Commit Graph

258 Commits

Author SHA1 Message Date
Jakob Borg 756c5a2604 User fewer hasher routines when there are many folders. 2015-04-29 21:26:08 +02:00
dartraiden 32a76901a9 typos and spelling correction 2015-04-29 15:59:47 +02:00
Jakob Borg 60fcaebfdb Run vet and lint. Make us lint clean. 2015-04-29 10:38:02 +02:00
Audrius Butkevicius 19a2042746 Merge pull request #1723 from calmh/bug-1722
Handle conflict with local delete (fixes #1722)
2015-04-28 10:39:22 +01:00
Jakob Borg 57ec2ff915 Handle conflict with local delete (fixes #1722) 2015-04-28 11:34:16 +02:00
Audrius Butkevicius 77a161a087 Matcher checks nil receiver 2015-04-28 10:17:14 +01:00
Audrius Butkevicius 9afbca3001 Add pagination to Out of sync item list (fixes #1509) 2015-04-26 00:22:26 +01:00
Jakob Borg be7b3a9952 Configurable file pull order (alphabetic, random, by size or age) 2015-04-25 23:20:21 +09:00
Jakob Borg 33a4fb5a1a Fix folder check tests 2015-04-25 23:16:46 +09:00
Audrius Butkevicius 433b923ea7 Add mutex logging 2015-04-23 10:54:14 +01:00
Jakob Borg bf7d84c12a Clean up REST JSON a little further 2015-04-21 23:28:58 +09:00
KAMADA Ken'ichi 33d3ba1b45 Fix nil pointer dereferences in REST with non-existent folders 2015-04-18 22:41:47 +09:00
Jakob Borg cebe21a3af Don't buffer large files a long time while scanning 2015-04-17 16:40:09 +09:00
Jakob Borg 6d84443db8 Homogenize the LocalIndexUpdated event
It had two different formats, and we use "items" instead of "numFiles"
in other places.

(Discovered while documenting :)
2015-04-17 14:22:06 +09:00
Jakob Borg da8a1f242c Merge pull request #1646 from AudriusButkevicius/readonly
Make targets writeable before removal on Windows (fixes #1610)
2015-04-17 14:21:39 +09:00
Audrius Butkevicius dff51fc707 Make targets writeable before removal on Windows (fixes #1610) 2015-04-16 22:53:53 +01:00
Audrius Butkevicius 7d954dd5d1 Fix tests on Windows (fixes #1531) 2015-04-16 21:18:17 +01:00
Jakob Borg d1cc1828b8 Improve ItemStarted/ItemFinished events
- Remove full details from ItemStarted (unnecessary, incorrect CamelCase)

 - Add "type" ("file" or "dir") to both events

 - Add "action" (what we tried to do - "delete" or "update") to both
   events.
2015-04-14 23:31:39 +09:00
Jakob Borg 102a2db1f3 Work around broken Lstat on Android 2015-04-14 19:53:49 +09:00
Jakob Borg aa803ce2ff Move folder errors to state
The "Invalid" config attribute is retained for errors discovered during
config loading (empty path, duplicate ID). This can only be set or
cleared at config loading time.

Errors discovered during runtime (I/O problems, etc) are now in the
folder state instead. Changes to these are sent as any other folder
state change.
2015-04-13 07:43:45 +09:00
Jakob Borg 936c76119d Index reset should generate file conflicts (fixes #1613) 2015-04-09 13:06:09 +02:00
Audrius Butkevicius df6dbc5fa4 Only run deadlock detection if asked or non-release/beta (fixes #1536) 2015-04-08 13:40:05 +01:00
Jakob Borg b2a79855ae Handle long filenames on Windows (fixes #1295) 2015-04-08 14:05:39 +02:00
Audrius Butkevicius 8f05b8f982 Change /rest/system/connections output (fixes #1487) 2015-04-07 13:21:03 +01:00
Jakob Borg a17333d73e Override needs to twiddle the version a bit more (fixes #1564) 2015-04-06 20:55:40 +02:00
Jakob Borg ecadf30fe7 model: Use separate db commit routine (fixes #1558) 2015-04-05 16:19:14 +02:00
Audrius Butkevicius 6fb1e03ed4 Merge pull request #1576 from Zillode/reset-indexes
Update reset API to reflect new use cases.
2015-04-04 22:31:59 +01:00
Jakob Borg 739c525a98 model: TestIgnores should not randomly fail 2015-04-04 22:55:24 +02:00
Lode Hoste ab287ebf40 Update reset API to reflect new use cases.
/rest/reset clears the entire Syncthing DB and restart the program
/rest/reset&folder=default clears the indexes of the default folder
2015-04-04 22:45:11 +02:00
Audrius Butkevicius e1757ee726 Fix test 2015-03-30 22:49:16 +01:00
Audrius Butkevicius 5fe15475a4 Merge pull request #1540 from calmh/conflicts
Handle conflicts when pulling (fixes #220)
2015-03-30 21:20:23 +01:00
Audrius Butkevicius 7406176fad More graceful handling on folder errors (fixes #762)
Checks health before accepting every scanner batch, also
recovers from errors without having to restart.
2015-03-30 08:27:12 +02:00
Jakob Borg 47e64ae503 Handle conflicts when pulling (fixes #220) 2015-03-30 00:01:52 +02:00
Jakob Borg ba575f55ec Merge pull request #1530 from Zillode/multi-scan
Support multiple scan query strings at the same time
2015-03-29 16:02:45 +02:00
Lode Hoste 2012ce02e8 Support multiple scan query strings at the same time 2015-03-28 22:40:13 +01:00
Audrius Butkevicius 0401a07507 Change existingBlocks map type 2015-03-26 22:04:34 +00:00
Audrius Butkevicius c12265499a Response errors may be protocol defined errors 2015-03-26 22:04:34 +00:00
Audrius Butkevicius 0e341832e0 Handle unknown flags at the model 2015-03-26 22:04:34 +00:00
Audrius Butkevicius 489e2e6ad5 Update Model function signatures 2015-03-26 22:04:33 +00:00
Jakob Borg 6da7f17c4a Implement version vectors 2015-03-25 23:10:34 +01:00
Jakob Borg b4f45d1e79 Update tests for version vectors 2015-03-25 23:10:33 +01:00
KAMADA Ken'ichi ac7097b4d0 Preserve the permission of a newly created directory
We need an explicit chmod() when creating a new directory.
Otherwise a new directory may be created with a different permission
from the one received from an originating device, because the umask
is applied to the mode given to mkdir().
The incorrect permission is later sent back to the originating device
and the original permission will be lost.
2015-03-23 22:39:16 +09:00
Jakob Borg 8311162be3 Automatically fix file name normalization errors (fixes #430) 2015-03-19 00:21:48 +01:00
Jakob Borg 1c3158099c Rename files to match type names 2015-03-17 19:37:06 +01:00
Jakob Borg bdbca75dfa Refactor state tracking (...)
Move state tracking into the puller/scanner objects. This is a first
step towards resolving #1391.

Rename Puller and Scanner to roFolder and rwFolder as they have more
duties than just pulling and scanning, and don't need to be exported.
2015-03-17 19:37:06 +01:00
Audrius Butkevicius de38b46392 Fix build 2015-03-17 17:54:25 +00:00
Jakob Borg 32425c5561 MPLv2 2015-03-17 16:02:27 +01:00
Pascal Jungblut 49bc74e7a0 Use lowerCamelCase for the JSON API (fixes #1338)
Replace the current mix of UpperCamelCase und lowerCamelCase with
consistent lowerCamelCase keys for the JSON API. Also adapt the frontend
so it works with the changed API.

Attention: this will break existing consumers of the API.
2015-03-16 10:05:01 +01:00
Audrius Butkevicius 57fc0eb5b1 Make sure we start scanning at an indexed location (fixes #1399) 2015-03-08 19:45:47 +00:00
Lode Hoste 758233f001 Do not error when chmod failes when permissions are ignored (fixes #1404). 2015-03-07 21:38:16 +01:00
Audrius Butkevicius bf3e249237 Add GlobalDirectoryTree benchmarks 2015-03-04 23:39:33 +00:00
Audrius Butkevicius fb649e9525 Fix benchmarks, cleanup tests 2015-03-04 23:39:32 +00:00
Audrius Butkevicius 9d1e2d9f46 Add /rest/tree API call 2015-03-04 23:39:27 +00:00
Audrius Butkevicius 9876d93b60 Fix tests on Windows while running as a simple user 2015-03-04 22:39:33 +00:00
Jakob Borg 44d0da02d0 Ensure progress when delete-by-rename fails (fixes #1373) 2015-03-01 10:55:48 +01:00
Jakob Borg c25107eff3 Handle weird Lstat() returns for disappeared items (ref #1373) 2015-03-01 10:55:43 +01:00
Jakob Borg e7dbb8ccdc Fix test for unknown flags 2015-02-15 09:51:39 +01:00
Karol Różycki ec160f1f0a Button to rescan all folders, fixes #1151 2015-02-12 21:03:35 +01:00
Jakob Borg 6bbd74adcd Merge pull request #1321 from AudriusButkevicius/bitcheck
Refuse files with unknown bits set (fixes #1276)
2015-02-10 20:27:14 +01:00
Audrius Butkevicius b87ed97402 Refuse files with unknown bits set (fixes #1276) 2015-02-09 23:32:33 +00:00
Jakob Borg a73a011ee0 Merge pull request #1323 from AudriusButkevicius/finished
Add ItemFinished event (fixes #1258)
2015-02-09 15:24:10 +01:00
Jakob Borg 5d9a41f712 Merge pull request #1319 from AudriusButkevicius/renames
Fix issues with renames
2015-02-09 15:14:47 +01:00
Audrius Butkevicius 38eaefcabd Add ItemFinished event (fixes #1258) 2015-02-01 18:59:29 +00:00
Audrius Butkevicius 380d5dfa6d Remove fd cache (ref #1308) 2015-02-01 18:59:24 +00:00
Audrius Butkevicius 32af626630 Fix issues with renames (fixes #1302)
Extra comments explain current issues.
2015-02-01 18:58:27 +00:00
Jakob Borg 2c8b627008 Integer type policy
Integers are for numbers, enabling arithmetic like subtractions and for
loops without getting shot in the foot. Unsigneds are for bitfields.

- "int" for numbers that will always be laughably smaller than four
  billion, and where we don't care about the serialization format.

- "int32" for numbers that will always be laughably smaller than four
  billion, and will be serialized to four bytes.

- "int64" for numbers that may approach four billion or will be
  serialized to eight bytes.

- "uint32" and "uint64" for bitfields, depending on required number of
  bits and serialization format. Likewise "uint8" and "uint16", although
  rare in this project since they don't exist in XDR.

- "int8", "int16" and plain "uint" are almost never useful.
2015-01-19 10:34:36 -08:00
Jakob Borg 221e3eddd5 Remove leveldb panic workaround
Haven't seen this triggered for a long time...
2015-01-19 10:23:00 -08:00
Jakob Borg 74c39c677b Actually remove test file after test run 2015-01-19 10:23:00 -08:00
Audrius Butkevicius cba38b15a9 Check for deleted files 2015-01-18 13:44:10 +00:00
Jakob Borg 53cd289b90 Verify folder<->device permission in Request
Requests from valid devices for valid folders should be rejected if the
folder is not shared with that device.
2015-01-16 12:50:51 +01:00
Audrius Butkevicius 74d7c8e625 Efficient renames (fixes #1217) 2015-01-13 22:06:13 +00:00
Jakob Borg eef1aebe8c Refactor out protocol and luhn (protocol dependency) packages 2015-01-13 13:22:56 +01:00
Jakob Borg e6c9baf6ef Rename db.Set to db.FileSet 2015-01-12 20:57:39 +01:00
Jakob Borg 8d6db7be31 Rename package internal/files to internal/db 2015-01-12 20:57:22 +01:00
Jakob Borg 571cf7d490 Merge pull request #1182 from AudriusButkevicius/autoauto
Connecting to a newer node triggers autoupgrade check (fixes #1177)
2015-01-11 09:16:12 +01:00
Audrius Butkevicius 5034a41c08 Connecting to a newer node triggers autoupgrade check (fixes #1177) 2015-01-10 17:05:19 +00:00
Jakob Borg 219ef996f5 Merge pull request #1226 from syncthing/deregister-fix
All roads lead to Finisher (fixes #1201)
2015-01-10 17:53:01 +01:00
Audrius Butkevicius 5827a686b8 Make parallel hasher configurable, remove finisher setting (fixes #1199) 2015-01-10 14:32:20 +00:00
Audrius Butkevicius dec479532e All roads lead to Finisher (fixes #1201) 2015-01-10 13:45:48 +00:00
Jakob Borg d46ce5003c Implement GetGlobalTruncated 2015-01-09 08:41:02 +01:00
Jakob Borg 4c4143d9be Move FileInfoTruncated to files package
This is where it's used, and it clarifies that it's never used over the
wire.
2015-01-09 08:28:24 +01:00
Jakob Borg 8bc7d259f4 Move FileIntf to files package, expose Iterator type
This is where FileIntf is used, so it should be defined here (it's not
a protocol thing, really).
2015-01-09 08:18:42 +01:00
Audrius Butkevicius 9d6ef24660 Merge pull request #1194 from syncthing/fix-1186
Use comma-ok idiom to signal files missing in database (fixes #1186)
2015-01-06 21:54:13 +00:00
Brendan Long 7a9fc6dbd3 Don't use Go 1.4 range syntax in queue_test.go, since the listed requirement is Go 1.3. 2015-01-06 15:45:58 -06:00
Jakob Borg 75d0dc251e Use comma-ok idiom to signal files missing in database (fixes #1186)
Prevents us from doing stupid things to the folder root (empty file
path) when nodes disconnect...
2015-01-06 22:40:20 +01:00
Audrius Butkevicius 5ac01a3af4 Hash blocks after receipt, try multiple peers (fixes #1166) 2015-01-03 23:21:57 +00:00
Audrius Butkevicius 56ccb5b2ab New device, folder prompts (fixes #120, fixes #330) 2015-01-03 23:06:41 +00:00
Audrius Butkevicius 9c6cb0f630 Merge pull request #1172 from syncthing/random-scanintv
Add a random perturbation to the scan interval (fixes #1150)
2015-01-02 15:25:22 +00:00
Jakob Borg 54c3caad53 Add a random perturbation to the scan interval (fixes #1150) 2015-01-02 16:16:16 +01:00
Jakob Borg 9b5e8aaf83 Repair buggy BringToFront 2015-01-02 15:54:04 +01:00
Jakob Borg 5143c09bcf Refactor / cleanup 2015-01-02 15:54:04 +01:00
Jakob Borg 2496185629 Only buffer file names, not full &FileInfo 2015-01-02 15:33:39 +01:00
Jakob Borg 34deb82aea Use slice instead of list, no map
benchmark                           old ns/op     new ns/op     delta
BenchmarkJobQueueBump               345           154498        +44682.03%
BenchmarkJobQueuePushPopDone10k     9437373       3258204       -65.48%

benchmark                           old allocs     new allocs     delta
BenchmarkJobQueueBump               0              0              +0.00%
BenchmarkJobQueuePushPopDone10k     10565          22             -99.79%

benchmark                           old bytes     new bytes     delta
BenchmarkJobQueueBump               0             0             +0.00%
BenchmarkJobQueuePushPopDone10k     1452498       385869        -73.43%
2015-01-02 15:33:39 +01:00
Jakob Borg 8f72ae9da2 Add some benchmarks 2015-01-02 15:33:39 +01:00
Audrius Butkevicius b753f01ac1 Add tests 2015-01-02 15:33:39 +01:00
Audrius Butkevicius fd0a147ae6 Add job queue (fixes #629)
Request to terminate currently ongoing downloads and jump to the bumped file
incoming in 3, 2, 1.

Also, has a slightly strange effect where we pop a job off the queue, but
the copyChannel is still busy and blocks, though it gets moved to the
progress slice in the jobqueue, and looks like it's in progress which it isn't
as it's waiting to be picked up from the copyChan.

As a result, the progress emitter doesn't register on the task, and hence the file
doesn't have a progress bar, but cannot be replaced by a bump.

I guess I can fix progress bar issue by moving the progressEmiter.Register just
before passing the file to the copyChan, but then we are back to the initial
problem of a file with a progress bar, but no progress happening as it's stuck
 on write to copyChan

I checked if there is a way to check for channel writeability (before popping)
but got struck by lightning just for bringing the idea up in #go-nuts.

My ideal scenario would be to check if copyChan is writeable, pop job from the
queue and shove it down handleFile. This way jobs would stay in the queue while
they cannot be handled, meaning that the `Bump` could bring your file up higher.
2015-01-02 15:33:39 +01:00
Jakob Borg ce4b897d0e Handle read only temp files after crash/restart 2014-12-31 13:06:28 +01:00
Jakob Borg 22f39be197 Exit before attempting to use nil variables on scanning nonexistent folder 2014-12-23 14:14:05 +01:00
Jakob Borg cadbb6bbce Move ignore handling from index recv to puller (fixes #1133)
With this change we accept updates for ignored files from other devices,
and check the ignore patterns at pull time. When we detect that the
ignore patterns have changed we do a full check of files that we might
now need to pull.
2014-12-23 10:46:02 +01:00
Jakob Borg 2c89f04be7 Refactor ignore handling (...)
This uses persistent Matcher objects that can reload their content and
provide a hash string that can be used to check if it's changed. The
cache is local to each Matcher object instead of kept globally.
2014-12-23 10:46:02 +01:00