Commit Graph

648 Commits

Author SHA1 Message Date
Simon Frei e19728d8cc
lib/model: Correct completion perc. when globalBytes is 0 (#6963) 2020-09-07 20:03:18 +02:00
Simon Frei 3dd13c3994
test, lib/model: Various integration test updates & improvements (#6956) 2020-09-07 09:35:37 +02:00
Simon Frei 6365e6108f
lib/model: Don't acquire I/O token for send-only pull (#6951) 2020-09-03 13:59:45 +02:00
Simon Frei 540518a7b7
lib/model: Add missing lock on download-state (fixes #6880) (#6945) 2020-08-30 08:03:10 +02:00
Jakob Borg 3ffbe45a6d
lib/db, lib/model: Avoid accounting mishap on folder restart (fixes #6938) (#6939)
We created a new fileset before stopping the folder during restart. When
we create that fileset it loads the current metadata and sequence
numbers from the database. But the folder may have time to update those
before stopping, leaving the new fileset with bad data.

This would cause wrong accounting (forgotten files) and potentially
sequence reuse causing files not sent to other devices.

This change reuses the fileset on restart, skipping the issue entirely.
It also moves the creation of the fileset back under the lock so there
should be no chance of concurrency issues here.
2020-08-27 18:33:27 +02:00
Jakob Borg 3393db1f69
lib/model: Save config after auto accepting folders (fixes #6922) (#6937) 2020-08-27 16:28:19 +02:00
Jakob Borg 145d87ce70 lib/model: Fixup spelling of protocol.FileIntf in test (ref #6463) 2020-08-27 16:21:04 +02:00
Simon Frei 06ac631351
lib/model: Reduce fmut locking in ClusterConfig (#6913)
The FileSet.Drop operation in there needs to potentially update a whole lot of global lists, which can take a while (longer than the deadlock interval apparently)
2020-08-27 16:01:46 +02:00
Simon Frei 1fc2dbdeeb
lib/model: Add test for recv-only changes undone by remote (#6463)
Co-authored-by: Audrius Butkevicius <audrius.butkevicius@gmail.com>
2020-08-27 15:54:00 +02:00
Audrius Butkevicius d507d932b8
all: Use protobuf to generate config structs (fixes #6734) (#6900) 2020-08-25 08:11:14 +02:00
Simon Frei a65b46debd
lib/model: Don't mark globally deleted items as recv-only (fixes #6910) (#6911) 2020-08-21 12:34:22 +02:00
Simon Frei b628460930
lib/model: Prevent localflag flipflopping on del. item (ref #6865) (#6919) 2020-08-21 10:09:48 +02:00
Simon Frei cc1f6e4d4a
lib/db, lib/model: Cover exec-paths with debug logging (#6918) 2020-08-20 16:11:20 +02:00
Simon Frei 88599bc154
lib/model: Handle del. dir with locally changed items on pull (fixes #6873) (#6914) 2020-08-20 10:56:29 +02:00
Audrius Butkevicius bf9ff17267
all: Remove need to restart syncthing (#6883) 2020-08-18 09:26:33 +02:00
Audrius Butkevicius e9bb17307d
lib/fs: Unwrap mtimeFile, get fd the "correct" way (ref #6875) (#6877) 2020-08-07 07:47:48 +02:00
Audrius Butkevicius 96e197e502
lib/model: Don't close file early (fixes #6875) (#6876) 2020-08-03 21:54:42 +02:00
Simon Frei a85bc1c3a6
lib/model: Check folder error before sync-waiting (fixes #6793) (#6847) 2020-07-31 19:26:09 +02:00
Simon Frei 850dd4cd25
lib/db: Don't count invalid items to global state (ref #6850) (#6863) 2020-07-30 13:49:14 +02:00
Simon Frei e46c8ab9ee
lib/model: Detect deleted RO items scanning on non-RO (fixes #6864) (#6865) 2020-07-30 13:41:45 +02:00
Simon Frei 932d8c69de lib/fs: Properly handle case insensitive systems (fixes #1787, fixes #2739, fixes #5708)
With this change we emulate a case sensitive filesystem on top of
insensitive filesystems. This means we correctly pick up case-only renames
and throw a case conflict error when there would be multiple files differing
only in case.

This safety check has a small performance hit (about 20% more filesystem
operations when scanning for changes). The new advanced folder option
`caseSensitiveFS` can be used to disable the safety checks, retaining the
previous behavior on systems known to be fully case sensitive.

Co-authored-by: Jakob Borg <jakob@kastelo.net>
2020-07-28 11:15:11 +02:00
Jakob Borg aedc2d788f
lib/model, lib/versioner: Drive version cleanup from scanner (fixes #6313) (#6693)
This change adds a separate config for the cleanup interval, and runs that cleanup from the main folder loop.
2020-07-14 10:48:50 +02:00
Jakob Borg 0c61c66511
lib/api, lib/model: Improve folder completion API (fixes #6075) (#6808) 2020-07-03 08:48:37 +02:00
Simon Frei 9c45ac381c
lib/model: Consider weak hash on recheckFile (#6797) 2020-06-26 16:47:03 +02:00
Simon Frei 4f06708330
lib/api, lib/model: Fixes around event request tracking (#6774) 2020-06-25 21:48:48 +02:00
Simon Frei 0648fb0626
lib/model: Don't force rescan already changed items (#6798) 2020-06-25 20:23:59 +02:00
Simon Frei 90e248615f
lib/scanner: Test weak hash consistency (ref #5556) (#6794)
Relevant much earlier changes:
  9b1c592fb7
  bd1c29ee32

Make sure vanilla and rolling adler are consistent. And that they match
with scanner.Validate.
2020-06-25 14:47:35 +02:00
Audrius Butkevicius 689cf2a5ee
lib/ur: Normalise contract between syncthing and ursrv (#6770)
* Fix ui, hide report date

* Undo Goland madness

* UR now web scale

* Fix migration

* Fix marshaling, force tick on start

* Fix tests

* Darwin build

* Split "all" build target, add package name as a tag

* Remove pq and sql dep from syncthing, split build targets

* Empty line

* Revert "Empty line"

This reverts commit f74af2b067dadda8a343714123512bd545a643c3.

* Revert "Remove pq and sql dep from syncthing, split build targets"

This reverts commit 8fc295ad007c5bb7886c557f492dacf51be307ad.

* Revert "Split "all" build target, add package name as a tag"

This reverts commit f4dc88995106d2b06042f30bea781a0feb08e55f.

* Normalise contract types

* Fix build add more logging
2020-06-23 09:47:15 +01:00
Simon Frei 85794933d3
lib/model: Remove double error handling in performFinish (#6780) 2020-06-23 07:30:16 +02:00
Audrius Butkevicius 5fb3992275
lib/model: More connection closing (#6778)
* lib/model: More connection closing

* Revert "lib/model: More connection closing"

This reverts commit 5397c3a55c0de25fab93d4ec622530818f1b2a46.

* Add tests, fix one broken-ness

* Update lib/model/model.go

Co-authored-by: Simon Frei <freisim93@gmail.com>

* Update model.go

* Update model.go

Co-authored-by: Simon Frei <freisim93@gmail.com>
2020-06-22 21:26:26 +01:00
Simon Frei f66e57947b
lib/model: Use right db snap when scanning recvonly folder (#6769) 2020-06-21 09:28:29 +02:00
Simon Frei 2716898cb9
lib/model: Don't ignore stat failure in performFinish (#6766) 2020-06-19 23:47:29 +02:00
Simon Frei 22f0077262
lib/model: Don't stay scanning forever on fail (#6761) 2020-06-18 14:13:46 +01:00
Jakob Borg 946170f3fc
gui, lib/ignore: Handle editing ignores with error (fixes #5425) (#6757)
This changes the error handling in loading ignores slightly:

- There is a new ParseError type that is returned as the error
  (somewhere in the chain) when the problem was not an I/O error loading
  the file, but some issue with the contents.

- If the file was read successfully but not parsed successfully we still
  return the lines read (in addition to nil patterns and a ParseError).

- In the API, if the error IsParseError then we return a successful
  HTTP response with the lines and the actual error included in the JSON
  object.

- In the GUI, as long as the HTTP call to load the ignores was
  successful we can edit the ignores. If there was an error we show this
  as a validation error on the dialog.

Also some cleanup on the Javascript side as it for some reason used
jQuery instead of Angular for this editor...
2020-06-18 11:04:00 +02:00
Simon Frei 8cf9d91ed4
lib: Print nicely rounded durations (#6756) 2020-06-18 10:55:41 +02:00
Audrius Butkevicius 4812fd3ec1
all: Add copy-on-write filesystem support (fixes #4271) (#6746) 2020-06-18 08:15:47 +02:00
Simon Frei 6976219d6d
lib/model: Check dir before deletion when pulling (#6741) 2020-06-16 15:20:08 +02:00
Simon Frei a47546a1f1
lib/scanner, lib/model: Improve error handling when scanning (#6736) 2020-06-16 09:25:41 +02:00
Jakob Borg 5be13f62a2 lib/model: Fix minor flakiness in TestModTimeWindow 2020-06-16 06:32:37 +02:00
Simon Frei 1f8e6c55f6
lib/db: Refactor to use global list by version (fixes #6372) (#6638)
Group the global list of files by version, instead of having one flat list for all devices. This removes lots of duplicate protocol.Vectors.

Co-authored-by: Jakob Borg <jakob@kastelo.net>
2020-05-30 09:50:23 +02:00
Simon Frei ed6bfc5417
lib/model: Don't increase pull pause on triggered pull (#6690) 2020-05-29 09:52:28 +02:00
Simon Frei b784f5b9e3
lib/config, lib/model: Commit auto-accepted folders all at once (#6684) 2020-05-27 08:05:26 +02:00
Simon Frei c3b5eba205
lib/model: Fix checking children when trying to delete a dir (fixes #6646) (#6647) 2020-05-25 08:46:24 +02:00
Simon Frei cf75329067
lib/model: Handle concurrently removed device in ClusterConfig (#6666) 2020-05-20 11:13:55 +02:00
Jakob Borg 5ffa012410
lib/model: Don't crash when taking rename shortcut (fixes #6654) (#6657)
If we fail to take the rename shortcut we may crash on a later loop,
because we do trickiness with the indexes but the original buckets[key]
in "range buckets[key]" isn't re-evaluated so i exceeds the max index.
2020-05-17 08:48:35 +01:00
Jakob Borg 651ee2ce74
lib/model: Partial revert of rename fix (fixes #6653) (#6656)
We can't just drop the snap because it's in use elsewhere. This should
be equally functional.
2020-05-16 23:05:33 +02:00
Audrius Butkevicius a1c5b44c74
lib/model: Fix rename handling (ref #6650) (#6652) 2020-05-16 14:39:27 +02:00
Simon Frei e5cc55ce09
lib/model: Close conns when devices are removed (fixes #6564) (#6641) 2020-05-14 07:50:53 +02:00
Simon Frei 299b9d8883
lib/model: Adjust remote-rename-test to timer-based versions (fixes #6625) (#6644) 2020-05-14 00:31:05 +02:00
Simon Frei 974551375e
lib/db: Dont add symlinks to blocks map (fixes #6637) (#6639) 2020-05-13 20:38:21 +02:00