Commit Graph

1559 Commits

Author SHA1 Message Date
Audrius Butkevicius e9bb17307d
lib/fs: Unwrap mtimeFile, get fd the "correct" way (ref #6875) (#6877) 2020-08-07 07:47:48 +02:00
Jakob Borg 5e1f39b6f6 lib/fs: Fix WatchRename test for FreeBSD (fixes #6613) 2020-08-03 23:24:01 +02:00
Audrius Butkevicius 96e197e502
lib/model: Don't close file early (fixes #6875) (#6876) 2020-08-03 21:54:42 +02:00
Simon Frei bbf25e2d02
lib/db: Log context on panic (#6872) 2020-08-01 17:32:36 +02:00
Simon Frei a85bc1c3a6
lib/model: Check folder error before sync-waiting (fixes #6793) (#6847) 2020-07-31 19:26:09 +02:00
Simon Frei 850dd4cd25
lib/db: Don't count invalid items to global state (ref #6850) (#6863) 2020-07-30 13:49:14 +02:00
Simon Frei e46c8ab9ee
lib/model: Detect deleted RO items scanning on non-RO (fixes #6864) (#6865) 2020-07-30 13:41:45 +02:00
Jakob Borg 2dc2aa5d21
lib/connections, lib/tlsutil: Handle certName in Go 1.15 (fixes #6867) (#6868)
Our authentication is based on device ID (certificate fingerprint) but
we also check the certificate name for ... historical extra security
reasons. (I don't think this adds anything but it is what it is.) Since
that check breaks in Go 1.15 this change does two things:

- Adds a manual check for the peer certificate CommonName, and if they
  are equal we are happy and don't call the more advanced
  VerifyHostname() function. This allows our old style certificates to
  still pass the check.

- Adds the cert name "syncthing" as a DNS SAN when generating the
  certificate. This is the correct way nowadays and makes VerifyHostname()
  happy in Go 1.15 as well, even without the above patch.
2020-07-30 13:36:11 +02:00
Jakob Borg d53a2567a4
build: Actually set build tags (#6866)
Apparently our Tags field depended on having specific files react to
tags and add themselves there. This, instead, works for all tags.

Also, pass tags to the test command line.
2020-07-30 10:58:43 +02:00
Jakob Borg 6f4671ed27 lib/connections: Add noquic tag
The QUIC package is notorious for being incompatible with either too
old or too new Go releases. Currently it doesn't build with Go 1.15 RC
and I want to test the rest with Go 1.15. With this I can do `go run
build.go --tags noquic` to do that.
2020-07-30 09:42:02 +02:00
Simon Frei 424d1b1608
lib/db: Commit meta when dropping device (#6862) 2020-07-28 16:46:42 +02:00
Simon Frei 1b9e5c0937
lib/db: Include blocks in db check (ref #6855) (#6861) 2020-07-28 16:25:07 +02:00
Jakob Borg 62cff26edf lib/config: Repair versioning XML serialization (fixes #6859, ref #6693) 2020-07-28 12:08:37 +02:00
Simon Frei 932d8c69de lib/fs: Properly handle case insensitive systems (fixes #1787, fixes #2739, fixes #5708)
With this change we emulate a case sensitive filesystem on top of
insensitive filesystems. This means we correctly pick up case-only renames
and throw a case conflict error when there would be multiple files differing
only in case.

This safety check has a small performance hit (about 20% more filesystem
operations when scanning for changes). The new advanced folder option
`caseSensitiveFS` can be used to disable the safety checks, retaining the
previous behavior on systems known to be fully case sensitive.

Co-authored-by: Jakob Borg <jakob@kastelo.net>
2020-07-28 11:15:11 +02:00
Simon Frei 08e0f938a9
lib/db: Update global meta even if unchanged (fixes #6850) (#6852) 2020-07-24 12:36:16 +02:00
Simon Frei ebead944b5
lib/fs: Pass infinite recursion error on instead of warning (#6846)
Prompted by https://forum.syncthing.net/t/infinite-filesystem-recursion-detected/15285. In my opinion the filesystem shouldn't throw warnings but pass on errors for the caller to decide what's to be happening with it. Right now in this PR an infinite recursion is a normal scan error, i.e. folder is in failed state and displays failed items, but no warning. I think that's appropriate but if deemed appropriate an additional warning can be thrown in the scanner.
2020-07-22 22:10:24 +02:00
Audrius Butkevicius 55147f5901
lib/db: Rework flush hooks (#6838) 2020-07-19 08:55:27 +02:00
Jakob Borg aedc2d788f
lib/model, lib/versioner: Drive version cleanup from scanner (fixes #6313) (#6693)
This change adds a separate config for the cleanup interval, and runs that cleanup from the main folder loop.
2020-07-14 10:48:50 +02:00
Simon Frei 16f4921c50
lib/nat: Make sure nat keeps being disabled (fixes #6823) (#6824) 2020-07-14 08:16:08 +02:00
Jakob Borg fc1dac5196
lib/scanner: Less strict validation (fixes #6827) (#6828)
This fixes the change in #6674 where the weak hash became a deciding
factor. Now we again just use it to accept a block, but don't take a
negative as meaning the block is bad.
2020-07-11 14:48:45 +02:00
greatroar 9f92f8c609
lib/db: Use SipHash to deal with hash collision in GC (#6826)
If the GC finds a key k that it wants to keep, it records that in a
Bloom filter. If a key k' can be removed but its hash collides with k,
it will be kept. Since the old Bloom filter code was completely
deterministic, the next run would encounter the same collision, assuming
k must still be kept.

A randomized hash function that uses all the SHA-256 bits solves this
problem: the second run has a non-zero probability of removing k', as
long as the Bloom filter is not completely full.
2020-07-11 09:36:09 +02:00
Jakob Borg bbda58a29f lib/db: Add test for GC run 2020-07-11 09:22:15 +02:00
Simon Frei 8d67235a75
lib/config: Don't cache device map (fixes #6816) (#6817) 2020-07-07 23:44:49 +02:00
Jakob Borg 0c61c66511
lib/api, lib/model: Improve folder completion API (fixes #6075) (#6808) 2020-07-03 08:48:37 +02:00
Simon Frei ebbe1abe28
lib/protocol: Include WeakHash in BlocksHash (#6799) 2020-06-29 09:33:07 +02:00
Audrius Butkevicius 9d4a700829
lib/ur: Track new folder feature usage (#6803) 2020-06-28 20:35:22 +02:00
Simon Frei d7fc7008af
lib/protocol: Add BlocksHash to FileInfo string (#6802) 2020-06-27 07:44:33 +02:00
Simon Frei 9c45ac381c
lib/model: Consider weak hash on recheckFile (#6797) 2020-06-26 16:47:03 +02:00
Simon Frei 4f06708330
lib/api, lib/model: Fixes around event request tracking (#6774) 2020-06-25 21:48:48 +02:00
Simon Frei 80ada1bb6c
lib/protocol: Don't warn when dropping deleted invalid item (fixes #6795) (#6796) 2020-06-25 20:24:31 +02:00
Simon Frei 0648fb0626
lib/model: Don't force rescan already changed items (#6798) 2020-06-25 20:23:59 +02:00
Simon Frei 90e248615f
lib/scanner: Test weak hash consistency (ref #5556) (#6794)
Relevant much earlier changes:
  9b1c592fb7
  bd1c29ee32

Make sure vanilla and rolling adler are consistent. And that they match
with scanner.Validate.
2020-06-25 14:47:35 +02:00
Audrius Butkevicius bb76311ec6
lib/ur: Zero our the value before unmarshal (#6790)
* lib/ur: Zero our the value before unmarshal

* Comment

* Complete rewrite
2020-06-23 21:23:32 +01:00
Audrius Butkevicius 689cf2a5ee
lib/ur: Normalise contract between syncthing and ursrv (#6770)
* Fix ui, hide report date

* Undo Goland madness

* UR now web scale

* Fix migration

* Fix marshaling, force tick on start

* Fix tests

* Darwin build

* Split "all" build target, add package name as a tag

* Remove pq and sql dep from syncthing, split build targets

* Empty line

* Revert "Empty line"

This reverts commit f74af2b067dadda8a343714123512bd545a643c3.

* Revert "Remove pq and sql dep from syncthing, split build targets"

This reverts commit 8fc295ad007c5bb7886c557f492dacf51be307ad.

* Revert "Split "all" build target, add package name as a tag"

This reverts commit f4dc88995106d2b06042f30bea781a0feb08e55f.

* Normalise contract types

* Fix build add more logging
2020-06-23 09:47:15 +01:00
Simon Mwepu 72f954dcab
lib/versioner: Create versioning directory recursively (fixes #6565) (#6678)
Co-authored-by: Jakob Borg <jakob@kastelo.net>
2020-06-23 08:37:29 +02:00
Simon Frei 85794933d3
lib/model: Remove double error handling in performFinish (#6780) 2020-06-23 07:30:16 +02:00
Simon Frei 4881a6336f
lib/db: Correct need accounting on reset (fixes #6784) (#6785) 2020-06-23 07:29:27 +02:00
Audrius Butkevicius 5fb3992275
lib/model: More connection closing (#6778)
* lib/model: More connection closing

* Revert "lib/model: More connection closing"

This reverts commit 5397c3a55c0de25fab93d4ec622530818f1b2a46.

* Add tests, fix one broken-ness

* Update lib/model/model.go

Co-authored-by: Simon Frei <freisim93@gmail.com>

* Update model.go

* Update model.go

Co-authored-by: Simon Frei <freisim93@gmail.com>
2020-06-22 21:26:26 +01:00
Jakob Borg b62b7d269e
lib/connections: Correctly fixup port in IPv6 listen URLs (#6786)
Previously tcp://[fe80::1] would get "fixed" to tcp://[[fe80::1]]:22000
which is not great.
2020-06-22 16:47:15 +01:00
Audrius Butkevicius cbaef624cf
lib/nat: Make service termination faster (#6777)
* lib/nat: Make service termination faster

* Newline
2020-06-22 09:01:57 +01:00
greatroar dc145bfad7
lib/protocol: Use sha256.Sum256 in NewDeviceID (#6775)
This is shorter, skips two allocations, makes the function inlineable
and is safer, since the compiler now check whether
DeviceIDLength == sha256.Size.
2020-06-21 20:38:06 +02:00
greatroar d985aa9e4b
lib/scanner: Fix Validate docs (#6776)
Co-authored-by: greatroar <@>
2020-06-21 19:23:06 +01:00
Jakob Borg 705710b1a1
gui, lib/api: Add more version info to about dialog (#6773) 2020-06-21 12:20:19 +02:00
Simon Frei f66e57947b
lib/model: Use right db snap when scanning recvonly folder (#6769) 2020-06-21 09:28:29 +02:00
Simon Frei 2716898cb9
lib/model: Don't ignore stat failure in performFinish (#6766) 2020-06-19 23:47:29 +02:00
Audrius Butkevicius deaccc7f8d
lib/fs: Add support for Windows duplicate extents (#6764) 2020-06-18 22:32:26 +01:00
Simon Frei 22f0077262
lib/model: Don't stay scanning forever on fail (#6761) 2020-06-18 14:13:46 +01:00
Jakob Borg 946170f3fc
gui, lib/ignore: Handle editing ignores with error (fixes #5425) (#6757)
This changes the error handling in loading ignores slightly:

- There is a new ParseError type that is returned as the error
  (somewhere in the chain) when the problem was not an I/O error loading
  the file, but some issue with the contents.

- If the file was read successfully but not parsed successfully we still
  return the lines read (in addition to nil patterns and a ParseError).

- In the API, if the error IsParseError then we return a successful
  HTTP response with the lines and the actual error included in the JSON
  object.

- In the GUI, as long as the HTTP call to load the ignores was
  successful we can edit the ignores. If there was an error we show this
  as a validation error on the dialog.

Also some cleanup on the Javascript side as it for some reason used
jQuery instead of Angular for this editor...
2020-06-18 11:04:00 +02:00
Simon Frei 8cf9d91ed4
lib: Print nicely rounded durations (#6756) 2020-06-18 10:55:41 +02:00
Audrius Butkevicius 4812fd3ec1
all: Add copy-on-write filesystem support (fixes #4271) (#6746) 2020-06-18 08:15:47 +02:00