Commit Graph

1747 Commits

Author SHA1 Message Date
greatroar
9c88efd55f
lib/util: Don't modify input in UniqueTrimmedStrings (#7288)
Also clarified the comment.
2021-01-16 17:39:15 +01:00
Jakob Borg
ffcb57580f
cmd/syncthing: Provide early startup for config service (ref #7188) (#7285) 2021-01-16 12:58:02 +01:00
Simon Frei
f63cdbfcfa
lib: Apply config changes sequentially (ref #5298) (#7188) 2021-01-15 15:43:34 +01:00
Simon Frei
b2d82da20d
lib/model: Pull when folder leaves error state (fixes #7280) (#7281) 2021-01-14 13:29:01 +01:00
Jakob Borg
36acaf5e21
lib/fs: Avoid blocking new caseFs creation while waiting to drop cache (fixes #7273) (#7275) 2021-01-13 17:45:29 +01:00
Jakob Borg
253049a4cc
lib/api, lib/model: Avoid contention on filesystem for DB status call (ref #7270) (#7271)
This splits the ignore getting to two methods, one that loads from disk
(the old one) and one that just returns whatever is already loaded (the
new one). The folder summary service which is just interested in stats
now uses the latter method. This means that it, and API calls that call
it, does not get blocked by folder I/O.
2021-01-12 16:25:21 +01:00
Jakob Borg
8f199e12b3
lib/fs, lib/model: Reduce lock contention on NewCaseFilesystem (fixes #7268) (#7269) 2021-01-12 16:22:21 +01:00
greatroar
f6fac3e949
lib/ur: Plug file descriptor leak in Linux memorySize (#7266) 2021-01-11 15:15:21 +01:00
Jakob Borg
0b193b76c2
lib/config, lib/connections: Add optional connection limits (fixes #7176) (#7223)
This adds two new configuration options:

    // The number of connections at which we stop trying to connect to more
    // devices, zero meaning no limit. Does not affect incoming connections.
    ConnectionLimitEnough int

    // The maximum number of connections which we will allow in total, zero
    // meaning no limit. Affects incoming connections and prevents
    // attempting outgoing connections.
    ConnectionLimitMax int

These can be used to limit the number of concurrent connections in
various ways.
2021-01-11 15:14:44 +01:00
Jakob Borg
b13b15758d
lib/connections, lib/model: Track last connection duration (ref #7223) (#7242)
This adds a statistic to track the last connection duration per device.
It isn't used for much in this PR, but it's available for #7223 to use
in deciding how to order device connection attempts (deprioritizing
devices that just dropped our connection the last time).
2021-01-05 17:45:07 +01:00
Simon Frei
c48eb4241a
lib/model: Fix child-check when deleting dirs in pull (#7236) 2021-01-02 21:40:37 +01:00
Simon Frei
0f8290485e
lib/model: Handle index sender terminating due to error (fixes #7231) (#7232) 2020-12-30 09:59:11 +01:00
André Colomb
3d1edd2492
lib/fs: Fix TestChmodDir depending on umask (fixes #6551) (#7241)
The test would fail if the umask on UNIX is greater than 0022, because
the OS transparently subtracts it from the mode passed to Mkdir(), as
the Go documentation confirms.

Our goal here is not to test os.Mkdir(), so just make sure the desired
mode is actually set by forcing it afterwards.
2020-12-30 09:56:10 +01:00
Simon Frei
a05dc6cc47
lib/model: Cleanup redundant filesystem variables in folders (#7237) 2020-12-27 22:26:25 +01:00
Simon Frei
f13e6ca631
lib/model: Remove obsolete return val from ccHandleFolders (ref #6443) (#7229) 2020-12-23 13:10:08 +01:00
Simon Frei
a20a5f61f0
lib/ur: Send unreported failures on shutdown (#7164) 2020-12-22 20:17:14 +01:00
Jakob Borg
05f25e600e
lib/connections: Refactor connection loop (#7177)
This breaks out some methods from the connection loop to make it simpler
to manage and understand.

Some slight simplifications to remove the `seen` variable (we can filter
`nextDial` based on times are in the future or not, so we don't need to
track `seen`) and adding a minimum loop interval (5s) in case some
dialer goes haywire and requests a 0s redial interval or such.

Otherwise no significant behavioral changes.
2020-12-21 16:40:13 +01:00
Simon Frei
a744dee94c
lib/fs: Correct wrapping order for meaningful log-caller (#7209) 2020-12-21 13:01:34 +01:00
Simon Frei
78bd0341a8
all: Handle errors opening db/creating file-set (ref #5907) (#7150) 2020-12-21 12:59:22 +01:00
Simon Frei
c845e245a1
lib: Close underlying conn in protocol (fixes #7165) (#7212) 2020-12-21 11:40:51 +01:00
Simon Frei
4a787986cd
lib/db: Prevent IndexID creation race (#7211) 2020-12-21 11:32:59 +01:00
Simon Frei
bd0c9913cf
lib/db: Remove index ids when dropping folder (#7200) 2020-12-21 11:10:59 +01:00
Simon Frei
d904dfa191
lib/model: Fix flaky test and add some scanning debug (#7214) 2020-12-20 18:13:35 +01:00
Simon Frei
fa40ccece1
lib: Consistently set suture logging (#7202) 2020-12-18 19:44:00 +01:00
Simon Frei
7919310dc6
lib/model: Unflake TestIgnoreDeleteUnignore (#7208) 2020-12-18 18:42:09 +01:00
Simon Frei
739e99c4d9
lib/config: Remove deprecated pending entries from config (ref #6443) (#7204) 2020-12-17 22:49:29 +01:00
André Colomb
7502997e7e
all: Store pending devices and folders in database (fixes #7178) (#6443) 2020-12-17 19:54:31 +01:00
Jakob Borg
4142a431b5 model: Actually print folder description in "Overriding" log message 2020-12-12 12:32:24 +01:00
Simon Frei
0db3b7a530
build: Switch to gopsutil's v3 module (#7191) 2020-12-10 16:43:15 +01:00
Jakob Borg
ec5a5d5218
lib/api: Returns tags in version as list (#7190) 2020-12-10 12:22:09 +01:00
Jakob Borg
e9b68a224c
lib/connections: Handle QUIC not being available (#7186)
This does two things:

- Exclude QUIC from go1.16 builds, automatically, for now, since it
  doesn't work and just panics.

- Provide some fake listeners and dialers when QUIC is disabled.

These fake listeners and dialers indicate that they are disabled and
unsupported, which silences "Dialing $address: unknown address scheme:
quic" type of stuff which is not super helpful to the user.
2020-12-09 19:23:50 +01:00
Simon Frei
8fd6b1d428
lib/protocol: Handle slashified paths in IsEncryptedParent (fixes #7184) (#7187) 2020-12-09 18:16:14 +01:00
Simon Frei
cccbb0bd5e
lib/ur: Reset timer when there's nothing to report (#7169) 2020-11-28 20:09:22 +01:00
Simon Frei
240ae0c14f
lib/model: Unflake TestRequestReceiveEncryptedLocalNoSend (#7167) 2020-11-27 20:53:03 +01:00
Simon Frei
bbb22c8c80
lib/protocol: Send Close message on read error (#7141) 2020-11-27 11:31:20 +01:00
Simon Frei
a9764fc16c
lib: Skip deleted, locally changed on recv-enc folders (fixes #7153) (#7154) 2020-11-27 11:26:36 +01:00
Audrius Butkevicius
af13f0cd35
lib/stun: Don't notify about address changes if it's not useful (fixes #7144) (#7159) 2020-11-27 07:34:30 +01:00
Simon Frei
e1b958284e
lib/api: Shut the api down gracefully (fixes #7138) (#7157) 2020-11-26 15:49:39 +01:00
Simon Frei
bf7d03d029
lib/model: Fix enc file size when pulling (fixes #7152) (#7155) 2020-11-25 22:57:25 +01:00
Simon Frei
3169212046
lib/db: Do not reset index-id when dropping device (ref #7135) (#7156)
This reverts commit 641b7aee38.
2020-11-25 22:54:05 +01:00
Simon Frei
54b50e3d52
lib/fs, lib/model: Cover more windowsyness sanitizing paths (fixes #7075) (#7158) 2020-11-25 22:52:46 +01:00
André Colomb
e6595c1ab9
lib/model: Simplify access to Folder and Device configuration. (#7151)
Use the accessor methods with a device argument instead of looking up
in a temporary map of the same data.
2020-11-24 22:20:50 +01:00
Simon Frei
100ef10d84
lib/model: Don't send locally changed on recv-enc (fixes #7137) (#7147) 2020-11-24 21:49:45 +01:00
Simon Frei
5d2c83a7e9
lib/protocol: Fix OOR panic on recv-only folders (#7143) 2020-11-23 18:37:27 +01:00
Simon Frei
8ebd893349
lib/db: Store versions for last successful db migration (#7140) 2020-11-23 18:31:32 +01:00
Simon Frei
db1f20603a
lib/model: Retain index info for new folders/devs (ref #7100) (#7133) 2020-11-20 15:53:13 +01:00
Simon Frei
24af89c8e2
all: Refactor preparing configuration (#7127) 2020-11-20 14:21:54 +01:00
Simon Frei
641b7aee38
lib/db: Reset index-id when dropping device (#7135) 2020-11-20 14:17:09 +01:00
Simon Frei
b43eccf2fe
lib/model: Never send unpaused folder without index info (#7134) 2020-11-20 14:13:50 +01:00
Simon Frei
e11b309379
lib/model, lib/util: Replace IsComplete with NoRestartErr (ref #6947) (#7126) 2020-11-18 13:43:57 +01:00
Simon Frei
2d3a535ced
lib/model: Handle cluster-config before folder start (fixes #7122) (#7123) 2020-11-17 15:30:21 +01:00
Simon Frei
9524b51708
all: Implement suture v4-api (#6947) 2020-11-17 13:19:04 +01:00
Simon Frei
e8fc465ea8
lib/config: Use correct var in MaxConcurrentIncomingRequestKiB() (#7121) 2020-11-16 16:51:51 +01:00
Jakob Borg
cca73de6a1 lib/osutil: Consider sync() best effort (fixes #7117) 2020-11-14 09:23:27 +01:00
Simon Frei
b8051fb37e
lib/model: Prevent test deadlock (#7110) 2020-11-10 12:32:07 +01:00
Simon Frei
1d3b9876f6
lib/api, lib/db: Add file debug endpoint (#7095) 2020-11-10 09:24:45 +01:00
Rahmi Pruitt
2f6a25a56f
gui: Add advance config port mapping to gui (fixes #4824) (#7017) 2020-11-10 09:24:11 +01:00
Jakob Borg
846b265430
lib/tlsutil: Add O and OU to generated certificates (fixes #7108) (#7109) 2020-11-09 17:02:56 +01:00
Simon Frei
31559e908b
all: Add untrusted folders behind feature flag (ref #62) (#7055) 2020-11-09 15:33:32 +01:00
Simon Frei
54643e86b5
lib/model: Fix locking when resending cluster-configs (#7107) 2020-11-09 14:05:21 +01:00
Simon Frei
1f1729ba43
lib/model: Add done chan to track folder-lifetime (fixes #6664) (#7094) 2020-11-09 09:05:48 +01:00
Simon Frei
d4ce0dfd84
lib/model: Send indexes for newly shared folder (fixes #7098) (#7100) 2020-11-09 08:58:46 +01:00
Simon Frei
cc9ea9db89
lib/folder: Clear pull errors when nothing is needed anymore (#7093) 2020-11-06 14:22:20 +01:00
Simon Frei
a08a1b6998
lib/api: Fix debug endpoints (ref #7001) (#7092) 2020-11-06 14:21:37 +01:00
Simon Frei
d0ccea0404
lib/config: Sanity checks on MaxConcurrentWrites (ref #7064) (#7069) 2020-11-03 19:09:32 +01:00
Simon Frei
a38b370c8d
lib/ur: Fix panics in failure-reporting (fixes #7090) (#7091) 2020-11-03 12:29:33 +01:00
Jakob Borg
7892547873
lib: Remove USE_BADGER experiment (#7089)
This removes the switch for using a Badger database, because it has bugs
that it seems there is no interest in fixing, and no actual bug tracker
to track them in.

It retains the actual implementation for the sole purpose of being able
to do the conversion back to LevelDB if anyone is actually running with
USE_BADGER. At some point in a couple of versions we can remove the
implementation as well.
2020-11-03 09:10:35 +01:00
Simon Frei
4d1bcd718c
lib/api: Fix /rest/config path and add methods to cors (ref #7001) (#7081) 2020-11-01 21:36:54 +01:00
André Colomb
7dc0c6ab43
lib/api: Allow OPTIONS method in CORS preflight request handling (ref #7017) (#7079)
This allows for checking GUI / API availability without actually doing
a GET or POST request.
2020-11-01 14:29:55 +01:00
Tomasz Wilczyński
4a616f3cb2
lib/config: Check for "msdos" when detecting FAT FS in Android (#7072) 2020-10-30 15:13:56 +01:00
Simon Frei
9189c79d74
lib/api: Add missing config mod. locks (ref #7001) (#7053) 2020-10-23 10:34:20 +02:00
Simon Frei
f0f60ba2e7
lib/api: Add /rest/config endpoint (fixes #6540) (#7001) 2020-10-22 19:54:35 +02:00
Simon Frei
1c2be84e4e
lib/model: Pass device infos as struct (fixes #7051) (#7052) 2020-10-22 13:05:31 +02:00
Simon Frei
2ba3be5e4d
lib/db: Add mechanism to repair db without schema update (ref #7044) (#7047) 2020-10-21 14:21:09 +02:00
Simon Frei
5c91723ef2
lib/model: Handle index sender lifetime (fixes #7034) (#7038) 2020-10-21 11:51:53 +02:00
Audrius Butkevicius
a17a8cd48b
lib/connections: Fix LAN addresses begin advertised even when disabled (fixes #7035) (#7045) 2020-10-21 09:16:44 +02:00
Simon Frei
27c91c57d5
lib/db: Remove need for the right dev removing globals (fixes #7036) (#7044) 2020-10-21 08:26:10 +02:00
Simon Frei
1a8c10a8d0
lib/model: Use winning version instead of merge on conflict (#6995) 2020-10-19 08:53:19 +02:00
Simon Frei
01a7ef3b0f
lib/db: Undo adding user info to panic msgs (ref #7029) (#7040) 2020-10-19 08:40:37 +02:00
Simon Frei
23c935b05a
lib/db: Ignore not found on delete in recalcGlobal (ref #7026) (#7041) 2020-10-19 08:28:53 +02:00
Simon Frei
d828adb648
cmd/stcrashreceiver, lib/db: Improve panic message handling (#7029) 2020-10-08 17:37:45 +02:00
Simon Frei
9d09fd6af3
all: Add failure reporting (fixes #7028) (#6925) 2020-10-07 10:05:13 +02:00
Simon Frei
14ae330eff
lib/db: Improve error handling checking db (ref #7026) (#7027) 2020-10-06 20:14:09 +02:00
Simon Frei
42de53c6c9
lib/model: Fix race in GlobalDirectory tests (fixes #7021) (#7022) 2020-10-03 20:46:17 +02:00
Simon Frei
48da6f0f22
lib: Use uint64 for disk stats (ref #3930) (#7019) 2020-10-02 15:22:28 +02:00
Simon Frei
a20c6ca868
lib/model, lib/protocol: Send ClusterConfig on config change (fixes #7020) (#7018) 2020-10-02 11:49:51 +02:00
Audrius Butkevicius
e027175446
all: Move remaining protos to use the vanity plugin (#7009) 2020-10-02 08:07:05 +02:00
Jakob Borg
7774932302
lib/config, lib/syncthing: Only drop delta index on upgrade if so ordered (fixes #6982) (#6983) 2020-09-30 20:16:30 +02:00
Simon Frei
1b1d38183d
lib: Remove HelloResult type, same as Hello (#7015) 2020-09-29 12:17:38 +01:00
Tomasz Wilczyński
fb3281b647
lib/fs: Add COM0 and LPT0 to invalid Windows file names (ref #7008) (#7013)
COM0 and LPT0 are not listed in the official Microsoft's documentation
at https://docs.microsoft.com/windows/win32/fileio/naming-a-file, but
in reality are also invalid in Windows Explorer.

Signed-off-by: Tomasz Wilczyński <twilczynski@naver.com>
2020-09-28 20:35:54 +02:00
Jakob Borg
e6b1f67ecf
lib/fs: Be even more strict about Windows names (ref #7008) (#7012)
Things like nul.whatever.txt are also not allowed.
2020-09-28 10:42:37 +02:00
Jakob Borg
9e0b924d57
lib/fs: Be more clear about invalid file names (ref #7010) (#7011)
Add specific errors for the failures, resulting in this rather than just
the generic "invalid filename":

[MRIW7] 08:50:50 INFO: Puller (folder default, item "NUL"): syncing: filename is invalid: name is reserved
[MRIW7] 08:50:50 INFO: Puller (folder default, item "fail."): syncing: filename is invalid: name ends with space or period
[MRIW7] 08:50:50 INFO: Puller (folder default, item "sup:yo"): syncing: filename is invalid: name contains reserved character
[MRIW7] 08:50:50 INFO: default: Failed to sync 3 items
2020-09-28 10:22:50 +02:00
Jakob Borg
df99237a7f
lib/model: Disallow reserved names, and names ending with period on Windows (fixes #7008) (#7010)
Like we already do with names ending in space or containing invalid
characters.
2020-09-28 08:25:43 +01:00
Simon Frei
8452fd2ab4
lib/model, lib/scanner: Prevent races aborting scans (fixes #6994) (#6997) 2020-09-25 11:27:44 +02:00
Tobias Klauser
c390565eef
lib/fs: Use file clone ioctl wrappers and types from golang.org/x/sys/unix (#7000)
Use the IoctlFileClone and IoctlFileCloneRange ioctl wrappers and the
FileCloneRange type provided by golang.org/x/sys/unix instead of
locally implementing them. This also allows to re-enable the code for
ppc/ppc64/ppc64le again (see commit 758a1a6a37 ("lib/fs: Disable ioctl
on ppc (fixes #6898) (#6901)")) since golang.org/x/sys/unix internally
uses the correct FICLONE and FICLONERANGE values depending on $GOARCH.
2020-09-24 10:29:32 +02:00
Simon Frei
8210466b03
lib/model: Consider case conflicts when checking to be deleted items (#6986) 2020-09-12 07:45:50 +02:00
greatroar
3e24d82513
lib/fs: Optimize UnicodeLowercase (#6979)
Most notably, it now detects all-lowercase files and returns these
as-is. The tests have been expanded with two cases and are now used
as a benchmark (admittedly a rather trivial one).

name                           old time/op    new time/op    delta
UnicodeLowercaseMaybeChange-8    4.59µs ± 2%    4.57µs ± 1%    ~     (p=0.197 n=10+10)
UnicodeLowercaseNoChange-8       3.26µs ± 1%    3.09µs ± 1%  -5.27%  (p=0.000 n=9+10)
2020-09-11 09:16:10 +02:00
Simon Frei
08bebbe59b
lib/db, lib/syncthing: Don't repair DB on upgrade, but on error (fixes #6917) (#6971) 2020-09-10 10:54:41 +02:00
Simon Frei
c5c23ed10f
lib/model: Consider existing file when handling symlink on windows (#6977) 2020-09-10 10:52:38 +02:00
Simon Frei
286698ccb1
lib/model: Handle symlink deletion on windows (fixes #6972) (#6976) 2020-09-10 08:25:44 +02:00