Commit Graph

1915 Commits

Author SHA1 Message Date
Jakob Borg b6d1e16b4e
lib/protocol: Switch to a newer lz4 package (#8122) 2022-01-17 18:52:43 +01:00
Simon Frei 21d04b895a
lib, gui: Default ignores for new folders (fixes #7428) (#7530) 2022-01-13 23:38:21 +01:00
André Colomb 0cba3154f0
lib/model: Remove bogus fields from connections API endpoint (fixes #8103) (#8104)
* lib/model: Remove bogus fields from connections API endpoint.

Switch the returned data type for the /rest/system/connections element
"total" to use only the Statistics struct.  The other fields of the
ConnectionInfo struct are not populated and misleading.

* Lowercase JSON field names.

* lib/model: Get rid of ConnectionInfo.MarshalJSON().

It was missing the StartedAt field from the embedded Statistics
struct.  Just lowercasing the JSON attribute names can be done just as
easily with annotations.

* lib/model: Remove bogus startedAt field from totals.

Instead of using the Statistics type with one field empty, just switch
to a free-form map with the three needed fields.
2022-01-10 10:26:45 +01:00
André Colomb 5237337626
cmd/syncthing: Add --skip-port-probing (fixes #8090) (#8099)
* cmd/syncthing: Remove unnecessary function arguments.

The openGUI() function does not need a device ID to work, and there is
only one caller anyway which uses EmptyDeviceID.

The loadOrDefaultConfig() function is always called with the same
dummy values.

* cmd/syncthing: Avoid misleading info messages from monitor process.

In order to check whether panic reporting is enabled, the monitor
process utilizes the loadOrDefaultConfig() function.  In case there is
no config file yet, info messages may be logged during creation if the
config Wrapper, which is discarded immediately after.

Stop using the DefaultConfig() utility function from lib/syncthing and
directly generate a minimal config instead to avoid these.

Add comments to loadOrDefaultConfig() explaining its limited purpose.

* cmd/syncthing/generate: Always write updated config file.

Previously, an existing config file was left untouched unless either
of the --gui-user or --gui-password options was given.  Remove that
condition and simplify the checking code.

* lib/config: Factor out ProbeFreePorts().

* cmd/syncthing: Add option --skip-port-probing.

Applies to both the "generate" and "serve" subcommands, as well as the
deprecated --generate option, just as the --no-default-folder flag.
2022-01-07 11:19:17 +01:00
Gahl Saraf cc39341eb9
lib: Fix panic due to closed event subscriptions on shutdown (#8079) 2021-12-22 20:16:21 +01:00
greatroar eb857dbc45 lib/osutil: Use x/sys/windows for SetLowPriority 2021-11-27 15:35:07 +01:00
greatroar 286a25ae49 lib/upgrade: Use strings.Reader instead of bytes.Buffer 2021-11-27 15:35:07 +01:00
greatroar ae70046b49 lib/protocol: Remove unused sorting boilerplate 2021-11-27 15:35:07 +01:00
greatroar c366933416 lib/sync: Make the clock a function pointer 2021-11-27 15:35:07 +01:00
greatroar 6a9716e8a1 lib/model: Return index from deviceActivity.leastBusy
This way, we don't need a second loop over the Availabilities to remove
the selected item.
2021-11-26 12:07:43 +01:00
greatroar 8a1e54d58a lib/fs: Optimize Canonicalize
When pathSep is a constant, the compiler precomputes pathSep+pathSep and
".."+pathSep instead of emitting function calls to compute "//" and
"../". Benchmark results in lib/osutil:

name                old time/op    new time/op    delta
TraversesSymlink-8    8.86µs ± 3%    8.53µs ± 4%  -3.79%  (p=0.000 n=18+20)

name                old alloc/op   new alloc/op   delta
TraversesSymlink-8    1.06kB ± 0%    1.06kB ± 0%    ~     (all equal)

name                old allocs/op  new allocs/op  delta
TraversesSymlink-8      15.0 ± 0%      15.0 ± 0%    ~     (all equal)
2021-11-23 21:25:29 +01:00
greatroar 3e032c4da6 lib/fs: optimize Windows path checking/sanitizing
name                          old time/op    new time/op    delta
WindowsInvalidFilenameValid-8     875ns ± 1%     150ns ± 1%  -82.84%  (p=0.000 n=9+9)
WindowsInvalidFilenameNUL-8       276ns ± 4%     121ns ± 3%  -56.26%  (p=0.000 n=10+10)

name                          old alloc/op   new alloc/op   delta
WindowsInvalidFilenameValid-8     32.0B ± 0%     16.0B ± 0%  -50.00%  (p=0.000 n=10+10)
WindowsInvalidFilenameNUL-8       32.0B ± 0%     19.0B ± 0%  -40.62%  (p=0.000 n=10+10)

name                          old allocs/op  new allocs/op  delta
WindowsInvalidFilenameValid-8      2.00 ± 0%      1.00 ± 0%  -50.00%  (p=0.000 n=10+10)
WindowsInvalidFilenameNUL-8        2.00 ± 0%      2.00 ± 0%     ~     (all equal)
2021-11-23 21:25:29 +01:00
Jakob Borg 1754c93370
lib/config, lib/ignore: Write Windows line endings (fixes #7115) (#8052) 2021-11-22 09:38:24 +01:00
Jakob Borg 4b750b6dc3
all: Remove usage of deprecated io/ioutil (#7971)
As of Go 1.16 io/ioutil is deprecated. This replaces usage with the
corresponding functions in package os and package io.
2021-11-22 08:59:47 +01:00
greatroar bf89bffb0b
lib/config: Decouple VerifyConfiguration from Committer (#7939)
... and remove 8/10 implementations, which were no-ops. This saves code
and time copying configurations.
2021-11-22 08:45:29 +01:00
Jakob Borg e2288fe441
lib/relay: Send SNI when the address is a host name (fixes #8014) (#8015) 2021-11-22 08:31:03 +01:00
greatroar 8265dac127
lib/nat: Fix race condition in Mapping (#8042)
The locking protocol in nat.Mapping was racy:

* Mapping.addressMap RLock'd, but then returned a map shared between
  caller and Mapping, so the lock didn't do anything.

* Operations inside Service.{verifyExistingMappings,acquireNewMappings}
  would lock the map for every update, but that means callers to
  Mapping.ExternalAddresses can be looping over the map while the
  Service methods are concurrently modifying it. When the Go runtime
  detects that happening, it panics.

* Mapping.expires was read and updated without locking.

The Service methods now lock the map once and release the lock only when
done.

Also, subscribers no longer get the added and removed addresses, because
none of them were using the information. This was changed for a previous
attempt to retain the fine-grained locking and not reverted because it
simplifies the code.
2021-11-22 08:29:44 +01:00
André Colomb 100870e142
cmd/syncthing: Implement generate as a subcommand with optional API credential setting (fixes #8021) (#8043)
Accept a subcommand as an alternative to the --generate option.  It
accepts a custom config directory through either the --home or
--config options, using the default location if neither is given.

Add the options --gui-user and --gui-password to "generate", but not
the "serve --generate" option form.  If either is given, an existing
config will not abort the command, but rather load, modify and save it
with the new credentials.  The password can be read from standard
input by passing only a single dash as argument.

Config modification is skipped if the value matches what's already in
the config.

* cmd/syncthing: Utilize lib/locations package in generate().
Instead of manually joining paths with "magic" file names, get them
from the centralized locations helper lib.

* cmd/syncthing: Simplify logging for --generate option.
Visible change: No more timestamp prefixes.
2021-11-18 22:57:59 +01:00
Jakob Borg 12fb7f2a0a lib/model: Correct "reverting folder" log entry 2021-11-17 12:52:10 +01:00
Jakob Borg f1bf4d899a lib/model: Correct handling of fakefs cache
We looked under one cache key, then stored under another...
2021-11-17 12:52:10 +01:00
Simon Frei 591e4d8af1
gui, lib: Fix tracking deleted locally-changed on encrypted (fixes #7715) (#7726) 2021-11-10 09:46:21 +01:00
André Colomb dec6f80d2b
lib/config: Move the bcrypt password hashing to GUIConfiguration (#8028)
What hash is used to store the password should ideally be an
implementation detail, so that every user of the GUIConfiguration
object automatically agrees on how to handle it.  That is currently
distribututed over the confighandler.go and api_auth.go files, plus
tests.

Add the SetHasedPassword() / CompareHashedPassword() API to keep the
hashing method encapsulated.  Add a separate test for it and adjust
other users and tests.  Remove all deprecated imports of the bcrypt
package.
2021-11-08 13:32:04 +01:00
André Colomb ec8a748514
lib/syncthing: Clean up / refactor LoadOrGenerateCertificate() utility function. (#8025)
LoadOrGenerateCertificate() takes two file path arguments, but then
uses the locations package to determine the actual path.  Fix that
with a minimally invasive change, by using the arguments instead.
Factor out GenerateCertificate().

The only caller of this function is cmd/syncthing, which passes the
same values, so this is technically a no-op.

* lib/tlsutil: Make storing generated certificate optional.  Avoid
  temporary cert and key files in tests, keep cert in memory.
2021-11-07 23:59:48 +01:00
greatroar db15e52743
lib/api: http.Request.BasicAuth instead of custom code (#8039) 2021-11-06 12:38:08 +01:00
André Colomb 41bfb7a330
Normalize CLI options to always use two dashes. (#8037)
Consistently use double dashes and fix typos -conf, -data-dir and
-verify.

Applies also to tests running the syncthing binary for consistency.

* Fix mismatched option name --conf in cli subcommand.

According to the source code comments, the cli option flags should
mirror those from the serve subcommand where applicable.  That one is
actually called --config though.

* cli: Fix help text option placeholders.

The urfave/cli package uses the Value field of StringFlag to provide a
default value, not to name the placeholder.  That is instead done with
backticks around some part of the Usage field.

* cli: Add missing --data flag in subcommand help text.

The urfave/cli based option parsing uses a fake flags collection to
generate help texts matching the used global options.  But the --data
option was omitted from it, although it is definitely required when
using --config as well.  Note that it cannot just be ignored, as some
debug stuff actually uses the DB:

syncthing cli --data=/bar --config=/foo debug index dump
2021-11-04 08:42:55 +01:00
greatroar 807a6b1022
lib/model: Optimize jobQueue performance and memory use (#8023)
By truncating time.Time to an int64 nanosecond count, we lose the
ability to precisely order timestamps before 1678 or after 2262, but we
gain (linux/amd64, Go 1.17.1):

name                      old time/op    new time/op    delta
JobQueuePushPopDone10k-8    2.85ms ± 5%    2.29ms ± 2%  -19.80%  (p=0.000 n=20+18)
JobQueueBump-8              34.0µs ± 1%    29.8µs ± 1%  -12.35%  (p=0.000 n=19+19)

name                      old alloc/op   new alloc/op   delta
JobQueuePushPopDone10k-8    2.56MB ± 0%    1.76MB ± 0%  -31.31%  (p=0.000 n=18+13)

name                      old allocs/op  new allocs/op  delta
JobQueuePushPopDone10k-8      23.0 ± 0%      23.0 ± 0%     ~     (all equal)

Results for BenchmarkJobQueueBump are with the fixed version, which no
longer depends on b.N for the amount of work performed. rand.Rand.Intn
is cheap at ~10ns per iteration.
2021-10-29 20:20:46 +02:00
Tomasz Wilczyński 296cc1bca2 lib/model: Limit the number of default hashers on Android (ref #2220)
Like Windows and Mac, Android is also an interactive operating system.
On top of that, it usually runs on much slower hardware than the other
two. Because of that, it makes sense to limit the number of hashes used
by default there too.

Signed-off-by: Tomasz Wilczyński <twilczynski@naver.com>
2021-10-29 10:06:52 +02:00
Simon Frei 951b058952 lib/model: Set mod. time after writing trailer in shortcut (ref #7992) 2021-10-24 16:50:30 +02:00
greatroar 8f8e8a9285 lib/protocol: Simplify codeToError, errorToCode
Also be explicit about the fact that ErrNoError is nil. That name isn't
used anywhere outside this file.
2021-10-22 18:40:40 +02:00
greatroar 46082f194c lib/protocol: Eliminate nativeModel on Unix 2021-10-22 18:40:40 +02:00
Simon Frei 517667c590
lib/model: Pull when a new connection is established (fixes #8012) (#8013) 2021-10-20 18:55:22 +02:00
greatroar 7c292cc812
lib/connections: Fix and optimize registry (#7996)
Registry.Get used a full sort to get the minimum of a list, and the sort
was broken because util.AddressUnspecifiedLess assumed it could find out
whether an address is IPv4 or IPv6 from its Network method. However,
net.(TCP|UDP)Addr.Network always returns "tcp"/"udp".
2021-10-06 10:52:51 +02:00
Simon Frei c94b797f00
lib/protocol: Preserve sequence decrypting fileinfos (fixes #7994) (#7995) 2021-10-06 10:26:54 +02:00
Simon Frei 708a5c2070
lib/model: Write trailer when shortcutting on recv-enc (fixes #7991) (#7992) 2021-10-05 11:23:27 +02:00
Jakob Borg 92eaf52c21 lib/protocol: Test to lock down encryption key derivation
So that we don't inadvertently change how these things work.
2021-10-04 14:36:28 +02:00
tomasz1986 8d8f331a4a
lib/config: Always set mtime window to 2 on Android unless ext detected (ref #7963) (#7966)
The current detection is flawed, because it looks for a few specific
file systems like "msdos" or "fat" to set the mtime window, while in
reality Android seems to report names like "fuseblk", which can stand
for fat, ext4, or even f2fs.

At the moment, we set the mtime window only for a few known names used
for the fat filesystem. With this change, we take a safer approach of
always setting the time window unless we explicitly detect file systems
like ext2/ext3/ex4, which are known not to experience issues with moving
timestamps on Android.

Signed-off-by: Tomasz Wilczyński <twilczynski@naver.com>
2021-09-27 21:29:51 +02:00
greatroar 198028d627
lib/rand: Optimizations (#7964)
rand.secureSource.Uint64 no longer allocates. rand.String uses a
strings.Builder. Benchmark results on linux/amd64:

name            old time/op    new time/op    delta
SecureSource-8    69.1ns ± 3%    51.7ns ± 3%   -25.21%  (p=0.000 n=20+10)
String-8          2.66µs ± 2%    1.95µs ± 1%   -26.61%  (p=0.000 n=10+10)

name            old alloc/op   new alloc/op   delta
SecureSource-8     8.00B ± 0%     0.00B       -100.00%  (p=0.000 n=20+10)
String-8            288B ± 0%       32B ± 0%   -88.89%  (p=0.000 n=10+10)

name            old allocs/op  new allocs/op  delta
SecureSource-8      1.00 ± 0%      0.00       -100.00%  (p=0.000 n=20+10)
String-8            33.0 ± 0%       1.0 ± 0%   -96.97%  (p=0.000 n=10+10)
2021-09-26 12:15:39 +02:00
Jakob Borg 73c5184518
build: Update all dependencies (#7968)
Also add a script step for how to do this, as frankly I forget each time
what the canonical way is.

`go run build.go update-deps`
2021-09-26 12:08:23 +02:00
greatroar f96c211198
lib/db: Replace SipHash with hash/maphash (#7962) 2021-09-24 21:26:07 +02:00
greatroar a4489dec30
lib/tlsutil: Allocate UnionedConnection in one go (#7912) 2021-09-21 08:40:34 +02:00
Simon Frei 721cd740d8
lib/model: Don't reset db while folder is running (fixes #7935) (#7936) 2021-09-11 17:14:47 +02:00
greatroar de719ac409
lib/db: Inline sync.Once in releaser (#7938)
Having a pointer to a Once suggests that it is shared with other
objects, but it never is.
2021-09-10 09:58:17 +02:00
jtagcat 0ffa7f3f57
all: Clarify 'Cleaning data for folder' log message (#7937)
Instead of data (could be read as user data), use metadata.
2021-09-10 09:56:44 +02:00
greatroar 7fa141ea39
all: Unused args, retvals, assignments (#7926) 2021-09-08 00:11:16 +02:00
greatroar 13196ddd92 lib/relay/protocol: Merge two Sprintf calls 2021-09-06 15:30:56 +02:00
greatroar eafb40460d lib/discover, lib/protocol: Buffer allocation 2021-09-06 15:30:56 +02:00
Simon Frei 4e2a9bb139
lib/model: Remove encryption pw on folder stop (#7925) 2021-09-03 09:54:47 +02:00
greatroar 3b2239357f
lib/model: Add missing error assignment + Remove unused argument (#7922) 2021-09-01 22:03:06 +02:00
Ross Smith II 7501bee430
lib/model: Typo in folder_recvonly.go (#7919) 2021-09-01 07:41:18 +02:00
greatroar ed98039aa5
lib/fs: Optimize TempName + some cosmetic changes (#7911) 2021-08-29 10:47:53 +02:00