Commit Graph

218 Commits

Author SHA1 Message Date
greatroar 49488c0e71
all: Clean up fmt.Errorf usage (#8309) 2022-04-27 20:30:13 +02:00
greatroar bd0acd04b1
lib/protocol: Use one mutex for rawConnect.awaiting and nextID (#8198)
Having a separate mutex for the three or four instructions needed to
fetch and increment nextID means the overhead exceeds the cost of this
operation.  nextID is now handled inside the critical section for
awaiting instead, while the more expensive channel creation has been
moved outside it.

This is mostly a simplification, though it may have minor performance
benefits in some situations. The single-threaded sender benchmark shows
no significant difference:

name               old speed      new speed      delta
RequestsRawTCP-8   55.3MB/s ± 7%  56.6MB/s ± 6%   ~     (p=0.190 n=10+10)
RequestsTLSoTCP-8  20.5MB/s ±20%  20.8MB/s ± 8%   ~     (p=0.604 n=10+9)
2022-02-28 09:13:30 +01:00
greatroar a0fd619df3
lib/protocol: Require at least 3.125% savings from compression (#8133)
* lib/protocol: Require at least 3.125% savings from compression

The new lz4 library doesn't need its output buffer to be the maximum
size, unlike the old one (which would allocate if it weren't). It can
take a buffer that is of a smaller size and will report if compressed
data can fit inside the buffer (with a small chance of reporting a false
negative). Use that property to our advantage by requiring compressed
data to be at most n-n/32 = .96875*n bytes long for n input bytes.

* lib/protocol: Remove unused receivers

To make DeepSource happy.

* lib/protocol: Micro-optimize lz4Compress

Only write the length if compression was successful. This is a memory
write, so the compiler can't reorder it.

Only check the return value of lz4.CompressBlock.  Length-zero inputs
are always expanded by LZ4 compression (the library documents this),
so the check on len(src) isn't needed.
2022-01-24 19:36:58 +01:00
Jakob Borg b6d1e16b4e
lib/protocol: Switch to a newer lz4 package (#8122) 2022-01-17 18:52:43 +01:00
André Colomb 0cba3154f0
lib/model: Remove bogus fields from connections API endpoint (fixes #8103) (#8104)
* lib/model: Remove bogus fields from connections API endpoint.

Switch the returned data type for the /rest/system/connections element
"total" to use only the Statistics struct.  The other fields of the
ConnectionInfo struct are not populated and misleading.

* Lowercase JSON field names.

* lib/model: Get rid of ConnectionInfo.MarshalJSON().

It was missing the StartedAt field from the embedded Statistics
struct.  Just lowercasing the JSON attribute names can be done just as
easily with annotations.

* lib/model: Remove bogus startedAt field from totals.

Instead of using the Statistics type with one field empty, just switch
to a free-form map with the three needed fields.
2022-01-10 10:26:45 +01:00
greatroar ae70046b49 lib/protocol: Remove unused sorting boilerplate 2021-11-27 15:35:07 +01:00
Jakob Borg 4b750b6dc3
all: Remove usage of deprecated io/ioutil (#7971)
As of Go 1.16 io/ioutil is deprecated. This replaces usage with the
corresponding functions in package os and package io.
2021-11-22 08:59:47 +01:00
greatroar 8f8e8a9285 lib/protocol: Simplify codeToError, errorToCode
Also be explicit about the fact that ErrNoError is nil. That name isn't
used anywhere outside this file.
2021-10-22 18:40:40 +02:00
greatroar 46082f194c lib/protocol: Eliminate nativeModel on Unix 2021-10-22 18:40:40 +02:00
Simon Frei c94b797f00
lib/protocol: Preserve sequence decrypting fileinfos (fixes #7994) (#7995) 2021-10-06 10:26:54 +02:00
Jakob Borg 92eaf52c21 lib/protocol: Test to lock down encryption key derivation
So that we don't inadvertently change how these things work.
2021-10-04 14:36:28 +02:00
greatroar eafb40460d lib/discover, lib/protocol: Buffer allocation 2021-09-06 15:30:56 +02:00
Jakob Borg d47745a86b
all: Update build constraints to Go 1.17 style (#7894) 2021-08-17 10:10:41 +02:00
Simon Frei 50aacdf1f0
lib/protocol: Ensure correct blocksize on enc. fileinfo (ref #7861) (#7870) 2021-08-04 23:12:01 +02:00
greatroar bd363fe0b7
lib/protocol: Write uncompressible messages uncompressed (#7790) 2021-06-27 17:59:30 +02:00
Simon Frei 857caf3637
lib/connections: Trigger dialer when connection gets closed (#7753) 2021-06-17 13:57:44 +02:00
Simon Frei 5bdb6798a9
all: Regenerate proto (#7696) 2021-05-19 13:30:20 +02:00
greatroar 3746c899b7
build: List go:generate tools in tools.go (#7599) 2021-05-08 12:52:06 +02:00
Simon Frei 1a00ea7c6e
lib: Prevent using protocol method with native path (fixes #7557) (#7563) 2021-04-11 15:29:43 +02:00
Simon Frei 0dcd9794d4
lib/protocol: Deterministic encrypted version (fixes #7533) (#7538) 2021-03-31 08:59:15 +02:00
Simon Frei 924b96856f
lib: Handle adding enc folders on an existing conn (fixes #7509) (#7510) 2021-03-22 21:50:19 +01:00
Jakob Borg 8ef504f745
all: Simplify some method calls (#7499)
strings.Replace(a, b, c, -1) -> strings.ReplaceAll(a, b, c)

(Go 1.12) and who knows what was up with that dialQueue.Sort() thing.
2021-03-17 23:12:26 +01:00
Simon Frei d2d4fcc1df
lib/protocol: Improve messages when an error occurs receiving (ref #7466) (#7470) 2021-03-15 19:14:09 +01:00
Simon Frei f80ee472c2
lib/protocol: Set invalid flag on encrypted file infos (fixes #7466) (#7467) 2021-03-13 16:57:36 +01:00
Jakob Borg 4d979a1ce9
all: Truncate some timestamps (fixes #7457) (#7459)
This truncates times meant for API consumption to second precision,
where fractions won't typically matter or add any value. Exception to
this is timestamps on logs and events, and of course I'm not touching
things like file metadata.

I'm not 100% certain this is an exhaustive change, but it's the things I
found by grepping and following the breadcrumbs from lib/api...

I also considered general-but-ugly solutions, like having the API
serializer itself do reflection magic or even regexps on returned
objects, but decided against it because aurgh...
2021-03-12 10:35:10 +01:00
greatroar c00520281b
lib/protocol: Optimize FileKey (#7440) 2021-03-07 18:44:21 +01:00
Simon Frei 3d91f7c975
lib: Use counterfeiter to mock interfaces in tests (#7375) 2021-03-03 08:53:50 +01:00
greatroar 56b5352f64
all: Use crypt/rand through its buffered version, but not in benchmarks (#7420) 2021-03-02 19:17:20 +01:00
greatroar ffcaffa32f
lib/protocol: Optimize encrypted filename handling + make it more strict (#7408) 2021-02-27 08:57:12 +01:00
Simon Frei 0ffd80f380
lib/protocol: Alwasy return buffers to the pool (#7409) 2021-02-27 08:55:51 +01:00
Simon Frei c0f353c0e8
lib: Do not set ModifiedBy on meta only changes (#7345) 2021-02-08 15:30:39 +01:00
Jakob Borg 3b7a57d108
lib/protocol: Hide repeated data blocks in a given file (#7319) 2021-02-02 20:15:14 +01:00
Audrius Butkevicius a7d9268e4d
lib/model: Make /browse endpoint return sane objects (#7306) 2021-02-01 09:27:34 +01:00
Jakob Borg b13b15758d
lib/connections, lib/model: Track last connection duration (ref #7223) (#7242)
This adds a statistic to track the last connection duration per device.
It isn't used for much in this PR, but it's available for #7223 to use
in deciding how to order device connection attempts (deprioritizing
devices that just dropped our connection the last time).
2021-01-05 17:45:07 +01:00
Simon Frei c845e245a1
lib: Close underlying conn in protocol (fixes #7165) (#7212) 2020-12-21 11:40:51 +01:00
Simon Frei 8fd6b1d428
lib/protocol: Handle slashified paths in IsEncryptedParent (fixes #7184) (#7187) 2020-12-09 18:16:14 +01:00
Simon Frei bbb22c8c80
lib/protocol: Send Close message on read error (#7141) 2020-11-27 11:31:20 +01:00
Simon Frei 5d2c83a7e9
lib/protocol: Fix OOR panic on recv-only folders (#7143) 2020-11-23 18:37:27 +01:00
Simon Frei 31559e908b
all: Add untrusted folders behind feature flag (ref #62) (#7055) 2020-11-09 15:33:32 +01:00
Simon Frei f0f60ba2e7
lib/api: Add /rest/config endpoint (fixes #6540) (#7001) 2020-10-22 19:54:35 +02:00
Simon Frei a20c6ca868
lib/model, lib/protocol: Send ClusterConfig on config change (fixes #7020) (#7018) 2020-10-02 11:49:51 +02:00
Audrius Butkevicius e027175446
all: Move remaining protos to use the vanity plugin (#7009) 2020-10-02 08:07:05 +02:00
Simon Frei 1b1d38183d
lib: Remove HelloResult type, same as Hello (#7015) 2020-09-29 12:17:38 +01:00
Jakob Borg 674fca3868
lib/db, lib/protocol: Never need empty-version entries (fixes #6961) (#6962)
Avoid havoc when discovering locally-deleted-upgraded files during repair / need calculation...

Co-authored-by: Simon Frei <freisim93@gmail.com>
2020-09-07 20:18:25 +02:00
Jakob Borg b7986801cd cmd/stindex: Print more hashes and versions 2020-09-04 09:10:33 +02:00
Audrius Butkevicius d507d932b8
all: Use protobuf to generate config structs (fixes #6734) (#6900) 2020-08-25 08:11:14 +02:00
Simon Frei ebbe1abe28
lib/protocol: Include WeakHash in BlocksHash (#6799) 2020-06-29 09:33:07 +02:00
Simon Frei d7fc7008af
lib/protocol: Add BlocksHash to FileInfo string (#6802) 2020-06-27 07:44:33 +02:00
Simon Frei 80ada1bb6c
lib/protocol: Don't warn when dropping deleted invalid item (fixes #6795) (#6796) 2020-06-25 20:24:31 +02:00
greatroar dc145bfad7
lib/protocol: Use sha256.Sum256 in NewDeviceID (#6775)
This is shorter, skips two allocations, makes the function inlineable
and is safer, since the compiler now check whether
DeviceIDLength == sha256.Size.
2020-06-21 20:38:06 +02:00