Commit Graph

1077 Commits

Author SHA1 Message Date
Simon Frei 42bd42df5a lib/db: Do all update operations on a single item at once (#5441)
To do so the BlockMap struct has been removed. It behaves like any other prefixed
part of the database, but was not integrated in the recent keyer refactor. Now
the database is only flushed when files are in a consistent state.
2019-01-23 10:22:33 +01:00
Simon Frei a371b15398 lib/db: Various polish (#5471)
naming: buf -> keyBuf
dedup: use getFileTrunc
manually inline insertFile
2019-01-20 10:24:39 +01:00
Simon Frei 00fa77dd47 lib/db: Consistent use of buffers (#5470) 2019-01-20 08:47:20 +01:00
Simon Frei df4d754197 lib/db: Minor polish (#5469) 2019-01-19 20:26:46 +01:00
Simon Frei f3d735c56a lib/db: Fix, optimize and extend benchmarks (#5467) 2019-01-19 20:24:44 +01:00
Simon Frei 1d99db9bc6 lib/db: Deduplicate getFile* code (#5468) 2019-01-19 20:21:58 +01:00
Audrius Butkevicius 96bd691f55 lib/fs: Skip some tests on OpenBSD (fixes #5077) (#5466) 2019-01-19 08:28:57 +01:00
Simon Frei 22e133cce6 lib/db: Deduplicate comparing old and new items (#5465) 2019-01-18 21:19:56 +00:00
Simon Frei 1f87b874af lib/db: Add "dirty" function terminology to getGlobal (ref #5462) (#5463) 2019-01-18 13:01:39 +01:00
Jakob Borg 1e69997ecd
lib/db: Fix iterating sequence index (fixes #5340) (#5462)
There was a problem in iterating the sequence index that could result
in missing updates. The issue is that while the index was (correctly)
iterated in a snapshot, the actual file infos were read dirty outside of
the snapshot. This fixes this by doing the reads inside the snapshot,
and also updates a couple of other places that did the same thing more
or less harmfully (I didn't investigate).

To avoid similar issues in the future I did some renaming of the
getFile* methods - the ones in a transaction are just getFile, while the
ones directly on the database are variants of getFileDirty to highlight
what's going on.
2019-01-18 11:34:18 +01:00
Jakob Borg 76af0cf07b
lib/protocol: Remove support for v0.13 hello messages (#5461) 2019-01-17 20:48:43 +01:00
Jakob Borg b01edca420
all: Update protobuf package 1.0.0 -> 1.2.0 (#5452)
Also adds a few file global options to keep the generated code similar
to what we already had.
2019-01-14 11:53:36 +01:00
Simon Frei c87411851d lib/protocol: Fix potential deadlock when closing connection (ref #5440) (#5442) 2019-01-14 08:32:37 +01:00
Simon Frei 51f65bd23a lib/model: Reset pull errors when succeeding (fix #5450) (#5451) 2019-01-14 08:30:52 +01:00
Jakob Borg 93ae9a1c2e lib/logger: Clean up LOGGER_DISCARD handing to unbreak tests
When using newLogger() with an io.Writer, that writer should in fact be
used...
2019-01-13 10:22:01 +01:00
Simon Frei 0b03b6a9ec lib/model: Improve filesystem operations during tests (fixes #5422)
* lib/fs, lib/model: Improve filesystem operations during tests (fixes #5422)

Introduces MustFilesystem that panics on errors and should be used for operations
during testing which must never fail.
Create temporary directories outside of testdata.

* don't do a filesystem, just a wrapper around os for testing

* fix copyright
2019-01-11 12:56:05 +00:00
Simon Frei 24ffd8be99 all: Send Close BEP msg on intentional disconnect (#5440)
This avoids waiting until next ping and timeout until the connection is actually
closed both by notifying the peer of the disconnect and by immediately closing
the local end of the connection after that. As a nice side effect, info level
logging about dropped connections now have the actual reason in it, not a generic
timeout error which looks like a real problem with the connection.
2019-01-09 17:31:09 +01:00
Jakob Borg 1e71b00936
lib/model: Sanitize paths used for auto accepted folders (fixes #5411) (#5435) 2019-01-05 18:10:02 +01:00
Jakob Borg 5503175854
lib/logger: Strip control characters from log output (fixes #5428) (#5434) 2019-01-05 11:31:02 +01:00
Simon Frei 158559023e lib/db: Fix sequence updating for remote invalid items (#5420)
* lib/db: Fix sequence updating for remote invalid items

* fix for the unit test introduced in the previous commit

* lib/db: Polish blockmap
2019-01-04 20:19:10 +01:00
Jakob Borg 04070b4848
lib/logger: Add test for stack level (#5430) 2019-01-04 17:51:58 +01:00
Simon Frei 47e08797cb lib/model: Don't pull if ignores failed to load and cleanup (#5418) 2019-01-01 10:17:14 +01:00
Simon Frei 643cfe2e98 lib/config: Revert #5415 (#5417)
This reverts commit 9d075781add838200cb1e50537aa48d3ac1916f4:
"cmd/syncthing: Improve messages when free space is running out (#5415)"
2018-12-30 21:57:41 +01:00
Simon Frei 8bb9878f26 lib/model: Check folder context before setting error state (#5416) 2018-12-30 21:56:16 +01:00
Maurizio Tomasi 9d075781ad cmd/syncthing: Improve messages when free space is running out (#5415) 2018-12-29 20:48:20 +01:00
Jakob Borg 41469c5393 lib/rc: Remove dead lock (fixes #5403) 2018-12-23 09:11:15 +01:00
Simon Frei 4783294a09 lib/model: Don't pass nil *Matcher to performFinish (fixes #5401) (#5402) 2018-12-22 21:58:17 +01:00
Simon Frei 99c9d65ddf lib/scanner: Check ignore patterns before reporting error (fixes #5397) (#5398) 2018-12-21 12:08:15 +01:00
Simon Frei fc81e2b3d7 lib/model: Do folder watch operations under lock (fixes #5392) (#5395) 2018-12-21 12:06:21 +01:00
Simon Frei 2626143fc5 lib/rc: Fix hangups when Syncthing process ends unexpectedly (#5383) 2018-12-21 11:49:04 +01:00
Jakob Borg 944ddcf768
all: Become a Go module (fixes #5148) (#5384)
* go mod init; rm -rf vendor

* tweak proto files and generation

* go mod vendor

* clean up build.go

* protobuf literals in tests

* downgrade gogo/protobuf
2018-12-18 12:36:38 +01:00
Simon Frei 3cc8918eb4 lib/db: Defer unlock to avoid hangup on panic (#5388) 2018-12-17 14:59:09 +01:00
Simon Frei c40c9a8d6a
lib/scanner: Don't report error on missing items (fixes #5385) (#5387) 2018-12-17 14:52:15 +01:00
Jakob Borg abb3fb8a31
lib/events: Become a service (fixes #5372) (#5373)
Here the event Logger is rewritten as a service with a main loop instead
of mutexes. This loop has a select with essentially two legs: incoming
events, and subscription changes. When both are possible select will
chose one randomly, thus ensuring that in practice unsubscribes will
happen timely and not block the system.
2018-12-13 13:42:28 +01:00
Jakob Borg fc860df514 lib/watchaggregator: Properly unsubscribe from events when stopping (ref #5372) (#5374) 2018-12-13 08:11:51 +01:00
Simon Frei c934918347 lib/scanner: Fix empty path on normalization error (fixes #5369) (#5370) 2018-12-12 12:09:39 +01:00
Simon Frei a09079ed25 all: Display list of locally changed items in UI (fixes #5336) (#5337) 2018-12-11 09:59:04 +01:00
Jakob Borg 1b59960ff9
lib/model: Attempt to unflake Deregister tests (fixes #5362) (#5366) 2018-12-11 09:42:03 +01:00
Jakob Borg f04d054b5a lib/db: Document current keyspace 2018-12-10 09:55:21 +01:00
Audrius Butkevicius ff2cde469e lib/model: Allow limiting number of concurrent scans (fixes #2760) (#4888) 2018-12-05 08:40:05 +01:00
Jakob Borg 0fe4c01a28
lib/db: Fix broken test on Windows (#5347) 2018-12-05 08:24:14 +01:00
Jakob Borg e89c4c053a lib/db: Simple case sensitivity test 2018-11-26 17:58:00 +01:00
Simon Frei 2f9840ddae lib: Introduce fs.IsParent (fixes #5324) (#5326) 2018-11-22 11:16:45 +01:00
Jakob Borg c0a26c918a
lib/scanner, vendor: Update github.com/chmduquesne/rollinghash (fixes #5334) (#5335)
Updates the package and fixes a test that depended on the old behavior
of Write() being equivalent to Reset()+Write() which is no longer the
case. The scanner already did resets after each block write, so this is
fine.
2018-11-22 08:50:06 +01:00
Simon Frei db8777c29e lib/model: Add test for #5323 (#5325) 2018-11-13 08:36:16 +00:00
Simon Frei 33bed5b1ec lib/model: Don't compare permissions if IgnorePerms is true (fixes #5323) (#5322) 2018-11-13 08:54:49 +01:00
Simon Frei 4f27bdfc27 lib/model, lib/protocol: Handle request concurrency in model (#5216) 2018-11-13 08:53:55 +01:00
Simon Frei 603da2dce2 cmd/syncthing, lib/relay: Fixes regarding stopping of services (#5293) 2018-11-07 11:05:07 +01:00
Simon Frei d510e3cca3 all: Display errors while scanning in web UI (fixes #4480) (#5215) 2018-11-07 11:04:41 +01:00
Simon Frei 53b0f36be6 lib/fs: Use os.FileMode.String for fs.FileMode (#5302) 2018-10-31 12:49:50 +01:00