Commit Graph

779 Commits

Author SHA1 Message Date
Simon Frei e70003737b lib/fs: make watcher tests even more darwin slowness resistant
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/4439
2017-10-20 15:59:18 +00:00
Michael Ploujnikov f98c21b68e all: Add filesystem notification support
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3986
2017-10-20 14:52:55 +00:00
Audrius Butkevicius fb7264a663 cmd/syncthing: Enable KCP by default
Also, use upstream library, as my changes have been merged.
2017-10-17 23:17:10 +01:00
Audrius Butkevicius becbb3b123 lib/model: Fix tests not to require Go 1.9 2017-10-12 22:36:20 +01:00
Jakob Borg 2f08f8021f lib/beacon: Don't exit after a single write failure (fixes #4414)
With VPNs and stuff we can get a single failure on an interface that
supposedly supports broadcasts without it being fatal.

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/4415
2017-10-12 07:13:44 +00:00
Audrius Butkevicius 9d3f3847ed lib/model: Fix removal of paused folders, improve tests (fixes #4405) 2017-10-12 08:23:33 +02:00
Audrius Butkevicius 74c8d34805 lib/model: Centralize error reporting, modified files are errors (fixes #4392) 2017-10-12 08:23:33 +02:00
Audrius Butkevicius 5ec1490be0 lib/fs: Ignore directory fsync failures 2017-10-12 08:22:29 +02:00
Audrius Butkevicius 2760d032ca cmd/syncthing: Add more stats to usage reports (ref #3628)
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/4347
2017-10-12 06:16:46 +00:00
Jakob Borg a365ae51c4 lib/model: Hide temporary files on Windows while they are in use (fixes #4382)
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/4394
2017-09-23 13:47:51 +00:00
Jakob Borg 97222797a0 lib/config: Make folder marker change non fatal 2017-09-23 15:29:55 +02:00
Simon Frei e588bb29b9 lib/model: remove unused folderFs member from Model
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/4389
2017-09-21 09:03:36 +00:00
Jakob Borg 2dd9450793 lib/protocol, vendor: Import luhn code directly
I've changed it incompatibly to fix a correctness bug. Nonetheless, we
should remain incorrect indefinitely.
2017-09-20 21:34:32 +02:00
Jakob Borg 3ee12464b4 lib/config, lib/model: Make sure to hide our special files (fixes #4382)
The folder marker conversion forgot to hide the .stfolder. This adds
that, for those who have not yet been converted.

Also adds Hide() calls to the folder start, to mend historical
unhidedness. (I'm sure this will upset someone who is manually managing
their .stignores in the other direction...)

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/4384
2017-09-20 06:49:04 +00:00
Simon Frei 1088eb12ea lib/model: Fix logging inconsistencies (fixes #4375)
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/4376
2017-09-18 11:56:19 +00:00
Jakob Borg 429cc20eb7 cmd/syncthing: Add some common security releated HTTP headers (fixes #4360)
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/4361
2017-09-10 08:28:12 +00:00
Audrius Butkevicius e85ce7c94e lib/model: Support removing paused folders (fixes #4357)
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/4358
LGTM: imsodin, calmh
2017-09-09 15:08:59 +00:00
Jakob Borg 283c8d95e2 lib/protocol: Comment typo 2017-09-08 15:25:14 +02:00
wangguoliang a9aa375109 lib/fs: Comment typo
Skip-check: authors

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/4354
2017-09-07 09:40:46 +00:00
Simon Frei f7d2c58783 lib/model: Deduplicate folder loops
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/4352
LGTM: AudriusButkevicius, calmh
2017-09-07 06:17:47 +00:00
Jakob Borg 4d3e0de4ba cmd/syncthing, lib/sha256: Skip CPU benchmarks when user decided (fixes #4348)
When STHASHING is set, don't benchmark as it's already decided. If weak
hashing isn't set to "auto", don't benchmark that either.

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/4349
2017-09-06 06:55:47 +00:00
Simon Frei 9dbc509996 lib/ignore: Consistent behaviour for nil *Matcher
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/4310
2017-09-06 06:39:18 +00:00
Simon Frei c41aaad3bb lib/ignore: Ignore duplicate lines in .stignore
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/4350
LGTM: AudriusButkevicius, calmh
2017-09-04 12:46:19 +00:00
Jakob Borg 9682bbfbda lib/protocol: Optimize luhn and chunk functions
These functions were very naive and slow. We haven't done much about
them because they pretty much don't matter at all for Syncthing
performance. They are however called very often in the discovery server
and these optimizations have a huge effect on the CPU load on the
public discovery servers.

The code isn't exactly obvious, but we have good test coverage on all
these functions.

benchmark                 old ns/op     new ns/op     delta
BenchmarkLuhnify-8        12458         1045          -91.61%
BenchmarkUnluhnify-8      12598         1074          -91.47%
BenchmarkChunkify-8       10792         104           -99.04%

benchmark                 old allocs     new allocs     delta
BenchmarkLuhnify-8        18             1              -94.44%
BenchmarkUnluhnify-8      18             1              -94.44%
BenchmarkChunkify-8       44             2              -95.45%

benchmark                 old bytes     new bytes     delta
BenchmarkLuhnify-8        1278          64            -94.99%
BenchmarkUnluhnify-8      1278          64            -94.99%
BenchmarkChunkify-8       42552         128           -99.70%

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/4346
2017-09-03 10:26:12 +00:00
Audrius Butkevicius 9e6a1fdcd4 vendor: Update kcp, removes closeConn (fixes #4343) 2017-09-02 16:11:48 +02:00
Simon Frei 55e0ac3e24 lib/model: Make puller retrying and logging less aggressive
Currently all errors during pulling and the first of these errors again on
finishing are logged to info. Besides that the errors logged when finishing
are stored in f.errors. This PR moves all logging during pulling to the debug
channel (they might still be relevant in some obscure debugging case) and
uses the stored errors to log the main error per fail when all pulling
iterations are done and failed.

Additional instead of trying 11 times it now only tries 3 times.

This is the first part of what is discussed here:
https://forum.syncthing.net/t/reduce-verboseness-of-puller/10261

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/4338
2017-09-02 06:05:55 +00:00
Audrius Butkevicius cbcc3ea132 lib/connections: Use our own fork of kcp (fixes #4063)
This updates kcp and uses our own fork which:

1. Keys sessions not just by remote address, but by remote address +
conversation id 2. Allows not to close connections that were passed directly
to the library. 3. Resets cache key if the session gets terminated.

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/4339
LGTM: calmh
2017-09-02 06:04:35 +00:00
Audrius Butkevicius ab132ff6fe lib: Folder marker is now a folder
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/4341
LGTM: calmh
2017-09-02 05:52:38 +00:00
Jakob Borg a8145e187f lib/model: Fixup because I suck and shouldn't commit directly to master 2017-08-31 10:49:17 +02:00
Jakob Borg e33fa10115 lib/model: Use same batch size constants in db updater as for protocol 2017-08-31 10:47:39 +02:00
Jakob Borg 4b6e7e7867 lib/tlsutil: Remove undesired bufio from UnionedConnection (ref #4245)
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/4335
2017-08-31 07:34:48 +00:00
Simon Frei ddf6d64faa lib/model: Create folders via newFolder
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/4329
2017-08-25 19:47:01 +00:00
Audrius Butkevicius a69ba18f62 lib/model: Some platforms do not support usage checks (fixes #4321)
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/4322
2017-08-22 18:13:58 +00:00
Simon Frei 0ca0e3e9bd lib/model: GetIgnores: Don't return error for no .stignore file
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/4311
2017-08-22 06:48:25 +00:00
Audrius Butkevicius 0a96a1150b lib/model, lib/ignores: Properly handle out of folder ignores and free space checks (fixes #4313, fixes #4314)
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/4318
2017-08-22 06:45:00 +00:00
Audrius Butkevicius b8c249cddc lib/model: Move stale scan check info finisher (ref #4305, fix #3742)
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/4317
2017-08-22 06:42:09 +00:00
Audrius Butkevicius e8ba6d4771 lib/upnp: Fix build
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/4316
2017-08-21 11:41:40 +00:00
Audrius Butkevicius 606fce09ca lib/upnp: Disable confusing messages
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/4312
2017-08-21 10:03:25 +00:00
Audrius Butkevicius 3d8b4a42b7 all: Convert folders to use filesystem abstraction
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/4228
2017-08-19 14:36:56 +00:00
Simon Frei ab8c2fb5c7 lib/model: Fix race in GetIgnores (fixes #4300)
In addition this function returned an error when .stignore file was not
present, which is perfectly valid. Also removed inconsistent nil check in
ignores.go (only relevant for tests) and adjusted walk.go to do what it says
in comments (check if Matcher is nil) to prevent nil deref in tests.

Originally reported in:
https://forum.syncthing.net/t/reason-for-panic-maybe-too-little-ram/10346

Regression from #3996

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/4301
2017-08-12 17:10:43 +00:00
Jakob Borg 1fc2ab444b lib/model: Remove ineffective symlink recovery attempt 2017-08-08 15:30:28 +02:00
Jakob Borg fa5c890ff6 lib/versioner: Clean the versions dir of symlinks, not the full folder
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/4289
2017-08-08 13:13:08 +00:00
Jakob Borg a3c17f8f81 lib/model: Disable symlink attack test on Windows 2017-08-08 08:05:24 +02:00
Jakob Borg f1f21bf220 lib/model, lib/versioner: Prevent symlink attack via versioning (fixes #4286)
Prior to this, the following is possible:

- Create a symlink "foo -> /somewhere", it gets synced
- Delete "foo", it gets versioned
- Create "foo/bar", it gets synced
- Delete "foo/bar", it gets versioned in "/somewhere/bar"

With this change, versioners should never version symlinks.
2017-08-07 07:57:10 +02:00
Simon Frei a04b92332f gui, lib/config: Add default path for new folders (fixes #2157)
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/4192
2017-07-20 13:16:54 +00:00
HairyFotr 0ad10b0fee all: Typos
Skip-check: authors

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/4263
2017-07-20 13:10:46 +00:00
Simon Frei b75c9f2bbb lib/ignores: Don't add text from includes to lines (fixes #4249)
Otherwise all the lines from includes will be shown in the web UI instead of
just the #include ... line. This problem was introduced in #3996.

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/4248
LGTM: calmh
2017-07-06 11:44:11 +00:00
Siyuan Liu 03c678a810 lib/versioner: Interpret versions path relative to folder path (fixes #4188)
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/4243
2017-07-03 14:50:51 +00:00
Jakob Borg add10c98fa lib/db: Fix test for Go 1.9's smarter time.Time values 2017-06-28 14:36:04 +02:00
Jakob Borg db1dc9985a lib/upgrade: 0.x to 1.0 is a minor upgrade
This removes the special handling of minor versions as major when the
actual major is zero, and adds the special case that upgrades from 0.x
to 1.x are considered minor. 0.x to 2.x or 1.x to 2.x etc are still
considered major.

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/4226
2017-06-25 14:17:43 +00:00