Commit Graph

703 Commits

Author SHA1 Message Date
Jakob Borg 987718baf8 vendor: Update github.com/gogo/protobuf
Also tweaks the proto definitions:

 - [packed=false] on the block_indexes field to retain compat with
   v0.14.16 and earlier.

 - Uses the vendored protobuf package in include paths.

And, "build.go setup" will install the vendored protoc-gen-gogofast.
This should ensure that a proto rebuild isn't so dependent on whatever
version of the compiler and package the developer has installed...

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3864
2017-01-03 00:16:21 +00:00
Jakob Borg ec62888539 lib/connections: Allow on the fly changes to rate limits (fixes #3846)
Also replaces github.com/juju/ratelimit with golang.org/x/time/rate as
the latter supports changing the rate on the fly.

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3862
2017-01-02 11:29:20 +00:00
Jakob Borg 6809d38cde lib/protocol: Revert protobuf encoder changes in v0.14.17 (fixes #3855)
The protobuf encoder now produces packed arrays for things like []int32,
which is actually correct according to the proto3 spec. However
Syncthing v0.14.16 and earlier doesn't support this. This reverts the
encoding change, but keeps the updated decoder so that we are both more
compatible with other proto3 implementations and can move to the updated
encoder in the future.

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3856
2017-01-01 17:19:00 +00:00
Jakob Borg 0a118d2979 lib/config, lib/model: Temporarily disable bad tests (ref #3834, #3843) 2017-01-01 13:27:18 +01:00
Jakob Borg c69c3c7c36 lib/sha256: Smoke test the implementation on startup (hello OpenSUSE!) 2016-12-28 12:15:51 +01:00
Jakob Borg f5a310ad64 Revert "lib/model: Handle filename conflicts on Windows."
This reverts commit 01e50eb3fa.
2016-12-23 11:10:58 +01:00
Unrud 01e50eb3fa lib/model: Handle filename conflicts on Windows.
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3810
LGTM: calmh
2016-12-22 23:04:53 +00:00
Audrius Butkevicius bab7c8ebbf all: Add folder pause, make pauses permanent (fixes #3407, fixes #215, fixes #3001)
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3520
2016-12-21 18:41:25 +00:00
Nathan Morrison 0725e3af38 all: Add a global change list
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3694
2016-12-21 16:35:20 +00:00
Jakob Borg dd7bb6c4b8 lib/model: Fix tests, clean up pool usage in protocol 2016-12-21 14:53:45 +01:00
Jakob Borg d41c131364 build: Enable gometalinter "gosimple" check, improve build.go 2016-12-21 14:53:45 +01:00
Jakob Borg 47f22ff3e5 build: Enable gometalinter "unconvert" check 2016-12-21 14:53:45 +01:00
Jakob Borg 744c2e82b5 build: Enable gometalinter "staticcheck" check 2016-12-21 14:53:45 +01:00
Jakob Borg ead7281c20 build: Enable gometalinter "unused" check 2016-12-21 14:53:45 +01:00
AudriusButkevicius aa3ef49dd7 lib/model: Fix lock order
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3823
2016-12-21 12:22:18 +00:00
Jakob Borg 5c067661f4 lib/model: Consistently show folder description in startup messages
Since we anyway need the folderConfig for this I'm skipping the copying
of all it's attributes that rwfolder did and just keeping the original
around instead.

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3825
2016-12-21 11:23:20 +00:00
Jakob Borg 226da976dc lib/model: Allow empty subdirs in scan request (fixes #3829)
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3833
2016-12-21 10:33:07 +00:00
AudriusButkevicius 9e0afb7d8a lib/connections: Support setting traffic class (fixes #3790) 2016-12-21 11:01:15 +01:00
Jakob Borg d7d5687faa lib/protocol: Warnln should have been Warnf 2016-12-21 10:43:12 +01:00
Jakob Borg 2f770f8bfb lib/config, lib/protocol: Improve folder description with empty label 2016-12-19 10:12:06 +01:00
Jakob Borg b8c1c0e048 cmd/syncthing: Enable better crypto, print negotiated cipher suite
This adds support for AES_256_GCM_SHA384 (in there since Go 1.5, a bit
of a shame we missed it) and ChaCha20-Poly1305 (if built with Go 1.8;
ignored on older Gos).

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3822
2016-12-18 21:07:44 +00:00
Jakob Borg 9c67bd2550 lib/connections: Fix port fixup in Go 1.8 (fixes #3817)
The test for the error string is fragile, and the error string changed
in Go 1.8 so the relevant part is no longer a prefix. This covers it
with a test though, so it should be fine in the future as well.

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3818
2016-12-18 11:28:18 +00:00
Jakob Borg f67c5a2fd6 lib/model: Don't send symlinks to old devices that can't handle them (fixes #3802)
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3811
2016-12-17 19:48:33 +00:00
Heiko Zuerker 398c356f22 lib/model: Clarify master terminology (fixes #2679)
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3793
2016-12-16 22:23:35 +00:00
Audrius Butkevicius 542b76f687 lib/model: Moar sleep
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3807
2016-12-16 12:05:27 +00:00
Jakob Borg abb8a1914a lib/model: Accept scan requests of paths ending in slash (fixes #3804)
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3805
2016-12-16 11:21:22 +00:00
Audrius Butkevicius 0582836820 lib/model, lib/scanner: Efficient inserts/deletes in the middle of the file
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3527
2016-12-14 23:30:29 +00:00
Jakob Borg dde9d4c9eb lib/rc: Remove pause to aggregate events (ref #3527) 2016-12-14 10:24:36 +01:00
Jakob Borg 3582783972 lib/model, lib/osutil: Verify target directory before pulling / requesting
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3798
2016-12-13 10:24:10 +00:00
Jakob Borg 5070d52f2f lib/model: Add benchmark for model.Request() 2016-12-09 23:14:56 +01:00
Jakob Borg 7b07ed6580 lib/model, lib/protocol, lib/scanner: Include symlink target in index, pull symlinks synchronously
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3792
2016-12-09 18:02:18 +00:00
Jakob Borg a9b03de99a gui, lib/db: Correct space accounting of symlinks, for "out of sync" status 2016-12-09 10:38:36 +01:00
Lars K.W. Gohlke 8ce9b026e9 lib/model: Minor cleanups
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3765
2016-12-06 08:54:04 +00:00
Jakob Borg 2fd1dca905 lib/connections: Fix odd logging, forgot to call function 2016-12-02 12:56:14 +01:00
Jakob Borg e3cf718998 lib/ignore: Add central check for internal files, used in scanning, pulling and requests
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3779
2016-12-01 14:00:11 +00:00
Jakob Borg 3266aae1c3 lib/protocol: Apply input filtering on file names
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3775
2016-12-01 12:35:32 +00:00
Jakob Borg 63194a37f6 lib/model: Double check results in filepath.Join where needed
Wherever we have untrusted relative paths, make sure they are not
escaping their folder root.

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3776
2016-12-01 12:35:11 +00:00
Unrud cabe94552a lib/model: Prevent collisions in the progressemitter registry
Using filepath.Join can cause collisions. The folder ID could be something
like ".." or "../..".

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3778
2016-12-01 12:34:20 +00:00
Jakob Borg 48a229a0cd lib/model: Temp names from all platforms should be recognized as such
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3777
LGTM: AudriusButkevicius
2016-11-30 21:23:24 +00:00
Jakob Borg e4db86836b lib/model: Locking in the request test 2016-11-30 13:11:06 +01:00
Jakob Borg 913a85c571 lib/model: Add simple file syncing test
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3772
2016-11-30 09:32:28 +00:00
Jakob Borg ed4f6fc4b3 lib/connections, lib/model: Connection service should expose a single interface
Makes testing easier, which we'll need

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3771
2016-11-30 07:54:20 +00:00
Jakob Borg fc1430aa92 lib/fs: The interface and basicfs
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3748
2016-11-24 12:07:14 +00:00
Jakob Borg 3cde608eda lib/db: Fix ineffassign lint issue 2016-11-24 12:08:44 +01:00
Jakob Borg 91568a173a lib/model: Remove ineffectual assignment in test 2016-11-24 11:33:27 +01:00
Jakob Borg e57f5499a1 lib/sync: Remove unused struct field 2016-11-24 11:30:55 +01:00
Jakob Borg 9abb7b71a9 lib/osutil: Fix lint warning on error formatting (fixes #3760) 2016-11-24 11:20:51 +01:00
Jakob Borg 8559e20237 lib/osutil: Don't chmod in atomic file creation (fixes #2472)
Instead, trust (and test) that the temp file has appropriate permissions
from the start. The only place where this changes our behavior is for
ignores which go from 0644 to 0600. I'm OK with that.

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3756
2016-11-23 14:06:08 +00:00
Jakob Borg 26730eb083 lib/model: Fix test that relies on ignore reloading 2016-11-23 14:42:29 +01:00
Simon Frei 4160ce674d model: consistently use cfg when referring to config instance and not package
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3755
2016-11-22 23:14:20 +00:00
Jakob Borg 2dbeea21c4 lib/ignore: Don't slow down tests by sleeping 2016-11-22 22:44:04 +01:00
Jakob Borg a2b8485a89 lib/ignore: Fast reload of unchanged ignores (fixes #3394)
This changes the "seen" map that we're anyway keeping around to track
the modtimes of loaded files instead. When doing a Load() we check that
1) the file we are loading is in the modtime set, and 2) that none of
the files in the modtime set have changed modtimes. If that's the case
we do a quick return without parsing anything or clearing the cache.

This required adding two one seconds sleeps in the tests to make sure
the modtimes were updated when we expect cache reloads, because I'm on a
crappy filesystem with one second timestamp granularity. That also
proves it works...

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3754
2016-11-22 21:30:45 +00:00
Unrud f1e83a57cd lib/osutil: Remove unnecessary fsync in Copy()
Fsyncing the file has a small performance penalty and seems unnecessary. The
file will be fsynced anyway, when the changes are commited to the database.

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3749
2016-11-22 07:59:54 +00:00
Jakob Borg cc9a9fb390 lib/model, lib/protocol: Add Folder.Description() for logging (ref #3741) 2016-11-22 08:36:14 +01:00
Roman Zaynetdinov d3a251e6d9 lib/model: Log folder IDs and labels (fixes #3724)
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3741
2016-11-21 20:09:18 +01:00
Unrud 1574b7d834 lib/model: Add fsync of files and directories, option to disable (fixes #3711) 2016-11-21 18:09:51 +01:00
Jakob Borg 548a324256 lib/connections: Slow down failing listeners
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3745
2016-11-19 12:37:14 +00:00
Jakob Borg a8a0bc356a lib/model: Minor cleanup to not fondle cfg.Raw things in handleDeintroductions
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3739
2016-11-17 08:56:55 +00:00
Jakob Borg faee1d5a8d lib/model: Fix locking around introduction handling (fixes #3737)
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3738
2016-11-17 08:50:24 +00:00
Jakob Borg 3088dac33b lib/model: Clean up generateClusterConfig, fix spurious test failure by sorting 2016-11-17 07:45:45 +01:00
Jakob Borg 95c738ea28 lib/protocol: Serialize the all zeroes device ID to the empty string
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3734
2016-11-15 06:22:36 +00:00
Audrius Butkevicius 38d28c3f4a lib/relay: Close invitation channel in all error cases (fixes #3726) 2016-11-13 09:32:05 +01:00
Audrius Butkevicius f60b424d70 lib/config: Raw() -> RawCopy() 2016-11-13 09:29:35 +01:00
Audrius Butkevicius a1a91d5ef4 lib/model: Introducer can remove stuff it introduced (fixes #1015)
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3522
2016-11-13 09:29:33 +01:00
Jakob Borg 59f3d1445f Revert "lib/model: Introducer can remove stuff it introduced (fixes #1015)"
This reverts commit 0b88cf1d03.
2016-11-12 08:38:29 +01:00
Audrius Butkevicius 0b88cf1d03 lib/model: Introducer can remove stuff it introduced (fixes #1015)
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3522
2016-11-11 15:54:25 +01:00
Audrius Butkevicius 56e2ba29d0 lib/config: Subscribers get a copy of the config
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3722
2016-11-11 14:52:23 +00:00
Jakob Borg ae4cc94a9d lib/model: Fix locking order in Availability() (fixes #3634)
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3714
2016-11-08 06:38:50 +00:00
Jakob Borg 3f9b75b7b3 Revert "lib/model: Introducer can remove stuff it introduced (fixes #1015)"
This reverts commit ec2b097313.
2016-11-08 14:27:32 +08:00
Audrius Butkevicius ec2b097313 lib/model: Introducer can remove stuff it introduced (fixes #1015)
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3522
2016-11-08 00:40:48 +08:00
Audrius Butkevicius caaab462bc lib/sync: Fix broken build 2016-11-05 02:31:52 +00:00
Audrius Butkevicius da413b823b lib/sync: Add option for sasha-s/go-deadlock 2016-11-05 02:24:53 +00:00
Audrius Butkevicius 14937e7dd2 build: Fix proto builder on Windows 2016-11-03 22:06:51 +00:00
Audrius Butkevicius 3418497f3d lib/sync: Log everything... 2016-11-03 21:33:33 +00:00
Audrius Butkevicius 7fba8cf759 lib/sync: Print all lockers, add holder to RWMutex 2016-10-30 00:17:25 +01:00
Jakob Borg 0296c23685 lib/protocol: Use DeviceID in protocol messages, with custom marshalling
This makes the device ID a real type that can be used in the protobuf
schema. That avoids the juggling back and forth from []byte in a bunch
of places and simplifies the code.

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3695
2016-10-29 21:56:24 +00:00
MikeLund 74dd051d51 all: Update docs.s.n links to use https
Skip-check: authors

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3691
2016-10-27 17:02:19 +00:00
Jakob Borg 4b3adfa21c vendor: Update gobwas/glob to fix question mark handling 2016-10-23 15:47:31 +02:00
Simon Frei 7c37301c91 lib/ignore: Add directory separator to glob.Compile call
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3674
LGTM: calmh
2016-10-21 07:33:40 +00:00
Audrius Butkevicius 815588daba lib/sync, lib/model: Capture locker routine ID, print locker details on deadlock 2016-10-18 21:00:01 +01:00
Jakob Borg 4e8c8d7e2c cmd/syncthing, lib/db, lib/model: Track more detailed file/dirs/links/deleted counts 2016-10-17 23:57:43 +02:00
Jakob Borg b8a90b7eaa lib/model: Remove chatty delta index confirmation
This had its moment when making sure it worked initially, but nowadays
there is not point discussing the obvious on every folder on every
connect.
2016-10-17 09:21:58 +02:00
Benny Ng 05c37e58c1 lib/osutil: Prevent infinite Glob recursion (fixes #3577)
Skip-check: authors

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3665
2016-10-12 20:55:38 +00:00
Tim Howes fe9c2b9857 lib/ignore: Match directory contents for patterns ending in / (fixes #3639)
Appends "**" to patterns with a terminal slash, so that directory
contents are ignored, but not the directory itself.
2016-10-04 08:12:55 +09:00
Nathan Morrison d322ebd0b9 Add API service for local disk changes
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3626
LGTM: calmh, AudriusButkevicius
2016-09-28 15:54:13 +00:00
Jakob Borg 467c1b26fb cmd/syncthing, lib/config: Log errors replacing or saving config (ref #3567) 2016-09-24 09:59:09 +02:00
Jakob Borg 3cabecda04 lib/upnp: Correct the result deduplication mechanism (fixes #3578)
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3618
2016-09-24 07:33:56 +00:00
Jakob Borg d328e0fb75 cmd/syncthing: Add selectable sha256 package (fixes #3613, fixes #3614)
This adds autodetection of the fastest hashing library on startup, thus
handling the performance regression. It also adds an environment
variable to control the selection, STHASHING=standard (Go standard
library version, avoids SIGILL crash when the minio library has bugs on
odd CPUs), STHASHING=minio (to force using the minio version) or unset
for the default autodetection.

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3617
2016-09-23 19:33:54 +00:00
Jakob Borg b371b1fe34 lib/versioner: Test both spaces and parens in ext versioner paths
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3610
2016-09-18 12:24:55 +00:00
Jakob Borg 90c0a39df8 lib/versioner: Test for external versioner
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3609
2016-09-17 20:34:50 +00:00
Lars K.W. Gohlke 70c5a5dff1 lib/versioner: Rename versioner_test to simple_test
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3603
2016-09-16 11:01:43 +00:00
Jakob Borg da0b7cc7f2 lib/model: Correct lock taking order in ConnectionStats (fixes #3596)
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3597
2016-09-14 19:38:55 +00:00
Jakob Borg 139e9b144e lib/config: Fix tests for changes in previous commit 2016-09-13 22:20:22 +02:00
Jakob Borg 7fb53ec954 lib/config: Correct name of discovery-v6-4 server 2016-09-12 11:30:06 +02:00
Jakob Borg d8b5070ca8 lib/config: Update default set of discovery servers 2016-09-12 09:55:45 +02:00
Jakob Borg 5e99d38412 all: Use github.com/minio/sha256-simd
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3581
2016-09-09 09:57:51 +00:00
Jakob Borg 49910a1d85 lib/config, cmd/syncthing: Enforce localhost only connections
When the GUI/API is bound to localhost, we enforce that the Host header
looks like localhost. This can be disabled by setting
insecureSkipHostCheck in the GUI config.

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3558
2016-09-03 08:33:34 +00:00
Jakob Borg 46a143e80e lib/model: Handle deleted-then-ignored files (fixes #3502)
When files that were previously marked as deleted became ignored, we
used to do nothing at all. This changes that behavior to set the Invalid
bit (that we should rename to Ignored). This then becomes an update to
other devices that they should not trust our knowledge about the file in
question.

Read this diff without whitespace...

Tested by
- creating a bunch of files on s1
- letting them sync to s2
- shutting down s2
- deleting the files on s1 and rescanning
- adding the files to .stignore on s1 and rescanning
- starting up s2 and letting it sync
- observing the files are not deleted on s2, and it considers itself up
  to date.

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3557
2016-09-02 13:23:24 +00:00
Jakob Borg 69b7f26e4c lib/model, cmd/syncthing: Also account for deleted files in folder summary events (ref #3496)
This should probably be reflected in the GUI somewhere as well...
2016-09-02 10:45:39 +02:00
Jakob Borg 5b37d0356c lib/model, gui: Correct completion percentages when there are lots of deletes (fixes #3496)
We used to consider deleted files & directories 128 bytes large. After
the delta indexes change a bug slipped in where deleted files would be
weighted according to their old non-deleted size. Both ways are
incorrect (but the latest change made it worse), as if there are more
files deleted than remaining data in the repo the needSize can be
greater than the globalSize, resulting in a negative completion
percentage.

This change makes it so that deleted items are zero bytes large, which
makes more sense. Instead we expose the number of files that we need to
delete as a separate field in the Completion() result, and hack the
percentage down to 95% complete if it was 100% complete but we need to
delete files. This latter part is sort of ugly, but necessary to give
the user some sort of feedback.

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3556
2016-09-02 06:45:46 +00:00