Commit Graph

234 Commits

Author SHA1 Message Date
Jakob Borg 4253f22680 lib/scanner: Use fs.Filesystem for all operations
One more step on the path of the great refactoring. Touches rwfolder a
little bit since it uses the Lstat from fs as well, but mostly this is
just on the scanner as rwfolder is scheduled for a later refactor.

There are a couple of usages of fs.DefaultFilesystem that will in the
end become a filesystem injected from the top, but that comes later.

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/4070
LGTM: AudriusButkevicius, imsodin
2017-04-01 09:04:11 +00:00
Jakob Borg bdb56d91b9 lib/model: Honor umask when creating folder directories on Unix (fixes #2519)
Doesn't change the behavior on Windows.

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/4076
2017-03-31 07:51:23 +00:00
Jakob Borg 5c27796471 lib/model, lib/scanner: Properly ignore symlinks on Windows (fixes #4035)
Adds a unit test to ensure we don't scan symlinks on Windows. For the
rwfolder, trusts that the logic in the invalid check is correct and that
the check is actually called from the need loop.

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/4042
2017-03-18 00:25:47 +00:00
Audrius Butkevicius 0da0774ce4 lib/connections: Add KCP support (fixes #804)
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3489
2017-03-07 12:44:16 +00:00
Simon Frei c20d612736 cmd/syncthing, lib/model: Handle rel/abs paths for config/protected paths (fixes #3183)
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3995
2017-03-04 07:49:48 +00:00
HairyFotr c56c48a777 all: Correct various typos
Skip-check: authors

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/4005
2017-02-25 08:12:13 +00:00
Jakob Borg 89f8be40c6 lib/model: Use ignore patterns in remote completion calculation
Basically, if we don't care about the sync status of the file we should
not tag someone else out of sync because they don't have the latest
version. This solves *my* "Syncing - 100%" scenario at least.

The reason this happens seems to be like this, in my situation. I have
three devices, connected in a "line": A-B-C. A is a Mac and litters
.DS_Store files everywhere. I've ignored these, but some escaped into
the folders before I did so. I've also ignored them on B and C but at
different stages. B was flagging C as out of sync, because at the point
the ignores were introduced C had a lower version of .DS_Store than A.
Now none of them are sending updates about it any more since it's
ignored...

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3981
2017-02-10 11:22:09 +00:00
Simon Frei 22903df2c1 lib/model: Meaningful error messages for paused folders
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3977
LGTM: AudriusButkevicius, calmh
2017-02-09 20:29:56 +00:00
Jakob Borg f7fc0c1d3e all: Update license url to https (ref #3976) 2017-02-09 08:04:16 +01:00
Audrius Butkevicius 72de47df00 lib/model: Increase in-flight buffer from 2mb to 8mb, download 2 files at a time
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3972
2017-02-08 20:00:55 +00:00
Jakob Borg c4ba580cbb all: Remove symlink support on Windows, SymlinksEnabled config
After this change,

- Symlinks on Windows are always unsupported. Sorry.

- Symlinks are always enabled on other platforms. They are just a small
  file like anything else. There is no need to special case them. If you
  don't want to sync some symlinks, ignore them.

- The protocol doesn't differentiate between different "types" of
  symlinks. If that distinction ever does become relevant the individual
  devices can figure it out by looking at the destination when they
  create the link.

It's backwards compatible in that all the old symlink types are still
understood to be symlinks, and the new SYMLINK type is equivalent to the
old SYMLINK_UNKNOWN which was always a valid way to do it.

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3962
LGTM: AudriusButkevicius
2017-02-07 08:34:24 +00:00
Audrius Butkevicius 67acef1794 lib/weakhash, lib/model, cmd/syncthing: Decide if to use weakhash on startup (fixes #3938)
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3949
2017-02-06 10:27:11 +00:00
Audrius Butkevicius dd78177ae0 scanner: Allow disabling weak hash in scanning (fixes #3891)
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3905
2017-01-23 13:50:32 +00:00
Simon Frei dbb3a34887 lib/ignore: Centralize handling of temporary filenames (fixes #3899)
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3901
LGTM: calmh, AudriusButkevicius
2017-01-17 07:33:48 +00:00
Audrius Butkevicius 1a1e35d998 lib/osutil: Replace IsDir with TraversesSymlink (fixes #3839)
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3883
LGTM: calmh
2017-01-10 07:09:31 +00:00
Adel Qalieh 8d2a31e38e lib/model: Remove syncthing-specific files (fixes #3819)
Syncthing adds some hidden files when a folder is added, but there is currently
no equivalent cleanup procedure. This change is conservative as not to
accidentally cause data loss.

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3874
2017-01-07 17:05:30 +00:00
Audrius Butkevicius 29d010ec0e lib/model, lib/weakhash: Hash using adler32, add heuristic in puller
Adler32 is much faster, and the heuristic avoid the obvious cases where it
will not help.

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3872
2017-01-04 21:04:13 +00:00
Jakob Borg ec62888539 lib/connections: Allow on the fly changes to rate limits (fixes #3846)
Also replaces github.com/juju/ratelimit with golang.org/x/time/rate as
the latter supports changing the rate on the fly.

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3862
2017-01-02 11:29:20 +00:00
Jakob Borg 0a118d2979 lib/config, lib/model: Temporarily disable bad tests (ref #3834, #3843) 2017-01-01 13:27:18 +01:00
Jakob Borg f5a310ad64 Revert "lib/model: Handle filename conflicts on Windows."
This reverts commit 01e50eb3fa.
2016-12-23 11:10:58 +01:00
Unrud 01e50eb3fa lib/model: Handle filename conflicts on Windows.
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3810
LGTM: calmh
2016-12-22 23:04:53 +00:00
Audrius Butkevicius bab7c8ebbf all: Add folder pause, make pauses permanent (fixes #3407, fixes #215, fixes #3001)
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3520
2016-12-21 18:41:25 +00:00
Nathan Morrison 0725e3af38 all: Add a global change list
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3694
2016-12-21 16:35:20 +00:00
Jakob Borg dd7bb6c4b8 lib/model: Fix tests, clean up pool usage in protocol 2016-12-21 14:53:45 +01:00
Jakob Borg d41c131364 build: Enable gometalinter "gosimple" check, improve build.go 2016-12-21 14:53:45 +01:00
Jakob Borg 744c2e82b5 build: Enable gometalinter "staticcheck" check 2016-12-21 14:53:45 +01:00
Jakob Borg ead7281c20 build: Enable gometalinter "unused" check 2016-12-21 14:53:45 +01:00
AudriusButkevicius aa3ef49dd7 lib/model: Fix lock order
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3823
2016-12-21 12:22:18 +00:00
Jakob Borg 5c067661f4 lib/model: Consistently show folder description in startup messages
Since we anyway need the folderConfig for this I'm skipping the copying
of all it's attributes that rwfolder did and just keeping the original
around instead.

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3825
2016-12-21 11:23:20 +00:00
Jakob Borg 226da976dc lib/model: Allow empty subdirs in scan request (fixes #3829)
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3833
2016-12-21 10:33:07 +00:00
Jakob Borg 2f770f8bfb lib/config, lib/protocol: Improve folder description with empty label 2016-12-19 10:12:06 +01:00
Jakob Borg f67c5a2fd6 lib/model: Don't send symlinks to old devices that can't handle them (fixes #3802)
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3811
2016-12-17 19:48:33 +00:00
Heiko Zuerker 398c356f22 lib/model: Clarify master terminology (fixes #2679)
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3793
2016-12-16 22:23:35 +00:00
Audrius Butkevicius 542b76f687 lib/model: Moar sleep
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3807
2016-12-16 12:05:27 +00:00
Jakob Borg abb8a1914a lib/model: Accept scan requests of paths ending in slash (fixes #3804)
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3805
2016-12-16 11:21:22 +00:00
Audrius Butkevicius 0582836820 lib/model, lib/scanner: Efficient inserts/deletes in the middle of the file
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3527
2016-12-14 23:30:29 +00:00
Jakob Borg 3582783972 lib/model, lib/osutil: Verify target directory before pulling / requesting
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3798
2016-12-13 10:24:10 +00:00
Jakob Borg 5070d52f2f lib/model: Add benchmark for model.Request() 2016-12-09 23:14:56 +01:00
Jakob Borg 7b07ed6580 lib/model, lib/protocol, lib/scanner: Include symlink target in index, pull symlinks synchronously
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3792
2016-12-09 18:02:18 +00:00
Lars K.W. Gohlke 8ce9b026e9 lib/model: Minor cleanups
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3765
2016-12-06 08:54:04 +00:00
Jakob Borg e3cf718998 lib/ignore: Add central check for internal files, used in scanning, pulling and requests
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3779
2016-12-01 14:00:11 +00:00
Jakob Borg 63194a37f6 lib/model: Double check results in filepath.Join where needed
Wherever we have untrusted relative paths, make sure they are not
escaping their folder root.

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3776
2016-12-01 12:35:11 +00:00
Unrud cabe94552a lib/model: Prevent collisions in the progressemitter registry
Using filepath.Join can cause collisions. The folder ID could be something
like ".." or "../..".

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3778
2016-12-01 12:34:20 +00:00
Jakob Borg 48a229a0cd lib/model: Temp names from all platforms should be recognized as such
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3777
LGTM: AudriusButkevicius
2016-11-30 21:23:24 +00:00
Jakob Borg e4db86836b lib/model: Locking in the request test 2016-11-30 13:11:06 +01:00
Jakob Borg 913a85c571 lib/model: Add simple file syncing test
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3772
2016-11-30 09:32:28 +00:00
Jakob Borg ed4f6fc4b3 lib/connections, lib/model: Connection service should expose a single interface
Makes testing easier, which we'll need

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3771
2016-11-30 07:54:20 +00:00
Jakob Borg 91568a173a lib/model: Remove ineffectual assignment in test 2016-11-24 11:33:27 +01:00
Jakob Borg 8559e20237 lib/osutil: Don't chmod in atomic file creation (fixes #2472)
Instead, trust (and test) that the temp file has appropriate permissions
from the start. The only place where this changes our behavior is for
ignores which go from 0644 to 0600. I'm OK with that.

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3756
2016-11-23 14:06:08 +00:00
Jakob Borg 26730eb083 lib/model: Fix test that relies on ignore reloading 2016-11-23 14:42:29 +01:00