Commit Graph

177 Commits

Author SHA1 Message Date
Majed Abdulaziz
48245effdf lib/model, lib/stats: Keep track of folder's last scan time (ref #3143)
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3250
LGTM: calmh, AudriusButkevicius
2016-06-02 19:26:52 +00:00
Audrius Butkevicius
92a23da3ec lib/model: Make the (?d) prefix actually work
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3205
2016-05-28 04:17:34 +00:00
Jakob Borg
9bb5988b4e lib/model: Don't deadlock when returning temp index block counts
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3194
2016-05-26 09:16:08 +00:00
Audrius Butkevicius
fc173bf679 lib/model: Fix wild completion percentages
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3188
2016-05-26 06:53:27 +00:00
Audrius Butkevicius
915e1ac7de lib/model: Handle (?d) deletes of directories (fixes #3164)
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3170
2016-05-23 23:32:08 +00:00
Audrius Butkevicius
00be2bf18d lib/model: Track puller creation times (fixes #3145)
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3150
2016-05-22 10:16:09 +00:00
Jakob Borg
44290a66b7 lib/model: Leave temp file in place when final rename fails (fixes #3146)
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3148
2016-05-22 09:06:07 +00:00
Jakob Borg
a0ec4467fd cmd/syncthing: Emit new RemoteDownloadProgress event to track remote download progress
Without this the summary service doesn't know to recalculate completion
percentage for remote devices when DownloadProgress messages come in.
That means that completion percentage isn't updated in the GUI while
transfers of large files are ongoing. With this change, it updates
correctly.

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3144
2016-05-22 07:52:08 +00:00
Jakob Borg
1dbc310c9b cmd/syncthing: Rename event LocalDiskUpdated -> LocalChangeDetected
I think this better reflects what it means. Also tweaks the verbose
format to be more like our other things and lightly refactors the code
to not have the boolean and include the folder in the event.

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3121
2016-05-19 07:01:43 +00:00
Nate Morrison
86ca58e2a9 lib/model: Emit LocalDiskUpdated events on detecting local changes
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3055
2016-05-19 00:19:26 +00:00
Jakob Borg
2c1323ece6 lib/connections: Un-deprecate relaysEnabled (fixes #3074)
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3098
2016-05-17 00:05:38 +00:00
Jakob Borg
6f743f3138 Revert "lib/model: Emit LocalDiskUpdated events on detecting local changes"
This reverts commit 5a7fad0bcd.
2016-05-14 10:55:24 +02:00
Nate Morrison
5a7fad0bcd lib/model: Emit LocalDiskUpdated events on detecting local changes
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3055
2016-05-14 08:37:07 +00:00
Jakob Borg
b993b41847 lib/config: Minor attribute updates
As discussed in
https://github.com/syncthing/docs/pull/169

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3082
2016-05-12 08:23:18 +00:00
Jakob Borg
21e116aa45 lib/scanner: Refactor scanner.Walk API
The old usage pattern was to create a Walker with a bunch of attributes,
then call Walk() on it and nothing else. This extracts the attributes
into a Config struct and exposes a Walk(cfg Config) method instead, as
there was no reason to expose the state-holding walker type.

Also creates a few no-op implementations of the necessary interfaces
so that we can skip nil checks and simiplify things here and there.

Definitely look at this diff without whitespace.

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3060
2016-05-09 18:25:39 +00:00
Jakob Borg
1a703efa78 lib/model: Fix accounting error in rescan with multiple subs (fixes #3028)
When doing prefix scans in the database, "foo" should not be considered
a prefix of "foo2". Instead, it should match "foo" exactly and also
strings with the prefix "foo/". This is more restrictive than what the
standard leveldb prefix scan does so we add some code to enforce it.

Also exposes the initialScanCompleted on the rwfolder for testing, and
change it to be a channel (so we can wait for it from another
goroutine). Otherwise we can't be sure when the initial scan has
completed, and we need to wait for that or it might pick up changes
we're doing at an unexpected time.

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3067
2016-05-09 12:56:21 +00:00
Jakob Borg
8b7b0a03eb lib/config: Don't require restart when adding folders/devices or changing listen address
The VersioningConfig change is because it defaults to nil but gets
deserialized to map[string]string{}. Now prepare() enforces a single
representation of the empty map.

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3065
2016-05-09 11:30:19 +00:00
klemens
bd41e21c26 all: Correct spelling in comments
Skip-check: authors pr-build-mac

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3056
2016-05-08 10:54:22 +00:00
Audrius Butkevicius
674fc566bb lib/connections: Refactor
1. Removes separate relay lists and relay clients/services, just makes it a listen address
2. Easier plugging-in of other transports
3. Allows "hot" disabling and enabling NAT services
4. Allows "hot" listen address changes
5. Changes listen address list with a preferable "default" value just like for discovery
6. Debounces global discovery announcements as external addresses change (which it might alot upon starting)
7. Stops this whole "pick other peers relay by latency". This information is no longer available,
   but I don't think it matters as most of the time other peer only has one relay.
8. Rename ListenAddress to ListenAddresses, as well as in javascript land.
9. Stop serializing deprecated values to JSON

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/2982
2016-05-04 19:38:12 +00:00
Jakob Borg
09832abe50 lib/config: Change folder type attribute to a FolderType type
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3032
2016-05-04 11:26:36 +00:00
Audrius Butkevicius
eabd2fc936 lib/model: Use factories for creating folders
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3029
2016-05-04 10:47:33 +00:00
Audrius Butkevicius
29fa05ae05 lib/model: Discard download progress upon receiving an index update (fixes #2993)
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3023
2016-05-01 06:49:29 +00:00
Audrius Butkevicius
8cf3a7aeda lib/model: Prettify tests (fixes #3014)
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3016
2016-04-26 20:19:30 +00:00
Jakob Borg
b8c5cf1142 lib/model: Refactor: complete renaming of p -> f
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3015
2016-04-26 15:11:19 +00:00
Lars K.W. Gohlke
236f121c4e lib/model: Refactor out folder and folderscan types, simplify somewhat
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3007
2016-04-26 14:01:46 +00:00
Lars K.W. Gohlke
e87c1abd4e all: Clean up dead code
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3005
2016-04-22 21:15:36 +00:00
Audrius Butkevicius
4aa6ecb122 lib/model: Do not use WRONLY (ref #2584)
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/2994
2016-04-22 08:12:10 +00:00
Audrius Butkevicius
ea54525a33 lib/connections: Try not to deadlock (fixes #2987)
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/2991
2016-04-18 20:25:31 +00:00
Jakob Borg
b81c8d2e1b lib/model: Drop incoming updates for ignored items (fixes #1701)
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/2975
2016-04-18 18:35:31 +00:00
Jakob Borg
f6953624dd lib/model: Test should pass go vet inspections
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/2977
2016-04-15 11:41:18 +00:00
AudriusButkevicius
1a5f524ae4 lib/model, lib/protocol: Implement temporary indexes (fixes #950)
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/2252
2016-04-15 10:59:41 +00:00
Michael Ploujnikov
7533a61203 unifySubs: add two trivial test cases 2016-04-10 02:41:28 +00:00
Michael Ploujnikov
467d338fe4 lib/model: Scanning unknown items is OK as long as the parent is known (fixes #2915) 2016-04-09 11:25:06 +00:00
Audrius Butkevicius
4389bb037d lib/model: Add option for overwriting names on connect (fixes #2912) 2016-04-09 07:43:47 +00:00
Audrius Butkevicius
2eb8a9ef56 all: Dead code cleanup 2016-04-09 01:10:31 +00:00
Audrius Butkevicius
5a98af622d lib/ignore: Implement deletable ignores using (?d) prefix (fixes #1362) 2016-04-07 09:34:07 +00:00
Jakob Borg
cc1d122352 lib/model: Correctly detect deleted but previously ignored files as deleted 2016-04-04 11:53:55 +01:00
Jakob Borg
f5f0e46016 lib: Use bytes.Equal instead of bytes.Compare where possible 2016-03-31 15:12:46 +00:00
Michael Ploujnikov
66f480519b lib/model: Refactor out scanning method from rwfolder.Serve loop 2016-03-30 06:53:47 +00:00
Audrius Butkevicius
b4f9a55e6e protocol: Add "Hello" message at connection start, also for unauthed peers 2016-03-25 20:29:07 +00:00
Jakob Borg
aba2cc4db2 lib/model: Properly handle deleting multiple files when doing scans with subs (fixes #2851) 2016-03-18 12:16:33 +00:00
Jakob Borg
2df001fe5c lib/model: Correct handling of multiple subs when scanning (fixes #2851)
Previously the code failed in that it would return top-level plus a sub,
i.e. ["", "foo"], and it would consider "usr/lib" a prefix of
"usr/libexec" which it is not.
2016-03-18 08:28:44 +00:00
Jakob Borg
c439c543d0 tests: messagediff argument order should be expected, actual
So that the diff describes the changes that happened in actual as
compared to expected. The opposite is confusing.
2016-03-17 08:03:29 +01:00
Jakob Borg
17517bcc3d Don't show restart prompt when changing folder label (fixes #2840) 2016-03-16 12:18:21 +01:00
Lars K.W. Gohlke
16c3d39fd2 Add folder label in addition to ID (fixes #966)
An auto generated ID is suggested on folder creation to reduce conflicts with
folders created on other devices.
2016-03-11 09:48:46 +00:00
Audrius Butkevicius
a8ffde6f21 Add deps 2016-03-06 20:32:10 +00:00
Michael Ploujnikov
49601a63c8 Model.internalScanFolder: Don't ignore special .stfolder and .stignore files.
Fixes #2151.

Since Walk.walkAndHashFiles ignores .stfolder and .stignore, they will
never be found by fs.Get(protocol.LocalDeviceID, sub) in
Model.internalScanFolder. As a result, when asked to scan those subs
we end up scanning the whole folder.
2016-01-22 23:27:47 -05:00
Michael Ploujnikov
6c33188af3 Model.internalScanFolderSubs: Scan only requested subs.
This reverts the change introduced in 9b9fe0d Reduce scanning effort.
That commit caused us to automatically ignore the basename of the
specified subs and instead scan closest known root folder. For
example, in a folder that looks like:

Sync/
├── 00
│   ├── one
│   ├── three
│   └── two
├── 01
│   ├── one
│   ├── three
│   └── two
├── 02
│   ├── one
│   ├── three
│   └── two
└── one

calling '/rest/db/scan?folder=default&sub=01' called filepath.Walk on
the whole Sync/ folder instead of just the desired subfolder. This
contradicts the scan behavior promised by the documentation.

This is related to #2151.
2016-01-22 23:27:38 -05:00
Jakob Borg
11d4986517 Humanize serialization of version vectors (again) 2016-01-20 11:14:08 -08:00
Jakob Borg
6c1faa4bdb Don't require restart for usage reporting changes (fixes #2704) 2016-01-18 10:06:31 -08:00
Jakob Borg
2919b76947 The "OK" log level is silly and should not exist 2016-01-16 23:04:41 +01:00
Jakob Borg
acaf134dfe Don't crash on folder remove while pulling (fixes #2705) 2016-01-16 21:42:32 +01:00
Audrius Butkevicius
c4c6df179b Handle race within the job queue (fixes #1263) 2016-01-16 17:20:21 +00:00
Jakob Borg
8b3d75b339 Undo the hash algorithm additions; retain flag checks 2016-01-12 14:35:00 +01:00
Jakob Borg
acdddc0b79 Don't leak sendIndexes on disconnect (fixes #2589)
Adds a Closed() method on protocol.Connection and clears up
wireformatConnection a little too.
2016-01-11 17:57:25 +01:00
Jakob Borg
096b2d73cd Don't conflict copy conflict copies (fixes #2605) 2016-01-03 21:16:31 +01:00
Jakob Borg
0db80710aa Detect nonstandard hash algo and stop folder (ref #2314) 2016-01-01 20:14:31 +01:00
Audrius Butkevicius
80b9a3e00b Don't warn about failed ignores if folder unhealthy (fixes #2630) 2016-01-01 12:59:13 +00:00
Anderson Mesquita
6d94a3be05 Refactor rwfolder tests
This creates a few utility functions to avoid repetition and removes
some redundant checks.
2015-12-22 23:43:07 -05:00
Anderson Mesquita
3f94e70488 WIP: Consider tempfile when checking for free space (fixes #2598)
Checks the existing blocks that can be reused when downloading a file so
that it only requires the space corresponding to the missing blocks.
This will prevent syncthing from claiming the folder doesn't have enough
space when resuming download of large files after they have been
partially downloaded.
2015-12-21 13:36:08 -05:00
Jakob Borg
03935b2d64 Don't verify free space for files when folder MinDiskFreePct==0 (fixes #2600) 2015-12-19 13:59:08 +01:00
Lode Hoste
cc0b9e5088 Additional output on insufficient error (ref #2580) 2015-12-15 08:45:33 +01:00
Nate Morrison
86e8e5199e Filenames added to audit log in the LocalIndexUpdated event (fixes #2549) 2015-12-04 08:42:00 +01:00
buinsky
05a31c2686 Fix deleting folders on WinXP (fixes #2522) 2015-11-30 19:09:36 +03:00
Audrius Butkevicius
4474d200b0 Fix symlinks (fixes #2524) 2015-11-28 20:23:08 +00:00
Jakob Borg
d46f267663 Handle sparse files (fixes #245) 2015-11-21 17:58:09 +01:00
Jakob Borg
9cf653d673 Rename Model.clusterConfig to Model.generateClusterConfig 2015-11-17 12:10:08 +01:00
Jakob Borg
9fbdb6b305 Cancel a running scan 2015-11-13 15:30:21 +01:00
Jakob Borg
73285cadb6 Remove folder without restart (fixes #2262) 2015-11-13 13:32:52 +01:00
AudriusButkevicius
f7edd36931 Use virtualMtime when deciding if a file is up to date 2015-11-12 03:40:06 +00:00
Mike Boone
342036408e Fix typos. 2015-11-11 21:20:34 -05:00
Zillode
593632045d Correct commentary for ConnectionStats
Related to https://github.com/syncthing/syncthing-android/issues/473 and 944d9c84a0
ConnectionStats returns connection information for all devices, even when disconnected.
2015-11-09 23:58:06 +01:00
Jakob Borg
59565fd1d1 Woops 2015-11-07 11:25:00 +01:00
Jakob Borg
55592137a2 Use constructor functions for FolderConfiguration and DeviceConfiguration 2015-11-07 09:50:04 +01:00
Jakob Borg
686f91777c Don't force rescan dirs and symlinks
We can't look for changed modtime on these as we don't track the modtime
to start with.
2015-11-04 19:53:07 +00:00
Jakob Borg
2a4fc28318 We should pass around db.Instance instead of leveldb.DB
We're going to need the db.Instance to keep some state, and for that to
work we need the same one passed around everywhere. Hence this moves the
leveldb-specific file opening stuff into the db package and exports the
dbInstance type.
2015-10-31 12:35:30 +01:00
Jakob Borg
e1a8ea7dec Rescan changed files before pulling on top of them (fixes #2416) 2015-10-29 09:12:37 +01:00
Audrius Butkevicius
5b03c2d949 Remove dead code 2015-10-25 20:46:09 +00:00
Audrius Butkevicius
f629a998a0 Change errNoDevice message to something more human 2015-10-25 13:27:26 +00:00
Jakob Borg
c19eff4872 Revive remote client version in the GUI 2015-10-22 08:53:28 +02:00
Jakob Borg
c268e4ad1b Also keep GlobalSize in RAM 2015-10-21 21:58:48 +02:00
Jakob Borg
d4f81e8791 Keep LocalSize data in RAM 2015-10-21 21:58:48 +02:00
Matt Burke
c2a5e180b8 Emit warning when sync could overwrite configuration
Overwriting configuration files is likely to happen if a
user syncs their home directories across computers. In this
case, the biggest risk is that all nodes will end up with
the same certificate and thus Device ID.

When the model prepares a folder for syncing, it checks to
see if the configuration files this instance is using are
getting synced. If the are getting synced, and they aren't
getting ignored, a warning is emitted. The model is used
so that when a new folder is added dynamically, a warning
is also emitted.

This will not prevent a user from shooting themselves in
the foot, and will not cover all cases (e.g. symlinks).
It should provide _something_ for many users in this
situation to go on, though.
2015-10-20 12:22:27 -04:00
Jakob Borg
0f9fa9507e Tests must use locking to avoid race (fixes #2394) 2015-10-20 08:51:31 +02:00
Jakob Borg
90e0141ac5 Request() should return protocol errors 2015-10-19 15:14:41 +02:00
Jakob Borg
8435a8678e Don't validate requests against the database 2015-10-19 15:14:41 +02:00
Jakob Borg
6578ffe2c9 Merge pull request #2320 from AudriusButkevicius/proto
More proto changes
2015-10-18 08:47:30 +02:00
Audrius Butkevicius
afc917b582 Fix tests 2015-10-17 09:48:41 +01:00
Audrius Butkevicius
9f4cd7716e Add more information about the folders to ClusterConfig 2015-10-17 09:46:46 +01:00
Jakob Borg
910a7c619a Merge pull request #2381 from AudriusButkevicius/maxcon
Allow limiting max conflicts (fixes #2282)
2015-10-17 09:10:31 +09:00
Audrius Butkevicius
a323d85d32 Add more information about the device to ClusterConfig 2015-10-16 19:40:12 +01:00
Audrius Butkevicius
491a33de0b Move device name into the protocol messages 2015-10-16 19:40:12 +01:00
Audrius Butkevicius
752533489a Allow limiting max conflicts (fixes #2282) 2015-10-16 19:26:38 +01:00
Jakob Borg
368cd44558 Fix race conditions in model tests 2015-10-14 14:41:16 +09:00
Audrius Butkevicius
d985ed553a Make puller pause configurable 2015-10-10 11:36:09 +01:00
Jakob Borg
76af9ba53d Implement facility based logger, debugging via REST API
This implements a new debug/trace infrastructure based on a slightly
hacked up logger. Instead of the traditional "if debug { ... }" I've
rewritten the logger to have no-op Debugln and Debugf, unless debugging
has been enabled for a given "facility". The "facility" is just a
string, typically a package name.

This will be slightly slower than before; but not that much as it's
mostly a function call that returns immediately. For the cases where it
matters (the Debugln takes a hex.Dump() of something for example, and
it's not in a very occasional "if err != nil" branch) there is an
l.ShouldDebug(facility) that is fast enough to be used like the old "if
debug".

The point of all this is that we can now toggle debugging for the
various packages on and off at runtime. There's a new method
/rest/system/debug that can be POSTed a set of facilities to enable and
disable debug for, or GET from to get a list of facilities with
descriptions and their current debug status.

Similarly a /rest/system/log?since=... can grab the latest log entries,
up to 250 of them (hardcoded constant in main.go) plus the initial few.

Not implemented in this commit (but planned) is a simple debug GUI
available on /debug that shows the current log in an easily pasteable
format and has checkboxes to enable the various debug facilities.

The debug instructions to a user then becomes "visit this URL, check
these boxes, reproduce your problem, copy and paste the log". The actual
log viewer on the hypothetical /debug URL can poll regularly for new log
entries and this bypass the 250 line limit.

The existing STTRACE=foo variable is still obeyed and just sets the
start state of the system.
2015-10-03 18:09:53 +02:00
Jakob Borg
36c746bd9f Create missing directories 2015-10-01 09:43:16 +02:00
Jakob Borg
38a3bf3ada Merge pull request #2317 from simplypeachy/patch-1
Add missing parameter in disk space warning
2015-10-01 08:10:36 +02:00
Jakob Borg
be2ca0ea22 Revert "Case insensitive renames, part 1" 2015-09-30 21:40:04 +02:00
Jakob Borg
ddfebb17cf Case insensitive renames, part 1 2015-09-30 12:41:29 +02:00