Commit Graph

189 Commits

Author SHA1 Message Date
Jakob Borg 913a85c571 lib/model: Add simple file syncing test
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3772
2016-11-30 09:32:28 +00:00
Jakob Borg ed4f6fc4b3 lib/connections, lib/model: Connection service should expose a single interface
Makes testing easier, which we'll need

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3771
2016-11-30 07:54:20 +00:00
Jakob Borg 91568a173a lib/model: Remove ineffectual assignment in test 2016-11-24 11:33:27 +01:00
Jakob Borg 8559e20237 lib/osutil: Don't chmod in atomic file creation (fixes #2472)
Instead, trust (and test) that the temp file has appropriate permissions
from the start. The only place where this changes our behavior is for
ignores which go from 0644 to 0600. I'm OK with that.

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3756
2016-11-23 14:06:08 +00:00
Jakob Borg 26730eb083 lib/model: Fix test that relies on ignore reloading 2016-11-23 14:42:29 +01:00
Simon Frei 4160ce674d model: consistently use cfg when referring to config instance and not package
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3755
2016-11-22 23:14:20 +00:00
Jakob Borg cc9a9fb390 lib/model, lib/protocol: Add Folder.Description() for logging (ref #3741) 2016-11-22 08:36:14 +01:00
Roman Zaynetdinov d3a251e6d9 lib/model: Log folder IDs and labels (fixes #3724)
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3741
2016-11-21 20:09:18 +01:00
Unrud 1574b7d834 lib/model: Add fsync of files and directories, option to disable (fixes #3711) 2016-11-21 18:09:51 +01:00
Jakob Borg a8a0bc356a lib/model: Minor cleanup to not fondle cfg.Raw things in handleDeintroductions
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3739
2016-11-17 08:56:55 +00:00
Jakob Borg faee1d5a8d lib/model: Fix locking around introduction handling (fixes #3737)
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3738
2016-11-17 08:50:24 +00:00
Jakob Borg 3088dac33b lib/model: Clean up generateClusterConfig, fix spurious test failure by sorting 2016-11-17 07:45:45 +01:00
Audrius Butkevicius f60b424d70 lib/config: Raw() -> RawCopy() 2016-11-13 09:29:35 +01:00
Audrius Butkevicius a1a91d5ef4 lib/model: Introducer can remove stuff it introduced (fixes #1015)
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3522
2016-11-13 09:29:33 +01:00
Jakob Borg 59f3d1445f Revert "lib/model: Introducer can remove stuff it introduced (fixes #1015)"
This reverts commit 0b88cf1d03.
2016-11-12 08:38:29 +01:00
Audrius Butkevicius 0b88cf1d03 lib/model: Introducer can remove stuff it introduced (fixes #1015)
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3522
2016-11-11 15:54:25 +01:00
Jakob Borg ae4cc94a9d lib/model: Fix locking order in Availability() (fixes #3634)
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3714
2016-11-08 06:38:50 +00:00
Jakob Borg 3f9b75b7b3 Revert "lib/model: Introducer can remove stuff it introduced (fixes #1015)"
This reverts commit ec2b097313.
2016-11-08 14:27:32 +08:00
Audrius Butkevicius ec2b097313 lib/model: Introducer can remove stuff it introduced (fixes #1015)
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3522
2016-11-08 00:40:48 +08:00
Audrius Butkevicius 3418497f3d lib/sync: Log everything... 2016-11-03 21:33:33 +00:00
Audrius Butkevicius 7fba8cf759 lib/sync: Print all lockers, add holder to RWMutex 2016-10-30 00:17:25 +01:00
Jakob Borg 0296c23685 lib/protocol: Use DeviceID in protocol messages, with custom marshalling
This makes the device ID a real type that can be used in the protobuf
schema. That avoids the juggling back and forth from []byte in a bunch
of places and simplifies the code.

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3695
2016-10-29 21:56:24 +00:00
MikeLund 74dd051d51 all: Update docs.s.n links to use https
Skip-check: authors

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3691
2016-10-27 17:02:19 +00:00
Audrius Butkevicius 815588daba lib/sync, lib/model: Capture locker routine ID, print locker details on deadlock 2016-10-18 21:00:01 +01:00
Jakob Borg 4e8c8d7e2c cmd/syncthing, lib/db, lib/model: Track more detailed file/dirs/links/deleted counts 2016-10-17 23:57:43 +02:00
Jakob Borg b8a90b7eaa lib/model: Remove chatty delta index confirmation
This had its moment when making sure it worked initially, but nowadays
there is not point discussing the obvious on every folder on every
connect.
2016-10-17 09:21:58 +02:00
Nathan Morrison d322ebd0b9 Add API service for local disk changes
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3626
LGTM: calmh, AudriusButkevicius
2016-09-28 15:54:13 +00:00
Jakob Borg da0b7cc7f2 lib/model: Correct lock taking order in ConnectionStats (fixes #3596)
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3597
2016-09-14 19:38:55 +00:00
Jakob Borg 46a143e80e lib/model: Handle deleted-then-ignored files (fixes #3502)
When files that were previously marked as deleted became ignored, we
used to do nothing at all. This changes that behavior to set the Invalid
bit (that we should rename to Ignored). This then becomes an update to
other devices that they should not trust our knowledge about the file in
question.

Read this diff without whitespace...

Tested by
- creating a bunch of files on s1
- letting them sync to s2
- shutting down s2
- deleting the files on s1 and rescanning
- adding the files to .stignore on s1 and rescanning
- starting up s2 and letting it sync
- observing the files are not deleted on s2, and it considers itself up
  to date.

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3557
2016-09-02 13:23:24 +00:00
Jakob Borg 69b7f26e4c lib/model, cmd/syncthing: Also account for deleted files in folder summary events (ref #3496)
This should probably be reflected in the GUI somewhere as well...
2016-09-02 10:45:39 +02:00
Jakob Borg 5b37d0356c lib/model, gui: Correct completion percentages when there are lots of deletes (fixes #3496)
We used to consider deleted files & directories 128 bytes large. After
the delta indexes change a bug slipped in where deleted files would be
weighted according to their old non-deleted size. Both ways are
incorrect (but the latest change made it worse), as if there are more
files deleted than remaining data in the repo the needSize can be
greater than the globalSize, resulting in a negative completion
percentage.

This change makes it so that deleted items are zero bytes large, which
makes more sense. Instead we expose the number of files that we need to
delete as a separate field in the Completion() result, and hack the
percentage down to 95% complete if it was 100% complete but we need to
delete files. This latter part is sort of ugly, but necessary to give
the user some sort of feedback.

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3556
2016-09-02 06:45:46 +00:00
Jakob Borg 0b1475169f lib/model: Correct virtual mtime handling (fixes #3516)
We previously set the mtime on the temp file, and then renamed it to the
real path. Unfortunately that means we'd save the real timestamp under
the under the temp name ".syncthing.foo.tmp" when the actual file that
we will look up on the next scan is "foo". This moves the Chtimes later,
ensuring that it gets recorded correctly under the right name.

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3519
2016-08-16 18:22:19 +00:00
Audrius Butkevicius 6ec4fbc82b lib/model: Add minumum interval for progress emitter (fixes #3517)
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3518
2016-08-16 18:22:01 +00:00
Jakob Borg 18cc7a663b lib: Remove osutil.Remove & osutil.RemoveAll (fixes #3513)
These are no longer required with Go 1.7. Change made by removing the
functions, doing a global s/osutil.Remove/os.Remove/.

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3514
2016-08-16 10:01:58 +00:00
Jakob Borg cf5febad47 build, cmd, lib: Minimum supported compiler version is Go 1.5 2016-08-15 08:37:32 +02:00
Jakob Borg 7776839c82 cmd/syncthing, gui: Improve completion calculation (fixes #3492)
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3493
2016-08-12 06:41:43 +00:00
Jakob Borg e52be3d83e lib/connections, lib/model: Refactor connection close handling (fixes #3466)
So there were some issues here. The main problem was that
model.Close(deviceID) was overloaded to mean "the connection was closed
by the protocol layer" and "i want to close this connection". That meant
it could get called twice - once *to* close the connection and then once
more when the connection *was* closed.

After this refactor there is instead a Closed(conn) method that is the
callback. I didn't need to change the parameter in the end, but I think
it's clearer what it means when it takes the connection that was closed
instead of a device ID. To close a connection, the new close(deviceID)
method is used instead, which only closes the underlying connection and
leaves the cleanup to the Closed() callback.

I also changed how we do connection switching. Instead of the connection
service calling close and then adding the connection, it just adds the
new connection. The model knows that it already has a connection and
makes sure to close and clean out that one before adding the new
connection.

To make sure to sequence this properly I added a new map of channels
that get created on connection add and closed by Closed(), so that
AddConnection() can do the close and wait for the cleanup to happen
before proceeding.

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3490
2016-08-10 09:37:32 +00:00
Audrius Butkevicius af3b6f9c83 lib/model, lib/config: Support "live" device removal, folder unsharing and folder configuration changes
Furthermore:
1. Cleans configs received, migrates them as we receive them.
2. Clears indexes of devices we no longer share the folder with

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3478
2016-08-07 16:21:59 +00:00
Jakob Borg ea87bcefd6 lib/protocol, lib/model: Implement high precision time stamps (fixes #3305)
This adds a new nanoseconds field to the FileInfo, populates it during
scans and sets the non-truncated time in Chtimes calls.

The actual file modification time is defined as modified_s seconds +
modified_ns nanoseconds. It's expected that the modified_ns field is <=
1e9 (that is, all whole seconds should go in the modified_s field) but
not really enforced. Given that it's an int32 the timestamp can be
adjusted += ~2.9 seconds by the modified_ns field...

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3431
2016-08-06 13:05:59 +00:00
Jakob Borg 0655991a19 lib/db, lib/fs, lib/model: Introduce fs.MtimeFS, remove VirtualMtimeRepo
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3479
2016-08-05 17:45:45 +00:00
Jakob Borg f368d2278f lib/config, lib/connections: Refactor handling of ignored devices (fixes #3470)
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3471
2016-08-05 09:29:49 +00:00
Jakob Borg 1eb6db6ca8 cmd/syncthing, lib/...: Correctly handle ignores & invalid file names (fixes #3012, fixes #3457, fixes #3458)
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3464
2016-08-05 07:13:52 +00:00
Audrius Butkevicius 08b5a7908f gui: Add one-off notifications that need to be acked
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3254
2016-08-02 08:07:30 +00:00
Jakob Borg 297240facf all: Rename LocalVersion to Sequence (fixes #3461)
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3462
2016-07-29 19:54:24 +00:00
Jakob Borg 72026db599 lib/db, lib/model: Create temp sorting database in config dir (fixes #3449)
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3454
2016-07-27 21:38:43 +00:00
Jakob Borg aafc96f58f lib/model, lib/protocol: Sequence ClusterConfig properly (fixes #3448)
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3452
2016-07-27 21:36:25 +00:00
Jakob Borg 7c7e8648ff lib/model: Trigger a puller iteration on connection (fixes #3451)
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3453
2016-07-27 21:35:41 +00:00
Jakob Borg 7aaa1dd8a3 lib/scanner: Recheck file size and modification time after hashing (ref #3440)
To catch the case where the file changed. Also make sure we never let a
size-vs-blocklist mismatch slip through.

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3443
2016-07-26 08:51:39 +00:00
Jakob Borg 6ed22d0885 lib/model: Stricter temporary file permissions
We could have a file to sync with permissions rw------- but we'd create
the temp file with rw-rw-rw- minus umask, usually rw-r--r--. This
potentially exposes private data while the file is being synced.

Similarly, when ignorePerms was set and we were reusing a temp files we
would set the permissions to rw-r--r-- explicitly, potentially
overriding a strict umask that would otherwise have had the file be
rw-------.

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3437
2016-07-25 10:18:05 +00:00
Jakob Borg 47fa4b0a2c cmd/syncthing, lib/db, lib/model, lib/protocol: Implement delta indexes (fixes #438)
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3427
2016-07-23 12:46:31 +00:00