Commit Graph

227 Commits

Author SHA1 Message Date
Jakob Borg 8bc7d259f4 Move FileIntf to files package, expose Iterator type
This is where FileIntf is used, so it should be defined here (it's not
a protocol thing, really).
2015-01-09 08:18:42 +01:00
Audrius Butkevicius 9d6ef24660 Merge pull request #1194 from syncthing/fix-1186
Use comma-ok idiom to signal files missing in database (fixes #1186)
2015-01-06 21:54:13 +00:00
Brendan Long 7a9fc6dbd3 Don't use Go 1.4 range syntax in queue_test.go, since the listed requirement is Go 1.3. 2015-01-06 15:45:58 -06:00
Jakob Borg 75d0dc251e Use comma-ok idiom to signal files missing in database (fixes #1186)
Prevents us from doing stupid things to the folder root (empty file
path) when nodes disconnect...
2015-01-06 22:40:20 +01:00
Audrius Butkevicius 5ac01a3af4 Hash blocks after receipt, try multiple peers (fixes #1166) 2015-01-03 23:21:57 +00:00
Audrius Butkevicius 56ccb5b2ab New device, folder prompts (fixes #120, fixes #330) 2015-01-03 23:06:41 +00:00
Audrius Butkevicius 9c6cb0f630 Merge pull request #1172 from syncthing/random-scanintv
Add a random perturbation to the scan interval (fixes #1150)
2015-01-02 15:25:22 +00:00
Jakob Borg 54c3caad53 Add a random perturbation to the scan interval (fixes #1150) 2015-01-02 16:16:16 +01:00
Jakob Borg 9b5e8aaf83 Repair buggy BringToFront 2015-01-02 15:54:04 +01:00
Jakob Borg 5143c09bcf Refactor / cleanup 2015-01-02 15:54:04 +01:00
Jakob Borg 2496185629 Only buffer file names, not full &FileInfo 2015-01-02 15:33:39 +01:00
Jakob Borg 34deb82aea Use slice instead of list, no map
benchmark                           old ns/op     new ns/op     delta
BenchmarkJobQueueBump               345           154498        +44682.03%
BenchmarkJobQueuePushPopDone10k     9437373       3258204       -65.48%

benchmark                           old allocs     new allocs     delta
BenchmarkJobQueueBump               0              0              +0.00%
BenchmarkJobQueuePushPopDone10k     10565          22             -99.79%

benchmark                           old bytes     new bytes     delta
BenchmarkJobQueueBump               0             0             +0.00%
BenchmarkJobQueuePushPopDone10k     1452498       385869        -73.43%
2015-01-02 15:33:39 +01:00
Jakob Borg 8f72ae9da2 Add some benchmarks 2015-01-02 15:33:39 +01:00
Audrius Butkevicius b753f01ac1 Add tests 2015-01-02 15:33:39 +01:00
Audrius Butkevicius fd0a147ae6 Add job queue (fixes #629)
Request to terminate currently ongoing downloads and jump to the bumped file
incoming in 3, 2, 1.

Also, has a slightly strange effect where we pop a job off the queue, but
the copyChannel is still busy and blocks, though it gets moved to the
progress slice in the jobqueue, and looks like it's in progress which it isn't
as it's waiting to be picked up from the copyChan.

As a result, the progress emitter doesn't register on the task, and hence the file
doesn't have a progress bar, but cannot be replaced by a bump.

I guess I can fix progress bar issue by moving the progressEmiter.Register just
before passing the file to the copyChan, but then we are back to the initial
problem of a file with a progress bar, but no progress happening as it's stuck
 on write to copyChan

I checked if there is a way to check for channel writeability (before popping)
but got struck by lightning just for bringing the idea up in #go-nuts.

My ideal scenario would be to check if copyChan is writeable, pop job from the
queue and shove it down handleFile. This way jobs would stay in the queue while
they cannot be handled, meaning that the `Bump` could bring your file up higher.
2015-01-02 15:33:39 +01:00
Jakob Borg ce4b897d0e Handle read only temp files after crash/restart 2014-12-31 13:06:28 +01:00
Jakob Borg 22f39be197 Exit before attempting to use nil variables on scanning nonexistent folder 2014-12-23 14:14:05 +01:00
Jakob Borg cadbb6bbce Move ignore handling from index recv to puller (fixes #1133)
With this change we accept updates for ignored files from other devices,
and check the ignore patterns at pull time. When we detect that the
ignore patterns have changed we do a full check of files that we might
now need to pull.
2014-12-23 10:46:02 +01:00
Jakob Borg 2c89f04be7 Refactor ignore handling (...)
This uses persistent Matcher objects that can reload their content and
provide a hash string that can be used to check if it's changed. The
cache is local to each Matcher object instead of kept globally.
2014-12-23 10:46:02 +01:00
Jakob Borg 618a8682b7 golint style tweaks 2014-12-16 23:33:56 +01:00
Jakob Borg d452b7593f Merge branch 'pr-1094'
* pr-1094:
  GUI tweaks for last file synced
  Display last received file and time (fixes #292, fixes #801)
2014-12-12 14:25:12 +01:00
Audrius Butkevicius 7a1aaaf5c4 Remove temporaries during scan (fixes #1092) 2014-12-09 23:58:58 +00:00
Audrius Butkevicius bde92d5cfe Display last received file and time (fixes #292, fixes #801) 2014-12-09 20:24:48 +00:00
Jakob Borg febeed3277 config.ConfigWrapper -> config.Wrapper 2014-12-08 16:39:11 +01:00
Jakob Borg 9d07aa006d Various style fixes 2014-12-08 16:36:15 +01:00
Jakob Borg 12d69e25dd Fixes for go vet 2014-12-08 16:19:08 +01:00
Jakob Borg c9492e54f7 Copyright notice 2014-12-08 15:42:53 +01:00
Audrius Butkevicius a9339d0627 Revert "Cache file descriptors" (fixes #1096)
This reverts commit 992ad97ad5.

Causes issues on Windows which uses file locking.
Meaning we cannot archive or modify the file while it's open.
2014-12-08 11:56:14 +00:00
Audrius Butkevicius b00264b594 Copy compression setting while introducing 2014-12-07 22:43:30 +00:00
Jakob Borg 581f4b89bd Merge remote-tracking branch 'origin/pr/977'
* origin/pr/977:
  Cache file descriptors
2014-12-07 08:03:34 +01:00
Audrius Butkevicius 604a4e7dbc Scrap IsSymlink for native support on Go 1.4
Obviously needs Go 1.4 to go back in.

I am still open to doing fix-up's on rescan interval on Windows, which
would still allow getting rid of all the Windows code.

Frankly, we could just defer creations of links (like we defer deletions of files)
in hopes that the target gets created, and if it doesn't, well tough luck, you'll
get a file symlink.

To be honest, nobody would even notice this 'issue' as I am sure nobody on
Windows uses symlinks.

But at the same time, this ugly code is hidden away in some creppy file in
it's own module far far away, and the interface that it exports is fine'ish,
so I wouldn't mind keeping it as it is.
2014-12-04 23:02:57 +00:00
Jakob Borg 7f97037190 Revert "Merge pull request #1078 from syncthing/go1.4"
This reverts commit b658afd857, reversing
changes made to 591c5dabf4.
2014-12-04 22:24:49 +01:00
Audrius Butkevicius 992ad97ad5 Cache file descriptors 2014-12-04 16:18:47 +00:00
Jakob Borg 2abe792f36 Use same order of parameters as os.Symlink 2014-12-04 11:53:55 +01:00
Jakob Borg e7441ff6e8 DisableSymlinks -> !SymlinksEnabled 2014-12-01 11:27:07 +01:00
Audrius Butkevicius 7ad711f554 Add option to disable symlinks (fixes #1017) 2014-11-30 22:10:32 +00:00
Audrius Butkevicius fd2d2c035e Add support for multiple announce servers (fixes #677)
Somebody owes me a beer.
2014-11-30 13:25:06 +01:00
Jakob Borg 90de5659ea Data race: sharedPullerState WriteAt+Close 2014-11-29 23:51:53 +01:00
Jakob Borg 42b8dafafe Data race: can't access sharedPullerState.closed from the outside 2014-11-29 23:18:56 +01:00
Jakob Borg 07cdf0364c Data race: ProgressEmitter (debug output only) 2014-11-29 22:51:13 +01:00
Jakob Borg 7f829f0159 Data race: broken locking on model.folderIgnores 2014-11-29 22:38:08 +01:00
Jakob Borg a918aa97d9 Data race: deviceActivity methods with value receiver :( 2014-11-29 22:38:08 +01:00
Audrius Butkevicius ea17542e4b Change progress emitter
1. Do not use cached value for BytesCompleted
2. Refactor JS a bit
3. Allow disabling progress emitter
2014-11-25 22:07:18 +00:00
piobpl 90a31589bb Showing detailed sync progress (fixes #476)
based on commit by Audrius Butkevicius <audrius.butkevicius@gmail.com>
2014-11-25 20:18:35 +01:00
Audrius Butkevicius 262cf63956 Populate correct defaults 2014-11-23 18:45:45 +00:00
Jakob Borg fe2ae4c6c3 Merge pull request #997 from syncthing/lig
Minor fixes
2014-11-23 11:35:19 +01:00
Jakob Borg 16d9944dbb Merge pull request #1002 from AudriusButkevicius/routine-cfg
Make copiers, pullers and finishers configurable
2014-11-23 11:29:58 +01:00
Audrius Butkevicius 59a85c1d75 Use custom structure for /need calls (fixes #1001)
Also, remove trimming by number of blocks as this no longer affects the size
of the response.
2014-11-23 00:52:48 +00:00
Audrius Butkevicius 4427149a38 Make copiers, pullers and finishers configurable
Compliments #999
2014-11-23 00:02:12 +00:00
Audrius Butkevicius 20dee618ea Populate ignores upon adding a folder (fixes #996) 2014-11-22 02:22:09 +00:00
Audrius Butkevicius 37ebbb53be Replace directories/links with files (fixes #580) 2014-11-22 02:22:03 +00:00
Jakob Borg ce948fc512 Don't leave read only dir around, fails clean 2014-11-20 23:34:14 +01:00
Audrius Butkevicius c325ffd0f8 Add symlink support (fixes #873) 2014-11-20 16:32:00 +01:00
Audrius Butkevicius 938e287501 Code smell 2014-11-20 16:32:00 +01:00
Jakob Borg 82c6caef85 Use more inclusive copyright header 2014-11-17 12:54:42 +01:00
Audrius Butkevicius 7106fc5304 Remove existing file while handling dir (fixes #952) 2014-11-13 22:59:40 +00:00
Audrius Butkevicius d16dcb9f19 Lock while accessing folder configs 2014-11-13 22:30:49 +00:00
Audrius Butkevicius 1aaf34b0ed Fix typo 2014-11-13 22:30:43 +00:00
Audrius Butkevicius 39a3b8922d Save config on device rename (fixes #957) 2014-11-12 23:42:17 +00:00
Emil Hessman 2592ba7399 internal/model: fix formatting directive in test 2014-11-12 12:32:25 +01:00
Jakob Borg c959f59581 Quick and dirty fix for Ignores test failures 2014-11-09 22:20:20 +01:00
Jakob Borg 2449723a1c Add list of compiled regexps to /rest/ignores (fixes #942) 2014-11-08 22:13:12 +01:00
Jakob Borg 8449a65cdf Merge pull request #930 from AudriusButkevicius/avail
Only connected devices are available devices
2014-11-03 21:56:40 -06:00
Audrius Butkevicius fc47562983 Only connected devices are available devices 2014-11-03 21:25:36 +00:00
Audrius Butkevicius 76900ae291 Fix and relax locking 2014-11-03 21:14:57 +00:00
Audrius Butkevicius 9ea8b6f659 Do not delete temp files on error (fixes #849) 2014-11-01 23:33:49 +00:00
Audrius Butkevicius d305752749 Cleanup temporary files during directory removal (fixes #919) 2014-10-29 11:19:48 +00:00
Jakob Borg d3ddfa31f7 Merge branch 'pr/909'
* pr/909:
  Add Vilbrekin
  Correctly check whether parent directory is writable for current user. "&04" was checking if file is readable by others, while "&0200" checks if it's writable for current user.
2014-10-26 13:59:09 +01:00
Jakob Borg 15ee9a5cac Break out logger as a reusable component 2014-10-26 13:16:54 +01:00
Audrius Butkevicius 58945a429f Revert removal of test files 2014-10-26 11:36:40 +00:00
Lode Hoste 33267f2178 Ignore all paths within .stversions folder 2014-10-26 11:47:02 +01:00
Vilbrekin 970e50d1f1 Correctly check whether parent directory is writable for current user.
"&04" was checking if file is readable by others, while "&0200" checks
if it's writable for current user.

(Fixes #904)
2014-10-26 02:26:40 +01:00
Audrius Butkevicius d4199c2d08 Recover from corrupt block maps 2014-10-24 23:20:08 +01:00
Jakob Borg d8b335ce65 Don't panic on queries for nonexistent folders (fixes #765) 2014-10-24 14:54:36 +02:00
Audrius Butkevicius 25bb55491a Cleanup blockmap on update (fixes #889) 2014-10-22 22:18:07 +01:00
Jakob Borg 198cbacc3e Be lenient towards malformed UPnP IGD UUIDs (fixes #890) 2014-10-21 17:07:11 +02:00
Jakob Borg c55fee69de Devices added by introducer should have dynamic address (fixes #866) 2014-10-18 20:40:31 +02:00
Audrius Butkevicius aa168ec2d6 Populate block offsets even if the blocks are not diffed 2014-10-17 23:16:29 +01:00
Jakob Borg 86a57d8b56 Hash mismatch in general doesn't merit a warning 2014-10-17 10:33:02 +02:00
Jakob Borg 5488ae5b89 Don't log inscrutable 'recovered: leveldb: not found' to web GUI 2014-10-16 13:45:42 +02:00
Audrius Butkevicius dedf835aa6 Delete files and directories after pulling 2014-10-16 12:26:28 +02:00
Audrius Butkevicius e62b9c6009 Add fetcher tests 2014-10-16 12:26:28 +02:00
Audrius Butkevicius 53da778506 Track total block counts, count copier blocks
Will eventually allow us to track progress per file
2014-10-16 12:26:28 +02:00
Audrius Butkevicius 4360b2c815 Fix tests 2014-10-16 12:26:28 +02:00
Audrius Butkevicius 1e15b1e0be Implement block fetcher (fixes #781, fixes #3) 2014-10-16 12:26:28 +02:00
Jakob Borg 8818c4785b Fix debug and fmt poopoo 2014-10-16 12:23:33 +02:00
Jakob Borg 2fb8eb755b Add a few more debug prints 2014-10-16 12:12:54 +02:00
Jakob Borg cd05f1c3d7 Merge pull request #860 from AudriusButkevicius/ticker
Cleanup temporaries once an hour (fixes #858)
2014-10-15 18:07:15 +02:00
Audrius Butkevicius 86346aa332 Add further nil checks (fixes #862, ref #864) 2014-10-15 15:54:55 +01:00
Audrius Butkevicius ea9f8b0ceb Cleanup temporaries once an hour (fixes #858) 2014-10-15 10:30:10 +01:00
Jakob Borg a778b410b9 Only do initial scan if scanInterval==0 (fixes #856) 2014-10-15 10:51:09 +02:00
Audrius Butkevicius aaf625c624 Merge pull request #854 from AudriusButkevicius/nils
Revert and replace 31d95ac, 65acc7c, 87780a5
2014-10-15 08:24:19 +01:00
Audrius Butkevicius 5419ff9a71 Keep temporaries for reuse, cleanup before pull (fixes #849, fixes #841) 2014-10-14 22:00:40 +01:00
Audrius Butkevicius ade437d625 Revert and replace 31d95ac, 65acc7c, 87780a5 2014-10-14 21:35:30 +01:00
Jakob Borg 964d17d05a Merge pull request #842 from AudriusButkevicius/ignorecache
Cache ignore file matches
2014-10-14 12:43:21 +02:00
Audrius Butkevicius 665c5992f0 Cache ignore file matches 2014-10-14 10:30:37 +01:00
Jakob Borg 5f52e0581d Add linientMtimes workaround for Android brokenness (ref #831) 2014-10-14 08:48:43 +02:00
Jakob Borg 7fb1a470ce Temporary workaround for panics in GUI/Usage reporting (ref #811) 2014-10-13 14:45:40 +02:00
Jakob Borg 5656be5206 Folders queried by the REST interface might not exist in the model (fixes #823) 2014-10-12 10:47:23 +02:00
Jakob Borg d65302742c Emit info level log when deleting directory fails 2014-10-10 21:20:46 +02:00
Jakob Borg 41e20bb6b7 Reinstate ignore permissions 2014-10-10 00:34:32 +02:00
Jakob Borg 64b42bba2e Remove dead code 2014-10-07 19:26:55 +02:00
Jakob Borg d089436546 Removing a nonexistent file is OK (fixes #792) 2014-10-06 21:47:42 +02:00
Jakob Borg 9b11609b63 Use a configuration wrapper to handle loads and saves 2014-10-06 17:57:35 +02:00
Jakob Borg 2fde82528d Fix tests for previous 2014-10-06 10:19:27 +02:00
Jakob Borg 6c383e279f Handle corner case where we can reuse a temp file totally 2014-10-06 10:15:57 +02:00
Audrius Butkevicius a8faeeac73 Add tests 2014-10-04 16:01:05 +01:00
Audrius Butkevicius 69e385e4cd Reuse temporary files (fixes #4) 2014-10-04 16:00:59 +01:00
Audrius Butkevicius 41b8dd2863 Have only one block size 2014-10-04 13:20:46 +01:00
Audrius Butkevicius 493dc8fcd5 Remove unused argument/variable 2014-10-04 13:20:07 +01:00
Jakob Borg 87764445e8 Some debug prints in the puller loop skips 2014-10-03 17:55:51 +02:00
Jakob Borg 72c90abe36 Reschedule pulls when skipping due to scan incomplete (fixes #777) 2014-10-03 16:10:35 +02:00
Jakob Borg 555e70ebec Remove temporary file on hash mismatch (fixes #771) 2014-10-01 14:43:22 +02:00
Jakob Borg 9edce23e76 Relicense to GPL 2014-10-01 07:53:59 +02:00
Jakob Borg 756a8a35e3 Add file I forgot in previous 2014-09-30 17:56:02 +02:00
Jakob Borg f3057c61a7 Prepare for being able to start and stop folders at any time 2014-09-30 17:52:05 +02:00
Jakob Borg 2091e12e82 Perform initial scan asynchronously (fixes #509, fixes #464) 2014-09-30 17:35:04 +02:00
Jakob Borg 991dc32a0b Fix config tests for autoUpgradeIntervalH 2014-09-28 22:38:10 +02:00
Audrius Butkevicius f561d3261a Rename FolderConfiguration.Directory to FolderConfiguration.Path 2014-09-28 14:36:16 +01:00
Audrius Butkevicius fdf8ee7015 Manual fixup 2014-09-28 14:23:08 +01:00
Audrius Butkevicius 5ec95086f2 Run go fmt -w -s 2014-09-28 14:23:08 +01:00
Audrius Butkevicius 26e4669316 Run go fmt -w 2014-09-28 14:23:08 +01:00
Audrius Butkevicius 6c352dca74 Rename Repository -> Folder, Node -> Device (fixes #739) 2014-09-28 14:23:07 +01:00
Jakob Borg b8ed135183 Don't get stuck in idle while syncing from a disconnecting node 2014-09-28 07:56:05 +02:00
Jakob Borg 0e2653b7dd Correctly handle ro dirs in ro dirs etc 2014-09-28 01:54:25 +02:00
Jakob Borg 99427d649e Complete rewrite of the puller (fixes #638, fixes #715, fixes #701) 2014-09-27 21:51:08 +02:00
Jakob Borg 14817e31f6 Move top level packages to internal. 2014-09-27 09:42:10 +02:00