Commit Graph

1266 Commits

Author SHA1 Message Date
Simon Frei 1cb55904bc lib/model: Prevent panic in NeedFolderFiles (fixes #5872) (#5875) 2019-07-19 19:39:52 +02:00
Simon Frei 2b622d0774 lib/model: Close conn on dev pause (fixes #5873) (#5874) 2019-07-19 19:37:29 +02:00
Simon Frei 1894123d3c lib/syncthing: Modify exit status before stopping (fixes #5869) (#5870) 2019-07-18 20:49:00 +02:00
Jakob Borg e7e177a6fa lib/relay: Prevent spurious relay error message (fixes #5861) (#5864) 2019-07-17 10:55:28 +02:00
Simon Frei eed1edcca0 cmd/syncthing: Ensure myID is set by making it local (fixes #5859) (#5862) 2019-07-17 07:19:14 +02:00
Simon Frei 0025e9ccfb all: Refactor cmd/syncthing creating lib/syncthing (ref #4085) (#5805)
* add skeleton for lib/syncthing

* copy syncthingMain to lib/syncthing (verbatim)

* Remove code to deduplicate copies of syncthingMain

* fix simple build errors

* move stuff from main to syncthing with minimal mod

* merge runtime options

* actually use syncthing.App

* pass io.writer to lib/syncthing for auditing

* get rid of env stuff in lib/syncthing

* add .Error() and comments

* review: Remove fs interactions from lib

* and go 1.13 happened

* utility functions
2019-07-14 11:43:13 +01:00
Simon Frei 82b70b9fae lib/model, lib/protocol: Track closing connections (fixes #5828) (#5829) 2019-07-14 11:03:55 +02:00
Simon Frei def4b8cee5 lib/config: Error on empty folder path (fixes #5853) (#5854) 2019-07-14 11:03:14 +02:00
Aurélien Rainone f1a7dd766e all: Add comment to ensure correct atomics alignment (fixes #5813)
Per the sync/atomic bug note:

> On ARM, x86-32, and 32-bit MIPS, it is the caller's
> responsibility to arrange for 64-bit alignment of 64-bit words
> accessed atomically. The first word in a variable or in an
> allocated struct, array, or slice can be relied upon to be
> 64-bit aligned.

All atomic accesses of 64-bit variables in syncthing code base are
currently ok (i.e they are all 64-bit aligned).

Generally, the bug is triggered because of incorrect alignement
of struct fields. Free variables (declared in a function) are
guaranteed to be 64-bit aligned by the Go compiler.

To ensure the code remains correct upon further addition/removal
of fields, which would change the currently correct alignment, I
added the following comment where required:

     // atomic, must remain 64-bit aligned

See https://golang.org/pkg/sync/atomic/#pkg-note-BUG.
2019-07-13 14:05:39 +01:00
Simon Frei 20c8dbd9ed lib/model: Fix integer conversion (fixes #5837) (#5851) 2019-07-12 16:37:12 +02:00
xduugu 4b3f9b1af9 lib/versioner: Replace multiple placeholders in a single token in external command (fixes #5849)
* lib/versioner: Add placeholder to provide the absolute file path to external commands

This commit adds support for a new placeholder, %FILE_PATH_FULL%, to the
command of the external versioner. The placeholder will be replaced by
the absolute path of the file that should be deleted.

* Revert "lib/versioner: Add placeholder to provide the absolute file path to external commands"

This reverts commit fb48962b947358e90a76e87a7794adb6057de476.

* lib/versioner: Replace all placeholders in external command (fixes #5849)

Before this commit, only these placeholders were replaced that span a
whole word, for example "%FOLDER_PATH%". Words that consisted of more
than one placeholder or additional characters, for example
"%FOLDER_PATH%/%FILE_PATH%", were left untouched.

* fixup! lib/versioner: Replace all placeholders in external command (fixes #5849)
2019-07-12 08:45:39 +01:00
Simon Frei 3446d50201 lib/model: Remove pointless error that watch hasn't started (fixes #5833) (#5834) 2019-07-10 11:00:06 +02:00
Simon Frei 9fef1552fc lib/db, lib/model: Remove folder info from panics (ref #5839) (#5840) 2019-07-10 10:57:49 +02:00
Simon Frei 485acda63b lib/relay: Call the proper Error method (ref #5806) (#5841) 2019-07-09 22:29:19 +01:00
Simon Frei ba056578ec
lib: Add util.Service as suture.Service template (fixes #5801) (#5806) 2019-07-09 11:40:30 +02:00
Simon Frei 863fe23347 gui, lib/model: Fix download progress accounting (fixes #5811) (#5815) 2019-06-30 09:23:47 +02:00
Simon Frei 1cf352a722 lib/model: NewFileSet outside fmut (#5818) 2019-06-29 08:49:30 +02:00
Simon Frei b58f6ca886 lib/model: Correct/unify check if item changed (#5819) 2019-06-29 07:45:41 +02:00
Simon Frei 3c7e7e971d lib/model: Make jobQueue.Jobs paginated (fixes #5754) (#5804)
* lib/model: Make jobQueue.Jobs paginated (fixes #5754)

* fix, no test yet

* add test
2019-06-27 19:25:38 +01:00
Audrius Butkevicius afde0727fe lib/versioner: Revert naming change (fixes #5807) (#5808) 2019-06-25 08:56:11 +03:00
Simon Frei bf744ded31 cmd/syncthing, lib/db: Exit/close db faster (fixes #5781) (#5782)
This adds a 10s timeout on closing the db and additionally cancels active
db iterators and waits for them to terminate before closing the db.
2019-06-17 15:27:25 +03:00
Simon Frei cea5962417 lib/model: Unflake TestPullInvalidIgnoredSR/SO (fixes #5796) (#5799) 2019-06-17 15:23:28 +03:00
Simon Frei 02752af862 lib/protocol: Don't block on Close (fixes #5794) (#5795) 2019-06-14 19:04:41 +02:00
Simon Frei abd363e8bb
lib/model: Don't error on pulling deletion of invalid file (fixes #5791) (#5792) 2019-06-14 08:48:14 +02:00
Simon Frei 13a746e0fb lib/model: Prevent nil deref if folder stopped (fixes #5780) (#5778) 2019-06-11 11:48:51 +02:00
Audrius Butkevicius 21f50e2f8f lib/versioner: Use mtime for version cleanup (fixes #5765) (#5769) 2019-06-11 09:16:55 +02:00
Audrius Butkevicius b7c70a9817 lib/fs: Enhance mtimefs, use everywhere (fixes #5777) (#5776) 2019-06-11 08:27:12 +02:00
Jakob Borg 42ce6be9b9
lib/ur: Implement crash (panic) reporting (fixes #959) (#5702)
* lib/ur: Implement crash (panic) reporting (fixes #959)

This implements a simple crash reporting method. It piggybacks on the
panic log files created by the monitor process, picking these up and
uploading them from the usage reporting routine.

A new config value points to the crash receiver base URL, which defaults
to "https://crash.syncthing.net/newcrash" (following the pattern of
"https://data.syncthing.net/newdata" for usage reports, but allowing us
to separate the service as required).
2019-06-11 08:19:11 +02:00
Mingxuan Lin eb4fe808c5 lib/fs: Fallback EvalSymlinks method on windows (fixes #5609) (#5611) 2019-06-10 14:33:53 +02:00
Simon Frei 1054ce9354
lib/model: Refactor sending indexes as suture service (#5757) 2019-06-10 13:27:22 +02:00
Audrius Butkevicius ee746263fb
lib/connections: Do not leak FDs, fix address copy (fixes #5767) (#5768)
* lib/connections: Do not leak FDs, fix address copy (fixes #5767)

* build

* Update quic_listen.go

* Update quic_listen.go
2019-06-09 22:14:00 +01:00
Jakob Borg 41ff4b323e lib/api: Correct logic for errors.jons in support bundle (fixes #5766)
The err check should always be done with != and have the error case
first, to follow the pattern and avoid mistakes like these.
2019-06-09 09:35:05 +02:00
Jakob Borg 997bb5e7e1 all: Remove "large blocks" config (#5763)
We now always use large / variable blocks.
2019-06-06 15:57:38 +01:00
Jakob Borg ca2fa8de4e lib/build: Version 1.2 will be the Fermium Flea 2019-06-06 14:45:07 +02:00
Simon Frei e39d3f95dd lib/protocol: Prioritize close msg and add close timeout (#5746) 2019-06-05 14:01:59 +08:00
Simon Frei 6664e01acf lib/protocol: Return from ClusterConfig when closed (#5752) 2019-05-29 12:14:00 +02:00
Jakob Borg 5ce5b2c94a
lib/config: Refactor migrations a bit (#5750)
This breaks out config migrations to a separate concept, making it
(imho) slightly easier to maintain and get an overview.
2019-05-29 11:37:44 +02:00
Audrius Butkevicius e714df013f lib/connections: Add QUIC protocol support (fixes #5377) (#5737) 2019-05-29 09:56:40 +02:00
Simon Frei 3775a64d5c lib/protocol: Don't send anything else before cluster config (#5741) 2019-05-27 11:15:34 +01:00
Simon Frei 129df0613b lib/model: Unflake folder restart with blocking conn test (ref #5707) (#5744) 2019-05-27 10:58:09 +01:00
Simon Frei 486230768e lib/fs, lib/model: Add error channel to Watch to avoid panics (fixes #5697) (#5734)
* lib/fs, lib/model: Add error channel to Watch to avoid panics (fixes #5697)

* forgot unsupported watch

* and more non(-standard)-unixy fixes

* and windows test

* review
2019-05-25 20:08:26 +01:00
Simon Frei 9e6db72535 lib/protocol: Don't call receiver after calling Closed (fixes #4170) (#5742)
* lib/protocol: Don't call receiver after calling Closed (fixes #4170)

* review
2019-05-25 20:08:07 +01:00
Simon Frei d91da8feee lib/model: Readd special handling of conn close in TestIssue5063 (#5743)
This partially reverts commit 64518b0f7e.
2019-05-25 18:51:13 +01:00
Simon Frei 64518b0f7e
lib/model: Close connections when model is stopped (#5733) 2019-05-25 16:00:32 +02:00
Simon Frei 5d35b2c540 lib/protocol: Test for Close on blocking send deadlock (ref #5442) (#5732) 2019-05-23 21:42:02 +01:00
Simon Frei 5b306510a0
lib/model: Consistently cleanup model in tests (#5724) 2019-05-19 14:29:07 +02:00
Simon Frei 1b2b970f32 lib/model, lib/testutils: Test closing a connection on folder restart (#5707) 2019-05-18 08:53:59 +02:00
Simon Frei 5ffbb7668d lib/model: Fix test flakyness regression (ref #5592) (#5718) 2019-05-18 08:52:50 +02:00
Simon Frei 441ea109a1
lib/model: Refactor file deletions when pulling (#5699) 2019-05-17 18:29:54 +02:00
Jakob Borg 2c866277a2 lib/api, lib/connections, gui: Show connection error for disconnected devices (fixes #3345) (#5727)
* lib/api, lib/connections, gui: Show connection error for disconnected devices (fixes #3345)

This adds functionality in the connetions service to track the last
error per address. That is in turn exposed in the /rest/system/status
API method, as that is also where we already show the listener status
from the connection service.

The GUI uses this info where it lists addresses, showing errors (if any)
in red underneath each address.

I also slightly refactored the existing status method on the connection
service to have a better name and return typed information.

* ok

* review

* formatting

* review
2019-05-16 22:11:45 +01:00
André Colomb e4956358fb lib/model: Remove superfluous check for IndexID in remote ClusterConfig (#5717)
The check in ClusterConfig() when iterating through announced devices
in a folder explicitly skips entries without a non-zero IndexID.
Therefore, the check for IndexID == 0 just below will never be true
and the intended cleanup of local index data will not happen.

Plainly remove that check to make the intended case distinction work.
2019-05-12 21:17:55 +02:00
Simon Frei dfbbb286fc lib/fs: Consider win83 for root path as well when watching (ref #5706) (#5709) 2019-05-11 10:06:04 +02:00
Simon Frei fbd445fe0a lib/api: Clean up after test, enables test caching (#5710) 2019-05-11 08:41:32 +02:00
Simon Frei 2b246eeb52
lib/model: Move test utilities to separate files (#5694) 2019-05-10 13:33:45 +02:00
Simon Frei 2558b021e5 lib/fs: Remove \\?\ for drive letters when watching (fixes #5578) (#5701) 2019-05-10 09:09:58 +02:00
Jakob Borg 31be810eb6 lib/model: Don't fail operation when fsync() fails (fixes #5704) (#5705) 2019-05-09 21:20:29 +01:00
Jakob Borg 62a6d619e7 Merge branch 'release'
* release:
  lib/fs: Revert removal of \\?\ for drive letters (fixes #5695)
2019-05-08 17:38:08 +02:00
Jakob Borg a04fcfe749 lib/fs: Revert removal of \\?\ for drive letters (fixes #5695)
This reverts commit ca823bd591 from #5633.

This un-fixes bug #5578.
2019-05-08 17:31:52 +02:00
Simon Frei 283f39ae5f lib/protocol: Revert unreleased changes related to closing connections (#5688)
This reverts commits:
    ec7c88ca55
    19b51c9b92
    5da41f75fa
    04b927104f
2019-05-08 08:08:26 +02:00
Simon Frei b45d77b6be lib/model: Fix regression deleting directories on pull (ref #5690) (#5691) 2019-05-06 20:55:26 +02:00
Simon Frei 79e67b7f79 lib/model: Remove individual pull errors on retry (fixes #5659) (#5690) 2019-05-06 17:21:56 +02:00
Simon Frei 5fa8467756 lib/model: Consistent use of locks (#5684) 2019-05-02 18:55:39 +01:00
Simon Frei 5954b105cd
lib/model: Let fakeConnection call Model.Closed on close (#5682) 2019-05-02 14:24:55 +02:00
Simon Frei defc5dca65
lib/model: Use correct lock (#5683) 2019-05-02 14:09:42 +02:00
Simon Frei 9f358ecae0
lib/protocol: Refactor to pass only relevant argument to writeMessage (#5681) 2019-05-02 14:09:19 +02:00
Simon Frei fe4daf242b
cmd, lib/db: Actually close goleveldb (fixes #5505) (#5671) 2019-05-02 11:15:00 +02:00
Simon Frei ec7c88ca55 lib/protocol: Fix yet another deadlock (fixes #5678) (#5679)
* lib/protocol: Fix yet another deadlock (fixes #5678)

* more consistency

* read deadlock

* naming

* more naming
2019-05-02 09:21:07 +01:00
Simon Frei 19b51c9b92 lib/protocol: Don't close asyncMessage.done on success (fixes #5674) (#5675) 2019-04-29 17:52:57 +02:00
Audrius Butkevicius 0ca1f26ff8
lib/versioner: Restore for all versioners, cross-device support (#5514)
* lib/versioner: Restore for all versioners, cross-device support

Fixes #4631
Fixes #4586
Fixes #1634
Fixes #5338
Fixes #5419
2019-04-28 23:30:16 +01:00
Jakob Borg 2984d40641 lib/ignore: Additional test case (#5672) 2019-04-28 21:20:11 +01:00
Simon Frei 5da41f75fa lib/model, lib/protocol: Wait for reader/writer loops on close (fixes #4170) (#5657)
* lib/protocol: Wait for reader/writer loops on close (fixes #4170)

* waitgroup

* lib/model: Don't hold lock while closing connection

* fix comments

* review (lock once, func argument) and naming
2019-04-28 11:58:51 +01:00
Simon Frei 04b927104f lib/protocol: Don't send any messages before cluster config (#5646)
* lib/model: Send cluster config before releasing pmut

* reshuffle

* add model.connReady to track cluster-config status

* Corrected comments/strings

* do it in protocol
2019-04-23 20:47:11 +01:00
Simon Frei 110806842c lib/model: Pass the old not new fileinfo to deleteItemOnDisk (fixes #5654) (#5655) 2019-04-23 20:46:28 +01:00
Simon Frei e302ccf4b4
lib/fs: Improve IsParent (#5658) 2019-04-22 11:12:32 +02:00
Simon Frei 926e9228ed lib/scanner, lib/model: File -> item when logging error (#5664) 2019-04-21 16:19:59 +01:00
Simon Frei 86e72d9973 lib/model: Use RLock and legacy polish (#5663) 2019-04-21 13:21:36 +01:00
Simon Frei 3bea59b0d9
lib/model: Refactor progressEmitter to de-/activate by config (fixes #4613) (#5623) 2019-04-13 14:20:51 +02:00
Simon Frei fca895a632 lib/model: Fix block index calculation for recheckFile (fixes #5649) (#5648) 2019-04-12 15:21:07 +02:00
Simon Frei 79360e2205
lib/fs: Add test that symlinks are skipped on walk (#5644) 2019-04-10 22:36:37 +02:00
Simon Frei 9b2a73f9ab
lib/model: Pause pull for at least as long as failed pull took (fixes #5641) (#5643) 2019-04-10 16:22:23 +02:00
Simon Frei c305265c62 lib/model: Request errors conforming to BEP specs (#5642) 2019-04-10 11:47:24 +02:00
Simon Frei ca823bd591
lib/fs: When watching remove \\?\ for drive letters (fixes #5578) (#5633) 2019-04-09 09:02:04 +02:00
Simon Frei 395e524e2d lib/model: Update db on scan/pull in folder (#5608) 2019-04-07 13:29:17 +02:00
Jakob Borg f0f8bf7784 lib/config: Round times stored for pending folders/devices (fixes #5554) 2019-03-27 20:35:42 +01:00
Jakob Borg bf3834e367
lib/protocol: Use constants instead of init time hashing (fixes #5624) (#5625)
This constructs the map of hashes of zero blocks from constants instead
of calculating it at startup time. A new test verifies that the map is
correct.
2019-03-27 20:20:30 +01:00
Simon Frei 8d1eff7e41 lib/model: Missing queue.Done and reset between pulls (fixes #5332) (#5626) 2019-03-27 20:19:35 +01:00
Simon Frei 0cff66fcbc lib/model: Optimize puller for meta only changes (#5622) 2019-03-27 08:36:58 +00:00
Simon Frei 43a5be1c4b lib/model: Send item finished even after deregistering (fixes #5362) (#5620) 2019-03-26 21:31:33 +01:00
Simon Frei b50039a920 cmd/syncthing, lib/api: Separate api/gui into own package (ref #4085) (#5529)
* cmd/syncthing, lib/gui: Separate gui into own package (ref #4085)

* fix tests

* Don't use main as interface name (make old go happy)

* gui->api

* don't leak state via locations and use in-tree config

* let api (un-)subscribe to config

* interface naming and exporting

* lib/ur

* fix tests and lib/foldersummary

* shorter URVersion and ur debug fix

* review

* model.JsonCompletion(FolderCompletion) -> FolderCompletion.Map()

* rename debug facility https -> api

* folder summaries in model

* disassociate unrelated constants

* fix merge fail

* missing id assignement
2019-03-26 19:53:58 +00:00
Simon Frei d4e81fff8a lib/model: Remove unnecessary arguments in pulling functions (#5619) 2019-03-25 12:59:22 +01:00
Simon Frei e31a116e6e lib/model: Pass correct file info to deleteItemOnDisk (fixes #5616) (#5617) 2019-03-25 12:42:39 +01:00
Simon Frei e7ae851900 lib/model: Debug and test fixes (#5613) 2019-03-22 14:43:47 +01:00
Simon Frei 1a6d023ba8 lib/model: Run recvonly tests in temporary dirs (#5587) 2019-03-20 17:38:03 +00:00
Simon Frei 50d8c43e7c lib/config: Set UseLargeBlocks to true by default (fixes #5599) (#5600) 2019-03-12 12:59:26 +00:00
Simon Frei 289a02e994 lib/model: Integrate stat refs in folder (#5596) 2019-03-11 16:57:21 +00:00
Simon Frei 445637ebec lib/model: Pass fset & ignores on folder creation (#5592) 2019-03-11 07:28:54 +01:00
Simon Frei 3f3d2c814b
lib/model: Remove unused code (#5591) 2019-03-10 17:05:39 +01:00
Simon Frei 189e44488e lib/model: Introduce must test utility (#5586)
* lib/model: Introduce must test utility

* nice
2019-03-09 18:45:36 +00:00
Simon Frei 27ff20faa3 lib/model: Introduce waitForState test utility (#5585)
* lib/model: Introduce waitForState test utility

* folder id as param to waitForState
2019-03-09 10:36:55 +00:00
Simon Frei b1564e53e4 lib/model: Improve test utilities (#5584) 2019-03-08 20:29:09 +00:00