Commit Graph

200 Commits

Author SHA1 Message Date
Jakob Borg 2a4fc28318 We should pass around db.Instance instead of leveldb.DB
We're going to need the db.Instance to keep some state, and for that to
work we need the same one passed around everywhere. Hence this moves the
leveldb-specific file opening stuff into the db package and exports the
dbInstance type.
2015-10-31 12:35:30 +01:00
Jakob Borg 313485e406 Remove file that snuck in by mistake 2015-10-31 11:38:59 +01:00
Jakob Borg faf4267c73 Refactor: the various db key functions should be instance methods 2015-10-31 11:27:04 +01:00
Jakob Borg cdbc8004fb Comment pedantry 2015-10-31 11:16:07 +01:00
Jakob Borg 08c8d679ac Create a correct URL is more difficult than just slapping on a scheme (fixes #2316) 2015-10-30 21:22:40 +01:00
Jakob Borg 48c34b7234 Translation update 2015-10-30 10:23:09 +01:00
Jakob Borg e1a8ea7dec Rescan changed files before pulling on top of them (fixes #2416) 2015-10-29 09:12:37 +01:00
Jakob Borg c4ad97136f Move leveldb instance and transactions into separate files 2015-10-29 08:07:51 +01:00
Jakob Borg fd7b8ec77e Neater transaction handling 2015-10-28 22:04:00 +01:00
Jakob Borg e28c991331 Create an instance type to tie database methods to 2015-10-28 21:03:05 +01:00
Jakob Borg 9e210d705d The PublicKey() method is an addition in Go 1.4 2015-10-27 16:03:14 +01:00
Jakob Borg c42f1b53ab pulorder.go -> pullorder.go 2015-10-27 12:14:14 +01:00
Jakob Borg d171173e90 AlwaysLocalNets should not default to null 2015-10-27 12:04:51 +01:00
Jakob Borg 679f0f9363 Fix some config Copy() things we had forgotten 2015-10-27 11:53:42 +01:00
Jakob Borg 724c1e297f Remove handling of config versions < 10 (v0.11.0) 2015-10-27 11:46:33 +01:00
Jakob Borg 83154569b1 Refactor config types into separate files 2015-10-27 11:37:03 +01:00
Jakob Borg e3c0fba34b Must not call hex.Dump in non-debug mode... 2015-10-27 10:27:18 +01:00
Jakob Borg 2b6a6b91f3 Remove unused struct field 2015-10-27 09:55:05 +01:00
Jakob Borg dc32f7f0a3 Reduce allocations in HashFile
By using copyBuffer we avoid a buffer allocation for each block we hash,
and by allocating space for the hashes up front we get one large backing
array instead of a small one for each block. For a 17 MiB file this
makes quite a difference in the amount of memory allocated:

	benchmark               old ns/op     new ns/op     delta
	BenchmarkHashFile-8     102045110     100459158     -1.55%

	benchmark               old allocs     new allocs     delta
	BenchmarkHashFile-8     415            144            -65.30%

	benchmark               old bytes     new bytes     delta
	BenchmarkHashFile-8     4504296       48104         -98.93%
2015-10-27 09:37:27 +01:00
Jakob Borg 1efd8d6c75 Add benchmark of HashFile 2015-10-27 09:30:34 +01:00
Jakob Borg 464e6bec95 Log lines in REST should have lower case keys 2015-10-27 08:22:35 +01:00
Audrius Butkevicius 5b03c2d949 Remove dead code 2015-10-25 20:46:09 +00:00
Audrius Butkevicius f629a998a0 Change errNoDevice message to something more human 2015-10-25 13:27:26 +00:00
Audrius Butkevicius 73236e58c5 Close channel after the client is stopped 2015-10-22 23:09:02 +01:00
Jakob Borg 32414853c6 Fix Raleway font 2015-10-22 21:08:24 +02:00
Jakob Borg f3dc78d457 Don't deadlock after checking relay client status (fixes #2404) 2015-10-22 20:32:15 +02:00
Jakob Borg a32ac62208 Don't expect ending slash on Windows 2015-10-22 13:49:41 +02:00
Jakob Borg d7a934cf0e Paths must not end with slash on Windows 2015-10-22 11:39:34 +02:00
Jakob Borg 503491392d Correct amount of stack unwinding for debug prints 2015-10-22 11:38:45 +02:00
Jakob Borg c19eff4872 Revive remote client version in the GUI 2015-10-22 08:53:28 +02:00
Jakob Borg 2941a813c2 Fix upgrade tests 2015-10-22 08:35:48 +02:00
Jakob Borg 0a022d38fa Upgrade lib should use same criteria for beta check as main 2015-10-22 08:28:35 +02:00
Jakob Borg 7ed3b3dd3a Docs update 2015-10-22 08:14:43 +02:00
Jakob Borg 9a1922fdc6 Merge pull request #2385 from AudriusButkevicius/you-are-next
Add separate client for dynamic relays (fixes #2368)
2015-10-22 08:01:22 +02:00
Jakob Borg 83131103cf Don't load block list in ...Truncated methods
Speeds up and reduces allocations on those operations, at the price of
having a manually tweaked XDR decoder for FileInfoTruncated.

benchmark                         old ns/op      new ns/op      delta
BenchmarkReplaceAll-8             1868198122     1880206886     +0.64%
BenchmarkUpdateOneChanged-8       231852         172695         -25.51%
BenchmarkUpdateOneUnchanged-8     230624         179341         -22.24%
BenchmarkNeedHalf-8               104601744      109461427      +4.65%
BenchmarkHave-8                   29102480       34105026       +17.19%
BenchmarkGlobal-8                 150547687      172778045      +14.77%
BenchmarkNeedHalfTruncated-8      102471355      76564986       -25.28%
BenchmarkHaveTruncated-8          28758368       14277481       -50.35%
BenchmarkGlobalTruncated-8        151192913      106070136      -29.84%

benchmark                         old allocs     new allocs     delta
BenchmarkReplaceAll-8             555577         557554         +0.36%
BenchmarkUpdateOneChanged-8       1135           587            -48.28%
BenchmarkUpdateOneUnchanged-8     1135           587            -48.28%
BenchmarkNeedHalf-8               374780         374775         -0.00%
BenchmarkHave-8                   151992         152085         +0.06%
BenchmarkGlobal-8                 530033         530135         +0.02%
BenchmarkNeedHalfTruncated-8      374699         22160          -94.09%
BenchmarkHaveTruncated-8          151834         4904           -96.77%
BenchmarkGlobalTruncated-8        530037         30536          -94.24%

benchmark                         old bytes      new bytes      delta
BenchmarkReplaceAll-8             1765116216     1765305376     +0.01%
BenchmarkUpdateOneChanged-8       135085         93043          -31.12%
BenchmarkUpdateOneUnchanged-8     134976         92928          -31.15%
BenchmarkNeedHalf-8               44758752       44751791       -0.02%
BenchmarkHave-8                   11845052       11967172       +1.03%
BenchmarkGlobal-8                 80431136       80431065       -0.00%
BenchmarkNeedHalfTruncated-8      46526459       18243543       -60.79%
BenchmarkHaveTruncated-8          11348357       418998         -96.31%
BenchmarkGlobalTruncated-8        80977672       43116991       -46.75%
2015-10-21 23:49:10 +02:00
Jakob Borg c1591a5efd Only run benchmarks with -tags benchmark
Avoids creating temp database and stuff on a normal test run
2015-10-21 23:19:26 +02:00
Jakob Borg 918ef4dff8 Use batches in blockmap, speeds up and reduces memory usage on large Replace and Update ops
benchmark                         old ns/op      new ns/op      delta
BenchmarkReplaceAll-8             2880834572     1868198122     -35.15%
BenchmarkUpdateOneChanged-8       236596         231852         -2.01%
BenchmarkUpdateOneUnchanged-8     227326         230624         +1.45%
BenchmarkNeedHalf-8               105151538      104601744      -0.52%
BenchmarkHave-8                   28827492       29102480       +0.95%
BenchmarkGlobal-8                 150768724      150547687      -0.15%
BenchmarkNeedHalfTruncated-8      104434216      102471355      -1.88%
BenchmarkHaveTruncated-8          27860093       28758368       +3.22%
BenchmarkGlobalTruncated-8        149972888      151192913      +0.81%

benchmark                         old allocs     new allocs     delta
BenchmarkReplaceAll-8             555451         555577         +0.02%
BenchmarkUpdateOneChanged-8       1135           1135           +0.00%
BenchmarkUpdateOneUnchanged-8     1135           1135           +0.00%
BenchmarkNeedHalf-8               374779         374780         +0.00%
BenchmarkHave-8                   151996         151992         -0.00%
BenchmarkGlobal-8                 530066         530033         -0.01%
BenchmarkNeedHalfTruncated-8      374702         374699         -0.00%
BenchmarkHaveTruncated-8          151834         151834         +0.00%
BenchmarkGlobalTruncated-8        530049         530037         -0.00%

benchmark                         old bytes      new bytes      delta
BenchmarkReplaceAll-8             5018351912     1765116216     -64.83%
BenchmarkUpdateOneChanged-8       135085         135085         +0.00%
BenchmarkUpdateOneUnchanged-8     134976         134976         +0.00%
BenchmarkNeedHalf-8               44769400       44758752       -0.02%
BenchmarkHave-8                   11930612       11845052       -0.72%
BenchmarkGlobal-8                 81523668       80431136       -1.34%
BenchmarkNeedHalfTruncated-8      46692342       46526459       -0.36%
BenchmarkHaveTruncated-8          11348357       11348357       +0.00%
BenchmarkGlobalTruncated-8        81843956       80977672       -1.06%
2015-10-21 23:05:23 +02:00
Jakob Borg 0d9a04c713 Reuse blockkey, speeds up large Update and Replace calls
benchmark                         old ns/op      new ns/op      delta
BenchmarkReplaceAll-8             2866418930     2880834572     +0.50%
BenchmarkUpdateOneChanged-8       226635         236596         +4.40%
BenchmarkUpdateOneUnchanged-8     229090         227326         -0.77%
BenchmarkNeedHalf-8               104483393      105151538      +0.64%
BenchmarkHave-8                   29288220       28827492       -1.57%
BenchmarkGlobal-8                 159269126      150768724      -5.34%
BenchmarkNeedHalfTruncated-8      108235000      104434216      -3.51%
BenchmarkHaveTruncated-8          28945489       27860093       -3.75%
BenchmarkGlobalTruncated-8        149355833      149972888      +0.41%

benchmark                         old allocs     new allocs     delta
BenchmarkReplaceAll-8             1054944        555451         -47.35%
BenchmarkUpdateOneChanged-8       1135           1135           +0.00%
BenchmarkUpdateOneUnchanged-8     1135           1135           +0.00%
BenchmarkNeedHalf-8               374777         374779         +0.00%
BenchmarkHave-8                   151995         151996         +0.00%
BenchmarkGlobal-8                 530063         530066         +0.00%
BenchmarkNeedHalfTruncated-8      374699         374702         +0.00%
BenchmarkHaveTruncated-8          151834         151834         +0.00%
BenchmarkGlobalTruncated-8        530021         530049         +0.01%

benchmark                         old bytes      new bytes      delta
BenchmarkReplaceAll-8             5074297112     5018351912     -1.10%
BenchmarkUpdateOneChanged-8       135097         135085         -0.01%
BenchmarkUpdateOneUnchanged-8     134976         134976         +0.00%
BenchmarkNeedHalf-8               44759436       44769400       +0.02%
BenchmarkHave-8                   11911138       11930612       +0.16%
BenchmarkGlobal-8                 81609867       81523668       -0.11%
BenchmarkNeedHalfTruncated-8      46588024       46692342       +0.22%
BenchmarkHaveTruncated-8          11348354       11348357       +0.00%
BenchmarkGlobalTruncated-8        79485168       81843956       +2.97%
2015-10-21 23:05:23 +02:00
Jakob Borg 0c0c69f0cf The GC runs are legacy and slows things down quite a bit
benchmark                         old ns/op      new ns/op      delta
BenchmarkReplaceAll-8             2942370526     2866418930     -2.58%
BenchmarkUpdateOneChanged-8       7402489        226635         -96.94%
BenchmarkUpdateOneUnchanged-8     7298777        229090         -96.86%
BenchmarkNeedHalf-8               113608416      104483393      -8.03%
BenchmarkHave-8                   29834263       29288220       -1.83%
BenchmarkGlobal-8                 162773699      159269126      -2.15%
BenchmarkNeedHalfTruncated-8      111943400      108235000      -3.31%
BenchmarkHaveTruncated-8          29490369       28945489       -1.85%
BenchmarkGlobalTruncated-8        165841081      149355833      -9.94%

benchmark                         old allocs     new allocs     delta
BenchmarkReplaceAll-8             1054942        1054944        +0.00%
BenchmarkUpdateOneChanged-8       1149           1135           -1.22%
BenchmarkUpdateOneUnchanged-8     1135           1135           +0.00%
BenchmarkNeedHalf-8               374774         374777         +0.00%
BenchmarkHave-8                   151995         151995         +0.00%
BenchmarkGlobal-8                 530042         530063         +0.00%
BenchmarkNeedHalfTruncated-8      374697         374699         +0.00%
BenchmarkHaveTruncated-8          151834         151834         +0.00%
BenchmarkGlobalTruncated-8        530050         530021         -0.01%

benchmark                         old bytes      new bytes      delta
BenchmarkReplaceAll-8             5074294728     5074297112     +0.00%
BenchmarkUpdateOneChanged-8       141048         135097         -4.22%
BenchmarkUpdateOneUnchanged-8     134976         134976         +0.00%
BenchmarkNeedHalf-8               44734813       44759436       +0.06%
BenchmarkHave-8                   11911634       11911138       -0.00%
BenchmarkGlobal-8                 80436854       81609867       +1.46%
BenchmarkNeedHalfTruncated-8      46514673       46588024       +0.16%
BenchmarkHaveTruncated-8          11348357       11348354       -0.00%
BenchmarkGlobalTruncated-8        81730740       79485168       -2.75%
2015-10-21 23:05:22 +02:00
Jakob Borg 943e80e26c Make benchmarks more realistic 2015-10-21 23:04:29 +02:00
Jakob Borg 1eca4170f7 Add test for LocalSize/GlobalSize results 2015-10-21 21:58:48 +02:00
Jakob Borg c268e4ad1b Also keep GlobalSize in RAM 2015-10-21 21:58:48 +02:00
Jakob Borg d4f81e8791 Keep LocalSize data in RAM 2015-10-21 21:58:48 +02:00
Audrius Butkevicius 4f0680c3c8 Add separate client for dynamic relays (fixes #2368)
Did some manual tests in the playground, such as kicking off two clients in parallel, first connecting,
second one getting a message about already being connected, falling back to the second address.
2015-10-21 20:08:14 +01:00
Jakob Borg 8c26fe44c3 Actually run protocol tests faster with -short (on Go 1.5...) 2015-10-21 14:45:18 +02:00
Jakob Borg 8c7d9f3dd2 Protocol tests should run faster with -short 2015-10-21 14:35:59 +02:00
Jakob Borg f241b7e79a Global discovery should time out (fixes #2389) 2015-10-21 14:24:55 +02:00
Matt Burke c2a5e180b8 Emit warning when sync could overwrite configuration
Overwriting configuration files is likely to happen if a
user syncs their home directories across computers. In this
case, the biggest risk is that all nodes will end up with
the same certificate and thus Device ID.

When the model prepares a folder for syncing, it checks to
see if the configuration files this instance is using are
getting synced. If the are getting synced, and they aren't
getting ignored, a warning is emitted. The model is used
so that when a new folder is added dynamically, a warning
is also emitted.

This will not prevent a user from shooting themselves in
the foot, and will not cover all cases (e.g. symlinks).
It should provide _something_ for many users in this
situation to go on, though.
2015-10-20 12:22:27 -04:00
Jakob Borg aa42aafe33 Don't panic on clean shutdown 2015-10-20 15:59:37 +02:00
Jakob Borg 32c1a9bc45 Docs & translation update 2015-10-20 09:59:50 +02:00