2015-10-20 09:59:50 +02:00
|
|
|
|
.\" Man page generated from reStructuredText.
|
|
|
|
|
.
|
2018-09-05 07:45:24 +02:00
|
|
|
|
.TH "SYNCTHING-BEP" "7" "Sep 02, 2018" "v0.14" "Syncthing"
|
2015-10-20 09:59:50 +02:00
|
|
|
|
.SH NAME
|
|
|
|
|
syncthing-bep \- Block Exchange Protocol v1
|
|
|
|
|
.
|
|
|
|
|
.nr rst2man-indent-level 0
|
|
|
|
|
.
|
|
|
|
|
.de1 rstReportMargin
|
|
|
|
|
\\$1 \\n[an-margin]
|
|
|
|
|
level \\n[rst2man-indent-level]
|
|
|
|
|
level margin: \\n[rst2man-indent\\n[rst2man-indent-level]]
|
|
|
|
|
-
|
|
|
|
|
\\n[rst2man-indent0]
|
|
|
|
|
\\n[rst2man-indent1]
|
|
|
|
|
\\n[rst2man-indent2]
|
|
|
|
|
..
|
|
|
|
|
.de1 INDENT
|
|
|
|
|
.\" .rstReportMargin pre:
|
|
|
|
|
. RS \\$1
|
|
|
|
|
. nr rst2man-indent\\n[rst2man-indent-level] \\n[an-margin]
|
|
|
|
|
. nr rst2man-indent-level +1
|
|
|
|
|
.\" .rstReportMargin post:
|
|
|
|
|
..
|
|
|
|
|
.de UNINDENT
|
|
|
|
|
. RE
|
|
|
|
|
.\" indent \\n[an-margin]
|
|
|
|
|
.\" old: \\n[rst2man-indent\\n[rst2man-indent-level]]
|
|
|
|
|
.nr rst2man-indent-level -1
|
|
|
|
|
.\" new: \\n[rst2man-indent\\n[rst2man-indent-level]]
|
|
|
|
|
.in \\n[rst2man-indent\\n[rst2man-indent-level]]u
|
|
|
|
|
..
|
|
|
|
|
.SH INTRODUCTION AND DEFINITIONS
|
|
|
|
|
.sp
|
2017-02-05 13:54:49 +01:00
|
|
|
|
The Block Exchange Protocol (BEP) is used between two or more \fIdevices\fP thus
|
|
|
|
|
forming a \fIcluster\fP\&. Each device has one or more \fIfolders\fP of files
|
|
|
|
|
described by the \fIlocal model\fP, containing metadata and block hashes. The
|
|
|
|
|
local model is sent to the other devices in the cluster. The union of all
|
|
|
|
|
files in the local models, with files selected for highest change version,
|
|
|
|
|
forms the \fIglobal model\fP\&. Each device strives to get its folders in sync
|
|
|
|
|
with the global model by requesting missing or outdated blocks from the
|
|
|
|
|
other devices in the cluster.
|
2015-10-20 09:59:50 +02:00
|
|
|
|
.sp
|
2018-05-09 07:45:15 +02:00
|
|
|
|
File data is described and transferred in units of \fIblocks\fP, each being from
|
|
|
|
|
128 KiB (131072 bytes) to 16 MiB in size, in steps of powers of two. The
|
|
|
|
|
block size may vary between files but is constant in any given file, except
|
|
|
|
|
for the last block which may be smaller.
|
2015-10-20 09:59:50 +02:00
|
|
|
|
.sp
|
2017-11-29 07:45:17 +01:00
|
|
|
|
The key words “MUST”, “MUST NOT”, “REQUIRED”, “SHALL”, “SHALL NOT”,
|
|
|
|
|
“SHOULD”, “SHOULD NOT”, “RECOMMENDED”, “MAY”, and “OPTIONAL” in this
|
2015-10-20 09:59:50 +02:00
|
|
|
|
document are to be interpreted as described in RFC 2119.
|
|
|
|
|
.SH TRANSPORT AND AUTHENTICATION
|
|
|
|
|
.sp
|
|
|
|
|
BEP is deployed as the highest level in a protocol stack, with the lower
|
|
|
|
|
level protocols providing encryption and authentication.
|
|
|
|
|
.INDENT 0.0
|
|
|
|
|
.INDENT 3.5
|
|
|
|
|
.sp
|
|
|
|
|
.nf
|
|
|
|
|
.ft C
|
2016-04-01 09:33:42 +02:00
|
|
|
|
+\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-+
|
2015-10-20 09:59:50 +02:00
|
|
|
|
| Block Exchange Protocol |
|
|
|
|
|
|\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-|
|
|
|
|
|
| Encryption & Auth (TLS 1.2) |
|
|
|
|
|
|\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-|
|
2016-07-10 09:23:58 +02:00
|
|
|
|
| Reliable Transport |
|
2015-10-20 09:59:50 +02:00
|
|
|
|
|\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-|
|
|
|
|
|
v ... v
|
|
|
|
|
.ft P
|
|
|
|
|
.fi
|
|
|
|
|
.UNINDENT
|
|
|
|
|
.UNINDENT
|
|
|
|
|
.sp
|
|
|
|
|
The encryption and authentication layer SHALL use TLS 1.2 or a higher
|
2017-11-29 07:45:17 +01:00
|
|
|
|
revision. A strong cipher suite SHALL be used, with “strong cipher
|
|
|
|
|
suite” being defined as being without known weaknesses and providing
|
2015-10-20 09:59:50 +02:00
|
|
|
|
Perfect Forward Secrecy (PFS). Examples of strong cipher suites are
|
|
|
|
|
given at the end of this document. This is not to be taken as an
|
|
|
|
|
exhaustive list of allowed cipher suites but represents best practices
|
|
|
|
|
at the time of writing.
|
|
|
|
|
.sp
|
|
|
|
|
The exact nature of the authentication is up to the application, however
|
|
|
|
|
it SHALL be based on the TLS certificate presented at the start of the
|
|
|
|
|
connection. Possibilities include certificates signed by a common
|
|
|
|
|
trusted CA, preshared certificates, preshared certificate fingerprints
|
|
|
|
|
or certificate pinning combined with some out of band first
|
|
|
|
|
verification. The reference implementation uses preshared certificate
|
2017-11-29 07:45:17 +01:00
|
|
|
|
fingerprints (SHA\-256) referred to as “Device IDs”.
|
2015-10-20 09:59:50 +02:00
|
|
|
|
.sp
|
|
|
|
|
There is no required order or synchronization among BEP messages except
|
|
|
|
|
as noted per message type \- any message type may be sent at any time and
|
|
|
|
|
the sender need not await a response to one message before sending
|
2015-10-22 08:14:43 +02:00
|
|
|
|
another.
|
2015-10-20 09:59:50 +02:00
|
|
|
|
.sp
|
2016-05-04 21:47:08 +02:00
|
|
|
|
The underlying transport protocol MUST guarantee reliable packet delivery.
|
2016-07-10 09:23:58 +02:00
|
|
|
|
.sp
|
2017-11-29 07:45:17 +01:00
|
|
|
|
In this document, in diagrams and text, “bit 0” refers to the \fImost
|
|
|
|
|
significant\fP bit of a word; “bit 15” is thus the least significant bit of a
|
|
|
|
|
16 bit word (int16) and “bit 31” is the least significant bit of a 32 bit
|
2016-07-10 09:23:58 +02:00
|
|
|
|
word (int32). Non protocol buffer integers are always represented in network
|
|
|
|
|
byte order (i.e., big endian) and are signed unless stated otherwise, but
|
|
|
|
|
when describing message lengths negative values do not make sense and the
|
|
|
|
|
most significant bit MUST be zero.
|
|
|
|
|
.sp
|
|
|
|
|
The protocol buffer schemas in this document are in \fBproto3\fP syntax. This
|
|
|
|
|
means, among other things, that all fields are optional and will assume
|
2018-05-23 07:45:25 +02:00
|
|
|
|
their default value when missing. This does not necessarily mean that a
|
2016-07-10 09:23:58 +02:00
|
|
|
|
message is \fIvalid\fP with all fields empty \- for example, an index entry for a
|
|
|
|
|
file that does not have a name is not useful and MAY be rejected by the
|
|
|
|
|
implementation. However the folder label is for human consumption only so an
|
|
|
|
|
empty label should be accepted \- the implementation will have to choose some
|
2017-11-29 07:45:17 +01:00
|
|
|
|
way to represent the folder, perhaps by using the ID in it’s place or
|
2016-07-10 09:23:58 +02:00
|
|
|
|
automatically generating a label.
|
2016-05-04 21:47:08 +02:00
|
|
|
|
.SH PRE-AUTHENTICATION MESSAGES
|
|
|
|
|
.sp
|
|
|
|
|
AFTER establishing a connection, but BEFORE performing any authentication,
|
2016-07-10 09:23:58 +02:00
|
|
|
|
devices MUST exchange Hello messages.
|
2016-05-04 21:47:08 +02:00
|
|
|
|
.sp
|
2016-07-10 09:23:58 +02:00
|
|
|
|
Hello messages are used to carry additional information about the peer,
|
|
|
|
|
which might be of interest to the user even if the peer is not permitted to
|
|
|
|
|
communicate due to failing authentication. Note that the certificate based
|
|
|
|
|
authentication may be considered part of the TLS handshake that precedes the
|
|
|
|
|
Hello message exchange, but even in the case that a connection is rejected a
|
|
|
|
|
Hello message must be sent before the connection is terminated.
|
2016-05-04 21:47:08 +02:00
|
|
|
|
.sp
|
2016-07-10 09:23:58 +02:00
|
|
|
|
Hello messages MUST be prefixed with an int32 containing the magic number
|
|
|
|
|
\fB0x2EA7D90B\fP, followed by an int16 representing the size of the message,
|
|
|
|
|
followed by the contents of the Hello message itself.
|
2016-05-04 21:47:08 +02:00
|
|
|
|
.INDENT 0.0
|
|
|
|
|
.INDENT 3.5
|
|
|
|
|
.sp
|
|
|
|
|
.nf
|
|
|
|
|
.ft C
|
2016-07-10 09:23:58 +02:00
|
|
|
|
0 1
|
|
|
|
|
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5
|
|
|
|
|
+\-+\-+\-+\-+\-+\-+\-+\-+\-+\-+\-+\-+\-+\-+\-+\-+
|
|
|
|
|
| Magic |
|
|
|
|
|
| (32 bits) |
|
|
|
|
|
+\-+\-+\-+\-+\-+\-+\-+\-+\-+\-+\-+\-+\-+\-+\-+\-+
|
|
|
|
|
| Length |
|
|
|
|
|
+\-+\-+\-+\-+\-+\-+\-+\-+\-+\-+\-+\-+\-+\-+\-+\-+
|
|
|
|
|
/ /
|
|
|
|
|
\e Hello \e
|
|
|
|
|
/ /
|
|
|
|
|
+\-+\-+\-+\-+\-+\-+\-+\-+\-+\-+\-+\-+\-+\-+\-+\-+
|
2016-05-04 21:47:08 +02:00
|
|
|
|
.ft P
|
|
|
|
|
.fi
|
|
|
|
|
.UNINDENT
|
|
|
|
|
.UNINDENT
|
|
|
|
|
.sp
|
2016-07-10 09:23:58 +02:00
|
|
|
|
The Hello message itself is in protocol buffer format with the following schema:
|
2016-05-04 21:47:08 +02:00
|
|
|
|
.INDENT 0.0
|
|
|
|
|
.INDENT 3.5
|
|
|
|
|
.sp
|
|
|
|
|
.nf
|
|
|
|
|
.ft C
|
2016-07-10 09:23:58 +02:00
|
|
|
|
message Hello {
|
|
|
|
|
string device_name = 1;
|
|
|
|
|
string client_name = 2;
|
|
|
|
|
string client_version = 3;
|
|
|
|
|
}
|
2016-05-04 21:47:08 +02:00
|
|
|
|
.ft P
|
|
|
|
|
.fi
|
|
|
|
|
.UNINDENT
|
|
|
|
|
.UNINDENT
|
2016-07-10 09:23:58 +02:00
|
|
|
|
.SS Fields (Hello message)
|
2016-05-04 21:47:08 +02:00
|
|
|
|
.sp
|
2016-07-10 09:23:58 +02:00
|
|
|
|
The \fBdevice_name\fP is a human readable (configured or auto detected) device
|
|
|
|
|
name or host name, for the remote device.
|
|
|
|
|
.sp
|
|
|
|
|
The \fBclient_name\fP and \fBclient_version\fP identifies the implementation. The
|
|
|
|
|
values SHOULD be simple strings identifying the implementation name, as a
|
|
|
|
|
user would expect to see it, and the version string in the same manner. An
|
2017-11-29 07:45:17 +01:00
|
|
|
|
example client name is “syncthing” and an example client version is “v0.7.2”.
|
2016-07-10 09:23:58 +02:00
|
|
|
|
The client version field SHOULD follow the patterns laid out in the \fI\%Semantic
|
|
|
|
|
Versioning\fP <\fBhttp://semver.org/\fP> standard.
|
|
|
|
|
.sp
|
|
|
|
|
Immediately after exchanging Hello messages, the connection MUST be dropped
|
|
|
|
|
if the remote device does not pass authentication.
|
2016-05-04 21:47:08 +02:00
|
|
|
|
.SH POST-AUTHENTICATION MESSAGES
|
2015-10-20 09:59:50 +02:00
|
|
|
|
.sp
|
2016-07-10 09:23:58 +02:00
|
|
|
|
Every message post authentication is made up of several parts:
|
|
|
|
|
.INDENT 0.0
|
|
|
|
|
.IP \(bu 2
|
|
|
|
|
A header length word
|
|
|
|
|
.IP \(bu 2
|
|
|
|
|
A \fBHeader\fP
|
|
|
|
|
.IP \(bu 2
|
|
|
|
|
A message length word
|
|
|
|
|
.IP \(bu 2
|
|
|
|
|
A \fBMessage\fP
|
|
|
|
|
.UNINDENT
|
2015-10-20 09:59:50 +02:00
|
|
|
|
.INDENT 0.0
|
|
|
|
|
.INDENT 3.5
|
|
|
|
|
.sp
|
|
|
|
|
.nf
|
|
|
|
|
.ft C
|
2016-07-10 09:23:58 +02:00
|
|
|
|
0 1
|
|
|
|
|
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5
|
|
|
|
|
+\-+\-+\-+\-+\-+\-+\-+\-+\-+\-+\-+\-+\-+\-+\-+\-+
|
|
|
|
|
| Header Length |
|
|
|
|
|
+\-+\-+\-+\-+\-+\-+\-+\-+\-+\-+\-+\-+\-+\-+\-+\-+
|
|
|
|
|
/ /
|
|
|
|
|
\e Header \e
|
|
|
|
|
/ /
|
|
|
|
|
+\-+\-+\-+\-+\-+\-+\-+\-+\-+\-+\-+\-+\-+\-+\-+\-+
|
|
|
|
|
| Message Length |
|
|
|
|
|
| (32 bits) |
|
|
|
|
|
+\-+\-+\-+\-+\-+\-+\-+\-+\-+\-+\-+\-+\-+\-+\-+\-+
|
|
|
|
|
/ /
|
|
|
|
|
\e Message \e
|
|
|
|
|
/ /
|
|
|
|
|
+\-+\-+\-+\-+\-+\-+\-+\-+\-+\-+\-+\-+\-+\-+\-+\-+
|
2015-10-20 09:59:50 +02:00
|
|
|
|
.ft P
|
|
|
|
|
.fi
|
|
|
|
|
.UNINDENT
|
|
|
|
|
.UNINDENT
|
|
|
|
|
.sp
|
2016-07-10 09:23:58 +02:00
|
|
|
|
The header length word is 16 bits. It indicates the length of the following
|
|
|
|
|
\fBHeader\fP message. The Header is in protocol buffer format. The Header
|
|
|
|
|
describes the type and compression status of the following message.
|
2015-10-20 09:59:50 +02:00
|
|
|
|
.sp
|
2016-07-10 09:23:58 +02:00
|
|
|
|
The message is preceded by the 32 bit message length word and is one of the
|
|
|
|
|
concrete BEP messages described below, identified by the \fBtype\fP field of
|
|
|
|
|
the Header.
|
2015-10-22 08:14:43 +02:00
|
|
|
|
.sp
|
2016-07-10 09:23:58 +02:00
|
|
|
|
As always, the length words are in network byte order (big endian).
|
2015-10-22 08:14:43 +02:00
|
|
|
|
.INDENT 0.0
|
2016-07-10 09:23:58 +02:00
|
|
|
|
.INDENT 3.5
|
2015-10-20 09:59:50 +02:00
|
|
|
|
.sp
|
2016-07-10 09:23:58 +02:00
|
|
|
|
.nf
|
|
|
|
|
.ft C
|
|
|
|
|
message Header {
|
|
|
|
|
MessageType type = 1;
|
|
|
|
|
MessageCompression compression = 2;
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
enum MessageType {
|
|
|
|
|
CLUSTER_CONFIG = 0;
|
|
|
|
|
INDEX = 1;
|
|
|
|
|
INDEX_UPDATE = 2;
|
|
|
|
|
REQUEST = 3;
|
|
|
|
|
RESPONSE = 4;
|
|
|
|
|
DOWNLOAD_PROGRESS = 5;
|
|
|
|
|
PING = 6;
|
|
|
|
|
CLOSE = 7;
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
enum MessageCompression {
|
|
|
|
|
NONE = 0;
|
|
|
|
|
LZ4 = 1;
|
|
|
|
|
}
|
|
|
|
|
.ft P
|
|
|
|
|
.fi
|
2015-10-20 09:59:50 +02:00
|
|
|
|
.UNINDENT
|
|
|
|
|
.UNINDENT
|
|
|
|
|
.sp
|
2016-07-10 09:23:58 +02:00
|
|
|
|
When the \fBcompression\fP field is \fBNONE\fP, the message is directly in
|
|
|
|
|
protocol buffer format.
|
|
|
|
|
.sp
|
|
|
|
|
When the compression field is \fBLZ4\fP, the message consists of a 32 bit
|
|
|
|
|
integer describing the uncompressed message length followed by a single LZ4
|
|
|
|
|
block. After decompressing the LZ4 block it should be interpreted as a
|
|
|
|
|
protocol buffer message just as in the uncompressed case.
|
|
|
|
|
.SH MESSAGE SUBTYPES
|
|
|
|
|
.SS Cluster Config
|
2015-10-20 09:59:50 +02:00
|
|
|
|
.sp
|
|
|
|
|
This informational message provides information about the cluster
|
|
|
|
|
configuration as it pertains to the current connection. A Cluster Config
|
2016-07-10 09:23:58 +02:00
|
|
|
|
message MUST be the first post authentication message sent on a BEP
|
|
|
|
|
connection. Additional Cluster Config messages MUST NOT be sent after the
|
|
|
|
|
initial exchange.
|
|
|
|
|
.SS Protocol Buffer Schema
|
2015-10-20 09:59:50 +02:00
|
|
|
|
.INDENT 0.0
|
|
|
|
|
.INDENT 3.5
|
|
|
|
|
.sp
|
|
|
|
|
.nf
|
|
|
|
|
.ft C
|
2016-07-10 09:23:58 +02:00
|
|
|
|
message ClusterConfig {
|
|
|
|
|
repeated Folder folders = 1;
|
|
|
|
|
}
|
2015-10-20 09:59:50 +02:00
|
|
|
|
|
2016-07-10 09:23:58 +02:00
|
|
|
|
message Folder {
|
|
|
|
|
string id = 1;
|
|
|
|
|
string label = 2;
|
|
|
|
|
bool read_only = 3;
|
|
|
|
|
bool ignore_permissions = 4;
|
|
|
|
|
bool ignore_delete = 5;
|
|
|
|
|
bool disable_temp_indexes = 6;
|
2018-02-07 07:45:17 +01:00
|
|
|
|
bool paused = 7;
|
2015-10-20 09:59:50 +02:00
|
|
|
|
|
2016-07-10 09:23:58 +02:00
|
|
|
|
repeated Device devices = 16;
|
|
|
|
|
}
|
2015-10-20 09:59:50 +02:00
|
|
|
|
|
2016-07-10 09:23:58 +02:00
|
|
|
|
message Device {
|
2016-11-22 09:32:57 +01:00
|
|
|
|
bytes id = 1;
|
|
|
|
|
string name = 2;
|
|
|
|
|
repeated string addresses = 3;
|
|
|
|
|
Compression compression = 4;
|
|
|
|
|
string cert_name = 5;
|
|
|
|
|
int64 max_sequence = 6;
|
|
|
|
|
bool introducer = 7;
|
|
|
|
|
uint64 index_id = 8;
|
|
|
|
|
bool skip_introduction_removals = 9;
|
2016-07-10 09:23:58 +02:00
|
|
|
|
}
|
2015-10-20 09:59:50 +02:00
|
|
|
|
|
2016-07-10 09:23:58 +02:00
|
|
|
|
enum Compression {
|
|
|
|
|
METADATA = 0;
|
|
|
|
|
NEVER = 1;
|
|
|
|
|
ALWAYS = 2;
|
|
|
|
|
}
|
2015-10-20 09:59:50 +02:00
|
|
|
|
.ft P
|
|
|
|
|
.fi
|
|
|
|
|
.UNINDENT
|
|
|
|
|
.UNINDENT
|
2016-07-10 09:23:58 +02:00
|
|
|
|
.SS Fields (Cluster Config Message)
|
2015-10-22 08:14:43 +02:00
|
|
|
|
.sp
|
2016-07-10 09:23:58 +02:00
|
|
|
|
The \fBfolders\fP field contains the list of folders that will be synchronized
|
2015-10-22 08:14:43 +02:00
|
|
|
|
over the current connection.
|
2016-07-10 09:23:58 +02:00
|
|
|
|
.SS Fields (Folder Message)
|
2015-10-22 08:14:43 +02:00
|
|
|
|
.sp
|
2016-07-10 09:23:58 +02:00
|
|
|
|
The \fBid\fP field contains the folder ID, which is the unique identifier of
|
|
|
|
|
the folder.
|
2015-10-22 08:14:43 +02:00
|
|
|
|
.sp
|
2016-07-10 09:23:58 +02:00
|
|
|
|
The \fBlabel\fP field contains the folder label, the human readable name of
|
|
|
|
|
the folder.
|
2015-10-22 08:14:43 +02:00
|
|
|
|
.sp
|
2016-07-10 09:23:58 +02:00
|
|
|
|
The \fBread only\fP field is set for folders that the device will accept no
|
|
|
|
|
updates from the network for.
|
2016-04-01 09:33:42 +02:00
|
|
|
|
.sp
|
2016-07-10 09:23:58 +02:00
|
|
|
|
The \fBignore permissions\fP field is set for folders that the device will not
|
|
|
|
|
accept or announce file permissions for.
|
2015-10-22 08:14:43 +02:00
|
|
|
|
.sp
|
2016-07-10 09:23:58 +02:00
|
|
|
|
The \fBignore delete\fP field is set for folders that the device will ignore
|
|
|
|
|
deletes for.
|
2015-10-20 09:59:50 +02:00
|
|
|
|
.sp
|
2016-07-10 09:23:58 +02:00
|
|
|
|
The \fBdisable temp indexes\fP field is set for folders that will not dispatch
|
|
|
|
|
and do not wish to receive progress updates about partially downloaded files
|
|
|
|
|
via Download Progress messages.
|
2015-10-22 08:14:43 +02:00
|
|
|
|
.sp
|
2018-02-07 07:45:17 +01:00
|
|
|
|
The \fBpaused\fP field is set for folders that are currently paused.
|
|
|
|
|
.sp
|
2016-07-10 09:23:58 +02:00
|
|
|
|
The \fBdevices\fP field is a list of devices participating in sharing this
|
|
|
|
|
folder.
|
|
|
|
|
.SS Fields (Device Message)
|
2015-10-22 08:14:43 +02:00
|
|
|
|
.sp
|
2016-07-10 09:23:58 +02:00
|
|
|
|
The device \fBid\fP field is a 32 byte number that uniquely identifies the
|
2015-10-22 08:14:43 +02:00
|
|
|
|
device. For instance, the reference implementation uses the SHA\-256 of the
|
2015-10-20 09:59:50 +02:00
|
|
|
|
device X.509 certificate.
|
|
|
|
|
.sp
|
2016-07-10 09:23:58 +02:00
|
|
|
|
The \fBname\fP field is a human readable name assigned to the described device
|
2015-10-22 08:14:43 +02:00
|
|
|
|
by the sending device. It MAY be empty and it need not be unique.
|
|
|
|
|
.sp
|
2016-07-10 09:23:58 +02:00
|
|
|
|
The list of \fBaddresses\fP is that used by the sending device to connect to
|
2015-10-22 08:14:43 +02:00
|
|
|
|
the described device.
|
|
|
|
|
.sp
|
2016-07-10 09:23:58 +02:00
|
|
|
|
The \fBcompression\fP field indicates the compression mode in use for this
|
2015-10-22 08:14:43 +02:00
|
|
|
|
device and folder. The following values are valid:
|
|
|
|
|
.INDENT 0.0
|
|
|
|
|
.TP
|
|
|
|
|
.B 0
|
|
|
|
|
Compress metadata. This enables compression of metadata messages such as Index.
|
|
|
|
|
.TP
|
|
|
|
|
.B 1
|
|
|
|
|
Compression disabled. No compression is used on any message.
|
|
|
|
|
.TP
|
|
|
|
|
.B 2
|
|
|
|
|
Compress always. Metadata messages as well as Response messages are compressed.
|
|
|
|
|
.UNINDENT
|
|
|
|
|
.sp
|
2016-07-10 09:23:58 +02:00
|
|
|
|
The \fBcert name\fP field indicates the expected certificate name for this
|
2015-10-22 08:14:43 +02:00
|
|
|
|
device. It is commonly blank, indicating to use the implementation default.
|
|
|
|
|
.sp
|
2016-08-10 11:41:46 +02:00
|
|
|
|
The \fBmax sequence\fP field contains the highest sequence number of the files
|
|
|
|
|
in the index. See \fI\%Delta Index Exchange\fP for the usage of this field.
|
2015-10-22 08:14:43 +02:00
|
|
|
|
.sp
|
2016-07-10 09:23:58 +02:00
|
|
|
|
The \fBintroducer\fP field is set for devices that are trusted as cluster
|
|
|
|
|
introducers.
|
2016-07-26 10:53:00 +02:00
|
|
|
|
.sp
|
|
|
|
|
The \fBindex id\fP field contains the unique identifier for the current set of
|
|
|
|
|
index data. See \fI\%Delta Index Exchange\fP for the usage of this field.
|
2016-11-22 09:32:57 +01:00
|
|
|
|
.sp
|
|
|
|
|
The \fBskip introduction removals\fP field signifies if the remote device has
|
|
|
|
|
opted to ignore introduction removals for the given device. This setting is
|
|
|
|
|
copied across as we are being introduced to a new device.
|
2016-07-10 09:23:58 +02:00
|
|
|
|
.SS Index and Index Update
|
2015-10-20 09:59:50 +02:00
|
|
|
|
.sp
|
|
|
|
|
The Index and Index Update messages define the contents of the senders
|
|
|
|
|
folder. An Index message represents the full contents of the folder and
|
|
|
|
|
thus supersedes any previous index. An Index Update amends an existing
|
|
|
|
|
index with new information, not affecting any entries not included in
|
|
|
|
|
the message. An Index Update MAY NOT be sent unless preceded by an
|
2016-08-10 11:41:46 +02:00
|
|
|
|
Index, unless a non\-zero Max Sequence has been announced for the
|
2015-10-20 09:59:50 +02:00
|
|
|
|
given folder by the peer device.
|
2016-07-10 09:23:58 +02:00
|
|
|
|
.sp
|
|
|
|
|
The Index and Index Update messages are currently identical in format,
|
|
|
|
|
although this is not guaranteed to be the case in the future.
|
|
|
|
|
.SS Protocol Buffer Schema
|
2015-10-20 09:59:50 +02:00
|
|
|
|
.INDENT 0.0
|
|
|
|
|
.INDENT 3.5
|
|
|
|
|
.sp
|
|
|
|
|
.nf
|
|
|
|
|
.ft C
|
2016-07-10 09:23:58 +02:00
|
|
|
|
message Index {
|
|
|
|
|
string folder = 1;
|
|
|
|
|
repeated FileInfo files = 2;
|
|
|
|
|
}
|
2015-10-20 09:59:50 +02:00
|
|
|
|
|
2016-07-10 09:23:58 +02:00
|
|
|
|
message IndexUpdate {
|
|
|
|
|
string folder = 1;
|
|
|
|
|
repeated FileInfo files = 2;
|
|
|
|
|
}
|
2015-10-20 09:59:50 +02:00
|
|
|
|
|
2016-07-10 09:23:58 +02:00
|
|
|
|
message FileInfo {
|
|
|
|
|
string name = 1;
|
|
|
|
|
FileInfoType type = 2;
|
|
|
|
|
int64 size = 3;
|
|
|
|
|
uint32 permissions = 4;
|
2016-08-10 11:41:46 +02:00
|
|
|
|
int64 modified_s = 5;
|
|
|
|
|
int32 modified_ns = 11;
|
2017-01-23 21:58:53 +01:00
|
|
|
|
uint64 modified_by = 12;
|
2016-07-10 09:23:58 +02:00
|
|
|
|
bool deleted = 6;
|
|
|
|
|
bool invalid = 7;
|
|
|
|
|
bool no_permissions = 8;
|
|
|
|
|
Vector version = 9;
|
2018-05-09 07:45:15 +02:00
|
|
|
|
int64 sequence = 10;
|
|
|
|
|
int32 block_size = 13;
|
2016-07-10 09:23:58 +02:00
|
|
|
|
|
2016-12-13 11:29:40 +01:00
|
|
|
|
repeated BlockInfo Blocks = 16;
|
|
|
|
|
string symlink_target = 17;
|
2016-07-10 09:23:58 +02:00
|
|
|
|
}
|
2015-10-20 09:59:50 +02:00
|
|
|
|
|
2016-07-10 09:23:58 +02:00
|
|
|
|
enum FileInfoType {
|
|
|
|
|
FILE = 0;
|
|
|
|
|
DIRECTORY = 1;
|
2017-02-23 07:47:07 +01:00
|
|
|
|
SYMLINK_FILE = 2 [deprecated = true];
|
|
|
|
|
SYMLINK_DIRECTORY = 3 [deprecated = true];
|
|
|
|
|
SYMLINK = 4;
|
2016-07-10 09:23:58 +02:00
|
|
|
|
}
|
2015-10-20 09:59:50 +02:00
|
|
|
|
|
2016-07-10 09:23:58 +02:00
|
|
|
|
message BlockInfo {
|
2018-02-07 07:45:17 +01:00
|
|
|
|
int64 offset = 1;
|
|
|
|
|
int32 size = 2;
|
|
|
|
|
bytes hash = 3;
|
|
|
|
|
uint32 weak_hash = 4;
|
2016-07-10 09:23:58 +02:00
|
|
|
|
}
|
2015-10-20 09:59:50 +02:00
|
|
|
|
|
2016-07-10 09:23:58 +02:00
|
|
|
|
message Vector {
|
|
|
|
|
repeated Counter counters = 1;
|
|
|
|
|
}
|
2015-10-20 09:59:50 +02:00
|
|
|
|
|
2016-07-10 09:23:58 +02:00
|
|
|
|
message Counter {
|
|
|
|
|
uint64 id = 1;
|
|
|
|
|
uint64 value = 2;
|
|
|
|
|
}
|
2015-10-20 09:59:50 +02:00
|
|
|
|
.ft P
|
|
|
|
|
.fi
|
|
|
|
|
.UNINDENT
|
|
|
|
|
.UNINDENT
|
2015-10-22 08:14:43 +02:00
|
|
|
|
.SS Fields (Index Message)
|
2015-10-20 09:59:50 +02:00
|
|
|
|
.sp
|
2016-07-10 09:23:58 +02:00
|
|
|
|
The \fBfolder\fP field identifies the folder that the index message pertains to.
|
2015-10-20 09:59:50 +02:00
|
|
|
|
.sp
|
2016-07-10 09:23:58 +02:00
|
|
|
|
The \fBfiles\fP field is a list of files making up the index information.
|
|
|
|
|
.SS Fields (FileInfo Message)
|
2015-10-22 08:14:43 +02:00
|
|
|
|
.sp
|
2016-07-10 09:23:58 +02:00
|
|
|
|
The \fBname\fP is the file name path relative to the folder root. Like all
|
2015-10-20 09:59:50 +02:00
|
|
|
|
strings in BEP, the Name is always in UTF\-8 NFC regardless of operating
|
2016-07-10 09:23:58 +02:00
|
|
|
|
system or file system specific conventions. The name field uses the slash
|
2017-11-29 07:45:17 +01:00
|
|
|
|
character (“/”) as path separator, regardless of the implementation’s
|
2016-07-10 09:23:58 +02:00
|
|
|
|
operating system conventions. The combination of folder and name uniquely
|
|
|
|
|
identifies each file in a cluster.
|
|
|
|
|
.sp
|
|
|
|
|
The \fBtype\fP field contains the type of the described item. The type is one
|
2017-02-23 07:47:07 +01:00
|
|
|
|
of \fBfile (0)\fP, \fBdirectory (1)\fP, or \fBsymlink (4)\fP\&.
|
2016-07-10 09:23:58 +02:00
|
|
|
|
.sp
|
|
|
|
|
The \fBsize\fP field contains the size of the file, in bytes. For directories
|
2016-12-13 11:29:40 +01:00
|
|
|
|
and symlinks the size is zero.
|
2016-07-10 09:23:58 +02:00
|
|
|
|
.sp
|
|
|
|
|
The \fBpermissions\fP field holds the common Unix permission bits. An
|
|
|
|
|
implementation MAY ignore or interpret these as is suitable on the host
|
|
|
|
|
operating system.
|
|
|
|
|
.sp
|
2017-02-23 07:47:07 +01:00
|
|
|
|
The \fBmodified_s\fP time is expressed as the number of seconds since the Unix
|
2016-08-10 11:41:46 +02:00
|
|
|
|
Epoch (1970\-01\-01 00:00:00 UTC). The \fBmodified_ns\fP field holds the
|
|
|
|
|
nanosecond part of the modification time.
|
2015-10-20 09:59:50 +02:00
|
|
|
|
.sp
|
2017-01-23 21:58:53 +01:00
|
|
|
|
The \fBmodified_by\fP field holds the short id of the client that last made
|
|
|
|
|
any modification to the file whether add, change or delete. This will be
|
|
|
|
|
overwritten every time a change is made to the file by the last client to do
|
|
|
|
|
so and so does not hold history.
|
|
|
|
|
.sp
|
2016-07-10 09:23:58 +02:00
|
|
|
|
The \fBdeleted\fP field is set when the file has been deleted. The block list
|
|
|
|
|
SHALL be of length zero and the modification time indicates the time of
|
|
|
|
|
deletion or, if the time of deletion is not reliably determinable, the last
|
|
|
|
|
known modification time.
|
|
|
|
|
.sp
|
|
|
|
|
The \fBinvalid\fP field is set when the file is invalid and unavailable for
|
|
|
|
|
synchronization. A peer MAY set this bit to indicate that it can temporarily
|
|
|
|
|
not serve data for the file.
|
|
|
|
|
.sp
|
|
|
|
|
The \fBno permissions\fP field is set when there is no permission information
|
|
|
|
|
for the file. This is the case when it originates on a file system which
|
|
|
|
|
does not support permissions. Changes to only permission bits SHOULD be
|
|
|
|
|
disregarded on files with this bit set. The permissions bits MUST be set to
|
|
|
|
|
the octal value 0666.
|
|
|
|
|
.sp
|
|
|
|
|
The \fBversion\fP field is a version vector describing the updates performed
|
|
|
|
|
to a file by all members in the cluster. Each counter in the version vector
|
|
|
|
|
is an ID\-Value tuple. The ID is the first 64 bits of the device ID. The
|
|
|
|
|
Value is a simple incrementing counter, starting at zero. The combination of
|
|
|
|
|
Folder, Name and Version uniquely identifies the contents of a file at a
|
|
|
|
|
given point in time.
|
|
|
|
|
.sp
|
2016-08-10 11:41:46 +02:00
|
|
|
|
The \fBsequence\fP field is the value of a device local monotonic clock at the
|
|
|
|
|
time of last local database update to a file. The clock ticks on every local
|
|
|
|
|
database update, thus forming a sequence number over database updates.
|
2015-10-20 09:59:50 +02:00
|
|
|
|
.sp
|
2018-05-09 07:45:15 +02:00
|
|
|
|
The \fBblock_size\fP field is the size, in bytes, of each individual block in
|
|
|
|
|
the block list (except, possibly, the last block). If this field is missing
|
|
|
|
|
or zero, the block size is assumed to be 128 KiB (131072 bytes). Valid
|
|
|
|
|
values of this field are the powers of two from 128 KiB through 16 MiB. See
|
|
|
|
|
also \fI\%Selection of Block Size\fP\&.
|
|
|
|
|
.sp
|
2016-07-10 09:23:58 +02:00
|
|
|
|
The \fBblocks\fP list contains the size and hash for each block in the file.
|
2018-05-09 07:45:15 +02:00
|
|
|
|
Each block represents a \fBblock_size\fP\-sized slice of the file, except for
|
|
|
|
|
the last block which may represent a smaller amount of data. The block list
|
|
|
|
|
is empty for directories and symlinks.
|
2016-12-13 11:29:40 +01:00
|
|
|
|
.sp
|
|
|
|
|
The \fBsymlink_target\fP field contains the symlink target, for entries of
|
|
|
|
|
symlink type. It is empty for all other entry types.
|
2016-07-10 09:23:58 +02:00
|
|
|
|
.SS Request
|
2015-10-20 09:59:50 +02:00
|
|
|
|
.sp
|
|
|
|
|
The Request message expresses the desire to receive a data block
|
2017-11-29 07:45:17 +01:00
|
|
|
|
corresponding to a part of a certain file in the peer’s folder.
|
2016-07-10 09:23:58 +02:00
|
|
|
|
.SS Protocol Buffer Schema
|
2015-10-20 09:59:50 +02:00
|
|
|
|
.INDENT 0.0
|
|
|
|
|
.INDENT 3.5
|
|
|
|
|
.sp
|
|
|
|
|
.nf
|
|
|
|
|
.ft C
|
2016-07-10 09:23:58 +02:00
|
|
|
|
message Request {
|
|
|
|
|
int32 id = 1;
|
|
|
|
|
string folder = 2;
|
|
|
|
|
string name = 3;
|
|
|
|
|
int64 offset = 4;
|
|
|
|
|
int32 size = 5;
|
|
|
|
|
bytes hash = 6;
|
|
|
|
|
bool from_temporary = 7;
|
|
|
|
|
}
|
2015-10-20 09:59:50 +02:00
|
|
|
|
.ft P
|
|
|
|
|
.fi
|
|
|
|
|
.UNINDENT
|
|
|
|
|
.UNINDENT
|
|
|
|
|
.SS Fields
|
|
|
|
|
.sp
|
2016-07-10 09:23:58 +02:00
|
|
|
|
The \fBid\fP is the request identifier. It will be matched in the
|
2017-11-29 07:45:17 +01:00
|
|
|
|
corresponding \fBResponse\fP message. Each outstanding request must have a
|
2016-07-10 09:23:58 +02:00
|
|
|
|
unique ID.
|
2015-10-20 09:59:50 +02:00
|
|
|
|
.sp
|
2016-07-10 09:23:58 +02:00
|
|
|
|
The \fBfolder\fP and \fBname\fP fields are as documented for the Index message.
|
|
|
|
|
The \fBoffset\fP and \fBsize\fP fields specify the region of the file to be
|
|
|
|
|
transferred. This SHOULD equate to exactly one block as seen in an Index
|
|
|
|
|
message.
|
2016-05-04 21:47:08 +02:00
|
|
|
|
.sp
|
2016-07-10 09:23:58 +02:00
|
|
|
|
The \fIhash\fP field MAY be set to the expected hash value of the block. If set,
|
|
|
|
|
the other device SHOULD ensure that the transmitted block matches the
|
|
|
|
|
requested hash. The other device MAY reuse a block from a different file and
|
|
|
|
|
offset having the same size and hash, if one exists.
|
2015-10-20 09:59:50 +02:00
|
|
|
|
.sp
|
2016-07-10 09:23:58 +02:00
|
|
|
|
The \fBfrom temporary\fP field is set to indicate that the read should be
|
2017-11-29 07:45:17 +01:00
|
|
|
|
performed from the temporary file (converting name to it’s temporary form)
|
2016-07-10 09:23:58 +02:00
|
|
|
|
and falling back to the non temporary file if any error occurs. Knowledge of
|
|
|
|
|
contents of temporary files comes from DownloadProgress messages.
|
|
|
|
|
.SS Response
|
2015-10-20 09:59:50 +02:00
|
|
|
|
.sp
|
|
|
|
|
The Response message is sent in response to a Request message.
|
2016-07-10 09:23:58 +02:00
|
|
|
|
.SS Protocol Buffer Schema
|
2015-10-20 09:59:50 +02:00
|
|
|
|
.INDENT 0.0
|
|
|
|
|
.INDENT 3.5
|
|
|
|
|
.sp
|
|
|
|
|
.nf
|
|
|
|
|
.ft C
|
2016-07-10 09:23:58 +02:00
|
|
|
|
message Response {
|
|
|
|
|
int32 id = 1;
|
|
|
|
|
bytes data = 2;
|
|
|
|
|
ErrorCode code = 3;
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
enum ErrorCode {
|
|
|
|
|
NO_ERROR = 0;
|
|
|
|
|
GENERIC = 1;
|
|
|
|
|
NO_SUCH_FILE = 2;
|
|
|
|
|
INVALID_FILE = 3;
|
|
|
|
|
}
|
2015-10-20 09:59:50 +02:00
|
|
|
|
.ft P
|
|
|
|
|
.fi
|
|
|
|
|
.UNINDENT
|
|
|
|
|
.UNINDENT
|
|
|
|
|
.SS Fields
|
|
|
|
|
.sp
|
2016-07-10 09:23:58 +02:00
|
|
|
|
The \fBid\fP field is the request identifier. It must match the ID of the
|
|
|
|
|
\fBRequest\fP that is being responded to.
|
|
|
|
|
.sp
|
|
|
|
|
The \fBdata\fP field contains either the requested data block or is empty if
|
|
|
|
|
the requested block is not available.
|
2015-10-20 09:59:50 +02:00
|
|
|
|
.sp
|
2016-07-10 09:23:58 +02:00
|
|
|
|
The \fBcode\fP field contains an error code describing the reason a Request
|
|
|
|
|
could not be fulfilled, in the case where zero length data was returned. The
|
|
|
|
|
following values are defined:
|
2015-10-20 09:59:50 +02:00
|
|
|
|
.INDENT 0.0
|
|
|
|
|
.TP
|
|
|
|
|
.B 0
|
2016-07-10 09:23:58 +02:00
|
|
|
|
No Error (data should be present)
|
2015-10-20 09:59:50 +02:00
|
|
|
|
.TP
|
|
|
|
|
.B 1
|
|
|
|
|
Generic Error
|
|
|
|
|
.TP
|
|
|
|
|
.B 2
|
|
|
|
|
No Such File (the requested file does not exist, or the offset is
|
|
|
|
|
outside the acceptable range for the file)
|
|
|
|
|
.TP
|
|
|
|
|
.B 3
|
|
|
|
|
Invalid (file exists but has invalid bit set or is otherwise
|
|
|
|
|
unavailable)
|
|
|
|
|
.UNINDENT
|
2016-07-10 09:23:58 +02:00
|
|
|
|
.SS DownloadProgress
|
2016-05-04 21:47:08 +02:00
|
|
|
|
.sp
|
|
|
|
|
The DownloadProgress message is used to notify remote devices about partial
|
|
|
|
|
availability of files. By default, these messages are sent every 5 seconds,
|
|
|
|
|
and only in the cases where progress or state changes have been detected.
|
|
|
|
|
Each DownloadProgress message is addressed to a specific folder and MUST
|
2016-07-10 09:23:58 +02:00
|
|
|
|
contain zero or more FileDownloadProgressUpdate messages.
|
|
|
|
|
.SS Protocol Buffer Schema
|
2016-05-04 21:47:08 +02:00
|
|
|
|
.INDENT 0.0
|
|
|
|
|
.INDENT 3.5
|
|
|
|
|
.sp
|
|
|
|
|
.nf
|
|
|
|
|
.ft C
|
2016-07-10 09:23:58 +02:00
|
|
|
|
message DownloadProgress {
|
|
|
|
|
string folder = 1;
|
|
|
|
|
repeated FileDownloadProgressUpdate updates = 2;
|
|
|
|
|
}
|
2016-05-04 21:47:08 +02:00
|
|
|
|
|
2016-07-10 09:23:58 +02:00
|
|
|
|
message FileDownloadProgressUpdate {
|
|
|
|
|
FileDownloadProgressUpdateType update_type = 1;
|
|
|
|
|
string name = 2;
|
|
|
|
|
Vector version = 3;
|
|
|
|
|
repeated int32 block_indexes = 4;
|
|
|
|
|
}
|
2016-05-04 21:47:08 +02:00
|
|
|
|
|
2016-07-10 09:23:58 +02:00
|
|
|
|
enum FileDownloadProgressUpdateType {
|
|
|
|
|
APPEND = 0;
|
|
|
|
|
FORGET = 1;
|
|
|
|
|
}
|
2016-05-04 21:47:08 +02:00
|
|
|
|
.ft P
|
|
|
|
|
.fi
|
|
|
|
|
.UNINDENT
|
|
|
|
|
.UNINDENT
|
|
|
|
|
.SS Fields (DownloadProgress Message)
|
|
|
|
|
.sp
|
2016-07-10 09:23:58 +02:00
|
|
|
|
The \fBfolder\fP field represents the ID of the folder for which the update is
|
|
|
|
|
being provided.
|
2016-05-04 21:47:08 +02:00
|
|
|
|
.sp
|
2016-07-10 09:23:58 +02:00
|
|
|
|
The \fBupdates\fP field is a list of progress update messages.
|
|
|
|
|
.SS Fields (FileDownloadProgressUpdate Message)
|
2016-05-04 21:47:08 +02:00
|
|
|
|
.sp
|
2016-07-10 09:23:58 +02:00
|
|
|
|
The \fBupdate type\fP indicates whether the update is of type \fBappend (0)\fP
|
|
|
|
|
(new blocks are available) or \fBforget (1)\fP (the file transfer has
|
|
|
|
|
completed or failed).
|
2016-05-04 21:47:08 +02:00
|
|
|
|
.sp
|
2016-07-10 09:23:58 +02:00
|
|
|
|
The \fBname\fP field defines the file name from the global index for which
|
|
|
|
|
this update is being sent.
|
2016-05-04 21:47:08 +02:00
|
|
|
|
.sp
|
2016-07-10 09:23:58 +02:00
|
|
|
|
The \fBversion\fP message defines the version of the file for which this
|
2016-05-04 21:47:08 +02:00
|
|
|
|
update is being sent.
|
|
|
|
|
.sp
|
2016-07-10 09:23:58 +02:00
|
|
|
|
The \fBblock indexes\fP field is a list of positive integers, where each
|
|
|
|
|
integer represents the index of the block in the FileInfo message Blocks
|
|
|
|
|
array that has become available for download.
|
2016-05-04 21:47:08 +02:00
|
|
|
|
.sp
|
2016-07-10 09:23:58 +02:00
|
|
|
|
For example an integer with value 3 represents that the data defined in the
|
|
|
|
|
fourth BlockInfo message of the FileInfo message of that file is now
|
|
|
|
|
available. Please note that matching should be done on \fBname\fP AND
|
|
|
|
|
\fBversion\fP\&. Furthermore, each update received is incremental, for example
|
|
|
|
|
the initial update message might contain indexes 0, 1, 2, an update 5
|
|
|
|
|
seconds later might contain indexes 3, 4, 5 which should be appended to the
|
|
|
|
|
original list, which implies that blocks 0\-5 are currently available.
|
|
|
|
|
.sp
|
|
|
|
|
Block indexes MAY be added in any order. An implementation MUST NOT assume
|
|
|
|
|
that block indexes are added in any specific order.
|
|
|
|
|
.sp
|
|
|
|
|
The \fBforget\fP field being set implies that previously advertised file is no
|
|
|
|
|
longer available, therefore the list of block indexes should be truncated.
|
|
|
|
|
.sp
|
|
|
|
|
Messages with the \fBforget\fP field set MUST NOT have any block indexes.
|
|
|
|
|
.sp
|
|
|
|
|
Any update message which is being sent for a different \fBversion\fP of the
|
|
|
|
|
same file name must be preceded with an update message for the old version
|
|
|
|
|
of that file with the \fBforget\fP field set.
|
|
|
|
|
.sp
|
|
|
|
|
As a safeguard on the receiving side, the value of \fBversion\fP changing
|
|
|
|
|
between update messages implies that the file has changed and that any
|
|
|
|
|
indexes previously advertised are no longer available. The list of available
|
|
|
|
|
block indexes MUST be replaced (rather than appended) with the indexes
|
|
|
|
|
specified in this message.
|
|
|
|
|
.SS Ping
|
|
|
|
|
.sp
|
|
|
|
|
The Ping message is used to determine that a connection is alive, and to
|
|
|
|
|
keep connections alive through state tracking network elements such as
|
|
|
|
|
firewalls and NAT gateways. A Ping message is sent every 90 seconds, if no
|
|
|
|
|
other message has been sent in the preceding 90 seconds.
|
|
|
|
|
.SS Protocol Buffer Schema
|
2015-10-20 09:59:50 +02:00
|
|
|
|
.INDENT 0.0
|
|
|
|
|
.INDENT 3.5
|
|
|
|
|
.sp
|
|
|
|
|
.nf
|
|
|
|
|
.ft C
|
2016-07-10 09:23:58 +02:00
|
|
|
|
message Ping {
|
|
|
|
|
}
|
2015-10-20 09:59:50 +02:00
|
|
|
|
.ft P
|
|
|
|
|
.fi
|
|
|
|
|
.UNINDENT
|
|
|
|
|
.UNINDENT
|
2016-07-10 09:23:58 +02:00
|
|
|
|
.SS Close
|
2015-10-20 09:59:50 +02:00
|
|
|
|
.sp
|
2016-07-10 09:23:58 +02:00
|
|
|
|
The Close message MAY be sent to indicate that the connection will be torn
|
|
|
|
|
down due to an error condition. A Close message MUST NOT be followed by
|
|
|
|
|
further messages.
|
|
|
|
|
.SS Protocol Buffer Schema
|
2015-10-20 09:59:50 +02:00
|
|
|
|
.INDENT 0.0
|
|
|
|
|
.INDENT 3.5
|
|
|
|
|
.sp
|
|
|
|
|
.nf
|
|
|
|
|
.ft C
|
2016-07-10 09:23:58 +02:00
|
|
|
|
message Close {
|
|
|
|
|
string reason = 1;
|
2015-10-20 09:59:50 +02:00
|
|
|
|
}
|
|
|
|
|
.ft P
|
|
|
|
|
.fi
|
|
|
|
|
.UNINDENT
|
|
|
|
|
.UNINDENT
|
2016-07-10 09:23:58 +02:00
|
|
|
|
.SS Fields
|
|
|
|
|
.sp
|
|
|
|
|
The \fBreason\fP field contains a human readable description of the error
|
|
|
|
|
condition.
|
2015-10-20 09:59:50 +02:00
|
|
|
|
.SH SHARING MODES
|
|
|
|
|
.SS Trusted
|
|
|
|
|
.sp
|
|
|
|
|
Trusted mode is the default sharing mode. Updates are exchanged in both
|
|
|
|
|
directions.
|
|
|
|
|
.INDENT 0.0
|
|
|
|
|
.INDENT 3.5
|
|
|
|
|
.sp
|
|
|
|
|
.nf
|
|
|
|
|
.ft C
|
|
|
|
|
+\-\-\-\-\-\-\-\-\-\-\-\-+ Updates /\-\-\-\-\-\-\-\-\-\e
|
|
|
|
|
| | \-\-\-\-\-\-\-\-\-\-\-> / \e
|
|
|
|
|
| Device | | Cluster |
|
|
|
|
|
| | <\-\-\-\-\-\-\-\-\-\-\- \e /
|
|
|
|
|
+\-\-\-\-\-\-\-\-\-\-\-\-+ Updates \e\-\-\-\-\-\-\-\-\-/
|
|
|
|
|
.ft P
|
|
|
|
|
.fi
|
|
|
|
|
.UNINDENT
|
|
|
|
|
.UNINDENT
|
2017-01-10 08:41:14 +01:00
|
|
|
|
.SS Send Only
|
2015-10-20 09:59:50 +02:00
|
|
|
|
.sp
|
2018-07-25 07:45:29 +02:00
|
|
|
|
In send only mode, a device does not apply any updates from the cluster, but
|
|
|
|
|
publishes changes of its local folder to the cluster as usual.
|
2015-10-20 09:59:50 +02:00
|
|
|
|
.INDENT 0.0
|
|
|
|
|
.INDENT 3.5
|
|
|
|
|
.sp
|
|
|
|
|
.nf
|
|
|
|
|
.ft C
|
|
|
|
|
+\-\-\-\-\-\-\-\-\-\-\-\-+ Updates /\-\-\-\-\-\-\-\-\-\e
|
|
|
|
|
| | \-\-\-\-\-\-\-\-\-\-\-> / \e
|
|
|
|
|
| Device | | Cluster |
|
|
|
|
|
| | \e /
|
2018-07-25 07:45:29 +02:00
|
|
|
|
+\-\-\-\-\-\-\-\-\-\-\-\-+ \e\-\-\-\-\-\-\-\-\-/
|
|
|
|
|
.ft P
|
|
|
|
|
.fi
|
|
|
|
|
.UNINDENT
|
|
|
|
|
.UNINDENT
|
|
|
|
|
.SS Receive Only
|
|
|
|
|
.sp
|
|
|
|
|
In receive only mode, a device does not send any updates to the cluster, but
|
|
|
|
|
accepts changes to its local folder from the cluster as usual.
|
|
|
|
|
.INDENT 0.0
|
|
|
|
|
.INDENT 3.5
|
|
|
|
|
.sp
|
|
|
|
|
.nf
|
|
|
|
|
.ft C
|
|
|
|
|
+\-\-\-\-\-\-\-\-\-\-\-\-+ Updates /\-\-\-\-\-\-\-\-\-\e
|
|
|
|
|
| | <\-\-\-\-\-\-\-\-\-\-\- / \e
|
|
|
|
|
| Device | | Cluster |
|
|
|
|
|
| | \e /
|
2015-10-20 09:59:50 +02:00
|
|
|
|
+\-\-\-\-\-\-\-\-\-\-\-\-+ \e\-\-\-\-\-\-\-\-\-/
|
|
|
|
|
.ft P
|
|
|
|
|
.fi
|
|
|
|
|
.UNINDENT
|
|
|
|
|
.UNINDENT
|
2016-07-26 10:53:00 +02:00
|
|
|
|
.SH DELTA INDEX EXCHANGE
|
|
|
|
|
.sp
|
|
|
|
|
Index data must be exchanged whenever two devices connect so that one knows
|
|
|
|
|
the files available on the other. In the most basic case this happens by way
|
|
|
|
|
of sending an \fBIndex\fP message followed by one or more \fBIndex Update\fP
|
|
|
|
|
messages. Any previous index data known for a remote device is removed and
|
|
|
|
|
replaced with the new index data received in an \fBIndex\fP message, while the
|
|
|
|
|
contents of an \fBIndex Update\fP message is simply added to the existing
|
|
|
|
|
index data.
|
|
|
|
|
.sp
|
|
|
|
|
For situations with large indexes or frequent reconnects this can be quite
|
|
|
|
|
inefficient. A mechanism can then be used to retain index data between
|
|
|
|
|
connections and only transmit any changes since that data on connection
|
2017-11-29 07:45:17 +01:00
|
|
|
|
start. This is called “delta indexes”. To enable this mechanism the
|
2016-08-10 11:41:46 +02:00
|
|
|
|
\fBsequence\fP and \fBindex ID\fP fields are used.
|
2016-07-26 10:53:00 +02:00
|
|
|
|
.INDENT 0.0
|
|
|
|
|
.TP
|
2016-08-10 11:41:46 +02:00
|
|
|
|
.B Sequence:
|
|
|
|
|
Each index item (i.e., file, directory or symlink) has a sequence number
|
2016-07-26 10:53:00 +02:00
|
|
|
|
field. It contains the value of a counter at the time the index item was
|
|
|
|
|
updated. The counter increments by one for each change. That is, as files
|
2016-08-10 11:41:46 +02:00
|
|
|
|
are scanned and added to the index they get assigned sequence numbers
|
|
|
|
|
1, 2, 3 and so on. The next file to be changed or detected gets sequence
|
|
|
|
|
number 4, and future updates continue in the same fashion.
|
2016-07-26 10:53:00 +02:00
|
|
|
|
.TP
|
|
|
|
|
.B Index ID:
|
|
|
|
|
Each folder has an Index ID. This is a 64 bit random identifier set at
|
|
|
|
|
index creation time.
|
|
|
|
|
.UNINDENT
|
|
|
|
|
.sp
|
2016-08-10 11:41:46 +02:00
|
|
|
|
Given the above, we know that the tuple {index ID, maximum sequence number}
|
2016-07-26 10:53:00 +02:00
|
|
|
|
uniquely identifies a point in time of a given index. Any further changes
|
2016-08-10 11:41:46 +02:00
|
|
|
|
will increase the sequence number of some item, and thus the maximum
|
|
|
|
|
sequence number for the index itself. Should the index be reset or removed
|
|
|
|
|
(i.e., the sequence number reset to zero), a new index ID must be generated.
|
2016-07-26 10:53:00 +02:00
|
|
|
|
.sp
|
2016-08-10 11:41:46 +02:00
|
|
|
|
By letting a device know the {index ID, maximum sequence number} we have for
|
2016-07-26 10:53:00 +02:00
|
|
|
|
their index data, that device can arrange to only transmit \fBIndex Update\fP
|
2016-08-10 11:41:46 +02:00
|
|
|
|
messages for items with a higher sequence number. This is the delta index
|
|
|
|
|
mechanism.
|
2016-07-26 10:53:00 +02:00
|
|
|
|
.sp
|
2016-08-10 11:41:46 +02:00
|
|
|
|
The index ID and maximum sequence number known for each device is
|
|
|
|
|
transmitted in the \fBCluster Config\fP message at connection start.
|
2016-07-26 10:53:00 +02:00
|
|
|
|
.sp
|
|
|
|
|
For this mechanism to be reliable it is essential that outgoing index
|
2016-08-10 11:41:46 +02:00
|
|
|
|
information is ordered by increasing sequence number. Devices announcing a
|
|
|
|
|
non\-zero index ID in the \fBCluster Config\fP message MUST send all index data
|
|
|
|
|
ordered by increasing sequence number. Devices not intending to participate
|
|
|
|
|
in delta index exchange MUST send a zero index ID or, equivalently, not send
|
|
|
|
|
the \fBindex_id\fP attribute at all.
|
2015-10-20 09:59:50 +02:00
|
|
|
|
.SH MESSAGE LIMITS
|
|
|
|
|
.sp
|
2016-07-10 09:23:58 +02:00
|
|
|
|
An implementation MAY impose reasonable limits on the length of messages and
|
|
|
|
|
message fields to aid robustness in the face of corruption or broken
|
|
|
|
|
implementations. An implementation should strive to keep messages short
|
|
|
|
|
and to the point, favouring more and smaller messages over fewer and larger.
|
|
|
|
|
For example, favour a smaller Index message followed by one or more Index
|
|
|
|
|
Update messages rather than sending a very large Index message.
|
|
|
|
|
.sp
|
|
|
|
|
The Syncthing implementation imposes a hard limit of 500,000,000 bytes on
|
|
|
|
|
all messages. Attempting to send or receive a larger message will result in
|
2016-07-26 10:53:00 +02:00
|
|
|
|
a connection close. This size was chosen to accommodate Index messages
|
2017-11-29 07:45:17 +01:00
|
|
|
|
containing a large block list. It’s intended that the limit may be further
|
2016-07-10 09:23:58 +02:00
|
|
|
|
reduced in a future protocol update supporting variable block sizes (and
|
|
|
|
|
thus shorter block lists for large files).
|
2018-05-09 07:45:15 +02:00
|
|
|
|
.SH SELECTION OF BLOCK SIZE
|
|
|
|
|
.sp
|
|
|
|
|
The desired block size for any given file is the smallest block size that
|
|
|
|
|
results in fewer than 2000 blocks, or the maximum block size for larger
|
|
|
|
|
files. This rule results in the following table of block sizes per file
|
|
|
|
|
size:
|
|
|
|
|
.TS
|
|
|
|
|
center;
|
|
|
|
|
|l|l|.
|
|
|
|
|
_
|
|
|
|
|
T{
|
|
|
|
|
File Size
|
|
|
|
|
T} T{
|
|
|
|
|
Block Size
|
|
|
|
|
T}
|
|
|
|
|
_
|
|
|
|
|
T{
|
|
|
|
|
0 \- 250 MiB
|
|
|
|
|
T} T{
|
|
|
|
|
128 KiB
|
|
|
|
|
T}
|
|
|
|
|
_
|
|
|
|
|
T{
|
|
|
|
|
250 MiB \- 500 MiB
|
|
|
|
|
T} T{
|
|
|
|
|
256 KiB
|
|
|
|
|
T}
|
|
|
|
|
_
|
|
|
|
|
T{
|
|
|
|
|
500 MiB \- 1 GiB
|
|
|
|
|
T} T{
|
|
|
|
|
512 KiB
|
|
|
|
|
T}
|
|
|
|
|
_
|
|
|
|
|
T{
|
|
|
|
|
1 GiB \- 2 GiB
|
|
|
|
|
T} T{
|
|
|
|
|
1 MiB
|
|
|
|
|
T}
|
|
|
|
|
_
|
|
|
|
|
T{
|
|
|
|
|
2 GiB \- 4 GiB
|
|
|
|
|
T} T{
|
|
|
|
|
2 MiB
|
|
|
|
|
T}
|
|
|
|
|
_
|
|
|
|
|
T{
|
|
|
|
|
4 GiB \- 8 GiB
|
|
|
|
|
T} T{
|
|
|
|
|
4 MiB
|
|
|
|
|
T}
|
|
|
|
|
_
|
|
|
|
|
T{
|
|
|
|
|
8 GiB \- 16 GiB
|
|
|
|
|
T} T{
|
|
|
|
|
8 MiB
|
|
|
|
|
T}
|
|
|
|
|
_
|
|
|
|
|
T{
|
|
|
|
|
16 GiB \- up
|
|
|
|
|
T} T{
|
|
|
|
|
16 MiB
|
|
|
|
|
T}
|
|
|
|
|
_
|
|
|
|
|
.TE
|
|
|
|
|
.sp
|
|
|
|
|
An implementation MAY deviate from the block size rule when there is good
|
|
|
|
|
reason to do so. For example, if a file has been indexed at a certain block
|
|
|
|
|
size and grows beyond 2000 blocks it may be retained at the current block
|
|
|
|
|
size for practical reasons. When there is no overriding reason to the
|
|
|
|
|
contrary, such as when indexing a new file for the first time, the block
|
|
|
|
|
size rule above SHOULD be followed.
|
|
|
|
|
.sp
|
|
|
|
|
An implementation MUST therefore accept files with a block size differing
|
|
|
|
|
from the above rule. This does not mean that arbitrary block sizes are
|
|
|
|
|
allowed. The block size used MUST be exactly one of the power\-of\-two block
|
|
|
|
|
sizes listed in the table above.
|
2015-10-20 09:59:50 +02:00
|
|
|
|
.SH EXAMPLE EXCHANGE
|
|
|
|
|
.TS
|
|
|
|
|
center;
|
|
|
|
|
|l|l|l|.
|
|
|
|
|
_
|
|
|
|
|
T{
|
|
|
|
|
#
|
|
|
|
|
T} T{
|
|
|
|
|
A
|
|
|
|
|
T} T{
|
|
|
|
|
B
|
|
|
|
|
T}
|
|
|
|
|
_
|
|
|
|
|
T{
|
|
|
|
|
1
|
|
|
|
|
T} T{
|
|
|
|
|
ClusterConfiguration\->
|
|
|
|
|
T} T{
|
|
|
|
|
<\-ClusterConfiguration
|
|
|
|
|
T}
|
|
|
|
|
_
|
|
|
|
|
T{
|
|
|
|
|
2
|
|
|
|
|
T} T{
|
|
|
|
|
Index\->
|
|
|
|
|
T} T{
|
|
|
|
|
<\-Index
|
|
|
|
|
T}
|
|
|
|
|
_
|
|
|
|
|
T{
|
|
|
|
|
3
|
|
|
|
|
T} T{
|
|
|
|
|
IndexUpdate\->
|
|
|
|
|
T} T{
|
|
|
|
|
<\-IndexUpdate
|
|
|
|
|
T}
|
|
|
|
|
_
|
|
|
|
|
T{
|
|
|
|
|
4
|
|
|
|
|
T} T{
|
|
|
|
|
IndexUpdate\->
|
|
|
|
|
T} T{
|
|
|
|
|
T}
|
|
|
|
|
_
|
|
|
|
|
T{
|
|
|
|
|
5
|
|
|
|
|
T} T{
|
|
|
|
|
Request\->
|
|
|
|
|
T} T{
|
|
|
|
|
T}
|
|
|
|
|
_
|
|
|
|
|
T{
|
|
|
|
|
6
|
|
|
|
|
T} T{
|
|
|
|
|
Request\->
|
|
|
|
|
T} T{
|
|
|
|
|
T}
|
|
|
|
|
_
|
|
|
|
|
T{
|
|
|
|
|
7
|
|
|
|
|
T} T{
|
|
|
|
|
Request\->
|
|
|
|
|
T} T{
|
|
|
|
|
T}
|
|
|
|
|
_
|
|
|
|
|
T{
|
|
|
|
|
8
|
|
|
|
|
T} T{
|
|
|
|
|
Request\->
|
|
|
|
|
T} T{
|
|
|
|
|
T}
|
|
|
|
|
_
|
|
|
|
|
T{
|
|
|
|
|
9
|
|
|
|
|
T} T{
|
|
|
|
|
T} T{
|
|
|
|
|
<\-Response
|
|
|
|
|
T}
|
|
|
|
|
_
|
|
|
|
|
T{
|
|
|
|
|
10
|
|
|
|
|
T} T{
|
|
|
|
|
T} T{
|
|
|
|
|
<\-Response
|
|
|
|
|
T}
|
|
|
|
|
_
|
|
|
|
|
T{
|
|
|
|
|
11
|
|
|
|
|
T} T{
|
|
|
|
|
T} T{
|
|
|
|
|
<\-Response
|
|
|
|
|
T}
|
|
|
|
|
_
|
|
|
|
|
T{
|
|
|
|
|
12
|
|
|
|
|
T} T{
|
|
|
|
|
T} T{
|
|
|
|
|
<\-Response
|
|
|
|
|
T}
|
|
|
|
|
_
|
|
|
|
|
T{
|
|
|
|
|
13
|
|
|
|
|
T} T{
|
|
|
|
|
Index Update\->
|
|
|
|
|
T} T{
|
|
|
|
|
T}
|
|
|
|
|
_
|
|
|
|
|
T{
|
2017-11-29 07:45:17 +01:00
|
|
|
|
…
|
2015-10-20 09:59:50 +02:00
|
|
|
|
T} T{
|
|
|
|
|
T} T{
|
|
|
|
|
T}
|
|
|
|
|
_
|
|
|
|
|
T{
|
|
|
|
|
14
|
|
|
|
|
T} T{
|
|
|
|
|
T} T{
|
|
|
|
|
<\-Ping
|
|
|
|
|
T}
|
|
|
|
|
_
|
|
|
|
|
T{
|
|
|
|
|
15
|
|
|
|
|
T} T{
|
|
|
|
|
Ping\->
|
|
|
|
|
T} T{
|
|
|
|
|
T}
|
|
|
|
|
_
|
|
|
|
|
.TE
|
|
|
|
|
.sp
|
2015-10-22 08:14:43 +02:00
|
|
|
|
The connection is established and at 1. both peers send ClusterConfiguration
|
2016-07-10 09:23:58 +02:00
|
|
|
|
messages and then Index records. The Index records are received and both
|
|
|
|
|
peers recompute their knowledge of the data in the cluster. In this example,
|
|
|
|
|
peer A has four missing or outdated blocks. At 5 through 8 peer A sends
|
|
|
|
|
requests for these blocks. The requests are received by peer B, who
|
|
|
|
|
retrieves the data from the folder and transmits Response records (9 through
|
|
|
|
|
12). Device A updates their folder contents and transmits an Index Update
|
|
|
|
|
message (13). Both peers enter idle state after 13. At some later time 14,
|
|
|
|
|
the ping timer on device B expires and a Ping message is sent. The same
|
|
|
|
|
process occurs for device A at 15.
|
2015-10-20 09:59:50 +02:00
|
|
|
|
.SH EXAMPLES OF STRONG CIPHER SUITES
|
|
|
|
|
.TS
|
|
|
|
|
center;
|
|
|
|
|
|l|l|l|.
|
|
|
|
|
_
|
|
|
|
|
T{
|
|
|
|
|
ID
|
|
|
|
|
T} T{
|
|
|
|
|
Name
|
|
|
|
|
T} T{
|
|
|
|
|
Description
|
|
|
|
|
T}
|
|
|
|
|
_
|
|
|
|
|
T{
|
|
|
|
|
0x009F
|
|
|
|
|
T} T{
|
|
|
|
|
DHE\-RSA\-AES256\-GCM\-SHA384
|
|
|
|
|
T} T{
|
|
|
|
|
TLSv1.2 DH RSA AESGCM(256) AEAD
|
|
|
|
|
T}
|
|
|
|
|
_
|
|
|
|
|
T{
|
|
|
|
|
0x006B
|
|
|
|
|
T} T{
|
|
|
|
|
DHE\-RSA\-AES256\-SHA256
|
|
|
|
|
T} T{
|
|
|
|
|
TLSv1.2 DH RSA AES(256) SHA256
|
|
|
|
|
T}
|
|
|
|
|
_
|
|
|
|
|
T{
|
|
|
|
|
0xC030
|
|
|
|
|
T} T{
|
|
|
|
|
ECDHE\-RSA\-AES256\-GCM\-SHA384
|
|
|
|
|
T} T{
|
|
|
|
|
TLSv1.2 ECDH RSA AESGCM(256) AEAD
|
|
|
|
|
T}
|
|
|
|
|
_
|
|
|
|
|
T{
|
|
|
|
|
0xC028
|
|
|
|
|
T} T{
|
|
|
|
|
ECDHE\-RSA\-AES256\-SHA384
|
|
|
|
|
T} T{
|
|
|
|
|
TLSv1.2 ECDH RSA AES(256) SHA384
|
|
|
|
|
T}
|
|
|
|
|
_
|
|
|
|
|
T{
|
|
|
|
|
0x009E
|
|
|
|
|
T} T{
|
|
|
|
|
DHE\-RSA\-AES128\-GCM\-SHA256
|
|
|
|
|
T} T{
|
|
|
|
|
TLSv1.2 DH RSA AESGCM(128) AEAD
|
|
|
|
|
T}
|
|
|
|
|
_
|
|
|
|
|
T{
|
|
|
|
|
0x0067
|
|
|
|
|
T} T{
|
|
|
|
|
DHE\-RSA\-AES128\-SHA256
|
|
|
|
|
T} T{
|
|
|
|
|
TLSv1.2 DH RSA AES(128) SHA256
|
|
|
|
|
T}
|
|
|
|
|
_
|
|
|
|
|
T{
|
|
|
|
|
0xC02F
|
|
|
|
|
T} T{
|
|
|
|
|
ECDHE\-RSA\-AES128\-GCM\-SHA256
|
|
|
|
|
T} T{
|
|
|
|
|
TLSv1.2 ECDH RSA AESGCM(128) AEAD
|
|
|
|
|
T}
|
|
|
|
|
_
|
|
|
|
|
T{
|
|
|
|
|
0xC027
|
|
|
|
|
T} T{
|
|
|
|
|
ECDHE\-RSA\-AES128\-SHA256
|
|
|
|
|
T} T{
|
|
|
|
|
TLSv1.2 ECDH RSA AES(128) SHA256
|
|
|
|
|
T}
|
|
|
|
|
_
|
|
|
|
|
.TE
|
|
|
|
|
.SH AUTHOR
|
|
|
|
|
The Syncthing Authors
|
|
|
|
|
.SH COPYRIGHT
|
2018-02-07 07:45:17 +01:00
|
|
|
|
2014-2018, The Syncthing Authors
|
2015-10-20 09:59:50 +02:00
|
|
|
|
.\" Generated by docutils manpage writer.
|
|
|
|
|
.
|