syncthing/vendor/github.com/gogo/protobuf
Jakob Borg 916ec63af6 cmd/stdiscosrv: New discovery server (fixes #4618)
This is a new revision of the discovery server. Relevant changes and
non-changes:

- Protocol towards clients is unchanged.

- Recommended large scale design is still to be deployed nehind nginx (I
  tested, and it's still a lot faster at terminating TLS).

- Database backend is leveldb again, only. It scales enough, is easy to
  setup, and we don't need any backend to take care of.

- Server supports replication. This is a simple TCP channel - protect it
  with a firewall when deploying over the internet. (We deploy this within
  the same datacenter, and with firewall.) Any incoming client announces
  are sent over the replication channel(s) to other peer discosrvs.
  Incoming replication changes are applied to the database as if they came
  from clients, but without the TLS/certificate overhead.

- Metrics are exposed using the prometheus library, when enabled.

- The database values and replication protocol is protobuf, because JSON
  was quite CPU intensive when I tried that and benchmarked it.

- The "Retry-After" value for failed lookups gets slowly increased from
  a default of 120 seconds, by 5 seconds for each failed lookup,
  independently by each discosrv. This lowers the query load over time for
  clients that are never seen. The Retry-After maxes out at 3600 after a
  couple of weeks of this increase. The number of failed lookups is
  stored in the database, now and then (avoiding making each lookup a
  database put).

All in all this means clients can be pointed towards a cluster using
just multiple A / AAAA records to gain both load sharing and redundancy
(if one is down, clients will talk to the remaining ones).

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/4648
2018-01-14 08:52:31 +00:00
..
_conformance cmd/stdiscosrv: New discovery server (fixes #4618) 2018-01-14 08:52:31 +00:00
codec cmd/stdiscosrv: New discovery server (fixes #4618) 2018-01-14 08:52:31 +00:00
gogoproto cmd/stdiscosrv: New discovery server (fixes #4618) 2018-01-14 08:52:31 +00:00
gogoreplace vendor: Update github.com/gogo/protobuf, keeping all files 2017-06-14 05:24:09 +02:00
io cmd/stdiscosrv: New discovery server (fixes #4618) 2018-01-14 08:52:31 +00:00
jsonpb cmd/stdiscosrv: New discovery server (fixes #4618) 2018-01-14 08:52:31 +00:00
plugin vendor: Update everything 2017-12-29 11:38:00 +00:00
proto cmd/stdiscosrv: New discovery server (fixes #4618) 2018-01-14 08:52:31 +00:00
protobuf cmd/stdiscosrv: New discovery server (fixes #4618) 2018-01-14 08:52:31 +00:00
protoc-gen-combo vendor: Update everything 2017-12-29 11:38:00 +00:00
protoc-gen-gofast vendor: Update github.com/gogo/protobuf, keeping all files 2017-06-14 05:24:09 +02:00
protoc-gen-gogo cmd/stdiscosrv: New discovery server (fixes #4618) 2018-01-14 08:52:31 +00:00
protoc-gen-gogofast vendor: Update github.com/gogo/protobuf, keeping all files 2017-06-14 05:24:09 +02:00
protoc-gen-gogofaster vendor: Update github.com/gogo/protobuf, keeping all files 2017-06-14 05:24:09 +02:00
protoc-gen-gogoslick vendor: Update github.com/gogo/protobuf, keeping all files 2017-06-14 05:24:09 +02:00
protoc-gen-gogotypes vendor: Update github.com/gogo/protobuf, keeping all files 2017-06-14 05:24:09 +02:00
protoc-gen-gostring vendor: Update github.com/gogo/protobuf, keeping all files 2017-06-14 05:24:09 +02:00
protoc-min-version vendor: Update github.com/gogo/protobuf, keeping all files 2017-06-14 05:24:09 +02:00
sortkeys vendor: Update github.com/gogo/protobuf, keeping all files 2017-06-14 05:24:09 +02:00
test cmd/stdiscosrv: New discovery server (fixes #4618) 2018-01-14 08:52:31 +00:00
types cmd/stdiscosrv: New discovery server (fixes #4618) 2018-01-14 08:52:31 +00:00
vanity cmd/stdiscosrv: New discovery server (fixes #4618) 2018-01-14 08:52:31 +00:00
version vendor: Update github.com/gogo/protobuf, keeping all files 2017-06-14 05:24:09 +02:00
.gitignore cmd/stdiscosrv: New discovery server (fixes #4618) 2018-01-14 08:52:31 +00:00
.mailmap cmd/stdiscosrv: New discovery server (fixes #4618) 2018-01-14 08:52:31 +00:00
.travis.yml cmd/stdiscosrv: New discovery server (fixes #4618) 2018-01-14 08:52:31 +00:00
AUTHORS cmd/stdiscosrv: New discovery server (fixes #4618) 2018-01-14 08:52:31 +00:00
CONTRIBUTORS cmd/stdiscosrv: New discovery server (fixes #4618) 2018-01-14 08:52:31 +00:00
GOLANG_CONTRIBUTORS cmd/stdiscosrv: New discovery server (fixes #4618) 2018-01-14 08:52:31 +00:00
LICENSE vendor: Update github.com/gogo/protobuf, keeping all files 2017-06-14 05:24:09 +02:00
Makefile cmd/stdiscosrv: New discovery server (fixes #4618) 2018-01-14 08:52:31 +00:00
README cmd/stdiscosrv: New discovery server (fixes #4618) 2018-01-14 08:52:31 +00:00
Readme.md cmd/stdiscosrv: New discovery server (fixes #4618) 2018-01-14 08:52:31 +00:00
bench.md cmd/stdiscosrv: New discovery server (fixes #4618) 2018-01-14 08:52:31 +00:00
custom_types.md cmd/stdiscosrv: New discovery server (fixes #4618) 2018-01-14 08:52:31 +00:00
extensions.md cmd/stdiscosrv: New discovery server (fixes #4618) 2018-01-14 08:52:31 +00:00
install-protobuf.sh cmd/stdiscosrv: New discovery server (fixes #4618) 2018-01-14 08:52:31 +00:00

Readme.md

Protocol Buffers for Go with Gadgets

Build Status

gogoprotobuf is a fork of golang/protobuf with extra code generation features.

This code generation is used to achieve:

  • fast marshalling and unmarshalling
  • more canonical Go structures
  • goprotobuf compatibility
  • less typing by optionally generating extra helper code
  • peace of mind by optionally generating test and benchmark code
  • other serialization formats

Keeping track of how up to date gogoprotobuf is relative to golang/protobuf is done in this issue

Users

These projects use gogoprotobuf:

Please let us know if you are using gogoprotobuf by posting on our GoogleGroup.

Mentioned

Getting Started

There are several ways to use gogoprotobuf, but for all you need to install go and protoc. After that you can choose:

  • Speed
  • More Speed and more generated code
  • Most Speed and most customization

Installation

To install it, you must first have Go (at least version 1.6.3) installed (see http://golang.org/doc/install). Latest patch versions of Go 1.8, 1.9 and 1.10 are continuously tested.

Next, install the standard protocol buffer implementation from https://github.com/google/protobuf. Most versions from 2.3.1 should not give any problems, but 2.6.1, 3.0.2 and 3.5.1 are continuously tested.

Speed

Install the protoc-gen-gofast binary

go get github.com/gogo/protobuf/protoc-gen-gofast

Use it to generate faster marshaling and unmarshaling go code for your protocol buffers.

protoc --gofast_out=. myproto.proto

This does not allow you to use any of the other gogoprotobuf extensions.

More Speed and more generated code

Fields without pointers cause less time in the garbage collector. More code generation results in more convenient methods.

Other binaries are also included:

protoc-gen-gogofast (same as gofast, but imports gogoprotobuf)
protoc-gen-gogofaster (same as gogofast, without XXX_unrecognized, less pointer fields)
protoc-gen-gogoslick (same as gogofaster, but with generated string, gostring and equal methods)

Installing any of these binaries is easy. Simply run:

go get github.com/gogo/protobuf/proto
go get github.com/gogo/protobuf/{binary}
go get github.com/gogo/protobuf/gogoproto

These binaries allow you to use gogoprotobuf extensions. You can also use your own binary.

To generate the code, you also need to set the include path properly.

protoc -I=. -I=$GOPATH/src -I=$GOPATH/src/github.com/gogo/protobuf/protobuf --{binary}_out=. myproto.proto

Note that in the protoc command, {binary} does not contain the initial prefix of "protoc-gen".

Most Speed and most customization

Customizing the fields of the messages to be the fields that you actually want to use removes the need to copy between the structs you use and structs you use to serialize. gogoprotobuf also offers more serialization formats and generation of tests and even more methods.

Please visit the extensions page for more documentation.

Install protoc-gen-gogo:

go get github.com/gogo/protobuf/proto
go get github.com/gogo/protobuf/jsonpb
go get github.com/gogo/protobuf/protoc-gen-gogo
go get github.com/gogo/protobuf/gogoproto

GRPC

It works the same as golang/protobuf, simply specify the plugin. Here is an example using gofast:

protoc --gofast_out=plugins=grpc:. my.proto