syncthing/vendor/github.com/gogo/protobuf/extensions.md
Jakob Borg 987718baf8 vendor: Update github.com/gogo/protobuf
Also tweaks the proto definitions:

 - [packed=false] on the block_indexes field to retain compat with
   v0.14.16 and earlier.

 - Uses the vendored protobuf package in include paths.

And, "build.go setup" will install the vendored protoc-gen-gogofast.
This should ensure that a proto rebuild isn't so dependent on whatever
version of the compiler and package the developer has installed...

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3864
2017-01-03 00:16:21 +00:00

16 KiB

gogoprotobuf Extensions

Here is an example.proto which uses most of the gogoprotobuf code generation plugins.

Please also look at the example Makefile which shows how to specify the descriptor.proto and gogo.proto in your proto_path

The documentation at http://godoc.org/github.com/gogo/protobuf/gogoproto describes the extensions made to goprotobuf in more detail.

Also see http://godoc.org/github.com/gogo/protobuf/plugin/ for documentation of each of the extensions which have their own plugins.

Fast Marshalling and Unmarshalling

Generating a Marshal, MarshalTo, Size (or ProtoSize) and Unmarshal method for a struct results in faster marshalling and unmarshalling than when using reflect.

See BenchComparison for a comparison between reflect and generated code used for marshalling and unmarshalling.

NameOptionTypeDescriptionDefault
marshalerMessageboolif true, a Marshal and MarshalTo method is generated for the specific messagefalse
sizerMessageboolif true, a Size method is generated for the specific messagefalse
unmarshaler Message bool if true, an Unmarshal method is generated for the specific message false
protosizerMessageboolif true, a ProtoSize method is generated for the specific messagefalse
unsafe_marshaler Message bool if true, a Marshal and MarshalTo method is generated for the specific message. The generated code uses the unsafe package. false
unsafe_unmarshaler Message bool if true, an Unmarshal method is generated for the specific message. The generated code uses the unsafe package. false
stable_marshaler Message bool if true, a Marshal and MarshalTo method is generated for the specific message, but unlike marshaler the output is guaranteed to be deterministic, at the sacrifice of some speed false

More Canonical Go Structures

Lots of times working with a goprotobuf struct will lead you to a place where you create another struct that is easier to work with and then have a function to copy the values between the two structs.

You might also find that basic structs that started their life as part of an API need to be sent over the wire. With gob, you could just send it. With goprotobuf, you need to make a new struct.

gogoprotobuf tries to fix these problems with the nullable, embed, customtype, customname, casttype, castkey and castvalue field extensions.

NameOptionTypeDescriptionDefault
nullable Field bool if false, a field is generated without a pointer (see warning below). true
embed Field bool if true, the field is generated as an embedded field. false
customtype Field string It works with the Marshal and Unmarshal methods, to allow you to have your own types in your struct, but marshal to bytes. For example, custom.Uuid or custom.Fixed128 goprotobuf type
customname (beta) Field string Changes the generated fieldname. This is especially useful when generated methods conflict with fieldnames. goprotobuf field name
casttype (beta) Field string Changes the generated field type. It assumes that this type is castable to the original goprotobuf field type. It currently does not support maps, structs or enums. goprotobuf field type
castkey (beta) Field string Changes the generated fieldtype for a map key. All generated code assumes that this type is castable to the protocol buffer field type. Only supported on maps. goprotobuf field type
castvalue (beta) Field string Changes the generated fieldtype for a map value. All generated code assumes that this type is castable to the protocol buffer field type. Only supported on maps. goprotobuf field type
enum_customname (beta) Enum string Sets the type name of an enum. If goproto_enum_prefix is enabled, this value will be used as a prefix when generating enum values.goprotobuf enum type name. Helps with golint issues.
enumvalue_customname (beta) Enum Value string Changes the generated enum name. Helps with golint issues.goprotobuf enum value name
stdtime Timestamp Field bool Changes the Well Known Timestamp Type to time.TimeTimestamp
stdduration Duration Field bool Changes the Well Known Duration Type to time.DurationDuration

Warning about nullable: according to the Protocol Buffer specification, you should be able to tell whether a field is set or unset. With the option nullable=false this feature is lost, since your non-nullable fields will always be set.

Warning about customtype: It is your responsibility to test all cases of your marshaling, unmarshaling and size methods implemented for your custom type.

Issues with customtype include:

Goprotobuf Compatibility

Gogoprotobuf is compatible with Goprotobuf, because it is compatible with protocol buffers (see the section on tests below).

Gogoprotobuf generates the same code as goprotobuf if no extensions are used.

The enumprefix, getters and stringer extensions can be used to remove some of the unnecessary code generated by goprotobuf.

NameOptionTypeDescriptionDefault
gogoproto_import File bool if false, the generated code imports github.com/golang/protobuf/proto instead of github.com/gogo/protobuf/proto. true
goproto_enum_prefix Enum bool if false, generates the enum constant names without the messagetype prefix true
goproto_getters Message bool if false, the message is generated without get methods, this is useful when you would rather want to use face true
goproto_stringer Message bool if false, the message is generated without the default string method, this is useful for rather using stringer true
goproto_enum_stringer (experimental) Enum bool if false, the enum is generated without the default string method, this is useful for rather using enum_stringer true
goproto_extensions_map (beta) Message bool if false, the extensions field is generated as type []byte instead of type map[int32]proto.Extension true
goproto_unrecognized (beta) Message bool if false, XXX_unrecognized field is not generated. This is useful to reduce GC pressure at the cost of losing information about unrecognized fields. true

Less Typing

The Protocol Buffer language is very parseable and extra code can be easily generated for structures.

Helper methods, functions and interfaces can be generated by triggering certain extensions like gostring.

NameOptionTypeDescriptionDefault
gostring Message bool if true, a `GoString` method is generated. This returns a string representing valid go code to reproduce the current state of the struct. false
onlyone (deprecated) Message bool if true, all fields must be nullable and only one of the fields may be set, like a union. Two methods are generated: `GetValue() interface{}` and `SetValue(v interface{}) (set bool)`. These provide easier interaction with a union. false
equal Message bool if true, an Equal method is generated false
compare Message bool if true, a Compare method is generated. This is very useful for quickly implementing sort on a list of protobuf structs false
verbose_equal Message bool if true, a verbose equal method is generated for the message. This returns an error which describes the exact element which is not equal to the exact element in the other struct. false
stringer Message bool if true, a String method is generated for the message. false
face Message bool if true, a function will be generated which can convert a structure which satisfies an interface (face) to the specified structure. This interface contains getters for each of the fields in the struct. The specified struct is also generated with the getters. This allows it to satisfy its own face. false
description (beta) Message bool if true, a Description method is generated for the message. false
populate Message bool if true, a `NewPopulated` function is generated. This is necessary for generated tests. false
enum_stringer (experimental) Enum bool if true, a String method is generated for an Enum false

Issues with Compare include:

#Peace of Mind

Test and Benchmark generation is done with the following extensions:

testgen Message bool if true, tests are generated for proto, json and prototext marshalling as well as for some of the other enabled plugins false
benchgen Message bool if true, benchmarks are generated for proto, json and prototext marshalling as well as for some of the other enabled plugins false

More Serialization Formats

Other serialization formats like xml and json typically use reflect to marshal and unmarshal structured data. Manipulating these structs into something other than the default Go requires editing tags. The following extensions provide ways of editing these tags for the generated protobuf structs.

jsontag (beta) Field string if set, the json tag value between the double quotes is replaced with this string fieldname
moretags (beta) Field string if set, this string is appended to the tag string empty

Here is a longer explanation of jsontag and moretags

File Options

Each of the boolean message and enum extensions also have a file extension:

  • marshaler_all
  • sizer_all
  • protosizer_all
  • unmarshaler_all
  • unsafe_marshaler_all
  • unsafe_unmarshaler_all
  • stable_marshaler_all
  • goproto_enum_prefix_all
  • goproto_getters_all
  • goproto_stringer_all
  • goproto_enum_stringer_all
  • goproto_extensions_map_all
  • goproto_unrecognized_all
  • gostring_all
  • onlyone_all
  • equal_all
  • compare_all
  • verbose_equal_all
  • stringer_all
  • enum_stringer_all
  • face_all
  • description_all
  • populate_all
  • testgen_all
  • benchgen_all

Each of these are the same as their Message Option counterparts, except they apply to all messages in the file. Their Message option counterparts can also be used to overwrite their effect.

Tests

  • The normal barrage of tests are run with: make tests
  • A few weird tests: make testall
  • Tests for compatibility with golang/protobuf are handled by a different project harmonytests, since it requires goprotobuf.
  • Cross version tests are made with Travis CI.
  • GRPC Tests are also handled by a different project grpctests, since it depends on a lot of grpc libraries.
  • Thanks to go-fuzz we have proper fuzztests.