v0.6.0

This release contains several breaking changes. Such changes were deemed acceptable since this module has not yet reached v1.0 and the impact on existing user code is likely to be very low.

Fixes:

1. If a protocompile.Resolver returns search results that include descriptor protos, the compilation step that interprets options could inexplicably fail. This was because the logic was relying on an AST for some metadata about the option declarations in the source. This has been fixed: if a descriptor proto is provided that contains uninterpreted options, they can now be successfully interpreted.

Changes:

1. Numerous elements have been renamed so that references to "oneof" elements now use the spelling "Oneof" instead of "OneOf" (second "O" now lower-case). This is for consistency with how this is spelled in the core protobuf runtime, in the "google.golang.org/protobuf" module).
   • This is a breaking change. Fixes to user code that may have been broken by this change should be straight-forward and mechanical.
2. The grammar used to generate the parser has been improved. In particular, all conflicts in the yacc configuration have been eliminated. This does not fix any known bug, but it does provide higher confidence that no latent bugs exist since the generated parser could previously have chosen the wrong action in the face of a conflict.
3. The function sourceinfo.GenerateSourceInfo now accepts a sourceinfo.OptionIndex instead of an options.Index. It also now accepts variadic options. The function sourceinfo.GenerateSourceInfoWithExtraComments has been removed; its functionality can now be accomplished by passing an option to sourceinfo.GenerateSourceInfo.
   • This is a breaking change. It is expected that these functions, despite being exported, were only used internally by the compiler implementation.
4. The type options.Index has been removed. Code should instead use the new sourceinfo.OptionIndex type.
   • This is a breaking change. It is expected that this type, despite being exported, was only used internally by the compiler implementation.

Additions:

1. A new protocompile.SourceInfoMode value has been added: SourceInfoExtraOptionLocations. This can be combined with other values (via bitwise OR) to have the compiler include additional source code info for locations inside message literals in custom option values.
2. A new sourceinfo.OptionIndex type has been added, along with related types in the same package: OptionSourceInfo, OptionChildrenSourceInfo, ArrayLiteralSourceInfo, and MessageLiteralSourceInfo. These provide a model of source info paths in the interior of an option declaration.
3. A new sourceinfo.GenerateOption type has been added, allowing the behavior of sourceinfo.GenerateSourceInfo to be customized via variadic options. Options are available for retaining extra comments (sourceinfo.WithExtraComments) and for producing extra locations for option declarations with message literal values (sourceinfo.WithExtraOptionLocations).

v0.5.1

This is an urgent bug-fix release, to remedy a potential deadlock error in the previous release (v0.5.0).

The only change in this release is a fix to this deadlock issue. With v0.5.0, if a compile operation was using an override version of google/protobuf/descriptor.proto, it was possible for the call to compiler.Compile to halt, only returning if the context was cancelled or times out. This is now fixed.

v0.5.0

DO NOT USE!

This release introduced a potential deadlock in calls to the Compile method of a protocompile.Compiler. Use v0.5.1 instead, which fixes the deadlock issue.

---

This release addresses an issue introduced in v0.3.0 wherein the results of compilation could sometimes have custom options represented as unrecognized fields, instead of as known extension fields.

While fixing that issue, several other changes were made opportunistically to improve code maintainability. This resulted in some changes that are technically not backwards compatible. However, these incompatible changes are highly unlikely to pose source compatibility issues for users. Methods were removed from an interface that were highly unlikely to be called outside of this repo.

Fixes:

1. It was previously possible for descriptors to be returned with unrecognized fields if an override version of google/protobuf/descriptor.proto were in use. This has been fixed so that all custom options interpreted by the compiler should be represented as known extension fields.

Changes:

1. The linker.Result interface has had several methods removed:
   • ResolveEnumType(protoreflect.FullName) protoreflect.EnumDescriptor
   • ResolveMessageType(protoreflect.FullName) protoreflect.MessageDescriptor
   • ResolveOptionsType(protoreflect.FullName) protoreflect.MessageDescriptor (just recently introduced in v0.3.0)
   • ResolveExtension(protoreflect.FullName) protoreflect.ExtensionTypeDescriptor
2. The linker.ResolverFromFile function has been updated so that the returned resolver properly implements visibility rules of the Protobuf language. It can resolve any element that is visible to the given file, including elements visible via transitive public imports.

Additions:

1. The various entry-point functions in the options sub-package now accept optional variadic InterpreterOption params. A single such option is now provided, WithOverrideDescriptorProto, which allows the caller to provide an override version of google/protobuf/descriptor.proto to use for interpreting options.

v0.4.0

Fixes:

1. If a protocompile.Resolver returns search results with parser results or descriptor protos that are shared across more than one compile operation, a data race could occur. This happens when a descriptor proto needs to be modified during compilation (like to replace unqualified references with fully-qualified references during linking, to interpret options, or to add source code info). In that case the modification could be made to the same descriptor, concurrently from the different compile operations. If the race detector is enabled, this will result in a crash.
2. If a program directly calls parser.ResultFromAST, passing false for whether to validate the result, it could get back a different file descriptor proto than if true were passed, even if the result is in fact valid. In particular, when validation was skipped, field descriptors may be missing values for the label field. This is now fixed such that there is no difference in the result (other than the possibility of an error when validating, due to extra validation checks).
3. If a protocompiler.Resolver returns a search result with an invalid type reference, such that a field's type_name field resolved to a message but the type field was incorrectly set to TYPE_ENUM, an invalid compile result would be produced. And if the converse occurred (type_name resolved to an enum, but type indicated TYPE_MESSAGE), it would be silently ignored but produce a valid result (by correcting the errant type field value). Now, if either of these kinds of mismatches are detected, an error will occur.
4. The binary format of a CanonicalProto from a linker.Result did not match the serialization format that protoc uses in some obscure cases. If the compile operation contained an override version of "google/protobuf/descriptor.proto" that re-declared a known field (e.g. same tag number as a field statically known to the program) with an incompatible type, protoc includes it with other unrecognized fields after recognized fields are serialized. But this repo was including it with recognized fields. This has been fixed.

Changes:

1. When computing source code info, comments that would previously have been dropped due to ambiguity may not be attributed. When a sequence of comments is ambiguous (i.e. unclear whether they should be attributed as trailing comment to prior token, leading comment to subsequent token, or detached), instead of being dropped they will now be attributed as follows:

   • If there is one comment, it will be detached.
   • If there are two or more comments, the first will be a trailing comment on prior token, the last will be a leading comment on subsequent token, and any other comments in between will be detached.

   This mirrors the behavior of protoc as of v22.0, in which an equivalent change was made to comment attribution logic.

Additions:

1. A new function, Clone(parser.Result) parser.Result, has been added to the parser sub-package. This makes a deep copy of a parser.Result, which can be used to make defensive copies (such as to prevent data races). This clones the result's file descriptor proto and also re-creates an internal index that maps elements of the file descriptor proto to nodes in the AST.

v0.3.0

This release contains a few fixes but is mostly to align with the recent release of Protocol Buffers v22.0.

Fixes:

1. The reference compiler protoc allows the use of integer literals for enum values inside of a message literal, for custom options whose value is a message type. But this repo did not allow this, so some sources that could be compiled with protoc would result in errors with this repo. This has been fixed.
2. This repo allowed the message_set_wire_format option to be used in files that declare a syntax of proto3. However, this is not allowed; that option is only acceptable in the proto2 syntax. So this repo would accept some sources that were actually valid and would fail if compiled with protoc. This has been fixed.
3. This repo allowed certain kinds of relative references when resolving extension names in message literals, for custom options whose value is a message type, that are not allowed by the reference compiler protoc. So this repo would accept some sources that were actually valid and would fail if compiled with protoc. This has been fixed.
4. Fuzz testing identified a couple of patterns of pathological input that caused degenerate performance in the lexer and parser. These have been fixed.

Changes:

1. When using protocompile.WithStandardImports, the version of google/protobuf/descriptor.proto has been updated to match the version included with Protocol Buffers v22.0. Even if not using protocompile.WithStandardImports, sources may refer to new options added in v22.0 and the compiler will accept them.
2. The above v22.0 release included some changes to behavior when validating custom JSON names. In particular, checking for conflicting JSON names is now case-sensitive. Also, custom JSON names may not be enclosed brackets ([ and ]). These changes are now also reflected in validation done by this repo.
3. If a compile operation provides an override version of google/protobuf/descriptor.proto, it will be used during the options interpretation step. Previously, the was only supported in files that imported override definitions of the options types. If override definitions were not directly imported, options interpretation would always use the version of the options types linked into the compiler program (via the google.golang.org/protobuf/types/descriptorpb package). To provide an override version, implement a protocompile.Resolver that returns a result for google/protobuf/descriptor.proto that is different from the result supplied when using protocompile.WithStandardImports.

Additions:

1. The linker.Result interface has a new method named ResolveOptionsType. This is used to implement the above new strategy for resolving options message types when a compile operation includes an override version of google/protobuf/descriptor.proto. This is technically a backwards-incompatible change. Such an incompatible change was made because (1) this repo is still pre-v1.0, and (2) this interface is not intended to be implemented by code outside of this repo.

v0.2.0

This release contains a couple of bug-fixes and one minor addition.

Fixes:

1. If a protocompile.SourceResolver has an Accessor function that returns an error that wraps an ErrNotExist, it could incorrectly abort searching through import paths. It instead required that the function directly return that error sentinel, without wrapping. Now it correctly handles if the ErrNotExist error is wrapped.
2. When using an error reporter that accumulates many errors (vs. failing after the first one) and parsing source with an invalid string literal that contained incorrect escape sequences, the lexer could get into an incorrect state and report many confusing errors. It effectively behaved as if the incorrect escape sequence was the end of the string literal; it would then report tokenization errors about the rest of the string literal contents, as if they were regular source code not inside a string literal. This has been fixed, and error messages in these situations are now intuitive.

Additions:

1. The protocompile.SearchResult type has a new field ParseResult. This allows the compiler to make more efficient use of the result if the resolver already has an instance of parser.Result. Previously, the resolver would have to decide to return either the file descriptor proto or the AST. Now it can return both.

v0.1.0

This is an initial release of this new Protobuf compiler.

It is thoroughly tested and should be considered production-ready. The only reason this is a v0 and not a v1 is to get a little more mileage on the new API. Since this is an entirely different interface to the compiler compared to this repo's predecessor, we want to keep a little flexibility for now to change and improve the shape of things. Once we've established more comfort with the ergonomics of the API, we'll lock it in with a v1.0.0.