Skip to content

Releases: jhump/protoreflect

v2.0.0-beta.2

14 Sep 01:41
d7318a2
Compare
Choose a tag to compare
v2.0.0-beta.2 Pre-release
Pre-release

This is a pre-release for an upcoming v2.0.0 of this module.

Once we have more test coverage we'll create a proper release candidate, but for now we're considering this a "beta" of the next iteration of this repo.

(Note: there was also a v2.0.0-beta.1 tag, but it should be ignored, hence there being no release information associated with it.)

What's Changed?

A lot has changed!

This version finally drops the use of the old "v1 API" of the Protobuf Runtime for Go (github.com/golang/protobuf) and instead is built on the current "v2 API": google.golang.org/protobuf.

The v2 API introduced support for descriptors, reflection, and dynamic messages into the core runtime. That means several packages from v1 of this repo no longer exist.

Removed Packages

These packages were marked deprecated in v1.17.0. They have all been dropped for v2. See the list below for suitable replacements:

Renamed/Moved Packages

The other functionality in v1 of this module, the stuff that is retained in v2, has been moved around. Because the desc and dynamic directories were removed (since they were home to now-removed packages), most other packages have been shuffled around a little:

New Packages

  • github.com/jhump/protoreflect/v2/protodescs
    This package provides a few random helpers for working with descriptors and descriptor protos.

  • github.com/jhump/protoreflect/v2/protomessage
    This package provides some helpers for working with proto messages.

  • github.com/jhump/protoreflect/v2/protoresolve
    This package provides numerous nominal interface types for kinds of "resolvers". Resolvers are types that can answer queries about descriptors. This package also provides a *Registry type, which builds on the standard API's *protoregistry.Files and *protoregistry.Types types and provides new functionality:

    • It is thread-safe and can be used from multiple goroutines.
    • It can track the original descriptor proto from which a descriptor was built. This allows more convenient and efficient access than have to re-construct the descriptor proto (via the standard runtime's protodesc package).
    • It is a single value that can be used to query for descriptors as well as extension and message types. (The type implementations provided are all dynamic types.)

    This package has the broadest new API and is the most likely to potentially change between this beta and a final v2.0.0 release.

  • github.com/jhump/protoreflect/v2/sourceloc
    This package provides several helpers for working with instances of protoreflect.SourceLocation and protoreflect.SourcePath.

v1.17.0

05 Sep 18:58
06e5c08
Compare
Choose a tag to compare

This release deprecates some packages that have newer alternatives in the Protobuf runtime. It also includes some small additions to the grpcreflect and desc/sourceinfo packages. The rest of the changes are bug fixes.

"github.com/jhump/protoreflect/desc"

Changes:

  • This package is now deprecated. Though use of this package is still needed in order to use a handful of other packages in this repo, many usages should instead prefer the "google.golang.org/protobuf/reflect/protoreflect" package in the Protobuf runtime. In a v2 of this repo, this package will no longer be present.
  • A performance improvement was contributed that speeds up the MessageDescriptor.FindFieldByName method.

"github.com/jhump/protoreflect/desc/protoparse"

Changes and bugfixes:

  • This package is now deprecated. Most usages should instead prefer the "github.com/bufbuild/protocompile" package. There are some behavioral differences that we hope to address with additional API and enhancements to protocompile. But the protoparse package here is just a veneer on top of protocompile and we plan to do as little maintenance as possible here. In a v2 of this repo, this package will no longer be present.
  • The protocompile dependency has been updated to use a more recent version of that package. This addresses potential panics when calling Parser.ParseButDontLink.

"github.com/jhump/protoreflect/desc/sourceinfo"

Changes and bugfixes:

  • A possible panic when calling sourceinfo.GlobalFiles.FindDescriptorByName has been fixed.
  • Descriptors returned from the sourceinfo.GlobalFiles registry or from the various Wrap* functions could induce panics in user code if querying for non-existent elements. For example, calling messageDescriptor.Fields().ByNumber(123) could return a typed-nil value on failure, instead of a nil interface. This would result in nil-dereference panics if methods were called on this later. These have been fixed.
  • Descriptors returned from the sourceinfo.GlobalFiles registry and from the various Wrap* functions had concrete types defined in this package, not in the Protobuf runtime. But the protoreflect.Descriptor interface has a "do not implement" marker, indicating that concrete types implementing the interface should not be implemented outside of that module. To avoid future issues with the Protobuf runtime, there are no more implementations in this repo. Instead of wrapping other descriptor implementations (via embedding, and then overriding some methods), this package now builds new descriptors, using the protodesc package in the Protobuf runtime module.
  • The various Wrap* functions are now deprecated. The name is no longer accurate since they no longer return wrapper values. Also the new implementation, which builds new descriptors, can fail (though unlikely), but the existing signature of these functions did not allow for returning an error. So these functions could possibly panic in the face of malformed input descriptors. Users should use the new AddSourceInfoTo* versions of these functions instead.

Additions:

  • Adds AddSourceInfoTo* functions to replace the Wrap* functions. These functions augment the input descriptors by adding source code information to them.

"github.com/jhump/protoreflect/dynamic"

Changes:

  • This package is now deprecated. Most usages should instead prefer the "google.golang.org/protobuf/types/dynamicpb" package in the Protobuf runtime. In a v2 of this repo, this package will no longer be present.

"github.com/jhump/protoreflect/grpcreflect"

Additions:

  • Adds a new NewClientV1 function to return a client that uses v1 of the server reflection service. This fills the gap in the existing factory functions, NewClientV1Alpha and NewClientAuto.
  • Adds a new AllowFallbackResolver method to the *grpcreflect.Client type. This allows the client to be configured with an alternate resolver that can be used as a backup when trying to satisfy dependencies, for cases where the remote server is non-authoritative and unable to provide all of them.

v1.16.0

09 Apr 20:50
48670ae
Compare
Choose a tag to compare

This release adds support for Protobuf Editions to many of the packages. Note, however, that the desc/protoparse package does not yet work to parse Protobuf Editions source files. This release also includes a handful of bug-fixes.

"github.com/jhump/protoreflect/desc"

Additions:

  • Adds an Edition method to *desc.FileDescriptor, which returns non-zero for files that use Protobuf Editions (instead of syntax "proto2" or "proto3").

Changes/fixes:

  • Fixes an issue in the various Wrap* functions, that could result in a descriptor hierarchy where the same descriptors were incorrectly present multiple times. This could happen if a file in the transitive dependency graph being wrapped had more than one incoming edge (e.g. same file appearing in import statements from two or more other files).

"github.com/jhump/protoreflect/desc/builder"

Additions:

  • Adds an Editions field and SetEditions method to *builder.FileBuilder, for creating files that use Protobuf Editions.

"github.com/jhump/protoreflect/desc/protoprint"

Changes/fixes:

  • Now correctly prints the contents for files that use Protobuf Editions. Previously, if given a descriptor for a file used Protobuf Editions, it could generate invalid source output.
  • Fixed a bug when Printer.CustomSortFunction was set wherein enum values could be printed in incorrect order, resulting in invalid source output. In syntax "proto3" and in open enums in Protobuf Editions, the first value must be the one with a zero numeric value.

"github.com/jhump/protoreflect/grpcreflect"

Additions:

  • Adds a new AllowMissingFileDescriptors method to *grpcreflect.Client that allows users to opt-in to a lenient mode that will try to return schemas even when some files are unavailable. This will only be successful in cases where the missing files only provide custom options or are unused imports. This restores some lenience that was inadvertently provided (due to bugs and missing validation checks) when versions of this repo prior to v1.15 were combined with versions of the google.golang.org/grpc module prior to v1.61.

v1.15.6

29 Jan 22:29
e6d8c42
Compare
Choose a tag to compare

This is a bugfix release to address recently identified regressions in the desc/protoparse package -- things that worked in v1.14.1 but did not work in v1.15.x.

"github.com/jhump/protoreflect/desc/protoparse"

Changes/fixes:

  • The documentation for the Parser.ImportPaths field specifies that the field is not used for calls to the Parser.ParseFilesButDoNotLink method. This is because import paths do not need to be resolved if not linking. But as of v1.15.0, the ImportPaths field was being used for that method, and the difference in behavior could break existing programs. This has been fixed, and the behavior once again matches the spec.
  • As of v1.15.0, if a path provided to Parser.ParseFiles were resolved as descriptors (via the Parser.LookupImport or Parser.LookupImportProto functions) instead of as source code (via the Parser.Accessor function or the default action of os.Open), the parse operation would fail with a "no such file" error. This would succeed with v1.14.1, with the descriptor being used as the input for the file instead of the file being parsed from source. So working programs could have been broken by an upgrade to v1.15.x. The package has been changed back to the v1.14.1 behavior.

v1.15.5

22 Jan 18:50
7000dd2
Compare
Choose a tag to compare

This is a bugfix release to address bugs in the desc/builder, desc/protoparse, and grpcreflect packages and a compatibility issue with the most recent version of github.com/bufbuild/protocompile.

"github.com/jhump/protoreflect/desc/builder"

Changes/fixes:

  • A regression was introduced in v1.15.0 of this package that could cause it to fail to include some imports in a file for custom options. If the custom option in question was in a known transitive dependency, the builder could fail to add the necessary import. This has been fixed.

"github.com/jhump/protoreflect/desc/protoparse"

Changes/fixes:

  • As of v1.15.0, this package uses the github.com/bufbuild/protocompile package under the hood as the parser. But that package is still using major version zero, since it's API has not completely stabilized. A recent release (v0.8.0) included backwards-incompatible changes that caused this repo to no longer compile. This release addresses the compile errors: this repo now uses that latest release of protocompile and correctly compiles against it.
  • The use of v0.8.0 of protocompile fixes some issues in the desc/protoparse package that could cause it to incorrectly accept or reject some Protobuf source files. See the release notes for protocompile v0.8.0 for more details.

"github.com/jhump/protoreflect/grpcreflect"

Changes/fixes:

  • The grpcreflect.NewClientAuto function (added in v1.14.0) returns a client that automatically falls back to using v1alpha of the reflection protocol if the server does not support v1. However, it relied on the server returning an "unimplemented" error to decide to fall back, but this condition has been observed to sometimes instead manifest as an "unavailable" error (in proxies that fail to write a 404 error before closing the connection). In those cases, the client was not correctly falling back to v1alpha. This release remedies that, so it can fallback to v1alpha when this category of error occurs.

v1.15.4

08 Dec 00:22
f139a6d
Compare
Choose a tag to compare

This is a bugfix release to address bugs in the desc/protoprint package and a compatibility issue with the most recent version of github.com/bufbuild/protocompile.

"github.com/jhump/protoreflect/desc/protoparse"

Changes/fixes:

  • As of v1.15.0, this package uses the github.com/bufbuild/protocompile package under the hood as the parser. But that package is still using major version zero, since it's API has not completely stabilized. A recent release (v0.7.0) included backwards-incompatible changes that caused this repo to no longer compile. This release addresses the compile errors: this repo now uses that latest release of protocompile and correctly compiles against it.

"github.com/jhump/protoreflect/desc/protoprint"

Changes/fixes:

  • When a string literal included a double-quote character ("), it would get printed incorrectly leading to either an incorrect string literal in some cases or an invalid and uncompilable source file in others. This has been fixed.
  • When sorting elements in the file, if the file used proto3 syntax and had an enum with negative numeric values, the result would include the negative values first, before the zero value. This is not allowed in proto3 syntax, which requires that the zero value be the first value, so the result would be an invalid and uncompilable source file. This has been fixed.
  • When a source file is parsed into a FileDescriptorProto but never linked, the descriptor may be missing the type field of a FieldDescriptorProto (since it is not known whether a named type refers to a message, group, or enum), and it may have options left in the uninterpreted_options field of the various options messages, which has a peculiar representation for aggregate values. When printing a descriptor that had these characteristics, they were not correctly handled, so the output of the printer would be an invalid and uncompilable source file. Both of these cases have been fixed.

v1.15.3

02 Oct 20:17
35c5957
Compare
Choose a tag to compare

This is a bugfix release to address a panics in the desc/builder and desc/protoparse packages.

"github.com/jhump/protoreflect/desc/builder"

Changes/fixes:

  • When building an enum where some values in an EnumBuilder had numbers explicitly assigned and others did not, a panic could occur. This panic has now been fixed. When there is a mix of explicit value numbers and implicit numbers, the implicit values will be auto-assigned available numbers (starting at zero) while the explicit values will use the configured number.

"github.com/jhump/protoreflect/desc/protoparse"

Changes/fixes:

  • When the protoparse package was overhauled in v1.15.0, a bug was inadvertently introduced to the functionality related to the Parser.InferImportPaths flag. If the files being processed imported a file for which an AST was not available (which is actually quite common, since they may import a standard file such as "google/protobuf/empty.proto"), a panic would occur. This panic has been fixed.
  • This release includes some other improvements related to the same Parser.InferImportPaths flag which now allow it to better detect file layouts and remedy import mismatches in more scenarios.

v1.15.2

23 Aug 18:20
8baa453
Compare
Choose a tag to compare

This is a bugfix release to address issues found in the desc/protoparse package.

"github.com/jhump/protoreflect/desc/protoparse"

Changes/fixes:

  • A fix was added to v1.15.1 so that descriptors returned from this package would use statically known extension types for custom options, and custom options in the source that were not statically known would be represented as unrecognized fields. However, the fix was incomplete: only the files whose names were explicitly given to the Parser were updated in this fashion. Other files in the transitive dependency (i.e. imports) did not have options represented this way. As of this release, the fix is complete and all descriptors returned by this package should have the same representation for custom options as in v1.14 and earlier.
  • As of v1.15.0, this package uses the github.com/bufbuild/protocompile package under the hood as the parser. But that package is still using major version zero, since it's API has not completely stabilized. A recent release (v0.6.0) included backwards-incompatible changes that caused this repo to no longer compile. This release addresses the compile errors: this repo now uses that latest release of protocompile and correctly compiles against it.

v1.15.1

04 Mar 01:39
81aa758
Compare
Choose a tag to compare

This is small bugfix release to address regressions reported in v1.15.0.

It also fixes a panic bug that can occur when a program includes both the grpcreflect package from this module and v1.52.0 or newer of the "google.golang.org/grpc" module.

"github.com/jhump/protoreflect/desc/protoparse"

Changes/fixes:

  • As of v1.15.0, descriptors returned from this package would include dynamic extension fields to represent option values for custom options that were defined in the set of files being compiled. This was a regression from all previous versions, where custom options were represented using statically known extension fields (i.e. extensions that were generated to Go code and linked into the calling program) or as unrecognized fields. This breaks some usages, where calling code is using type assertions of the extension value to a generated struct (which fails when the value is a dynamic message value). This has been fixed, and the behavior from v1.14.1 and before has been restored.
  • As of v1.15.0, a data race was possible. This is was a regression, where #236 revealed its ugly head again. The issue would occur if a parse operation used the Parser.LookupImport or Parser.LookupImportProto field to supply some dependencies as descriptors instead of as source. The race occurred when the same descriptor was referenced from different, concurrent parse operations: one goroutine would modify the descriptor during linking, while another goroutine was trying to access it (without synchronization). It has now been fixed (via a fix in the underlying protocompile library: bufbuild/protocompile#103).

"github.com/jhump/protoreflect/grpcreflect"

Changes/fixes:

  • This package includes its own internal copy of generated Go code corresponding to grpc/reflection/v1/reflection.proto. This was because, historically, there were no released Go packages that included it. But as of v1.52.0 of the grpc-go runtime, another package now provides this. If both packages are linked into the same program, a panic will occur during program initialization due to there being multiple, conflicting definitions of these types. As of v1.57.0 of grpc-go, this init-time panic is guaranteed to happen. This panic is resolved in this version (the internal copy is now registered using a non-conflicting name). An example panic stack-trace follows:
    panic: proto: file "reflection.proto" has a name conflict over grpc.reflection.v1.ErrorResponse
    	previously from: "google.golang.org/grpc/reflection/grpc_reflection_v1"
    	currently from:  "github.com/jhump/protoreflect/grpcreflect/internal/grpc_reflection_v1"
    See https://protobuf.dev/reference/go/faq#namespace-conflict
    
    goroutine 1 [running]:
    google.golang.org/protobuf/reflect/protoregistry.glob..func1({0x1075580e0?, 0xc0005e2800?}, {0x1075580e0?, 0xc0005e2840})
            /.../go/pkg/mod/google.golang.org/[email protected]/reflect/protoregistry/registry.go:56 +0x264
    google.golang.org/protobuf/reflect/protoregistry.(*Files).RegisterFile.func1({0x107569940, 0xc0006fc5b0})
            /.../go/pkg/mod/google.golang.org/[email protected]/reflect/protoregistry/registry.go:154 +0x2ec
    google.golang.org/protobuf/reflect/protoregistry.rangeTopLevelDescriptors({0x10756d6c8, 0xc0004e76c0}, 0xc00071ce28)
            /.../go/pkg/mod/google.golang.org/[email protected]/reflect/protoregistry/registry.go:417 +0x148
    google.golang.org/protobuf/reflect/protoregistry.(*Files).RegisterFile(0xc0005ce258, {0x10756d6c8?, 0xc0004e76c0?})
            /.../go/pkg/mod/google.golang.org/[email protected]/reflect/protoregistry/registry.go:149 +0x708
    google.golang.org/protobuf/internal/filedesc.Builder.Build({{0x1072414ee, 0x45}, {0x108070e00, 0x6bd, 0x6bd}, 0x0, 0x8, 0x0, 0x1, {0x10755ec98, ...}, ...})
            /.../go/pkg/mod/google.golang.org/[email protected]/internal/filedesc/build.go:112 +0x22c
    google.golang.org/protobuf/internal/filetype.Builder.Build({{{0x1072414ee, 0x45}, {0x108070e00, 0x6bd, 0x6bd}, 0x0, 0x8, 0x0, 0x1, {0x0, ...}, ...}, ...})
            /.../go/pkg/mod/google.golang.org/[email protected]/internal/filetype/build.go:138 +0x22c
    github.com/jhump/protoreflect/grpcreflect/internal/grpc_reflection_v1.file_reflection_proto_init()
            /.../go/pkg/mod/github.com/jhump/[email protected]/grpcreflect/internal/grpc_reflection_v1/reflection.pb.go:947 +0x558
    github.com/jhump/protoreflect/grpcreflect/internal/grpc_reflection_v1.init.0()
            /.../go/pkg/mod/github.com/jhump/[email protected]/grpcreflect/internal/grpc_reflection_v1/reflection.pb.go:818 +0x24
    

v1.15.0

26 Feb 22:39
bc601ae
Compare
Choose a tag to compare

This is a significant and admittedly long overdue release. This release finally provides a bridge for interoperability between packages and functionality in this repo and the new reflection packages and APIs added in the "v2 API" of the Protobuf runtime for Go (google.golang.org/protobuf), particularly the protoreflect, protodesc, protoregistry, and dynamicpb packages.

This repo began because the Protobuf runtime (at the time, the "v1 API" in github.com/golang/protobuf) had no support for Protobuf reflection and dynamic messages. The functionality slowly grew to include dynamic gRPC, interesting meta features like parsing Protobuf sources or printing Protobuf source from a descriptor, etc.

But the v2 API was released nearly three years ago. And it does provide some functionality that this repo provided, but now in the core runtime library. And yet this repo still refers to the now-deprecated v1 API.

Under the hood, this is a very big change, mainly due to the big changes (described below) in the github.com/jhump/protoreflect/desc and github.com/jhump/protoreflect/desc/protoparse packages.

This release is planned to be the final v1 release. Any other changes significant enough to warrant a new minor version will instead be added in an upcoming v2 version. A new v2 version will have zero references to the v1 API of the Protobuf runtime, and it will not duplicate any of the functionality that now exists in the v2 API.

The changes are all detailed below and include all of the changes described in the past two release candidates (v1.15.0-rc1 and v1.15.0-rc2).


NOTE: Zero defects were reported in the nearly six weeks since the first release candidate was created. It is entirely possible that very few users tried it, so there still could be some material bugs and compatibility issues lurking in this release. If you encounter any marked changes in behavior between the previous release (v1.14.1) and this one, please file an issue as soon as possible! However, if the issue is a performance degradation, it may not be addressed until a v2. So If your use case is very sensitive to performance, you may need to stay on v1.14.1. The under-the-hood changes in this repo for this release are expected to have some performance impact, though it will hopefully be minor or even negligible for most users.


"github.com/jhump/protoreflect/desc"

Changes/fixes:

  • This package, the core of this whole repo, with its descriptor interfaces and related functions, has been substantially overhauled. The descriptor values provided by this package are now backed by the descriptor implementations in the google.golang.org/protobuf/reflect/protoreflect package.
    • The primary advantage of this change is that it is now easy to convert a protoreflect.Descriptor to a desc.Descriptor (via the various Wrap* functions in this package). It is similarly easy to unwrap a desc.Descriptor value, to recover the underlying protoreflectDescriptor.
    • This allows this package to be easily used in conjunction with protoreflect and accompanying packages. You can take a protoreflect.Descriptor and then easily use it with the desc/builder and desc/protoprint packages in this repo. Similarly, you can create a desc.Descriptor using the desc/protoparse package and easily turn that into a protoreflect.Descriptor, for use with the google.golang.org/protobuf/... packages.

Additions:

  • This adds a new DescriptorWrapper interface, which is implemented by all descriptor implementations in this package. It contains an Unwrap() protoreflect.Descriptor function, to recover the underlying protoreflect.Descriptor.
  • Also, all descriptor implementations in this package also have additional methods that are more strongly typed. For example, *desc.FileDescriptor has a method UnwrapFile() protoreflect.FileDescriptor.
  • Finally, this package now has numerous Wrap* functions, which accept a protoreflect.Descriptor and wrap it, returning a desc.Descriptor. There is one function for each concrete type, for example for messages there is WrapMessage(d protoreflect.MessageDescriptor) (*MessageDescriptor, error)

"github.com/jhump/protoreflect/desc/builder"

Changes/fixes:

  • Previously, not all rules of the Protobuf language were enforced when building a new descriptor with this package. Since the desc.Descriptor values returned by this package are now backed by protoreflect.Descriptor values, more rules are enforced. This is because the implementation of descriptors in the protoreflect package does perform all of those validation checks. The new checks that are now enforced that previously were not:
    1. Files with a syntax of proto3 are not allowed to have required fields.
    2. Files with a syntax of proto3 are not allowed to have messages that define extension ranges.
    3. Files with a syntax of proto3 are not allowed to use groups.
    4. Files with a syntax of proto3 are not allowed to declare default values for fields.
    5. Extension fields must use tag numbers that are in an extension range defined on the extended message.
    6. Non-extension fields are not allowed to use tags that lie in a message's extension ranges or reserved ranges.
    7. Non-extension fields are not allowed to use names that the message has marked as reserved.
    8. Extension ranges and reserved ranges must not overlap.

"github.com/jhump/protoreflect/desc/protoparse"

Changes/fixes:

  • This package has been overhauled perhaps even more than the desc package. The implementation in this package has been completely replaced with the functionality of the github.com/bufbuild/protocompile package. So the exported APIs in this package are now just adapters. The actual parser/compiler is implemented in this other dependency. Most of this adaptation logic is trivial with the exception being for the Parser.ParseToAST method, which must convert from protocompile's AST model to the model defined in desc/protoparse/ast.

"github.com/jhump/protoreflect/desc/protoprint"

Changes/fixes:

  • Formatting of message literals in custom option values has been improved.
    • It now encloses nested messages in curly braces ({ and }) instead of angle brackets (< and >), which is better aligned with the commonly-used and preferred syntax style.
    • Also, nested messages now respect the Printer.MessageLiteralExpansionThresholdLength, instead of this threshold only being applied for a top-level message. So small nested messages can be emitted in compact form, even if enclosed within a message literal that has been expanded.
  • If printing descriptors that had no source code info and no other sort configuration was used on the Printer, then elements with multiple option declarations would have those options printed in seemingly random order. The actual order was based on a map iteration, and was thus non-deterministic. This has been fixed. In such a case, options will now be ordered by name.

"github.com/jhump/protoreflect/desc/sourceinfo"

Additions:

  • A new TypeResolver interface has been added, as well as a GlobalTypes package variable that implements that interface. This is analogous to protoregistry.GlobalTypes, just as the existing sourceinfo.GlobalFiles corresponds to protoregistry.GlobalFiles.
  • This adds new Wrap* functions, that accept various protoreflect types and return values that are identical to the input values except that they include source code info that was registered with this package.

"github.com/jhump/protoreflect/dynamic"

Changes/fixes:

  • Previously, if protoreflect APIs were used to store a value in an extension field, using a protoreflect.FieldDescriptor that was not a generated extension (i.e. not known at compile-time), trying to convert the resulting message to a *dynamic.Message would overlook that field; it would be absent from the result, not even appearing in the unrecognized fields. This has been fixed. Such extension values will be in the resulting *dynamic.Message and appear as recognized/known fields.