AIP-146

Generic fields

Most fields in any API, whether in a request, a resource, or a custom response, have a specific type or schema. This schema is part of the contract that developers write their code against.

However, occasionally it is appropriate to have a generic or polymorphic field of some kind that can conform to multiple schemata, or even be entirely free-form.

Guidance

While generic fields are generally rare, a service may introduce generic field where necessary. There are several approaches to this depending on how generic the field needs to be; in general, services should attempt to introduce the "least generic" approach that is able to satisfy the use case.

Oneof

A oneof may be used to introduce a type union: the user or service is able to specify one of the fields inside the oneof. Additionally, a oneof may be used with the same type (usually strings) to represent a semantic difference between the options.

Because the individual fields in the oneof have different keys, a developer can programmatically determine which (if any) of the fields is populated.

A oneof preserves the largest degree of type safety and semantic meaning for each option, and services should generally prefer them over other generic or polymorphic options when feasible. However, the oneof construct is ill-suited when there is a large (or unlimited) number of potential options, or when there is a large resource structure that would require a long series of "cascading oneofs".

Note: Adding additional possible fields to an existing oneof is a non-breaking change, but moving existing fields into or out of a oneof is breaking (it creates a backwards-incompatible change in Go protobuf stubs).

Maps

Maps may be used in situations where many values of the same type are needed, but the keys are unknown or user-determined.

Maps are usually not appropriate for generic fields because the map values all share a type, but occasionally they are useful. In particular, a map can sometimes be suited to a situation where many objects of the same type are needed, with different behavior based on the names of their keys (for example, using keys as environment names).

Struct

The google.protobuf.Struct object may be used to represent arbitrary nested JSON. Keys can be strings, and values can be floats, strings, booleans, arrays, or additional nested structs, allowing for an arbitrarily nested structure that can be represented as JSON (and is automatically represented as JSON when using REST/JSON).

A Struct is most useful when the service does not know the schema in advance, or when a service needs to store and retrieve arbitrary but structured user data. Using a Struct is convenient for users in this case because they can easily get JSON objects that can be natively manipulated in their environment of choice.

If a service needs to reason about the schema of a Struct, it should use JSONSchema for this purpose. Because JSONSchema is itself JSON, a valid JSONSchema document can itself be stored in a Struct.

Any

The google.protobuf.Any object can be used to send an arbitrary serialized protocol buffer and a type definition.

However, this introduces complexity, because an Any becomes useless for any task other than blind data propagation if the consumer does not have access to the proto. Additionally, even if the consumer does have the proto, the consumer has to ensure the type is registered and then deserialize manually, which is an often-unfamiliar process.

Because of this, Any should not be used unless other options are infeasible.