AIP-203

Field behavior documentation

When defining fields in protocol buffers, it is customary to explain to users certain aspects of the field's behavior (such as whether it is required or optional). Additionally, it can be useful for other tools to understand this behavior (for example, to optimize client library signatures).

Guidance

APIs use the google.api.field_behavior annotation to describe well-understood field behavior, such as a field being required or immutable.

// The audio data to be recognized.
RecognitionAudio audio = 2 [(google.api.field_behavior) = REQUIRED];
  • APIs must apply the google.api.field_behavior annotation on every field on a message or sub-message used in a request.
  • The annotation must include any google.api.FieldBehavior values that accurately describe the behavior of the field.
    • FIELD_BEHAVIOR_UNSPECIFIED must not be used.
  • APIs must at minimum use one of REQUIRED, OPTIONAL, or OUTPUT_ONLY.

Warning: Although field_behavior does not impact proto-level behavior, many clients (e.g. CLIs and SDKs) rely on them to generate code. Thoroughly review and consider which values are relevant when adding a new field.

Fields with no annotation are interpreted as OPTIONAL for backwards-compatility. Nontheless, this annotation must not be omitted.

Note: The vocabulary given in this document is for descriptive purposes only, and does not itself add any validation. The purpose is to consistently document this behavior for clients.

field behavior of nested messages

google.api.field_behavior annotations on a nested message are independent of the annotations of the parent.

For example, a nested message can have a field behavior of REQUIRED while the parent field can be OPTIONAL:

message Title {
  string text = 1 [(google.api.field_behavior) = REQUIRED];
}

message Slide {
  Title title = 1 [(google.api.field_behavior) = OPTIONAL];
}

In the case above, if a title is specified, the text field is required.

Vocabulary

Identifier

The use of IDENTIFIER indicates that a field within a resource message is used to identify the resource. It must be attached to the name field and must not be attached to any other field (see fields representing resource names).

This annotation must not be applied to references to other resources within a message.

Immutable

The use of IMMUTABLE indicates that a field on a resource cannot be changed after it's creation. This can apply to either fields that are input or outputs, required or optional.

When a service receives an immutable field in an update request (or similar), even if included in the update mask, the service should ignore the field if the value matches, but should error with INVALID_ARGUMENT if a change is requested.

Potential use cases for immutable fields (this is not an exhaustive list) are:

  • Attributes of resources that are not modifiable for the lifetime of the application (e.g. a disk type).

Note: Fields which are "conditionally immutable" must not be given the immutable annotation.

Input only

The use of INPUT_ONLY indicates that the field is provided in requests and that the corresponding field will not be included in output.

Additionally, a field should only be described as input only if it is a field in a resource message or a field of a message included within a resource message. Notably, fields in request messages (a message which only ever acts as an argument to an RPC, with a name usually ending in Request) should not be described as input only because this is already implied.

Potential use cases for input only fields (this is not an exhaustive list) are:

  • The ttl field as described in AIP-214.

Warning: Input only fields are rare and should be considered carefully before use.

Optional

The use of OPTIONAL indicates that a field is not required.

A field may be described as optional if it is a field on a request message (a message that is an argument to an RPC, usually ending in Request), or a field on a submessage.

Output only

The use of OUTPUT_ONLY indicates that the field is provided in responses, but that including the field in a message in a request does nothing (the server must clear out any value in this field and must not throw an error as a result of the presence of a value in this field on input). Similarly, services must ignore the presence of output only fields in update field masks.

Additionally, a field should only be described as output only if it is a field in a resource message, or a field of a message farther down the tree. Notably, fields in response messages (a message which only ever acts as a return value to an RPC, usually ending in Response) should not be described as output only because this is already implied.

Output only fields may be set to empty values if appropriate to the API.

Potential use cases for output only fields (this is not an exhaustive list) are:

  • Create or update timestamps.
  • Derived or structured information based on original user input.
  • Properties of a resource assigned by the service which can not be altered.

Required

The use of REQUIRED indicates that the field must be present (and set to a non-empty value) on the request or resource.

A field should only be described as required if either:

  • It is a field on a resource that a user provides somewhere as input. In this case, the resource is only valid if a "truthy" value is stored.
    • When creating the resource, a value must be provided for the field on the create request.
    • When updating the resource, the user may omit the field provided that the field is also absent from the field mask, indicating no change to the field (otherwise it must be provided).
  • It is a field on a request message (a message that is an argument to an RPC, with a name usually ending in Request). In this case, a value must be provided as part of the request, and failure to do so must cause an error (usually INVALID_ARGUMENT).

We define the term "truthy" above as follows:

  • For primitives, values other than 0, 0.0, empty string/bytes, and false
  • For repeated fields maps, values with at least one entry
  • For messages, any message with at least one "truthy" field.

Fields should not be described as required in order to signify:

  • A field which will always be present in a response.
  • A field which is conditionally required in some situations.
  • A field on any message (including messages that are resources) which is never used as user input.

Note: In most cases, empty values (such as false for booleans, 0 for integers, or the unspecified value for enums) are indistinguishable from unset values, and therefore setting a required field to a falsy value yields an error. A corollary to this is that a required boolean must be set to true.

Unordered List

The use of UNORDERED_LIST on a repeated field of a resource indicates that the service does not guarantee the order of the items in the list.

A field should be described as an unordered list if the service does not guarantee that the order of the elements in the list will match the order that the user sent, including a situation where the service will sort the list on the user's behalf.

A resource with an unordered list may return the list in a stable order, or may return the list in a randomized, unstable order.

Backwards compatibility

Adding or changing google.api.field_behavior values can represent a semantic change in the API that is perceived as incompatible for existing clients. The following are examples of backwards incompatible changes with google.api.field_behavior:

  • Adding REQUIRED to an existing field previously considered OPTIONAL (implicitly or otherwise)
  • Adding a new field annotated as REQUIRED to an existing request message
  • Adding OUTPUT_ONLY to an existing field previously accepted as input
  • Adding INPUT_ONLY to an existing field previously emitted as output
  • Adding IMMUTABLE to an existing field previously considered mutable
  • Removing OUTPUT_ONLY from an existing field previously ignored as input
  • Removing IDENTIFIER from an existing field.

There are some changes that are backwards compatible, which are as follows:

  • Adding OPTIONAL to an existing field
  • Adding IDENTIFIER to an existing name field
  • Changing from REQUIRED to OPTIONAL on an existing field
  • Changing from OUTPUT_ONLY and/or IMMUTABLE to IDENTIFIER on an existing field
  • Removing REQUIRED from an existing field
  • Removing INPUT_ONLY from an existing field previously excluded in responses
  • Removing IMMUTABLE from an existing field previously considered immutable

Rationale

Required set of annotations

A field used in a request message must be either an input or an output.

In the case of an output, the OUTPUT_ONLY annotation is sufficient.

In the case of an input, a field is either required or optional, and therefore should have at least the REQUIRED or OPTIONAL annotation, respectively.

Requiring field behavior

By including the field behavior annotation for each field, the overall behavior that the resource exhibits is more clearly defined. Clearly defined field behavior improves programmatic clients and user understanding.

Requiring the annotation also forces the API author to explicitly consider the behavior when initially authoring of the API.

Modifying field behavior after initial authoring can result in backwards-incompatible changes in clients. For example, making an optional field required results in backwards-incompatible changes in the method signature of an RPC or a resource in a Declarative client. See the Backwards compatibility section for more detailed compatibility guidance.

History

In 2023-05 field_behavior was made mandatory. Prior to this change, the annotation was often omitted. Its values are relied upon to produce high quality clients. Furthermore, adding or changing some of the field_behavior values after the fact within a major version can be backwards-incompatible. See the Backwards compatibility section for more detailed compatibility guidance.

The benefits of requiring field_behavior at the time that the API is authored surpass the costs to clients and API users of not doing so.

Changelog

  • 2023-09-14: Clarify that nested behavior and parent behavior are independent.
  • 2023-08-25: Add guidance on IDENTIFIER.
  • 2023-07-20: Describe compatibility guidance with new section.
  • 2023-05-24: Clarify that IMMUTABLE does not imply input nor required.
  • 2023-05-10: Added guidance to require the annotation.
  • 2020-12-15: Added guidance for UNORDERED_LIST.
  • 2020-05-27: Clarify behavior when receiving an immutable field in an update.
  • 2019-12-05: Added guidance on output only fields in field masks.
  • 2019-06-18: Use the machine-readable annotation, not comments.