Welcome to the AIP Newsletter

Welcome to the first (hopefully monthly) edition of the AIP newsletter, which is designed to keep you up to date about the AIP program, and particular proposals making their way through the system.

In this edition, we will discuss five pending proposals. This is intended as a "public comment" period: the AIP editors are happy with these proposals, but we want to ensure that you are too. Assuming feedback is sufficiently positive, we intend to formally approve these proposals on Friday, March 27, 2020.

API linter

As many readers know, we have published an API linter, which is able to check much of the guidance provided in API Improvement Proposals for APIs defined in protocol buffers.

The linter now has over 150 rules, and is running on many new and existing APIs at Google, hopefully pushing us toward greater consistency.

AIPs under review

AIP-153: Import and export

Many users want to be able to load data into an API, or get their existing data out of an API. This is particularly important for enterprise users, who are often concerned about vendor lock.

Import and export are particularly important concepts for enterprise APIs, although they are increasing in importance for consumer and SaaS APIs as well.

We have actually had a reasonably consistent import and export story in Google Cloud for a long time, but it has largely been passed along by tribal knowledge ("go look at X and copy what they did"). Therefore, this AIP attempts to codify what we believe is a long-standing practice.

Summary: AIP-153 mandates that both import and export methods return long-running operations. Unlike most API methods, they do support partial failures, which are handled through the LRO metadata. Finally, they have a specified place for per-source and per-destination configuration, distinct from configuration for the data itself.

We are aiming to approve AIP-153 on March 27. If you have feedback, please leave a comment.

AIP-162: Resource revisions

Some APIs need to have resources with a revision history, where users can reason about the state of the resource over time.

Resource revisions is a concept we are seeing crop up more and more, and we are seeing an increasing number of situations where audit logging is a necessary but insufficient solution. Several APIs have struggled with how to model a resource's history over time, and handle common use cases like commit and rollback, which is what prompted this AIP.

This is new guidance, put together by the AIP team as well as several API teams who needed this functionality near the end of 2019. It has been positively received by the teams that needed it, and so we would like to codify it.

Summary: AIP-162 mandates the use of revision IDs that are appended to a resource name with the @ character for signifying revisions, and sets out method structures for most common operations (listing revisions, rollback, etc.). It also proposes a tagging mechanism for user-friendly revision names.

We are aiming to approve AIP-162 on March 27. If you have feedback, please leave a comment.

AIP-163: Change validation

Occasionally, a user wants to validate an intended change to see what the result will be before actually making the change. For example, a request to provision new servers in a fleet will have an impact on the overall fleet size and cost, and could potentially have unexpected downstream effects.

Change validation is the concept of performing a "dry run" on an API request, and returning back the result without making any changes.

This is effectively a migration of long-standing guidance that was not very fleshed out. The previous API design guide mentioned validate_only as an entry in the standard fields table with minimal explanation. This proposal fleshes that out with an example, and addresses some questions we have received over time.

Summary: AIP-163 offers guidance for "dry run" requests using a bool validate_only field in the request message, and addresses some common situations around permissions and incomplete output.

We are aiming to approve AIP-163 on March 27. If you have feedback, please leave a comment.

AIP-165: Criteria-based delete

Occasionally, an API may need to provide a mechanism to delete a large number of resources based on some set of filter parameters, rather than requiring the individual resource name of the resources to be deleted.

Criteria-based delete is designed to address problems where a user needs to delete so many items that batch delete simply becomes infeasible.

This proposal represents a concession of sorts: we have long tried to avoid delete-by-filter because of concerns that users would accidentally delete important records and be unable to recover them. However, several APIs have had legitimate reasons to need to do this, because the number of records the user would want to delete was so high.

Summary: AIP-165 offers guidance for criteria-based delete, with a common name (Purge) and structure, and mandates a dry-run by default with a force flag to actually perform the purge.

We are aiming to approve AIP-165 on March 27. If you have feedback, please leave a comment.

AIP-194: Retries for gRPC clients

RPCs sometimes fail. When one does, the client performing the RPC needs to know whether it is safe to retry the operation. When status codes are used consistently across multiple APIs, clients can respond to failures appropriately.

Retries for gRPC clients handles how to deal with errors, and in particular when a client can transparently retry those errors.

This proposal came about because of a need for first-party client libraries to be able to transparently retry some server errors. gRPC offers a configuration strategy for this, which is defined by API producers.

Summary: AIP-194 offers guidance for configuring what error codes from an API are safe to retry. It recommends usually retrying UNAVAILABLE and nothing else.

We are aiming to approve AIP-194 on March 27. If you have feedback, please leave a comment.

Recent updates

In addition to the new AIPs under review, we have added the following guidance to existing AIPs:

  • AIP-8: AIPs have reverse-chronological changelogs (#425)
  • AIP-135: Delete methods send 403 errors if the user lacks access (#409)
  • AIP-140: display_name and title standard fields (#405)
  • AIP-158: Field order for pagination response messages (#419)
  • AIP-193: Added reference to new ErrorInfo message (#396)

What is coming?

We have more coming over the course of the next few months, including long-awaited public guidance for filtering and versioning.

You also may notice some pull requests around making AIPs more generic. We are exploring the possibility of bringing AIPs to even more API producers across multiple companies, so that others can publish their standards under a common information architecture. This effort is still nascent, but we are excited and hopeful. Stay tuned!