Data Series

Welcome to the second edition of the AIP newsletter, which is designed to keep you up to date about the AIP program, and particular proposals making their way through the system.

This month's new AIPs (unintentionally) share a common theme: handling lists and series of data, whether in fields, resource relationships, or actual list methods. These new AIPs are long overdue: they codify knowledge that has long been reasonably consistent, but tribal.

The AIP newsletter kicks off what is effectively a "public comment" period: the AIP editors are happy with these proposals, but we want to ensure that you are too. Assuming feedback is sufficiently positive, we intend to formally approve these proposals on Friday, April 24, 2020.

AIPs under review

AIP-124: Resource associations

APIs sometimes have resource hierarchies that can not be cleanly expressed in the usual tree structure.

Resource association comes into play when a resource needs something other than a direct parent-child relationship. This might mean a many-to-many relationship, or it might be a one-to-many association that is distinct from the parent-child relationship.

We have had several APIs struggle with unusual associations over the past year or so. In general, this has been handled reasonably consistently, but we never codified the guidance into an official AIP.

Summary: AIP-124 provides patterns for the two most common associations: a resource with multiple different one-to-many relationships, and a resource with a many-to-many association with another resource.

We are aiming to approve AIP-124 on April 24. If you have feedback, please leave a comment.

AIP-144: Repeated fields

Representing lists of data in an API is trickier than it often appears. Users often need to modify lists in place, and longer data series within a single resource pose a challenge for pagination.

Repeated fields are somewhat common in APIs, and a common question that we get is how to handle adding or removing a single item from them atomically. Additionally, repeated fields can become a trap if they are unbounded in size and become too large.

This is codification of guidance that we have been giving on a case-by-case basis for some time. In particular, this documents the various restrictions on the two update strategies.

Summary: AIP-144 lays out guidance for repeated fields, and details two distinct strategies for updating repeated fields.

We are aiming to approve AIP-144 on April 24. If you have feedback, please leave a comment.

AIP-160: Filtering

Often, when listing resources, it is desirable to filter over the collection and only return results that the user is interested in.

Filtering involves taking a list of resources (usually) and only providing the results that the user wants, specified using a familiar syntax similar to that of Google search.

This is likely the most-requested AIP of all time. We have had internal filtering documents for ages, but we have not had a public version, and one has been sorely needed for years. Also, this work would likely still be unfinished if not for the efforts of Tristan Swadell; we are grateful for his efforts.

Summary: AIP-160 offers a detailed breakdown of the filtering syntax and how to provide filtering results in list and search methods.

We are aiming to approve AIP-160 on April 24. If you have feedback, please leave a comment.

Recent updates

In addition to the new AIPs under review, we have added the following guidance to existing AIPs:

  • AIP-133: Clarify which fields can be required (#460)
  • AIP-140: Add guidance about message and field name conflicts (#470)
  • AIP-151: Clarify that both response_type and metadata_type are expected (#469)
  • AIP-231: Clarify Batch Get behavior if no resource names are sent (#474)