AIP-235

Batch methods: Delete

Some APIs need to allow users to delete a set of resources in a single transaction. A batch delete method provides this functionality.

Guidance

APIs may support Batch Delete using the following two patterns:

Returning the response synchronously

rpc BatchDeleteBooks(BatchDeleteBooksRequest) returns (google.protobuf.Empty) {
  option (google.api.http) = {
    post: "/v1/{parent=publishers/*}/books:batchDelete"
    body: "*"
  };
}

Returning an Operation which resolves to the response asynchronously

rpc BatchDeleteBooks(BatchDeleteBooksRequest) returns (google.longrunning.Operation) {
  option (google.api.http) = {
    post: "/v1/{parent=publishers/*}/books:batchDelete"
    body: "*"
  };
  option (google.longrunning.operation_info) = {
    response_type: "google.protobuf.Empty"
    metadata_type: "BatchDeleteBooksOperationMetadata"
  };
}
  • The RPC's name must begin with BatchDelete. The remainder of the RPC name should be the plural form of the resource being deleted.
  • The request message must match the RPC name, with a Request suffix.
  • The response message should be google.protobuf.Empty.
    • If the resource is soft deleted, the response message should be a response message containing the updated resources.
  • If the batch method returns an google.longrunning.Operation, both the response_type and metadata_type fields must be specified.
    • If the resource is soft deleted, the response_type should be a response message containing the updated resources.
  • The HTTP verb must be POST (not DELETE).
  • The HTTP URI must end with :batchDelete.
  • The URI path should represent the collection for the resource, matching the collection used for simple CRUD operations. If the operation spans parents, a dash (-) may be accepted as a wildcard.
  • The body clause in the google.api.http annotation should be "*".

Atomic vs. Partial Success

  • The batch delete method may support atomic (all resources deleted or none are) or partial success behavior. To make a choice, consider the following factors:
    • Complexity of Ensuring Atomicity: Operations that are simple passthrough database transactions should use an atomic operation, while operations that manage complex resources should use partial success operations.
    • End-User Experience: Consider the perspective of the API consumer. Would atomic behavior be preferable for the given use case, even if it means that a large batch could fail due to issues with a single or a few entries?
  • Synchronous batch delete must be atomic.
  • Asynchronous batch delete may support atomic or partial success.

Request message

The request for a batch delete method should be specified with the following pattern:

message BatchDeleteBooksRequest {
  // The parent resource shared by all books being deleted.
  // Format: publishers/{publisher}
  // If this is set, the parent of all of the books specified in `names`
  // must match this field.
  string parent = 1 [
    (google.api.resource_reference) = {
      child_type: "library.googleapis.com/Book"
    }];

  // The names of the books to delete.
  // A maximum of 1000 books can be deleted in a batch.
  // format: publishers/{publisher}/books/{book}
  repeated string names = 2 [
    (google.api.field_behavior) = REQUIRED,
    (google.api.resource_reference) = {
      type: "library.googleapis.com/Book"
    }];
}
  • A parent field should be included, unless the resource being deleted is a top-level resource. If a caller sets this field, and the parent collection in the name of any resource being deleted does not match, the request must fail.
    • This field should be required if only 1 parent per request is allowed.
    • The field should identify the resource type that it references.
    • The comment for the field should document the resource pattern.
  • The request message must include a repeated field which accepts the resource names specifying the resources to delete. The field should be named names.
    • The field should be required.
    • The field should identify the resource type that it references.
    • The comment for the field should document the resource pattern.
  • Other fields besides name may be "hoisted" from the standard Delete request. There is no way to allow for these fields to accept different values for different resources; if this is needed, use the alternative request message form.
  • The request message must not contain any other required fields, and should not contain other optional fields except those described in this or another AIP.
  • The comment above the names field should document the maximum number of requests allowed.
  • Filter-based matching must not be supported.

Request message containing standard delete request messages

If the standard Delete request message contains a field besides the resource name that needs to be different between different resources being requested, the batch message may alternatively hold a repeated field of the standard Delete request message. This is generally discouraged unless your use case really requires it.

The request for a batch delete method should be specified with the following pattern:

message BatchDeleteBooksRequest {
  // The parent resource shared by all books being deleted.
  // Format: publishers/{publisher}
  // If this is set, the parent of all of the books specified in the
  // DeleteBookRequest messages must match this field.
  string parent = 1 [
    (google.api.resource_reference) = {
      child_type: "library.googleapis.com/Book"
    }];

  // The requests specifying the books to delete.
  // A maximum of 1000 books can be deleted in a batch.
  repeated DeleteBookRequest requests = 2
    [(google.api.field_behavior) = REQUIRED];
}
  • A parent field should be included. If a caller sets this field, and the parent collection in the name of any resource being deleted does not match, the request must fail.
    • This field should be required if only 1 parent per request is allowed.
    • The field should identify the resource type that it references.
    • The comment for the field should document the resource pattern.
  • The request message must include a repeated field which accepts the request messages specifying the resources to delete, as specified for standard Delete methods. The field should be named requests.
    • The field should be required.
  • Other fields may be "hoisted" from the standard Delete request, which means that the field can be set at either the batch level or child request level. Similar to parent, if both the batch level and child request level are set for the same field, the values must match.
    • Fields which must be unique cannot be hoisted (e.g. etag).
  • The request message must not contain any other required fields, and should not contain other optional fields except those described in this or another AIP.
  • The comment above the requests field should document the maximum number of requests allowed.
  • Filter-based matching must not be supported unless it is infeasible to support critical use cases without it, because it makes it too easy for users to accidentally delete important data. If it is unavoidable, see AIP-165.

Response message (soft-delete only)

In the case where a response message is necessary because the resource is soft-deleted, the response should be specified with the following pattern:

message BatchDeleteBooksResponse {
  // Books deleted.
  repeated Book books = 1;
}
  • The response message must include one repeated field corresponding to the resources that were soft-deleted.

Operation metadata message

  • The metadata_type message must either match the RPC name with OperationMetadata suffix, or be named with Batch prefix and OperationMetadata suffix if the type is shared by multiple Batch methods.
  • If batch delete method supports partial success, the metadata message must include a map<int32, google.rpc.Status> failed_requests field to communicate the partial failures.
    • The key in this map is the index of the request in the requests field in the batch request.
    • The value in each map entry must mirror the error(s) that would normally be returned by the singular Standard Delete method.
    • If a failed request can eventually succeed due to server side retries, such transient errors must not be communicated using failed_requests.
    • When all requests in the batch fail, Operation.error must be set with code = google.rpc.Code.Aborted and message = "None of the requests succeeded, refer to the BatchDeleteBooksOperationMetadata.failed_requests for individual error details"
  • The metadata message may include other fields to communicate the operation progress.

Adopting Partial Success

In order for an existing Batch API to adopt the partial success pattern, the API must do the following:

  • The default behavior must be retained to avoid incompatible behavioral changes.
  • If the API returns an Operation:
    • The request message must have a bool return_partial_success field.
    • The Operation metadata_type must include a map<int32, google.rpc.Status> failed_requests field.
    • When the bool return_partial_success field is set to true in a request, the API should allow partial success behavior, otherwise it should continue with atomic behavior as default.
  • If the API returns a direct response synchronously:
    • Since the existing clients will treat a success response as an atomic operation, the existing version of the API must not adopt the partial success pattern.
    • A new version must be created instead that returns an Operation and follows the partial success pattern described in this AIP.

Rationale

Restricting synchronous batch methods to be atomic

The restriction that synchronous batch methods must be atomic is a result of the following considerations.

The previous iteration of this AIP recommended batch methods must be atomic. There is no clear way to convey partial failure in a sync response status code because an OK implies it all worked. Therefore, adding a new field to the response to indicate partial failure would be a breaking change because the existing clients would interpret an OK response as all resources created.

On the other hand, as described in AIP-193, Operations are more capable of presenting partial states. The response status code for an Operation does not convey anything about the outcome of the underlying operation and a client has to check the response body to determine if the operation was successful.

Communicating partial failures

The AIP recommends using a map<int32, google.rpc.Status> failed_requests field to communicate partial failures, where the key is the index of the failed request in the original batch request. The other options considered were:

  • A repeated google.rpc.Status field. This was rejected because it is not clear which entry corresponds to which request.
  • A map<string, google.rpc.Status> field, where the key is the request id of the failed request. This was rejected because:
    • Client will need to maintain a map of request_id -> request in order to use the partial success response.
    • Populating a request id for the purpose of communicating errors could conflict with AIP-155 if the service can not guarantee idempotency for an individual request across multiple batch requests.
  • A repeated FailedRequest field, where FailedRequest contains the individual create request and the google.rpc.Status. This was rejected because echoing the request payload back in response is discouraged due to additional challenges around user data sensitivity.

Changelog

  • 2025-03-06: Changed recommendation to allow partial success, along with detailed guidance
  • 2022-06-02: Changed suffix descriptions to eliminate superfluous "-".
  • 2020-09-16: Suggested annotating parent, names, and requests fields.
  • 2020-08-27: Removed parent recommendations for top-level resources.
  • 2020-03-27: Added reference to AIP-165 for criteria-based deletion.
  • 2019-10-11: Changed the primary recommendation to specify a repeated string instead of a repeated standard Delete message. Moved the original recommendation into its own section.
  • 2019-09-11: Fixed the wording about which child field the parent field should match.
  • 2019-08-01: Changed the examples from "shelves" to "publishers", to present a better example of resource ownership.