Iterator Guidelines

Introduction

The Google Cloud Client Libraries for Go should be as consistent as possible. Since Go has no standard iterator pattern, this document establishes guidelines for iterators.

Most iterators will result from the standard Google APIs List method with the pagination pattern: each call to a List method for a resource returns a sequence (“page”) of resource items (e.g. Books) along with a “next-page token” that can be passed to the List method to retrieve the next page.

Each List method will result in an iterator (which we call a List iterator) with two methods, one for iterating over individual items and one to support paging. Iterators may also arise from other sources, like streaming RPCs or Cloud PubSub message subscriptions. These may or may not support page-by-page iteration.

Examples

Here is what iterators written according to these guidelines will look like to users. Here is the List iterator for the Book resource in the library example:

it := client.Books(ctx, shelfName)
for {
	book, err := it.Next()
	if err == iterator.Done {
		break
	}
	if err != nil {
		return err
	}
	process(book)
}

The unique error Done indicating the end of iteration is in the google.golang.org/api/iterator package.

Here's the same code using a switch:

it := client.Books(ctx, shelfName)
loop:
for {
	book, err := it.Next()
	switch err {
	case nil:
		process(book)
	case iterator.Done:
		break loop
	default:
		return err
	}
}

Iteration by pages is done with an iterator.Pager:

it := client.Books(ctx, shelfName)
p := iterator.NewPager(it, pageSize, "") 
for {
    var books []*library.Book
    nextPageToken, err := p.NextPage(&books)
    if err != nil {
		return err
    }
	for _, b := range books {
		process(b)
	}
	if nextPageToken == "" {
		break
	}
}

Here we retrieve the first page of 25 books and display the page and the next-page token:

it := client.Books(ctx, shelfName)
var books []*library.Book
nextPageToken, err := iterator.NewPager(it, 25, "").NextPage(&books)
if err != nil {
    return err
}
display(books, nextPageToken)

When the next-page token is handed back to us later (possibly in another process), we can get the next page:

it := client.Books(ctx, shelfName)
var books []*library.Book
nextPageToken, err := iterator.NewPager(it, 25, token).NextPage(&books)
if err != nil {
    return err
}
display(books, nextPageToken)

The Iterator Type

An iterator should be represented by a type whose name ends in Iterator. If the iterator is a List iterator, the type's name should be ResourceIterator, E.g. BookIterator. The type should have at least one method, called Next. Next is described below.

Example:

type BookIterator struct { ... }

The Creating Method

Typically, the client will have a single method that returns an iterator of a particular type. We will call this the creating method.

The name of the creating method for a List iterator should be the plural of the resource, e.g. Books (not ListBooks, which is a bit verbose for Go). For other kinds of iterators, the name of the creating method should be a plural noun, but a different name can be used if it makes more sense.

The first argument to the creating method should be a context.Context. The iterator will use that context throughout the iteration. In the unlikely event that neither the creating method nor the iterator makes any RPCs, the context can be omitted.

The creating method may accept other arguments after the context, as needed.

In most cases, the creating method will simply create an instance of the iterator type and return it, leaving the initial work, including any RPCs, to the first call to Next:

func (c *Client) Books(ctx context.Context, shelf string) *BookIterator { ... }

A creating method may return an error along with the iterator.

The Next Method

An iterator over values of type T will have a method called Next that returns (T, error). For example,

func (it *BookIterator) Next() (*Book, error) { ... }

Next will typically have no arguments, but it may in some cases. (For example, the Datastore iterator's Next method takes an argument into which it copies the entity.) None of the arguments should be a context, because Next should use the context passed when the iterator was created.

Following standard Go convention, if Next’s second return value is non-nil, then the first must be the zero value for T.

A special error value returned by Next signals the successful end of the iteration. This sentinel value is the value of the variable Done in the package google.golang.org/api/iterator. After Next returns iterator.Done, all subsequent calls to it will return Done.

If feasible, the user should be able to continue calling Next even if it returns an error that is not Done. If that is not feasible, it should be so documented.

The documentation comment for Next should be as follows:

Next returns the next result. Its second return value is iterator.Done if there are no more results. Once Next returns Done, all subsequent calls will return Done.

The PageInfo Method

Iterators that support pagination, such as List iterators, should also have a PageInfo method that returns an *iterator.PageInfo. For details, see the iterator package documentation and source.

The documentation comment for PageInfo should be as follows:

PageInfo supports pagination. See the google.golang.org/api/iterator package for details.

The Close and Stop Methods

An iterator may require clean-up work to happen after it completes, or if the user abandons it before reaching the end. The iterator type should have a method of no arguments that performs this clean-up.

If the clean-up work can result in an error that should be returned, the method should be named Close. If no error is possible (or if the error is to be exposed in another way), the method should be named Stop:

func (it *CleanupWithErrorIterator) Close() error { ... }

func (it *CleanupNoErrorIterator) Stop() { ... }

If an iterator does not require clean-up, it should define neither Close nor Stop.

If Close or Stop does not support being called multiple times, that should be documented.

Other Guidelines

None of the iterator methods are required to be safe for concurrent use by different goroutines, and Close (or Stop) and Next are not required to be concurrently callable.

Iterators may have fields and methods other than those described here.