Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: export PaginationIterator helper class #273

Draft
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

mvolfik
Copy link
Contributor

@mvolfik mvolfik commented Sep 7, 2022

No description provided.

@mvolfik mvolfik requested a review from B4nan September 7, 2022 14:26
@mvolfik
Copy link
Contributor Author

mvolfik commented Sep 7, 2022

wait, i realized this probably isn't really what I wanted. do we have some util for easily iterating over all values from a PaginatedList (e.g. returned by await client.tasks().list()?

@mvolfik
Copy link
Contributor Author

mvolfik commented Sep 7, 2022

Here's what I (well, GitHub Copilot mostly) came up with:

import type { PaginatedList } from 'apify-client';

export async function* iterateList<T>(
    listGetter: (opts: { limit: number; offset: number }) => Promise<PaginatedList<T>>,
    limit: number,
): AsyncGenerator<T, void, never> {
    let offset = 0;
    let total = 0;
    do {
        const {
            items,
            count: returnedCount,
            total: returnedTotal,
        } = await listGetter({ limit, offset });

        offset += returnedCount;
        total = returnedTotal;
        for (const item of items) yield item;
    } while (offset < total);
}

Would it be worth including and exporting this util somewhere?

@mvolfik mvolfik marked this pull request as draft September 7, 2022 14:47
@gippy gippy added the t-tooling Issues with this label are in the ownership of the tooling team. label Nov 20, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
t-tooling Issues with this label are in the ownership of the tooling team.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants