- Updated internals and libraries. The scraper should be even more efficient now.
- Added
max_request_retries
field and retrying failed requests in general.
- Removed sync mode and
run_async
option. Everything is async now. - Added
max_concurrency
field. This fixes all memory problems with previous async implementation.
- Added support of async scraping. Can be turned on with
"run_async": true
. - Added buffering of results before pushing into dataset (to not overwhelm Apify API). Can be changed via
push_data_size
.