All notable changes to this project will be documented in this file.
2.11.0 (2024-12-16)
- actor-build: Add actorDefinition type for actor build detail, deprecate inputSchema and readme. (#611) (123c2b8) by @drobnikj
- Add
charge
method to the run client for "pay per event" (#613) (3d9c64d) by @Jkuzz - request-queue: Add queueHasLockedRequests and clientKey into RequestQueueClientListAndLockHeadResult (#617) (f58ce98) by @drobnikj
- actor: Correctly set type for ActorTaggedBuilds (#612) (3bda7ee) by @metalwarrior665
2.10.0 (2024-11-01)
- Add user.updateLimits (#595) (bf97c0f) by @MFori, closes #329
- Allow appending custom parts to the user agent (#602) (d07452b) by @B4nan
2.9.7 (2024-10-14)
2.9.5 (2024-08-19)
- Add Actor Standby types (#569) (d3ba82b) by @jirimoravcik
- Allow
unwind
param toDatasetClient.listItems()
to be an array (#576) (7ef3b14) by @fnesveda - (client) Add maxCostPerRun param (#578) (854e776) by @stetizu1
2.9.4 (2024-06-26)
- Add smartlook token to docs build [internal] (#540) (4b408bd) by @HonzaTuron
- Add notifications field to Schedule (#545) (c6f9429) by @m-murasovs
- JavaScript client code examples for platform (#548) (bac3692) by @HonzaTuron
- Added data property to API error object (#559) (3b7c4b7) by @gippy
2.9.2 (2024-02-23)
- Add monthlyUsage() and limits() endpoints to UserClients (#517) (2767c8d) by @tobice
- Parse monthlyUsage.dailyServiceUsages[].date as Date (#519) (980d958) by @tobice
2.9.1 (2024-02-20)
2.9.0 (2024-02-16)
- Add
KeyValueStore.recordExists()
method (#510, closes #507) (069d620) by @barjin - Add log() method to BuildClient (#509) (8821df6) by @tobice
- Add
runs()
andbuilds()
top level endpoints (#468, closes #296) (252d2ac) by @foxt451
- Publish browser bundle (#506) (01f9fe1) by @B4nan
- Update axios to v1.6 (#505, closes #501) (6567e0c) by @B4nan
2.8.6 (2024-02-02)
- (request-queue) Limit payload size for batchAddRequests() (#489, closes #446) (164c9c7) by @drobnikj
- Replace ReadableStream with Readable (#463, closes #240) (6f6deb7) by @foxt451
- Add missing properties to
ActorCollectionCreateOptions
type (#486) (623b516) by @jirimoravcik
2.8.4 (2023-11-20)
2.8.2 (2023-10-30)
- Add how to install javascript Apify client (#440) (b79e463) by @webrdaniel
- (types)
ScheduleCreateOrUpdateData
should haveid
as optional (#276) (e59ff10) by @magne4000
2.8.1 (2023-10-11)
- Add new webhook fields (#426) (44ced4d) by @m-murasovs
- Add delete to runs and builds (#428) (a399663) by @Jkuzz
2.8.0 (2023-09-08)
- Add Actor reboot method (#408) (be21c82) by @jirimoravcik
2.7.2 (2023-08-28)
- Rename APIFY_ACTOR_MAX_ITEMS to ACTOR_MAX_PAID_DATASET_ITEMS (#353) (36d6c20) by @novotnyj
- (runs, builds) Add usage usd into actor run and build types (#355) (a6be0b8) by @drobnikj
- Add shouldInterpolateStrings field to webhook type (#358) (ad1f204) by @valekjo
- Use Actor/Apify env vars instead of
ENV_VARS
(#373) (b2743f5) by @jirimoravcik - Added StoreCollectionClient class useful for listing Actors in Apify Store (#395) (5ffd98b) by @drobnikj
2.7.1 (2023-04-06)
2.7.0 (2023-03-14)
- Add support for
maxItems
in run options (#330) (9442187) by @novotnyj - (repo) Repository links (#335) (8c30566) by @drobnikj
- (requestQueue) Request queue v2 features release (#334) (bde2ac7) by @drobnikj
- Add "standard" handing for setStatusMessage (#333) (4dab74a) by @barjin
- Fix isAtHome value in User-Agent header (#286) (5232c1b) by @mvolfik
- (actor) Fix types for actor run (#331) (8440f2f) by @drobnikj
- Improve reading of the version when using bundlers (#332, closes #235) (4ac1ba4) by @vladfrangu
2.6.3 (2023-02-14)
- Updating pull request toolkit config [INTERNAL] (3fc9f85) by @mtrunkat
- Updating pull request toolkit config [INTERNAL] (80008ea) by @mtrunkat
- IsStatusMessageTerminal in RunUpdate interface (#306) (e60d9a6) by @barjin
2.6.2 (2023-01-04)
- Re-export useful types and classes (#285, closes #279) (868c420) by @vladfrangu
Actor.call
andTask.call
acceptwaitSecs
notwaitForFinish
(#283, closes #282) (1ce8ed5) by @vladfrangu- (types) Correct extends clause for Dataset entries (#284, closes #267) (cb07c3a) by @vladfrangu
- Correct docs links for actor env vars, some refactoring (#287) (669d7ac) by @jirimoravcik
- Make ActorUpdateOptions type have optional fields (#288) (46a0e4f) by @metalwarrior665
- Correctly set default client headers (#290) (08eeae1) by @valekjo
2.6.1 (2022-10-13)
- Drop single file support (#257) (2b8b3af) by @valekjo
- Update actor types (#263) (cb57822) by @HonzaTuron
- Add flatten param to Dataset items listing (#264) (0c40ea7) by @Strajk
- Add optional title field to task, schedule, key-value store, dataset, and request queue (#271) (46d625b) by @valekjo
- Add tslib dependency (35d633f) by @B4nan
- Add
defaultRequestQueueId
property toActorRun
type (#268) (2a78dde) by @fnesveda
- Add
run.update()
method for setting fields on runs. You can update statusMessage of run using this method. - Fix parsing ApifyError for method called with
forceBuffer
param totrue
. - Fix stream support, related with axios/axios#1045
- Adjust default parallels and retries for batch add requests
- Add methods to list all requests in queue,
requestQueueClient.listRequests()
andrequestQueueClient.paginateRequests()
. The paginate requests returns async iterator, you can use it:
for await (const page of rqClient.paginateRequests()) {
// Do something with page.items
}
- Fix ActorRun - correct type for
status
- Add
origin
param to the last actor/task run endpoints
- Add request queue methods
listAndLockHead()
,prolongRequestLock()
,deleteRequestLock()
- Add exponential backoff to
batchAddRequests
- Add option
minDelayBetweenUnprocessedRequestsRetriesMillis
tobatchAddRequests
- Fix: Retries in batch requests insert endpoint
- Add batch delete requests method for request queue (for now experimental)
- Add support for
schema
parameter in key-value store and datasetgetOrCreate
function - Make
RequestQueueClientListItem.method
strictly typed - Fix: Correct types and validation for
tasks().create
- Fix: Add missing override modifier for error cause
- Fix: Updated incorrect RunCollectionClient list status type
- Added support for the (for now experimental)
view
parameter to the dataset items endpoints - For TypeScript users, the type for the fields that end in
At
(ex.:createdAt
) have been correctly typed as a Date object instead of a string - Add option to set custom timeout for RequestQueue client calls.
- Added
ActorEnvVarCollectionClient
andActorEnvVarClient
- Add export for
PaginatedList
interface. - Experimental: Add method for request queue batch requests insert endpoint
- For TypeScript users, the input type for Task#start and Task#call is now correctly marked as optional
- For TypeScript users, the input type for Task#start and Task#call have been corrected (these methods expect an optional input of an object, not an object or an array of objects)
- For TypeScript users, the overloads for KeyValueStore#getRecord have been relaxed and their order has been corrected
- For TypeScript users, the
WebhookEventType
type was corrected to represent its correct value.
- Fix: Unnamed storages can now be created again.
- For TypeScript users, DatasetClients can now take in a generic type parameter that defines the data present in a dataset.
- BREAKING CHANGE: Remove default export for ApifyClient. Migration from v1.x.x to v2.x.x:
// v2.x.x
const { ApifyClient } = require('apify-client');
// v1.x.x
const ApifyClient = require('apify-client');
- TypeScript rewrite
- Fix: Exposing token on ApifyClient instance
- Changed passing token using request HTTP header instead of the request parameter for every API calls
- Fix: Exposing token on ApifyClient instance
- Changed passing token using request HTTP header instead of the request parameter for every API calls
- Added new method
.test()
to theWebhookClient
class
- Added gracefully parameter for abort run function
- Enabled access to actor run storages via
RunClient
- use new
apify-shared
packages to reduce bundle size
- Fixed invalid max body length setting thanks to a transitive default in
axios
.
- Fixed double stringification of JSON inputs in
.start()
,.call()
and.metamorph()
functions.
- Added missing function serialization to
.metamorph()
. See 1.2.0 release.
- Added function serialization to
.start()
and.call()
function inputs. You can now define input functions as JS functions instead of having to type them out as a string. - Added validation for resource IDs to be non-empty. This is non-breaking and prevents cryptic errors like
We have bad news: there is no API endpoint at this URL.
- Fixed slow parsing of large responses due to a bug in
axios
.
- Added timeout, memory, and build parameters to
client.run.resurrect()
- Deprecated
disableRedirect
option forkvs.setRecord()
- Updated
apify-shared
version to resolve a sub-dependency vulnerability.
- Fix requests timing out too early due to socket timeouts.
- Fix
.call
and.waitForFinish
waiting infinitely withwaitSecs: 0
. - Add missing validations to
getOrCreate
methods. - Remove signed URL uploads from
kvs.setRecord()
- Omit query params for direct upload URL calls
- Removed limits on max content/body length.
- Fix
axios
security vulnerability by updating version. - Update
ow
to0.22.0
. - Improve docs.
- Removed the
desc
option fromkeyValueStoreClient.listKeys()
because Apify API does not support it.
- A complete rewrite of the client. See README.
- Removed legacy Apify Crawler methods that are no longer supported by Apify API.
- Added update method for request queues, datasets and key-value stores.
- Added new method
client.acts.resurrectRun()
that resurrects finished (even failed) actor run.
body
andcontentType
parameters ofclient.tasks.runTask()
were deprecated in favor of newinput
parameter.- Added new methods to get and update input of actor task -
client.tasks.getInput()
andclient.tasks.updateInput()
.
- Added
simplified
andskipFailedPages
parameters todatasets.getItems()
to support legacy crawler produced datasets.
- Added parameter retryOnStatusCodes to define an array of HTTP status codes on which
client retries the request. Default value is
[429]
(only rate limit exceeded).
- Fix of non-JSON crawler execution results retrieval
- Added support for clientKey parameter to request queue endpoints.
- Fixed a bug where last retry stats would not be saved correctly.
- Client now retries request in a case of an invalid JSON response (incomplete response payload).
apifyClient.stats.rateLimitErrors
is now anArray
and tracks errors per retry count.
- Added
client.tasks.listWebhooks()
to list task webhooks.
- Added
client.webhooks
providing access to Apify webhooks. - Added
client.webhookDispatches
providing access to Apify webhook dispatches. - Added
client.acts.listWebhooks()
to list actor webhooks.
- Added exponential backoff to
datasets.getItems()
in a case of "Unexpected end of JSON input" error
- Added more details to
ApifyClientError
for easier debugging - Added
client.acts.metamorphRun()
, see documentation for more information.
- Improve
.toString()
message ofApifyClientError
.
- Added
clean
,skipHidden
andskipEmpty
parameters toclient.datasets.getItems()
method.
- Increased number of retries for request queue endpoints that may be slower to scale up.
- Fixed getActorVersion method, now returns
null
if version does not exist
- Added
ApifyClient.stats
object that collects various statistics of the API client
- Methods for Apify storages (i.e. key-value stores, datasets and request queues) that use other than GET HTTP method now require token parameter.
tasks.runTask()
method now allows to overload input and options from actor task configuration.
- All key-value store records with content type
text/*
are now parsed into string. - Option
promise
to customize Promise implementation is not supported any more. - All methods now use native promises instead of Bluebird implementation. Make sure that your code doesn't depend on Bluebird.
- All Boolean parameters of v2 endpoints (Actor, Storages) are now truly Boolean and don't accept
1
astrue
. Legacy crawler API hasn't changed. - Added support for actor versions API.
- Endpoint to get items from dataset now passes
encoding: null
to support XSLX format.
- Added support for actor tasks API.
- Requests repeated more than
expBackOffMaxRepeats/2
times are logged.
- Added
client.tasks
providing access to Apify tasks.
- Upgraded NPM dependencies
- Renamed
ApifyError
toApifyClientError
- Added support for more content types to
utils.parseBody()
.
- Added support for pre-serialized data (strings) to
Apify.datasets.putItems()
.
- Added
executionId
parameter to getCrawlerSettings method. You can get crawler settings for specific execution with that.
- Thrown error in a case of failed request now contains details such as
URL
,method
, ... .
- All date fields (ending with
At
) such asmodifiedAt
,createdAt
, etc. are now parsed toDate
object.
- Added
client.requestQueues
providing access to Apify Request Queue. - RequestQueue / KeyValueStore / Dataset now support
[username]~[store-name]
instead of store ID.
- Added
client.users.getUser()
method that retrieves own accout details including usage and limits.
- WARNING: Method
datasets.getItems()
now returns object PaginationList with items wrapped inside instead of plain items array. This helps to iterate through all the items using pagination. This change is not backward compatible!
- Support for Function type added to
utils.checkParamOrThrow()
- Updated GitHub repo and Travis CI links
- Datasets group addded