You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I want to copy 8k files to an external FS. This is right now pretty unuseable with Flysystem as the interface only allows one copy at once. So I cannot take any advantage of the real adapter, like async copying at once and so on.
My suggestion so far would be to have a interface: BatchFilesystemOperator which introduces new methods like:
writeBatch and takes a list of files and the adapter is able to write these files in a automatic.
when the adapter does not have this interface, we can just call the regular write in a loop.
Feature Request
Scenario / Use-case
I want to copy 8k files to an external FS. This is right now pretty unuseable with Flysystem as the interface only allows one copy at once. So I cannot take any advantage of the real adapter, like async copying at once and so on.
My suggestion so far would be to have a interface: BatchFilesystemOperator which introduces new methods like:
writeBatch
and takes a list of files and the adapter is able to write these files in a automatic.when the adapter does not have this interface, we can just call the regular write in a loop.
In the example AsyncAWS s3 adapter, we can implement this method and make use of async requests to make this faster for this adapter.
The text was updated successfully, but these errors were encountered: