-
Notifications
You must be signed in to change notification settings - Fork 206
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Update evaporate.js #401
base: master
Are you sure you want to change the base?
Update evaporate.js #401
Conversation
In our project, we faced with the possibility to upload several identical files to one s3 bucket to the separate subfolders. This cause the s3 upload error: was uploaded the only first file from the list of identical files. This patch for the fix parallel uploads of identical files.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The available option allowS3ExistenceOptimization
might work for your needs, but then you would not benefit from that optimization. Refer to the wiki for more information.
evaporate.js
Outdated
@@ -2066,6 +2066,7 @@ | |||
// The key tries to give a signature to a file in the absence of its path. | |||
// "<filename>-<mimetype>-<modifieddate>-<filesize>" | |||
return [ | |||
fileUpload.name, // fix for parallel uploads of identical files |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Unfortunately, this breaks compatibility and a key feature of EvaporateJS: the ability to retry failed transfers. You can see which specs fail by running the test suite.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Maybe I did something wrong, but in my case when I tried to upload the same file simultaneously to the one bucket to the separated directories. Evaporate's library tried to retry upload that file and I had one result in all tries - the first file upload was done successfully, all others - not. In this process, the library sends me notifications about file uploading progress and the success event. But files didn't upload.
I changed fileUpload.name to the fileUpload.awsKey. Maybe this will be okay for your tests? I tried use allowS3ExistenceOptimization with true and false values without success. |
In our project, we faced with the possibility to upload several identical files to one s3 bucket to the separate subfolders. This cause the s3 upload error: was uploaded the only first file from the list of identical files. This patch for the fix parallel uploads of identical files.