Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adds Automatic Support to Not Use Multipart Uploads If File Size <= partSize #387

Open
wants to merge 8 commits into
base: master
Choose a base branch
from

Conversation

bikeath1337
Copy link
Collaborator

Thanks to @jakubzitny for #362.

This PR has not be tested yet, but specs pass (I will remove the WIP after doing more testing).

What this intends to do: Enhance Evaporate to avoid the overhead of using multipart uploads if the file size is equal to the part size. This option avoids two round trips to AWS (initiate and complete).

Description:

  • The feature is enabled by default. To disable, use the provided option enablePartSizeOptimization
  • The started, progress and complete callbacks should all be invoked
  • computeContentMd5 should be respected
  • Retry features should be maintained

Changes:

  • The complete callback's xhr parameter will be undefined.

@jakubzitny
Copy link
Collaborator

jakubzitny commented Nov 13, 2017

Hey @bikeath1337, so I found some time to try this out and plugged this branch into our code. However I am getting SignatureDoesNotMatch errors. Not sure if there are some differences in auth when using Multipart or this simple PUT API. We're using the customAuthMethod for signing requests.

I'll try a simpler example to see if that works.

EDIT: Yeah I seem to be getting the same for the simple example. Let me dig into it some other time again 😛

@bikeath1337
Copy link
Collaborator Author

Hi, @jakubzitny , I'm back from being away... :) Any update on this?

@fer90
Copy link

fer90 commented Oct 31, 2024

However I am getting SignatureDoesNotMatch errors

Hey there! I know this is SUPER old, but for anyone trying to do the same thing => I applied this changes to the evaporate lib in my project in order to improve performance a little bit and, hopefully, get rid of a sporadic bug in an already deprecated ( 😞 ) MinIO gateway, and I figured the SignatureDoesNotMatch error arises because the string to sign is malformed.

The string to sign generator uses this.request.path which is undefined in the new PutObject. So, it literally concats an undefined at the end of the string sent to the custom signing endpoint. You may define a path: '' in PutObject to fix it quickly

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants