Allow differentiating blobstores by buckets #376
Conversation
| @@ -66,6 +66,9 @@ | |||
| "s3proxy.max-single-part-object-size"; | |||
| public static final String PROPERTY_V4_MAX_NON_CHUNKED_REQUEST_SIZE = | |||
| "s3proxy.v4-max-non-chunked-request-size"; | |||
| /** Used to locate blobstores by specified bucket names. */ | |||
| public static final String PROPERTY_BUCKET_LOCATOR = | |||
| "s3proxy.bucket-locator"; | |||
There was a problem hiding this comment.
I am unclear how to use this feature -- can you add some more documentation? Does each properties file have a unique identifier? Is it used for something?
There was a problem hiding this comment.
Expanded the description. The bucket-locator property specifies the bucket names associated with the backend specified in the properties file. Different properties files can specify different buckets, allowing a client to use the same S3Proxy credentials, but have the requests route to different backends.
| localCredential, blobStore)); | ||
| Map.Entry<String, String> key = Maps.immutableEntry( | ||
| localIdentity, null); | ||
| if (!parsedLocations.contains(key)) { |
There was a problem hiding this comment.
You can use the return value from parsedLocations.add.
|
@berlinsaint @jiffysrc there seems to be some overlap between #372, #374, and this PR. Can you guys sort out which approach is the best? |
timuralp
force-pushed
the
feature/bucket-locator
branch
from
917fc68
to
3fce2d3
Compare
Allows differentiating the backend blobstores by configured buckets. The commit allows using the same s3proxy credentials across the different backends and relies on the bucket names to disambiguate them. The "default" blobstore is the one that occurs first. Fixes: gaul#371
timuralp
force-pushed
the
feature/bucket-locator
branch
from
3fce2d3
to
9be15e1
Compare
| localCredential, blobStore)); | ||
| } | ||
| } | ||
| if (!buckets.isEmpty()) { |
There was a problem hiding this comment.
Do you need to check whether buckets is empty? The foreach loop should handle this already.
|
Tested this with two local filesystem configs and verified that objects go where they should. Also tried some duplicate buckets and the error message is comprehensible. |
|
Superseded by #378. |
Allows differentiating the backend blobstores by configured buckets.
The commit allows using the same s3proxy credentials across the
different backends and relies on the bucket names to disambiguate them.
The "default" blobstore is the one that occurs first.
Fixes: #371