-
Notifications
You must be signed in to change notification settings - Fork 41
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
S3 Endpoint Resolver #300
Merged
Merged
S3 Endpoint Resolver #300
Changes from 17 commits
Commits
Show all changes
41 commits
Select commit
Hold shift + click to select a range
346038d
wip
waahm7 a58567a
initial wip
waahm7 f5d2ce0
first working test
waahm7 bb2b3c4
update names
waahm7 794e0a3
add cmake flag
waahm7 68ef762
update cmake config
waahm7 01007bf
add the include folder
waahm7 cccf470
remove unused length
waahm7 929d4a7
Shuffle code around and have a compile time flag
waahm7 d041c20
Add docs and error handling
waahm7 b12ba58
Update CI
waahm7 cfaace0
clean up
waahm7 385e09f
cleanup structs
waahm7 c8dcecf
remove from header cheacker for now
waahm7 fe8332e
Add generated comment
waahm7 950c7bb
clean up
waahm7 1741966
remove define from c files
waahm7 70a42aa
PR feedback
waahm7 b770a1b
Merge branch 'main' into s3-endpoint-generate
waahm7 0c959b1
update generated files with latest script
waahm7 e4fd685
fix clang
waahm7 f0048ac
add script
waahm7 a6b4dbe
add test modify lines
waahm7 846337e
cleanup and remove resolve endpoint function as well
waahm7 3b62109
update names and script
waahm7 45f196c
update script name
waahm7 69a6337
update script
waahm7 82cb6f1
lint fix
waahm7 cd06f6a
fix script
waahm7 a616a20
update bucket name
waahm7 054758b
Update to join
waahm7 fe78015
Fix O(n) complexity
waahm7 afe543e
move static to beginning
waahm7 fcfe1ec
Update tests/s3_endpoint_resolver_tests.c
waahm7 705f87c
Remove simple test
waahm7 7d7f7b2
Use dict to escape
waahm7 8a236ff
Update cmake flag
waahm7 c7afe7c
Add the flag in test
waahm7 bb5edb4
Add 2 more tests
waahm7 e92aff5
Update endpoint artifacts script (#306)
waahm7 49a3912
Merge branch 'main' into s3-endpoint-generate
waahm7 File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
49 changes: 49 additions & 0 deletions
49
include/aws/s3/s3_endpoint_resolver/s3_endpoint_resolver.h
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,49 @@ | ||
#ifndef AWS_S3_ENDPOINT_RESOLVER_H | ||
waahm7 marked this conversation as resolved.
Show resolved
Hide resolved
|
||
#define AWS_S3_ENDPOINT_RESOLVER_H | ||
|
||
/** | ||
* Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. | ||
* SPDX-License-Identifier: Apache-2.0. | ||
*/ | ||
|
||
#include <aws/s3/s3.h> | ||
#include <aws/sdkutils/endpoints_rule_engine.h> | ||
waahm7 marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
||
AWS_PUSH_SANE_WARNING_LEVEL | ||
extern const char aws_s3_endpoint_resolver_partitions[]; | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. do we need it customer header? or can we have it in private header? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Thanks, fixed. |
||
extern const char aws_s3_endpoint_rule_set[]; | ||
waahm7 marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
||
AWS_EXTERN_C_BEGIN | ||
|
||
/** | ||
* Creates a new S3 endpoint resolver | ||
*/ | ||
AWS_S3_API | ||
struct aws_s3_endpoint_resolver *aws_s3_endpoint_resolver_new(struct aws_allocator *allocator); | ||
waahm7 marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
||
/** | ||
* Increments the reference count on the aws_s3_endpoint_resolver, allowing the caller to take a reference to it. | ||
* Returns the same aws_s3_endpoint_resolver passed in. | ||
*/ | ||
AWS_S3_API | ||
struct aws_s3_endpoint_resolver *aws_s3_endpoint_resolver_acquire(struct aws_s3_endpoint_resolver *endpoint_resolver); | ||
|
||
/** | ||
* Decrements a aws_s3_endpoint_resolver's ref count. When the ref count drops to zero, the resolver will be destroyed. | ||
* Returns NULL. | ||
*/ | ||
AWS_S3_API | ||
struct aws_s3_endpoint_resolver *aws_s3_endpoint_resolver_release(struct aws_s3_endpoint_resolver *endpoint_resolver); | ||
|
||
/* | ||
* Resolve an s3 endpoint given s3 request context. | ||
* Resolved endpoint is ref counter and caller is responsible for releasing it. | ||
*/ | ||
AWS_S3_API | ||
struct aws_endpoints_resolved_endpoint *aws_s3_endpoint_resolver_resolve_endpoint( | ||
struct aws_s3_endpoint_resolver *endpoint_resolver, | ||
struct aws_endpoints_request_context *request_context); | ||
|
||
AWS_EXTERN_C_END | ||
AWS_POP_SANE_WARNING_LEVEL | ||
#endif /* AWS_S3_ENDPOINT_RESOLVER_H */ |
102 changes: 102 additions & 0 deletions
102
source/s3_endpoint_resolver/aws_s3_endpoint_resolver_partition.c
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,102 @@ | ||
/** | ||
* Copyright Amazon.com, Inc. or its affiliates. | ||
* All Rights Reserved. SPDX-License-Identifier: Apache-2.0. | ||
*/ | ||
|
||
#include <aws/s3/s3_endpoint_resolver/s3_endpoint_resolver.h> | ||
|
||
/** | ||
* This file is generated using a Python script. | ||
waahm7 marked this conversation as resolved.
Show resolved
Hide resolved
|
||
* Do not modify directly. | ||
*/ | ||
const char aws_s3_endpoint_resolver_partitions[] = { | ||
'{', '"', 'v', 'e', 'r', 's', 'i', 'o', 'n', '"', ':', '"', '1', '.', '1', '"', ',', '"', 'p', | ||
'a', 'r', 't', 'i', 't', 'i', 'o', 'n', 's', '"', ':', '[', '{', '"', 'i', 'd', '"', ':', '"', | ||
'a', 'w', 's', '"', ',', '"', 'r', 'e', 'g', 'i', 'o', 'n', 'R', 'e', 'g', 'e', 'x', '"', ':', | ||
'"', '^', '(', 'u', 's', '|', 'e', 'u', '|', 'a', 'p', '|', 's', 'a', '|', 'c', 'a', '|', 'm', | ||
'e', '|', 'a', 'f', ')', '-', '\\', '\\', 'w', '+', '-', '\\', '\\', 'd', '+', '$', '"', ',', '"', | ||
'r', 'e', 'g', 'i', 'o', 'n', 's', '"', ':', '{', '"', 'a', 'f', '-', 's', 'o', 'u', 't', 'h', | ||
'-', '1', '"', ':', '{', '}', ',', '"', 'a', 'p', '-', 'e', 'a', 's', 't', '-', '1', '"', ':', | ||
'{', '}', ',', '"', 'a', 'p', '-', 'n', 'o', 'r', 't', 'h', 'e', 'a', 's', 't', '-', '1', '"', | ||
':', '{', '}', ',', '"', 'a', 'p', '-', 'n', 'o', 'r', 't', 'h', 'e', 'a', 's', 't', '-', '2', | ||
'"', ':', '{', '}', ',', '"', 'a', 'p', '-', 'n', 'o', 'r', 't', 'h', 'e', 'a', 's', 't', '-', | ||
'3', '"', ':', '{', '}', ',', '"', 'a', 'p', '-', 's', 'o', 'u', 't', 'h', '-', '1', '"', ':', | ||
'{', '}', ',', '"', 'a', 'p', '-', 's', 'o', 'u', 't', 'h', 'e', 'a', 's', 't', '-', '1', '"', | ||
':', '{', '}', ',', '"', 'a', 'p', '-', 's', 'o', 'u', 't', 'h', 'e', 'a', 's', 't', '-', '2', | ||
'"', ':', '{', '}', ',', '"', 'a', 'p', '-', 's', 'o', 'u', 't', 'h', 'e', 'a', 's', 't', '-', | ||
'3', '"', ':', '{', '}', ',', '"', 'c', 'a', '-', 'c', 'e', 'n', 't', 'r', 'a', 'l', '-', '1', | ||
'"', ':', '{', '}', ',', '"', 'e', 'u', '-', 'c', 'e', 'n', 't', 'r', 'a', 'l', '-', '1', '"', | ||
':', '{', '}', ',', '"', 'e', 'u', '-', 'n', 'o', 'r', 't', 'h', '-', '1', '"', ':', '{', '}', | ||
',', '"', 'e', 'u', '-', 's', 'o', 'u', 't', 'h', '-', '1', '"', ':', '{', '}', ',', '"', 'e', | ||
'u', '-', 'w', 'e', 's', 't', '-', '1', '"', ':', '{', '}', ',', '"', 'e', 'u', '-', 'w', 'e', | ||
's', 't', '-', '2', '"', ':', '{', '}', ',', '"', 'e', 'u', '-', 'w', 'e', 's', 't', '-', '3', | ||
'"', ':', '{', '}', ',', '"', 'm', 'e', '-', 'c', 'e', 'n', 't', 'r', 'a', 'l', '-', '1', '"', | ||
':', '{', '}', ',', '"', 'm', 'e', '-', 's', 'o', 'u', 't', 'h', '-', '1', '"', ':', '{', '}', | ||
',', '"', 's', 'a', '-', 'e', 'a', 's', 't', '-', '1', '"', ':', '{', '}', ',', '"', 'u', 's', | ||
'-', 'e', 'a', 's', 't', '-', '1', '"', ':', '{', '}', ',', '"', 'u', 's', '-', 'e', 'a', 's', | ||
't', '-', '2', '"', ':', '{', '}', ',', '"', 'u', 's', '-', 'w', 'e', 's', 't', '-', '1', '"', | ||
':', '{', '}', ',', '"', 'u', 's', '-', 'w', 'e', 's', 't', '-', '2', '"', ':', '{', '}', ',', | ||
'"', 'a', 'w', 's', '-', 'g', 'l', 'o', 'b', 'a', 'l', '"', ':', '{', '}', '}', ',', '"', 'o', | ||
'u', 't', 'p', 'u', 't', 's', '"', ':', '{', '"', 'n', 'a', 'm', 'e', '"', ':', '"', 'a', 'w', | ||
's', '"', ',', '"', 'd', 'n', 's', 'S', 'u', 'f', 'f', 'i', 'x', '"', ':', '"', 'a', 'm', 'a', | ||
'z', 'o', 'n', 'a', 'w', 's', '.', 'c', 'o', 'm', '"', ',', '"', 'd', 'u', 'a', 'l', 'S', 't', | ||
'a', 'c', 'k', 'D', 'n', 's', 'S', 'u', 'f', 'f', 'i', 'x', '"', ':', '"', 'a', 'p', 'i', '.', | ||
'a', 'w', 's', '"', ',', '"', 's', 'u', 'p', 'p', 'o', 'r', 't', 's', 'F', 'I', 'P', 'S', '"', | ||
':', 't', 'r', 'u', 'e', ',', '"', 's', 'u', 'p', 'p', 'o', 'r', 't', 's', 'D', 'u', 'a', 'l', | ||
'S', 't', 'a', 'c', 'k', '"', ':', 't', 'r', 'u', 'e', '}', '}', ',', '{', '"', 'i', 'd', '"', | ||
':', '"', 'a', 'w', 's', '-', 'u', 's', '-', 'g', 'o', 'v', '"', ',', '"', 'r', 'e', 'g', 'i', | ||
'o', 'n', 'R', 'e', 'g', 'e', 'x', '"', ':', '"', '^', 'u', 's', '\\', '\\', '-', 'g', 'o', 'v', | ||
'\\', '\\', '-', '\\', '\\', 'w', '+', '\\', '\\', '-', '\\', '\\', 'd', '+', '$', '"', ',', '"', 'r', | ||
'e', 'g', 'i', 'o', 'n', 's', '"', ':', '{', '"', 'u', 's', '-', 'g', 'o', 'v', '-', 'w', 'e', | ||
's', 't', '-', '1', '"', ':', '{', '}', ',', '"', 'u', 's', '-', 'g', 'o', 'v', '-', 'e', 'a', | ||
's', 't', '-', '1', '"', ':', '{', '}', ',', '"', 'a', 'w', 's', '-', 'u', 's', '-', 'g', 'o', | ||
'v', '-', 'g', 'l', 'o', 'b', 'a', 'l', '"', ':', '{', '}', '}', ',', '"', 'o', 'u', 't', 'p', | ||
'u', 't', 's', '"', ':', '{', '"', 'n', 'a', 'm', 'e', '"', ':', '"', 'a', 'w', 's', '-', 'u', | ||
's', '-', 'g', 'o', 'v', '"', ',', '"', 'd', 'n', 's', 'S', 'u', 'f', 'f', 'i', 'x', '"', ':', | ||
'"', 'a', 'm', 'a', 'z', 'o', 'n', 'a', 'w', 's', '.', 'c', 'o', 'm', '"', ',', '"', 'd', 'u', | ||
'a', 'l', 'S', 't', 'a', 'c', 'k', 'D', 'n', 's', 'S', 'u', 'f', 'f', 'i', 'x', '"', ':', '"', | ||
'a', 'p', 'i', '.', 'a', 'w', 's', '"', ',', '"', 's', 'u', 'p', 'p', 'o', 'r', 't', 's', 'F', | ||
'I', 'P', 'S', '"', ':', 't', 'r', 'u', 'e', ',', '"', 's', 'u', 'p', 'p', 'o', 'r', 't', 's', | ||
'D', 'u', 'a', 'l', 'S', 't', 'a', 'c', 'k', '"', ':', 't', 'r', 'u', 'e', '}', '}', ',', '{', | ||
'"', 'i', 'd', '"', ':', '"', 'a', 'w', 's', '-', 'c', 'n', '"', ',', '"', 'r', 'e', 'g', 'i', | ||
'o', 'n', 'R', 'e', 'g', 'e', 'x', '"', ':', '"', '^', 'c', 'n', '\\', '\\', '-', '\\', '\\', 'w', | ||
'+', '\\', '\\', '-', '\\', '\\', 'd', '+', '$', '"', ',', '"', 'r', 'e', 'g', 'i', 'o', 'n', 's', | ||
'"', ':', '{', '"', 'c', 'n', '-', 'n', 'o', 'r', 't', 'h', '-', '1', '"', ':', '{', '}', ',', | ||
'"', 'c', 'n', '-', 'n', 'o', 'r', 't', 'h', 'w', 'e', 's', 't', '-', '1', '"', ':', '{', '}', | ||
',', '"', 'a', 'w', 's', '-', 'c', 'n', '-', 'g', 'l', 'o', 'b', 'a', 'l', '"', ':', '{', '}', | ||
'}', ',', '"', 'o', 'u', 't', 'p', 'u', 't', 's', '"', ':', '{', '"', 'n', 'a', 'm', 'e', '"', | ||
':', '"', 'a', 'w', 's', '-', 'c', 'n', '"', ',', '"', 'd', 'n', 's', 'S', 'u', 'f', 'f', 'i', | ||
'x', '"', ':', '"', 'a', 'm', 'a', 'z', 'o', 'n', 'a', 'w', 's', '.', 'c', 'o', 'm', '.', 'c', | ||
'n', '"', ',', '"', 'd', 'u', 'a', 'l', 'S', 't', 'a', 'c', 'k', 'D', 'n', 's', 'S', 'u', 'f', | ||
'f', 'i', 'x', '"', ':', '"', 'a', 'p', 'i', '.', 'a', 'm', 'a', 'z', 'o', 'n', 'w', 'e', 'b', | ||
's', 'e', 'r', 'v', 'i', 'c', 'e', 's', '.', 'c', 'o', 'm', '.', 'c', 'n', '"', ',', '"', 's', | ||
'u', 'p', 'p', 'o', 'r', 't', 's', 'F', 'I', 'P', 'S', '"', ':', 't', 'r', 'u', 'e', ',', '"', | ||
's', 'u', 'p', 'p', 'o', 'r', 't', 's', 'D', 'u', 'a', 'l', 'S', 't', 'a', 'c', 'k', '"', ':', | ||
't', 'r', 'u', 'e', '}', '}', ',', '{', '"', 'i', 'd', '"', ':', '"', 'a', 'w', 's', '-', 'i', | ||
's', 'o', '"', ',', '"', 'r', 'e', 'g', 'i', 'o', 'n', 'R', 'e', 'g', 'e', 'x', '"', ':', '"', | ||
'^', 'u', 's', '\\', '\\', '-', 'i', 's', 'o', '\\', '\\', '-', '\\', '\\', 'w', '+', '\\', '\\', '-', | ||
'\\', '\\', 'd', '+', '$', '"', ',', '"', 'o', 'u', 't', 'p', 'u', 't', 's', '"', ':', '{', '"', | ||
'n', 'a', 'm', 'e', '"', ':', '"', 'a', 'w', 's', '-', 'i', 's', 'o', '"', ',', '"', 'd', 'n', | ||
's', 'S', 'u', 'f', 'f', 'i', 'x', '"', ':', '"', 'c', '2', 's', '.', 'i', 'c', '.', 'g', 'o', | ||
'v', '"', ',', '"', 's', 'u', 'p', 'p', 'o', 'r', 't', 's', 'F', 'I', 'P', 'S', '"', ':', 't', | ||
'r', 'u', 'e', ',', '"', 's', 'u', 'p', 'p', 'o', 'r', 't', 's', 'D', 'u', 'a', 'l', 'S', 't', | ||
'a', 'c', 'k', '"', ':', 'f', 'a', 'l', 's', 'e', ',', '"', 'd', 'u', 'a', 'l', 'S', 't', 'a', | ||
'c', 'k', 'D', 'n', 's', 'S', 'u', 'f', 'f', 'i', 'x', '"', ':', '"', 'c', '2', 's', '.', 'i', | ||
'c', '.', 'g', 'o', 'v', '"', '}', ',', '"', 'r', 'e', 'g', 'i', 'o', 'n', 's', '"', ':', '{', | ||
'"', 'u', 's', '-', 'i', 's', 'o', '-', 'e', 'a', 's', 't', '-', '1', '"', ':', '{', '}', ',', | ||
'"', 'u', 's', '-', 'i', 's', 'o', '-', 'w', 'e', 's', 't', '-', '1', '"', ':', '{', '}', ',', | ||
'"', 'a', 'w', 's', '-', 'i', 's', 'o', '-', 'g', 'l', 'o', 'b', 'a', 'l', '"', ':', '{', '}', | ||
'}', '}', ',', '{', '"', 'i', 'd', '"', ':', '"', 'a', 'w', 's', '-', 'i', 's', 'o', '-', 'b', | ||
'"', ',', '"', 'r', 'e', 'g', 'i', 'o', 'n', 'R', 'e', 'g', 'e', 'x', '"', ':', '"', '^', 'u', | ||
's', '\\', '\\', '-', 'i', 's', 'o', 'b', '\\', '\\', '-', '\\', '\\', 'w', '+', '\\', '\\', '-', '\\', | ||
'\\', 'd', '+', '$', '"', ',', '"', 'o', 'u', 't', 'p', 'u', 't', 's', '"', ':', '{', '"', 'n', | ||
'a', 'm', 'e', '"', ':', '"', 'a', 'w', 's', '-', 'i', 's', 'o', '-', 'b', '"', ',', '"', 'd', | ||
'n', 's', 'S', 'u', 'f', 'f', 'i', 'x', '"', ':', '"', 's', 'c', '2', 's', '.', 's', 'g', 'o', | ||
'v', '.', 'g', 'o', 'v', '"', ',', '"', 's', 'u', 'p', 'p', 'o', 'r', 't', 's', 'F', 'I', 'P', | ||
'S', '"', ':', 't', 'r', 'u', 'e', ',', '"', 's', 'u', 'p', 'p', 'o', 'r', 't', 's', 'D', 'u', | ||
'a', 'l', 'S', 't', 'a', 'c', 'k', '"', ':', 'f', 'a', 'l', 's', 'e', ',', '"', 'd', 'u', 'a', | ||
'l', 'S', 't', 'a', 'c', 'k', 'D', 'n', 's', 'S', 'u', 'f', 'f', 'i', 'x', '"', ':', '"', 's', | ||
'c', '2', 's', '.', 's', 'g', 'o', 'v', '.', 'g', 'o', 'v', '"', '}', ',', '"', 'r', 'e', 'g', | ||
'i', 'o', 'n', 's', '"', ':', '{', '"', 'u', 's', '-', 'i', 's', 'o', 'b', '-', 'e', 'a', 's', | ||
't', '-', '1', '"', ':', '{', '}', ',', '"', 'a', 'w', 's', '-', 'i', 's', 'o', '-', 'b', '-', | ||
'g', 'l', 'o', 'b', 'a', 'l', '"', ':', '{', '}', '}', '}', ']', '}', '\0'}; |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't understand how this works, just throw out my question:
So, this will only compile the
source/s3_endpoint_resolver/*.c
when the cmake config is set. What about the header file? Will Cmake not compile the header files as well?There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, it will only include the source when the CMake configuration is set. The header file is always included to simplify things and we warn users not to use it unless it's enabled.