Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Grand dispatch queue #661

Open
wants to merge 78 commits into
base: runtime_select_event_loop
Choose a base branch
from
Open
Changes from 69 commits
Commits
Show all changes
78 commits
Select commit Hold shift + click to select a range
9389f14
CMakeLists.txt
sbSteveK Jul 29, 2024
e825924
event_loop.h
sbSteveK Jul 29, 2024
452217c
io.h
sbSteveK Jul 29, 2024
c0d4086
kqueue_event_loop.c
sbSteveK Jul 29, 2024
84fd773
dispatch_queue_event_loop.c
sbSteveK Jul 29, 2024
98c8ef0
event_loop.c
sbSteveK Jul 29, 2024
89e8ece
epoll_event_loop.c
sbSteveK Jul 29, 2024
9c27144
iocp_event_loop.c
sbSteveK Jul 29, 2024
f375bb2
event_loop_test.c
sbSteveK Jul 29, 2024
41a5fa1
try ifdef on TargetConditionals.h and comment failing event loop test
sbSteveK Jul 29, 2024
04c2b93
reorder the event loop config, so apple platform would use dispatch q…
xiazhvera Jul 30, 2024
0d301d2
bring in dispatch queue changes
xiazhvera Sep 3, 2024
4afaea6
clangformat
xiazhvera Sep 3, 2024
a7f6904
remove unused args
xiazhvera Sep 3, 2024
89635db
clean up
xiazhvera Sep 11, 2024
195ca1c
clean up dispatch queue
xiazhvera Sep 11, 2024
24ce65b
Merge branch 'main' of https://github.com/awslabs/aws-c-io into grand…
xiazhvera Sep 11, 2024
287094f
clang-format
xiazhvera Sep 11, 2024
bd58da0
more comments and format clean up
xiazhvera Sep 11, 2024
f0e5dde
quick windows test
xiazhvera Sep 16, 2024
aef1b14
TEST: quick error verification
xiazhvera Sep 16, 2024
41bb257
Revert "TEST: quick error verification"
xiazhvera Sep 16, 2024
22e68b2
TEST: using struct instead of union
xiazhvera Sep 16, 2024
a28cb37
Revert "TEST: using struct instead of union"
xiazhvera Sep 16, 2024
c67e966
add back definition for union
xiazhvera Sep 16, 2024
3ca34ce
WINDOWS: rename function
xiazhvera Sep 16, 2024
f8c26f5
fix compile error
xiazhvera Sep 16, 2024
a428cd8
remove unused finalize functions
xiazhvera Sep 16, 2024
5ab8f24
fix event loop schedule future
xiazhvera Sep 16, 2024
0918e76
improve dispatch caller's thread check
xiazhvera Sep 26, 2024
a55f14f
update caller's thread changes
xiazhvera Sep 26, 2024
06fb206
use lock to protect the thread id info
xiazhvera Sep 26, 2024
ed04764
lint
xiazhvera Sep 26, 2024
e8fe46d
fix thread related test/disable pipe tests
xiazhvera Sep 26, 2024
a84cb5a
AWS_USE_DISPATCH_QUEUE updates (#679)
sbSteveK Oct 3, 2024
977bb8a
Merge branch 'main' into grand_dispatch_queue
xiazhvera Oct 15, 2024
ce07c5a
bring in event loop changes
xiazhvera Oct 15, 2024
0210346
bring in CI changes
xiazhvera Oct 15, 2024
a751d76
Merge branch 'main' into grand_dispatch_queue
xiazhvera Oct 16, 2024
b44c510
update comments
xiazhvera Oct 25, 2024
b0f85f2
remove is_executing check
xiazhvera Oct 25, 2024
7bc39ee
improve comments
xiazhvera Oct 25, 2024
475c1f2
make all private function static
xiazhvera Oct 25, 2024
cf592a7
init variables
xiazhvera Oct 25, 2024
1803c0f
clang-format
xiazhvera Oct 25, 2024
977c4f7
Merge branch 'runtime_select_event_loop' of github.com:awslabs/aws-c-…
xiazhvera Nov 7, 2024
eab14fa
fix header update
xiazhvera Nov 7, 2024
55de953
Merge branch 'runtime_select_event_loop' of github.com:awslabs/aws-c-…
xiazhvera Nov 8, 2024
74019cf
update ci flags
xiazhvera Nov 8, 2024
5d22a69
update setup switch default event loop
xiazhvera Nov 8, 2024
600421e
revert function rename
xiazhvera Nov 8, 2024
f808b35
Merge branch 'runtime_select_event_loop' of github.com:awslabs/aws-c-…
xiazhvera Nov 8, 2024
4a8007a
Merge branch 'runtime_select_event_loop' of github.com:awslabs/aws-c-…
xiazhvera Nov 9, 2024
70008b1
fix implicit function call
xiazhvera Nov 9, 2024
8bd9808
Merge branch 'runtime_select_event_loop' of github.com:awslabs/aws-c-…
xiazhvera Nov 9, 2024
8d946db
temporary remove dispatch queue wrap
xiazhvera Nov 9, 2024
1212977
Merge branch 'runtime_select_event_loop' of github.com:awslabs/aws-c-…
xiazhvera Nov 9, 2024
afd634d
update dispatch queue related flags
xiazhvera Nov 9, 2024
6b92e59
hide dispatch queue header
xiazhvera Nov 11, 2024
94f95c1
Merge branch 'runtime_select_event_loop' of github.com:awslabs/aws-c-…
xiazhvera Nov 12, 2024
22c6363
Merge branch 'runtime_select_event_loop' of github.com:awslabs/aws-c-…
xiazhvera Nov 12, 2024
c507d13
update comments
xiazhvera Nov 12, 2024
c54b99e
rename enum
xiazhvera Nov 12, 2024
69cbb09
lint
xiazhvera Nov 12, 2024
5fc5270
Merge branch 'runtime_select_event_loop' of github.com:awslabs/aws-c-…
xiazhvera Nov 12, 2024
7d20796
Merge branch 'runtime_select_event_loop' of github.com:awslabs/aws-c-…
xiazhvera Nov 12, 2024
691aa17
Merge branch 'runtime_select_event_loop' of github.com:awslabs/aws-c-…
xiazhvera Nov 25, 2024
2461b43
Merge branch 'runtime_select_event_loop' of github.com:awslabs/aws-c-…
xiazhvera Nov 25, 2024
6783915
Grand dispatch queue context (#697)
xiazhvera Nov 29, 2024
85bf6ce
move aws_io_handle
xiazhvera Dec 3, 2024
ef012d3
schedule service entry on dispatch queue resume
xiazhvera Dec 3, 2024
8bb613e
Merge branch 'runtime_select_event_loop' of github.com:awslabs/aws-c-…
xiazhvera Dec 4, 2024
7cdd319
WIP code review update
xiazhvera Dec 5, 2024
d45eb98
wip update cr
xiazhvera Dec 10, 2024
cfa0399
Merge branch 'runtime_select_event_loop' of github.com:awslabs/aws-c-…
xiazhvera Dec 18, 2024
f4414aa
refactor dispatch queue release process/update iteartion schedule pro…
xiazhvera Dec 18, 2024
e1ce086
set up impl_data of event loop ealier
xiazhvera Dec 18, 2024
a93216b
revert ci change
xiazhvera Dec 18, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 0 additions & 1 deletion CMakeLists.txt
Original file line number Diff line number Diff line change
@@ -44,7 +44,6 @@ file(GLOB AWS_IO_TESTING_HEADERS
"include/aws/testing/*.h"
)


file(GLOB AWS_IO_PRIV_HEADERS
"include/aws/io/private/*.h"
)
8 changes: 8 additions & 0 deletions include/aws/io/io.h
Original file line number Diff line number Diff line change
@@ -14,12 +14,20 @@ AWS_PUSH_SANE_WARNING_LEVEL

#define AWS_C_IO_PACKAGE_ID 1

struct aws_io_handle;
xiazhvera marked this conversation as resolved.
Show resolved Hide resolved

typedef void aws_io_set_queue_on_handle_fn(struct aws_io_handle *handle, void *queue);
typedef void aws_io_clear_queue_on_handle_fn(struct aws_io_handle *handle);

struct aws_io_handle {
union {
int fd;
/* on Apple systems, handle is of type nw_connection_t. On Windows, it's a SOCKET handle. */
void *handle;
} data;
void *additional_data;
aws_io_set_queue_on_handle_fn *set_queue;
aws_io_clear_queue_on_handle_fn *clear_queue;
};

enum aws_io_message_type {
10 changes: 10 additions & 0 deletions include/aws/io/private/event_loop_impl.h
Original file line number Diff line number Diff line change
@@ -152,6 +152,16 @@ AWS_IO_API
struct _OVERLAPPED *aws_overlapped_to_windows_overlapped(struct aws_overlapped *overlapped);
#endif /* AWS_ENABLE_IO_COMPLETION_PORTS */

/**
* @internal - Don't use outside of testing.
*
* Return the default event loop type. If the return value is `AWS_ELT_PLATFORM_DEFAULT`, the function failed to
* retrieve the default type value.
* If `aws_event_loop_override_default_type` has been called, return the override default type.
*/
AWS_IO_API
enum aws_event_loop_type aws_event_loop_get_default_type(void);

/**
* Associates an aws_io_handle with the event loop's I/O Completion Port.
*
74 changes: 74 additions & 0 deletions source/darwin/dispatch_queue.h
Original file line number Diff line number Diff line change
@@ -0,0 +1,74 @@
#ifndef AWS_IO_DARWIN_DISPATCH_QUEUE_H
#define AWS_IO_DARWIN_DISPATCH_QUEUE_H
/**
* Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.
* SPDX-License-Identifier: Apache-2.0.
*/

#include <Security/Security.h>
#include <aws/common/mutex.h>
#include <aws/common/thread.h>
#include <aws/io/tls_channel_handler.h>
#include <dispatch/dispatch.h>

struct secure_transport_ctx {
struct aws_tls_ctx ctx;
CFAllocatorRef wrapped_allocator;
CFArrayRef certs;
SecIdentityRef secitem_identity;
CFArrayRef ca_cert;
enum aws_tls_versions minimum_version;
struct aws_string *alpn_list;
bool verify_peer;
};

struct dispatch_scheduling_state {
/**
* Let's us skip processing an iteration task if one is already in the middle of executing
*/
bool will_schedule;
xiazhvera marked this conversation as resolved.
Show resolved Hide resolved

/**
* List<scheduled_service_entry> in sorted order by timestamp
*
* When we go to schedule a new iteration, we check here first to see
* if our scheduling attempt is redundant
*/
struct aws_linked_list scheduled_services;
};

struct dispatch_loop;
struct dispatch_loop_context;

struct dispatch_loop {
struct aws_allocator *allocator;
dispatch_queue_t dispatch_queue;
struct aws_task_scheduler scheduler;
struct aws_linked_list local_cross_thread_tasks;
struct aws_event_loop *base_loop;

/* Apple dispatch queue uses the id string to identify the dispatch queue */
struct aws_string *dispatch_queue_id;
xiazhvera marked this conversation as resolved.
Show resolved Hide resolved

/* Synced data handle cross thread tasks and events, and event loop operations*/
struct {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is this block separate from synced_thread_data? Does the lock in synced_thread_data guard this too? I would not split them like this

struct aws_linked_list cross_thread_tasks;
struct dispatch_loop_context *context;
bool suspended;
} synced_task_data;

/* Synced thread data handles the thread related info. `is_executing` flag and `current_thread_id` together are used
* to identify the executing thread id for dispatch queue. See `static bool s_is_on_callers_thread(struct
* aws_event_loop *event_loop)` for details.
*/
struct {

struct aws_mutex thread_data_lock;
bool is_executing;
aws_thread_id_t current_thread_id;
} synced_thread_data;

bool is_destroying;
};

#endif /* #ifndef AWS_IO_DARWIN_DISPATCH_QUEUE_H */
Loading