Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make get_or_set_index immune to memory address reuse #5

Open
wants to merge 1 commit into
base: for_pull_requests
Choose a base branch
from

Conversation

CatboxParadox
Copy link

@CatboxParadox CatboxParadox commented Jul 27, 2021

If a mutex instance gets used, deleted and then recreated in the same thread - especially on the stack, the new instance might get the same memory address as the old instance. Because of this, a lookup in thread_local_index_hashmap can't be based on the instance address (this), but rather on the (heap allocated, shared) shared_locks_array instance address, because that one will not be reused even after the mutex instance is deleted since all unregister_t structs still hold a std::shared_ptr to it until they get erased from thread_local_index_hashmap.

I only ran the benchmark from object_threadsafe/benchmark, hope that's fine, I would expect the fix to have no impact on performance:

Before the change:

CPU Cores: 8
Benchmark thread-safe associative containers with size = 100000
Threads = 8, iterations per thread = 2000000
Time & MOps - steady_clock is_steady = 1, num/den = 1 / 1000000000
Latency     - high_resolution_clock is_steady = 0, num/den = 1 / 1000000000
burn_cpu() for each multithread operations took: 3.26855 nano-sec 

Filling of containers... filled containers.
0        % of write operations (1/3 insert, 1/3 delete, 1/3 update) 
                                        (1 Operation latency, usec)
                     time, sec   MOps    Median  Min     Max 
std::map - 1 thread:    9.02    1.77
std::map & std::mutex:  20.8    0.77
safe_ptr<map,mutex>:    20.4    0.785
safe_ptr<map,shared>:   2.29    6.99
safe_ptr<map,contfree>: 1.3     12.3
safe<map,contf>rowlock: 1.35    11.8
safe part<mutex>:       3.83    4.18
safe part<contfree>:    1.51    10.6
15       % of write operations (1/3 insert, 1/3 delete, 1/3 update) 
                                        (1 Operation latency, usec)
                     time, sec   MOps    Median  Min     Max 
std::map - 1 thread:    5.35    2.99
std::map & std::mutex:  14.4    1.11
safe_ptr<map,mutex>:    17      0.944
safe_ptr<map,shared>:   29.5    0.543
safe_ptr<map,contfree>: 3.42    4.68
safe<map,contf>rowlock: 2.68    5.96
safe part<mutex>:       3.22    4.97
safe part<contfree>:    1.59    10
30       % of write operations (1/3 insert, 1/3 delete, 1/3 update) 
                                        (1 Operation latency, usec)
                     time, sec   MOps    Median  Min     Max 
std::map - 1 thread:    4.48    3.58
std::map & std::mutex:  15.4    1.04
safe_ptr<map,mutex>:    14.2    1.13
safe_ptr<map,shared>:   35.6    0.449
safe_ptr<map,contfree>: 4.83    3.32
safe<map,contf>rowlock: 3.78    4.24
safe part<mutex>:       3.7     4.32
safe part<contfree>:    1.74    9.21
45       % of write operations (1/3 insert, 1/3 delete, 1/3 update) 
                                        (1 Operation latency, usec)
                     time, sec   MOps    Median  Min     Max 
std::map - 1 thread:    4.57    3.5
std::map & std::mutex:  14.7    1.09
safe_ptr<map,mutex>:    13.6    1.18
safe_ptr<map,shared>:   35.6    0.45
safe_ptr<map,contfree>: 6.27    2.55
safe<map,contf>rowlock: 5.03    3.18
safe part<mutex>:       3.83    4.18
safe part<contfree>:    1.98    8.08
60       % of write operations (1/3 insert, 1/3 delete, 1/3 update) 
                                        (1 Operation latency, usec)
                                        (1 Operation latency, usec)
                     time, sec   MOps    Median  Min     Max 
std::map - 1 thread:    5.09    3.14
std::map & std::mutex:  15.1    1.06
safe_ptr<map,mutex>:    15      1.07
safe_ptr<map,shared>:   33.9    0.472
safe_ptr<map,contfree>: 7.97    2.01
safe<map,contf>rowlock: 6.53    2.45
safe part<mutex>:       3.97    4.03
safe part<contfree>:    2.17    7.36
75       % of write operations (1/3 insert, 1/3 delete, 1/3 update) 
                                        (1 Operation latency, usec)
                     time, sec   MOps    Median  Min     Max 
std::map - 1 thread:    4.66    3.43
std::map & std::mutex:  14.9    1.07
safe_ptr<map,mutex>:    14.8    1.08
safe_ptr<map,shared>:   30      0.533
safe_ptr<map,contfree>: 8.79    1.82
safe<map,contf>rowlock: 7.6     2.1
safe part<mutex>:       4.11    3.89
safe part<contfree>:    2.36    6.79
90       % of write operations (1/3 insert, 1/3 delete, 1/3 update) 
                                        (1 Operation latency, usec)
                     time, sec   MOps    Median  Min     Max 
std::map - 1 thread:    4.77    3.36
std::map & std::mutex:  15.2    1.05
safe_ptr<map,mutex>:    15.4    1.04
safe_ptr<map,shared>:   23.5    0.68
safe_ptr<map,contfree>: 9.7     1.65
safe<map,contf>rowlock: 8.96    1.79
safe part<mutex>:       4.33    3.7
safe part<contfree>:    2.68    5.97
end

After the change:

CPU Cores: 8
Benchmark thread-safe associative containers with size = 100000
Threads = 8, iterations per thread = 2000000
Time & MOps - steady_clock is_steady = 1, num/den = 1 / 1000000000
Latency     - high_resolution_clock is_steady = 0, num/den = 1 / 1000000000
burn_cpu() for each multithread operations took: 3.89929 nano-sec
Filling of containers... filled containers.
0        % of write operations (1/3 insert, 1/3 delete, 1/3 update)
                                        (1 Operation latency, usec)
                     time, sec   MOps    Median  Min     Max
std::map - 1 thread:    9.13    1.75
std::map & std::mutex:  21.6    0.739
safe_ptr<map,mutex>:    21      0.76
safe_ptr<map,shared>:   2.25    7.12
safe_ptr<map,contfree>: 1.51    10.6
safe<map,contf>rowlock: 1.47    10.9
safe part<mutex>:       4.11    3.9
safe part<contfree>:    1.72    9.28
15       % of write operations (1/3 insert, 1/3 delete, 1/3 update)
                                        (1 Operation latency, usec)
                     time, sec   MOps    Median  Min     Max
std::map - 1 thread:    5.28    3.03
std::map & std::mutex:  15.4    1.04
safe_ptr<map,mutex>:    18.5    0.867
safe_ptr<map,shared>:   28.8    0.556
safe_ptr<map,contfree>: 3.95    4.05
safe<map,contf>rowlock: 2.74    5.85
safe part<mutex>:       3.41    4.7
safe part<contfree>:    1.76    9.07
30       % of write operations (1/3 insert, 1/3 delete, 1/3 update)
                                        (1 Operation latency, usec)
                     time, sec   MOps    Median  Min     Max
std::map - 1 thread:    4.45    3.59
std::map & std::mutex:  15.5    1.03
safe_ptr<map,mutex>:    15.3    1.05
safe_ptr<map,shared>:   34.7    0.461
safe_ptr<map,contfree>: 5.13    3.12
safe<map,contf>rowlock: 4.07    3.94
safe part<mutex>:       3.85    4.16
safe part<contfree>:    1.83    8.76
45       % of write operations (1/3 insert, 1/3 delete, 1/3 update)
                                        (1 Operation latency, usec)
                     time, sec   MOps    Median  Min     Max
std::map - 1 thread:    4.67    3.43
std::map & std::mutex:  15.4    1.04
safe_ptr<map,mutex>:    15      1.07
safe_ptr<map,shared>:   35.3    0.454
safe_ptr<map,contfree>: 6.41    2.5
safe<map,contf>rowlock: 5.25    3.05
safe part<mutex>:       4.01    3.99
safe part<contfree>:    2.03    7.88
60       % of write operations (1/3 insert, 1/3 delete, 1/3 update)
                                        (1 Operation latency, usec)
std::map - 1 thread:    4.83    3.32
std::map & std::mutex:  15.6    1.03
safe_ptr<map,mutex>:    15.3    1.04
safe_ptr<map,shared>:   32.7    0.489
safe_ptr<map,contfree>: 7.74    2.07
safe<map,contf>rowlock: 6.36    2.52
safe part<mutex>:       4.04    3.96
safe part<contfree>:    2.69    5.94
75       % of write operations (1/3 insert, 1/3 delete, 1/3 update) 
                                        (1 Operation latency, usec)
                     time, sec   MOps    Median  Min     Max 
std::map - 1 thread:    4.89    3.27
std::map & std::mutex:  15.3    1.05
safe_ptr<map,mutex>:    15.4    1.04
safe_ptr<map,shared>:   29.8    0.538
safe_ptr<map,contfree>: 8.79    1.82
safe<map,contf>rowlock: 7.65    2.09
safe part<mutex>:       4.26    3.75
safe part<contfree>:    2.45    6.54
90       % of write operations (1/3 insert, 1/3 delete, 1/3 update) 
                                        (1 Operation latency, usec)
                     time, sec   MOps    Median  Min     Max 
std::map - 1 thread:    4.73    3.38
std::map & std::mutex:  15.7    1.02
safe_ptr<map,mutex>:    15.9    1.01
safe_ptr<map,shared>:   23.7    0.674
safe_ptr<map,contfree>: 9.5     1.68
safe<map,contf>rowlock: 8.77    1.83
safe part<mutex>:       4.48    3.58
safe part<contfree>:    2.64    6.07
end

@CatboxParadox CatboxParadox force-pushed the fix/contfree-mtx-recycled-this branch from f6df557 to 96ed683 Compare July 27, 2021 11:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant