chore(tiering): Faster small bins serialization #2 #3396
Conversation
dranikpg
changed the title
dranikpg
force-pushed
the
tiering-offload-opt-2
branch
from
4b10617 to
0b4a673
Compare
Signed-off-by: Vladislav Oleshko <[email protected]>
| // If tiering is enabled, try saving the received page on disk | ||
| // Fall back to memory in case of errors | ||
| if (EngineShard::tlocal() && EngineShard::tlocal()->tiered_storage()) { | ||
| auto& storage = EngineShard::tlocal()->tiered_storage()->BorrowStorage(); | ||
|
|
||
| util::fb2::Done done; | ||
| std::error_code ec; | ||
| auto cb = [this, offset, &ec, &done](io::Result<tiering::DiskSegment> res) { | ||
| if (res.has_value()) | ||
| small_items_pages_[offset] = res.value(); | ||
| else | ||
| ec = res.error(); | ||
| done.Notify(); | ||
| }; | ||
| ec = storage.Stash(io::Buffer(page), {}, cb); | ||
|
|
||
| done.Wait(); | ||
| if (!ec) | ||
| return {}; | ||
| } | ||
|
|
||
| small_items_pages_[offset] = page; | ||
| return {}; | ||
| } |
There was a problem hiding this comment.
I really wish to use "virtual" keys to store pages. This would not require having custom code for their serialization, as well custom code for loading and stashing them. We could simply rely on the tiered storage for doing this
There was a problem hiding this comment.
Second, a missing optimization is that we don't re-use the offloaded page, we only use it to read from it
There was a problem hiding this comment.
The idea would be to create a mapping table between new and old offsets and re-create those offsets in small bins
There was a problem hiding this comment.
you can add TODOs to detail your ideas..
Regarding virtual keys, I think it's fine to use a predefined prefix like __df__ for dragonfly specific keys if it helps you. one thing to note - how it works with multiple databases. pages are shared between entities from differrent databases,imho
There was a problem hiding this comment.
if it helps you
I want to use the existing code for serialization and offloading, so pages will be "just values" and managed automatically
|
All in all I don't like my code, but not sure where we can simplify or cut corners |
src/server/rdb_load.cc
Outdated
| @@ -481,6 +484,10 @@ void RdbLoaderBase::OpaqueObjLoader::operator()(const RdbSBF& src) { | |||
| pv_->SetSBF(sbf); | |||
| } | |||
|
|
|||
| void RdbLoaderBase::OpaqueObjLoader::operator()(const RdbTieredSegment& src) { | |||
| CHECK(false) << "unreachable"; | |||
| // If tiering is enabled, try saving the received page on disk | ||
| // Fall back to memory in case of errors | ||
| if (EngineShard::tlocal() && EngineShard::tlocal()->tiered_storage()) { | ||
| auto& storage = EngineShard::tlocal()->tiered_storage()->BorrowStorage(); | ||
|
|
||
| util::fb2::Done done; | ||
| std::error_code ec; | ||
| auto cb = [this, offset, &ec, &done](io::Result<tiering::DiskSegment> res) { | ||
| if (res.has_value()) | ||
| small_items_pages_[offset] = res.value(); | ||
| else | ||
| ec = res.error(); | ||
| done.Notify(); | ||
| }; | ||
| ec = storage.Stash(io::Buffer(page), {}, cb); | ||
|
|
||
| done.Wait(); | ||
| if (!ec) | ||
| return {}; | ||
| } | ||
|
|
||
| small_items_pages_[offset] = page; | ||
| return {}; | ||
| } |
There was a problem hiding this comment.
you can add TODOs to detail your ideas..
Regarding virtual keys, I think it's fine to use a predefined prefix like __df__ for dragonfly specific keys if it helps you. one thing to note - how it works with multiple databases. pages are shared between entities from differrent databases,imho
dranikpg
changed the title
Faster small bins serialization by extending the rdb format with RDB_TYPE_TIERED_SEGMENT and RDB_OPCODE_TIERED_PAGE