feat: impl put_multipart in object_store
#4793
Conversation
Rachelint
force-pushed
the
impl-put-multipart
branch
from
522dfae
to
9990f5f
Compare
|
@Xuanwo hi, have a moment to review? |
| } | ||
| } | ||
|
|
||
| impl fmt::Debug for OpendalUpload { |
There was a problem hiding this comment.
Debug has been included, so we can remove the usage of fmt. Or we can remove all Debug / Formatter instead.
| @@ -412,6 +417,74 @@ impl ObjectStore for OpendalStore { | |||
| } | |||
| } | |||
|
|
|||
| /// `MultipartUpload`'s impl based on `Writer` in opendal | |||
| struct OpendalUpload { | |||
There was a problem hiding this comment.
How about following the same naming pattern like object_store: OpendalMultiPartUpload?
| // Refer to the fs impl in `object_store`: | ||
| // https://github.com/apache/arrow-rs/blob/a35214f92ad7c3bce19875bb091cb776447aa49e/object_store/src/local.rs#L715 | ||
| // IO exists during locking, so use tokio::sync::Mutex here | ||
| writer: Arc<Mutex<Writer>>, |
There was a problem hiding this comment.
Sorry for not take deep research on this API first.
Writer handles the write internally, all data must be written in order. Using a lock here is incorrect. Writer needs to make some changes to make it's possible to implement MultipartUpload.
Let me take a look first.
There was a problem hiding this comment.
Hi, @Rachelint. I'm currently working on a fix for this; let's mark this PR as a draft for now. Thanks for your patience.
There was a problem hiding this comment.
Hi, @Rachelint. I'm currently working on a fix for this; let's mark this PR as a draft for now. Thanks for your patience.
Got it. Somethings I can help to fix? I am willing to do more and understand the project better.
There was a problem hiding this comment.
Got it. Somethings I can help to fix? I am willing to do more and understand the project better.
Hi, I'm still working on how to address this. Would you like to look at other issues first? Thanks for your understanding!
Rachelint
force-pushed
the
impl-put-multipart
branch
2 times, most recently
from
819fce4
to
d4391e3
Compare
Rachelint
force-pushed
the
impl-put-multipart
branch
from
d4391e3
to
124788e
Compare
close: #4748