Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[gdrive] potential regression of folder name handling #4722

Open
imWildCat opened this issue Jun 12, 2024 · 14 comments
Open

[gdrive] potential regression of folder name handling #4722

imWildCat opened this issue Jun 12, 2024 · 14 comments

Comments

@imWildCat
Copy link
Contributor

in my app, I found folders would have extra / for google drive

image

might be a regression recently

@imWildCat
Copy link
Contributor Author

20:18:04 [WARN] service=gdrive operation=stat path=apps/infoflow/default.ifvault/v1/data/48aaec2f-ed21-4f37-a3ca-90f9ac5d6872 -> NotFound (permanent) at stat, context: { service: gdrive, path: apps/infoflow/default.ifvault/v1/data/48aaec2f-ed21-4f37-a3ca-90f9ac5d6872 } => path not found: apps/infoflow/default.ifvault/v1/data/48aaec2f-ed21-4f37-a3ca-90f9ac5d6872
20:18:06 [WARN] service=gdrive operation=stat path=apps/infoflow/default.ifvault/v1/data/1dff98d4-a8c6-4b08-a9e3-78e45d4aa034 -> NotFound (permanent) at stat, context: { service: gdrive, path: apps/infoflow/default.ifvault/v1/data/1dff98d4-a8c6-4b08-a9e3-78e45d4aa034 } => path not found: apps/infoflow/default.ifvault/v1/data/1dff98d4-a8c6-4b08-a9e3-78e45d4aa034

yes. it is a regression.
the path created by the same open dal client cannot be read.
image

@imWildCat
Copy link
Contributor Author

this regression is happening between v0.45.1 and v0.46.0

this is the last correct version: https://github.com/apache/opendal/blob/v0.45.1/core/src/services/gdrive/core.rs#L396

@imWildCat
Copy link
Contributor Author

Interesting. It also repros using v0.45.1
image

@Xuanwo
Copy link
Member

Xuanwo commented Jun 13, 2024

async fn create_dir(&self, parent_id: &str, name: &str) -> Result<String> {
let url = "https://www.googleapis.com/drive/v3/files";
let content = serde_json::to_vec(&json!({
"name": name,
"mimeType": "application/vnd.google-apps.folder",
// If the parent is not provided, the folder will be created in the root folder.
"parents": [parent_id],
}))
.map_err(new_json_serialize_error)?;

Maybe we should trim the / suffix for dir?

@imWildCat
Copy link
Contributor Author

let me try!

@imWildCat
Copy link
Contributor Author

it seems to be fixed by this change.

image

https://github.com/imWildCat/opendal/commits/main/

I also found another issue shown above: opendal now create folder with same names...

@Xuanwo
Copy link
Member

Xuanwo commented Jun 13, 2024

I also found another issue shown above: opendal now create folder with same names...

We should already have checks for the folder exist before creating. Maybe some logic not adapated yet? Like creating abc folder but check with abc/?

@imWildCat
Copy link
Contributor Author

Could it be related to cache?
I'm not calling opendal concurrently.

@Xuanwo
Copy link
Member

Xuanwo commented Jun 13, 2024

I'm not calling opendal concurrently.

The cache is thread-safe, so concurrent calls should not pose a problem. I believe this issue is related to how we handle paths.

@imWildCat
Copy link
Contributor Author

after the fix you proposed above, the duplication problem still exists.

@imWildCat
Copy link
Contributor Author

anyway, let me create a PR to fix the naming issue first.
let's see if the test can pass

@imWildCat
Copy link
Contributor Author

@Xuanwo any suggestion on the direction to fix the tests on main first?

I believe we have many regressions since #3975

@Xuanwo
Copy link
Member

Xuanwo commented Jul 12, 2024

@Xuanwo any suggestion on the direction to fix the tests on main first?

Hi, I've been quite busy recently. I plan to set aside some time next week to debug this issue.

@imWildCat
Copy link
Contributor Author

@Xuanwo any suggestion on the direction to fix the tests on main first?

Hi, I've been quite busy recently. I plan to set aside some time next week to debug this issue.

Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants