-
Notifications
You must be signed in to change notification settings - Fork 1.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Cannot see items in vault #2561
Comments
I was able to reproduce this. I created a new instance (both postgres and bitwarden). Logged into the web portal. Was moving items around, and I moved an item to a different folder, and then tried to move the same item to my organization. Gave me the cipher error...I logged out and back in and my vault is gone again. User gives 404 in the admin portal again. This is similar to what I was doing the first time. I think I had renamed an item and then tried editing the same item again (either to move to a folder or the organization). It's not specific to the desktop app, though, since I was able to reproduce it on web portal this time. |
Thanks for that helpful stack trace, looks like the issue is in this method here: server/src/Infrastructure.EntityFramework/Repositories/Queries/UserCipherDetailsQuery.cs Lines 79 to 90 in 5036304
To add more details it looks like we may be saving folder data in the database in a different format than we are expecting it to be in this method and we will need to find out which is the proper format (based on how the MSSQL implementation) is doing it and copy that. |
maybe the same issue as #2512 (comment) the solution can be: |
@juliokele It should have no problem reading a Guid that is all uppercase or all lowercase (see here). I think something else is going on where the data is being input wrong. @Jalancar Could you connect to your database and get me the data that comes from this query: select "Folders"
from public."Cipher"
where "Folders" is not null
limit 1 This should just be ids to both your user and to the folder but use your discretion if you think it's displaying identifiable information. |
I believe I am having the same issue. The organization drops out. I was running stable for a few days, I was moving passwords to my Organization, today I cannot sync and in admin portal I get 404 when clicking on my user. This also happened a few days ago, but I thought it was stable now. I am running Podman on Fedora 37 to host the Unified beta. Environment Details
Database Image mariadb:10 This is my api.log when I try to sync (fail) from the Bitwarden macos client: fail: Microsoft.EntityFrameworkCore.Query[10100] |
"Folders" |
MariaDB [bitwarden_vault]> select Folders from Cipher where Folders is not null limit 1; MariaDB [bitwarden_vault]> |
Will the fix require a repair in the database? |
I have the same issue. When I remove the user from organization, he can see all his private passwords again. When he joins organization again, he can't see anything and from his point of view he is not assigned to organization, just like a clean new account. |
After it happened to me the first time in my setup, it took a number of days to break again. So, I haven't tried to start over again until more is known about this issue. To me, though I don't have solid evidence, it occurs once I started making changes to my passwords by moving passwords from my vault to the Organization Collections. |
Hi, is there any manual fix I can perform on my db? Running on postgres
|
Figured out a manual fix - looks like the issue is that some "Folders" values in the Ciphers table have a blank key. I updated the values and added the key values manually. Now my vault is loading again! |
@ngalluzzo Can you describe the queries you executed to fix it? |
Someone may be able to give a smarter query, I didn't have time to figure out how to parse the json object properly. FWIW I'm running Postgres 14
Look for rows that have a Folders value with an empty object key, like this:
Copy the key from one of the other rows (it seems to be the same for every row) and edit the empty values
cc @justindbaur Haven't figured out how to reproduce what causes the key to go missing, but wanted to call it out to you in case you have a hunch. I'm 80% certain that it has something to do with updating an Org folder. |
I can confirm this. It appears that when moving items from my personal folder to an organization folder one of the items was missing the the uuid key in the first value of the string like {"": "some_uuid_value"}. After inputing the value everything reappeared. |
I was also able to reproduce on a fresh self-host installation. Create Folder Everything fine ✅: Create Item -> Select newly created Folder -> Select Organisation/Collection -> Save Will break ❌: Create Item -> Select Organisation/Collection -> Save -> Edit Item -> Select newly created Folder -> Save |
@dionysiusmarquis Thank you so much for the reproduction steps, I have just made a PR to fix this issue and wrote up some SQL you could all run on your databases to fix the folders. After that PR merges and it gets released to the beta tag it should allow for vault items to start showing up. Without running the below cleanup scripts items still won't be in the folders you had chosen. If you don't want to run the scripts you can just manually add items to the folders you want and it should work from then on. MySQL: UPDATE Cipher AS c
JOIN Folder AS f ON f.Id = JSON_UNQUOTE(JSON_EXTRACT(c.Folders, '$.""'))
SET c.Folders = JSON_SET(JSON_REMOVE(c.Folders, '$.""'), CONCAT('$."', f.UserId, '"'), f.Id)
WHERE JSON_KEYS(c.Folders) = JSON_ARRAY(""); Postgres: UPDATE "Cipher"
SET "Folders" = (c."Folders"::jsonb - '')::jsonb || jsonb_build_object(f."UserId", f."Id")
FROM "Cipher" c
JOIN "Folder" f ON f."Id" = (TRIM('"' FROM (c."Folders"::jsonb->'')::text)::uuid)
WHERE (c."Folders"::jsonb->'') is not null; |
This has happened twice now on the latest dev branch of the unified docker container. |
Steps To Reproduce
Expected Result
See all of my saved logins and secure notes
Actual Result
There are no items to list.
Screenshots or Videos
No response
Additional Context
After installing self-hosted Bitwarden, I imported my passwords from Lastpass. A lot of them were duplicated for some reason. I was going through and editing/de-duplicating. After changing the name of one, I tried to move it to my Organization. It gave me an error about the cipher being out of date, and to re-sync. I tried to re-sync and it says Sync failed. I logged out and back in, and now my vault shows as being empty. As far as I can tell, it all still exists in the database. I have over 1800 entries in the Cipher table, my vault shows empty in the web UI, the desktop application, and the Chrome extension. I added my email to the the Admin users so that I could get to the admin console. It shows my users and Organization, but if I click on either user, it gives me a 404 and this error shows up in api.log:
2023-01-10 16:28:27.463 +00:00 [ERR] The JSON value could not be converted to System.Collections.Generic.Dictionary`2[System.Guid,System.Guid]. Path: $ | LineNumber: 0 | BytePositionInLine: 4. System.Text.Json.JsonException: The JSON value could not be converted to System.Collections.Generic.Dictionary`2[System.Guid,System.Guid]. Path: $ | LineNumber: 0 | BytePositionInLine: 4. ---> System.FormatException: The JSON value is not in a supported Guid format. at System.Text.Json.Utf8JsonReader.GetGuidNoValidation() at System.Text.Json.Serialization.Converters.GuidConverter.ReadAsPropertyNameCore(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options) at System.Text.Json.Serialization.JsonDictionaryConverter`3.<OnTryRead>g__ReadDictionaryKey|10_0(Utf8JsonReader& reader, ReadStack& state, <>c__DisplayClass10_0& ) at System.Text.Json.Serialization.JsonDictionaryConverter`3.OnTryRead(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options, ReadStack& state, TDictionary& value) at System.Text.Json.Serialization.JsonConverter`1.TryRead(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options, ReadStack& state, T& value) at System.Text.Json.Serialization.JsonConverter`1.ReadCore(Utf8JsonReader& reader, JsonSerializerOptions options, ReadStack& state) --- End of inner exception stack trace --- at System.Text.Json.ThrowHelper.ReThrowWithPath(ReadStack& state, Utf8JsonReader& reader, Exception ex) at System.Text.Json.Serialization.JsonConverter`1.ReadCore(Utf8JsonReader& reader, JsonSerializerOptions options, ReadStack& state) at System.Text.Json.JsonSerializer.ReadFromSpan[TValue](ReadOnlySpan`1 utf8Json, JsonTypeInfo jsonTypeInfo, Nullable`1 actualByteCount) at System.Text.Json.JsonSerializer.ReadFromSpan[TValue](ReadOnlySpan`1 json, JsonTypeInfo jsonTypeInfo) at System.Text.Json.JsonSerializer.Deserialize[TValue](String json, JsonSerializerOptions options) at Bit.Infrastructure.EntityFramework.Repositories.Queries.UserCipherDetailsQuery.GetFolderId(Nullable`1 userId, Cipher cipher) in /source/src/Infrastructure.EntityFramework/Repositories/Queries/UserCipherDetailsQuery.cs:line 83 at lambda_method420(Closure , QueryContext , DbDataReader , ResultContext , SingleQueryResultCoordinator ) at Microsoft.EntityFrameworkCore.Query.Internal.SingleQueryingEnumerable`1.AsyncEnumerator.MoveNextAsync() at Microsoft.EntityFrameworkCore.EntityFrameworkQueryableExtensions.ToListAsync[TSource](IQueryable`1 source, CancellationToken cancellationToken) at Microsoft.EntityFrameworkCore.EntityFrameworkQueryableExtensions.ToListAsync[TSource](IQueryable`1 source, CancellationToken cancellationToken) at Bit.Infrastructure.EntityFramework.Repositories.CipherRepository.GetManyByUserIdAsync(Guid userId, Boolean withOrganizations) in /source/src/Infrastructure.EntityFramework/Repositories/CipherRepository.cs:line 353 at Bit.Api.Controllers.SyncController.Get(Boolean excludeDomains) in /source/src/Api/Controllers/SyncController.cs:line 72 at lambda_method87(Closure , Object ) at Microsoft.AspNetCore.Mvc.Infrastructure.ActionMethodExecutor.AwaitableObjectResultExecutor.Execute(IActionResultTypeMapper mapper, ObjectMethodExecutor executor, Object controller, Object[] arguments) at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.<InvokeActionMethodAsync>g__Logged|12_1(ControllerActionInvoker invoker) at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.<InvokeNextActionFilterAsync>g__Awaited|10_0(ControllerActionInvoker invoker, Task lastTask, State next, Scope scope, Object state, Boolean isCompleted) at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.Rethrow(ActionExecutedContextSealed context) at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.Next(State& next, Scope& scope, Object& state, Boolean& isCompleted) at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.<InvokeInnerFilterAsync>g__Awaited|13_0(ControllerActionInvoker invoker, Task lastTask, State next, Scope scope, Object state, Boolean isCompleted) at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeNextExceptionFilterAsync>g__Awaited|26_0(ResourceInvoker invoker, Task lastTask, State next, Scope scope, Object state, Boolean isCompleted)
Githash Version
3997def-dirty
Environment Details
Database Image
postgres:15
Issue-Link
#2480
Issue Tracking Info
The text was updated successfully, but these errors were encountered: