Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

798 change url scheme from kiwix to zim #961

Merged
merged 5 commits into from
Sep 8, 2024

Conversation

BPerlakiH
Copy link
Collaborator

@BPerlakiH BPerlakiH commented Sep 1, 2024

Fixes: #798

Note: changes applied on top of the #956 PR.

Solution:

Update the code to create "zim://" urls

Whenever we re-constructing urls (eg. to create bookmarks), the code has been updated.

Migrating bookmarks:

This was the easy part, once the app starts we do a one-off migration, selecting the bookmarks that start with the url "kiwix://" and update those to be "zim://"

Migrating tabs:

This was the hard part. We store the tabs together with the so called "interactiveState" of the webView loaded. Which is a serialised data provided by Apple. This contains the loaded url, and the browsing history (eg. former urls).

Formerly we used a "trick" to update such urls, for the migration of ZIM files, which is in short utf8 decode the data, and from that regex replace the urls.

Now the same trick is not working to change the url scheme, possibly because the length of the string is changing. Formerly we updated the zim id of the urls, which is the same length. (The scheme change from "kiwix" to "zim" is not the same length).
Editing the data as utf8 decoded string (as before), breaks the tabs, and they won’t load.

This "interactiveState" of the webView is not properly modifiable, or decodable in a reliable fashion, details of serialisation here.

It seems to be plist file, but cannot be decoded neither with python (plistlib), nor with swift (PropertyListDecoder), nor with macOS (plutil), without the detailed serialisation knowledge, it just won’t work.

TLDR: For these reasons, it should not be modified directly.

Solution for tab urls:

Since the tab urls, and interactive state is "saved" once we leave a tab (or the application), the solution is to migrate those urls "on demand", meaning once we want to load such a URL, we check if it's an old "kiwix" one, and modify it to be a new "zim" url, and load that instead.
Once loaded as a new "zim" URL, it will be saved correctly.

Possible followup:

Since we have a new DWDS content, in a separate PR, I can remove this "utf-8" decoding "trick" introduced by this older PR, and replace it with a similar "on demand" migration solution to change the hosts (ZIM file id migration from old to new).

@@ -42,4 +42,9 @@ final class BookmarkMigrationTests: XCTestCase {
let expectedString = "\0\0\0\u{02}bplist00�\u{01}\u{02}\u{03}\u{04}\u{05}\u{06}^RenderTreeSize^IsAppInitiated^SessionHistory\u{10}\u{03}\t�\u{07}\u{08}\t\n\u{1C}\u{1C}_\u{10}\u{15}SessionHistoryEntries_\u{10}\u{1A}SessionHistoryCurrentIndex_\u{10}\u{15}SessionHistoryVersion�\u{0B}\u{16}�\u{0C}\r\u{0E}\u{0F}\u{10}\u{11}\u{12}\u{13}\u{14}\u{15}_\u{10}\u{17}SessionHistoryEntryData_\u{10}\u{18}SessionHistoryEntryTitle_\u{10}2SessionHistoryEntryShouldOpenExternalURLsPolicyKey_\u{10}\u{16}SessionHistoryEntryURL_\u{10}\u{1E}SessionHistoryEntryOriginalURLO\u{10}P\0\0\0\0\0\0\0\0\u{02}\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0�\u{07}�@�\u{13}\u{06}\0\0\0\0\0\0\0\0\0����\0\0\0\0�\u{07}�@�\u{13}\u{06}\0����\0\0\0\0�\u{03}\0\0\0\0�?\0\0\0\0����TDWDS\u{10}\u{01}_\u{10}2kiwix://A992BF76-CA94-6B60-A762-9B5BC89B5BBF/index_\u{10}2kiwix://A992BF76-CA94-6B60-A762-9B5BC89B5BBF/index�\u{0C}\r\u{0E}\u{0F}\u{10}\u{17}\u{18}\u{19}\u{1A}\u{1B}O\u{10}P\0\0\0\0\0\0\0\0\u{02}\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0V<�A�\u{13}\u{06}\0\0\0\0\0\0\0\0\0����\0\0\0\0U<�A�\u{13}\u{06}\0����\0\0\0\0\0\0\0\0\0\0�?\0\0\0\0����[hier und da\u{10}\u{02}_\u{10}<kiwix://A992BF76-CA94-6B60-A762-9B5BC89B5BBF/wb/hie_.und_.da_\u{10}<kiwix://A992BF76-CA94-6B60-A762-9B5BC89B5BBF/wb/hie_.und_.da\u{10}\u{01}\0\u{08}\0\u{0F}\0\u{1E}\0-\0<\0>\0?\0F\0^\0{\0�\0�\0�\0�\0�\u{01}\u{0B}\u{01}$\u{01}E\u{01}�\u{01}�\u{01}�\u{01}�\u{02}\t\u{02}\u{14}\u{02}g\u{02}s\u{02}u\u{02}�\u{02}�\0\0\0\0\0\0\u{02}\u{01}\0\0\0\0\0\0\0\u{1D}\0\0\0\0\0\0\0\0\0\0\0\0\0\0\u{02}�"
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is how the "interactiveState" looks like in pure data.

@codecov-commenter
Copy link

codecov-commenter commented Sep 1, 2024

Codecov Report

Attention: Patch coverage is 42.59259% with 31 lines in your changes missing coverage. Please review.

Project coverage is 37.23%. Comparing base (9306122) to head (2392364).
Report is 6 commits behind head on main.

Files with missing lines Patch % Lines
Model/Migration/MigrateSchemeToZIM.swift 0.00% 11 Missing ⚠️
ViewModel/BrowserViewModel.swift 45.45% 6 Missing ⚠️
Model/Utilities/URL.swift 50.00% 5 Missing ⚠️
App/App_iOS.swift 0.00% 4 Missing ⚠️
App/App_macOS.swift 50.00% 2 Missing ⚠️
Model/Entities/SearchResult.m 0.00% 1 Missing ⚠️
Model/Utilities/WebKitHandler.swift 0.00% 1 Missing ⚠️
Views/ViewModifiers/SaveContentHandler.swift 0.00% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #961      +/-   ##
==========================================
- Coverage   39.12%   37.23%   -1.89%     
==========================================
  Files         110      111       +1     
  Lines        6311     6346      +35     
==========================================
- Hits         2469     2363     -106     
- Misses       3842     3983     +141     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@rgaudin
Copy link
Member

rgaudin commented Sep 6, 2024

I see no change to a plist file. Is the new scheme properly announced to the OS?

Base automatically changed from 798-migration-service to main September 6, 2024 08:15
@BPerlakiH
Copy link
Collaborator Author

I see no change to a plist file. Is the new scheme properly announced to the OS?
Good catch, thank you @rgaudin.

I have updated the plist file.

@BPerlakiH BPerlakiH linked an issue Sep 6, 2024 that may be closed by this pull request
rgaudin
rgaudin previously approved these changes Sep 6, 2024
Copy link
Member

@rgaudin rgaudin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM now.

@kelson42 kelson42 force-pushed the 798-change-url-scheme-from-kiwix-to-zim branch from 94b9ee4 to 2392364 Compare September 8, 2024 18:40
@kelson42 kelson42 merged commit 91f8161 into main Sep 8, 2024
4 checks passed
@kelson42 kelson42 deleted the 798-change-url-scheme-from-kiwix-to-zim branch September 8, 2024 19:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Change url scheme from "kiwix://" to "zim://"
4 participants