[WIP] Qt6 Conversion #1601

greghope667 · 2023-04-14T15:13:25Z

Summary of changes

Updates Plover to use Qt6 instead of Qt5
This is work-in-progress, see below for details

Pull Request Checklist

Changes have tests
News fragment added in news.d. See documentation for details

Progress

Convert Qt5 api calls to Qt6
Remove resource compiler (not supported in PyQt6)
Test for Windows/Mac/Linux
Include resources (icons) in built package
Update Windows/Mac/Linux build scripts
Figure out how this breaks plugins and fix them (Plugin manger updated, other plugins to follow)

Release plan

Moved to Qt6 scoped enums Enabled PyQt6 pyuic

mkrnr · 2023-04-17T18:33:44Z

Thank you so much for working on this! This migration is incredibly important for the maintainability of plover.

I'm happy to test this the next days on Windows, Linux, and macOS.

Let me know if there's anything else I can do to help.

greghope667 · 2023-04-17T21:58:34Z

@mkrnr Any testing you can do would be very helpful. Post any error messages or not-working features here.

If there are any plugins you use that you can test that would be useful, although I expect only non-graphical plugins have a chance of working without updates.

The packaging systems need fixing too (see all the red in the CI right now...) . I'll take a look into that over the next few days/weeks but if anyone knows how all the packaging scripts work then help would be appreciated.

Robberduckzilla · 2023-04-22T20:10:00Z

Launches successfully on Apple Silicon Macbook Air (M2 chip). Qt6 takes system theme (Dark) too, neat!

mkrnr · 2023-04-28T05:58:10Z

Here are some tests on Windows 11 64 bit using a Georgi:

tox r -e launch

all built-in functionality works flawlessly:

Add translation
Lookup
Paper Tape
Suggestions
Add/create/edit/delete dictionary
Writing steno
Anything else I tried

Warnings during start, maybe the one regarding gui_gt can be fixed:

warning: no previously-included files found matching '.gitignore'
adding license file 'LICENSE.txt'
warning: no previously-included files found matching 'plover\gui_qt\*_rc.py'
warning: no previously-included files found matching 'plover\gui_qt\*_ui.py'
warning: no previously-included files found matching 'plover\gui_qt\.gitignore'
no previously-included directories found matching '.github'

tox r -e test

only failures are in:

test\gui_qt\test_dictionaries_widget.py ..................................FFF

This fails on the master as well and is likely fixed by #1599.

tox r -e plugins_install

Executed successfully. Is there a way to test this at this point? Because when I then run tox r -e launch it recreates the env because the env type changes.

tox r -e plugins_install -- C:\Users\mkoer\git\plover_regenpfeifer

Executed successfully. Don't know how to test this, see above.

tox r -e packaging_checks

packaging_checks: commands[0]> bash --noprofile --norc -eo pipefail -c " . ./plover_build_utils/functions.sh; python=python; \"$@\"" -- packaging_checks
packaging_checks: exit 2 (.02 seconds) C:\Users\mkoer\git\greghope667-plover> bash --noprofile --norc -eo pipefail -c " . ./plover_build_utils/functions.sh; python=python; \"$@\"" -- packaging_checks
.pkg: _exit> python c:\users\mkoer\git\greghope667-plover\venv\lib\site-packages\pyproject_api\_backend.py True setuptools.build_meta
  packaging_checks: FAIL code 2 (81.95=setup[81.94]+cmd[0.02] seconds)
  evaluation failed :( (82.56 seconds)

tox r -e release_prepare

release_prepare: commands[0]> bash --noprofile --norc -eo pipefail -c " . ./plover_build_utils/functions.sh; python=python; \"$@\"" -- release_prepare
release_prepare: exit 2 (0.01 seconds) C:\Users\mkoer\git\greghope667-plover> bash --noprofile --norc -eo pipefail -c " . ./plover_build_utils/functions.sh; python=python; \"$@\"" -- release_prepare
.pkg: _exit> python c:\users\mkoer\git\greghope667-plover\venv\lib\site-packages\pyproject_api\_backend.py True setuptools.build_meta
  release_prepare: FAIL code 2 (75.69=setup[75.67]+cmd[0.01] seconds)
  evaluation failed :( (76.28 seconds)

greghope667 · 2023-04-28T14:55:18Z

Thanks for running the tests, that's great to know this works for other people.

Warnings during start, maybe the one regarding gui_gt can be fixed:

Don't think these warnings happened for Qt5 - I'll look at what's causing these messages

Executed successfully. Is there a way to test this at this point? Because when I then run tox r -e launch it recreates the env because the env type changes.

You might be able to launch plover directly using the environment created by tox. Enter the environment (something like source .tox/dev/bin/activate.bat, might be a slightly different path), then run with python -m plover. I think there's some way to directly run commands through tox in a given environment too but will have to check.

I'm going to look at packaging this, then hopefully we'll be able to share this and get some beta testers for a Qt6 Plover release.

mkrnr · 2023-05-02T18:16:11Z

I'll give that a try next days, thanks!

We'll also have to look into updating plover_plugins_manager to Qt6 since that's such an integral part of plover. Did you look into that already by any chance?

Switch builds to PyQt6 Include resources folder in distribution Remove extra MANIFEST.in entries to silence warnings Pin PyQt6-Qt6 - googlefonts/fontra-pak#27 Add pyqt6rc to test/build requirements

greghope667 · 2023-05-03T22:32:54Z

I've had a quick look at the plugin manager - there are a few small changes needed there for Qt6 but shouldn't be anything too major. We'll need to update both this project and the plugin manager together though as they're mutually dependent.

mkrnr · 2023-06-25T18:56:23Z

Awesome to see your progress on this, now with the plugin manager migration!

I'd like to help you testing this but when I launch with tox (tox r -e launch) the plugin manager is still not included. Could you share how you test this? I also tried building the packages on windows but was not successful.

greghope667 · 2023-06-26T14:25:52Z

Most of my recent testing has been through the CI system in GitHub, just fixing errors and getting the builds working. The CI versions have the plugin manager bundled (which was causing build failures using a non-patched Qt5 plugin manager version) - you can download these as build artefacts. Seems to work for the Linux AppImage, but the Windows exe doesn't yet run. I've yet to sort out how to use the plugin manager successfully when launching locally from tox.

Getting local builds to work is not easy - you have to have almost exactly the same setup (including library versions) that the build scripts need else you'll get a bunch of unhelpful error messages from pip. I've also run into issues where some of the download sources for build scripts are rate limited so stop working after a few attempts. Honestly the build system needs work to be simplified, it's a little unmanageable right now.

mkrnr · 2023-06-26T19:45:41Z

That makes a lot of sense to focus on getting the Github CI build system to work.

The macOS DMG installs and starts perfectly fine (macOS 13.4.1, M1 chip) but the Plugins Manager is not showing up. The CI build looked fine to me but I'll continue looking into that one.

On Win11 64bit I get ValueError: 'plugins_manager/icon.svg' must be only a file name for both the Installer and ZIP version. You probably get the same. I also get this error message when running the AppImage on Ubuntu.

Let me know if I can do more tests. I'll also try to work on solutions. Thanks again for working on this, I would love to buy you a coffee!

greghope667 · 2023-07-13T17:33:11Z

I've sorted out the bundling of assets now, and I think mostly everything works (well, it's likely there's some bugs hiding somewhere). All the CI versions seem to run ok for me as far as I can test. The plugin manager works using openstenoproject/plover_plugins_manager#22 , but no plugins will work as they're all PyQt5 - I'm not aware of any non-graphical plugins.

I'm sure some work is still needed (reviews are welcome) but I'm going to mark this as ready.

mkrnr · 2023-07-13T18:53:46Z

This is really exciting!

Here are the results of my first tests:

Win11 64bit opened via ZIP

I'm able to install plover-regenpfeifer via the plugin manager and it works (except a known bug within the plugin itself regarding file paths)
In Configuration > Machine when I press the Port dropdown menu, I get ERROR: Qt GUI error AttributeError: type object 'Qt' has no attribute 'UserRole'
Everthing else I tested works

Win11 64bit via installer

The installation works but when starting plover I get the following stacktrace:

2023-07-13 20:49:49,386 [MainThread] WARNING: Qt: QGuiApplication::font(): no QGuiApplication instance and no application font set.
2023-07-13 20:49:49,389 [MainThread] ERROR: Qt GUI error
Traceback (most recent call last):
  File "build/windist/data\Lib\site-packages\plover\scripts\main.py", line 131, in main
  File "build/windist/data\Lib\site-packages\plover\gui_qt\main.py", line 126, in main
  File "build/windist/data\Lib\site-packages\plover\gui_qt\main.py", line 77, in __init__
  File "build/windist/data\Lib\site-packages\plover\gui_qt\main_window.py", line 96, in __init__
  File "build/windist/data\Lib\site-packages\plover\gui_qt\utils.py", line 59, in Icon
  File "importlib\resources.py", line 161, in path
  File "importlib\resources.py", line 49, in _get_package
  File "importlib\resources.py", line 40, in _resolve
  File "importlib\__init__.py", line 127, in import_module
  File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
  File "<frozen importlib._bootstrap>", line 984, in _find_and_load_unlocked
ModuleNotFoundError: No module named 'plover_plugins_manager.gui_qt.resources'

MacOS m1 (Ventura 13.4.1) via DMG

Everything looks to work well but the Plugins Manager is not shown in the GUI

Ubuntu 64bit via AppImage

The plugin manager is shown and I'm able to install plugins
Same error as with Windows above: In Configuration > Machine when I press the Port dropdown menu, I get ERROR: Qt GUI error AttributeError: type object 'Qt' has no attribute 'UserRole'
Everything else I tested works

It would be awesome to have more people testing this.

greghope667 · 2023-07-13T19:27:12Z

Thanks for the help testing, especially across multiple systems.

ERROR: Qt GUI error AttributeError: type object 'Qt' has no attribute 'UserRole'

This is a PyQt api change, I think it's meant to be Qt.ItemDataRole.UserRole in PyQt6. Possibly from here? I'll fix this.

The Windows installer error looks like a missing file in the bundle. Maybe it's not extracting correctly? This is literally just an icon failing to load however, so maybe we can catch + log instead of aborting.

Not sure why the plugin manager isn't working on MacOS though. Do you have multiple plover installs at all? Plover uses the default (global) python system import system so can get confused quite easily. Will take a look at the DMG contents as well.

mkrnr

The UserRole fix works, no more errors around that 👍

Also I found out the reason for the plugins manager not showing up on MacOS (M1): There's a plugins folder where the plover configs are stored. This plugins folder also contains the Plugins Manager Plugin #inception
Deleting this folder fixed the issue and now the manager shows up.

Test summary:

MacOS M1 ✅
Windows 11 64 bit ✅
Ubuntu 64 bit ✅

Amazing work Greg!

mkrnr · 2023-07-16T06:57:49Z

.github/workflows/ci.yml

@@ -36,15 +36,15 @@ jobs:
        id: set_cache
        run: |
          job_cache_extra_deps=(); job_id=test_linux; job_name='Test (Linux)'; job_needs=(); job_os=Linux; job_platform=ubuntu-22.04; job_python=3.9; job_reqs=(reqs/dist.txt reqs/test.txt); job_skip_cache_name=skip_test_linux_py-3.9_ubuntu-22.04; job_skip_cache_path=.skip_cache_test_linux; job_skiplists=(job_test os_linux); job_test_args='-p no:pytest-qt --ignore=test/gui_qt'; job_type=test; job_variant=Linux; analyze_set_job_skip_cache_key
-          job_cache_extra_deps=(osx/deps.sh); job_id=test_macos; job_name='Test (macOS)'; job_needs=(); job_os=macOS; job_platform=macos-10.15; job_python=3.9; job_reqs=(reqs/dist.txt reqs/test.txt); job_skip_cache_name=skip_test_macos_py-3.9_macos-10.15; job_skip_cache_path=.skip_cache_test_macos; job_skiplists=(job_test os_macos); job_test_args='-p no:pytest-qt --ignore=test/gui_qt'; job_type=test; job_variant=macOS; analyze_set_job_skip_cache_key


You'll get merge conflicts when rebasing/merging master into this branch because I bumped that version. I didn't go for latest though to prevent "random" issues in the future with new macOS versions. In my view freezing this version makes the build more robust. (Same for the other places in this file).

Pinning the version is a good idea. I bumped it up to latest because there was some issue with old macOS versions in the CI, and was too lazy to check which versions actually worked. I'll fix the version to match what you had in your patch.

mkrnr · 2023-07-16T11:36:31Z

One more thing I noticed now: when installing via DMG on mac (M1), the dark mode doesn't work (😱).
Looking through the step Build distribution (macOS DMG) I saw this which might be related:

Warning:  This wheel needs a higher macOS version than is set in MACOSX_DEPLOYMENT_TARGET variable.  To silence this warning, set MACOSX_DEPLOYMENT_TARGET to at least 10_14 or recreate these files with lower MACOSX_DEPLOYMENT_TARGET:

In terms of general functionality it works though.

Robberduckzilla · 2023-07-16T19:42:31Z

@mkrnr
looks like this gets passed in .github/workflows/ci/helpers.sh, line 80
I think we can just bump this number to 10.14?
i.e.
run_eval "echo MACOSX_DEPLOYMENT_TARGET=10.14 >>\$GITHUB_ENV"

It's just a warning though, so I don't think it's causing the issue of plover not picking up default theme. Does it work as expected on Linux/Windows machines?

mkrnr · 2023-07-16T20:18:53Z

@Robberduckzilla makes sense. Another thought: maybe it's because the DMG is compiled for intel Macs and Rosetta doesn't care about theming?

Anyhow, I wouldn't consider this a show stopper at all. Definitely something that we can take care of in a follow-up PR.

…nto pyqt6-migration

Robberduckzilla · 2023-08-09T12:03:07Z

I don't have write access to your repo, so a quick note on a small change you should make:
/plover/__init__.py, line 15

- __version__ = '4.0.0.dev12'
+ __version__ = '5.0.0'

This will show version 5 in the about window, as well as (I think) install it into a plover 5.0.0 folder, rather than the misleading 4.0.0dev12

greghope667 added 10 commits April 3, 2023 17:47

Change imports to Qt6

d0ba2f3

Changes for PyQt6 compatibility

78baa69

Moved to Qt6 scoped enums Enabled PyQt6 pyuic

Update tests for pyqt6

0020e77

Switch to pyqt6rc for ui files

e192159

Use new Icon() function to import svg

3ace7fe

Remove PyQt5 rcc usage

d74ab50

Remove Qt5 from dependencies

388c201

Add empty __init__ file for import detection in pyqt6rc

fcbc5c8

More Qt5->Qt6 tweaks

2c7f162

Bump versions for building on Mac

1652b03

greghope667 added the in-progress label Apr 14, 2023

greghope667 changed the title ~~Qt6 Conversion~~ [WIP] Qt6 Conversion Apr 14, 2023

greghope667 force-pushed the pyqt6-migration branch 4 times, most recently from bed0d2b to d7065ba Compare May 3, 2023 22:05

Fixes for CI system

c428339

Switch builds to PyQt6 Include resources folder in distribution Remove extra MANIFEST.in entries to silence warnings Pin PyQt6-Qt6 - googlefonts/fontra-pak#27 Add pyqt6rc to test/build requirements

greghope667 force-pushed the pyqt6-migration branch from d7065ba to c428339 Compare May 3, 2023 22:17

greghope667 added 2 commits June 17, 2023 18:47

Use temporary qt6 plugin manager

750870f

CI file updates

327a818

greghope667 force-pushed the pyqt6-migration branch from edec04b to 327a818 Compare June 17, 2023 18:26

Change icon code to allow imports from other modules

e292cc0

greghope667 mentioned this pull request Jul 13, 2023

Pyqt6 migration openstenoproject/plover_plugins_manager#22

Open

greghope667 marked this pull request as ready for review July 13, 2023 17:33

UserRole -> ItemDataRole.UserRole

6482889

mkrnr approved these changes Jul 16, 2023

View reviewed changes

greghope667 added 2 commits July 18, 2023 17:55

Merge branch 'master' of https://github.com/openstenoproject/plover i…

359c02a

…nto pyqt6-migration

Add news fragment

d215bea

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[WIP] Qt6 Conversion #1601

[WIP] Qt6 Conversion #1601

greghope667 commented Apr 14, 2023 •

edited

Loading

mkrnr commented Apr 17, 2023

greghope667 commented Apr 17, 2023

Robberduckzilla commented Apr 22, 2023

mkrnr commented Apr 28, 2023

greghope667 commented Apr 28, 2023

mkrnr commented May 2, 2023

greghope667 commented May 3, 2023

mkrnr commented Jun 25, 2023

greghope667 commented Jun 26, 2023

mkrnr commented Jun 26, 2023 •

edited

Loading

greghope667 commented Jul 13, 2023

mkrnr commented Jul 13, 2023

greghope667 commented Jul 13, 2023

mkrnr left a comment

mkrnr Jul 16, 2023

greghope667 Jul 18, 2023

mkrnr commented Jul 16, 2023

Robberduckzilla commented Jul 16, 2023 •

edited

Loading

mkrnr commented Jul 16, 2023

Robberduckzilla commented Aug 9, 2023 •

edited

Loading

[WIP] Qt6 Conversion #1601

Are you sure you want to change the base?

[WIP] Qt6 Conversion #1601

Conversation

greghope667 commented Apr 14, 2023 • edited Loading

Summary of changes

Pull Request Checklist

Progress

Release plan

mkrnr commented Apr 17, 2023

greghope667 commented Apr 17, 2023

Robberduckzilla commented Apr 22, 2023

mkrnr commented Apr 28, 2023

tox r -e launch

tox r -e test

tox r -e plugins_install

tox r -e plugins_install -- C:\Users\mkoer\git\plover_regenpfeifer

tox r -e packaging_checks

tox r -e release_prepare

greghope667 commented Apr 28, 2023

mkrnr commented May 2, 2023

greghope667 commented May 3, 2023

mkrnr commented Jun 25, 2023

greghope667 commented Jun 26, 2023

mkrnr commented Jun 26, 2023 • edited Loading

greghope667 commented Jul 13, 2023

mkrnr commented Jul 13, 2023

Win11 64bit opened via ZIP

Win11 64bit via installer

MacOS m1 (Ventura 13.4.1) via DMG

Ubuntu 64bit via AppImage

greghope667 commented Jul 13, 2023

mkrnr left a comment

Choose a reason for hiding this comment

mkrnr Jul 16, 2023

Choose a reason for hiding this comment

greghope667 Jul 18, 2023

Choose a reason for hiding this comment

mkrnr commented Jul 16, 2023

Robberduckzilla commented Jul 16, 2023 • edited Loading

mkrnr commented Jul 16, 2023

Robberduckzilla commented Aug 9, 2023 • edited Loading

greghope667 commented Apr 14, 2023 •

edited

Loading

mkrnr commented Jun 26, 2023 •

edited

Loading

Robberduckzilla commented Jul 16, 2023 •

edited

Loading

Robberduckzilla commented Aug 9, 2023 •

edited

Loading