setup_jukebox_core.sh on device doesn't use pre-built pyzmq on future3

[DivineDominion]

Describe your problem

_jukebox_core_download_prebuilt_pyzmq, introduced in e8945ed, is never called/used.

So in the end, after libzmq is fetched from Google Drive, pyzmq is then fetched without WebSockets support via pip3. So the intended workaround to use a pre-built binary mentioned here is not being utilized:

RPi-Jukebox-RFID/installation/routines/setup_jukebox_core.sh

Lines 72 to 78 in 5e35ce8

	# ZMQ
	# Because the latest stable release of ZMQ does not support WebSockets
	# we need to compile the latest version in Github
	# As soon WebSockets support is stable in ZMQ, this can be removed
	# Sources:
	# https://pyzmq.readthedocs.io/en/latest/draft.html
	# https://github.com/MonsieurV/ZeroMQ-RPi/blob/master/README.md

What's your hardware set up?

• Raspberry Pi 3 Model B Plus Rev 1.3
• RPi OS Lite, 32 Bit, v11 (Bullseye)

[DivineDominion]

Maybe actually using the pre-built binary would also fix #2050

Or build pyzmq properly as mentioned there. I'll explore both and report back

[pabera]

The reason why we implemented both options was the following: 1) Building zmq on low performance Pi's fails. This was the case for a Pi 3a or 2, 1 or Zero. They don't have enough memory. Therefore we pre-built the binaries and offered them via download. It's arguable whether a Google Drive link is a great download resource (maybe this should be a Github link). 2) Building zmq takes a long time on any Pi and it slows down the installation process of the Phoniebox a lot.
The reason why we offer building it on the Pi was the variations of the systems, Pi 4, now Pi 5 and 32bit vs. 64bit.
Maybe ZMQ now supports WebSockets by default and all of this is not required. This should be explored :-)

[AlvinSchiller]

@pabera i currently have a 3A+ i work with. And the installation failed at the first try. But after increasing the swapfile the build was successfull (still takes a while ^^).
This could also be part of the installation, if it makes things easier.

[DivineDominion]

@pabera Thanks! I understand the reasons. I could not find any call of _jukebox_core_download_prebuilt_pyzmq in git (via git -S"_jukebox_core_download_prebuilt_pyzmq"). So it looks to me like this function was never used after it was added? That is the confusing part: because the idea to offer a binary download sounds sensible, but it's not taking any effect and the installation boils down to use pip3 install pyqmt in the end, including download progress and such in the log.

[DivineDominion]

@AlvinSchiller

i currently have a 3A+ i work with. And the installation failed at the first try. But after increasing the swapfile the build was successfull (still takes a while ^^). This could also be part of the installation, if it makes things easier.

May I ask how you did install, what error you encountered, and how and increased the swap file? (My attempt to install doesn't try to compile anything and I didn't run out of swap space AFAICT, so maybe I'm doing something different?)

[AlvinSchiller]

@AlvinSchiller

i currently have a 3A+ i work with. And the installation failed at the first try. But after increasing the swapfile the build was successfull (still takes a while ^^). This could also be part of the installation, if it makes things easier.

May I ask how you did install, what error you encountered, and how and increased the swap file? (My attempt to install doesn't try to compile anything and I didn't run out of swap space AFAICT, so maybe I'm doing something different?)

I installed the develop branch, and choose to build locally. But i can try again later and get more details for you. The 3A+ only has 500 MB Ram which is slightly to low. The 3B+ with 1 GB shouldn't need the swapfile change.
Haven't tried main yet. Which branch you are using?

[pabera]

@DivineDominion I really don't remember why this code was added (e8945ed) but never called. I remember the reasons (explained above), but maybe it was more for testing purposes. I recall that it was pretty costly to prebuild all these environments etc, maybe it never worked out well. 🤷‍♀️

[DivineDominion]

@AlvinSchiller

I installed the develop branch, and choose to build locally. But i can try again later and get more details for you. The 3A+ only has 500 MB Ram which is slightly to low. The 3B+ with 1 GB shouldn't need the swapfile change. Haven't tried main yet. Which branch you are using?

Sorry for repeating myself so bluntly -- how did you do that exactly?

I'm on feature3/develop, too, because of the more recent compatibility changes to pip3 usage.

I would love to rebuild pyzmq, too, but it never does. All I get is a download. I found how to rebuild the web app in the update instructions but that's not affecting pyzmq.

[DivineDominion]

Good news. The command should include --no-binary "pyzmq" to avoid downloading the latest version and be this:

sudo ZMQ_PREFIX="${ZMQ_PREFIX}" ZMQ_DRAFT_API=1 \
  pip3 install --no-cache-dir --no-binary "pyzmq" --pre pyzmq

See the updated pyzmq docs:

https://github.com/zeromq/pyzmq/blob/main/docs/source/howto/draft.md

I'm currently executing everything to double check.

[DivineDominion]

Took a while! But the build instructions work 🎉

---

See #2096 for a fix.

Also see #2097 to remove the unused function. That would close this issue, because the observation was that this helper isn't used, and the build process didn't work. Now the build process works with the fix. 👍

We could still re-enable using the binary, though? But that sounds like an annoying CI task -- one we can hardly test manually for all architectures. I would suggest to use the manual build process that does what it is supposed to do and leave the convenience download preparation for an enhancement (new issue) for later, because it'd be such a tedious blocker.

[AlvinSchiller]

@AlvinSchiller

I installed the develop branch, and choose to build locally. But i can try again later and get more details for you. The 3A+ only has 500 MB Ram which is slightly to low. The 3B+ with 1 GB shouldn't need the swapfile change. Haven't tried main yet. Which branch you are using?

Sorry for repeating myself so bluntly -- how did you do that exactly?

I'm on feature3/develop, too, because of the more recent compatibility changes to pip3 usage.

I would love to rebuild pyzmq, too, but it never does. All I get is a download. I found how to rebuild the web app in the update instructions but that's not affecting pyzmq.

Sorry for the delay and not sure if it is still relevant, but here my steps for the installation:

• Run installer (from my own fork)
  • cd; GIT_USER='AlvinSchiller' GIT_BRANCH='future3/develop' bash <(wget -qO- https://raw.githubusercontent.com/AlvinSchiller/RPi-Jukebox-RFID/future3/develop/installation/install-jukebox.sh)
• Choose to build WebApp
• Reboot after installation
• Extend Swapfile to 512 MB
  • sudo dphys-swapfile swapoff
  • sudo nano /etc/dphys-swapfile -> change CONF_SWAPSIZE=100 to CONF_SWAPSIZE=512
  • sudo dphys-swapfile setup
  • sudo dphys-swapfile swapon
• Reboot
• Run WebApp installer
  • cd ~/RPi-Jukebox-RFID/src/webapp
  • ./run_rebuild.sh -u
• Run command for pyzmq from Websocket failure on future3 #2050 (comment)
  • sudo pip uninstall pyzmq
  • sudo apt-get install libzmq3-dev
  • MQ_DRAFT_API=1 sudo pip3 install --no-binary=pyzmq pyzmq

Phoniebox is Running, WebApp is usable, Songs can be player and RC522 Rfid Card reader is also working

[DivineDominion]

@AlvinSchiller Thank you!

---

With recent changes to the repo, this issue has been fixed. Thanks everyone for the help!