py-neuro GUI development

[timonmerk]

Overall Goal:

• Creating a Graphical User Interface (GUI) for the [[py_neuromodulation]] toolbox (GitHub - neuromodulation/py_neuromodulation: Real-time analysis of intracranial neurophysiology recordings.)
• There are many ways to do this but a good start would be a REACT Application
  You can find out more about React here: React

Milestones:

Ground work (basic):

• setup a development environment #subtask
  It's basically like a digital work place for each project. There's plenty of resources online telling you how you can setup an environment for your use case
• Learning about git, signing up on Github #subtask
  Git is something everybody working with code in a team needs to know. It allows us to work together without messing up someone else's work.
  You can compare it with something like google docs. People can work together over the internet and see the changes some else has done.
  You don't need to know everything here, just the basics: clone/create a repository, pull changes, commit your changes, push your changes, working on branches
• Get to know the React framework #subtask
  React and the underlying languages (javascript, HTML, CSS) are used all over the world. There's tons of tutorials online and projects on Github.
  My recommendation: Learning by doing. Try to build something. It's a great way of learning quick and it's very rewarding to see your results. This is the offical documentation: (React)
• Set up the project and start building the application #subtask
  Now that you know the basics, let's create this project. You will get the layout drafts of the GUI.
  You can start by implementing the components without a lot of functionality. There's a lot of things to consider:
  should a button be "clickable" at any time?
  Will the Web-App look and work normal on different screen sizes? (responsiveness)
  How can you structure the input fields so users will like and understand it best?
  Feel free to make it as beautiful and fresh looking as you can
• Bind the GUI to the backend #subtask
  With this point you will get some help from us. we will give you the data you need (probably JSON format) coming from py_neuromodulation.
  At this point it makes sense to already understand how users are utilizing the toolbox.
  Your task is to change read this data where it makes sense and update the JSON files. Basically implementing the logic into the GUI and giving it the possibility to communicate with the backend.

Refine (intermediate):

• Make all static screens/visualizations of the GUI work and deal with edge cases. #subtask
  The user won't make any mistakes if we don't let them! Try to eliminate all possible sources of error. Button's that shouldn't be pressed at some point shouldn't be active.
  Can you get from one screen to the next without errors (always?) How do you deal with e.g. wrong inputs?

Perfectionise (PRO):

• Visualising the actual features in a live visualisation. #subtask
  Does the application run with all static features (settings, channels,....) ? Freakin Awesome!!!
  Now let's get to the really tricky stuff: How can we visualize the features coming from the backend in real time?
  Try implementing graphs giving useful insights into the data. Like raw signal, fft, mov detect,....
• Any cool ideas to add on top? Let's do it! #subtask

Useful tools:

• (to test very basic component examples and see the file structure you could have a look at StackBlitz | Instant Dev Environments | Click. Code. Done.)

[toni-neurosc]

Hi @timonmerk and @elizaveta-terek, since I am the one who proposed building the GUI as a web-app, I would like to give here a brief explanation of the tech-stack we will be using and the reason for each of the choices:

Frontend (React SPA)

• Javascript runtime + package manager: Developing web apps that use Javascript as a programming language typically requires not only running code on the browser (which has a JS engine build-in), but also running development tools locally on the computer, which also need to be installed from a repository along with their dependencies. The typical runtime that allows running JS apps locally is Node.js, and the standard repository for JS packages is NPM (Node Package Manager). Therefore, typically one would install Node.js and NPM by using a tool called NVM (Node Version Manager). As you can see, this is already getting fairly complex, for that reason I usually prefer using Bun (https://bun.sh/) which is an all-in-one JS runtime and package manager written in Zig that is super-fast and has a syntax almost identical to npm/node. But the choice on this tool does not really matter as it does not affect the rest of the workflow.

• Development environment: A typical local JS tool is a development environment that takes care of a series of things:

  • Create the scaffolding required for the app (folder structure + boilerplate code)
  • Running a development server that builds the app real-time so you can see the changes as you change your code
  • Compiling and packing all the code into "static" files that are then served by the web-server.

  The most popular dev-env for JS is Vite. It is a combination of several tools: a project setup tool, a dev server, a plugin manager, a Typescript compiler (but it supports plain JS too) and a webpacking tool to build and deploy the app. You can install and run Vite using any JS package manager: https://vitejs.dev/guide/#scaffolding-your-first-vite-project

• UI framework: When developing an app in JS, there are a couple things that you want to do that are a pain to do with plain HTML/JS/CSS. This would include:

  • Creating custom UI components with encapsulated behavior and style that can be re-used in different places, like for example creating a button that you can put in your HTML by writing <MyButton />.
  • Passing JS variables into the HTML and CSS code.
  • Maintaining client-side state: meaning that the state of the UI is maintained client-side and updated as the user interacts with the app without the need to pass data to the server or request new HTML from the server. Of course, we can still pass data to the server when we need to.
  • Creating reactive UI's: this means basically that the UI updates itself as the data changes without us needing to do anything except handle the underlying state in the JS code.

  We have chosen React as a framework for 2 reasons mainly:

  • It's the most used UI framework, so as @timonmerk mentioned, there is a lot of support for it and teaching resources, as well as ready-made, open-sourced UI components that we can copy or adapt to do things quickly.
  • Plotly maintains a React component library that we can immediately start using to display plots: https://github.com/plotly/react-plotly.js

  React is, nonetheless, not without some disadvantages:

  • It's a batteries not incluced kind of framework, which means that there are some things that it can't do out of the box (e.g. client-side routing, database-connection, etc.) so it relies on other open-source tools to do those things such as React Router.
  • It's one of the oldest frameworks and has accumulated a lot of complexity with the years, so the learning-curve is a bit harder than other frameworks.
  • It's not the most performant, so we will have to see how it handles the plotting of real-time data to a reasonable framerate.

• Client-side routing: This is a bit more technical, and it has to do with the concept of SPA or Single-Page Application. The conept of "routing" will become clearer when I explain the backend, but it basically means that the app can move between different sections of the app (e.g. pynm.app/open-stream and pynm.app/view-stream) without reloading the interface, replacing only the necessary parts of the interface at each time.

Backend (Python server)

Now let's talk a bit of backend, which will be handled entirely in Python:

• Web app framework: Usually a web-site has different pages inside it, for example, and https://en.wikipedia.org/ and https://en.wikipedia.org/wiki/JavaScript are 2 different URLs that contain different HTML code. In the context of a web-application, sometimes this URLs are not just folders with HTML files inside, but rather are API-endpoints that return data that the frontend requests to the server. For example, if we wanted to get data from our server to show it in a graph, we could create an API-endpoint called "http://myapp.com/data", and when we make an HTTP request to this URL, instead of returning HTML it will return data, typically in the form of a serialized data format such as JSON. From the perspective of the web-app server, these URLs together with the associated response are called "server routes" and the functionality is called "routing".

  So this is more or less what the server does in the context of a web-app and for doing this we need a Python package that handles everything necessary to make this happen, including routing. Usually we call this a backend framework and Python has different options, with Django being the most popular but also being fairly big and complex and requiring a lot of boilerplate. Instead, at least for now, we will be using Flask, which is a very minimal framework that works with very few lines of code.

• Web-sockets: communication between backend and frontend is typically done by using API-endpoint routes that return serialized data, but in the event that we need an event-based method that allows real-time bi-directional communication between frontend and backend that also supports sending binary-data, we can use the Websockets protocol. I'm still new to this as well so I'm learning how to use it, but I think this is the best way to share the raw data and calculated feature data stream between the backend and the frontend for graphing purposes. To avoid using the low-level Websockets protocol directly, a typical solution is to use a library called SocketIO that has Python support and particularly a Flask integration package called flask-socketio that we will be using