New Control Panel with Flutter

[Dampfwalze]

Since it seemed like you are a bit uncertain, which framework suits this purpose well, I've gone ahead and put in a day of work, to build a prototype with Flutter:

2024-02-14.09-37-35.mp4

I have used the fluent_ui package, so it looks similar to what you have already implemented using WPF.

I've also gone ahead and integrated it into the installer.

I think Flutter serves this purpose well, because it has good performance and a very good and easy development experience. It builds to a small application package and does not have dependency hell.

The Dart programming language has a similar ecosystem than Rust, while being similar to C#, but with more polished language features and good type safety (sound null safety). Recently, it also got new powerful functional programming features, like pattern matching, similar to Rust. Sometimes, I like to call it "Managed Rust".

Source: https://github.com/Dampfwalze/virtual-display-rs/tree/control-panel-flutter

I have published a release with this Control Panel

I am curious about your opinion and if this interests you, I can develop this a bit further and open a PR.

[MolotovCherry]

Hi!

First of all, thank you for your effort and willingness to contribute!

I'm not fundamentally opposed to going this direction, but I have a few questions/thoughts first:

• I am pleased with the startup time (even more so than the c# version in fact)
• How mature is fluent-ui compared to the native winui3? I do see the repository has a lot of stars and development, so this gives me a good first impression. Does it also support themes to the extent I did in the original c# version (I assume you built and tested the original gui app, so you know what it looks like)
• If we did go this route, I'm most likely to want to do some Rust->Dart ffi via a dylib (mainly for the named pipe and ipc libraries). The purpose of this is the logic can be more tightly coupled with the original Rust version, and allows it to be 1 single source of truth. It may even be possible to go so far as to do all the pipe and ipc communication in rust and expose this to dart with an api + callbacks. Such a setup would also make it easy to expose other native functionality if the need arises.
• I see there is some c++ code in the repo. I assume this is only for generated/required stuff, and not something that would actually need to be touched once set up? The reason I'm asking is because I don't want c++ knowledge to be a requirement for project development going forward (at least in the sense of having to program in it), though I can accept it if it's only a pure dependency/generated code. (One of the main reasons Rust was used was to keep c++ out of the picture)
• There are a few things under the hood that the gui app is expected to do in order to work with the driver. Mainly, this is saving/loading from a registry key for persistence; the driver only reads the registry, it does not save to it, so it's completely the gui's job to manage persistence. Another thing is letting the gui know the maximum amount of monitors it can add/enable at once time (there's a set upper limit, but it's possible it may dynamically change due to some lower level details in the OS!)
• If my thoughts sound reasonable to you, I'd be willing to collaborate on the implementation so that the two parts work properly together. If you're interested, it would be convenient to use my discord support server for the collaboration

[Dampfwalze]

Hi!

I am glad you like it!

Here are my thoughts on your questions:

I am pleased with the startup time (even more so than the c# version in fact)

Dart archives this using snapshots. When you compile Dart to a snapshot, it starts a fully functional VM and initializes it with your source code (either using JIT or AOT). But instead of running your code, it stops there and takes a snapshot. This snapshot can later be used to initialize a new VM. (This is all handled automatically)

Also, Dart is tree shaken, when compiled AOT, which contributes to a small package size, compared to C#.

How mature is fluent-ui compared to the native winui3?

Since I am not too deep into this package, I don't want to lean too far out of the window, but as far as I know, this is a pretty well supported project. It has been around since a long time, so it already had years to mature and as far as I know is the best community build alternative to Flutters native Material Design. In my time using it, it felt pretty inline with Flutters design principles. Also, it is a Flutter Favorite, which is a title given to packages that are recommended by the Flutter team itself.

There is a demo available in the browser, maybe you have already seen it: https://bdlukaa.github.io/fluent_ui/

One of Flutter's superpowers is that you have control over each pixel on screen. So if something is not to your liking, it is always possible to do something about it.

Does it also support themes to the extent I did in the original c# version

I did test your c# version, but I never noticed the theming feature, because when I clicked on the settings symbol, the application just crashed... 😅 I commented out some lines, so now It works.

Yes, theming is one of Flutters main features. In Flutter, the theme is a set of fixed properties that widgets inherit from its ancestors. If a property does not get defined explicitly, it will be used from the theme. The app's theme is separated into multiple themes, usually per widget type. So there is a ButtonTheme, SliderTheme, ScrollbarTheme and so on. This is the main design principle behind themes. But to which extent this is implemented, of course, depends on the design library used. But as far as I can tell, this is implemented pretty well in fluent_ui.

For the window effects, there is a package called flutter_acrylic . It has a wide variety of window effects to use. I never used it, but this is definitely a possibility.

I'm most likely to want to do some Rust->Dart ffi via a dylib

Dart has native support to load and call dynamically linked c-style libraries. It is called Dart ffi. You can read more about it here. There is still active development around this feature, though it is a proven solution. Dart ffi is the basis for any interop with other languages in Dart (If you factor out the traditional way in Flutter using Method Channels, but they are slowly phased out).

Using a package called ffigen, you can generate function bindings using a C header file.

Also, there is a package to generate Rust bindings directly, called flutter_rust_bridge. I don't know how mature it is, since I never used it, but it looks to be pretty good. It is a Flutter Favorite too.

I see there is some c++ code in the repo.

Flutter always generates some platform specific code, that can be found in the platform specific subfolder. For example, in the windows subfolder, you can find a CMake project that is used to build your app. This project also handles window and engine initialization, which is a good thing, because this is in control of the user! But there is absolutely no need to do anything here. It is just that you can do something, if you need to.

There are a few things under the hood that the gui app is expected to do in order to work with the driver.

I want to emphasize on the ability to integrate this driver into other projects as well. This means that any logic handled outside the driver must be either very simple, or must be implemented in some form of library and abstracted away from the library user.

I like the idea that the client only deals with the IPC. Is there any disadvantage, when the driver writes to the registry itself, or any benefit, if it doesn't?

[MolotovCherry]

Also, there is a package to generate Rust bindings directly, called flutter_rust_bridge. I don't know how mature it is, since I never used it, but it looks to be pretty good. It is a Flutter Favorite too.

I took a look at this, and the project looks like a viable path forward. The project is actively maintained, well developed, well used, and has a lot of attention (and makes ffi easier). I can handle all the ipc, callbacks (and even registry stuff) in the plugin and make it easy to use on the dart side, so the app can focus mostly on the actual ui.

[...] It is just that you can do something, if you need to.

Sounds okay then!

I want to emphasize on the ability to integrate this driver into other projects as well. This means that any logic handled outside the driver must be either very simple, or must be implemented in some form of library and abstracted away from the library user.

I like the idea that the client only deals with the IPC. Is there any disadvantage, when the driver writes to the registry itself, or any benefit, if it doesn't?

Putting some logic in a library is pretty easy to do (and in fact, I prefer it since it cleans up the logic in a nice area, without having to try to reinvent things in new languages; hmm, on a another thought, maybe I should crate python bindings to make it scriptable).

In any case, the reason the driver doesn't do it is simply because it can't. Drivers are heavily restricted in what they can and can't do with the registry, and it's very complex when it gets to writing (at least for umdf drivers, not sure about kmdf drivers). Reading however is easy enough to do. However, in a separate library, it's very easy to write to the registry, so handling this elsewhere is pretty ideal compared to the alternative.

---

After taking a look at this and the possibilities, there looks to be a viable path forward! I think I am interested yes. And I'd be happy to collaborate and take care of all the Rust ipc/driver logic side of it.

[Dampfwalze]

OK, nice! Let's do this then!

the reason the driver doesn't do it is simply because it can't

That's a bummer! But Ok.

maybe I should crate python bindings to make it scriptable

I think C bindings would also be beneficial as well, since C-style ABIs are the golden standart to make interop possible and almost any language has some form of support for it. Maybe there is a somewhat simple solution using cbindgen? I know that it is always a hassle to do, when you have to pass around dynamically sized data structures... Or it may be easier to create C++ Bindings instead using the cxx rust crate?

I imagine a core IPC library written in Rust, which has different language bindings built on top of it.

I propose the following roadmap:

• Create IPC library
  • Create API spec for core library, Dart and C
  • Build core library
  • Create Dart bindings
  • Create C bindings (low priority)
  • Create Python bindings (low priority)
  • Create ... bindings (low priority)
• Create user interface
  • Create UI spec
  • Build app

The API should be unbiased from the actual IPC protocol.

The pipe name should not be completely hardcoded into the library, but rather default to the standard name.

Do you want to notify the client of changes in the configuration or other events? Maybe you can support multiple clients. So for example, if you use the Control Panel and the CLI at the same time, every client gets notified when the configuration changes. This also ensures that the state of the Control Panel always reflects the correct state of the driver.

I can do the API spec for Dart.

[MolotovCherry]

Sounds good!

I'll try building out a core ipc library this week. In my off time before I make the implementation, I'm working on thinking through what the easiest and most reasonable implementation would be like; if you have any ideas as a ui designer of an api that's most useful to you, that's also helpful. I'll get back to you once I have more thoughts and ideas on it.

(Planning a nice api takes a majority of the time. The actual time it takes to write an implementation would be fairly fast, as it's not actually that difficult. Bindings aren't that difficult either considering)

Also, feel free to ping/message me on the discord server if you have any more thoughts/questions!

[MolotovCherry]

Maybe you can support multiple clients.

I don't plan to support this right now, because, while possible, in practice it can get quite messy. And in the end, it requires the client to properly handle things. When you get to the scripting side, it gets even more ambiguous. If the client can assume that whatever information it obtains never changes after requesting the state, it makes things easy.

I can do the API spec for Dart.

You mean like what kind of api you need? That would definitely be helpful to help me plan the best way to structure the api for your use-case

--
Also, I'm sorry it's taking so long. I've been busy with so many things. In either case, knowing what api structure would be useful to you would be a help for sure! Once you let me know the api structure that suits you, I can get right on a workable library for your case

(I've also converted this to an issue report @ #101 )

[Dampfwalze]

I have build a prototype for a Dart library and Flutter application in Dampfwalze/virtual-display-rs at dart-vdd-prototype. It has the following structure:

root
├───rust
│   └───dart_vdd                        # Rust - Dart bridge (Dart package)
│       ├───example
│       │   └───dart_vdd_example.dart
│       ├───lib
│       │   └───src
│       │       └───generated
│       │           └───api.dart        # Dart API generated from Rust
│       └───rust                        # Cargo project
│           └───src
│               └───api.rs              # Rust API
└───vdd_control                         # Flutter app
    └───lib
        └───main.dart

There are a few points of interest:

• rust/dart_vdd/rust/src/api.rs: This is where the library is implemented on the Rust side
• rust/dart_vdd/lib/src/generated/api.dart: These are the generated bindings by flutter_rust_bridge
• rust/dart_vdd/example/dart_vdd_example.dart: A (non exhaustive) usage example

I used a Dart feature called native-assets to build and link the Rust library, which is currently not stable and behind an experimental flag. It is essentially a build.dart file, similar to Rusts build.rs. Traditionally, it was not possible to distribute binaries with just Dart. Flutter has its own mechanisms for this, but then, the package can only be used in Flutter projects. The native-assets feature is meant to replace these mechanisms in the future. But for now, it is only available in Flutter on the main channel (like nightly in Rust).

The API uses a Dart Stream as the main way of communicating the driver's state (as an immutable data structure). It would stream any changes made to the driver, regardless of who did it, so it would always reflect the correct state of the driver. This pattern is quite native to Flutter to build on top of.

I saw that you tried to create Python bindings. You have built all the logic directly into the library. I think there should be one core library, written completely in Rust, that we can build bindings on top of. So the bindings are just bindings and have no logic themself and are only forwarding calls to the core library.

(I've also converted this to an issue report @ #101 )

Should we move over then for discussions?

[MolotovCherry]

Hi! Thanks for this!

Would you like to pr what you have so far for easier collaboration? (I can't really test things until then)

Thanks for the api stubs you have. I'll go fill those out.

Everything looks good to me.*

And regarding the API. I do plan to make it a core library. But for now, my focus is to just make something quick that works (while keeping the API the same). Since we have some API stubs, you can expect any changes to be internal only.

And yeah, discussing this on the issue instead would be much preferable!

• One note, I'm likely to change the modes struct and put a unique hash per mode (for refresh rates too). The reason why is that I realized that there's no way for library users to keep track of what's what, especially with inserting, deleting, (and saying which specific mode/refresh rate to remove). Using indexes is a pretty bad and error prone way.