FYI: Comment and question

[PallHaraldsson]

Structure of the packages part of the Lux.jl Universe[1]

The graph below doesn't display well, for me at least. I enabled dark mode in my web browser and can't barely see the text, or that there was text at all. I assumed there would be and zoomed in to see it. This is going to become more and more common, so I would wish worked better, not just for your repo, and people test both ways... I used some hack to get dark mode in Microsoft Edge, I don't even recall how, thus how to turn of to look at your graph...

Related Projects
Flux.jl -- We share most of the backend infrastructure with Flux (FluxML/Flux.jl#1829 hints towards making Flux explicit-parameter first)

I know you share backend infrastructure, are more similar than different(?) but "explicit-parameter" is the main difference? And then if Flux going there, to be more like Lux, then no very large difference? I wanted to reference your package: "Flux.jl and Lux.jl", starting with Flux since more established, but now thinking of doing "Lux, Flux and BetaML" (if I knew a clear winner I would just reference your). I thought BetaML more for educational, not even sure by now, maybe a competitor too (or complementary? or just for different things, at least partially?)? At least seemingly very valuable, should you also list it under related?

[avik-pal]

The graph below doesn't display well, for me at least.

The graph was made using Github and mermaid.js integration, so it is not something we can control. Also, it renders fine on Brave & Firefox.

so it is probably a edge problem. Regardless, I will remove the graph soon, because hasn't been updated over time.

I know you share backend infrastructure, are more similar than different(?) but "explicit-parameter" is the main difference?

Even if flux moves to explicit parameters (in the Zygote sense of the term), it is still via a nested functor style. That doesn't play well with any external framework (unless you use Optimisers.destructure). A lot of work has gone into making that work, but it still have fundamental problems of dealing with non-standard eltypes. See https://discourse.julialang.org/t/comparison-to-numpyro/106495/7 for example where Flux takes 10hrs compared to 7.5 mins of Lux.

Additionally, Flux has some strange internal mechanism to override user inputs and do what it thinks makes the "most sense" (which is great for complete beginners but after getting some experience it is a pain trying to override that behavior), but that means you can't switch AD backends easily, and most importantly, makes it undesirable for anything that is not routine (definitely not new research).

SciML repos mostly have Lux first-class support (Flux models are converted to Lux for compatibility https://github.com/SciML/DiffEqFlux.jl#diffeqfluxjl).

Also, many features (that are quite popular among users) like @compact (FluxML/Fluxperimental.jl#12 & #540 & #516 (comment) -- see these for the general comments on @compact being useful) never get "shipped" in flux -- rather they are put in repositories like https://github.com/FluxML/Fluxperimental.jl which no one really knows about. So I would say they don't do a great job of pushing new features in.