Redesign template from ground up, with new framework

[vincerubinetti]

Introduction

Lab Website Template (LWT) was originally built upon Jekyll, a tool for generating static websites. This was chosen at the time due to its ubiquity and its convenient "automatic" integration with GitHub Pages. Otherwise, Jekyll is quite old and limited, and has many other shortcomings discussed below. With the release of v1.0.0, the template moved to a "manual" GitHub Pages integration, which, while opening up many new possibilities, essentially removed one of the main benefits of using Jekyll. This would've been an ideal time to switch to a completely different tool, but the opportunity was missed. Though, now we have the benefit of more lessons-learned that we can incorporate into a new design from the start.

The goal of a redesign would be to make the template even more simple, flexible, robust, and modern. It would be based on Astro to hopefully alleviate most, if not all, of the problems with Jekyll. It's uncertain when or if we will have time for this redesign, but this issue attempts to document and discuss the motivations and ideas involved. Please leave a 👍 if you would like to see this redesign happen, or a 👎 if you are not interested and don't think it's worth the effort.

Eleventy was another considered option, but was decided against. Astro is more inline with modern web-design paradigms, making things like writing reusable components, templating with JSX/MDX, and having type safety more clean and easy. Astro's developer experience is top-notch, building most of these things in, whereas Eleventy requires more configuration and plugins (some still in alpha) to achieve the same functionality. Eleventy is also older (though not as old as Jekyll) and seems to have fewer users than Astro (going by npm installs and GitHub stars).

Next, Nuxt, Remix, and other similar tools were also considered, but Astro was deemed to be more simple (better DX, batteries included), more flexible (can use any frontend framework), and more suited to the particular use case of the template (95% static content with occasional interactivity) with its "island architecture".

Current Design (Jekyll)

Benefits

Jekyll is established

Jekyll was first released in 2008 and has likely been used to create millions of static websites. There's a large ecosystem of plugins for it. However, Astro has a comparable and still-growing ecosystem of plugins and users as well. See: Jekyll installs over time vs Astro installs over time.

Jekyll integrates with GitHub Pages automatically

…if you're willing to use an incredibly stripped-down version of it. As aforementioned though, the template no longer takes advantage of this.

Challenges

Too many languages/technologies

• Ruby - for running Jekyll, managing packages, and writing custom Jekyll plugins
• Jekyll - for compiling site
• Markdown - for writing content
• Liquid - for using components in content and other templating
• HTML - for writing components and layouts
• JavaScript - for interactive plugins
• CSS/Sass - for styling
• Python - for cite process scripts
• Docker - for running entire stack consistently on any platform, with one installation

Even though we try to hide under-the-hood complexities from the user as much as possible, having so many pieces to the puzzle adds mental overhead. More things can go wrong, and the user has to understand more disparate technologies to troubleshoot or customize things. It also makes maintaining the template more difficult.

Ruby tooling is bad

Most of this is opinionated and anecdotal, but some of it can be supported by documented issues in the Jekyll and template repos. The ecosystem of tools around Ruby has several oddities:

• Installing Ruby itself can be complicated, especially on Windows, and instructions and resources for installing vary between operating systems.
• Installing Ruby packages ("gems") requires a separate "bundler" tool with its own setup. We have run into many frustrating and confusing issues like: incompatibility between bundler version and lockfiles or gem versions, installation permission restrictions, behavior differences when running on CI vs docker vs natively, and more. MacOS has a system installation of Ruby that the system uses, which can cause conflicts.
• The Ruby lock file is often intended to be edited (unusual), e.g. to target specific platforms (a problem in and of itself). This makes package management a bit more confusing.
• Several sub-dependencies/gems of Jekyll have long-standing platform-specific issues, especially on Windows, requiring special workarounds that add extra complexity for the template maintainers and/or users.
• Installing Jekyll is sometimes incredibly slow, sometimes > 1 minute on CI. Perhaps this is innate to Ruby, or due to the particular dependencies of Jekyll.

Jekyll is abandonware

It may be too harsh to call Jekyll "abandonware", but because web development is a field that moves so quickly, a framework needs to stay very up-to-date to be competitive. The fate of Jekyll is uncertain and its official commitments are a bit unclear, but judging by the hundreds of unresolved issues and feature requests that are closed (candidly attributed to lack of resources), it only receives the most critical of bug-fix updates. No new features or enhancements.

Compare this to Astro, which is very actively maintained (as of writing).

Liquid not flexible enough

Liquid is not a full-fledged templating language, it only supports very simple templating. We have found that users ultimately need much more than Liquid provides. We need flexibility of expression that only a full programming language can offer.

To make matters worse, Jekyll has its own special "flavor" of Liquid that it implements, which is different from the "official" Liquid docs in some cases, causing hard-to-debug issues.

Hard to lint, format, syntax highlight, etc.

There are tools that can lint, format, and syntax-highlight the Liquid language, but being able to do those things in a single file that has a combination of Liquid + HTML + Markdown proved to be impractical (perhaps impossible) after some experimentation.

Lack of Jekyll subject-matter-experts

Though Jekyll has been around for a long time, it would still be easier to find someone knowledgeable in general web technology (which Astro stays close to) than someone knowledgeable in Jekyll. Getting people to review or maintain a complex Jekyll project can be difficult. It is particularly difficult with the current team setup of the template maintainers. The template has odd workarounds/hacks that are not very transparent at first glance and would require historical knowledge to understand the importance of.

New Design (Astro)

Benefits

Same language/technologies for everything

• Node - for running Astro and managing packages
• Astro - for compiling site
• MDX - for writing content
• JavaScript - for interactive plugins, components, cite-process scripts, and any other code
• HTML - for writing components and layouts
• CSS - for styling
• Python - installation only, needed for running running Manubot

Since the ultimate product of the template is a website, we feel it makes sense to use web technologies and languages wherever possible. For consistency and less mental overhead, JavaScript will be used in basically any case where coding/scripting is needed. While the template's primary audience may not know JavaScript very well, it is only one language they would need to learn to make customizations, compared to Liquid + Ruby + Python + JavaScript.

Regarding users writing their content: MDX is essentially a combination of HTML, JavaScript, and Markdown that is hopefully intuitive enough for users to learn rapidly. Nesting content, a common need, should become easier and clearer with MDX as compared to Liquid capture tags.

Since the maintainers of the template overlap with the maintainers of Manubot, we may also make an attempt to maintain a public Manubot API that could be queried over HTTP. This would mean the template wouldn't even need Python installed, and instead could just make a request to e.g. manubot.org/api?cite=doi:12345 to get a citation. It would also mean someone would need to make sure to maintain this service in perpetuity.

Better tooling

Astro is based on Node, which should avoid the problems of Ruby's tooling. Node comes with a package manager, so no separate installation or setup is needed. Node also in a sense acts as a compatibility layer with the operating system. Many (if not most) Node packages are regular JavaScript (non-compiled) that Node translates (interprets) into OS-specific code as appropriate, meaning that there should seldom be problems with any plugin/package/sub-dependency working on one OS but not another. By experience, Astro and its sub-dependencies work exactly the same on any platform without hiccups.

Because of Node's good cross-platform compatibility, a Docker container will hopefully not be needed any more. A strong effort will be made to have the template (using Node scripts) automate the installation of Python (for running Manubot) so that Docker would not be required.

Type safety

If using VSCode, users should automatically get type safety and type hints. They'll be able to immediately tell if they're using a component and mis-typed a parameter name or gave it the wrong type of value. They'll also be able to get some quick explanation of what each parameter does as a popup, reducing the need to reference the official template documentation.

This would be a massive improvement, and solve many issues before they even become bugs, and answer user questions before they even go to ask on GitHub.

Easy linting, formatting, syntax highlighting, etc.

Many tools and editor integrations exist to lint, format, and highlight MDX. Users will be able to get rapid feedback on their code and keep it consistent and readable.

This would be a massive improvement, in the same ways as type safety.

A component can be one file

In the current design, a component's HTML/CSS/JS have to be split into separate files in _includes/_styles/_scripts, respectively. With Astro, all of that can be encapsulated in one file, i.e. a single file component. This makes customization and updating more easy for the user, and maintenance easier for the template maintainers.

It is not practical to do this in Jekyll without a lot of duplication and performance issues in the compiled site, or without complicated plugins.

More control over separation of content

With Jekyll, certain things have to go in certain folders and have certain names. This mixes and blurs the distinction between what is "template content" ("under-the-hood" stuff needed for the general functioning of the template) and "user content" (stuff for your specific website).

Astro allows more flexibility, meaning we can arrange things in such a way it's more clear which is which. Users will have an easier time focusing on what is really important to them (their user content), and updating the template version could be simpler, perhaps even automated (see below).

Cite process can be more seamless

Astro makes it easy to write plugins that can hook into its lifecycle steps. That means that instead of the cite process being a separate script that is run completely independent of building the site, it can become an actual part of the build process. This makes many things easier, like watching for updates, making sure the site is built any time citations change, CI workflows, etc.

Jekyll can do this to some extent too, but in a more limited fashion and with worse DX. The cite process script would also need to be written in Ruby.

Citations and other data inclusion can be simpler

In the current design, the cite process is very much structured around Jekyll's limitation that data must be in the \_data folder as a single file to access it from content and loop over it.

With Astro, it should be possible (if it is desirable from a design standpoint), to allow users to import and loop through data from anywhere in the repo. For example, perhaps you could specify ORCIDs in-situ, like <Citations orcids={["xxx-xxx-xxx", "xxx-xxx-xxx", …]}/>, and the template would just generate the appropriate citations right there, without the intermediate citations.yaml compiled list (but still having a global cache shared across pages). Or, a list of manual sources could be in a yaml file collocated in folder of the page its meant to be displayed on.

The exact syntax and mechanics of how this will work will need much experimentation, but it seems like there is much opportunity for improved intuitiveness and flexibility.

Script for updating template version

Updating a user instance of the template to the latest version has always been difficult, even for the maintainers of the template. This is not a trivial task, due to being a template and not a package as discussed in the docs, and is very error-prone. However, a strong effort will be made to make a script that does this automatically for the user. The lack of such a script in the current design isn't completely due to Jekyll shortcomings, but it is hindered by them. With the other new design benefits discussed above, writing such a script should be more doable.

Accessibility and other important testing

This isn't prohibited by the current Jekyll design, but it would require adding yet another technology to the stack: Node/npm. With the new Astro design, we'll already have these things.

With Playwright and Axe, we can essentially automatically run Lighthouse accessibility tests on each one of the user's pages. This can help them catch if they've written content (or made an under-the-hood modification to the template) that breaks accessibility on their site.

We could also just use Lighthouse fully to also test for performance and such.

Easier integration with React/Vue/etc. libraries

Using Astro may open up a new world of possibilities for users to integrate existing components/widgets/whatever that were written in React, Vue, Svelte, or other web frameworks. A strong effort will be made to make the installation and use of these libraries in the template easy.

You could technically do this with Jekyll, but it would be impractical.

Better CSS

• More use of flexbox and grid to improve the consistency and flexibility of layouts.
• More use of variables to aid in user customization.
• Various lessons learned from years of recent CSS experience.

With modern additions to native CSS, SASS is much less necessary, and can be ditched, reducing a bit of complexity.

This improvement isn't really due to the limitations of Jekyll, but just by the nature of doing a ground-up rewrite.

Faster installation, dev preview, and builds

Instead of a bulky Docker container, Ruby/Jekyll's slow installation and build times, and Jekyll's outdated dev preview mechanisms, we'll have speedy installations/previews/builds with Node/Astro/Vite.

Better multi-language support

Astro seems to have internationalization built-in, whereas Jekyll needs a (now deprecated) plugin: https://github.com/kurtsson/jekyll-multiple-languages-plugin

Challenges

"Lab Website Template" is an established "brand"

The template has hundreds of forks and stars. Getting many of these users to migrate to a new and very different template will be difficult. The new design will be so different that we should probably call it by a new name, which will likely hinder adoption even further.

It's also hard to think of a good/appropriate new name. Suggestions welcome.

Users will not make the effort to migrate

By observation, most users do not update their template version once they've forked/generated their instance of it. It will be even harder to get people to upgrade to a completely new design.

As such, a strong effort will be made to make a special v1 to v2 migration script that does its best to automatically upgrade them. It should be possible to automatically translate Liquid syntax to MDX, migrate image assets and data lists, and etc.