Move away from browserify?

[outofambit]

While working on #3385, (and from discussions in #3409) I can't help but wonder if this project would be better served by a babel + webpack build process over the current browserify + uglify infrastructure.

I think we can replicate the existing behavior with babel, using babel-preset-env and babel-minify, and webpack to do the bundling.

We might also be able to use something as simple as rollup, though I am less familiar with that tool.

What do y'all think on a migration like this? Are there other tools worth considering?

[limzykenneth]

What would be the benefit of moving from browserify to webpack or rollup? What would be the drawbacks, if any?

[bigmstone]

Disclaimer: I am a webpack/babel user which artificially inflates my desire to see it in other places. I also help maintain a gulp/browserify app as part of my day job, and honestly don't have much bad to say about it. It's mainly set it and forget it. We have all the features I would want from a normal webpack setup. Including hot reload. Also use babelify, so we have access latest language features.

One weak(ish) reason to switch to webpack would be mind share and development resources. webpack has orders of magnitude more usage based on GH stars (meh) and NPM downloads (less meh). Having more usage will lead towards the usual suspects: trend towards having new features, able to respond quickly to competing features, quick bug discovery & patching, etc.

I just low-key want to help P5 get better ES6/"Instance" support since ES6 actually made me "okay" with JS. So mainly +1 to bringing babel into the mix. Either through webpack + babel or existing build system + babelify.

[Spongman]

with webpack, how easy is it to run custom build steps? for example, something like the grunt task defined in eslint-samples.js ?

BTW: there's an example of a babelified, ES6 version of p5 over here: #3394. still uses grunt, but hey.

[limzykenneth]

If there isn't a strong benefit to switch build tool then the effort may not be worth the reward so that's why I'm asking.

Out of curiosity, because I haven't have a project that uses webpack or rollup but I want to try, how do they integrate with test runner like mocha or linters like eslint? Do they run with plugins on top or are they completely seperate, ie run with two shell commands (or one npm script with something like &&)?

[bigmstone]

Out of curiosity, because I haven't have a project that uses webpack or rollup but I want to try, how do they integrate with test runner like mocha or linters like eslint? Do they run with plugins on top or are they completely seperate, ie run with two shell commands (or one npm script with something like &&)?

eslint will take care of itself. It doesn't transpile prior to linting since it's linting the untranspiled code anyway.

Tests all depends on the test suite, but with mocha there's a plugin or you can use a script as you described.

with webpack, how easy is it to run custom build steps? for example, something like the grunt task defined in eslint-samples.js ?

"easy" is a hard word to nail down. 😁 It's done by building a plugin and referencing that plugin in your set of build rules. See eslint-loader as an example, and how to build a plugin on webpack's wiki. I will disclaim that I've never built in a custom build step. So YMMV.

[outofambit]

with webpack, how easy is it to run custom build steps? for example, something like the grunt task defined in eslint-samples.js ?

@Spongman webpack wouldn't replace all of the scripts in the build folder, it would mostly replace build/browserify.js and build/combineModules.js. I think eslinting should be separate, so I'm not sure we need any custom build steps? But maybe I'm missing something.

What would be the benefit of moving from browserify to webpack or rollup? What would be the drawbacks, if any?

@limzykenneth I agree with @bigmstone, I think the main benefit is that these tools are have a larger ecosystem. They would also be easier to integrate with tools like babel, which may be useful with some of the es6ing coming up.

There are also some parts of our build process that would be simplified. Right now we use derequire as an additional build step after browserify, which wouldn't be required by a rollup or webpack setup. There are also rollup plugins for minifying that would remove the uglify grunt task entirely.

I think the main drawback of webpack is that the learning curve is a bit steep and configuration can be challenging. Rollup and Parcel (which I forgot to mention earlier) have significantly less configuration (mostly because they have sensible defaults) and are much more approachable.

[limzykenneth]

When I was looking through the dependencies to find obsolete ones derequire do stuck out as something I don't understand why it is there (still don't, really). I have heard of the complex configuration file of webpack and grunt seems to be comparable in terms of config complexity. Parcel on the other hand was something that I dipped my toe into recently and only gave up because it doesn't support Vue out of the box, however, if Parcel can deliver its promise of zero configuration for our build I think that will be a great improvement, we don't need to write new configurations to migrate over and the build process cannot be simpler. I don't know how Rollup compare to Parcel but I'm prepared to support something that significantly reduce the complexity of the dev setup with minimal effort to migrate.

---

On another note, I've seen Babel being thrown around in these discussion, and maybe I'm not familiar with some aspect of it, what I don't understand is why do we need it? Most of the biggest ES6 features are already supported by all major browsers for several years and so well covered by our compatibility targets (discounting IE 😞 😠 ) so I assume the aim is not to transpile to ES5?

Then if I'm not missing anything else Babel will then be used to transpile features not yet widely implemented by browsers to something current browsers can run, in this case I would be more wary. I see the source code of p5.js as a learning resource for everyone including beginners as well but if we include code that doesn't run in the user's environment out of the box when they copied it, it becomes a source of confusion and another barrier to entry because now they need to know about Babel before continuing.

[bigmstone]

what I don't understand is why do we need it?

There are commonly used features that haven't been fully standardized yet. Field declarations being one. Now that the industry seems to have centered around babel this will likely be a "norm" for the foreseeable future as the language evolves. It's also easy to get language feature support for lots of browsers with little effort without needing to manually check against your supported browser list. You just build to what babel supports and sleep well.

in this case I would be more wary. I see the source code of p5.js as a learning resource for everyone including beginners as well but if we include code that doesn't run in the user's environment out of the box when they copied it, it becomes a source of confusion and another barrier to entry because now they need to know about Babel before continuing.

I can see your point, but we can all admit javascript has historically been fractured. Take your pick: dependency management, build tooling, framework flavor of the week (I'm looking at you jquery, angular, react, vue), browser support circus. So I can appreciate what the popularity of webpack/babel has brought to the JS community. Namely removing the browser circus. Sadly we're still fighting a few of those (node/yarn, vue/react).

Rereading that comment I think you might not be talking about beginners working on the library (which 1. would likely not be realistic and 2. babel would be "abstract" from them by the build process), but rather consumption of the library. My assumption with ES6+ support w/ babel is the library would still ship a transpiled distribution. So a beginner can use P5 as they always have.

[lmccart]

I don't currently have a strong opinion here, I need to do some more research into the different options before I can weigh in more substantively. But I'm excited about exploring possibilities for improving the build system. One note @bigmstone, it is indeed a goal that beginner programmers find the p5.js ecosystem friendly enough to make contributions. This is different than a lot of software projects, and requires us to make certain decisions privileging readability over software extreme heavy lifting. We try to find a balance. (Sidenote: this makes me realize that the list of goals we hashed out in a meeting last spring never made it to the github, so I need to work on getting this up there! Currently, there's not any way to know the extent to which we aim to make this beginner accessible.)

[bigmstone]

I can appreciate that and I can see both sides. On one hand it might be worthwhile being opinionated on some tooling. I find newcomers to JS get overwhelmed by all the options and are unsure where to start. So they read an article from a few years ago and are convinced jquery and bower is the way of the future. On the other hand being able to drop into an editor and in 5 seconds have something running was one of my favorite first impressions of languages like Go.

[outofambit]

There's a lot of other things we could have babel do, but I think the strongest case for babel in p5js is to ensure backward compatibility with older browser versions (or ones that are the minority in not supporting an ES syntax/feature) using babel-preset-env.

Supporting a range of browsers is much easier with babel-preset-env. When added to your build process, it will polyfill and transpile your javascript to work on the browser versions listed in a configuration file (which we maintain as part of our repo). It's super cool and would allow us to embrace whatever features of es6+ we find appropriate without limiting the library's (browser) reach.

---

On the topic of build processes and readability: I did a quick proof of concept in #3429 to explore what using rollup might look like. There's still a couple details that need to be worked out, but I think the core is there? I find removing the need for a separate uglify and derequire step is pretty compelling (simpler and easier to follow), but I would love to hear what others think! 💭

(Also, the promise chain at the end is a little wonky. That could be cleaned up with async/await but I don't think grunt supports promises, so we could just separate the .then's out instead of chaining them all at once like I did.)

[hydrosquall]

One key benefit of rollup over parcel that I haven't seen mentioned in this discussion yet is that because of techniques like tree-shaking, it has the potential to grant significantly (10x) faster build times + smaller bundle sizes.

https://x-team.com/blog/rollup-webpack-parcel-comparison/

[limzykenneth]

@hydrosquall Pacel also has tree shaking support albeit experimental still, and Parcel also caches builds so according to them provides even faster builds, not sure how it compares for us though.

[coreygo]

I've too have been trying to reduce the size of p5 as much as possible for a single sketch which uses very few of the features in p5 and am limited by the current build process.

I'm also having a separate issue using p5 and Gatsby together. I've tried various workarounds as part of the debug process but no such luck in my case… @Aendrew has also mentioned this in #2244 and #2503—with additional general discussion in #522.

It seems using rollup, webpack, or parcel would help.

I thought maybe if I tried npm run grunt combineModules:core/shape:color:math:image that would do the trick but the generated p5Custom.js is larger than the complete p5.min.js. 😕

The current grunt task for combineModules doesn't also create a p5Custom.min.js using uglify like the normal grunt build task. 😉

[lmccart]

@coreygo have you tried this? p5.js developer docs: creating a custom build of p5.js with select components

[coreygo]

@lmccart yep… 😃 I was actually about to update — I've created a pull request for that doc since it's not currently accurate.

• grunt fails, it should be npm run grunt combineModules
• core/shape* isn't included as part of core by default — so 2d_primitives like line() won't work
• npm run grunt is a missing step after npm install, without that step running npm run grunt combineModules fails with Fatal error: Cannot find module '../../docs/reference/data.json' from '~/Projects/p5.js/src/core'

[outofambit]

hey all, #3431 is ready for review as a first step to simplifying the build process with parcel!

[lmccart]

As discussed in #3431, we've explored the options and unfortunately none of them provide what we need right now. parcel seemed like the best fit, but because of transpiling limitations it doesn't suit our purposes. I'm closing this for organization, we can revisit in the future as the state of the web evolves :) Thanks all for the discussion and @outofambit for investigating the parcel solution and other alternatives.