DX: Implement DX for second contact

[BioPhoton]

The "second contact" focuses on more advanced tests and CI integration.

It should focus on a good user experience from the CLI perspective, as well as a smooth setup of the GitHub action to integrate user-flow into the CI. This means the user should be guided by good documentation and maybe a command with questions and options to set it up.

In general this issue targets direct users of @push-based/user-flows, developer, performance engineers etc. that have user-flow already installed and running, but not implemented a real live integration.
This story is a follow up on #32

Furthermore it will help to separate the CLI and the GH action repositories.

Related repositories:

• 💻 the CLI - @push-based/user-flow
• ⚙ the CI Action - CI related code e.g. GitHub Action @push-based/user-flow-gh-action

Persona

Personal Data

Chris, 26, full-stack developer, self educated

Background

Needs an easy way to stop regression on an ongoing JS application. He tried user flow already #32 and remembered it after another uncaught regression in his live system.

Knowledge

He knows lighthouse and puppeteer, also knows user-flow.

Environment and Tooling

There are no performance tests in the application
There are very few tests in the CI

Second contact

Chris already tested out user-flow and was impressed by this new method of testing the performance of user actions. However, he only tested the basics and left it at that point as there was no more time.

Some 1,5 month later, while testing the performance of the application he was working on, he noticed the performance of the application got worse and he had to figure out why.

The app is pretty busy maintained from many developers and there is no good release management in place to track changes, nor any continuous performance measurement, so it was quite a hassle to spot the issue.

After all the issue was found, and additionally, not only the performance that had gotten worse, but had also a couple of SEO critical bugs introduced.
What a nice ending of that Friday... and hello to a weekend of cleaning up others mess.

In the middle of the next week, not quite recovered from the work intense weekend he started to reason about possible solutions to this unfortunately recurring problem and user-flow jumped in his mind again.

He suggested to set up using user-flow to avoid regressions, but the conversation with his CTO was unpleasant.
Just by looking at some GitHub repository and no clear time estimation on how much it takes to get it running it just got zero attention next to all the high priority tickets in the queue.

But Chris's curiosity got the best of him and he keep playing around with it, knowing that he could convince his CTO to implement user-flow if a PoC could catch some regression in the CI.

Problems from test run

To fix for second contact

• Using puppeteer to manipulate the application was more time consuming that expected - fixed by Include puppeteer/replay to execute steps from a .json export  #66
• Every time I wanted to add a test I had to run the whole test white which took way to long. - fixed by Add can execute a single user-flow file to documentation #141
• I still had no idea how to integrate it into the CI - fixed by Include puppeteer/replay to execute steps from a .json export  #66
• New results overwrote the previous ones and didn’t allow me to check if there was a difference in results - fixed by flow file name should get derived from url #109
• I wasn’t really sure where to put files and what should be where and realized it would eventually become a massive code base or a bowl of spaghetti code - fixed by Update ufo-architecture.md #64
• If I wanted to import recording from chrome new recorder I had to introduce a bunch of new functions and didn’t know where they should go, also since they are written in js and not TS I ended up just adding //@ts-ignore all over the place since it would require effort I wasn’t willing to invest at the moment to fix the type errors - fixed by Include puppeteer/replay to execute steps from a .json export  #66
• A couple of times I had to change puppeteer configurations and just kind of dumped the new configs in the file. - fixed
• ~generate multiple formats - fixed by fix: use all persist format items in array #181 ~

To fix for third contact

• Some action required the user to log in and it added unnecessary steps, plus I could figure out how to maintain the user logged in between different user flow test runs - fixed by docs: authenticated pages #20
• To set up a new test I ended up duplicating a user flow file and modifying it.

DX to Implement

• 💻complete docs
  • Add details to command docs  #88
• ⚙ GH actions - Roadmap user-flow-gh-action#9
  • 💻⚙ good docs on CI integrating
    • feat: extend init param with file creation for gh action #198
    • Create detailed docs on the setup user-flow-gh-action#86
  • 💻 command to generate GH action e.g. init --action
    • feat: extend init param with file creation for gh action #198
• ⚙ get information about the performance metrics of a PR in the comments - feat: Implement the comment interaction user-flow-gh-action#10

Next steps

• Add more docs for LH configuration #161
• ⚙ run it against the main branch to check
• 💻 get a warning / error when a PR causes performance regression that below the budget
• 💻⚙docs for authentication docs: authenticated pages #20
• 💻 setup scalable testing architecture
  • add a test
  • add a page
  • add a UI element
  • add fixtures
  • add budget