@grafana/toolkit - implement common tooling for plugin developers

[dprokop]

The work started in: #17445

TODO:

• Enable building app/plugin using shared rollup config
• If there is more than one module.ts entry then add manual chunks
• Enable running tests on app/plugin
• Introduce common tsconfig
• Introduce common tslint config
• Do not expose internal cli commands
• Validate plugin name/org
  - OK for plugins in grafana repo to skip the org
• Run typecheck before linting
• Add watch option to enable easy development (rollup watch, plugin:[dev|watch])

Areas to cover

Plugin development tasks

• grafana-toolkit plugin:test - performs tests for a given plugin
• grafana-toolkit plugin:dev - runs plugin in development mode, meaning that Rollup is run in watch mode
• grafana-toolkit plugin:build - prepares plugin distribution package
  • Runs linter
  • Runs tests
  • Bundles plugin
• grafana-toolkit plugin:create - creates plugin seed from a template (grafana/toolkit: Introduce scaffolding task #18506)

Testing plugin

• Make sure toolkit allows working with enzyme/react-testing-library
• Enable custom Jest config (via package.json file as a start)
• Enable running test task in watch mode (grafana/toolkit: plugin:test improvements #18137)
• run a single test from command line (@briangann) (grafana/toolkit: plugin:test improvements #18137)

Linting / Prettier

• Enable linting plugin via tslint
  • Enable lint fix in plugin:dev (grafana/toolkit: prettier and lint fix in dev mode #18131)
• Enable custom tslint config
• Prettier (grafana/toolkit: Add early support for Prettier #17964)
  • Enable Prettier check in 'plugin:build'
  • Enable Prettier write in 'plugin:dev' (grafana/toolkit: prettier and lint fix in dev mode #18131)

Bundling plugin

• Verifies if plugin follows common practices
  • Check if plugin.json naming follows naming convention (@grafana/toolkit: validate type and id #17975)
• Bundles plugin using Webpack
• CSS generation
• Show webpack stats after build is finished (both plugin:dev and plugin:build) (grafana/toolkit: show webpack stats on successful compilation #18138)

### Documentation

• generate docs from /docs folder
  a different task should do this

CircleCI helpers

package a plugin in circleci. this will be super helpful for backend plugins.

See: #17967

Plugin stylesheet (#17938)

• Add support for common conventions:
  • Single css/scss file (used via import)
  • Separate files for themes (used via loadPluginCss)
  • Emotion
• Enable PostCSS
• Verify SASS setup

With new react panels, we should push for emotion. However we still need to support old/existing panels that tend to either just import a .css file, or use: loadPluginCss

loadPluginCss({
  dark: 'plugins/grafana-kubernetes-app/css/dark.css',
  light: 'plugins/grafana-kubernetes-app/css/light.css'
});

v2 backlog:

• Make Enzyme a default test tool in toolkit
• plugin:ci - gather stats from webpack task

Current (known) problems to be figured out

1. How to handle imports from core? import XXX from 'app/plugins/sdk'
   This problem surfaced in k8s and ajax panel. Can't see any clear path how to fight this one. Problem is that those plugins import implementations from core (indirectyl, via pligins/sdk). It's not about interfaces anymore but about actual implementations being extended by apps/plugins. Thing is that we possibly could extract app/plugins/sdk to a separate package, i.e. @grafana/plugin-sdk, but it's super hard because of app/plugins/sdk referencing core modules.... Crap

Toolkit wishlist:

• prettier fix (in dev)
• test ask @brian.gann
• external/lib support. Include, but don't compile:
  https://github.com/grafana/grafana-polystat-panel/blob/use-toolkit/src/d3wrapper.ts#L1
• enzyme out-of-the-box

[dprokop]

@ryantxu could you please update Check if plugin.json naming follows naming convention above? I remember you had an idea about plugin naming convention in regards to org name and plugin name

[ryantxu]

rollup vs webpack? so I originally though rollup was a good path because it makes nicer bundles... but having tried this with a bunch of plugins, I think it is going to be more difficult that in is worth :(

rollup seems to be much stricter about how it imports and packages libraries. This is likely more "correct", but I am struggling to find something that works.

I think we should consider webpack. @dprokop, could you take a stab a matching this webpack.config.js -- If you get close, I can poke on the details till it works with a few of the plugins

[dprokop]

@ryantxu sure, as we talked on friday, we gonna try with webpack here.

[dprokop]

@ryantxu webpack related work: #17850

[ryantxu]

Current Status:

✅ https://github.com/ryantxu/ajax-panel
✅ https://github.com/NatelEnergy/grafana-discrete-panel
✅ https://github.com/grafana/kentik-app
✅ https://github.com/grafana/kubernetes-app
✅ grafana/simple-json-datasource#129
❌ https://github.com/ryantxu/grafana-graphql-datasource: fails with:

https://medium.com/@gavin.blog/how-to-solve-ng-annotate-stringmap-expected-string-key-error-f0711c7c84db

I think the next priorities should be:

1. prettier --fix?

   • yarn dev -- auto fix prettier and lint errors
   • yarn build -- failes if prettier or lint fails

   Alternativly we could add an explicit call with --fix, but I think fewer options is better

2. @grafana/data build/publish?

[dprokop]

@ryantxu me need to publish @grafana/data in order to make those ^^ problems disappear(grapgql ds and simple-json ds). Opened #17981 to see how we can improve the process there

[dprokop]

@ryantxu the grafana-graphql-datasource plugin should build correctly with #18005 applied and @grafana/data added as dependency

[briangann]

Currently the toolkit doesn't watch .html files, which are common in plugins for the query editors/datasource configuration.

It would be great to have 'yarn dev --watch' detect changes to these files and drop them into dist, just like the rest of the process.

The "hack" i've used so far is to add htmlwebpackplugin, though it leaves an artifact 'index.html' that never gets used, but does detect and copy the partials/* correctly.

[torkelo]

Anything left for this issue? Maybe close this and create an issue for remaining issues

[dprokop]

@torkelo yup, closing this one. Other toolkit tasks are tracked in separate issues.