Add config validation CLI

[SeanBryan51]

Fixes #186

[codecov]

Codecov Report

Merging #191 (505bc50) into main (474d183) will decrease coverage by 0.01%.
Report is 1 commits behind head on main.
The diff coverage is 83.33%.

@@            Coverage Diff             @@
##             main     #191      +/-   ##
==========================================
- Coverage   85.19%   85.19%   -0.01%     
==========================================
  Files          27       27              
  Lines        1385     1391       +6     
==========================================
+ Hits         1180     1185       +5     
- Misses        205      206       +1     

Files	Coverage Δ
benchcab/cli.py	100.00% <100.00%> (ø)
tests/test_cli.py	100.00% <100.00%> (ø)
benchcab/benchcab.py	23.42% <50.00%> (+0.30%)	⬆️

[SeanBryan51]

This is a bit of a hack since the validation occurs in Benchcab.__init__() which will consequently run anything else that is inside the __init__ method. E.g. benchcab will yell at you if we are not running on Gadi due to the _validate_environment method being called.

Perhaps this issue will go away once we are able to port benchcab to other machines?

[bschroeter]

So there is a way that this can be improved using a "parse and dispatch" strategy. Whereby you don't need to run a conditional to work out which subcommand to use.

See: https://github.com/AusClimateService/axiom/blob/54de78d3c490800bdf6806e15a1e8d23f2e82430/bin/axiom#L207

I basically set the desired command as a default on the sub parser and then pop it to call when dispatching.

https://github.com/AusClimateService/axiom/blob/54de78d3c490800bdf6806e15a1e8d23f2e82430/bin/axiom#L15

[bschroeter]

As per yesterdays meeting, there is a validate_config method now present in config.py.

benchcab/benchcab/config.py

Line 32 in 42e66b6

def validate_config(config: dict) -> bool:

Returns True if valid and raises an exception if not.

It would be worthwhile wrapping this in a local method to return something useful for the user in either case.

[SeanBryan51]

The parse and dispatch strategy looks like a good option. I've made a new issue for this (see #196) as it is a bit out of the scope of this PR and a bit more involved.

[ccarouge]

I am wondering if it is time to introduce click. It's quite a bit of work so probably not necessary right away but it should remove the needs for either conditionals or a parse and dispatch strategy.