Add script for visualizing RaFTS predictions of performance metrics #32
Conversation
pkg/fs_algo/fs_algo/fs_perf_viz.py
Outdated
| plt.title("Predicted Performance: {}".format(ds), fontsize = 28) | ||
|
|
||
| # Save the plot as a .png file | ||
| output_path = f'{dir_out}/data_visualizations/{ds}_{algo}_{metric}_performance_map.png' |
There was a problem hiding this comment.
I suggest we 1) create a function for writing data visualization files and 2) a function for generating the data visualization directory and filepath (e.g. std_viz_path). 1) When we switch to the cloud, we'll know to modify the read/write functions to provide a) local read/write or b) cloud read/write (no need to develop this yet). 2) A single function to create the file path will eliminate guesswork on filenames when reading whatever was written. Prioritize item #2 and we can get back to #1 if you're limited in time - we'll need a major overhaul to prepare for cloud integration.
There was a problem hiding this comment.
Return a matplot lib figure as an object (plt.gcf())
Create functions that define paths
Create functions that do the file I/O so that when we migrate to AWS we can just update the function
Move the creation of the data_visualizations folder into a function that creates the path for it
dir_ * if a directory
path_* if a path
Could include smaller functions for creating the path to the performance map, for example: fsate lines 492, 474
Look at plot learning curve for example: https://github.com/glitt13/formulation-selector/blob/agu_24/pkg/fs_algo/fs_algo/fs_algo_train_eval.py
There was a problem hiding this comment.
Classes are mostly for conveniently passing along data, so you can probably just do functions for now and either make a separate plotting file or add to fsate
| plt.title('Observed vs. Predicted Performance: {}'.format(ds)) | ||
|
|
||
| # Save the plot as a .png file | ||
| output_path = f'{dir_out}/data_visualizations/{ds}_{algo}_{metric}_obs_vs_sim_scatter.png' |
There was a problem hiding this comment.
Since this is a different visualization, seems like the std_viz_path function could use the true_keys object for creating the full file name.
| plt.title('Observed vs. Predicted Performance: {}'.format(ds)) | ||
|
|
||
| # Save the plot as a .png file | ||
| output_path = f'{dir_out}/data_visualizations/{ds}_{algo}_{metric}_obs_vs_sim_scatter.png' |
There was a problem hiding this comment.
@bolotinl A new suggestion - create a subdirectory based on the dataset inside output/data_visualizations, mimicking the structures created by fs_algo_train_eval.std_pred_path, e.g.
Path(dir_out)/Path('data_visualizations')/Path(dataset_id)/Path(insert specific plottype_algo_metric_dataset_id here)
A new script allows the user to plot a map of RaFTS predicted module performance and/or a scatter plot comparing RaFTS predicted performances to actual performances
Additions
fs_perf_viz.py, which just requires aviz_config.yamlviz_config.yamlTesting
The viz_config.yaml should require little to no customization, though it is customizeable in terms of which plots will be generated.
python fs_perf_viz.py "/full/path/to/viz_config.yaml"Todos
fs_perf_viz.py(in progress)