feat(anomaly detection): Dynamic Window for Matrix Profiling

[aayush-se]

• Use 2 windows (1 determined by the SuSS algorithm, and a "fixed" window at size 10) to perform detection reducing the recovery time for the algorithm ultimately making alerts persist for a shorter period of time after anomalous behavior is over
• Performing detection twice for each window to maintain the matrix profiles
• Updated the anomaly_algo_data to store matrix profile related information for both windows for a given alert
• Created MPTimeSeriesAnomaliesSingleWindow class and updated the MPTimeSeriesAnomalies class to effectively manage and store the proper matrix profile related information and the boolean for determining which window logic to use
• Recalculates both matrix profiles and update the respective rows if they are not present to ensure backwards compatibility with the older MPTimeSeriesAnomalies class
• Updated current tests to match the new classes and their fields along with shapes for anomaly_algo_data

[aayush-se]

Waiting for codecov to give report -- will update test coverage upon receiving

[ram-senth]

Seems like this is not backward compatible as both mp_suss and mp_fixed are not there. So detection for existing alerts in production will fail, right?

[aayush-se]

This was one thing I wanted to ask about -- if we were to use this class for prod, could we rerun calculation to populate the objects appropriately?

[aayush-se]

Because to get the algo data for both suss and fixed, we would need to store the MP as a field in the object

[ram-senth]

I think it will be best to just let the recalculation happen when data pruning happens. So this code should be backward compatible. We will need an unit test to check that. Also, test it locally with an alert pre-populated.

[aayush-se]

Sure, will include those tests and confirm through local testing

[ram-senth]

We should add this 10 as a constant to the MPConfig class?

[ram-senth]

So we recalculate but we do not save to the database here. However, the new time step that will be stored in the db will have these additional information. So next detect call will actually fail, right?

[aayush-se]

Within _recalculate_batch_detection, another function update_timeseries is called that directly updates the DbDynamicAlertTimeSeries with the new anomaly_algo_data

[ram-senth]

I'm good with the changes. Just wanted to confirm that you tested locally with a saved alert that does not have the fixed window history to confirm the backward compatibility. We do not want production failures when we deploy. Also, seems like the unit test coverage needs increasing. There are a few low hanging fruits like testing for length mismatch etc that should take it over the goal.