You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
The main cpanstats table (CPAN::Testers::Schema::Result::Stats) has two existing date/time-ish fields:
postdate, which is a YYYYMM stored as a mediumint
fulldate, which is a YYYYMMDDHHNN stored as a varchar
Neither of these is ideal, since date/time comparisons in MySQL want ISO8601 date/time strings (which the datetime column can store). Using non-standard date/time formats makes it much harder to do ad-hoc reporting based on time periods (like pulling sections of the report summaries in Grafana dashboards, or for generating historical statistics for stats.cpantesters.org).
We should add a new column, created, to the cpanstats table. This column should be a datetime column, and should be initially populated based on the fulldate column (using 00 for the seconds field). This new column should be automatically populated by CPAN::Testers::Schema::ResultSet::Stats->insert_test_report, the standard API for inserting data into the cpanstats table.
It is possible to get a more accurate date/time for the report by using the test_reports table for new reports, or the metabase.metabase table for older reports, but the metabase table will make it much more difficult (and if knowing the seconds is that important, we can always fix it later).
Once we have this new column, we can start updating any other code that uses postdate or fulldate to instead use created (like CPAN::Testers::Schema::ResultSet::Release->since and CPAN::Testers::API::Controller::Summary->summary). While visiting other repositories, consider moving any useful code here to the schema.
The text was updated successfully, but these errors were encountered:
For example, here's a SQL query I needed to write in Grafana:
SELECT
CAST( UNIX_TIMESTAMP( STR_TO_DATE( fulldate, '%Y%m%d%H%i' ) ) /900AS signed ) *900astime,
COUNT(tester) AS value,
tester AS metric
FROM cpanstats
WHERE fulldate >= DATE_FORMAT( $__timeFrom(), '%Y%m%d%H%i' )
AND fulldate <= DATE_FORMAT( $__timeTo(), '%Y%m%d%H%i' )
GROUP BY1, tester
ORDER BY1
I had to upgrade to Grafana 5.1, because the 4.3 we were on doesn't support $__timeFrom() and $__timeTo(), without which this would not be possible. If we had a simple created column, this query would be able to take full advantage of Grafana's SQL macros, and would look like:
SELECT
$__timeGroup( created, '15m' ) astime,
COUNT(tester) AS value,
tester AS metric
FROM cpanstats
WHERE $__timeFilter( created )
GROUP BY1, tester
ORDER BY1
The main
cpanstats
table (CPAN::Testers::Schema::Result::Stats
) has two existing date/time-ish fields:postdate
, which is aYYYYMM
stored as amediumint
fulldate
, which is aYYYYMMDDHHNN
stored as avarchar
Neither of these is ideal, since date/time comparisons in MySQL want ISO8601 date/time strings (which the
datetime
column can store). Using non-standard date/time formats makes it much harder to do ad-hoc reporting based on time periods (like pulling sections of the report summaries in Grafana dashboards, or for generating historical statistics for stats.cpantesters.org).We should add a new column,
created
, to thecpanstats
table. This column should be adatetime
column, and should be initially populated based on thefulldate
column (using 00 for the seconds field). This new column should be automatically populated byCPAN::Testers::Schema::ResultSet::Stats->insert_test_report
, the standard API for inserting data into thecpanstats
table.It is possible to get a more accurate date/time for the report by using the
test_reports
table for new reports, or themetabase.metabase
table for older reports, but the metabase table will make it much more difficult (and if knowing the seconds is that important, we can always fix it later).Once we have this new column, we can start updating any other code that uses
postdate
orfulldate
to instead usecreated
(likeCPAN::Testers::Schema::ResultSet::Release->since
andCPAN::Testers::API::Controller::Summary->summary
). While visiting other repositories, consider moving any useful code here to the schema.The text was updated successfully, but these errors were encountered: