Merge remote-tracking branch 'upstream/master'

CHANGELOG.md

@@ -2,18 +2,26 @@
 
 ## [**Next release**](https://github.com/netdata/netdata/tree/HEAD)
 
-[Full Changelog](https://github.com/netdata/netdata/compare/v1.42.1...HEAD)
+[Full Changelog](https://github.com/netdata/netdata/compare/v1.42.2...HEAD)
 
 **Merged pull requests:**
 
+- Add index to ACLK table to improve update statements [\#15890](https://github.com/netdata/netdata/pull/15890) ([stelfrag](https://github.com/stelfrag))
+- allow any field to be a facet [\#15880](https://github.com/netdata/netdata/pull/15880) ([ktsaou](https://github.com/ktsaou))
+- Regenerate integrations.js [\#15879](https://github.com/netdata/netdata/pull/15879) ([netdatabot](https://github.com/netdatabot))
+- use the newer XXH3 128bits algorithm, instead of the classic XXH128 [\#15878](https://github.com/netdata/netdata/pull/15878) ([ktsaou](https://github.com/ktsaou))
+- ML add new `delete old models param` to readme [\#15873](https://github.com/netdata/netdata/pull/15873) ([andrewm4894](https://github.com/andrewm4894))
+- Update SQLITE version to 3.42.0 [\#15870](https://github.com/netdata/netdata/pull/15870) ([stelfrag](https://github.com/stelfrag))
 - Regenerate integrations.js [\#15867](https://github.com/netdata/netdata/pull/15867) ([netdatabot](https://github.com/netdatabot))
+- Add a fail reason to pinpoint exactly what went wrong [\#15866](https://github.com/netdata/netdata/pull/15866) ([stelfrag](https://github.com/stelfrag))
 - Add plugin and module information to collector integrations. [\#15864](https://github.com/netdata/netdata/pull/15864) ([Ferroin](https://github.com/Ferroin))
 - Regenerate integrations.js [\#15862](https://github.com/netdata/netdata/pull/15862) ([netdatabot](https://github.com/netdatabot))
 - Explicitly depend on version-matched plugins in native packages. [\#15861](https://github.com/netdata/netdata/pull/15861) ([Ferroin](https://github.com/Ferroin))
 - Apply a label prefix for netdata labels [\#15860](https://github.com/netdata/netdata/pull/15860) ([kevin-fwu](https://github.com/kevin-fwu))
 - fix proc meminfo cached calculation [\#15859](https://github.com/netdata/netdata/pull/15859) ([ilyam8](https://github.com/ilyam8))
 - Fix compilation warnings [\#15858](https://github.com/netdata/netdata/pull/15858) ([stelfrag](https://github.com/stelfrag))
 - packaging cleanup after \#15842 [\#15857](https://github.com/netdata/netdata/pull/15857) ([ilyam8](https://github.com/ilyam8))
+- Add a chart that groups anomaly rate by chart type. [\#15856](https://github.com/netdata/netdata/pull/15856) ([vkalintiris](https://github.com/vkalintiris))
 - fix packaging static build openssl 32bit [\#15855](https://github.com/netdata/netdata/pull/15855) ([ilyam8](https://github.com/ilyam8))
 - fix packaging mark stable static build [\#15854](https://github.com/netdata/netdata/pull/15854) ([ilyam8](https://github.com/ilyam8))
 - Facets histograms [\#15846](https://github.com/netdata/netdata/pull/15846) ([ktsaou](https://github.com/ktsaou))
@@ -62,7 +70,13 @@
 - fix: the cleanup was not performed during the kickstart.sh dry run [\#15775](https://github.com/netdata/netdata/pull/15775) ([ilyam8](https://github.com/ilyam8))
 - Increase alert snapshot chunk size [\#15748](https://github.com/netdata/netdata/pull/15748) ([MrZammler](https://github.com/MrZammler))
 - Added CentOS-Stream to distros [\#15742](https://github.com/netdata/netdata/pull/15742) ([k0ste](https://github.com/k0ste))
+- Unconditionally delete very old models. [\#15720](https://github.com/netdata/netdata/pull/15720) ([vkalintiris](https://github.com/vkalintiris))
 - Misc code cleanup [\#15665](https://github.com/netdata/netdata/pull/15665) ([stelfrag](https://github.com/stelfrag))
+- Metadata cleanup improvements [\#15462](https://github.com/netdata/netdata/pull/15462) ([stelfrag](https://github.com/stelfrag))
+
+## [v1.42.2](https://github.com/netdata/netdata/tree/v1.42.2) (2023-08-28)
+
+[Full Changelog](https://github.com/netdata/netdata/compare/v1.42.1...v1.42.2)
 
 ## [v1.42.1](https://github.com/netdata/netdata/tree/v1.42.1) (2023-08-16)
 
@@ -415,18 +429,6 @@
 - multi-threaded version of freeipmi.plugin [\#15327](https://github.com/netdata/netdata/pull/15327) ([ktsaou](https://github.com/ktsaou))
 - Single module schema, add required properties [\#15326](https://github.com/netdata/netdata/pull/15326) ([Ancairon](https://github.com/Ancairon))
 - Fix coverity issue 394862 - Argument cannot be negative [\#15324](https://github.com/netdata/netdata/pull/15324) ([stelfrag](https://github.com/stelfrag))
-- Rename log Macros \(debug\) [\#15322](https://github.com/netdata/netdata/pull/15322) ([thiagoftsm](https://github.com/thiagoftsm))
-- bearer authorization API [\#15321](https://github.com/netdata/netdata/pull/15321) ([ktsaou](https://github.com/ktsaou))
-- local-listeners: use host prefix in read\_cmdline [\#15320](https://github.com/netdata/netdata/pull/15320) ([ilyam8](https://github.com/ilyam8))
-- local-listener using libnetdata [\#15319](https://github.com/netdata/netdata/pull/15319) ([ktsaou](https://github.com/ktsaou))
-- avoid memory allocations for alert transitions facets processing [\#15318](https://github.com/netdata/netdata/pull/15318) ([ktsaou](https://github.com/ktsaou))
-- add add summary linking to alert instances \(ati\) when options=summary,values is requested [\#15317](https://github.com/netdata/netdata/pull/15317) ([ktsaou](https://github.com/ktsaou))
-- fix alerts transitions sorting [\#15315](https://github.com/netdata/netdata/pull/15315) ([ktsaou](https://github.com/ktsaou))
-- Keep health log history in seconds [\#15314](https://github.com/netdata/netdata/pull/15314) ([MrZammler](https://github.com/MrZammler))
-- stale vitual hosts [\#15313](https://github.com/netdata/netdata/pull/15313) ([ktsaou](https://github.com/ktsaou))
-- bump go.d.plugin to v0.54.0 [\#15312](https://github.com/netdata/netdata/pull/15312) ([ilyam8](https://github.com/ilyam8))
-- health: respect overriding nc binary for IRC notifications [\#15310](https://github.com/netdata/netdata/pull/15310) ([ilyam8](https://github.com/ilyam8))
-- hide not available for viewers charts when exporting in shell format [\#15309](https://github.com/netdata/netdata/pull/15309) ([ilyam8](https://github.com/ilyam8))
 
 ## [v1.40.1](https://github.com/netdata/netdata/tree/v1.40.1) (2023-06-27)
 

collectors/all.h

@@ -403,7 +403,8 @@
 // [ml] charts
 #define ML_CHART_PRIO_DIMENSIONS                      39181
 #define ML_CHART_PRIO_ANOMALY_RATE                    39182
-#define ML_CHART_PRIO_DETECTOR_EVENTS                 39183
+#define ML_CHART_PRIO_TYPE_ANOMALY_RATE               39183
+#define ML_CHART_PRIO_DETECTOR_EVENTS                 39184
 
 // [netdata.ml] charts
 #define NETDATA_ML_CHART_RUNNING                      890001

collectors/python.d.plugin/go_expvar/metadata.yaml

@@ -4,7 +4,7 @@ modules:
       plugin_name: python.d.plugin
       module_name: go_expvar
       monitored_instance:
-        name: Go applications
+        name: Go applications (EXPVAR)
         link: "https://pkg.go.dev/expvar"
         categories:
           - data-collection.apm

collectors/systemd-journal.plugin/systemd-journal.c

@@ -29,6 +29,7 @@
 #define JOURNAL_PARAMETER_ANCHOR                "anchor"
 #define JOURNAL_PARAMETER_LAST                  "last"
 #define JOURNAL_PARAMETER_QUERY                 "query"
+#define JOURNAL_PARAMETER_FACETS                "facets"
 #define JOURNAL_PARAMETER_HISTOGRAM             "histogram"
 
 #define SYSTEMD_ALWAYS_VISIBLE_KEYS             NULL
@@ -52,6 +53,8 @@
     "|_SYSTEMD_USER_UNIT"                       \
     "|USER_UNIT"                                \
     "|UNIT"                                     \
+    "|CONTAINER_NAME"                           \
+    "|IMAGE_NAME"                               \
     ""
 
 static netdata_mutex_t mutex = NETDATA_MUTEX_INITIALIZER;
@@ -232,8 +235,7 @@ static char *uid_to_username(uid_t uid, char *buffer, size_t buffer_size) {
     if (getpwuid_r(uid, &pw, tmp, 1024, &result) != 0 || result == NULL)
         return NULL;
 
-    strncpy(buffer, pw.pw_name, buffer_size - 1);
-    buffer[buffer_size - 1] = '\0'; // Null-terminate just in case
+    snprintfz(buffer, buffer_size - 1, "%u (%s)", uid, pw.pw_name);
     return buffer;
 }
 
@@ -244,8 +246,7 @@ static char *gid_to_groupname(gid_t gid, char* buffer, size_t buffer_size) {
     if (getgrgid_r(gid, &grp, tmp, 1024, &result) != 0 || result == NULL)
         return NULL;
 
-    strncpy(buffer, grp.gr_name, buffer_size - 1);
-    buffer[buffer_size - 1] = '\0'; // Null-terminate just in case
+    snprintfz(buffer, buffer_size - 1, "%u (%s)", gid, grp.gr_name);
     return buffer;
 }
 
@@ -336,9 +337,6 @@ static void systemd_journal_dynamic_row_id(FACETS *facets __maybe_unused, BUFFER
 }
 
 static void function_systemd_journal(const char *transaction, char *function, char *line_buffer __maybe_unused, int line_max __maybe_unused, int timeout __maybe_unused) {
-    char *words[SYSTEMD_JOURNAL_MAX_PARAMS] = { NULL };
-    size_t num_words = quoted_strings_splitter_pluginsd(function, words, SYSTEMD_JOURNAL_MAX_PARAMS);
-
     BUFFER *wb = buffer_create(0, NULL);
     buffer_flush(wb);
     buffer_json_initialize(wb, "\"", "\"", 0, true, BUFFER_JSON_OPTIONS_NEWLINE_ON_ARRAY_ITEMS);
@@ -353,31 +351,34 @@ static void function_systemd_journal(const char *transaction, char *function, ch
     facets_accepted_param(facets, JOURNAL_PARAMETER_ANCHOR);
     facets_accepted_param(facets, JOURNAL_PARAMETER_LAST);
     facets_accepted_param(facets, JOURNAL_PARAMETER_QUERY);
+    facets_accepted_param(facets, JOURNAL_PARAMETER_FACETS);
     facets_accepted_param(facets, JOURNAL_PARAMETER_HISTOGRAM);
 
     // register the fields in the order you want them on the dashboard
 
-    facets_register_dynamic_key(facets, "ND_JOURNAL_PROCESS", FACET_KEY_OPTION_NO_FACET|FACET_KEY_OPTION_VISIBLE|FACET_KEY_OPTION_FTS,
-                                systemd_journal_dynamic_row_id, NULL);
+    facets_register_dynamic_key_name(facets, "ND_JOURNAL_PROCESS",
+                                     FACET_KEY_OPTION_NO_FACET | FACET_KEY_OPTION_VISIBLE | FACET_KEY_OPTION_FTS,
+                                     systemd_journal_dynamic_row_id, NULL);
 
-    facets_register_key(facets, "MESSAGE",
-                        FACET_KEY_OPTION_NO_FACET|FACET_KEY_OPTION_MAIN_TEXT|FACET_KEY_OPTION_VISIBLE|FACET_KEY_OPTION_FTS);
+    facets_register_key_name(facets, "MESSAGE",
+                             FACET_KEY_OPTION_NO_FACET | FACET_KEY_OPTION_MAIN_TEXT | FACET_KEY_OPTION_VISIBLE |
+                             FACET_KEY_OPTION_FTS);
 
-    facets_register_key_transformation(facets, "PRIORITY", FACET_KEY_OPTION_FACET|FACET_KEY_OPTION_FTS,
-                                       systemd_journal_transform_priority, NULL);
+    facets_register_key_name_transformation(facets, "PRIORITY", FACET_KEY_OPTION_FACET | FACET_KEY_OPTION_FTS,
+                                            systemd_journal_transform_priority, NULL);
 
-    facets_register_key_transformation(facets, "SYSLOG_FACILITY", FACET_KEY_OPTION_FACET|FACET_KEY_OPTION_FTS,
-                                       systemd_journal_transform_syslog_facility, NULL);
+    facets_register_key_name_transformation(facets, "SYSLOG_FACILITY", FACET_KEY_OPTION_FACET | FACET_KEY_OPTION_FTS,
+                                            systemd_journal_transform_syslog_facility, NULL);
 
-    facets_register_key(facets, "SYSLOG_IDENTIFIER", FACET_KEY_OPTION_FACET|FACET_KEY_OPTION_FTS);
-    facets_register_key(facets, "UNIT", FACET_KEY_OPTION_FACET|FACET_KEY_OPTION_FTS);
-    facets_register_key(facets, "USER_UNIT", FACET_KEY_OPTION_FACET|FACET_KEY_OPTION_FTS);
+    facets_register_key_name(facets, "SYSLOG_IDENTIFIER", FACET_KEY_OPTION_FACET | FACET_KEY_OPTION_FTS);
+    facets_register_key_name(facets, "UNIT", FACET_KEY_OPTION_FACET | FACET_KEY_OPTION_FTS);
+    facets_register_key_name(facets, "USER_UNIT", FACET_KEY_OPTION_FACET | FACET_KEY_OPTION_FTS);
 
-    facets_register_key_transformation(facets, "_UID", FACET_KEY_OPTION_FACET|FACET_KEY_OPTION_FTS,
-                                       systemd_journal_transform_uid, uids);
+    facets_register_key_name_transformation(facets, "_UID", FACET_KEY_OPTION_FACET | FACET_KEY_OPTION_FTS,
+                                            systemd_journal_transform_uid, uids);
 
-    facets_register_key_transformation(facets, "_GID", FACET_KEY_OPTION_FACET|FACET_KEY_OPTION_FTS,
-                                       systemd_journal_transform_gid, gids);
+    facets_register_key_name_transformation(facets, "_GID", FACET_KEY_OPTION_FACET | FACET_KEY_OPTION_FTS,
+                                            systemd_journal_transform_gid, gids);
 
     time_t after_s = 0, before_s = 0;
     usec_t anchor = 0;
@@ -386,33 +387,53 @@ static void function_systemd_journal(const char *transaction, char *function, ch
     const char *chart = NULL;
 
     buffer_json_member_add_object(wb, "request");
-    buffer_json_member_add_object(wb, "filters");
 
+    char *words[SYSTEMD_JOURNAL_MAX_PARAMS] = { NULL };
+    size_t num_words = quoted_strings_splitter_pluginsd(function, words, SYSTEMD_JOURNAL_MAX_PARAMS);
     for(int i = 1; i < SYSTEMD_JOURNAL_MAX_PARAMS ;i++) {
-        const char *keyword = get_word(words, num_words, i);
+        char *keyword = get_word(words, num_words, i);
         if(!keyword) break;
 
         if(strcmp(keyword, JOURNAL_PARAMETER_HELP) == 0) {
             systemd_journal_function_help(transaction);
             goto cleanup;
         }
-        else if(strncmp(keyword, JOURNAL_PARAMETER_AFTER ":", strlen(JOURNAL_PARAMETER_AFTER ":")) == 0) {
-            after_s = str2l(&keyword[strlen(JOURNAL_PARAMETER_AFTER ":")]);
+        else if(strncmp(keyword, JOURNAL_PARAMETER_AFTER ":", sizeof(JOURNAL_PARAMETER_AFTER ":") - 1) == 0) {
+            after_s = str2l(&keyword[sizeof(JOURNAL_PARAMETER_AFTER ":") - 1]);
         }
-        else if(strncmp(keyword, JOURNAL_PARAMETER_BEFORE ":", strlen(JOURNAL_PARAMETER_BEFORE ":")) == 0) {
-            before_s = str2l(&keyword[strlen(JOURNAL_PARAMETER_BEFORE ":")]);
+        else if(strncmp(keyword, JOURNAL_PARAMETER_BEFORE ":", sizeof(JOURNAL_PARAMETER_BEFORE ":") - 1) == 0) {
+            before_s = str2l(&keyword[sizeof(JOURNAL_PARAMETER_BEFORE ":") - 1]);
         }
-        else if(strncmp(keyword, JOURNAL_PARAMETER_ANCHOR ":", strlen(JOURNAL_PARAMETER_ANCHOR ":")) == 0) {
-            anchor = str2ull(&keyword[strlen(JOURNAL_PARAMETER_ANCHOR ":")], NULL);
+        else if(strncmp(keyword, JOURNAL_PARAMETER_ANCHOR ":", sizeof(JOURNAL_PARAMETER_ANCHOR ":") - 1) == 0) {
+            anchor = str2ull(&keyword[sizeof(JOURNAL_PARAMETER_ANCHOR ":") - 1], NULL);
         }
-        else if(strncmp(keyword, JOURNAL_PARAMETER_LAST ":", strlen(JOURNAL_PARAMETER_LAST ":")) == 0) {
-            last = str2ul(&keyword[strlen(JOURNAL_PARAMETER_LAST ":")]);
+        else if(strncmp(keyword, JOURNAL_PARAMETER_LAST ":", sizeof(JOURNAL_PARAMETER_LAST ":") - 1) == 0) {
+            last = str2ul(&keyword[sizeof(JOURNAL_PARAMETER_LAST ":") - 1]);
         }
-        else if(strncmp(keyword, JOURNAL_PARAMETER_QUERY ":", strlen(JOURNAL_PARAMETER_QUERY ":")) == 0) {
-            query= &keyword[strlen(JOURNAL_PARAMETER_QUERY ":")];
+        else if(strncmp(keyword, JOURNAL_PARAMETER_QUERY ":", sizeof(JOURNAL_PARAMETER_QUERY ":") - 1) == 0) {
+            query= &keyword[sizeof(JOURNAL_PARAMETER_QUERY ":") - 1];
         }
-        else if(strncmp(keyword, JOURNAL_PARAMETER_HISTOGRAM ":", strlen(JOURNAL_PARAMETER_HISTOGRAM ":")) == 0) {
-            chart = &keyword[strlen(JOURNAL_PARAMETER_HISTOGRAM ":")];
+        else if(strncmp(keyword, JOURNAL_PARAMETER_HISTOGRAM ":", sizeof(JOURNAL_PARAMETER_HISTOGRAM ":") - 1) == 0) {
+            chart = &keyword[sizeof(JOURNAL_PARAMETER_HISTOGRAM ":") - 1];
+        }
+        else if(strncmp(keyword, JOURNAL_PARAMETER_FACETS ":", sizeof(JOURNAL_PARAMETER_FACETS ":") - 1) == 0) {
+            char *value = &keyword[sizeof(JOURNAL_PARAMETER_FACETS ":") - 1];
+            if(*value) {
+                buffer_json_member_add_array(wb, JOURNAL_PARAMETER_FACETS);
+
+                while(value) {
+                    char *sep = strchr(value, ',');
+                    if(sep)
+                        *sep++ = '\0';
+
+                    facets_register_facet_id(facets, value, FACET_KEY_OPTION_FACET|FACET_KEY_OPTION_FTS|FACET_KEY_OPTION_REORDER);
+                    buffer_json_add_array_item_string(wb, value);
+
+                    value = sep;
+                }
+
+                buffer_json_array_close(wb); // JOURNAL_PARAMETER_FACETS
+            }
         }
         else {
             char *value = strchr(keyword, ':');
@@ -426,7 +447,7 @@ static void function_systemd_journal(const char *transaction, char *function, ch
                     if(sep)
                         *sep++ = '\0';
 
-                    facets_register_facet_filter(facets, keyword, value, FACET_KEY_OPTION_REORDER);
+                    facets_register_facet_id_filter(facets, keyword, value, FACET_KEY_OPTION_FACET|FACET_KEY_OPTION_FTS|FACET_KEY_OPTION_REORDER);
                     buffer_json_add_array_item_string(wb, value);
 
                     value = sep;
@@ -437,8 +458,6 @@ static void function_systemd_journal(const char *transaction, char *function, ch
         }
     }
 
-    buffer_json_object_close(wb); // filters
-
     time_t expires = now_realtime_sec() + 1;
     time_t now_s;