Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[8.17](backport #6329) Prevent leaking secrets when logging component model #6383

Open
wants to merge 5 commits into
base: 8.17
Choose a base branch
from

Conversation

mergify[bot]
Copy link
Contributor

@mergify mergify bot commented Dec 18, 2024

What does this PR do?

Implements MarshalJSON method on the component.Component struct, so that when the component model is logged, the output does not contain any secrets that might be part of the component model (e.g. API keys to HTTP endpoints etc.).

This change also restores the debug log dumping the component model that was removed in #5201.

Why is it important?

This is part of the effort to prevent leaking secrets in logs as much as possible.

Checklist

  • My code follows the style guidelines of this project
  • [ ] I have commented my code, particularly in hard-to-understand areas
  • [ ] I have made corresponding changes to the documentation
  • [ ] I have made corresponding change to the default configuration files
  • [ ] I have added tests that prove my fix is effective or that my feature works
  • I have added an entry in ./changelog/fragments using the changelog tool
  • [ ] I have added an integration test or an E2E test

How to test this PR locally

  • Build the agent binary (go build .)
  • Change agent.logging.level to debug in elastic-agent.yml file
  • Run the built agent (sudo ./elastic-agent)
  • Look for "Updating running components" debug log in the output.

The output I'm seeing is:

{"log.level":"debug","@timestamp":"2024-12-13T13:40:20.271Z","log.origin":{"function":"github.com/elastic/elastic-agent/internal/pkg/agent/application/coordinator.(*Coordinator).refreshComponentModel","file.name":"coordinator/coordinator.go","file.line":1403},"message":"Updating running component model","log":{"source":"elastic-agent"},"components":[{"ID":"system/metrics-default","InputType":"system/metrics","OutputType":"elasticsearch","ErrMsg":"input not supported","Units":[{"ID":"system/metrics-default-unique-system-metrics-input"},{"ID":"system/metrics-default"}]},{"ID":"filestream-monitoring","InputType":"filestream","OutputType":"elasticsearch","ErrMsg":"input not supported","Units":[{"ID":"filestream-monitoring-filestream-monitoring-agent"},{"ID":"filestream-monitoring"}]},{"ID":"beat/metrics-monitoring","InputType":"beat/metrics","OutputType":"elasticsearch","ErrMsg":"input not supported","Units":[{"ID":"beat/metrics-monitoring-metrics-monitoring-beats"},{"ID":"beat/metrics-monitoring"}]},{"ID":"http/metrics-monitoring","InputType":"http/metrics","OutputType":"elasticsearch","ErrMsg":"input not supported","Units":[{"ID":"http/metrics-monitoring-metrics-monitoring-agent"},{"ID":"http/metrics-monitoring"}]}],"ecs.version":"1.6.0"}

Here's the formatted and JSON-sorted version of this debug log:

logged component model - after change
{
    "@timestamp": "2024-12-13T13:40:20.271Z",
    "components": [
        {
            "ErrMsg": "input not supported",
            "ID": "system/metrics-default",
            "InputType": "system/metrics",
            "OutputType": "elasticsearch",
            "Units": [
                {
                    "ID": "system/metrics-default-unique-system-metrics-input"
                },
                {
                    "ID": "system/metrics-default"
                }
            ]
        },
        {
            "ErrMsg": "input not supported",
            "ID": "filestream-monitoring",
            "InputType": "filestream",
            "OutputType": "elasticsearch",
            "Units": [
                {
                    "ID": "filestream-monitoring-filestream-monitoring-agent"
                },
                {
                    "ID": "filestream-monitoring"
                }
            ]
        },
        {
            "ErrMsg": "input not supported",
            "ID": "beat/metrics-monitoring",
            "InputType": "beat/metrics",
            "OutputType": "elasticsearch",
            "Units": [
                {
                    "ID": "beat/metrics-monitoring-metrics-monitoring-beats"
                },
                {
                    "ID": "beat/metrics-monitoring"
                }
            ]
        },
        {
            "ErrMsg": "input not supported",
            "ID": "http/metrics-monitoring",
            "InputType": "http/metrics",
            "OutputType": "elasticsearch",
            "Units": [
                {
                    "ID": "http/metrics-monitoring-metrics-monitoring-agent"
                },
                {
                    "ID": "http/metrics-monitoring"
                }
            ]
        }
    ],
    "ecs.version": "1.6.0",
    "log": {
        "source": "elastic-agent"
    },
    "log.level": "debug",
    "log.origin": {
        "file.line": 1403,
        "file.name": "coordinator/coordinator.go",
        "function": "github.com/elastic/elastic-agent/internal/pkg/agent/application/coordinator.(*Coordinator).refreshComponentModel"
    },
    "message": "Updating running component model"
}

Here's the same debug log before MarshalJSON was implemented - note that it includes details like "api_key": "example-key", which is not present in the modified output.

logged component model - before change
{
    "@timestamp": "2024-12-10T13:36:42.542Z",
    "components": [
        {
            "Component": {
                "limits": {
                    "source": {
                        "go_max_procs": 0
                    }
                }
            },
            "Err": {
                "Reason": "input not supported"
            },
            "ErrMsg": "",
            "Features": {
                "fqdn": {},
                "source": {
                    "agent": {
                        "features": {
                            "fqdn": {
                                "enabled": false
                            }
                        }
                    }
                }
            },
            "ID": "system/metrics-default",
            "InputSpec": {
                "BinaryName": "",
                "BinaryPath": "",
                "InputType": "",
                "Spec": {
                    "Aliases": null,
                    "Command": null,
                    "Description": "",
                    "IsolateUnits": false,
                    "Name": "",
                    "Outputs": null,
                    "Platforms": null,
                    "ProxiedActions": null,
                    "Runtime": {
                        "Preventions": null
                    },
                    "Service": null
                }
            },
            "InputType": "system/metrics",
            "OutputType": "elasticsearch",
            "Units": [
                {
                    "Config": {
                        "data_stream": {
                            "namespace": "default"
                        },
                        "id": "unique-system-metrics-input",
                        "source": {
                            "data_stream.namespace": "default",
                            "id": "unique-system-metrics-input",
                            "streams": [
                                {
                                    "data_stream.dataset": "system.cpu",
                                    "metricsets": [
                                        "cpu"
                                    ]
                                },
                                {
                                    "data_stream.dataset": "system.memory",
                                    "metricsets": [
                                        "memory"
                                    ]
                                },
                                {
                                    "data_stream.dataset": "system.network",
                                    "metricsets": [
                                        "network"
                                    ]
                                },
                                {
                                    "data_stream.dataset": "system.filesystem",
                                    "metricsets": [
                                        "filesystem"
                                    ]
                                }
                            ],
                            "type": "system/metrics"
                        },
                        "streams": [
                            {
                                "data_stream": {
                                    "dataset": "system.cpu"
                                },
                                "source": {
                                    "data_stream.dataset": "system.cpu",
                                    "metricsets": [
                                        "cpu"
                                    ]
                                }
                            },
                            {
                                "data_stream": {
                                    "dataset": "system.memory"
                                },
                                "source": {
                                    "data_stream.dataset": "system.memory",
                                    "metricsets": [
                                        "memory"
                                    ]
                                }
                            },
                            {
                                "data_stream": {
                                    "dataset": "system.network"
                                },
                                "source": {
                                    "data_stream.dataset": "system.network",
                                    "metricsets": [
                                        "network"
                                    ]
                                }
                            },
                            {
                                "data_stream": {
                                    "dataset": "system.filesystem"
                                },
                                "source": {
                                    "data_stream.dataset": "system.filesystem",
                                    "metricsets": [
                                        "filesystem"
                                    ]
                                }
                            }
                        ],
                        "type": "system/metrics"
                    },
                    "Err": null,
                    "ID": "system/metrics-default-unique-system-metrics-input",
                    "LogLevel": 3,
                    "Type": 0
                },
                {
                    "Config": {
                        "data_stream": {},
                        "source": {
                            "api_key": "example-key",
                            "hosts": [
                                "127.0.0.1:9200"
                            ],
                            "preset": "balanced",
                            "type": "elasticsearch"
                        },
                        "type": "elasticsearch"
                    },
                    "Err": null,
                    "ID": "system/metrics-default",
                    "LogLevel": 3,
                    "Type": 1
                }
            ]
        },
        {
            "Component": {
                "limits": {
                    "source": {
                        "go_max_procs": 0
                    }
                }
            },
            "Err": {
                "Reason": "input not supported"
            },
            "ErrMsg": "",
            "Features": {
                "fqdn": {},
                "source": {
                    "agent": {
                        "features": {
                            "fqdn": {
                                "enabled": false
                            }
                        }
                    }
                }
            },
            "ID": "beat/metrics-monitoring",
            "InputSpec": {
                "BinaryName": "",
                "BinaryPath": "",
                "InputType": "",
                "Spec": {
                    "Aliases": null,
                    "Command": null,
                    "Description": "",
                    "IsolateUnits": false,
                    "Name": "",
                    "Outputs": null,
                    "Platforms": null,
                    "ProxiedActions": null,
                    "Runtime": {
                        "Preventions": null
                    },
                    "Service": null
                }
            },
            "InputType": "beat/metrics",
            "OutputType": "elasticsearch",
            "Units": [
                {
                    "Config": {
                        "data_stream": {
                            "namespace": "default",
                            "source": {
                                "namespace": "default"
                            }
                        },
                        "id": "metrics-monitoring-beats",
                        "name": "metrics-monitoring-beats",
                        "source": {
                            "data_stream": {
                                "namespace": "default"
                            },
                            "id": "metrics-monitoring-beats",
                            "name": "metrics-monitoring-beats",
                            "streams": [
                                {
                                    "data_stream": {
                                        "dataset": "elastic_agent.metricbeat",
                                        "namespace": "default",
                                        "type": "metrics"
                                    },
                                    "failure_threshold": 5,
                                    "hosts": [
                                        "http+unix:///vagrant/data/tmp/Hk6rvk9TDibMPcDvpl0jkLE-qDsHWVYL.sock"
                                    ],
                                    "id": "metrics-monitoring-metricbeat",
                                    "index": "metrics-elastic_agent.metricbeat-default",
                                    "metricsets": [
                                        "stats"
                                    ],
                                    "period": "1m0s",
                                    "processors": [
                                        {
                                            "add_fields": {
                                                "fields": {
                                                    "dataset": "elastic_agent.metricbeat",
                                                    "namespace": "default",
                                                    "type": "metrics"
                                                },
                                                "target": "data_stream"
                                            }
                                        },
                                        {
                                            "add_fields": {
                                                "fields": {
                                                    "dataset": "elastic_agent.metricbeat"
                                                },
                                                "target": "event"
                                            }
                                        },
                                        {
                                            "add_fields": {
                                                "fields": {
                                                    "id": "7f563b50-11f1-4e7c-8361-31458142a891",
                                                    "process": "metricbeat",
                                                    "snapshot": false,
                                                    "version": "9.0.0"
                                                },
                                                "target": "elastic_agent"
                                            }
                                        },
                                        {
                                            "add_fields": {
                                                "fields": {
                                                    "id": "7f563b50-11f1-4e7c-8361-31458142a891"
                                                },
                                                "target": "agent"
                                            }
                                        },
                                        {
                                            "add_fields": {
                                                "fields": {
                                                    "binary": "metricbeat",
                                                    "id": "beat/metrics-monitoring"
                                                },
                                                "target": "component"
                                            }
                                        }
                                    ]
                                },
                                {
                                    "data_stream": {
                                        "dataset": "elastic_agent.metricbeat",
                                        "namespace": "default",
                                        "type": "metrics"
                                    },
                                    "failure_threshold": 5,
                                    "hosts": [
                                        "http+unix:///vagrant/data/tmp/akSPbdqgaHaTY0_J01-dsfYK6JpMz2zn.sock"
                                    ],
                                    "id": "metrics-monitoring-metricbeat",
                                    "index": "metrics-elastic_agent.metricbeat-default",
                                    "metricsets": [
                                        "stats"
                                    ],
                                    "period": "1m0s",
                                    "processors": [
                                        {
                                            "add_fields": {
                                                "fields": {
                                                    "dataset": "elastic_agent.metricbeat",
                                                    "namespace": "default",
                                                    "type": "metrics"
                                                },
                                                "target": "data_stream"
                                            }
                                        },
                                        {
                                            "add_fields": {
                                                "fields": {
                                                    "dataset": "elastic_agent.metricbeat"
                                                },
                                                "target": "event"
                                            }
                                        },
                                        {
                                            "add_fields": {
                                                "fields": {
                                                    "id": "7f563b50-11f1-4e7c-8361-31458142a891",
                                                    "process": "metricbeat",
                                                    "snapshot": false,
                                                    "version": "9.0.0"
                                                },
                                                "target": "elastic_agent"
                                            }
                                        },
                                        {
                                            "add_fields": {
                                                "fields": {
                                                    "id": "7f563b50-11f1-4e7c-8361-31458142a891"
                                                },
                                                "target": "agent"
                                            }
                                        },
                                        {
                                            "add_fields": {
                                                "fields": {
                                                    "binary": "metricbeat",
                                                    "id": "http/metrics-monitoring"
                                                },
                                                "target": "component"
                                            }
                                        }
                                    ]
                                },
                                {
                                    "data_stream": {
                                        "dataset": "elastic_agent.filebeat",
                                        "namespace": "default",
                                        "type": "metrics"
                                    },
                                    "failure_threshold": 5,
                                    "hosts": [
                                        "http+unix:///vagrant/data/tmp/xTEtpJ7117ppc6OYvJCaYHbDW8mLjXGe.sock"
                                    ],
                                    "id": "metrics-monitoring-filebeat",
                                    "index": "metrics-elastic_agent.filebeat-default",
                                    "metricsets": [
                                        "stats"
                                    ],
                                    "period": "1m0s",
                                    "processors": [
                                        {
                                            "add_fields": {
                                                "fields": {
                                                    "dataset": "elastic_agent.filebeat",
                                                    "namespace": "default",
                                                    "type": "metrics"
                                                },
                                                "target": "data_stream"
                                            }
                                        },
                                        {
                                            "add_fields": {
                                                "fields": {
                                                    "dataset": "elastic_agent.filebeat"
                                                },
                                                "target": "event"
                                            }
                                        },
                                        {
                                            "add_fields": {
                                                "fields": {
                                                    "id": "7f563b50-11f1-4e7c-8361-31458142a891",
                                                    "process": "filebeat",
                                                    "snapshot": false,
                                                    "version": "9.0.0"
                                                },
                                                "target": "elastic_agent"
                                            }
                                        },
                                        {
                                            "add_fields": {
                                                "fields": {
                                                    "id": "7f563b50-11f1-4e7c-8361-31458142a891"
                                                },
                                                "target": "agent"
                                            }
                                        },
                                        {
                                            "add_fields": {
                                                "fields": {
                                                    "binary": "filebeat",
                                                    "id": "filestream-monitoring"
                                                },
                                                "target": "component"
                                            }
                                        }
                                    ]
                                }
                            ],
                            "type": "beat/metrics"
                        },
                        "streams": [
                            {
                                "data_stream": {
                                    "dataset": "elastic_agent.metricbeat",
                                    "namespace": "default",
                                    "source": {
                                        "dataset": "elastic_agent.metricbeat",
                                        "namespace": "default",
                                        "type": "metrics"
                                    },
                                    "type": "metrics"
                                },
                                "id": "metrics-monitoring-metricbeat",
                                "source": {
                                    "data_stream": {
                                        "dataset": "elastic_agent.metricbeat",
                                        "namespace": "default",
                                        "type": "metrics"
                                    },
                                    "failure_threshold": 5,
                                    "hosts": [
                                        "http+unix:///vagrant/data/tmp/Hk6rvk9TDibMPcDvpl0jkLE-qDsHWVYL.sock"
                                    ],
                                    "id": "metrics-monitoring-metricbeat",
                                    "index": "metrics-elastic_agent.metricbeat-default",
                                    "metricsets": [
                                        "stats"
                                    ],
                                    "period": "1m0s",
                                    "processors": [
                                        {
                                            "add_fields": {
                                                "fields": {
                                                    "dataset": "elastic_agent.metricbeat",
                                                    "namespace": "default",
                                                    "type": "metrics"
                                                },
                                                "target": "data_stream"
                                            }
                                        },
                                        {
                                            "add_fields": {
                                                "fields": {
                                                    "dataset": "elastic_agent.metricbeat"
                                                },
                                                "target": "event"
                                            }
                                        },
                                        {
                                            "add_fields": {
                                                "fields": {
                                                    "id": "7f563b50-11f1-4e7c-8361-31458142a891",
                                                    "process": "metricbeat",
                                                    "snapshot": false,
                                                    "version": "9.0.0"
                                                },
                                                "target": "elastic_agent"
                                            }
                                        },
                                        {
                                            "add_fields": {
                                                "fields": {
                                                    "id": "7f563b50-11f1-4e7c-8361-31458142a891"
                                                },
                                                "target": "agent"
                                            }
                                        },
                                        {
                                            "add_fields": {
                                                "fields": {
                                                    "binary": "metricbeat",
                                                    "id": "beat/metrics-monitoring"
                                                },
                                                "target": "component"
                                            }
                                        }
                                    ]
                                }
                            },
                            {
                                "data_stream": {
                                    "dataset": "elastic_agent.metricbeat",
                                    "namespace": "default",
                                    "source": {
                                        "dataset": "elastic_agent.metricbeat",
                                        "namespace": "default",
                                        "type": "metrics"
                                    },
                                    "type": "metrics"
                                },
                                "id": "metrics-monitoring-metricbeat",
                                "source": {
                                    "data_stream": {
                                        "dataset": "elastic_agent.metricbeat",
                                        "namespace": "default",
                                        "type": "metrics"
                                    },
                                    "failure_threshold": 5,
                                    "hosts": [
                                        "http+unix:///vagrant/data/tmp/akSPbdqgaHaTY0_J01-dsfYK6JpMz2zn.sock"
                                    ],
                                    "id": "metrics-monitoring-metricbeat",
                                    "index": "metrics-elastic_agent.metricbeat-default",
                                    "metricsets": [
                                        "stats"
                                    ],
                                    "period": "1m0s",
                                    "processors": [
                                        {
                                            "add_fields": {
                                                "fields": {
                                                    "dataset": "elastic_agent.metricbeat",
                                                    "namespace": "default",
                                                    "type": "metrics"
                                                },
                                                "target": "data_stream"
                                            }
                                        },
                                        {
                                            "add_fields": {
                                                "fields": {
                                                    "dataset": "elastic_agent.metricbeat"
                                                },
                                                "target": "event"
                                            }
                                        },
                                        {
                                            "add_fields": {
                                                "fields": {
                                                    "id": "7f563b50-11f1-4e7c-8361-31458142a891",
                                                    "process": "metricbeat",
                                                    "snapshot": false,
                                                    "version": "9.0.0"
                                                },
                                                "target": "elastic_agent"
                                            }
                                        },
                                        {
                                            "add_fields": {
                                                "fields": {
                                                    "id": "7f563b50-11f1-4e7c-8361-31458142a891"
                                                },
                                                "target": "agent"
                                            }
                                        },
                                        {
                                            "add_fields": {
                                                "fields": {
                                                    "binary": "metricbeat",
                                                    "id": "http/metrics-monitoring"
                                                },
                                                "target": "component"
                                            }
                                        }
                                    ]
                                }
                            },
                            {
                                "data_stream": {
                                    "dataset": "elastic_agent.filebeat",
                                    "namespace": "default",
                                    "source": {
                                        "dataset": "elastic_agent.filebeat",
                                        "namespace": "default",
                                        "type": "metrics"
                                    },
                                    "type": "metrics"
                                },
                                "id": "metrics-monitoring-filebeat",
                                "source": {
                                    "data_stream": {
                                        "dataset": "elastic_agent.filebeat",
                                        "namespace": "default",
                                        "type": "metrics"
                                    },
                                    "failure_threshold": 5,
                                    "hosts": [
                                        "http+unix:///vagrant/data/tmp/xTEtpJ7117ppc6OYvJCaYHbDW8mLjXGe.sock"
                                    ],
                                    "id": "metrics-monitoring-filebeat",
                                    "index": "metrics-elastic_agent.filebeat-default",
                                    "metricsets": [
                                        "stats"
                                    ],
                                    "period": "1m0s",
                                    "processors": [
                                        {
                                            "add_fields": {
                                                "fields": {
                                                    "dataset": "elastic_agent.filebeat",
                                                    "namespace": "default",
                                                    "type": "metrics"
                                                },
                                                "target": "data_stream"
                                            }
                                        },
                                        {
                                            "add_fields": {
                                                "fields": {
                                                    "dataset": "elastic_agent.filebeat"
                                                },
                                                "target": "event"
                                            }
                                        },
                                        {
                                            "add_fields": {
                                                "fields": {
                                                    "id": "7f563b50-11f1-4e7c-8361-31458142a891",
                                                    "process": "filebeat",
                                                    "snapshot": false,
                                                    "version": "9.0.0"
                                                },
                                                "target": "elastic_agent"
                                            }
                                        },
                                        {
                                            "add_fields": {
                                                "fields": {
                                                    "id": "7f563b50-11f1-4e7c-8361-31458142a891"
                                                },
                                                "target": "agent"
                                            }
                                        },
                                        {
                                            "add_fields": {
                                                "fields": {
                                                    "binary": "filebeat",
                                                    "id": "filestream-monitoring"
                                                },
                                                "target": "component"
                                            }
                                        }
                                    ]
                                }
                            }
                        ],
                        "type": "beat/metrics"
                    },
                    "Err": null,
                    "ID": "beat/metrics-monitoring-metrics-monitoring-beats",
                    "LogLevel": 3,
                    "Type": 0
                },
                {
                    "Config": {
                        "data_stream": {},
                        "source": {
                            "api_key": "example-key",
                            "hosts": [
                                "127.0.0.1:9200"
                            ],
                            "preset": "balanced",
                            "type": "elasticsearch"
                        },
                        "type": "elasticsearch"
                    },
                    "Err": null,
                    "ID": "beat/metrics-monitoring",
                    "LogLevel": 3,
                    "Type": 1
                }
            ]
        },
        {
            "Component": {
                "limits": {
                    "source": {
                        "go_max_procs": 0
                    }
                }
            },
            "Err": {
                "Reason": "input not supported"
            },
            "ErrMsg": "",
            "Features": {
                "fqdn": {},
                "source": {
                    "agent": {
                        "features": {
                            "fqdn": {
                                "enabled": false
                            }
                        }
                    }
                }
            },
            "ID": "http/metrics-monitoring",
            "InputSpec": {
                "BinaryName": "",
                "BinaryPath": "",
                "InputType": "",
                "Spec": {
                    "Aliases": null,
                    "Command": null,
                    "Description": "",
                    "IsolateUnits": false,
                    "Name": "",
                    "Outputs": null,
                    "Platforms": null,
                    "ProxiedActions": null,
                    "Runtime": {
                        "Preventions": null
                    },
                    "Service": null
                }
            },
            "InputType": "http/metrics",
            "OutputType": "elasticsearch",
            "Units": [
                {
                    "Config": {
                        "data_stream": {
                            "namespace": "default",
                            "source": {
                                "namespace": "default"
                            }
                        },
                        "id": "metrics-monitoring-agent",
                        "name": "metrics-monitoring-agent",
                        "source": {
                            "data_stream": {
                                "namespace": "default"
                            },
                            "id": "metrics-monitoring-agent",
                            "name": "metrics-monitoring-agent",
                            "streams": [
                                {
                                    "data_stream": {
                                        "dataset": "elastic_agent.elastic_agent",
                                        "namespace": "default",
                                        "type": "metrics"
                                    },
                                    "failure_threshold": 5,
                                    "hosts": [
                                        "http://localhost:6791"
                                    ],
                                    "id": "metrics-monitoring-agent",
                                    "index": "metrics-elastic_agent.elastic_agent-default",
                                    "metricsets": [
                                        "json"
                                    ],
                                    "namespace": "agent",
                                    "path": "/stats",
                                    "period": "1m0s",
                                    "processors": [
                                        {
                                            "add_fields": {
                                                "fields": {
                                                    "dataset": "elastic_agent.elastic_agent",
                                                    "namespace": "default",
                                                    "type": "metrics"
                                                },
                                                "target": "data_stream"
                                            }
                                        },
                                        {
                                            "add_fields": {
                                                "fields": {
                                                    "dataset": "elastic_agent.elastic_agent"
                                                },
                                                "target": "event"
                                            }
                                        },
                                        {
                                            "add_fields": {
                                                "fields": {
                                                    "id": "7f563b50-11f1-4e7c-8361-31458142a891",
                                                    "process": "elastic-agent",
                                                    "snapshot": false,
                                                    "version": "9.0.0"
                                                },
                                                "target": "elastic_agent"
                                            }
                                        },
                                        {
                                            "add_fields": {
                                                "fields": {
                                                    "id": "7f563b50-11f1-4e7c-8361-31458142a891"
                                                },
                                                "target": "agent"
                                            }
                                        },
                                        {
                                            "copy_fields": {
                                                "fail_on_error": false,
                                                "fields": [
                                                    {
                                                        "from": "http.agent.beat.cpu",
                                                        "to": "system.process.cpu"
                                                    },
                                                    {
                                                        "from": "http.agent.beat.memstats.memory_sys",
                                                        "to": "system.process.memory.size"
                                                    },
                                                    {
                                                        "from": "http.agent.beat.handles",
                                                        "to": "system.process.fd"
                                                    },
                                                    {
                                                        "from": "http.agent.beat.cgroup",
                                                        "to": "system.process.cgroup"
                                                    },
                                                    {
                                                        "from": "http.agent.apm-server",
                                                        "to": "apm-server"
                                                    },
                                                    {
                                                        "from": "http.filebeat_input",
                                                        "to": "filebeat_input"
                                                    }
                                                ],
                                                "ignore_missing": true
                                            }
                                        },
                                        {
                                            "drop_fields": {
                                                "fields": [
                                                    "http"
                                                ],
                                                "ignore_missing": true
                                            }
                                        },
                                        {
                                            "add_fields": {
                                                "fields": {
                                                    "binary": "elastic-agent",
                                                    "id": "elastic-agent"
                                                },
                                                "target": "component"
                                            }
                                        }
                                    ]
                                },
                                {
                                    "data_stream": {
                                        "dataset": "elastic_agent.elastic_agent",
                                        "namespace": "default",
                                        "type": "metrics"
                                    },
                                    "failure_threshold": 5,
                                    "hosts": [
                                        "http+unix:///vagrant/data/tmp/Hk6rvk9TDibMPcDvpl0jkLE-qDsHWVYL.sock"
                                    ],
                                    "id": "metrics-monitoring-metricbeat-1",
                                    "index": "metrics-elastic_agent.elastic_agent-default",
                                    "metricsets": [
                                        "json"
                                    ],
                                    "namespace": "agent",
                                    "path": "/stats",
                                    "period": "1m0s",
                                    "processors": [
                                        {
                                            "add_fields": {
                                                "fields": {
                                                    "dataset": "elastic_agent.elastic_agent"
                                                },
                                                "target": "event"
                                            }
                                        },
                                        {
                                            "add_fields": {
                                                "fields": {
                                                    "id": "7f563b50-11f1-4e7c-8361-31458142a891",
                                                    "process": "metricbeat",
                                                    "snapshot": false,
                                                    "version": "9.0.0"
                                                },
                                                "target": "elastic_agent"
                                            }
                                        },
                                        {
                                            "add_fields": {
                                                "fields": {
                                                    "id": "7f563b50-11f1-4e7c-8361-31458142a891"
                                                },
                                                "target": "agent"
                                            }
                                        },
                                        {
                                            "copy_fields": {
                                                "fail_on_error": false,
                                                "fields": [
                                                    {
                                                        "from": "http.agent.beat.cpu",
                                                        "to": "system.process.cpu"
                                                    },
                                                    {
                                                        "from": "http.agent.beat.memstats.memory_sys",
                                                        "to": "system.process.memory.size"
                                                    },
                                                    {
                                                        "from": "http.agent.beat.handles",
                                                        "to": "system.process.fd"
                                                    },
                                                    {
                                                        "from": "http.agent.beat.cgroup",
                                                        "to": "system.process.cgroup"
                                                    },
                                                    {
                                                        "from": "http.agent.apm-server",
                                                        "to": "apm-server"
                                                    },
                                                    {
                                                        "from": "http.filebeat_input",
                                                        "to": "filebeat_input"
                                                    }
                                                ],
                                                "ignore_missing": true
                                            }
                                        },
                                        {
                                            "drop_fields": {
                                                "fields": [
                                                    "http"
                                                ],
                                                "ignore_missing": true
                                            }
                                        },
                                        {
                                            "add_fields": {
                                                "fields": {
                                                    "binary": "metricbeat",
                                                    "id": "beat/metrics-monitoring"
                                                },
                                                "target": "component"
                                            }
                                        }
                                    ]
                                },
                                {
                                    "data_stream": {
                                        "dataset": "elastic_agent.elastic_agent",
                                        "namespace": "default",
                                        "type": "metrics"
                                    },
                                    "failure_threshold": 5,
                                    "hosts": [
                                        "http+unix:///vagrant/data/tmp/akSPbdqgaHaTY0_J01-dsfYK6JpMz2zn.sock"
                                    ],
                                    "id": "metrics-monitoring-metricbeat-1",
                                    "index": "metrics-elastic_agent.elastic_agent-default",
                                    "metricsets": [
                                        "json"
                                    ],
                                    "namespace": "agent",
                                    "path": "/stats",
                                    "period": "1m0s",
                                    "processors": [
                                        {
                                            "add_fields": {
                                                "fields": {
                                                    "dataset": "elastic_agent.elastic_agent"
                                                },
                                                "target": "event"
                                            }
                                        },
                                        {
                                            "add_fields": {
                                                "fields": {
                                                    "id": "7f563b50-11f1-4e7c-8361-31458142a891",
                                                    "process": "metricbeat",
                                                    "snapshot": false,
                                                    "version": "9.0.0"
                                                },
                                                "target": "elastic_agent"
                                            }
                                        },
                                        {
                                            "add_fields": {
                                                "fields": {
                                                    "id": "7f563b50-11f1-4e7c-8361-31458142a891"
                                                },
                                                "target": "agent"
                                            }
                                        },
                                        {
                                            "copy_fields": {
                                                "fail_on_error": false,
                                                "fields": [
                                                    {
                                                        "from": "http.agent.beat.cpu",
                                                        "to": "system.process.cpu"
                                                    },
                                                    {
                                                        "from": "http.agent.beat.memstats.memory_sys",
                                                        "to": "system.process.memory.size"
                                                    },
                                                    {
                                                        "from": "http.agent.beat.handles",
                                                        "to": "system.process.fd"
                                                    },
                                                    {
                                                        "from": "http.agent.beat.cgroup",
                                                        "to": "system.process.cgroup"
                                                    },
                                                    {
                                                        "from": "http.agent.apm-server",
                                                        "to": "apm-server"
                                                    },
                                                    {
                                                        "from": "http.filebeat_input",
                                                        "to": "filebeat_input"
                                                    }
                                                ],
                                                "ignore_missing": true
                                            }
                                        },
                                        {
                                            "drop_fields": {
                                                "fields": [
                                                    "http"
                                                ],
                                                "ignore_missing": true
                                            }
                                        },
                                        {
                                            "add_fields": {
                                                "fields": {
                                                    "binary": "metricbeat",
                                                    "id": "http/metrics-monitoring"
                                                },
                                                "target": "component"
                                            }
                                        }
                                    ]
                                },
                                {
                                    "data_stream": {
                                        "dataset": "elastic_agent.elastic_agent",
                                        "namespace": "default",
                                        "type": "metrics"
                                    },
                                    "failure_threshold": 5,
                                    "hosts": [
                                        "http+unix:///vagrant/data/tmp/xTEtpJ7117ppc6OYvJCaYHbDW8mLjXGe.sock"
                                    ],
                                    "id": "metrics-monitoring-filebeat-1",
                                    "index": "metrics-elastic_agent.elastic_agent-default",
                                    "metricsets": [
                                        "json"
                                    ],
                                    "namespace": "agent",
                                    "path": "/stats",
                                    "period": "1m0s",
                                    "processors": [
                                        {
                                            "add_fields": {
                                                "fields": {
                                                    "dataset": "elastic_agent.elastic_agent"
                                                },
                                                "target": "event"
                                            }
                                        },
                                        {
                                            "add_fields": {
                                                "fields": {
                                                    "id": "7f563b50-11f1-4e7c-8361-31458142a891",
                                                    "process": "filebeat",
                                                    "snapshot": false,
                                                    "version": "9.0.0"
                                                },
                                                "target": "elastic_agent"
                                            }
                                        },
                                        {
                                            "add_fields": {
                                                "fields": {
                                                    "id": "7f563b50-11f1-4e7c-8361-31458142a891"
                                                },
                                                "target": "agent"
                                            }
                                        },
                                        {
                                            "copy_fields": {
                                                "fail_on_error": false,
                                                "fields": [
                                                    {
                                                        "from": "http.agent.beat.cpu",
                                                        "to": "system.process.cpu"
                                                    },
                                                    {
                                                        "from": "http.agent.beat.memstats.memory_sys",
                                                        "to": "system.process.memory.size"
                                                    },
                                                    {
                                                        "from": "http.agent.beat.handles",
                                                        "to": "system.process.fd"
                                                    },
                                                    {
                                                        "from": "http.agent.beat.cgroup",
                                                        "to": "system.process.cgroup"
                                                    },
                                                    {
                                                        "from": "http.agent.apm-server",
                                                        "to": "apm-server"
                                                    },
                                                    {
                                                        "from": "http.filebeat_input",
                                                        "to": "filebeat_input"
                                                    }
                                                ],
                                                "ignore_missing": true
                                            }
                                        },
                                        {
                                            "drop_fields": {
                                                "fields": [
                                                    "http"
                                                ],
                                                "ignore_missing": true
                                            }
                
(…)

* feat: Prevent leaking secrets when logging component model

* add changelog entry

* test: add test checking that secrets are not leaked when components are marhsaled to JSON

(cherry picked from commit d359ccd)
@mergify mergify bot requested a review from a team as a code owner December 18, 2024 09:18
@mergify mergify bot added the backport label Dec 18, 2024
@mergify mergify bot requested review from michalpristas and blakerouse and removed request for a team December 18, 2024 09:18
@andrzej-stencel andrzej-stencel enabled auto-merge (squash) December 19, 2024 12:16
Copy link
Contributor Author

mergify bot commented Dec 23, 2024

This pull request has not been merged yet. Could you please review and merge it @andrzej-stencel? 🙏

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant