Run query based on URL POST Request

[mrueg]

It would be a nice feature, if prometheus-es-exporter could execute a query based on a POST request, similar to blackbox_exporter (compare e.g. https://github.com/prometheus/blackbox_exporter#checking-the-results )
e.g. https://es-exporter/metrics?query=$MY_QUERY

This has certain advantages, as one could quickly replace the query without redeploying the exporter or run multiple queries (based on different requests) at different intervals.

[braedon]

Hi @mrueg, could you clarify what kind of queries you'd like to run?

We could use an endpoint like this to run a query string query and get basic count metrics, but I don't know of a reasonable way to do aggregations?

[askz]

we could support urlencoded or base64 encoded query string to achieve this

[braedon]

Yeah, it's definitely possible to encode the query JSON in a URL, but it doesn't seem like it'd be very user friendly...

Just supporting doc count metrics from a query string query would be a lot friendlier - it's a direct translation from the ES API - but I'm not sure if that'd satisfy @mrueg's use case.

To be honest, I'm a little reluctant to add a custom metrics endpoint full stop, as it'd require moving away from the built-in Prometheus library HTTP server. There's also potentially extra security implications - the ability to run arbitrary queries on the cluster could be used to extract sensitive data if an attacker has access to the exporter metrics endpoint, but not the Elasticsearch API itself.

I'm open to adding it if there's a clear benefit and user demand, but need to be convinced 😄.

[mrueg]

@braedon The setup I image would use prometheus-operator on Kubernetes with trusted users being able to hit the prometheus-es-exporter with custom queries by creating their own ServiceMonitor objects. I agree that the endpoint needs protection from abuse, similar to blackbox-exporter or others that allow running custom queries.

The advantages I see from this approach is getting an easy way to update/change queries without redeploying the whole exporter and its configuration as well as allowing multiple users to share the same exporter instance.