nginx scrap not working

[souhaiebtar]

i added

- job_name: 'nginx'
    scrape_interval: 1s
    static_configs:
         - targets: ['51.158.67.59:9913']

to prometheus/conf/prometheus.yml

but when i open 51.158.67.59:9090 and query to get data for this job, nothing is showen,

i already tested the same config on docker-compose locally using prometheus and everything worked properly,
just to now the address is hosted on the internet and everything work properly

[mh720]

Prometheus doesn't know how to scrape metrics directly from nginx, you'll need to use a Prometheus 'exporter' service that knows how to talk to nginx to read it's internal metrics format and present it as a scrape-able source to Prometheus. It looks like nginx provides one:

https://github.com/nginxinc/nginx-prometheus-exporter

And I see other community Prometheus exporters for nginx stats such as https://github.com/hnlq715/nginx-vts-exporter

[anthraxn8b]

https://www.martin-helmich.de/de/blog/monitoring-nginx.html

Try ths one!

If you need a separate logfile: nginx allows you to add multiple access_log entries on the same level!

[shimerg]

Hello!
I have a same problem with posgres scrap.
I use 'exporter' service (this one https://github.com/wrouesnel/postgres_exporter/issues) and add to prometheus/conf/prometheus.yml:

  - job_name: 'postgre_exporter'
    static_configs:
      - targets: ['localhost:9187']

and as next step - rebuild docker stack (docker stack deploy -c docker-compose.yml mon)

but when i try to get data for this job in prometheus, nothing is showen, and in web (Prometheus -> Status -> Configuration).
in http://xxx.xxx.xxx.xxx:9187/metrics i can see all metric info.
What am i doing wrong?

[mh720]

If your prom is running in a container, localhost:9187 will resolve to that prom container's 127.0.0.1 address. If you are running prom and postgres as Docker compose services inside the same Docker swarm overlay network - you'll likely want to enable encryption for the overlay network, see swarmstack's compose for examples if needed. You should be able to have prom scrape the service name as listed in your swarm compose file, .e.g. postgres:9187.

Otherwise, you may need to expose the postgres container's port 9187 to the Docker host network, and address prom towards either a Docker node IP or DNS record, or a round-robin DNS record for some or all of your swarm hosts.

[shimerg]

Thank you for your answer!
I changed that section in prometheus/conf/prometheus.yml to

  - job_name: 'postgre_exporter'
    static_configs:
      - targets: ['xxx.xxx.xxx.xxx:9187']

and did docker stack deploy -c docker-compose.yml mon
but nothing happened - seems like prometheus doesn't recieve a new configuration

[shimerg]

Ok, i did it!
But:

1. I needed to restart stack with --web.enable-lifecycle flag in docker-compose.yml (prometheus section) - for reload prometheus configuration with curl
2. In container with Prometheus (with docker exec) i added in config file in /etc/prometheus/prometheus.yml this:

  - job_name: 'postgres-exporter'
    dns_sd_configs:
    - names:
      - 'cranky_keldysh'
      type: 'A'
      port: 9187

where is cranky-keldysh is a name of container with postgres_exporter
I hope, my expirience will help someone.