springboot-log-elastic

POC with SpringBoot (2.5.0) simple API App with Elastic stack (7.13.0) / logstash -> elasticsearch -> kibana (everything in docker)

Run

• git clone https://github.com/keuss/springboot-log-elastic
• mvn clean package
• docker-compose build (from docker-elk folder with the docker-compose.yml file)
• docker-compose up (from docker-elk folder with the docker-compose.yml file)
• To shutdown the stack and remove all persisted data docker-compose down -v (from docker-elk folder with the docker-compose.yml file)

Stack

• Logback with LogstashEncoder
• Elastic with docker with xpack.license basic
• Logstash with GELF log driver (UDP protocol and port 5000 to send messages from Docker services)
• Add Index Pattern (from Home/Manage) : Index "logback-*" and time filter field name @timestamp (must have some data first for this index !)

• Filter by tag (bookapi_app_1 or bookapi_app_2)

• Dashboard example with custom tag

Search Query examples

• tag : bookapi_app_2 AND logcontent.level : ERROR

URLs

• API Swagger to add data (create some books) : app1 / app2
• Kibana : http://localhost:5601/ (elastic/changeme), add sample data first, then create logback-* index

Links

Logback

• https://www.baeldung.com/logback
• https://github.com/logstash/logstash-logback-encoder
• https://www.innoq.com/en/blog/structured-logging/ (nice feature like keyValue, ...)
• https://dev.to/anandsunderraman/json-logging-in-spring-boot-applications-2j33

Send the Logs of a Java App to the Elastic Stack

• https://www.baeldung.com/java-application-logs-to-elastic-stack
• https://medium.com/@ridwanfajar/send-your-container-logs-to-elk-elasticsearch-logstash-and-kibana-with-gelf-driver-7995714fbbad

Gelf

• https://docs.docker.com/config/containers/logging/gelf/
• https://stackoverflow.com/questions/44050803/unable-to-connect-docker-container-to-logstash-via-gelf-driver
• https://www.docker.com/blog/adventures-in-gelf/
• https://blog.zenika.com/2016/02/15/consolider-les-logs-docker-dans-un-elk/

Other Docker logging driver

• https://docs.docker.com/config/containers/logging/configure/
• Example : https://docs.docker.com/config/containers/logging/splunk/

Docker Port Binding

• https://docs.docker.com/config/containers/container-networking/
• https://betterprogramming.pub/how-does-docker-port-binding-work-b089f23ca4c8
• 0.0.0.0 vs localhost, see here and here, using 0.0.0.0 to listen inside the docker application will work (by default, Docker binds published container ports to the 0.0.0.0 IP address, with matches any IP address on the system) 0.0.0.0 <=> 127.0.0.1 (localhost) !

Disable compose v2

• fastapi/full-stack-fastapi-template#427