Aegis for MinIO

This service manages the scanning of new files uploaded to MinIO. It is connected to Kafka to read PUT events, download the file into a cache and then scan. Depending on the output, the files tags will be updated with the result of the scan.

Contents

• Setup
• Config
• Make Commands
• Usage

Setup

Local Environment

External dependencies that need to be pre-installed:

• Go
• MinIO
• Prometheus
• Kafka and Zookeeper
• Clamav
• Postgresql
• Grafana (optional)

Once installed, start the external dependencies. This can be done using the commands below. Prometheus requires a config file to be passed in shown below. Any changes to the following commands may require changes to the config.env.

MinIO:

minio server <path_to_data>

Kafka:

zookeeper-server-start ./local-configs/zoo.cfg
kafka-server-start ./local-configs/kafka.properties
kafka-topics --create --topic minio-put-events --bootstrap-server localhost:9092

ClamAV. Change ./clamd.conf to use localhost

clamd

Prometheus

prometheus --config.file=./local-configs/prometheus.yml

Grafana

grafana server --config ./local-configs/grafana.ini --homepath /opt/homebrew/share/grafana

Postgresql

pg_ctl -D <postgresql_location> start

Finally, start the Aegis service.

make build && make run

Thats it! Navigate to http://localhost:9000 to view interact with MinIO. Checkout the usage section for how to configure the MinIO client.

Kubernetes Environment

External dependencies that need to be pre-installed:

• k3d
• Helm
• Docker
• Kubernetes

Make sure dependencies are installed and configured correctly.

From within project root you can use the make command to launch the kubernetes environment created by k3d.

make docker-build && make create-cluster

Using kubectl you can verify that all pods and services have started correctly.

kubectl get pods
kubectl get svc

Navigate to http://localhost:9001 to view interact with MinIO. Checkout the usage section for how to configure the MinIO client.

Default credentials: user: minioadmin password: minioadmin

To connect to the exposed postgresql database

psql --host localhost --username postgres

To connect Prometheus to Grafana first start the Grafana service.

grafana server --config ./local-services/grafana.ini

Then connect to the exposed Prometheus service.

Prometheus endpoint: localhost:9001/prometheus

Testing

Dependencies:

• Go
• mockery

make mock && make test

Config

All configuration is stored within the config.env file but can be overridden by environment variables.

Default configuration

### AEGIS CONFIG ###
# String: info or debug
AEGIS_LOGGER_LEVEL=info

# String: Console or json
AEGIS_LOGGER_ENCODING=json

# Bool: remove files from cache after scan
AEGIS_REMOVE_AFTER_SCAN=true

# String: tag, remove or quarantine CAUTION WHEN USING REMOVE
AEGIS_CLEANUP_POLICY=tag

# String: Name of quanrantine bucket. Required if AEGIS_CLEANUP_POLICY=quarantine
AEGIS_QUARANTINE_BUCKET=

MINIO_ENDPOINT=127.0.0.1:9000
MINIO_ACCESS_KEY=minioadmin
MINIO_SECRET_KEY=minioadmin
MINIO_USE_SSL=false

KAFKA_BROKERS="127.0.0.1:9092"
KAFKA_TOPIC=minio-put-events
KAFKA_GROUP_ID=g1
KAFKA_MAX_BYTES=10

CLAMAV_REMOVE_AFTER_SCAN=true
CLAMAV_DATETIME_FORMAT="01-02-2006 15:04:05"
CLAMAV_PATH=cache/

PROMETHEUS_ENDPOINT=127.0.0.1:2112
PROMETHEUS_PATH=/metrics

POSTGRESQL_USERNAME=postgres
POSTGRESQL_PASSWORD=postgres
POSTGRESQL_ENDPOINT=127.0.0.1:5432
POSTGRESQL_DATABASE=aegis_antivirus
POSTGRESQL_TABLE=aegis_audit_logs

Make Commands

help             Print this message
build            Create the binary
run              Run the binary
vendor           Download the vendored dependencies
test             Run the tests
mock             Generate the mocks for testing
docker-build     Build the docker image
create-cluster   Create the k3d cluster
delete-cluster   Delete the k3d cluster
rebuild-cluster  Delete and recreate the cluster

Usage

Overview

Aegis is designed to bring security to the open-source object store MinIO. Aegis listens for PUT notifications from MinIO which passed on to Kafkas event queue. These notifications are then read in by Aegis where files are retrieved into a temporary cache. Aegis then distributes the files to various antivirus engines for scanning. The results are collated and then the file is tagged with the results. A Postgresql audit log is also collecting to track change over time. Metrics are also collected and exposed to Prometheus which can be consumed in Grafana for visualisation.

MinIO

When the Aegis local or cluster setup has been created, MinIO need to be configured to send notifications to Kafka. In dashboard > Events > Add Event Destination. Select Kafka as the queue type and enter the brokers address and topic name. The brokers address change depending on your setup but these are normally 127.0.0.1:9092 for local or aegis-kafka.default.svc.cluster.local:9092 for kubernetes. The topic name can be anything you like but it must match the topic name in the config.env file.

Next, you should configure every bucket that requires file scanning. Buckets > > Settings > Events > Subscribe to Event. From the drop down, select the Kafka event queue ARN and also the PUT event type. This will send a notification to Kafka every time a file is uploaded to the bucket.

This should be all the setup you need to utilise Aegis in your object storage stack.

Grafana

Prometheus metrics are exposed to http://localhost:9090. These can be consumed in Grafana to visualise the data.

ClamAV

When starting the deployment within kubernetes the clamav will start and request a signature database update. Sometimes this update can trigger too many times causing a request cooldown. This will cause errors with object scanning but can be solved by simply waiting for this cooldown to expire.