Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature/691 Add Bucket and Task Creation Support to Helm Chart #692

Open
wants to merge 4 commits into
base: master
Choose a base branch
from

Conversation

davidpepm
Copy link

Pull Request Summary: Add Bucket and Task Creation Support to Helm Chart

Overview

This pull request enhances the Helm chart for InfluxDB by introducing the ability to create aggregated buckets and tasks for data aggregation. The feature addresses performance considerations by enabling the creation of pre-aggregated data.

Key Changes

  1. New Aggregated Buckets

    • Adds support for creating the following buckets:
      • default_agg_topic
      • default_agg_stats
      • default_agg_kpi
    • Each bucket is configured with a retention policy of "forever" (0s).
  2. Data Aggregation Task

    • Introduces a scheduled task to aggregate data into the new buckets.
    • The task runs every 6 hours and processes data from the default bucket into the aggregated buckets.
  3. Helm Chart Modifications

    • values.yaml: Includes new configuration to toggle bucket and task creation using the createBucketsAndTask.enabled flag.
    • ConfigMap: Stores the aggregation task script for maintainability.
    • Job: Automates bucket and task creation during deployment via a Kubernetes Job.

Testing and Validation

  • I am unable to set up the required Kubernetes environment (e.g., Minikube) for testing the implementation locally. Assistance is needed to validate these changes in a test environment.

Next Steps

  • Review the changes and test the setup to confirm functionality.
  • Merge this pull request into the main branch after testing is successful.

Related Issue

This pull request addresses [Issue #691].
Including the keyword Closes #691 in the pull request description will link the issue and close it upon merging.


Request for Assistance

  • Testing: Help is needed to validate the changes in a Kubernetes environment since I lack the setup and expertise in tools like Minikube.
  • Merge: Once reviewed and validated, either grant me permission to merge or let someone with access proceed with the merge.

Thanks for reviewing this!

- Added configuration for `default_agg_topic`, `default_agg_stats`, and `default_agg_kpi` buckets with retention policy set to forever.
- Added support for task creation to aggregate data into the new buckets.(the flux query to be added)
- Introduced `createBucketsAndTask.enabled` flag to toggle this feature.
- Implemented task script to populate data into these buckets with a 6-hour interval.
- Introduced a ConfigMap template to store the task script (`task.flux`) from values.yaml.
- Ensures the task script can be used by the job for bucket and task creation.
- Added a Job template to create aggregated buckets and the task using the provided ConfigMap script.
- Ensures buckets and tasks are initialized during deployment.
- Supports toggling the feature via `createBucketsAndTask.enabled`.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
2 participants