Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Metrics and clearer logs for denied scaling #940

Open
sharnoff opened this issue May 21, 2024 · 0 comments
Open

Metrics and clearer logs for denied scaling #940

sharnoff opened this issue May 21, 2024 · 0 comments
Labels
c/autoscaling/autoscaler-agent Component: autoscaling: autoscaler-agent

Comments

@sharnoff
Copy link
Member

sharnoff commented May 21, 2024

Problem description / Motivation

We don't have clear signals for when the scheduler denies upscaling, or the vm-monitor denies downscaling.

This came up here: https://neondb.slack.com/archives/C03TN5G758R/p1716167732610919?thread_ts=1716166712.233079

Feature idea(s) / DoD

The autoscaler-agent can:

  1. Expose new instance-level metrics for amount of scaling requests that were not satisfied
  2. Change the log level of scheduler responses that didn't grant the full request to "warn", rather than just "info"

Implementation ideas

Should be simple enough to inject these alongside the existing logs & metrics, wherever they're defined (IIRC exec_bridge / executor ?)

@sharnoff sharnoff added the c/autoscaling/autoscaler-agent Component: autoscaling: autoscaler-agent label May 21, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
c/autoscaling/autoscaler-agent Component: autoscaling: autoscaler-agent
Projects
None yet
Development

No branches or pull requests

1 participant