Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: allow to reset histograms #305

Open
vmarchaud opened this issue Nov 28, 2022 · 5 comments
Open

feat: allow to reset histograms #305

vmarchaud opened this issue Nov 28, 2022 · 5 comments
Labels
enhancement New feature or request

Comments

@vmarchaud
Copy link

I'm collecting runTime and waitTime metrics for my pool with otel:

const metrics = {
  workersQueueWaitTime: meter.createObservableGauge('content_workers_queue_wait_time'),
  workersQueueLatency: meter.createObservableGauge('content_workers_queue_latency')
} as const

// later on
const percentiles = ['p50', 'p99', 'p99_9'] as const
metrics.workersQueueWaitTime.addCallback((result) => {
  for (const percentile of percentiles) {
    result.observe(this.workers.waitTime[percentile], { percentile })
  }
})
metrics.workersQueueLatency.addCallback((result) => {
  for (const percentile of percentiles) {
    result.observe(this.workers.runTime[percentile], { percentile })
  }
})

However over the lifetime of the pool, those histogram are never reset so the p99 and p99_9 take a lot of time to get back to a current value since it stores old one. I would like to be able to call this.workers.runTime.reset() to clear the buckets after each value collection.

If that's something you are okay with, i can make a PR for it

@metcoder95
Copy link
Member

Just out of curiosity, what would be the shape of the API?
Just to reset the histograms at a given point in time?

Along with that, do you have a distributed setup? How do you measure the distribution among hosts?

@metcoder95 metcoder95 added the enhancement New feature or request label Jun 12, 2023
Copy link

This issue has been marked as stale because it has been opened 30 days without activity. Remove stale label or comment or this will be closed in 5 days.

@github-actions github-actions bot added the stale label May 12, 2024
Copy link

This issue was closed because it has been stalled for 5 days with no activity.

@github-actions github-actions bot closed this as not planned Won't fix, can't repro, duplicate, stale May 26, 2024
@metcoder95 metcoder95 reopened this May 29, 2024
@metcoder95 metcoder95 added never-stale Issues/PRs that are not stale and removed stale labels May 29, 2024
@metcoder95 metcoder95 removed the never-stale Issues/PRs that are not stale label Oct 11, 2024
Copy link

This issue has been marked as stale because it has been opened 30 days without activity. Remove stale label or comment or this will be closed in 5 days.

@github-actions github-actions bot added the stale label Nov 10, 2024
@vmarchaud
Copy link
Author

@metcoder95 Sorry for some reason i never say your comment

Just out of curiosity, what would be the shape of the API?

i've suggested this.workers.runTime.reset()

Just to reset the histograms at a given point in time?

Yes i don't think it requires something more, it match the underlying implementation used (https://nodejs.org/api/perf_hooks.html#histogramreset)

Along with that, do you have a distributed setup? How do you measure the distribution among hosts?

Yes, prometheus is actually pretty good at re-constructing histogram across different instances

@github-actions github-actions bot removed the stale label Nov 17, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants