Outdated dependency librato-metrics

[jibberia]

librato-rack depends on librato-metrics ~> 1.6. The current version is 2.0.1. In my project, we require librato-metrics and librato-rails (whether this is correct is up for debate, but it's what we have now) and we're unable to use current versions because of this issue.

Backstory: we updated Ruby versions from 2.0.0 to 2.3.1 (keeping librato-metrics (1.5.1), librato-rails (0.11.1), librato-rack (0.4.5)) and saw significantly increased response times to metrics-api.librato.com:

We rolled back and are attempting to update librato gems. I see that librato-metrics fixed a bug related to Ruby 2.3.1. It doesn't seem likely to fix our problem but it's a good first step.

[niklibrato]

Hi @jibberia - thanks for reporting this. I'll share this with the engineering team.

[chancefeick]

Thanks for the report, @jibberia.

Wanted to sum up the issue: saw an increase in response time when running older versions (librato-metrics 1.5.1, librato-rack 0.4.5, librato-rails 0.11.1) on ruby 2.3.1, but did not when upgrading the clients to the latest versions?

Unable to reproduce on ruby 2.3.1 using librato-rack ~> 1.0, which has a dependency on librato-metrics ~> 1.6.1 as you mentioned.

As for the screenshot, we did see a temporary increase in POST response time overlapping with multiple deploys, but nothing to indicate an ongoing issue. Can you provide another screenshot using a wider time range (7 days or greater) to determine if it was consistent when reproducing?

[jibberia]

Hi @chancefeick, thanks for getting back to me. Sorry if I wasn't clear. Here's what happened:

1. Updated Ruby to from 2.0.0-p648 to 2.3.1. Some dependencies were updated, this had nothing to do with Librato (still using librato-metrics 1.5.1, librato-rack 0.4.5, librato-rails 0.11.1).
2. Tested successfully enough in dev and staging.
3. Deployed to production and noticed some changes in performance. Our web server starts thrashing after ~15 minutes and restarting worker processes. While this is happening, we also notice elevated response time from Librato. Other external services' performance remains the same.
4. Figured we'd update Librato to potentially eliminate it, unsure of whether it's a symptom of a cause. Noticed the gem dependency problem and filed this issue.

Since then, I forked it (https://github.com/jibberia/librato-rack), updated the dependency, tested in dev, staging, and production, and decided that Librato works fine for our purposes with the updated dependency -- but it doesn't fix our response time issue. I'm still unsure of whether it's a symptom or cause, but I'm slightly leaning towards symptom. This has been a very tricky issue to debug.

The chart below shows response time to metrics-api.librato.com. The peaks happen when we deploy various versions of Librato gems (as above) on Ruby 2.3.1.

If you want to keep this GitHub issue on topic, I'm happy to resume the response time discussion over email -- [email protected]