Tools and resources regarding Node.js and Performance Tools Integration

[thlorenz]

goal

I'm trying to gather resources and tools that make it easier to profile Node.js apps ideally by integrating with Chrome Dev tools and chrome://tracing.

what I found so far

Documentation of tools to analyse the data has been started here but the integration part with Node.js is still missing.

I tried to gather some free and open source tools that try to make this easier and would love to get some feedback and/or further tool suggestions. Pointers to any resources that document anything related are also greatly appreciated.

tools

Name	In app initialization	Instrumentation	Description
memwatch	Yes	No	Leak detection and heap usage and diffing.
heapdump	Yes	No	Make a dump of the V8 heap for later inspection
profiler	Yes	Resume/Pause	Access the V8 profiler from node.js DEPRECATED
v8-profiler	Yes	Start/Stop	Provides node bindings for the v8 profiler and integration with node-inspector
node-gc	Yes	Handle Events	Emits events from the V8 garbage collector to let you know when garbage collection occurs
gc-stats	Yes	Handle Events	Emits GC stats events after GC occurred
webkit-devtools-agent	Yes	No	Implementation of Chrome developer tools protocol. Only the heap and CPU profilers are working right now.
trace-viewer	Yes	Yes	JavaScript frontend for Chrome about:tracing and Android systrace. It provides rich analysis and visualization capabilities for trace files, supporting both the linux kernel trace format and Chrome's base/trace_event
tracing-framework	No	Yes	A set of libraries, tools, and visualizers for the tracing and investigation of complex web applications. It helps discover performance problems, track regressions, and build buttery-smooth 60fps web apps.
irhydra	No	No	Tool that can display intermediate representations used by V8 and Dart VM optimizing compilers
cpu-profiler	Yes	Start/Stop	Node bindings for the v8 cpu profiler
chrome-cpu-profiler	Yes	Start/Stop	Allows viewing top down, bottom up and callgraphs of a node app in Google Chrome
perf-cpuprofile	No	No	Converts linux perf tool output to .cpuprofile files readable by chromiums devtools

webkit-devtools-agent looks very promising, but doesn't show flamegraphs and has no time line support.

techniques

With a version of node that has a --log-timer-events as part of v8-options, run the app in profiling mode and load the resulting v8.log into Chrome at chrome://tracing.

Check for supported flag:

➝  node --v8-options | grep log_timer_events
  --log_timer_events (Time events including external callbacks.)

Profile app to generate v8.log including GC events.

node --prof --track_gc_object_stats --trace_gc_verbose --log_timer_events app.js

Loading into chrome://tracing will yield a flame chart similar to the below:

A detailed explanation is outlined in creating v8 profiling timeline plots although some of the information seems a bit outdated.

what I'm interested in

The idea is to find a way to use the built in Chrome tools to profile Node.js apps the same way as is currently possible
for JavaScript apps running in the browser.

Interesting metrics to gather:

• CPU profile including flamecharts - what's described here but for Node.js apps
• Heap profile, snapshots and diffs - what's described here but for Node.js apps
• GC events and timeline

Additionally integration with chrome://tracing by passing flags like --log_timer_events are of interest.

Both, CLI and GUI support would be nice to have as well as real-time and historical data.

[c4milo]

FWIW, Strongloop folks are revamping v8-profiler, I'm not sure if they are releasing their UI tools open source, though. From where I left off in node-webkit-agent, debugger agent is pretty close, the rest of the agents are fairly straight forward. Although, I'm not entirely sure about flamegraphs. Unfortunately, I switched gears and I'm not actively working on the project. However, I'm willing to re-take efforts and be incentivized. ;D

[thlorenz]

@c4milo thanks, will have a look into it. From ease of use perspective it's the best I found so far. Too bad timeline and metrics like GC events are not there.

[thlorenz]

Pulling @bnoordhuis in here since he authored some tools I found, hopefully he's got some useful advice.

[c4milo]

@thlorenz BTW, ongoing work with the debugger is in the dev branch, it might need to be also re-sync'd with master.

[bnoordhuis]

Our (StrongLoop's) tools aren't libre but we'll be releasing an on-premise version in the near future that is gratis for non-commercial use and covers most of the things mentioned.

By the way, memwatch is effectively unmaintained and has at least one serious bug, see lloyd/node-memwatch#50.

[thlorenz]

Thanks @bnoordhuis I saw the link to a strong-agent tool on one of your repos, but saw that it requires a STRONGLOOP_KEY and doesn't seem to be available on github.

I'm trying to gather completely open tools and techniques here that everyone can use without signing up for anything.

Also thanks for the warning related to memwatch. I'll add that warning to the resources.

[deepak1556]

just did an upgrade to memwatch with nan https://github.com/deepak1556/node-memwatch , @thlorenz thanks for this repo, totally helpful 👍

[thlorenz]

@deepak1556 ah, thanks for the info I'll have a detailed read of whatcha doing in there ;)

[sidorares]

@thlorenz https://github.com/sidorares/node-tick/ ?

[thlorenz]

@sidorares I think I saw this before, but I am not sure how this helps me integrate with chrome tools.
I can just open v8.log directly in chrome://tracing.

What does node-tick give me extra? From the readme I cannot see what it is parsed into and how I can use the resulting output.

[sidorares]

I was actually unable to use v8.log with chrome://tracing even with additional flags. I'd like to add html output to node-tick with ui similar to chrome://tracing

[thlorenz]

which version of node are you using (you need to turn on log_timer_events flag)?
The screenshot above was taken from chrome://tracing.

[sidorares]

I used 0.10.29 and didn't notice Error: unrecognized flag --log_timer_events
With 0.11.13 I can see v8.log rendered correctly in chrome, thanks!

[thlorenz]

Happy things work for you now.

I had started an issue to collect chrome://tracing related info. Feel free to add your findings there.

[brendangregg]

It's worth noting that what Chrome have implemented is what they call a "flame chart", which is not the same as a "flame graph". A flame chart shows the passage of time on the x-axis. A flame graph sorts alphabetically on the x-axis, for maximum merging. I wish Chrome would add an option so that you could view either, since they have the data anyway.

Flame graphs are explained here: http://www.brendangregg.com/FlameGraphs/cpuflamegraphs.html

I think flame charts can solve some time-based problems that flame graphs cannot, but the first profile I want to see is the flame graph, as it can better fit the profile data on one screen.

[thlorenz]

@brendangregg totally agreed and corrected.
Once I create documentation from this I'll make sure to indicate that the flame-chart is a time line of functions calls vs. time spent in function aggregation.

I feel like a more fitting name would be call graph, but flame chart it is.

[arunoda]

@brendangregg yep. It's super hard to see things with a timescale.

[yamsellem]

@thlorenz I'm trying to use --log-timer-events as explained in your first post with node 0.12.
The flag is supported but, launching the command seems to block anything from execution (even a simple console.log('Hello World')).

This command will block anything for execution:
node --prof --track_gc_object_stats --trace_gc_verbose --log_timer_events app.js

Any idea on the matter?

[cblair]

I'm very interested in this topic, I need some metrics from our node.js/mocha test builds like the flame charts from chrome://tracing (actually I'd like the flame graphs like @brendangregg mentioned), but I need them pretty passively so I can convince my team to review them during PRs. Working through this list starting with trace-viewer, will post what I find, but interested if anyone's already done this.