This week I needed to dive deeper into some performance issues I have been experiencing with dante. The first section will detail the struggle that lead me to this point. If you want the quick-n-dirty DIY instructions for profiling Docker, feel free to skip ahead.
At NodeSource, I'm building hundreds (literally hundreds) of Docker Images. This has lead to the pressing need of parallelizing these. Currently, I'm provisioning a 32-core box and triggering 100 simultaneous builds to burn through my 600+ images. I've noticed though that, at any one time, there is never more than ~25 layers actively running. Somewhere, there is a bottleneck. Running
ps shows me that Dante is indeed spawning 100
docker build commands, so it would seem the docker daemon isn't keeping pace.
Diving into the Daemon
So golang has a built-in profiler called [pprof