Sometimes when you’re investigating server performance issues, you don’t have the luxury of a full development suite installed with tools like a profiler.
It turns out that the linux ts command, included in moreutils, can be surprisingly helpful. It can timestamp stdout in microseconds, clearly showing timing gaps:
time ./prog_excerpt.cgi sp_token=1 | ts "%.S"
In the above situation, installing the Perl NYT Profiler failed, but ts and a few print statements gave me the info I needed (a database query was taking almost one-third second when it was expected to take under 100 ms.)