unix log file analysis

If you want to get gaps in your log file that are > 14000 (for example) miliseconds than the previous line, then this awk script is for you.  This does some gnarly calcs over the JD Edwards time stamps and tells you which two lines have a gap greater than X.  I know that you can do this with “performance workbench”, but that’s not as much fun.

cat R42565* | awk -F. '{if(substr($2,0,6)-prev>14000&&prev>0&&substr($2,0,6)*1==substr($2,0,6))printf("GAP(%d)\nPREV%s\nCURR%s\n",substr($2,0,6)-prev,prevline,$0);prev=substr($2,0,6);prevline=$0}' > Gaps.txt

GAP(15992)
PREVMay 27 09:21:58.293008 - 2388/4500 MAIN_THREAD                              Try to send the message to security server AUULTDWNWAD013
CURRMay 27 09:21:58.309000 - 2388/4500 MAIN_THREAD                              Try to free sec list
GAP(31911)
PREVMay 27 09:21:58.324089 - 2388/4500 WRK:Starting jdeCallObject               METADATA - Using Environment [JPY910] for user [00160531] in sendCnvMsgToMetadataServer()
CURRMay 27 09:21:58.356000 - 2388/4500 WRK:Starting jdeCallObject               Spec Data returned from Metadata server. size = 508.
GAP(15997)
PREVMay 27 09:21:58.371003 - 2388/4500 WRK:Starting jdeCallObject               METADATA - Using Environment [JPY910] for user [00160531] in sendCnvMsgToMetadataServer()
CURRMay 27 09:21:58.387000 - 2388/4500 WRK:Starting jdeCallObject               Spec Data returned from Metadata server. size = 732.

So the output is something like the above.  My threshold was 14000 ms.  Note that this is a common issue with my clients having a consistent wait of over 14000ms on traffic.  It’s pure network too – on windows 2008R2 – as some of the traffic is DB and some is local (to it’s own metadata kernel!)

If anyone has a fix for this 15000ms wait, I’d love to know!