Wednesday 3 April 2019

never underestimate the power of stderr

JDE debugging is lots of fun, especially on Unix based environments.  No irony, I seriously like it.

What especially rings my bell is the fact that the OS is nice and basic and you can find things easily.  I want to talk about 2 specific items that CNC people often miss when dealing with logging on Linux / Unix.

Firstly, the root dir for anything dodgey is $SYSTEM/bin64 ($SYSTEM is an environment variable that is defined for the user that runs the JDE services).  So if you are looking for output files that are not in the correct place or other strange things (see below)

-rw-rw-r--.  1 jde920 jde920   29639 Mar 18 15:25 \JDEdwards\E910\output\WorkOrders\CPG\WO_20500305_20190318_152535.csv
-rw-rw-r--.  1 jde920 jde920   29639 Mar 18 15:25 \JDEdwards\E910\output\WorkOrders\CPG\WO_20500305_20190318_152539.csv
-rw-rw-r--.  1 jde920 jde920   29639 Mar 18 15:25 \JDEdwards\E910\output\WorkOrders\CPG\WO_20500305_20190318_152543.csv
-rw-rw-r--.  1 jde920 jde920   29639 Mar 18 15:25 \JDEdwards\E910\output\WorkOrders\CPG\WO_20500305_20190318_152546.csv
-rw-rw-r--.  1 jde920 jde920   29639 Mar 18 15:26 \JDEdwards\E910\output\WorkOrders\CPG\WO_20500305_20190318_152608.csv
-rw-rw-r--.  1 jde920 jde920   29639 Mar 18 15:26 \JDEdwards\E910\output\WorkOrders\CPG\WO_20500305_20190318_152612.csv
-rw-rw-r--.  1 jde920 jde920   29639 Mar 18 15:26 \JDEdwards\E910\output\WorkOrders\CPG\WO_20500305_20190318_152616.csv
-rw-rw-r--.  1 jde920 jde920   29639 Mar 18 15:26 \JDEdwards\E910\output\WorkOrders\CPG\WO_20500305_20190318_152630.csv
-rw-rw-r--.  1 jde920 jde920   29639 Mar 18 15:26 \JDEdwards\E910\output\WorkOrders\CPG\WO_20500305_20190318_152636.csv
-rw-rw-r--.  1 jde920 jde920   29639 Mar 18 15:26 \JDEdwards\E910\output\WorkOrders\CPG\WO_20500305_20190318_152637.csv


Okay, so we have found the missing export files, they are using the windows \ not / and therefore are created under the root directory.

Secondly, $SYSTEM/bin64/jdenet_n.log is a valuable place for stderr.  What is stderr you ask?  Well, I refer to google for a good answer.  But in simple terms it's all of the OS errors that you generally do not see in stdout.

For example, I was recently chasing a little problem around about an OSA.  And specifically whether JDE could load a 32bit OSA using a 64 bit version of  JDE...   Any guesses?

LoadLibrary - dlopen: No such file or directory
/jdedwards/e920/system/lib/libFSosa.so: wrong ELF class: ELFCLASS32: No such file or directory
libFSosa.so: No such file or directory
LoadLibrary - dlopen: No such file or directory
/jdedwards/e920/system/lib/libFSosa.so: wrong ELF class: ELFCLASS32: No such file or directory
libFSosa.so: No such file or directory
LoadLibrary - dlopen: No such file or directory
/jdedwards/e920/system/lib/libFSosa.so: wrong ELF class: ELFCLASS32: No such file or directory
libFSosa.so: No such file or directory
LoadLibrary - dlopen: No such file or directory
/jdedwards/e920/system/lib/libFSosa.so: wrong ELF class: ELFCLASS32: No such file or directory
libFSosa.so: No such file or directory
LoadLibrary - dlopen: No such file or directory
/jdedwards/e920/system/lib/libFSosa.so: wrong ELF class: ELFCLASS32: No such file or directory
libFSosa.so: No such file or directory
LoadLibrary - dlopen: No such file or directory
/jdedwards/e920/system/lib/libFSosa.so: wrong ELF class: ELFCLASS32: No such file or directory
libFSosa.so: No such file or directory
LoadLibrary - dlopen: No such file or directory
/jdedwards/e920/system/lib/libFSosa.so: wrong ELF class: ELFCLASS32: No such file or directory
libFSosa.so: No such file or directory

Well, that would tell you the answer - NO!

The interesting thing that you need to remember is that this is NOT in the jde.log for the UBE nor was it in the debug logs, all we got was:

Apr  3 12:28:27.740781  winansi.c1737    -      LoadLibrary("/jdedwards/e920/system/bin64/libFSosa.so")
Apr  3 12:28:27.740793  winansi.c1737    -      LoadLibrary("/usr/java/jdk1.8.0_192-amd64/jre/lib/amd64/server/libFSosa.so")
Apr  3 12:28:27.740799  winansi.c1737    -      LoadLibrary("/usr/java/jdk1.8.0_192-amd64/jre/lib/amd64/libFSosa.so")
Apr  3 12:28:27.740803  winansi.c1737    -      LoadLibrary("/jdedwards/e920/system/lib/libFSosa.so")
Apr  3 12:28:27.740826  winansi.c1737    -      LoadLibrary("/jdedwards/e920/system/libv64/libFSosa.so")
Apr  3 12:28:27.740830  winansi.c1737    -      LoadLibrary("/oraclient/product/12.2.0/client_64/lib/libFSosa.so")
Apr  3 12:28:27.740836  winansi.c1737    -      LoadLibrary("/oraclient/product/12.2.0/client_64/lib/libFSosa.so")
Apr  3 12:28:27.740840  winansi.c1737    -      LoadLibrary("/usr/java/jdk1.8.0_192-amd64/jre/lib/amd64/server/libFSosa.so")
Apr  3 12:28:27.740845  winansi.c1737    -      LoadLibrary("/usr/java/jdk1.8.0_192-amd64/jre/lib/amd64/libFSosa.so")
Apr  3 12:28:27.740849  winansi.c1737    -      LoadLibrary("/jdedwards/e920/system/lib/libFSosa.so")

As you can see the debug logs were not enough.

The wonderful command that spits this source of truth is found in RunOneWorld.sh
...
print "     Starting jdenet_n..." >> $LOGFILE
cd $SYSTEM/$BIN_FOLDER
$SYSTEM/$BIN_FOLDER/jdenet_n > $SYSTEM/$BIN_FOLDER/jdenet_n.log 2>&1 &

Awesome, see the 2>&1  that is the important line.  This is saying that any OS errors, please send them to the $SYSTEM/$BIN_FOLDER/jdenet_n.log file.

Awesome.  Because runube is essentially started from the environment of a kernel, then it inherits the same settings, despite being a separate PID.

See more here https://www.computerhope.com/jargon/s/stderr.htm

Stderr, also known as standard error, is the default file descriptor where a process can write error messages.
In Unix-like operating systems, such as LinuxmacOS X, and BSDstderr is defined by the POSIX standard. Its default file descriptor number is 2.
In the terminal, standard error defaults to the user's screen.
So, as someone funnier than me just said, we don't have a big problem - we have a bit problem!  We need a new OSA for this bad boy to work.



No comments: