Thursday 26 August 2021

Load Testing JD Edwards orchestrations

I've been tasked to help a very large client work out whether they can use JD Edwards orchestration to facilitate transactions from their ecommerce solution.  This is a very common question that I'm being asked more and more.  Customers want to connect their ecommerce solution to JDE for stock availability and also for pricing. 

The problem is that pricing is complex for B2B and a little less complex for B2C - as there is generally only a single price.  When you need to price on volume and customer - then you need to tap into the JD Edwards advanced pricing rules.

How are you going to do this effectively?

My choice is calling the JD Edwards BSFN's for pricing via orchestration.  This is a super easy process.  There are some nasty product out there that masquerade as middleware and expose BSFN's - but you can do this through AIS simply and free.  The other advantages are that it'll upgrade with your JD Edwards application stack.  I would go native AIS / orchestration every time.

I could do a blog (and might) on how easy it is to create an orchestration that allows you to call a BSFN.  It takes minutes.

My highly complex orchestration in 9.2.5.3

?xml version='1.0' encoding='UTF-8'?>
<ServiceRequest>
  <omwObjectName>SRE_2108230001F5</omwObjectName>
  <studioVersion>9.2.5.3</studioVersion>
  <name>request_CallB4201500</name>
  <shortDesc>call B4201500</shortDesc>
  <productCode>55</productCode>
  <locale>en</locale>
  <updateTime>1629696722845</updateTime>
  <description></description>
  <group>0</group>
  <appStack>false</appStack>
  <returnFromAllForms>false</returnFromAllForms>
  <bypassER>false</bypassER>
  <serviceRequestSteps>
    <serviceRequestSteps type="customServiceRequest">
      <scriptLanguage>groovy</scriptLanguage>
      <objectName>B4201500</objectName>
      <functionName>CalculateSalesPricesAndCosts</functionName>
      <dataStructureName>D4201500</dataStructureName>
      <bsfnInputs>
        <bsfnInputs>
          <name>szAdjustmentSchedule</name>
          <input>szAdjustmentSchedule</input>
          <defaultValue></defaultValue>
          <controlID>9</controlID>
          <type>String</type>
        </bsfnInputs>
        <bsfnInputs>
          <name>mnAddressNo</name>
          <input>mnAddressNo</input>
          <defaultValue></defaultValue>
          <controlID>10</controlID>
          <type>Numeric</type>
        </bsfnInputs>
        <bsfnInputs>
          <name>mnShipToNo</name>
          <input>mnShipToNo</input>
          <defaultValue></defaultValue>
          <controlID>11</controlID>
          <type>Numeric</type>
        </bsfnInputs>
        <bsfnInputs>
          <name>mnShortItemNo</name>
          <input>mnShortItemNo</input>
          <defaultValue></defaultValue>
          <controlID>12</controlID>
          <type>Numeric</type>
        </bsfnInputs>
        <bsfnInputs>
          <name>szBaseCurrencyCode</name>
          <input></input>
          <defaultValue>USD</defaultValue>
          <controlID>13</controlID>
          <type>String</type>
        </bsfnInputs>
        <bsfnInputs>
          <name>szCustomerCurrencyCode</name>
          <input></input>
          <defaultValue>USD</defaultValue>
          <controlID>14</controlID>
          <type>String</type>
        </bsfnInputs>
        <bsfnInputs>
          <name>cCurrencyConversionMethod</name>
          <input></input>
          <defaultValue>Z</defaultValue>
          <controlID>18</controlID>
          <type>String</type>
        </bsfnInputs>
        <bsfnInputs>
          <name>szBranchPlantDtl</name>
          <input>szBranchPlantDtl</input>
          <defaultValue>M30</defaultValue>
          <controlID>34</controlID>
          <type>String</type>
        </bsfnInputs>
        <bsfnInputs>
          <name>szCompany</name>
          <input>szCompany</input>
          <defaultValue>00411</defaultValue>
          <controlID>43</controlID>
          <type>String</type>
        </bsfnInputs>
        <bsfnInputs>
          <name>mnQtyShipped</name>
          <input>mnQtyShipped</input>
          <defaultValue></defaultValue>
          <controlID>51</controlID>
          <type>Numeric</type>
        </bsfnInputs>
        <bsfnInputs>
          <name>mnQtyOrdered</name>
          <input></input>
          <defaultValue>1</defaultValue>
          <controlID>54</controlID>
          <type>Numeric</type>
        </bsfnInputs>
        <bsfnInputs>
          <name>mnConvFactorTransToPrim</name>
          <input></input>
          <defaultValue>1</defaultValue>
          <controlID>55</controlID>
          <type>Numeric</type>
        </bsfnInputs>
        <bsfnInputs>
          <name>mnConvFactorPricingToPrim</name>
          <input></input>
          <defaultValue>1</defaultValue>
          <controlID>56</controlID>
          <type>Numeric</type>
        </bsfnInputs>
        <bsfnInputs>
          <name>szTransactionUom</name>
          <input></input>
          <defaultValue>EA</defaultValue>
          <controlID>66</controlID>
          <type>String</type>
        </bsfnInputs>
        <bsfnInputs>
          <name>szPricingUom</name>
          <input></input>
          <defaultValue>EA</defaultValue>
          <controlID>67</controlID>
          <type>String</type>
        </bsfnInputs>
        <bsfnInputs>
          <name>jdPriceEffectiveDate</name>
          <input>jdPriceEffectiveDate</input>
          <defaultValue>20210823</defaultValue>
          <controlID>68</controlID>
          <type>Date - yyyyMMdd</type>
        </bsfnInputs>
        <bsfnInputs>
          <name>szCustomerPricingGroup</name>
          <input>szCustomerPricingGroup</input>
          <defaultValue></defaultValue>
          <controlID>73</controlID>
          <type>String</type>
        </bsfnInputs>
        <bsfnInputs>
          <name>szLineType</name>
          <input>szLineType</input>
          <defaultValue></defaultValue>
          <controlID>87</controlID>
          <type>String</type>
        </bsfnInputs>
      </bsfnInputs>
      <bsfnReturnControls>
        <bsfnReturnControls>
          <controlID>20</controlID>
          <variable>mnUnitPrice</variable>
          <title>mnUnitPrice</title>
          <type>Numeric</type>
        </bsfnReturnControls>
        <bsfnReturnControls>
          <controlID>21</controlID>
          <variable>mnExtendedPrice</variable>
          <title>mnExtendedPrice</title>
          <type>Numeric</type>
        </bsfnReturnControls>
        <bsfnReturnControls>
          <controlID>24</controlID>
          <variable>mnListPrice</variable>
          <title>mnListPrice</title>
          <type>Numeric</type>
        </bsfnReturnControls>
        <bsfnReturnControls>
          <controlID>26</controlID>
          <variable>szListPriceUOM</variable>
          <title>szListPriceUOM</title>
          <type>String</type>
        </bsfnReturnControls>
        <bsfnReturnControls>
          <controlID>44</controlID>
          <variable>jdTransactionDate</variable>
          <title>jdTransactionDate</title>
          <type>Date - yyyy-MM-dd</type>
        </bsfnReturnControls>
        <bsfnReturnControls>
          <controlID>66</controlID>
          <variable>szTransactionUom</variable>
          <title>szTransactionUom</title>
          <type>String</type>
        </bsfnReturnControls>
        <bsfnReturnControls>
          <controlID>67</controlID>
          <variable>szPricingUom</variable>
          <title>szPricingUom</title>
          <type>String</type>
        </bsfnReturnControls>
        <bsfnReturnControls>
          <controlID>68</controlID>
          <variable>jdPriceEffectiveDate</variable>
          <title>jdPriceEffectiveDate</title>
          <type>Date - yyyy-MM-dd</type>
        </bsfnReturnControls>
      </bsfnReturnControls>
      <isAsynch>false</isAsynch>
    </serviceRequestSteps>
  </serviceRequestSteps>
</ServiceRequest>



Above is the code to the SR, as you might want to know the parameters that you need to fill out to get this working

I therefore have the ability to run this with the following active parameters:

{

  "szAdjustmentSchedule": "string",

  "mnAddressNo": "string",

  "mnShipToNo": "string",

  "mnShortItemNo": "string",

  "szBranchPlantDtl": "string",

  "szCompany": "string",

  "mnQtyShipped": "string",

  "jdPriceEffectiveDate": "string",

  "szCustomerPricingGroup": "string",

  "szLineType": "string"

}

Awesome and easy so far.

I can use curl to run this and put a bunch of &'s at the end of the queries to try and fudge some performance statistics, but that is not really going to help anyone.  I want to do a proper job of testing something that is massively parallel - as we also know that advanced pricing - even in a simple form - is going to take too long as a linear transaction.

Here is a complete script that will allow you to call the orchestration that I have created above.

There are a number of nuances in this that you need to get right and took me quite a while (lucky you).  The use of compressed as native compression at my tools level.  The use of insecure because I was using the JMeter proxy and did not load the certificate properly.  Using curlFormat for some nice timing information for your load testing.  And the use of a here file to load the JSON into a variable.

Note that the first command is not proxied and the second command is proxied.

data=$(cat <<EOF
{
  "mnAddressNumber": "4242",
  "jdDateEffective": "1/1/2021",
  "mnQtyOrdered": "12",
  "szUnitOfMeasure": "EA",
  "szCostCenter": "M30",
  "szItemNo": "220"
}
EOF
)
echo $data
#there is some native compression there, need to turn that off / account for it
#-w "@curl-format.txt" -o /dev/null
set -x
#curl -v --output - --compressed --request POST \
  #--url https://f5dv.fusion5.cloud/jderest/orchestrator/orch_getPrice2 \
  #--header 'Accept: application/json' \
  #--header 'Authorization: Basic Something==' \
  #--header 'Cache-Control: no-cache' \
  #--header 'Connection: keep-alive' \
  #--header 'Content-Type: application/json' \
  #--header 'Host: f5dv.fusion5.cloud' \
  #--header 'accept-encoding: gzip, deflate' \
  #--header 'cache-control: no-cache' \
  #--data "${data}"
countMax=5
i=0
while [ $i -lt ${countMax} ]
do
curl --proxy localhost:8888 --insecure -o /dev/null -w "@curlFormat.txt" --compressed --request POST \
  --url https://f5dv.fusion5.cloud/jderest/orchestrator/orch_getPrice2 \
  --header 'Accept: application/json' \
  --header 'Authorization: Basic Something==' \
  --header 'Cache-Control: no-cache' \
  --header 'Connection: keep-alive' \
  --header 'Content-Type: application/json' \
  --header 'Host: f5dv.fusion5.cloud' \
  --header 'accept-encoding: gzip, deflate' \
  --header 'cache-control: no-cache' \
  --data "${data}" &
  i=$(($i+1))
done

Cool - so I can now do rough testing.

I also created some additional scripts that ripped out the auth token and saved more time there.

Even if I can get 20 items on a page, and my amazing JDE can return a price in .4 of a second...  I need to wait 20x.4 or 8 seconds for the thing to complete - it is not going to fly.  We need to move on from a linear equation.

JMeter to the rescue

Unfortunately (or fortunately) OATS is nearing end of life.  unfortunately as I understand the platform and limitation well and have executed MANY load testing scenarios very successfully.  Fortunately we were given enough notice and I was able to not pay the maintenance bill this year for the companies subscription and find an alternative solution.

JMeter is totally awesome and nerdy - I'm loving it.  It does everything that I need and I can load test JD Edwards [with a number of tweaks] and also importantly orchestrations.

There is a lot of steep learning for JMeter, but it's not my first rodeo.


Above are the results for a 20 thread linear load test, finishing in a eye watering 12 seconds.  I know my internet can be slow, but a customer waiting 12 seconds to render a 20 item page might be too much.

Let's try another step.  How about caching and handle reuse (token specifically)

WOW = 2.8 seconds for 20 calls.  Look at the session initialisation overhead for what we are doing - that is crazy.  It's good to know how long the actual BSFN is taking, that is .12 of a second.  So we are dealing with .28 of overhead.  Fair enough I say.

Let's start to look into parallel processing:

What we are doing here is breaking down the AIS calling into 20 separate HTML calls or threads.  If there is one thing that the internet is pretty good at - that is threading.




We have 2.05 seconds for the 20 threads to complete.  You can see the distinctions in the start times for the above two scenarios.  Immediately above we can see all 20 threads leave JMeter at the same time [but interestingly seem to come back one at a time [seems like we are single threaded somewhere]?].  Although we have all fire off at the same time, their responses are only retrieved at about .12 seconds [magic number] after the previous one... Hmmm - might need to look closer at the parallel processing in WLS and also my server.

But, the theory is good (if not great) when I have lots of servers.  So imagine that I had 20 AIS servers, the I'm only going to wait as long as the longest request.  I'm going to use JSESSIONID in my load balancer and not need stateless load balancing in 9.2.5.3 and I'm going to maintain open connections to the AIS servers.  So, when my ecommerce solution requests a lot of pricing or a lot of availability - I'm going to reply in spades!

I'll write more about this load testing exercise and the use of JMeter for your load testing needs.  I don't see a lot of point in having additional tools. 




















No comments: