Wednesday, 13 February 2019

IoT in action

JDE IoT is easy, let's look at a use case and see how you can implement this solution.

Firstly, if you don't know how IoT can help you – think of something that you measure manually and let's automate that.

So, if you need to go and measure the temperature in a control room, then why not use a sensor to do this.  Then you can have that measured all of the time.  What about water quality?  What about lidar to measure the height of a stack of coal?  No worries!

At Fusion5, we've actually been through a process of designing this complete solution.

We have been using particle io boards to send sensor data to JD Edwards, but there is some cool tech in the middle.


We are using the particle 3G boards, like the boron below


but also dabbling with the latest mesh modules that they have released.  These allow us to have less 3G, but more sensors.

This is allowing us to produce any device, using a myriad of sensors and being able to react to the data immediately.



The build at the moment is above, with a remote temp sensor and also a water proof and dust proof housing

Below is a set of devices for a PoC we've completed here in Australia.  We have a couple of production POC's working with our JD Edwards clients.



These devices are waking up and sending temperature data & humidity data every 15 minutes to our particle cloud.  We are using particle as the middleman, as the particle complete device solution (including data plan) is perfect.  Added value is that if the power is disconnected, this also triggers an event that you can use to raise a work order, let's be honest – if the power goes, this is not good.  The device has a battery which allows 2 weeks of disconnected data to be sent.

We are using 3G, as it does not create any holes in the client's networks.  3G also ensures that if the power goes out, we can still send the sensor data out to the cloud and react.  Note that mesh devices can be connected to the 3G board, which extends its range without additional 3G data plans.


Simply, the diagram look like the above, but we need more layers to be future proof, there also needs to be some explanation to the logic layer.  You can see that all of the items are standard and the JDE orchestrations are configuration not code.  The solution is implemented without increasing your technical debt.

The above shows a little more detail of where we are adding strategic value, how this is an enterprise solution -  not a tactical one.  We use JDE for what it is good for – raising work orders or storing central data – it's not good for storing all of the trend data.  We use cheap cloud storage (2.3 c per GB per month, or a massive 28 cents a year).  Even with readings every 15 seconds and saying 256 bytes of data per read [date, time, GPS, temp, humidity], we are only going to store 24x60x4 - 5760 readings per day or 1.4 MB a day or 511MB a year per device.  This means that you can store the data from 200 devices a year for about 28 dollars a year.  You don't want to add 100GB to your JDE database do you?

You can see there are a number of items on this diagram.  We are using public cloud (in many cases AWS lambda) in to run logic over the top of the data, we do this as an initial triage – as we don't want to call orchestrations every time we get a meter reading (read the numbers above).

The data is interpreted in this cloud logic layer, which then can determine whether a business rule has been breached and call and orchestration accordingly.  Once again, the cloud costs of lambda are tiny and this is a cheap solution to run at scale.

Note that the values (actual breach values - high and low temperatures) ARE stored in JDE.  So there is a single place to go to maintain the threshold data.  The lambda functions query this on a scheduled basis – keeping everything fresh and efficient.

You'll also see mention of campfire in the diagram, this is a Fusion5 integration solution for enabling on premise software to be connected to the cloud.  It's our super light middleware, that is no fuss and build on AWS highly available constructs.  Highly available and disaster recoverable integration solution in the cloud!

All of the measurement data is being stored in a bucket (whether this is azure, google cloud storage or s3) to enable training AI down the track.  Imagine pointing quicksight at your data, and extracting insights.

This is a complete solution that can scale to any number of devices.  It delivers TRUE real-time ERP functionality – raising work orders when things get too hot – but also allows for future prediction analysis by providing the training data for AI.  This AI can deliver true actionable insights back into JDE and perhaps raise work orders before something actually gets too hot (based upon all of the training data).

This is all decoupled from the ERP.  This is all getting clients cloud ready.  BUT, JDE is doing all of the things that it is great at!

We have a self service web portal that users can view any of the configured IoT devices:

This can easily be hosted as a cafe1 window and accessible from JDE


You can see from above, that all of the information is at your fingertips from JDE, perfect.


We can drill down to the history too


This is all enabled with a simple orchestration and cafe1 pages. 

You also don't need JDE, we have the capability to call "webhooks" that can really go to anywhere, but also use campfire to call bespoke and on premise functionality.

Fusion5 and the innovation team are plugging in IoT and also the ability to embrace AI into our ERP.  This implementation means clients are not making large capital investments, and you are able to prototype these quickly and easily.

Devices are costing $150 each, data subscriptions are around $5 a month and then cloud costs (logic and website hosting) are costing another $5 a month.  Sure, the consulting might cost something, but you can see that the ongoing investment is minimal!

If you want to plug IoT into your JDE in a strategic way, Fusion5 would love to help!

Wednesday, 6 February 2019

Fraud detection in JDE using AI and orchestration

Okay, I’ve posted about it a number of times, but we all just need to admit it – orchestration is cool for JD Edwards.  Yeah, AIS was good, but it’s just got better and better.  This really is a challenge for us (the community that thrives on JD Edwards) to extend the use of JD Edwards beyond the traditional boundaries.  I’d like to challenge people to look outside of the square when they are solving business problems.   I’m going to step you through and really simple example of big gains from small investments using AI, Cloud and orchestration.

I do tend to talk a little more technical than most, unashamedly to be honest.  What I’m promoting in this post is to get the nitty gritty of web-hooks and API’s.  Understand that this is all really easy technology, easy to find and easy to implement.  Let’s say that you wanted a super simple solution that ripped the text of ANY attachment in JDE – no matter if it was a picture, PDF or anything.  You could then create a simple table in JDE and store the text in some sort of text search field (this can be implemented in so many ways, generally triggers over tables  etc etc).  This table could index the text in EVERY attachment.  Therefore every scanned PO, WO, every PDF attachment, every special instruction, every hand written note that was a photo could be converted to the TEXT value and be made searchable.  WOW, that would be great. 

I can only imagine finding a vendors part number or special notes of a PO from a couple of years ago could really save your bacon at some stage.  Being able to search attachments to work orders for serial numbers would be amazing for so many clients.  What if I said that something like this could be cobbled together < 1 week?  Treating your ERP like a searchable DMS – wow!

What if you were to look at an amazing extension of this idea to prevent fraud (I have to thank a good contributor Matthew S for this extension).  What if I took a hash of the resulting text from the scan of every document and looked for duplicate invoices (or fraud).  What if I could search for duplicate anything (let’s not get right into locality sensitive hashing https://en.wikipedia.org/wiki/Locality-sensitive_hashing), but there are some really amazing things that you could do.  This would be a complete bolt on solution that uses orchestration and some “APIs”.  We will talk about said API’s later.

Fusion5 have gone a long way to make this a reality.  We’ve created public facing API’s that are capable of many different interactions with a media object, for example:

  • Custom AI algorithm to recognize particular objects that you have trained the algorithm with
  • Character recognition algorithm that can turn something like this:
Into this  (My mind boggles that I can do such deep analysis of a photo like this.)

700N $99 TCOME E $45 $66 EACH 57 20% 20 PARTICIPATING BEER, PRE-MIX AND CIDER OFF 1 LITRE Vad Cruiser Captain Morgan Spiced Gold Rum, Smirnoff Red Vodka OR Canadian Club 700mi Whole team over? Covered. Baron Samedi Spiced Rum OR Jack Daniel's Old No 7 700ml HAHN Bombay Sapphire Gin 1 Litre 2 FOR BUY 2 OR MORE CASES OR 10 $28 CASE CASES OR 10 PACKS $41 CASE SA $10 EACH Coron "Extra SEST SERVED OVER VI Dew MAVE UP TO CASE DE PORTO ΚXXX XX GOLD GOLD CERVER MA BOURBON HAHN Heineken WHISKEY MIXED WITH PREMIUM CIDE MERCURY 4 LITRES PERON STRO AZZU Aceand hotely CERVECERIA MODELO, S.A MEXICO, D.F. PREMIUM Cola KOPPARBERI 'BUILT TO LAST 22. SIZ ITU TELS ASTUNDA perDry EST RUM 18 PREMIUM Scotch 700m De Bortoll Premium 4 Litre Casks Excludes fortified Apple IRON JACK 6.3% LARD CIDEI ORIGINAL XXXX Gold Bottles or Cans 24x375ml 24 PACK 6.9% ALC/VOL & COLA 190 mL 45% ALCNVOL HISP AUSTRALIAN LAGER EST 1911 HODART, TAS Heineken Premium Lager Bottles 24x330ml OR Coopers Premium Lager Bottles 24x355ml SUPER Peroni Nastro Azzurro Bottles 24x330ml 10 CANS 24 PACK SUPER SAVER S SUPER SAVER 10 CANS CANS SUPER SAVER SUPER SAVER EACH EACH SA EACH EACH EACH S BREW OOPERS HRB WERY TOOHEYS VODKA COOPERS PURE CI SAD BLONDE Um Low Carb Logo LOW CARU DRY VELVE ARK AL WOODSTOCK CRUISER BOURBON ORCHARD AND COLA THIEVES RASPBERRY Special - FACE GAMUN BOKLESS CARBOHYDRATES Houghton Classic Shingleback Red Knot AUS Twelve New Zealand Pinot Noir CLEAN CRISP TASTE APPLE CIDER Hardys HRB 2750 46% ALCANO Mumm Cordon Rouge Champagne NV 24 6.0 PACK 30 CANS IN-STORE @ your local BWS PICK UP Shop online. Collect instore. WITH YOUR GROCERIES @ woolworths.com.au/bws SIMPLY SHOP, SCAN AND SAVE with Woolworths Rewards at BWS rewards 17" We support the responsible service of alcohol. Available in SA from Wednesday 11 April until Tuesday 17 April 2018 unless sold out prior Savings are based on offers apply to the quantity advertised only. Limit rights reserved. Specials may not be available in all stores including Alice Springs. "Standard local call charges y by store. Wine is 750ml unless otherwise stated. At this great price no further discounts apply Casks not available in Adelaide City or Rundle Mall. See www.woolworthsrewards.com.au for terms and conditions Selected cases may not be available in all stores. WC110418/SA Page 35

  • Standard object detection
  • Landmark detection

We’ve embodied this in a simple to use GUI, to allow for interactive discovery (before plugging it in).

So all of these options can be plugged into ANY analysis in JD Edwards and scheduled using orchestration.  Extracting more value from the plain old photos.

In action:

My use case is is simply to find duplicate invoices, okay – this is going to be easy.

  1. I create my custom table (F55DUPINV) and view combo in my “database of choice” that is going to hold the text retrieved out of any attachment.  Note that there is going to be smarts here, because the text will be long.
  2. I create an orchestration that takes a parameter of MO data structure name and calls my API (webhook) via a connection.  This retrieves the text and then inserts this into a JDE form (or via a database connector if too big)
  3. This orchestration will also hash the text to a unique value for quick uniqueness checks.
  4. I can then have another scheduled orchestration that looks for new duplicates (or fuzzy logic like) and sends an email to the fraud officer of the instance, with a link to the actual transactions.

4 steps which can extract the text out of every attachment in JDE, compare them and automate the delivery of exceptions to your business rules.

I have not changed a single line of standard code, I’ve not contributed to my technical debt.  I’ve got actionable results from the implementation of a couple of nice orchestrations, some smart database features and some scheduling of orchestrations with notifications.

Although similar to a previous post, this shows real transactions and data going into JD Edwards in a different way.  I guess the perspective is more business focused.

Fusion5 are in the API & orchestration space.  If you’d like to trial some of this technology at your site – please do not hesitate to reach out.  We can get demo’s like this running in no time and allow you to harness the power of orchestrations at your organisation to extract real business value.  We can provide you a key to use our API’s and orchestrations that are ready to go.


Monday, 4 February 2019

Orchestration enhancements - SFTP

We hear about enhancements in orchestration studio, and sometimes just …yawn… because we think there is not much in it.  I’ve been looking into 9.2.2.4 (I know, behind the times) and the functionality is great.
Below is a sample connector defined to a free ftp server (that did not work, keep reading)…  but this is available from the tools menu in orchestration studio. 
That’s pretty simple.  Shame I cannot use key’s for authentication, that might be coming in subsequent releases?
But, when you go to use this connection in a service request, there is some magic.  You have a bunch of options (not just put and get) for your file IO needs. 
That is really neat.  You can get files easy enough and use variables for filenames, as you can see from below, just use the ${variable} syntax and they can be passed into the service request.
This is really functional (and unexpected), that you can get the native output from a UBE (or CSV or OSA).  This is a native get.  Also get the BIP output

Note that this will grab the output of the UBE and FTP the file in one fell swoop .  Then schedule it using the AIS scheduler .  Imagine what might have been lots of modifications and scripts is now a single operation orchestration studio that can be completed by a simpleton like me.   Once again, an amazing use case of UDO’s and orchestration working hard to reduce clients technical debt.  We all just need to start the process of converting the old ways we do things.

I tried a couple of free FTP servers on the internet to get this working, with various success.  The rebex.net one above had a problem negotiating security algorithms.  You cannot pass options into the FTP command with the interface that has been provided, so you might need to ensure that they play nice.  Of course there are work arounds to add items to the java.security files for the JRE that is running AIS, but it might be easier to change the server (or not).

I ended up being able to test everything with the following connection details:
goto this link to find the password, they need some credit for providing this! https://www.wftpserver.com/onlinedemo.htm 

You can upload and download, which is cool.  I got both of them working immediately (thanks JDE)
I then had a crack at the GetUBEOutput, which I thought was a bit of a joke initially, but it’s really good.
As I stated before, 1 SR, 1 schedule and 1 orchestration could launch a job then send the output from the job to a remote SFTP server.  It’s that easy.  How many mods are we going to save with this alone?
The return JSON is nice and informative too:
{
  "Connectors" : [ {
    "reportName" : "R0004C",
    "reportVersion" : "XJDE0001",
    "jobNumber" : 24,
    "executionServer" : "F5ENT",
    "jobStatus" : "D",
    "objectType" : "UBE",
    "user" : "SM00001",
    "environment" : "JPLAY920",
    "submitDate" : "20181030",
    "lastDate" : "20181030",
    "submitTime" : "174358",
    "lastTime" : "174400",
    "oid" : "R0004C_XJDE0001",
    "queueName" : "QBATCH",
    "fileName" : "/upload/R0004C_XJDE0001_24_PDF.pdf"
  } ]
}
In summary, the SFTP worked out of the box first time.  Created a connection, created an SR – wrapped the SR in an orchestration and boom!  I was able to sftp files from anywhere to anywhere.  Making this parameter driven is easy and then submitting the same to cURL is also easy – so that you could EASILY call this orchestration from a BSFN (see B98ORCH) you probably need 9.2.3 for this.
jdeOrchestrationManager *orchMgr = (jdeOrchestrationManager *)jdeOrchestrationManagerInit(lpBhvrCom);
The magic exists in the function pointer above





Friday, 18 January 2019

Using AI and image recognition with JD Edwards

Introduction

This blog post is hopefully going to demonstrate how Fusion5 (quite specifically William in my team) have been able to exploit some really cool AI cloud constructs and link them in with JD Edwards.  We’ve been looking around for a while for use cases for proper AI and JD Edwards, and we think that we have something pretty cool.

I want to also point out that a lot of people are claiming AI, when what they are doing is not AI.  I think that true AI is able to do evaluations (calculations) based upon a set of parameters that it has not necessarily seen before.  It's not comparing the current images to 1000000's of other images, it has been trained on MANY other images and in it's internals it has the ability to apply that reference logic.  That model that has been built from all of it's training can be run offline, it's essentially autonomous - this is a critical element in the understanding of AI.


We are using JD Edwards orchestration to call out to an external web service that we’ve written (web hook).  This web hook has been programmed to call a number of different AI models to interpret images that have been attached to JD Edwards data.  So, if you use generic media object attachments – this mechanism can be used to interpret what is actually in those images.  This can greatly increase the ability for a JD Edwards customer to react to situations that need it.


For example if you used JD Edwards for health and safety incidents and you wanted some additional help in making sure that the images being attached did not contain certain critical objects – and perhaps if they do, you’d raise the severity or send a message based upon the results… You could also analyse frames of video with the same logic and detect certain objects.


We’ve decided to test the cloud and are using different models for our object detection.  we are using google, Microsoft and AWS to see if they are better or worse at object detection.

Object detection vs. Object Recognition

Note that there is a large difference between object detection and object recognition. – stolen from https://dsp.stackexchange.com/questions/12940/object-detection-versus-object-recognition
Object Recognition: which object is depicted in the image?

  • input: an image containing unknown object(s)
    Possibly, the position of the object can be marked in the input, or the input might be only a clear image of (not-occluded) object.
  • output: position(s) and label(s) (names) of the objects in the image
    The positions of objects are either acquired form the input, or determined based on the input image.
    When labelling objects, there is usually a set of categories/labels which the system "knows" and between which the system can differentiate (e.g. object is either dog, car, horse, cow or bird).
Object detection: where is this object in the image?
  • input: a clear image of an object, or some kind of model of an object (e.g. duck) and an image (possibly) containing the object of interest
  • output: position, or a bounding box of the input object if it exists in the image (e.g. the duck is in the upper left corner of the image
We’ve spent time training our algorithms to look for certain objects in images, so using object recognition.  We’ve trained 3 separate algorithms with the same 200 training images (we know that this is tiny, but the results are surprising!)


My PowerPoint skills are really being shown off in the above diagram of what has been done.

At this stage we’ve used a couple of the major public cloud providers and have created our own models that are specifically designed to detect objects that we are interested in, namely trains, graffiti and syringes.  This is quite topical in a public safety environment.
We’ve created an orchestration and a connector that are able to interrogate JDE and send the various attachments to the AI models and have some verification of what is actually in the images.  Note that this could easily be put into a schedule or a notification to ensure that this was being run for any new images that are uploaded to our system.

Testing

Let’s scroll deep into google images for train graffiti.  The reason I scroll deep is that these algorithms were trained on 70 pics of trains and 70 pics of graffiti and also 40 pics of syringes.  I want to ensure that I'm showing the algorithm something that it has never seen before.


And attach this to an address book entry in JD Edwards as a URL type attachment.



In this instance we are using the above parameters. 300 as the AN8 for ABGT and only want type 5’s.


William has written an orchestration which can run through the media objects (F00165) for ANY attachments.  We're currently processing image attachments, but really - this could be anything.

To call our custom orchestration, our input JSON looks like this:

{
   "inputs" : [ {
     "name" : "Object Name 1",
     "value" : "ABGT"
   }, {
     "name" : "Generic Text Key 1",
     "value" : "300"
   }, {
     "name" : "MO Type 1",
     "value" : "5"
   }, {
     "name" : "Provider",
     "value" : "customVision"
   }, {
     "name" : "ModelName",
     "value" : ""
   } ]
}
The provider in this instance is a model that we have trained in customVision in Azure.  We've trained this model with the same images as Google, and also our completely custom model.

Let’s run it and see what it thinks.  Remember that the orchestration is calling an Service Request which is essentially running a Lambda function though a connection.  There is some basic authentication in the header to ensure that the right people are calling it.  The Lambda function is capable of extending HOW it interprets the photo, how many different AI engines and models that it will return.
{
   "Data Requests" : [ {
     "Data Browser - F00165 [Media Objects storage]" : [ {
       "Media Object Sequence Number" : "1",
       "GT File Name" : "
https://c1.staticflickr.com/6/5206/5350489683_e7cdca43ba_b.jpg"
     } ]
   } ],
   "Connectors" : [ {
     "graffiti" : 0.9162581,
     "train" : 0.599198341,
     "syringe" : 0.00253078272
   } ]
}
Not too bad, it’s 91% sure that there is graffiti, 60% sure that there is a train and pretty sure that there are no syringes.  Not too bad, let’s try google now
A simple change to the provider parameter allows us to use google next.  Note I did have some issues with my screen shots, so that might reference some different pictures, but revealed these results.

{
   "inputs" : [ {
     "name" : "Object Name 1",
     "value" : "ABGT"
   }, {
     "name" : "Generic Text Key 1",
     "value" : "300"
   }, {
     "name" : "MO Type 1",
     "value" : "5"
   }, {
     "name" : "Provider",
     "value" : "autoML"
   }, {
     "name" : "ModelName",
     "value" : ""
   } ]
}
Results
{
   "Data Requests" : [ {
     "Data Browser - F00165 [Media Objects storage]" : [ {
       "Media Object Sequence Number" : "1",
       "GT File Name" : "
https://c1.staticflickr.com/6/5206/5350489683_e7cdca43ba_b.jpg"
     } ]
   } ],
   "Connectors" : [ {
     "graffiti" : 0.9999526739120483,
     "train" : 0.8213397860527039
   } ]
}
Similar, it thinks that there is 99.99% chance of graffiti and 82% chance of a train – more certain than Microsoft.


Finally, let’s try a hosted model that we are running on google cloud:
We drive that with the following parameters

{
   "inputs" : [ {
     "name" : "Object Name 1",
     "value" : "ABGT"
   }, {
     "name" : "Generic Text Key 1",
     "value" : "300"
   }, {
     "name" : "MO Type 1",
     "value" : "5"
   }, {
     "name" : "Provider",
     "value" : "customModel"
   }, {
     "name" : "ModelName",
     "value" : "multi_label_train_syringe_graffiti"
   } ]
}
And the output is:
{
   "Data Requests" : [ {
     "Data Browser - F00165 [Media Objects storage]" : [ {
       "Media Object Sequence Number" : "1",
       "GT File Name" : "
https://c1.staticflickr.com/6/5206/5350489683_e7cdca43ba_b.jpg"
     } ]
   } ],
   "Connectors" : [ {
     "graffiti" : 0.9984090924263,
     "syringe" : 7.345536141656339E-4,
     "train" : 0.9948076605796814
   } ]
}

So it’s very certain there is a train and graffiti, but very certain there is no syringe.

What does this mean?

We are able to do some advanced image recognition over native JD Edwards attachments using some pretty cool cloud constructs.  We’ve trained these models with limited data, and have some great results.  Although we should really try some images without trains or graffiti (trust me, this does also work).  We are paying a fraction of a cent for some massive compute to be able to load our models and process our specific AI needs.

You could be on premise or in the cloud and STILL use all of these techniques to understand your non structured data better.  This is all done with a single orchestration.


Fusion5 have the ability to create custom models for you and find actionable insights that you are interested in and ensure that this information is ALERTING users to ACT.

What does out AI algorithm think of me?

As you know, it's been trained (excuse the put) to look for trains, graffiti and syringes.  What if I pass in a picture of me?

Image result for "shannon moir"


Let’s process this for a joke: "https://fusion5.com.au/media/1303/shannonmoir-300.jpg"
{
   "Data Requests" : [ {
     "Data Browser - F00165 [Media Objects storage]" : [ {
       "Media Object Sequence Number" : "1",
       "GT File Name" : "
https://c1.staticflickr.com/6/5206/5350489683_e7cdca43ba_b.jpg"
     }, {
       "Media Object Sequence Number" : "2",
       "GT File Name" : "
https://fusion5.com.au/media/1303/shannonmoir-300.jpg"
     } ]
   } ],
   "Connectors" : [ {
     "graffiti" : 0.9162581,
     "train" : 0.599198341,
     "syringe" : 0.00253078272
   }, {
     "graffiti" : 0.100777447,
     "syringe" : 0.006329027,
     "train" : 0.00221525226

   } ]
}
Above is an example of processing a picture of “yours truly”, to see what Azure thinks…
10% chance graffiti and no syringe or train…  not bad…

Great, so if this was attached to an incident in JDE, you might want to still raise the priority of the case, but not becasue there is graffiti or a train!

What’s next

We (Innovation team at Fusion5) are going to integrate this into JD Edwards and increase the priority of incidents based upon what is seen in the images.  

These algorithms can be trained to look for anything we want in images and we can automatically react to situations without human intervention.  

Another very simple extension of what you see here is using AI to rip all of the text out of an image, OCR if you like.  It’d be super simple to look through ALL images and convert them to text attachments or searchable text.


Imagine that you wanted to verify a label or an ID stamp on an object, this could all be done through AI very simply!


Friday, 21 December 2018

Technical Debt - again

In my last post I showed you some advanced reporting over ERP analytics data to understand what applications are being used, how often they are being used and who is using them.  This is the start of understanding your JD Edwards modifications and therefore technical debt.

At Fusion5 we are doing lots of upgrades all of the time, so we need to understand our clients technical debt.  We strive to make every upgrade more cost efficient and easier.  This is easier said than done, but let me mention a couple of ways which we do this:

Intelligent and consistent use of category codes for objects.  One of the code is specifically about retrofit and needs to be completed when the object is created.  This is "retrofit needed" - sounds simple I know.  But, if you create something bespoke - that never needs to be retrofitted - the best thing you can do it mark it like that.  Therefore lots of time will be saved looking at this object in the future (again and again).

Replace modifications with configuration.  UDO's have made this better and easier and continue to do so.  If you are retrofitting and you think - hey - I could do this with a UDO - please do yourself a favour and configure a UDO and don't touch the code!  Security is also an important concept for developers to understand completely.  Because - guess what?  You can use security to force people to enter something into the QBE line - you don't need to use code.  (Application Query Security)



  1. Everyone needs to understand UDO's well.  We all have a role in simplification.
If you don't know what EVERY one of these are - you need to know!

OCM's can be used for force keyed queries.  Wow!!!  Did you know that you can create a specific OCM that forces people to only use keyed fields for QBE - awesome.  So simple.  I know that there is code out there that enforces this.   This is like the above tip for security.



System enhancement knowledge.  This is harder (takes time), but knowledge of how modules are enhanced over time is going to hopefully retire some custom code.  Oracle do a great job of giving us the power to find this, you just need to know where to look:



Compare releases


Calculate the financial impact.  Once you know all of this, you can start to use a calculator like fusion5 have developed, this is going to assist you understand your technical debt and do research around it.  We have developed a comprehensive suite of reports that allow you to slice and dice your modification data and understand what modifications are going to cost you money and which ones will not.  Here are a couple of screen grabs.  All we need to create your personalised and interactive dashboard is the results of a couple of SQL statements that we provide (or you run our agent - though ppl don't like running agents).


You can see that I have selected 5 system codes and I can see how much the worst case and best case estimates for the retrofit of those 5 system codes is.  I can see how often the apps are used and therefore make an appropriate finance based decision on whether that should be kept or not.  You are able to see the cost estimates by object type, system code and more.  Everything can also be downloaded for excel analysis.







Wednesday, 19 December 2018

To keep a modification or not–that be the question

The cost of a modification grows and grows.  If you look at your modifications, especially if you are modifying core objects – retrofit is going to continue to cost you money going forward.

How can you work out how often your modified code (or custom code) for that matter is being used?

One method is to use object identification, but this is only part of the story.

You’ll see below that ERP analytics is able to provide you things like number of session, number of unique users, Average time on page and total time on page for each of your JD Edwards applications.  This can be based on application, form or version – which can assist you find out more.

With this information, you can see how often your modifications are used, and for how long and make a call on whether they are worth their metal.


image

Our reporting suite allows you to choose date ranges and also system codes to further refine the analysis.

image


You are then able to slice and dice your mods (note that we can determine modified objects too, but this is using data blending with data studio) to give you a complete picture:

image


Of course, we can augment this list with batch and then calculate secondary objects from cross reference to begin to build the complete picture.  You want to narrow down both retrofit and testing if you can.


image


See below for how we look at queue concurrency and wait times to work out job scheduling opportunities and efficiencies.

image

Thursday, 13 December 2018

JDE scheduler problems

Who loves seeing logs like this for their scheduler kernel?

108/1168     Tue Dec 11 21:49:02.125002        jdbodbc.C7611
       ODB0000164 - STMT:00 [08S01][10054][2] [Microsoft][SQL Server Native Client 11.0]TCP Provider: An existing connection was forcibly closed by the remote host.
108/1168     Tue Dec 11 21:49:02.125003        jdbodbc.C7611
       ODB0000164 - STMT:01 [08S01][10054][2] [Microsoft][SQL Server Native Client 11.0]Communication link failure

108/1168     Tue Dec 11 21:49:02.125004        JDB_DRVM.C998
       JDB9900401 - Failed to execute db request

108/1168     Tue Dec 11 21:49:02.125005        JTP_CM.C1335
       JDB9900255 - Database connection to F98611 (PJDEENT02 - 920 Server Map) has been lost.

108/1168     Tue Dec 11 21:49:02.125006        JTP_CM.C1295
       JDB9900256 - Database connection to (PJDEENT02 - 920 Server Map) has been re-established.

108/1168     Tue Dec 11 21:49:02.125007        jdbodbc.C2702
       ODB0000020 - DBInitRequest failed - lost database connection.

108/1168     Tue Dec 11 21:49:02.125008        JDB_DRVM.C908
       JDB9900168 - Failed to initialize db request

Who loves spending the morning fixing jobs from the night before and moving batch queues and UBE's until things are back to normal?  Noone!

Here is something that may help, not I must admit I gotta thank an amazing colleague for this, not my SQL - but I go like it.

What you need to do is write a basic shell script (say that was on the ent server) that runs this:

select count (*) from SY910.F91300
    where SJSCHJBTYP = '1'
    and SJSCHSTTIME > (select
                      ((extract(day from (current_timestamp-timestamp '1970-01-01 00:00:00 +00:00'))*86400+
                        extract(hour from (current_timestamp-timestamp '1970-01-01 00:00:00 +00:00'))*3600+
                        extract(minute from (current_timestamp-timestamp '1970-01-01 00:00:00 +00:00'))*60+
                        extract(second from (current_timestamp-timestamp '1970-01-01 00:00:00 +00:00')))/60)-60 current_utime_minus_1hour
                        from dual);

If you get a 1 that is good, if you get 0 that is bad.  You probably need to recycle your scheduler kernel  (that control  record should change every 15 mins at least).

So, if you have a script that runs that, you can tell if the kernel is updating the control record...

Then you can grep through the logs to find the PID of the scheduler kernel and kill it from the OS.  Then I write a little executable that gives the scheduler kernel a kick in the pants (start a new one) - and BOOM!  You have a resiliant JD Edwards scheduler.