Wednesday, 18 December 2019

Modernizing the shop floor - AI + camera = actionable insights

I was told a fact that the average piece of equipment on the shop floor in Australia is 21 years old, wow – they just don't make them like they used to.  I tried to back this up with some quotes (https://www.hpe.com/us/en/insights/articles/4-ways-to-bring-legacy-manufacturing-equipment-into-the-iot-age-1903.html) says 20-25, that'll do.

So how can a traditional manufacturing organisation adopt some modern processing capabilities that are being facilitated by modern cloud compute and of course IoT?  The shop floor needs processes to tell you when something is wrong, or validate that an even has occurred without expensive retrofitting of custom hardware?  I think it's very possible - to the point of being simple.  For the purposes of this blog entry, I'm going to specifically talk about image processing to facilitate shop floor productivity.

We've been working on some pretty cool technology in the background.  As a group at ww.fusion5.com.au we have been focused on keeping up to date on the latest cloud algorithms that we can call to create value.   One that I really like is all of the AI processing that you can call on your images (in bulk)

Here is a sample image - let's use the cloud to extract some value from it...  I appreciate that this is not the shop floor, but I'm getting there...  And I love to explain things with images of beer!


Nice, this is a beer list from a great pub that I like to visit.  If I wanted to index the contents so that it was searchable, I need to employ some OCR to rip the text out, I have a demo here:


You should be able to login and use it.

Look what it extracts - amazing!

[ "carringbush hotel victoria pale ale 4.2% 4,6% 3.5% temp bad shepher sold out! thund road sorry.. * drayght bodriggy * speccy juice * krush pale * summer ale *mornington \"twc bir stc gr. 4.7% * kaiju! tropical 'ko 4.7% mt.goat solts out \"r session ipa 5.5% kaiju! signal sun s fixation ipa * xxxx gold 6.4% of 3.5% *coldstream sold pple cider 5% 45% american pale aleale 5.7% billy the mid & raspberry 3.57 grapefruit 150ml 7/\" out \"mt.goat * steam ale sorry! carringbush hotel pe closed on xmas boxing day, new *monteiths r's eve and new year's day. will be open all mt.goat infysed days from 12pm until late. ballast sculpin point ipa * feral biggie juice neipa 6% matso's hard lemon lemonade alcoholic 4,5%" ]

Let's look at what else can be done.



This is a great example of some serialized boxes from my medicine cabinet, probably expired.  But it had me thinking about how this technology really can make things easier.  People could take a picture of their medicine cabinet and this could tell you what is expired or quickly what they do!

[
"prescription on keep out of read 30 tab solone prednisolone each table prednisolo inova l10,12 helpo nsw 2067 code no gujdrugs- la064303 190311b 2022.02 lot 6104550 gtinc01) : 18901079058253 b (10):gvh017006 exp(17):06/2020 sr no (21) : tbukns31589w your assurance of quality healthcare. this product has been developed and manufactured to the highest standards. johnson & johnson pacific australia · new zealand 45 jones street, ultimo nsw 2007, australia and auckland, new zealand for more information call toll free: 1800 029 979 australia, 0800 446 147 new zealand or visit www.codral.com.au or www.codral.co.nz made in australia from imported and local components. 9 300607 1805 10\" batch no: 2 8 7963 11/21 expiry: 60027607 lot 01-2020 exp 2402000706 5152506623 astrazeneca py77012 may2020"
]

As you can see from the above, you get the serials and everything from the labels.  You can easily turn this into indexed data and search on it, as well as audit-able data.



Imagine that you have a camera mounted on a production line and this camera is going to take a video feed of everything in it'd field of vision.

Then, when the data changes, it is going to call a webhook.  Simple…  Right? 

The power of this very simple process is amazing.

We could have a basic web camera on the shop floor that can then ANY analogue  information (a dial or a readout) to a digital signal.  This information is then ready for your data lake or your big data locations for storage.  This data is going to provide you with a wealth of information on the shop floor process.  For example, just say you were a food exporter and you needed to ensure that all of your labels had been printed properly – or you are going to lose your export licence?  This simple solution could do this, very simply.

Running an AI fuelled OCR process on the label above gives the following output:

[
"any meat works co long flat road anytown nsw boneless beef la(*l)+ product of australia *ygs* striploin znuy701 over 4.5kg iw/vac 32307 keep chilled o°c (01) 9 9316710123453 (3101) 000262 (13) 150310 (21)41457354 品質保持期限090613 s/o: 3345rt5 austral 9999 nspecte pkd on 10-mar-2015 17:26 best before 22-may-2015 26.2kg net weight carton id 41457354 est. no.9999"
]

Great, that was simple.  I got my development team to then create a bit of an interface to enable me (or anyone) to extract the relevant data from what OCR wants, of course I call back on a regex:

.*\ss\/o:\s(?<serial>\S+)\s.*pkd\son\s(?<packedDate>\d{1,2}-\S{3}-\d{4}).*\s(?<weight>\d+\.\d+)kg.*

This has been defined to take any picture, AI tears that down, and then the regex is applied to extract 3 named groups of data:

Named Groups
{
"serial": "3345rt5",
"packedDate": "10-mar-2015",
"weight": "26.2"
}

Awesome, so we have 3 named groups identified and extracted.  We now also have an interface to program our orchestration end point:


Okay, so you do not have my entire basic Auth string, but you can see what is being done.

From my named extraction groups, I can place those variables in the orchestration call in JD Edwards (of course this could be any web hook).

And with this simple piece of code, I'm able to complete finished goods in JD Edwards from the installation of a web camera and some smart code behind it.

Imagine reading gauges, temperatures, movement of devices etc – all could be done with the same simple steps.

Plugging the power of the cloud (into the edge if you must) and then helping this data with forecasting and shop floor efficiency.

This is completely generic too, applying this to stock takes very simply.  Using the power of AI to do the counting for you!

I take this to the next level with another example: box counting
produces
[ "aperf free free shipping free free free shipping free shipping shipping free free free shipping free free shipping shipping shipping shipping free free shipping free shipping free shipping free free shipping shipping shipping free stipping free free free shipping free free free shipping erf ship shipping shipping shipping free free lee shipping ping free free shipping shipping shipping drf" ]


With a quick bit of excel, I find that there is 28 instances of the word free...  So I can assume that there is about 28 boxes.  This is a trivial example.  But imagine a single high quality image of a warehouse shelf, if it was properly faced - the ability to stocktake with a single image could be amazing.

You can see that we have stood up the ability to extract ANY data from an image (I've shown some simple examples, but we can train specific models to look for quality issues or interpret images as you need) and record that data immediately into JD Edwards.  ALL you need is the camera (we partner with meraki if you need some help) and we can do all the rest...  Extract the data from the image and get that into JD Edwards (or any system).

At fusion5 we have our #fastfive innovation offering where we come to you and assist you uncover innovation opportunities rapidly.  We run some workshops and then can put together some very impressive POC's with small amounts of money!






Extending JDE to generative AI