Barco Labs Graphical Content Analysis Server

Company
Barco Labs division of Barco NV
Location
Milpitas, CA
Start
June 2018
End
August 2018

Later on in the DNN anomaly detection project we had the opportunity to temporarily integrate our A.I. technology into a Barco control-room product called OpSpace that would be demoed at the 2018 ISE Expo. The OpSpace system contained a subsystem called the EDP Analytics Service (EAS) which used a graphical IoT builder call Node-RED. At the request of some of our potential customers we implemented a way for operators to manually define regions and anomaly rules even though this went against our machine-learning approach.

I developed a new Docker image called the Alice GCA (Graphical Content Analysis) or VCA (Video Content Analysis) server for integration with OpSpace-EAS. (Hence, GCA = VCA.) It was developed on top of the NodeJS/Hapi platform rather than ExpressJS. My GCA server Docker could manage clusters of Alice Docker containers using the "socat" (unix socket) trick to interact directly with the Docker daemon to start and stop the Alice Docker containers. The Alice DNN dockers themselves were modified by Jonas to work with my GCA server and Ankit did the OpSpace side of the GCA integration, so this was a group effort between the Enterprise EAS team and the Barco Labs A.I. team. The whole thing was up and running in time for the ISE Expo where we were able to show the enhanced analytic capabilities of the Alice DNNs within the OpSpace product using the Node-RED graphical GUI to connect video sources to A.I. analytics. I idea was to make it easy to customize OpSpace to customer needs by simply dragging and connecting nodes on a graph. This resulted in a very cool teaser of things to look forward to in the OpSpace product.

Here is an example of the Node-RED GUI being used to graphically connect a SCADA video source to an Alice color anomaly analysis node which in turn sends MQTT anomaly detection messages to a message debug node that shows JSON payloads in the right panel. In production use, the message debug node would be replaced with a multi-point node forwarding the anomaly message to other nodes such as a storage node (front-ending the Casandra DB) and various dashboard nodes for immediate display.

NodeRed GUI for GCA

Note that several GCA server "analysis nodes" were added to the EAS Node-RED GUI so the OpSpace-EAS could connect with my GCA/VCA server docker. Each node has a configuration panel for entering various details. Here is the configuration for the color anomaly analysis node. (Node-RED auto-fills most of the connection info.)

Node-RED Color Anomaly Setup

Here is a component diagram of my GSA/VCA server docker with OpSpace-EAS shown on the far left and the Alice Engine Dockers shown on the far right. Due to some limitations with the way Node-RED needed to communicate we were limited to using Server-sent Events (SSE) where my GCA/VCA server would send one-way events to Node-RED and Node-RED would send HTTP REST requests back to my GCA/VCA server. Once things were running we used MQTT.

Alice GCA component detail

Since OpSpace was a huge dedicated hardware system with only one system in-house, I needed a way to simulate that part of OpSpace which communicated with my GCA/VCA server. So, I created a SSE tester that I could use in place of OpSpace-EAS for debugging my GCA/VCA server code. The rest of this page shows how I used my SSE Tester to run my GSA/VCA server docker which in turn ran the Alice DNN dockers.

The SSE Tester has several GUI modules which are shown here.

SSE Tester screen

On the left, one or more video feeds can be captured. Instead of converting RTSP to WebRTC, I punted and used FFMPEG to do frame captures overwriting the image displayed by the SSE Tester every second. Source stream regions can be defined by simply dragging a rectangle over the region of interest in the video feed. Once a region is defined, clicking on a region displays a configuration dialog where different anomaly rules can be applied. When all the regions are setup with anomaly rules, the tester dialog can be used to run clusters of Alice DNN dockers which will detect anomalies when they occur. In the middle, is an Alice cluster management dialog which displays the current state of all the Alice dockers (along with controls to stop them as needed).

When an anomaly event is detected, an image capture is taken along with a bunch of anomaly capture data listed below it. Detected anomaly events appear on the right with the region causing the anomaly highlighted. (When running within the actual OpSpace system, the Node-Red "analysis nodes" have a similar set of anomaly configuration dialogs.)

Once a region is drawn it can be deleted by clicking on the [x]. (In the OpSpace GUI those regions can also be moved and resized).

SSE Tester regions

The configuration summary dialog displays multiple color rules, multiple OCR rules and value range rules for a given region.

SSE Tester configuration summary dialog

Predefined sets of anomaly rules for multiple regions can be saved and retrieved later for quick reuse.

SSE Tester pre-defined rule sets

Then the Alice DNN dockers can be started by clicking [Start].

SSE Tester start

Once started, the cluster of Alice DNNs will now look for anomalies within the video feeds.

SSE Tester running

The Alice clusters can also be monitored and managed from the Alice monitoring dialog.

SSE Tester Alice cluster dialog

And there you have it -- my Graphical Content Analysis server and SSE Tester.