game engines

Skill level
1 Basic skills

In 2017, I researched the possibility of combining our DNN anomaly-detection technology of live videos with 3-D common-operational views that would show the location of the anomaly on realistic 3-D terrain models (from Mapbox or Open Street Maps) combined with fly-over capabilities. We were targeting customers with large highway/waterway transportation infrastructures monitored by hundreds of traffic cams. The use of a game engine looked like a viable way to achieve this goal. After I researched a number of technologies and compared Unreal with Unity, it appeared that the open-source Unreal game engine would be the best fit for this use case because:

  • Unreal had better support for 3-D worlds than Unity had at the time. (That is no longer the case today.)
  • Unreal (as-well-as Unity) supported WebAssembly with direct Chrome and Firefox support.
  • Unreal uses C++ rather than C#. Therefore, Unity required an extra cross-compile step to create WebAssemblies build by Emscripten's C++ LLVM compiler. That extra C# to C++ cross-compile step in Unity made the implementation of custom modules for WebAssembly a bit more problematic.
  • Unreal has a built-in graphical world-building tool called BluePrints that could be extended with custom C++ modules. It looked like this 4GL build tool might be a good way for integrators to customize our solution to specific situations without having to write a lot of custom code.
  • It was possible to import terrain maps and infrastructure models using open-source tools like Blender.

Once Unreal was decided upon, I spent some time working with Blender and the Unreal 4 game engine targeted for WebAssembly and the BluePrints 4GL build tool to see what was possible. I got as far a building basic models, terrains and animations using the Unreal game engine and Blender before the stakeholders at Barco decided that this project wasn't inline with the company's revised business direction. In addition, we were not receiving the kind of enthusiastic feedback from prospective customers that we were hoping for. Consequently, the entire anomaly-detection project was shelved.

In 2018, I used the Unity game engine to create realistic 3-D renderings of people's faces to produce roughly 25,000 images of labeled training data for one of our deep-learning network projects. The labelled training data needed to contain precise location vectors of several facial features. Creating this kind of training data from actual photos would have been too costly. Several facial features were randomly varied within the Unity game engine to include as many realistic facial variations as possible. The face assets were provided by Columbia University. I merely programmed the Unity game engine to assemble and generate the labelled image sets using those assets within specific training parameters.

Experiences using this skill are shown below:

Barco Labs (research)

[I know, this section just echos the same stuff as on the résumé. I plan to expand later.] Worked with PhDs, staff and university interns researching disruptive technologies. Barco Labs deliverables are research papers, patents and demos. Any research that might become a viable product in 2 to 5 years is then passed off to one of the product divisions. (Due to the trade secret nature of this research some details cannot be revealed.) Accomplishments: