AJ is an experienced technical leader specializing in AI, mechanic development, and Unreal Engine 4/5. He has an extensive track record of working with and leading teams world-wide both software and game development as well as in the governmental and military spheres.
AJ’s primary works in the video game and simulation industries have been in Unreal 4/5, though he has significant experience with Python, SQL, NoSQL, R, and Node.JS, and is experienced with using AWS products for both in-engine usage as well as standalone.
Below is a subset of relevant projects AJ has completed:
AI Integration with Sequencer for Cinematics
Time Period:
Client / Purpose :
Skillsets
Engine / Platform
2014 – 2015
International Committee of the Red Cross
Training materials for field personnel to manage elections and poll watching in Africa
- Client migration to sequencer
- AI consolidation to crowd controller
- Deterministic AI Pathing (for film repeatability)
- Art / Animation Design and Direction
The task was to incorporate AI using in-house assets in order to be able to record a number of scenes to drive training and discussion around the topic of election workers in African countries.
Key Challenge:
How to depict a large crowd of 250+ individuals moving in a way that felt realistic and didn’t take away from training, and yet was deterministic enough to be filmed from multiple angles and be consistent.
Solution:
Instead of 250+ AI Controllers, I created one group controller that consolidated command of the AI and gave discrete directions to ensure movement was reliable through every course of filming. In order to ensure the movement was deterministic, I wrote positions to a persistent map to ensure I could repeat ad nauseum the movement of the AI.
To Refine: Give ability to regenerate crowd positions based on random seed
Large Scale Multiplayer AI Training Sandbox
Time Period:
Client / Purpose :
Skillsets
Engine / Platform
2016
International Committee of the Red Cross
Develop a large scale simulation “sandbox” that can handle multiple open world AI interactions in a replicated setting with persistence
- Replication
- AI Scene Design
- AI Virtualization / Optimization
- Systems Engineering
The task was to create a large sandbox training area depicting a village, military installation, and humanitarian mission with multiple AI factions and social interactions. The sandbox was to support at minimum 16 trainees at once for different modular training scenarios.
Key Challenge:
How to depict large amounts of individuals conducting consistent and repeatable patterns of life while managing performance on a dedicated server architecture and allowing trainees to explore / experience any part of the open training world environment.
Solution:
I implemented a self-referenced cell system that was generated out of the editor that would allow the server to understand and virtualized / enable AI according to proximity to the player. Virtualized AI were transported to a pool (to avoid hitching from object instantiation) and their movement was then tracked mathematically based on time of day. Enabled AI are controlled by a central dispatcher until they enter a “scene” (building or such) which would then control their immediate actions within the scope of the scene (such as cooking, cleaning, or selling items). The SceneController could manage interactions between AI as well as be spawned dynamically to enable two AI to talk in a public area. Result was 2000+ AI in the training simulation with a minimal performance impact.
To Refine:
Enable the cell generation system to more efficiently generate cells based on lines of sight, levels of detail in area, etc.
Auto-Cover Detection Test using EQS
Time Period:
Client / Purpose :
Skillsets
Engine / Platform
2019
Personal Use / Contract Toolbox
Develop an easily turned AI EQS setup allowing for cover detection dynamically accounting for terrain and object differences with minimal to no prep.
- AI Navigation
- EQS
- Combat AI
The task was to create an environmental query setup that enabled cover / terrain detection returning appropriate cover locations for.
Key Challenge:
Creating a quick, modular EQS query that would readily recognize collision objects and be able to return appropriate cover locations, regardless of whether collision was terrain based or object based.
Solution:
Utilize offset collision boxes per sampled point with a z offset, and non-visibility from target point to manage default cover locations. Additional tests can be added to give higher weight to cover locations that include a possible “shoot from” location by running a trace with offset. Result is an easily scaleable AI cover system that requires zero preparation of scene or other assets.
To Refine:
Write the EQS Test logic in C++ for performance improvements, as well as an asynchronous trace handler to return to the AI what positions are “shootable”.
Melee Combat AI Setup with Prediction
Time Period:
Client / Purpose :
Skillsets
Engine / Platform
2019 – 2021
Internal / Odin Softworks
Develop reactive and predictive AI to enrich a melee combat simulator
- Replication
- Gameplay Ability System
- AI Systems Design
Utilizing a quality of life plugin for virtual reality development (VRExpansion Plugin) extend the functionality of the plugin to allow for complex AI interactions during combat, to include being stabbed, grabbed, and for the AI to engage with dodge / parry / counter actions.
Key Challenge:
Given the sheer amount of player interactions possible, design a system to abstract these interactions to a few “actionable” interactions in emplace the interfaces to trigger said interactions.
Solution:
Utilizing velocity based tracing on the player’s weapon, I triggered a response check against the gameplay ability system’s attributes to ensure the AI had enough stamina for the selection interaction and then triggered said interaction via gameplay ability and montage play. For appropriate montages, I sampled the player weapon location and direction to ensure the AI could react accordingly (dodge the proper direction, or move their weapon to parry accordingly).
To Refine:
Graduate tracing script to C++, employ procedural animations for greater AI interaction and fidelity
AWS Integration with Unreal Engine for Persistent Player Profiles
Time Period:
Client / Purpose :
Skillsets
Engine / Platform
2022
Internal / Odin Softworks
Create multiplayer backend capable of supporting, storing, and retrieving large amounts of user data in a secure and CAP maximized manner
- Cloud Systems Design
- Database Organization
- AWS Integration
- Systems Engineering
The task was to incorporate create the multiplayer backend able to securely store player account data, character data, inventory, as well as a actions / quest log.
Key Challenge:
Incorporate a database method that could securely and at scale handle read-write operations in a manner that supported onward development but could also be used for internal analytics.
Solution:
Employing an Amazon Web Services stack consisting of Gamelift (Player and Server management), DynamoDB (backend database), Cognito (login and credentialing for players), SQS (messaging between Gamelift Servers). DynamoDB backend is capable of scaling to tens of thousands of writes / reads a second which is more than ample throughput for the multiplayer backend. Using a Struct to JSON to database workflow allows for minimal parsing of data while also minimizing actual replication payloads (even for large amounts of data, such as inventories).
To Refine:
Graduate all DB Operations to C++ and enhance the server side ability to conduct operations in parallel
Replicated Player / AI Interactions in Mobile VR
Time Period:
Client / Purpose :
Skillsets
Engine / Platform
2022 – ongoing
Oculus Launchpad
Convert single player MVP VR project to multiplayer with fully replicated interactions
- Replication
- AI Interactions
- VR Development
- Systems Engineering
Following the selection of FromLight as an Oculus Launchpad grant awardee, we determined the need to bring the current, single player MVP into a multiplayer setting employing the AWS framework
Key Challenge:
Numerous interactions which are easily implemented in single player become exponentially more difficult in multiplayer and require specific implementations (such as physical animations, dialogue) etc. Furthermore, development and actions have to account for multiple players interacting with the same actor at once.
Solution:
Utilizing Epic’s gameplay ability system, we were able to implement the vast majority of replication tasks without needing to expand out into vast replication graphs in the individual actors. This expanded into a GAS-enabled AI scene actor system which allows the AI to follow a number of lively realistic states providing enhanced immersion for AI scenes.
To Refine:
Include level based master controllers to handle time differentiated patterns of life for the AI with respect to availability for goods and services in multiplayer.
Project Management / Art Direction
Time Period:
Client / Purpose :
Skillsets
Engine / Platform
2014 – 2017
International Committee of the Red Cross
Advise and set up program to enable onward virtual training scenarios and anticipate client art needs
- Project Management and Direction
- JIRA / Confluence
- Source Control
- Knowledge Management
Provide a sustainable model for art direction and software/game development project management that could be replicated by the client organization and maintained for a long-term commitment to virtual training.
Key Challenge:
How to socialize and document key project management tenets and software design concepts to client organizations with minimal if any exposure to these concepts while actively in a production environment for that organization.
Solution:
Mirrored internal project management systems and setup to client organization and conducted left seat / right seat ride instruction alongside documentation to ensure client could repeat and understand the necessary operations for the software project management. Additionally developed internal workflows to gather client art requirements and implement those as finished products.
Autonomous Learning AI Integration with Unreal Engine
Time Period:
Client / Purpose :
Skillsets
Engine / Platform
2018
Education
Develop an autonomous AI agent in Unreal Engine that could use actor critic network to target a specific colored model and engage a shoot mechanic
- Machine Learning
- Python / Unreal Integration
- Abstracted AI Systems
For a capstone AI course I intended to implement an autonomous agent that could take the camera render as input and appropriately (and in real time) target an opposing player and shoot them in game.
Key Challenge:
Training an AI using reinforcement learning on an action space that is nearly infinite.
Solution:
I discretized the action space to make for specific movements the AI was able to engage and increased the speed at which the AI was able to process the images, allowing for near real time training. With the training time as the limiting factor, I then scaled the world time to adjust allowing for a compressed training cycle in which I was able to training the AI for roughly 1.4 “years” in a single day. The result was an autonomously trained AI that could locate and target the player in a maze while only interacting with the same controls the player had access to.
To Refine:
I would remove the control access and have the AI learn from a subset of behaviors that were scripted in a conventional manner.
Marketing and Game Analytics
Time Period:
Client / Purpose :
Skillsets
Engine / Platform
2022
University of Wisconsin – Eau Claire Data Science Masters Program
Conduct a detailed analysis of video game sales data in which to recommend particular traits that would enhance potential sales
- Data Cleaning
- Analysis
- Machine Learning and Statistics
- Communication
The assignment was to conduct an analysis on video data from 2020 to 2022 in order to identify particular common traits in videos with positive sales and provide feedback on development and marketing focus
Key Challenge:
Acquiring, deconflicting, and normalizing data from multiple sources, of which there was considerable overlap in the possible data points.
Solution:
Deconflicting the readily available commercial datasets from sources like Steam, VGInsights, and Kaggle resulted in a far more robust data set. Once the established data set was created, I normalized between AAA studios and indie studios, then separated by genre and then how the game was summarized (using the developer’s summary on Steam, tokened for relevant words).
To Refine:
I would do more exploration within the variables, particularly the game summary and reviews, and look to further define games that have lasting player counts. Another challenge that needs to be managed in this is the relative change of playing and buying habits over COVID lockdown.
AWS Front End / Back End Game Development using Lambda
Time Period:
Client / Purpose :
Skillsets
Engine / Platform
2022
Rivals Media
Develop and manage full stack production and team utilizing AWS to provide event based trivia games at scale
- Front End Project Management
- Backend Tools Design and Implementation
- PostGres SQL DB Management and Design
- AWS Engineering
Rivals Media creates simple trivia games to run sweepstakes for televised or in-person events. Using leveraging AWS, I implemented the integration of a third party backend application with PostGres SQL hosted an AWS Aurora serverless instances, through the use of AWS Lambda which allows us to simultaneously serve 50,000+ concurrent users.
Key Challenge:
Ensuring data management and read/writes to a relational database at scale with comparatively expensive read-writes and the subsequent analytics that can be drawn from that database.
Solution:
With optimized leverage of AWS Lambda and event based scaling up of servers, the bottlenecks were nearly completely eliminated.
To Refine:
Would migrate the active transaction elements of the DB as well as the dynamic front end service to NoSQL (likely DynamoDB) allowing for scale orders of magnitude higher than current, while implementing post event writes to the production relational DB allowing for the more complex analytics available there.