Boosting AUKUS innovation with AI-powered synthetic environments

After officially announcing our October Sky participation at DSEI, our team flew over to Sydney, Australia, to attend the inaugural industry-led event and help tackle critical technology adoption challenges currently faced by the Australian Defence Force. Orchestrated by innovation execution company, Chaos1, AUKUS-aligned organisations from the US, UK, and Australia came together with the aim to rapidly bridge the gap between emerging technology and mission-viable capability in priority areas for the Australian Department of Defence.

The technological advancement and proliferation of drones in the modern operating environment is a major point of concern for the Australian Armed Forces due to the hybrid role they play on the battlefield and the threat they pose to national critical infrastructure. Hence, exploring Counter-UAS (Unmanned Aircraft System) systems was one of the five technology objectives curated by Chaos1 and the one Hadean was tasked to address for the duration of the two-week long event. In particular, our team zeroed in on the protection of an airport from the “Gatwick problem” , public safety from drone attacks, and the safety and viability of heat-powered systems for counter-drone operations.

By incorporating multiple graphical engines, including VBS and Unreal Engine, with a meticulously crafted pattern of life simulation designed to reflect the bustling metropolis of Melbourne, with thousands of citizens, Hadean set the stage for a realistic scenario of a drone flying in an urban environment and being shot down by a laser system. The drone seen in the virtual representation of Melbourne seamlessly mirrored the physical motion of a drone flying in a warehouse and was multiplied virtually to create a dynamic drone swarm inside the synthetic environment.

Provided with a realistic and complex backdrop, the demo used data of a drone being shot down by a high-energy turret. It consisted of XYZ coordinates of the drone, as well as the firing angle of a laser turret that was tracking it, firing state (laser on/off) and drone state (alive/dead). With this data, the Hadean team conducted a series of experiments, placing the drone swarm in various locations, including Melbourne Airport, the MCG, and even above a local fish and chips shop. The drone’s position was fed through Kafka, mimicking real-time data, and it was visible in both 2D in the pattern of life and simultaneously in 3D in Unreal Engine and VBS.

To infuse the scenario with a lifelike aura, Hadean also introduced a social media hub, complete with a series of LLM-generated drone-related tweets. These tweets added a layer of authenticity to the virtual world, revealing the power of AI-generated data. Initially, the drone became a curiosity to people in the pattern of life who tweeted about seeing it nearby, and then became a threat when a warning was issued, prompting scared tweets from terrified bystanders. An interest management system which imposes regions of interest called “interest rings” were used to control and filter the amount of information available to participants in the training environment, affecting what they could see across Unreal Engine, VBS and in the pattern of life. By moving interest rings from location to location the team was able to dictate the spatial availability of information about the drone available within the simulation and dynamically observe the shifting reaction of citizens as they moved from “looking for” the drone to “finding it and feeling threatened”.

The demo showcased how Hadean’s spatial computing technology can bridge physical and virtual worlds and help explore the impact of shooting down a real-world drone across many systems in a common operating picture. By bringing physical assets into the digital world it provided a safe and cost effective environment to test the capabilities of a high-energy C-UAS system while addressing and assessing fundamental questions about public safety during a drone shootdown, considering the potential for laser back-reflection. It also demonstrated how an LLM integration can be used to create free-form, natural language control for the interest management system, harnessing a geospatial understanding of Melbourne.

The reception from the audience and experts was overwhelmingly positive, marking the viability of the Hadean Platform for Defence to drive impactful training and decision support and the commitment of the Hadean way to support the push for innovation across AUKUS as a trusted partner with an international footprint.

Interested to learn more about the Hadean Platform for Defence? Continue reading here.