We talked with one of the companies whose AI agent flew the X-62A VISTA with Frank Kendall in the cockpit.
Founded in 2015, Shield AI is a pioneering company dedicated to advancing autonomous systems for military aviation. With a mission to protect service members and civilians through intelligent technology, Shield AI develops cutting-edge AI pilots and autonomy products.
Their innovative solutions have flown on various aircraft, from quadcopters to advanced military jets like the X-62A VISTA. By combining traditional heuristic methods with neural network reinforcement learning.
We recently had the chance to interview the Shield AI team, led by Brandon Tseng, former Navy SEAL and co-founder/president of the company, and delve into their pivotal role in the historic flight of the X-62A VISTA with U.S. Secretary of the Air Force Frank Kendall aboard.
The discussion you will read in the interview covers the innovative AI agents developed by Shield AI, their current and future projects, and the transformative impact of AI on military aviation.
Highlighting the intricate balance between human pilots and AI systems, the interview sheds light on the advancements and ethical considerations driving the future of autonomous military aircraft.
Question: Can you describe the role played by Shield AI in the flight of Frank Kendall?
Shield AI was one of the companies whose AI agent flew the X-62 with Frank Kendall in the cockpit.
Q: What’s an AI agent and why did several different agents fly with the X-62A?
An AI agent in aviation is a software program that interacts with its environment, collects data, and autonomously performs tasks to achieve predetermined goals. Each AI agent is analogous to a different AI pilot. The X-62A ACE Team is composed of U.S.-based companies, government organizations, and academic institutions, and the program demonstrates the strength of Defense Department organizations working closely with industry and academia toward shared objectives.
DARPA uses a competitive down-select approach to assessing technology in its programs. These start with 3-4 companies and over the course of the program they are down selected based off of their technical performance. Shield AI was the only AI company down selected for the final phase of the program.
Q: Can you tell us a bit more about Shield AI and what projects is the company working on?
Shield AI was founded in 2015 with the mission to protect service members and civilians with intelligent systems. We are working to bring an autonomy product to market, which can be accomplished via working with a number of different companies that build uncrewed jets and through our own R&D efforts. Our autonomy has flown 6 aircraft: 3 quadcopters, MQ-35 V-BAT, X-62 VISTA, MQM-178 Firejet.
We are developing autonomy products to solve operational problems for defense customers. The key to making this operational is the ability to fly consistently and integrate more sensor and mission systems into the autonomous solution.
Q: What future capabilities are you planning to develop for the AI systems used in autonomous aircraft like the X-62A?
As an example, our V-BAT Teams product enables teams of V-BATs (our medium-size vertical takeoff and landing drone) to autonomously perform missions in electronically contested environments, responding dynamically to adversaries and environmental changes. It utilizes a modular Nvidia GPU equipped with Shield AI’s Hivemind AI pilot. Starting with four drones, we plan to double the team size yearly. V-BAT Teams can autonomously operate without GPS or communications, providing 24/7 coverage and targeting enemy assets in operational areas. The system is scalable, supports rapid updates, and includes a ruggedized mobile control station for managing large teams efficiently.
Q: How do you envision the role of AI in military aviation evolving over the next decade?
In the next decade, AI will dramatically transform military aviation. Leading air forces anticipate that Autonomous Collaborative Platforms (ACPs) will significantly enhance their forces’ size, lethality, and survivability. These platforms will revolutionize capabilities and reshape warfare strategies. AI pilots are central to this change, capable of handling both traditional and more complex missions. AI Pilots will team w/ Human pilots to provide mass and increase survivability. Moving forward, the effectiveness of AI pilots will become crucial, positioning advanced software as a key differentiator in future military technology.
Q: Can you elaborate on the current capabilities of the AI agents used in the X-62A, particularly in terms of their tactical maneuvering and decision-making processes?
The X-62A ACE Team marked a milestone in military aviation by executing the first AI vs. human dogfight with a manned F-16, showcasing the practical use of machine learning-based autonomy in flight-critical systems. AI dogfighting is a means to an end – the highly complex, high-risk environment of a dogfight was a compelling problem to leverage for attacking the much bigger issue: What is the most efficient and effective path to optimize the performance and safety of artificial intelligence in aerospace vehicles?
The X-62A’s open system architecture and robust safety sandbox allowed the team to rapidly iterate software and AI agents during a single four-week test window with more than 30,000 lines of flight-critical software changed and up to 8 million neural network parameters refined per AI agent during the window. In total, the team made over 100,000 lines of flight-critical software changes across 21 test flights during three test windows from December 2022 through September 2023 – an unprecedented pace of development.
Q: What distinguishes your AI system from other autonomous aviation technologies currently in development?
Shield AI builds autonomy using a warfighter centric approach. We start with highly experienced current and former fighter pilots who decompose the mission into tasks and establish objectives used to establish the autonomy’s reward structure. AI engineers then leverage and combine traditional heuristic (or rule of thumb) methods with cutting-edge, neural network reinforcement learning techniques. This process creates autonomy that is explainable and intuitive to warfighters. Shield AI’s autonomy is unique because we focus solely on maneuvering the aircraft and employing effects in a timely manner based on an observation space provided by the platform systems and sensors. This approach is directly analogous to the aircrew’s role in traditional aircraft and allows verification, validation, and airworthiness certification of the aircraft separate from the autonomous pilot. It also allows aircraft and mission system manufacturers the ability to upgrade capabilities on separate timelines.
Q: In what scenarios do human pilots still outperform AI, and how do you plan to bridge these gaps?
While AI pilots are revolutionizing military aviation, human-machine teaming remains the ultimate force multiplier. Human pilots still outperform AI in scenarios requiring complex judgment under unpredictable conditions. To bridge these gaps, we focus on synergy through ancillary tools that enhance the operational capabilities of Autonomous Collaborative Platforms (ACPs). This includes advanced mission planning, interaction, training, and debriefing systems. Our approach begins with comprehensive education programs to prepare aviators for effective interaction with AI pilots. We integrate digital twins of our AI pilots into aviator simulators, facilitating realistic training scenarios that improve predictability and performance familiarity. These simulations allow aviators to rehearse with the exact AI counterparts they will partner with in flight, ensuring seamless coordination.
Post-mission, we utilize specialized tools for detailed debriefs, analyzing every aspect of the mission to understand perception, decision-making, and execution discrepancies. This continuous feedback loop enhances both human and AI performance, steadily closing the gap where human pilots currently excel.
Q: What ethical considerations have been taken into account in the development and deployment of AI for autonomous military aircraft?
First and foremost, we subscribe to the principles laid out in the Department of Defense’s AI ethical principles. We are committed to building AI technologies that are responsible, equitable, traceable, reliable, and governable. As long as humans understand what a system is capable of, how to constrain its operational envelope and its potential failures, they are in a position to employ it in accordance with the principles of the law of war.
New technologies in the areas of deterrence and warfare are bound to face criticism and controversy as they advance. Yet there is a discernable trendline, from unguided bombs that were lucky to land within one thousand feet of their target in WWII to precision-guided munitions, that is leading to enhanced control over the effects of warfare. AI and autonomy promise to continue this progression by providing humans with greater influence while simultaneously reducing the number of humans required to be present on the battlefield. Wishing these algorithmic capabilities away in the name of ensuring more human control is akin to removing the laser guidance kit from a Hellfire missile and demanding that the pilot provide all the aiming – the operator’s toolkit is just poorer, and civilians are likely to pay the price. Even if an adversary were to stoop to a lower level, we believe the U.S. and allies would maintain the moral and ethical high ground. If our technology, systems and capabilities are superior, we can still deter major conflict and maintain that moral and ethical high ground.
Q: Can you describe the software development process for the AI systems used in the X-62A, particularly in terms of testing and validation?
We can speak to our software development processes at Shield AI, but we cannot get into any more detail about the AI agents used in the X-62 flights. Shield AI is building the world’s best AI pilot focused on specific operational mission scenarios. To do this we combine many Autonomy & AI techniques – Heuristic rules of thumb (ROT), decision trees, machine learning, and deep learning (aka neural networks). Hivemind is a software framework that delivers resilient and portable artificial intelligence (AI) capabilities for edge systems. Products within Hivemind build upon a modular open system architecture (MOSA) provided by Hivemind Foundations and EdgeOS, integrate seamlessly to deliver the world’s best AI Pilot (Hivemind Edge), provide a robust AI Factory for training, introspection, analysis and evolvement (Hivemind Design), and deliver fluid human engagement with these systems (Hivemind Commander).
Q: How do you manage the continuous integration and deployment of new features and updates to the AI software?
To manage the continuous integration and deployment of new features and updates to our AI software, we utilize the Hivemind Foundations—a highly modular and extensible system designed for seamless integration into various platforms. This system connects sensor data to AI pipelines and actuation systems, enhancing interoperability and ease of adaptation.
At the core of our operations is the Hivemind Design framework, or AI factory, which features plugin-based simulators and analyzers for rigorous testing from software-in-the-loop to hardware-in-the-loop scenarios. This infrastructure is finely tuned for rapid deployment, allowing for quick configuration changes and updates. By leveraging AI and machine learning for testing and verification, we ensure fast development cycles and robust deployment of updates and new functionalities, making our system both resilient and adaptable to new platforms swiftly.
