Military

An F-22 Pilot in a Real Jet Just Took on China’s J-20 in Augmented Reality

December 7, 2020Sandboxx News
An F-22 Raptor demonstrates its warfighting capabilities during an F-22 demonstration at the biennial Arctic Thunder Open House on Joint Base Elmendorf-Richardson, Alaska, July 26, 2014. The event features more than 40 Air Force, Army and civilian aerial acts, July 25-27, and has an expected crowd of more than 200,000 people. It is the largest two-day event in the state and one of the premier aerial demonstrations in the world. The 2014 Arctic Thunder Open House is a proud part of the Anchorage Centennial Celebration.

An F-22 Raptor demonstrates its warfighting capabilities during an F-22 demonstration at the biennial Arctic Thunder Open House on Joint Base Elmendorf-Richardson, Alaska, July 26, 2014. The event features more than 40 Air Force, Army and civilian aerial acts, July 25-27, and has an expected crowd of more than 200,000 people. It is the largest two-day event in the state and one of the premier aerial demonstrations in the world. The 2014 Arctic Thunder Open House is a proud part of the Anchorage Centennial Celebration.

This article was originally published Nov. 16, 2020, on Sandboxx News. Follow Sandboxx News on Instagram


By Alex Hollings


An American veteran F-22 pilot just took on a virtual representation of China’s Chengdu J-20 stealth fighter from the seat of a very real aircraft in flight. The Chinese fighter was projected on the pilot’s augmented reality helmet-mounted display, giving a real pilot in a real aircraft the opportunity to train against a seemingly real foreign opponent. According to the two firms responsible for the test, this marks the first time such an engagement with a virtual adversary has ever been achieved.


In a story first covered by Thomas Newdick at The Warzone, U.S. companies Red 6 and EpiSci announced the successful test of their augmented reality training program earlier this week. Their veteran F-22 pilot participated in the test from the stick of a Freeflight Composites Berkut 560 experimental aircraft. Inside the cockpit, he wore a helmet that included an augmented reality display that projected the image of his opponent, China’s J-20, in his field of view. The enemy aircraft was not controlled by another aviator, but was instead a reactive adversary controlled by EpiSci’s Tactical AI technology.


Red 6 CEO Dan Robinson on Instagram

“With this first-ever within-visual-range dogfight against an AI bandit, EpiSci’s Tactical AI demonstrated the ability to work on a real aircraft, with flight-ready hardware and sensors,” said Chris Gentile, EpiSci’s Vice President for Tactical Autonomous Systems.


“While fielding autonomous systems that control fighters may be in the future, this system is ready now to bring next-generation capability to our training programs, providing immediate benefit to the USAF’s ability to develop and maintain world-class fighter pilots. By introducing them to this technology now, they’ll be even more prepared to use a range of unmanned tools in the future.”


The AI-enabled, augmented reality J-20 pilot leveraged EpiSci’s experience in previous virtual dogfights. The firm was one of eight teams that competed in the U.S. Defense Advanced Research Projects Agency’s AlphaDogfight trials, which pitted a real U.S. Air Force F-16 pilot against virtual fighters crewed only by artificial intelligence. Unlike this recent test, the AlphaDogfight trials were held entirely within a virtual world, with the human pilot managing his aircraft through a virtual reality headset and game controls. The AI pilot handily defeated its human counterpart an impressive five times in a row, though it’s important to note that the test’s virtual environment and unusual methodology favored the AI in a way real combat may not have.


DARPA’s AlphaDogfight trials

Red 6 worked in conjunction with EpiSci’s AI pilot with their Airborne Tactical Augmented Reality System (ATARS), which was responsible for the pilot interface that projected the virtual opponent into the pilot’s augmented reality view, effectively serving as the point in which the real pilot and aircraft could interact with the virtual J-20 opponent.


“Red 6’s ATARS system enables real pilots to fly real airplanes connected together in an augmented world,” explained Dan Robinson, CEO and founder of Red 6.


“With the additional integration of Tactical AI into our platform, we are now able to interact and respond to any threat aircraft. This opens spectacular possibilities for training.”


This technology could offer American fighter pilots a significant advantage in any potential 21st century conflict. While aggressor squadrons offer a very realistic simulation of the combat environment, there are real limitations to what can be accomplished through the use of aggressors. Aggressor squadrons, sometimes called “adversary squadrons,” are made up of aviators and aircraft used specifically for the purpose of simulating enemy forces in war games. Aggressors, many of whom are veteran military aviators working in the private sector, fly actual combat aircraft in simulated missions against U.S. forces.


Here’s a video of an aggressor A-4 Skyhawk running for its (simulated) life as a U.S. Air Force F-22 Raptor closes in.



While training against one another and aggressor aircraft offers American aviators with a very realistic facsimile of combat operations, this approach to training often does not offer the ability to fly against the sorts of jets American pilots will actually square off against in a near-peer conflict against a nation like China. The use of the J-20 for this test speaks to that limitation, as American pilots have never had to square off against an enemy force operating their own stealth aircraft before.


While F-117 Nighthawks have been seen in the skies over America’s southwest in recent months, prompting some to wonder if they are serving in an adversary capacity, the “stealth fighter” was never actually a fighter at all. The F-117 was designed to carry a relatively small payload of two bombs deep into enemy territory while avoiding detection. Aircraft like the J-20, on the other hand, are purpose-built to conduct fighter operations at long distances, including engaging enemy fighters. Put simply; the F-117 just isn’t a good stand-in for 5th generation fighters like the aforementioned J-20 or Russia’s Su-57.


While it is believed to leverage stolen plans for the U.S. Air Force’s F-22 Raptor, the J-20 is much longer and has front canards, both of which affect the aircraft’s performance in a fight. These, along with a number of other differences, make it difficult to emulate the J-20’s performance in aggressor squadrons. (WikiMedia Commons)

Virtual adversaries, on the other hand, can be programmed to fly just like we anticipate enemy fighters to, using publicly sourced information about their aerodynamic capabilities and projected stealth profile. If leveraged by the U.S. Air Force, these AI models could be updated using classified intelligence regarding opponent aircraft, potentially making the virtual fighter’s tactics and performance even more accurate. That experience against enemy fighters could make all the difference when it comes to the split-second decisions fighter pilots are faced with in a dogfight.


“Dogfighting, or Basic Fighter Maneuvers as we call it, is an incredibly complex and dynamic environment. The most difficult part is perceiving what the adversary is doing,” F-35 Pilot instructor Justin Lee told Sandboxx News.


“You’re looking for minute changes in their lift-vector which foreshadows their next move. That’s why it’s important to have good vision (which can be corrected with glasses or surgery).”


This sort of training is just one of a number of areas where artificial intelligence is likely going to dramatically change the world of military aviation. While AI-controlled virtual aircraft can change the way fighter pilots square off against opponents in training, artificial intelligence will eventually also supplement the pilots themselves. While DARPA’s AlphaDogfight trials pitted an AI pilot against a human one, the stated goal of their Air Combat Evolution (ACE) endeavor was to help pilots grow more accustomed to sharing the sky, and even the cockpit, with AI.


“Regardless of whether the human or machine wins the final dogfight, the AlphaDogfight Trials is all about increasing trust in AI. If the champion AI earns the respect of an F-16 pilot, we’ll have come one step closer to achieving effective human-machine teaming in air combat, which is the goal of the ACE program.”


Artificial intelligence can supplement human pilots within the same aircraft in a number of ways: from more capable auto-pilot systems to taking quick action when an inbound missile is detected. However, in order for pilots to be able to fully leverage the capabilities AI can offer, they need to learn to work with these systems, to trust the decisions they make, and to adjust their approach to combat operations. The more simple tasks the AI can take over for the human pilot, the more focus that pilot can place on complex tasks like managing the battles space.


The Kratos XQ-58A Valkyrie demonstrator (U.S. Air Force)

Pilots can also soon expect similar AI systems to control autonomous wingmen flying alongside them into battle. Program’s like the U.S. Air Force’s Skyborg are already aiming to field armed UCAVs alongside crewed aircraft. By using low-cost attritable platforms like the Kratos Valkyrie, Skyborg drones could support fighter operations by taking orders from human pilots and using AI to quickly determine the best ways to achieve their objectives. These drones could engage ground targets, fly ahead of the manned fighter to increase its sensor range, or even sacrifice themselves to save the the pilot and his or her aircraft.


“Pilots of the future will need to be comfortable teaming with AI — and training is the ideal place to introduce this technology,” said EpiSci’s Gentile.



Sandboxx News
Sandboxx News

Sandboxx News provides the latest news, entertainment, and motivation from all around the force.

More from Coffee or Die Magazine
Coffee Or Die Photo
From the Team Room to Team Room Design: An Operator’s Creative Journey

BRCC partners with Team Room Design for an exclusive T-shirt release!

Coffee Or Die Photo
Get Your Viking On: The Exclusive 30 Sec Out BRCC Shirt Club Design

Thirty Seconds Out has partnered with BRCC for an exclusive shirt design invoking the God of Winter.

Grizzly Forge BRCC shirt
Limited Edition: Grizzly Forge Blades on an Awesome BRCC Shirt

Lucas O'Hara of Grizzly Forge has teamed up with BRCC for a badass, exclusive Shirt Club T-shirt design featuring his most popular knife and tiomahawk.

BRCC Limited Edition Josh Raulerson Blackbeard Skull Shirt
From Naval Service to Creative Canvas: BRCC Veteran Artist Josh Raulerson

Coffee or Die sits down with one of the graphic designers behind Black Rifle Coffee's signature look and vibe.

Medal of Honor is held up.
Biden Will Award Medal of Honor to Army Helicopter Pilot Who Rescued Soldiers in Vietnam Firefight

Biden will award the Medal of Honor to a Vietnam War Army helicopter pilot who risked his life to save a reconnaissance team from almost certain death.

dear jack mandaville
Dear Jack: Which Historic Battle Would You Want To Witness?

Ever wonder how much Jack Mandaville would f*ck sh*t up if he went back in time? The American Revolution didn't even see him coming.

west point time capsule
West Point Time Capsule Yields Centuries-Old Coins

A nearly 200-year-old West Point time capsule that at first appeared to yield little more than dust contains hidden treasure, the US Military Academy said.

  • About Us
  • Privacy Policy
  • Careers
Contact Us
Contact Us
© 2024 Coffee or Die Magazine. All Rights Reserved