Background Info
About Pokémon Game
Pokémon means Pocket Monsters in Japan. The Pokémon is a role-playing game based around building a small team of monsters to battle other monsters. Pokémon are divided into types, such as water and fire, each with different strengths. Contests between them can be somewhat similar to the simple hand game of rock-paper-scissors. Pokémon launched the Pokémon Go Augmented Reality (AR) game, which became one of the most popular AR games in the world with over 100 million monthly active users at its peak and has generated billions of dollars in revenue.
About NVidia Edge AI: Jetson Nano
The NVidia Jetson Nano is a portable but powerful computer designed for artificial intelligence (AI) and machine learning applications. It helps with image classification, object detection, speech processing, and segmentation. NVidia’s Jetson Nano is a remarkable computational device to make AI work locally.
About Integem Holographic AR Technology
Users can interact with other users and virtual creatures thanks to Integem’s holographic AR technology, which transports them from the real world into a virtual one. It works with any computer or phone device with a webcam connected to the internet. No special goggles or glasses are needed. Integem Holographic AR technology includes the holographic AR experience platform iPlayer® and the holographic AR creation software iCreator®.
In 2018, Integem launched holographic AR education programs for K–12 students. Students ages 5 to 18 at Integem can learn holographic AR coding, STEM, AI, 2D/3D game design, robotics engineering, digital art, 2D/3D animation, 3D modeling, fashion, design, interactive storytelling, space/nature exploration, and more. Teleport into their creation to experience the magic of their holographic AR design and prepare them for future success, all while having a lot of fun! Since 2018, thousands of students have attended Integem education programs from 20+ countries.
Our Design: Intelligent Pokémon Holographic AR Game with NVidia Jetson Nano and Integem Holographic AR Technology
In this project, we leverage the power of NVidia Edge AI for automatic real-time Pokémon recognition and create a magical Pokémon world where players can step into the game and fight each other using Pokémon monsters.
This project aims to inspire children and teenagers to learn holographic AR and AI. They love Pokémon games, and many of them are Pokémon fans. Using Pokémon makes them feel AI is relevant, understandable, and fun. With Integem Holographic AR technology, young students will see the magic and power of AI.
AI Training
We trained the NVidia Jetson Nano to recognize four Pokémon monster toys: Abomasnow, Charizard, Sandslash, and Pikachu, and their locations in the camera view. These Pokémon monsters are carefully selected so that each monster has its own unique strengths and weaknesses, as shown below:
Charizard can beat Abomassnow, Pikachu can beat Charizard, Sandslash can beat Pikachu, and Abomassnow can beat Sandslash.
Intelligent Pokémon Holographic AR Game Design
We use Integem iCreator® to design the Pokémon Holographic AR Game. Two people can join the game remotely. Each player has two Pokémon monster toys.
The design is as below:
Every player in the game secretly chooses one Pokémon monster and places it in the camera’s view area. The camera connects to an NVidia Jetson Nano. NVidia Jetson Nano runs in real-time (30-frame per second) to recognize what Pokémon monsters are in the camera view and the location of the Pokémon monsters. Based on the location of the Pokémon monster, NVidia Jetson Nano will intelligently recognize which player selected which Pokémon monster.
NVidia Jetson Nano will send the real-time recognition results to Integem iPlayer®.
iPlayer® will bring the correlated Pokémon monster to each player, and the players will be inside the Pokémon monster holographic AR fight game where their selected Pokémon monster will show up and fight with each other. Both players will be inside the game, watching how their Pokémon are fighting during the fight. The winner will get a game star and enjoy the winner’s glorification in the game: the winner will show up on the big screen of the Pokémon battlefield and ride Charizard around while being worshiped by the other three Pokémon monsters. A tie is declared, and nobody gets a star reward if two Pokémon monsters cannot defeat one another.
After each Pokémon fight, the players will be prompted to select their Pokémon monster for the next battle match. And it repeats the same game-playing process as before.
The player who first earned three stars will be declared the game’s ultimate winner and receive extensive winner glorification then the game is over.
The technical details are posted on GitHub as follows:
https://github.com/sean-bayarea/Holographic-AR-AI-NVidia-Nano-Project.git
It includes all the training image data and a retrained AI detection model. Follow the steps below to retrain your model and infer it in NVidia Nano. The GitHub site also includes the Integem iCreator® project, which includes 159 stages designed for two people to play the interactive AR Pokémon game. The iCreator project also allows players to remix their Pokémon fighting games.
You can see a sample play video here: https://vimeo.com/779115797/7688887e1d.
Conclusion
The NVidia Jetson Nano is a small but powerful computer designed for Edge AI computing. Real-time AI applications are made possible by their real-time capability. It is a terrific way for young students to learn AI with holographic AR and prepare them for future success.
