What if painting a digital world was at your fingertips?
A body tracking "Splatoon type" painting game to demonstrate the power of Azure Kinect DK technology. Players utilize their bodies to digitally paint on the walls with projections - bypassing joysticks and traditional gaming controllers.
The Process
This was a unique and fun project in collaboration with the Create(X) projection space in The University of Hawaii at West Oahu and the Laboratory for Advanced Visualization & Applications (LAVA). Our small team was tasked with providing a new interactive demo to help visitors interact with projection screens.
The objective of this game is similar to the popular Nintendo Switch game "Splatoon", however what made our version unique was that it immersed the players in a fun and colorful world where they could physically interact with projections.During the gameplay groups of players could control a paint sprayer with their arms to add as much color to the world as possible. A score keeping mechanic tracked and quantified the levels of paint coverage achieved by each player (it kept a score of who had covered the most area with their paint nozzles).
We developed this game in Unity. The hardest part about creating this game was the utilization of multiple Azure Kinect DKs. The tracking system had a tendency to lose track of the player when they moved from one sensor to another, or if sensors were placed near each other. We kept having issues of players being detected twice on two separate kinects and had to come up with reliable ways to keep track of players as they transitioned between cameras.
One approach was to calculate the ratio of distances between body parts to identify players (based on either the physical distances between body parts or the angles at which the players limbs are being held). However, the success of this method was at risk if two nearby players were the same height and making similar poses. Ultimately, we used player position data to identify unique players. If a player was being tracked by more than one camera the game would only display the player from the camera they are located closest to.Learn more about Create(X) here
Multiple projectors were set up with Azure Kinect DKs tracking every section of wall around the lab. These technologies have promising applications at art exhibits, advertisements and conventions.
Everyone who walked by wanted to be a part of the testing process
Full Body tracking utilizing the Azure Kinect DK in Unity
