DIY AR Interfaces
2020 Design Platform, NUS DIDThe COVID-19 pandemic has amplified the role of different digital contexts for living, working, learning, and playing. At the same time, it has exposed the fragility of the global supply network and fueled an already active DIY culture. In this design platform, students were tasked to pull these trends together and develop DIY tangible interactive experiences for a specific context, supported by fiducial markers and computer vision (a facet of augmented reality).
Built on top of Printed Paper Markers
Student Projects
AruControls
Carina Lim, Choo Yuan Jie, Prasanth Kumaar Kunasilan
AruControls is an app that connects DIY physical modules to a particular digital action, such as a keyboard press. It consists of 6 core modules made with AruCo markers, which enables the detection of a physical interaction using computer vision. When a marker is detected, the computer will respond according to a digital action as indicated on the AruControls app. By coding according to marker detection methods, AruControls also expands the possibilities of physical interactions: anyone can personalise their own productivity board with keyboard shortcuts; create their own interfaces as embodied learning tools; or construct their own controllers for new gameplay experiences. AruControls is open-source and designed with affordance, tactile feedback, and ease of construction. Find construction instructions at instructables.com/arucontrols, and access the app at arucontrols.github.io.
Carina Lim, Choo Yuan Jie, Prasanth Kumaar Kunasilan
AruControls is an app that connects DIY physical modules to a particular digital action, such as a keyboard press. It consists of 6 core modules made with AruCo markers, which enables the detection of a physical interaction using computer vision. When a marker is detected, the computer will respond according to a digital action as indicated on the AruControls app. By coding according to marker detection methods, AruControls also expands the possibilities of physical interactions: anyone can personalise their own productivity board with keyboard shortcuts; create their own interfaces as embodied learning tools; or construct their own controllers for new gameplay experiences. AruControls is open-source and designed with affordance, tactile feedback, and ease of construction. Find construction instructions at instructables.com/arucontrols, and access the app at arucontrols.github.io.
Elucidate
Lim Jing Jie, Nysha Tan, Wu Junyi
Our phone always tries to grab our attention for hours on end, making us miss the fleeting moments around us. Have you ever wondered how light sounds like? How about the wind? Elucidate encapsulates the phone and turns it into an ambient interface that detects wind and light. In the wind version, flaps have been designed to be sensitive to changes in air movement. Adhered to these flaps are ArUco markers which are assigned to a sound sample. The phone plays music when the wind pushes the flaps into view. In the light version, an optimised number of cut-outs were determined in order to control light streaming in. The music changes according to how the light shines onto the markers. Elucidate is flat-packed and can be assembled in one motion, making it easy to tune into temporal tunes at any time.
Lim Jing Jie, Nysha Tan, Wu Junyi
Our phone always tries to grab our attention for hours on end, making us miss the fleeting moments around us. Have you ever wondered how light sounds like? How about the wind? Elucidate encapsulates the phone and turns it into an ambient interface that detects wind and light. In the wind version, flaps have been designed to be sensitive to changes in air movement. Adhered to these flaps are ArUco markers which are assigned to a sound sample. The phone plays music when the wind pushes the flaps into view. In the light version, an optimised number of cut-outs were determined in order to control light streaming in. The music changes according to how the light shines onto the markers. Elucidate is flat-packed and can be assembled in one motion, making it easy to tune into temporal tunes at any time.
FunFund
Winnie Lim, Siti Sahrah Binte Juari, Rain Tay
FunFund is a DIY coin bank designed to educate children the concept of tangible and virtual money as Singapore transits to become a cashless society. Leveraging the potential of fiducial markers, the money-saving journey is extended with added interactions. Through making it a dynamic experience, FunFund aims to motivate children to cultivate a habit of saving money and taking charge of keeping records of the balance. As an open-source design, FunFund can be adapted for other currencies simply through adjusting the sizes of coin slider filter, thus spreading the virtue of saving to other parts of the world.
Winnie Lim, Siti Sahrah Binte Juari, Rain Tay
FunFund is a DIY coin bank designed to educate children the concept of tangible and virtual money as Singapore transits to become a cashless society. Leveraging the potential of fiducial markers, the money-saving journey is extended with added interactions. Through making it a dynamic experience, FunFund aims to motivate children to cultivate a habit of saving money and taking charge of keeping records of the balance. As an open-source design, FunFund can be adapted for other currencies simply through adjusting the sizes of coin slider filter, thus spreading the virtue of saving to other parts of the world.
HEXBOX
Yong Zhen Zhou, Feng Guozi, Claira Chiw
The HEXBOX is a foray into engineering a touch of ‘juicy’ physicality into gameplay, without the likes of expensive hardware. Consisting of an ecosystem of four DIY controllers + console, each HEXBOX controller is designed for different physical interactions (pinch, push, slide and twist), made possible with sturdy yet malleable low-cost materials like cardboard and elastic bands. When paired with computer vision and the motion-tracking capabilities of the ArUco markers, these interactions are seamlessly brought to life. Of the four hexagonal controllers—one is made of two units. As the housing console is designed to intentionally hold up to three controller units at any given time, so the player is required to strategise their next move.
Trim ‘N’ Trouble is HEXBOX’s accompanying flagship game about a wizard who simultaneously needs to trim his beard and defend himself against assorted enemies. To win the game, the wizard's beard has to be completely trimmed off. Its core gaming mechanic, best enjoyed with two players, focuses on the swapping of the HEXBOX controllers to accomplish incoming gaming objectives.
Yong Zhen Zhou, Feng Guozi, Claira Chiw
The HEXBOX is a foray into engineering a touch of ‘juicy’ physicality into gameplay, without the likes of expensive hardware. Consisting of an ecosystem of four DIY controllers + console, each HEXBOX controller is designed for different physical interactions (pinch, push, slide and twist), made possible with sturdy yet malleable low-cost materials like cardboard and elastic bands. When paired with computer vision and the motion-tracking capabilities of the ArUco markers, these interactions are seamlessly brought to life. Of the four hexagonal controllers—one is made of two units. As the housing console is designed to intentionally hold up to three controller units at any given time, so the player is required to strategise their next move.
Trim ‘N’ Trouble is HEXBOX’s accompanying flagship game about a wizard who simultaneously needs to trim his beard and defend himself against assorted enemies. To win the game, the wizard's beard has to be completely trimmed off. Its core gaming mechanic, best enjoyed with two players, focuses on the swapping of the HEXBOX controllers to accomplish incoming gaming objectives.
Stickibeats
Lim Shi Yun, Kianne Lim, Chua Zixin
Stickibeats is a DIY augmented reality toy kit for children to build and create their own musical beats. Stickibeats embraces computer vision at the core of its toy mechanics, and teaches children how cause and effect work. During the making process, children get to decide which markers they would like to stick on the toy wheel. Differently coloured stickers come with pre-loaded music samples. Using computer vision to detect movement, the detection of each unique ArUco markers is translated into musical beats. Stickibeats is an ensemble of hardware and software which makes up a system of tangible music composition through a cardboard interface. As children roll the musical toy across the room, the music visualisation on the phone shows how physical play is translated into musical beats.
Lim Shi Yun, Kianne Lim, Chua Zixin
Stickibeats is a DIY augmented reality toy kit for children to build and create their own musical beats. Stickibeats embraces computer vision at the core of its toy mechanics, and teaches children how cause and effect work. During the making process, children get to decide which markers they would like to stick on the toy wheel. Differently coloured stickers come with pre-loaded music samples. Using computer vision to detect movement, the detection of each unique ArUco markers is translated into musical beats. Stickibeats is an ensemble of hardware and software which makes up a system of tangible music composition through a cardboard interface. As children roll the musical toy across the room, the music visualisation on the phone shows how physical play is translated into musical beats.
TILT
Brandon Ong, Cherryl Thant, Tan Hwee Peng
TILT is a DIY cardboard game console that comes with three co-op games for kids, aged five years old and above, to interact and play with by tilting, flipping and lifting. These three physical interactions are captured using computer vision to detect ArUco markers. Parents without hands-on experience or prior knowledge in software can easily build this simple console, and children can bring it around to keep themselves entertained as long as a laptop is set up to run the browser game. The cardboard model was inspired by the form of a traditional seesaw, where it promotes the concept of collaboration and sharing between two children in the midst of completing tasks during gameplay. The tasks include sorting fruits, serving pancakes, collecting stars and avoiding obstacles.
Brandon Ong, Cherryl Thant, Tan Hwee Peng
TILT is a DIY cardboard game console that comes with three co-op games for kids, aged five years old and above, to interact and play with by tilting, flipping and lifting. These three physical interactions are captured using computer vision to detect ArUco markers. Parents without hands-on experience or prior knowledge in software can easily build this simple console, and children can bring it around to keep themselves entertained as long as a laptop is set up to run the browser game. The cardboard model was inspired by the form of a traditional seesaw, where it promotes the concept of collaboration and sharing between two children in the midst of completing tasks during gameplay. The tasks include sorting fruits, serving pancakes, collecting stars and avoiding obstacles.
TRACK
Pang Shi Jie, Yap Zi Ning, Elicia Low
Leveraging on a dependable means of tracking motion via computer vision, TRACK is an accessible and low-cost DIY tracking device for Occupational Therapy. It aims to assist the Occupational Therapists (OTs) in monitoring the elderly’s performance and progress while also improving the elderly’s rehabilitation experience. TRACK comprises a physical setup, a game interface, along with a data visualization dashboard for the OT. The physical setup is an add-on device to existing shoulder pulley which is complemented with “Coconut Climb”, a game we designed to engage and motivate the elderly to complete their exercises. Upon game completion, the OTs on site can view a summary of the elderly’s session to evaluate their progress. Additionally, every session’s data is saved on a separate database where the OTs can access to gain more insights about the elderly’s monthly performance, thus making more accurate prescriptions.
Pang Shi Jie, Yap Zi Ning, Elicia Low
Leveraging on a dependable means of tracking motion via computer vision, TRACK is an accessible and low-cost DIY tracking device for Occupational Therapy. It aims to assist the Occupational Therapists (OTs) in monitoring the elderly’s performance and progress while also improving the elderly’s rehabilitation experience. TRACK comprises a physical setup, a game interface, along with a data visualization dashboard for the OT. The physical setup is an add-on device to existing shoulder pulley which is complemented with “Coconut Climb”, a game we designed to engage and motivate the elderly to complete their exercises. Upon game completion, the OTs on site can view a summary of the elderly’s session to evaluate their progress. Additionally, every session’s data is saved on a separate database where the OTs can access to gain more insights about the elderly’s monthly performance, thus making more accurate prescriptions.