This page contains info on the virtual buddy applications that I built through the course of my PhD work, which includes three iterations of the Virtual Fitness Buddy system, the Virtual STEM Buddy system, and the Virtual Buddy Fruit and Vegetable system. These systems encouraged children to engage in healthier exercise habits, to learn about various STEM concepts, and to engage in healthier fruit and vegetable consumption habits respectively. Each subsequent virtual buddy system built upon knowledge gleaned from all the previous virtual buddy systems.
Virtual Fitness Buddy at Home
Details
The Virtual Fitness Buddy Home (VFB-H)1 project was designed to encourage children to engage in healthier physical activity (PA) habits through the use of a virtual dog buddy. As a child completed PA, tracked using a Fitbit PA tracker, their VFB’s health would increase, which meant that they could play for longer, unlock new games and tricks, and earn points to spend on various customizations. VFB-H was built as an iPad app that children could use at home, utilizing touchscreen, voice-activation, and real-life motion as ways to interact with the system.
VFB-H Stats
| Name | Virtual Fitness Buddy At-Home (VFB-H) |
| Dates | Fall 2021 to Spring 2022 |
| Location | At-Home |
| Cohorts | 2 |
| Duration | 4.5 months each |
| Participants | 25 Treatment (44 Total) |
| Focus | Physical Activity |
| Controls | Motion, Touch, and Voice |
| Hardware | iPad |
| Publication | Dissertation Ch. 5 |
As part of this app, I designed six different minigames: agility, bark it, basketball, frisbee, soccer, and volleyball. See below for more details on these games. I implemented leaderboards that would track their top 3 personal scores in addition to the top 3 scores for their family and across the entire study for each minigame. In addition to these games, children were able to get their VFB to perform tricks using their voice and could play fetch without having to interact with a minigame.
VFB-H Minigames
- Type: Obstacle course
- Goal: Guide VFB through obstacles as quickly as possible
- Scoring: Time remaining on 60 second timer
- Controls: Virtual joystick (touchscreen)
- Aids: Guide arrow to next obstacle
Agility
- Type: Trick memory game
- Goal: Complete longest trick streak
- Scoring: Number of correct tricks performed
- Controls: Voice or touchscreen
- Aids: Guide arrow to stage
Bark It
- Type: Goal throws from random locations game
- Goal: Maximize basketballs scored in 60 seconds
- Scoring: Goals scored
- Controls: Virtual joystick and buttons (touchscreen) and iPad AR movement
- Aids: Guide arrow to next location and hoop
Basketball
- Type: Balloon popping game
- Goal: Maximize balloons popped in 60 seconds
- Scoring: Sum of balloon values plus VFB catch bonuses
- Controls: Virtual buttons (touchscreen) and iPad AR movement
- Aids: Guide arrow to current balloon
Frisbee
- Type: Goal kicks from random locations game
- Goal: Maximize soccer balls scored in 60 seconds
- Scoring: Goals scored
- Controls: Virtual joystick and buttons (touchscreen) and virtual paddle using iPad AR movement
- Aids: Guide arrow to next location and goal
Soccer
- Type: Serve streak game
- Goal: Perform longest serve streak in 60 seconds
- Scoring: Length of streak
- Controls: Virtual paddle using iPad AR movement
- Aids: Guide arrow to net
Volleyball
For the VFB-H app, I was responsible for the following:
Responsibilities
- Lead game designer and software engineer
- Designing and implementing:
- Minigames
- New park layout to allow for exploration
- Virtual kiosks to start/stop games, display instructions, and show leaderboards
- Speech-to-text (STT) for VFB tricks using Google’s STT API
- Arrow navigation system to show players where to go for their next objective
- Trophy system that dynamically changed size as players met higher and higher physical activity goals
- New iPad-based input system (touchscreen, AR motion, and voice)
- User interfaces (UI)
- Player control system
- App software architecture
- Adding sound effects to the park
- Add sense of realism to app
- Add feedback when completing tasks correctly
- Recording and editing video guide on how to use app
- Initializing iPads and Fitbits for player use
- Troubleshooting and bug fixes for app and Fitbit syncing
- Tech support and troubleshooting iPads and physical activity trackers
- Data analysis on gameplay metrics
Virtual STEM Buddy at Museum
Details
The Virtual STEM Buddy Museum (VSB-M)2 project was designed to encourage children to learn more about various STEM concepts through the use of a virtual robot buddy. As a child played the minigames, their VSB would provide instructions on how to play in addition to information on the underlying STEM concept being used, such as levers and trajectories. This info was conveyed both through text and audio as it could not be assumed that all of its players would be able to read. VSB-M was built as an exhibit at the Children’s Museum of Atlanta (CMOA) consisting of a large screen display, Android tablet, and Microsoft Kinect.
VSB-M Stats
| Name | Virtual STEM Buddy Museum (VSB-M) |
| Dates | January 2017 to May 2022 |
| Location | Children’s Museum of Atlanta (CMOA) |
| Sites | 1 |
| Cohorts | 1 |
| Duration | 5 years |
| Participants | Unknown – 28,122 recorded plays over 2 years |
| Focus | STEM Education |
| Controls | Motion and Touch |
| Hardware | TV, PC, Kinect, Android Tablet |
| Publications | Paper Dissertation Ch. 3 |
The tablet provided a familiar museum exhibit interface that was used for complex selection tasks, such as customizing their VSB. While the Kinect provided a more novel interface where players used their real world movements to interact with their VSB and play the minigames.
As part of this app, I designed two different minigames: Lever Hero and Slingshot. They were designed to target the concepts of lever balancing and trajectories respectively. See below for more details on these games.
VSB-M Minigames
- Type: Lever balancing game
- STEM Concept: Levers
- Goal: Balance levers as quickly and accurately as possible
- Scoring: Remaining time to solve problems
- Controls: Kinect grabbing gesture and movement
Lever Hero
- Type: Trajectory game
- STEM Concept: Trajectories
- Goal: Complete picture with minimal paint splats
- Scoring: Remaining time to solve problems
- Controls: Kinect grabbing gesture and movement
Slingshot
For the VSB-M app, I was responsible for the following:
Responsibilities
- Lead game designer and software engineer
- Designing and implementing:
- Minigames
- Overall game controls
- VSB interactivity and behavior
- New player skeletal avatar system
- Android tablet game selection system
- Bluetooth communication between tablet and pc
- User interfaces (UI)
- Player control system
- App software architecture
- Worked with CMOA staff to:
- Scale interactions to exhibit’s allocated physical space
- Design game mechanics that would fit well and safely into museum environment
- Observing and recording interactions on-site for bug fixes and data collection
- Tech support and troubleshooting kiosk
- Data Engineering:
- Data logging system and its data structures
- Generate codebook describing all variables
- Utilize ETL (extract, transform, load) processes to provide data in analyzable format
- Data analysis on gameplay metrics
Virtual Fitness Buddy Afterschool
Details
The Virtual Fitness Buddy Afterschool (VFB-A)3 project was designed to encourage children to engage in healthier physical activity (PA) habits through the use of a virtual dog buddy. As a child completed PA, tracked using a Fitbit PA tracker, their VFB’s health would increase, which meant that they could play for longer, unlock new games and tricks, and earn points to spend on various customizations. VFB-A was built as a mobile kiosk station to be used in afterschool programs consisting of a large screen display, touchscreen, and Microsoft Kinect.
VFB-A Stats
| Name | Virtual Fitness Buddy Afterschool (VFB-A) |
| Dates | Spring 2018 to Fall 2019 |
| Location | Afterschool Programs |
| Sites | 11 Treatment (21 Total) |
| Cohorts | 3 |
| Duration | 6 months each |
| Participants | 318 Treatment (464 Total) |
| Focus | Physical Activity |
| Controls | Motion, Touch, and Voice |
| Hardware | TV, PC, Kinect, Touchscreen |
| Publications | Paper Dissertation Ch. 4 |
As part of this app, I designed six different minigames: agility, bark it, basketball, frisbee, soccer, and volleyball. See below for more details on these games. In addition to these games, children were able to get their VFB to perform tricks using their voice and could play fetch without having to interact with a minigame. To achieve this, I implemented various midair gesture-based systems that players would use to perform the necessary actions for the minigames, tricks, and fetch.
VFB-A Minigames
- Type: Goal scoring game
- Goal: Maximize basketballs scored in 60 seconds
- Scoring: Goals scored
- Controls: Shot zone system and Kinect movement
Basketball
- Type: Balloon popping game
- Goal: Maximize balloons popped in 60 seconds
- Scoring: Sum of balloon values
- Controls: Kinect grabbing gesture and movement
Slingshot
- Type: Goal scoring game
- Goal: Maximize soccer balls scored in 60 seconds
- Scoring: Goals scored
- Controls: Virtual paddle attached to foot for kicking
Soccer
- Type: Serve streak game
- Goal: Perform longest serve streak in 60 seconds
- Scoring: Length of streak
- Controls: Virtual paddle attached to hand to return serve
Volleyball
For the VFB-A app, I was responsible for the following:
Responsibilities
- Lead game designer and software engineer
- Lead small team of artists and programmers
- Designing and implementing:
- Skeletal Avatar
- Touchscreen selection system
- Two throwing and trick gesture systems
- Leaderboards
- Control kiosk app
- User interfaces (UI)
- Player control system
- App software architecture
- Setting up and maintaining control and treatment kiosks
- Tech support and troubleshooting for both kiosks and physical activity trackers
- Data Engineering:
- Data logging system and its data structures
- Generate codebook describing all variables
- Utilize ETL (extract, transform, load) processes to provide data in analyzable format
- Data analysis on gameplay metrics
Virtual Fitness Buddy Summer Camp
Details
The Virtual Fitness Buddy Summer Camp (VFB-C)4 project was designed to encourage children to engage in healthier physical activity (PA) habits through the use of a virtual dog buddy. As a child completed PA goals, tracked using a Fitbit PA tracker, they would earn points to use to unlock new tricks and their VFB’s health would increase, meaning that they were slimmer and faster. Unlike the other VFB projects, children could only interact with their VFB by playing fetch and completing tricks without any minigames. VFB-C was built as a mobile kiosk station to be used in a week-long summer camp consisting of a large screen display and Microsoft Kinect. Lastly, the control group for this study also had their own VFB, but they did not earn points to spend.
VFB-C Stats
| Name | Virtual Fitness Buddy Camp (VFB-C) |
| Dates | Summer 2015 |
| Location | Summer Camp |
| Sites | 1 |
| Cohorts | 1 |
| Duration | 3 Days |
| Participants | 39 Treatment (67 Total) |
| Focus | Physical Activity |
| Controls | Motion and Voice |
| Hardware | TV, PC, Kinect |
| Publications | Paper Dissertation Ch. 2.2 |
For the VFB-C app, I was responsible for the following:
Responsibilities
- Designing and implementing:
- Updated user interface (UI)
- Hover selection technique for menu selection
- Trick shop
- Trick and Fitbit selection systems
- Observing and troubleshooting system throughout study
- Fitbit setup and troubleshooting
- Administering post-treatment questionnaires
Virtual Buddy Fruit and Vegetable Summer Camp
Details
The Virtual Buddy Fruit and Vegetable (VF&V)5 project was designed to encourage children to engage in healthier fruit and vegetable (F&V) habits through the use of a virtual dog buddy. As a child ate more fruits and vegetables, tracked by camp counselors, their VB’s health would increase, which meant that their heart would be easier to pump, their major artery would be more elastic, and they could complete fetching and tricks faster. As that child met their F&V goals, they would earn credits to spend on performing tricks. VB-F&V was built as a station to be used in a week-long summer camp consisting of a Mac and Novint Falcon (haptic joystick).
VB-F&V Stats
| Name | Virtual Buddy Fruit and Vegetable (VB-F&V) |
| Dates | Summer 2014 |
| Location | Summer Camp |
| Sites | 1 |
| Cohorts | 1 |
| Duration | 3 Days |
| Participants | 25 Treatment (68 Total) |
| Focus | Fruit and Vegetable Consumption |
| Controls | Haptic Joystick |
| Hardware | Mac, Novint Falcon |
| Publications | Paper Dissertation Ch. 2.1 |
As part of this app, I implemented two different VB health checks: arterial elasticity and heart pumping. They were designed to represent different aspects of a child’s VB. See below for more details on these health checks. In addition to these health checks, children were able to get their VFB to perform tricks and/or play fetch by spending trick credits earned through meeting their F&V goals.
VB-F&V Health Checks
- Type: Arterial elasticity check
- Goal: Determine the rigidity of a major artery
- Controls: Use haptic joystick to press on artery to see how much it depresses
Arterial Elasticity
- Type: Heart pumping check
- Goal: Determine how difficult is it to pump the heart
- Controls: Use haptic joystick to pump heart and feel resistance
Heart Pumping
For the VB-F&V app, I was responsible for the following:
Responsibilities
- Designing and implementing:
- VB health checks
- Haptic joystick input system
- Gesture system utilizing haptic joystick
- Reward system based on fruit and vegetable consumption
- RFID login system
- User interfaces (UI)
- Player control system
- Observing and troubleshooting system throughout study
- Administering post-treatment questionnaires
Footnotes
-
Ball, Catherine. Design and Field Implementation of Virtual Buddy-Based Serious Games for Children. Diss. University of Georgia, 2023. ↩
-
Ball, Catherine, Sun Joo Ahn, and Kyle Johnsen. “Design and field study of motion-based informal learning games for a children’s museum.” 2019 IEEE 5th workshop on everyday virtual reality (WEVR). IEEE, 2019. ↩
-
Ball, Catherine, et al. “Scaling the virtual fitness buddy ecosystem as a school-based physical activity intervention for children.” IEEE computer graphics and applications 42.1 (2021): 105-115. ↩
-
Ahn, Sun Joo, Kyle Johnsen, and Catherine Ball. “Points-based reward systems in gamification impact children’s physical activity strategies and psychological needs.” Health Education & Behavior 46.3 (2019): 417-425. ↩
-
Ahn, Sun Joo Grace, et al. “Using Virtual Pets to Increase Fruit and Vegetable Consumption in Children: A Technology-Assisted Social Cognitive Theory Approach.” Cyberpsychology, behavior and social networking 19.2 (2016): 86-92. ↩