TandemTrack: Shaping Consistent Exercise Experience by Complementing a Mobile App with a Smart Speaker

by | Apr 1, 2020 | Research Publications

Author(s):

  • Yuhan Luo
  • Bongshin Lee
  • Eun Kyoung Choe

Abstract:


Smart speakers such as Amazon Echo present promising opportunities for exploring voice interaction in the domain of in-home exercise tracking. In this work, we examine if and how voice interaction complements and augments a mobile app in promoting consistent exercise. We designed and developed TandemTrack, which combines a mobile app and an Alexa skill to support exercise regimen, data capture, feedback, and reminder. We then conducted a four-week between-subjects study deploying TandemTrack to 22 participants who were instructed to follow a short daily exercise regimen: one group used only the mobile app and the other group used both the app and the skill. We collected rich data on individuals’ exercise adherence and performance, and their use of voice and visual interactions, while examining how TandemTrack as a whole influenced their exercise experience. Reflecting on these data, we discuss the benefits and challenges of incorporating voice interaction to assist daily exercise, and implications for designing effective multimodal systems to support self-tracking and promote consistent exercise.

Documentation:

https://doi.org/10.1145/3313831.3376616

References:
  1. 30 Day Pushup Challenge. https://www.amazon.com/dp/B078Z5QMH7. Accessed: 2020-01-08.
  2. 5-Minute Plank Workout. https://www.amazon.com/cubic-ai-5-Minute-Plank-Workout/dp/B06XHTCB3Z. Accessed: 2020-01-08.
  3. 7 Min Fitness Challenge. http://7minworkoutapp.net/. Accessed: 2020-01-08.
  4. Alexa Skill Kit. https://developer.amazon.com/en-US/alexa/alexa-skills-kit. Accessed: 2020-01-08.
  5. Amazon Echo. https://www.amazon.com/echo. Accessed: 2020-01-08.
  6. Amazon Web Service (AWS) Lambda. https://aws.amazon.com/lambda/. Accessed: 2020-01-08.
  7. Centers for Disease Control and Prevention: Physical Activity Basics. https://www.cdc.gov/physicalactivity/basics/index.htm. Accessed: 2020-01-08.
  8. Fitbit Skill. https://www.amazon.com/dp/B01CH4BP28. Accessed: 2020-01-08.
  9. Gartner Predicts 75% of US Households will Have Smart Speakers by 2020. https://tinyurl.com/yfzt33jm. Accessed: 2020-01-08.
  10. Google Firebase. https://firebase.google.com/. Accessed: 2020-01-08.
  11. Google Home. https://store.google.com/gb/product/google_home. Accessed: 2020-01-08.
  12. HealthBot. https://healthbot.in/. Accessed: 2020-01-08.
  13. JEFIT. https://www.jefit.com/. Accessed: 2020-01-08.
  14. Keep Workout and Fitness Trainer. https://www.keepkeep.com/. Accessed: 2020-01-08.
  15. Kotlin. https://kotlinlang.org/. Accessed: 2020-01-08.
  16. Lark. https://www.lark.com/. Accessed: 2020-01-08.
  17. My fitnesspal Amazon Alexa Skill. https://www.amazon.com/gp/product/B07QN179C5. Accessed: 2020-01-08.
  18. Nike Training Club App. https://www.nike.com/us/en_us/c/nike-plus/training-app. Accessed: 2020-01-08.
  19. Node.js. https://nodejs.org/en/. Accessed: 2020-01-08.
  20. RunKeepper. https://runkeeper.com/running-app. Accessed: 2020-01-08.
  21. SWORKIT. https://sworkit.com/. Accessed: 2020-01-08.
  22. Janice M Brown, William R Miller, and Lauren A Lawendowski. 1999. The self-regulation questionnaire. (1999).
  23. Jennifer K Carroll, Anne Moorhead, Raymond Bond, William G LeBlanc, Robert J Petrella, and Kevin Fiscella. 2017. Who uses mobile phone health apps and does use matter? A secondary data analytics approach. Journal of medical Internet research 19, 4 (2017), e125. http://doi.org/10.2196/jmir.5604
  24. Eun Kyoung Choe, Bongshin Lee, and Seung-Won Hwang. 2018. Personal Data Exploration with Speech on Mobile Devices. In AVI 2018 Workshop on Multimodal Interactoin for Data Visualization.
  25. Arlene E Chung, Ashley C Griffin, Dasha Selezneva, and David Gotz. 2018. Health and Fitness Apps for Hands-Free Voice-Activated Assistants: Content Analysis. JMIR mHealth and uHealth 6, 9 (2018), e174. http://doi.org/10.2196/mhealth.9705
  26. David E Conroy, Chih-Hsiang Yang, and Jaclyn P Maher. 2014. Behavior change techniques in top-ranked mobile apps for physical activity. American journal of preventive medicine 46, 6 (2014), 649–652. http://doi.org/10.1016/j.amepre.2014.01.010
  27. Sunny Consolvo, Katherine Everitt, Ian Smith, and James A Landay. 2006. Design requirements for technologies that encourage physical activity. In Proceedings of the SIGCHI conference on Human Factors in computing systems. ACM, 457–466. http://doi.org/10.1145/1124772.1124840
  28. Stefania Druga, Randi Williams, Cynthia Breazeal, and Mitchel Resnick. 2017. Hey Google is it OK if I eat you?: Initial Explorations in Child-Agent Interaction. In Proceedings of the 2017 Conference on Interaction Design and Children. ACM, 595–600. http://doi.org/10.1145/3078072.3084330
  29. Brian J Fogg. 2009. A behavior model for persuasive design. In Proceedings of the 4th international Conference on Persuasive Technology. ACM, 40. http://doi.org/10.1145/1541948.1541999
  30. Macro Gutierrez, Ramon Moreno, and Marina Rebelo. 2017. Information and communication technologies and global health challenges. In Global Health Informatics. Elsevier, 50–93. http://doi.org/10.1016/B978-0–12–804591–6.00004–5
  31. Florence-Emilie Kinnafick, Cecilie Thøgersen-Ntoumani, and Joan L Duda. 2014. Physical activity adoption to adherence, lapse, and dropout: a self-determination theory perspective. Qualitative health research 24, 5 (2014), 706–718. http://doi.org/10.1177/1049732314528811
  32. Rafal Kocielnik, Lillian Xiao, Daniel Avrahami, and Gary Hsieh. 2018. Reflection Companion: A Conversational System for Engaging Users in Reflection on Physical Activity. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 2, 2 (2018), 70. http://doi.org/10.1145/3214273
  33. Eveliina E Korkiakangas, Maija A Alahuhta, and Jaana H Laitinen. 2009. Barriers to regular exercise among adults at high risk or diagnosed with type 2 diabetes: a systematic review. Health promotion international 24, 4 (2009), 416–427. http://doi.org/10.1093/heapro/dap031
  34. Paul Krebs and Dustin T Duncan. 2015. Health app use among US mobile phone owners: a national survey. JMIR mHealth and uHealth 3, 4 (2015), e101. http://doi.org/10.2196/mhealth.4924
  35. Phillippa Lally and Benjamin Gardner. 2013. Promoting habit formation. Health Psychology Review 7, sup1 (2013), S137–S158. http://doi.org/10.1080/17437199.2011.603640
  36. Minha Lee, Sander Ackermans, Nena van As, Hanwen Chang, Enzo Lucas, and Wijnand IJsselsteijn. 2019. Caring for Vincent: A Chatbot for Self-Compassion. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. http://doi.org/10.1145/3290605.3300932
  37. Leib Litman, Zohn Rosen, David Spierer, Sarah Weinberger-Litman, Akiva Goldschein, and Jonathan Robinson. 2015. Mobile exercise apps and increased leisure time exercise activity: a moderated mediation analysis of the role of self-efficacy and barriers. Journal of medical Internet research 17, 8 (2015), e195. http://doi.org/10.2196/jmir.4142
  38. Irene Lopatovska and Harriet Williams. 2018. Personification of the Amazon Alexa: BFF or a mindless companion. In Proceedings of the 2018 Conference on Human Information Interaction & Retrieval. ACM, 265–268. http://doi.org/10.1145/3176349.3176868
  39. Paula M Ludewig, Molly S Hoff, Erin E Osowski, Shane A Meschke, and Peter J Rundquist. 2004. Relative balance of serratus anterior and upper trapezius muscle activity during push-up exercises. The American journal of sports medicine 32, 2 (2004), 484–493. https://doi.org/10.1177/0363546503258911
  40. Elizabeth Jane Lyons and Claire Hatkevich. 2013. Prevalence of behavior changing strategies in fitness video games: theory-based content analysis. Journal of medical Internet research 15, 5 (2013), e81. http://doi.org/10.2196/jmir.2403
  41. Harm op den Akker, Valerie M Jones, and Hermie J Hermens. 2014. Tailoring real-time physical activity coaching systems: a literature survey and model. User modeling and user-adapted interaction 24, 5 (2014), 351–392. https://doi.org/10.1007/s11257-014–9146-y
  42. Rosemary Pimm, Corneel Vandelanotte, Ryan E Rhodes, Camille Short, Mitch J Duncan, and Amanda L Rebar. 2016. Cue consistency associated with physical activity automaticity and behavior. Behavioral Medicine 42, 4 (2016), 248–253. http://doi.org/10.1080/08964289.2015.1017549
  43. Markus Pohl. 325,000 mobile health apps available in 2017. https://research2guidance.com/325000-mobile-health-apps-available-in-2017/. Accessed: 2020-01-08.
  44. Martin Porcheron, Joel E Fischer, Stuart Reeves, and Sarah Sharples. 2018. Voice interfaces in everyday life. In proceedings of the 2018 CHI conference on human factors in computing systems. ACM, 640. http://doi.org/10.1145/3173574.3174214
  45. Alisha Pradhan, Kanika Mehta, and Leah Findlater. 2018. Accessibility came by accident: use of voice-controlled intelligent personal assistants by people with disabilities. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems. ACM, 459. http://doi.org/10.1145/3173574.3174033
  46. Andrew Prestwich, Macro Perugini, and Robert Hurling. 2009. Can the effects of implementation intentions on exercise be enhanced using text messages? Psychology and Health 24, 6 (2009), 677–687. http://doi.org/10.1080/08870440802040715
  47. Amanda Purington, Jessie G Taft, Shruti Sannon, Natalya N Bazarova, and Samuel Hardman Taylor. 2017. Alexa is my new BFF: social roles, user satisfaction, and personification of the amazon echo. In Proceedings of the 2017 CHI Conference Extended Abstracts on Human Factors in Computing Systems. ACM, 2853–2859. http://doi.org/10.1145/3027063.3053246
  48. IQVIA Institute Report. The Growing Value of Digital Health. https://www.iqvia.com/institute/reports/the-growing-value-of-digital-health. Accessed: 2020-01-08.
  49. B Ricci, M Marchetti, and F Figura. 1981. Biomechanics of sit-up exercises. Medicine and science in sports and exercise 13, 1 (1981), 54–59. http://www.ncbi.nlm.nih.gov/pubmed/6452564
  50. Ron Shor and Anat Shalev. 2014. Barriers to involvement in physical activities of persons with mental illness. Health promotion international 31, 1 (2014), 116–123. http://doi.org/10.1093/heapro/dau078
  51. Cindy HP Sit, John H Kerr, and Irene TF Wong. 2008. Motives for and barriers to physical activity participation in middle-aged Chinese women. Psychology of Sport and exercise 9, 3 (2008), 266–283. http://doi.org/10.1016/j.psychsport.2007.04.006
  52. Daniel Smolyak, Bongshin Lee, and Eun Kyoung Choe. 2018. TandemTrack: Promoting Consistent Exercise Leveraging Multimodal Training and Tracking. In Extended Abstracts of the 2018 CHI Conference on Human Factors in Computing Systems. ACM, LBW543. http://doi.org/10.1145/3170427.3188536
  53. Emmanuel Stamatakis, I-Min Lee, Jason Bennie, Jonathan Freeston, Mark Hamer, Gary O’Donovan, Ding Ding, Adrian Bauman, and Yorgi Mavros. 2017. Does strength-promoting exercise confer unique health benefits? A pooled analysis of data on 11 population cohorts with all-cause, cancer, and cardiovascular mortality endpoints. American journal of epidemiology 187, 5 (2017), 1102–1112. http://doi.org/10.1093/aje/kwx345
  54. Katarzyna Stawarz, Anna L Cox, and Ann Blandford. 2014. Don’t forget your pill!: designing effective medication reminder apps that support users’ daily routines. In Proceedings of the 32nd annual ACM conference on Human factors in computing systems. ACM, 2269–2278. http://doi.org/10.1145/2556288.2557079
  55. Thomas Stütz, Michael Domhardt, Gerlinde Emsenhuber, Daniela Huber, Martin Tiefengrabner, Nicholas Matis, and Simon Ginzinger. 2017a. An interactive 3D health app with multimodal information representation for frozen shoulder. In Proceedings of the 19th International Conference on Human-Computer Interaction with Mobile Devices and Services. ACM, 3. https://doi.org10.2196/10.1145/3098279.3098562
  56. Thomas Stütz, Gerlinde Emsenhuber, Daniela Huber, Michael Domhardt, Martin Tiefengrabner, Gertie Janneke Oostingh, Ulrike Fötschl, Nicholas Matis, and Simon Ginzinger. 2017b. Mobile Phone–Supported Physiotherapy for Frozen Shoulder: Feasibility Assessment Based on a Usability Study. JMIR rehabilitation and assistive technologies 4, 2 (2017), e6. https://doi.org10.2196/rehab.7085
  57. Lie Ming Tang and Judy Kay. 2017. Harnessing Long Term Physical Activity Data — How Long-term Trackers Use Data and How an Adherence-based Interface Supports New Insights. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 1, 2 (2017), 26. https://doi.org/10.1145/3090091
  58. Georgina Torbet. Smart speakers used primarily for music and information – and shopping. https://www.digitaltrends.com/home/smart-speakers-survey/. Accessed: 2020-01-08.
  59. Markku Turunen, Jaakko Hakulinen, Olov Ståhl, Björn Gambäck, Preben Hansen, Mari C Rodríguez Gancedo, Raúl Santos de La Cámara, Cameron Smith, Daniel Charlton, and Marc Cavazza. 2011. Multimodal and mobile conversational health and fitness companions. Computer Speech & Language 25, 2 (2011), 192–209. https://doi.org/10.1016/j.csl.2010.04.004
  60. Upkar Varshney. 2014. Mobile health: Four emerging themes of research. Decision Support Systems 66 (2014), 20–35. https://doi.org/10.1016/j.dss.2014.06.001
  61. Bas Verplanken and Sheina Orbell. 2003. Reflections on past behavior: a self-report index of habit strength. Journal of Applied Social Psychology 33, 6 (2003), 1313–1330. http://doi.org/10.1111/j.1559–1816.2003.tb01951.x
  62. Joshua H West, P Cougar Hall, Carl L Hanson, Michael D Barnes, Christophe Giraud-Carrier, and James Barrett. 2012. There’s an app for that: content analysis of paid health and fitness apps. Journal of medical Internet research 14, 3 (2012), e72. http://doi.org/10.2196/jmir.1977
  63. Justin Yang, Costas A Christophi, Andrea Farioli, Dorothee M Baur, Steven Moffatt, Terrell W Zollinger, and Stefanos N Kales. 2019. Association Between Push-up Exercise Capacity and Future Cardiovascular Events Among Active Adult Men. JAMA network open 2, 2 (2019), e188341–e188341. http://doi.org/10.1001/jamanetworkopen.2018.8341
  64. Ugan Yasavur, Christine Lisetti, and Naphtali Rishe. 2014. Let’s talk! speaking virtual counselor offers you a brief intervention. Journal on Multimodal User Interfaces 8, 4 (2014), 381–398. https://doi.org/10.1007/s12193-014-0169–9.