Assessing the Privacy of mHealth Apps for Self-Tracking: Heuristic Evaluation Approach

by | Oct 22, 2018 | Research Publications

Author(s):

  • Hutton, Luke
  • Price, Blaine A.
  • Kelly, Ryan
  • McCormick, Ciaran
  • Bandara, Arosha
  • Hatzakis, Tally
  • Meadows, Maureen
  • Nuseibeh, Bashar

Abstract:

Background: The recent proliferation of self-tracking technologies has allowed individuals to generate significant quantities of data about their lifestyle. These data can be used to support health interventions and monitor outcomes. However, these data are often stored and processed by vendors who have commercial motivations, and thus, they may not be treated with the sensitivity with which other medical data are treated. As sensors and apps that enable self-tracking continue to become more sophisticated, the privacy implications become more severe in turn. However, methods for systematically identifying privacy issues in such apps are currently lacking.

Objective: The objective of our study was to understand how current mass-market apps perform with respect to privacy. We did this by introducing a set of heuristics for evaluating privacy characteristics of self-tracking services.

Methods: Using our heuristics, we conducted an analysis of 64 popular self-tracking services to determine the extent to which the services satisfy various dimensions of privacy. We then used descriptive statistics and statistical models to explore whether any particular categories of an app perform better than others in terms of privacy.

Results: We found that the majority of services examined failed to provide users with full access to their own data, did not acquire sufficient consent for the use of the data, or inadequately extended controls over disclosures to third parties. Furthermore, the type of app, in terms of the category of data collected, was not a useful predictor of its privacy. However, we found that apps that collected health-related data (eg, exercise and weight) performed worse for privacy than those designed for other types of self-tracking.

Conclusions: Our study draws attention to the poor performance of current self-tracking technologies in terms of privacy, motivating the need for standards that can ensure that future self-tracking apps are stronger with respect to upholding users’ privacy. Our heuristic evaluation method supports the retrospective evaluation of privacy in self-tracking apps and can be used as a prescriptive framework to achieve privacy-by-design in future apps.

Documentation:

https://doi.org/10.2196/mhealth.9217

References:
  1. Choe E, Lee N, Lee B, Pratt W, Kientz J. Understanding quantified-selfers? practices in collecting and exploring personal data. New York: ACM Press; 2014 Presented at: CHI ’14 the SIGCHI Conference on Human Factors in Computing Systems; April 26 – May 01, 2014; Toronto, Canada p. 1143-1152. [CrossRef]
  2. Jones SL, Kelly R. Dealing With Information Overload in Multifaceted Personal Informatics Systems. Human–Computer Interaction 2017 May 16;33(1):1-48. [CrossRef]
  3. Rooksby J, Rost M, Morrison A, Chalmers M. Personal tracking as lived informatics. : ACM Press; 2014 Presented at: CHI ’14 the SIGCHI Conference on Human Factors in Computing Systems; April 26 – May 01, 2014; Toronto, Canada p. 1163-1172.
  4. Stone AA, Shiffman S, Schwartz JE, Broderick JE, Hufford MR. Patient compliance with paper and electronic diaries. Controlled Clinical Trials 2003 Apr;24(2):182-199. [CrossRef]
  5. Price BA, Kelly R, Mehta V, McCormick C, Ahmed H, Pearce O. Feel My Pain: Design and Evaluation of Painpad, a Tangible Device for Supporting Inpatient Self-Logging of Pain. New York: ACM Press; 2018 Presented at: CHI ’18 the 2018 CHI Conference on Human Factors in Computing Systems; April 21 – 26, 2018; Montreal, Canada p. 169.
  6. Shklovski I, Mainwaring S, Skúladóttir H, Borgthorsson H. Leakiness and creepiness in app space: perceptions of privacy and mobile app use. New York: ACM Press; 2014 Presented at: CHI ’14 the SIGCHI Conference on Human Factors in Computing Systems; April 26 – May 01, 2014; Toronto, Canada p. 2347-2356.
  7. Rosenfeld L, Torous J, Vahia IV. Data Security and Privacy in Apps for Dementia: An Analysis of Existing Privacy Policies. Am J Geriatr Psychiatry 2017 Aug;25(8):873-877. [CrossRef] [Medline]
  8. Dehling T, Gao F, Schneider S, Sunyaev A. Exploring the Far Side of Mobile Health: Information Security and Privacy of Mobile Health Apps on iOS and Android. JMIR Mhealth Uhealth 2015 Jan 19;3(1):e8 [FREE Full text] [CrossRef] [Medline]
  9. Paul G, Irvine J. Privacy Implications of Wearable Health Devices. New York: ACM Press; 2014 Presented at: SIN ’14 the 7th International Conference on Security of Information and Networks; September 09 – 11, 2014; Glascow, Scotland p. 117. [CrossRef]
  10. Drozdiak N, Nicas J. Google Privacy-Policy Change Faces New Scrutiny in EU. New York: The Wall Street Journal; 2017 Jan 24.   URL: https://www.wsj.com/articles/oracle-expresses-concern-to-eu-over-google-privacy-policy-1485263548 [accessed 2018-09-18]
  11. Thielman S. current edition: UK edition The Guardian – Back to home Support The Guardian Subscribe Find a job Dating Sign in / Register Search News Opinion Sport Culture Lifestyle Show More UK World Business Football UK politics Environment Education Society Science Tech Global development Cities Obituaries Hacking Yahoo hack: 1bn accounts compromised by biggest data breach in history. London: The Guardian; 2016 Dec 15.   URL: https:/​/www.​theguardian.com/​technology/​2016/​dec/​14/​yahoo-hack-security-of-one-billion-accounts-breached [accessed 2018-09-17] [WebCite Cache]
  12. Digital advertisers battle over online privacy. New York: The Economist; 2016 Nov 05.   URL: http:/​/www.​economist.com/​news/​business/​21709584-escalating-fight-over-users-data-and-targeted-ads-digital-advertisers-battle-over-online [accessed 2017-09-05]
  13. Hutton L, Henderson T. Beyond the EULA: Improving Consent for Data Mining. In: Cerquitelli T, Quercia D, Pasquale F, editors. Transparent Data Mining for Big and Small Data. Studies in Big Data. New York: Springer; 2017:147-167.
  14. Bellotti V, Sellen A. Design for privacy in ubiquitous computing environments. Dordrecht: Springer; 1993 Presented at: ECSCW’93 the Third European Conference on Computer-Supported Cooperative Work; 13–17 September 1993; Milan, Italy p. 77-92.
  15. Cavoukian A, Stoddart J, Dix A, Nemec I, Peep V, Shroff M. Resolution on Privacy by Design. 2010 Oct 27 Presented at: 32nd International Conference of Data Protection and Privacy Commissioners; 27-29 Oct 2010; Jerusalem, Israel   URL: https:/​/edps.​europa.eu/​sites/​edp/​files/​publication/​10-10-27_jerusalem_resolutionon_privacybydesign_en.​pdf
  16. Hoepman JH. Privacy Design Strategies. In: Cuppens-Boulahia N, Cuppens F, Jajodia S, Abou El Kalam A, Sans T, editors. ICT Systems Security and Privacy Protection. SEC 2014. IFIP Advances in Information and Communication Technology. Berlin: Springer; 2014:446-459.
  17. Clarke R. Privacy impact assessment: Its origins and development. Computer Law & Security Review 2009 Jan;25(2):123-135. [CrossRef]
  18. Oetzel MC, Spiekermann S. A systematic methodology for privacy impact assessments: a design science approach. Eur J Inf Syst 2013 Jul 23;23(2):126-150. [CrossRef]
  19. Wright D. Should privacy impact assessments be mandatory? Commun. ACM 2011 Aug 01;54(8):121-131. [CrossRef]
  20. European Union. Regulation (EU) 2016/679 of the European Parliament and of the Council of 27 April 2016 on the protection of natural persons with regard to the processing of personal data and on the free movement of such data, and repealing Directive 95/46/EC (General Data Protection Regulation). Off J Eur Union 2016 Apr 27;L119:1-88.
  21. Perera C, McCormick C, Bandara A, Price B, Nuseibeh B. Privacy-by-Design Framework for Assessing Internet of Things Applications and Platforms. New York: ACM Press; 2016 Presented at: 6th Int Conf on the Internet of Things; 7-9 Nov 2016; Stuttgart, Germany p. 83-92. [CrossRef]
  22. Jensen C, Potts C. Experimental Evaluation of a Lightweight Method for Augmenting Requirements Analysis. New York: ACM Press; 2007 Presented at: 1st ACM Int Workshop Empir Assess Softw Eng Lang Technol; 5 Nov 2007; Atlanta, Georgia p. 49-54. [CrossRef]
  23. Jamal A, Cole M. A Heuristic Evaluation of the Facebook’s Advertising Tool Beacon. 2009 Presented at: First Int Conf Inf Sci Eng Internet IEEE; 2009; Nanjing, China p. 1527-1530. [CrossRef]
  24. Nielsen J, Molich R. Heuristic evaluation of user interfaces. New York: ACM Press; 1990 Presented at: CHI ’90 the SIGCHI Conference on Human Factors in Computing Systems; 1-5 April 1990; Seattle, Washington. [CrossRef]
  25. Sutcliffe A, Gault B. Heuristic evaluation of virtual reality applications. Interacting with Computers 2004 Aug;16(4):831-849. [CrossRef]
  26. Furano RF, Kushniruk A, Barnett J. Deriving a Set of Privacy Specific Heuristics for the Assessment of PHRs (Personal Health Records). Stud Health Technol Inform 2017;234:125-130. [Medline]
  27. Hutton L. Quantified Self Privacy Heuristics v1. 2017 Oct 23.   URL: https://doi.org/10.21954/ou.rd.5514082 [accessed 2018-09-19]
  28. Pitofsky R, Anthony S, Thompson M, Swindle O, Leary T. FTC Reports. 2000 May. Privacy Online: Fair Information Practices in the Electronic Marketplace: A Report to Congress   URL: https:/​/www.​ftc.gov/​reports/​privacy-online-fair-information-practices-electronic-marketplace-federal-trade-commission [accessed 2018-09-19] [WebCite Cache]
  29. Paul G, Irvine J. Privacy Implications of Wearable Health Devices. In: ACM Press. New York: ACM Press; 2014 Presented at: SIN ’14 the 7th International Conference on Security of Information and Networks; 9-11 Sept 2014; Glascow, Scotland.
  30. Tankard C. What the GDPR means for businesses. Network Security 2016 Jun;2016(6):5-8. [CrossRef]
  31. Inostroza R, Rusu C, Roncagliolo S, Rusu V. Usability Heuristics for Touchscreen-based Mobile Devices: Update. New York: ACM Press; 2013 Presented at: ChileCHI ’13 the 2013 Chilean Conference on Human – Computer Interaction; 11-15 Nov 2013; Temuco, Chile p. 24-29. [CrossRef]
  32. IDC. Worldwide Smartphone OS Market Share. 2017.   URL: https://www.idc.com/promo/smartphone-market-share/os [accessed 2018-09-19] [WebCite Cache]
  33. Mohamed I, Patel D. Android vs iOS Security: A Comparative Study. New York: IEEE; 2015 Presented at: 12th International Conference on Information Technology – New Generations; 13-15 April 2015; Las Vegas p. A. [CrossRef]
  34. London Quantified Self Meetup Group 2014 Survey Data Internet. 2014. 2014   URL: https://doi.org/10.21954/ou.rd.5505805
  35. Wearables unit shipments worldwide by vendor from 1Q’14 to 2Q’18 (in millions).: Statista; 2018.   URL: https://www.statista.com/statistics/435933/quarterly-wearables-shipments-worldwide-by-vendor/ [accessed 2009-09-20]
  36. McCormick C. Google Play Store Scrapper. 2017.   URL: https://bitbucket.org/ou-rse/playstorescraper/ [accessed 2018-09-20]
  37. Popular Apps Evaluated. 2017 Oct 18.   URL: https://figshare.com/articles/Popular_Apps_Evaluated/5507146 [accessed 2018-09-20] [WebCite Cache]
  38. Android Distribution Dashboard. 2018.   URL: https://developer.android.com/about/dashboards/ [accessed 2018-09-20] [WebCite Cache]
  39. Cohen J. A Coefficient of Agreement for Nominal Scales. Educational and Psychological Measurement 1960 Apr 01;20(1):37-46. [CrossRef]
  40. Agresti A. Categorical Data Analysis, 3rd Edition. London: Wiley; Jan 2013.
  41. Racherla P, Furner C, Babb J. Conceptualizing the Implications of Mobile App Usage and Stickiness: A Research Agenda. SSRN Journal 2012 Dec 9 [FREE Full text] [CrossRef]
  42. Luger E, Moran S, Rodden T. Consent for all: revealing the hidden complexity of terms and conditions. New York: ACM Press; 2013 Presented at: CHI ’13 the SIGCHI Conference on Human Factors in Computing Systems; April 27 – May 02, 2013; Paris, France p. 2687-2696. [CrossRef]
  43. Obar JA, Oeldorf-Hirsch A. The Biggest Lie on the Internet: Ignoring the Privacy Policies and Terms of Service Policies of Social Networking Services. 2018 Aug 18 Presented at: TPRC 44: The 44th Research Conference on Communication, Information and Internet Policy; 30 Sep – 1 Oct 2016; Arlington, VA p. 1-20   URL: https://papers.ssrn.com/sol3/papers.cfm?abstract_id=2757465 [CrossRef]
  44. Under Armour Inc. Privacy Shield Certification. 2017 Feb 09.   URL: https://www.privacyshield.gov/participant?id=a2zt0000000TNOuAAO [accessed 2018-09-20]
  45. Ayalon O, Toch E. Retrospective Privacy: Managing Longitudinal Privacy in Online Social Networks. New York: ACM Press; 2013 Presented at: SOUPS ’13 the Ninth Symposium on Usable Privacy and Security; 24-26 July 2013; Newcastle, UK p. 4. [CrossRef]
  46. Hutton L, Henderson T. I didn’t sign up for this!: Informed consent in social network research. Palo Alto, CA: The AAAI Press; 2015 Presented at: the 9th International AAAI Conference on Web and Social Media (ICWSM); 26-29 May 2015; Oxford, UK p. 178-187   URL: http://www.aaai.org/ocs/index.php/ICWSM/ICWSM15/paper/view/10493
  47. Phua J, Jin SV, Kim JJ. Uses and gratifications of social networking sites for bridging and bonding social capital: A comparison of Facebook, Twitter, Instagram, and Snapchat. Computers in Human Behavior 2017 Jul;72:115-122. [CrossRef]
  48. Kamal N, Fels S, Ho K. Online Social Networks for Personal Informatics to Promote Positive Health Behavior. New York: ACM Press; 2010 Presented at: WSM ’10 the second ACM SIGMM workshop on Social media; 25 Oct 2010; Firenze, Italy p. 47-52. [CrossRef]
  49. Google Play Store: Mint: Budget, Bills, Finance.: www.mint.com   URL: https://play.google.com/store/apps/details?id=com.mint [accessed 2018-09-20] [WebCite Cache]
  50. Martin K, Nissenbaum H. Measuring privacy: an empirical test using context to expose confounding variables. Columbia Science and Technology Law Review 2016;18:176.
  51. Nissenbaum H. Privacy in context: technology, policy, and the integrity of social life. Stanford, CA: Stanford University Press; 2010.
  52. Slavin R, Wang X, Hosseini M, Hester J, Krishnan R, Bhatia J, et al. Toward a framework for detecting privacy policy violations in android application code. New York: ACM Press; 2016 Presented at: ICSE ’16 the 38th International Conference on Software Engineering; May 14 – 22, 2016; Austin, TX p. 25-36.