Author(s):
- Zilu Liang
- Mario Alberto Chapa Martell
Abstract:
Consumer sleep tracking devices are known to be inaccurate, but there is a lack of understanding of how user characteristics may affect the accuracy of these devices. This study aims to examine the effect of age, gender, subjective sleep quality, sleep hygiene and sleep structure on the accuracy of two consumer sleep trackers, i.e. Fitbit Charge 2 and Neuroon. Sleep data were collected from 27 healthy participants using consumer devices and a medical device concurrently. Analysis found that age, sleep hygiene and sleep structure were significantly associated to the accuracy of consumer sleep trackers, whereas no association was found on gender and subjective sleep quality. Both consumer devices had improved accuracy on total sleep time and sleep efficiency for participants who had longer, deeper and less interrupted sleep. Our findings suggest that consumer devices may not be suited for young adults and for people with short and fragmented sleep.
Documentation:
http://dx.doi.org/10.4108/eai.24-7-2018.159404
References:
[1] Piwek, L., Ellis, D. A., Andrews, S., and Joinson, A.
(2016) The rise of consumer health wearables: promises
and barriers. PLoS Med 13(2): e1001953
[2] Liang, Z., and Ploderer, B. (2016) Sleep tracking in the
real world: a qualitative study into barriers for improving
sleep. In Proceedings of the 28th Australian Conference on
Computer-Human Interaction, Launceston, Tasmania,
Australia, 537-541.
[3] Liang, Z., and Chapa-Martell, M. A. (2015) Framing
self-quantification for individual-level preventive health
care. In Proceedings of the International Joint Conference
on Biomedical Engineering Systems and Technologies,
Lisbon, Portugal, 336-343.
[4] Liang, Z., Ploderer, B, and Chapa-Martell, MA, “Is fitbit
fit for sleep-tracking? sources of measurement errors and
proposed countermeasures. In Proceedings of the 11th EAI
International Conference on Pervasive Computing
Technologies for Healthcare, Barcelona, Spain, 476-479.
[5] West, P, Van Kleek, M., Giordano, R., et al. (2017)
Information quality challenges of patient-generated data in
clinical practice. Front Public Health 5:284.
[6] De Zambotti, M., Goldstone, A., Claudatos, S., et al.
(2017) A validation study of Fitbit Charge 2 compared
with polysomnography in adults. Chronobiology
International 35(4): 465-476.
[7] Meltzer, L., Hiruma, L., Avis, K., et al. (2015) Comparison
of a commercial accelerometer with polysomnography and
actigraphy in children and adolescents. Sleep 38(8): 1323-
1330.
[8] De Zambotti, M., Baker, F.C., and Colrain, I.M. (2015)
Validation of sleep-tracking technology compared with
polysomnography in adolescents. Sleep 38(9):1461-1468.
[9] De Zambotti, M., Claudatos, S., Inkelis, S., et al. (2015)
Evaluation of a consumer fitness-tracking device to assess
sleep in adults. Chronobiology International 32(7): 1024-
1028.
[10] Montgomery-Downs, H., Insana, S., and Bond, J. (2012)
Movement toward a novel activity monitoring device.
Sleep and Breathing 16(3):913-917.
[11] Liang, Z., and Chapa-Martell, M.A. (2018) Validity of
consumer activity wristbands and wearable EEG for
measurement overall sleep parameters and sleep structure
in free-living conditions. Journal of Healthcare
Informatics Research 2(1-2):152-178.
[12] Marino, M., Li, Y., Rueschman, M., et al. (2013)
Measurement sleep accuracy, sensitivity, and specificity of
wrist actigraphy compared to polysomnography. Sleep
36(11):1747-1755.
[13] Blackwell, T., Ancoli-Israel, S., Redline, S., et al. (2011)
Factors that may influence the classification of sleep-wake
by wrist actigraphy: the MrOS sleep study. J Clin Sleep
Med 7(4): 357-367.
[14] Buysse, D., Reynolds, C., Monk, T., et al. (1989) The
Pittsburgh sleep quality index: a new instrument for
psychiatric practice and research. Psychiatry Res 28(2):
193-213.
[15] Natale, V., Leger, D., Martoni, M., et al. (2014) The role of
actigraphy in the assessment of primary insomnia: a
retrospective study. Sleep Medicine 15(1): 111-115.
[16] Martin, J., and Hakim, A. (2011) wrist actigraphy. CHEST
139(6): 1514-1527.
[17] Liang, Z., Ploderer, B., Chapa-Martell, M. A., and
Nishimura, T. (2016) A cloud-based intelligent computing
system for contextual exploration on personal sleep-
tracking data using association rule mining. Intelligent
Computing Systems. Communications in Computer and
Information Science, Springer, Cham.
[18] Liang, Z., Chapa-Martell, M. A., and Nishimura, T. (2016)
Mining hidden correlations between sleep and lifestyle
factors from quantified-self data. In Proceedings of the
2016 ACM International Joint Conference on Pervasive
and Ubiquitous Computing: Adjunct, Heidelberg,
Germany, 547-552.
[19] Liang, Z., Chapa-Martell, M. A., and Nishimura, T. (2016)
A personalized approach for detecting unusual sleep from
time series sleep-tracking data. In Proceedings of the IEEE
International Conference on Health Informatics (ICHI),
Chicago, US.
[20] Liang, Z., Ploderer, B., Liu, W., et al. (2016)
SleepExplorer: a visualization tool to make sense of
correlations between personal sleep data and contextual
factors. Personal Ubiquitous Comput 20(6): 985-1000.
[21] Weatherall, J., Paprocki, Y., Meyer, T. M., Kudel, I., and
Witt, E. A. (2018) Sleep tracking and exercise in patients
with type 2 diabetes mellitus (step-D): pilot study to
determine correlations between Fitbit data and patient-
reported outcomes. JMIR Mhealth Uhealth 6(6): e131.
[22] Haeuber, E., Shaughnessy, M., Forrester, L., Coleman, K.
L., and Macko, R. F. (2004) Accelerometer monitoring of
home- and community-based ambulatory activity after
stroke. Archives of Physical Medicine and Rehabilitation
85(12):1997-2001.
[23] Kanga, S.-G., Kanga, J. M., and Kob, K.-P.
(2017) Validity of a commercial wearable sleep tracker in
adult insomnia disorder patients and good sleepers.
Journal of Psychosomatic Research 97:38-44.
[24] Dickinson, D., Cazier, J., and Cech, T. (2016) A practical
validation study of a commercial accelerometer using good
and poor sleepers. Health Psychol Open 3(2): 1-10.
[25] Liang, Z., and Chapa-Martell, M. A. (2019) Accuracy of
Fitbit wristbands in measurement sleep stage transitions
and the effect of user-specific factors. JMIR Mhealth
Uhealth 7(6): e13384.
[26] Liang, Z., and Chapa-Martell, M. A. (2019) Combining
numerical and visual approaches in validating sleep data
Not All Errors Are Created Equal: Influence of User Characteristics on Measurement Errors of Consumer Wearable Devices for Sleep Tracking 9EAI Endorsed Transactions
on Pervasive Health and Technology
07 2018 | Volume 4 | Issue 15 | e4quality of consumer wearable wristbands. In Proceedings
of PerCom Workshops, Kyoto, Japan, 777-782.
[27] Liang, Z., and Chapa-Martell, M. A. (2019) Achieving
accurate ubiquitous sleep sensing with consumer wearable
activity wristbands using multi-class imbalanced
classification. In Proceedings of the 17th IEEE Intl Conf
Perv Intel Comp (PICOM 2019), Fukuoka, Japan.
[28] Liang, Z., and Chapa-Martell, M. A. (2019) Combining
resampling and machine learning to improve sleep-wake
detection of Fitbit wristbands. In Proceedings of the 7th
International Conference on Healthcare Informatics,
Xi’an, China.
[29] Bellone, G., Plano, S., Cardinali, D., Perez Chada, D.,
Vigo, D., and Golombek, D. (2016) Comparative analysis
of actigraphy performance in healthy young subjects. Sleep
Science 9:272-279.
[30] Lucey, B., McLeland, J., Toedebusch, C., Boyd, J., et al
(2016) Comparison of a single-channel EEG sleep study to
polysomnography. J Sleep Res 25(6):625-635.
[31] Ferguson, T., Rowlands, A. V., Olds, T., and Maher, C.
(2015) The validity of consumer-level, activity monitors in
healthy adults worn in free-living conditions: a cross-
sectional study. International Journal of Behavioral
Nutrition and Physical Activity 12(42): 1-9.
[32] O’Hara, E., Flanagan, D., Penzel, T., Garcia, C., Frohberg,
D., and Heneghan, C. (2015) A comparison of radio-
frequency biomotion sensors and actigraphy versus
polysonography for the assessment of sleep in normal
subjects. Sleep Breath 19:91-98.
[33] Taibi, D., Landis, C., and Vitiello, M. (2013) Concordance
of polysomnogaphic and actigraphic measurement of sleep
and wake in older women with insomnia. J Clin Sleep Med
9(3):217-225.
[34] Suh, S., Nowakowski, S., Bernet, R., et al. (2012) Clinical
significance of night-to-night sleep variability in insomnia.
Sleep Medicine 13(5): 469-475.
[35] Buysse, D., Cheng, Y., Germain, A., et al. (2010) Night-to-
night sleep variability in older adults with and without
chronic insomnia. Sleep Medicine 11(1): 56-64.
[36] Natale, V., Plazzi, G., and Martoni, M. (2009) Actigraphy
in the assessment of insomnia: a quantitative approach.
Sleep 32(6): 767-771.
[37] Buysse, DJ. (2014) Sleep Health: Can We Define It? Does
It Matter? Sleep 37(1): 9-17.
[38] Yoda, K., Inaba, M., Hamanoto, K., et al. (2015)
Association between poor Glycemic control, impaired
sleep quality, and increased arterial thickening in type 2
diabetic patents. Plus One 10(4): e0122521.
[39] Shrivastava, D., Jung, S., Saadat, M., et al. (2014) How to
interpret the results of a sleep study. J Community Hosp
Intern Med Perspect 4:24983.
[40] Fung, M., Peters, K., Ancoli-Israel, I., et al. (2013) Total
sleep time and other sleep characteristics measured by
actigraphy do not predict incident hypertension in a cohort
of community-dwelling older men. J Clin Sleep Med 9(6):
585-591.
[41] Harvey, A., Stinson, K., Whitaker, K., et al. (2008) The
subjective meaning of sleep quality: a comparison of
individuals with and without insomnia. Sleep 31(3): 383-
393.
[42] Matsuo, M., Masuda, F., Sumi, Y., et al. (2016)
Comparison of portable sleep monitors of different
modalities: potential as naturalistic sleep recorders. Front
Neurol 7:110.
[43] Yoshida, M., Shinohara, H., and Kodama, H. (2015)
Assessment of nocturnal sleep architecture by actigraphyand one-channel electroencephalography in early infancy,”
Early Human Development 91(9): 519-526.
[44] McCall, C., and McCall, V. (2012) Objective vs subjective
measurements of sleep in depressed insomniacs: first night
effect or reverse first night effect? J Clin Sleep Med 8(1):
59-65.
[45] Ahmadi, N., Shapiro, G., Chung, S., et al. (2009) Clinical
diagnosis of sleep apnea based on single night of
polysomnography vs two nights of polysomnography,”
Sleep and Breathing 13(3): 221-226.
[46] Tworoger, S., Davis, S., Vitiello, M., et al. (2005) Factors
associated with objective (actigraphic) and subjective sleep
quality in young adult women. Journal of Psychosomatic
Research 59(1): 11-19.
[47] Ancoli-Israel, I., Chesson, A., and Quan, S. (2017) for the
American Academy of Sleep Medicine. The AASM
manual for the scoring of sleep and associated events rules,
terminology and technical specifications. Darien, IL:
American Academy of Sleep Medicine, Version 2.4.
[48] Shcherbina, A., Mattsson, M., Waggott, D., et al. (2017)
Accuracy in wrist-worn, sensor-based measurements of
heart rate and energy expenditure in a diverse cohort. J
Pers Med 7(2): 3.
[49] Takacs, J., Pollock, C., Guenther, J., et al. (2014)
Validation of the Fitbit One activity monitor device during
treadmill walking. Journal of Science and Medicine in
Sport 17(5): 496-500.
[50] Meltzer, L., Walsh, C., Traylor, J., et al. (2012) Direct
comparison of two new actigraphs and polysomnography
in children and adolescents. Sleep 35(1):159-166.
[51] Werner, H., Molinari, L., Guyer, C., et al. (2008)
Agreement rates between actigraphy, diary, and
questionnaire for children’s sleep patterns. JAMA
Pediatrics 162(4): 350-358.
[52] Rosenberger, M., Buman, M., Haskell, W., et al. (2016)
Twenty-four hours of sleep, sedentary behavior, and
physical activity with nine wearable devices. Medicine &
Science in Sports & Exercise 48(3): 457-465.
[53] Cox, D. (1958) The regression analysis of binary
sequences. JSTOR 20(2): 215-242.
[54] Blackwell, T., Redline, S., Ancoli-Israel, I., et al. (2008)
Comparison of sleep parameters from actigraphy and
polysomnography in older women: the SOF study. Sleep
31(2):283-291.
[55] Renk, E., Collins, W., Rizzo, M., et al. (2005) Calibrating
a triaxial accelerometer-magnetometer. IEEE Control
Systems Magazine 25(6):86-95.
[56] Masurier, G.L. (2004) Pedometer sensitivity and
specificity. Medicine and Science in Sports and Exercise
36(2): 346.
[57] Roylance, L., and Angell, J. (1979) A batch-fabricated
silicon accelerometer. IEEE Transactions on Electron
Devices 26(12): 1911-1917.