Author(s):
- Ricardo Ribeiro
- Alina Trifan
- António J R Neves
Abstract:
Background: Over the past decade, the wide availability and small size of different types of sensors, together with the decrease in pricing, have allowed the acquisition of a substantial amount of data about a person’s life in real time. These sensors can be incorporated into personal electronic devices available at a reasonable cost, such as smartphones and small wearable devices. They allow the acquisition of images, audio, location, physical activity, and physiological signals among other data. With these data, usually denoted as lifelog data, we can then analyze and understand personal experiences and behaviors. This process is called lifelogging.
Objective: The objective of this paper was to present a narrative review of the existing literature about lifelogging over the past decade. To achieve this goal, we analyzed lifelogging applications used to retrieve relevant information from daily digital data, some of them with the purpose of monitoring and assisting people with memory issues and others designed for memory augmentation. We aimed for this review to be used by researchers to obtain a broad idea of the type of data used, methodologies, and applications available in this research field.
Methods: We followed a narrative review methodology to conduct a comprehensive search for relevant publications in Google Scholar and Scopus databases using lifelog topic-related keywords. A total of 411 publications were retrieved and screened. Of these 411 publications, 114 (27.7%) publications were fully reviewed. In addition, 30 publications were manually included based on our bibliographical knowledge of this research field.
Results: From the 144 reviewed publications, a total of 113 (78.5%) were selected and included in this narrative review based on content analysis. The findings of this narrative review suggest that lifelogs are prone to become powerful tools to retrieve memories or increase knowledge about an individual’s experiences or behaviors. Several computational tools are already available for a considerable range of applications. These tools use multimodal data of different natures, with visual lifelogs being one of the most used and rich sources of information. Different approaches and algorithms to process these data are currently in use, as this review will unravel. Moreover, we identified several open questions and possible lines of investigation in lifelogging.
Conclusions: The use of personal lifelogs can be beneficial to improve the quality of our life, as they can serve as tools for memory augmentation or for providing support to people with memory issues. Through the acquisition and analysis of lifelog data, lifelogging systems can create digital memories that can be potentially used as surrogate memory. Through this narrative review, we understand that contextual information can be extracted from lifelogs, which provides an understanding of the daily life of a person based on events, experiences, and behaviors.
Documentation:
https://doi.org/10.2196/30517
References:
Dodge M, Kitchin R. ‘Outlines of a world coming into existence’: pervasive computing and the ethics of forgetting. Environ Plann B Plann Des. 2016 Jul 22;34(3):431–45. doi: 10.1068/b32041t. [CrossRef] [Google Scholar]
2. Gurrin C, Smeaton AF, Doherty AR. LifeLogging: personal big data. FNT Inf Retrieval. 2014;8(1):1–125. doi: 10.1561/1500000033. [CrossRef] [Google Scholar]
3. Bush V. As we may think. Interactions. 1996 Mar;3(2):35–46. doi: 10.1145/227181.227186. [CrossRef] [Google Scholar]
4. Mann S. Wearable computing: a first step toward personal imaging. Computer. 1997;30(2):25–32. doi: 10.1109/2.566147. [CrossRef] [Google Scholar]
5. Mann S. ‘WearCam’ (The wearable camera): personal imaging systems for long-term use in wearable tetherless computer-mediated reality and personal photo/videographic memory prosthesis. Proceedings of the Digest of Papers. Second International Symposium on Wearable Computers (Cat. No.98EX215); Digest of Papers. Second International Symposium on Wearable Computers (Cat. No.98EX215); Oct 19-20, 1998; Pittsburgh, PA, USA. 1998. [CrossRef] [Google Scholar]
6. Mann S. Continuous lifelong capture of personal experience with EyeTap. Proceedings of the the 1st ACM workshop on Continuous archival and retrieval of personal experiences; MM04: 2004 12th Annual ACM International Conference on Multimedia; Oct 15, 2004; New York, USA. 2004. [CrossRef] [Google Scholar]
7. Mann S, Fung J, Aimone C, Sehgal A, Chen D. Designing EyeTap digital eyeglasses for continuous lifelong capture and sharing of personal experiences. CHI 2005; April 2-7, 2005; Portland, Oregon. 2005. [Google Scholar]
8. Mann S, Huang J, Janzen R, Lo R, Rampersad V, Chen A, Doha T. Blind navigation with a wearable range camera and vibrotactile helmet. Proceedings of the 19th ACM international conference on Multimedia; MM ’11: ACM Multimedia Conference; Nov 28 -Dec 1, 2011; Scottsdale Arizona USA. 2011. [CrossRef] [Google Scholar]
9. Mann S, Lo R, Ovtcharov K, Gu S, Dai D, Ngan C, Ai T. Realtime HDR (High Dynamic Range) video for eyetap wearable computers, FPGA-based seeing aids, and glasseyes (EyeTaps). Proceedings of the 2012 25th IEEE Canadian Conference on Electrical and Computer Engineering (CCECE); 2012 25th IEEE Canadian Conference on Electrical and Computer Engineering (CCECE); Apr 29- May 2, 2012; Montreal, QC, Canada. 2012. [CrossRef] [Google Scholar]
10. Mann S, Ali M, Lo R, Wu H. FreeGlass for developers, “haccessibility”, and Digital Eye Glass + Lifeglogging research in a (sur/sous)veillance society. Proceedings of the International Conference on Information Society (i-Society 2013); International Conference on Information Society (i-Society 2013); Jun 24-26, 2013; Toronto, ON, Canada. 2013. [Google Scholar]
11. Mann S, Mann C, Lam D, Mathewson K, Stairs J, Pierce C, Hernandez J, Kanaan G, Piette L, Khokhar H. The human eye as a camera. Proceedings of the 2019 IEEE International Conference on E-health Networking, Application & Services (HealthCom); 2019 IEEE International Conference on E-health Networking, Application & Services (HealthCom); Oct 14-16, 2019; Bogota, Colombia. 2019. [CrossRef] [Google Scholar]
12. Gemmell J, Bell G, Lueder R, Drucker S, Wong C. Mylifebits: fulfilling the memex vision. Proceedings of the tenth ACM international conference on Multimedia; MM02: ACMM02: ACM Multimedia 2002M Multimedia 2002; Dec 1 – 6, 2002; Juan-les-Pins France. 2002. [CrossRef] [Google Scholar]
13. Gemmell J, Lueder R, Bell G. The mylifebits lifetime store. Proceedings of the 2003 ACM SIGMM workshop on Experiential telepresence; ETP ’03: Proceedings of the 2003 ACM SIGMM workshop on Experiential telepresence; Nov 7, 2003; Berkeley California. 2003. [CrossRef] [Google Scholar]
14. Gemmell J, Bell G, Lueder R. MyLifeBits. Commun ACM. 2006 Jan;49(1):88–95. doi: 10.1145/1107458.1107460. [CrossRef] [Google Scholar]
15. Bell G, Gemmell J. A digital life. Sci Am. 2007 Mar;296(3):58–65. doi: 10.1038/scientificamerican0307-58. [PubMed] [CrossRef] [Google Scholar]
16. Bell G, Gemmell J. Total Recall: How the E-Memory Revolution Will Change Everything. New York, NY: Dutton Penguin; 2009. [Google Scholar]
17. Hodges S, Williams L, Berry E, Izadi S, Srinivasan J, Butler A, Smyth G, Kapur N, Wood K. SenseCam: a retrospective memory aid. In: Dourish P, Friday A, editors. UbiComp 2006: Ubiquitous Computing. UbiComp 2006. Lecture Notes in Computer Science, vol 4206. Berlin, Heidelberg: Springer; 2006. pp. 177–193. [Google Scholar]
18. Hodges S, Berry E, Wood K. SenseCam: a wearable camera that stimulates and rehabilitates autobiographical memory. Memory. 2011 Oct;19(7):685–96. doi: 10.1080/09658211.2011.605591. [PubMed] [CrossRef] [Google Scholar]
19. Chen M, Mao S, Liu Y. Big data: a survey. Mobile Netw Appl. 2014 Jan 22;19(2):171–209. doi: 10.1007/s11036-013-0489-0. [CrossRef] [Google Scholar]
20. Gurrin C, Smeaton AF, Byrne D, O’Hare N, Jones GJF, O’Connor N. An examination of a large visual lifelog. In: Li H, Liu T, Ma WY, Sakai T, Wong KF, Zhou G, editors. Information Retrieval Technology. AIRS 2008: Lecture Notes in Computer Science, vol 4993. Berlin, Heidelberg: Springer; 2008. pp. 537–542. [Google Scholar]
21. Harvey M, Langheinrich M, Ward G. Remembering through lifelogging: a survey of human memory augmentation. Pervasive Mobile Comput. 2016 Apr;27:14–26. doi: 10.1016/j.pmcj.2015.12.002. [CrossRef] [Google Scholar]
22. World Population Ageing 2017 Highlights. Geneva: United Nations; 2019. [Google Scholar]
23. Global Action Plan on the Public Health Response to Dementia 2017 – 2025. Geneva: World Health Organization; 2017. [Google Scholar]
24. Piasek P, Irving K, Smeaton AF. Case study in SenseCam use as an intervention technology for early-stage dementia. Int J Comput Healthcare. 2012;1(4):304. doi: 10.1504/ijcih.2012.051798. [CrossRef] [Google Scholar]
25. Ferrari R. Writing narrative style literature reviews. Med Writing. 2015 Dec 23;24(4):230–5. doi: 10.1179/2047480615z.000000000329. [CrossRef] [Google Scholar]
26. Greenhalgh T, Thorne S, Malterud K. Time to challenge the spurious hierarchy of systematic over narrative reviews? Eur J Clin Invest. 2018 Jun 16;48(6):e12931. doi: 10.1111/eci.12931. http://europepmc.org/abstract/MED/29578574 . [PMC free article] [PubMed] [CrossRef] [Google Scholar]
27. Page M, McKenzie J, Bossuyt P, Boutron I, Hoffmann T, Mulrow C, Shamseer L, Tetzlaff JM, Moher D. Updating guidance for reporting systematic reviews: development of the PRISMA 2020 statement. J Clin Epidemiol. 2021 doi: 10.31222/osf.io/jb4dx. (forthcoming) [PubMed] [CrossRef] [Google Scholar]
28. Doherty AR, Smeaton AF. Automatically segmenting lifelog data into events. Proceedings of the 2008 Ninth International Workshop on Image Analysis for Multimedia Interactive Services; 2008 Ninth International Workshop on Image Analysis for Multimedia Interactive Services; May 7-9, 2008; Klagenfurt, Austria. 2008. [CrossRef] [Google Scholar]
29. Doherty AR, Caprani N, Conaire CO, Kalnikaite V, Gurrin C, Smeaton AF, O’Connor NE. Passively recognising human activities through lifelogging. Comput Human Behav. 2011 Sep;27(5):1948–58. doi: 10.1016/j.chb.2011.05.002. [CrossRef] [Google Scholar]
30. Gurrin C, Qiu Z, Hughes M, Caprani N, Doherty AR, Hodges SE, Smeaton AF. The smartphone as a platform for wearable cameras in health research. Am J Prev Med. 2013 Mar;44(3):308–13. doi: 10.1016/j.amepre.2012.11.010.S0749-3797(12)00869-0 [PubMed] [CrossRef] [Google Scholar]
31. Pauly-Takacs K, Moulin CJ, Estlin EJ. SenseCam as a rehabilitation tool in a child with anterograde amnesia. Memory. 2011 Oct;19(7):705–12. doi: 10.1080/09658211.2010.494046.924717534 [PubMed] [CrossRef] [Google Scholar]
32. Wang P, Smeaton AF. Using visual lifelogs to automatically characterize everyday activities. Inf Sci. 2013 May;230:147–61. doi: 10.1016/j.ins.2012.12.028. [CrossRef] [Google Scholar]
33. Wang P, Sun L, Yang S, F. Smeaton A, Gurrin C. Characterizing everyday activities from visual lifelogs based on enhancing concept representation. Comput Vision Image Understanding. 2016 Jul;148:181–92. doi: 10.1016/j.cviu.2015.09.014. [CrossRef] [Google Scholar]
34. Song S, Chandrasekhar V, Cheung N, Narayan S, Li L, Lim J. Computer Vision – ACCV 2014 Workshops. Cham: Springer; 2015. Activity recognition in egocentric life-logging videos. [Google Scholar]
35. Li N, Crane M, Ruskin HJ, Gurrin C. Application of statistical physics for the identification of important events in visual lifelogs. Proceedings of the 2013 IEEE International Conference on Bioinformatics and Biomedicine; 2013 IEEE International Conference on Bioinformatics and Biomedicine; Dec 18-21, 2013; Shanghai, China. 2013. [CrossRef] [Google Scholar]
36. Balonas M, Mestre B, Talavera E, Giró-i-Nieto X, Radeva P. Visual summary of egocentric photostreams by representative keyframes. Proceedings of the 2015 IEEE International Conference on Multimedia & Expo Workshops (ICMEW); 2015 IEEE International Conference on Multimedia & Expo Workshops (ICMEW); Jun 29- Jul 3, 2015; Turin, Italy. 2015. [CrossRef] [Google Scholar]
37. Talavera E, Bolanos M, Dimiccoli M, Aghaei M, Radeva P. R-clustering for egocentric video segmentation. Pattern Recognition and Image Analysis; 7th Iberian Conference, IbPRIA 2015; June 17-19, 2015; Santiago de Compostela, Spain. Cham: Springer; 2015. [CrossRef] [Google Scholar]
38. Dimiccoli M, Bolaños M, Talavera E, Aghaei M, Nikolov SG, Radeva P. SR-clustering: semantic regularized clustering for egocentric photo streams segmentation. Comput Vision Image Understanding. 2017 Feb;155:55–69. doi: 10.1016/j.cviu.2016.10.005. [CrossRef] [Google Scholar]
39. Gupta R, Gurrin C. MultiMedia Modeling. MMM 2018. Lecture Notes in Computer Science, vol 10704. Cham: Springer; 2018. Approaches for event segmentation of visual lifelog data. [Google Scholar]
40. Fan C, Zhang Z, Crandall DJ. Deepdiary: lifelogging image captioning and summarization. J Visual Commun Image Representation. 2018 Aug;55:40–55. doi: 10.1016/j.jvcir.2018.05.008. [CrossRef] [Google Scholar]
41. Garcia del Molino A, Lim J-H, Tan A-H. Predicting visual context for unsupervised event segmentation in continuous photostreams. Proceedings of the 26th ACM international conference on Multimedia; MM ’18: ACM Multimedia Conference; Oct 22 – 26, 2018; Seoul Republic of Korea. 2018. [CrossRef] [Google Scholar]
42. Furnari A, Battiato S, Farinella GM. Personal-location-based temporal segmentation of egocentric videos for lifelogging applications. J Visual Commun Image Representation. 2018 Apr;52:1–12. doi: 10.1016/j.jvcir.2018.01.019. [CrossRef] [Google Scholar]
43. Oliveira-Barra G, Bolanos M, Talavera E, Gelonch O, Garolera M, Radeva P. Lifelog retrieval for memory stimulation of people with memory impairment. In: Alameda-Pineda X, Ricci E, Sebe N, editors. Multimodal Behavior Analysis in the Wild: Advances and Challenges. Cambridge, Massachusetts: Academic Press; 2019. pp. 135–158. [Google Scholar]
44. Ellis D, Lee K. Minimal-impact audio-based personal archives. Proceedings of the the 1st ACM workshop on Continuous archival and retrieval of personal experiences; MM04: 2004 12th Annual ACM International Conference on Multimedia; Oct 15, 2004; New York, USA. 2004. [CrossRef] [Google Scholar]
45. Shaikh MA, Molla MK, Hirose K. Automatic life-logging: a novel approach to sense real-world activities by environmental sound cues and common sense. Proceedings of the 2008 11th International Conference on Computer and Information Technology; 2008 11th International Conference on Computer and Information Technology; Dec 24-27, 2008; Khulna, Bangladesh. 2008. [CrossRef] [Google Scholar]
46. Shah M, Mears B, Chakraborty C, Spanias A. Lifelogging: archival and retrieval of continuously recorded audio using wearable devices. Proceedings of the 2012 IEEE International Conference on Emerging Signal Processing Applications; 2012 IEEE International Conference on Emerging Signal Processing Applications; Jan 12-14, 2012; Las Vegas, NV, USA. 2012. [CrossRef] [Google Scholar]
47. Yamano K, Itou K. Browsing audio lifelog data using acoustic and location information. Proceedings of the 2009 Third International Conference on Mobile Ubiquitous Computing, Systems, Services and Technologies; 2009 Third International Conference on Mobile Ubiquitous Computing, Systems, Services and Technologies; Oct 11-16, 2009; Sliema, Malta. 2009. [CrossRef] [Google Scholar]
48. Ziaei A, Sangwan A, Hansen JH. Prof-life-log: Personal interaction analysis for naturalistic audio streams. Proceedings of the 2013 IEEE International Conference on Acoustics, Speech and Signal Processing; 2013 IEEE International Conference on Acoustics, Speech and Signal Processing; May 26-31, 2013; Vancouver, BC, Canada. 2013. [CrossRef] [Google Scholar]
49. Li D, Gu Y, Kamijo S. Smartphone based lifelog with meaningful place detection. Proceedings of the 2018 IEEE International Conference on Consumer Electronics (ICCE); 2018 IEEE International Conference on Consumer Electronics (ICCE); Jan 12-14, 2018; Las Vegas, NV, USA. 2018. [CrossRef] [Google Scholar]
50. Tanaka G, Okada M, Mineno H. GPS-based daily context recognition for lifelog generation using smartphone. Int J Advanced Comput Sci App. 2015;6(2):104–12. doi: 10.14569/ijacsa.2015.060216. [CrossRef] [Google Scholar]
51. Aizawa K, Ishijima K, Shiina M. Summarizing wearable video. Proceedings 2001 International Conference on Image Processing (Cat. No.01CH37205); Proceedings 2001 International Conference on Image Processing (Cat. No.01CH37205); Oct 7-10, 2001; Thessaloniki, Greece. 2001. [CrossRef] [Google Scholar]
52. Hori T, Aizawa K. Context-based video retrieval system for the life-log applications. Proceedings of the 5th ACM SIGMM international workshop on Multimedia information retrieval; Proceedings of the 5th ACM SIGMM international workshop on Multimedia information retrieval; Nov 7, 2003; Berkeley California. 2003. [CrossRef] [Google Scholar]
53. Datchakorn T, Toshihiko Y, Kiyoharu A. Practical experience recording and indexing of life log video. Proceedings of the 2nd ACM workshop on Continuous archival and retrieval of personal experiences; ICMI05: Seventh International Conference on Multimodal Interfaces 2005; Nov 11, 2005; Hilton Singapore. 2005. [CrossRef] [Google Scholar]
54. Doherty AR, Kelly P, Kerr J, Marshall S, Oliver M, Badland H, Hamilton A, Foster C. Using wearable cameras to categorise type and context of accelerometer-identified episodes of physical activity. Int J Behav Nutr Phys Act. 2013 Feb 13;10(1):22. doi: 10.1186/1479-5868-10-22. https://ijbnpa.biomedcentral.com/articles/10.1186/1479-5868-10-22 .1479-5868-10-22 [PMC free article] [PubMed] [CrossRef] [Google Scholar]
55. Hurvitz PM, Moudon AV, Kang B, Saelens BE, Duncan GE. Emerging technologies for assessing physical activity behaviors in space and time. Front Public Health. 2014;2:2. doi: 10.3389/fpubh.2014.00002. [PMC free article] [PubMed] [CrossRef] [Google Scholar]
56. Yang P, Hanneghan M, Qi J, Deng Z, Dong F, Fan D. Improving the validity of lifelogging physical activity measures in an internet of things environment. Proceedings of the 2015 IEEE International Conference on Computer and Information Technology; Ubiquitous Computing and Communications; Dependable, Autonomic and Secure Computing; Pervasive Intelligence and Computing; 2015 IEEE International Conference on Computer and Information Technology; Ubiquitous Computing and Communications; Dependable, Autonomic and Secure Computing; Pervasive Intelligence and Computing; Oct 26-28, 2015; Liverpool, UK. 2015. [CrossRef] [Google Scholar]
57. Yang P, Stankevicius D, Marozas V, Deng Z, Liu E, Lukosevicius A, Dong F, Xu L, Min G. Lifelogging data validation model for internet of things enabled personalized healthcare. IEEE Trans Syst Man Cybern Syst. 2018 Jan;48(1):50–64. doi: 10.1109/tsmc.2016.2586075. [CrossRef] [Google Scholar]
58. Dobbins C, Rawassizadeh R, Momeni E. Detecting physical activity within lifelogs towards preventing obesity and aiding ambient assisted living. Neurocomputing. 2017 Mar;230:110–32. doi: 10.1016/j.neucom.2016.02.088. [CrossRef] [Google Scholar]
59. Ni J, Chen B, Allinson NM, Ye X. A hybrid model for predicting human physical activity status from lifelogging data. Eur J Operational Res. 2020 Mar;281(3):532–42. doi: 10.1016/j.ejor.2019.05.035. [CrossRef] [Google Scholar]
60. Kim JW, Lim JH, Moon SM, Jang B. Collecting health lifelog data from smartwatch users in a privacy-preserving manner. IEEE Trans Consum Electron. 2019 Aug;65(3):369–78. doi: 10.1109/tce.2019.2924466. [CrossRef] [Google Scholar]
61. Choi J, Choi C, Ko H, Kim P. Intelligent healthcare service using health lifelog analysis. J Med Syst. 2016 Aug 28;40(8):188. doi: 10.1007/s10916-016-0534-1.10.1007/s10916-016-0534-1 [PubMed] [CrossRef] [Google Scholar]
62. Dobbins C, Fairclough S. Signal processing of multimodal mobile lifelogging data towards detecting stress in real-world driving. IEEE Trans Mobile Comput. 2019 Mar 1;18(3):632–44. doi: 10.1109/tmc.2018.2840153. [CrossRef] [Google Scholar]
63. Doherty A, Pauly-Takacs K, Caprani N, Gurrin C, Moulin CJ, O’Connor N, Smeaton AF. Experiences of aiding autobiographical memory using the SenseCam. Human Comput Interact. 2012;27(1-2):174. doi: 10.1080/07370024.2012.656050. [CrossRef] [Google Scholar]
64. Dobbins C, Fairclough S. Applied Computing in Medicine and Health. Burlington, Massachusetts: Morgan Kaufmann; 2016. Lifelogging technologies to detect negative emotions associated with cardiovascular disease; pp. 27–44. [Google Scholar]
65. Lee H, Smeaton AF, O’Connor NE, Jones G, Blighe M, Byrne D, Doherty A, Gurrin C. Constructing a SenseCam visual diary as a media process. Multimedia Syst. 2008 Jul 12;14(6):341–9. doi: 10.1007/s00530-008-0129-x. [CrossRef] [Google Scholar]
66. Gibson G, Newton L, Pritchard G, Finch T, Brittain K, Robinson L. The provision of assistive technology products and services for people with dementia in the United Kingdom. Dementia (London) 2016 Jul 05;15(4):681–701. doi: 10.1177/1471301214532643.1471301214532643 [PubMed] [CrossRef] [Google Scholar]
67. Sanders D, Scott P. Literature review: technological interventions and their impact on quality of life for people living with dementia. BMJ Health Care Inform. 2020 Jan 15;27(1):e100064. doi: 10.1136/bmjhci-2019-100064. https://informatics.bmj.com/lookup/pmidlookup?view=long&pmid=31948938 .bmjhci-2019-100064 [PMC free article] [PubMed] [CrossRef] [Google Scholar]
68. Rodgers MM, Pai VM, Conroy RS. Recent advances in wearable sensors for health monitoring. IEEE Sensors J. 2015 Jun;15(6):3119–26. doi: 10.1109/jsen.2014.2357257. [CrossRef] [Google Scholar]
69. Zheng Y, Ding X, Poon CC, Lo BP, Zhang H, Zhou X, Yang G, Zhao N, Zhang Y. Unobtrusive sensing and wearable devices for health informatics. IEEE Trans Biomed Eng. 2014 May;61(5):1538–54. doi: 10.1109/tbme.2014.2309951. [PMC free article] [PubMed] [CrossRef] [Google Scholar]
70. Gemmell J, Sundaram H. CARPE’04: Proceedings of the the 1st ACM workshop on Continuous archival and retrieval of personal experiences. New York, NY, USA: Association for Computing Machinery; 2004. [Google Scholar]
71. Gemmell J, Sundaram H. CARPE ’05: Proceedings of the 2nd ACM workshop on Continuous archival and retrieval of personal experiences. New York, NY, USA: Association for Computing Machinery; 2005. [Google Scholar]
72. Mase K. CARPE ’06: Proceedings of the 3rd ACM workshop on Continuous archival and retrival of personal experences. New York, NY, USA: Association for Computing Machinery; 2006. [Google Scholar]
73. GemmellJ G, Williams L, Wood K, Lueder R, Bell G. Passive capture and ensuing issues for a personal lifetime store. Proceedings of the the 1st ACM workshop on Continuous archival and retrieval of personal experiences; MM04: 2004 12th Annual ACM International Conference on Multimedia; Oct 15, 2004; New York USA. 2004. [CrossRef] [Google Scholar]
74. Aizawa K, Tancharoen D, Kawasaki S, Yamasaki T. Efficient retrieval of life log based on context and content. Proceedings of the the 1st ACM workshop on Continuous archival and retrieval of personal experiences; MM04: 2004 12th Annual ACM International Conference on Multimedia; Oct 15, 2004; New York, USA. 2004. [CrossRef] [Google Scholar]
75. Gurrin C, Joho H, Hopfgartner F, Zhou L, Albatal R. NTCIR lifelog: the first test collection for lifelog research. Proceedings of the 39th International ACM SIGIR conference on Research and Development in Information Retrieval; SIGIR ’16: The 39th International ACM SIGIR conference on research and development in Information Retrieval; Jul 17 – 21, 2016; Pisa Italy. 2016. [CrossRef] [Google Scholar]
76. Gurrin C, Joho H, Hopfgartner F, Zhou L, Gupta R, Albatal R, Dang-Nguyen D-T. Overview of NTCIR-13 Lifelog-2 Task. Proceedings of the The Thirteenth NTCIR conference (NTCIR-13); The Thirteenth NTCIR conference (NTCIR-13); Dec 5-8, 2017,; Tokyo, Japan. 2017. [Google Scholar]
77. Gurrin C, Joho H, Hopfgartner F, Zhou L, Ninh H-T, Le T-K, Albatal R, Dang-Nguyen D-T, Healy G. Overview of the NTCIR-14 Lifelog-3 task. Proceedings of the 14th NTCIR Conference on Evaluation of Information Access Technologies; 14th NTCIR Conference on Evaluation of Information Access Technologies; Jun 10-13, 2019; Tokyo Japan. 2019. https://doras.dcu.ie/25867/ [Google Scholar]
78. Bolanos M, Dimiccoli M, Radeva P. Toward storytelling from visual lifelogging: an overview. IEEE Trans Human Mach Syst. 2017;47(1):77–90. doi: 10.1109/thms.2016.2616296. [CrossRef] [Google Scholar]
79. Gurrin C, Joho H, Hopfgartner F, Zhou L, Albatal R. Overview of NTCIR-12 lifelog task. Proceedings of the 12th NTCIR Conference on Evaluation of Information Access Technologies; Proceedings of the 12th NTCIR Conference on Evaluation of Information Access Technologies; Jun 7-10, 2016; Tokyo Japan. 2016. https://research.nii.ac.jp/ntcir/workshop/OnlineProceedings12/pdf/ntcir/OVERVIEW/01-NTCIR12-OV-LIFELOG-GurrinC.pdf . [CrossRef] [Google Scholar]
80. Gurrin C, Giro-i-Nieto X, Radeva P, Dimiccoli M, Johansen H, Joho H, Singh VK. LTA 2016: the first workshop on lifelogging tools and applications. Proceedings of the 24th ACM international conference on Multimedia; MM ’16: ACM Multimedia Conference; Oct 15 – 19, 2016; Amsterdam The Netherlands. 2016. [CrossRef] [Google Scholar]
81. Dang-Nguyen D, Piras P, Riegler M, Boato G, Zhou L, Gurrin C. ImageCLEF. Central Europe: CEUR-WS; 2017. Overview of ImageCLEF lifelog 2017: lifelog retrieval and summarization. [Google Scholar]
82. Dang-Nguyen DT, Piras L, Riegler M, Zhou L, Lux M, Gurrin C. Overview of ImageCLEFlifelog 2018: daily living understanding and lifelog moment retrieval. Proceedings of the Conference and Labs of the Evaluation Forum; Conference and Labs of the Evaluation Forum; Sep 10-14, 2018; Avignon, France. 2018. Google Scholar]
83. Dang-Nguyen DT, Piras L, Riegler M, Zhou L, Lux M, Tran M-T, Le T-K, Ninh V-T, Gurrin C. Overview of ImageCLEFlifelog 2019: solve my life puzzle and lifelog moment retrieval. Proceedings of CLEF 2019; CLEF 2019; Sep 9-12, 2019; Lugano, Switzerland. 2019. http://ceur-ws.org/Vol-2380/paper_223.pdf . [Google Scholar]
84. Ninh V-T, Le T-K, Zhou L, Piras L, Riegler M, Halvorsen P, Lux M, Tran M-T, Gurrin C, Dang-Nguyen D-T. Overview of ImageCLEF Lifelog 2020: lifelog moment retrieval and sport performance lifelog. Proceedings of CLEF 2020; CLEF 2020; Sep 20-25, 2020; Thessalonik, Greece. 2020. http://ceur-ws.org/Vol-2696/paper_65.pdf . [CrossRef] [Google Scholar]
85. Gurrin C, Schoeffmann K, Joho H, Dang-Nguyen DT, Riegler M, Piras L. LSC ’18: Proceedings of the 2018 ACM Workshop on The Lifelog Search Challenge. New York, NY, USA: Association for Computing Machinery; 2018. [Google Scholar]
86. Gurrin C, Schoeffmann K, Joho H, Dang-Nguyen DT, Riegler M, Piras L. LSC ’19: Proceedings of the ACM Workshop on Lifelog Search Challenge. New York, NY, USA: Association for Computing Machinery; 2019. [Google Scholar]
87. Gurrin C, Schoeffmann K, Jónsson BÞ, Dang-Nguyen DT, Lokoč J, Tran MT, Hürst W. LSC ’20: Proceedings of the Third Annual Workshop on Lifelog Search Challenge. New York, NY, USA: Association for Computing Machinery; 2020. [Google Scholar]
88. Rawassizadeh R, Tomitsch M, Wac K, Tjoa AM. UbiqLog: a generic mobile phone-based life-log framework. Pers Ubiquit Comput. 2012 Apr 3;17(4):621–37. doi: 10.1007/s00779-012-0511-8. [CrossRef] [Google Scholar]
89. Bolaños M, Radeva P. Ego-object discovery. arXiv. 2015:1639. [Google Scholar]
90. Bolaños M, Peris A, Casacuberta F, Soler S, Radeva P. Egocentric video description based on temporally-linked sequences. J Visual Commun Image Representation. 2018 Jan;50:205–16. doi: 10.1016/j.jvcir.2017.11.022. [CrossRef] [Google Scholar]
91. Münzer B, Leibetseder A, Kletz S, Primus MJ, Schoeffmann K. lifeXplore at the lifelog search challenge 2018. Proceedings of the 2018 ACM Workshop on The Lifelog Search Challenge; ICMR ’18: International Conference on Multimedia Retrieval; Jun 11, 2018; Yokohama Japan. 2018. [CrossRef] [Google Scholar]
92. Schoeffmann K, Primus MJ, Muenzer B, Petscharnig S, Karisch C, Qing X, Huerst W. Collaborative feature maps for interactive video search. In: Amsaleg L, Guðmundsson G, Gurrin C, Jónsson B, Satoh S, editors. MultiMedia Modeling. MMM 2017. Lecture Notes in Computer Science, vol 10133. Cham: Springer; 2017. pp. 457–462. [Google Scholar]
93. Primus MJ, Münzer B, Leibetseder A, Schoeffmann K. MultiMedia Modeling. MMM 2018. Lecture Notes in Computer Science, vol 10705. Cham: Springer; 2018. The ITEC collaborative video search system at the video browser showdown 2018; pp. 438–443. [Google Scholar]
94. Leibetseder A, Münzer B, Primus MJ, Kletz S, Schoeffmann K. diveXplore 4.0: the ITEC deep interactive video exploration system at VBS 2020. MultiMedia Modeling; 26th International Conference on MultiMedia Modeling; January 5-8; Daejeon, South Korea. Berlin, Heidelberg: Springer-Verlag; 2020. pp. 753–759. [CrossRef] [Google Scholar]
95. Leibetseder A, Münzer B, Primus MJ, Kletz S, Schoeffmann K, Berns F, Beecks C. Lifexplore at the lifelog search challenge 2019. Proceedings of the ACM Workshop on Lifelog Search Challenge; ICMR ’19: International Conference on Multimedia Retrieval; Jun 10 – 13, 2019; Ottawa ON Canada. 2019. [CrossRef] [Google Scholar]
96. Leibetseder A, Schoeffmann K. Lifexplore at the lifelog search challenge 2020. Proceedings of the Third Annual Workshop on Lifelog Search Challenge; ICMR ’20: International Conference on Multimedia Retrieval; Jun 9, 2020; Dublin Ireland. 2020. [CrossRef] [Google Scholar]
97. Lokoč J, Souček T, Kovalčik G. Using an interactive video retrieval tool for lifelog data. Proceedings of the 2018 ACM Workshop on The Lifelog Search Challenge; ICMR ’18: International Conference on Multimedia Retrieval; Jun 11, 2018; Yokohama Japan. 2018. [CrossRef] [Google Scholar]
98. Lokoč J, Kovalčík G, Souček T, Moravec J, Čech P. Viret: a video retrieval tool for interactive known-item search. Proceedings of the 2019 on International Conference on Multimedia Retrieval; ICMR ’19: International Conference on Multimedia Retrieval; Jun 10 – 13, 2019; Ottawa ON Canada. 2019. [CrossRef] [Google Scholar]
99. Lokoč J, Souček T, Čech P, Kovalčík G. Enhanced VIRET tool for lifelog data. Proceedings of the ACM Workshop on Lifelog Search Challenge; ICMR ’19: International Conference on Multimedia Retrieval; Jun 10 – 13, 2019; Ottawa ON Canada. 2019. [CrossRef] [Google Scholar]
100. Kovalčík G, Škrhak V, Souček T, Lokoč J. VIRET tool with advanced visual browsing and feedback. Proceedings of the Third Annual Workshop on Lifelog Search Challenge; ICMR ’20: International Conference on Multimedia Retrieval; Jun 9, 2020; Dublin Ireland. 2020. [CrossRef] [Google Scholar]
101. Lokoč J, Kovalcík G, Soucek T. International Conference on Multimedia Modeling. Cham: Springer; 2018. Revisiting SIRET video retrieval tool; pp. 419–424. [Google Scholar]
102. Rossetto L, Gasser R, Heller S, Parian M, Schuldt H. Retrieval of structured and unstructured data with vitrivr. Proceedings of the ACM Workshop on Lifelog Search Challenge; ICMR ’19: International Conference on Multimedia Retrieval; Jun 10 – 13, 2019; Ottawa ON Canada. 2019. [CrossRef] [Google Scholar]
103. Heller S, Parian M, Gasser R, Sauter L, Schuldt H. Interactive lifelog retrieval with vitrivr. Proceedings of the Third Annual Workshop on Lifelog Search Challenge; ICMR ’20: International Conference on Multimedia Retrieval; Jun 9, 2020; Dublin Ireland. 2020. [CrossRef] [Google Scholar]
104. Rossetto L, Giangreco I, Tanase C, Schuldt H. Vitrivr: a flexible retrieval stack supporting multiple query modes for searching in multimedia collections. Proceedings of the 24th ACM international conference on Multimedia; MM ’16: ACM Multimedia Conference; Oct 15 – 19, 2016; Amsterdam The Netherlands. 2016. [CrossRef] [Google Scholar]
105. Gasser R, Rossetto L, Schuldt H. Multimodal multimedia retrieval with vitrivr. Proceedings of the 2019 on International Conference on Multimedia Retrieval; ICMR ’19: International Conference on Multimedia Retrieval; Jun 10 – 13, 2019; Ottawa ON Canada. 2019. [CrossRef] [Google Scholar]
106. Zhou L, Hinbarji Z, Dang-Nguyen D-T, Gurrin C. Lifer: an interactive lifelog retrieval system. Proceedings of the 2018 ACM Workshop on The Lifelog Search Challenge; ICMR ’18: International Conference on Multimedia Retrieval; Jun 11, 2018; Yokohama Japan. 2018. [CrossRef] [Google Scholar]
107. Ninh V-T, Le T-K, Zhou L, Piras L, Riegler M, Lux M, Tran M-T, Gurrin C, Dang-Nguyen D. LIFER 2.0: discovering personal lifelog insights using an interactive lifelog retrieval system. Proceedings of the CLEF 2019; CLEF 2019; Sep 9-12, 2019; Lugano, Switzerland. 2019. [CrossRef] [Google Scholar]
108. Le T-K, Ninh V-T, Zhou L, Nguyen-Ngoc M-H, Trinh H-D, Tran N-H, Piras L, Riegler M, Halvorsen P, Lux M, Tran M-T, Healy G, Gurrin C, Dang-Nguyen D-T. Organiser team at ImageCLEF Lifelog 2020: a baseline approach for moment retrieval and athlete performance prediction using lifelog data. Proceedings of the CLEF 2020; CLEF 2020; Sep 22-25, 2020; Thessaloniki, Greece. 2020. http://ceur-ws.org/Vol-2696/paper_104.pdf . [CrossRef] [Google Scholar]
109. Ribeiro R, Silva J, Trifan A, Oliveira JL, Neves AJ. UA.PT Bioinformatics at ImageCLEF 2020: lifelog moment retrieval web based tool. Proceedings of the CLEF 2020; CLEF 2020; Sep 22-25, 2020; Thessaloniki, Greece. 2020. [Google Scholar]
110. Le T-K, Ninh V-T, Tran M-T, Nguyen T-A, Nguyen H-D, Zhou L, Healy G, Gurrin C. LifeSeeker 2.0: interactive lifelog search engine at LSC 2020. Proceedings of the Third Annual Workshop on Lifelog Search Challenge; ICMR ’20: International Conference on Multimedia Retrieval; Jun 9, 2020; Dublin Ireland. 2020. [CrossRef] [Google Scholar]
111. Mai-Nguyen A-V, Phan T-D, Vo A-K, Tran V-L, Dao M-S, Zettsu K. BIDAL-HCMUS@LSC2020: an interactive multimodal lifelog retrieval with query-to-sample attention-based search engine. Proceedings of the Third Annual Workshop on Lifelog Search Challenge; ICMR ’20: International Conference on Multimedia Retrieval; Jun 9, 2020; Dublin Ireland. 2020. [CrossRef] [Google Scholar]
112. Tran M-T, Nguyen T-A, Tran Q-C, Tran M-K, Nguyen K, Ninh V-T, Le T-K, Trang-Trung H-P, Le H-A, Nguyen HD. FIRST – Flexible Interactive Retrieval SysTem for visual lifelog exploration at LSC 2020. Proceedings of the Third Annual Workshop on Lifelog Search Challenge; ICMR ’20: International Conference on Multimedia Retrieval; Jun 9, 2020; Dublin Ireland. 2020. [CrossRef] [Google Scholar]
113. Rossetto L, Baumgartner M, Ashena N, Ruosch F, Pernischová R, Bernstein A. LifeGraph: a knowledge graph for lifelogs. Proceedings of the Third Annual Workshop on Lifelog Search Challenge; ICMR ’20: International Conference on Multimedia Retrieval; Jun 9, 2020; Dublin Ireland. 2020. [CrossRef] [Google Scholar]
114. Khan OS, Larsen MD, Poulsen LA, Jónsson B, Zahálka J, Rudinac S, Koelma D, Worring M. Exquisitor at the lifelog search challenge 2020. Proceedings of the Third Annual Workshop on Lifelog Search Challenge; ICMR ’20: International Conference on Multimedia Retrieval; Jun 9, 2020; Dublin Ireland. 2020. [CrossRef] [Google Scholar]
115. Karako K, Chen Y, Song P, Tang W. Super-aged society: constructing an integrated information platform of self-recording lifelogs and medical records to support health care in Japan. Biosci Trends. 2019;13(3):276–8. doi: 10.5582/bst.2019.01124. doi: 10.5582/bst.2019.01124. [PubMed] [CrossRef] [CrossRef] [Google Scholar]
116. Sugawara J, Ochi D, Yamashita R, Yamauchi T, Saigusa D, Wagata M, Obara T, Ishikuro M, Tsunemoto Y, Harada Y, Shibata T, Mimori T, Kawashima J, Katsuoka F, Igarashi-Takai T, Ogishima S, Metoki H, Hashizume H, Fuse N, Minegishi N, Koshiba S, Tanabe O, Kuriyama S, Kinoshita K, Kure S, Yaegashi N, Yamamoto M, Hiyama S, Nagasaki M. Maternity Log study: a longitudinal lifelog monitoring and multiomics analysis for the early prediction of complicated pregnancy. BMJ Open. 2019 Feb 19;9(2):e025939. doi: 10.1136/bmjopen-2018-025939. https://bmjopen.bmj.com/lookup/pmidlookup?view=long&pmid=30782942 .bmjopen-2018-025939 [PMC free article] [PubMed] [CrossRef] [Google Scholar]
117. Dobbins C, Merabti M, Fergus P, Llewellyn-Jones D, Bouhafs F. Exploiting linked data to create rich human digital memories. Comput Commun. 2013 Sep;36(15-16):1639–56. doi: 10.1016/j.comcom.2013.06.008. [CrossRef] [Google Scholar]
118. Kim S, Yeom S, Kwon O, Shin D, Shin D. Ubiquitous healthcare system for analysis of chronic patients’ biological and lifelog data. IEEE Access. 2018;6:8909–15. doi: 10.1109/access.2018.2805304. [CrossRef] [Google Scholar]
119. Jacquemard T, Novitzky P, O’Brolcháin F, Smeaton AF, Gordijn B. Challenges and opportunities of lifelog technologies: a literature review and critical analysis. Sci Eng Ethics. 2014 Jun 9;20(2):379–409. doi: 10.1007/s11948-013-9456-1. [PubMed] [CrossRef] [Google Scholar]
120. Doherty AR, Hodges SE, King AC, Smeaton AF, Berry E, Moulin CJ, Lindley S, Kelly P, Foster C. Wearable cameras in health: the state of the art and future possibilities. Am J Prev Med. 2013 Mar;44(3):320–3. doi: 10.1016/j.amepre.2012.11.008.S0749-3797(12)00866-5 [PubMed] [CrossRef] [Google Scholar]
121. Florez-Revuelta F, Mihailidis A, Ziefle M, Colonna L, Spinsante S. Privacy-aware and acceptable lifelogging services for older and frail people: The PAAL project. Proceedings of the 2018 IEEE 8th International Conference on Consumer Electronics – Berlin (ICCE-Berlin); 2018 IEEE 8th International Conference on Consumer Electronics – Berlin (ICCE-Berlin); Sep 2-5, 2018; Berlin, Germany. 2018. [CrossRef] [Google Scholar]
122. Sellen AJ, Whittaker S. Beyond total capture. Commun ACM. 2010 May;53(5):70–7. doi: 10.1145/1735223.1735243. [CrossRef] [Google Scholar]
123. Dobbins C, Merabti M, Fergus P, Llewellyn-Jones D. Creating human digital memories with the aid of pervasive mobile devices. Pervasive Mobile Comput. 2014 Jun;12:160–78. doi: 10.1016/j.pmcj.2013.10.009. [CrossRef] [Google Scholar]