LifeLogging: Personal big data

by | Jun 16, 2014 | Research Publications

Author(s)

  • Gurrin, Cathal
  • Smeaton, Alan F.
  • Doherty, Aiden R.

Abstract:

We have recently observed a convergence of technologies to foster the emergence of lifelogging as a mainstream activity. Computer storage has become significantly cheaper, and advancements in sensing technology allows for the efficient sensing of personal activities, locations and the environment. This is best seen in the growing popularity of the quantified self movement, in which life activities are tracked using wearable sensors in the hope of better understanding human performance in a variety of tasks. This review aims to provide a comprehensive summary of lifelogging, to cover its research history, current technologies, and applications. Thus far, most of the lifelogging research has focused predominantly on visual lifelogging, hence we maintain this focus in this review. However, we also reflect on the challenges lifelogging poses for information access and retrieval in general. This review is a suitable reference for those seeking an information retrieval scientist’s perspective on lifelogging and the quantified self.

Document:

https://www.nowpublishers.com/article/Details/INR-033

References:
  1. Brindley, R., Bateman, A., and Gracey, F. (2011). Exploration of use of SenseCam to support autobiographical memory retrieval within a cognitive- behavioural therapeutic intervention following acquired brain injury. Mem- ory, 19(7):745–757.
  2. Bristow, H. W., Baber, C., Cross, J., Knight, J. F., and Woolley, S. I. (2004). Defining and evaluating context for wearable computing. International Journal of Human Computer Studies, 60(5-6):798–819.
  3. Bukhin, M. and DelGaudio, M. (2006). WayMarkr: Acquiring perspective through continuous documentation. In MUM ’06: Proceedings of the 5th international conference on mobile and ubiquitous multimedia, page 9, New York, NY, USA. ACM.
  4. Burke, J., Estrin, D., Hansen, M., Parker, A., Ramanathan, N., Reddy, S., and Srivastava, M. B. (2006). Participatory Sensing. Boulder, Colorado, USA.
  5. Bush, V. (1945). As we may think. The Atlantic Monthly, 176(1):101–108.
  6. Byrne, D., Doherty, A. R., Smeaton, A. F., Jones, G. J., Kumpulainen, S., and Järvelin, K. (2008a). The SenseCam as a Tool for Task Observation. In HCI 2008 – 22nd BCS HCI Group Conference.
  7. Byrne, D. and Jones, G. J. (2009). Exploring narrative presentation for large multimodal lifelog collections through card sorting. In Proceedings of the 2nd Joint International Conference on Interactive Digital Storytelling: In- teractive Storytelling, ICIDS ’09, pages 92–97, Berlin, Heidelberg. Springer- Verlag.
  8. Byrne, D. and Jones, G. J. F. (2008). Towards computational autobiographical narratives through human digital memories. In Proceeding of the 2nd ACM international workshop on Story representation, mechanism and context, pages 9–12. ACM.
  9. Byrne, D., Kelliher, A., and Jones, G. J. (2011). Life editing: third-party perspectives on lifelog content. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI ’11, pages 1501–1510, New York, NY, USA. ACM.
  10. Byrne, D., Lavelle, B., Jones, G. J., and Smeaton, A. F. (2007). Visualising bluetooth interactions: Combining the arc diagram and docuburst tech- niques. In HCI 2007 – Proceedings of the 21st BCS HCI Group Conference.
  11. Byrne, D., Lee, H., Jones, G. J. F., and Smeaton, A. F. (2008b). Guidelines for the presentation and visualisation of lifelog content. In iHCI 2008 – Irish HCI 2008
  12. Byrne, R. and Diamond, D. (2006). Chemo/bio-sensor networks. Nature Materials, 5(6):421–424.
  13. Callan, J., Allan, J., Clarke, C., Dumais, S., Evans, D., Sanderson, M., and Zhai, C. (2007). Meeting of the MINDS: an information retrieval research agenda. ACM SIGIR Forum, 41(2):25–34. Caprani, N., Doherty, A. R., Lee, H., Smeaton, A. F., O’Connor, N. E., and Gurrin, C. (2010). Designing a touch-screen SenseCam browser to support an aging population. In CHI ’10 Extended Abstracts on Human Factors in Computing Systems, CHI EA ’10, pages 4291–4296, New York, NY, USA. ACM.
  14. Cavoukian, A. (2010). Privacy by design: The 7 foundational principles. im- plementation and mapping of fair information practices. Information and Privacy Commissioner of Ontario, Canada. Chatfield, K. and Zisserman, A. (2012). VISOR: Towards On-the-Fly Large- Scale Object Category Retrieval. In Asian Conference on Computer Vision, Lecture Notes in Computer Science. Springer.
  15. Cheng, W. C., Golubchik, L., and Kay, D. G. (2004). Total recall: are privacy changes inevitable? In Proceedings of the the 1st ACM workshop on Con- tinuous archival and retrieval of personal experiences, CARPE’04, pages 86–92, New York, NY, USA. ACM.
  16. Cohen, G. and Conway, M. A. (2008). Memory in the real world. Routledge. Conway, M. A. and Loveday, C. (2011). SenseCam: The Future of Everyday Memory Research. Memory, 19(7):685–807.
  17. d’Aquin, M., Elahi, S., and Motta, E. (2010). Personal monitoring of web information exchange: Towards web lifelogging. In Proceedings of the Web- Sci10: Extending the Frontiers of Society On-Line.
  18. De Jager, D., Wood, A. L., Merrett, G. V., Al-Hashimi, B. M., O’Hara, K., Shadbolt, N. R., and Hall, W. (2011). A low-power, distributed, pervasive healthcare system for supporting memory. In Proceedings of the First ACM MobiHoc Workshop on Pervasive Wireless Healthcare, page 5. ACM.
  19. de Silva, G. C., Yamasaki, T., and Aizawa, K. (2007). An interactive multi- media diary for the home. Computer, 40(5):52–59.
  20. Dean, T., Ruzon, M., Segal, M., Shlens, J., Vijayanarasimhan, S., and Yagnik, J. (2013). Fast, Accurate Detection of 100,000 Object Classes on a Single Machine. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, (CVPR), Washington, DC, USA.
  21. Deterding, S., Sicart, M., Nacke, L., O’Hara, K., and Dixon, D. (2011). Gam- ification. Using game-design elements in non-gaming contexts. In PART 2: Proceedings of the 2011 annual conference extended abstracts on Human factors in computing systems, pages 2425–2428. ACM.
  22. Dey, A. K. and Abowd, G. D. (1999). Towards a better understanding of context and context-awareness. In HUC’99: Proceedings of the 1st interna- tional symposium on Handheld and Ubiquitous Computing, pages 304–307, London, UK. Springer-Verlag.
  23. Dickie, C., Vertegaal, R., Fono, D., Sohn, C., Chen, D., Cheng, D., Shell, J. S., and Aoudeh, O. (2004). Augmenting and sharing memory with eyeblog. In CARPE’04. Proceedings of the the 1st ACM workshop on Continuous archival and retrieval of personal experiences, pages 105–109, New York, NY, USA. ACM.
  24. Dodge, M. and Kitchin, R. (2007). “Outlines of a world coming into exis- tence”: Pervasive computing and the ethics of forgetting. Environment and Planning B, 34(3):431–445.
  25. Doherty, A. and Smeaton, A. (2008a). Automatically segmenting lifelog data into events. In Ninth International Workshop on Image Analysis for Mul- timedia Interactive Services, pages 20–23. IEEE.
  26. Doherty, A. R., Byrne, D., Smeaton, A. F., Jones, G. J., and Hughes, M. (2008). Investigating keyframe selection methods in the novel domain of passively captured visual lifelogs. In CIVR ’08: Proceedings of the 2008 international conference on Content-based image and video retrieval, pages 259–268, Niagara Falls, Canada. ACM.
  27. Doherty, A. R., Caprani, N., Ó’Conaire, C., Kalnikaite, V., Gurrin, C., Smeaton, A. F., and Connor, N. E. O. (2011a). Passively recognising human activities through lifelogging. Computers in Human Behavior, 27(5):1948– 1958.
  28. Doherty, A. R., Gurrin, C., and Smeaton, A. F. (2009). An investigation into event decay from large personal media archives. In Proceedings of the 1st ACM international workshop on Events in multimedia, EiMM ’09, pages 49–56, New York, NY, USA. ACM.
  29. Doherty, A. R., Hodges, S. E., King, A. C., Smeaton, A. F., Berry, E., Moulin, C. J., Lindley, S., Kelly, P., and Foster, C. (2013a). Wearable cameras in health: The state of the art and future possibilities. Am J Prev Med, 44(3):320–323
  30. Doherty, A. R., Kelly, P., and Foster, C. (2013b). Wearable Cameras: Identify- ing Healthy Transportation Choices. IEEE Pervasive Computing, 12(1):44– 47.
  31. Doherty, A. R., Kelly, P., Kerr, J., Marshall, S., Oliver, M., Badland, H., Hamilton, A., and Foster, C. (2013c). Using wearable cameras to categorise type and context of accelerometer-identified episodes of physical activity. International Journal Behavioural Nutrition Physical Activity, 10:22.
  32. Doherty, A. R., Moulin, C. J., and Smeaton, A. F. (2011b). Automatically assisting human memory: A SenseCam browser. Memory, 7(19):785–795.
  33. Doherty, A. R., Pauly-Takacs, K., Caprani, N., Gurrin, C., Moulin, C. J. A., O’Connor, N. E., and Smeaton, A. F. (2012). Experiences of Aiding Auto- biographical Memory Using the SenseCam. Human-Computer Interaction, 27(1-2):151–174.
  34. Doherty, A. R. and Smeaton, A. F. (2008b). Automatically segmenting lifelog data into events. In WIAMIS: 9th International Workshop on Image Anal- ysis for Multimedia Interactive Services, pages 20–23, Washington, DC, USA. IEEE Computer Society.
  35. Doherty, A. R. and Smeaton, A. F. (2008c). Combining face detection and novelty to identify important events in a visual lifelog. In CIT: 8th Interna- tional Conference on Computer and Information Technology, Workshop on Image- and Video-based Pattern Analysis and Applications, Sydney, Aus- tralia. IEEE Computer Society.
  36. Doherty, A. R. and Smeaton, A. F. (2010). Automatically augmenting lifelog events using pervasively generated content from millions of people. Sensors, 10(3):1423–1446.
  37. Doherty, A. R., Smeaton, A. F., Lee, K., and Ellis, D. P. (2007). Multimodal segmentation of lifelog data. In RIAO 2007 – Large-Scale Semantic Access to Content (Text, Image, Video and Sound), Pittsburg, PA, USA.
  38. Dong, W., Lepri, B., and Pentland, A. S. (2011). Modeling the co-evolution of behaviors and social relationships using mobile phone data. In Proceedings of the 10th International Conference on Mobile and Ubiquitous Multimedia, MUM ’11, pages 134–143, Beijing, China. ACM.
  39. Doulamis, A., van Gool, L., Nixon, M., Varvarigou, T., and Doulamis, N. (2008). Area ’08: Proceedings of the 1st acm workshop on analysis and retrieval of events/actions and workflows in video streams. In Proceedings of the 16th ACM International Conference on Multimedia, MM ’08, pages 1147–1148, New York, NY, USA. ACM. 433087.
  40. Dumais, S., Cutrell, E., Cadiz, J., Jancke, G., Sarin, R., and Robbins, D. C. (2003). Stuff I’ve seen: a system for personal information retrieval and re- use. In Proceedings ofthe 26th annual international ACMSIGIR conference on Research and development in informaion retrieval, pages 72–79, ACM. New York, NY, USA.
  41. Dunton, G. F., Liao, Y., Intille, S. S., Spruijt-Metz, D., and Pentz, M. (2011). Investigating children’s physical activity and sedentary behavior using eco- logical momentary assessment with mobile phones. Obesity (Silver Spring), 19(6):1205–1212.
  42. Eagle, N. and Pentland, A. (2005). Social serendipity: Mobilizing social soft- ware. Pervasive Computing, IEEE, 4(2):28–34.
  43. Eagle, N. and Pentland, A. S. (2006). Reality mining: Sensing complex social systems. Personal Ubiquitous Comput., 10(4):255–268. Economist, T. (2010). The data deluge. Special Supplement.
  44. Ellis, D. P. and Lee, K. (2004). Minimal-impact audio-based personal archives. In CARPE’04: Proceedings of the the 1st ACM workshop on Continuous archival and retrieval of personal experiences, pages 39–47, New York, NY, USA. ACM.
  45. Ellis, D. P. and Lee, K. (2006). Accessing minimal-impact personal audio archives. IEEE Multimedia, 13(4):30–38.
  46. Elsweiler, D. and Ruthven, I. (2007). Towards task-based personal information management evaluations. In Proceedings of the 30th annual international ACM SIGIR conference on Research and development in information re- trieval, SIGIR ’07, pages 23–30, Amsterdam, The Netherlands. ACM.
  47. Everingham, M., Van Gool, L., Williams, C. K., Winn, J., and Zisserman, A. (2010). The PASCAL Visual Object Classes (VOC) Challenge. Interna- tional Journal of Computer Vision, 88(2):303–338.
  48. Fallah, N., Apostolopoulos, I., Bekris, K., and Folmer, E. (2013). Indoor human navigation systems: A survey. Interacting with Computers, 25(1):21– 33.
  49. Fleck, R. (2005). Exploring SenseCam to Inform the Design of Image Capture and Replay Devices for Supporting Reflection. Cognitive Science Research Paper – University of Sussex CSRP, 576:22.
  50. Fleck, R. and Fitzpatrick, G. (2006). Supporting collaborative reflection with passive image capture. In Supplementary proceedings of COOP’06, pages 41–48.
  51. Fox, E. and Shaw, J. (1993). Combination of multiple searches. In TREC 2: Text REtrieval Conference, pages 243–252, Gaithersberg, Md., USA.
  52. Fuller, R. B., Hays, K. M., and Miller, D. A. (2008). Buckminster Fuller: starting with the Universe. Yale University Press.
  53. Ganeriwal, S., Balzano, L. K., and Srivastava, M. B. (2008). Reputation- based framework for high integrity sensor networks. ACMTransactions on Sensor Networks, 4(3):1–37.
  54. Gemmell, J., Bell, G., and Lueder, R. (2006). MyLifeBits: a personal database for everything. Communications of the ACM, 49(1):88–95.
  55. Gemmell, J., Bell, G., Lueder, R., Drucker, S., and Wong, C. (2002). MyLifeBits: fulfilling the Memex vision. In Proceedings of the tenth ACM international conference on Multimedia, pages 235–238. ACM.
  56. Gemmell, J., Williams, L., Wood, K., Lueder, R., and Bell, G. (2004). Passive capture and ensuing issues for a personal lifetime store. In CARPE’04: Pro- ceedings of the the 1st ACM workshop on Continuous archival and retrieval of personal experiences, pages 48–55, New York, NY, USA. ACM.
  57. Gemming, L., Doherty, A. R., Kelly, P., Utter, J., and Ni Mhurchu, C. (2013). Feasibility of a SenseCam-assisted 24-h recall to reduce under-reporting of energy intake. European journal of clinical nutrition, 67(10):1095–1099.
  58. Girgensohn, A. and Boreczky, J. (2000). Time-constrained keyframe selection technique. Multimedia Tools Appl., 11(3):347–358.
  59. Graham, A., Garcia-Molina, H., Paepcke, A., and Winograd, T. (2002). Time as essence for photo browsing through personal digital libraries. In JCDL ’02: Proceedings of the 2nd ACM/IEEE-CS joint conference on Digital li- braries, pages 326–335, New York, NY, USA. ACM.
  60. Gurrin, C., Aarflot, T., Sav, S., and Johansen, D. (2009). Addressing the chal- lenge of managing large-scale digital multimedia libraries. JDIM, 7(5):262– 270.
  61. Gurrin, C., Byrne, D., Jones, G. J. F., and Smeaton, A. F. (2008a). Archi- tecture and challenges of maintaining a large-scale, context-aware human digital memory. In Visual Information Engineering, 2008. VIE 2008. 5th International Conference on, pages 158–163.
  62. Gurrin, C., Lee, H., Caprani, N., Zhang, Z., O’Connor, N., and Carthy, D. (2010a). Browsing large personal multimedia archives in a lean-back envi- ronment. In Advances in Multimedia Modeling, volume Boll, Susanne and Tian, Qi and Zhang, Lei and Zhang, Zili and Chen, Yi-PingPhoebe of Lec- ture Notes in Computer Science, pages 98–109. Springer Berlin Heidelberg.
  63. Gurrin, C., Lee, H., and Hayes, J. (2010b). iforgot: a model of forgetting in robotic memories. In Proceedings of the 5th ACM/IEEE international conference on Human-robot interaction, HRI ’10, pages 93–94, Piscataway, NJ, USA. IEEE Press.
  64. Gurrin, C., Qiu, Z., Hughes, M., Caprani, N., Doherty, A. R., Hodges, S. E., and Smeaton, A. F. (2013). The SmartPhone as a Platform for Wearable Cameras in Preventative Medicine. Am J Prev Med, 44(3):308–313.
  65. Gurrin, C. and Smeaton, A. (2004). Replicating web structure in small-scale test collections. Information Retrieval, 7(3-4):239–263.
  66. Gurrin, C., Smeaton, A. F., Byrne, D., O’Hare, N., Jones, G. J., and O’Connor., N. E. (2008b). An examination of a large visual lifelog. In AIRS 2008 – Asia Information Retrieval Symposium.
  67. Gürses, S., Troncoso, C., and Diaz, C. (2011). Engineering privacy by design. Computers, Privacy & Data Protection.
  68. Hamilakis, Y. and Labanyi, J. (2008). Introduction: time, materiality, and the work of memory. History & Memory, 20(2):5–17.
  69. Hanjalic, A. (2012). New grand challenge for multimedia information retrieval: bridging the utility gap. International Journal of Multimedia Information Retrieval, 1:139–152.
  70. Hauptmann, A., Yan, R., and Lin, W.-H. (2007). How many high-level con- cepts will fill the semantic gap in news video retrieval? In Proceedings of the 6th ACM international conference on Image and video retrieval, CIVR ’07, pages 627–634, New York, NY, USA. ACM.
  71. Haveliwala, T. H. (2002). Topic-sensitive pagerank. In Proceedings of the 11th International Conference on World Wide Web, WWW ’02, pages 517–526, New York, NY, USA. ACM.
  72. Hayes, G. R., Patel, S. N., Truong, K. N., Iachello, G., Kientz, J. A., Farmer, R., and Abowd, G. D. (2004). The personal audio loop: Designing a ubiq- uitous audio-based memory aid. In In Proceedings of Mobile HCI, pages 168–179. Springer Verlag.
  73. Hearst, M. A. and Plaunt, C. (1993). Subtopic structuring for full-length document access. In SIGIR ’93: Proceedings of the 16th annual interna- tional ACMSIGIR conference on Research and development in information retrieval, pages 59–68, New York, NY, USA. ACM.
  74. Hebbalaguppe, R., McGuinness, K., Kuklyte, J., Healy, G., O’Connor, N., and Smeaton, A. (2013). How interaction methods affect image segmentation: user experience in the task. In The 1st IEEE Workshop on User-Centered ComputerVision (UCCV), 16-18 Jan 2013, Tampa, Florida, U.S.A.. IEEE.
  75. Heittola, T., Mesaros, A., Eronen, A., and Virtanen, T. (2010). Audio con- text recognition using audio event histograms. In 18th European Signal Processing Conference, pages 1272–1276.
  76. Hodges, S., Berry, E., and Wood, K. (2011). SenseCam: A wearable cam- era that stimulates and rehabilitates autobiographical memory. Memory, 7(19):685–696.
  77. Hodges, S., Williams, L., Berry, E., Izadi, S., Srinivasan, J., Butler, A., Smyth, G., Kapur, N., and Wood, K. (2006). SenseCam: A retrospective memory aid. In UbiComp: 8th International Conference on Ubiquitous Computing, volume 4602 of LNCS, pages 177–193, Berlin, Heidelberg. Springer.
  78. Hopfgartner, F., Yang, Y., Zhou, L., and Gurrin, C. (2013). Semantic Models for Adaptive Interactive Systems, chapter User Interaction Templates for the Design of Lifelogging Systems, pages 187–204. Springer London.
  79. Hori, T. and Aizawa, K. (2003). Context-based video retrieval system for the life-log applications. In MIR ’03: Proceedings of the 5th ACM SIGMM international workshop on Multimedia information retrieval, pages 31–38, New York, NY, USA. ACM Press.
  80. Hughes, M., Newman, E., Smeaton, A. F., and O’Connor, N. E. (2012). A lifelogging approach to automated market research. In Proceedings of the SenseCam Symposium 2012.
  81. Ingwersen, P. and Järvelin, K. (2005). The Turn: Integration of information seeking and retrieval in context. Springer.
  82. Jacquemard, T., Novitzky, P., O’Brolcháin, F., Smeaton, A. F., and Gordijn, B. (2013). Challenges and opportunities of lifelog technologies: A literature review and critical analysis. Science and engineering ethics, pages 1–31.
  83. Jaimes, R., Omura, K., Nagamine, T., and Hirata, K. (2004). Memory cues for meeting video retrieval. In In CARPE’04: Proceedings of the the 1st ACM workshop on Continuous archival and retrieval of personal experi- ences, pages 74–85. ACM Press.
  84. Järvelin, K. (2011). IR research: systems, interaction, evaluation and theories. In Proceedings of the 33rd European conference on Advances in information retrieval, ECIR’11, pages 1–3, Berlin, Heidelberg. Springer-Verlag.
  85. Järvelin, K. and Wilson, T. (2003). On conceptual models for information seeking and retrieval research. Information Research, 9(1):9–1.
  86. Joachims, T. (2002). SVM light, http://svmlight.joachims.org.
  87. Jones, G. J., Gurrin, C., Kelly, L., Byrne, D., and Chen, Y. (2008). Informa- tion access tasks and evaluation for personal lifelogs. In 2nd International workshop on Evaluating Information Access (EVIA), Tokyo, Japan.
  88. Jones, W. (2007). Personal information management. Annual Review of In- formation Science and Technology, 41(1):453–504.
  89. Kalnikaite, V., Sellen, A., Whittaker, S., and Kirk, D. (2010). Now let me see where i was: Understanding how lifelogs mediate memory. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI ’10, pages 2045–2054, New York, NY, USA. ACM.
  90. Kansal, A., Hsu, J., Zahedi, S., and Srivastava, M. B. (2007). Power manage- ment in energy harvesting sensor networks. ACMTransactions on Embed- ded Computing Systems (TECS), 6(4):32.
  91. Kelly, P., Doherty, A. R., Berry, E., Hodges, S., Batterham, A. M., and Foster, C. (2011). Can we use digital life-log images to investigate active and sedentary travel behaviour ? Results from a Pilot Study. International Journal Behavioural Nutrition Physical Activity, 8:44.
  92. Kelly, P., Doherty, A. R., Hamilton, A., Matthews, A., Batterham, A. M., Nelson, M., Foster, C., and Cowburn, G. (2012). Investigating the error on self-reported journey durations in school children. Am J Prev Med, 43(5):546–550.
  93. Kelly, P., Doherty, A. R., Smeaton, A. F., Gurrin, C., and O’Connor, N. E. (2010). The colour of life: novel visualisations of population lifestyles. In MM ’10 Proceedings of the international conference on Multimedia. ACM.
  94. Kelly, P., Marshall, S., Badland, H., Kerr, J., Oliver, M., Doherty, A. R., and Foster, C. (2013). Ethics of using wearable cameras devices in health behaviour research. Am J Prev Med, 44(3):314–319.
  95. Kennedy, L. and Hauptmann, A. (2006). LSCOM lexicon definitions and annotations version 1.0. Technical report, Columbia University.
  96. Kent, P. (2012). Search engine optimization for dummies. Wiley. Kern, N., Schiele, B., and Schmidt, A. (2007). Recognizing context for anno-
  97. tating a live life recording. Personal and Ubiquitous Computing, 11(4):287– 298.
  98. Kerr, J., Marshall, S. J., Godbole, S., Chen, J., Legge, A., Doherty, A. R., Kelly, P., Oliver, M., Badland, H., and Foster, C. (2013). Using the Sense- Cam to improve classifications of sedentary behavior in free-living settings. American Journal of Preventive Medicine, 44(3):290–296.
  99. Kikhia, B., Hallberg, J., Bengtsson, J. E., Savenstedt, S., and Synnes, K. (2010). Building digital life stories for memory support. Int. J. Comput. Healthc.,1(2):161–176.
  100. Multimedia, MM ’08, pages 999–1000, New York, NY, USA. ACM.
  101. Klimt, B. and Yang, Y. (2004). Introducing the Enron corpus. In First Conference on Email and Anti-Spam (CEAS).
  102. Kokare, M., Chatterji, B., and Biswas, P. (2003). Comparison of similar- ity metrics for texture image retrieval. In TENCON 2003: Conference on Convergent Technologies for Asia-Pacific Region, pages 571–575.
  103. Kuhlthau, C. (1991). Inside the search process: Information seeking from the user’s perspective. Journal of the American Society for Information Science, 42(5):361–371.
  104. Kumpulainen, S., Järvelin, K., Serola, S., Doherty, A. R., Smeaton, A. F., Byrne, D., and Jones, G. J. F. (2009). Data collection methods for task- based information access in molecular medicine. pages 1–10.
  105. Langheinrich, M. (2001). Privacy by Design — Principles of Privacy-Aware Ubiquitous Systems. In Abowd, G., Brumitt, B., and Shafer, S., editors, Ubicomp 2001: Ubiquitous Computing, volume 2201 of Lecture Notes in Computer Science, pages 273–291. Springer Berlin Heidelberg.
  106. Lavelle, B., Byrne, D., Gurrin, C., Smeaton, A. F., and Jones, G. J. (2007). Bluetooth Familiarity: Methods of Calculation, Applications and Limita- tions. In MIRW2007 – Mobile Interaction with the Real World, Workshop at the MobileHCI07: 9th International Conference on Human Computer Interaction with Mobile Devices and Services.
  107. Lazer, D., Pentland, A. S., Adamic, L., Aral, S., Barabasi, A. L., Brewer, D., Christakis, N., Contractor, N., Fowler, J., Gutmann, M., Jebara, T., King, G., Macy, M., Roy, D., and Alstyne, M. V. (2009). Life in the network: the coming age of computational social science. Science (New York, NY), 323(5915):721–723.
  108. Lee, H., Smeaton, A. F., O’Connor, N. E., Jones, G. J., Blighe, M., Byrne, D., Doherty, A., and Gurrin., C. (2008). Constructing a SenseCam Visual Diary as a Media Process Multimedia Systems. Multimedia Systems Journal, Special Issue on Canonical Processes of Media Production, 14(6):341–349.
  109. Lee, M. L. and Dey, A. K. (2007). Providing good memory cues for people with episodic memory impairment. In Assets ’07: Proceedings of the 9th In- ternational ACMSIGACCESS Conference on Computers and Accessibility, pages 131–138, Tempe, Az., USA. ACM.
  110. Lee, M. L. and Dey, A. K. (2008). Using lifelogging to support recollection for people with episodic memory impairment and their caregivers. In Pro- ceedings of the 2nd International Workshop on Systems and Networking Support for Health Care and Assisted Living Environments, HealthNet ’08, pages 14:1–14:3, New York, NY, USA. ACM.
  111. Li, J. and Wang, J. Z. (2003). Automatic linguistic indexing of pictures by a statistical modeling approach. IEEE Transactions on Pattern Analysis and Machine Intelligence, 25:1075–1088.
  112. Lim, J.-H., Tian, Q., and Mulhem, P. (2003). Home photo content modeling for personalized event-based retrieval. Multimedia, IEEE, 10(4):28–37.
  113. Lin, W.-H. and Hauptmann, A. (2006). Structuring continuous video record- ings of everyday life using time-constrained clustering. In Multimedia Con- tent Analysis, Management, and Retieval: SPIE-IST Electronic Imaging, volume 6073, pages 111–119.
  114. Lindley, S. E., Glancy, M., Harper, R., Randall, D., and Smyth, N. (2011). “Oh and how things just don’t change, the more things stay the same”: Reflections on SenseCam images 18 months after capture. International Journal of Human-Computer Studies, 69(5):311–323. Designing for Reflec- tion on Personal Experience.
  115. Lowe, D. (2004). Distinctive image features from scale-invariant keypoints. International Journal of Computer Vision, 60(2):91–110.
  116. Machajdik, J., Hanbury, A., Garz, A., and Sablatnig, R. (2011). Affective computing for wearable diary and lifelogging systems: An overview. In Mayer, H., Uray, M., and Ganster, H., editors, Machine Vision-Research for High Quality Processes and Products-35th Workshop of the Austrian Association for Pattern Recognition. Austrian Computer Society.
  117. Mann, S. (1997). Wearable computing: A first step toward personal imaging. Computer, 30(2):25–32.
  118. Mann, S. (2004). Continuous lifelong capture of personal experience with Eye- Tap. In Proceedings of the the 1st ACM workshop on Continuous archival and retrieval of personal experiences, CARPE’04, pages 1–21, New York, NY, USA. ACM.
  119. Mann, S., Fung, J., Aimone, C., Sehgal, A., and Chen, D. (2005). Designing EyeTap Digital Eyeglasses for Continuous Lifelong Capture and Sharing of Personal Experiences. In Alt. Chi, Proc. CHI 2005
  120. Mann, S., Huang, J., Janzen, R., Lo, R., Rampersad, V., Chen, A., and Doha, T. (2011). Blind navigation with a wearable range camera and vibrotac- tile helmet. In Proceedings of the 19th ACM international conference on Multimedia, pages 1325–1328, New York, NY, USA. ACM.
  121. Mann, S., Lo, R., Ovtcharov, K., Gu, S., Dai, D., Ngan, C., and Ai, T. (2012). Realtime HDR (High Dynamic Range) video for EyeTap wearable comput- ers, FPGA-based seeing aids, and glasseyes (EyeTaps). In Electrical Com- puter Engineering (CCECE), 2012 25th IEEE Canadian Conference on, pages 1–6.
  122. Mantiuk, R., Kowalik, M., Nowosielski, A., and Bazyluk, B. (2012). Do-it- yourself eye tracker: low-cost pupil-based eye tracker for computer graphics applications. In Proceedings of the 18th international conference on Ad- vances in Multimedia Modeling, MMM’12, pages 115–125, Berlin, Heidel- berg. Springer-Verlag.
  123. Massimi, M. and Baecker, R. M. (2010). A death in the family: Opportunities for designing technologies for the bereaved. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI ’10, pages 1821– 1830, New York, NY, USA. ACM.
  124. May, G. and Warrington, G. (2012). Assessment of physical activity in search and rescue operations using accelerometer based technologies. In SenseCam Symposium 2012, Oxford, UK.
  125. Mayer-Schonberger, V. (2011). Delete: the virtue of forgetting in the digital age. Princeton University Press. Melucci, M. (2012). Contextual Search: A Computational Framework., vol- ume 6. NOW Publishers.
  126. Michie, S., van Stralen, M., and West, R. (2011). The behaviour change wheel: A new method for characterising and designing behaviour change interventions. Implementation Science, 6(1):1–12.
  127. Montague, M. and Aslam, J. A. (2001). Relevance score normalization for metasearch. In CIKM ’01: Proceedings of the tenth international conference on Information and knowledge management, pages 427–433, New York, NY, USA. ACM Press.
  128. Naaman, M., Harada, S., Wang, Q., Garcia-Molina, H., and Paepcke, A. (2004). Context data in geo-referenced digital photo collections. In MULTI- MEDIA ’04: Proceedings of the 12th annual ACM international conference on Multimedia, pages 196–203, New York, NY, USA. ACM
  129. Naphade, M., Smith, J. R., Tesic, J., Chang, S.-F., Hsu, W., Kennedy, L., Hauptmann, A., and Curtis, J. (2006). LSCOM: Large-Scale Concept On- tology for Multimedia. IEEE Multimedia, 13(3):86–91.
  130. Nelson, R. O. and Hayes, S. C. (1981). Theoretical explanations for reactivity in self-monitoring. Behavior Modification, 5(1):3–14.
  131. Nelson, T. H. (1988). Managing immense storage. BYTE, 13(1):225–238. Neo, S.-Y., Zhao, J., Kan, M.-Y., and Chua, T.-S. (2006). Video retrieval
  132. using high level features: Exploiting query matching and confidence-based weighting. In CIVR’06: Proc. Conf. Image and Video Retrieval, pages 143– 152. Springer-Verlag, Tempe, AZ, USA.
  133. Nistér, D. and Stewnius, H. (2006). Scalable recognition with a vocabulary tree. In IEEE Conference on Computer Vision and Pattern Recognition, pages 2161–2168, New York, USA.
  134. Oard, D. W. and Malionek, J. (2013). The apollo archive explorer. In Proceedings of the 13th ACM/IEEE-CS joint conference on Digital libraries, pages 453–454. ACM, ACM.
  135. O’Brien, A., McDaid, K., Loane, J., Doyle, J., and O’Mullane, B. (2012). Visualisation of movement of older adults within their homes based on pir sensor data. In Pervasive Computing Technologies for Healthcare (PervasiveHealth), 2012 6th International Conference on, pages 252–259. IEEE.
  136. Ó’Conaire, C., O’Connor, N. E., Smeaton, A. F., and Jones., G. J. (2007). Organising a daily Visual Diary Using Multi-Feature Clustering. In SPIE Electronic Imaging – Multimedia Content Access: Algorithms and Systems (EI121), San Jose, CA. O’Connor, E., Hayes, J., Smeaton, A. F., O’Connor, N. E., and Diamond, D. (2009). Environmental monitoring of galway bay: fusing data from remote and in-situ sources. In SPIE Europe Remote Sensing, pages 74780X– 74780X. International Society for Optics and Photonics.
  137. O’Connor, M. F., Conroy, K., Roantree, M., Smeaton, A. F., and Moyna, N. M. (2009). Querying XML data streams from wireless sensor networks: An evaluation of query engines. In Research Challenges in Information Science, 2009. RCIS 2009. Third International Conference on, pages 23– 30. IEEE.
  138. O’Hara, K., Tuffield, M., and Shadbolt, N. (2009). Lifelogging: Privacy and empowerment with memories for life. In Identity in the Information Society, volume 1.
  139. O’Hare, N., Gurrin, C., Jones, G. J., and Smeaton., A. F. (2005). Combination of content analysis and context features for digital photograph retrieval. In 2nd IEE European Workshop on the Integration of Knowledge, Semantic and Digital Media Technologies, pages 323–328, Washington, DC, USA. IEEE Computer Society.
  140. O’Hare, N., Gurrin, C., Jones, G. J. F., Lee, H., O’Connor, N. E., and Smeaton, A. F. (2007). Using text search for personal photo collections with the mediassist system. In SAC ’07: Proceedings of the 2007 ACM symposium on Applied computing, pages 880–881, New York, NY, USA. ACM Press.
  141. O’Loughlin, G., Cullen, S., McGoldrick, A., O’Connor, S., Blain, R., O’Malley, S., and Warrington, G. D. (2013). Using a Wearable Camera to Increase the Accuracy of Dietary Analysis. American Journal of Preventive Medicine, 44(3):297–301.
  142. Oneata, D., Douze, M., Revaud, J., Jochen, S., Potapov, D., Wang, H., Har- chaoui, Z., Verbeek, J., Schmid, C., Aly, R., McGuiness, K., Chen, S., O’Connor, N. E., Chatfield, K., Parkhi, O., Arandjelovic, R., Zisserman, A., Basura, F., and Tuytelaars, T. (2012). AXES at TRECVid 2012: KIS, INS, and MED. In TRECVID Workshop, Gaithersburg, Md., USA.
  143. Page, L., Brin, S., Motwani, R., and Winograd, T. (1999). The pagerank citation ranking: Bringing order to the web.
  144. Pauly-Takacs, K., Moulin, C. J., and Estlin, E. J. (2011). SenseCam as a rehabilitation tool in a child with anterograde amnesia. Memory, 19(7):705– 712.
  145. Petrelli, D. and Whittaker, S. (2010). Family memories in the home: con- trasting physical and digital mementos. Personal Ubiquitous Comput., 14(2):153–169.
  146. Price, B. A. (2010). Challenges in eliciting privacy and usability requirements for lifelogging. ACM.
  147. Qiu, Z., Doherty, A. R., Gurrin, C., and Smeaton, A. F. (2010). Turning raw sensecam accelerometer data into meaningful user activities. In SenseCam 2010: Proceedings of the Second Annual SenseCam Symposium, Dublin, Ireland.
  148. Qiu, Z., Gurrin, C., Doherty, A., and Smeaton, A. F. (2012). A Real-Time life experience logging tool. Advances in Multimedia Modeling, pages 636–638.
  149. Reddy, S., Parker, A., Hyman, J., Burke, J., Estrin, D., and Hansen, M. (2007a). Image browsing, processing, and clustering for participatory sens- ing: lessons from a dietsense prototype. In EmNets’07: Proceedings of the 4th workshop on Embedded networked sensors, pages 13–17, Cork, Ireland. ACM Press.
  150. Reddy, S., Parker, A., Hyman, J., Burke, J., Estrin, D., and Hansen, M. (2007b). Image browsing, processing, and clustering for participatory sens- ing: lessons from a dietsense prototype. In EmNets ’07: Proceedings of the 4th workshop on Embedded networked sensors, pages 13–17, New York, NY, USA. ACM.
  151. Richardson, M. and Domingos, P. (2002). The intelligent surfer: Probabilistic combination of link and content information in pagerank. In Advances in Neural Information Processing Systems ’14, volume 14.
  152. Riedl, M. O. and Young, R. M. (2010). Narrative planning: Balancing plot and character. Journal of Artificial Intelligence Research, 39(1):217–268.
  153. Roantree, M., Shi, J., Cappellari, P., O’Connor, M., and Moyna, N. M. (2012). Data transformation and query management in personal health sensor net- works. Journal of Network and Computer Applications, 35(4):1191–1202.
  154. Rogers, E. M. (2003). Diffusion of Innovations. The Free Press, 5 edition.
  155. Rubner, Y., Tomasi, C., and Guibas, L. J. (2000). The earth mover’s distance as a metric for image retrieval. Int. J. Comput. Vision, 40(2):99–121.
  156. Sadlier, D. A. and O’Connor, N. E. (2005). Event detection in field sports video using audio-visual features and a support vector machine. Circuits and Systems for Video Technology, IEEE Transactions on, 15(10):1225– 1233.
  157. Salembier, P. and Sikora, T. (2002). Introduction to MPEG-7: Multimedia Content Description Interface. John Wiley & Sons, Inc., New York, NY, USA.
  158. Scherp, A., Jain, R., Kankanhalli, M., and Mezaris, V., editors (2010). EiMM ’10: Proceedings of the 2nd ACM international workshop on Events in mul- timedia, New York, NY, USA. ACM. 433107.
  159. Schoeffmann, K., Ahlström, D., Bailer, W., Cobârzan, C., Hopfgartner, F., McGuinness, K., Gurrin, C., Frisson, C., Le, D.-D., Fabro, M., Bai, H., and Weiss, W. (2013). The video browser showdown: a live evaluation of inter- active video search tools. International Journal of Multimedia Information Retrieval, pages 1-15.
  160. Sellen, A. J., Fogg, A., Aitken, M., Hodges, S., Rother, C., and Wood, K. (2007). Do life-logging technologies support memory for the past?: an exper- imental study using SenseCam. In CHI’07: Proceedings ofthe SIGCHIcon- ference on Human factors in computing systems, pages 81–90, New York, NY, USA. ACM.
  161. Sellen, A. J. and Whittaker, S. (2010). Beyond total capture: a constructive critique of lifelogging. Comm. ACM, 53(5):70–77.
  162. Shah, M., Mears, B., Chakrabarti, C., and Spanias, A. (2012). Lifelogging: Archival and retrieval of continuously recorded audio using wearable de- vices. In Emerging Signal Processing Applications (ESPA), 2012 IEEE International Conference on, pages 99–102.
  163. Silva, A. R., Pinho, S., Macedo, L., and Moulin, C. J. (2013). Does SenseCam improve general cognitive performance? Am J Prev Med, 44(3):302–307.
  164. Sinha, P. and Jain, R. (2008). Semanics in digital photos: A contextual anal- ysis. International Journal of Semantic Computing, 2(3):311–325. Smeaton, A. F. and Browne., P. (2006). A usage study of retrieval modal- ities for video shot retrieval. Information Processing and Management, 42(5):1330–1344. Smeaton, A. F., Over, P., and Doherty, A. R. (2010). Video shot boundary detection: Seven years of TRECVid activity. Computer Vision and Image Understanding, 114(4):411–418. Smeaton, A. F., Over, P., and Kraaij, W. (2006). Evaluation campaigns and TRECVid. In Proceedings of the 8th ACM international workshop on Mul- timedia information retrieval, MIR ’06, pages 321–330, Santa Barbara, Cal- ifornia, USA. ACM. Smeaton, A. F., Over, P., and Kraaij, W. (2009). High-level feature detec- tion from video in TRECVid: a 5-year retrospective of achievements. In Divakaran, A., editor, Multimedia Content Analysis, Theory and Applica- tions, pages 151–174. Springer Verlag, Berlin. Smeulders, A. W., Worring, M., Santini, S., Gupta, A., and Jain, R. (2000). Content-based image retrieval at the end of the early years. IEEE Trans- actions on Pattern Analysis and Machine Intelligence, 22:1349–1380.
  165. Snoek, C. G. M. and Worring, M. (2009). Concept-based video retrieval. Foundations and Trends in Information Retrieval, 2(4):215–322.
  166. Snoek, C. G. M., Worring, M., van Gemert, J. C., Geusebroek, J.-M., and Smeulders, A. W. M. (2006). The challenge problem for automated detec- tion of 101 semantic concepts in multimedia. In Proceedings of the 14th annual ACM international conference on Multimedia, MULTIMEDIA ’06, pages 421–430, New York, NY, USA. ACM.
  167. Spence, R. (2002). Rapid, serial and visual: A presentation technique with potential. Information Visualization, 1(1):13–19.
  168. Stanley, K. G. and Osgood, N. D. (2011). The potential of sensor-based monitoring as a tool for health care, health promotion, and research. The Annals of Family Medicine, 9(4):296–298.
  169. Starner, T., Mann, S., Rhodes, B., Levine, J., Healey, J., Kirsch, D., Picard, R. W., and Pentland, A. (1997). Augmented reality through wearable com- puting. Presence: Teleoperators and Virtual Environments, 6(4):386–398.
  170. Stein, E. W. (1995). Organization memory: Review of concepts and recom- mendations for management. International Journal of Information Man- agement, 15(1):17–32.
  171. Stix, G. (2011). Photographic memory: Wearable cam could help patients stave off effects of impaired recall. Scientific American.
  172. Takahashi, M., Ito, S., Sumi, Y., Tsuchikawa, M., Kogure, K., Mase, K., and Nishida, T. (2004). A layered interpretation of human interactions cap- tured by ubiquitous sensors. In Proceedings of the the 1st ACM workshop on Continuous archival and retrieval of personal experiences, CARPE’04, pages 32–38, New York, NY, USA. ACM.
  173. Tancharoen, D. and Aizawa, K. (2004). Novel concept for video retrieval in life log application. In PCM: Pacific Rim Conference on Multimedia, pages 915–923.
  174. Tancharoen, D., Yamasaki, T., and Aizawa, K. (2005). Practical experience recording and indexing of life log video. In CARPE ’05: Proceedings of the 2nd ACM workshop on Continuous archival and retrieval of personal experiences, pages 61–66, New York, NY, USA. ACM.
  175. Tancharoen, D., Yamasaki, T., and Aizawa, K. (2006). Practical life log video indexing based on content and context. In Multimedia Content Analysis, Management, and Retieval: In Proceedings ofSPIE-IST Electronic Imaging.
  176. Taneja, H. and Gupta, R. (2010). Web information retrieval using query independent page rank algorithm. In Advances in Computer Engineering (ACE), 2010 International Conference on, pages 178–182.
  177. Troiano, R. P., Berrigan, D., Dodd, K. W., Masse, L. C., Tilert, T., and McDowell, M. (2008). Physical activity in the United States measured by accelerometer. Med Sci Sports Exerc, 40:181–188.
  178. van den Hoven, E., Sas, C., and Whittaker, S. (2012). Introduction to this special issue on designing for personal memories: Past, present, and future. Human-Computer Interaction, 27(1-2):1–12.
  179. Vemuri, S. and Bender, W. (2004). Next-generation personal memory aids. BT Technology Journal, 22(4):125–138.
  180. Vemuri, S., Schmandt, C., Bender, W., Tellex, S., and Lassey, B. (2004). An audio-based personal memory aid. In Ubicomp, pages 400–417.
  181. Vicedo, J. L. and Gómez, J. (2007). Trec: Experiment and evaluation in infor- mation retrieval: Book reviews. J. Am. Soc. Inf. Sci. Technol., 58(6):910– 911.
  182. Walter, C. (2005). Kryder’s law. Scientific American, 293(2):32–33. Wang, P. and Smeaton, A. F. (2011). Aggregating semantic concepts for event representation in lifelogging. In Proceedings of the International Workshop on Semantic Web Information Management, SWIM ’11, pages 8:1–8:6, New York, NY, USA. ACM.
  183. Wang, P. and Smeaton, A. F. (2012). Semantics-based selection of everyday concepts in visual lifelogging. International Journal of Multimedia Infor- mation Retrieval, 1:87–101.
  184. Wang, Z., Hoffman, M. D., Cook, P. R., and Li, K. (2006). Vferret: content- based similarity search tool for continuous archived video. In CARPE ’06: Proceedings of the 3rd ACM workshop on Continuous archival and retrival of personal experences, pages 19–26, New York, NY, USA. ACM.
  185. Watson, H. J. and Wixom, B. H. (2007). The current state of business intel- ligence. Computer, 40(9):96–99. Whittaker, S., Kalnikait˙e, V., Petrelli, D., Sellen, A., Villar, N., Bergman, O., Ilan, B., Clough, P., Brockmeier, J., and Whittaker, S. (2012). Socio- technical lifelogging: Deriving design principles for a future proof digital past. Human-Computer Interaction (Special Issue on Designing for Per- sonal Memories), 27(1-2):37.62.
  186. Woodman, O. and Harle, R. (2008). Pedestrian localisation for indoor envi- ronments. In Proceedings ofthe 10th international conference on Ubiquitous computing, UbiComp ’08, pages 114–123, New York, NY, USA. ACM.
  187. Yang, Y., Zhou, R., and Gurrin, C. (2012). A mechanical memory: Proto- type digital memories in everyday devices. In 6th Irish Human Computer Interaction Conference (iHCI2012), Galway, Ireland.
  188. Yeung, M. and Yeo, B.-L. (1996). Time-constrained clustering for segmenta- tion of video into story units. Pattern Recognition, 1996., Proceedings of the 13th International Conference on, 3:375–380.
  189. Zacks, J. M., Speer, N. K., Vettel, J. M., and Jacoby, L. L. (2006). Event Un- derstanding and Memory in Healthy Aging and Dementia of the Alzheimer Type. Psychology and Aging, 21(3):466–482.
  190. Zacks, J. M. and Tversky, B. (2001). Event structure in perception and con- ception. Psychological Bulletin, 127(1):3–21.
  191. Zhang, S., Rowlands, A. V., Murray, P., and Hurst, T. L. (2012). Physical activity classification using the GENEA wrist-worn accelerometer. Med Sci Sports Exerc, 44(4):742–748.
  192. Zheng, Y.-T., Zhao, M., Song, Y., Adam, H., Buddemeier, U., Bissacco, A., Brucher, F., Chua, T.-S., and Neven, H. (2009). Tour the world: Build- ing a web-scale landmark recognition engine. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, (CVPR), pages 1085–1092.
  193. Zhou, L., Caprani, N., Gurrin, C., and O’Connor, N. E. (2013). ShareDay: A novel lifelog management system for group sharing. In Li, S., Saddik, A., Wang, M., Mei, T., Sebe, N., Yan, S., Hong, R., and Gurrin, C., ed- itors, Advances in Multimedia Modeling, volume 7733 of Lecture Notes in Computer Science, pages 490–492. Springer Berlin Heidelberg.