E-Log

The E-log project aims to develop the back-end event-base system for life logging application domain that covers healthcare, fitness, surveillance, security and enterprise management.

memex_application.png

E-log_image_samples.png Logo_Marie-Curie.jpg

The origin of the E-log stems from the MEMEX (a compound word of “memory” and “index”) system proposed by Vannevar Bush in 1945. MEMEX refers the theoretical proto-hypertext computer system in which an individual compresses and stores all of their books, records, and communications which is then mechanized so that it may be consulted with exceeding speed and flexibility. Its necessity in the modern era is still growing over wide application domains and the rapid progress of computer systems and mobile devices makes it more realistic to implement. Technically we can now record when and where we are, what we do and what we see in real-time and we will see in this report in which application domains market demands mostly dominate.

memex.png

The concept of the MEMEX system can be broadly applied on the personal digital collection like MyLifeBit project of Microsoft, to the military application like ASSIST project of USA DARPA, to the public security that needs to monitor and record suspicious activities and report illegal events, and to health-care services that need to monitor patients, infants or the aged. Also, smart-phones are getting very popular and their embedded sensors such as cameras, GPS sensors or microphones can record user status in real-time and send it to anywhere by wireless network. They are becoming the strong and solid foundation for the MEMEX-like system development.

memex_research.png

The E-log project is a sort of life logging project while the research focus is more set on the analysis of life events pouring efforts on designing the information management system easy to archive the life data and supports the query of real human who are interested in retrieving high-level events. This needs the mathematical formulation of real-life events with the foundation of the way that can treat digitized data objects in a unified way to permit the traversal of complex relations to retrieve all events of user’s interests.

memex_system_factors.png

  • Collaboration
  • Technical Meeting 2011-02-16 Presentation by Pil Ho Kim on the current status of e-Log project Location: Ofek 1, POVO 1 Attendants: Pil Ho Kim (pilho.kim@disi.unitn.it) Minh-Son Dao (minhson.dao@disi.unitn.it) Riccardo Mattivi <rmattivi@disi.unitn.it> Mattia <broilo@disi.unitn.it> Thien <dangnguyen@disi.unitn.it> jacopostaiano <jacopostaiano@gmail.com>, Ivan Tankoyeu <tankoyeu@disi.unitn.it>, Pierre Andrews <andrews@disi.unitn.it> Simon Andersson <andersson@disi.unitn.it> 2011-02-22 Attendants: Minh-Son Dao (PhD in Prof. Francesco De Natale’s group), Pil Ho Kim Agenda: Research object: Event recognition from continuous image streams of personal life using various metadata including GPS and SN activities.

  • Communication
  • Contacts: pilhokim AT gmail DOT com People I met The e-Log project has been getting the broad attention from many people in various fields. If you are interested, please reach me by email. I would love to share my experiences on the e-Log project. I have been explaining the concept and the progress of e-Log to: Dr. Ramesh Jain, Donald Bren professor Donald Bren School of Information and Sciences, University of Irvine, USA.

  • Data Structure
  • This pages manages the most recent e-Log data structure – mostly focused on the detail structure of logging data objects. Data structure iPhone iPhone photos CREATE TABLE `iphone_photos` ( `filename` varchar(255) NOT NULL, `iphone_unix_timestamp` bigint(20) NOT NULL DEFAULT '0', `content` mediumblob, `iphone_timestamp` timestamp NULL DEFAULT NULL, `iphone_UTC_timestamp` timestamp NULL DEFAULT NULL, PRIMARY KEY (`filename`,`iphone_unix_timestamp`), KEY `timestamp_key` (`iphone_timestamp`), KEY `unix_timestamp_index` (`iphone_unix_timestamp`), KEY `filename` (`filename`), KEY `sensecam_timestamp` (`iphone_timestamp`,`filename`), KEY `image_UTC_timestamp` (`iphone_UTC_timestamp`) ) ENGINE=MyISAM DEFAULT CHARSET=utf8 AVG_ROW_LENGTH=21204; iPhone photo EXIF information CREATE TABLE `iphone_photos_exif` ( `filename` varchar(100) NOT NULL, `iphone_unix_timestamp` bigint(20) NOT NULL DEFAULT '0', `section` varchar(50) NOT NULL DEFAULT '', `section_key` varchar(50) NOT NULL DEFAULT '', `key_value` text, PRIMARY KEY (`filename`,`iphone_unix_timestamp`,`section`,`section_key`), KEY `name` (`section`), FULLTEXT KEY `value` (`key_value`) ) ENGINE=MyISAM DEFAULT CHARSET=utf8 AVG_ROW_LENGTH=21204; iPhone GPS CREATE TABLE `iphone_gps` ( `latitude` double NOT NULL, `longitude` double NOT NULL, `elevation` double NOT NULL, `gps_timestamp` timestamp NOT NULL DEFAULT '0000-00-00 00:00:00', `gps_timezone` varchar(255) DEFAULT NULL, `gps_UTC_timestamp` timestamp NULL DEFAULT NULL, PRIMARY KEY (`latitude`,`longitude`,`elevation`,`gps_timestamp`), KEY `timestamp_key` (`gps_timestamp`), KEY `latitude` (`latitude`), KEY `longitude` (`longitude`), KEY `elevation` (`elevation`), KEY `gps_timestamp` (`gps_timestamp`), KEY `gps_timezone` (`gps_timezone`), KEY `gps_UTC_timestamp` (`gps_UTC_timestamp`) ) ENGINE=MyISAM DEFAULT CHARSET=latin1; Communication GMail IMAP CREATE TABLE `gmail_allmail` ( `gmail_msgno` int(11) unsigned NOT NULL DEFAULT '0', `gmail_subject` varchar(255) DEFAULT NULL, `gmail_from` varchar(255) DEFAULT NULL, `gmail_to` varchar(255) DEFAULT NULL, `gmail_UTC_date` datetime DEFAULT NULL, `gmail_message_id` tinytext, `gmail_references` text, `gmail_in_reply_to` text, `gmail_size` int(11) DEFAULT NULL, `gmail_uid` int(11) DEFAULT NULL, `gmail_recent` int(11) DEFAULT NULL, `gmail_flagged` int(11) DEFAULT NULL, `gmail_answered` int(11) DEFAULT NULL, `gmail_deleted` int(11) DEFAULT NULL, `gmail_seen` int(11) DEFAULT NULL, `gmail_draft` int(11) DEFAULT NULL, `gmail_body` longtext, `gmail_body_striped` longtext, PRIMARY KEY (`gmail_msgno`), KEY `gmail_uid` (`gmail_uid`), KEY `gmail_to` (`gmail_to`), KEY `gmail_from` (`gmail_from`), FULLTEXT KEY `gmail_subject` (`gmail_subject`), FULLTEXT KEY `gmail_body` (`gmail_body`), FULLTEXT KEY `gmail_message_id` (`gmail_message_id`), FULLTEXT KEY `gmail_references` (`gmail_references`), FULLTEXT KEY `gmail_in_reply_to` (`gmail_in_reply_to`), FULLTEXT KEY `gmail_body_striped` (`gmail_body_striped`) ) ENGINE=MyISAM DEFAULT CHARSET=utf8; Phone record CREATE TABLE `phone_record` ( `id` int(11) NOT NULL AUTO_INCREMENT, `call_type` varchar(255) DEFAULT NULL, `call_name` varchar(255) DEFAULT NULL, `call_phone number` varchar(255) DEFAULT NULL, `call_duration` varchar(255) DEFAULT NULL, `call_timestamp` timestamp NULL DEFAULT '0000-00-00 00:00:00', `call_date` varchar(255) DEFAULT NULL, `call_time` varchar(255) DEFAULT NULL, `call_am_pm` varchar(3) DEFAULT NULL, `call_message` varchar(255) DEFAULT NULL, `call_date_backup` varchar(255) DEFAULT NULL, `call_UTC_timestamp` timestamp NULL DEFAULT NULL, PRIMARY KEY (`id`) ) ENGINE=MyISAM AUTO_INCREMENT=244 DEFAULT CHARSET=utf8; People Sensecam people information face-identified from pictures CREATE TABLE `sensecam_people` ( `id` int(11) NOT NULL AUTO_INCREMENT, `name` char(100) NOT NULL, `filename` char(125) NOT NULL, `face` mediumblob, PRIMARY KEY (`id`), UNIQUE KEY `filename` (`filename`,`name`) ) ENGINE=MyISAM AUTO_INCREMENT=7115 DEFAULT CHARSET=utf8; Google contacts CREATE TABLE `gmail_contacts` ( `Name` varchar(255) DEFAULT NULL, `Given Name` varchar(255) DEFAULT NULL, `Additional Name` varchar(255) DEFAULT NULL, `Family Name` varchar(255) DEFAULT NULL, `Yomi Name` varchar(255) DEFAULT NULL, `Given Name Yomi` varchar(255) DEFAULT NULL, `Additional Name Yomi` varchar(255) DEFAULT NULL, `Family Name Yomi` varchar(255) DEFAULT NULL, `Name Prefix` varchar(255) DEFAULT NULL, `Name Suffix` varchar(255) DEFAULT NULL, `Initials` varchar(255) DEFAULT NULL, `Nickname` varchar(255) DEFAULT NULL, `Short Name` varchar(255) DEFAULT NULL, `Maiden Name` varchar(255) DEFAULT NULL, `Birthday` varchar(255) DEFAULT NULL, `Gender` varchar(255) DEFAULT NULL, `Location` varchar(255) DEFAULT NULL, `Billing Information` varchar(255) DEFAULT NULL, `Directory Server` varchar(255) DEFAULT NULL, `Mileage` varchar(255) DEFAULT NULL, `Occupation` varchar(255) DEFAULT NULL, `Hobby` varchar(255) DEFAULT NULL, `Sensitivity` varchar(255) DEFAULT NULL, `Priority` varchar(255) DEFAULT NULL, `Subject` varchar(255) DEFAULT NULL, `Notes` text, `Group Membership` varchar(255) DEFAULT NULL, `E-mail 1 - Type` varchar(255) DEFAULT NULL, `E-mail 1 - Value` varchar(255) DEFAULT NULL, `E-mail 2 - Type` varchar(255) DEFAULT NULL, `E-mail 2 - Value` varchar(255) DEFAULT NULL, `E-mail 3 - Type` varchar(255) DEFAULT NULL, `E-mail 3 - Value` varchar(255) DEFAULT NULL, `Phone 1 - Type` varchar(255) DEFAULT NULL, `Phone 1 - Value` varchar(255) DEFAULT NULL, `Phone 2 - Type` varchar(255) DEFAULT NULL, `Phone 2 - Value` varchar(255) DEFAULT NULL, `Phone 3 - Type` varchar(255) DEFAULT NULL, `Phone 3 - Value` varchar(255) DEFAULT NULL, `Phone 4 - Type` varchar(255) DEFAULT NULL, `Phone 4 - Value` varchar(255) DEFAULT NULL, `Phone 5 - Type` varchar(255) DEFAULT NULL, `Phone 5 - Value` varchar(255) DEFAULT NULL, `Address 1 - Type` varchar(255) DEFAULT NULL, `Address 1 - Formatted` varchar(255) DEFAULT NULL, `Address 1 - Street` varchar(255) DEFAULT NULL, `Address 1 - City` varchar(255) DEFAULT NULL, `Address 1 - PO Box` varchar(255) DEFAULT NULL, `Address 1 - Region` varchar(255) DEFAULT NULL, `Address 1 - Postal Code` varchar(255) DEFAULT NULL, `Address 1 - Country` varchar(255) DEFAULT NULL, `Address 1 - Extended Address` varchar(255) DEFAULT NULL, `Organization 1 - Type` varchar(255) DEFAULT NULL, `Organization 1 - Name` varchar(255) DEFAULT NULL, `Organization 1 - Yomi Name` varchar(255) DEFAULT NULL, `Organization 1 - Title` varchar(255) DEFAULT NULL, `Organization 1 - Department` varchar(255) DEFAULT NULL, `Organization 1 - Symbol` varchar(255) DEFAULT NULL, `Organization 1 - Location` varchar(255) DEFAULT NULL, `Organization 1 - Job Description` varchar(255) DEFAULT NULL, `Website 1 - Type` varchar(255) DEFAULT NULL, `Website 1 - Value` varchar(255) DEFAULT NULL, `Custom Field 1 - Type` varchar(255) DEFAULT NULL, `Custom Field 1 - Value` varchar(255) DEFAULT NULL, `Jot 1 - Type` varchar(255) DEFAULT NULL, `Jot 1 - Value` varchar(255) DEFAULT NULL ) ENGINE=MyISAM DEFAULT CHARSET=utf8; String normalization for comparison CREATE TABLE `people_name` ( `id` int(11) NOT NULL AUTO_INCREMENT, `source` varchar(100) DEFAULT NULL, `name` varchar(255) DEFAULT NULL, `normalized_name` varchar(255) DEFAULT NULL, `name_hex` varchar(255) DEFAULT NULL, `normalized_name_hex` varchar(255) DEFAULT NULL, PRIMARY KEY (`id`) ) ENGINE=MyISAM AUTO_INCREMENT=2571 DEFAULT CHARSET=utf8; See the usage

  • Documentation
  • This page archives the e-log documents. The first question from whom saw the e-log system might belike “What we can do with archived data? Does it deserve the efforts? Or does your system smart to answer my question per archived data?” Can’t be clear enough, just accumulating everything in a digitized form can never answer such question neither is that fantastic job to do everyday. We, as the researcher, should provide the answer clear enough for the user that their gain will be much higher than their investment by any means.

  • Download
  • This page will archive software codes or binaries opened for the public developed during the course of the e-log project. E-Log:K-mean GPS spatial database E-Log:K-mean GPS spatial classification code

  • iPhone Image Data Processing
  • Photos taken by iPhone (3G or more) embed the GPS location if the location service was turned on then. Thus let us extract all information including GPS. See the below PHP that I wrote. <?PHP // Utility to read data from SenseCam and put them into // the MySQL Database. // // Developed by Pil Ho Kim, PhD, UNITN. // Last update: November 9th, 2010 // // Below two functions are from http://stackoverflow.

  • K-mean GPS Points Classification
  • To meet the Google Geocoding API server limit, we use the OpenCV K-mean method to classify the GPS points into 2500 area. I slightly modified the OpenCV K-means algorithm to perform the hierarchical classification by space and time. Hierarchical GPS spatial clustering Prepare GPS data table See E-Log:K-mean GPS spatial database. Program codes See E-Log:K-mean GPS spatial classification code. Running results Hierarchical GPS spatial-temporal clustering To-do Extend the search using the Google Places API

  • K-mean GPS Spatial Classification Code
  • I developed the below code within the environment: OpenCV 2.2 Mac OS X 10.6.6 Xcode 3.2.5 64 bits GCC 4.2 /* * Hierchical K-mean algorithm for GPS spatial clustering * * Developed by Pil Ho Kim, 2011. * * History: * 2011-02-11 First revision * */ #include <iostream>#include <opencv.hpp>#include <sstream>#include <fstream>#include <vector>#include <string>#include <unistd.h> using namespace std; char m_sSourceFile[1024]; char m_sTargetFile[1024]; char m_sRegionFile[1024]; #define ID_MAX 0 #define ID_MIN 1 #define ID_LAT 0 #define ID_LOG 1 int m_iRegionID = 0; #define CLUSTER_MIN_RADIUS 0.

  • K-mean GPS Spatial Database
  • Table schemas CREATE TABLE `iphone_gps` ( `latitude` double NOT NULL, `longitude` double NOT NULL, `elevation` double NOT NULL, `gps_timestamp` timestamp NOT NULL DEFAULT '0000-00-00 00:00:00', `gps_timezone` varchar(255) DEFAULT NULL, `gps_UTC_timestamp` timestamp NULL DEFAULT NULL, `gps_point` geometry NOT NULL, PRIMARY KEY (`latitude`,`longitude`,`elevation`,`gps_timestamp`), KEY `timestamp_key` (`gps_timestamp`), KEY `latitude` (`latitude`), KEY `longitude` (`longitude`), KEY `elevation` (`elevation`), KEY `gps_timestamp` (`gps_timestamp`), KEY `gps_timezone` (`gps_timezone`), KEY `gps_UTC_timestamp` (`gps_UTC_timestamp`), SPATIAL KEY `gps_point` (`gps_point`) ) ENGINE=MyISAM DEFAULT CHARSET=latin1; CREATE TABLE `iphone_gps_clusters` ( `parent_region_id` int(11) NOT NULL, `region_id` int(11) DEFAULT NULL, `latitude` double NOT NULL, `longitude` double NOT NULL, KEY `parent_region_id` (`parent_region_id`), KEY `region_id` (`region_id`), KEY `latitude` (`latitude`), KEY `longitude` (`longitude`) ) ENGINE=MyISAM DEFAULT CHARSET=latin1; CREATE TABLE `iphone_gps_region` ( `region_id` int(11) NOT NULL DEFAULT '0', `count` int(11) DEFAULT NULL, `radius_km` double DEFAULT NULL, `latitude_center` double DEFAULT NULL, `longitude_center` double DEFAULT NULL, `latitude_max` double DEFAULT NULL, `longitude_max` double DEFAULT NULL, `latitude_min` double DEFAULT NULL, `longitude_min` double DEFAULT NULL, PRIMARY KEY (`region_id`), KEY `count` (`count`) ) ENGINE=MyISAM DEFAULT CHARSET=latin1; CREATE TABLE `iphone_gps_region_hierarchy` ( `parent_region_id` int(11) NOT NULL, `region_id` int(11) NOT NULL DEFAULT '0', PRIMARY KEY (`parent_region_id`,`region_id`), KEY `parent_region_id` (`parent_region_id`), KEY `region_id` (`region_id`) ) ENGINE=MyISAM DEFAULT CHARSET=latin1; Queries Load clustered data After E-Log:K-mean GPS spatial classification code, load data from the text file.

  • Presentation
  • 2011/02/16 KnowDive seminar Abstract: The current status of e-log project collecting the personal life logs using ViconRevue and iPhone will be presented in the seminar. Experience from daily life logging will be shared with the audience with the emphasis on promising research topics. The talk includes the demo on 5 months of spatio-temporally correlated life logs and will discuss technical challenges in privacy protection, complex queries and life log system implementation.

  • Privacy Protection
  • Privacy concerns on life logs involve all sorts of collected data. Let us discuss all such issues and technical solutions on this page Image data For images taken by ViconRevue or iPhone, let us focus on the people inside rather than objects as Google does on the street view image data like below. Google detects faces from the scene and blurs them to protect the privacy. We may do the similar approach but the things to consider compared with Google is that photos have boundaries.

  • Research Focus
  • The E-log project aims to develop the back-end event-base system for life logging application domain that covers healthcare, fitness, surveillance, security and enterprise management. The origin of the E-log stems from the MEMEX (a compound word of “memory” and “index”) system proposed by Vannevar Bush in 1945. MEMEX refers the theoretical proto-hypertext computer system in which an individual compresses and stores all of their books, records, and communications which is then mechanized so that it may be consulted with exceeding speed and flexibility.

  • Status
  • This page records the current status of e-log archives with technical analysis. Data collection ViconRevue ViconRevue is the wearable digital camera designed to take photos passively without intervention whilst being worn by the user. Based on Microsoft SenseCam technology, Revue is a research tool aimed at researchers as an aid for people with memory loss. Specification Vicon Revue contains a 1230mAh lithium-ion battery that lasts enough for one-day continuous shot. It is built around a PIC microcontroller.

  • UTC Timezone
  • If we need to manage time-sensitive data coming from distant areas in different timezone, then we need a method to convert the local timestamp into a unified timezone. I selected coordinated universal time, or in short UTC. What we need is a database to retrieve the timezone from the given location information.This section develops the data table and the stored procedure to meet the above demand. First, create the the time_zone table and populate it using .