Designing for Quantified Self: The Moment -Phase 1 (as of December, 2013)
(2014-2015 in collaboration with the MIT Media Lab) Role: Research, UI, UX, Prototyping, Testing

The Moment is an application aimed to help people with depression or bipolar disorder to monitor their emotional ups and downs, reveal their emotional patterns, and eventually find a peaceful way to live with their emotions, rather than fighting with them. The system consists of two main components: A smart-phone application for the users to track their feelings and memories about events, and a sensor recording their physiological response. The data will then be visualized in several ways and available to be shared with their trusted ones or doctors. The system is designed to make the user more aware of their mood swings and the precursors to them; to reveal patterns about what could lead to emotional ups and downs, including physiological data such as arousal and sleep habits; to provide a record for healthcare providers; to build a library of personalized interventions for future use; and to create an effective network of social supports.

Why
For people with depression or bi-polar disorder, a proper and positive intervention is needed and helpful. However, an effective intervention should not be universal, because everyone has his/her unique experiences. A customized intervention based on these unique experiences will be of significant help. This project will try to design these personalized interventions with multiple data inputs including self-assessments, biological records(EDA) and daily activities records.

Target User: People with depression or bi-polar disorder.

Project Goal:

- Find patterns between self-reports and biological data
- Create personalized intervention
- Explore new ways to visualize data about one's emotion
- Improve the experience of self-reporting and self-assessment of emotion status.

Theoretical Context and Previous Researches

Positive psychology is currently emerging as a new way of behavioral changes by amplifying the positive value in life as opposed to fix the negative experiences. This idea has lead me to think about how I can change the way I live with my emotional episodes with the rapidly improving technology — especially our smartphones and wearable devices, things that we carry with us everyday on the go. Previous research about behavioral intervention technologies have been showing promising opportunities of psychological intervention strategies by using digital media to target behaviors, cognitions, and emotions in support of physical and mental health. As an example, CalmMeNow, which uses mobile devices for stress mitigation based on three types of interventions, including social networking, has shown great show great potential for interventions. Researches on self-reporting and logging systems has been showing more opportunities with wearable devices integrated into the design of intervention strategies. Researches on emotional expressivity has offered valuable qualities for designing self-reporting interfaces for emotional activities. From reported clinical experience of bipolar disorder sufferers, a proper and positive intervention is can be helpful for treating symptoms. From our user studies, however, we found an effective intervention should not be universal because everyone has his/her unique experiences that are different from one another. A personalized intervention based on their annotations and sensor data and their unique experiences will be of significant help to discover what could lead to or improve emotional ups and downs as personalized interventions when needed. In addition to these personalized interventions, forming connections with trusted and/or closed ones is another feature that helps the user when experiencing depression as social interactions play an important role in intervening mental health.

Contextual Reasearch

Relevant Technology and Research

  • Jennifer Healey et al, Out of the Lab and into the Fray: Towards Modeling Emotion in Everyday Life
  • Anna Ståhl,Designing for Emotional Expressivity
  • Dacher Keltner et al,Functional Accounts of Emotions
  • Jamie Ward,Emotional Mediated Synaesthesia
  • James W. Pennebaker and Cindy K. Chung , Expressive Writing: Connections to Physical and Mental Health
  • Pablo Paredes et al,CalmMeNow: Exploratory Research and Design of Stress Mitigating Mobile Interventions
  • Pedro Sanches et al,Mind the Body! Designing a Mobile Stress Management Application Encouraging Personal Reflection
  • Anna Ståhl et al,Reflecting on the Design Process of the Affective Diary
  • Dan Morris et al,SuperBreak: Using Interactivity to Enhance Ergonomic Typing Breaks
  • Elsa Kosmack Vaara et al,Temporal Relations in Affective Health
  • Akane Sano et al,Autonomic Sleep Patterns in Visual Discrimination Task Improvement
  • Yadid Ayzenberg et al,FEEL: Frequent EDA and Event Logging – A Mobile Social Interaction Stress Monitoring System
  • Akane Sano et al,Stress Recognition using Wearable Sensors and Mobile Phones
  • Robert LiKamWa et al,Can Your Smartphone Infer Your Mood?

Existing Similar Applications

A) Interface: Input

A lot of them use emoji-like interface for input, some of them use slide bars, some of them give words to choose, some of them have you type in your mood. None of them are using a color scheme.

B) Interface: Logs:
In most of them, the logs are displayed visually in more than one form, including, timelines, numbers or charts of an overview, The Moment has both timeline and overview, (but the overview is still in process). And most importantly, the moment includes physiological data which is from the Q sensor, so the user get to compare those two. This is important because sometimes we exaggerate our feelings.

C) Social Supports
I personally feel this is very important because supports from friends and closed ones can be very powerful. But very few of the current apps have this feature.

D) Interventions
Very few of them generate interventions, and only oneof them generate "personalized" interventions. I'm still working on this part but I believe it's feasible as long as the app collects enough data.


How does The Moment work?

The system consists of 2 main components, 1 Q sensor worn by the user, and an app on the user's iphone. The Q sensor monitors the user's EDA and sends the data to the iPhone app. Besides, the user can take notes of her feelings and activities and do some simple self-assessments on the app.


Initial Idea


Interface

 

The top part is self-report data while bottom part is the data from Q sensor.

 


Data analysis and visulization

A) Get the data from Q sensor and make markers with Q Software

B) Export the data to csv format and open it with Microsoft Excel

C) Calculate the points at each range, then translate the points into time duration

D) The background color on the bottom part ( Interface ) is the mean value of one day's data. There are 6 shades of greay representing different energy intensity.

E) Generate the histogram. The x axis is the EDA value and y is the time duration (Here the unit is a minute). This histogram shows up when you tap on a dot on the timeline ( Interface ), tells you the EDA value and where it is on a day-based histogram.

F) The dots represent specific peaks collected by the Q sensor, and are mapped to the self-reporting logs on the timeline. (Interface ). The bigger the dot is, the higher the EDA value is.

G)The Q sensor records the EDA during your sleep. Thus, you can get a quick idea of the quality of your sleep on the timeline in addition to the logs.

*The ongoing research includes the relationship between sleep quality and emotions experienced on the next day. Data from a longer period of time is required for more detailed discussion.


Problems encountered

About Data:
  • Time: Data da files don't always start at midnight, and sometimes I got a long over-night file, while sometimes I got more than 10 small pieces in a day. It's difficult to calculate the data in this case.
  • Time: I never go to bed a midnight, and sometimes I stay overnight while sometimes I sleep whole day, so it's a problem to define "one-day" because it's apparently not 24 hours.
  • Tool: I use excel to analyze data, and it has limits. And very often the eda file exceed the limits.
  • EDA doesn't show "negative" value: If I am very depressed, eda wouldn't be able to show how negative I am, it would look like I am at ease.
About Interface:
  • From the usability test, most of the users feel the color scheme is intuitive. If they are confused, there's a map for their reference. However, there're some users, although very few, said this map doesn't fit their feelings. For example I define yellow as happy and positive, but some think blue is happy. So I'm working on a "personalized" map.
  • Need to find a better way to show sleeping data


Behind the scenes...



Still working on...

Social Supports:
  • How to connect friends and/or closed ones? Need to check Facebook api or other possible source.
Personalized Intervention:
  • Working with Chloe Mun Yee Kwan from Department of Psychological Clinical Science at North Dakota State University.
  • Need to develop an algorithm for recognizing the situation that triggers intervention
  • What kind of intervention would appears? (For example if I got too many green-blue-ish in 2 days, a picture that I took at a happy moment would pop up, or if I got too many deep blue in a week, send notification to my friends. )
Other Ways of Visualization:
  • A month overview
  • An year overview
  • Sleeping data

Special thanks to Rosalind W. Picard, Brian Lucid, Yu-Wei Chang, Akane Sano, Chloe Mun Yee Kwan, Javier Hernandez, Tony Tong, Yadid Ayzenberg.