Wednesday, January 9, 2013

Hypothesis Generation Systems

Imagine you're tracking something about your health. Say it's your weight. You have weight measurements from every day and you want to generate some hypotheses to test, like "my weight this month is significantly higher than last month." Or say you're tracking how many steps you take each day, and want to know some things about how that correlates with when you woke up, or any number of other data streams you've got.

I'm picturing some kind of visual analysis tool. Something that would help with hypothesis generation and feature selection. What's in this space? I started with the 150 most popular tools on the Quantified Self guide, as well as all 204 tools on Ian Li's personalinformatics.org. Some things I found:

Systems that pull in data from a lot of places and generate visualizations
Notch pulls in data streams from (currently) Fitbit, Runkeeper, and Bodymedia. Then it generates some pretty basic (although definitely pretty) visualizations.
Cosm (formerly Pachube) lets you look at a bunch of graphs together (one example here). Doesn't do any statistics, though, AFAICT. Focuses on Arduino/Internet-of-Things applications.
Trackify seems nonexistent.
Fluxtream seems promising, but I don't have access yet.
Also The Locker Project (Memo to myself: they seem to be very connected with Singly, which looks like a way to easily integrate apps with lots of social and/or quantified-self applications), TheCarrot, DataDepot
These all would be much cooler if they did stats (although just visualization is useful too!)

Tools that let you track things yourself by hand (and then they generate graphs)
ChartMyself (meant for QS-type use), TallyZoo (for anything), MycrocosmDaytum (anything), MercuryApp (mood-ish things), Dayta, Tonic, DidThis (things you've done), rTracker, Graphomatic, DailyDiary, LifeMee (maybe? signup is broken), Limits, DailyTracker, Grafitter, your.flowingdata, lifemetric (for many people)
These all seem like not what I'm looking for. The data entry is manual, and the output is still mostly visualization.

There's Quantified Mind, which apparently shows you all the significant results in your mental-test data automatically. Only for mental-test data, but still, this is the kind of thing I'm looking at.

There's media annotation and editing tools as jumping-off points, if you wanted to build this. Elan is one, but it's based on the quirky Java Media Framework; Pitivi is another, but for Linux only.

Then there's the whole field of information visualization, big-data type stuff (recent startup Trifacta comes to mind), which I'm just starting to dig in to.

More to come! There's a conference called IEEE VIS (formerly VAST) that might have some interesting stuff, I'll trawl through that. Also, any ideas you have would be quite welcome.