Analytics, Big Data, and More Questions

March 15, 2013

I’ve been using various analytics tools for quite a bit with products I’ve built and clients I’ve worked with. They are the usual suspects: MixPanel, KISSmetrics, Google Analytics, New Relic, and so forth. But I think they all have one critical flaw: they all require you to know the question to ask.

At the end of the day, I don’t think any service has really tackled this problem. They all seem to want to focus on either getting more data in from more sources (iOS, Android, web, desktop), more customer interaction (push notifications, email messages, text messages), or dealing with just a much larger scale of data (social gaming, enterprise). These are all worthy problems to solve but they seem to avoid the biggest problem of all.

When I pipe all my product usage data and subscriptions data into a service, I still can’t say “find my most profitable users and show me a profile of what likely makes them fall into that category”. Then, have it show me maybe that 90% of them came from Reddit and they all created “Projects” within 4 hours of signing up and that they all upgraded on day 23. Potentially, I could connect my other services to this tool so we could see that I send a marketing mailer around then or released a new feature at that time.

What we do now is we have to propose the question already: “split up my users by sources”. “Hmm, wow, Reddit sends a lot of my paying customers over… I’m going to optimize for Reddit’s traffic.” But that misses the whole point of Big Data: having access to the cross-sections and complexities of multiple data sources.

I’ve been banging this drum for quite some time starting back when I worked at a Personal Finance Management tool. Especially in the financial world, there aren’t a lot of services that can take a ton of data and use it all at once to produce a good quality question and then give a great hypothesis.

I should be clear I’m not knocking what these tools do and the complexities that they take out of my hands as a customer. I personally use the tools I’ve listed here and would recommend them to most businesses.

It may be time to put up or shut up. This is definitely a hard problem to solve. But, unless I’m missing some key player who serves this niche, I feel like it’s 2013 and we were promised jet-packs and all we got were bigger databases.