How Good Is Your Information Metabolism?

Information Metabolism. Ever heard of it?

I came across this phrase yesterday for the first time in the latest issue of Sloan Management Review. In an interesting interview about the changing role of IT in innovation, MIT Sloan Professor Erik Brynjolfsson says

smart companies have learned to tap the flood of data created by information technology and process it with what he calls a “higher information metabolism”.

I  like “information metabolism”. It sounds much more glamorous and important than “data analysis”.  That said, I am not sure I agree with the comment. Companies that are primarily online like Google have certainly demonstrated the ability to “metabolize” data at an impressive rate. However, if my experience with bricks-and-mortar companies is any indication, the majority are suffering from severe “information indigestion”.

Back to the interview. Prof. Brynjolfsson’s main thesis is that

IT is setting off a revolution in innovation on four dimensions simultaneously: measurement, experimentation, sharing, and replication. Each of these is important in and of itself, but, more profoundly, they reinforce each other.

Plausible. The importance of the “measurement” dimension is, of course, obvious for us Analytics types. It creates new, often much more detailed, data that we can use to solve old problems better and exploit new opportunities.

What about “experimentation”?

The big advantage of an experimental approach that uses IT is that you can get at causality in a way that you can’t with just pure measurement and observation. And that, of course, is the gold standard for being able to have actionable knowledge about what’s really happening in your business …

Have to agree. For knowledge that’s truly actionable, nothing beats what you get from a controlled experiment.

What we’re going to see in the coming decade are companies whose whole culture is based on continuous improvement and experimentation—not just of specific processes, but of the entire way the company runs. I think this revolution can be fairly compared to the scientific revolution that happened centuries ago.

That’s a grand statement. I like it – partly because I am a data-driven person and just love this sort of thing but mostly because this view, if it comes true, will drive incredible demand for analytics people for a long time to come.

He gives the example of casino chain Harrah’s as an example of such an organization.

Harrah’s runs dozens of experiments. For instance, they will see whether different kinds of discounts and coupons can entice people that normally come for two days to come for three days, or get people who normally bet the $5 machines to bet the $25 machines. They bring experimentation to figure out what work practices can get their waiters and waitresses to serve customers more effectively and get higher customer satisfaction scores. This is a mentality that they bring to every aspect of their business.

Impressive. That’s a lot of experiments.

I agree with and believe in many of the views expressed in the interview. However, I was VERY surprised by the lack of discussion regarding a key skill that’s needed to support the hyper-experimenting organization of the future: sophisticated analytical modeling.

Except for simple A/B tests, setting up and analyzing experiments takes a lot of modeling and analysis. A lot. When you believe that the response variables may be related in nonlinear ways to the treatment variables or when many treatment variables are involved, it can get complicated very quickly. To do all this, a high degree of statistical sophistication is needed in the organization (otherwise why would pharma companies employ biostatisticians?).

Without the stats and modeling skillset, only the simplest experiments will be done and there won’t be any enduring competitive edge to speak of.

In fact, I’d go so far as to say that the main reason most bricks-and-mortar organizations today suffer from “information digestion” is because they lack analytical modeling skills. Given this, if  they start running experiments, the gap between data volume and the ability to analyze it will grow even wider.

I think the right sequence for an aspiring “information metabolizer” is to increase their modeling bandwidth first, start analyzing their current data and demonstrate some value, and THEN start running experiments.

I like “Information metabolism”. But, just like in biological metabolism, key catalysts are needed to make it all happen. Analytical modeling is a critical catalyst. Let’s not forget about this in the rush to experiment.


52 thoughts on “How Good Is Your Information Metabolism?”

  1. Anthony Morrison’s genius was in recognizing these opportunities.
    With the step up from high school to college comes a commensurate increase in academic expectations.
    For the aspiring pilots out there the Bridge Limbo
    achievement is earned when you fly an airplane under 30
    unique bridges in Panau. To be the LIGHT and just do what she feels called to doing.
    The Lurcher is not a breed, as such, but more a type of dog.

Leave a Reply

Your email address will not be published. Required fields are marked *