Planning vs. Experimentation
By Arnold Kling
it is simple to run a controlled experiment with a Web page. Amazon can show a different page layout to every hundredth visitor and determine in a few days whether the new design increases sales.
Similarly, a search engine can run a controlled experiment to try out a new tweak to its search algorithm and discover in a few hours whether users find it an improvement on the old algorithm. On the Web, continuous improvement really is continuous.
…Old media just do not understand online kaizen. Their perceptions are tied to the print world, where design changes are costly. The Wall Street Journal spent years planning its recent redesign of the print edition and millions of dollars rolling it out. Yet it will be months before it becomes clear how successful these changes were.
The experimental mindset is more like direct mail than traditional mass media. Direct mail marketers experiment with different types of envelopes, different pitches, different offers, and so on.