By David Bowen
Financial Times; Apr 21, 2003
I spend my time helping organisations improve the effectiveness of their websites. And how, my clients ask, do you measure effectiveness? Or they don't, often, but they should, for in these budget-garrotting times the only way anyone will spend a bean is if the return can be measured.
It is not easy. When the commercial web first appeared, there was much excitement precisely because it seemed so measurable. Not only could you see how many people had visited your site, you could see exactly where they had been. And if you sold things online, you could get an even harder measure by counting your sales.
But it didn't quite work out like that. Some publishing sites have managed to generate auditable figures, but most organisations make little use of web statistics, for the simple reason that there are too many of them. E-commerce figures? Dell, famously successful online, sells much more via telephone and fax than it does from its website. Customers use the web en masse to educate themselves before buying; but the crucial sign of success for Dell is not higher web sales but fewer phone calls - nice hard numbers, but not so simple to measure.
So how can we measure the effectiveness of our sites? I am sceptical about "usability studies" that involve the general public. For one thing, few people are sufficiently aware of what is possible to be truly critical. They will struggle through, and feel that any difficulties they encounter are their own fault. For another, the studies are inevitably based on what a site is doing already, rather than what it might do.
Another sort of usability test is based on an analysis of the site, giving marks for different features. This can not only fail to test potential, but can also be downright dangerous. What happens if there is a catastrophic failure in one specific area, such as a broken search engine? That feature will receive no points, when it should be given minus 100: the catastrophe will barely be acknowledged.
Itis possible to put numbers on web effectiveness, but only by mixing in a generous dollop of selective judgment. We need to judge what might be done against what is being done; and that needs a fair understanding of the organisation and its goals, as well as the potential of the medium.
Take fitted kitchen manufacturers. You are not going to sell them online, so you should surely do your best to tempt web visitors to pick up the phone or visit a shop. MFI does this imaginatively, providing a step-by-step interactive planner that lets you plan your kitchen and then lures you cleverly into making contact. MFI has seen what is possible and executed it with style: top marks. Its rival Möben, by contrast, offers almost no details of its products, tiny pictures and, worst of all, a site slowed down by Flash animation software. It is a designer-driven brochure site that fails utterly to exploit the medium's potential: the return on investment is hard to spot.
Take corporations - what should they do with their central sites? As I pointed out last month, it is the companies that go beyond the basics - of media and investor relations and of recruitment - that show real imagination and deserve to be marked highly. It is fine that a corporation should serve investors well - but unless it uses the web in other ways too, it cannot be given a high overall rating.
More specifically, take Monsanto. I would expect it to use its site for "reputation management" - defending itself against the slings and arrows of outrageous anti-GM campaigners. It does not, so mark it down.
Or BT Group. As a small business manager, I want to locate services by telling BT my need, and to be led gently through to order online. This rarely happens. BT.com is much better than it used to be, but it has not got the hang of user-driven journeys. Again, lose marks.
Finally, take political parties. What can they do online? Provide a huge choice of policies? They don't want to do that. Provide an interactive space where people can help mould policies? No. There is certainly a role for the web as a closed information-sharing system for party members, with a simple public site laying out policies and crowing about successes. But what do we find instead? Overblown sites looking for a purpose. Mark them down for doing too much, not too little.
Websites are complex beasts, and measuring their effectiveness is a complex task. Organisations have their own conventions for measurement of all sorts of things, and the secret will be to create metrics that fit well with these conventions. But I would urge that directors accept that in this particular area a degree of fuzziness, or at least subjectivity, is not just unavoidable but is absolutely essential.
The writer is a website effectiveness consultant for Bowen Craggs & Co.