William Trochim
Professor, Cornell University
Partner, Concept Systems Incorporated
Groupware enables things we do every day – meeting or scheduling, for example – over an electronic medium. The value that it adds is limited because it doesn’t change the fundamental way you do business – it just uses technology to do things you normally did anyway. A much higher value comes from using groupware to do things that have never been possible before, such as using groupware-based decision software to set and achieve valuable, agreed upon goals and to assess later how well you met those goals.
Say you are going to set up a new Intranet, or a related training program. How do you measure the value of each? What is the return on investment? How do you quantify what you are going to get? How do you separate the effects of the Intranet from all the other factors? The questions become nightmarish.
In traditional ROI analysis, we want to know if we are getting our money’s worth – a nearly impossible question if you start thinking about the complexity of most organizations and the host of other factors that can affect returns. For example, did this Intranet training make a difference? Does our corporate website improve our bottom line? You can try to play this game with numbers and rough estimates, but in most cases, it’s doubtful that those numbers are meaningful.
Doing ROI analysis is expensive and tricky. Put simply, the ROI of doing ROI well is usually too low to justify the effort and expense. It’s smarter to focus instead on doing a good up-front job laying out what you are trying to do, establishing consensus, and tracking that initial vision all the way through the project to measures of performance and change. Then you evaluate the project based on how close it came to the original expectations. This "Return on Expectation" (ROE) is quite different from traditional "Return on Investment" analysis. And, it’s a perfect opportunity for the newest applications of groupware.
Let’s illustrate the ROE idea by showing how the groupware program I developed – The Concept System – could be used to develop and evaluate an organization’s Intranet. You’d begin by sitting down with a group of people, either in a room or over the Internet, and using the software to lay out all the topics the website should include. Key stakeholders would be asked about specific elements that they’d like to see. All their ideas would be included. The software would help them to organize the ideas and decide which features are relatively more important.
The Concept System will develop a map or picture of the group’s ideas for the intranet site. And, the graphic output enables you to see whether the group is in consensus. The software shows the group where there are "disconnects" and helps them come to a shared vision for the project. They’d then turn the map over to the development team to use as a framework or blueprint for constructing the website. Later, at regular intervals, the same maps are used to check the actual site against initial expectations – we assess the ROE to see if we got what we expected to get.
Measuring Return On Expectations is also powerful when using groupware to collaborate with customers or prospects. Telecommunications companies, for example, are interested in how to develop the communications devices of the 21st century. They ask potential customers what features they would like in future communications devices and develop product feature maps that show customer preferences. Product development teams use the maps to develop prototype models, and then evaluate how well the prototypes meet the expectations of the customers. They are using groupware to work directly with prospective customers to determine which products would best match customer expectations – which product gives the best "Return On Expectations."
Andersen Consulting, EDS , Motorola, Price Waterhouse, Nationwide and others are using this approach for evaluation of business training, performance management and measurement, strategic and operational planning, product development, and more.
The advantage of such an approach is obvious: A clear idea of critical success factors without endless and tiresome meetings. And no wasting precious corporate time and resources scrambling to "work up some numbers" of questionable utility. Did this project do what we wanted it to do? Did we get a Return On Expectations? That’s a far more meaningful question.