related case studies

Requirements analysis Training in use of a knowledge base

This project evaluated a training course designed to teach participants how to analyze the requirements of a business computer system. Twelve persons who were familiar with the content generated 53 statements which were then mapped into 10 clusters. The most important topics were located in the top right of the map and reflect the basic work of requirements analysis.

Figure 1. Concept map for a Systems Requirements Course

The training program was evaluated over the first three conducts. Participants were asked to rate both the opportunity they had to address each topic and the degree to which they achieved on that topic. Initial results looked promising. The outcome pattern match of importance with opportunity was .64 while the match with achievement was slightly lower at .58. However, these pattern matches declined over the next two conducts. For the opportunity outcome match the value was .58 for Conduct 2 and .54 for Conduct 3. For the achievement outcome match, the values were .42 for Conduct 2 and .34 for Conduct 3.

Figure 2. Outcome pattern matches for opportunity and achievement for the first three conducts of a systems requirements course.

To explore this pattern, it is useful to look in more detail at a specific match. For instance, the opportunity outcome match for Conduct 3 shows that one of the biggest disconnections occurred for the cluster Event initiated access of data work objects which was fourth highest in importance but tenth in opportunity. Another disconnection occurred with the cluster Data modeling concepts which was sixth in importance but second in opportunity.

Figure 3. Opportunity outcome pattern match for Conduct 3 of the systems requirements course.

When we look at the achievement outcome pattern match, we see a similar pattern, with the disconnects slightly more exaggerated. For instance, the cluster Event initiated access of data is even more extremely in tenth place here than it was with the opportunity match. In general, for both opportunity and achievement, clusters related to the model had higher outcomes, while those related to events were lower. This occurs despite the fact that the importance ratings do not follow this pattern.

Figure 4. Achievement outcome pattern match for Conduct 3 for the systems requirements course.

When we examine the patterns across the three conducts for the opportunity and achievement ratings, we can see more clearly what is disconnecting over time. There are two changes that stand out for both outcome measures. First, after Conduct 1, Data modeling concepts jumped from eighth place to first or second. Second, after Conduct 1, Event initiated access of data dropped gradually from highest down to one of the lowest (although in importance it is fifth).

Figure 5. Opportunity consistency pattern matches across conducts for systems requirements course.

Figure 6. Achievement consistency pattern matches across conducts for systems requirements course.

The last two figures offer some clues as to where the development team might look to identify why the program seems to be disconnecting over time. Two topics were rated fifth and six in importance initially. One of them became a much clearer focus of the course (Data modeling concepts) and one became much less a focus (Event initiated access of data). The development team can begin there and examine how the course is being implemented to confirm these initial hypotheses and attempt to get the program back on course.