related case studies

Self-study computer programming school Systems design course Training planning and evaluation

The purpose of this study was to evaluate the effectiveness of training in client/server computer application development that was based on a combination of instructor-led and team-based learning methods. Ten persons familiar with the course content generated 90 statements and each subsequently completed a similarity sort of the statements. The concept map showed eleven clusters ranging from contextual supports and general methodology on the top and left, to more specific client/server concepts on the right and bottom. The cluster describing the actual coding and testing of client/server applications was considered most important for this course.

Figure 1. Concept map for training in client/server computer application development.

The implementation of the program was assessed in three pilot administrations (labeled Pilot 2 through Pilot 4). The effectiveness of the training was evaluated in two separate conducts. The results across both pilots and conducts indicate that the program is moderately successful in achieving its expected results. For the three pilot runs, the outcome pattern matches were .51, .53, and .63, while for the two conducts they were .58 and .53 respectively. This indicates that, on the whole, the program was moderately successful in achieving desired outcomes.

Figure 2. Outcome pattern match of expectations and achievement for conduct 1 of the client/server computer application development training program.

Consistency of outcome patterns across the five instances of the training was extremely high with the pattern matches ranging from .9598 to .9897.

Figure 3. Consistency pattern matches across three pilot and two actual administrations of the client/server computer application development training program.

The major reason the matches were not stronger is that the highest rated area -- the coding and testing of the client/server applications -- consistently was rated between sixth and eighth in terms of achievement. The development team is investigating whether this topic can be made even more salient in subsequent implementations.

Figure 4. Time series of outcome and consistency pattern matches for the client/server computer application development training program.

One consistent pattern that emerged is that the highest achievement tended to be evidenced in clusters most related to the way the course was taught, a combination of instructor-led and team-based learning. The clusters related to communication, teaming and coaching consistently placed in the top five across all administrations. One implication may be that the training methods used may have been more salient than the content.