Casual Maps Validated For Software Development Research

FAYETTEVILLE, Ark. — Object-oriented (OO) software development has many business benefits, but the OO approach, which requires a substantial change in thinking, is difficult for most traditional developers. University of Arkansas researcher Deb Armstrong validated the use of causal mapping to explore these problems and developed an online survey instrument to facilitate further research.

"Because OO development is very new, little research has been done to characterize OO expertise," said Armstrong, assistant professor of information systems in the Walton College of Business. "We know that prior knowledge of the traditional procedural approach to development actually hinders transition to the OO approach, but we don’t know why. To understand that, we need to know what knowledge OO software developers have and how it is organized."

To identify these basic concepts, Armstrong and V.K. Narayanan, professor of strategy and entrepreneurship at Drexel University, used a technique called causal mapping. Although common in other disciplines, causal mapping has not been used previously in information technology (IT) research. It is a collection of techniques that can be used to assess and explain the mental models that form the way individuals think about a specific topic. The end-product is a revealed causal map (RCM). RCMs can be aggregated to show the cognitive structures of a group, which shows similarities in thinking about the topic.

They interviewed 24 OO development experts. In the course of each interview, the researchers asked numerous "structural" questions (eg., "What is the first thing you do when you get a requirements document?") and "content" questions ("How do you know someone is an expert in OO development?") On the basis of the answers, the researchers developed a RCM for each individual, which graphically represents how each expert thinks about the programming method.

"Causal mapping has been used a great deal in empirical research among cognition scholars, but not in the area of information technology (IT)," explained Armstrong. "We wanted to compare the RCMs with another method to see if the technique is valid in this setting."

To validate the causal maps, the researchers administered a survey to 142 experts in OO development. The developed the Software Development and Maintenance Approach (SDMA) to capture concepts that had emerged in the previous interviews. After pre-testing to determine internal and construct validity, the SDMA was administered to OO developers from 33 organizations in industries ranging from telecommunications to manufacturing and financial services.

A factor analysis conducted on the survey results revealed five OO constructs, which were then used to recast the aggregate RCM. Although the recast RCM had one additional construct, the researchers found that it was very similar to the original RCM in both structure and linkages.

"This is the first study designed to compare causal mapping with the survey method," Armstrong said. "The results indicate that causal mapping, which is considered to be a more appropriate tool for cognition research, can be used as a basis for other large-sample studies in this emerging field. In fact, it may be a necessity if we are to capture the true phenomena under investigation."

Armstrong presented the results of their research recently at the annual meeting of the Academy of Management in Denver.