Narayanan V.K., Armstrong D.J. - Causal Mapping for Research in Information Technology (2005)(en)

170 Vo, Poole and Courtney

Laukkanen, M. (1994). Comparative cause mapping of organizational cognitions. Organization Science, 5, 322-343.

Lee, S., Courtney, J. F., & O’Keefe, R. M. (1992). A system for organizational learning using causal maps. Omega, 20, 23-36.

Lee, S. Y. (1995). An integrated model of land use/transportation system performance: System dynamics modeling approach. Ph.D. Dissertation, University of Maryland, College Park.

Levi, A., & Telock, P. E. (1980). A cognitive analysis of Japan’s 1941 decision for war.

Journal of Conflict Resolution, 24, 195-211.

Linstone, H. A. (1984). Multiple perspectives for decision making: Bridging the gap between analysis and action. New York: North-Holland, Elsevier Science.

Markoczy, L., & Goldberg, J. (1995). A method for eliciting and comparing causal maps.

Journal of Management, 21, 305-333.

Massey, A. P., & O’Keefe, R. (1993). Insights from attempts to validate a multi-attribute model of problem definition quality. Decision Sciences, 24, 106-125.

Massey, A. P., & Wallace, W. A. (1996). Understanding and facilitating group problem structuring and formulation: Mental representations, interaction, and representation aids. Decision Support Systems ,17, 253-274.

Mitroff, I. I., & Linstone, H. A. (1993). The unbounded mind: Breaking the chains of traditional business thinking. New York: Oxford University Press.

Montazemi, A. R., & Conrath, D. W. (1986). The use of causal mapping for information requirements analysis. MIS Quarterly, 10, 44-55.

Nicolini, D. (1999). Comparing methods for mapping organizational cognition. Organization Studies, 20, 833-861.

O’Keefe, J., & Nadel, L. (1978). The hippocampus as a causal map. Oxford, UK: Clarendon.

Richardson, G. P. (1991). Feedback thought in social science and systems theory.

Philadelphia: University of Pennsylvania Press.

Sakman, M. (1985). An empirical study of three methods of problem definition in illstructured situations. Ph.D. dissertation, University of Wisconsin, Madison.

Schneider, S., & Angelmar, R. (1993). Cognition in organizational analysis: Who’s minding the store? Organization Studies 14, 352-374.

Senge, P. M. (1990). The fifth discipline: The art and practice of the learning organization. New York: Doubleday/Currency.

Sheetz, S. D., Tegarden, D. P., Kozar, K. A., & Zigurs, I. (1994). A group support systems approach to causal mapping. Journal of Management Information Systems, 11, 3157.

Steinbruner, J. D. (1974). The cybernetic theory of decision: New dimensions of political analysis. Princeton, NJ: Princeton University Press.

Tegarden, D. P., & Sheetz, S. D. (2003). Group causal mapping: A methodology and system for capturing and evaluating managerial and organizational cognition.

Omega, 31, 113-125.

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

Collective Causal Mapping Approaches 171

Vennix, J. A. M. (1996). Group model building: Facilitating team learning using system dynamic. Chichester, UK: Wiley.

Weick, K. E., & Bougon, M. G. (1986). Organizations as causal maps: Charting ways to success and failure. In H. Sims & D. Gioia (Eds.), The Thinking Organization (pp. 102-135). San Francisco: Jossey-Bass.

Zhang, W. R., Wang, W., &King, R. S. (1994). A-pool: An agent-oriented open system shell for distributed decision process modelling. Journal of Organizational Computing, 4, 127-154.

Endnotes

1In the analysis, we use their distance ratio (DR) formula to measure the distance between collective maps and individual maps.

2This framework and a prototype of sustainable decision support systems were developed to improve policy planning and decision making regarding urban infrastructure investments such as investments in roads and bridges, fresh water supply systems, waste water treatment, drainage and so forth.

3These interviews were conducted by the participants and other researchers with people who are involved with the city’s infrastructure management.

4It was our intention to limit the number of nodes to 15 (maximum).

5The power-related implications of a map can be surfaced if it can be used to show how one group of stakeholders is advantaged in the current situation and how this contributes to the problems or how one group can manipulate the organization to serve their interests.

6The map, which had more links, thus, higher degree of map density ratio, received a higher rating.

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

172 Vo, Poole and Courtney

Appendix

Cognitive Maps from Houston Case

Collective map created with aggregate method

Collective map created with congregate method

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

Collective Causal Mapping Approaches 173

Collective map created with workshop mapping method

-+

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

174 Armstrong and Narayanan

Chapter VII

Expanding Horizons:

Juxtaposing Causal

Mapping and Survey

Techniques1

Deborah J. Armstrong

University of Arkansas, USA

V.K. Narayanan

Drexel University, USA

Abstract

In this chapter, we compare the findings from causal maps derived from semi-structured interviews with that obtained from survey respondents, using a data set originally constructed to characterize object-oriented (OO) software development expertise. To compare the results, we invoke three different theories to capture evoked concepts in the interviews, but discover one theory provided more robust theoretical constructs in embracing the evoked concepts. The survey responses were factor analyzed to explore if the factor structure matched the structure derived from revealed causal maps. Although there was significant similarity between the results, the survey yielded more factors than predicted by the theory. The lessons learned from this process are discussed.

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

Juxtaposing Causal Mapping and Survey Techniques 175

Introduction

The original work of Axelrod (1976) on causal mapping has remained a standard adopted by others extending the technique to discovery (e.g., Fiol & Huff, 1992), evocative (e.g., Nelson, Nadkarni, Narayanan & Ghods, 2000) and intervention (Hodgkinson & Wright, 2002) settings to name a few. While some methodological properties of causal mapping have recently come under scrutiny (Mohammed, Klimoski & Rentsch, 2001; Nadkarni & Narayanan, in press), one fundamental question remains unanswered: How do the data yielded by causal mapping techniques compare with the data obtained by other methods? Without works comparing alternate methodologies, we cannot be fully confident of the meaning of the data yielded by causal mapping, much less the appropriateness of the technique to different research settings.

In this chapter we advance the causal mapping method using a comparative study that links causal mapping data with data obtained from surveys. We take a cue from Nelson et al. (2000), who argued that causal maps are a starting point for capturing concepts in an exploratory context, but the concepts then become the basis of constructing large sample surveys for validation and hypothesis testing. The domain used for this study is object-oriented (OO) software development, which is a developing domain, and as such robust theories that capture this domain are absent. This necessitates exploratory works and we employ causal mapping to characterize the domain. We validate the causal maps by administering a relatively large sample survey that in turn provides a basis of comparison for the causal maps.

Thus the central objective of this chapter is to demonstrate an approach to couple revealed causal maps (RCMs) developed in “evocative” domains (Nelson et al., 2000) and large scale surveys designed for statistically based hypothesis testing. Put another way, we propose a strategy for systematically linking discovery and hypothesis testing contexts. An ancillary objective is to demonstrate how the approach triangulates the causal mapping technique with survey methods, thus exploring the validity of the causal mapping technique

To meet these objectives we organize the paper as follows: First, we establish the context of the study, OO software development; second, we summarize the conceptual underpinnings of the study and the strategy for triangulation; third, we provide a detailed description of the methodology used; we then report the results of the study and finally discuss the lessons learned.

The Context: Object-Oriented (OO)

Software Development

“OO software development” refers to a set of principles guiding software development that emphasizes organization based on both information and processing, and that manipulates the information according to the real-world objects that the information

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

176 Armstrong and Narayanan

describes (Brown, 1997). Unlike other approaches such as procedural software development, OO has come into vogue more recently and theories about OO are still evolving. As software development is knowledge work, its most important resource is expertise (Faraj & Sproull, 2000). Yet, a systematic identification of the major constructs for objectoriented software development expertise has yet to receive significant attention. While there has been an exploratory study linking cognition and OO concepts (Sheetz & Tegarden, 2001), to date no research has empirically tested the evoked theories of OO software development. This provides the perfect context to evoke and empirically test theories of expertise. As noted by Nelson et al. (2000, p.482), “in evocative studies, domain experts are available, but work is needed to evoke their knowledge and cast it into available theoretical frameworks to construct domain specific theories.”

We conducted an extensive review of theoretical literature to identify the frameworks available to investigate OO software development. Theoretical sources we reviewed included textbooks on traditional software development (e.g., Dennis & Wixom, 1999; Hoffer, Valacich & George, 2001), OO software development (e.g., Brown, 1997; Martin & Odell, 1995; Norman, 1996), “classic” books on OO software development (e.g., Booch, 1994; Coad & Yourdon, 1991a, 1991b; Henderson-Sellers, 1992; Rumbaugh, Blaha, Premerlani, Eddi & Lorensen, 1991) and seminal articles in OO software development (e.g., Detienne, 1995; Fichman & Kemerer, 1992; Rosson & Alpert, 1990; Villeneuve & Fedorowicz, 1997). Table 1 lists a sample of the potential OO theories that could be used as a starting point for this research.

The abundance of theoretical frameworks is one indicator that OO theory development is in the early stages. To reduce the number of theories to a manageable set, we employed two criteria: comprehensiveness and diversity. Since this was an exploratory study, we wanted to use frameworks that represented the widest range of conceptualizations possible. Comprehensiveness was defined as the number of concepts identified in the

Table 1. Sample of potential OO theories used in classification scheme

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

Juxtaposing Causal Mapping and Survey Techniques 177

framework: The more concepts the framework had, the more comprehensive the framework. Several of the frameworks were too restricted in the number and variety concepts they included and thus were not appropriate for this study. Diversity was assessed by comparing the frameworks to each other. For example, the Booch (1994) framework was almost identical to the Coad and Yourdon (1991) framework and was thus eliminated as a candidate early in the selection process. As a result, we narrowed the theoretical frameworks to three, the major constructs of each are presented in Table 2.

•Theoretical Framework I was developed by grouping concepts based on the categorization scheme presented by Rosson and Alpert (1990) and utilized by Sheetz and Tegarden (2001) in their study linking cognitive activities to objectoriented design complexity. The categories Rosson and Alpert suggested were: Communicating Objects, Abstraction, Problem-Oriented Design, and Shared Behavior. The Communicating Objects category contains concepts such as message passing; the Abstraction category contains concepts such as abstraction and encapsulation; the Problem-Oriented Design category contains concepts such as modeling objects; and the Shared Behavior category contains concepts such as inheritance and class.

•Theoretical Framework II was adapted from Henderson-Sellers (1992). His book compiles several research articles into the idea of an object-oriented triangle (see Henderson-Sellers (1992) for a summarization of supporting literature). The first corner includes the concepts of encapsulation and information hiding. The second corner includes the concepts of abstraction, classes and objects. The third corner includes the concepts of inheritance and polymorphism.

•Theoretical Framework III was developed from Armstrong (in press). The categories suggested were based on the object-oriented model and consist of Behavior, Structure, OO Modeling/Analysis and OO Development. The Behavior construct contains concepts such as message passing and collaboration and is focused on the object actions within the system. The Structure construct contains concepts such as attribute, class and object and is focused on the relationships between classes and objects and the mechanisms that support the class/object structure. The OO Modeling/Analysis construct contains concepts such as identifying objects and is focused on the analysis phase and identifying the “things” or objects in the problem under study. The OO Development construct contains concepts such as framework and layer and is focused on the overall development of OO.

Table 2. Chosen theoretical framework constructs

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

178 Armstrong and Narayanan

The underlying strategy for triangulation of RCMs and surveys employed in this study is anchored in key epistemological underpinnings, which we summarize before we detail the method.

Conceptual Underpinnings of the

Methodological Strategy

Analysis of the truth claims of data by any method should rest on its conceptual underpinnings. As persuasively argued by Rescher (1992), knowledge development is itself a practice, the study of which is the domain of epistemology. Rescherian epistemology further isolates two interconnected cycles of practice underlying knowledge development, thus allowing us to categorize different approaches. The first one is a theoretical cycle which seeks to maintain internal consistency among theoretical ideas, or “theoretical self-substantiation.” The second, an applicative cycle, seeks “pragmatic validation” external to theory. Rescher underscores the tension between the two cycles, and maintains that in any vibrant domain of inquiry, the overall legitimization of a methodology for the substantiation of our factual beliefs must unite the two distinct cycles, one toward a systematic coherence at the theoretical level, and the other toward pragmatic validity at the empirical level. See Figure 1(a) for a graphical representation of the cycles.

Figure 1. Rescherian epistemology applied to validation

(a) Two-fold cycle of the legitimization of systemizing methodology (Adapted from Rescher, 1992)

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

Juxtaposing Causal Mapping and Survey Techniques 179

Figure 1. Rescherian epistemology applied to validation

(b) Four stage validation process for RCM

In this study we engage the two-fold cycle to develop a system of validated knowledge regarding OO software development expertise. In the theoretical cycle we develop theories of OO software development from revealed causal maps (RCMs) of OO experts. In the applicative cycle we compare the findings from the causal maps with those obtained from survey respondents.

Proposed Strategy

Our approach to the analysis of methodological validity of the RCMs is anchored in Rescherian epistemology. Specifically, we adopt a four-stage process as shown in Figure 1(b).

1.First, we engage the applicative cycle, by creating a text of what constitutes OO expertise to isolate concepts in the language of the practitioner. Semi-structured interviews were conducted with OO software development experts to gather their knowledge regarding the concepts that constitute OO development.

2.Second, through a conscious process of choice, we choose a theoretical framework to interpret the concepts, thus engaging the theoretical cycle. As stated above, three theoretical frameworks were initially chosen as candidates. Additional OO experts were asked to assess the congruence of the emergent concepts with each

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.