Appendix 1 - A Communication on Systems Engineering Case Studies
SSE Master’s Project and Thesis Guidelines Appendix 1
A Communication on Systems Engineering Case Studies
Dr. Andrew Sage, First Bank Professor, George Mason University
Systems engineering and management concepts – involving knowledge principles, practices, and perspectives - are very important for success today. It is highly desirable to be able to evaluate the worth of the use of these concepts. Towards this end, case studies are potentially very instructive in that they relate aspects of the real world of professional knowledge practices to the research world of knowledge principles. Often, the bad examples may be even more valuable than the good ones, since they emphasize penalties for not following proper concepts, including processes and systems management perspectives.
A case study is empirical inquiry that:
Investigates a contemporary phenomenon within its real-life context, especially
Boundaries between phenomenon and context are not clearly evident, and in
Multiple sources of evidence are generally used.
Many authors infer similar definitions, in particular Yin (2003 a, b). To have a worthy inquiry or study, it is very important that the results of a case study have:
Internal Validity – Were the findings actually justified by the research, or were there problems of researcher bias? Has the case study researcher demonstrated a causal relationship between factors by showing that other plausible factors could not equally well or perhaps better explain the observed relationships?
External Validity – Could the research findings be generalized?
Construct Validity - Do the measures used in the case study make the concepts involved operational? In other words have we used multiple evidence sources, have we sufficiently established chains of evidence and have those providing evidence to the case study been allowed to review the case report before finalizing it?
Reliability - To what extent would other researchers who are studying the same case in exactly the same way arrive at equivalent conclusions?
Each of these is particularly important in case study research. it is particularly important to avoid even the appearance of researcher bias in case study research. At a minimum, this suggests strongly that the case study research team members should not all be stakeholders of the programs being reviewed.
A recent revival of interest in case study research has occurred, especially in evaluation research in many areas such as enterprise management and, more recently, systems engineering. This has led to the recognition that case study research can fill important needs. In particular, it is often insufficient to know that X can cause Y. We also need to know the how and the why X causes Y, and for what specific X and Y. Also, we are interested in the what, the when, and the who associated with research outcomes. Case study research can potentially answer these interrogatives and thereby help us become more contextually aware. Modern case study research is able to deal with issues of internal and external validity if the case study is well defined, developed, and deployed, with appropriate formulation, analysis and assessment, and interpretation across these three phases of the case study effort.
A prototypical characteristic of case studies is that they support a holistic understanding and interpretation of the systems of action, or interrelated activities engaged in by the participants or actors in the case situation subject to study. These case studies more often lead to hypotheses about behavior rather than being useful for validating general claims about behavior. Also, case study research often reveals a rich detail of information that highlights critical contingencies that exist among variables in the case study. Finally, the case study method is especially useful for exploration of topics when there is not a strong theory or set of knowledge principles to which one can appeal. Even when there is a strong normative theory, case studies can often lead to valuable insights, and potentially even to revision of the normative theories and knowledge principles towards improved ones.
The works of Yin (2003a) and Stake (1995) concerning case study research are especially recommended. Yin (2003b) provides several illustrative examples of case studies in the social sciences. Friedman and Sage (2004) provide thoughts on the major relevance and use of cases in systems engineering. The Air Force Institute of Technology (2006) has sponsored development of several case studies based on the framework suggested in this paper.
- Air Force Institute of Technology, Four Case Study Research Efforts, http://www.afit.edu/cse/page.cfm?page=49, accessed 28 March 2006
- G. Friedman, G. and A. P. Sage, "Case Studies of Systems Engineering and Management in Systems Acquisition," Systems Engineering, Vol. 7, No. 1, 2004, pp. 84-97, R. E. Stake, The Art of Case Study Research, Sage Publications, Thousand Oaks CA (1995).
- R.K. Yin, Applications of Case Study Research, Second Edition, Sage Publications, Thousand Oaks CA, (2003a).
- R.K Yin, Case Study Research: Design and Methods, Third Edition, Sage Publications, Thousand Oaks CA, (2003b).