* * *
This article is part of the UnderstandingSociety research gateway. Please visit the site, where you will find other useful articles, blogs, and an international social network site on the philosophy of social science, Marxism, and globalization.
* * *
Daniel Little, University of Michigan-Dearborn
This essay turns to a consideration of the logic of causal inquiry: how can social scientists probe social phenomena so as to discern underlying causal relations? There are two general strategies in answers to this question, corresponding to comparative methodology and statistical inquiry. The following represents an effort to explore in detail the problems of identifying patterns of social causation. In the first part I offer a generalization of Mill's methods of difference and similarity (Mill 1950), as a basis for discussing the logic of comparative analysis. In the second part I relax the conditions required for Mill's methods and consider to what extent statistical methods will aid in identifying causal relations among factors.
The goal of this section is to see how far an inductive approach to causal analysis can take us. There is a general anti-Baconian assumption in the philosophy of science that doubts that important scientific truths can be discovered on a purely inductive basis. But this is not obviously true in the case of causal analysis. The goal of causal analysis is to determine which of a large set of observational variables are causally relevant to the occurrence of the effect. And causal relevance necessarily shows up in observed states of the world: if A&B causes C and nothing else does, then we will find that A and B are always present when C is present, and no other variable will show similar invariance. So a purely inductive approach is not impossible in principle. [1] The obstacle we face is rather the paucity of data: in order to inductively establish the causal diagram underlying a set of social processes we would need a vast data set of instances in which all the factors are known, And this is almost invariably not available in social science research.
It is also true, of course, that our scientific interest is not finished when we have established the causal diagram underlying this particular cancer. This is so because we will now want to know the physiological basis of the links in the diagram: how is it that exposure to this toxin in the presence of this genetic defect gives rise to this type of malignant cell growth? This task is exactly analogous to the task of social theory in causal analysis in social science; it involves explaining the causal mechanisms based on an account of the causal powers of the factors involved.
Much of the following will focus on the question, how can social scientists identify causal relations among social phenomena? It may be useful, however, to begin by formulating a set of assumptions that jointly constitute a theory of causal realism for the social sciences.
It is useful to introduce several concepts and distinctions that provide a fuller vocabulary in terms of which to describe causal relations.
causal field: a set of conditions identified or suspected to be causally relevant to the occurrence of a given effect.
causal closure: the causal field is complete; all factors that are causally relevant to the effect are included.
direct generic causal connection between A and B: A has a causal power to bring about B (in a specified set of causal fields). E.g.: Average levels of education in developing countries have a generic causal influence on rates of economic development. Such connections should be discernible by statistical tests, since it follows immediately from this definition that there will be strong correlations between the two factors. (This is a one-step causal diagram; it corresponds generally to Mill's distinction between direct and indirect causal connections.)
causal diagram: an analysis of the combinations of factors that in sequence lead to a given outcome. Each link in the causal diagram consists of direct generic causal connections. Multiple pathways may lead from factors to effect.
deterministic causation: causes work through exceptionless regularities.
probabilistic causation: causes alter the probability of occurrence of their effects relative to the prior probability of the effect.
causal consistency: if a complete setting of the factors in the causal field is once associated with the effect, then this setting will always be associated with the effect.
causal independence of factors: two causal factors are independent with respect to the effect if each transmits its effect to the result independently from the other. (This condition is equivalent to the two factors lying on different pathways within the causal diagram.)
standing condition: a causal condition which is present over extended periods of time, and which has not changed its state at the time of the effect.
instigating condition: a causal condition whose state changed at the time of the effect and whose change of state was causally involved in bringing about the effect.
This discussion I now present may be described as experimental philosophy of social science. This section is organized around a single hypothetical causal model. I stipulate that the world has certain causal properties and then construct various data sets that would result in such a world on the basis of several different forms of empirical inquiry. The goal is to investigate whether it is possible to use these several methods to discover underlying causal relations and what the central limitations of the methods are.
I have constructed this case around a simple causal model. For the purpose of the case I ask the reader to assume that the world is causally organized according to the causal diagram provided in figure 1. This diagram involves six independent variables (food crisis, local organization available, war, weak state institutions, exploitation, and economic crisis). It involves two intermediate causal variables (social unrest and state crisis). And it identifies one final variable (revolution). Variables A-F are assumed to be independent-that is, none is causally relevant to the occurrence of any of the others. The states of variables G and H are determined by the states of A-F. And the state of R is determined by the states of G and H. This causal structure can be represented as a complex truth functional law:
1 ((AvE)&B)&((CvF)&D) => R
This expression can be simplified:
2 B&D & [(AvE)&(CvF)] => R
Figure 1. Causal diagram for revolution
Thus B and D (local organization and weak state institutions) turn out to be necessary conditions for the occurrence of revolution (in this hypothetical world)-a circumstance that can be verified by inspection of the causal diagram directly.
The diagram is used to construct the data set; but the reader is now asked to assume that it is no longer available. The researcher is assumed to be able to collect information about outcomes, but cannot inspect the causal diagram directly.
How may we investigate the causal relations established by the causal diagram? A particularly direct approach is the comparative method: examine a number of different settings for the primary variables (A-F) and see whether we can identify necessary and jointly sufficient conditions for the occurrence of revolution. (Each setting of the variables constitutes a case.) This approach makes two central assumptions:
Exceptionless causation: causes work through exceptionless regularities.
Causal closure: we have identified the complete causal field; all causally relevant variables are considered.
These assumptions guarantee causal consistency: if a complete setting of the dependent variables is once associated with the effect, then this setting will always be associated with the effect. There are two ways a causal system might fail to be causally consistent: causes might be probabilistic (so that a causal setting sometimes fails to produce the effect), or the causal field might be incomplete (so that the effect sometimes appears in settings that are not causal settings). The first assumption guarantees that if some conjunction of properties is sometimes sufficient to produce an outcome, then it is always sufficient to produce that outcome. The second assumption guarantees that if the effect is present, it has been produced by the conjunction of properties present in the setting, not some causal factor external to the field. These assumptions are highly unrealistic, but they are unavoidable. It is simply impossible to apply a comparative method without making them. What the following will show, however, is that even if we make these highly charitable assumptions, causal reasoning in comparative research is much more problematic than usually admitted.
Let us begin, then, with a comparative study of revolution based on these variables. Figure 2 represents a hypothetical comparative study of revolution along the lines of Theda Skocpol's research (Skocpol 1979). Here I assume that the researcher has identified a set of cases in which revolution occurs and a set of cases in which it does not; he or she has then determined the status of all variables (A-H) for each case. The values of the dependent variables (G, H, and R) are generated on the basis of the causal diagram of figure 1; but the researcher is presumed to be ignorant of the real underlying causal relations. Each line of the table, then, represents the result of the researcher's examination of a single case.
| A | B | C | D | E | F | G | H | R | |
| food crisis | local organiz. | war | weak inst. | exploi- tation | econ. crisis | social unrest | state crisis | revolution | |
| Cuba | false | true | false | true | true | true | true | true | true |
| France | true | true | false | true | true | true | true | true | true |
| Russia | false | true | true | true | true | true | true | true | true |
| China | true | true | true | true | true | true | true | true | true |
| England | false | false | true | false | true | true | false | false | false |
| Italy | false | false | true | true | true | true | false | true | false |
| Sweden | true | true | false | false | false | true | true | false | false |
Figure 2. Comparative study of revolution
If we inspect the cases looking for necessary and/or sufficient conditions for the occurrence of revolution, we will notice various facts:
1. Social unrest and state crisis (G and H) appear to be necessary for R. Neither is sufficient, since each occurs singly in cases in which revolution does not occur (Italy and Sweden). But these two variables appear to be jointly sufficient for the occurrence of R.
2. We cannot tell much about the role of economic crisis, since it occurs in every case. It may be necessary; it cannot be sufficient.
3. Food crisis and war cannot be necessary conditions for revolution, since there are cases in which revolution occurs but one or the other of these conditions is absent (Cuba, France, Russia). We do not know, however, whether this is a minimal sufficient condition.
4. Local organization, weak institutions, exploitation, and economic crisis are jointly sufficient for revolution, since these conditions by themselves lead to revolution in one case (Cuba), and are present in each of the other cases in which revolution occurs.
Is it possible to systematize this analysis at all? It is, since we can express all the information about causal relations presented by this study in the form of a complex truth functional sentence. (This is a formulation of the causal relations in terms of the INUS conditions that govern these factors.) [2] Revolution occurs in conjunction with certain combinations of the variables but not others. So we can express the causal law represented by the study as a set of jointly sufficient conditions. Consider, then, the expression we get from considering only the four positives in figure 2 (Cuba, France, China, Russia):
{ABCDEF v AB~CDEF v ~ABCDEF v ~AB~CDEF }=> R
We notice that several factors are in common among each disjunct, so these can be pulled out of the expression:
BDEF & {AC v A~C v ~AC v ~A~C}=> R
Thus B, D, E, and F appear to be necessary conditions for the occurrence of revolution.
The expression in braces is a tautology, so we may simplify:
BDEF=> R
That is: local organization, weak institutions, exploitation, and economic crisis are sufficient to produce revolution-a correct conclusion given the underlying causal process.
What this data misses, however, is that there are other combinations of conditions that will produce revolution:
BDAF => R
BDAC => R
BDEC => R
So this method permits us to discover some of the sufficient conditions for the occurrence of revolution; it does not permit us to conclude that any condition is a necessary condition; and it is not guaranteed to eliminate conditions that are unnecessary for the outcome.
In short: the data provided in figure 2 is consistent with the causal diagram. But it is not sufficient to permit the researcher to infer the complete underlying causal structure. Rather, it would be necessary to arrive at a hypothesis about the causal relations among these conditions; and such a hypothesis most naturally emerges from a substantive theory of the causal mechanisms that are at work in the social phenomena under consideration. And to discern the other possible causal pathways it would be necessary to identify more cases in which different settings of the variables lead to revolution or no revolution.
The chief limitation of the preceding study is its incompleteness. However, if we assume exceptionless causal regularities and causal closure, then there is a research strategy available which permits complete information. [3] There are 6 independent variables here; this produces a truth table of 64 logically possible combinations (table 1). Some of these combinations may be excluded as logically possible but naturally or socially impossible. For each remaining possibility find a case that instantiates it; then see whether there was a revolution or not. (We may construe the truth table as an exhaustive list of possible worlds: all possible states of the world with respect to the independent variables.) This will give us a list of conjunctive conditions, each of which is sufficient to cause revolution. In the case of the model under discussion, that sentence is:
3 {ABCDEF v ABCDE~F v ABCD~EF v ABCD~E~F v AB~CDEF v AB~CD~EF v ~ABCDEF v ~ABCDE~F v ~AB~CDEF } => R
4 BD & {ACEF v ACE~F v AC~EF v AC~E~F v A~CEF v A~C~EF v ~ACEF v ~ACE~F v ~A~CEF}=> R
We can identify necessary conditions if there are any (as there are in this case-B and D), since these will occur in each disjunct. Each of the disjuncts in expression 3 is sufficient for the production of R-though the complexity and redundancy of the expression makes this somewhat uninteresting. Expression 4 could with ingenuity be simplified to expression 2 above, since the two are logically equivalent; but the simplification does not jump out at one.
So: if the causal relations in question were exceptionless and if we could find a case for each logically and socially possible world, then we would be able to come up with a truth-functional statement that captures the causal relations among the factors. This expression might or might not be simplifiable to a comprehensible causal hypothesis.
This truth-table method in a sense represents a complete method of causal analysis for comparative research. However, for actual comparative work it is clear that it will not be possible to find cases for every logically and socially possible world; this implies, however, that we will not be able to provide an exhaustive statement of the causal regularities. We would have cases corresponding only to a small subset of possible settings of the variables.
We can also examine the conditional probabilities associated with the truth-table analysis (figure 3). The conditional probability of revolution given the presence of food crisis is represented as P(R|A). This expression should be understood as representing the frequency of the occurrence of revolution in the subset of cases in which there is food crisis. Wesley Salmon (1984) shows that a factor A is causally relevant to an effect R if and only if P(R) P(R|A). The conditional probabilities reported in figure 3 show that each of the variables (A-F) produces a higher conditional probability of revolution than the antecedent probability of revolution-showing that each is causally relevant to the occurrence of revolution.
| P(R) | 9/64 | .141 | ||||
| P(R|A) | 6/32 | .188 | P(R|B) | 9/32 | .281 | |
| P(R|C) | 6/32 | .188 | P(R|D) | 9/32 | .281 | |
| P(R|E) | 6/32 | .188 | P(R|F) | 5/32 | .156 | |
| P(R|AB) | 6/16 | .375 | P(R|AC) | 4/16 | .250 | |
| P(R|AD) | 6/16 | .375 | P(R|AE) | 3/16 | .187 | |
| P(R|AF) | 4/16 | .250 | P(R|BC) | 6/16 | .375 | |
| P(R|BD) | 8/16 | .500 | P(R|BE) | 6/16 | .375 | |
| P(R|BF) | 6/16 | .375 | P(R|CD) | 6/16 | .375 | |
| P(R|CE) | 4/16 | .250 | P(R|CF) | 3/16 | .187 | |
| P(R|DE) | 6/16 | .375 | P(R|DF) | 6/16 | .375 | |
| P(R|EF) | 4/16 | .250 | P(R|GH) | 9/9 | 1.000 | |
| P(R|ABC) | 4/8 | .500 | P(R|ABD) | 6/8 | .750 | |
| P(R|ABE) | 3/8 | .375 | P(R|ABF) | 4/8 | .500 | |
| P(|R|ACD) | 4/8 | .500 | P(R|ACE) | 2/8 | .250 | |
| P(R|BCD) | 6/8 | .750 | P(R|BCE) | 4/8 | .500 | |
| P(R|ABCD) | 4/4 | 1.000 | P(R|ABDE) | 4/4 | 1.000 |
Figure 3. Conditional probabilities of revolution based on truth table
Several observations emerge from this consideration of comparative research.
1. Any actual comparative project will consider a number of cases that is substantially smaller than the total (though finite) number of possible states of the world with respect to the causal field. This means that conclusions will be less than exhaustive, and there may be causal relations that are not identified.
2. A given causal field (list of potentially relevant causal variables) cannot be known to be complete; there may be other variables that are causally relevant but not present. This means that there may be causal inconsistencies in the study (e.g. cases with the same setting of the variables but different outcomes for the dependent variable).
3. The assumption of exceptionless causal regularities is plainly unjustified. But this means that Mill's methods and their generalization here cannot be applied; we cannot infer from the presence of a set of factors and the absence of R, that those factors do not cause R. Instead, this may simply be one of the infrequent times in which the general causal regularity fails to materialize.
4. Note, further, that examining the conditional probabilities associated with the data (figure 3) might lead us to believe that these are statistical causal processes: that the occurrence of food crisis uniformly increases the probability of revolution by some amount. But this would be wrong; the model to this point embodies strictly exceptionless regularities. But since the model embodies disjunctive causal conditions there will only be weak correlations between those factors and the occurrence of revolution. Further, there is no simple additive effect among all the variables: the fact that R is weakly correlated with A and with E does not suggest that R will be more likely to occur when both A and E are present. For in the correct causal story these factors are on separate causal branches (note the conditional probabilities of R|A, R|E, and R|AE are all equal).
5. Given the large number of different combinations of variables (settings) that produce distinct outcomes and pathways, we may be somewhat skeptical about comparative studies that consider only a small number of cases; it is perhaps reasonable to worry that these cases will have only captured a few out of a large number of pathways and give a misleadingly simple idea of the causal processes at work.
6. Finally, and most importantly: these limitations demonstrate that purely inductive study of cases cannot suffice to fully identify causal relations. Rather, it is necessary to put forward hypotheses about the underlying causal relations. These hypotheses can be tested through comparative study (examination of specific cases with questionable settings of variables), but they cannot be deduced from the data. The data radically underdetermines the causal hypotheses. Where do such hypotheses come from? This is the function of social theory: rational choice theory, theory of organizations, theories of mobilization, theories of collective action, theories of institutional change, etc. All these theories represent empirically supported descriptions of processes of social causation; so when we look at the phenomena of a case, we may recognize instances of causal mechanisms at work. This is the deductive side of social science explanation. The data of the case studies represent a few data points to which our causal theories must conform, rather than a CAT-scan of social causation that we need only trace over mechanically.
The assumptions required for Mill's methods and its generalization are excessively demanding: exceptionless causal regularities and complete causal fields. The world seems to present us with numerous examples of causal relations that are probabilistic rather than exceptionless; and we are rarely in a position to be able to specify with confidence a complete list of factors that are causally relevant to a given kind of outcome. We now relax the first assumption by allowing probabilistic causal relations. Under these relaxed conditions Mill's methods are no longer available. We still have the same 6 independent variables; but now instead of exceptionless causal laws we have the circumstance that some combinations of conditions have the effect of changing the antecedent probability of the occurrence of revolution (upwards typically).
Figure 4. Probabilistic causal model for revolution.
I have incorporated statistical causation into this model in a somewhat more realistic way by assigning probabilities to each of the independent variables (A-F). This in turn permits us to calculate a probability of occurrence for each line of the truth table. Secondly, I have assigned probabilities to each of the causal links specified in the model. None of the regularities is now conceived to be exceptionless; rather, I assign a probability to the occurrence of the result given the occurrence of the antecedent conditions (a conditional probability). Here we have causal regularities that may be represented in the following form:
(A&B) =[p]=> Q,
where [p] represents the probability of Q's occurring given the occurrence of A&B. (Thus p is the conditional probability of Q given A&B: P(Q|AB).)
Figure 4 illustrates this set of assumptions. Probabilities of the independent variables are represented by decimal fractions above each factor. Conditional probabilities of outcomes given antecedent conditions are represented by decimal fractions attached to each causal link in the diagram. These assumptions define an infinite set of possible worlds, with these characteristics:
1. each world state appears in proportion to its overall probability (determined by the product of the probabilities of the states of each of the independent variables);
2. each world state is associated with revolution in proportion to the net probability of the particular causal links that produce revolution in that case.
Note, however, that there is a combinatorial explosion here. Most worlds will not lead to revolution; and there are a large number of pathways that do, corresponding to different settings of the six primary variables. [4]
I have explored this set of assumptions through a spreadsheet model that amounts to a simulation of the causal diagram represented in figure 4, incorporating the probabilistic assumptions now introduced. It is no longer possible to examine all possible combinations of outcomes. What is possible, however, is to examine random samples from this universe of possible worlds. The spreadsheet model incorporates the probabilistic assumptions of the model described above and constructs a random selection of 50 cases based on these assumptions. Each line of the spreadsheet contains a randomly assigned value for the independent variables; the model then computes values for dependent values, incorporating the conditional probabilities of association embodied in the diagram. (This is roughly analogous to a rather large comparative study in which we randomly select 50 cases-settings of societies in which the independent variables are potentially present-and determine the settings of independent and dependent variables.) Each run of the model represents a different random selection of cases; I will refer to these runs as small studies. Each run typically produces between 0 and 4 cases of revolution. (If we wanted to generate about 10 cases of revolution for scrutiny we would need a run of about 350 cases.) Table 2 represents one sample data set produced by the model.
We can also construct a large data set composed of a random selection from the small studies. The large data set described below is a set of 700 random settings of the variables A-F, along with a computed value for R in line with the statistical-causal relations between the independent variables, intermediate variables, and dependent variable. Statistical analysis of patterns of association within this data set will be provided below.
The question, now, is this: what inferences about causation can be derived from these samples? How could we interrogate this data so as to reconstruct the underlying causal processes? We can pose several interesting questions to this population of possible worlds:
1. Can we identify some or all of the causal relations among the variables?
2. Is it possible to identify any necessary conditions for the occurrence of R? (B and D are still necessary conditions; the question is whether this is detectable.)
3. What correlations exist between the occurrence of revolution and various of the variables?
4. What is the conditional probability of revolution, for all combinations of the independent variables?
5. Can we deduce the true causal story (the causal diagram that led to the construction of the set)?
A full account of the underlying causal relations requires:
a. the structure of the causal tree: disjunctions, conjunctions, and layers
b. the weights of the causal links (probability of transition)
c. the weights of the variables (probability of occurrence of each independent variable)
Are there analytical techniques that would permit the researcher to infer the underlying causal relations given an unlimited number of cases? And how quickly do these techniques degrade in face of limited data?
Let us begin by considering the utility of frequency and correlation information in constraining the causal analysis. It is possible to determine the weights and independence of the independent variables directly by computing frequencies and correlations. This establishes the weights of the variables. The more difficult problem is to determine (1) which variables are causally related to the dependent variable, and (2) how causal variables are grouped within the underlying causal diagram.
The most direct approach to determining the causal relevance of a set of factors is to compute the correlation matrix for all variables, and then attempt to identify causal connections by tracing non-zero correlations. The null hypothesis (the hypothesis that X is not causally related to R) implies that there will be no correlation between X and R. Figure 5 provides the correlation matrix of one run of the statistical model (N=51). And Figure 6 provides the corresponding correlation matrix for a large data set produced in the same way (N=700). Both tables show suggestive correlations between R and other variables. R is most strongly correlated with "social unrest" and "state crisis"-reflecting the fact that these are necessary and jointly sufficient for increasing the probability of R in this world. Moreover, the strong correlation would be suggestive of a causal relation even absent knowledge of the true causal laws of the model. R is also strongly correlated with "local organization" and "weak institutions"-once again reflecting that these variables are necessary conditions for R. R is significantly correlated with "food crisis", "war", "exploitation", and "economic crisis." This correlation is weaker, reflecting the fact that these are disjunctive conditions: either food crisis and exploitation is necessary, and either war or economic crisis is necessary, for R. So the correlation matrix does provide suggestive evidence of causal relations. In the small study, however, the level of statistical significance of these correlations is quite low. The statistical significance of the correlations in the large sample is high; however, it is unrealistic to suppose that it would ever be possible to empirically generate 700 separate cases for study. Finally, none of this data allows us to mechanically reproduce the true causal story, since these data do not permit us to sort out causal dependence among the variables. [5]
| A | B | C | D | E | F | G | H | R | |
| foodcr | locorg | war | weak inst | exp | econcr | socunr | statcr | rev | |
| foodcr | 1.000 | 0.142 | 0.007 | 0.051 | -0.007 | 0.021 | 0.212 | 0.101 | 0.072 |
| locorg | 0.142 | 1.000 | 0.031 | 0.052 | 0.178 | 0.242 | 0.735 | 0.089 | 0.299 |
| war | 0.007 | 0.031 | 1.000 | -0.031 | -0.190 | -0.153 | -0.054 | 0.290 | -0.116 |
| weakin | 0.051 | 0.052 | -0.031 | 1.000 | -0.178 | 0.215 | 0.158 | 0.413 | 0.209 |
| exp | -0.007 | 0.178 | -0.190 | -0.178 | 1.000 | -0.083 | 0.285 | -0.160 | 0.116 |
| econcr | 0.021 | 0.242 | -0.153 | 0.215 | -0.083 | 1.000 | 0.245 | 0.504 | 0.427 |
| socunr | 0.212 | 0.735 | -0.054 | 0.158 | 0.285 | 0.245 | 1.000 | 0.139 | 0.406 |
| statcr | 0.101 | 0.089 | 0.290 | 0.413 | -0.160 | 0.504 | 0.139 | 1.000 | 0.506 |
| rev | 0.072 | 0.299 | -0.116 | 0.209 | 0.116 | 0.427 | 0.406 | 0.506 | 1.000 |
Figure 5. correlation matrix-small study
| A | B | C | D | E | F | G | H | I | R | |
| foodcr | locorg | war | weak inst | exp | econcr | socunr | statcr | rand- om | rev | |
| foodcr | 1.000 | 0.004 | -0.026 | -0.010 | 0.021 | -0.034 | 0.046 | 0.004 | 0.002 | 0.033 |
| locorg | 0.004 | 1.000 | 0.022 | -0.035 | 0.002 | -0.015 | 0.679 | 0.011 | -0.054 | 0.208 |
| war | -0.026 | 0.022 | 1.000 | 0.002 | 0.019 | 0.069 | 0.084 | 0.411 | 0.007 | 0.240 |
| weakin | -0.010 | -0.035 | 0.002 | 1.000 | -0.020 | 0.036 | 0.009 | 0.403 | -0.058 | 0.191 |
| exp | 0.021 | 0.002 | 0.019 | -0.020 | 1.000 | -0.029 | 0.219 | 0.004 | -0.038 | 0.061 |
| econcr | -0.034 | -0.015 | 0.069 | 0.036 | -0.029 | 1.000 | -0.012 | 0.328 | -0.008 | 0.117 |
| socunr | 0.046 | 0.679 | 0.084 | 0.009 | 0.219 | -0.012 | 1.000 | 0.053 | -0.005 | 0.306 |
| statcr | 0.004 | 0.011 | 0.411 | 0.403 | 0.004 | 0.328 | 0.053 | 1.000 | -0.012 | 0.475 |
| random | 0.002 | -0.054 | 0.007 | -0.058 | -0.038 | -0.008 | -0.005 | -0.012 | 1.000 | -0.028 |
| rev | 0.033 | 0.208 | 0.240 | 0.191 | 0.061 | 0.117 | 0.306 | 0.475 | -0.028 | 1.000 |
Figure 6. correlation matrix-large study
| small study | large study | ||||||||
| freq | prob | chi sq | test | freq | prob | chi sq | test | ||
| R | 3/51 | 5.88% | 22/700 | 3.14% | |||||
| R|A | 1/11 | 9.09% | 0.260 | 0.610 | 9/227 | 3.96% | 0.75 | 0.388 | |
| R|B | 3/21 | 14.29% | 4.550 | 0.033 | 22/300 | 7.33% | 30.29 | <.0001 | |
| R|C | 0/9 | 0.00% | 0.680 | 0.409 | 14/108 | 12.96% | 40.46 | <.0001 | |
| R|D | 3/30 | 10.00% | 2.230 | 0.135 | 22/329 | 6.69% | 25.61 | <.0001 | |
| R|E | 3/42 | 7.14% | 0.680 | 0.409 | 21/578 | 3.63% | 2.62 | 0.106 | |
| R|F | 3/13 | 23.08% | 9.320 | 0.002 | 14/232 | 6.03% | 9.53 | 0.002 | |
| R|I | 5/209 | 2.39% | 0.55 | 0.458 | |||||
| R|~I | 17/491 | 3.46% | 0.55 | 0.458 | |||||
| R|A&B | 1/6 | 16.67% | 3.440 | 0.179 | 9/98 | 9.18% | 21.21 | <.0001 | |
| R|B&D | 3/13 | 23.08% | 9.320 | 0.009 | 22/135 | 16.30% | 95.06 | <.0001 | |
| R|D&F | 3/10 | 30.00% | 13.070 | 0.001 | 14/115 | 12.17% | 39.63 | <.0001 | |
| R|A&D | 1/7 | 14.29% | 2.070 | 0.355 | 9/105 | 8.57% | 18.67 | <.0001 | |
| R|C&D | 0/5 | 0.00% | 2.420 | 0.298 | 14/51 | 27.45% | 109.81 | <.0001 | |
| R|ABCD | 1/7 | 14.29% | 3.910 | 0.277 | 5/6 | 83.33% | 193.68 | <.0001 | |
| R|BE | 3/19 | 15.79% | 5.370 | 0.068 | |||||
| R|ABDF | 8/19 | 42.11% | 112.88 | <.0001 | |||||
| R|BECD | 14/20 | 70.00% | 311.08 | <.0001 | |||||
| R|BEDF | 13/41 | 31.71% | 128.53 | <.0001 |
Figure 7. Selected conditional probabilities of revolution-small and large study
A second statistical approach involves the idea of statistical relevance: if condition X is causally relevant to the occurrence of R, then it should be the case that P(R|X) P(R) (Salmon 1984). So we can look for causal factors by examining various conditional probabilities and looking for statistically significant divergences from the prior incidence of revolution. This approach permits us to discern the causal relevance of individual factors; but it also allows us to tease out relations of causal dependence within the system of causal factors. Let us now probe this data set by examining conditional probabilities for the various conditions identified here (figure 7).
Do the conditional probabilities provide any clue about the structure of the causal tree? We can make a start by sorting variables (as far as possible) into groups of factors probably lying on the same causal pathway. Two factors are causally independent if each transmits its effect to the result independently from the other. If X and Y lie on the same pathway they are not causally independent; rather, the influence of either depends on the presence of the other. If they lie on separate pathways, by contrast, they are causally independent; each exercises its influence independently from the presence or absence of the other. We can use conditional probabilities for all factors and combinations of factors to investigate causal independence. If X and Y are causally independent then we can compute the Baysian conditional probability of their conjunction according to the following formula:
p(R|X&Y) = 1 - [1-p(R|X)] * [1-p(R|Y]
(For low probabilities this simplifies to the sum of p(R|X) + p(R|Y).) If there is a 50% chance of revolution resulting from war and a 50% chance of revolution resulting from economic crisis and if each has its own independent effect on the probability of revolution, then when both occur there is a 75% chance of revolution (or in other words, a 1 in 4 chance of escaping revolution).
If the causal properties of X and Y are not independent, this means that the conditional probability of R given X&Y is different from (generally greater) than this Baysian expectation. Consider this simple example. Suppose that
X&Y&Z =[.75]=> R p(X)=.3, p(Y)=.4, and p(Z)=.02
We can now compute the expected conditional probabilities.
p(R|X) = .4*.02*.75 = .0060
p(R|Y) = .3*.02*.75 = .0045
But the conditional probability of R given X&Y is greater than expected (.0105):
p(R|X&Y)= .02*.75 = .0150
So X and Y are not causally independent in their effects on R; instead, we can infer that they lie on the same causal pathway (or fall within the same INUS condition).
We can now subject this set of conditional probabilities to a test of independence of factors. For each pair of factors look at the 3 relevant conditional probabilities. If p(R|X&Y) > expected, then provisionally conclude that X and Y fall on the same path and are necessary conditions on that path. If not then provisionally conclude that they lie on separate paths and neither is necessary condition to the causal powers of the other. Next look at conditional probabilities of sets of 3, then 4, then 5, then 6 factors. In each case rank sets by their frequencies. The strategy of this approach is to continue to test 2-tuples, 3-tuples, 4-tuples, 5-tuples, and 6-tuples until we reach the point where the conditional probabilities at level n are what would be expected from level n-1. This process should allow us to sort out the approximate structure of the causal diagram. Once all factors are sorted in this way we will have identified the main branches of the causal diagram.
The conditional probabilities provided in figure 7 point our causal analysis in the right direction. The conditional probabilities of R given each of the single factors (A-F) are statistically different from the background incidence of R; whereas the conditional probability of R given I (the control random variable) is-as it should be-approximately equal to the prior incidence of R-indicating that I is not causally relevant to R. Conditional probabilities for various pairs of factors increase the probability of R: for example, the incidence of R given war and weak institutions rises to 27.5% (from a prior incidence of 3.1%). Once again, however, we have a serious problem of statistical significance in the small study; only the conditional probabilities of revolution given economic crisis and revolution given weak institutions and economic crisis are statistically significant. (Moreover, these estimates substantially overestimate the true population probability, as we know by comparing these estimates with the corresponding values derived from the large study.)
The analysis provided here supports a mixed assessment of the potential of inductive methods to discover social causal relations. On the one hand, if we have abundant data (in the form of a large number of cases in which the values of all the variables are known), then the tools of correlation analysis and conditional probability comparison shed valuable light on the causal structure of the phenomena under study. On the other hand, these results fall far short of a full account of the underlying causal order. Several more specific observations follow as well.
1. If we had a sufficient number of cases then statistical associations among A-F and R would give a clear indication of the causal relations among these variables. These associations would strongly constrain hypotheses about the underlying causal relations. Analysis of conditional probability of R given various combinations of factors can in principle sort out relations of causal dependence among the variables (thus defining the groups of factors present in separate causal pathways). However, there are few social-science research problems which provide such a range of data.
2. Given data limitations it appears dubious that there are analytical techniques that permit us to infer the underlying causal relations. In spite of the fact that the causal properties of the system are entirely expressed in the data set, we could only arrive at the causal diagram through the formulation of hypotheses about the possible relations among factors. These hypotheses can be evaluated through data available in the data set, but they cannot be deduced from the set. These data serve to constrain causal hypotheses. But they do not suffice to replace such hypotheses. The researcher still needs to consider hypothetical causal chains, which can then be tested against new evidence.
The central conclusion of the discussion presented here is that the empirical procedures associated with NSC and IR standards (Mill's methods and its generalizations, and various tests of statistical association) almost always radically underdetermine the true causal story for a given ensemble of phenomena. Therefore it is necessary to put forward theories of causal mechanisms (hypotheses of causal pathways) whose implications for NSC and IR standards can then be tested. This brings us back to the topic of causal realism: the goal of causal analysis is to identify the causal mechanisms that link cause and effect. The upshot of the analysis provided here provides another compelling reason for adhering to causal realism, however: it is only on the basis of hypotheses about underlying causal mechanisms that social scientists will be able to use empirical evidence to establish causal connections. This in turn brings us to a better understanding of the role of social theory in social research; for it is a central function of social science theory to offer empirically justified accounts of a wide range of potential social mechanisms. Causal realism thus demands social theory-collective action theory, theory of bureaucracies and institutions, class conflict theory, economic geography, rational choice theory, theory of social-property regimes, etc.-since we need to have an analysis of the causal powers of the various factors in order to account for the links in the causal diagram.
This observation in turn sheds light on Theda Skocpol's method in Social Revolutions (1979). The analysis above suggested that her study depended on too few cases. But in fact her method really consists of two parts; it is not exclusively comparativist. First, she is unmistakably pursuing a comparative study of revolution, based on a small number of cases. But second, she is canvassing widely within social science theory to arrive at theories of particular causal links-e.g. between war and state stability, or between intra-elite conflict and state stability. These theories are not inherently based on comparative study. The result is that Skocpol is able to reconstruct the causal pathways represented by various cases, because she knows something about the causal powers of the various factors. Social theory thus provides a source of hypotheses about causal mechanisms that can then be probed and evaluated using a comparative methodology.
These observations shed light as well on the issue of whether social phenomena embody a causal order. We have surveyed several techniques that permit us to discern suggestive causal relations among observable social phenomena. And we have noted the importance of employing social theory to arrive at hypotheses about underlying causal mechanisms. The chief caution that emerges from this account, however, is that social scientists are rarely in the position of data richness that would permit them to make confident assertions about causal relations among social phenomena.
Another approach is to examine a reasonable set of positive cases (e.g. 20); form the truth-functional disjunction that results; and attempt to simplify this sentence. (This approach will only be fruitful if we can assume causal closure, and therefore no false positives). This approach generalizes Mill's method of similarity; it attempts to eliminate factors that are not INUS conditions through logical simplification of the extensive INUS statement. This approach should take us back in the direction of a causal diagram, with necessary and sufficient conditions emerging. This approach will only work on the assumption of causal closure-that we have identified all causally relevant factors. (Otherwise there will be occurrences of R that do not correspond to a causally productive setting of A-F.)
This approach does not rely on statistical regularities, but rather derives from logical analysis of the INUS assumptions. Factors that drop out of the causal sentence are not causally relevant to the production of R.
Consider Winston Hsieh's treatment of revolution in the Canton Delta: Hsieh finds that locale affects level of mobilization. In order to make sense of this link, however, he turns to Skinner's theory of the spatial organization of social activity imposed by the hierarchy of central places.
We ought not expect that we will produce a general theory of the effect; this will be unavailable if there is a complex causal diagram underlying the occurrence of rebellion. Rather, we will arrive at an analysis of the several pathways through which revolution can come about; and we will deploy a variety of theories to account for the causal linkages identified in the several pathways.
References
Little, Daniel. 1991. Varieties of Social Explanation: An Introduction to the Philosophy of Social Science. Boulder, Colorado: Westview Press.
Mackie, J. L. 1974. Cement of the Universe . London: Oxford University Press.
Mill, John Stuart. 1950. Philosophy of Scientific Method. New York : Hafner.
Ragin, Charles C. 1987. The Comparative Method: Moving Beyond Qualitative and Quantitative Strategies. Berkeley: University of California Press.
Rotberg, Robert I., and Theodore K. Rabb, ed. 1989. The Origin and Prevention of Major Wars. Cambridge: Cambridge University Press.
Salmon, Wesley C. 1984. Scientific Explanation and the Causal Structure of the World . Princeton: Princeton University Press.
Skocpol, Theda. 1979. States and Social Revolutions. Cambridge: Cambridge University Press.
Williamson, Jr., Samuel R. 1989. "The Origins of World War I". In The Origin and Prevention of Major Wars. See R. I. Rotberg and T. K. Rabb.
[1] This is rather different, then, from the problem of theoretical explanation in natural science, where the goal is to arrive at theoretical hypotheses about unobservable entities and processes. Here the anti-Baconian stand is highly persuasive; there is no mechanical procedure that could lead us from a data set to a theory of the underlying processes.
[2] John Mackie analyzes causal relations in terms of the notion of an INUS condition: a condition that is an "insufficient but non-redundant part of an unnecessary but sufficient condition" (Mackie1974:62).
[3] The following converges with Charles Ragin's "boolean treatment" of comparative methodology (Ragin 1987). I find his discussion more obscure than it need to be, however.
[4] To relax the assumption of causal closure it would be necessary to add one additional alternative path to the diagram, representing the fact that revolution may result from different causes altogether, and then assign a probability to this pathway. This would mean that revolution will sometimes occur even when each of the independent variables is absent. If this probability is low relative to the probability of revolution produced by the independent variables it should still be possible to discern the causal relations among those variables and revolution.
[5] It may be possible to use a neural network analysis to produce a causal diagram based on training supplied by the cases. This would give some clue about the number of layers and the structure of the causal tree. This approach uses extensive computational resources to provide non-parametric measures of association among a system of values, producing a predictive weighting of the variables in relation to the dependent variable.
