Return to Home Page

 

Evidence and Objectivity in the Social Sciences

 

            The concepts of objectivity, truth, and the authority of empirical standards have come under serious challenge by some critics of the social sciences in the past several decades.  Feminist critics charge that the concepts and methods of the social sciences reflect an essential patriarchalism that discredits the objectivity of social science knowledge.[1]  Marxist critics sometimes contend that the social sciences are enmeshed in a bourgeois worldview that makes objectivity impossible.[2]  And post-modernist writers seem to disdain the ideas of truth and objectivity in the social sciences altogether, preferring instead the slippery notions of multiple discourses and knowledge/power.

            Against these deconstructionist views I believe that a core aim of scientific research is to arrive at objective, true beliefs about the subject matter of the discipline: about what sorts of entities are to be found, what their properties are, and what causal relations obtain among them.  Science aims at producing knowledge about natural and social phenomena.  And this aim brings with it a concern for truth, a concern for rational standards of belief assessment, and a commitment to the notion that the standards of belief assessment are conducive to truth.

            These aspirations towards objectivity encounter a number of skeptical concerns.  For example:  Scientific disputes are inherently underdetermined by the evidence.  There are no pure "facts," but only facts as couched in one conceptual system or another.  There are no pure observations, but rather observations couched in a theory-laden vocabulary.  Theories bring with them their own empirical criteria, which bias the findings in support of them.  The relations between observation and theory are hopelessly circular, with theories generating the observations that supposedly support them.  Research projects are guided by antecedent assumptions about the structure of the phenomena which shape the eventual empirical findings in an arbitrary way. Scientific research communities are regulated by other criteria altogether (individual career advancement, the political demands of funding agencies, etc.) rather than epistemic criteria (evidence, logical coherence, etc.).  Social phenomena are not objective in the first place, but rather defined by the fluid and changing intentions, meanings, and beliefs of the participants and observers.  All observation in social science requires the interpretation of behavior, so there are no brute facts at all (C. Taylor, 1985); the investigator constructs the world he observes (Berger, 1966); or all social observation depends upon the perspective of the investigator, so that there are no perspective-independent facts. 

            These points are intended to reduce the appeal of the claim of objectivity of empirical reasoning in social science.  They contribute to a conception of social science that, if accepted, would radically undermine the claims of objectivity, empirical control of belief, and rigor which science claims for itself, and that would emphasize non-rational factors in the development of science.  However, examination of the current practice of social scientists in a wide range of disciplines does not support such a non-rational theory of science.  Instead, it is possible to discern a clear set of empirical procedures in the various disciplines that are well-designed to collect and analyze empirical data.  And it is possible to trace through the logical relations that obtain between the types of data that are collected and the more abstract or hypothetical claims of the disciplines.  These findings suggest a level of objectivity and empirical rigor that is consistent with a sophisticated empiricist theory of science.

            My purpose here, then, is to give a robust defense of an empiricist philosophy of social science.  I defend the idea that objectivity is attainable within the social sciences.  I maintain that it is possible to arrive at true hypotheses and theories in the social sciences, in the demanding sense that the social world is approximately as some of our hypotheses and theories describe it to be.  And I affirm the capacity of the multitude of empirical methods and procedures through which social scientists interrogate the social world to test, falsify, and confirm their hypotheses and theories.  None of these claims invokes the idea of certainty or infallibility, and my view of social science method does not deny that social scientists often manifest bias, preconception, and distortion in their investigations and concepts.  What I do maintain, however, is that the empirical methods of the social sciences serve as a substantial check on these deficiencies, and that over time it is reasonable to expect improvement in the verisimilitude of our social science hypotheses and theories.  Social science, then, contains a body of knowledge of the social world: true beliefs based on appropriate standards of warrant.

Objectivity

 

      What is objectivity in social science?  What might be meant by the claim that a given theory represents an objective scientific analysis of a range of social phenomena?  Debate over the objectivity of social science has often combined a variety of separate theses:

 

1.      There are social facts that are independent of the concepts and theories of the scientist which the theory is intended to uncover—that is, that there is an objective social world.  (ontological objectivity)

2.      It is possible for a theory of a given range of social facts to be well-grounded on the basis of the right sorts of reasons (empirical and theoretical adequacy).  (epistemic objectivity)

3.      Social facts are independent of the states of consciousness of participants.

4.      Scientific inquiry can be value-free and interest-neutral.

5.      Scientific inquiry tends to converge around a consensus among all researchers over the properties of the world as a result of further empirical and theoretical research.

 

Thesis (1) contradicts conceptual relativism.  Thesis (2) contradicts or finesses a family of underdetermination arguments within the philosophy of science.  Thesis (3) divides materialist social science from interpretation theory and verstehen sociology.  Thesis (4) upholds the position that it is possible to exclude value commitments from the conduct of science.  And thesis (5) asserts that science progresses towards a higher degree of agreement among researchers.

      We may dispense quickly with (4).  It is unquestionably true that scientific research is interest-relative: what particular features of the social system, what aspects of action, and what causal processes, are selected for scrutiny and explanation, are dependent on the interests—both intellectual and moral—of the investigator.  Further, it is plain that scientific reasoning presupposes a set of normative commitments—for example, to the primacy of empirical evidence over religious authority.  But Weber’s treatment of this issue is convincing; these points do not diminish the objectivity of science (Weber, 1949, pp. 74-80).  Once having defined the program of research, it is still possible to arrive at an objective analysis of the subject matter.

      Thesis (1) represents a general metaphysical view of the social world, in that it asserts the mind-independence of various kinds of social processes, structures, etc.  First, a scientific theory may attempt to identify underlying processes, structures, mechanisms, and the like, whose laws explain the observable data.  This goal presupposes that social phenomena are the result of a set of law-governed, objective social processes which the social scientist can discover and map out.  (Call this the realism component.)  (It might be noted that (1) does not commit one to methodological holism.  The objective social facts referred to may well supervene upon facts about individual actions.) 

      Thesis (2) represents the view that scientific theories are put forward as being justified on the basis of a “scientific method.”  There need to be objective procedures in terms of which to compare competing theories and to provide empirical and logical arguments favoring one such theory over its competitors.  (Call this the justification component.) 

      Thesis (5) represents the view that scientific inquiry progresses towards consensus among members of a given research community, and that this consensus is best explained on the hypothesis that the consensus theory is true, and has been arrived at through reliable procedures of scientific inquiry.  (Bernard Williams describes this view briefly in Ethics and the Limits of Philosophy; 1985, chapter 8.)

      Various combinations of these components of objectivity in social science are possible.  For example, Weber appears to affirm (1) and hold that there are social facts; he denies (3), asserting that these facts are “subjective” in the sense that they depend essentially on the states of mind of the persons whose meaningful behavior constitutes them; and he accepts (2), holding that it is possible to offer theoretically and empirically well-grounded descriptions of these facts (Weber, 1949, chapter 2).  Nelson Goodman (1978) appears to contradict (1), maintaining that there are as many social worlds as there are schemes of concepts in terms of which to organize and describe experience.  Such a view is forced to reject (2) as well, since it maintains that there is no uniquely best theory of the world.  It would be possible to reject (1) while maintaining (2)—that is, to hold that there is a best social scientific theory of a given range of social phenomena, but deny that such a theory describes an independently existing set of social facts.  (For example, Putnam’s anti-realist arguments might be deployed here.)  The form of objectivity of social science that I will defend here affirms (1) and (2).  Concerning thesis (3), I hold that there is no need to choose.  Some social facts may be constituted by the meanings attributed to them by participants, while others may be meaning-independent.  And finally, I will argue that the procedures internal to various social science disciplines are sufficient to produce the sort of convergence of theoretical beliefs described in thesis (5).

      What, then, is centrally involved in the idea of objectivity in social science research?  I will take epistemic objectivity as the central issue is this debate.  Objectivity invokes an assertion of rational credibility for the theories advanced within the social sciences.  Note, however, that objectivity is not the same as certainty.  To say that I have objective knowledge about the statement “the apple is in the refrigerator” is not to say that I am certain of its truth.  It is rather to say two things: first, that I have good empirical reasons to believe that the statement is true; and second (at the meta-level), the fact that I have this body of evidence makes it probable that the statement is true.  In other words, it is possible to embrace both the obvious truth that scientific knowledge is fallible and that it is objective.

      Nor is the concept of objectivity best understood as invoking a pure, impersonal reflection of the world as it really is—the “view from nowhere,” in Thomas Nagel’s words (Nagel, 1986).  It is plain enough that human knowledge of the world is always from some perspective or another.  In my understanding, at least, the notion of objectivity rather involves two more modest ideas: first, that human beliefs are potentially true, and second, that there are standards of belief evaluation that permit us to assess the likelihood of a given ensemble of beliefs.  The idea of truth invokes a correspondence between thought and the world.  The world is taken to have fixed, objective properties—properties that are independent of the beliefs that we have about them.  And a belief is true just in case it (approximately) captures some of these objective features of the world.  The idea of warrant invokes the notion that there are standards of evidence that successfully—though certainly imperfectly—permit us to winnow our beliefs in such a way as to increase the verisimilitude of our system of beliefs over time.

 

What is an empiricist philosophy of social science?

 

            The central question to be addressed, then, is the issue of epistemic objectivity for social science research.  This requires that we offer an account of a convincing approach to an empiricist philosophy of social science: an account of the methods and warrant of social science inquiry.  There are many strands of empiricist philosophy of science.  Empiricism is often associated with the doctrines of the unity of science, reductionism, the covering law model of explanation, insistence on the centrality of generalizations as the core of scientific knowledge, and skepticism about the scientific credentials of the social sciences.  I will not defend any of these views; in fact, I believe that each of them is misleading or wrong in consideration of the social sciences (Little, 1991; Little, 1993). 

            The core of the empiricist conception of science is a cluster of views concerned with the confirmation of scientific hypotheses and beliefs.  Empiricist philosophy of science demands that scientific assertions must be provided with appropriate empirical support.  And the purpose of empirical support is to enhance the likelihood that these assertions are true: that is, that the world is as the belief asserts.  Empiricists are confident that empirical standards serve to eliminate false hypotheses and to give us some basis of confidence in the truth of other hypotheses.  The core values of empiricist philosophy of science, then, are the centrality of empirical standards of belief assessment, the possibility of objectivity, and the possibility of scientific truth.

            These general observations conform to the actual practice of most social scientists.  Much social science literature takes the form of straightforward empirical disagreements and arguments.  Social scientists are concerned to provide an empirical case for the conclusions that they draw, and a significant portion of social science research goes into the gathering and assessing of empirical data; the critical assessment of empirical findings of other researchers; and the marshaling of data through extended arguments to support a variety of conclusions.  This section will analyze some of the empirical characteristics of these debates.  A principal aim of this discussion is to provide a more discriminating analysis of the use of empirical reasoning in social science than is typical in philosophy of social science today.[3]

            Let us begin, then, with a brief general treatment of empirical reasoning.  A central aim of research in social science is to arrive at true factual beliefs about social and historical phenomena, and to arrive at sound explanations of these phenomena.  These aims are pursued through empirical research: through a set of discipline-specific procedures for collecting, analyzing, and evaluating empirical data.  Ideally the empirical procedures of a discipline ought to be truth-enhancing: they ought to lead the discipline over time to a more veridical set of beliefs about the phenomena.[4]  They should permit investigators to assign a degree of credibility to factual assertions and explanatory claims.  And they should permit the research community to attempt to resolve factual and explanatory disagreements on the basis of empirical evidence (as well as a number of other relevant factors--coherence, fit with other accepted results, etc.).  Thus the empirical procedures of a discipline, and the body of empirical findings to which they give rise, are crucial to the epistemic standing of the discipline.

            The central empirical problems in social science research arise in two general areas.  First, there is a wide range of problems concerning the discovery and evaluation of matters of fact.  How are factual claims about the social world arrived at and corroborated?  It is plain that different social science disciplines have developed specialized techniques and methods of data discovery that conform to the exigencies of data sources in the domain of investigation.  Different disciplines probe their empirical domains using different empirical techniques: field interviews, participant observation, archival research, linguistic analysis, collection of price data, and so forth.  So it is clear that the idea of methodological pluralism fits this aspect of scientific research.  The techniques of empirical investigation that are suited to anthropology are not appropriate to the discipline of economic history, for example.  Second, there are more familiar problems having to do with the inferences that social scientists draw from a given range of empirical data.  And here it is not the case that there is fundamental diversity across disciplines.  For there are only a small number of logical relations that may obtain between a domain of evidence and a hypothesis or theory that permit us to say that the evidence confirms or tests the hypothesis.  These modes of inference fall in several general areas.  First, there are problems of induction.  What sort of empirical research is needed to arrive at and evaluate generalizations in social science?  Second, there are problems of causal reasoning.  How are causal hypotheses to be evaluated in the social sciences and history?  And finally, there are problems of evaluating complex theories designed to explain a range of social phenomena.  How is empirical evidence used to evaluate theories in social science?

 

Primary research data

 

            In order to know anything about a domain of phenomena it is necessary to be able to make observations within this domain.  How do we know that there was a Roman empire?  We are able to observe a wide range of circumstances in the present that permit us to make inferences about the past: Roman monuments and buildings, documents dating from the Roman era, documents from subsequent eras reporting contemporary beliefs about Rome, the current properties of Romance languages, and the like.  None of these contemporary observations wear their implications for the past on their sleeve; but it is unproblematic to confirm that the monuments in the Roman Forum are constructed from a certain kind of stone, or that a given manuscript is about 2400 years old.  These contemporary observations constitute the domain of evidence available to the social science researcher; they constitute the data on the basis of which to evaluate social hypotheses.

            Consider some examples of some of the sorts of data available in current social science research.

 

    Archives of documents of the Chinese Communist Party during the Sino-Japanese War.

    Records of Chinese Imperial Maritime authorities recording grain prices over time.

    English parish registers recording births, marriages, and deaths over time.

    Interviews with participants in the Cuban missile crisis.

    Observation of agricultural practices in a Moroccan village over a period of six months.

    Interviews with corporate executives about their decision-making processes.

    Criminal court records.

    Archives of French census data.

    Maps of French landholdings in a region over time.

 

I assert that these data sources are representative of the empirical core of all social science research.  It is only on the basis of empirical investigation of matters that can be directly observed that we are able ultimately to arrive at and evaluate hypotheses about unobservable properties and relations.

            Use of primary data sources require a specialized set of skills and knowledge.  China historians, for example, make primary use of various types of archives--local county gazetteers, Imperial tax and legal records, rebel confessions compiled by official investigators, Communist Party documents, and the like.  Anthropologists, by contrast, are less dependent on written records, and more dependent on direct observation and conversation with members of the target community; and their training and specialized skills differ accordingly.  These examples suggest that the specific techniques of data-gathering and assessment vary widely from discipline to discipline, depending on the exigencies of the subject matter; and there are few generalizations that would characterize all these techniques.

            Primary data research has several important characteristics.  First, the object of examination must be directly observable.  [DL1] Second, it must be possible in principle for other investigators to reproduce the results.  And finally, the discovery and documentation of primary data depends on research methods that are highly discipline-specific.  Much of the training that researchers in the social sciences undergo takes the form of specialized techniques for acquiring and analyzing primary data: archives, interviews, artifacts, price and tariff records, parish registers, and the like.  And these sorts of sources constitute the empirical basis of all social science knowledge.

 

Facts

 

            Turn now to factual inquiry.  A fact is a singular statement that identifies one or more entities and attributes a property or relation to the entity or entities, with or without a reference to time.  A factual assertion is true just in case the entities to which it refers possess the properties and relations identified by the property and relation terms of the assertion.  The logical form of such a statement may be represented as follows:

 

            'a has the property P at time t.'

            'The population of Beijing in 1800 was 6 million.'

 

            'a bears the relation R to b at time t.'

            'Brazil had a higher infant mortality rate than Sri Lanka in 1980.'

 

There is no a priori limitation on what sorts of entities may be involved--in particular, no requirement of ontological individualism.  It is possible, therefore, to make factual assertions about abstract or complex entities--General Motors as a business organization, the rate of unemployment in the English economy, etc.  (From this analysis it follows that a factual assertion may go wrong in two distinct ways; it may fail to refer to an existent entity, and it may attribute a property to a real entity that the entity does not possess.)  An important type of factual assertion in social science is quantitative: a statement assigning a magnitude to some feature of economy or society (rate of profit, rate of unemployment, real wage).

            It is apparent that most social facts do not fall in the category of primary data.  We cannot directly observe the population of Beijing in 1850, the pattern of tenancy in northern France in the ancien régime, or the savings dispositions of American workers.  Rather, it is necessary to make use of primary data in such a way as to permit us to arrive at estimates of these factual circumstances.  The discovery of factual data is itself an important part of social scientific knowledge and research.  Thus establishing the techniques of irrigation in use in nineteenth-century Bali (Geertz, 1980), the dowry arrangements practiced in nineteenth-century Hong Kong (Watson, 1985), or the population history of England (Wrigley and Schofield, 1981) are significant tasks for empirical research, and the outcome of these researches will be the discovery of a set of empirical facts.  Our knowledge of the social organization of Bali, the lineage systems of Hong Kong, and the population of England is furthered by these research projects.

            The results of these investigations are put forward as true statements about the social world; they are offered as factual descriptions.  Typically, however, they cannot be directly confirmed or falsified.  Instead, it is necessary for the investigator to engage in an extensive program of empirical research in order to arrive at a credible factual judgment.  The outcome of such a program will be an organized and logically structured argument that details a range of empirical data and shows why this data makes the factual claim probable, plausible, or credible.  This circumstance draws our attention to an important feature of empirical reasoning in social science.  A great many of the findings of social science research are factual claims (e.g., "the population of Beijing in 1853 was thus and so") which can, however, only be empirically evaluated or supported by weighing a wide range of other empirical evidence, and which remain both approximate and contestable.  Even at this level, moreover, disagreements arise over what the facts are, and what techniques of data gathering and analysis are appropriate.

            How are factual claims arrived at and defended?  As argued with respect to primary data, here too the diverse disciplines contained within the social sciences possess a specialized set of research tools and techniques through which factual questions are investigated.  A chief component of graduate training in these fields involves acquiring mastery of the skills needed to apply these techniques--language skills, interviewing techniques, mathematical techniques employed in aggregating data, etc.  None of these techniques guarantees objectivity and truth; at the same time, however, practitioners in virtually all the social science disciplines provide convincing evidence of rigorous empirical investigation, and we generally have good reason to suppose that the specialist's research will arrive at reliable factual judgments (subject to the usual qualifications of imperfection, fallibility, and incompleteness).

            Let us consider a representative problem of methodology associated with factual research.  A fundamental problem in research design for a given empirical program is the formulation of a scheme of concepts in terms of which to describe and aggregate the data.  (This problem is analogous to the problem of "operationalizing" an abstract concept in other areas of science.)  This process unavoidably introduces an element of arbitrariness into the process.  The "standard of living" debate in English economic history is instructive in this context.  The fundamental question is this: did the working class standard of living improve or decline during the first fifty years of industrialization (1750-1800)? This question does not directly admit of evaluation, however, since the concept of the standard of living is a vague one.  One line of thought in the debate therefore uses the real wage of unskilled workers as a proxy for the standard of living.  To estimate the real wage we need information about working class consumption patterns (goods consumed), prices of these goods, and money wages during the period.  With accurate information on these variables it is possible to compute the real wage.  But each variable raises its own difficulties.  It is necessary to answer the following questions:

 

    What portions of the working class are to be considered?

    Do we mean to include the conditions of the working poor in Ireland?

    Do we mean to include agricultural labor as well as industrial labor?

    Is it necessary to consider the welfare of different strata (regional, occupational) of the working class--with the possible result that some benefited and some lost ground?

    What does the expression "standard of living" refer to?  Is the real wage an acceptable proxy?

    Is it desirable to give an account of the quality of life--deterioration of living conditions, conditions of work, health and sanitation conditions--in addition to purchasing power?

    If we are satisfied to consider only the real wage, how shall we arrive at a time series concerning this variable?

 

Different decisions on these various questions lead to different ways of formulating the problem of measuring the real wage, and these differences sometimes lead to substantially different answers to the most general question; on one formulation the real wage may be found to rise, while on an alternative formulation it is found to fall.

            Is such a factual claim as "the real wage held constant during the eighteenth century" in principle decidable, given full access to historical data?  At least these problems arise.  First, the particular interpretation given to the concept of the real wage is inherently contestable; any given formulation is put forward as being plausible, but other interpreters may favor constructions that differ in ways that turn out to make a difference.  This is true both at the level of providing a quantitative index (Crafts, 1985) and at the level of debate over the "quality of life" as a component of the real wage.  Second, the data available is radically incomplete, so it is necessary for the researcher to make various assumptions--e.g., that a given price series for grain is representative of other agricultural products, or that a batch of wage data for several towns is representative of the region as a whole--in order to make use of the data to support or criticize a given claim.  And finally, there are often alternative assumptions that can be made along the way of aggregating the data that lead to significantly different conclusions.

            These points show that factual claims like these can frequently only be interpreted in the context of the larger framework of analysis in which they are put forward, and that these issues are not decidable in any strong sense.  Instead, a conclusion on the behavior of the real wage is plausible or implausible, depending on (1) the plausibility of the research assumptions the investigator makes in formulating a particular representation of the real wage, and (2) the plausibility and comprehensiveness of the investigator's empirical argument, showing how the available stock of data leads to a particular result within his conceptual framework.

            It should also be noted that even highly factual research programs--projects aimed solely at uncovering the factual details of a given range of social or historical phenomena--are dependent on antecedent theoretical interests and assumptions.  Concerning any particular domain of social phenomena there are indefinitely many factual questions the researcher could pose.  Thus in considering the rural history of medieval England one might pursue any of these factual inquiries:  What were typical patterns of landholding?  What patterns of domestic life were to be found?  What sorts of demographic patterns were present--age of marriage, fertility rates, mortality rates, etc.?  What types of crops and cultivation techniques were in use?  How extensive was market activity?  What were typical patterns of consumption? How extensive was literacy?  What sorts of religious belief and institutions were to be found at the local level?  What sorts of regional variations were there on any of the above characteristics?  And so on indefinitely.  Each of these questions can be investigated through study of available primary sources; but it is obvious that no single study can pose all these questions.  It is not possible, that is, to engage in a comprehensive factual inquiry into medieval England; the investigator must select a manageable number of factual problems and research those.

            So the selection of a small number of topics for extensive empirical research is itself a significant choice, and one that is guided, among other things, by a judgment of the relative explanatory importance of various factors.  Historians within a Malthusian framework, for example, will be particularly interested in factors that affect fertility and real income; Marxist historians may be more concerned with detailed information about land tenure, cultivation techniques, and class relations; contemporary social historians may pay more attention to whatever information is available about popular lifestyles and domestic practices; and so on.[5]

            A common anti-positivist view about "facts" may be derived from these considerations: there are no facts that are independent from a framework of theory, quantitative models, background assumptions, and the like.  The facts and generalizations discussed here are unavoidably enmeshed in a set of assumptions and theoretical commitments--e.g., the concept of the real wage.  So it must be conceded that it is most often not possible to pose a factual inquiry except in the context of a developed system of specialized concepts, quantitative methods, etc. 

            This does not, however, undercut the rational standing of factual findings.  Once this machinery has been established, the investigator can interrogate the available sources and arrive at justified conclusions (for example, about what the secular trends in the real wage were).  The factual claim cannot be separated from its theoretical context; but taking this context into account, other researchers can make use of this data for developing and testing their own hypotheses and claims.

            Finally, it should be observed that factual beliefs may be assigned a range of levels of warrant within the research discipline, depending on the degree of confidence practitioners attribute to the belief on the basis of existing evidence.  Thus the historian may attach a very high level of warrant to the belief that there was a demographic shift from north to south in the Ming dynasty, a middling level of warrant to the belief that the Qing bureaucracy was stretched thin by population growth in the nineteenth century, and a low level of warrant to the view that China's imperial longevity depends on its organizational form.  What determines the level of warrant for a belief is the answer to this question: given the research available on the topic, how likely is it that the belief is nonetheless false?  That is, to what extent do existing results within the discipline constrain the belief?  (It should be noted that a low level of warrant for a belief does not imply that the belief is false, but rather that the evidence is insufficient to allow us to determine the truth or falsity.  Such a thesis might be described as "speculative" in the sense that it exceeds the current ability of the discipline to provide appropriate empirical evaluation.)  In consideration of this range of levels of warrant, a fact is a singular statement with a high level of warrant, and a good empirical argument is one that depends upon grounds with high warrant.

 

Inferences

 

            The procedures used in the discovery of primary data and facts are highly variable across social science disciplines.  There is not this wide diversity, however, when we turn to consideration of the problem of confirmation.  For there are only a few logical relations between a domain of evidence and a hypothesis that constitute warrant for the hypothesis.  First, it may be that the hypothesis or theory deductively entails a set of observational consequences.  So the hypothesis can be tested by examining the truth or falsity of its observable deductive consequences.  This is referred to as the "hypothetico-deductive" (H-D) method of testing non-observational hypotheses.[6]  A number of criticisms have been raised against the H-D method in points of detail.  First, there is the familiar logical problem that no amount of true observational consequences can establish the truth of the hypothesis that predicts these consequences.  At best the finding that the observational consequences of a hypothesis are born out in a wide variety of test circumstances confers inductive support on the hypothesis: we have greater warrant in believing the hypothesis to be true than was the case prior to gathering this evidence.

            Second, there is the "Duhem-Quine" problem: hypotheses and theories only have observational consequences when conjoined with large bodies of collateral assumptions (auxiliary assumptions, bits of theory from other areas, simplifying assumptions, idealizations, and the like).  So when a body of evidence is found to be inconsistent with deductive expectations, we have not succeeded in falsifying the hypothesis in question, but rather the long conjunction of hypothesis and collateral assumptions.  It is therefore possible to save the hypothesis by adjusting the collateral assumptions.  This raises the problem of adhocness and unfalsifiability: when is it scientifically legitimate to adjust assumptions, and when is this move simply an adhoc adjustment that renders the hypothesis unfalsifiable?

            These problems notwithstanding, the inferential force of deductive testing of scientific hypotheses is unquestionable.  The H-D method underlies the logic of experimentation and testing in both natural and social science research.  Hypotheses acquire warrant to the degree that they possess implications for observable circumstances and these implications are found to be true.  This is one of the chief means through which primary data and factual conclusions can be brought to bear on explanatory hypotheses in the social sciences.  Consider the problem of evaluating the neo-Malthusian hypothesis about population and economy (Wrigley and Schofield, 1981).  The hypothesis is that European population growth was regulated by shifting patterns of economic opportunity.  How might we attempt to empirically evaluate this hypothesis?  We would endeavor to arrive at a description of how the world would have been if this hypothesis were true; we would then attempt to identify test circumstances associated with that description; and we would design a research program to evaluate whether the test circumstances turn out as the hypothesis entails.  A reasonable test circumstance in the case of this hypothesis might be this: economic downturns will be followed by a decrease in fertility (with an appropriate time lag).  In order to evaluate the test circumstance we would need to identify one or more historical occasions for which primary data exists in which economic downturn occurs; we would then attempt to determine whether fertility behaved as expected.  If we find that fertility does in fact fall, then we have some support for the neo-Malthusian hypothesis.

            Consider another example.  General equilibrium theory conjoined with the assumption that China's traditional economy embodied a high level of competition entails that there will not be wage differentials for farm labor within a given market area.  It is possible to make use of various sources to estimate farm wage levels in different counties in North China in the 1930s.  If the resulting estimates show significant differentials, then GET and the assumption of competitiveness are jointly disconfirmed.

            So far we have considered the problem of testing a hypothesis through its deductive consequences.  Many instances of scientific confirmation involve probabilistic predictions, however.  Thus it may be that a hypothesis entails a probabilistic description of observational consequences: if the theory is true, then some outcomes are more probable and other outcomes are less probable.  In this case a single observation will not suffice to disconfirm the theory; however, if we collect a range of relevant evidence and find that the distribution of outcomes is substantially different from that predicted by the theory, then the theory is to some extent disconfirmed.  This is referred to as "inductive-statistical" confirmation.

            Suppose our hypothesis is that multi-ethnic societies are more prone to social unrest than single-ethnic societies.  This hypothesis does not entail that every multi-ethnic society is more prone to unrest than any single-ethnic society.  Rather, it implies a statistical distribution of cases.  If we construct a study in which there are a number of multi- and single-ethnic societies, our hypothesis entails that a partition of the cases along these lines will result in a significant difference in rates and levels of unrest in the two groups.  If we do not find this result to hold, then our hypothesis is disconfirmed.

            An important special case of empirical confirmation in the social sciences concerns causal hypotheses.  Mill's methods of difference and similarity (and more sophisticated versions of these methods) provide ways of using empirical evidence and observation to test causal hypotheses.  These are methods aimed at identifying the cause of an event by observing variations in antecedent conditions for repeated occurrences of the event.2  Suppose that we are interested in discovering the cause of an event P in a causal field of a range of possibly relevant factors {A,B,C,D,E}.  For vividness, suppose that the event P is the success of a union organizing drive, and the causal factors are: (A) falling real wages; (B) urban setting; (C) skilled labor force; (D) authoritarian management style; and (E) industrial company.  That is, we are interested in discovering a factor which is necessary and sufficient for the occurrence of P.  The method of agreement instructs us to find two or more cases in which P occurs and in which only one of the possible causal factors is present in all cases (factor A in figure 1).  (The letters 'p' and 'a' signify the presence or absence of the factor in question.)  In this example, then, we need to find two or more instances of union organization drives that lead to success, and then determine the state of factors A through E.  If the set of factors surveyed is exhaustive and if there is a single necessary and sufficient condition for the occurrence of P, then the factor that is present in every case must be the necessary and sufficient condition.  In this example, it is the "real wage" variable that is constant across the cases; so the method of similarity would lead us to conclude that the direction of change of real wages is the cause of success or failure of union organizing drives.

            Turn now to the method of difference.  In this instance we are instructed to find a pair of cases, in the first of which P occurs and in the second of which it is absent.  Once again we are to survey the set of relevant factors {A,B,C,D,E}.  If there is a single factor that covaries with P, we are authorized to conclude that A is the cause of P.  In figure 2 there are two cases, one (I1) in which P occurs and one (N1) in which P does not occur.  We now survey the two circumstances, and find that B, C, D, and E remain fixed through both cases, while P and A vary from the first case to the second.  We can conclude from this analysis that C and D are not necessary conditions for P, since they are absent in I1.  The only factor which is present when and only when P occurs is A.  If B were a sufficient condition for the occurrence of P, then P ought to have occurred in N1 as well. Therefore, the method of difference permits us to conclude that B is not a sufficient condition for the occurrence of P.

 

	P	A	B	C	D	E
I1	p	p	p	a	a	p
N1	p	p	a	p	a	a

Figure 1. Mill's method of agreement

 

	P	A	B	C	D	E
I1	p	p	p	a	a	p
N1	a	p	a	a	a	p

Figure 2. Mill's method of difference

 

            Do these findings permit us to conclude that A is a sufficient condition for P, though?  They do so only if we can assume that {A,B,C,D,E} is an exhaustive set of causal factors for the occurrence of P; otherwise it is entirely possible that the covariance of A and P is accidental.  But this is a highly unrealistic assumption; in the typical case it will be an open question whether there are other as yet unidentified causal factors.  If we do not know that {A,B,C,D,E} is exhaustive, then the best we can conclude is that only A out of the set {A,B,C,D,E} is potentially a necessary and sufficient cause of P; and only A, B, and E are potentially necessary conditions for P. In order to have further reason to suppose that A is sufficient and necessary we need to survey a number of other possible cases; ideally it will emerge that A always covaries with P, while none of B, C, D, or E are necessary for the occurrence of P.

            Moreover, it is apparent that these methods cannot handle complex causation and probabilistic causation.  Suppose that A causes P when in the presence of F and B causes P when in the presence of G.  Then there will be cases where A is absent, B is present, and P is present; there will be cases where A is present, B is absent, and P is present; and there will be cases where A, B, and P are all present.  The first such case would indicate that A is not a cause of P, while the second indicates that B is not a cause of P.  Likewise, suppose that A is the only cause of P, but it is a probabilistic cause: if A occurs, then there is a 90% chance that P will occur as well.  If our set of cases includes one of the rare instances where A occurs and P does not, the method of difference will exclude A as a cause of P.  Thus Mill's methods are well designed only for cases where we have single conditions that are necessary and sufficient for the occurrence of the outcome.  Moreover, these methods require relatively demanding conditions for their application: a complete list of potentially relevant causal conditions, a pair of observations in which P occurs and does not occur, and information about the occurrence or non-occurrence of each of the relevant conditions.  In spite of these limitations, however, Mill's methods underlies much reasoning about causation in the social sciences.

            Consider an example.  Suppose our hypothesis is that high unemployment in an election year is a sufficient condition for change of political regime in a democracy.  This hypothesis is strictly disconfirmed by a single negative instance--an election in a period of unemployment in which the incumbent wins reelection.  If our causal hypothesis is probabilistic rather than deterministic--e.g. "high unemployment makes change of regime more likely"--then we need a set of cases to evaluate the hypothesis.  We need to construct a study that identifies a number of instances of elections including a range of unemployment levels.  If we find that the incidence of change of regime is the same among cases in which there is high unemployment as in the general population of cases, then the data disconfirm the probabilistic causal hypothesis.

            In short, there are a small number of logical methods of empirical inference through which a hypothesis may be tested.  Evidence may be collected to examine whether the consequences of the hypothesis are born out in experience.  The central conclusion here is a positive one: there are rigorous and clear modes of inference through which scientific hypotheses can be tested, and hypotheses that have survived these tests have a greater probability of being true than do hypotheses that have failed them.

 

Underdetermination of theory

 

            A powerful basis for skepticism about scientific objectivity stems from the possibility of underdetermination: the idea that there are multiple systems of hypotheses that jointly possess all the same observational consequences.  On this view, there are always alternative theoretical hypotheses that would do equally well to explain and organize available empirical data.  This means that the theoretical order which the scientist "discovers" in the data is an artifact of the particular but arbitrary features of the theoretical presuppositions he brings to the investigation.  The problem of underdetermination can be put in a broadly philosophical form, as Quine does in Word and Object (Quine 1960)--Quine believes that it is logically possible that there are alternative and non-equivalent theories compatible with all possible observations, past, present, and future.[7]  But the problem can be put more tellingly in the circumstances of actual social science research.  Given the paucity of empirical evidence on most social science research topics, it is possible that the available evidence is insufficient to distinguish between competing hypotheses.  In this case the competing hypotheses can be said to be underdetermined by available evidence.

            Consider the hypothesis that agricultural productivity was rising in traditional China at a rate of about 2 percent per year.  Since population growth is estimated at about 2 percent, this hypothesis entails that per capita consumption of grain remained constant.  Now consider a competing hypothesis: agricultural productivity was rising at 3 percent; an elite landowning class was taking a rising share of the harvest; and peasant livelihoods were remaining constant.  Data showing that per capita consumption of grain was constant over a long period of time fails to distinguish between these hypotheses.  The data that would distinguish has to do with consumption over time of the landowning class.  But if this data is unavailable for some reason, then the basic hypothesis is underdetermined by available evidence.

            It is plain enough that many research questions in the social sciences are empirically underdetermined, in that available data fails to resolve the issue.  However, this point does not support currently fashionable forms of relativism and subjectivism in social science.  For the limits on empirical determinacy that arise in most areas of social research do not derive from general philosophical considerations (the ultimate indeterminacy of social phenomena, for example), but rather from the humdrum limits of practical research: limited availability of data on contested questions, imperfections of available data, limits on research resources, and the like. 

            Consider an example: competing explanations of the occurrence of peasant rebellion in late Qing China.  Elizabeth Perry (1980) analyzes the Red Spears rebellion in North China, while Susan Naquin (1976) analyzes the Eight Trigrams rebellion.  Both Perry and Naquin are concerned with peasant rebellions which have a strong component of a heterodox Buddhist sect (the White Lotus society) in circumstances of ecological crisis.  However, the two authors offer radically different explanations of these phenomena.  Perry analyzes her rebellion in terms which emphasize peasant rationality, survival strategies, ecological constraints and opportunities, and the like; while Naquin emphasizes the importance of the millenarian world-view, the religious motivations of the adherent, and particulars of White Lotus organizational resources.  Her account pays relatively little attention to material factors or rational decision making.  Our underdeterminist might maintain that this shows that there is no "true" analysis of either rebellion; rather, the data can be organized according to either framework.

            Against this view I would hold that the indeterminacy of such disputes is overstated.  There are ways of narrowing down the dispute between two such theories (e.g., Naquin's and Perry's).  The requirements of empirical and explanatory adequacy impose significant constraints on the correctness of a given line of interpretation.  The applicability of a given theoretical construct to a particular historical phenomenon can be evaluated with some degree of objectivity.  Two points in particular stand out, both supporting the possibility of objectivity.  First, concerning any particular point of controversy it is possible for social scientists to produce additional research findings that will serve to narrow the range of disagreement.  Recall our discussion above of the standard of living in the early Industrial Revolution. As this topic became controversial, additional empirical research shed new light on both the scope and the character of working class consumption.  The ultimate result is that the factual situation is now understood more clearly.  This example represents perhaps a paradigm example of progress within social science: a controversy arises which is theoretically important, further research is undertaken, and the controversy is narrowed or resolved.

            This example underlines the importance of distinguishing between local and global perspectives in considering the rationality of science.  In this case, on a particular empirical disagreement we see that unmistakable progress has emerged.  And it is probably true that most current empirical disagreements admit of eventual resolution through further research--if and insofar as the requisite research efforts are made.  But there is a Malthusian twist here: controversies multiply geometrically, while research resources multiply only arithmetically.  This implies, then, that, many questions, though resolvable in principle, must remain in fact unresolved; with the result that, from a global perspective, much current social science belief remains controvertible.

            These arguments show that there are major areas of social science where irresolvable underdetermination problems do not arise.  In these areas there continue to be local problems of empirical indeterminacy, insufficiency of data, and so forth; but these are problems that yield to further research and debate.  So it is worth emphasizing for philosophers that in many areas of social science, the problems of objectivity and rationality are the humdrum ordinary problems of scientific research anywhere--not special and intractable problems arising from the specifics of social phenomena or social method.

 

Conclusion

 

            This review of some of the empirical methods available to the social sciences is intended to support the idea that social science research can lead to hypotheses and theories that are approximately true of the social world, and that the empirical procedures of the social sciences often give us reason to accept those hypotheses and theories. 

            These arguments supporting the possibility of objective, empirically controlled social inquiry should not be understood overly broadly, however.  It is quite evident that there are areas of social science that are indeterminately vague, rhetorical, ideological, speculative, indifferent to empirical controls, and so forth.  My claim is only that social science can achieve a high level of rigor and empirical warrant, and that it should aspire to such standards.

 

References

 

Antony, Louise M. and Charlotte Witt, eds.  1993.  A Mind of One's Own: Feminist Essays on Reason and Objectivity.   Boulder, Colorado: Westview.

Berger, Peter L., and Thomas Luckmann. 1966.  The Social Construction of Reality.  Garden City, N.Y: Doubleday.

Braybrooke, David. 1987.  Philosophy of Social Science.  Englewood Cliffs, N.J.: Prentice-Hall.

Crafts, N. F. R. 1985.  British Economic Growth during the Industrial Revolution.  Oxford: Oxford University Press.

Geertz, Clifford. 1980.  Negara: The Theatre State in Nineteenth-Century Bali.  Princeton: Princeton University Press.

Glymour, Clark. 1980.  Theory and Evidence.  Princeton: Princeton University Press.

Hempel, Carl. 1966.  Philosophy of Natural Science.  Englewood Cliffs, N.J.: Prentice-Hall.

Lakatos, Imre.  1970a.  "Methodology of Scientific Research Programmes".  In Criticism and the Growth of Knowledge.  See I. Lakatos and A. Musgrave. 

Lakatos, Imre, and Alan Musgrave, eds.  1970b.  Criticism and the Growth of Knowledge.  Cambridge: Cambridge University Press.

Laudan, Larry. 1977.  Progress and its Problems.  Berkeley: University of California.

Little, Daniel. 1991.  Varieties of Social Explanation: An Introduction to the Philosophy of Social Science.  Boulder, Colorado: Westview Press.

Little, Daniel.  forthcoming.  "On the Scope and Limits of Generalizations in the Social Sciences."  Synthese.

Miller, Richard W. 1987.  Fact and Method.  Princeton: Princeton University Press.

Nagel, Ernest. 1961.  The Structure of Science.  New York: Harcourt, Brace & World, Inc.

Nagel, Thomas. 1986.  The View from Nowhere.  New York: Oxford University Press.

Naquin, Susan. 1976.  Millenarian Rebellion in China: The Eight Trigrams Uprising of 1813.  New Haven: Yale University Press.

Newton-Smith, W. H. 1981.  The Rationality of Science.  Boston: Routledge & Kegan Paul.

Perry, Elizabeth J. 1980.  Rebels and Revolutionaries in North China 1845-1945.  Stanford: Stanford University Press.

Popper, Karl. 1961.  The Poverty of Historicism.  London: Routledge & Kegan Paul.

Popper, Karl. 1965.  Conjectures and Refutations: The Growth of Scientific Knowledge.  New York : Basic Books.

Popper, Karl. 1968.  The Logic of Scientific Discovery.  New York : Harper.

Quine, Willard Van Orman. 1960.  Word and Object.  Cambridge: MIT Press.

Resnick, Stephen A. and Richard D. Wolff. 1987.  Knowledge and Class: A Marxian Critique of Political Economy.  Chicago: University of Chicago Press.

Rosenberg, Alexander. 1988.  Philosophy of Social Science.  Boulder, Colorado: Westview.

Roth, Paul A. 1987.  Meaning and Method in the Social Sciences: The Case for Methodological Pluralism.  Ithaca: Cornell University Press.

Taylor, Charles. 1985a.  "Interpretation and the Sciences of Man".  in Taylor (1985b).

Taylor, Charles. 1985b.  Philosophy and the Human Sciences: Philosophical Papers 2.  Cambridge: Cambridge University Press.

Watson, Ruby. 1985.  Inequality Among Brothers.  Cambridge: Cambridge University Press.

Wrigley, E. Anthony, and Roger S. Schofield. 1981.  The Population History of England, 1541-1871: A Reconstruction.  Cambridge: Harvard University Press.

Zalk, Sue Rosenberg and Janice Gordon-Kelter, eds.  1992.  Revolutions in Knowledge: Feminism in the Social Sciences.   Boulder, Colorado: Westview.

abstract

 

            The concepts of objectivity, truth, and the authority of empirical standards have come under serious challenge by some critics of the social sciences in the past several decades.  Feminist critics charge that the concepts and methods of the social sciences reflect an essential patriarchalism that discredits the objectivity of social science knowledge.  Marxist critics sometimes contend that the social sciences are enmeshed in a bourgeois worldview that makes objectivity impossible.  And post-modernist writers seem to disdain the ideas of truth and objectivity in the social sciences altogether, preferring instead the slippery notions of multiple discourses and knowledge/power.  This essay reaffirms the centrality of empirical constraints and the goal of objectivity within social science research.  It begins with an account of a defensible empiricist philsophy of social science.  It turns next to a consideration of the problems involved in making factual judgments in social science research.  It considers next the problems of inference faced by social science researchers: how do we derive conclusions about social facts on the basis of the sorts of evidence available to us?  The essay concludes with a reconsideration of Karl Popper's theory of falsifiability as a criterion of scientific credibility.