PhD Student in Information Science at University of Colorado Boulder
Ways of Knowing: Competing Methodologies in Social and Political ResearchMoses, J. W., & Knutsen, T. L. (2019). Ways of knowing: Competing methodologies in social and political research. Macmillan International Higher Education.
I take notes on this book differently than I have other books, using the reading guides provided by my Ways of Knowing course to document my reading here. As such, I leave open questions and opinions in my notes.
IntroductionThis book deals with the myriad of "ways of knowing," the debates surrounding how we come to understand the world and how this impacts the methods we choose to use.
This book uses the terms "naturalism" and "constructivism." In this case:
- Naturalism = Positivism
- Constructivism = Interpretivism
Methodology and MethodMoses and Knutsen refer to both naturalism/positivism and constructivism/interpretivism as methodologies. Methodologies impact how methods are chosen and used. Methodologies refers to the underlying logic of using certain methods. They lament that social scientists often “use the term ‘methodology’ as a fancy word for statistical methods. Methodology investigates “the concepts, theories and basic principles of reasoning on a subject” (pg. 4).
Methodology is often misused to imply method. Methods are research techniques, operational procedures.
They discuss how methodological dogmatism limits the ability of the scientist to see the different and often contrasting perspectives of reality. They cite interesting examples of Economics and Political scientists, which often present naturalist methodology as the only reality, particularly in the case of the 2008 recession, Brexit, and Donald Trump’s election (pg. 6) - how predictive can social science truly be?
The Three Muskateers of Metaphysics (The Philosophy of Science?)Metaphysics refers to the theory of nature or the structure of reality. How does this relate to the philosophy of science? Is the philosophy of science a metaphysics of science?
- Methodology: the way we acquire knowledge. The study of methods and which methods are appropriate for producing knowledge. The fundamental question driving methodology is: “How do we know?”
- Ontology: the study of being, the study of the basic building blocks of existence. The fundamental question driving ontology is: “What is the world really made of?” Marshall’s Concise Oxford Dictionary of Sociology (pg. 367) defines ontology as necessary to understanding the world, as how we understand the world is reliant on what assumptions we make about what can exist and how. This may include: persons, institutions, relations, norms, practices, structures, roles, etc.
- Epistemology: the study of knowledge and how it is justified or related to truth/belief. The fundamental question driving epistemology is: “What is knowledge?” How do we know what we know? Marshall’s Concise Oxford Dictionary of Sociology (pg. 153) points out two historically competing schools of thought: rationalism and empiricism. Rationalism seeks to reconstruct human knowledge through pure reasoning, reliant on formal demonstrations of logic and math. Empiricism prioritizes senses and experiences in constructing knowledge and human thoughts.
Open Questions About the Three MuskateersPrimarily, I find ontology confusing - how is ontology different from epistemology? If ontology asks what the world is really made of, and epistemology determines knowledge as, for example, experience, is the world not made of experiences? Methodology seems most clear, but also seems highly reliant on epistemology and ontology. After all, to ask how we know something, we also have to determine what kinds of knowledge we buy into and what the world is made of.
I had definitely not understood methodology as separate from method. I think I had attributed the definition of methodology to epistemology. Ontology was something I had associated only with the classification of things, such as a schematic order.
Thoughts on Methodological VarietyI like that they describe the “competing” methodologies of naturalism/positivism and constructivism/interpretivism as a spectrum, where the two polls are two “ideal” types that fit neatly into perfect categories. Often, scientists will not fit solely into one camp. As someone who leans interpretivist, I still find my approaches compatible with methods that may be most associated with a positivist methodology. For example, one can “interpret” quantitative data and statistics alongside qualitative data.
Naturalism/PositivismAlso known as: empiricism, positivism, behaviouralism.
Naturalism arose out of twentieth century approaches to social science which pushed for scientific legitimacy by embracing the worldview of the natural sciences (e.g., physics, chemistry) (pg. 7). The authors describe naturalism as dominating modern social science. Naturalism assumes there is a singular “Real World” independent of the human experience that can be uncovered through reliable and neutral observation. “Naturalists rely heavily on knowledge that is generated by sensual perception, such as observation and direct experience” (pg. 7). Reason and logic are meant to be supported by direct, recorded experiences in order to prove something is true. Naturalist social scientists have “embraced falsification, replication, and predictive capacity as the standards for evaluating their knowledge” (pg. 7).
The authors describe 6 features that summarize the naturalist approach:
- “There exist regularities and patterns in nature that are independent of the observer (e.g., a Real World)”
- The patterns can be observed and described objectively
- Observational statements about these patterns can be empirically tested “according to a falsification principle and a correspondence theory of truth”
- “It is possible to distinguish between value-laden and factual statements (and facts are, in principle, theoretically independent)
- In science, the general (nomothetic) is more important than the particular (idiographic) - generalizability?
- “Human knowledge is both singular and cumulative” (pg. 8)
Constructivism/InterpretivismInterpretivism is the most common alternative term, but the authors describe constructivism corresponding to “‘Gadamer’s hermeneutics, Habermas’s Critical Theory … French deconstructionists, post-structuralists’” (pg. 9).
Constructivist social scientists may be interested in the patterns not firmly rooted in some singular nature or reality, but of a world that seems ephemeral, cultural, and contingent on human agency. Constructivists view the observer’s perspective as influencing what is studied, how it is studied, and what is found/seen. They do not believe in a neutral or objective experience, but that all information/knowledge is channeled through the human mind, shaped by individual and cultural perspectives, challenging a singular Real World with “the possibility of multiple worlds” (pg. 9). When observing a singular something, constructivists believe different people see different things. They state that, due to the belief that social contexts fill all observation with meaning, constructivists generally use a broader set of epistemological tools (e.g., empathy, authority, mythology) than naturalists.
Unlike positivists, interpretivists try to understand the broader meanings and contexts of a phenomenon to better understand behaviors and artifacts, political understandings, etc. They are more hesitant to make claims about a singular truth.
They list the following qualities of constructivism:
- The world studied is not singular/independent of the observer and contains social facts
- Observations/experiences depend on the investigator’s perspectives and cannot be neutral or even consistent across studies
- Observations are biased and can be understood in different ways
- Factual statements are value-laden
- Knowledge gained through particular (idiographic) studies are valuable and do not need to be part of a larger generalizable (nomothetic) project
- There can be more than one way to understand and these understandings are valuable (pg. 10).
The Naturalist Philosophy of Science
Joists of Naturalism
- Ontology: the Real World independent of our senses. It exists regardless of if human beings can observe it or not. A statement is true if it accurately describes the Real World.
- Epistemology: Knowledge about nature is acquired through systematic observation. Knowledge about the laws of nature is acquired through the identification of associations (correlations). The ultimate purpose of science if to uncover regularities and define them in natural laws. Knowledge can be gained through observation and induction, reason and deduction, but most always be confirmed by empirical evidence. Human knowledge grows over time and is accumulated through confirmed correlations and increasingly accurate theories.
- Methodological: Regularities are observable through the systematic use of the human senses and sense perceptions can be analyzed through inductive/deductive reasoning.
Hierarchy of MethodsA firm hierarchy where the experimental method is best. When experimentation is not realistic, the second-best approach is the statistical method. If this is unreasonable, the scientist may use the weaker comparative method designed for a smaller number observations. Lastly, as a last case scenario, the scientists may resort to a case study or historiography.
Empiricists vs. RationalistsBritain’s 17th and 18th century philosophers were empiricists. Empiricists inductively rely on observations.
French 17th and 18th century philosophers were rationalists. Rationalists deductively based arguments on axioms, a statement for which is viewed as truth and thus no proof is required. The main role of this approach was to guarantee consistency through use of logic and a set of rules. Naturalists embrace rationalism.
Both schools believed in a Real World that is orderly and streamlined and that we have access to this world via senses.
Origins of NaturalismThe authors open the chapter by discussing the “first true application of the scientific method - of a process that involves systematic observation, scrupulous note-taking of things and patterns observed, and thoughtful efforts to make sense of it all” - Galileo Galilei’s “The Starry Messenger,” an observation of the night sky through a new tool called the telescope written in the early 15th century (pg. 15). He wrote it in contradiction to the traditional approaches to knowledge as deemed by the authority of the Church and Aristotle’s “Physica,” claiming the traditional approaches inhibited scientific discovery. Similarly, Johannes Kepler observed the movement on planets and tracked their orbit using mathematics, also against traditional knowledge procedures. Reliance on traditional authority impeded individual and free thought and observation. Isaac Newton would later synthesize Galilei and Kepler’s observations in “Mathematical Principles of Natural Philosophy.”
Moses and Knutsen trace a similar history through Francis Bacon’s work, which took issue with Aristotle’s “Organon,” inspiring later philosophers of science John Locke and David Hume (pg. 16). Bacon established a new methodology of science, which did not start from previous “truths” but instead relied on the process of induction and deduction.
Bacon saw the craftsman, who had few texts and wrote down notes to share and be tested and reworked, as an ant, gathering materials from its surroundings to build a larger whole. He saw the scientists as a spider, relying on deductive reasoning to build a web from pre-supposed knowledge. Both are attempting to formulate some known truth about the underlying reality of observations of the world.
- Deduction: builds on true and accepted claims (axoims), starting with general truths and proceedings through reasoning toward explanations. A top-down approach where theory guides empirical study.
- Induction: builds on sensory observation by starting with empirical observation to generate more general theories (a bottom-up approach).
However, Bacon believed in synthesizing these two perspectives (the perspective of the be) to both gather and observe, but also digest and refine observation into theory through reasoning. Bacon established the naturalist stance that “(i) only direct observations supply us with statements about the world and (ii) true knowledge is derived from observation statements” (pg. 18). He believed human senses could not always be trusted, and thus must also be complimented by common sense and reason.
Hume refined Bacon’s perspective that the human mind and senses were fallible, thus doubting the validity of induction and whether causal analysis was possible. He argued our imagination, which allows us to rearrange and combine the simplistic ideas we observed, are what lead to causality. We cannot actually observe causality itself (pg. 20-21). Hume believed science should not attempt to explain facts, but should focus on describing them and documenting their regularity. Hume believed we should only attempt to draw correlations among observed facts. For Hume, facts were empirical evidence and values were normative knowledge, based on values and beliefs. We can say nothing certain about values, therefore, Hume believed there could be no basis of science on values. Values were deemed philosophical and theologian, not scientific. This view on causality versus correlation is interesting to me from the perspective and arguments of causal inference in machine learning - which many seek for explainability, but many doubt, potentially due to this lack of direct observation?
TheoryA theory seems to be an overarching truth which is universal until proven untrue. Rorty’s representational theory of perception, also Popper’s correspondence theory of truth, states: “‘ a theory or a statement is true, if what it says corresponds to reality’” (pg. 24). Moses and Kutsen write that a theory is “a set of (verified) correlations , logically or systematically related to each other … [hinging] on a statement which says that one phenomenon (or one class of phenomenon) is connected in a certain way with another phenomenon (or class of phenomenon) … a map of associations” (pg. 24-25). Theory is not a lens in the way I am familiar with in interpretivism, but a guiding reality that has been formulated from many proven and tested observations.
Cartesian Scientific MethodDescartes developed the notion of Cartesian dualism, or that the world of the mind is separate from the world of the body. In terms of observation, he argued that sense experiences belonged to their own outer world, separate from the inner world of the mind. Like Bacon, Descartes believed in both a deductive and inductive synthesis and the fallibility of the senses. Descartes believed there was a simple streamlined reality beneath a superficially complex world. He recommended two epistemological principles for understanding the simple world: systematic doubt and reductionism.
- Systematic doubt: he asks what is possible to know? He states one must “cleanse [their] mind of all former beliefs, because many of these are bound to be false” (pg. 26). All claims should be treated as if they are false and we should only believe it if we are certain it is true. Any doubt about a claim at all should mean rejecting the claim.
- Reductionism: Begin investigations by dividing every argument into its component propositions and ask of every proposition: how do I know this is true? Reject any proposition that cannot be verified completely. Start with the simplest propositions and then move onto the most complex ones. This will reduce the number of propositions to a few true core claims, a solid foundation for building an argument (pg. 26).
He believed this method was the best method for reducing the clutter of sense perception so one could view the Real World. These principles are still basic foundations for naturalist methodology.
Durkeim's Sociology“Comte coined ‘sociology’ to designate the science that would synthesize all positive knowledge about society and guide humanity in its search for the ‘good society’” (pg. 29). He relied on the epistemological argument of empiricists and the historical argument of accumulating knowledge through historical evolution (pg. 28-29).
Durkheim urged sociologists to move away from the study or concepts to focus on the study of things. His most basic rule was to consider social facts as things, independent of individuals. A social fact had external power over individuals and groups. He argued social facts are more difficult to observe than natural facts, and what appears in our senses is often an illusion. His approach worked to fit positivism into social science. However, his view that sociology was inherently different from natural science actually opened the box to more constructivist approaches, leading to a severing of the link between the two (pg. 30).
Logical PositivismScientists of the German Vienna Circle in the 17th century sought a leaner scientific approach to more solid logical foundations. They used logic as their primarily tool, more narrowly defining Hume’s distinction between fact and value. They asked: When is the argument scientific? They wanted to settle scientific controversies using a “demarcation principle” to distinguish science from pseudo-science: the principle of verification. Any scientific statement could be subject to tests that would identify them as true or false.
The Experimental MethodNaturalist methods are based on control, comparison, and random assignment. “Control is to isolate the cause-effect relationship from other potential explanatory variables; comparison is used to map regularities with the aim of discovering general laws or patterns; and random assignment is used to secure stronger ground for causal inference” (pg. 44). This can be used in many methods, but works best in the experimental method, which places the control and comparison into the hands of the scientist. The purpose is to show that relationships between variables are real and direct and not accidental.
The experimental method is focused on proving that when the independent variable (x) is present, the dependent variable (y) is also present; similarly, when the independent variable (x) is not present, the dependent variable (y) is also not present. The independent variable exists on its own, regardless of the dependent variable, but the dependent variable must rely on the independent variable, focused on showing a relationship between the two. This is used to show correlations and covariations.
When testing on human subjects, the independent variable may be referred to as the treatment group (those exposed to some stimulus x), while the group not exposed becomes the control group (y).
- Causation describes what made something happen; it describes the underlying cause of an event or phenomenon. It is considered very difficult to prove.
- Correlation describes the relationship between two or more variables or events. It describes the strength or the relationship between variables, but does not claim to isolate what caused the relationship. Perfect correlation may be indistinguishable from causation.
- Hypothesis: A proposed explanation for an event of phenomenon, that is then tested to be either proven or disproven.
- Hawthorne Effect: When a scientist observes the world, they alter that world in some way through their observation. In the case of human subjects, this may be influencing the behavior of the subjects simply by observing them, which becomes a confounding variable when attempting to measure correlations between treatments and behaviors. Some workarounds may be not to isolate the treatment and control groups; in medicine, the control group does not know they are given a placebo.
- Random allocation: A method of randomly sampling to try to evenly distribute a sample and diminish confounding variables.
- Pre-test: Establishing a baseline for which to evaluate the effects of the independent variable.
- Post-test: An evaluation of the effects/lack of effects after the treatment. A comparison between the pre-test and the post-test will allow the researcher to understand that observable differences are attributed to the treatment.
- Internal validity: The scientist’s control over the experimental context. Strong external validity is the “crown jewel of experimentation.”
- External validity: Generalizability of the experimental results to the real world. Strong internal validity often decreases external validity, given the experiment becomes less like the real world.
- Natural experiments: “Observation-based research designs where the investigator uses design choices to control for confounding variables” (pg. 56)
- Field experiments: Experiments in natural settings where the investigator manipulates the environment in some way, but is often unable to control surrounding contextual variables
- Laboratory experiments: The most controlled method, which allows the researcher to control some elements of the natural environment and also control the independent variables, to the sacrifice of external validity.
- True experiments: Have three essential features. (1) Subjects are assigned at random to treatment and control groups. (2) The experimenter manipulates the treatment variable. (3) The experimenter compares the responses of those in the treatment group against the responses of those in the control group (pg. 60).
- Quasi-experiments: Much like a true experiment, except that subjects are not randomly assigned to treatment and control groups. Instead, one might do the same test again on a different group many times and compare with another test, as seen in the classroom example on page 58.
Critiques of Experimental Methods
- Ethics: On one hand, some experimental methods can be unethical, like the example of giving cigarettes to children to understand if they will develop lung cancer.
- Confounding variables: Particularly when working with human subjects, there may be confounding variables which cannot be controlled.
- Ontological concerns (the nature of the world and how we study it): The mechanistic approach of creating a controlled experiment is artificial and manipulated.
The Statistical MethodTwo types of statistical approaches: descriptive and inferrential. Descriptive statistics are rooted in observation, while inferrential statistics focus on uncovering causality. The ontological difference separating Hume and J.S. Mill point to the utility and belief in descriptive versus inferrential statistics: Hume did not believe causality could be observed. Mill, however, believed the world is a web of causal relationships and causality was underneath the surface of correlation, even if it could not be directly observed.
- Descriptive statistics are often used to supplement narratives and illustrate claims (pg. 69). They quantitatively describe some sample. They are also often used in constructivist work. Descriptive statistics have their origins in the state, particularly rulers attempting to gather information about their inhabitants and enemies (pg. 70).
- Inferential statistics seek to infer characteristics of a population to generate predictions, provide explanations, and test hypotheses (pg. 69). They are used to draw conclusions or insights beyond the values of the data alone, particularly to understand something not directly observed. Regression analysis allows scientists to use non-experimental data in controlled ways, by manipulating variables.
Some issues with statistical methods include: Statistical approaches cannot account for all variables. The logic of statistical approaches depends critically on both the logic of the experiment and the availability of sufficient data. Not everything that can be studied can be statistically studied, due to lack of available data or lack of sufficient data.
- Variable: Something that varies, “a phenomenon that assumes different (varying) values according to different cases” (pg. 74).
- Variable analysis: The breakdown of specific events or behaviors into variables to be analyzed by themselves or in relation to other variables.
- Univariate statistics: The statistical analysis of one variable at a time.
- Bivariate statistics: The statistical analysis of two variables at a time.
- Measures of central tendency: Arithmetic mean (average), median (the value in the middle of all ordered values), and mode (the most common value).
- Null hypothesis: The hypothesis the researcher is trying to disprove. This hypothesis states that the variables are not related in a manner that is statistically significant.
- Standard deviation: The measure of dispersion, capturing the spread of values in a set of values.
- Normal distribution: A curve in which the measures of central tendency coincide. (The bell curve.)
- Correlation coefficient: Varies between +1 and -1, where +1 is a perfect positive correlation (when one variable rises, the other does) between variables and -1 is a perfective negative correlation between variables (when one variable rises, the other falls).
- P-value: The most common measure of statistical significance (the probability of an outcome in a statistical study). .05 indicates a 5% chance of getting the results you did given the null hypothesis is true.
- Standard error: The standard deviation of its sampling distribution, providing the estimate of the precision of a parameter. It is used to infer from a particular sample to a relevant larger population.
- Degrees of freedom: The measure for the number of values in a final calculation that are free to vary.
- R^2: A summary statistic ranging between 0 and 1 to determine how well an equation fits the data. If R^2=1, then all variation in the dependent variable is explained by the model (a rare occurrence) (pg. 77).
- Regression analysis: Used to predict the value of dependent variable y given the value of independent variable x. Regression analysis has two types: bivariate and multivariate.
- Bivariate regression analysis: How changes in a single independent variable relate to changes in a dependent variable.
- Multivariate regression analysis: Allows expansion of the number of independent explanatory variables.
- Spurious relationship: When two+ variables are statistically co-varied but not causally linked; co-variation usually due to chance or some hidden variable (called a lurking variable).
- Significance: When a statistical relationship is unlikely to occur by pure chance. It does not mean the variable is meaningful or important.
The Comparative MethodComparative methods mirror experimental and statistical methods, as they involve variable analysis, are controlled, and compare empirical relationships between two or more variables. They are different from experimental and statistical methods in that they can trace proposed causal relationships. Comparative methods go by many names: different systems/similar systems, comparable case strategies, focused comparison, case-oriented comparisons, and the method of systematic comparative illustration (pg. 95). Comparative methods do not use random selection; cases are selected based on the dependent variables. Often, cases may be selected from real world examples, rather than controlled laboratory experiments (e.g., a study of revolutions or a study of companies). However, the variables selected from the cases when doing the comparative analysis remain controlled. They were developed as inductive tools for mapping empirical patterns in the Real World. Selection bias is the major concern for comparative methods, given that variables are carefully selected for analysis.
Selection bias / sampling bias: Can threaten the generalizability of results. Social scientists may select only the cases that support the chosen theory or only draw from limited sources. It is also an issue when selecting on the dependent variable.
N describes the number of cases to be studied. The N scale determines the number of comparisons for a comparative study. So, if a scientist wishes to compare n=9 cases, they must try 36 different combinations for comparison. Unlike statistical methods, comparative methods rely on a much smaller number of cases (Ns) and thus results are heavily weighted on variable selection. Therefore, over-determination based on a smaller number of variables becomes a concern.
Mill's Four Logical Designs
- The method of difference: Compares similar systems. Case selection is used to control for causal effects. By choosing similar cases, observed differences between cases cannot be explain by similarity. A difference in a key explanatory factor is then used to explain variations in outcomes. A common use of the method of difference is in longitudinal comparisons (e.g., comparing a nation before and after a catastrophic event); intra-state comparisons (e.g., policies across the 50 united states); contextual comparisons (e.g., focusing on hospital management in multiple similar countries); and counterfactual comparisons (e.g., imagining a purely (fictitious) similar case as Mill imagined, though it greatly forks away from Mill’s inductive approach).
- The method of agreement: Compares different systems. Much simpler than the method of difference, because it looks at cases which have only one circumstance in common and treats that circumstance as the cause or effect of the given phenomenon. Yet, while the simplest of the four methods, it is regarded as inferior, due to a tendency to lead to faulty generalizations.
- The indirect method of difference: Mirror application of the method of agreement. Considered the most reliable comparative method. Involves cross-tabulating causes and effects. Compares all cases systematically for both agreement and disagreement. Juxtaposes positive and negative cases to be more certain about the causal relationship of a phenomenon.
- The method of concomitant variations: “Focus on particular (concomitant) variation, where a quantitative change in an independent variable is associated with quantitative changes in the dependent variable” (pg. 97). It is not limited to binary cases like the previous applications. Tracks variation in magnitude, rather than in presence or absence.
Mill believed that, in the case of the method of difference, it was absurd to believe social realities could be exactly alike except along one key explanatory factor (pg. 98). (Similarly, Durkheim felt the social world was too messy and complex for the method of difference or the method of agreement.)
Over-DeterminationOver-determination is the claiming of causation in a study where there is not enough empirical evidence for such a claim.
The first main component of over-determination is the lack of examples. For example, only one explanatory variable has negative degrees of freedom, making any claims to causation worthless. Without more examples, you can say nothing about the larger phenomenon. The more examples you have, the more deterministic your data is about the larger phenomenon.
The second main component is too many variables. A method for handling this is to combine variables that have similar characteristics. Another option is more rigorously using theories to choose important variables.
History, Interviews, and Case StudiesCiometrics has borrowed statistical methods and theories from naturalist science and employed them to historical examinations. There has been a long standing rivalry between historians and naturalist social scientists who adopt historical methods, mostly along methodological lines. Naturalists use historical accounts as case studies, taking them as a case of some larger phenomenon that must be accompanied by theory, and views the historical method as a lower ring of scientific methods.
The social scientist employs deductive frameworks to understand lessons from history. Historians do not “usually accept a comparative framework that extends beyond a single historical period, nation or culture” (pg. 133).
Sources and the Rankean IdealThe historical method adopts a systematic doubt, characterized as “source criticism.” Sources are accounts from which an historical narrative is built. Leopold von Ranke, who established historiography as an objective discipline, distinguished between two types of sources. The Rankean ideal desires to root our forgeries in historical documents, to stick as close to primary sources as possible, to figure out inconsistencies, and to be as objective as possible. To only document “what actually happened” is the ideal goal.
Primary sources: the most reliable sources, as they come from direct accounts of historical events, such as eyewitness accounts and original documents.
Secondary sources: sources once removed from historical events, such as stories, newspaper reports, or statistics. They are useful for establishing timelines and themes, and understanding what is written about and what is not. Yet they are seen as less trustworthy as they are filtered through primary sources.
InterviewsInterviews can be one-on-one or in focus groups, but naturalist interviews follow the Rankean approach, treating participants as sources of data and facts that can be collected and cleaned of mere opinion. They are skeptical of human memory and rely on Ranke’s hierarchy of sources to identify reliable data. They structure questions so that they will produce the same answers under different conditions and with different participants. They try to triangulate sources around the same question. They seek to understand how the world really is, towards a Real World.
Structured interviews: Structured interviews are structured in the same order for every participant. Data can be used in quantitative analysis.
Unstructured interviews: Unstructured interviews do not have pre-determined questions or ordering, focused more on deeply probing contextual answers about a subject.
Semi-structured interviews: Semi-structured interviews combine these methods, with a protocol of questions that guides the interview, rather than determines it entirely.
Case StudiesCase studies are considered the lowest step in the naturalist hierarchy of methods, and are generally treated as “cases as something” - meaning the history being examined is being examined because of some larger theoretical concern (pg. 133). Some social scientists believe case studies are limited, thinking they are not useful for testing hypothesis or theories, while others find them useful as real world evidence.
Didactical case studies: Law, business, and other practical disciplines tend to take a didactical approach to case studies. In law, the case study is used to learn from; a student is presented with a case study, a detailed description of an issue, and seeks to work out a just and orderly solution. They often observe similar cases and compare them. They enculture students to specific disciplinary thinking and reasoning.
Fitting case studies: They usually use what are called fitting case studies. These are theory confirming case studies meant to assess the degree to which a case fits a general proposition (pg. 136). They usually describe a single event and fit that event into a conceptual scheme to demonstrate the explanatory power of a theory. A plausibility probe is a method of trial testing a given theory on a case, to test the plausibility of the theory.
Doubts About Naturalist MethodologySowing seeds of doubt opens up spaces for alternative ways of knowing. Specifically, it opens up constructivist approaches.
Ontological DoubtsThe ontological assumptions of naturalism are that a Real World exists, it exists independently of human interrogation, and that it is ordered. They have allowed scientists to elaborate theories that have taught us about the natural world and universe. They have allowed us to make great natural science and medical science achievements. However, the authors the authors discuss both physical and metaphysical ontological doubts about the natural world. Metaphysical: At a metaphysical level, doubts stem from whether there is a truly ordered nature in the Real World. The belief in an orderly natura if crucial to naturalist ontology, because it allows scientists to use inductive methods to look for laws and deductive methods to model universal patterns mathematically.
Physical: At a physical level, subjective elements of nature interfere with empirical observation.
Natural vs. Social WorldThe three differences between the natural and social world are: Unpredictability, Agency, and Perspectivism.
Unpredictability: The authors discuss how the social world, specifically, introduces a level of unpredictability that may be less apt to occur in the natural world (where I refer to animals as also social, but physics, astrophysics, chemistry, etc. as natural). They also discuss how social science discoveries have a “disciplining effect on society,” so that “shocks” to that known knowledge impact the overall world (pg. 149). The discovery of some new natural phenomena can also shock and reorient social science.
Agency: The reality of human agency leads to questioning whether observations of the social world are actually inherent to the world, or are simply a result of human agency and therefore change with our circumstances. We begin to question whether the patterns we have observed are actually of our own making.
Perspectivism: Perspective argues that what we observe changes when placed in different contexts or new perspectives. This leads to a breakdown of the belief in a singular Real World and opens up the possibility of postmodernist approaches of multiple worlds. Friedrich Nietzshe is considered the originator of postmodern ontological pluralism which not only states there are no patterns in the Real World, there is no Real World at all. Any intelligibility about the world can be attributed to the observer imposing their own conceptual framework (pg. 152).
Epistemological DoubtsOnce we begin to doubt the ontology of naturalism (how the world is organized), we begin to doubt empiricist epistemology (how observation alone can be used to understand the world).
The three doubts are: Presuppositions, Meaning, and Scientific Authority.
Presupposition: Presupposition challenges the notion of facts as simply out there in the world to be observed. Facts are social and historical phenomena made by humans. Presupposition means that before observation, we have some preexisting idea about what we are meant to see. Theory is presupposition, in that it is viewed as a preexisting truth from which to base observations.
Meaning: Empathy is used to better understand the underlying meaning of social and historical events. In order to distinguish between meanings, the observer must interpret the context of the observed phenomena.
Constitutive communication posits scientists study the relationships, identities, and tasks within communication - not on the communication itself. Rather than focusing on the transmission model of communication, which posits communication occurs between relational actors, constitutive communication argues that relationships exist because of communication. Scientific Authority: Positivist research derives scientific authority from both reason and sense perception, and from the rhetorical argument that science itself is a source of authority in the modern world. However, privileging reason in the positivist sense introduces and sustains biases into science, particularly through ignoring important aspects of the human experience (pg. 159). Rhetorically, the very argument that positivist reasoning is a superior form of reasoning upholds positivist scientific authority, convincing others that it is the true way of knowing about the world.
Methodological DoubtsMethodological doubts include doubting methodological holism, or the idea that a single methodology is appropriate for both the study of the social world and the study of the natural world. For example, the logical positivist believes all science be modelled as closely as possible on physics (pg. 164).
Those who criticize methodological holism/universalism generally subscribe to one of two alternatives: (1) methodological pluralism, the idea that some methodologies are more appropriate than others for certain problems, which still relies on demarcation principles for determining the appropriate methodology; and (2) methodological agnosticism, which believes methodological assumptions to be alien and violence and does not rely on any measure of appropriateness (pg. 164-165).
The Constructivist Philosophy of ScienceConstructivists differ from naturalists in that, instead of looking for universal patterns, they trace patterns back to the human mind that is observing said patterns. Constructivists doubt there is a universal truth. They are more interested in mapping different explanations to the origins of variance.
There is such a range of diversity in constructivist epistemology and methodology that it is difficult to say there is some singular epistemological vision or methodological stance. It might be more fair to say they share a similar ontological view, but the epistemologies and methodologies underneath that ontology are much more diverse.
There is much disagreement on methods. Where some will use methods that seem to suit any research question, from hypothesis testing to interviews, others reject any naturalist procedural design. However, generally, constructivists will use the same methods as naturalists, but with a different aim - not proving some generalizable truth, but understanding a phenomenon within a specific context.
Three Joists of ConstructivismOntology: Humans are malleable, and each of us participates in the construction of our own world. There is no fixed human nature.
Epistemology: In addition to sense perception and human reasoning, constructivism relies on a border range of epistemological devices (e.g., empathy).
Methodology: For identifying socially constructed patterns.
Principles of ExplanationMeaning, rules, institutions, and praxis separate humans from animals and inanimate objects. In this typology, humans can also be analyzed by other explanations, such as measurement and function.
Hume and Kant on CausalityHume felt we could not infer beyond our own limited experience. He felt there were natural limitations to what humans could know about causality.
Kant was very concerned about the implications for Hume’s arguments about causality unraveling the foundations of naturalist science. If causality was ungraspable by scientists, it might also indicate other metaphysical concepts were unprovable as well. In trying to save causality, and build on Hume’s theories, he expanded his ideas about causality “as part of a bigger and more general property of the nature of the human condition” (pg. 173). Kant began to treat the human mind as an interpreter of the observations (impressions) that come to it from the external world. Kant shifted the ontological terrain of science from nature to the human mind. Though Kant still believed in the existence of a Real World, we can never objectively observe or know the Real World.
Whewall's BeliefsWhewell focused on four framing devices which shape our perception of the world: history, society, ideas, and language. There is no accumulation of singular insights of obvious patterns of facts. Societies share pools of knowledge, which might jumpstart certain historical shifts. Science and human knowledge is historical in nature. Facts are framed and affected by an individual carrier of knowledge and by the societies from which they pool knowledge.
The belief that sense perception is the basis for all knowledge would make all knowledge contingent on individual perception. However, not all knowledge is contingent and it is more than a sum of individual perceptions. Whewell believed that facts, ideas, and arguments do not always originate in individuals, but are sustained and maintained through social relationships.
While human knowledge comes from sense perception, scientific knowledge needs more than perception alone. Perception is conditioned by ideas and they are connected by language and culture. An idea may not correspond to some true Real World, but carry important social value. Some ideas are not true, but carry such discursive power that people continue to believe them despite empirical evidence. When an idea is convincing enough, it can become a fact.
He criticized naturalism for the following:
Methodology: Naturalism’s methodology was wrong, and naturalists had misunderstood Bacon’s concept of induction. He did not believe scientists began with observations and then induced general theories from them, but rather than scientists began with a question. They imagine possible answers and then test those answers against available facts.
Empiricist epistemology: While he felt naturalists were correct to assume that sense perception is important to scientific knowledge, science also depends on the appropriate processing of perceptions.
Ontological arrogance: They are convinced there is a Real World, but have few if any metaphysical arguments to prove this is true. While the naturalists have gathered plenty of knowledge about the world, they can’t show it is true knowledge or that their knowledge is about a Real World.
Kuhnian ParadigmsA paradigm is a commonly accepted theoretical framework from which disciplines operate. “‘The entire constellation of beliefs, values, techniques and so on shared by the members of a given community’” (pg. 179). Puzzle-solving routine activities within a paradigm is called “normal science.” When paradigms break down, there is a scientific crisis and revolution of thought - a paradigm shift. Kuhn argues that scientists will believe one thing, regardless of mounting evidence to the contrary, until the mass of conflicting evidence reaches a tipping point.
Naturalists may accept the idea of sudden shifts over slow accumulation, but are much more resistant to it. Constructivists, on the other hand, embrace the idea of human knowledge not evolving through slow accumulation but through large bounds or shifts.
What makes a scientist?Scientists differ from other individuals who may be interested in scientific topics in three ways. First, the nature of their knowledge is methodological and theoretical; they master the methods and theories associated with their discipline. Second, the context of that knowledge know the history of their discipline, including the history of controversies and debates. Third, there is a social/communal aspect to scientific knowledge, as scientists are connected to institutionalized scholarly communities.
HermeneuticsStemming from theology, hermeneutic understanding states that any understanding must be shown to fit a distinct context. Where natural science seeks to explain natural phenomena in terms of cause and effect, human sciences seek to understand social phenomena in terms of relationships.
Giddens believed in a "double hermeneutics": The first level is history, or “structuration,” but the second level is the “cultural apparatus.”
Structuration states that “all human action is carried out within the context of a preexisting social structure governed by a set of norms and rules that are distinct from those of other social structures” (pg. 189). While sustained by human action, structures also change through a complex process of feedback.
Language“A primary tool for capturing phenomena in the world.” Language forms distinct discourses, and when we use these discourses, we recreate our worldviews. Discourses are connected to knowledge and power. For naturalists, language is a tool for expressing observation and knowledge. For constructivists, language shapes knowledge.
Knowledge is discursive and has the ability to shape our social world, politics, etc. Thus, constructivists approach knowledge with skepticism and self-awareness, trying to situate knowledge within specific worldviews and contexts. Some purposefully adopt activist stances in their approach to knowledge. This allows for multiple knowledges to exist, dispersing or reallocating power from some singular scientific authority.
Other TermsPraxis refers to human action on the natural and social world, and the transformative nature of action. It prioritizes action over thought, and is often associated with Marxism and the work of Gramsci.
Noumena: The Real World.
Phenoumena: Our perceptions of the Real World.
Retroduction: The forming and accepting of a hypothesis to explain surprising facts. It is similar to induction that it moves from individual observation to a connected proposition, but it was different in that it ended in a proposition which could be tested.
Cultural apparatus: A cultural apparatus is “the two-tiered, interpretive and dialectical relationship between social scientific knowledge and human practice, where social analysts are part of the social world they analyze” (pg. 189). It is the lens through which we view the world.
Secondary world: The secondary world is the world we live in due to the fact that knowledge is provided by observers we have never met and often never will meet. The world is neither formed by human beings nor are human beings formed by the world.
Practice: Practice “involves countless inherent acts that are repeated in everyday routines and accumulated over time - they become habits that both give order to our lives and imprison us” (pg. 191).
Habitus: Habitus is socialized subjectivity, “‘the internalization of externality and externalization of internality” [ibid].
Discourse: Discourse is the routine use of language that represents and maintains society.
Intersubjectivity: The psychological relationship between people, emphasizing an inherent sociality in humans. Knowledge is constructed through our social interactions with other human beings and their knowledge.
Historical ApproachesConstructivists feel that the naturalist goal of finding one true account of history is misinformed. They seek diverse accounts of histories to better interpret and illustrate the social reality of history. Constructivists take concern with three aspects of the naturalist approach to historical accounts: (1) That there is a singular past that can be captured by scholars. (2) That data available to scholars are objective or representative of the past. (3) Data is simply there to be analyzed.
Social DataFacts are both too plentiful and too sparse. One can be overwhelmed by the large number of facts, the potentially relevant information suffocating. Otherwise, there may be too little data, especially for historical data where the scope is narrow and data and sources degrade over time. Plus, any surviving historical records might not be representative.
EthnomethodologyThe study of how people produce the social world and their understanding of it. Ethnomethodology stems from the distinction between “noumena” and “phenoumena” in an attempt to objectively study human consciousness and experiences. Ethnomethodologists believe that “the order we see in social life is constructed in the minds of social actors” (pg. 212).
Documentary MethodIt is the organization of social impressions into a coherent pattern. Humans will select certain facts to make sense of broader patterns, which is then used to make sense of remaining facts. “The documentary method is used by ethnomethodologists to show how we use cultural competence and contextual knowledge to make sense of commonplace events” (pg. 212).
DiscourseThe way the world is represented through language. When a series of language representations appear together in a lasting way, a discourse, or system of meaning, is produced.
As discourse refers to specific patterns in language, discourse analysis is employed to examine dialogue (regardless of medium) to identify the patterns and reasons beneath them. One can study how ideas and discourses have real social consequences.
Comparing ContextsWhile naturalists are interested in variable correlations, constructivists seek thickly descriptive narratives, question existing generalizations, develop new associations, and interrogate biases. When it comes to case studies and comparisons, constructivists differ from naturalists in three ways.
(1) The author’s commitment to generalization. (2) The author’s approach to case selection. (3) The nature of the data employed by the author.
On (1), constructivists do not seek to make generalized claims or use comparison to develop an overarching theory, but prefer to bring to light different perspectives.
On (2), Moses and Knutsen claim careful case selection is not of interest to the constructivist, but I disagree. While examples in the chapter present a nonchalance about case study selection (e.g., Foucauldean discourse analysis of any random discipline), this does not mean all constructivist researchers are uncareful about selection.
On (3), while naturalists strive for public and reproducible data, constructivists might use data otherwise frowned upon by naturalists (e.g., subjective information, imagined examples). Constructivists tend to pull from broader data sources.
Hermeneutic ComparisonsThey require two levels of comparison. “The first level juxtaposes particular events with general forms/norms … The second level of comparison is necessary to determine the nature of these general norms” (pg. 248). One hops back between the particular and the more general, but with the purpose of understanding the unique characteristics of each case study.
History and Criticism of StatisticsThe authors discuss the rise of statistics in the 19th century. Prior to the 19th century, the world was hierarchical in terms of autocracy; so treating the king and his citizens as equal units to be measured was unthinkable. With the Enlightenment, treating citizens of equal worth began to be more appealing with ideals of equality among humankind. The initial establishment of the Statistical Society of London in 1834 desired statistics remain objective, and thus stay away from political and social issues. However, statistics became increasingly political, something that constructivists like Foucault defined and studied as human social constructions.
Francis Galton, one of the most important figures in developing statistics, including developing the regression analysis, was a eugenicist who studied human intelligence and was interested in improving the human race through genetic manipulation. He held deep-rooted beliefs in the inferiority of African peoples. (Karl Pearson, his mentee, developed the chi-squared test and standard deviation was recruited to pursue eugenicist research.) While eugenics became particularly taboo after the Holocaust, discussions of the roots of statistics being heavily tied to eugenics is often ignored.
The authors argue that statistics’ lack of context is “foreign to both the human and the humane … insensitive to ethics, morality, and politics” (pg. 256).
The statistical method requires the analyst to distance oneself from the context of study, making it difficult to understand the underlying meanings of data. Quantification necessitates abbreviation. Descriptive characteristics must be transformed into simpler numerical scales. Interpretation is assumed to conflict with the scientists’ need for objectivity. Objectivity is supposed despite the statistician quantifying data into discrete, numerical, or classificatory categories for analysis. Interpretation is also necessary to gather large data via a systematic codebook.
Critiquing the Statistical WorldviewThe statistical worldview is what critic McKeown argues is unselfconscious and vague to the point of being inaccurate and misleading. He believes that this is misleading because modern researchers do not start from a position of ignorance towards one of certainty regarding a single proposition. Instead, they attempt to learn something new about a world they already know, which often means (1) devising ways of leveraging existing understanding to extend knowledge and (2) deciding what revisions of prior understanding are sensible in light of new knowledge. In the example of regression analysis, which is used to manipulate partial correlations in a design that holds other variables constant, the authors write that it constructs a new version of reality ignorant of any social meaning. Thus, it is less about discovering facts of social life but about constructing a new version of that life through statistical manipulation. The authors then state that this constructed reality is only transferable to the social world through demanding and controversial assumptions.
Statistics for ConstructivistsThe authors state four ways statistics are used by constructivists:
- To develop graphics that reveal new descriptions of complex social phenomena
- To recognize and leverage the power of our presuppositions and initial knowledge in Bayesian approaches
- To document the important role that social context plays in generating patterns
- To systematically map and compare the nature of political and social discourse
Graphics and Descriptive StatisticsGraphics and descriptive statistics can help to understand the role of pieces of information. They can help with a hermeneutical tacking between the general and the particular. Descriptive statistics can also help identify outliers and inliers for deeper analysis. Graphics can be helpful for digesting large amounts of information. Readers can digest particular pieces of information and compare them to a larger whole. However, not all graphs work equally well for all data, and inappropriate graphs can be misleading.
Bayesian ApproachBayes’ method is to provide a mathematical rule for updating existing beliefs in light of new evidence. The idea behind it is to mix prior knowledge and experiences to better predict the probability of some event occurring. Constructivists can find this useful because they can incorporate prior knowledge about a subject when examining particulars.
A Bayesian scholar can incorporate prior information into an a priori distribution, then randomly sample some case, and incorporate those findings as an outcome into an a posteriori distribution for a firmer foundation for predictions. By continuously incorporating new observations, the researcher’s picture of a “whole” is constantly evolving.
Bayesian inference is different from traditional statistical inference in two ways:
- Bayesian inference is built on the concept of subjective probabilities - a level of certainty or confidence on the part of the researcher.
- It allows for the introduction of prior information in addition to the sample when making inferences.
Interestingly, there has been resistance among positivists to Bayesian methods, because embracing the subjective diminishes the scientific credibility of the method. Constructivists might find it less appealing because there is still a level of distance between Bayesian methods and the context of study.
Interpretive ExperimentsExperimental methods are a traditionally naturalist approach, attempting to control the environment to isolate causal relationships. But constructivists also use experiments. The authors define three ways they use experiments:
- Contextual experiments to uncover contextual features of perceptions about the world.
- Imaginative thought experiments to develop new understandings of the world.
- Action-Research experiments aimed at contextual familiarity and improving local conditions.
Contextual ExperimentsContextual experiments might be used to measure three types of contextual influences: the influence of groups, people in authority, and the passing of time (or generations).
Imaginary JourneysImaginative experiments might result from deep-rooted local knowledge and experiences that lead a researcher to imagine new hypotheses and experiments. Social thought experiments, as seen in Plato's The Republic or Hobbes' Leviathan, asks readers to imagine a social reality which does not exist. This also lends to a hermeneutic approach.
Action ResearchScholars often embrace experimental methods but they anchor their understanding to familiar contexts and are explicitly committed to using the resulting knowledge to change the world for the better. Action Research contains a broad number of methodological and ideological forms. Ideologically, Action Research is grounded in democracy, humanism, and individual welfare. Methodologically, it differs from naturalism due to differing ways of collecting and interpreting data. Action researchers accumulate data in a systematic manner and then develop interventions to problems experienced by people/communities.
Participatory Action Research, which involves participants or a community as socially involved in the project, is one of them. Naturalist scientists believe Action Research is incompatible with the norms of an objective science. Oddly, the founder of Action Research was a naturalist who utilized experimental methods and believed science should contribute to a better world: Kurt Lewin.
An approach to Action Research might follow the spiral of steps Lewin laid out: identifying an idea or problem, track down facts/solutions/plans, take action, evaluate, amend the plan, and take action again.
Pygmalion EffectThe Pygmalion effect is how an author influences the outcome of their study or narrative. This is the other side of the Hawthorne effect, in which the author influences the perceptions and behaviors of their participants or subjects. Neither of these effects are inherently positivist or naturalist, as they exist in both positivist and naturalist research. Both researcher/author and participant/subject will always be influenced in some way by the research at hand.
The purpose of the experimental example was to show how behavioral influences like the Pygmalion effect or the Hawthorne effect can lead one to question naturalist assumptions about a fixed social reality.
Qual vs. QuantQsuantitative and qualitative methods do not actually belong to one specific camp, and qualitative is often used to denote constructivism and also an inferiority. I believe the authors sought to showcase that there is nothing inherently quantitative with positivism, and nothing inherently qualitative with constructivism, and further still, neither is better or worse than the other.
Scientific/Critical RealismThe authors define scientific realism as an “attempt to meld naturalist methods into a more pliable ontological base” through “burying the most important ontological differences [between] the naturalist and constructivist perspectives” (pg. 297). Critical realism is to be critical of existing social realities and our understandings of both natural and social realities. They break down scientific realism as accepting three functions of naturalism: (1) there is a Real World independently of humans; (2) humans have access to this World through senses; and (3) humans can develop theories for patterns in this World. However, it also acknowledges the shortcomings of naturalism and embraces many constructivists points of view. It is torn between these two worldviews, but leans more naturalist.
Methodological BridgesAfter the paths of pure Naturalism, pure Constructivism, Scientific/Critical Realism, the authors see a fourth path: methodological bridge-building. This perspective “[acknowledges] the usefulness of maintaining different ontological points of departure and [embraces] the methodological diversity that results from interacting across the ontological divide” (pg. 299).
Postmodernism and Big DataNot a coherent school of thought, but agree on important points: (1) A pluralist society needs to acknowledge voices marginalized by modern oppressive power structures, whether those voices are defined in terms of gender, sexuality, ethnicity, or culture; (2) Explore the connection between power and knowledge.
The authors argue for a further exploration of these two points in the realm of Big Data. Interestingly, they do not treat Big Data or data science as a science in the same way they have discussed social science; instead, they argue the utility of social science in fighting the oppression of Big Data and the shortcomings of data science analyses.