In philosophy of science, theories are said to be “commensurable” if we can discuss them using a shared nomenclature that allows direct comparison of them to determine which one is more valid or useful.Recently, the book by “Theory and Credibility”, by Scott Asworth and co-authors, develops this concept further to bridge the discussion between applied game theorists and empiricists in the study of political economy. It seeks to find a middle ground between pure empirical agnosticism and the rigidness of structural estimation.
A research design is commensurable if the theory (formal model) and the empirical analysis are similar. That is, one should be a close as possible representation of the other. This has important implications for research design and the scientific endeavor in political economy:
Theoretically informed causal inference matters: Both models and causally identified regressions imply in essence “all-else-equal” comparisons. That is, the comparative statics are reflected in the marginal effect of the treatment, and viceversa. Models explicitly state what variables are endogenous and exogenous, and provide clear implications on the basis of equilibrium strategies and the comparative statics. Thus models provide clear testable hypotheses, reducing the latitude for HARKing.
Measurement matters: The measures for the variables used in the empirical analysis must be good representations of the theory. Further, data constraints can limit the scope conditions of the theory. Empirically speaking, commensurability provides us with a clear understanding of bias due to self-selection, measurement error, and allow us to define the scope conditions of our findings with more precision.
Heterogeneity matters: Models often point at differences in behavior conditional on the values of the parameters. This can be revealed for instance by cut-off strategies and/or an analysis of the comparative statics (cross-partials and second or higher order partial derivatives). Empirical analysis can (and must) test for the related implications by means of an analysis of the heterogeneity of the treatment effects.
Moderators and mediators are useful for assessing mechanisms: Models are explicit about the behavior of their endogenous variables. In this sense, they can can guide the research design to select and analyze moderators (i.e., interaction effects) and mediators (i.e., intermediate outcomes) to rigorously test theories. In this regard, relaxing assumptions in the model can reveal new and interesting mechanisms and additional testable hypotheses.
Abduction strengthens the research design: When empirics do not confirm the model, the empirics can be revised to check if it is properly testing the model (e.g., are measures commensurable? is the econometric specification commensurable?); the model can also be revised by relaxing/imposing assumptions if its predictions are not confirmed. This leads to a fruitful back and forth between theory and empirical analysis that strengthens research design and reveals new, and interesting results. In other words, theory guides empirics, and empirics helps revising theory.