"Open Science and Replicability in the Behavioural Social Sciences" Conference of MethodenKompetenzZentrum, Faculty of Behavioural and Social Science, Chemnitz University of Technology, 30 April 2021           

We often hear the self-correcting mechanisms in science invoked as a reason to trust science, but it is not always clear what these mechanisms are.  Some quality control mechanisms, such as peer review for journals, or vetting for textbooks or for public dissemination, have recently been found not to provide much of a safeguard against invalid claims.  Instead, I argue that we should look for visible signs of a scientific community's commitment to self-correction.  These signs include transparency in the research and peer review process, and investment in error detection and quality control.  I describe some specific examples of transparency and quality control practices. We should trust scientific claims more to the extent that they were produced by communities that have these hallmarks of credibility.  Fields that are more transparent and engage in more criticism and correction should earn more trust.  Meta-science can provide scientists and the public with valuable information in assessing the credibility of scientific fields.

Since several years, psychology has its so-called credibility crisis: Many results of experimental studies coult not be replicated by other scientists. In other disciplines this discussion has gained considerable momentum as well (e.g., economics and medicine). In this talk we will first discuss that there are good reasons to believe that social sciences that use non-experimental, observational data also have a credibility problem. Faulty results likely result from errors or misspecifications of regression models. These credibility problems were probably mostly overseen because most replication audits so far focused on experimental data. Using the example of sociology, we will illustrate the missing credibility of studies based on observational data by some examples: There is meanwhile some evidence for low reproducibility of sociological research and questionable research practices seemingly have been widely used. In the main part of the talk we will then argue that credible social sciences need to be more transparent, open, and replicating. We will suggest several measures by which such open science practices could be furthered. A common theme will be that open science practices show features of a social dilemma. Therefore, moral appeals alone will not be sufficient. We additionally need changes of institutions that increase the opportunities and incentives for open science practices.