The Military Wants to Build a Bullshit Detector for Social Science Studies
In recent years, rigorous retesting of social science theories, like the willpower theory of ego depletion, has come crashing down. The resulting “replication crisis” had led many scientists to wonder: How many findings in psychological science would hold up in rigorous retesting? They’ve been answering that question by embarking on large-scale replication efforts. But wouldn’t it be helpful — for deciding which ideas are worth funding, for deciding which ideas are worth publicizing — to be able to predict the answer without investing the time and resources to retest an experiment? Perhaps a machine-learning-derived computer program (a form of AI) that could rapidly assess and predict the reliably of a scientific finding would be helpful. As it turns out, DARPA (the military’s Defense Advanced Research Projects Agency) is trying to build one. The project is called SCORE (Systematizing Confidence in Open Research and Evidence), and it’s a collaboration with the Center for Open Science in Virginia.