Evan Nesterak
Behavioralscience.org
July 5, 2023
Harvard University professor Francesca Gino, whose research frequently focused on dishonesty, is under scrutiny for allegedly fabricating data in at least four studies. The news was first reported by Stephanie M. Lee in The Chronicle of Higher Education on Friday, June 16.
A day after Lee’s report, a trio of scientists—Uri Simonsohn of the Esade Business School, Joseph Simmons of the University of Pennsylvania, and Leif Nelson of the University of California, Berkeley—published the first of four posts detailing alleged data fraud by Gino on their website Data Colada.
The news sent shockwaves through the behavioral science community. By all appearances Gino has been a successful scholar—winning awards for her research and teaching, serving on the editorial board for a handful of well-respected journals, writing two popular books, and consulting and speaking at a number of large companies.
“There’s so many of us who were impacted by her scholarship, by her leadership in the field, and as a co-author, as a colleague, it’s deeply upsetting,” Maurice Schweitzer, a professor at the University of Pennsylvania and coauthor with Gino on eight papers, said in The Chronicle of the Higher Education.
Several of her coauthors told me over the phone and through email that they were shocked and devastated by the news. (They requested anonymity while they gather more information about the situation.) Syon Bhanot, who isn’t a coauthor, recalled the time and effort he spent a few years ago trying to replicate one of the studies, whose data integrity has now been called into question. (Disclosure: Bhanot is an advisor to Behavioral Scientist.) And one of Gino’s former advisees at Harvard called the allegations “disturbing,” but also said he “never experienced or suspected fraud.”
While the news broke recently, it seems to be the culmination of a multiyear inquiry into Gino’s research by Harvard and other scientists in the field.
“In 2021, we and a team of anonymous researchers examined a number of studies coauthored by Gino, because we had concerns that they contained fraudulent data,” Simonsohn, Simmons, and Nelson wrote on Data Colada, the website where they write about research practices in behavioral science. The website, founded in 2013, has played a significant role in unearthing shoddy research practices, like p-hacking, and they’ve also revealed other instances of fake data.
“In the Fall of 2021, we shared our concerns with Harvard Business School,” they said. “Specifically, we wrote a report about four studies for which we had accumulated the strongest evidence of fraud. We believe that many more Gino-authored papers contain fake data. Perhaps dozens.”
Gino is now on administrative leave from Harvard, according to the university’s website. A Harvard spokesperson declined to comment in response to my email.
When I reached out to Gino for comment, she directed me to a brief statement published on LinkedIn. “As I continue to evaluate these allegations and assess my options, I am limited into what I can say publicly. I want to assure you that I take them seriously and they will be addressed,” she wrote. “There will be more to come on all of this.” (Disclosure: Gino has served as an advisor to Behavioral Scientist.)
None of Gino’s coauthors, which number 148 by Data Colada’s count, has been implicated in the alleged fraud. “To the best of our knowledge, none of Gino’s co-authors carried out or assisted with the data collection for the studies in question,” wrote Simonsohn, Simmons, and Nelson.
The response from her coauthors and others in the field has been remarkably swift, though many details remain unknown publicly. Within two weeks of the allegations surfacing, the papers in question are in the process of being retracted by their respective journals and her coauthors have organized an initiative to evaluate the research she was involved in (more on each below). These actions indicate that there may be merit to the allegations, but with Gino and Harvard yet to put out statements with any substance, a number of questions remain unanswered.
The studies in question
Data Colada reported evidence of suspected fraud in four individual studies, each a part of a different multistudy paper, which were published between 2012 and 2020.
The four studies in question are:
1. Study 1 in “Signing at the Beginning Makes Ethics Salient and Decreases Dishonest Self-Reports in Comparison to Signing at the End” by Lisa Shu, Nina Mazar, Francesca Gino, Dan Ariely, and Max Bazerman, published by the Proceedings of the National Academy of Sciences (PNAS) in 2012.
This paper was already retracted in 2021 due to data anomalies discovered by an anonymous set of researchers (published by Data Colada), after a 2020 replication attempt failed and the original 2012 data was posted. The alleged fraud by Gino in Study 1 represents the second time this paper has had a study called into question for fabricated data.
2. Study 4 in “The Moral Virtue of Authenticity: How Inauthenticity Produces Feelings of Immorality and Impurity,” by Francesca Gino, Maryam Kouchaki, and Adam Galinsky, published by Psychological Science in 2015.
3. Study 4 in “Evil Genius? How Dishonesty Can Lead to Greater Creativity,” by Francesca Gino and Scott Wiltermuth, published by Psychological Science in 2014.
4. Study 3a in “Why Connect? Moral Consequences of Networking with a Promotion or Prevention Focus,” by Francesca Gino, Maryam Kouchaki, and Tiziana Casciaro, published by Journal of Personality and Social Psychology in 2020.
The evidence
Data Colada investigated the data file from each study. In three cases, they retrieved the data from the Open Science Framework (an online science management tool developed by the Center for Open Science that emerged in response to the replication crisis), where researchers post materials related to their research, including data, code, analysis plans, and results. In one case, Gino provided Data Colada with the data directly.
Across the four studies, Simonsohn, Simmons, and Nelson report anomalies that they say indicate fraud. This includes data sorted in an order only possible through manual editing, implausible responses, and a mismatch between participants’ survey and text-based responses. When they looked into each of these anomalies further, they report finding instances where participants’ experimental conditions or responses were changed to provide support for the authors’ hypothesis.
In one case, participants with extreme responses to an honesty task appear to have had their experimental conditions changed to artificially lend support to the authors’ hypothesis that signing at the top of a document produces more honest responses than signing at the bottom.
In another case, the data file listed 20 participants’ class year as “Harvard,” rather than “junior,” “third year,” or “2016,” which gave the Data Colada group pause. When they looked into these responses, they found all but one of the responses were at the extreme end of a scale and in a direction that supported the authors’ hypothesis.
In a study on creativity, Data Colada’s analysis suggests that the number of creative ideas generated by several participants who cheated on a coin-toss earlier in the study was artificially increased to support the authors’ hypothesis that cheaters were more likely to be creative.
For a full look into Data Colada’s analysis, please see the series of posts, which correspond to the four studies above, respectively: post 109, post 110, post 111, and post 112.
An additional indication of fraud comes from Lee’s first story. She reported that Max Bazerman, a professor at Harvard Business School, and a coauthor on the original 2012 paper and 2020 replication, received “a 14-page document with what he described as ‘compelling evidence’ of data alterations,” in the 2012 paper.
“According to Bazerman, Harvard is recommending to the Proceedings of the National Academy of Sciences that it update the study’s retraction notice to reflect its new concerns,” Lee reported.
Lee also said in a tweet that Psychological Science plans to retract both articles in question, which the editor-in-chief of the journal, Patricia Bauer, confirmed to me in an email.
Update July 6, 2023: Psychological Science has retracted both articles: “The Moral Virtue of Authenticity: How Inauthenticity Produces Feelings of Immorality and Impurity” (read the retraction note) and “Evil Genius? How Dishonesty Can Lead to Greater Creativity” (read the retraction note). In both cases, the retraction reads that the Harvard University’s Research Integrity Office reached out to the journal editor after finding anomalies in the data, which the respective retraction notes explain further. The authors for each paper agreed to retract the article.
Each retraction ends with the same statement from Gino’s counsel: “Counsel for the first author informed the journal that whereas Dr. Gino viewed the retraction as necessary, she disagreed with references to original data, stating that ‘there is no original data available’ pertaining to the research reported in the article.”
In the case of the fourth study, “A retraction is slated to be published in the September issue of the Journal of Personality and Social Psychology,” read a statement emailed to me from the American Psychological Association’s publisher, Rose Sokol. “At this time, we believe this to be an isolated incident. However, we continue to be in touch with the institution and if we have reason to consider retracting other articles, we will take appropriate steps.”
---------
Forbes
Serious questions are swirling around the past behavior of New York Times best-selling author Dan Ariely. At the center of the controversy is whether Ariely played a role in fraudulently manipulating data in a famous study that subsequently changed how governments and businesses went about promoting honesty.
Ariely is the James B. Duke Professor of Psychology and Behavioral Economics at Duke University, the author of several well-known books including Predictably Irrational and The (Honest) Truth About Dishonesty, and the founder of the Center for Advanced Hindsight. To say he is an academic rock star is no exaggeration.
Along with four other researchers, Ariely published in 2012 what became a highly influential paper in the Proceedings of the National Academy of Sciences called “Signing at the Beginning Makes Ethics Salient and Decreases Dishonest Self-Reports in Comparison to Signing at the End.” As the title suggests, the main theme of the paper is that pledging to respond truthfully at the beginning of a form can lead to more honest completion of the form, as opposed to waiting until the very end for people to sign.
The researchers presented three studies which they claimed found support for this conclusion. The first two were experimental studies in the lab. In both, participants had to solve math problems, and then report their performance on a form knowing that they would get paid more the better they said they did. In the version where they had to sign an honesty pledge first before reporting how they did, 37% of participants cheated in each of the two studies. When the honesty pledge was on the bottom of the form, 79% cheated in the first study and 63% in the second.
It is the third study which turned out to be fraudulent. In it, the researchers worked with an unnamed “insurance company in the southeastern United States,” which was willing to give half of a group of their customers a sign-at-the-top honesty pledge form for reporting their current mileage, and the other half their traditional sign-at-the-bottom form. The difference in miles reported was 2,427.8, or 10.25% more for the top pledge form, suggesting greater honesty since more miles meant a higher insurance premium.
For years these findings were not questioned, and began to make their way into popular culture. But then doubts began to emerge. Attempts to replicate the findings of the experimental studies were not successful. The authors of the original 2012 paper even tried to replicate their first lab experiment themselves, but they couldn’t. So in 2020 they published a paper in the same journal reporting their failure.
At about that time some oddities were noticed about study three from the 2012 paper. Discussion of those oddities eventually led to a blog post on August 17 which has set the world of human behavior research abuzz. The post on the blog Data Colada, by Uri Simonsohn, Leif Nelson and Joe Simmons, summaries various analyses of the original data that was supposedly accumulated by the insurance company, and found massive irregularities. These were not subtle problems either, but blatant ones. The end result was not only that the data was not correct, but that it was statistically impossible.
Each author of the 2012 paper quickly agreed with the finding of the blog post that the data in experiment three was junk. And they have asked for the paper to be retracted from the Proceedings.
What does this have to do with Ariely specifically? Well, three of the five authors claim that they were only responsible for the first two lab studies. They allegedly didn’t have anything to do with the problematic third study. A fourth author, Nina Mazar, sent to the blog authors a 2011 email from Ariely with an Excel spreadsheet attached that contained the data supposedly from the insurance company. The properties of the Excel file apparently showed that Ariely was the one who created the file and also the one who changed it last.
So it looks like there are two main suspects for the source of the fraud: the insurance company and Dan Ariely. At this point, it seems that there is not much concrete information to go on in assigning blame.
On the one hand, Ariely has been adamant that he did nothing wrong. And if he were to commit fraud, one might think that he would have gone about it in a more subtle way. As he says, “I’m not an idiot. This is a very easy fraud to catch.”
On the other hand, as has been noted on social media, questions could be raised about why an insurance company would find it desirable to manufacture false data. Not only that, but why would it fabricate the data in a way that helps out the researcher’s hypothesis, yet does such a clearly bad job that it is flat out statistically impossible? In addition, according to an article in Science, concerns are starting to be raised about some of Ariely’s other work.
The goal here is not to investigate this further and try to assign guilt and innocence. I am not an investigative reporter. Instead, three more general observations are worth making.
The first is how rich with irony this whole affair is. After all, the whole point of the studies originally was to investigate dishonesty. So we end up getting dishonesty in the study of dishonesty. But it gets even more meta. For the bloggers allegedly engaged in an honest investigation into the dishonesty of a study of dishonesty. And either Ariely himself is being honest or dishonest. Suppose that he is being dishonest (again, I am not taking a stand one way or the other). Then he would be dishonest in his coverup of his dishonesty in a study of dishonesty!
The second point is about the damage to the public’s trust in the reliability of scientific research on human moral behavior. There is already a replication crisis that has been going on for several years. The 2012 paper contributes to this crisis with several researchers, including the original authors themselves, failing to replicate the findings of the lab studies.
Now the discovery of outright fraudulent data will serve to only make matters worse when it comes to public trust. Non-academics already are starting to worry about replicability. Now they have to worry about outright fabrication too. As the three bloggers note, “We have worked on enough fraud cases in the last decade to know that scientific fraud is more common than is convenient to believe, and that it does not happen only on the periphery of science.”
The third and final observation is about what can be done to prevent fraud like this from happening the next time. One suggestion that the bloggers offer, is to make public all the relevant data. As they write, “[t]he fabrication in this paper was discovered because the data were posted. If more data were posted, fraud would be easier to catch. And if fraud is easier to catch, some potential fraudsters may be more reluctant to do it.” This makes sense, but also seems like it could require a lot more time on behalf of already overstretched journal reviewers to ask them to examine carefully the original data of the studies for fraud.
At this point, we can only hope that this will be the last dishonest study of dishonesty for quite some time.
--------------------
Mussar Haskels:
1] The best approach to science - in all of its vast subject matter - is the approach we should have towards people: כבדהו וחשדהו - Respect them but with a mixture of wariness and suspicion.
2] When Hashem was going to create the world the middah of Emes came before Him and said "Don't do it!! Man is all falsehood!!!" Now, I wasn't there at the time [I was only created in 1971...] but so relates the Medrash [ב"ר פ"ח ועי' מהר"ל נתיב האמת פ"ג]. Sad but true. Everybody lies from time to time when necessary. The only people who don't lie are those who really work hard on themselves and make it a life project. It is EXTREMELY helpful if the person has Yiras Shomayim which is a very effective deterrent. But some professor from Harvard? Why shouldn't he/she lie if it will get them ahead??? No G-d to answer to [in their narrative] and it will bring them money and glory. Morality? Ethics? That is very nice but falls by the wayside if the stakes are high enough. If a woman would be מזנה for 100 million dollars then she is a זונה in a sense. It is just a matter of price. ודי למבין....
The sad reality is that people also steal chiddushei Torah. They say or even publish other people's Torah ideas and pass them off as their own. Just like religious people lie about other things. It is sad but true. Religious people often tell the truth as well. They are people. Sometimes good. Sometime - improvement is called for.
The antidote: I've said it once I've said it a thousand times. I only follow in the footsteps of our leaders, guides and masters.
MUSSAR!!!! [And often - therapy].
"החזק במוסר אל תרף נצרה כי היא חייך!!!!!!"
