Center for Leadership Science team earns award for research

Findings expected to advance organizational science studies

Trust in scientific research is critical, and trust that is lost can be difficult to rebuild. For research papers to be trustworthy and also useful, authors need to follow best practices, including how they cite past research in their current studies.

UNC Charlotte organizational science researchers in August received an international Best Paper Award from the journal Organizational Research Methods for new research on this issue. The journal’s editorial board chose the team’s paper for best advancing research methods in organizational scholarship among papers published in the journal the previous year.

Scott Tonidandel, director of the interdisciplinary Organizational Science Program and a management professor in the Belk College of Business, collaborated with authors Liana M. Kreamer and Betsy H. Albritton – organizational science Ph.D. students at the time – and Steven G. Rogelberg, Chancellor’s Professor and professor of organizational science, management and psychological science. Tonidandel and Rogelberg also are faculty with the Center for Leadership Science, part of the Belk College at The Dubois Center at UNC Charlotte Center City.

“This research matters because it will improve the overall quality of our science. So, when a business person reads an organizational science article, they can have more confidence in the results because the methods behind those findings were correctly executed.”

– Scott Tonidandel

Variations in methodological best practices

The study specifically explores how researchers in the organizational sciences use the methodological best practices they cite. In academic research, a methodological best practice is a method that has been shown to reliably yield a desired result. Examples include the design of a study, data collection and analysis, and other research steps.

The Charlotte researchers studied whether scholars are adhering fully to the research practices they cite, or if they are cherry picking from recommended practices without disclosing they are doing so, or even inaccurately following the methodological best practices they cite.

The team found substantial variation in how authors use the methodological best practices they cite. Close to half — 47.7% — have minor inaccuracies, and 34.5% do not adhere to the methodological best practices they cite. Just 17.4% are appropriately following the best practices from the cited articles.

“Essentially, our work shines a light on some common behaviors by researchers that are suboptimal,” Tonidandel said. “The hope is by identifying these practices and providing concrete recommendations, we can enhance the quality and integrity of the work being published.”

future recommendations

Previous studies have made all sorts of methodological recommendations, but little was known about how researchers were implementing those recommendations. After crunching the data and seeing the scope of the problem, the Charlotte team considered practical ways to address the issue.

“We provide a set of nine recommendations that we think will help improve the research that our science is built on,” Tonidandel said. “These recommendations are targeted at three distinct groups. One set is targeted at the researchers that are using the methodologies in their own research. But we also provide recommendations to reviewers and editors, who can help ensure that the correct methodological practices are being employed. Finally, we provide some recommendations to those folks who are developing these methodological best practices to help them as they write future articles.”

The journal published the paper in 2023. The award was presented at the Academy of Management’s August meeting.