SLICING AND DICING: EX POST APPROACHES
By Lorraine Eden, Professor of Management,
Texas A&M University, leden@tamu.edu
Blog No. 2012-03 (May 4, 2012)
NOTE: This posting first appeared on THE ETHICIST, the Academy of Management's ethics blog, http://connect.aomonline.org/resources/614c81fe4f/summary, in May 2012. The AOM's THE ETHICIST has three contributors, each one writing a monthly blog in sequence, on (1) research (Lorraine Eden), (2) teaching (Kathy Lund Dean) and (3) professional life (Paul Vaaler). Additional copies of my blog posts, in PDF format, are downloadable from my LinkedIn page at http://www.linkedin.com/in/transferpricingaggies.
KEY
INSIGHT: In my February 2012 THE ETHICIST: RESEARCH posting, I
discussed an important issue facing researchers: How do authors determine
whether the papers coming out of one project are sufficiently different from
one another so that they can be considered to be new papers? In my earlier posting, I looked at ex ante methods that
authors could use to determine whether a paper was sufficiently new. Here I
follow up with ex post methods for determining novelty; that is, once the paper
has been written and submitted for review, how can reviewers and editors be
assured of its originality?
1.
INTRODUCTION
Let me start by saying that I am not
against working in teams and developing multiple papers from a big research
project that are then published in different journals. In fact, I am very much in
favor of team-based research projects, and have been actively doing that for
some years now with various co-authors. Clearly, we want scholars to gain the
advantages of economies of scale, scope and learning that come from developing
big projects that lead to multiple publications. Working in teams offers the
advantages we typically associate with strategic alliances: the ability to
bring together and leverage complementary resources, the creation of routines
and capabilities that generate efficiencies and synergies, and the opportunity
for greater flexibility and speed, among other advantages.
Big projects with two or more co-researchers
are now common, perhaps even dominant, in terms of how we do research now
within the academy, at least based on my observation of authorship in our
scholarly journals where the norm now appears to be two, three and even four or
five co-authors. The April 2102
issue of AMJ, for example, has 10 articles; only one is single authored. The April 2012
issue of AMR has six articles; two are single authored.
When a team of researchers work together
for several years, each new paper builds off the previous ones as the authors
develop a deeper understanding of their research area, see more nuances and
find more puzzles to solve. As that building and expansion occur, basic ideas
are likely to be repeated across the papers, and may look repetitious to an
outsider – and to the authors themselves.
In my February blog, I addressed the issue of how authors themselves can
determine the “bright line” between “sufficiently new” and “excessive overlap”.
I recommended that authors do a originality analysis to compare papers, and
that authors share this analysis with journal editors at the time of submission.
[1]The
originality analysis would then help the editor (and reviewers if the editor chooses
to share the information) determine the degree of novelty. My general advice
was: be transparent. Assuming an author has been transparent, it is
up to the reviewers and editors to assess the extent of overlap and
unique contribution.
However, what if the authors are not
transparent and do not provide this information at the time of submission. How can
a journal editor and the paper’s reviewers separate acceptable overlap from “slicing
and dicing”? More generally, how can
editors and reviewers identify instances of scholarly dishonesty, and what
should they do if they do find such evidence?
2.
CASE EXAMPLE
Perhaps you have read the new article
by Honig and Bedi, “The Fox in the Hen House”, published in Academy of Management Learning &
Education. The authors examined the 279 papers presented at the 2009 AOM
meetings, in the International Management division, for evidence of plagiarism.
They used regression analysis to test hypotheses about possible antecedents to
plagiarism (for example, gender, degrees from non-English speaking countries,
junior/untenured faculty, authors located in emerging (“noncore”) economies). The
software program Turnitin.com was used
to determine the amount of plagiarized material in each paper. [2]
The results were startling: One-quarter of the presented
papers (71 of the 279 papers) showed some evidence of plagiarism, and 13.6% had
an average of 5% or more of the text plagiarized (approximately 1,000 words).
The highest offenders appear to have been papers by an author or co-author located
in non-core countries. Over 40% of the
papers written by scholars in noncore countries showed some evidence of
plagiarism; half of those had more than 5% of the text plagiarized (Honig &
Bedi, 2012: 113). For authors from “core” (developed) countries the percentages
were lower but still higher than I think most faculty would have expected: 21%
of papers showed some evidence of plagiarism and a quarter of those had more
than 5% of the text plagiarized. Education in a noncore rather than a core country
was also a differentiating factor (27% noncore vs 21% core); however, neither
gender (male vs female) nor rank (untenured/junior vs tenured/senior) appeared
to matter (Honig & Bedi, 2012: 116).
While one might argue that authors are typically less
careful on conference submissions than they are on submissions to scholarly
journals, the plagiarism issue clearly matters for both. Moreover, the
plagiarism estimates of Honig and Bedi are underreports because the authors
deliberately excluded self-plagiarism, stating that “If authors used sections
from their own previous work or cited the primary source, then it was not
considered plagiarism” (p. 112). If the authors had taken “slicing and dicing”
into account, I suspect that the percentages – across the board – would have
been much higher.
3. EX POST
APPROACHES #1: SOFTWARE SOLUTIONS
If plagiarism is a real problem,
and it apparently is, what should AOM do? More generally, what should we do as
a community of scholars about scholarly dishonesty? Honig and Bedi conclude that
“Institutional norms that many of us take for granted are clearly and brazenly
being disregarded”; they call for AOM to “implement more rigorous standards in
order to reduce plagiarism and to ensure high-quality and original scholarship”
(p. 119).
Jean Bartunek, in her introduction to
the Honig and Bedi article, notes that, starting in the spring of 2012, the AOM
journals will begin using the software program CrossCheck to detect
plagiarism in submissions. It is not
clear to me whether all submissions or only conditionally accepted papers will
go through the CrossCheck process. Obviously putting all papers through would
substantially increase the total cost.
However, regardless of the financial
cost, submitting papers to some form of software such as Turnitin.com or
CrossCheck seems to me to have now become a necessary part of the journal
review procedure. We may not like this
nor like the psychosocial (lack of trust) message that it sends to prospective
authors, but --- given the huge numbers of papers now being submitted to our
journals and the evidence in Hong & Bedi (2012) -- some form of plagiarism checking software
appears to be inevitable. AOM and
other scholarly associations must move to adopt stronger ex post approaches to
combating both plagiarism and self-plagiarism.
In addition to software options such as CrossCheck, I
believe AOM and its scholarly journals would benefit from membership in national
and international organizations that are devoted to improving ethics in
professional associations and scholarly publishing. Through our memberships, we
can signal our commitment to scholarly and professional ethics, learn and adopt
best practices, and bring the weight and voice of AOM -- 18,000 members from
110 countries --- to bear on these issues. One of the best known of these
associations is COPE.
4.
EX POST APPROACHES #2: COPE TO THE RESCUE
COPE (the Committee on Publication Ethics) is a
non-governmental organization that was set up by a group of journal editors to
share best practices for handling ethical violations. Over time, the
organization has developed a whole set of procedures that it recommends editors
follow when faced with problems such as plagiarism and self-plagiarism. More than 7,000 universities, journals and
publishers are now COPE
members.[3]
As an example of how COPE could be helpful,
the Committee has a whole section on “salami publishing,” that is, two
overlapping publications arising from the same research project. (I call this “slicing and dicing”.) The key
issues identified by COPE are (1) the degree of overlap between the two
publications and (2) whether the author sought to hide the overlap.
COPE provides flowcharts to help
editors determine best practice in identifying and handling cases of suspected
misconduct. Flowchart 1 deals with
suspected redundant publication in a submitted manuscript; flowchart 2 with the
same issue in a published article. In both flowcharts, the key issue is whether
there is major or minor overlap/redundancy between the two papers. Major
redundancy is defined as both papers having the “same dataset with identical
findings and/or evidence that authors have sought to hide redundancy, e.g. by changing
title, author order or not referring to previous papers”. Minor overlap is
defined as “salami publishing with some element of redundancy or legitimate
re-analysis (e.g. sub-group/extended follow-up/discussion aimed at different
audience)”. Note that evidence of
appearing to hide the overlap can raise the level from minor to major
redundancy.
Some examples of actual cases
submitted to COPE may be helpful here in distinguishing between major and minor
redundancy (search the COPE website for other examples):
- Salami publication: Four papers were completed by the same research team, with each paper referencing the prior publications. The fourth paper in the series was rejected on the grounds that there was significant overlap between the new paper and the earlier publications. COPE recommended, first, distinguishing between salami and redundant publication; arguing that if there were 2/3 overlap between the two papers, this was a redundant publication. COPE defines a salami publication as covering “the same population, methods, and question”. Second, if the two papers asked related questions, they should be published as one paper; if the two papers asked separate questions, they could be separate publications. Splitting up papers by outcomes was not legitimate.
- Duplicate publication or salami publication? A paper submitted to a journal is discovered by a reviewer to have been already published in another journal. When the editor contacts the author, he responds that the two papers are different. COPE’s advice was to focus on the overlap between the two papers, determining whether the overlap was major (2/3 would make it a duplicate publication) or minor (a salami publication) and then follow COPE’s rules for one or the other event.
- Duplicate submission: Two papers based on the same research project on pathogens in school children were submitted to different journals. One analyzed the data by socio-economic class; the other by school attended. Substantial portion of the texts were the same in the two manuscripts, especially in the data description and research methods. COPE recommended determining whether this was a duplicate or salami publication.
We have much to learn from other
associations and journals that have faced the same or similar issues to the
ones identified in Honig and Bedi (2012). In addition, we have much to offer to other
associations and journals by being at the table and exercising voice. I believe
that AOM, with its Code of
Ethics and well-developed ethical policies and procedures, can help shape
international best practices in the research ethics arena.
5. EX POST
APPROACHES #3: EDUCATION
A third ex post method to handle
scholarly dishonesty is to create a library of resources for sharing key
reports among authors, editors and reviewers. For example, in 2009, the
OECD’s Global Science Forum published a committee report and practical guide to
Best Practices for
Ensuring Scientific Integrity and Preventing Research Misconduct. Reports
and guides of best practices, such as the OECD one, could be brought together
in one location (a website), making them more widely and quickly available to
scholars. AOM already does some of this; its Ethics@AOM web page includes
links to articles on ethics in research published in AOM journals, the Ethics
Education Video Series, and THE ETHICIST blog posts, among other resources.
Ultimately, while software
programs like CrossCheck and organizations like COPE can help journal editors
and reviewers find and evaluate possible cases of scholarly misconduct, the
“rubber meets the road” through student and faculty education. This is where
AOM’s Ethics and Ethics Education Committees can and does already play an
important role, by activities such as:
- Providing ethical training of PhD students and junior faculty in the Professional Development Workshops at the AOM annual meetings;
- Organizing panels and roundtables on Ethics in the Academy at the AOM meetings and elsewhere;
I hope to see you in Boston in
August where we can continue this conversation and look forward to your
responses posted on THE ETHICIST.
6. QUESTIONS FOR DISCUSSION
- What is your response to the Honig and Bedi article?
- If you are a journal editor, have you been faced with situations similar to those described in this blog? How did you handle them? Do you agree with the COPE templates? Is your journal a member of COPE?
- As a reviewer, what would you do if you were presented with evidence of two overlapping papers? Should you inform the editors?
- Should AOM and its journals be a member of COPE?
REFERENCES
Bartunek, Jean. 2012. Introduction:
Plagiarism in Submissions to the AOM Conference. Academy of Management Learning & Education, 11.1: 99-100.
Eden, Lorraine. 2012. Slicing and
Dicing: Ex Ante Approaches. THE ETHICIST. Blog No. 2012-02 (February). http://connect.aomonline.org/posts/c72b3d87ac.
Honig, Benson and Akanksha Bedi. 2012.
The Fox in the Hen House: A Critical Examination of Plagiarism Among Members of
the Academy of Management. Academy of
Management Learning & Education, 11.1: 101-123.
Kirkman, Brad and Gilad Chen. 2011. Maximizing your data or data slicing?
Recommendations for managing multiple submissions from the same dataset. Management
and Organization Review 7:3: 433-446.
OECD Global Science Forum. 2009a.
Facilitating International Research
Misconduct Investigations. Final Report of the Co-ordinating Committee.
Paris: OECD. May 7. Accessed May 3,
2012. http://www.oecd.org/dataoecd/29/4/42713295.pdf
OECD Global Science Forum. 2009b. Practical Guide for Investigating Research
Misconduct Allegations in International Collaborative Research Projects.
Paris: OECD. May 15. Accessed May 3,
2012. http://www.oecd.org/dataoecd/42/34/42770261.pdf
ACKNOWLEDGEMENTS: I gratefully thank Chi Anyansi-Archibong, Kathy
Lund Dean, Kim Eggleton, Deidra Schleicher and Paul Vaaler for helpful comments
in connection with an earlier draft of this blog posting, while acknowledging
that the views expressed here are my own.
-------------------------------------------------------------------------------------------------------------------------------
[1] In my February 2012 blog, I discuss
the transparency matrix in Kirkman and Chen (2011) and the originality matrix
used by the Journal of Applied Psychology,
and how both matrices can be used in an originality analysis. I am grateful to Deidra Schleicher and Steve
Kozlowski for providing access to the JAP form.
[2] A brief aside: I require my undergraduate and
graduate (masters and PhD) students to turn their papers into Turnitin.com before
they are submitted to me. The students can submit as often as they like, and in
this manner, they can learn what is and is not plagiarism. I accept the papers
only after they have received a “green” rating from Turnitin.com. My cases of
plagiarism have basically disappeared since I started this practice some years
ago, and I highly recommend it to other instructors.
[3] I give away my
bias here by noting that the Journal of
International Business Studies joined COPE (as did all the Palgrave
journals) during my editorship. I believe that AOM should also join COPE and
adopt (where appropriate) its strategies and procedures for handling ethical
violations.
No comments:
Post a Comment