SLICING AND DICING: EX ANTE APPROACHES
Blog No. 2012-02 (February 2, 2012)
NOTE: This posting first appeared on THE ETHICIST, the Academy of Management's ethics blog, http://connect.aomonline.org/resources/614c81fe4f/summary, in Feb 2012. The
AOM's THE ETHICIST has three contributors, each one writing a monthly blog in sequence, on (1) research (Lorraine Eden), (2) teaching (Kathy Lund Dean) and (3) professional life (Paul Vaaler). Additional copies of my blog posts, in PDF format, are downloadable from my LinkedIn page at http://www.linkedin.com/in/transferpricingaggies.
Key Insight: Research projects are often huge undertakings
that lead to more than one publication. How do authors determine whether the
papers coming out of one project are sufficiently different from one another to
be considered new papers? In this blog, I look at some ex ante methods
that authors can use to determine whether a paper is new.
1. Case Examples
Example 1: Two co-authors have a major project underway
and want to maximize the number of publications from the project. They
recognize that journal editors frown on “slicing and dicing” and want to make
sure that the papers are sufficiently different so they really are different papers. However, the
co-authors don’t know what makes one paper sufficiently
different from another. Is it the dataset? The hypotheses? The empirical
findings? They search for information on what makes one paper sufficiently
different from another and cannot find a definitive answer.
Example 2: A new assistant professor is carving up his dissertation
into journal articles. He sends the
first article, which has the major theoretical and empirical contributions of
the dissertation, to Academy of
Management Journal (AMJ). The second
article, which looks at two moderators of the main effects in the first paper,
is sent to the Journal of International
Business Studies (JIBS). Both papers
use the same dataset and variables, with the exception that the moderator variables
in the JIBS submission are treated as control variables in the AMJ submission. The
hypotheses in the JIBS submission include some of the same hypotheses that
appear in the AMJ submission, with the addition of new ones for the moderator
effects. The AMJ submission goes in
first; the JIBS submission follows a month later. In the JIBS submission, the
author makes no mention of the prior submission to AMJ, either in the letter to
the editor or in the body of the paper. Nor does the author tell AMJ that a
second submission to JIBS is planned. The author reasons that he does not need
to mention either submission to the other journal because neither submission
has been published and, even if both papers should eventually be accepted for
publication, they will be so changed during the reviewing process that the
likelihood of duplicate material is low.
Example 3: Two years earlier, a professor published an
article in JIBS. She has now refined her
thinking and has a follow-up article building on the first one, but still using
the same dataset. Can she also submit the follow-up paper to JIBS?
Example 4: Three
co-authors submitted a paper to AMJ, which was rejected after the first round
of review. The co-authors spent a year
significantly revising the paper based on the reviewers’ and editor’s feedback.
The co-authors believed that the revised paper was sufficiently different that
they could submit the revised paper to AMJ as a “new submission”. In their cover letter to the journal, they
made no mention of the previously rejected submission.
2. The Problem
All four of the
above examples are slightly disguised real-world examples with which I am
familiar, either from my own research, my term as a JIBS editor and/or from
discussions with other journal editors. I suspect you can add more examples to
the ones I have above.
All four examples
involve what I refer to as the potential for “slicing and dicing” and the Committee on Publication Ethics (COPE) calls “salami publishing” or “redundancy”;
that is, the excessive cutting up of a research project into multiple papers
where each paper overlaps significantly with other papers from the same project.
Examples 1 and 2 involve situations where the papers come simultaneously out of
the same project. Example 3 (closely related papers follow one another in
sequence) and Example 4 (the authors revise a previously rejected paper and
send the revised paper to the same journal) involve sequential slicing and
dicing.
The core issue in
all of these cases is determining when a
paper is really or sufficiently new. How do we know when it is OK to
publish two or more papers out of one project? Where do we cross the line from being
OK to engaging in slicing and dicing?
We all know how
heavy are the publish-or-perish pressures, especially for junior faculty. (On
this topic, see my first THE ETHICIST: RESEARCH blog posting (Eden, August 2012), “Scientists behaving badly: Lessons from
the Fraud Triangle”.) An author’s desire
to segment his or her big research project into multiple, stand-alone papers
aimed at different journals is therefore not surprising. The key issue is where
to draw the line between two papers that are “siblings” (same intellectual
parents but different children) and those that are “clones”?
In this blog, I
address ex ante approaches to handling the ethical dilemma of slicing and
dicing. In my May 2012 blog, I will look at ex post approaches as recommended
by COPE.
3. Ex Ante Approaches
I
see four possible ex ante approaches for handling the slicing-and-dicing
problem.
a.
Craft Different Papers at Project Inception
Brad Kirkman and Gilad Chen in
their article, “Maximizing your data or data slicing? Recommendations for
managing multiple submissions from the same dataset” (Management and Organization Review 7:3, 2011, 433-446), provide our
first ex ante approach. When authors are starting out at the beginning of a
project, it is easier if they “intentionally craft and design [… ] separate
papers from the inception of the project” (Kirkman & Chen, 2011: 437). By starting at the beginning, authors have a
roadmap that helps keep the papers separate.
The papers can, for example, be aimed at different audiences, start with
different research questions and theoretical approaches, and/or use different
datasets.
b. Follow the Journal’s Instructions
What if “the horse is out of the barn” and you
didn’t craft separate papers from the beginning? What can you do? My advice is
to first turn to what the journal editors say on this topic. Editors want
innovative, thought-provoking, original articles published in their journals. They know about the pressures to engage in
slicing and dicing, and that authors may check the box that an article is “original”
even if it comes out a big research project.
Most journals therefore have an explicit policy defining originality and
asking authors to confirm at the time of submission that their paper is
original.
The AMJ
submission requirements are probably typical of most journals. AMJ requires authors to “check the box” that
(1) their manuscript is original, (2) not published or under review at another
journal, and (3) will not be submitted to another journal during the review
process. In addition, the submission
requirements ask authors to check the box to “confirm that their manuscripts
have not previously been submitted to AMJ for review.” Some journals go beyond this list to define
what they see as an original manuscript; the JIBS Code
of Ethics for Authors, for example, devotes
several paragraphs to what the editors see as original and what the journal
considers to be self-plagiarism or redundancy.
In my Example 1 (carving out papers from a
project) and Example 2 (carving up a dissertation into papers), the authors should
therefore look to the journals for definitions of originality, both to the
Instructions to Authors and to the Code of Ethics (if the journal has one).
Example 3 (sequential papers) and Example 4
(revised and submitted to the same journal after rejection) are slightly
different problems. Journals also do
provide instructions that are helpful for these situations. Michele
Kacmar’s 2009 AMJ “From
the Editors” letter, “An Ethical Quiz”, specifically addresses these cases in
her Scenario 2: Data Reuse. Kacmar
(2009: 432) explains that
AMJ requires authors to answer two questions, which I quote below:
·
Has another manuscript from this same database
ever been previously submitted to AMJ? If yes, please note this in your cover
letter, explain how this paper differs from the earlier one, and attach a copy
of the previous manuscript.
·
Has another manuscript from this same database
been accepted by or previously published at AMJ or at another journal? If yes,
please note this in your cover letter, explain how this paper differs from the
previous one, and attach a copy of the accepted or published manuscript.
The first question addresses papers that have
been previously submitted to AMJ (Example 4); the second addresses articles
previously published in AMJ (Example 3). In a situation where the author says
“yes” to either question, AMJ requires the author to add an explanation to the
cover letter at the time of submission and attach the other manuscript.
JIBS also has an FAQ posted on http://www.jibs.net that discourages resubmissions when a
manuscript has been rejected after review, except in special circumstances that
are outlined in the FAQ. The JIBS Code
of Ethics for Authors does not
specifically address Example 3 (sequential publications) but I believe the case
would fall within the section on Self-Plagiarism.
b. Do an Originality Analysis
Both AMJ and JIBS, interestingly, do allow a
bit of wiggle room for exceptions from what might be called the “no second kick
at the can” rule deterring authors from making sequential slice-and-dice submissions
to the same journal. It is this wiggle room – as mapped out by AMJ and JIBS ---
that I see as really helpful to authors in determining when carving up a
project into papers moves from OK into the unacceptable realm of slicing and dicing.
The JIBS FAQ would allow the authors in Example 4 to make
a new submission if “the revised manuscript becomes a new manuscript through
significant revision in terms of theory development, empirical work and
discussion, and also uses a substantially different dataset”. The FAQ adds that “the addition of one or two
new variables to an old dataset does not make a new dataset.”
Example
4 was also directly addressed in a 2009 AMJ “From the Editors” letter by (at the time) editor-in-chief Duane Ireland. His editorial,
“When Is a “New” Paper Really New?”, specifically lists three criteria that
must be met for a previously rejected manuscript to be considered a new
submission to AMJ: “The new manuscript
must (1) address modified or new research questions, (2) use new theoretical
arguments, and (3) use additional or new data to test the proposed
relationships. Satisfying or meeting one or two of the three criteria is not
sufficient.” (Ireland,
2009: 10) So, if the authors in Examples 3 and 4 were
to meet all three criteria, it would be OK for them to make a new submission to
AMJ.
These
two editorial policy statements suggest a useful way for authors to determine
when a paper that is part of a project is sufficiently new to be separately
published. If the authors create an originality matrix comparing the two papers
in terms of their component parts, it should be clear both authors and to the
journal editors whether there is sufficient differentiation to justify
separation of the papers. The statements by the two journals suggest that AMJ
would require differentiation in three areas: research question, theoretical
arguments and dataset. To this list, JIBS would add empirical tests and discussion.
Brad Kirkman and Gilad Chen (2011) develop a similar matrix, which they call a uniqueness analysis, based on five
components: research question, theories used, constructs/variables, and theoretical
implications and managerial implications. Kirkman and Chen provide two helpful tables, using
their own published papers, to show how authors can compare manuscripts in
terms of originality.
Based
on these three sets of criteria for originality identified by Ireland, Kirkman
and Chen, and the JIBS FAQ, I recommend that authors set up a matrix where the columns
are papers and the rows are criteria used to judge originality. See Table 1
below.
Table 1: An
Originality Matrix (Part 1)
|
||||
Paper 1
|
Paper 2
|
Overlap
|
Difference
|
|
Research Question(s)
|
||||
Theoretical
Arguments
|
||||
Dataset
|
||||
Constructs /
Variables Used
|
||||
Empirical
Tests
|
||||
Discussion:
Theoretical Implications
|
||||
Discussion:
Managerial Implications
|
||||
To
make the comparison even sharper, I argue that it is important not only to fill
in the table cells, but also to examine differences
and similarities or overlap between the
two papers. My matrix therefore also has two columns at the end where the
author must assess overlap and differences. (When I was JIBS editor-in-chief, I
occasionally asked authors to complete a version of Table 1 in cases that
appeared to possibly involve slicing and dicing. The authors and I then engaged
in a dialogue, based on the matrix, to determine whether the new submission was
sufficiently new. These dialogues led to the JIBS FAQ.)
Completing
Table 1, of course, forces the author to stand back and be ruthlessly honest
about both papers – not an easy task. Authors must use a self-critical eye or
the exercise is pointless. Once the exercise has been completed, the author
should look hard at the answers, particularly the last two columns on overlaps
and differences. These columns may well suggest ways that the two papers could
be further revised so as to make them even more separate.
Is
Table 1 sufficient to determine originality? Probably not for empirical papers.
To do this rigorously and thoroughly, I believe that one must go further than
Table 1 and delve into the issue of what makes two datasets different. Journal of Applied Psychology (JAP) has
a long-standing policy of publishing original
data. JAP requires authors to inform the editorial team, either in their
cover letter or in the methods section, if their dataset has or will be used in
other journal submissions not only to JAP but to other journals. If an author
says “yes”, JAP sends the author a separate form to complete, which I attach
below as Table 2. [i] Based on the completed Table 2, together with
any accompanying documentation (e.g., other manuscripts), the JAP editors can
more easily and accurately determine whether or not to accept the manuscript as
a new submission.
Table 2: JAP Original Data Appendix
(Originality Matrix Part 2)
|
||||||||||||||||||||||||||||||
Instructions: Authors
should edit accordingly to describe what has been done and what is planned.
Use as many columns as necessary. Provide any additional information
necessary to clarify the unique contribution of each manuscript. Please note the STATUS of each manuscript
connected to the data collection:
under review, in press, published, current ms, planned (anything
else).
----------------------------
The data
reported in this manuscript have been previously published and / or were
collected as part of a larger data collection (at one or more points in
time). Findings from the data collection have been reported in separate
manuscripts. MS 1 (status) focuses on
variables _____; MS 2 (status) focuses on variables _____. MS 3 (status) focuses on variables
_____. MS 4 (status) will focus on
variables ____. The table below
displays where each data variable appears in each study, as well as the
current status of each study.
|
I
therefore view a completed originality matrix as consisting of carefully and
honestly prepared Tables 1 and 2. A
completed originality matrix can help with all four of my examples, whether
starting a project and trying to determine the optimal number and content of
the papers (Example 1), carving up a dissertation (Example 2), or the
sequential issues discussed in Examples 3 and 4. By completing both tables, authors can
determine whether two papers are sufficiently different that they can ethically
be submitted to the same or different journals and published separately.
d.
Transparency Matters
In
addition to determining originality, there is an additional ethical issue
involved in slicing and dicing: transparency.
What should the author tell the journal editors at the time of
submission? I recommend that authors
“spill the beans”. Transparency is the best policy. Go for full disclosure, as Marshall Schminke, (at the
time) chair of the AOM Ethics Education Committee, argued in his AMR Editor’s
Comments, on “The Better Angels of Our Nature – Ethics and Integrity in the Publishing
Process”. Submitting
a cover letter with the completed originality matrix and the relevant papers to
the journal editors accomplishes full transparency.
Kirkman and Chen (2011: 442) are also
strong advocates for full transparency. They recognize that transparency
jeopardizes the double blind review process, but argue that transparency
matters more, given that the “ultimate goal of science is to build and advance our knowledge base”, which requires a clear assessment of
unique contribution of each paper. I recognize the problem (see my November 2011 THE ETHICIST: RESEARCH blog on double blind review), but also
agree with their assessment and have argued so elsewhere (Eden, JIBS Editorial, “Scientists behaving
badly, 2010). Transparency
– in the case of possible slicing and dicing – matters. Journal editors should
be provided with full information by authors, and then the decision on whether
to share the originality matrix and papers with the journal’s reviewers (and
thus violate one side of double blind review) should be left up to the journal editors.
e.
Summary
To
recap, I argue that there are established criteria for determining whether two
papers from the same research project are sufficiently different to be
considered that both can be treated as new papers. Authors should use these criteria to
determine whether a paper is original. I summarize the implications for my four
Examples below:
·
Example 1 (carving up a project into papers): Best to do this
at the beginning, not the end, of a project. Authors should deconstruct their
papers using the originality matrix (Tables 1 and 2) and share these results
with journal editors of the different papers at the time of submission.
·
Example 2 (main effect paper followed by moderator
paper): The case, as described, would not pass the originality matrix test;
moreover, there is a lack of transparency. The author should revise both
papers, using the originality matrix, until they are sufficiently different to
be treated as separate. He should provide both the matrix and papers to both
journals.
·
Example 3 (Second paper that grew out of the first
one): The answer is this depends on how
different the two papers are from one another. Again, an originality matrix is
needed and all information should be supplied to the journal if the author
decides to go forward with submission.
·
Example 4 (Once rejected after review, can I revise and
make a new submission?): The answer is probably not, except in exceptional
circumstances. These circumstances require an assessment that the two papers
were sufficiently different, based on a completed originality matrix, and
submission of the matrix and papers to the journal editors.
4. Questions for discussion
I
hope that this blog posting provides some food for thought. I return to this topic in my next THE
ETHICIST: RESEARCH blog, looking at ex post solutions. Some possible questions
for discussion might be:
- Do you think that “slicing and dicing”
(or “salami publishing”) is a problem?
- Have you been faced with situations such
as the ones described in this blog posting? How did you or you and your
co-authors handle them?
- What criteria would you use to determine
when a manuscript is new?
- Is the policy advocated here (an
originality matrix plus transparency) too onerous a burden to place on
authors?
- What advice do you give to your doctoral
students and junior faculty about managing a big research project like a
dissertation through to publication?
ENDNOTES
[1] I thank
the many members of THE ETHICIST advisory board who provided helpful comments
and shared resources on this topic, especially Kathy Lund Dean, Susan Jackson,
Deidra Schleicher and Anne Tsui. The views expressed here are my own.
[i] I thank Deidra
Schleicher and Steve Kozlowski at JAP for providing access to this form and
allowing its publication in THE ETHICIST.
The
Ethicist Terms of Use: AOM, contributors to THE ETHICIST, and
AOM officers, staff and volunteers accept no responsibility for the content of
all postings on THE ETHICIST, including the opinions and information posted or
circulated by users on THE ETHICIST. The content of all postings is
solely the responsibility of the users. AOM cannot warrant the accuracy of any
information posted on THE ETHICIST and disclaims all warranties with regard to
information circulated on THE ETHICIST. This disclaimer includes all
warranties of merchantability and fitness.
No comments:
Post a Comment