In my first year of my graduate program at the Manship School of Mass Communication, I took a public opinion course (my favorite course in the whole program, taught by the brilliant Rosanne Scholl, @Rosanne Scholl on Twitter). For that class, I designed and conducted an experiment to test which environmental message worked better: a persuasive message on climate change action based on scientific evidence, or a persuasive message based on fundamental values such as public health and environmental justice? A "fact" frame or a "value" frame?
One semester later, I conducted another experiment in the same vein for another class, in which I tested this "fact vs. value" framing hypothesis for a second environmental issue - hydraulic fracturing, or fracking. I compiled both papers together in a Study 1, Study 2 format, with interesting results: value framing of the environmental issues seemed to generally be more effective, but only for students who had adequate knowledge of the issue to relate it to their prior values, and for students who weren't already highly concerned about the environment.
Armed with what I thought were interesting results, I formatted my paper and submitted for publication to Science Communication. Following months of waiting, I excitedly opened the following letter in my inbox:
"Dear Miss Brown:
I am writing you in regard to manuscript # SC-13-0023 entitled "Science ‘Fact’ vs. ‘Value’ Framing Effects on Attitudes toward Environmental Issues," which you submitted to Science Communication.
All reviewers felt your topic was very important and well-chosen, and I agree. However, the recommendations were mixed. Two reviewers recommended acceptance and one recommended rejection. I try to go with majority opinion in most instances but in this case, after studying your paper and its arguments over many hours, I am concerned that there are fatal flaws in the paper as presently conceptualized. While your data are certainly valuable and interesting, at least as pilot data, the methodological flaws which the third reviewer identified cannot readily be overcome – and at an absolute minimum need to be discussed as important limitations. Please read the review for details; all three reviews are pasted later in this letter, and all provide recommendations worth carefully considering. Unfortunately, in addition to agreeing with Reviewer 3 that many different characteristics distinguish your stimuli “frames” – not just “facts” and “values” – I feel that the comparison between fracking and climate change is also flawed and does not involve simply (or even primarily) a “familiar” versus an “unfamiliar” issue. See other details related to this question below.
For these reasons I feel I have no choice but to reject this paper. However, I would strongly encourage you to consider reconceptualizing your study and rewriting the paper from the ground up, whether for submission to this journal or another one. You need to make your interpretation of these data much clearer and take into account Reviewer 3’s concerns, at a minimum. At this point I believe that you will also need to add new data or redo the entire experiment in a way that helps sort out what is actually going on. At a minimum, new analysis is likely to be required, meaning that an ordinary revision will not be enough to make this publishable. You will need to rewrite in a way that fully acknowledges the problems with the existing data if you want to salvage it. And even so, Reviewer 3 is right that the only way to really figure out what elements are causing your respondents to react differently is to design a new experiment. Perhaps this becomes Study 3?"
If you aren't familiar with manuscript reviews, this particular decision is somewhat of a strange beast. The fact that two reviewers rather enthusiastically recommended acceptance, but the third recommended rejection, is strange enough, but this editor clearly took the time to rigorously review my manuscript and provide me with in-depth advice (about 1 page more than shown) that is uncommon in such a decision letter. I am obviously onto something good in this research, but my concept of "media framing" is by nature a bit controversial, as many communication scholars can't decide exactly what "framing" entails, which hasn't helped the review process here.
So back to the drawing board. I completely re-write the paper and submit it to Journalism and Mass Communication Quarterly - which involves the head-ache task of switching from APA-style references to Chicago-style (learn from me, DO create your reference list in Word using Endnote!!).
After another few months, in short:
"Dear Miss Brown:
I write you in regards to manuscript # JMCQ-13-0079, titled "Science ‘Fact’ vs. ‘Value’ Framing Effects on Attitudes toward Environmental Issues," which you submitted to Journalism & Mass Communication Quarterly.
In view of the criticisms of the reviewer(s) found at the bottom of this letter, your manuscript has been denied publication in Journalism & MassCommunication Quarterly. Like the reviewers, I believe the study's topic is of great potential importance, and hope you will continue to pursue it."
Dang. At this point, I'm almost ready to give up on this paper. But in the mean time I've been working on a THIRD study, that not only tests the fact vs. value framing hypothesis for a third environmental issue - coast wetland land loss in Louisiana - but tests this hypothesis in light of respondents' scores on pre-existing fundamental human value orientations. I again get results confirming and expanding my results from the first two studies, so I buck up and decide to submit the paper one more time, to the journal Environmental Communication: A Journal of Nature & Culture.
Again, several months later:
"Dear Miss Brown,
Your manuscript entitled "Fact vs. Value Framing Effects on Attitudes toward Environmental Issues" which you submitted to Environmental Communication: A Journal of Nature and Culture, has been reviewed. The reviewer comments are included at the bottom of this letter.
I regret to inform you that the reviewers have raised serious concerns, and therefore your paper cannot be accepted for publication in Environmental Communication: A Journal of Nature and Culture. However since the reviewers do find some merit in the paper, I would be willing to reconsider if you wish to undertake major revisions and re-submit, addressing the referees' concerns.
Please note that resubmitting your manuscript does not guarantee eventual acceptance, and that your resubmission will be subject to re-review before a decision is rendered."
At first, I nearly cry and tear my paper to shreds (or break my computer). But after fowarding the e-mail angrily to my faculty advisor, she tells me to re-read the email: this is a revise and resubmit, a success!! The journal has given me five months to revise and resubmit my manuscript for re-consideration for publication.
So what do I do? After talking with my advisor, I decide that the only way to move forward is to combine my third study into the manuscript, in a Study 1, 2 & 3 format. I also decide, per the reviewers' recommendations, to conduct a manipulation check on my "framed" articles to make sure the "value" frame is actually making audiences think more about their values than the "fact" frame. The manipulation check works as expected... despite reviewers' concerns, I am still convinced that my "value" article and my "fact" article are working as intended.
I take several months to again completely re-write the literature review section of my manuscript. I add the third experimental study results and re-vamp my discussion section. I re-do tables and figures, and discuss my methods more clearly. I carefully go through the letter from the editor of Environmental Communication, to make sure that I address each reviewer's comments. I re-write, re-write and re-write until I have a manuscript that flows and tells a story.
Finally, I write a long re-submission cover letter describing exactly how my revised manuscript addresses the peer-reviewed comments, concerns and suggestions, and I re-submit my paper to the journal on Oct. 31st, 2013 (yes, on Halloween!)
All this to say, this paper has been nearly 2 years in the making, and the process isn't even close to being complete. If my revised manuscript DOES get accepted to Environmental Communication (keep your fingers crossed), reviewers will almost surely want more edits prior to official publication. But in the process, I've learned a lot about the peer-review process, how to write a cover letter, and how to persist in the publishing of an academic paper. Wish me luck!