December 21, 2010 by David Taylor
When I was doing my PhD in the 1960s the world of science was a different place; for example, computers were very large machines housed in computer laboratories and my thesis had to be typed and figures drawn by hand. However, although there have been many beneficial advances in scientific research over the last 50 years there have been some serious retrograde steps as well, many relating to the publication of results.
My supervisor, Professor John Riley, not only taught me how to do research work; he also taught me how to communicate the results of that research to the scientific community and, incidentally, how to peer review the work of others. In those days, a significant set of results would be written up for publication in an appropriate journal by those who had actually undertaken the intellectual and experimental work being reported, acknowledging the help of others at the end of the article. The draft paper would then be submitted to the appropriate journal; the choice of journal was usually limited and the editorial standards were high. The paper would only be accepted for publication, after peer review, provided that it represented a significant advance, contained enough detail for it to be repeated by other scientists and had not been previously published.
Today it is possible to identify three retrograde trends: the explosion in the number of journals, the pressure on scientists, particularly in academe to publish and the decline in journalistic standards in the mainstream media.
Never mind the quality, feel the width
In the 1970s there was a veritable explosion in the number of scientific journals, although there was not a concurrent increase in the amount of scientific work that was worthy of publication. The result was inevitable; the driving force for publication was no longer the scientific need to disseminate new insights but the journal publishers’ need to sell their products; more nearly always means worse and, as the number of journals increased, the quality and relevance of published work slowly declined. Now, in order to keep up with any specialist field a scientist has to scan more and more journals to separate the wheat from the chaff. As a peer reviewer for analytical chemistry journals in the 1980’s, it was noticeable that the familiarity of authors with the established literature gradually declined as the task of keeping up with the mountain of published literature became simply too time consuming.
Publish or Perish
At the beginning of the last century it was not uncommon for a scientist only to publish a handful of papers during an entire career. Since scientific advances are not necessarily proportionate to activity, lack of publication was not then seen as idleness. In addition this self imposed rigour meant that any paper in your field of interest was worth reading. Today the increasing use of publications as a metric for research endeavour and quality has had the entirely predictable, if unintended effect, of creating a crisis of obesity in the scientific literature. Scientists in academia now have an overriding need to publish as frequently as possible regardless of whether this is necessary for the advancement of science. So now we see the following.
- Incremental research that will lead to lots of publishing opportunities being favoured over more worthwhile fundamental in-depth work that may take a long time to produce anything publishable, and may ultimately be unsuccessful.
- More and more “me too” work that appears to have little scientific merit, but can be justified as novel because no one has done that precise work before. For example, papers continue to be published showing that pharmaceutical residues can be found in the effluent from sewage treatment plants. Well d’oh! We know that now. When the first couple of papers were published this was indeed both novel and surprising but today a significant paper would be one that showed that there were no residues in the effluent of a particular treatment plant!
- Research work being split into the smallest number of parts possible with each one being presented for publication.
- An increase in the number of authors on each paper so that each one adds to his publication record. People who previously would have been mentioned in the acknowledgements now often appear in the author list.
- The same work being published repeatedly in different journals, often with the names of the investigators in a different order.
The combination of the increase in the number of journals with the pressure on academic scientists to publish has led to an enormous increase in the volume of scientific literature but little if any increase in quality. In the 1960s most published papers in your field were well worth reading, but today probably only 1 in 100 falls into that category. However, although regrettable, this is an internal problem for the scientific community. The final part of this article is much more serious because it affects the standing of the scientific community with the public.
Publication by Press Release
When Einstein produced the theory of general relativity and Crick & Watson solved the structure of DNA their work was communicated very effectively, and with great excitement, to the scientific community by short papers in the journal Nature. Despite the paradigm changing nature of their results they did not consider a Press Release to be in any way appropriate. However, today it seems that any trivial piece of elementary scientific research has to be broadcast to the waiting media.
The reason for this change appears to be twofold; firstly everyone now wants to be a “celebrity” and scientists who make a name for themselves in the media now appear to climb the greasy pole just that little bit faster. Secondly, more and more universities are competing for a shrinking amount of funding so it is in their interests, aided by in-house press officers, to ensure that their name appears in the media as frequently as possible.
At the same time the mainstream media is under threat from innovative information platforms and has responded to this largely financial challenge by reducing both the standards and number of their journalists. As a consequence academic press releases, especially those that tell “exciting” stories, are likely to be regurgitated straight into the newspaper without even the most cursory research or fact checking by the journalist.
This unholy alliance means that a trivial, and often speculative and unproven, piece of scientific work gets exaggerated by the university press office. The resultant press release is then seized on by an overstretched journalist who hypes it further to make it into a “good” story.
Scare stories sell newspapers, so it is not surprising that press officers try to make their press releases emulate the Fat Boy in Dickens Pickwick Papers who “wanted to make your flesh creep”. However this is doing a major disservice to the community by turning every minor, and often unverified, scientific observation into an apocalyptic vision.
A recent example of the genre
What I describe above is becoming the norm in scientific publications. Take this recent paper by Steinemann et al. as an example. My only criticism of the paper itself relates to its lack of any significantly new scientific advances, unfortunately an increasingly common issue in the scientific literature. Previous work, discussed in the text, shows that the main findings of the paper had previously been published by others in 1991. There was nothing novel in the analytical methods used and nothing surprising in the data generated when compared to previous studies. This study was merely a “stamp collecting” exercise; a bit more data has now been acquired that is equivalent to that which was previously known. It is also interesting to note that there are seven authors. It is not clear why quite so many scientists were required for such a small pice of work.
However the Press Release which was syndicated across a large number of outlets contains none of the caveats that are quite properly in the published paper. Nothing in the Press Release is incorrect, but the overall tone is misleading and alarmist. The author of the press release could readily find a job as a political spin doctor where facts are often used in a thoroughly misleading manner.
For example the press release concentrates on the words “carcinogenic” and “toxic” but provides no context and no indication of either the likely exposure or the probable margin of safety. Typical quotes are:
“The sweet smell of fresh laundry may contain a sour note. Widely used fragranced products – including those that claim to be “green” – give off many chemicals that are not listed on the label, including some that are classified as toxic”
“All products emitted at least one chemical classified as toxic or hazardous. Eleven products emitted at least one probable carcinogen according to the EPA”
“Of the 133 different chemicals detected, nearly a quarter are classified as toxic or hazardous under at least one federal law”
Nor does the press release inform the reader that a large number of the compounds detected are actually natural products. For example, the most frequently detected substance was limonene described accurately in the Press Release as a compound with citrus scents without pointing out that it is a naturally occurring substance found in extremely high concentrations in the rind of oranges, lemons and limes.
However, my biggest criticism of the press release is how it deals with risk. The original paper does contain the following disclaimer, albeit buried at the end of the paper
“this study did not seek to assess, and makes no claims regarding, whether product usage would be associated with any risks.”
This is of course true, the authors could not say anything whatsoever about consumer risks because they have no information about consumer exposure under realistic conditions. The experimental conditions from which their data were produced bore absolutely no relation to any realistic consumer exposure. In addition, they show no toxicological data other than rudimentary classification and labelling information. However, the press release is more nuanced, using this form of words:
“The study establishes the presence of various chemicals but makes no claims about the possible health effects”
The effect of this sentence is, however, seriously undermined by the two sentences which immediately follow:
“Two national surveys published by Steinemann and a colleague in 2009 found that about 20 percent of the population reported adverse health effects from air fresheners, and about 10 percent complained of adverse effects from laundry products vented to the outdoors. Among asthmatics, such complaints were roughly twice as common.”
Other quotations in the Press Release from the principal author also tend to give an impression that goes well beyond the scientific data contained in the paper to which this press release refers. For example:
“We don’t want to give people the impression that if we reported on product ‘A’ and they buy product ‘B,’ that they’re safe,” Steinemann said. “We found potentially hazardous chemicals in all of the fragranced products we tested.”
“In the past two years, I’ve received more than 1,000 e-mails, messages, and telephone calls from people saying: ‘Thank you for doing this research, these products are making me sick, and now I can start to understand why,’” Steinemann said.”
We have here a relatively innocuous scientific publication which, on its own, would have contributed a small amount to our scientific understanding and would have merited little comment in the scientific community. It is, however, accompanied by a Press Release, probably written by the University press office, which exaggerates its significance and appears to have been designed to cause alarm in consumers, thus guaranteeing a high profile.
I think this sort of activity does science and the community a disservice and it is time it ceased.