For Science's Gatekeepers, a Credibility Gap (Re those flawed mold studies)

[Guest guest]

http://www.nytimes.com/2006/05/02/health/02docs.html?_r=1 & oref=slogin & pagewa

nted=all

The New York Times

The Doctor's World

For Science's Gatekeepers, a Credibility Gap

By LAWRENCE K. ALTMAN, M.D.

Published: May 2, 2006

Recent disclosures of fraudulent or flawed studies in

medical and scientific

journals have called into question as never before the

merits of their

peer-review system.

The system is based on journals inviting independent

experts to critique

submitted manuscripts. The stated aim is to weed out

sloppy and bad

research, ensuring the integrity of what it has

published.

Because findings published in peer-reviewed journals

affect patient care,

public policy and the authors' academic promotions,

journal editors contend

that new scientific information should be published in

a peer-reviewed

journal before it is presented to doctors and the

public.

That message, however, has created a widespread

misimpression that passing

peer review is the scientific equivalent of the Good

Housekeeping seal of

approval.

Virtually every major scientific and medical journal

has been humbled

recently by publishing findings that are later

discredited. The flurry of

episodes has led many people to ask why authors,

editors and independent

expert reviewers all failed to detect the problems

before publication.

The publication process is complex. Many factors can

allow error, even

fraud, to slip through. They include economic

pressures for journals to

avoid investigating suspected errors; the desire to

avoid displeasing the

authors and the experts who review manuscripts; and

the fear that angry

scientists will withhold the manuscripts that are the

lifeline of the

journals, putting them out of business.By promoting

the sanctity of peer

review and using it to justify a number of their

actions in recent years,

journals have added to their enormous power.

The release of news about scientific and medical

findings is among the most

tightly managed in country. Journals control when the

public learns about

findings from taxpayer-supported research by setting

dates when the research

can be published. They also impose severe restrictions

on what authors can

say publicly, even before they submit a manuscript,

and they have penalized

authors for infractions by refusing to publish their

papers. Exceptions are

made for scientific meetings and health emergencies.

But many authors have still withheld information for

fear that journals

would pull their papers for an infraction.

Increasingly, journals and

authors' institutions also send out news releases

ahead of time about a

peer-reviewed discovery so that reports from news

organizations coincide

with a journal's date of issue.

A barrage of news reports can follow. But often the

news release is sent

without the full paper, so reports may be based only

on the spin created by

a journal or an institution.

Journal editors say publicity about corrections and

retractions distorts and

erodes confidence in science, which is an honorable

business. Editors also

say they are gatekeepers, not detectives, and that

even though peer review

is not intended to detect fraud, it catches flawed

research and improves the

quality of the thousands of published papers.

However, even the system's most ardent supporters

acknowledge that peer

review does not eliminate mediocre and inferior papers

and has never passed

the very test for which it is used. Studies have found

that journals publish

findings based on sloppy statistics. If peer review

were a drug, it would

never be marketed, say critics, including journal

editors.

None of the recent flawed studies have been as

humiliating as an article in

1972 in the journal Pediatrics that labeled sudden

infant death syndrome a

hereditary disorder, when, in the case examined, the

real cause was murder.

Twenty-three years later, the mother was convicted of

smothering her five

children. Scientific naïveté surely contributed to the

false conclusion, but

a forensic pathologist was not one of the reviewers.

The faulty research in

part prompted the National Institutes of Health to

spend millions of dollars

on a wrong line of research.

Fraud, flawed articles and corrections have haunted

general interest news

organizations. But such problems are far more

embarrassing for scientific

journals because of their claims for the superiority

of their system of

editing.

A widespread belief among nonscientists is that

journal editors and their

reviewers check authors' research firsthand and even

repeat the research. In

fact, journal editors do not routinely examine

authors' scientific

notebooks. Instead, they rely on peer reviewers'

criticisms, which are based

on the information submitted by the authors.

While editors and reviewers may ask authors for more

information, journals

and their invited experts examine raw data only under

the most unusual

circumstances.

In that respect, journal editors are like newspaper

editors, who check the

content of reporters' copy for facts and internal

inconsistencies but

generally not their notes. Still, journal editors have

refused to call peer

review what many others say it is — a form of vetting

or technical editing.

In spot checks, many scientists and nonscientists said

they believed that

editors decided what to publish by counting reviewers'

votes. But journal

editors say that they are not tally clerks and that

decisions to publish are

theirs, not the reviewers'.

Editors say they have accepted a number of papers that

reviewers have

harshly criticized as unworthy of publication and have

rejected many that

received high plaudits.

Many nonscientists perceive reviewers to be impartial.

But the reviewers,

called independent experts, in fact are often

competitors of the authors of

the papers they scrutinize, raising potential

conflicts of interest.

Except when gaffes are publicized, there is little

scrutiny of the quality

of what journals publish.

Journals have rejected calls to make the process

scientific by conducting

random audits like those used to monitor quality

control in medicine. The

costs and the potential for creating distrust are the

most commonly cited

reasons for not auditing.

In defending themselves, journal editors often shift

blame to the authors

and excuse themselves and their peer reviewers.

Journals seldom investigate frauds that they have

published, contending that

they are not investigative bodies and that they could

not afford the costs.

Instead, the journals say that the investigations are

up to the accused

authors' employers and agencies that financed the

research.

Editors also insist that science corrects its errors.

But corrections often

require whistle-blowers or prodding by lawyers.

Editors at The New England

Journal of Medicine said they would not have learned

about a problem that

led them to publish two letters of concern about

omission of data concerning

the arthritis drug Vioxx unless lawyers for the drug's

manufacturer, Merck,

had asked them questions in depositions. Fraud has

also slipped through in

part because editors have long been loath to question

the authors.

" A request from an editor for primary data to support

the honesty of an

author's findings in a manuscript under review would

probably poison the air

and make civil discourse between authors and editors

even more difficult

than it is now, " Dr. Arnold S. Relman wrote in 1983.

At the time, he was

editor of The New England Journal of Medicine, and it

had published a

fraudulent paper.

Fraud is a substantial problem, and the attitude

toward it has changed

little over the years, other editors say. Some

journals fail to retract

known cases of fraud for fear of lawsuits.

Journals have no widely accepted way to retract

papers, said Kennedy,

editor in chief of Science, after the it retracted two

papers by the South

Korean researcher Dr. Hwang Woo Suk , who fabricated

evidence that he had

cloned human cells.

In the April 18 issue of ls of Internal Medicine,

its editor, Dr. Harold

C. Sox, wrote about lessons learned after the journal

retracted an article

on menopause by Dr. Poehlman of the University

of Vermont.

When an author is found to have fabricated data in one

paper, scientists

rarely examine all of that author's publications, so

the scientific

literature may be more polluted than believed, Dr. Sox

said.

Dr. Sox and other scientists have documented that

invalid work is not

effectively purged from the scientific literature

because the authors of new

papers continue to cite retracted ones.

When journals try to retract discredited papers, Dr.

Sox said, the process

is slow, and the system used to inform readers faulty.

Authors often use

euphemisms instead of the words " fabrication " or

" research misconduct, " and

finding published retractions can be costly because

some affected journals

charge readers a fee to visit their Web sites to learn

about them, Dr. Sox

said.

Despite its flaws, scientists favor the system in part

because they need to

publish or perish. The institutions where the

scientists work and the

private and government agencies that pay for their

grants seek publicity in

their eagerness to show financial backers results for

their efforts.

The public and many scientists tend to overlook the

journals' economic

benefits that stem from linking their embargo policies

to peer review. Some

journals are owned by private for-profit companies,

while others are owned

by professional societies that rely on income from the

journals. The costs

of running journals are low because authors and

reviewers are generally not

paid.

A few journals that not long ago measured profits in

the tens of thousands

of dollars a year now make millions, according to at

least three editors who

agreed to discuss finances only if granted anonymity,

because they were not

authorized to speak about finances.

Any influential system that profits from

taxpayer-financed research should

be held publicly accountable for how the revenues are

spent. Journals

generally decline to disclose such data.

Although editors of some journals say they demand

statements from their

editing staff members that they have no financial

conflicts of interest,

there is no way to be sure. At least one editor of a

leading American

journal had to resign because of conflicts of interest

with industry.

Journals have devolved into information-laundering

operations for the

pharmaceutical industry, say Dr. , the

former editor of BMJ,

the British medical journal, and Dr. Horton,

the editor of The

Lancet, also based in Britain.

The journals rely on revenues from industry

advertisements. But because

journals also profit handsomely by selling drug

companies reprints of

articles reporting findings from large clinical trials

involving their

products, editors may " face a frighteningly stark

conflict of interest " in

deciding whether to publish such a study, Dr.

said.

FAIR USE NOTICE: This may contain copyrighted (© )

material the use of which

has not always been specifically authorized by the

copyright owner. Such

material is made available for educational purposes,

to advance

understanding of human rights, democracy, scientific,

moral, ethical, and

social justice issues, etc. It is believed that this

constitutes a 'fair

use' of any such copyrighted material as provided for

in Title 17 U.S.C.

section 107 of the US Copyright Law. This material is

distributed without

profit.

__________________________________________________