The Fallacy of Hard Tests

[Guest guest]

In a message dated 6/17/2007 6:55:15 P.M. Central Daylight Time,

jkaymdc@... writes:

They test more on how well the candidate CAN test.....but maybe that has

changed since *I* took them..<G>

Jules

I believe there is more tan ample proof that once can " teach " a person to

perform well on any " standardized test " after all Sylvan learning Centers got

my kid 222 points on her SAT's and she took the tests a few months apart so no

one poured smarts in her. She was also the " good student " that did crap on

any " standardized test. She went to Sylvan and poof she's 222 points smarter,

not.

Louis N. Molino, Sr., CET

FF/NREMT-B/FSI/EMSI

Freelance Consultant/Trainer/Author/Journalist/Fire Protection Consultant

LNMolino@...

(Cell Phone)

(IFW/TFW/FSS Office)

(IFW/TFW/FSS Fax)

" A Texan with a Jersey Attitude "

" Great minds discuss ideas; Average minds discuss events; Small minds

discuss people " Eleanor Roosevelt - US diplomat & reformer (1884 - 1962)

The comments contained in this E-mail are the opinions of the author and the

author alone. I in no way ever intend to speak for any person or

organization that I am in any way whatsoever involved or associated with unless

I

specifically state that I am doing so. Further this E-mail is intended only for

its

stated recipient and may contain private and or confidential materials

retransmission is strictly prohibited unless placed in the public domain by the

original author.

************************************** See what's free at http://www.aol.com.

[Guest guest]

Forwarded to me via email...

Mike

*********************************

The Fallacy of Hard Tests

A great deal of fuss is often made about failing the bar exam. The

news a few weeks ago was that Governor Patakis daughter passed the

exam, but it is always mentioned that it was her second try.

Similarly, Kennedy, Jr. failed the New York bar exam twice,

before finally passing it on his third try.

As one who took several medical licensure and specialist exams, and

the Virginia bar exam, passing all, I might be inclined to pat myself

on the back, but my former background as a mathematician won't let me

do that. I do remember, however, some remarks from a noted orthopedic

surgeon about his own specialty exam: " It was a hellishly hard test,

and went on for hours, " he said, " but I'm really glad I passed the

first time I took it. Only about 35 percent who took it passed the

exam. "

He was describing, with only the slightest tinge of boastfulness, the

qualifying exam for specialists in orthopedic surgery. Passing the

exam entitled one to join the " college " of orthopedic surgeons, and

list oneself as specialist.

" Was it all multiple choice? " I asked. " And how did they grade it? " I

was thinking of my own exams. " Did they count only the right

answers.? "

When he said Yes to all the questions questions, I did not have the

heart to tell him what I knew as a mathematical certainty—that the

exam was, like most graduate medical exams, and large parts of legal

licensing bar exams in most states , virtually a complete fraud.

The reason these tests are fraudulent—and the harder they are, the

more they are fraudulent—is that for an extremely difficult test

graded in that way, guessing tends to count much more than knowledge.

A simple example will describe why this is the case. To illustrate

this, consider an extreme case.

Suppose you and I take a test, and you know twice as much as I do. For

simplicity (this is the extreme case) suppose the test consists of 100

questions, each True or False, and moreover (this is the key point),

let us agree that the test will be graded by only counting the number

right.

Naturally, both of us will guess at an answer for those questions that stump us.

Now suppose the test is very hard. As hard as it could be actually.

Suppose the test is so hard that I, with lesser knowledge, can only

answer one question based on actual knowledge. I answer that question,

and guess at the other 99. You, who know twice as much as I, can

answer two questions based on knowledge. So you guess at 98 answers.

As you can readily imagine, the odds of you getting a higher grade

than I are very slight. In fact, over 45 percent of the time, in

repeated trials, I would outscore you, even though my knowledge is

half that of yours.

I chose a True-False test for this example, but it doesn't make any

real difference were the test to be multiple choice with several

choices in each question. The only thing that makes a difference is

how hard is the test. Your advantage would grow substantially as the

test was weakened.

For further example, if the test was so easy, and you so well-versed

in the subject that you could get a perfect score, and I knew half as

much, I would answer 50 questions based on knowledge, and guess at 50.

In the long run, I would get half of those 50 correct, for a final

score of 75. So you get 100, and I get 75, on the average.

Were the test to be multiple choice, with four choices for each

question, and your knowledge was also 100 percent and mine half that,

I would then (guessing at 50) get a score of 50 + (1/4 times 50), or

62.5. on the average.

These extreme cases demonstrate the point, that truly hard multiple

choice tests, graded by counting only the number right and ignoring

guessing, are fraudulent.

[Guest guest]

They test more on how well the candidate CAN test.....but maybe that has changed

since *I* took them..<G>

Jules

Re: Re: The Fallacy of Hard Tests

Remember, most exams, whether the state bar exam or the NREMT exam, are

focused on knowledge, not upon judgment. We all too often test on what a

profession or occupation allows its members to do, not upon what they SHOULD do.

For example, just because you can push a med, does it mean you should?

-Wes Ogilvie

In a message dated 6/17/2007 5:12:58 PM Central Daylight Time,

lnmolino@... writes:

Very good points, every disbarred lawyer passed at least one Bar and every

MD, RN even hairdresser that was forced out of practice passed some test

somewhere and yet they failed to measure up over time. The question however,

remains as to what one would use to measure ability to hit the streets if

not a

test?

Louis N. Molino, Sr., CET

FF/NREMT-B/FSI/FF/N

Freelance Consultant/Trainer/Freelance CoFreelance Consultant/Trainer

_LNMolino@..._ (mailto:LNMolino@...)

(Cell Phone)

(IFW/TFW/FSS Office)

(IFW/TFW/FSS Fax)

" A Texan with a Jersey Attitude "

" Great minds discuss ideas; Average minds discuss events; Small minds

discuss people " Eleanor Roosevelt - US diplomat & reformer (1884 - 1962)

The comments contained in this E-mail are the opinions of the author and the

author alone. I in no way ever intend to speak for any person or

organization that I am in any way whatsoever involved or associated with

unless I

specifically state that I am doing so. Further this E-mail is intended only

for its

stated recipient and may contain private and or confidential materials

retransmission is strictly prohibited unless placed in the public domain by

the

original author.

******************************************<WBR>*********<WBR>*_http://www.aol.

http_ (http://www.aol.com./)