Protocol Testing

May 29, 2006

Forget about test banks and written tests. The only way to determine

competency to perform advanced sequences such as RSI is through scenario

testing.

Develop a realistic scenario and have the medics work through it as team

leader. There should be no level 1s and 2s for RSI. Either you're competent

to

do it or you're not.

Sit down and do a realistic task analysis showing exactly what items must be

performed. That means writing down every single and separate move that must

be done in performing the whole skill sequence.

Then go through and put an asterisk next to " absolutes " that must be done or

the person automatically fails.

There can be some skipped items, but no items should be ranked less than Yes

or NO. Did they do it or did they not.

If they did all the items marked absolute, then they ought to be able to

perform the skill sequence. If they did not do them all, then they need to

practice and retest until they can do them all.

This sort of testing is competency-based, and either you're competent or

you're not. But there should be no set limit on how many times the candidate

can

test. When competency level is achieved, then it is achieved. Period.

However, periodic competency retests should be done randomly, and often.

Build your didactic knowledge competencies into the scenario. For example,

it does no good for a medic to be able to figure out the number of gtts per

minute for a dopamine drip on paper. All that matters is for that medic to be

able to set up the dopamine drip and set the gtts or the pump correctly.

So make the candidates do everything in real time, and don't allow any

verbalization of anything. Require that all steps be done on the manikin and

done

in real time. Require that all drugs be drawn up, labeled, and arranged in

sequence before the scenario starts. Same for equipment. Require that the

bougie and Combitube/LMA/CobraPLA, et cetera be available at arms length.

This is the only way to know whether or not a medic can perform the skill.

Written tests tell you NOTHING other than that this person can guess the right

answer.

Use nothing less than a panel of three persons to evaluate these

performances. One of the panel ought to be the medical director, another the

EMS

education director, and another the QI person or an experienced EMS instructor

from

outside the organization. NOBODY from administration or management should

ever be allowed in the room.

This sort of testing costs money. It takes time. Management hates it

because they can't see it as making them a dime. But it might save them their

company if a lawsuit were to arise. Few managers have the sense to understand

that, but kudos to those who do. You know who you are, and the rest of you

don't read this list anyway.

Bottom line: Demand real-time performance and keep demanding it. Screw

written tests. They are worthless in the context that you're talking about.

Gene G.

> Here is a question for those of you that require testing of personnel on

> service protocols. My company wants to establish this type of testing, as

they

> want to develop a level 1 & 2 for each level. Ex: Level 2 paramedics would

> be allowed to do more advanced procedures, such as RSI, while the level 1

> paramedics would not be able to do this unless tested and cleared by the MD.

>

> Where do most of you come up with your testing questions? Is there a good

> test bank that can be bought, or does each service develop their own

> questions and skills testing for protocols?

>

> Your input would be appreciated.

>

> Wayne

>

> ---------------------------------

> Blab-away for as little as 1¢/min. Make PC-to-Phone Calls using Yahoo!

> Messenger with Voice.

>

May 29, 2006

Here is a question for those of you that require testing of personnel on service

protocols. My company wants to establish this type of testing, as they want to

develop a level 1 & 2 for each level. Ex: Level 2 paramedics would be allowed

to do more advanced procedures, such as RSI, while the level 1 paramedics would

not be able to do this unless tested and cleared by the MD.

Where do most of you come up with your testing questions? Is there a good

test bank that can be bought, or does each service develop their own questions

and skills testing for protocols?

Your input would be appreciated.

Wayne

---------------------------------

Blab-away for as little as 1¢/min. Make PC-to-Phone Calls using Yahoo!

Messenger with Voice.

May 29, 2006

Dr Yamada makes his own tests. Very difficult...Not just based on

the written protocols, which makes you think long and hard about each

question. That is where he wants your critical thinking to kick in.

D. Stone

>

> Here is a question for those of you that require testing of

personnel on service protocols. My company wants to establish this

type of testing, as they want to develop a level 1 & 2 for each

level. Ex: Level 2 paramedics would be allowed to do more advanced

procedures, such as RSI, while the level 1 paramedics would not be

able to do this unless tested and cleared by the MD.

>

> Where do most of you come up with your testing questions? Is

there a good test bank that can be bought, or does each service

develop their own questions and skills testing for protocols?

>

> Your input would be appreciated.

>

> Wayne

>

> ---------------------------------

> Blab-away for as little as 1¢/min. Make PC-to-Phone Calls using

Yahoo! Messenger with Voice.

>

May 30, 2006

Wayne,

Contact Dr. Racht at Austin/ County EMS. We just went through a system

wide retest for updated protocols.

Barry Sharp

--------------------------

Sent from my Wireless Handheld

Protocol Testing

Here is a question for those of you that require testing of personnel on

service protocols. My company wants to establish this type of testing, as

they want to develop a level 1 & 2 for each level. Ex: Level 2 paramedics

would be allowed to do more advanced procedures, such as RSI, while the

level 1 paramedics would not be able to do this unless tested and cleared by

the MD.

Where do most of you come up with your testing questions? Is there a good

test bank that can be bought, or does each service develop their own

questions and skills testing for protocols?

Your input would be appreciated.

Wayne

---------------------------------

Blab-away for as little as 1¢/min. Make PC-to-Phone Calls using Yahoo!

Messenger with Voice.

May 30, 2006

Just to play devil's advocate, what is the point of a provider's medical

director to authorize all of these various skills and procedures? One of the

biggest challenges I have seen in the past 15 years is ensuring that the medic

or EMT in question has the ability to correctly assess a patient. One can

have all of the skill knowledge in the world but it is useless (or dangerous)

if one's assessment skills are insufficient or inaccurate. Thoughts??

May 30, 2006

Wayne,

I agree with Gene, competency based evaluation is the only way to go.

My question to your agency is why create 2 different leveles? Why not expect

all your paramedics to be able to perform at an expected level? This sounds

like you are moving back to the EMT/Paramedic team approach, which while it

does have limited advantages, dual medics is much more desired. It seems

like it would confuse staffing problems, with differenct levels of

paramedics instead of just 1 standard.

Protocol Testing

Here is a question for those of you that require testing of personnel on

service protocols. My company wants to establish this type of testing, as

they want to develop a level 1 & 2 for each level. Ex: Level 2 paramedics

would be allowed to do more advanced procedures, such as RSI, while the

level 1 paramedics would not be able to do this unless tested and cleared by

the MD.

Where do most of you come up with your testing questions? Is there a good

test bank that can be bought, or does each service develop their own

questions and skills testing for protocols?

Your input would be appreciated.

Wayne

---------------------------------

Blab-away for as little as 1¢/min. Make PC-to-Phone Calls using Yahoo!

Messenger with Voice.

May 30, 2006

I agree, your either able to do the job or not. You don't need to

have a set of paramedics who can do one thing, and those who can't

because they weren't able to pass the test. What is the point of

having them around at all if they can't do all of the tasks they are

allowed or required to perform by their medical director.

You can be the best paramedic in the world when it comes to knowing

everything, but what good are you if you can't start an IV to give

the medications that you need? The same can be said for the cook

book medics who can do the skills but have no real clue as to when

and why they are doing them.

If we want to advance this profession, we need to stop making

exceptions for those who can't. It is okay to tell someone they

suck. We do it every year when people get fired, don't make a sports

team, etc. Why should we be " nice " and make exceptions in EMS?

>

> Forget about test banks and written tests. The only way to

determine

> competency to perform advanced sequences such as RSI is through

scenario testing.

>

> Develop a realistic scenario and have the medics work through it

as team

> leader. There should be no level 1s and 2s for RSI. Either

you're competent to

> do it or you're not.

>

> Sit down and do a realistic task analysis showing exactly what

items must be

> performed. That means writing down every single and separate

move that must

> be done in performing the whole skill sequence.

>

> Then go through and put an asterisk next to " absolutes " that must

be done or

> the person automatically fails.

>

> There can be some skipped items, but no items should be ranked

less than Yes

> or NO. Did they do it or did they not.

>

> If they did all the items marked absolute, then they ought to be

able to

> perform the skill sequence. If they did not do them all, then

they need to

> practice and retest until they can do them all.

>

> This sort of testing is competency-based, and either you're

competent or

> you're not. But there should be no set limit on how many times

the candidate can

> test. When competency level is achieved, then it is achieved.

Period.

> However, periodic competency retests should be done randomly, and

often.

>

> Build your didactic knowledge competencies into the scenario.

For example,

> it does no good for a medic to be able to figure out the number of

gtts per

> minute for a dopamine drip on paper. All that matters is for

that medic to be

> able to set up the dopamine drip and set the gtts or the pump

correctly.

>

> So make the candidates do everything in real time, and don't allow

any

> verbalization of anything. Require that all steps be done on the

manikin and done

> in real time. Require that all drugs be drawn up, labeled, and

arranged in

> sequence before the scenario starts. Same for equipment.

Require that the

> bougie and Combitube/LMA/CobraPLA, et cetera be available at arms

length.

>

> This is the only way to know whether or not a medic can perform

the skill.

> Written tests tell you NOTHING other than that this person can

guess the right

> answer.

>

> Use nothing less than a panel of three persons to evaluate these

> performances. One of the panel ought to be the medical director,

another the EMS

> education director, and another the QI person or an experienced

EMS instructor from

> outside the organization. NOBODY from administration or

management should

> ever be allowed in the room.

>

> This sort of testing costs money. It takes time. Management

hates it

> because they can't see it as making them a dime. But it might

save them their

> company if a lawsuit were to arise. Few managers have the sense

to understand

> that, but kudos to those who do. You know who you are, and the

rest of you

> don't read this list anyway.

>

> Bottom line: Demand real-time performance and keep demanding

it. Screw

> written tests. They are worthless in the context that you're

talking about.

>

> Gene G.

>

> > Here is a question for those of you that require testing of

personnel on

> > service protocols. My company wants to establish this type of

testing, as they

> > want to develop a level 1 & 2 for each level. Ex: Level 2

paramedics would

> > be allowed to do more advanced procedures, such as RSI, while

the level 1

> > paramedics would not be able to do this unless tested and

cleared by the MD.

> >

> > Where do most of you come up with your testing questions? Is

there a good

> > test bank that can be bought, or does each service develop their

own

> > questions and skills testing for protocols?

> >

> > Your input would be appreciated.

> >

> > Wayne

> >

> > ---------------------------------

> > Blab-away for as little as 1¢/min. Make PC-to-Phone Calls using

Yahoo!

> > Messenger with Voice.

> >

May 30, 2006

Our dept. uses written & practical evaluations. We do this once a

year. We also have someone from Medical Control (maybe nurse or

Dr.) that will ride out periodically to evaluate us. I think a good

bit of evaluation can be done by reviewing reports.

As for having different levels of Medics, I don't think it is a bad

idea. For example, I work for a fire dept. and we have approx. 80

paramedics. The drivers and officers are given the option of

dropping their paramedic. They do not get rid of their cert through

TDH, but they are EMTs under medical direction. For our Batallion

Chiefs, it is mandatory. If a service uses RSI, I think it is

reasonable to not make the drivers and officers responsible for that

skill. One advantage that we have is that there are no EMTs except

for a few (5 or so) that are drivers and officers.

>

> Wayne,

>

> I agree with Gene, competency based evaluation is the only way to

go.

>

> My question to your agency is why create 2 different leveles? Why

not expect

> all your paramedics to be able to perform at an expected level?

This sounds

> like you are moving back to the EMT/Paramedic team approach, which

while it

> does have limited advantages, dual medics is much more desired. It

seems

> like it would confuse staffing problems, with differenct levels of

> paramedics instead of just 1 standard.

>

> Protocol Testing

>

> Here is a question for those of you that require testing of

personnel on

> service protocols. My company wants to establish this type of

testing, as

> they want to develop a level 1 & 2 for each level. Ex: Level 2

paramedics

> would be allowed to do more advanced procedures, such as RSI,

while the

> level 1 paramedics would not be able to do this unless tested and

cleared by

> the MD.

>

> Where do most of you come up with your testing questions? Is

there a good

> test bank that can be bought, or does each service develop their

own

> questions and skills testing for protocols?

>

> Your input would be appreciated.

>

> Wayne

>

> ---------------------------------

> Blab-away for as little as 1¢/min. Make PC-to-Phone Calls using

Yahoo!

> Messenger with Voice.

>

May 30, 2006

,

Firat, the levels of EMTs where in place before I started there, and it was

actually put in there for the LP level, which is now what is going to be called

the Level 2 Paramedic, or the more advanced paramedic (CCEMTP type thinking).

As was stated, you either can or you can't, and there are some that are in the

middle of the road, for lack of better terms. Most want to do all they can,

while 1-2 just want to be able to do what they need to, this is where the

testing comes into play.

In the beginning, EMT's were working at the EMT-I level by starting IV's and

giving D50, which was all approved by the MD. Now, all of this is going away,

except for the EMT basic skills that are taught in school, nebs, epi pen, nitro,

etc. Now, they can be tested on these within the service and be approved by the

MD to give these.

Most of these people are rural fire-based EMTs and Paramedics working

part-time for this service, but have no additional training on more advanced

paramedic procedures like RSI, advanced cardiac drips, etc., this is also part

of the reason for level 1 & 2 in the paramedic setting.

What Gene stated does sound like a good plan, and they are willing to do what

is needed to test these people for evaluation of skills and knowledge. They

want good quality people and not just warm bodies like many providers do.

Anyone have more input, it would be appreciated.

Wayne

Wiseman wrote:

Wayne,

I agree with Gene, competency based evaluation is the only way to go.

My question to your agency is why create 2 different leveles? Why not expect

all your paramedics to be able to perform at an expected level? This sounds

like you are moving back to the EMT/Paramedic team approach, which while it

does have limited advantages, dual medics is much more desired. It seems

like it would confuse staffing problems, with differenct levels of

paramedics instead of just 1 standard.

Protocol Testing

Here is a question for those of you that require testing of personnel on

service protocols. My company wants to establish this type of testing, as

they want to develop a level 1 & 2 for each level. Ex: Level 2 paramedics

would be allowed to do more advanced procedures, such as RSI, while the

level 1 paramedics would not be able to do this unless tested and cleared by

the MD.

Where do most of you come up with your testing questions? Is there a good

test bank that can be bought, or does each service develop their own

questions and skills testing for protocols?

Your input would be appreciated.

Wayne

---------------------------------

Blab-away for as little as 1¢/min. Make PC-to-Phone Calls using Yahoo!

Messenger with Voice.

May 30, 2006

Gene, I do like what you say!

I have no problem with protocol testing as long as it's not done by a

" PARAGOD " , someone who belittles anyone, or someone who sits behind a desk who

can't even perform out in the field, and scrutinizes every little thing as if he

were a lawyer.

In other words---------- Let a REAL to life individual who knows what it's

like out in the field do the testing

wegandy1938@... wrote: