RICHARD PETTY:  Now I would 
like to introduce our presenter.

Many of you already know him from 
the wonderful work he has done with

the on-location trainings, 
the webinars,

the work he is doing in outcome 
measures with centers with Bob

and the centers for 
independent living,

and we will say more 
about that later in the day.

We have a completely separate session 
that updates you on what is going on

with centers and outcome measures, 
but Mike is someone who has been

involved in performance 
improvement, evaluation.

He has brought with his 
colleagues performance

improvement and evaluation 
and outcome measures to

many parts of the federal government 
over the past few decades.

He is well respected, 
if you read the literature.

He's someone that his peers 
acknowledge as for the role

that he has played in 
this important work.

He has done several seminars and 
trainings for the IL-NET and all

of you have given him high marks for 
clarity and being able to help all

of us have a much clearer 
understanding about outcome

measures and things that we 
can do to improve performance.

And right now he is getting ready to 
go to Thailand to work with a group

that's sponsored by USAID, they are 
federal officials in the far east.

He will be training them on 
performance improvement,

results oriented management and that 
certainly is out -- excuse me.

Evaluation is certainly just 
a central part of that.

He's part of the JAO advisory 
committee, he's a member of that

committee that works on performance 
audits, which again is evaluation.

So he is someone who is very 
much a part of what's going 
on in the federal,

at the federal level on the very 
topics that we are going be talking 
about today.

He has just concluded a term as a 
member of the board of directors of 
the American Evaluation Association.

And our training today builds 
on what we did last year

which was preparing your 
outcomes-focused SPIL.

This training this year is another 
key responsibility of Statewide 
Independent Living Councils.

It addresses one of the key 
requirements and obligations that

councils have and that is evaluating 
your outcomes focused SPIL.

And we are going to all learn 
a great deal today and it is

my great pleasure to introduce our 
presenter, Mike Hendricks.

Mike

[applause]

MIKE HENDRICKS:  How is that?
Is that good?

Can you hear me?
Nice.
Okay.

Well, thank you, Richard, 
nice was also the introduction,

I appreciate that very much.

I think the thing Richard left 
out was that today is a 
monumental day in my life.

Today is January 10th, 2011, 
the day the University of Oregon

is going to win the BCS 
championship against Auburn.

I am from Oregon.
I am wearing Oregon green.

And I have got my fingers really 
crossed because Auburn is an 
incredibly tough team.

It is like I say to my friends, 
my heart is with Oregon,

if I had to bet my 
wallet I am not so sure.

If you want to continue today's 
discussion I will be in the bar

from 8:00 to midnight either having 
a celebratory beer or drowning my 
sorrows, one or the other.

So I am really happy to be here.

How many of you were subjected 
to some time with me last year 
in Las Vegas?

Oh you poor souls, 
you are back again.
Ohhh, too bad.

Well the rest of you were lucky, 
this is your first time.

But I have had a chance to 
work with this group before,

and really liked it a lot.

For a couple of reasons,
one I think their heart, 
I think your heart

I should say I think your heart I 
think is really in the right place.

You're trying to do really good work, 
and that makes me really highly

motivated to try to help anyway I can.

So that's good.

The second is how quickly you 
pick up the kinds of things 
we try to share with you.

I mean you are a very sharp 
group and you pick up the

concepts and you 
work with them.

You know from last year 
that you will be working.

This isn't a lecture, 
this is a workshop,

where you will be working 
and you know that.

So it is always really 
rewarding to be here.

So anyway, thanks for that.

All right.

Let's talk about what we are going 
to do if we can because we are

going to be together, by the way, 
for you newbies we are going

to be together all day today 
and tomorrow morning.

This is sort of the skill 
building part of the 
SILC Congress, if you will.

So, it says that we will be 
talking about evaluating 
your outcomes-focused SPIL.

Let's just think about that because I 
don't know if you know this,

but just January 4th, six days ago, 
President Obama signed a new piece

of legislation that, 
and I am sure that Tim and RSA

are very familiar with this, 
signed a new piece of legislation

that basically enhances even more the 
importance of results and outcomes

and performance improvement and that 
sorts of things that we have all been

talking about for five or 
six years now right,

it enhances it even more 
within the federal government.

So if you thought, I don't think any 
of us did, but if you thought that

outcomes was kind of like a passing 
fad or the flavor of the month and

we weren't going be 
caring about it anymore,

this piece of legislation says no.

And if you think that the republicans 
coming in might do away with that

piece of legislation because it was 
from, you know, the last congress,

I would suggest to you, to go back and 
look at the history of which

congresses have passed that kind of 
legislation in the federal government

and it tends to be the republicans 
who are very interested in kind 
of hard-nosed management.

So I think if anything, this new 
congress is going to re-enforce that 
and perhaps push it further.

I don't know.

But we are still in the era of 
outcomes and we are going get 
deeper into it, I think.

So, the fact that it is an 
outcomes-focused SPIL is a good thing.

And now, I will say with Tim in 
the room or Tim not in the room,

I have been known in the past to 
be a little critical of some of

the things RSA has either told 
us or not told us about this.

But I have also been very willing 
to give praise where credit is due.

And I think RSA has done a very 
good thing by telling us that our 
SPILs need to be outcomes-focused.

I think that was excellent.

And I think those of you who did 
it this year, I hope got that 
kind of feedback from them.

So the fact that they have pushed 
us to do an outcomes-focused SPIL 
is a really good thing.

Now, they're pushing us to take 
the next step and I think 
that's a good thing too.

They're saying don't just sit 
back for three years, you know,

and then see 
how you did.

No, no, no, the idea 
is to keep watching.

The idea is to stay on top of it.

The idea is to be evaluating it 
and keep looking at how you are doing 
to find ways to make it better.

So I think that's another thing 
that they're pushing us to do.

We don't have any choice, but it is 
good, that we don't have any choice.

So I am a big fan of what 
we are going to be doing now,

the next day and a half, 
is thinking now,

that we have done our outcomes-focused 
SPIL, how are we going to evaluate it.

And let's take a little poll.

How many of you love being evaluated?

Yeah, that's kind of…

Oh we have one person.

Good.

Two people.

I am hoping that by the end of 
tomorrow you might feel a little

differently about it because my 
definition of evaluation is maybe 
not the same you've got.

Your definition of evaluation might 
be like these new x-ray machines

at the airport that I personally 
really hate myself,

where you go like that and 
your picture is flashed,

your nude picture is 
flashed to some room.

Has anybody had to do that yet?

I have.  You haven't.

I haven't had to do that.

It is not much fun I don't think.

That's one definition 
of evaluation; right?

There's another 
definition of evaluation.

And that's holding up a mirror, 
do you ever hold up a mirror to

yourself for any reason whatsoever?

That's kind of handy, isn't it.

Holding up a mirror is also
a form of evaluation.

So if you think of evaluation as the 
x-ray machine that is going to expose

every wart to strangers then 
you are not going to like what

I am going to 
be talking about.

But if you think of 
evaluation as a mirror,

that you can use for your own 
benefit and your own value to see

how things are going, and to get even 
better then you are going to like it,

because that's the good kind of 
evaluation in my mind.

I think that's the kind of 
evaluation RSA wants us to do.

If not I will argue with them.

Because that's what 
we should be doing.

So, and notice one 
more thing on this, 
I think.

It says facilitated by…

Richard was kind to 
use the word trainer,

but I don't think you can 
teach or train adults anything.

It is just my philosophy.

You just can't do it.

You know, Andi is shaking her head.

I should single her out perhaps, 
you can't.

Adults can learn, and my job is not to 
train you in any way, shape or form

because I can't do that, but my job 
is to hopefully create the conditions

under which you can learn something 
if you are motivated too.

I am going to work my butt off to try 
to create those conditions for you

and let's hope we are all on the same 
page and we can learn a lot together.

Before we even go to the next 
page I have got to say one thing.

I could not have done this by myself 
and I did not do this by myself.

There was a team of us who 
put this training together,

I was a part of it, yes.

Richard Petty from ILRU, Darrell Jones 
from ILRU, Tim Fuchs from NCIL.

But also we had a three person 
advisory group from your ranks

because we wanted to make sure we 
weren't totally off wall.

So, Brad Williams from New York 
put your hand up, please, 
was a part of it.

Pat Puckett from Georgia is.
She's out of the room right now.

Ann McDaniel from West Virginia.
Those three people were great.

[applause]

They took time out of the busy 
schedule that you all have to talk

with us on phone calls, 
to review materials,

to give feed back and talk with us 
again so we really appreciate that

and thanks all three for that.
Okay.

Our time together.

Like I said, we are going be 
together today and tomorrow morning

and by the way, let me say this, you 
have in Tim Fuchs wonderful packet,

you have a folder that looks just 
like this and inside that folder,

is may I reach for one?

Inside that folder is this which 
is exactly every single slide

that I am going to show you.

So if you feel that you can't see 
what's up on the screen you've got

the hand out right here in front 
of you that has everything,

take away for you.

So today and tomorrow, we are going 
to cover, we are going to have three

different sessions together of 
three hours each.

And each session will 
cover a different topic.

Now they all fit together, okay, 
but they are all going to cover 
a different topic.

But each session is going 
to use the exact same format.

We tried this last year in Las Vegas, 
and it seemed like it worked.

The first part is I will kind of go 
over stuff and explain some concepts

and procedures and give some examples.

Then we will do q&a.

Then we will take a break.

When we come back from the break, 
then it is your turn.

This is that working part I say of the 
workshop, you know.

This is when you roll up your sleeves,

although, maybe you don't 
roll them up today.

Anyway, you roll up your sleeves 
and you do a small group exercise,

and then together, we will discuss 
what you did, and we will learn

about it and we will refine it.

Now, overview, big picture.

You are saying evaluating our SPIL.

Oh gosh, that sounds big and hard.

Here is exactly what we 
are going to propose to you.

We are going to think that it is 
really a good idea to think of 
a three legged stool.

You know a three-legged stool is 
actually a very stable structure,

quite stable, if each leg 
is solid by itself, right.

We will suggest your first leg could 
be that it is important to implement,

to evaluate the implementation 
of SPIL activities.

Emphasis on implementation, 
we will go over each of these 
obviously more.

But this is the overview.

The second leg we think 
ought to be your progress

toward achieving SPIL 
objectives.

Okay, different thing from 
implementation isn't it.

The third thing-consumer satisfaction.

You can see those are three very 
different things but if you put

them all together they make a very 
nice strong three legged stool.

We think each is important 
and together they create a 
very stable foundation.

All right.

Evaluating.

You ain't got no choice.

Let's be honest about it, okay.

You will do something.

RSA's making us and they should.

But please don't just think of 
this as a requirement that

RSA is making us do.

Think of it as an opportunity 
to turn that requirement into

something useful for you.

And I really think we can.

See, I believe 
evaluation is 
important.

And let me give you an example.

Okay, if somebody said, 
well Andi is sitting there.

So I say Andi, you're got to drive 
back home to Alaska,

you're going to start here in Atlanta,

and you are going drive 
back home to Alaska and

by the way your car is 
going to have a gps in it.

Well, that's my question to you Andi.

Would you turn that gps on, 
or would you leave it off?

Oh, I think you are a smart woman.

I think we all should 
turn that gps on.

What does that have to do evaluation?

I think it is exactly the same idea.

You are starting from, 
to run a program.

You are starting right here, 
for three years you are going

to be running this program, okay.

You can leave that gps off and 
you won't know where you are,

you won't know how you are doing,

you won't know if there's 
a better direction to go.

You won't know if there's a nice 
short cut you have just missed.

Or you can turn that gps 
on and see where you are,

and try to get some feedback about 
your progress as you get along

there to see if maybe you 
want to alter your course.

That's the whole point.

Turn that darn gps on, 
turn that evaluation on.

It is important.

You can use it to focus on 
improvement, not reporting.

Now, if you use it to focus on 
reporting, I am sorry to focus

on improvement you will automatically 
generate the information RSA wants.

Okay.

You will automatically generate 
the information that RSA wants.

So turn it on, use it for yourself, 
tell RSA what you learned.

Let me warn you, 
if you do it the other way around,

if you rig your system to produce 
something that looks good to RSA,

you will not get the stuff you 
need to improve your program.

So it doesn't work both ways.

Huh/uh.

If you focus on accountability and 
reporting you won't necessarily

get all get the information 
for improvement.

If you focus on improvement, 
you will get the information 
for reporting.

So focus on improvement, 
not reporting.

Another guiding principle 
is that evaluations will 
vary from state to state.

You will not do things exactly as we 
suggest over the next day and a half.

And that's fine.

But listen please, 
take what works for you

and adapt, you know.

We do think that certain things 
are essential and we will point

that out as we go along.

Another guiding principle that you 
will see is we think evaluation

is a very collaborative activity.

This is not something you do, 
the three of you in the SILC 
offices somewhere.

This is a collaborative 
process within your state.

All right.

Now, this is an important table.

I would ask you to probably maybe open 
up your hand out and look at this if

you can because we created 
this just for this workshop.

It has not existed anywhere else.

You have not seen this before.

You will probably look at it and 
say I can improve it which is fine.

That's what we want.

But let's talk about it and let you 
know where we are coming from.

So this is called-who 
should do what.

And this is important because 
it is a collaborative activity.

Evaluation is, and that means 
different people are going 
to do different things.

Well, who's going to do what.

Very, very important.

Lets go down and we will show 
you what we think, or at least

some ideas we are throwing 
out for your consideration.

How is that.

What you see here, 
what we have on the left,

we have these tasks needed 
to evaluate a state SPIL.

We tried to sort of 
think very carefully,

what exactly are you 
going to do have to do.

Then across the right we 
have put which IL partner

might do which of these tasks.

So you see, we have got different 
partners, we have you guys,

of course, the SILC but also the 
DSU or the DSUs in your state maybe.

There's the CILs, those are 
definitely, those are

definitely three people that have 
to be involved in this; right.

And maybe, depending on how 
you decide to do things there 
may be a contractor.

So we left a column for that.

So let's go down these different 
tasks and we will just give you 
our first take of what we think.

First of all, first task, 
convene all IL stake holders

interested in evaluating the SPIL.

Like it or not we 
think that's your job.

We think it is the SILCs 
job to convene that group.

Second one.

Plan what will be done to 
evaluate implementation,

progress on objectives,

and consumer satisfaction.

That's the three legged stool; 
right, the three parts of it.

Plan what will be done.

We do not think this is something 
you do by yourself by any means.

We think it is something you do, 
the DSU and the CILs do.

That has got to be, we think, 
very collaborative.

Coordinate the information 
gathering efforts.

There is going to be some 
kind of information gathering.

We think that's back on you.

We think that's back on you – 
you may disagree,  that that's fine.

But we think that's back on 
you to coordinate and make 
sure it all happens.

Because you are the ones required, 
you know, by RSA very officially

required to make sure 
the evaluation happens.

All right.

Gather the needed information.

We don't know.

You can see we have question 
marks in here.

That's going to vary by state, 
not sure.

It will be interesting in a year or so 
won't it to do a little survey to see

who is gathering information 
for all of these evaluations,

cause I think right now we don't 
know do we, none of us know.

Then after you have got it,

compile and analyze 
the needed information.

Again we don't know.

Notice this is how the 
contractor column kicks in,

cause the contractor might 
be part of gathering it,

might be part of compiling it.

Discuss and interpret the findings.

You know there are people who say 
that data speak for themselves.

No, ain't true.

Data don't speak for themselves.

You have to interpret them.

You have to talk about them and 
figure out what they mean for you.

That's what we mean by discuss 
and interpret the findings.

And we definitely think that you, 
the DSU and the CILs have to be

involved in that and maybe the 
contractor, if you had a contractor.

Okay, decide what steps to take 
to improve any weaknesses found 
in the IL system.

Notice that word improve?

All three of you again.

Again, that's not something you can 
do by yourself or the DSU or the CILs.

All of you together.

Implement those improvements 
to the IL system.

Don't know, we don't know what they're 
going to be so how can we say who's 
supposed to implement it.

We don't know.

And then finally, 
amend the SPIL if appropriate.

That's all three of you again.

Let me just take a stop right here 
because this is something brand new.

Let's take a couple of minutes to see 
if anybody has anything they want

to say about what we 
have gotten to so far.

So far, we are just getting into it, 
but you know, we have given you,

first of all we said you 
have got to do evaluation.

We said we think you have got to do 
three parts of it for the three legged

stool and we have said this is kind of 
how we envision sort of the different

people collaboratively working on it.

Not a big pause but a little 
pause in case something is on 
somebody's mind.

Any hand up there yet?

That's fine.

I just wanted to give you the chance.

Let's go on.

Let me go back because 
it is really important.

I should have said 
something more important.

Here is that overview of the three 
legged stool; right.

Implementation of SPIL objectives, 
notice I didn't say implementation 
of the SPIL.

There's a big difference.

This is an important thing,

I am glad I remembered 
to point it out.

The implementation of the SPIL 
objectives, progress towards

achieving SPIL objectives, okay,

not progress toward 
achieving the SPIL.

Now, you can -- those newbies
among you may be wondering what

the heck is he talking about.

Let me show you because this is 
important, we think.

And I think RSA, by the way, 
is on the right wave length

on this one because this 
is their language too.

If you were with us in Las Vegas or 
even yesterday when we talked about

the logic model, you remember 
we talked about different 
levels of outcomes.

Some outcomes sort of 
happen right away logically,

some have to be developed 
and worked up to.

We have -- we have worked 
with RSA and we are in 
agreement on this language.

We were in agreement 
on this language a year ago.

That when RSA talks about mission,

they're talking about the 
top of the logic model.

See the example mission up here.

When RSA is talking about goals, 
they're talking about a middle

section of the logic model and when 
RSA is talking about objectives,

they're talking about the 
lowest level of outcomes.

Let me say that again.

When, and Tim by the way if I 
have any of this wrong or if

something has changed in 
the last year, stop me.

But when -- unless something has 
happened I don't know about.

When RSA talks about objectives,

they're talking about the lowest 
level of the logic model.

Why is that important?

That's important because 
as I just said,

we think you should focus your 
SPILs evaluation on the objectives.

I am going to walk up 
here and actually point.

In other words, we think you 
should evaluate things in here,

not things above it.

It is not the things 
above it aren't important.

Don't get me wrong.

They're very important.

But it is, they're going to flow out,

they're going to flow from the 
nice achievements you do in 
achieving your objectives.

Okay.

We will be working with that so 
if that isn't fully clear to you

don't worry, you will have plenty 
of time to wrestle with it.

But the key thing is here, 
all day long today, especially,

when we talk about evaluating 
your SPIL objectives,

we are specifically 
saying this initial,

lowest level of the logic model.

Now, again you remember from 
yesterday, again just to remind

ourselves what the heck this is,

remember what we said yesterday 
was that objective level is the

level right above the activities.

So if you are doing various 
things to achieve an objective,

the activities should just 
flow and create that objective 
happening right away.

That will become important 
in just a second.

Okay.

Now, we looked at the SPILs.

And we have seen some good objectives.

We have seen some bad objectives, 
you know I will be honest about

RSA and I will be 
honest about ourselves,

we have seen some awful objectives.

I will just say that right out to you 
and please don't be offended but I'm

an outsider and my job is to tell 
you my opinion and truthfully so.

There are some terrible 
objectives out there,

but there are some good objectives.

Here are some of them that we saw.

Formerly underserved 
populations are served.

PWDs have increased employment 
opportunities.

Policy makers financially 
support the IL network.

Persons with disabilities 
register to vote.

IL network key partners collaborate to 
provide delivery of services.

PWDs, which is our way of saving 
space, can access transportation.

There are good objectives out there.

But each of these objectives 
is unique, okay.

Each objective is unique.

It tries to achieve something unique.

Let's go back.

Each one of these tries to achieve 
something different, doesn't it?

It is trying to achieve something 
different.

It needs to be implemented uniquely, 
doesn't it?

So for instance, if you wanted to 
help the formerly underserved

populations be served, 
you would do different

activities wouldn't you than 
if you wanted to get persons

to register to vote.

There are different things 
you would do; right?

Well, since they achieve something 
differently and they're implemented

differently, guess what,

they're obviously going to have 
to be evaluated uniquely.

So you are going to have 
to do your evaluation

objective by objective by objective,

not the whole SPIL at once.

There's -- you can't grasp 
the whole SPIL at once.

It is just too much Jello.

You've got to grasp it 
objective by objective.

So we say here, think in terms of 
individual SPIL objectives, 
not your SPIL as a whole.

That's really important.

All right.

So I said that there – 
let me stop there.

Any questions yet?

Yes, we do have.

Hang on a second because they are 
taping it so they want to get every

question into the microphone as well.

AUDIENCE MEMBER: I'm a little 
confused.

Okay.

MIKE HENDRICKS: First morning, so no 
problem.

AUDIENCE MEMBER: Can you explain 
the difference between an 
objective and a goal?

MIKE HENDRICKS: I will do my best.

It is simply level of the logic model.

Each one, so the question you 
obviously heard was the difference 
between an objective and a goal.

Each one is a desired outcome.

No doubt about it.

So in that way they're identical.

So for instance, this objective 
which just happens to be,

what a coincidence, 
formerly underserved 
populations are served.

That is definitely a desired outcome.

And the goal right above it, 
the IL system is expanding.

That's also very definitely 
a desired outcome, 
no doubt about it.

And even the mission, 
PWDs in our state live

independently and participant 
fully in their community.

That also is a desired outcome.

So in that aspect, 
I can see why you are confused.

They're all desired outcomes, 
aren't they.

It is all about the levels.

And that's where this comes in.

Your activities down here, okay.

Say you do a bunch of really good 
work, just wonderful, wonderful, 
wonderful work.

The first thing that's going to 
achieve is not going be that thing 
at the very top, you know.

People in your state live 
independently and participate 
fully in their community.

You have got to achieve some other 
stuff before you can get there.

That is the best way to put it.

There's like interim achievements 
you have to make,

interim outcomes you have to get done,

before you can get up there.

And that's what these are.

These are the interim steps that have 
to happen on the way up there.

So, and again, it is simply jargon, 
but it is useful jargon.

It is simply jargon to label 
these different levels with 
different jargons,

and that is what it is, just  to let 
us know what we are talking about.

So it is a great question.

I suppose instead of using the word 
objective we could just say lowest

level of your logic model or first 
outcomes you are going to achieve

or something like that.

But RSA doesn't use that, 
because it is just saves a

lot of typing to just say objective, 
you know and know what we mean.

So great question.

Does that help?

Okay.

And listen by the way, I am glad you 
asked that.

This isn't about me for the next day 
and a half.

This is about you.

So if you don't understand something, 
then I am not being clear enough,

so don't ever hesitate to ask a question.

You are the reason 
we are in this room.

So any question you've got, 
put your hand up.

Okay.

Anymore right now?

Okay.

Let's go on then.

Let me check my time.

I don't want to keep 
us past our break.

Time to start talking about 
the three-legged stool.

That was the overview, 
that was kind of 
where we are going.

And some of that stuff you 
will hear again for sure.

But let's start talking about the 
three legged stool.

Here is the first leg.

The first leg of the stool is 
evaluating implementation

and this is not quite as obvious 
as it seems as you will see.

First of all, it means we have 
got to define what we mean by
implementation, doesn't it.

We may all have different definitions 
of what that word even means.

Well here is one, maybe not the 
best, we will see what you think.

We are defining implementation as 
making happen all the things that

need to happen in order to achieve 
all of the different objectives,

notice the word objectives 
not goal, not mission,

objectives, in your SPIL.

Say it again.

'making happen all the things that need 
to happen in order to achieve all of the 
different objectives in your SPIL.'

So another way to put it is to say 
doing whatever is needed to achieve 
that objective.

Or I used to live in India for a few 
years, they would say doing the 
needful to achieve this.

But you know, I think we all 
basically know what we mean 
by implementation, okay.

Now, again, I will use my 
parents Kentucky jargon, 
you ain't got no choice.

RSA says you will do this.

Here is where it says it.

Section 364.21 G of something 
but I am not exactly sure what
it is, I am sure you know what it is.

Here is the quote, the SILC shall, 
not the SILC might or the SILC

if it wants to, or the SILC might 
consider, no, no, the SILC

shall monitor, review and 
evaluate the implementation 
of the state plan, okay.

So this is going to be a 
leg in your stool.

One way or another.

It is going to be in there.

And what we would suggest, 
please, is to take your SPIL

and ask yourself how well 
have you implemented things

for each of your objectives.

Remember, it is that objective by 
objective by objective thing okay.

So, you take it, you take it in bite 
size chunks of objectives,

and you ask for each objective, 
how well has that been

implemented, how well that has 
one been implemented,

how well has that one been 
implemented, okay.

Well, but what do we mean 
by how well you have 
implemented things?

Okay.

We wrestled with this.

Our task force and our three 
advisors, we all wrestled with this.

Because, you can tackle this 
in a bunch of different ways.

I have a good friend who is like the 
expert on evaluating implementations,

she told me she just published 
an article in the American Journal 
of Public Health.

Which is a pretty high 
powered journal,

you know you gotta be pretty 
good to be published in there.

I called her and said what do 
we do here because there's

so much we could 
do by implementation.

So we got some advice
from her and our group.

And here is what we came up with.

We could list hundreds of 
aspects of implementation.

I mean, if you start thinking 
well I have to sign this contract,

I have to recruite these people 
and I gotta make sure there's

gas in the van and I mean, 
you know, my gosh,

the list would be huge.

But that's just too much.

That's overwhelming.

We suggest you focus on five key 
aspects, resources, staff, 
participants, activities, management.

That's not just a list we sort of 
rattled off the top of our head.

We spent a lot of time looking 
at stuff and talking to people
and thinking about it.

You can improve it maybe, 
which is fine, but as a

starting point, we think 
those five are not bad.

Here is what we mean by the five.

Resources.

The resources allocated?

Are they the right amount and 
kinds of money, materials, 
facilities et cetera?

I am a very bad baker but I like to 
bake stuff and sort of what I think

about this is like all of the stuff 
you have to get on the counter

before you start baking, 
at least that is what I do.

I put it all on the counter at once.

So it is like all of the resources, 
are they all out there or not?

The second one, appropriate 
staff in place, staff is an

important part of 
implementation; right.

Like the right numbers and 
the credentials and the training 
and the experience.

That's an important part 
of implementation.

Well, none of those do any 
good unless you have any

people right, 
any participants.

So, do you have the relevant number 
and type of participants identified?

Have you recruited them, 
have you  motivated them?

Are the participants there, 
in other words?

How about the activities?

The activities underway?

Are there sufficient types, 
numbers, intensity?

I do a fair amount, as Richard 
mentioned, I do some of my

work overseas and one of 
the big problems with

development evaluation is 
that there ain't no there there.

I mean there are classic evaluations 
of world bank education programs

that they would send people 
out to villages and the 
schools just ain't been built.

They're just not there.

You have to make sure there's 
activities underway.

That's an important part of 
implementation.

And not to be forgotten, 
although you might,

if you don't sort of 
consciously think about
it is management.

How about the 
management 
infrastructure.

You know, is there an effective 
organization, communication, 
et cetera stuff like that.

So again we have no problem 
with you improving this list.

In fact the more you wrestle with 
this list and the more you can

maybe come up with 
something better, 
we all win.

So don't hesitate 
to try to improve it.

But as a starting point, 
we think these five are 
not too bad.

The resources, the staff, 
the participants,

the activities, 
the management.

Okay, so where are we so far?

Well, so far I think we are 
saying you can't chew off

the whole SPIL at once, 
you have to take objective 
by objective.

And we are saying well what 
do you do for each objective.

We are saying, well ask 
about these five things.

If you are with me so far, let's see 
if you are with me on the next page.

This is what we think you ought to do.

Evaluate each of those 
aspects separately.

Let's take an example here.

Down the left is those 
five -- is or are, are.

Down the left are those five key 
aspects of implementation.

The resources, staff, participants, 
activities, management.

And our suggestion, and if Tim 
has any comments on this

now or later I'll be happy
to know if RSA likes this.

I think they do.

Is how well is it implemented 
and we thought about, you know

should we do a 0 to 100 scale, 
should we do a 0 to 10 scale?

I mean there's lots of things 
you can put in there; right.

And again with talking to experts 
and consulting and talking with

our advisory group, 
we decided that what

made most sense for this 
situation was to simply give

a score of high, medium, 
low, or none yet.

So just hypothetically here what we 
have done, we have looked about

this objective, I don't know 
what objective it is, some

objective we have looked 
at it and we said well

resources high, yeah, 
they got the resources 
in place, absolutely.

Staff high.

Good people, lots of 
them in place, excellent.

Thumbs up.

Participants low, they do not 
have the participants recruited

or maybe even 
identified, who knows.

Activities none yet.

Haven't started yet.

This is good to know the distinction, 
to see why we separate into five

things, cause implementation 
is not just one thing.

It has got different parts of it.

And management, 
medium, we will say.

I don't know why we said 
medium, but let's say we 
said medium, okay.

Okay.

So we have got each aspect 
evaluated separately, just like this.

So right away we are ahead of the ball 
game because we are able to see

the different parts but we 
think - see what you think.

We think we ought to take next 
step and then kind of look back,

let me go back here, 
kind of look over this

whole picture and kind 
of conclude, what do

we think is the overall 
implementation.

Now you may not want 
to take this next step.

You may say, you know it is 
more useful to me to have 
the level of detail of the five.

That's fine.

But we think it is useful 
to have that five plus 
the next step, the next step

where you, see at the bottom where 
it says overall implementation.

Now, this is going to be kind of 
subjective, isn't it?

Because you are going to have to look 
at five different things and come up 
with an overall.

But here is our thinking.

If you -- it is kind of like 
a so-what in this case.

So what if we have all the resources 
and all the staff because look how

high those are and so what if the 
management is medium.

You know, there are very few 
participants and nothing has

happened yet, the activities 
haven't happened yet.

So if you really want to be honest 
with yourself and say how is the

overall implementation, 
you would have to say 
that so far it is low.

You know, I mean I think that's the 
only fair thing you can conclude.

Again, I will leave it to you -- we 
will leave it to your decision,

whether you think you want 
that overall conclusion,

I think it is 
important.

So this is what we propose, 
we propose you do for one objective.

But I am sure you all remember 
I said you have to do this for

each objective or we think you 
have to do this, I think RSA

says you have to do this.

No, I guess they don't.

Let me take that back.

I don't want to put 
words in their mouth.

I don't think RSA has ever 
said you have to do this 
objective by objective.

They have said just what I 
wrote up there, that you

have to evaluate the 
implementation of the SPIL.

It is our suggestion to you 
that this is a good way to go.

So here is what we would then say.

If your SPIL has eight objectives, 
and probably that's a common

number for a lot of them, 
you essentially do that

very same thing we just 
did eight different times.

You will do it for objective 
one just like we did.

That's the column that shows what 
we just talked about in objective one.

And then you will do it for 
objective 2, 3, 4, 5, 6, 7, 8.

So you will end up with an overall 
implementation of each of your 
eight different objectives.

That, in our opinion is a useful 
evaluation of implementation 
of your SPIL objectives.

Let me go back.

I want to try a little example here.

We are good on time.

Does anybody have -- anybody 
want to tell me about an 
objective from their SPIL?

Somebody tell me about one objective 
from their SPIL.

Yes, we have a hand right here.

Thank you.

AUDIENCE MEMBER: Support a 
statewide network of centers 
of independent living.

MIKE HENDRICS: I am going 
to put you on the spot.

I didn't warn you in advance 
that I was going to do this.

I am going to put you on the 
spot and ask you to consider 
that's objective number two.

AUDIENCE MEMBER: Okay.

MIKE HENDRICS: I am going to 
ask you to just walk down this

column and do your thinking 
out loud about how you might

rate that objective on each 
of these five areas and 
then also the overall.

So once again, 
it was the objective was.

AUDIENCE MEMBER: To support a 
statewide network of centers of 
independent living.

There's more to it, would you like me to?

It says to deliver consistent 
CIL independent living

services that meet the standards 
and assurances of the rehab act.

MIKE HENDRICKS: I am going to ask 
you not to do the second part for 
various reasons, just the first part.

AUDIENCE MEMBER: Okay.

MIKE HENDRICKS: To support a 
statewide network of 
independent living centers.

AUDIENCE MEMBER: Right.

MIKE HENDRICKS: Okay, now 
that's an objective of yours.

Presumably you are doing 
some activities to 
make that happen.

At least I hope you are.

AUDIENCE MEMBER: Yes.

MIKE HENDRICKS: And presumably 
there's some implementation 
of all of that.

AUDIENCE MEMBER: Correct.

MIKE HENDRICKS: And now 
you are you are going to tell us,

you are going to be brutally 
honest with yourself.

You will hold that mirror up, 
you are not at the airport with

the x-ray machine someone 
else is going to see it,

you are holding up a mirror for 
your own benefit and you are

looking in your own soul 
and heart and asking yourself

how am I doing on 
my implementation?

And the answers are.

AUDIENCE MEMBER: Okay.
Let me ask you another question first.

MIKE HENDRICKS: Sure, sure.

AUDIENCE MEMBER: Because if 
we are going to quantify

this in some way, then it is 
going to be much easier

to quantify the activities than the objective.

MIKE HENDRICKS: Yup.
We are not quantifying.

We are doing only high, 
medium, low, none yet.

That's all we are 
asking you to do.

AUDIENCE MEMBER: Okay.

But what -- I mean I have got four 
activities within this objective.

What if I am doing well on activity 
one but I am doing really really bad 
on activity four.

MIKE HENDRICKS: That's an excellent 
excellent excellent question.

So, you are saying -- hold that until 
you get to activities.

Can you work your way down 
and then when we get there 
we will go back to that question.

So resources.

AUDIENCE MEMBER: All right.

MIKE HENDRICKS: For this 
objective and only this objective.

AUDIENCE MEMBER: Okay.

I would say assuming 
that 8 is high and that 2 is…

MIKE HENDRICKS: No, I am sorry.
My mistake.

These columns are different SPIL 
objectives.

Your objective fits in this column.

AUDIENCE MEMBER: High, medium, low,
got it.

MIKE HENDRICKS: Okay, yeah, 
high, medium, low, none yet

for the resources for that 
particular objective.

AUDIENCE MEMBER: Okay.
I would say medium.

MIKE HENDRICKS: Okay.

So you are saying that for
that particular objective,

you're honest holding up 
the mirror, says you've got 
a medium level of resources.

AUDIENCE MEMBER: Correct.

MIKE HENDRICKS: Okay.
Good thing to know; right.

Good thing to know.

Keep going.

AUDIENCE MEMBER: Okay.
Staff, I would say probably high.

MIKE HENDRICKS: Okay.

AUDIENCE MEMBER: Participants, 
I would say probably low.

MIKE HENDRICKS: Okay.

AUDIENCE MEMBER: Activities.

MIKE HENDRICKS: And now we 
get to your activities part, 
and what I hear you saying

is there's four different 
major activities you've got 
going on in here; correct.

AUDIENCE MEMBER: Right.

MIKE HENDRICKS: Okay, and the 
question is should I sort of take

column two and divide it into 
four parts, you know activity 
one, two, three, four.

Our best answer is no.
No.

Think of it as an objective.

Think of the objective 
level, you know.

A nerd like me might say the unit of 
analysis here is the objectives, okay.

So wrap together whatever activities 
you've got for that objective and

together come up with your rating 
of how those activities are going.

AUDIENCE MEMBER: You mean 
like at this moment today.

MIKE HENDRICKS: At this instant.

AUDIENCE MEMBER: I would 
say really, really low.

MIKE HENDRICKS: Oh so a 
new category, really, really low.

Would you perhaps 
even call that none yet?

AUDIENCE MEMBER: Something like that.

MIKE HENDRICKS: Okay.
We might call it none yet.

AUDIENCE MEMBER: None yet.

MIKE HENDRICKS: Okay.

See now, this is that honest mirror, 
you are holding it up to yourself

and saying we just ain't done anything yet, okay.

Management?

AUDIENCE MEMBER: All right.

Now is this management from a -- I am 
SILC Kansas -- right, you guys know that.

You know me.

This is from the SILC perspective.

MICHAEL HENDRICKS: No. 
Because remember we are evaluating

the SPIL and the SPIL is the 
total IL program in your state.

Remember, the total program 
in the state.

So the SPIL is about 
everything that's going on there.

So it is not just from 
your perspective.

It is how well is the total IL 
system in your state

management wise set up 
to achieve this objective.

AUDIENCE MEMBER: Set up 
or functioning?
Sorry.

MIKE HENDRICKS: Well, no, no, 
it is a darn good question.

You know, we say here effective 
organization communication,

so effective means it is 
actually happening.

So how good is the 
management right now?

AUDIENCE MEMBER: I would 
say it is pretty high.

MIKE HENDRICKS: Okay.
So you would give it a high.

AUDIENCE MEMBER: Yes.

MIKE HENDRICKS: Let me go 
point to it, I will be like Vana White here.

You would give it a high right 
here in that management box.

Okay.

Now, let me push you if you 
don't mind and I understand

that in your state you may 
not want to do this.

But, if you take our advice 
and you take the next step

and you then try the to add, 
you don't really add them,

do you, you sort of 
have to juggle them.

That's a better way to think about it.

You have five ratings and 
you are going to juggle them.

AUDIENCE MEMBER: But see 
that's different because I'm

an accountant so what I do is
I would assign a numerical

value to each of those things 
and I would average it which

might or might not be the overall perception.

That's very different too, very 
different ways of doing this.

MIKE HENDRICKS: Yep.

Let's just -- you have 
raised an excellent point.

Let's deal with it for a second.

What was your one for resources?

AUDIENCE MEMBER: You are 
going to make me remember?

MIKE HENDRICKS: Anybody remember 
what she said for resources?

AUDIENCE MEMBER: It was medium, 
thank you.
I can't remember.

MIKE HENDRICKS: So we 
got a medium, what was the staff.

AUDIENCE MEMBER: See, that 
would be a three on the scale

of one to five that 
would be three.

MIKE HENDRICKS: Staff was high.

What was participants.

AUDIENCE MEMBER: Low.

MIKE HENDRICKS: That was 
like -- oh no no no that wasn't 
the really, really low.

That was low.

Then we had essentially 
a none yet; right.

AUDIENCE MEMBER: Right.

MIKE HENDRICKS: And then 
what was the last one?

AUDIENCE MEMBER: High.

MIKE HENDRICKS: Okay, 
and now what are you saying

you want to -- we do not know 
the right or wrong way, 
there is no right or wrong way.

You are proposing an 
interesting alternative.

You are saying you would give 
a number, this would be a 2.

AUDIENCE MEMBER: Well, 
if you start with one as high.
Let's say

MIKE HENDRICKS: I will 
start with none is 0.

AUDIENCE MEMBER: none is 0, 
don't you think that makes 
intuitive sense, right.

AUDIENCE MEMBER: None is 0; 
low is 1.  I mean --

MIKE HENDRICKS: None is 0; low is 1.

Medium is 2.

High is three, and high is three.

Somebody have a calculator?

Without a calculator, that is,
it is nine.

Nine divided by 5 is what.

1.8 maybe?

Round it up, round it up to a 2, which 
is a medium.

Is that what you think?

AUDIENCE MEMBER: Actually 
yeah I think that's a fair 
analysis of where we are.

MIKE HENDRICKS: Okay.

So just to point out, just to point 
out, you are saying there is medium

implementation of an objective that 
has not yet had its first activity?

I am just pointing that out.

AUDIENCE MEMBER: Well, 
but see that's why I asked you

about whether we were going 
to look at the activities before 
we started this process.

MIKE HENDRICKS: It is a 
good question and it is tricky, 
this is where it gets tricky.

This is why I am not a fan of adding 
them up personally and dividing them 
because the problem is weighting.

To be nerdy about it, 
when you add them up,

and divide by 5 you are 
giving equal importance 
to each of the five.

That's the way addition works, 
you know every one has equal

importance but I am not sure 
reality works that way.

Some are more important and 
you know it is subjective 
but what do you think.

AUDIENCE MEMBER: In all fairness 
in something like this which is

supporting this, supporting 
the statewide CILs,

a large part of it is getting the assets 
in place to make it happen.

So yes, I feel like that is, even 
though we are saying that the

activities are low, I mean or none 
yet, still, getting ready to make

this happen is not reflected up there.

MIKE HENDRICKS: And that 
is exactly why I would never

say your conclusion of medium 
is wrong because how would I know,

an outsider, how would I know?

All we are asking is that, 
and suggesting, is that

you decide what you think 
the overall implementation

would be from 
something like that.

There's a hand in 
the back, I think.

And thank you for that.

That's an excellent objective, 
excellent example and you did that,

frankly you did that exactly 
the way I would love other

people to do it in the future.

AUDIENCE MEMBER: Mike, this is Molly.

I have a question about the 
definition of management.

MIKE HENDRICKS: Yeah, 
let's go back because I am not 
saying we have got that perfect.

AUDIENCE MEMBER: No, 
I mean are you kind of talking 
about who's doing what?

I mean to be more concrete about it, 
like who's going to take the lead on

one section, who's going the take 
the lead on another, when is it due.

It just seems kind of nebulous, 
the organization, communication,

and I am just wondering if you can explain.

MIKE HENDRICKS: I had to get,
it is an excellent point and you

may be exposing a flaw in our 
work here that maybe we haven't

specified exactly clearly enough 
what we mean by management.

Maybe we should do 
more work on that.

I think you are raising 
a really good point.

And I guess I don't have 
a good answer for you.

But other than thanks
for raising it.

AUDIENCE MEMBER: Sorry.

MIKE HENDRICKS: No, no, no.

Don't be sorry because, 
you know, this is all work 
in progress.

We are all in this together.

We are putting out these ideas.

If you can improve them, great.

If you can show where they need 
improvement, that's great too.

We are all in this together.

So that's fine.

That's good.

AUDIENCE MEMBER: I have 
a couple of questions about 
kind of the same idea, I guess.

So with resources, are those 
resources that are allocated

through the SPIL to accomplish 
the objective or are they the

resources in total of the IL 
network because obviously

not all of those resources 
are being dedicated to 
that objective?

And I mean it is hard for 
the SILC to know how

much resources either 
staff time or any other

that a center is 
dedicating to that objective.

Similarly with staff time.

MIKE HENDRICKS: You raise 
a great point and something

that's going to feed into 
something we will talk 
about a bit later.

You raised two 
great points really:

One of them is when we 
are talking about resources,

are we talking about 
resources for the whole

SPIL or are we talking 
about resources for 
this objective.

That was one of 
your questions; right.

AUDIENCE MEMBER: Well, 
the whole SPIL or the whole

IL network because even 
those are different.

MIKE HENDRICKS: Okay.
So three possibilities.

And what I am going 
to say is remember,

since this is implementation 
objective by objective by objective,

it is resources for this objective, 
this particular objective.

So that's one point.

The second one though you 
raised was great because

you said how are we 
going to know that and

the answer is please don't 
think it is entirely your job to do it.

Remember that part 
about evaluation

being a collaborative activity.

So you should -- I suspect 
I would hope that you

shouldn't try to do this 
kind of analysis just by yourself.

Get information from other people.

Make it a collaborative activity.

Is there another question?

Yeah, uh-huh.

AUDIENCE MEMBER: I was 
just thinking that we probably

ought to do some more thinking about 
what is that definition of management.

MIKE HENDRICKS: Sure.

AUDIENCE MEMBER: Or maybe 
I am not sure that it is even the

right term because what 
I was thinking of is more

along the lines, is there 
a steward of this objective.

Because that may be a more 
apt description because I

was thinking about some of 
the things in our plan

about expanding services 
to people with visual impairments.

The steward of that objective 
may not be anybody on the SILC.

It may very well be the services 
within the VR system for blind folks.

So maybe stewardship might be a more 
appropriate word than management?

MIKE HENDRICKS: You know 
I am quite willing, in fact I would

love to see this list taken and 
improved, nothing would make

me happier than you all to 
have a little subgroup that

comes back and says you 
know, good try, but here 
it is a little bit better.

That would be great.

Stewardship I am balking 
a little bit at if you mean

instead of management 
because we all know that 
management matters.

Management is important.

Quality of management matters.

And I think if you are talking 
about implementation I would

hate to see us lose completely 
the idea of management.

Now maybe stewardship
is a sixth or I don't really know.

Like I say any improvements 
are welcome.

Let's just go on because I 
think we are almost to a

point that would be a 
logical break.

AUDIENCE MEMBER: We have 
one more question.

MIKE HENDRICKS: Then let's not go on.

AUDIENCE MEMBER: When we
talk about management

because I do read that this is 
crucial, are we talking about

the individual that would 
be holding people accountable,

making sure that this 
activity is getting done?

Is that what management 
is all about?

Because we are doing it activity, 
objective by objective.

MIKE HENDRICKS: Thank you, 
thank you for correcting yourself.

I love that.

It is not activity by activity, 
it is objective by objective.

Thank you. Well, I personally think 
that's certainly part of management,

you know the person who -- person 
or group who is responsible for

making this happen, 
that's certainly part

of management but I 
don't think that's all of it.

Because like we say on here 
there's communication too.

Communication is certainly 
a part of management.

So I think it is 
certainly part of it, yes.

AUDIENCE MEMBER: I think 
we need to define it more.

MIKE HENDRICKS: That's great.
I would love to see an improvement.

Bob Michaels has
his hand up over here.

AUDIENCE MEMBER: I think one 
of the problems that we have is

that most statewide independent 
living councils don't have any

real authority over the people that
are responsible to carry out that.

[applause]

So -- well I mean so you have
you have I can hear what people

are saying and that is well I am 
in charge, and we have

somebody that is taking the 
lead on this, and it might even be us.

But I am responsible for the state 
agency to make sure they carry it

out and responsible to report 
on centers to make sure they

carry it out and some other 
agency that has never done

anything with us before, 
to make sure they carry it out.

I have no authority to make 
sure that they do it.

I think that's the problem
we are encountering.

MIKE HENDRICKS: Okay.
That's good to hear, Bob.

Remember though, 
what we are saying.

We are not saying that -- we 
are not saying that you have

the authority to make 
somebody manage better.

All we are doing now is trying to hold 
that mirror up to the management

and see how it is, and remember 
I am not even saying you hold

that mirror up by yourself.

Remember the 
collaborative part of it.

Gathering the 
needed information.

In other words, doing this 
analysis of implementation,

in my opinion would be 
best done very collaboratively.

So, several groups of people 
would be saying, around a

table, how is the 
management, you know.

So I am not trying to put this 
on the SILC to even do

the analysis, much less 
have responsibility for it.

Yeah.

AUDIENCE MEMBER: Just in response 
to Bob, we are not

we don't have the authority to make 
people do anything or to tell them

how they have to do it but we 
do have the responsibility

for monitoring, reviewing, and 
evaluating the implementation

of the SPIL and by looking at it 
like this, we are not even saying

that anything or anybody is good or bad.

We are just saying how the 
implementation is going.

We are not responsible 
for the implementation.

In fact we have been told we are not 
supposed to implement the SPIL.

But we can at least show how
it is going and to me that fits 
within our responsibility.

AUDIENCE MEMBER: Yeah, 
further on that one is that for

the last two years I know I have 
but many of us have been

sitting here going I don't want 
to tell you what my objective

is because you are going 
to make me evaluate it,

and I am not 
doing very well.

MIKE HENDRICKS: Well, 
doomsday has come 
because you are right.

AUDIENCE MEMBER: We are 
not doing and we are not

evaluating ourselves, 
we are evaluating the

entire process, 
the entire in IL.

MIKE HENDRICKS: The SPIL.
you are evaluating the SPIL.

AUDIENCE MEMBER: Anns right, 
we are evaluating whether

anything is being done 
or happening or anything.

And if we can't get them to give 
us information on how they are

doing with an objective, 
then we have to start

saying management is 
low or nonexistent.

MIKE HENDRICKS: Remember 
though, this is only the first leg 
of the stool.

AUDIENCE MEMBER: But I am 
talking about anyone [inaudible]

MIKE HENDRICKS: This is 
just implementation.

AUDIENCE MEMBER: So if I 
can't if I am not getting that

collaboration, if I am not getting 
that communication within one

of my agencies, my agency, 
one of the agencies involved

in the implementation of the 
SPIL then I am going to have

to look at management 
and say it is low.

MIKE HENDRICKS: From your 
point of view as you sit around

the table and look at the 
management of that

implementation of that 
particular objective,

you would say that 
was pretty low.

AUDIENCE MEMBER: yeah.

MIKE HENDRICKS: Now 
somebody else sitting

around the table may say 
I think it is pretty high.

That would be an interesting 
discussion around the table

but, yeah, I hear 
what you are saying.

AUDIENCE MEMBER: And for
some of us the problem is 
there's nobody to discuss it with.

MIKE HENDRICKS: oh, well.

AUDIENCE MEMBER: If we are 
sitting as a SILC and are we

not able to get your partners 
in this process to talk to us

about it, then there's 
nobody to discuss it with.

MIKE HENDRICKS: Then in 
that case you will have to do

it as collaboratively as you can which 
may not be very collaboratively.

I hear you.
Okay.

AUDIENCE MEMBER: Mike, 
I think you kind of said what

I was going to say, but I think 
that there's a difference between

management as a noun, 
people are managers, management.

And management as a verb.

You said that we are -- this is 
management of an implementation.

MIKE HENDRICKS: Management, 
that's what we are looking at here.

AUDIENCE MEMBER: It is not 
that I am a manager,

other people are managers and 
how it is being managed in that capacity.

It would be a staff resource, 
wouldn't it?

MIKE HENDRICKS: I don't know 
how it would be in your state,

I don't want to say.

But you are right, what we are 
looking at here is the

management of one particular 
objective and how well 
that's being implemented, yes.

Back here?

AUDIENCE MEMBER: I am glad 
that Tim is here because I

think this is something important 
for RSA to realize, that this

structure that's set up under 
the rehab act doesn't put 
SILCs in charge.

It doesn't put DSUs in charge.

And without at least those two 
willing partners and hopefully

other willing partners, 
this doesn't work.

So things aren't working in a state, 
it -- it is not easy to pin point

who's responsible and who
can fix it because it could

get a bad 
relationship.

It could be an unwilling partner.

It could be a lot of things.

And I think for us, 
it is important to 
realize that this,

if you do have a partner in the 
DSU that is concerned about

improving outcomes, this is an 
opportunity to build that relationship.

MIKE HENDRICKS: Yeah 
I am glad you said that because

our suggestion here is to be very 
collaborative with all of these

different partners 
as much as possible.

I am firmly convinced that's 
the best way to do evaluation,

but you are saying there may 
be cases in which the partners

don't want to be collaborative with us.

And I hear you.

AUDIENCE MEMBER: This is 
Valarie from Michigan,

and I would like to take what 
Mike said one step further,

is the reality in our state is we 
have done a lot of cutting 
as many other states do.

Service providers, that provide 
IL services as well as CILs have 
reduced their staff.

Our partners may believe 100% in 
what's in the SPIL and in what's

going on, but with their limited staff, 
they don't have the resource to bring

that individual to the table to help 
us in the evaluation project and process.

And therefore they're -- they're 
withdrawing from it, not because

of desire but because of the extra 
demands that we're placing on

them in the monitoring and evaluation.

And that's going to be something 
that I know in Michigan we are

going to deal with at least for 
the next couple of years.

MIKE HENDRICKS: Okay.

So I think what I hear you saying 
is like you say extended the same

ideas it might be nice to be 
collaboratively, to be collaborative

but if there ain't nobody 
there then what do you do.

AUDIENCE MEMBER: They have
put the priorities on the service.

MIKE HENDRICKS: Okay, well, 
I would say, I would bet that

RSA's reply might be that's 
a shame, we hate to hear that.

But somebody still needs to be looking 
at the SPIL and it may just have

to be you with as many people 
as you can get to look at it with you, 
I am guessing.

Richard?

AUDIENCE MEMBER: I would like 
to come back to the point I think 
that I was hearing.

One perspective and that is that the 
configuration of the SILCs roles and

responsibilities makes it impossible 
to achieve the objectives of the SPIL

because no one, no one 
entity is responsible.

If I heard that I guess I would just 
offer a different perspective and

that is that yes it is impossible

if you hold to the hierarchical
one entity is responsible

approach to operate it.

But on the other hand we have 
talked about collaboration and 
evaluation which is great.

Mike alluded to this earlier,
if the objectives are developed

collaboratively and if the people 
who will have the responsibility

for implementation of that 
particular objective are at the

table at the time that makes it 
more likely that the, that the

objective will be implemented 
and there is some mutual

accountability that comes through 
in the evaluation process.

So it is a different way of looking at 
it and Bob perhaps can help us

in some of the thinking in the 
historical perspective about

how that language came to be there:

But there is perhaps just another way 
to look at that, another way that

SILCs can go forward by 
working collaboratively and 
getting things done.

MIKE HENDRICKS: Okay.

I am -- this is a lively discussion, 
exactly the kind of discussion 
we want but it is break time.