PAULA MCELWEE: We're going to talk 
a little bit about outcomes that have

been selected by a taskforce working 
on this on all of our behalf.

But before we do that, let me say that 
you must make decisions for yourselves

at your center what you 
want your outcomes to be.

It may be these same outcomes, but it 
may be something completely different.

So you want so make sure that you know 
what you want your outcomes to be.

That's an important piece of this.

So as we talk about outcomes,

some of this is going to come 
out of a national conversation,

but you still need to have 
your local conversation.

Does that make sense?

Somebody was asking me during 
the break about what we mean 
by transformational,

looking at quality outcomes 
as transformational.

Well, it means that you at your 
center need to look into the future and

imagine how you want to transform things,
and you need to structure

outcomes around where you want to go.

We're going to talk a little about 
the nuts and bolts of doing this,

but -- and we're going to give you 
examples from the national conversation,

but it is a local conversation that 
also has to happen when you get home.

You need to, as a center, 
your board, 
your staff, 
your consumers,

buy in together to where you 
are wanting to go, and then to 
measuring what that means to you.

So quality outcomes are not 
identical from one center to the 
next or one community to the next.

There are some similarities that 
you'll see because we have a similar

funding structure which 
requires similar services.

We've developed a network, and so 
we believe that we have some good 
answers across the board.

But it is absolutely -- an essential 
part of that is the local conversation.

You may still end up with the same 
agreement on what those outcomes

should be, but if you don't have the 
local conversation, then you're in

a situation where someone else gave 
them to you, and they aren't your own.

You need to own these when you're done.

So we're going to give you tips 
and tools and ideas as we go along,

but you're not done 
when you leave here.

You don't get to just 
come to the training.

Right?

Because this is something you 
have to put into effect back home.

This conversation needs to keep on 
going on with your center and your

entire community, whatever 
your disability community is,

and your community back home.

So keep that in mind as we go 
through here, how you're going

to make it happen, how you're 
going to make the work happen.

All right.

Let's see if I can make this work.

This is Bob Michaels, and he's talking 
about the NCIL taskforce on outcomes

and about the process of 
narrowing down the 
outcomes from 16 to the 
ones they came up with.

[video playing.]

VIDEO-BOB MICHAELS:  We 
have got this process now.

We have identified 
the desired outcomes.

We put them into a logic model.

But, right away we realize, hey,
we have got way more than we 
can deal with here.

We are gonna have 
to pare this down.

So how did we go about doing that?

Of course, being who we are,
we said we wanted to go back

out to the field and find out 
what the field wants to do.

So we went through a number 
of different exercises we needed

to go through realizing that 
whatever we came up with

eventually we are gonna have 
to go back to the task force and

say okay, this is what people are 
telling us out in the community, 
that they want to do.

So what we did, is we went out 
there and said, now we are

going to try to, we are going 
to have to choose which 
outcome we want to measure.

So we wanted, as I said, 
to make the process 
as inclusive as possible.

What we asked people 
to do was, we would like

you to look at these 16 
and pare it down to 6.

What are the 6 centers that 
you want, er the 6 desired

outcomes that you want 
to work on the most?

And what we did, was we put 
a link in a letter, in a email

that we sent out to people 
and we said when you go

on the site for the outcome 
measures, there is a link

there that says last chance, 
please give us feedback
on the desired outcomes.

And you click on that and we 
have a survey and we are

going to ask you to identify the 
six that you think are most important.

What you did, 
was you identified 8.

There were about 8 that 
were clearly out ahead 
of everybody else.

So we had 8 of them 
instead of 6 of them.

What we did was take them 
back to the task force and

we said, okay, we think 
we should go with these 8
rather than the 6.

Because these all seem 
equally important to everbody else.

So lets look at 8.

And they agreed to do that.

So here are 
the 8 that we have.

We picked the IL services.

They picked the first one.

Persons with disabilities have 
skills, knowledge, resources 
to support their choices.

They picked the fourth one, people 
with disabilities are more independent.

On the I&R stream, people with 
disabilities get the information

they need and they picked people 
with disabilities to advocate for 
increased community supports.

On the systems advocacy, 
they picked four.

The barriers and problems 
identified in the community.

A consumer agenda for change exists.

Decision-makers act on our agenda.

And methods and practices 
promote independence.

Now, the packet I gave you this morning, 
I referred to this morning,

the one in color that also has the 
logic models, on page number 8 of

that packet, you have a picture 
of all of the eight that they chose.

Now, the last three, all three, 
the service streams, the very top, 
none of those were picked.

They did not pick people participate 
in communities to the extent they wish.

Communities are more accessible 
or that people with disabilities are 
integrated into American society.

Didn't pick any of those.

If you look at that chart, 
what you see is, 
eight of them are --

PAULA MCELWEE:  We'll have this 
chart for you a little later.

VIDEO-BOB MICHAELS:  You see 
there was at least one or two,

there were at least two in each stream, 
and there was four in the final stream

that has to do with systems advocacy.

Okay?  Any questions?

PAULA MCELWEE:  I'm sure you 
have lots of questions about this.

We'll be coming back to it, 
but I wanted to get you a sense of this.

VIDEO-BOB MICHAELS:  What you 
said is most important.

These are the ones that, after you 
told us they were most important,

then we began to work on them 
to identify -- to identify ways 
to measure them.

PAULA MCELWEE: But what you see is,

there's a national conversation around 
outcomes that really do define quality.

A center is only -- some 
of you identified this.

A center is only providing quality 
services if the result is,

people are making choices 
and becoming more independent.

And you saw that reflected then in 
the national conversation as well.

So we are part of this national 
conversation around outcomes.

We can be very intentional, 
then, about quality.

We can make solid decisions on 
a day-to-day basis that assist

us with not just imagining what 
the future will be but drilling

it down to what are we doing every 
day that affects that and how can

we make sure quality happens.

William Foster said: "quality is never 
an accident. It is always the result

of high intention, sincere effort, 
intelligent direction, 
and skillful execution.

It represents the wise 
choice of many alternatives."

So as you look at quality for your 
center and look at the national

conversation for CILs around quality, 
you will see that there can be very

high intention which results, then, 
in good choices around all of the

different alternatives for your 
time and for how you put your 
efforts together.

And, in fact, we need to be 
intentional about quality for 
lots of different reasons.

One is, we are a movement and 
we can never forget our roots.

We come out of disability 
rights efforts in this country.

We have come out of a rich history of 
making sure that the rights of people

with disabilities are honored wherever 
we go and we are part of that advocacy

and we are a very vital 
player in that activity.

And movement keeps moving.

A movement seeks to change the world, 
and you have to be intentional to 
make that happen.

Now, here is the other 
side of this story.

As I said when we started out 
this morning, there has been a 
lot of attention on compliance.

There have been more compliance 
visits than ever before.

We are seeing a lot of attention on 
compliance and we are identifying

through those compliance 
efforts some weak centers.

Maybe you know of a center in your 
state or your region that is weak.

When a center is weak and ineffectual, 
or in some cases, even dishonest,

that contributes to society's 
negative perceptions about 
people with disabilities.

Oh, "those people," and we 
are "those people," right.

So not only do we have a 
responsibility to work toward

quality in our own centers, 
we also have a responsibility

to do it as much as we are able in 
collaboration with other centers,

as much as they are willing to buy in, 
as much as we can participate in that

together so, thus, both the national 
and the local conversations and

sometimes that conversation in 
the middle is the conversation within 
a state regarding that.

You know, those mediocre centers will 
not facilitate change in their community.

And it's not to anyone's advantage 
in this movement for those centers to

continue to be ineffectual and we need 
to find ways to drive quality wherever

we can but one of the first things we 
have to do is define it for ourselves

and then demonstrate it because that's 
how we are going to change the world.

So does your CIL have the 
tools you need to do that?

That's why we are 
talking about quality.

We want to make a difference.

We want to change our world, so what 
are the tools that you need to have?

We are going to be looking 
at some of those.

Some of the tools are already in 
your hands and see how they help

you set up and measure 
some quality indicators.

We will show you how other centers 
are approaching quality outcomes

so you are going to hear from a 
couple of centers' perspectives 
related to that and then we will

discuss the shift to more 
outcome-based measures rather

than demographic measures 
as we assess quality.

Do you know what I mean by that?

I will tell you what I mean by that, 
I see kind of both.

You are familiar with your 704 report?

You completed your 704 report and 
sent it in at the end of the year, 
as you always do.

Much of that information 
is demographic in nature.

A lot of it is.

Who were the people that you served, 
what are your numbers by disability,

by numbers of, you know, 
types of goals set, 
goals met, 
different demographic information.

One of the things that's happening in 
this national conversation is a move

towards making sure that what we report 
as a field is more outcomes based

rather than just demographically based.

Numbers are fine if they lead you 
to understanding better what you are 
or are not doing, but numbers

that just kind of sit out 
there are meaningless, 
can be meaningless.

I have the, sometimes dubious 
pleasure of working with centers 
that just have been reviewed by RSA.

That is one of the main things that I do.

I work with centers that are 
intensive support, meaning 
that they have had a review

either by their state if they are a 
Part B center or by the Feds if 
they are a Part C center

and in some cases both, and I see 
interesting comments in those reports

around the 704 report.

Often statements around the 
information in the 704 report 
not being accurate and useful.

Now, I am sure when RSA looks at that, 
they aren't looking at it the way

we will look at this week, 
but what they are saying is

some of what you have in this report is 
not congruent with who you are and they

can see that coming in for a few days' 
review, just kind of off the top,

off the top, the 704 report doesn't 
describe well who you are.

Well, if the 704 report doesn't describe 
well who we are, it needs to change.

We are working on that, but we also 
need to work on how we will describe 
who we are for ourselves.

Because between where we are now and 
where we hope to go with that 704 report

is time, quite a bit of it probably, 
there has been a long time already

that the conversation 
has been going on.

So we don't know when that will 
change or if it will change or 
how it will change exactly.

What we do know is it does not define 
us and it does not describe the

outcomes that we want to be able 
to describe to show that we are 
doing good quality services.

Does that make sense?

So we need to go beyond compliance.

We need to go beyond the other reports 
that we are putting together and we

need to ask those questions around 
quality and find ways to provide the

answers for ourselves and our consumers 
and then for our field so that we can

show the effectiveness of centers 
instead of always ending up being

kind of pulled down a little by 
the centers that are not effective.

So it's an important discussion, 
I think, for that reason.

So you set the standard 
for your community, 
for your state, 
for your region.

You are the yard stick of quality 
and some people aren't used to an

environment where 
excellence is expected.

That's a Steve Jobs' quote.

"Be a yardstick of quality.

Some people aren't used 
to an environment where 
excellence is expected."

And I think within our states and 
within our centers nationally,

we have both of those situations, 
don't we?

We have centers that are yardsticks of 
quality, are measuring it and getting out and

getting it done but we certainly have 
centers who aren't used to the

environment where excellence is expected.

We want to turn that around and one of 
the best ways for us to do that is for 
each of us to be an example of that.

I am probably preaching to the choir 
because you all guys are here and you

all have quite a bit of background 
in quality, but that's our framework.

That's going to frame our discussion.

Now, on your table, you have one 
of the definitions of quality.

And what we'd like to do is have some 
group discussion around -- around each

table and if your table is small, 
feel free to join another table.

I think you are the really only small 
group here if you want to -- somebody

else will let you in, unless you feel 
like you have got lots of conversation.

These are the questions you 
will answer around that.

Do you agree with the definition and 
how can CILs demonstrate the quality

by that definition, so there are 
several different definitions out

there and we will talk about each of 
those, and what is a way to measure

whether your center meets 
quality in that way?

And then what we are going to do at the 
end of this discussion time -- we will

give you a few minutes to do this -- we 
will go around and ask someone to read

which definition you have at your 
table and what you decided would

demonstrate quality in that way.

Is the assignment clear?
All right.

So have your discussion and we will 
come back together in a few minutes.

Okay lets come back together.

Some interesting discussions 
as I moved around the room, 
good conversations.

That's great.

Let's share the high 
points with each other.

We gave you guys first chance 
last time so we will start on 
the opposite corner this time.

AUDIENCE MEMBER: So we get it again?

PAULA MCELWEE: You get to 
close it out.

AUDIENCE MEMBER:  All right.

PAULA MCELWEE: So back here, 
let's hear what you found about

the definition -- which definition 
you had, whether you agree with it

and how you would demonstrate quality?

AUDIENCE MEMBER:  Well, we had 
definition 4 on quality is 
superiority to competitors.

We had a lot of discussion about this, 
and in some regards we feel like we don't

necessarily agree with this definition as 
it relates to programs and services in the

CIL but in other ways we feel like 
it does relate to what we are doing.

Some of our programs are fee for service 
programs and in those programs, I think,

you know, what you are doing and how well 
you are doing it, you want to do those

programs better than your competitors 
because maybe in some instance it -- the

more money that you can bring into the 
CIL, the more services and programs you

can implement, the more things you can do 
for your consumers, right?

So you want to do those fee for 
service programs better than 
your competitor but we also felt

like, in regard to some of the 
other programs, like core 
services, that we shouldn't

be in a competitive -- it shouldn't be 
competitive between the CILs,

we should be collaborating 
and networking with each other.

We should be sharing our thoughts and 
ideas and the things that work good so we

shouldn't be competing in that regard, 
but maybe, like I said, in some of the

other programs we do, 
it can be competitive.

PAULA MCELWEE: That's good.

I forgot to have you identify 
yourselves and I am gonna

get in trouble for that later, so If you 
would tell us who you are.

AUDIENCE MEMBER: Sure, I am 
Mitch Iddins and the independent 
living program director from the

CIL in Shreveport, New Horizons 
Independent Living Center.

PAULA MCELWEE: Okay.

Any other observations from your 
table about any of the other questions?

If you want to speak, grab a mic and 
identify yourself.

AUDIENCE MEMBER: I think that 
was a consensus over here?

PAULA MCELWEE: All righty.

Let's hear from this table over here.

AUDIENCE MEMBER: Ours is a quality 
exceeding -- quality is exceeding 
the customer expectations.

PAULA MCELWEE: Will you identify yourself, 
please, Audrey.

AUDIENCE MEMBER: Audrey Schremer 
from Three Rivers, Inc., in 
Wamego, Kansas.

Really short memory too.

So our first is that quality is the 
extent to which customers or

users believe the product or 
service surpasses their needs.

And we had a good 
conversation about

sometimes expectations aren't 
reality when they come in the door.

So this is only part of the definition but 
not all of the definition of quality and

so we talked about then 
your plan becomes helping 
them see what reality is.

And then how can we 
demonstrate the quality.

We felt like we need to know the 
expectations up front with consumers,

have a vision what their finished 
product looks like for them.

And for communities to develop 
an action plan so we have those

indicators at the end and define 
what that success looks like so

you've got your indicators, 
seeing what that achievement 
is, and then surpassing it.

How we measure it, we looked 
at satisfaction surveys, the

suggestion of the ratio of closed files

of individuals who met their goals versus 
those that didn't and that's a huge effort

for a whole lot of CILs and I was 
glad to hear that get brought up.

Regularly revisiting the goals 
and then focus groups also,

so we don't forget we go 
far beyond just our consumers.

Look at family members, 
other service providers

and community people.

PAULA MCELWEE: So look at the 
community at large as you

are determining your outcomes.

Other comments at that table?
Okay.

How about this group over here?

AUDIENCE MEMBER: We looked at how –

PAULA MCELWEE: Identify yourself for me.

AUDIENCE MEMBER: I am Jan from 
Independent Living Center in 
Colorado Springs.

We looked at number 2, how can CILs 
demonstrate a different quality and

I think we demonstrate this kind of 
quality by making sure that the services

we are providing meet the needs of the 
consumer, that their goals are met and

that we determine whether we are 
providing the services that our

particular state demographics 
meet -- need.

As for number three, the way to measure 
whether our services meet the quality is

determining whether our consumer goals are 
met, which we get when we look at the

number of consumer files that are closed 
and if not, we need to determine why the

goals aren't met and whether we need to 
provide additional services, whether we

need to maybe change the services that we 
provide in a different way, and if we need

to provide new services that 
a consumer needs in our region.

I have the consumer feedback that we  
get from the surveys that we send out.

PAULA MCELWEE: Okay.

And just to note, because a couple 
of you have mentioned this and not 
everybody is in the same place on this.

You have mentioned counting the 
closed files, counting the closed 
files as a measurement.

All closed files?

AUDIENCE MEMBER:  No.

PAULA MCELWEE: So you've got
to distinguish the reasons

why files might be closed and that's 
data versus good data conversation that we

will have as we go through the time but
I thought that is a real good example

of what we are talking about there, right.

We are looking at the 
reason that people close

the file because they met their goal 
and so's a good distinction, I think.

A good opportunity to say that.

AUDIENCE MEMBER: I think also we need 
to look at the reasons why a consumer

withdraws from the program, 
why they are withdrawn?

PAULA MCELWEE: So it is still important 
to know the reasons behind whatever their 
answer is, whenever possible.

Not everybody gives us that opportunity.

Whenever possible.

Anything else from that table?  
All right.

How about this table right here?

Identify yourself for us, please.

AUDIENCE MEMBER: I am Charlie Lane

from Coalition of Responsible Disabled 
in Spokane, Washington, 
and our group focused on number 4

the quality is a superiority through 
competitors and so we looked at the

definition, quality is how a company's 
product and services compared to those of

competitors or how they compared to those 
offered by the company in the past.

We found that the definition of "they" can 
mean different possible interpretations.

There needs to be a definition within 
a document, a definition, of how to 
you define competitors.

How do you define what 
products and services are.

There is so many different perspectives on 
what the definitions can be, so to answer

the first question, do you agree with 
this definition is both a yes and a no, 
depending upon how you look at it.

This definition is very gray.

It needs to be sharpened, more narrowly 
defined, and when you look at all of our

centers, within our states, there are lots of 
layers of interpretations of what we do,

how we do, when we do things and how 
we got to that point, so the first thing

that comes up are quality - several things really.

One is how our centers are responsive 
to the evolving changes in each of our 
communities.

Also, to respond sensitively to 
the diverse needs of the 
community with different disabilities

and how do we connect transparency and all 
that we do and how we do things and also

how can we use technology to our advantage 
to support our mission, our services in 
the needs of our consumers.

And what tools do we need to make quality 
a very clear element in how we define 
our centers.

PAULA MCELWEE: Okay.

But you came up with a 
whole bunch more questions.

Do you have any answers for those?  
I'm just kidding.

Okay.

Good.

Anybody have any other comments 
from this table?  
Okay.

AUDIENCE MEMBER: We were just feeling 
like it was very gray.

PAULA MCELWEE: Yeah, it is.

And you've hit a really important point, 
and that is that whatever you decide 
about outcomes needs to be well defined.

And we'll talk about some of the ways to 
make things well defined and measurable.

You know, we know we're supposed to do 
that with the plans for the people that

we provide services to, although sometimes 
I see them that aren't measurable either,

but we know we're supposed 
to measure everything.

How do we make it measurable.

Yeah, Charlie.

AUDIENCE MEMBER:  I've got to 
add one more thing.

We talked about the spectrum between 
the other end of process through outcomes, 
and --

PAULA MCELWEE: Right.

Right.

AUDIENCE MEMBER: That needs to 
be addressed as well.

PAULA MCELWEE: That's a really good point.

As we get into outcomes more, we'll talk 
about them, but there are things that

you measure that have to do with 
process rather than outcomes.

One of the first ways -- one of the 
first cuts when deciding, is this really

a measurable outcome, is if you intend,
where it says date accomplished or goal

date, if you tend to put "ongoing" there, 
it is not an outcome.

If you want to put "ongoing," 
which it is still a process.

And you may still measure your processes 
and that may still be part of your 
quality, but it is still not an outcome.

We'll hit some more tips 
like that as we go along.

How about this table back here.

Be sure to identify yourself.

AUDIENCE MEMBER: Sally Decker, ARCIL, Inc. 
Round Rock, Texas, 
satellite of Austin, Texas, ARCIL.

The definition of quality 
is fitness for use.

Do we agree.

No, we don't agree.

At least some of us don't agree.

Quality, to me, in my interpretation, 
is what to bring to the table from

the beginning, in the front door.

But quality can only be measured -- you 
measure it by what you do, the outcome,

the intake, the results, but -- if your 
part of the quality is restricted or limited,

then you come to barriers that not only 
affect you, but your consumers.

AUDIENCE MEMBER: John Meinkowsky 
also with ARCIL here in Austin.

The definition fitness for use means the 
service does what it's intended to do,

so only to a certain extent 
do I agree with that.

That's incomplete.

Second question, how do CILs demonstrate 
this quality?

You have to first look at that whole idea 
of what is it intended to do -- who's asking?

Because RSA has their expectations 
about what a CIL is intended to do,

and of course, you know, RSA doesn't 
do anything unless Congress tells 
them to do it and gives them money to do it.

We have the state legislature also, 
we have local city governments,

we have other local power structures in 
each of our communities.

We have the consumers.

So everybody has ideas about what 
we are intended to do, or what 
we should be doing.

So we have to develop some type of
- something we're going to report.

We have to have something that this 
is an activity, and there are some 
numbers attached to it.

And if you do those things and meet 
your numbers, most people are going to say 
you're doing what you intended to do.

You may not be doing what they wanted 
you to do to begin with, so again,

a very limited definition.

So how do you measure that?

Well, everything has a 
fairly obvious measure.

Probably most everybody does that.

You measure -- if you're doing what you 
intended to do, you can do a follow-up on

your I and R, and you can do 
satisfaction surveys with your

consumers, and you can 
write reports to the 
people that give you money.

That's the easy part is, you know, 
measuring that because you're probably

measuring things you can count, regardless 
of the relevance of those things.

PAULA MCELWEE: Other comments from 
this table?  Anyone else?  Okay.

How about this next table back.

AUDIENCE MEMBER: Tami Hoar, program director 
of Montana Independent Living Project.

We had definition 2, quality 
as meeting customer expectations.

And our first discussion was just around 
that, what customer expectations are,

and initially that there are a lot of 
folks that come in with a false

expectation or very, very, very high 
expectations of what a CIL is or 
does or can do.

So we talked a lot about changing that to say 
meeting informed customer expectations,

and we can demonstrate this 
quality through education to the 
public and other referral sources,

as well as educating the consumer 
when they initially come in, those

I and R calls and those kinds of things, 
and how we're going to measure

this quality is just what everybody else 
has said, again, through specific consumer

surveys, and not just through 
closed consumer service records,

but, more specifically, looking at 
accomplished goals; have they completed

those goals that they set out; 
what were they, to begin with,

were they, as you just said, were they 
measurable to begin with, and 
that's really the measure of quality.

Did the consumer in the beginning 
understand and have an expectation

of the service to begin with, and then 
how -- if they're meeting those goals,

then we're assuming through that then that 
we're meeting that expectation for them.

And through that process, there's an 
ongoing communication with that consumer.

PAULA MCELWEE: I like that you 
changed the definition.

Anybody else from that table 
have a comment to add?  Okay.

Thanks, Tami.

Far out there in that last 
table in the middle.

AUDIENCE MEMBER: All right.

Jason Beloungey, assistant 
director for Access to 
Independence in Madison, Wisconsin.

We had question number 3, and to 
answer the questions, do you agree with

the definition, the group really had more 
of a "no" answer, although as they talked

it out, there was kind of a mix 
of yes and no in there.

Earlier somebody mentioned about, 
you know, we can provide a

quality service, but yet the person may 
not feel their needs are met because the 
needs may not be realistic.

PAULA MCELWEE: Can you 
remind us which definition this is?

AUDIENCE MEMBER: The definition is:

Qualities exceeding the 
customer expectations.

PAULA MCELWEE: So customer 
expectation one.

Sorry to interrupt, 
but I think that helps.

AUDIENCE MEMBER: You bet.

The other one that was kind of -- that 
really was around the kind of "yes" side

is that often people realize that we have 
more than they thought, so as part of that

exceeding, they come in, they, you know, 
want -- somebody mentioned wanting a ramp,

then they realized we have our 
technologies, other skills training, 
things like that.

Another area more on the yes side was 
sometimes you provide a response to needs

and barriers that people 
weren't even sure they had.

So another example to that was somebody 
coming in for rental assistance,

looking for help with housing, and through 
quality services, we determine that they

actually had issues with income, 
or budgeting, things like that.

So really surpassing the need that way.

The second question was, how can sales 
demonstrate that kind of quality.

Really, providing services that the 
consumer was not aware that they 
needed was one area to do that.

Also, doing a lot of outreach 
and community presence, 
is really important to that.

And then somebody earlier mentioned 
again about knowing expectations from

the consumer is obviously important.

And the third part, we also talked about 
the question was, what is a way to measure 
whether your center meets the quality in this way.

We, too, felt surveys, satisfaction 
surveys were important.

But a couple of other things that 
came up, was recognition by other 
entities, like awards.

It could be an award from local 
government, could be things like

non-profits recognizing the work 
that you do, collaborative efforts, 
things like that.

And then success stories.

Certainly we do that as part of the 704, 
but using it for other means as well

when we do outreach, when we talk 
about our services to other people,

other agencies, putting a human 
face on the work that we do.

And then also getting people with 
disabilities more involved in the

community through that work, 
and having -- showing there is

a presence of people with 
disabilities in our communities.

PAULA MCELWEE: Okay.
Good.

Anybody else from that table 
want to add anything?

This third table back here.

AUDIENCE MEMBER: Hi 
Erin Fontaine
from Access North

Center for Independent Living 
of Northeastern Minnesota.

We had definition 1, quality is fitness 
for use, means the product or 
service does, what it is intended to do.

And we didn't necessarily say it was 
inaccurate but it was incomplete and

maybe missing some clarity, 
and if we knew what the intent

who was deciding the intent; 
is this something that is,

you know, directed from the consumers, 
from the staff, from -- you know,

where is that being -- coming from 
as a starting point because that's

really hard for us to determine if 
it's doing what it's intended to do if 
we don't know what it is intended to do.

So that's the biggest barrier 
with the definition for us.

How can CILs demonstrate 
this kind of quality?

Really, we said it really 
couldn't be just one method.

We needed to be able to collect data, 
we needed to be able to hear,

you know, the needs of our consumers, 
we needed to do more education in our

communities about what CILs do and 
what we can provide and how we can 
provide them and what our role is.

We talked about -- you know, too, 
so many different levels of different

funders or providers or different sources 
that might be telling you what you need

to do or how to do it or what those 
different things are, what your intents

should be with RSA or state rehab services 
or your statewide SILCs or consumers.

How family members, or other providers might 
be referring people in and have different

expectations or desires in what they 
want you to help that person with 
is not their goal.

And we talked about, too, you know, 
as far as quality, there's maybe

sometimes where someone is really 
satisfied with your services,

when it wasn't a quality service, 
and other times where it may be

a quality service where a person 
seems really unsatisfied.

It's just, you know, making sure that we 
have good awareness and understanding of, 
you know, how these goals are written.

How can we determine outcomes and go 
from there, and with a measurement.

We talked about, again, 
it had to be multiple formats.

And again, you're going to have -- each 
person is unique and individual and we

all have different expectations, 
and so for -- you know, to determine that

quality level, having the documentation of 
goals and outcomes and the reports and all

those things, along with satisfaction 
surveys, making sure that the survey

is something that can be useful and that 
can make sense to consumers so that they

can provide feedback that can be 
useful for you, focus groups,

listening sessions, you know, referrals.

You know, are you -- are you well-known 
in your community that other agencies or

programs can understand what you do so 
that they can -- and understand what you

intend to do, and what your programs are, 
so that they can see that.

PAULA MCELWEE: Okay.

Anybody else from that table 
want to add a comment?

I think it's really interesting, 
while you're thinking if you have

another comment, satisfaction surveys 
have come up over and over again,

but as you describe what a satisfaction 
survey is, it becomes pretty obvious

to my ear that not all satisfaction 
surveys measure the same thing.

And Ann is going to talk about 
satisfaction surveys and how a center

uses that satisfaction 
information a little bit later.

But I think that that's an important 
conversation as you go through this

process of being peer 
support to each other.

A really fascinating thing is to ask to 
see other people's satisfaction surveys

because your satisfaction survey may be 
completely different than your neighbor's,

and we could all learn from each other 
and find the best satisfaction tools out

there, and it could be really useful.
You see some that are way too long.

And you think, oh, my, there's no way.

And then you see some that are so short, 
you know, they're kind of like at the

end of the Taco Bell receipt, it says, 
you know, call and answer these three

questions, or the consumer card that 
you can pick up at long John silver's.

I am giving away my eating habits.

Wherever it is that you pick up the card.

That will tell you some things about 
how the world looks at satisfaction,

but where do those questions come from, 
and how do we determine them,

and how often do we ask those questions?

Do we ask those questions each time a 
service is delivered, or once a year?

I had an interesting conversation with 
an executive director, very small center, 
just getting started, three staff.

Weren't serving very many people yet.

And I said, "Well, tell me how your 
satisfaction information is going."

And he said, "You know, I've mailed 
that survey out three times, 
and I still haven't gotten an answer."

And I said, "did you think about just 
picking up the phone and calling them?"

I mean, we're not talking 20 people.

Each of you, each staff person 
could have done five and they'd be done.

"Did you think about 
picking up the phone?"

"Oh, no, I never thought about that."

What fits your organization and your 
community is an important thing to

talk about and figure out some answers 
because sometimes it's not the traditional

once-a-year, I'm going to 
mail it out, survey.

That may or may not get you anywhere, 
and those of you who are from large

centers know how much turnover you 
get back if you do it that way.

So you need to think about all the 
different things that have to do 
with satisfaction.

You had a question or comment?  
He'll get you.
Just start talking.

AUDIENCE MEMBER: Okay.
Hi.

Michelle Parker from Bay Area 
Center for Independent Living
in Salsbury, Maryland.

We're a new CIL.

We just have been practicing --

PAULA MCELWEE: You're a new name.

You confused us at first.

We couldn't figure out who you were.

AUDIENCE MEMBER: It's probably the accent.

We've sent out some satisfaction surveys, 
and one of the questions is one,

do you send them anonymously, and, 
two, when you get the feedback,

how do you properly measure -- if you 
don't know where you went wrong for that

particular consumer, 
how do you know 
how to fix it?

So that's just kind of one 
of the questions we tried to --

PAULA MCELWEE: Well, jot those down.

We'll cover those a little bit later 
this week, but we really do want to go

into some detail on satisfaction 
surveys because there is, truly,

a lot of variation on how 
satisfaction surveys are done.

Some states, the SILC defines how 
satisfaction is happening and it comes 
through to the centers through the SILCs.

Sometimes what the SILC is doing 
isn't satisfactory to the centers and

so they do both, and they do their own, 
plus they do a statewide whatever is 
happening at the statewide level.

We see lots of different options out 
there, but I think you need to be willing

to continually ask yourself questions 
about whether it's effective or not.

And I think that's true of all of these 
ways that we measure things and all 
of the goals that we set.

Now, people who are purely research 
based will tell you never can change

the questions because then you don't 
have like history and I tend to be a lot

more -- there may be some questions that 
you want a long history on, but I tend

to be a lot more of the camp of you need 
to be responsive to your folks and you

need to figure out what they want to 
know -- what you want to know and what

they want to tell you and find a way 
that works for you and that sometimes

means changing and 
tweaking it for a while.

So hopefully we will give each other  
ways to change and tweak.

All right, now, and I didn't ask.

Do you have any other comments 
or questions from that table?

All right.

This table over here.

AUDIENCE MEMBER: My name is Carl Berry.

I am from Community Resources in 
Pennsylvania.

In the interest of time, you've pretty 
much heard most of our discussion –

PAULA MCELWEE: Oh you have time.

Because otherwise I will 
get done too early for lunch.

AUDIENCE MEMBER: Then I will 
drag this out really slowly.
[LAUGHTER]

Pretty much our take away with our group 
was our definition that we were presented

with is quality is meeting 
consumer expectations.

Quality is satisfying the consumer, 
the consumer defines quality.

Do we agree with that definition?  
In part.

The take away we took from this 
is quality for us and for our centers 
is a multilayered thing.

First and foremost, quality is meeting 
the consumer expectations, certainly.

That's -- I will give you an 
example that our table presented.

Consumer calls and says I need food.

Okay.

We can provide them with food 
or we can point them in the 
direction of where they can get food.

We can measure that by, 
did the consumer get food.

We can measure that by, are you satisfied 
with your experience of getting food.

But that doesn't necessarily meet with 
our goal as a center for independent living.

Our goals across the table is 
to create self-sufficiency.

So we felt that this definition is a little bit low.

We would also want to have 
another goal that says did 
we create self-sufficiency,

independence in our consumer through this 
experience, can we measure that as well?

Certainly, we can, but to get back to 
our definition quality as meeting

consumer expectations, we felt that was 
rather -- that was only part of it.

We gave that person a fish, 
but we want to,

as you would imagine, 
teach that person how to fish.

AUDIENCE MEMBER: I am Nicole Craig, 
the program director at Disability 
Resource Center in Knoxville, Tennessee.

and our definition was quality is
exceeding the customer expectations.

Quality is the extent to which 
customers or users believe

the product or services surpasses 
needs or expectations.

Quality is delighting the customer.
And this has been answered over and

over and we agree with the definition.

Something that we would like to add 
to on how do you demonstrate this quality, 
we looked at the behaviors.

What did the people in the center do.

It's answering the phone with a smile.

It's smiling when someone 
walks in the door.

It's treating the person like they are 
a paying customer at your facility.

It's building trust and rapport.

It is sitting down with the person 
and establishing expectations, 
not only theirs, but also ours.

PAULA MCELWEE: And if all of you would 
do this for me -- I will ask if there are questions 
and other comments in a second.

Put that piece of paper in the center 
with the green sheets of paper where your 
notes were from the conversations.

We will put that together in a document 
because I think this is some really good 
information to be thinking about.

Other comments or questions from this 
table?  Anybody else?

AUDIENCE MEMBER: We also looked at – 
we are trying to make a better, nicer 
environment.

We want to look inviting, not like V.R. 
looks, like the institutional look.

Something warm and welcoming and we 
thought, gosh, could we have a pot

of soup cooking during that day 
with the soup smell in the air and just --

PAULA MCELWEE: Not if you pay for it with 
any of the federal money.
[LAUGHTER]

AUDIENCE MEMBER: All right.

Resource development funds.

Resource development and there 
are places that donate, too.

PAULA MCELWEE: I am just kidding.

AUDIENCE MEMBER: I know you are going to 
tell on me, I can tell.

We will get an audit next year?

PAULA MCELWEE: I won't tell on you 
but who knows who will get the audit 
next year, there is a lot of them.

Any other comments or observations about this 
before I pull it all together?  Right here.

AUDIENCE MEMBER: Hi, I used to be 
in the private sector and there

is a fabulous little book 
called "Raving Fans."

And it's a very quick read and I think 
it's by Ken Blanchard and it talks about

customer service and how to make raving 
fans out of the people you serve and so we

will be sharing that with our CIL and 
making it a mandatory book read and study.

Cool.

PAULA MCELWEE: Will you identify yourself, 
please?

AUDIENCE MEMBER: Yes, my name is 
Marylou Dycstra with Disability

Advocates of Kent County in the
Grand Rapids, Michigan area

and it is "Raving Fans" and 
I believe it is by Ken Blanchard.

PAULA MCELWEE: Okay.

Any other questions or questions 
from this conversation?

So if we pull this all together it 
seems to me that most of these 
definitions for quality were

not complete by themselves, right, 
and as you looked at them, you brought

to the table your own experience and 
your own ideas about what quality would

mean for your centers and you 
began to put those out there.

Some really interesting ideas popped up 
through that and I hope you kind of hold

on to those and like I said, I will put 
a document together that we can post on

the wiki page related to that but if you 
look at the different ideas around how

to define quality, you redefined
it for yourselves a lot.

In some cases you put practical ideas 
behind the definition, so you are

thinking about having hot soup for – 
I do know of a center, however,

that was cited for having -- for not --

Hot soup?

PAULA MCELWEE: No, it was coffee.

Now the issue was, do you determine the 
eligibility of the people you serve?

And the people that came in 
were not determined eligible ...

It is not I&R, you have to determine 
eligibility for everything except I&R.

And we will talk about everything 
when we get to the compliance piece.

An interesting conversation, though and 
I get to visit a lot of different centers.

You see very many different approaches 
to what a center looks like,

and that's an interesting conversation 
to have, how does what we look like to

the person coming in the door give them 
from the first moment an impression 
about us and about our quality.

Interesting conversation.

Some of you had really solid, concrete 
ways in which you would change the 
definition for your own situation.

In all of those situations, as we look 
at how we define quality, we can then

drill down to how we measure 
it from those definitions.

So I think that's a 
very useful conversation.

Does anybody have anything 
else they would like to say 
about defining quality for yourself?

We can let the presenters give a quick 
preview if they want to speak or if any of 
the rest of you want to say about that?

Because I am not supposed to quit for 
another half hour so I will start on this

afternoon's schedule if we don't have 
other comments.

Yes, back here.

AUDIENCE MEMBER: You missed some people.

PAULA MCELWEE: I missed some guys.

I don't know why -- you look like one 
table from this weird perspective.

They were not going to let you by with it either.
[LAUGHTER]
All right.

Let's hear from this table right here 
and then we will see if somebody 
else has something else to add.

Tell us what you had guys.

I am from the Midwest and 
use "guys" not meaning any gender.

AUDIENCE MEMBER: My name is Terry.

I work for Q90 corporation which 
produces the CIL Suite software.

And our definition was quality 
is meeting customers expectations

We had some of the same 
reservations as has been

expressed by other tables
about the expectations

possibly not being in alignment with what 
we are actually able to do for them or

what would help to foster increased 
independence on their part so it's

kind of a qualified agreement but 
we did have some reservations.

And in terms of how we measure 
that -- whether or not you met that

quality expectation, the major option that we 
came up with is the satisfaction surveys,

although we also agree with 
some suggestions from other 
tables, about also looking at goals

that were achieved and the reasons the 
records were being closed and so forth to

get more extensive than 
yet originally considered.

PAULA MCELWEE: All right.

Excellent.

I am so sorry, you guys should 
have thrown something at me.

Raise your hand and jump up and down, 
okay, so as we look at all of these

definitions of quality, we 
are building that framework.

Do any of our presenters have a comment.

Maureen?

AUDIENCE MEMBER: I was just thinking about 
the term "customer" and sometimes our 
customers are not people with disabilities 
as the consumer perspective.

It can also be the community.

It can also be an architectural 
assessment, things like that and

I wonder if people measure quality 
differently for those kinds of

customers versus the people customer 
or do we -- do people expect the same

kinds of things across the board, 
timeliness, respect.

It's an interesting thing to think about.

Do you use other tools, so do people 
send out satisfaction surveys with the 
same questions to non consumers?

I don't know if we are going to talk about 
that or if Anne is going to talk about

that but there is a lot of work done 
in the community that we all should 
somehow be looking at.

PAULA MCELWEE: That's a very important
it's a very important measure because how

we impact the community and how they see 
us has direct relationship, I believe,

to how responsive that community is to 
the needs of the people that we serve,

how responsive are they to the rights of 
people with disabilities and we sometimes

are the face they see, so I think 
we have a direct effect there.

Yes, that's a good point.

So who are your stakeholders?

You know, it might be people beyond 
just the persons that receive services.

Who are the other people you 
have an impact on as a center.
Good point.

Comment back there?

AUDIENCE MEMBER: Yes, maybe you can help me.

I am Patrick from Pouhkeepsie, 
New York Independent Living Center.

In question 2, quality is measured 
by customer expectation.

Quality is measured by the consumer, 
and then the next bullet is,

the customer defines quality, to me, 
I see they are two different questions,

because we spoke among ourselves because 
some of us said that the consumer wanted

us to do, we do not have the resources, 
and for example, kind of an example

some of the consumers would 
like to fly to the moon, and we -- but,

again, I  want 
to fly to the moon.

Therefore, does it define a quality 
or the independent living specialist?

Okay, let's work on that, then.

Define for me how you are 
going to get to the moon.

[LAUGHTER]

But there is quality -- quality of aspects 
of this, of how a consumer is defining how

to get to the moon, what kind of 
education, what do you need to do,

and then are you satisfied of the quality 
of service because I have been defining it

but they want to go to the moon and I need 
help, and there is a battle of maybe two

different questions because if I am 
puzzled, I think somewhere there is

some kind of cognitive disability that 
defines that question for themselves.

Because they say, I didn't go 
to the moon, I'm not happy.

Are you following me?

PAULA MCELWEE: Yeah, I do follow you.

We're not going to go a lot into the 
intake process, but let me drill down

a little to what you've just asked because 
I think it shows something that is another

important thing to remember 
when you get back home.

One of the things you need to remember 
when you get back home is that whatever

we come up with here, and then when you 
get back with your board and your staff

and the consumers and you develop your 
own quality measures, it's going to have

an impact on a lot of your other 
processes, your forms, how you 
train people, all kinds of things.

So let's take that intake, for example.

We say one of the things that we want 
to do is meet the expectations of this 
customer.

But I notice this group back here added 
a qualifier, informed expectations 
of the customer.

And I thought, boy, that solves some 
issues because then you're having 
more of a conversation.

So I'm not just saying what do you want, 
I'm getting to know you.

And I'm helping you to understand us.

And we have something going on there at 
the intake point that is more than just

filling out an application 
and checking some boxes.

And it's more than just 
telling you what I want.

It's also telling you what we provide 
and it's also helping you to see how

that might fit, asking the right 
questions, having that dialogue.

So suddenly it's changed -- could have 
a potential, if you're not already doing

that, to change that entire intake 
process so that you can get to where you 
want to get related to measuring quality.

Does that make sense?

So you're talking about something that 
will be changed and have an impact all

the way back to the beginning of that 
person's work with you as a center,

all the way back to the beginning.

And that's a big -- that's a big thing 
to remember, when you put in place these

things that will help you to identify and 
measure quality, it's not just a one-time

thing, it's a constant then of looking 
at how that impacts all the different

processes throughout your organization 
and how that can become a way that all

of us together know what we are 
measuring and are striving for a 
certain goal related to that.

And the conversation about going to 
the moon, I think, you know,

you might stop at the current prices, 
you know, ten million dollars, and

when you win the lottery, come back 
and we'll work with you on that.
[Laughter].
No, just kidding.

But do you see what I'm saying?

It has an impact that threads all the way 
through your different processes and your

different -- how you train staff so that 
they know how to do that, so that you get

to the outcomes that you want, 
everything is impacted.

Does that -- I think that kind of hits it.

We wanted to, you know, just talk about 
and wrap up this particular topic,

and I'm running a little ahead so I'm 
going to go ahead and give you a preview

of this afternoon's topic before we 
break for lunch.

But we really do have 
to do this ourselves.

It is not something that's external that 
we can hand to you in your packet and you

know exactly how to do 
it when you get home.

We will give you the actual outcome 
measures that NCIL has developed

through their taskforce, but that's 
only a starting point.

You have to decide at your local 
level what quality means to you.

And once you've decided that, 
there's a lot of work that you 
have to do in order to achieve it.

So the organization has to be driven 
from within for a desire for quality.

We believe that you're here 
because that's how you feel.

As we talked to some people as you came 
in or as we visited with you when you

asked about the course, we could really 
sense that, that there are people who

are -- who are tired of just the same-old, 
same-old, and they want to take their

center to another level, and they see 
this as a tool to accomplishing that,

and that's what we believe it is as well.

We believe that we can work together 
to take this whole realm of CILs 
across the country to another level.

So it's clear what we expect and 
it's clear what quality is to us,

and then we can make a difference 
in our communities in a way that

mediocre centers will never do.