ANNE WEEKS: I am 
the President and CEO of

Mountain State Centers 
For Independent Living.

We were the first center in 
West Virginia and we opened

the center in Huntington 
October 1, 1981, and then we

competed in one of the national 
competitions and opened up our

second center in June of 1988.

What I'm gonna talk to you 
about primarily today is our

process with gathering data, 
how we gather data,

our strategic planning process, 
and then our work plan and how

we use that to all feed into 
our consumer feedback.

There are several ways 
that we get outcome.

Or get input from consumers.

The first one here is a 
consumer request,

consumer request outcome 
feedback form.

And the form, 
I understand, 
is on the Wiki.

You, I have in here just 
a text version of it that I'll 
show in a couple minutes.

But this outcome feedback form 
is a consumer request outcome

feedback form that we use 
after every event, activity,

training, skills training 
class, et cetera, to get

feedback from the 
consumers about what 
they did and learned.

And I'll talk a little 
bit more about that.

We have comment cards.

We have them in the vans.

We have them in the centers.

They're always available 
for the consumers to use.

And the consumers pretty much 
have good staff access.

Sometimes it's a matter of just 
feeling comfortable enough to

talk to a staff and saying 
"I've got a problem with this,"

or "I would like to see this."

And so they have that 
opportunity to feel

comfortable in the fact that 
we'll listen to their feedback.

We also do annual consumer 
forums or open space exercises.

Anybody in here familiar 
with open space?

Okay.

I'll mention that just briefly.

A little bit.

And then there is a statewide 
consumer satisfaction survey

that is done in cooperation 
with the Division of Rehab

Services, the Statewide 
Independent Living Council,

and the Centers for 
Independence Living.

In fact, we were on a 
teleconference last week

because we were looking at 
the survey and changing it.

And true to form, as to what 
was said the other day,

researchers say, "Well, you 
have to have a history.

And if you change the questions 
you don't have that history."

But we've used the same 
questions for about six years,

and so she agreed that they 
felt that that was a good

history, because it 
was pretty stable.

So we're working on 
changing the questions.

But the current questions that 
we ask are also on the Wiki so

you can see what questions are 
on our satisfaction survey.

The Outcome Feedback Form, 
actually it's called

Request/Outcome Feedback Form, 
but we shorten it, came out

of a site review that we had 
from RSA back in 1997, and they

said, this is from the 
regional office, then.

They said that they wanted 
to be able to look at CSRs

and they wanted to be able to 
know what the consumer

requested, did they get what 
they requested, and were they

satisfied with the service 
that they received.

So instead, we start thinking 
about this, well, do we want

the staff person to say that so 
and so participated in this

activity and seemed to 
do well in this class?

We decided we wanted it from 
the consumer, because that's 
more consumer driven.

So we came up with this 
form and the procedure is,

that when people sign up for a 
class, that the staff member,

after the sign-up deadline, 
pulls the names of the people

who are in the class, and she 
goes ahead, he or she goes

ahead and fills out the 
consumer's name and their

date of birth and the, their 
name as being in charge,

and the date of the 
event or the class.

And the reason why she or he 
does that is because if the

consumer does not show up, 
or if the consumer calls

and says I'm sick, then they 
didn't get what they requested.

But we have, they were supposed 
to be here, they didn't get

what they requested, 
but then the staff write in

"they called in sick" or 
"they just didn't show up,"

so we can also track that.

And like I said, the consumer 
form is on the Wiki.

On that form we just have 
simple questions that we want

the consumers to answer.

And one of them was:  You 
requested to attend,

and we wanted in their own 
words what were they thinking

that they were coming to today.

What were they doing today.

And then we asked them:  
Did you accomplish what 
you wanted to do?

"Yes" or "No."

And regardless of what they 
say, yes or no, we want them

to explain more and 
tell us about it.

Then we also always ask:  What 
other classes, activities,

and things would you like 
to see at the center?

So everything that we do, 
like we go to the legislature,

they visit with legislators, 
they fill out an outcome 
feedback form.

They go to a training, that 
the Fair Shake Network put

on the legislative process, 
we didn't conduct that,

but they still fill out an 
outcome feedback form.

But if they come to a class 
on budgeting and financial

management at the center, 
they still fill out an

outcome feedback form.

And I will say that consumers 
who come to a lotta things

gripe and grumble about 
having to fill out this form,

and I always tell the staff 
you can just blame it on me

because, you know, 
"Anne's making us do it

because the feds are making 
her do it," and so. . . .

So sometimes we get good 
answers, and sometimes

it's just, "I gotta fill this 
out so it's not . . ."

But sometimes we get some very 
helpful information there.

The form also lets me know, 
sometimes, if the consumers

are honest, just what they 
thought about that class.

And, maybe they didn't 
particularly, they thought

the class was not what they 
wanted it to be, but maybe

particularly they had a problem 
with Kathy and the way she was

teaching the class or she was 
not as patient as she should

have been or whatever, and 
so sometimes they'll put 
that down and stuff.

And so that gives me 
information about, you know,

talking about what happened 
and so forth and so on.

But it gives them an 
opportunity to provide

as much information as they 
choose or as little as they

choose, unfortunately 
sometimes, about the

class or the activity, 
what they did, and other

things that they 
would like to see.

The comment cards that we have, 
we have at the center.

They're at the front 
of the lobbies.

So that the front-line staff 
can give them to them.

And they're also on the vans.

Because a lotta times it's 
the van drivers who hear 
what the problem is.

Huntington has a public 
transportation system,

and they also have 
Dial-a-Ride.

And so our policy in Huntington 
is if you live on the bus

route, are eligible for Dial-a-
Ride, we'll teach you how

to ride that, but you need to 
get to the center on your own

in order to come to classes.

Beckley does not have public 
transportation, so our van

brings consumers into the 
center for whatever activities

that they participate in, 
because there is no 
public transportation.

The consumers can complete 
those at any time.

And they come to me, 
and I always look at them,

and never divulge 
who said what.

And then we talk about them in 
the staff meeting and then,

we also can talk about them 
with the board if it happens

to be something that would 
be a significant issue that

we needed to talk about.

And then I've already 
mentioned the 
staff access.

The consumers feeling 
comfortable enough 
to talk to them.

We also have a calendar that we 
put out every month that lists

classes and activities 
that are on there.

And we do a quarterly 
planning meeting to kind

of preliminarily schedule 
the calendar for the next 
three months.

In fact, they just had one 
for April, May, and June.

So they come to this meeting 
because it's consumer driven

and they tell us the types of 
things that they want on there.

Now, we may put stuff on there 
like when we had disability 
advocacy day at the capital,

we already had that on the 
February calendar and that.

But we put on there the things 
that the consumers want.

If they complain that there's 
nothing on there for them,

they're always invited to come 
to the meeting so they can make 
sure something is on there.

I have one fella used to come 
to me month after month and

say, "There's nothing 
on there for me.

There's nothing on there."

I kept saying, "Come to the 
meetings so you can put the

kind of stuff on there that 
you want on there."

A consumer forum, what we do 
is we send out an email blast

to our email list, which is 
about 800 people, and

businesses and so forth, and we 
send, put it on the consumer

calendars, and basically it's 
just setting a time and,

having people come in and 
whoever comes, comes.

And then we write down 
everything that's said,

we don't, there's no filtering.

There's no discussion.

And we just compile 
that as raw data.

Open space is a process that 
you can go through where you

decide on a theme for the day 
and then you allow the group

to self-manage what 
they wanna do.

And so if you decide the theme 
is going to be on, outcomes,

you can say that's the theme, 
and then what you do is you

allow people to come and say, 
"I would like to talk about

how do you develop outcomes."

And they post that on the wall.

And then they're all 
given a time schedule.

And so people go to the topic 
that they want to go to.

And there might be three or 
four topics during the day,

depending on how many staff, 
or how many of our participants 
are there.

One of the neat things about 
open space is that it has some

significant rules and 
principles, but one of them

is that whoever shows up are 
the right people.

When it starts, it starts.

And when it's over, it's over.

But the good thing about 
it is that it's totally 
stakeholder driven.

Because if you're not 
interested in knowing

how to measure outcomes, 
but you want to go talk

about how outcomes are gonna 
impact the 704 report because

that's another topic somebody 
put up there, then you go 
over there.

And so it's basically allowing 
people to have their freedom

based on topics, and 
anybody can post a topic.

Sometimes there might 
be one person that shows up,

and sometimes the person 
who wanted the topic gets

disappointed and we try to 
remind them whoever shows 
up are the right people.

So go ahead and do your thing.

Again, we write down all the 
data and we collect it and

it's another form of 
data that we use.

We do not do an open space 
the same year we do a consumer

forum to gather information.

The Statewide Consumer 
Satisfaction Survey is

all the centers provide the 
Statewide Independent Living

Council on a quarterly basis 
with information about open 
and closed cases.

Who was open this quarter.

Who was closed this quarter.

They send out the surveys, and 
they are sent to the evaluator,

who is up at West Virginia 
University, and she is the one

who compiles the data 
and about every February 
the report is ready.

Which is a statewide report.

Plus she also takes out, they 
color-code them, so that we

know which center 
they came from.

So they'll send each center 
a compilation of what the

evaluation was for that center.

And one of the things we 
started doing about three years

ago is we started having her 
call randomly 50 people.

And so sometimes it takes her a 
long time to get to 50 people,

because people don't 
remember being there, people 
don't remember services.

Particularly with the Community 
Living Services Program,

because we serve a lot 
of elderly people.

And they'll go, "Well, honey, 
there were some really nice 
people there."

And they don't necessarily 
remember why they were there.

But she gets to 50 people, 
and engages them in

conversation more about 
whatever topic that we 
say is important.

And like when we had the 
discussion last week, we want

her this year to concentrate 
more on consumer control.

Do they feel like they have 
more consumer control over 
their lives?

Is their life being improved 
by what they are getting,

so that's another element 
of that survey.

When we have gathered data, 
we develop a ten-year strategic

plan and our last strategic 
plan was developed in 2011,

so it runs from 2011 to 2021.

And we were fortunate this time 
that it was about the time that

the ARRA money came out and 
I put in my budget staff

development money and we took 
staff members and board members

who could go to a state park in 
the middle of West Virginia and

we spent four days up there 
looking at all of the data and

hashing out a ten-year 
strategic plan, and we

identified the specific goals 
that we are going to put on

the strategic plan, and 
I attend a couple of my,

me and my staff person attended 
the outcome measures report

workshop they did in Portland, 
which Tim is telling me last

night was in 2010 and I can't 
believe it was that long ago

but we know that it, we know 
that something is coming,

that outcomes somehow is coming 
and so we are trying to get the

staff used to thinking about 
outcomes so we are not totally

doing outcomes like the task 
force has identified and like

the test groups did, but we are 
talking about it and using that

so the staff will get 
used to using that.

When they took, when the 
staff and the board took all

the raw data, and let me 
say one other thing.

This is where in developing the 
strategic plan, that we include 
the stakeholders.

We invite other agencies.

We invite contractors that 
we work with, vendors that 
we work with.

The librarian down the street.

Anybody who feels they have 
a stake in something that the

center does, we invite them 
to come to one of the consumer

forums that we use to develop 
some of the raw data when we 
are doing the strategic plan.

So it doesn't happen every year 
but it does happen when we are 
doing the strategic plan.

And we also share the data with 
the consumers because sometimes

staff are feeling, well, the 
consumer goes out there in

a dark hole and whatever 
happens to it or something

out there and they always have 
access to the data so they know

we are paying attention 
to what is being said.

Then we look at the 
overall state wide consumer

satisfaction survey and then we 
also look at the Mountain State

portion of the satisfaction 
survey and then, based on all

of this data that you've got 
so many in your head, we had

to hash out, prioritize what 
we wanted to do in our next 
ten-year strategic plan

The strategic plan that 
was developed for 2011-
2021 has these goals:

Grow to meet identified needs 
in our communities, maintain

highest standards, obtain 
and retain strong base

of human resources including 
staff, board, and volunteers.

Achieve and maintain 
fiscal responsibility,

increase awareness of 
Mountain State's philosophy

and programs, and 
resource development.

Those are the goals that we 
have set for the strategic

plan and those goals will 
never change for that ten-year

period, so every year when we 
do our annual work plan that

we submit with our 704 report, 
we are still looking at those

goals and those goals stay 
the same every year.

But we develop outcomes 
and indicators for each 
one of those goals.

The outcomes also do not change 
during that ten-year period but

the indicators do based upon 
accomplishments that we have

achieved, what that indicator 
has said, based upon new data

from a new year's consumer 
forum, and things that may

need to be changed because of 
the reality is there that it's

not going to be met, 
so it can change.

Activities, however, we put 
activities to go towards the

indicators, those are very 
fluid and they can change every

year and, again, it is based 
if you accomplished something,

that's what you are putting 
activities in there for, if you

accomplish the activities, then 
obviously you need to change them.

And, again, if it's 
something that is not valid

or not reasonable, and so 
forth, then we take a look

at those, and those do and can 
change every year although some 
times it's an ongoing thing.

For one thing, we have a safety 
committee, and so the safety 
committee is an ongoing thing.

They may have worked on a 
safety policy the first year

or so but they are still 
there so they can review that 
on an ongoing basis.

And our work plan that was 
submitted with the 704 report 
this year is on wiki, too.

We include that work plan 
every year in the 704 report

and stuff, but one of the 
things that we do, and the

quarterly report, when we 
make it, it's in a table and

we have everything done in 
a table and we put down like

which staff are assigned 
to it and so forth.

When you're putting stuff on 
the 704 report, it has to be 
the straight text version.

But we have a quarterly review 
and at the first month of the

next quarter at a staff 
meeting, we go over the

quarter, we go over the 
work plan and we look and 
see where we are.

Sometimes it's a matter of, 
like, oh, gosh, I forgot I

was supposed to be working on 
that and sometimes it's like,

well, we did do this or we 
didn't do this or we are

working on this, just so that 
it continually brings up,

this is what we said we were 
going to do in the work plan

and so where are we with this, 
so that we are not hit at the

end of the fiscal year with the 
work plan that may not have

a lot of stuff done because 
everybody got busy doing daily

things that you end up doing 
and forgot about the specific

things in the work plan that 
they said they were going 
to be working on.

AUDIENCE MEMBER: Anne, 
are all three of these

West Virginia centers doing 
the same ten-year plan?

ANNE WEEKS: No, we have two 
centers, one in Huntington and

one in Beckley and those are my 
two centers and we do that.

The other Center is in 
Charleston, Appalachian

Center for Indpendent 
Living and the other

one is up in Morgantown 
and it's the Northern West

Virginia Center and 
they do their own thing.

We have informal group called 
West Virginia Centers for 
Independent Living.

We are not incorporated.

We don't have staff or 
anything but we do get

together on a regular basis to 
talk about things but they do

their own thing when it comes 
to developing work plans.

AUDIENCE MEMBER: So every year 
during the ten years, you go

back over your priorities, 
do you re-shift your priorities

due to the activities 
that have happened?

ANNE WEEKS:  Yes.
Yes.

And we also use the goals that 
are in the ten-year strategic

plan as the agenda that 
drives the board meetings,

so that when we put on there 
about improving staff and board

and volunteers, if we're 
going to have any training, 
we put that in there.

Or to be responsive to 
community needs, we have

consumer advisory groups and 
they have a report and it

goes under there so we don't 
just have an agenda that says

old business, new 
business or whatever.

We do it based on 
what the six goals are 
from the strategic plan.

AUDIENCE MEMBER: A ten-year 
plan sounds like a long 
range thing.

I don't see the point of doing 
that instead of 3 or 5.

ANNE WEEKS:  Well, 
when we started it, which was

ten years before that, it was 
because we were involved with

some strategic planning that 
the centers did, the SILC did,

and the Division of Rehab 
Services did and that's what

they were striving for, 
was a ten-year plan.

It does seem like it's a long 
period, and there is a lot of

stuff that can change but for 
the most part, that's why we

think that our goals for the 
strategic plan are pretty broad

so that you can come up with a 
lot of indicators that could

fit under any one of those.

The one thing about this 
current plan, which ends

in 2021, is that I kept telling 
them you need to keep thinking

about and putting goals in 
there for the fact that I am

not going to be here forever 
and I am planning on leaving

and retiring in 2021 and they 
just ignored it, so anyway,

so that's something that they 
are going to have to deal with,

that they will have to come 
back and put an indicator

in and an outcome in under 
maintaining staff and board 
and volunteers.

Yes.

AUDIENCE MEMBER:  So you've got 
all of these different formats

that you are collecting the 
consumer feedback through the

cards and the surveys and 
all of those things, are

you kind of looking at those 
as they are coming in?

How are you, are you compiling 
all of those different pieces

all together on 
a monthly basis?

How are you 
incorporating them in?

ANNE WEEKS:  If we get a 
comment card, the comment cards

come to me and I look at them 
as soon as I get them and if

it's something that is very 
significant, that I feel I need

to take the board, then I hold 
it to the next board meeting.

If it's something that I feel 
I just need to share with

staff, then at the next staff 
meeting, we will discuss it.

The outcome feedback forms 
happen every time there is

an event and so the staff 
person who is in charge of

that will review those and 
see what they say and, again,

if there are some significant 
things, like somebody may have

come up with a real ah-ha for 
a class or some kind of

training or something, 
then they will share that 
at the next meeting.

The consumer satisfaction 
survey that we do with the

SILC, it's done once a year 
even though we submit quarterly 
information for that.

AUDIENCE MEMBER: Is there a 
time where you are looking

at all of those 
things together?

ANNE WEEKS:  Yes, when we do, 
when we look at, we also do,

the consumer like this year we 
did a consumer forum in August

and so we had all 
of that data typed up.

We had information from the 
outcome feedback forms and

from comment cards that we 
might have had and we have all

of that data and then the 
staff and some board members

go through that data and look 
at, how will this fit into our

work plan for the upcoming year 
and so that data is all looked

at before we write the work 
plan for the upcoming year.

AUDIENCE MEMBER: One last part.

ANNE WEEKS:  Sure.

AUDIENCE MEMBER: When you do 
the outcome survey for the

groups or programs, we do 
a lot of groups, like a series

that would be for like four 
weeks or so, working on 
advocacy skills like that.

Are your groups more individual 
class or programs or do you do 
some in series?

I am wondering if you do them 
at each session or the first

time or the last time or what 
experience you have with that?

ANNE WEEKS:  We do them every 
time and we do have things

that are done like, you know, 
maybe four-week sessions.

Now we've been doing 
pretest/post-test

for years so the pretest we 
will do in the first class

and the post-test won't be 
done until the last class,

but as far as the outcome 
feedback form, that's done 
every time they get together.

In fact, the secretary has 
taken outcome feedback forms

and created a separate file for 
the consumers because it just

made their files, their CSRs 
so large so anyway, we have

dual files for people 
because of that.

Yes.

AUDIENCE MEMBER: On your 
Request/Outcome Feedback

Form, I am curious, you 
asked for date of birth.

Is there a particular 
reason for that?

ANNE WEEKS: Just because 
that's how we were identifying

who they were, when some of 
the people have similar names.

AUDIENCE MEMBER: Okay.

ANNE WEEKS:  Any other 
questions?

One thing I will say about 
the Outcome Feedback Form,

on the bottom of the form, 
if you look at it, you will see

a bunch of numbers, or a bunch 
of letters blocks of letters,

and those are the 704 service 
areas, and so whenever the

staff person is finished with 
whatever class and collects

these, they are responsible for 
marking which service area that

goes to and stuff and then 
that's also where they make

a comment that if a person 
didn't show up for class,

didn't call, or whether they 
came to, or they called in

sick, whatever, that's also 
where we make those comments.

AUDIENCE MEMBER: I am just 
curious why you're collecting 
medical information.

It kind of rings 
to the medical model?

ANNE WEEKS:  This here?

The medical information?

AUDIENCE MEMBER: Yes you 
are collecting, you are updating 
medical records.

ANNE WEEKS:  We do that just 
for purposes of safety when

we've had to call 911 and need 
to know medical information as

far as medicines and 
those types of stuff.

Because it's been a, 
it's been an issue at times,

and so that's why we do that.

AUDIENCE MEMBER:  And 
that is optional?

ANNE WEEKS:  Yes.
Oh, yeah.
They can tell us or not tell us.