IL & Outcome Measurement:  What You Need To Know.
Part 2  March 14, 2007
Presenter:  Mike Hendricks.

TIM FUCHS:  Good afternoon and thanks for joining us for part 2 of IL NET's 
national teleconference and webcast, independent living and outcome measurement:  
What you need to know.

If you were on Monday's call, and I know a lot of you were, excuse me for going 
through this piece again but I've got to do the housekeeping at the beginning, a 
little bit about our training and our training program.

Again I'm Tim Fuchs, operations director at the NCIL words and as part of this 
position I oversee NCIL's training and technical assistance activities for the 
IL NET.  If you're not familiar with IL NET it's a collaborative project of 
ILRU, Independent Living Research Utilization in Houston, Texas and nation all 
council on independent living in Washington D. C.

We're a national training and technical assistance project, funded by RSA.  
Starting to strengthen the independent living movement by supporting centers for 
independent living and statewide independent living councils.

Our trainer today for part 2 is again Mike Hendricks and a bit about Mike.  
Mike's a program evaluation guru and having provided technical assistance to 
governments at all levels, nonprofits and other community organizations, but 
what's most important to us today is Mike's role as the facilitator in outcomes 
measure expert to NCIL's outcome measures tasks force.  I want to thank Mike for 
being here today and for prepaying for these two presentations.

Also on the line with us today is Bob Michaels, chair of the outcome measures 
task force and my thanks as well goes to Bob for his help preparing for the call 
and being with us today.

And before I turn it over to Mike, I want to again remind everyone where they 
can access both the training materials and evaluation forms for the call, those 
are located on the westbound at www.ncil.org/outcomes.html.

I'll be repeating that web address before the end of the call. The manual that's 
located at that web address is vital to following along with today's 
presentation.  And also on there are the evaluation forms which is equally vital 
to us.  The evaluation form you only need to fill out one for both calls, and it 
should only take three to five minutes to fill out and instructions for 
returning it to the NCIL office are included on the form.

I would appreciate it very much if you would take a minute to do so. And without 
further ado, let's begin part 2 and I'll turn it over to Mike.

MIKE:  Thanks, Tim.  Hi, everybody.  Nice to be back here again.  I certainly 
enjoyed Monday.  It turns out there were lot of us on our call on Monday, so I 
think there are a lot of us again today. So it's obviously a topic that a lot of 
us care about and I'm glad to be talking with you again.

We're going to do three things today.  Today we're going to have a lot less 
talking from me and we're going to have a lot more discussion among ourselves, 
if you will; but here are the three things we're going the try accomplish.  The 
first one, and I'm going on ask your indulgence on this, we'll have a little 
brief review of what we did on Monday.  And don't worry, when I say brief, I 
mean brief, but there are people joining us for the first time and I'll just 
quickly walk through some of the pages.  I would guess five or six minutes to do 
that if that's all right with you and then we'll have questions and answers 
because there may be some questions and answers from the new people and there 
may well be some questions and answers from those of us who were here Monday and 
if you're like me, suddenly in the sow weer this morning something came to your 
mind and you want to ask it. So we'll have good time for that.

The second thing we'll do, we really want to explain and show you and walk 
through the products of the task force, the outcomes measures task force.  There 
have been some products and we need to -- this is a good chance to show them to 
you and just talk about them.

And then third, and in some ways maybe -- well I guess they are all important, 
but certainly another important one -- third, we need your help.  And we're 
going to talk about exactly how we can get your help and how you can help us.  
So those will be the three things we cover today.  And as I say we'll have lots 
of times for questions and answers.

So indulge me for a few minutes if you will while we go through the training 
materials real quickly.  I think it's not only maybe good for the new people, 
but also let's -- if you're like me, you've been working on other things since 
Monday afternoon and it's probably good if we all get our heads back around this 
again in the same place before we look at some of our work products for 
independent living.

So I'm looking on Page ix of the training materials where it says training 
materials actually at the top.  And this is where we talked about how there were 
two forces pushing the independent living movement towards outcome.  One, 
internal, came from the CILs themselves.  You yourself have said for quite a few 
years you can't to be able to measure your outcomes.  The second one is nor 
external from OMB.  On Page 2 we have materials about the PART process showing 
that OMB has now looked at 977 federal programs and given each of them one of 
five different ratings and on Page 3 we see that they have in fact done what 
they call a PART.  They PARTed the CIL programs.  It falls under the category of 
not performing so this is not a place we want to be.

On Page 5 and 6 are the details of exactly why that happened, why the CIL 
program received that rating and we talked about the fact that a lot of it goes 
back to not having outcome measures.  Sort of a cascading effect from that and 
so we need to work on that.

And then on 7 and 8 and 9 and 10 we've got a little bit of comfort, I hope, in 
the fact that we're not alone in the Department of Education, these are actually 
pretty recent data.  It's only last month, February 5th, in fact, of just last 
month, showing that actually more than half of the programs in the Department of 
Education received results not demonstrated.  So we have definitely company.

But RSA is a savvy group of people and they know this is not an idea situation 
so on Page 11, you can read that they are putting outcome measures now in the 
704 report.  And I like the last line in fact on Page 11, it says ultimately the 
independent living program will be evaluated based on its outcomes.  And that's 
obviously why we're all on this call.  And on Page 12 here are the exact 
indicators, if you will, measures that RSA is using right now.  And we talk then 
on Page 13, 14 and 15 that Kelly Buckland, president of NCIL, appreciated the 
good work RSA had done, especially appreciates the collaboration, but had some 
quibbles to say the least with some of the indicators and wrote them a letter 
about that fact.

And on Page 16 and 17 and 18, RSA wrote back.  Again, having some quibbles with 
Kelly's points, but that's perfectly understandable, but the bottom line of both 
these letters is that both sides said let's do work together on this.  We're all 
in this together.  Let's collaborate.  So that's certainly the situation we're 
in right now.

And then 19 and 20, we talked about the fact that NCIL has created an outcomes 
measures task force.  Can you see the composition there on Page 20.  And Bob 
Michaels, who is on the call, is the task force chair, and this is his most 
recent report, update, if you will.

21, 22 and 23 are some pages also on the website and background -- background of 
the kinds of things we're talking about now, the little -- how do we say it -- 
little methodological introduction if you will to outcome measurement.

And then a page 24 we started making sure we're all on the same page when we're 
dealing with this topic.  It's not so good if one of us means one thing when 
they say the word outcome and another person means another thing.  We at least 
need to be using the same jargon, even if we disagree or agree, we need to make 
sure we're using the same terms in the same ways.  So we had a little mini 
session, if you will, and I stressed the point on Monday and I'll stress it 
again today, please humor me, that on Page 24 the reason we do this is a tool 
for ourselves to increase our effectiveness.  That's the main reason we should 
be doing it, I think.  And then if we do that, then we have automatically a 
story that we can tell to other people and communicate values.  So those to me 
is the order of priorities of why we're all doing this, I hope.

Page 25 we all know -- everyone knows that you just can't turn around these days 
without hearing about outcomes from one group or another.  And I mention on Page 
26 that in the nonprofit sector United Way of America is quite -- I say quite 
the leader in the nonprofit sector and this was a reference to a web page with 
some good materials for you.

Page 27 was I think we called this the way we used to think about ourselves.  We 
used to think that we took inputs and we did something with them.  Those were 
the activities, and then we had some outputs, the number of people we worked 
with, number of sessions we delivered, and here are examples on Page 27, number 
of classes taught, number of participants served, that's all still important, 
but it's not sufficient anymore.

Page 28 shows us that all those things on the left are still very important, but 
what really matters is that they lead to some good outcomes, right?  Benefits 
are changes for people, so together pages 27 and 28 describe the world we're all 
living in now and the way I think -- I think we need to think of our programs as 
pages 27 and 28 together.

29, were just some little tips to help us remember the differences between 
activity, output and outcome and then on 30 we had some examples of programs and 
possible outcomes.  And here I'd like to make a little apology, if you don't 
mind.  We got quite good feedback about the call on Monday.  We appreciate that 
very much, but a couple of people -- more than a couple said I would have really 
liked more examples about the IL world itself.

And I can understand that.  I really can.  I just like to explain why that -- 
why we didn't do that.  That was my decision, so you can blame me if you thought 
we should have done more, but I've just found over the years, and when I do 
training, that sometimes when you're introducing new concepts, if you can do it 
in a content-freeway, if you will, that is not use a specific content that a 
person is working in as part of the example, I've found that sometimes the first 
time around it can help train.  For instance, sometimes when I'm working with 
people who work on children's programs, I'll use examples from senior citizens 
programs.  So they won't get caught up in whether I'm right or wrong about the 
children's services and they'll pay attention more to the concepts.  So that was 
my decision.  Blame me for it, but there was a reason behind it.  I think you'll 
find that today we'll be talking very specifically about the IL world so 
hopefully that will balance some of that out.

On 31 we talked about the differences between outputs and outcomes and how 
that's a very important difference.  32, we started talking about the fact that 
as we all know outcomes don't happen all at once.  We do some good in some way 
and then that cascades into some other way.  And then on 33 and 34, and I 
understand from some of that you this was helpful, so I'm glad, on 33 and 34 I 
suggested that this way of thinking if/then is a useful thing to do and I like 
it myself. It helps me, too.

And then on 35, 36, we actually had some examples of logic models.  Again, they 
are not IL related, hopefully by following if/then as you work your way up these 
pages, hopefully you can see the logic.  That's why it's called a logic model.  
It's supposed to represent the logic or if you will the theory of the program.  
So hopefully that if/then helps you there.

Now, 37 we're going to come back to later when we talk about the logic model 
we've created right now, but 37 are the ways a logic model just by itself -- you 
remember no data what so heifer, just the piece of paper by itself -- how that 
could be useful.  And then on 38 we talked about indicators.  I think I 
mentioned that in my opinion even though I've been doing evaluation for, gosh, 
30 years now I guess, you cannot measure outcomes directly.  I certainly cannot.  
I'm incapable of measuring an outcome directly.  I need to have some indicator 
of that outcome.  Then can I go do some measurements.  So indicators are the 
specific things we measure, and we talked about that.

And on Page 39 here were some programs, possible outcomes, and possible 
indicators just to give you a sense of what I mean when I say the word 
indicator.

And then on Page 40 we talked about, very briefly, some of the key traits of a 
good indicator.  There obviously can be good indicators and not so good 
indicators and we want to have good ones. So I just said that SMART is a good 
way to be on indicators, specific, measurable, ambitious, realistic and timely.  
So thanks for indulging me there and let's take some Q. and A. now and see if 
people have questions about the kinds of things we did on Monday.

OPERATOR:  Thank you.  If anyone does have a question at this time, please press 
0, 1 on your telephone key pad.  You will be able to ask your question in the 
order it is received.  Once again, you may press 0, 1 on your telephone.  We do 
have two audio questions and the first one comes from Donna Yeager.

CALLER:  Yes, this is Ken Williams.  One of the things I want to ask that I 
didn't get a chance to ask yesterday, at the introductory question and answer 
session, how did the outcome team put together -- that's the first part of the 
question.  The second part is when I look at the names of people who are on the 
committee, why is there no identifier in terms of who they are and what they do 
in independent living or outside of independent living?

MIKE:  Ken, that's a perfectly good question and I'm going to let Bob Michaels 
answer it.  He's the chair of the task force.

BOB:  Sure, thanks, Ken.  What we did is we had put on a -- when RSA first came 
out with their attempt to address the PART issues and identified the 18 
transportation and health care areas as areas that we need to be looking at, we 
went ahead and we did a training.  IL NET did a training and they offered it 
around the country.  It was a training done by Gregg Newton and this training 
was done in -- it was first done in Portland and then it was done regionally in 
five other areas.

So when we got done with that training, people were saying where do we go from 
here?  And Kelly Buckland, who is the president of NCIL asked me if I would head 
up a task force to take a look at the development of indicators, of the 
measurable outcomes.  And so we sat down and tried to think of how we'd put 
together a task force.

So what we did is -- I was the head of the task force, and we said we wanted to 
come from the Rehab Act subcommittee that I chaired at NCIL, a couple of people 
that really know a lot about The 704 Report.  And so what we did is we put two 
people on from there, Steve Thovson and Ben Harville.  And then what we did is 
we took two other people, we wanted to get some representation from SILCs, and 
so we took two SILC directors who had expressed a lot of interest in this, Tina 
Treasure and Pat Puckett and then we had five other people who were at those 
regional trainings and we went back -- what we did is we went back to the people 
who put on those trainings and we said, okay, who really stood out in the 
training?  They asked good questions and all that kind of business?  And so we 
picked the five people they named and we brought them on to the task force as 
well.

So that's where the five people came.  In addition to that -- in addition to 
those ten then there was a couple of people -- John Lancaster from NCIL who is 
the Executive Director and Tim Fuchs is on staff there.  Richard Petty from ILRU 
and then Kelly Buckland from -- as the Executive Director of NCIL and so that 
kind of made up the work group.

In addition to that, then we had people from RSA and from OMB and Department of 
Education who participated.  So does that answer your question?

CALLER:  Yeah, and I guess the one little follow up is that since DSU's have a 
role in this process was there any thinking about having that segment of our 
community -- or of the community involved in that process at all?

BOB:  Well, we realized at the point of putting this together that there was no 
way -- you know, if we were really talking about having a task force of only ten 
people, there was no way we were going to really get representation of all the 
people who would be able to really contribute to this process.  So what we've 
tried to do is we've tried to open up the process so that whatever we come up 
with people have lots of opportunity to give input.  So you'll see that like we 
created a website up on the IL NET.  And we'll talk more about that in a little 
bit.

CALLER:  Okay.

BOB:  at the IL NET website so people constantly have a chance, and we've had 
input from people from the DSU's.  Ben Harville works at a DSU.

CALLER:  Okay.

BOB:  So he does represent them, but -- I don't know if he's on the task force 
as much as a representative of the DSU as he is as a representative of the NCIL 
subcommittee, but he does have that background.

CALLER:  Okay.  The second part of the question, I'm sorry, that wasn't answered 
is like why after people's names it wouldn't say a SILC member or so on and so 
forth so people can identify with that and know what areas are covered in the 
committee. Was that purposely done or just something that was omitted?

BOB:  Well, you'll notice if the first letter that was sent out, and I'm not 
sure if it is the one up on the site, it is on that website.  We identified 
where all the people came from.

CALLER:  Okay.

BOB:  Now, we didn't identify it by each person.  That's a real good suggestion, 
but we did identify somebody from the SILCs and so many from the committee and 
all that.

CALLER:  Okay.  Thanks.

OPERATOR:  Thank you.  The next question comes from [ INAUDIBLE ].

CALLER:  Hi, this is Mike from the disability network. If I could summarize what 
we're doing here, it sounds like being able to -- after you explain what CIL's 
do to be able to answer the question so what?  And then how do you know that you 
did it?  That's the indicator piece, but there is another question after that of 
so what again meaning I understand the need to justify if we want to grow 
additional dollars for the CILs, but are we in jeopardy if these goals are not 
achieved?  Will funding be cut?  What is the follow up step?

MIKE:  Again, let's turn to Bob Michaels for that question.

BOB:  You know, I'm not really in a position to say where this is going to go if 
we aren't able to do it.  You know, there seems to be a real sense in federal 
government from what Mike was saying on Monday that you better be performing, 
you know, and you need to show this somehow.  Hopefully OMB and Department of Ed 
and RSA is going to see we're serious about wanting to participate and I think 
it's one thing that has made us different from other people in that we have 
really taken the bull by the horns and come in here and said this is something 
that we want to do, not something -- even though it's being -- a lot of people 
are trying to figure out ways from being parted, we've been looking for ways to 
develop our outcome measures. So hopefully they're going to see we're serious 
about wanting to participate and be a part of this process and look favorably at 
that. Now, what people will do otherwise, I really can't -- I really can't tell 
you.  I wish I could.

CALLER:  Thank you.

OPERATOR:  Thank you.  Do we have any web questions at this time?

TIM:  No web questions at this time.  Thank you, Dawn.

OPERATOR:  Thank you.  We have reached our five-minute mark for the question 
period.  You can continue with the presentation?

MIKE:  if you have more questions, we can take a little more Q. and A. time 
today if you have more questions.

OPERATOR:  We do have one more question from Ann.

CALLER:  Hi, this is Susan web at the CIL in Arizona in Phoenix.  We're kind of 
confused about one thing.  All of our directors are sitting here in the room, 
and we do a lot of things at the CIL that might be considered innovative but 
don't necessarily fit the specific areas that we seem to be pushing through the 
indicators. How is this going to be designed in The 704 Report so that we really 
can shine and talk about the good stuff we do, but also talk about the things 
that we do that aren't, you know, the cut and paste stuff that someone else 
dictated that we do to meet our community's needs?

MIKE:  Susan, that's a good question, and if we can, can I defer that just a 
little bit, because later on in the call I think we'll be touching on some of 
that, although I think the answer at that time will be no one really knows that 
yet.  Are you willing to defer that for a minute, and if we don't cover it 
sufficiently, jump back in then?

CALLER:  Certainly, and we're also willing to help you figure it out.

MIKE:  That's what we want and need to hear.  Thank you.

OPERATOR:  Thank you.  There are no further audio questions at this time.

MIKE:  Okay, well, that's good then.  Thank you for those questions and for 
allowing me to go through that little review.

Why don't we now turn to page 41 because page 41 is a product from this outcomes 
task force.  You remember this is a task force that was created about a year ago 
now, with a dozen people as Bob mentioned.  There have been a couple of meetings 
in person, there have been phone calls and E-mails.  One of the early products 
they created was that letter on Page 13 to 15 and this is another of the 
products.

Now, what this is -- I would call this a working logic model.  I wouldn't say 
it's a draft.  In my opinion, it's beyond draft, but I also wouldn't say it's 
final because frankly I never say any logic models are final.  They are always 
human creations and some day someone might have a better idea and we change it, 
but it's more than draft.  So let's call it a working, and I'd like to just walk 
through it if you will.  Let's also remember that this is intended to capture 
the CIL program at large.  The CIL program as Tom Kelley and RSA might view it, 
the CIL program as OMB might view it.  This may or may not capture your 
individual CIL program at the local level, and that's understandable because 
that's not its purpose.  If you ever want to try one on your own, more power to 
you, but this one is trying to capture basically all 600 CILs.

And let's just walk through this and I'll try use the if/then if I can, and see 
if that works out.down along the bottom, the yellow boxes as you can see are the 
different types of activities that the CIL program conducts.  And you can see 
that we say there are three different kinds:  There are IL services, and that's 
your four core services, plus perhaps something else; and by the way, I'm always 
going to ask Bob Michaels here to jump in and correct me if I'm wrong on 
anything.  Bob, feel free to do that.

In the center you see the information and referral type activities, and then on 
the right you see the systems advocacy type activities.  As the task force 
developed this, dividing into these three made a lot of sense to them, captured 
what the CILs are doing.

Let's work our way up.  You learned the other day about initial outcomes and 
intermediate and working up to the top.  You achieve the initials first and 
because you've achieved those, it allows you to achieve others working your way 
up.  So it's a little hard to walk through this, but I'll do my best and help me 
out here, but let's say on the left-hand side, the IL services activities -- if 
we deliver good IL services, what we're saying here is that then, that's the box 
right above it, if/then, persons with disabilities has skills, knowledge to 
support their choices.  Other things, too, but let's stick with that line.  If 
they have those skills and knowledge, then they will make their own choices, 
going up to the red boxes now, if they make their own choices, then two things 
will happen, they'll regard themselves as more independent and in fact they'll 
be more independent, and if those two things happen, then persons with 
disabilities will participate in communities to the extent they wish. So this 
would be the logic or the theory, if you will, of why we're delivering IL 
services along this left side.

Now, let's jump to the right side.  How is that?  The systems advocacy -- 
completely different types of activities.  I think so.  If we do systems 
advocacy type activities, then barriers and problems will be identified.  If we 
do that, then a consumer agenda for change exists and active coalitions will 
exist around our issues.  Now I'm going to jump back down and say remember in 
the yellow box we have the information and referral type activities.  If we do 
those, can you see in the orange box persons with disabilities can get the 
information they need, and a combination of both that and the IL services means 
that persons with disabilities see different possibilities.  And then you can 
see three boxes go together.

If people see different possibilities, get the information they need and a 
consumer agenda for change exists, then persons with disabilities advocate for 
increased community supports, and if that happens, then immediately above it 
they have a community identity culture, but also to the right of it, if they do 
this advocating, then decision makers will agree with our agenda.  And if 
decision makers agree with our agenda, then communities will have more 
resources.  And then going back down to the farthest or furthest -- I never 
know, so forgive me -- the furthest -- the orange box, if that happens then 
methods and practices promote independence and if that then resources support 
independence and individual choice in the red, and then you can see the two red 
boxes, if communities have more resources and resources support independent 
choice, then communities are more accessible in a variety of ways, housing, 
transportation, information, employment, education, assistive technology, health 
care, et cetera, and then finally at the top, if persons with disabilities 
participate in communities to the extent they wish, and communities are more 
accessible, then persons with disabilities are integrated into American society.

So now as you're looking at this, I'm willing to bet that every single person on 
this call can see something on this logic model they would change.  I'll bet 
that.  And that's perfectly natural, perfectly understandable.

Hopefully, everyone doesn't see the same thing.  If so, we've made a glaring 
error, but there will be things that you'll look at in one way and say, gosh, I 
would have done that a little bit differently and that's perfectly fine.  There 
is no set in stone, there is no right and wrong, it's simply a creation of some 
people who did their best to try to capture the flavor of the program.  I happen 
to think they've done a pretty good job.  It's understandable to me.  They've 
taken it to other people and it's been understandable to them.  It seems to 
capture things.  As I say, it's a working document, but it is a document we're 
working from right now.  So it's the foundation that we're building from as we 
do our outcome measurement work.  And you might ask the question, well, why do 
we even have to do this?  Gosh, Mike, why did we take the time to develop this 
logic model on Page 41? And that's where I would refer us back to page 37 where 
it lists the different ways that a logic model is useful by itself and I 
honestly believe this because I've seen this happen so many times.

It can create a shared vision of the program.  If all of us come to feel that in 
fact this isn't a bad job of capturing what the CIL program is about, then we 
have a shared vision of the program and it makes the theory very expressive.  So 
it's more than just a vision at the top.  It's throughout, how things fit 
together and how they work. If we have new staff coming on or we're trying to 
recruit new staff, we show them this and say this is what we're about.  And once 
they come on board, this is what we need to be able to accomplish.  It can help 
allocate resources properly by look at the different kinds of outcomes and 
asking ourselves are we allocating enough resources to make it happen?  It can 
communicate the program's intent to stakeholders, all sorts of community 
partners we work with.  I've seen logic models be a wonderful tool to show other 
agencies in the community that here is what we're about and here is how we sit 
with what you are about.  It's a wonderful tool for that.

It can explain the program to potential clients.  When you have a person come in 
the door the first time, you can show them this, talk with them about it and say 
this is what we're about here.  If you're about this, then let's work together.  
If you're about something else, then we may not be the best group for you, but 
it can help working with potential clients.

Negotiate fair accountability with funders and others.  It's fair to be held 
accountable for what we're trying to do.  It's not fair I think to be held 
accountable for something we're not trying to do.

And then the last one, and the reason the task force is working off of this now 
is it guides the systematic review of outcome data in order to identify 
improvement opportunities.  And in a minute, we'll do some Q. and A. now, and 
then in a minute we'll talk about what exactly we're going to do or what we're 
planning to do with this logic model in order to go forward.  So why don't we 
stop here and again just take some discussion.

OPERATOR:  Thank you.  If you want to have a question at this time, please press 
the 0, 1 on your telephone.  The first question comes from Mike Brown.

CALLER:  Hi, this is Doris Ray from independent center in northern Virginia.  
I'm wondering whether in this logic model there shouldn't be a fourth 
foundation, consumer controlled, consumer direction.  It being the basic 
philosophy and also because CILs are marked different -- I mean, there are lots 
of organizations that can provide services, that can provide I&R, that can 
provide systems advocacy for that matter, but the fact that we are consumer 
controlled, consumer-directed is the base of the whole philosophy and is in the 
act and should it not be measured?

MIKE:  Bob, do you want to address that?

BOB:  You know, I'm trying to think how we'd do that, Doris, as we're talking.  
You know, I really don't know.  You know, we don't try to talk about any 
particular part of the philosophy, but that doesn't mean we shouldn't.  And I'm 
just trying to think about the implications of what you're suggesting.  What you 
could do, Doris, that would be really helpful would be to drop me a note and 
describe to me what -- how you would make the logic model look different than 
what we have right here.  I guess I'm saying that to everybody that's out there 
on the call.  If you have any kind of suggestions regarding the logic model, you 
know, you can use the website as a way to give us information or you can send it 
to me at BobMichaels@cox.net.

This is yours, this is your logic model, not NCIL's.

CALLER:  Now I was just thinking it's an overlay, not necessarily changing the 
model, but yes, I'd be glad to do that.  I think this comes from also trying to 
communicate to -- when we're seeking local funding and state funding how we're 
different from the competition.

BOB:  if you can think of a way to do that, I'd love to hear it.

MIKE:  Bob Michaels has forgotten more about IL than I will ever know, but let 
me just say from my ignorant perspective, Doris, it sounds to me that the 
decision on how to handle this would come down to whether the consumer control 
is an actual manifest itself in ways you -- I'm sorry, not just ways, I know it 
manifests itself in ways you work with people, but if it manifests in what you 
do were people.  If there are some specific activities you do and you can look 
at it and say that's an activity because we care about consumer controls, then I 
think you make a really good point that perhaps there might need to be a fourth 
one.

If, on the other hand, you say well really the activities are the three that are 
here, but the way we do them or how we do them is with a strong emphasis on 
consumer controls, then I might -- then I might echo your word overlay that 
that's something that overlays the entire thing.  So it might come down to 
exactly how does consumer control manifest itself when you're working with 
people.

CALLER:  I mean, I think that bears some discussion is all I'm suggesting.

MIKE:  I think it's an excellent point and I'm glad you made it.  I certainly 
noted it down.

OPERATOR:  Thank you.  The next question comes from Charlotte Stewart.

CALLER:  First of all, a comment to what was just brought up.  Wouldn't the 
consumer control issue come in under inputs?  This graphic model we're looking 
at on Page 41 does not detail any inputs or activities because obviously those 
would depend upon the center, but that's where I would think the consumer 
control issue coming in.

My two questions are -- first of all, will you give us, Mike, before you adjourn 
this meeting today or training session how to get in touch with you?

MIKE:  Sure, if you want.

CALLER:  Okay.

MIKE:  I'll give it right now if you want.  We can always reach me through Tim 
or Bob, but since you asked, let me give you my E-mail right now.

CALLER:  Okay.

MIKE:  It's MikeHendri@aol.com.

CALLER:  Thank you.  And I think I forgot my second question.  I'll think of it 
and ask it later.

MIKE:  Sure, you raised a really good point about consumer control as an input, 
and that's something I forgot to say. So thanks for correcting me on that.  All 
the other examples I pointed out in here have had inputs, activities, outputs, 
outcomes and yet we come to this one and we don't have it.  It's simply a space 
problem. That's simply what it is.  We would have loved to have put everything, 
including all the inputs at the bottom.  We would -- notice we don't have 
outputs either.  We would have loved to have some good outputs there.  We had to 
leave something out in order to fit it on one page and so based on my advice and 
I think it was not bad, that the key things are the activities and the outcomes.  
So that's why the other two aren't on there.

BOB:  Hey, Mike, one of the things that I find when I'm trying to explain the 
logic model to people is that I always note to them that this is not a flow 
chart.  That this is a logic model, and that, therefore, it just has desired 
outcomes, and it's not a flow chart meaning that, you know, it does have good 
things and bad things that happen, it just has the desired outcomes.  And so if 
people look at it just to understand it's the positive things, it's just what we 
wants to have happen, I think it makes it easier to understand.

MIKE:  I think that's an excellent point, Bob.  And I'm glad you made it.  I 
think I mentioned on Monday, I even call these desired outcomes.  So it's 
desired outcome that persons with disabilities see different possibilities and 
each of these is a desired outcome.  So as Bob says, this is a bit of an 
optimistic view, but that's okay.  This is our goal.  This is what we're aiming 
to have happen.  So it's a good point, Bob.

TIM:  Mike, this is Tim and before we move back to the presentation, I think 
it's always worthwhile mentioning the work that you did to create what we 
initially called the accessible logic model, but what is in reality a text-based 
version of this document.  Because it gets at that same idea that there is a 
verbal way to describe what's happening on the page here.  And so I just want to 
mention if anyone is using that logic model, and that's what they were referring 
to, we'd love to have your feedback on that as well and not just the content, 
but how we've laid it out in a text format.  And if anybody else is interested 
in having that version or looking at that version, we'll be happy to send it to 
you.

MIKE:  Tim, is there a website people can access that?

TIM:  Certainly, if you go to www -- in fact, let me give -- this is the web 
page can you access the manual and evaluation forms on.  And that's 
www.ncil.org/outcomes.html.

And when you get to the website and that web address, you'll see three links, 
one is to the training manual with graphics and pictures. The other is to the 
training manual that's in text only, and the third is to the evaluation form.  
Needless to say, the text only version of the training manual includes the text-
based version of the outcome -- excuse me -- of the logic model.

OPERATOR:  Thank you.  The next question comes from [ INAUDIBLE ].

CALLER:  This is Evelyn at the independent life center in Colorado and Lloyd is 
our -- is who was setting up the conference call.

You know, we went over this logic model at the April conference and it bothered 
me and I've been thinking since then why it bothered me.  And I got the tape 
recorded version of this training manual and the way it was read, when it got to 
the if/then, if people with disabilities have an agenda, then decision-makers 
will agree and that really bothers me.  You know, we spend considerable lengths 
of time trying to get decision-makers to agree.  And we ended up with the ADA 
after how many years and how much work where decision-makers did agree with us, 
at least enough of them to get the law passed.  They are still trying to unpass 
it.

So should there be some time frame here?  Yeah, in the biggest and best of all 
possible worlds, we would come to the Congress or to our state legislature or to 
our town council and show them logically why, yes, you can see if we had curb 
cuts here on all these corners, people in wheelchairs and people who use walkers 
and have other mobility problems would be able to get around better; but they 
don't always see that.  And I'm wondering if there can't be -- and I right now 
don't have a suggestion.  I'm sorry to say, as to how we could put that in a 
little more realistic term.

MIKE:  Evelyn, I love you.  (Laughter).

CALLER:  Thank you.

MIKE:  and I'll tell you why because you must have had more of an influence than 
you realize.  I think what happened, based on what you just described, I think 
you were looking at an earlier version of the logic model which we have actually 
changed possibly because of you.  So let me bring other people up on board.

If you look on the logic model on the orange at the bottom, the orange initial 
outcomes, you'll see a box that says a consumer agenda for change exists.  Now, 
in the past -- and I think, Evelyn tell me if I'm right -- I think in the past 
the version you were able to look at, an arrow went directly from that box 
straight up to the red box above it, decision-makers agree with your agenda.  So 
the if/then logic or ill logic at that moment was just as Evelyn said, if a 
consumer agenda for change exists, then decision-makers agree with your agenda.  
And in our work, we realized the same thing and maybe because you said it, 
Evelyn, we realized that is not logical.  That is not logic California.  So 
currently, and again, perhaps there is a better improvement here, but again 
we're trying on capture reality, how the world and this program really does 
work, so now you'll see what happens is a consumer agenda for change exists, and 
because of that and some other things, people advocate for increased community 
supports and then because of the advocacy work they do, then decision-makers 
agree with your agenda.  Now, Evelyn might say well, yes, but not always.

CALLER:  No, they don't.  That's right.

MIKE:  or it takes years, but remember Bob Michaels' good point is that this is 
a logic model to reflect the desired situation, what we are working towards, 
what we want to see be happening.  So, Evelyn, that's an excellent point and I 
hope this modification we've made to the logic model will reflect reality 
better.

CALLER:  Yeah, as long as we understand there is a time frame in there that may 
be 20 years.  Hopefully not, but it eventually happens that way, I agree.

MIKE:  Okay, good.  Thanks.

OPERATOR:  Thank you.  The next question comes from Luke.

CALLER:  Yes, this is Mike from the Display Network in Flint.  Just to comment 
back to Doris on the consumer control, I think there is just a foundation 
statement, a fundamental assumption that to be a CIL you must do this before you 
even get into the logic model and that is to run by government made up of people 
with disabilities who make the choice on what we do just as a statement before 
you enter into that.  I got her point though.

The question I have is appreciate the work that has been done on this.  It's a 
great start, but trying to make it real or relative for us or me, I'm trying to 
figure out, okay, let's take an outcome of employment.  We know -- VR works on 
that.  Many CILs assist on that in some fashion on the independent living side.  
That's a great outcome. And where would that fit in here?  I mean, just because 
people have skills on the IL services side and make their own choices, and they 
regard themselves as more independent, where does the getting a job or getting a 
house or getting health care as a real outcome in their lives fit in?

MIKE:  Again, I'm going to defer to Bob Michaels.

BOB:  I was afraid of that.  (Laughter).

You know, we mentioned employment as one of the ultimate outcomes at the top.

CALLER:  Right.

BOB:  You know, one of the things that we were trying to do here is we were 
trying to get all of this so that all the centers could use it and kind of use 
it on one page, and one of the states we looked at the closest was Michigan 
because we know you guys have taken -- you've been doing a lot of work for about 
ten years on this.

And one of the things that we've found out when we took at look at what you had 
done is that you'd developed a logic model for all different kinds of services 
that a center might provide and for different areas of need that you're trying 
to address.

And so pretty much it was just a decision to kind of keep it in a place where 
centers could understand and build off it and be as specific as they want to be.  
I mean, you're right, but if we get into employment as one of the activities 
early on, then we've got to get into literally dozens of other things because 
there are so many different areas that centers address and then once we get into 
that, then we're kind of looking past the need for centers to respond to what 
the needs are in the community.

You know, we were having this debate internally and we just decided to go with a 
more global approach.  But I would really encourage anybody that wants to see 
more to take a look at what Michigan has developed because it's really -- it's 
really -- there is a lot to it and a lot more in depth than we're going to go 
into or tend to go into here.

MIKE:  Bob, this is Mike again.  Thank you on behalf of Michigan, but maybe it 
is as you're following up a logic chart, IL services, people have knowledge, it 
doesn't mean you follow those desired outcomes up.  It could jump right up to 
just below people integrated because they have a job.  And whether they are 
making their own choices, whether they regard themselves or maybe there is an 
assumption that all happens when you get a job or when you get a house or when 
you get health care or on and on.  You see what I'm getting at?

MIKE:  I do, Mike, and I think that what you're starting to talk about are the 
different kinds of indicators of how we would know if an outcome has been 
achieved.  Great point and I think you're also right on the cutting edge of 
where the task force is trying to figure out, for instance -- I'm looking at the 
blue boxes, not at the top blue box, but the two below it and to the left, PWD 
participates in communities to the extent they wish.  Well, one indicator 
perhaps -- I'm saying perhaps -- one possible indicator of participating in a 
community to the extent they wish is that those people who want to be employed 
are in fact employed.  Or, on the other side, I'm looking at the other blue box 
now, communities are more accessible and you can see under that employment is 
listed there.  One way to visualize this is an indicator of accessibility of the 
community is the difficulty of getting employment or the chances of getting 
employment.  So I'm just brainstorming as is everyone on this call, but I think 
there might be some places to put those kinds of outcomes there as indicators, 
if you will, as indicators of larger outcomes.

CALLER:  You really helped me with that, Mike, it's missing on here and as a key 
point, how do you know.  And that's the indicator.  So that helps me figure it 
out.

MIKE:  Good.  I'm glad to hear that.  That's actually what we're going to be 
spending the rest of the call on.  Let me ask our moderator if we have one more 
call on the logic and then we'll move on to the indicators part.

OPERATOR:  the next question comes from John Kelly.

CALLER:  This is [ INAUDIBLE ] from the Boston Center for Independent Living.  
First I just want to say thanks to the task force because they've done some 
really good work.  And it gives us a good kind of model to throw out other ideas 
on.

My question is was there any thought to obtaining and maintaining independence 
in the logic model?  Like for example, if we do in fact see getting people out 
of nursing homes as a fifth core service, we're helping the folks to obtain 
independence by getting them out of a long term facility and then, say, through 
utilization of program services whether it be attendant services or whatever, 
maintain their independence in the community and you get ability in indicator 
right there, how many folks got out of nursing homes or long term fa sits and 
how many folks are staying out and being part of the community and then it gets 
to that top blue box, persons with disabilities with integrated into American 
society.

MIKE:  I'll give a relatively uninformed comment and then see if Bob has a much 
more informed comment.  A relatively uninformed one is that perhaps what you're 
talking about might fit within the red box, persons with disabilities are more 
independent.  I would think one indicator of being independent is not being in 
an institution, I would think.  Perhaps I'm wrong on that, but it could be 
something that would reflect that or be an indicator of that outcome.  Or, 
again, perhaps it might be an indicator of the box above it, the blue one, 
participate in community to the extent they wish.  What you're saying is here 
are specific things we're trying to achieve, how do they fit on this logic 
model?  Or do they fit?  Maybe there is something missing on the logic model, 
and that's exactly the point we're at right now.  So, good.

CALLER:  I see it as a value that people with disabilities are more independent, 
but I think there is also a value to saying through the utilization of ILC's 
they are able to maintain it, too.  And that's important.  Because if we get 
folks out, they are more independent, but then there aren't any ILC's to help 
them maintain it once they are out, then we're still going to have problems.

MIKE:  Okay.  Did you want to say something, Bob, about that?

BOB:  No, I think you're exactly right and I think the point you were making on 
the previous call was the same one here. What he's suggesting is great 
indicators for outcome measures and it's right where you were talking I think on 
participating in the community.  So, yes, it's a nice lead in to where you're 
just about to go.

MIKE:  in the meantime I've written down the point about maintaining 
independence is something we don't want to lose sight of.

CALLER:  Obtaining and maintaining.

MIKE:  I'll write it, obtaining and maintaining.  So thank you for that 
addition.  That's good.  Okay.

If it's okay with you all then just looking at the clock here, let's go to our 
next section, if you can.  That has to do with what now?  What now?

Now, a little background here, if I can, please.  If I were working with a local 
program, and we had this logic model, I would say to them it would be wonderful 
to be able to develop indicators of every single initial outcome and wonderful 
to measure every indicator so that you can know in every box how you're doing.  
That would be wonderful.  But you probably can't afford it and you probably 
don't have the people and you probably don't have the time, so I would work and 
I would say to people, let's make some conscious decisions about what are the 
really key desired outcomes we want to keep our finger on.  I talk about keeping 
our finger on the pulse of each of these desired outcomes.  We could select some 
key ones.

If you think about it though, a program like the CIL program at large across all 
600 CILs, it has an advantage 6 hopefully, and this is our plan, hopefully 
allowing local CILs to provide information on the boxes that are relevant to 
their work because not every CIL is going to be working in every box.  And when 
we aggregate those together across CILs, then we'll have information on all the 
boxes for the program as a whole.  That's at least a current thinking.  We're 
obviously open to all sorts of good ideas.  But the current thinking then is 
that at say Tom Kelly's levels and at OMB's level and at the NCIL level and 
probably at the SILC level, in fact, we would have information about each of 
these boxes and we'd be able to say, you know, we're doing a really wonderful 
job on this, but we've got to find some ways to do a better job here.  And 
that's of course why we're doing this, right, to know where we're succeeding and 
how we can increase our effectiveness.

So what we hoped to do ultimately, as a task force, is to develop ways to 
measure each and every one of these boxes.  Now, again, let me say, please don't 
get nervous at that.  To my knowledge, we're not going to be asking every CIL to 
provide information on every box; but we want to be able to develop indicators 
of every box so that when we do aggregate it, we can say something about the 
whole logic model.

So where does that leave us?  Well, that leaves us with the kind of discussion 
we were just starting to have of, okay, good idea, how do we measure each box 
though?  What would be the indicators that would tell us?  I'll just pick one 
out here -- communities have more resources.  That's a desired outcome here in 
the red.  Okay, what do we know -- how will we know if communities have more 
resources?  What are the indicators, what are the specific items of information 
we're going the measure to know if communities have more resources  Well, the 
bottom line answer is we don't know yet.  That's what we're trying to work on, 
but here is what we've done.

If you look on Page 42, what you see here is just basically the first page of a 
website or web page and here the web page is down at the bottom so can you go to 
it for yourself.  And if you do that, if you go to that web page, you'll find 
basically that every single box on this logic model is listed here.  Can you see 
this one says if we want to know if people with disabilities participate in 
communities to the extent they wish, now, let's see, that's our blue one looking 
back on Page 41, that's our blue box, not the very top, but at the left next to 
the top.  So that's the desired outcome, and then what this web page says, if we 
want to know that, then would it make sense to measure -- and we've got three 
candidates, three possibilities -- for instance, would it make sense to measure 
the number and percent of persons with disabilities served by the CIL within the 
past year who report they are satisfied or very satisfied with the extent of 
their participation in the community?  Well, that's a reported perception, isn't 
it, people are reporting that they feel satisfied.  That's one way to go and 
maybe that's what we end up with or maybe that's one of what we end up with, but 
maybe not.  So here it has use it, lose it. So you can go -- you can vote.  You 
can go to this web page and you can vote.  Do you like that idea or does that 
not sound like such a good idea to you?  And down below it there is a second 
one, B, number and percent of persons with disabilities served by the CIL within 
the past year who report they agree or strongly agree with the statement I take 
part in community activities to the extent I want to.  Again, you may like or 
dislike that one.

Or there could be a completely different kind, for instance, someone mentioned 
employment.  Maybe to you it's a very good indicator that people participate in 
communities to the extent they wish, maybe to you one very good indicator is 
whether or not those -- the number and percent of persons who want to be 
employed who in fact are employed, that may make sense to you.  Well, there is a 
place here to write that in because you see, an even better thing to measure 
would be -- and please suggest your ideas.  So there is room on this web page 
for you to basically improve on these initial ideas we've put on there.  And 
they are just initial ideas.

So we are right -- that's exactly where we are right now.  We are right in the 
middle of looking at all sources we can, checking out what other programs do, 
asking people like yourself to please add your ideas to here.  This is where we 
are right now.

Now, you notice, if you will, on the top of 42 you can actually win some money.  
You can win some money for doing this.  We have a little contest going, and in 
fact the five best ideas, the five best new ideas that get added to these web 
pages, each one of those will be worth $100 or a free registration to the NCIL 
conference, which I think is about a 265-dollar value or maybe more.  So if you 
come up with all five of the best new ideas, you'll win $500.  So hopefully a 
little appealing there, too.

So this is where we are right now.  We're in the -- we're right in the thick.  
We're right in the belly of the beast, if you will.  We are right in the belly 
of the beast of coming up with possible indicators for each of these desired 
outcomes on our logic model.

Bob, do you want to add something to that?

BOB:  Not really.  I think you did a really good job of explaining it.  So we 
have -- we have, if you go up on the website and there are several pages just 
like the ones that you've been pointing out that deal with each one of the 
different desired outcomes and give possible indicators.

CALLER: [ INAUDIBLE ].

MIKE:  Someone has an open line I think.

OPERATOR:  Thank you.  We'll take care of that for you.

MIKE:  Let me say one more thing, if I can.  You might be wondering what's the 
process from here on out?  Well, let me give you a little sense of that, 
although I hope you're getting the flavor from this call that we're all working 
on this together.  We're open to any ideas you've got.  You may have a better 
idea for a process, but here is the process we have right now.

We're currently casting this net as widely as we possibly can to get ideas to 
get candidate ideas, and in about two or three weeks, we're probably going to 
pull that net in and we're going to see what we've got and Bob Michaels and 
myself and I think two other people are going to take a first cut at least of 
it.  A first cut, and we're going to see -- and we don't know how hard that's 
going to be because we don't know what we'll have yet, but we'll take a first 
cut and come up with something we propose to the task force and we'll also of 
course show the task force what we're leaving out of what we process and they 
may decide to put things back in or change it around.  In any case, we'll offer 
something to the task force.  And the task force will think about it and talk 
about it and have some conference calls. And the current plan is that about mid 
May, maybe the end of May, the task force will come to an agreement of, well, 
this is our best, again, working set of indicators for the different desired 
outcomes on the logic model.  And those will be presented at the NCIL conference 
in July.  Again, it will be a working set.  People may come up with better ones.  
That's fine.  But it will be the best we've got by July.

And then what would happen -- of course we change things based on any feedback -
- and then what would happen at least my hopes is that we would then test those 
out because it's one thing to develop a set of desired outcomes and how we're 
going the measure, but it's another to actually try it out.  And to do this, we 
would need to develop some kind of data collection instrument.  We'd need to 
develop some procedures, and hopefully some CILs would be willing to say, you 
know, we'll be a guinea pig, let's try it out here a little bit.  I think that's 
the only way we're going to know if this really works and if it really captures 
what we want it to be capturing.  So this is a process.  This is a process 
that's going to be evolving and working out, but class ee where we are in terms 
of the indicators.

We are hoping that -- well, we know -- I won't say we're hoping. We know that at 
the NCIL conference in July we'll be presenting the best we've got at that 
point.  That's why I say we're right in the belly of the beast right now and we 
need your help right now.  If you have some candidates, this is absolutely the 
time to get them into the hopper.  Bob, anything to add there?

BOB:  Well, the only other thing that I wanted to say was that it's really 
important that people understand this that isn't the task force's or NCIL's or 
IL NET's or anybody else's project, this is centers.  And we're really looking 
to centers to tell whoas they want these desired outcomes and indicators to be.  
So people shouldn't hold back and shouldn't be -- I really hope that nobody is 
looking at this as this is something that we're coming down to tell people that 
they have to do.  It just doesn't work like that, not that anybody would listen 
to us anyhow, but we're really just trying to get as much input as we can and we 
want this to really be a reflex of what it is the centers are saying is 
important to them.  It takes us back to what we were saying on Monday which is 
our first reason for doing this is because centers said that they want to have a 
better way to tell their story.  And that's really what we're looking at first.  
So this is your project, not ours.

MIKE:  Okay, we've been talking at you quite a bit here for the last few 
minutes.  Let's stop and do some Q. and amount. We've been trying to show you 
the website, the web page for the indicators part.  So let's hear?  Comments, 
please.

OPERATOR:  Thank you.  If anyone has a question at this time, please press 0, 1 
on your telephone.  The first cee comes from Ann.

CALLER:  Yes, this is Mike at freedom valley disability in Pennsylvania.  I just 
want to add going backwards a little bit to what Doris said.  I do believe it's 
really important for all of us to lay in here the consumer control issue because 
consumer control is something we have to keep track of.  It isn't overlaying, it 
does affect all of these things as you had said, but it is also something we 
need to measure because if we're not doing that, we're not being CILs.  And I 
really think it's important that we make sure we keep that no matter where we're 
talking in these levels that we're all keeping that in front for indicators.  
Thank you.

MIKE:  Thanks, Mike, I've written that down again.

BOB:  It would really be good if you could take a look at all the indicators 
we've put up as candidate indicators that are up on the website and if you can 
tell us how it is that we might add consumer controls into those --

CALLER:  I'll be happy to look at that.  I know how tough it is.  So that 
everybody here, not just you guys on the opposite sends of the lines hear that 
we're all thinking about this and we all have to contribute to this.  We aren't 
obviously all bright enough to come up with these things.

MIKE:  Just because you don't have the answer yet doesn't mean that raising the 
issue and making sure we attend to it, that's a valuable contribution.

CALLER:  This is wonderful what you've got here.  It really does help us get a 
better grasp of what it is we're about so we can be about it better.

OPERATOR:  Thank you.  The next question comes from Charlotte Stewart.

CALLER:  Hi, Bob, this is Paula just giving you feedback on the text version of 
the logic model.  For the fourth time I've been introduced to this and it's the 
first time I've understood it.  So it works very well.  And I just wanted to ask 
would it not be possible for individual CILs to develop their own indicators 
give thane we all agreed on the outcomes?

MIKE:  I'll jump in there if I can and Bob may disagree. You know, I'm going to 
suggest that method logically that would not be the way we'd want to go.  For 
instance, let's look at one of these at random and says decision makers agree 
with your agenda.  Let's just say that with one CIL measures that by saying the 
number of city councils that put us on the agenda to talk about something, they 
say that's a success in that, and another CIL measures it as, well, let's see, 
no, it's they actually do at least 60 percent of the things we ask them to do 
this year.  Well, those are two completely different things, aren't they?  Two 
completely different things, yet both CILs will be saying yes in our areas 
decision-makers greed with our agenda, but we won't really know what that means, 
will we?  So that's my hesitation.

CALLER:  Okay.

 BOB:  I think realistically what can happen, Mike, is that there will be -- 
because these are your outcomes and your indicators that some centers will want 
to take more and do more with the indicators than other centers.  You know, 
other centers will maybe just want the basic of what is being required by 
somebody, but there will be some centers that want to do more.  And I think also 
that there will be -- there will be indicators that everybody will have.

You know, I would think that probably a good example of that would be in terms 
of identifying the barriers and the problems in the community and then making -- 
developing an agenda for change.  You know, those might be common things that 
will be measured for every center, but then when it gets down to actually 
looking at your community and making sure that it's more accessible, well, for 
one, it might be housing, transportation information, for another it might be 
employment, education, and for another it might be health care, housing and 
transportation.  You know, it would be different depending on what it is that 
your community is saying is important.  So it would be ones that would be all 
the same and other ones that would be a little bit different, but not radically 
different.

OPERATOR:  Thank you.  The next question comes from Peter.

CALLER:  Thank you.  One of the things that I'm hoping, and I do plan to go to 
the website and look at some of those indicators, but I think this also sort of 
addresses just the previous caller's issues because all of the CILs are so 
different and in Vermont we have one CIL for the entire state, very rural, 
except for some isolated pockets of population, basically.  So sometimes the 
measures that are developed may work better for some centers than others, and 
what occurred to me, and I would be really interested to know if something like 
this would work, and I'm looking at Page 42 where it's asking are these three 
indicators something you'd want to have?  Well, what I would suggest is perhaps 
having a bunch of indicators, maybe say five or even more for some things, and 
so then if you're measuring those and obviously you're going to have higher 
percentages with some of them and lower with others, so in this particular case 
if what we want to know is whether the person with disabilities participates in 
the community, if rather than looking at each of the indicators individually, 
take them collectively as a group and if you have a fairly high percentage, then 
you can say in that particular overall measure you've been successful where it 
may give you some more flexibility, depending on the different nature and make 
up for each Center for Independent Living.  Does that make any sense at all?  Or 
did I articulate that well?

MIKE:  I think you're saying -- tell me if I'm wrong -- I think you may be 
saying that on Page 42 what if trying to pick whether it's A. or B. or C. that 
we ought to ask people to ask about, you're saying we allow all three and then a 
local CIL can decide the one that makes sense for it?

CALLER:  Not so much the one that makes sense for it, say you have all three and 
when you come back, you know, you have 15 percent on one and 2 percent on 
another and, you know, 50 percent on another one.  Add those percentages up to 
come up with what your score or whatever way to do that is, rather than relying 
on fewer questions.  Have more questions so you can figure out the diversity of 
the different centers.

MIKE:  I see so maybe you're saying A. plus B. plus C, ask them all?

CALLER:  Ask them all and then you can use the measures collectively, yes.

MIKE:  It's an interesting point, and I certainly made a note of it.  I 
appreciate it.  As you can imagine, the task force has heard pretty clearly from 
the CIL perspective that we're busy people. We have a lot to do already.  We 
didn't take this job in order to be data collectors.  We realize the importance 
of outcomes, but we are not researchers and don't want to be researchers, thank 
you.  So they've definitely been encouraging us to go more minimal than maximal, 
if you will, but you raise a good point and we make a note of it.  So thanks.

OPERATOR:  Thank you.  Before we take any other audio questions, are there any 
web questions at this time?

TIM:  Yes, thank you, Dawn.  I have a few.  First question comes from [ 
INAUDIBLE ] and her question is would consumer satisfaction and self-analysis 
surveys be central to determining initial outcomes such as skills, knowledge, et 
cetera?

MIKE:  Well, it's a good question and it's starting to get into, actually -- 
leads in beautifully into the pages 43 and 44 because as you can imagine, once 
we decide what information we need, then we have to decide who we're going to 
get it from or where we're going to get it from, and also how we're going to get 
it.  So page 42 is possible sources of program outcome data, and you can see it 
can be existing information or individuals or the environment.  And on Page 44 
it lists 7 or 8 different possible ways to get that and we just don't know yet.  
We haven't reached the point where we know.  So, yes, it may be that a survey of 
CIL clients, if that's the right word, forgive me if it isn't, of CIL client 
would be a way to go or it may not be.  We just aren't at that point yet, but 
we're very open to any ideas anyone has.

TIM:  and before we go back to the presentation, Mike, one more question similar 
to what we heard regarding consumer control.

Peer support is so essential and sets us a part from other organizations, should 
it be spelled out on the logic model?

MIKE:  Let's ask Bob's reaction if we can to that.  Peer support.

BOB:  (Laughter) I don't know.  It's like consumer control and it certainly has 
to be reflected in the indicators that we developed.  I don't know -- you know, 
when they were talking about that overlay of the counseling, I was thinking you 
have certain other things you need to overlay -- not con selling, the consumer 
control. Because you have the peer relationships, you have the equal access, you 
have community-based services across disabilities issues.  They all need to be 
reflected.  I'm not sure yet.  Any ideas that anybody has about how to do that, 
boy, we'd love to hear them.

MIKE:  Bob, let me ask a relatively uninformed question. Under IL services on 
the logic model, the bottom left yellow box, IL services, those were the four 
core services, is that correct?

BOB:  Well, we looked -- you know what we did is we changed that because we 
started off talking about core services, but then I&R and systems advocacy were 
separated out.  And we started adding other things in as we were talking about 
it and then the last change that we made logic model was just to list IL 
services to take into consideration that there were lots of other things that 
centers do that are different that are not in IL services and somehow we need to 
be bringing them in to the discussion here.

So it's broader than that, but I don't think it -- when I look at peer stuff -- 
I don't know, maybe I'm miss interpreting what the question was, but when I look 
at peer stuff, I think more in terms of our overall philosophy about how we 
operate in peer relationships.

MIKE:  So peer support is not in the yellow box IL services?

BOB:  I don't look at it that way, no.  Maybe other people do and if they do, 
I'd love to hear that and see an example that would work into our mod model.

MIKE:  Obviously our questioner has raised a good point. So we appreciate that, 
Tim.

OPERATOR:  Thank you.  Do you have any other web questions or do you wish to 
move back to audio?

TIM:  Let's go back to the audio questions.  Thank you, Dawn.

OPERATOR:  The next question comes from Ann.

CALLER:  Hi, this is Susan Webb, and we are all still here.  Getting back to the 
question we asked earlier, you kind of addressed some of it and we generally 
have a consensus here that we really like the model, but you said that what 
centers really want was the ability to tell our story, and I think that that's 
true.

But the question still remains how do we tell our whole story and get 
recognition for innovative gs, not just a generic standard.  We're kind of 
grappling here with the need to keep the model as simple as possible and as 
generic as possible, which I think is a good goal in terms of the overall 
reporting to the powers that be that have to judge us and decide to give us 
millions and millions more dollars, but by the same token, my concern would be 
that if you make it too vanilla and too generic, you're really going to lose a 
lot of the really good, rich, innovative stuff that centers do across the 
nation.

That's really more of a comment as you go through this, please keep in mind that 
there has to be a place in here somewhere for us to tell our story, yes, but to 
tell the whole story about all the great things we do.

MIKE:  Susan, let me see if I can echo that back because I think you are raising 
a good point.  I think what you're saying -- correct me if I'm wrong -- you 
recognize the need for certain -- the standard, if you will, display of what 
we're doing like the logic model that will capture the whole program, but you 
want to make sure that an individual CIL will have the opportunity also in a 
meaningful way, too, not just another question and an end, but in a meaningful 
way will have the opportunity to tell about perhaps the unique or -- maybe 
unique kinds of things they are achieving at their own CILs, is that correct?

CALLER:  I think you summarized that quite well, it might not be unique stuff as 
one of the commenters mentioned earlier, some CILs do employment for example and 
others don't.  You might not want employment to be a generic indicator because 
not all CILs do that, but those CILs like ours that does do that, we want to be 
able to get recognition that that's part of what we spend our resources on and 
the outcomes that we do achieve from those efforts.

MIKE:  I think it's an excellent point and we've certainly made note of it.  So 
thanks for bringing it up.

CALLER:  You're welcome.

OPERATOR:  Thank you.  The next question comes from Mike Brown.

CALLER:  Hi, this is Doris Ray again, and I had a couple of questions or 
comments.  One is that in the logic model on services, IL services, I have 
always thought that I&R was a service and there may be reasons that you're 
extracting it out.  However, I was wondering where community Education and 
Technical Assistance fit into the model?  And another question I had was related 
to the systems advocacy at times things take a long time because after advocacy, 
decision-makers may agree with your agenda.  For example, recently here we 
worked with another organization and got a resolution through a housing 
authority that when they used local money that they would abide by accessibility 
standards and include universal design, but now we're at the place where will 
that become reality?  And it will require some monitoring to find that out.  So 
what I'm wondering is where will that be reflected in the logic model and the 
only other thing is the overlay also needs to include cross-disability.

MIKE:  Okay, Doris, you raise several good points and as you can guess, I'm 
going to defer to our IL expert, Bob Michaels.

BOB:  You know, I really don't know how to answer that.

CALLER: [ INAUDIBLE ].

BOB:  Okay, one thing I would say though is that we looked at the development of 
these types of logic models and first started looking at services and we started 
to talk initially about the four core services and then we realized, well, 
systems advocacy is really different and we separated that out and then we said, 
well, you know, information and referral is a little bit different, too, and so 
we went through this discussion and it's kind of morphed into the model where 
you have I&R and systems advocacy and you have everything else over on the love 
side and it's just one of the things that's evolved.  It seems to have made 
sense after awhile and, again, we'd love to hear if you can think of another way 
to say it, we ought to reflect in there.

CALLER:  Maybe information, education and training or something like that so it 
reflects -- I mean, because individual information, one on one pieces of 
information is won thing, but information to the community can serve an 
awareness and enlightening function, but it also serves the systems advocacy 
agenda.

BOB:  Yeah.

MIKE:  That's a good point, Doris.  We've made notes of it.  So thanks.

OPERATOR:  Thank you.  The next question comes from Donna Yeager.

CALLER:  Yeah, this is Ken Williams again from Illinois. One of the things that 
I had -- in terms of a question has been partially answered because I was 
looking at the yellow box at the bottom, and one of the things that I liked was 
because it looked like the whole model was based on the four core services, but 
here you have three and it sounds like peer relationships are embedded in the 
first one.  Am I right in assuming that, that that thought or concept is embed 
in the first one, IL services?

MIKE:. Bob?

BOB:  I don't look at it that way, Ken.  I see the peer relationships and peer 
role models as being something that's more tied to our philosophy and the 
principles of the way we conduct business. So I would see it as going across all 
three of those.  And like when Doris was talking before about overlaying 
philosophy, she was talking with consumer control.  Well, you should also 
probably do that with cross-disability, with peer roles and peer relationships 
and I mean all those things, you know, we do have peer counseling over in the IL 
services, and I can understand what you're driving at.  But I've never thought 
about it in terms of just overlay before and I'm not sure how we deal with that, 
whether we do an introductory paragraph or -- I don't know how we overlay that.  
It may be to make sure that the indicators reflect all of those things that 
really make us different.

CALLER:  Yeah, because see like we had started working on this a few years back 
and one of the things that we had thought here in Illinois is base them on the 
four core services because that's the one thing that every center has in common 
and you can have your variations of your outcome ladder relative to your 
particular center, the dynamics of those centers, and that's kind of why I asked 
that question.

But secondly, I had another question and I think that was probably addressed 
because I was looking at information and referral and everything talks about 
people with disabilities when in fact -- and you were talking about reality and 
so on and so forth, but the reality is that most information and referral comes 
from the community and a lot of the community are people without disabilities as 
well. So how does that infuse into the logic model that we have here?

MIKE:  Bob?

BOB:  Yeah, you know, that's an issue that has been raised over and over again.  
Most every time we make a presentation on this, that issue comes up, and 
basically the task force has looked back at this several times to try to figure 
this out and, you know, what are we doing about family members?  What are we 
doing about other people in the community that get I&R, and the decision was 
made pretty much to keep it the way it was and not address them because our 
primary issue here is not -- is not -- is working with a person with a 
disability, and it's trying to make a person with a disability be more 
mainstreamed into -- you know, into our society.  So as long as that's the 
ultimate goal, these other things are nice and things that we do, and they 
certainly benefit the community and the family, but they aren't our primary 
issue.  So that's why we left that out.  If a lot of people felt -- you know, 
felt differently about that, we'd need to discuss it again.

CALLER:  Well the reason I brought it up is one of the things I heard earlier is 
that we want this model to reflect reality and the reality is the majority of 
information and referral requested services don't seem to come from people with 
disabilities as much as people from the community of all varieties and to me 
looking at this, leaving that out is leaving a major part of what information 
and referral is realistically all about.  Don't you think?

MIKE:  Well, this is Mike.  I've made a point of that.  I think you raise a very 
good point and I think it's something we're going to need to look at.  I'm glad 
you raised it, thanks.

OPERATOR:  Thank you.  The next question comes from Ken.

CALLER:  Yeah, this is Ken in Wyoming.  I just want to say, Bob, don't hold out 
and only bring the task force indicator form or whatever you're going to call it 
to NCIL.  Remember, you're going to be here for Region VIII meeting the end of 
June and we'd love to have you demonstrate it and try it on us and you might 
even find some folks here in Region VIII who would be willing to play with it 
and see if it works for us.

BOB:  Yeah.  Yeah, I'm looking forward to that visit.

CALLER:  Okay.

BOB:  You know, along that line, one of the things we should make sure we 
reiterate here is that IL NET has helped subsidize the cost of me to go around 
and do brief presentations in the different regions and as long as it's 
regional, they will support that and I commit to like a three hour presentation 
about what it is we've accomplished and we use that as an opportunity to gather 
input from people.  If you have a conference going on and there is something we 
can tag this on to, IL NET will help cover the cost of coming up there and doing 
a short presentation, giving an opportunity to give input.

We're going to be doing it -- we did it in April at the SILC Congress and we did 
it also in Region IV and 6.  We're going to be doing it up in Region VIII and 
Region X and if anybody else wants it, just let me know and we'll try to figure 
it out.

CALLER:  See you in June.

BOB:  Yeah.

TIM:  All right, everyone, this is Tim at the NCIL office.  I apologize, but 
we're out of time today and out of respect for your time, the captioner's time, 
our trainer's time and everyone's time I'm going to have to cut things short.  I 
want to give my sincere thank you to all of you and to Mike and Bob for their 
hard work preparing for the call and for their time being on the call today. For 
making as much time as they have for questions and answers.

And I also wanted to let you all know that now that the training is over, the 
door is not closed.  I, luckily, have a very simple E-mail address and I will be 
happy to forward all questions along to Bob or Mike or respond to them myself.  
That E-mail address is Tim@ncil.org.

That's an open door.  You can E-mail me whether it's one hour or one month from 
now.  I'd also like to remind you that the materials and evaluations from today 
are located at www.ncil.org/outcomes.html.

The evaluation forms on there again are extremely important to us and I would 
appreciate it if you'd take a few moments to fill that out.  The instructions 
for returning it to the NCIL office are included on the form.

Also want to take an opportunity to thank the outcome measures task force, every 
member of that group has worked very hard.  I've been witness to that and also 
ILRU and NCIL for making the call possible today.

Again, if you have any other questions, requests for information, or if you'd 
like to order a do it yourself kit which includes an audio copy of both 
teleconferences from today and Monday, you can E-mail me at Tim@ncil.org.  Bob 
and Mike, any last comments?

MIKE:  Well, I'd like to say thanks to everybody, Tim. I've enjoyed these two 
calls immensely and I hope they've been useful and I hope you get the sense and 
we've certainly tried to convey it sincerely that this is a process we're 
basically trying to work as your agents.  We're trying to produce something that 
will be useful for you folks so please help us to do that.

BOB:  Yeah, Tim, I just want to thank IL NET, ILRU and NCIL for the work you've 
done and thank RSA for working along with us and they've really been a big help 
and been real open to our ideas. So I'm looking forward to getting input from 
everybody so we can share these.  Thanks.

TIM:  Wonderful.  Thank you all and to our participants if I can leave with you 
a final request to visit the ILRU website and give us your comments on our 
process.  If you need those addresses again feel free to write me at 
Tim@ncil.org.  Bye everyone.