ROUGH DRAFT 9-14-11, Outcome Measures for Centers for Independent Living – An IL NET Resource Presented by ILRU >> Mike Hendricks: OKAY, LET'S HEAR SOME OF THE ISSUES COMING UP AT YOUR TABLES. I BET EVERY ISSUE IS RELEVANT TO EVERY ONE OF US. AND ONLINE, IF WE HAVE ANY COMING IN FROM ONLINE, ALSO. TELL ME WHAT YOU'VE BEEN TALKING ABOUT AT YOUR TABLE BUSY STORING DATA. I'LL ASK MY FRIENDS BACK IN THE BACK. >>> WHAT WAS THE QUESTION? >> Mike Hendricks: THE QUESTION WAS WHAT HAVE YOU BEEN TALKING ABOUT, ABOUT STORING INFORMATION BACK AT YOUR CIL AND WHAT ISSUES HAVE COME UP. >> AND PLEASE, INTRODUCE YOURSELF, TOO, WHOEVER'S GOING 57 TO BE THE RESPONDER THERE. WE'VE GOT THIS TABLE JUST EAGERLY GOING FIRST, AND THEN YOU'RE SECOND OVER HERE. >> Audience Member: JEANNIE BARKLEY OF GEORGIA AND SOUTH CAROLINA. WE SPENT SOME TIME TALKING ABOUT OUR INDIVIDUAL -- HOW WE STARTED PRIMARILY IN OUR DATABASES, WHICH SOME OF US USE WITH CIL, SOME OF US USE CFAL. WE WERE JUST TALKING ABOUT HOW SOME OF OUR STAFF, YOU KNOW, HAS SOME ISSUES IN -- WHEN THEY ARE WORDING THE OUTCOMES, THE MEASUREMENTS. SOME OF THEM ARE NOT QUITE GETTING THE DATA IN THERE THE WAY IT NEEDS TO BE PRESENTED. >> THE ACTUAL PROCESS OF GETTING IT FROM THE PAPER INTO THE COMPUTER? THAT IS A WHAT YOU'RE SAYING? >> Audience Member: UH-HUH. >> Mike Hendricks: WE'VE GOT TO DO THAT WELL, IT'S A GOOD ISSUE TO START THINKING ABOUT. >> Audience Member: MIKE FROM MARYLAND, ALSO ORIGINALLY FROM OREGON. ANYBODY WANTS TO GO OUT TO THE PUB IN SOUTHEAST PORTLAND, I'LL SHOW YOU AROUND PORTLAND AND WE'LL GO HAVE SOME IRISH DINNER. NOW, HOW ABOUT 6:30 IN THE HOTEL LOBBY. >> WOOHOO! >> Audience Member: WE TALKED ABOUT THE DIFFERENT DOCUMENTING SYSTEMS THAT WE'RE USING. IT SEEMS AS THOUGH JUST ONE OF THE THINGS WE TALKED ABOUT IS JUST THIS WHOLE PRUSSIAN OF HAVING TWO DIFFERENT DATA LOCATIONS FOR THIS KIND OF INFORMATION WOULD SEEM TO BE EXTREMELY FRUSTRATING. WE NEED TO GET WITH RICK AND THE FOLKS AT Q-90 AND SAY, LOOK 58 WHAT, CAN WE DO TO MODIFY YOUR DATABASE SO WE CAN INTEGRATE ALL OF THIS INFORMATION IN ONE PLACE. OTHERWISE IT'S GOING TO BE A NIGHTMARE OF SYSTEMS THAT DON'T TALK TO EACH OTHER, WHICH WE DON'T WANT. THAT'S REALLY IT, WE WERE TALK BEING HOW TO GET OUR PROVIDERS TO MAKE SURE THEY INTEGRATE IT. >> THAT'S A DARN IMPORTANT TOPIC AND I'M GLAD YOU MENTIONED IT. >> Mike Hendricks: RATHER THAN HAVING 50 INDIVIDUAL CILs CONTACT EACH OF THESE COMMERCIAL COMPANIES, THERE OUGHT TO BE, SEEMS LIKE IT WOULD MAKE MORE SENSE TO ORGANIZE SOMETHING, MAYBE TIM, I'M SURE HE HAS NOTHING MORE TO DO, HE'S GETTING MARRIED IN SIX WEEKS AND PROBABLY HAS NOTHING ON HIS MIND. HE COULD PROBABLY ORGANIZE VARIOUS MEETINGS AND SIMPLIFY THAT PROCESS. >> Audience Member: THAT'S WHAT I WANTED TO ASK YOU, HAVE THERE BEEN MEETINGS SURELY IF THESE COMPANIES ARE AWARE THAT THE OUTCOME MEASURES ARE THE REALITY OF THE FUTURE, THEY ARE GOING TO WANT TO COMPETE WITH EACH OTHER TO MAKE SURE THEY HAVE THE SYSTEMS WE ALL WANT. HAS THERE BEEN ANY CONTACT? >> Mike Hendricks: DAN KESSLER HAS HIS HAND UP. HE MAY HAVE THE ANSWER. WE WERE AWFULLY SURPRISED THEY ALL SAID YES. THEY WERE ALL CERTAINLY AWARE OF THE NEED. >> Audience Member: I WAS GOING ADDRESS SOMETHING ELSE. >> Mike Hendricks: DOES ANYBODY KNOW IF ANYBODY ON A SYSTEMATIC WAY HAS HAD CONTACT WITH THESE COMMERCIAL COMPANIES ABOUT THIS? 59 RICHARD PETTY HAS HIS HAND UP. >> Audience Member: I JUST WANTED TO MENTION THAT ONE OF THE DEVELOPERS OF SYSTEMS PARTICIPATED PRETTY ACTIVELY IN THE WORK GROUP. I BELIEVE THERE WAS INTEREST ON THE PART OF OTHERS. ALTHOUGH LATER ON, PROBABLY TOO LATE TO PARTICIPATE IN THE GROUP, BUT THERE WAS ONE WHO I THINK WAS PRETTY CONSISTENTLY INVOLVED. I DON'T RECALL WHO THAT WAS RIGHT NOW, MAYBE SOMEONE ELSE IN THE WORK GROUP DOES. >> Mike Hendricks: BOB MICHAELS PROBABLY DOES. >> Bob Michaels: THAT WAS JIM MOORE FROM C FULL COVERAGE AL. >> Mike Hendricks: HE WAS APPARENTLY ON ONE OF THE TASK FORCES THAT HAVE BEEN ASKED TO GIVE SOME ADVICE THAT WAY. >> Bob Michaels: WIFE HAD KIND OF ONGOING CONVERSATIONS -- I SHOULDN'T SAY IT THAT WAY, I GUESS. I'VE TRIED TO WORK WITH EACH ONE OF THE GROUPS THAT IS A MAJOR PROVIDER OF INDEPENDENT LIVING DATABASE SERVICES. WE HAVE TO TALKED TO -- MENTIONED TO CFAL AND OTHERS THAT THIS IS GOING TO BE THE WAY OF THE FUTURE. THAT'S WHY WE CONTACTED ALL OF THEM TO FIND OUT -- ASK THEM THE SAME QUESTIONS TO FIND OUT HOW INTERESTED THEY WERE IN PROVIDING SERVICES. AND THEY ALL ARE. >> Mike Hendricks: CLEARLY THIS IS SOMETHING THAT NEEDS TO BE ON OUR AGENDA TO FIGURE OUT HOW TO MAKE THIS HAPPEN. DAN, YOU HAD SOMETHING ELSE TO RAISE. >> Audience Member: YEAH, DAN KESSLER, ALABAMA. 60 THE LIST YOU HAD UP THERE, YOU SAID THERE WERE PROBABLY A COUPLE OF OTHER SYSTEMS, AS WELL. IN ADDITION TO THOSE THAT SOME OF US HAVE TO USE WITHIN OUR OWN CENTERS, SEVERAL DIFFERENT MANAGEMENT INFORMATION SYSTEMS. FOR EXAMPLE, IF YOU HAVE HUB THROUGH YOUR LOCAL CONTINUUM. IF YOU HAVE MONEY OR FUNDS OR PARTICIPATE IN THE WORK INCENTIVES PROGRAM, THERE'S A MANAGEMENT INFORMATION SYSTEM THERE YOU HAVE TO USE. AND GETTING INTO THOSE AND MAKING CHANGES TO THOSE I THINK IS GOING TO BE PRETTY MUCH -- IT'S NOT GOING TO BE AT THE TOP OF THE RADAR SCREEN. WE'RE DEALING WITH PROGRAMS OVER CILs. >> Mike Hendricks: SURE, SURE. I HEAR YOU. ANY OTHER TOPIC THAT CAME UP? THESE ARE IMPORTANT TOPICS. YOU SEE WHAT I MEAN ABOUT THE STORING? THE FIRST THOUGHT MIGHT BE WHY DO WE HAVE TO SPEND ANY TIME ON STORING? THIS IS REALLY IMPORTANT STUFF AND OBVIOUSLY THERE'S STUFF WE HAVE TO DO. ANYTHING ELSE THAT CAME UP THAT WE OUGHT TO MENTION RIGHT NOW, TALK ABOUT? ANYTHING FROM ONLINE, CAROL? OKAY, GOOD. O WE HAVE ONE MORE, SORRY, MY MISTAKE. INTRODUCE YOURSELVES, PLEASE. >>> BRENDA DID A LAWSON, TULSA, OKLAHOMA. WHAT DOES CROSS-CIL MEAN IN THIS CONTEST? IT SAYS CROSS-CIL OUTCOME INFORMATION. >> Mike Hendricks: AH. HANG ON ONE SECOND. 61 OH. HMMM. OH. I KNOW YOU WERE EXPECTING "OH" TO LEAD TO THE ANSWER. OH. WHY DID WE PUT THAT IN? HERE, YOU'RE THE ONE WHO DID THAT! >> Bob Michaels: THERE WAS A LIST OF QUESTIONS THAT WE HAD. >> Mike Hendricks: I THINK I REMEMBER. WE DID THIS A WHILE AGO. BOB DID THIS JUST FOR YOU ALL. YOU SHOULD THANK HIM FOR THAT. I THINK IF I WROTE IT, WHICH I MAY HAVE ORIGINALLY, WHAT I MEAN BY THAT IS CAN YOU DO COMPARISONS ACROSS ONE CIL TO ANOTHER CIL. CAN YOU DO IT NOW? AND CAN YOU DO THAT IN THE FUTURE? SO THAT'S WHAT I MEN, LIKE MORE THAN JUST ONE CIL AT A TIME. AST AND THE ANSWER SEEMS TO BE YES FOR ALL OF THEM. YOU HAD ME SPEECHLESS FOR A MOMENT. LET'S MOVE INTO ANALYZING, BECAUSE THIS IS WHERE YOU'RE GOING TO BE THE FIRST PEOPLE TO HEAR ACTUALLY THE RESULTS OF THE OUTCOME INFORMATION THIS YEAR. BEFORE BOB DOES THAT, I'M GOING TO MAKE SURE WE'RE ALL ON THE SAME WAVELENGTH OF WHAT WE'RE TALKING ABOUT, ABOUT ANALYSIS, IF I CAN. LET ME ASK OUR FRIEND IN THE BACK. ARE WE TOWARD GO FOR OUR ONLINE FRIENDS? >> WE ARE, FOR ANALYZING OUTCOMES. >> THANK YOU, BUDDY. YOU MIGHT BE A LITTLE INTIMIDATED AT THIS NOTION OF ANALYZING DATA. 62 PROBABLY NOT TOO MANY OF YOU ARE IN OTHER WORDS LIKE I AM, AND PROBABLY DON'T WANT TO BE, WHICH IS GOOD. YOU MAY NOT HAVE HAD A LOT OF RESEARCH COURSES. IT'S REALLY NOT THAT COMPLICATED. IT'S NOT. IT'S THREE SIMPLE STEPS WE WILL SUGGEST TO YOU. THE FIRST ONE IS CLEAN THE RAW INFORMATION. THIS IS OFTEN OVERLOOKED, AND WE HAVE LEARNED FROM OUR FIELD TEST NOT TO. SECOND ONE IS RUN THE NUMBERS. THIRD IS ANALYZE OPEN-ENDED QUESTIONS. WHAT DO WE MEAN? FIRST OF ALL, CLEAN THE RAW INFORMATION. LOOK AT THE ANSWERS QUESTION BY QUESTION. WERE ANSWERS TO ANY QUESTIONS WACKY? DO ANY QUESTIONS NEED TO BE THROWN OUT? . IN FACT, WE STARTED THE FIELD TEST LAST YEAR WITH 12 INDICATORS. WE HAVE 11 NOW. ONE OF THEM JUST WAS WACKY. IT WAS TOTALLY WACKY. WE DIDN'T KNOW IT WAS GOING TO BE WACKY, WE FOUND OUT IT WAS WACKY. IT HAD TO DO WITH THE PERCENTAGE OF GOALS ACHIEVED BY CONSUMERS. IT MADE LOT OF SENSE TO US TO TRY TO MEASURE THAT, MADE A WHOLE BUNCH OF SENSE. IT'S ONE OF THOSE GOOD EXAMPLES OF DOING A FIELD TEST. ON PAPER THAT SURE SEEMED TO MAKE SENSE. THE PROBLEM IS, IN REALITY CILs DIFFER A LOT, WE DIDN'T KNOW THIS, A LOT IN HOW MANY 63 GOALS THEY HELP PEOPLE TO SET. WE HAD SOME CILs WHERE THEY TYPICALLY SET THREE OR FOUR GOALS. OTHERS SET 20 OR 30 GOALS. THAT MADE IT DIFFICULT TO INTERPRET THE INFORMATION WE GOT BACK. WHAT DID IT MEAN WHEN YOU HAD COMPLETED THREE? WAS IT THREE OF FOUR? OR THREE OF 30? HALF AND HALF, WAS IT TWO OR WAS IT 15? WE LOOKED AT IT MORE CLOSELY AND SAID WORKS DON'T KNOW WHAT TO DO WITH IT. WE STILL DON'T KNOW WHAT TO DO WITH IT, BY THE WAY. IF ANYBODY DOES, WE'RE OPEN TO IT. WE THINK IT'S AN IMPORTANT AREA, WE DON'T KNOW QUITE WHAT TO DO WITH IT. THAT'S AN EXAMPLE OF LOOKING AT EACH QUESTION AND SEEING IF WHAT YOU'RE GETTING BACK MADE SOME SENSE. ALSO LOOK AT THE ANSWERS CLIENT BY CLIENT. HOW MANY GAVE INCONSISTENT ANSWERS? WE HAD NOT A BUNCH BUT WE HAD SOME PEOPLE SAY, NO, I AM NOT MORE INDEPENDENT. AND HERE'S THE WAY I AM MORE INDEPENDENT. AS YOU CAN IMAGINE, THAT FELT A LITTLE INCONSISTENT TO US. WE HAD TO DECIDE WHAT TO DO WITH THAT. DO ANY CLIENTS NEED TO BE THROWN OUT? WE DIDN'T THROW OUT ANY CLIENTS COMPLETELY. BUT WE DID THROW OUT SOME OF THEIR INCONSISTENT INFORMATION LIKE THAT. OR A SPECIFIC QUESTION, THAT'S MORE THE KIND OF THINGS WE 64 SCRATCHED OUR HEADS AND SAID, THIS DOESN'T MAKE ANY SENSE. SO THAT'S -- AND THAT'S NOT ESPECIALLY COMPLICATED BUT DO YOU HAVE TO DO IT. WE'D SUGGEST YOU DO IT. RUNNING THE NUMBERS. OKAY. A LOT EASIER THAN YOU PROBABLY THINK. BASICALLY YOU JUST FOR EACH INDICATOR YOU DESCRIBE YOUR OVERALL PERFORMANCE. AND THEN YOU SEE WHAT AFFECTS THE OUTCOME. FOR INSTANCE, GENDER, ETHNICITY, DISABILITY. BY THE WAY, BOB IS NOT GOING TO BE GIVING YOU ANY INFORMATION ON MEN VERSUS WOMEN OR OLDER VERSUS YOUNGER BUT WE DID ALL OF THOSE ANALYSES. WE LOOKED AT THEM ALL BOWE LAST YEAR AND THIS YEAR, AND WE FOUND NO DIFFERENCES, AT LEAST WITH THE NUMBERS WE HAVE. WE HAVE NO TESTIFIED THAT AS A RULE WE'RE DOING BETTER OR WORSE WITH ONE GROUP OVER ANOTHER. WE LOOKED IT FOR, WE THOUGHT THERE MIGHT HAVE BEEN SOMETHING THERE, WE LOOKED MORE CLOSELY BUT IT DOESN'T SEEM SO. THAT'S ACTUALLY I THINK GOOD NEWS. GRAPHICS ARE ESPECIALLY HELPFUL. LET'S WALK THROUGH A QUICK EXAMPLE. YOU REMEMBER THIS, THE LOGIC MODEL WE LOOKED AT YESTERDAY ABOUT THE AT-RISK TEEN MENTORING PROGRAM WHERE WE HAD THE SIX OUTCOMES. AND YOU REMEMBER THIS: WHERE WE HAD THE DIFFERENT INDICATORS, DO YOU REMEMBER THAT? THESE WERE THE SPECIFIC THINGS TO MEASURE FOR EACH ONE. LET'S JUST WALK THROUGH AN EXAMPLE OF HOW SOMEONE MIGHT 65 ANALYZE THIS. FIRST OF ALL, YOU MIGHT WANT TO KNOW, HOW DID WE DO OVERALL ON THESE DIFFERENT OUTCOMES ON THESE INDICATORS? HERE'S SOME HYPOTHETICAL BUT QUITE PLAUSIBLE INFORMATION. MAYBE WE LEARNED -- LET'S JUST SAY WE DID LEARN THAT 80% OF THE TEENS COMPLETE THEIR HOMEWORK REGULARLY, BUT IT'S TOO EARLY TO KNOW IF THEY ARE GOING TO LEARN BETTER GREATS OR ACHIEVE PASSING GRADES, WE JUST DON'T KNOW YES. WE KNOW THAT 60% OF THEM ATTEND SCHOOL REGULARLY, AND 55% MEET DISTRICT ATTENDANCE REQUIREMENTS. IT'S TOO EARLY TO KNOW IF THEY ARE GOING GRADUATE. FOR MANAGEMENT PURPOSES, WHAT CAN WE DO WITH THIS? NOT TOO MUCH. LET'S LOOK A LITTLE MORE IN-DEPTH. THAT'S OFTEN THE KEY TO USING ANALYSIS FOR MANAGEMENT PURPOSES, IS LOOK A LITTLE MORE IN-DEPTH. LET'S ASK OURSELVES HOW ABOUT GIRLS VOUS BOYS. AND LOOK DOWN HERE, WE FIND THAT BOTH OF THEM ARE COMPLETING THEIR HOMEWORK AT 80%, GIRLS AND BOYS. WHEN IT COMES TO ATTENDING SCHOOL, GIRL ONLY 45%. BOYS 70%. THAT'S INTERESTING TO KNOW. YOU CAN START TO DO SOMETHING WITH THAT, CAN'T YOU? YOU CAN START TO FIGURE OUT WHY IS THAT? LET'S TALK TO SOME OF THE GIRLS OR THE PARENTS, LET'S TRY TO FIGURE OUT WHY THAT IS. HOW ABOUT KIDS WE'RE WORKING WITH THROUGH THE JUVENILE JUSTICE SYSTEM OR OTHER TEENS KIND OF SLICE IT AND DICE IT 66 ANOTHER WAY, IF YOU WILL. UH-OH, BIG DIFFERENCES. MOST OF THE KIDS OR OTHER KIDS, 95%, ARE COMPLETING THEIR HOMEWORK REGULARLY, AND ONLY 35% OF THE JUVENILE JUSTICE KIDS ARE DOING THEIR HOMEWORK. OVER HERE, 75% OF THE OTHER KIDS ARE ATTENDING SCHOOL REGULARLY, AND ONLY 30% OF JUVENILE JUSTICE KIDS ARE ATTENDING SCHOOL REGULARLY. THIS WOULD BE A REALLY USEFUL UNANIMOUS SIS FOR ME IF I'M RUNNING THIS PROGRAM. I WOULD SAY, I'VE GOT TO DO SOME DIGGING HERE. HOW ABOUT THE MENTORS? LET'S LOOK, ARE THE DIFFERENT MENTORS HAVING AN EFFECT? OUTCOMES FOR WOMEN VERSUS MEN MENTORS. IT LOOKS LIKE THE WOMEN MENTORS SEEM TO BE A LITTLE BETTER THAN THE MEN MENTORS AT A GETTING THEM TO COMPLETE THEIR HOMEWORK REGULARLY. LOOKS LIKE THE MEN MENTORS ARE QUITE A BIT BETTER AT GETTING THEM TO ATTEND SCHOOL THAN THE WOMEN MENTORS. >> THAT'S INTERESTING. YOU CAN WORK WITH THE MEN MENTORS ON HOMEWORK AND WORK WITH THE MEN MENTORS ON ATTENDANCE AND HOPEFULLY BRING THAT UP A LITTLE MORE. WHAT ABOUT PARENTS? WHAT ABOUT WHETHER THE PARENTS ATTEND TEACHER CONFERENCES OR NOT? WE'RE TALKING ABOUT PASSING GRADES HERE. LOOKS HERE LIKE 75% OF THE KIDS WHOSE PARENTS ATTEND CONFERENCES, ARE ACHIEVING PASSING GRADES. ONLY 45% IF THE PARENTS DON'T ATTEND. THERE'S SOMETHING YOU CAN WORK 67 WITH ON THE PARENTS, AND BRING CONSCIOUS THING TO WORK WITH ON THE PARENTS. THAT'S JUST SOME PRETTY BIG -- THAT'S NOT COMPLICATED STUFF, IS IT? THAT'S PRETTY BASIC STUFF BUT IT LEADS YOU TO INTERESTING INSIGHTS OR OPENS SOME PATHS FOR YOU TO FOLLOW THAT CAN PROVIDE SOME UNIVERSAL INFORMATION FOR YOU. THAT'S RUNNING THE NUMBERS. IT DOESN'T REALLY NEED TO GET ANY MORE COMPLICATED THAN THAT. FINALLY, YOU CAN ANALYZE THOSE OPEN-ENDED QUESTIONS, THE ONES LIKE GIVE ME AN EXAMPLE OF HOW YOU'RE MORE INDEPENDENT. AND THAT'S PRETTY STRAYED FORD STUFF ONCE YOU START TO DO IT. IT'S A WAY TO SUMMARIZE WORDS, NOT NUMBERS. IT'S OFTEN CALLED CONTENT ANALYSIS. YOU MAY HAVE HEARD THAT EXPRESSION BEFORE OR DONE IT YOURSELVES. IT CAN BE DONE MANUALLY, OR IT CAN BE DONE BY COMPUTER. IT JUST SIMPLY ADDS A DEEPER UNDERSTANDING TO THE NUMBERS YOU'VE ALREADY ANALYZED. FOR INSTANCE, WE KNOW IN THIS REPORT WE'RE WRITING, THAT THE PEOPLE WHO SAID THEY WERE MORE THAN INDEPENDENT, WE KNOW THE WAYS IN WHICH THEY MOSTLY SAID THEY WERE MORE INDEPENDENT. SO WE KNOW. ALL RIGHT. BOB. DRUM ROLL! TELL US WHAT OUR FIELD TEST TOLD US! >> Bob Michaels: OKAY. SO HERE WE ARE, WE'VE GATHERED THIS INFORMATION AND WE'VE PUT IT INTO SURVEY MONKEY. AND NOW WE HAVE TO ANALYZE IT. 68 WE TOOK A LOOK AT IT, AS MIKE SAID, FIRST TO SEE WHO WAS INVOLVED. THE AVERAGE CONSUMER THAT ANSWERED THE SURVEY WAS BETWEEN 46 AND 50 YEARS OLD. THIS SHOWS THE SPREAD. 20, 12% BETWEEN 42 AND 48, AND 14% AT 63-69. YOU SEE THE OBVIOUS SPREAD. ETHNICITY, MOST OF THE CONSUMERS WERE WHITE, 74% WERE WHITE, I'M SORRY, I'VE GOT GET ON THE OTHER SIDE, HERE. 50% DISABILITIES OF CONSUMERS, 40% WERE PHYSICAL, 32% MULTIPLE, 13% COGNITIVE, 10% MENTAL-EMOTIONAL, AND 4% SENSORY. OKAY. THAT WAS A LITTLE ABOUT WHO IT WAS THAT WE WERE WORKING WITH. LET'S TAKE A LOOK AT -- AS MIKE WAS SAYING, WE DIDN'T FIND A LOT OF DIFFERENCE BETWEEN THE PEOPLE WHO WERE SERVED, THE CORE OF SERVICES THAT PEOPLE RECEIVED OR ANYTHING LIKE THAT. THIS IS EACH ONE OF THE INDICATORS. YOU WILL GET NOW A LIST. INDICATORS THAT WAS IN YOUR FOLDER. REMEMBER, IT'S A ONE-PAGE FORM OF ALL THE INDICATORS. THIS IS THE FIRST ONE. THE NUMBER AND PERCENT OF CONSUMERS SERVED BY THE CIL WITHIN THE LAST NINE MONTHS OF THE PAST FEDERAL FISCAL YEAR, WHO CAN LIST AT LEAST ONE SPECIFIC SKILL, TYPE OF KNOWLEDGE OR RESOURCE THEY DIDN'T HAVE BEFORE THEY CAME TO THE CENTER. 70% SAID YES, THEY WERE ABLE TO DO THAT, 30% WERE NOT. THIS IS THE RESULT OF THE SECOND YEAR OF THE SURVEY. OUTCOME FINDINGS FOR NO. 2, 69 NUMBER AND PERCENT OF CONSUMERS SERVED BY THE CIL WITHIN THE LAST NINE MONTHS, WHO CAN LIST ONE SPECIFIC WAY IN WHICH THEY ARE MORE INDEPENDENT THAN THEY WERE WHEN THEY APPROACHED THE CIL. 2% SAID THEY WERE MORE INDEPENDENT, 39% SAID SAME DEGREE, 6% SAID LESS INDEPENDENT. OKAY? MAUREEN? >>> THIS IS MAUREEN FROM WISCONSIN. WE TRIED TO ANALYZE SOME OF OUR DATA AND ASKED PEOPLE OUT OF INSTITUTIONS OR DIVERSION GOING INTO THEM. SO THE PEOPLE HAD EXPERIENCE WITH A CIL. BUT WE CAN'T TAKE CREDIT FOR ALL THAT WENT ON IN THEIR LIFE AT THAT POINT. WE HAD SOME LEGISLATORS POKE HOLES IN THAT LOGIC. I HOPE WE CAN HAVE SOME GOOD RATIONALE FOR HOW THEY WERE ASKED THIS TYPE OF QUESTION, TO COME BACK WITH. THEY MAYBE BECOME LESS INDEPENDENT FOR REASONING WAY BEYOND OUR CONTROL, PARTICULARLY HEALTH ISSUES, CARE PROVIDER, ALL THOSE KIND OF THINGS. THEY MAY BE MORE INDEPENDENT FOR REASONS OUT OF WHAT THEY WERE SEEN THROUGH THE CENTER. I DON'T KNOW, I DON'T HAVE ANY SUGGESTION. BUT I'M SURE -- >> Bob Michaels: WE CAN ALWAYS RUN INTO THAT PROBLEM OF, YOU KNOW, WAS THAT INDEPENDENCE CAUSED BY THE CENTER OR NOT? YOU KNOW, WE DON'T KNOW OR WOULDN'T KNOW, THEY ARE NOW MORE THAN THEY WERE WHEN THEY CAME TO THE CENTER. AND THAT'S ABOUT ALL YOU CAN 70 MAKE OF IT. YEAH. >>> THIS IS JULIA FROM CHARLOTTE AGAIN. I APPRECIATE MAUREEN BRINGING THAT UP. I'LL TELL YOU, SEVERAL YEARS AGO THE HARRIS POLL DID A SURVEY OF PEOPLE WITH DISABILITIES IN THE COUNTRY AFTER THE ADA WAS SIGNED AND ASKED PEOPLE WITH DISABILITIES IF THEY WERE MORE INDEPENDENT OR WHATEVER. THE ANSWER WITH A NO. THEY WEREN'T -- PEOPLE WITH DISABILITIES GENERALLY WEREN'T. AND I CAN REMEMBER -- THIS WAS BEFORE THE CONSOLIDATION OF THE LOCAL REGIONAL RSA OFFICES BACK TO WASHINGTON WHEN WE HAD A REGIONAL RSA OFFICE. AFTER THAT HARRIS POLL WE HAD A MEETING WITH OUR LOCAL RSA REGIONAL PERSON WHO LOOKED AT US, HELD UP THAT EVALUATION FROM HARRIS AND SAID, WHAT'S WRONG WITH YOU PEOPLE? >> YEAH, YEAH. >> Audience Member: DAN WAS THERE, YOU REMEMBER THAT. >> WE WERE A SHY BUNCH DOWN IN THE SOUTH. BASICALLY WE LOOKED AT HER AND SAID, BUT THEY ARE HAPPY WITH PEER COUNSEL, INR, INDEPENDENT LIVING SKILLS AND ADVOCACY, WHICH IS WHAT WE DO. WHAT THEY ARE NOT HAPPY WITH IS THINGS YOU DON'T PAY US TO DO. SO I THINK THAT WAS AN AHA MOMENT FOR ME, THAT WE CAN BE PAINTED WITH A BRUSH THAT, OKAY, I DID NOT CREATE TOTAL INDEPENDENCE FOR THE WORLD AND WORLD PEACE AND HARMONY. BUT THEN I DID EVERYTHING I GOT PAID TO DO AND THEN SOME. SO WE HAVE -- >> Bob Michaels: THAT'S ALL WE CAN MEASURE. 71 >> Audience Member: THAT'S ALL WE CAN MEASURE, THAT'S NOT ALL THE FEDS CAN HOLD US ACCOUNTABLE FOR. I THINK RENEE IS IN THE ROOM AND HEARD ME SAY THAT. JUST THINKING. >> Mike Hendricks: CAN I MAKE A COMMENT? >> THANK YOU, LYNNAE. >> Mike Hendricks: I WANT TO MAKE A COMMENT IF I CAN. YOU'RE RAISING THE ISSUE OF CAUSABLE ATTRIBUTION OR WHAT SOME PEOPLE ARE WILLING TO CALL PLAUSIBLE ASSOCIATION, WHICH ISN'T QUITE AS RIGOROUS BUT IS STILL A STANDARD YOU HAVE TO BE HELD TO. THIS IS AN ISSUE, THIS ABSOLUTELY IS AN IMPORTANT ISSUE. IT'S NOT JUST AN ISSUE FOR CILs OR FOR THIS FIELD TEST, IT'S AN ISSUE FOR ANY EFFORT TO GO OUT AND MEASURE OUTCOMES AND TRY TO SAY THAT YOU HAD SOMETHING TO DO WITH THEM. THIS IS A BIG ISSUE NATIONALLY FOR THIS. NOW, ARE THERE THINGS YOU CAN DO ABOUT IT? YES, ABSOLUTELY. THERE ARE THINGS WE CAN DO TO PIN DOWN OR CONTRIBUTION TO THAT. DO WE WANT TO SPEND THE EFFORT AND MONEY? THAT'S THE QUESTION. IT'S NOT THAT IT'S METHOD LOGICALLY NOT POSSIBLE, IT'S DO WE WANT TO SPEND O DOES RSA WANT TO HELP US HAVE MONEY TO DO THAT KIND OF EFFORT. THAT'S GOING TO HAVE TO BE A JUDGMENT CALL AS TO HOW IMPORTANT IS THE RIG RIGOR OF THE PLAUSIBLE ASSOCIATION GOING TO BE? THE FIELD DOESN'T KNOW THE 72 ANSWER TO THAT YET BUT IT IS GOING TO BE SOMETHING WHICH IS GOING TO BE WRESTLED WITH. >> Bob Michaels: SO WE CAN TAKE QUESTIONS. SO THAT'S THE SECOND INDICATOR. THIRD ONE, THE NUMBER AND PERCENT OF PERSONS WITH DISABILITIES CONTACTING THE CIL DURING THE LAST NINE MONTHS OF THE PAST FEDERAL FISCAL YEAR WHO REPORT THAT THEY HAVE INFORMATION THEY REQUESTED FROM THE CIL. 72% SAID YES, AND 15% SAID NO, AND 13% DIDN'T REMEMBER. OKAY? NO. 4, NUMBER AND PERCENT OF PEOPLE WITH DISABILITIES CONTACTING THE CIL DURING THE LAST NINE MONTHS OF THE PAST FEDERAL FISCAL YEAR WHO REPORT THAT THEY USED A NEW RESOURCE THAT THEY LEARNED ABOUT FROM THE CILs. 52% SAID YES, 32% SAID NO AND 17% DON'T REMEMBER. A DIFFERS BETWEEN WHETHER THEY GOT -- >> Mike Hendricks: BOB? CAN I MENTION SOMETHING HERE? ONE OF THE THINGS BOB AND I REMEMBER WORRIED ABOUT WAS THAT PEOPLE WERE GOING JUST MAKE THEMSELVES LOOK GOOD. COME BACK WITH DATA, TRYING TO SKEW THE INTERVIEWING AND COME BACK WITH DATA IN SUCH A WAY THAT WE JUST LOOKED REALLY GOOD. WE WERE ALWAYS ON -- WE HAD OUR EYES OPEN FOR WHETHER THESE DATA WERE CREDIBLE OR NOT. DID THEY SEEM TO BE CREDIBLE. THERE WERE A COUPLE THINGS THAT TOLD THAWS THESE DID SEEM CREDIBLE. ONE OF THEM, IF I CAN POINDER POINT IT OUT, IF YOU WERE GOING TO SKEW IT AND MAKE YOURSELF LOOK ABOUT, YOU WOULDN'T SAY 72% 73 OF PEOPLE THAT CALLED YOUR I&R, USE MAKE IT 95%, REALLY HIGH. 13% OF THE PEOPLE SAID NO, I DIDN'T GET THE INFORMATION I NEEDED. THAT GIVES SOME CREDIBILITY TO US. 72% SAID THEY GOT IT, AND ONLY 52% AS SAID THEY USED IT. THAT MAKES SENSE, NOT EVERYBODY WHO GETS INFORMATION IS GOING TO USE IT. THE FACT THAT IT DROPPED FROM 72% TO 52% SAID TO US, MAYBE WE CAN TRUST THESE DATA. >> Bob Michaels: DAN. >> Audience Member: DAN KESSLER OUT OF ALABAMA, JUST TO CLARIFY THE RESULTS HERE. ON PAGE 21, THE NUMBER OF PEOPLE WITH DISABILITIES CONTACTING THE CIL, THAT REPORT THEY USED THE NEW RESOURCE: THOSE NUMBERS THEN ARE -- OR THOSE INDIVIDUALS WERE AT 72%, RIGHT? OBVIOUSLY ON THE PREVIOUS QUESTION, IF YOU DIDN'T GET ANY INFORMATION OR DON'T REMEMBER IT, YOU'RE NOT GOING USE A RIGHT THAT -- >> Bob Michaels: YEAH. WE ASKED THE QUESTIONS TO EVERYBODY, DIDN'T WE? >> Audience Member: THAT IS PART OF THE SKIP LOGIC? >> Bob Michaels: I DON'T THINK SO. I THINK WE ASKED BOTH QUESTIONS OF EVERYBODY. >> Audience Member: AND A RELATED QUESTION,, TOO IS THE PEOPLE WHO CONSUMED -- WE DO HAVE CONSUMERS WHO ON A REGULAR BASIS RECEIVE INFORMATION FROM CONSUMER SERVICES. WERE THEY INCLUDED IN THE I&R LIST? TOBACCO TOBACCO I'M SORRY. >> SOMEBODY WITH A CSR, WOULD THEY HAVE BEEN EXCLUDED? 74 >> Bob Michaels: YES, YES. >> Audience Member: THERE'S NO OVERLAP? >> Bob Michaels: RIGHT. QUESTION? >> Audience Member: THIS KIND OF DATA IS WHAT I WOULD DESCRIBE AS THE TIME TO SEE HOW EFFECTIVELY WE STAFF AND HOW MUCH DIFFERENCE WE'RE MAKING IN PEOPLE'S LIVES. I HOPE WE CAN PUT ON THE LIST, HOW MUCH MONEY ARE WE SAVING THE TAXPAYER WHEN WE DO WHAT WE DO. THAT KIND OF OUTCOME DATA, HOW MANY PEOPLE WE GOT OUT OF NURSING HOMES. SO WHEN WE GO TO CONGRESS WE CAN HAVE THAT KIND OF DATA. ARE WE GOING TO TALK ABOUT HOW YOU CREATE THOSE KINDS OF OUTCOME DATA? >> THE ONE TALKS ABOUT DEINSTITUTIONALIZATION. >> Bob Michaels: DEALING WITH THE MONEY AND ALL. WE CAN AND WE'LL MAKE SURE WE DO TALK MORE ABOUT IT. >> Mike Hendricks: NONE OF THE INDICATORS THIS YEAR DEALT DIRECTLY WITH FINANCIAL QUESTIONS, YOU'RE CORRECT. MAYBE WE SHOULD IN THE FUTURE. THAT'S WHY WE'RE ALL LEARNING. >> Bob Michaels: NEXT ONE, THE NUMBER AND PERCENT OF CONSUMERS WHO CAN LIST AT LEAST ONE SPECIFIC PERSONAL ADVOCACY ACTIVITY THEY ENGAGED IN, OKAY? 59% SAID YES. AND 41% SAID NO. OKAY? SO THAT MEANS THAT PEOPLE WHO SPOKE OR DID SOMETHING FOR THEMSELVES. TO ADVOCATE ON THEIR OWN BEHALF. WE ASKED THIS IN TWO PARTS. WE SAID DID YOU DO IT ON YOUR OWN BEHALF. THE NEXT ONE WAS, DID YOU TAKE WHAT YOU LEARNED AND USED IT FOR 75 SOMEBODY ELSE, DID YOU ADVOCATE FOR ANYBODY ELSE. THAT WAS THE NEXT ONE. SO NUMBER AND PERCENT OF CONSUMERS THAT ARE SERVED BY THE CENTER WITHIN THE LAST NINE MONTHS WHO CAN LIST AT LEAST ONE SPECIFIC SYSTEMS ADVOCACY ACTIVITY THEY ENGAGED IN. 72% HAVE SAID NO, AND ONLY 28% SAID YES. . SO THOSE 59% SAID THEY LOOK OUT FOR THEMSELVES. ONLY 28% LOOKED OUT FOR OTHER PEOPLE. SO WRITE COME FROM, THAT WOULD SAY, WE'VE GOT TO WORK ON THAT. WE KNOW THAT ONCE PEOPLE START TO LOOK OUT FOR THEMSELVES, THEY HAVE TO GO THROUGH THAT STAGE FIRST BEFORE THEY START TO SPEAK FOR OTHER PEOPLE. SO YOU REALLY NEED TO TAKE THAT BEYOND. IT'S PRETTY CLEAR HERE THAT LOTS OF OUR CONSUMERS DO NOT DO THAT. THE NUMBER AND PERCENT OF ACTIVITIES CONDUCTED SUCH AS SURVEYS, PUBLIC MEETINGS, FOCUS GROUPS AND SO FORTH DURING THE PAST CALENDAR YEAR TO IDENTIFY OR CONFIRM THE PRIMARY BARRIERS AND PROBLEMS IN THE COMMUNITY. REMEMBER, WE HAD ONE WHERE WE TALKED ABOUT THE BARRIERS AND PROBLEMS ARE IDENTIFIED, GOING OUT TO THE COMMUNITY AND FINDING OUT WHAT IT IS THEY WANT TO DO. WELL, WE FOUND THAT MOST OF THE CENTERS DID UP TO 13 CENTERS DID BETWEEN ZERO AND 10 THINGS TO GATHER INFORMATION. AND THEN YOU CAN SEE HOW IT'S SPREAD OUT. 11-20 IS THREE. AND THERE ARE 2 CENTERS THAT DID OVER 120 THINGS. I CAN'T IMAGINE WHAT THEY WERE. 76 MAYBE SURVEYED 120 PEOPLE. BUT WE REALLY TRIED HARD TO GET PEOPLE REALLY TO KNOW AND UNDERSTAND THAT IF YOU DID A SURVEY THAT WAS ONE. SO WE THINK 13 CENTERS DID BETWEEN 0-10. OKAY. THE PRESENCE WITHIN THE CENTER'S ANNUAL PLAN OF A SEPARATE SECTION CONTAINING AN EXPLICIT SYSTEMS ADVOCACY WORK PLAN. AND 57% OF THE PEOPLE, OF THE CENTERS HAD THAT, AND 43% OF THE CENTERS DID NOT. >> Mike Hendricks: OTHER WAY AROUND. >> Bob Michaels: O SORRY. 57% OF CENTERS DID NOT HAVE IT, AND 43% OF THE CENTERS DID. SO YOU SEE WHAT WE'RE TRYING TO DO THERE? YOU WENT OUT AND ADVOCATED, FOUND OUT WHAT IT IS THAT YOUR COMMUNITY WANTS YOU TO DO. THE NEXT STEP WOULD BE TO SET UP A PLAN FOR DOING IT. THAT PLAN WOULD BE, YOU KNOW, WORKING ON THOSE SPECIFIC THINGS THAT WERE IDENTIFIED BY YOUR COMMUNITY. AS YOU SEE, MOST CENTERS DON'T DO THAT. NOW, WHAT WE DID, AT FIRST WE SAID -- THE FIRST YEAR WE SAID IT HAD TO BE -- WAIT A MINUTE, NO, IT WASN'T. WE SAID IT HAD HAD TO BE FOUR DIFFERENT REQUIREMENTS THAT WOULD CONSTITUTE A PLAN. THAT WAS HAVING GOALS AND OBJECTIVES. WITH THE PEOPLE RESPONSIBLE WERE IDENTIFIED, IN SOME OTHER PLAN THAT YOU HAD. THAT COULD YOU COUNTED. STILL, WE HAD 57% OF THE CENTERS DO NOT DO THAT. OKAY? OUTCOME FINDING FOR INDICATOR 77 NO. 9. NUMBER OF POSITIVE CHANGES ACHIEVED OR NEGATIVE CHANGES PREVENTED DURING THE PAST CALENDAR YEAR IN LEGISLATION, POLICIES, PRACTICES OR SERVICES AT THE LOCAL, STATE OR FEDERAL LEVEL THAT ADDRESS THE BARRIERS AND PROBLEMS IDENTIFIED BY THE CENTER'S CONSUMERS. NOW, THIS IS THE PART THAT WAS REALLY TOUGH. AGAIN, GETTING THE CENTERS TO UNDERSTAND THIS. YOU CAN SAY TO PEOPLE, YOU KNOW, YOU'VE GOT THESE THREE OR FOUR THINGS NOW THAT HAVE BEEN IDENTIFIED. YOU DEVELOP A PLAN TO MEET THOSE NEEDS. NOW, WHAT SUCCESSES DID YOU HAVE RELATED TO THOSE THREE OR FOUR THINGS? NOT THE SUCCESSES YOU HAD OTHERWISING BUT JUST RELATED TO THOSE THREE OR FOUR THINGS. WE'RE REALLY GETTING CENTERS TO UNDERSTAND THIS AND GUN TAKE THAT INTO CONVERSATION, IT WAS TOUGH. WE THINK WE'RE PRETTY SUCCESSFUL IN DOING THAT. YOU KNOW, EVEN CENTERS LIKE -- WE HAD LOTS OF FUNDING PROBLEMS THIS LAST YEAR. A LOT OF CENTERS SPENT ALL THEIR TIME DOING THAT, YOU KNOW? THEY COULDN'T COUNT IT BECAUSE THEY WERE WORKING ON OTHER THINGS. SOMETIMES HAVE YOU THINGS, WHEN WE SAY POSITIVE THINGS DONE OR NEGLECT THINGS PREVENTED. SO MAYBE ONE OF YOUR THINGS IS FUNDING FOR A PAS PROGRAM, PERSONAL ASSISTANTS PROGRAM, AND YOU PREVENT THAT CUT FROM HAPPENING OR DRASTICALLY REDUCE THE CUT, SO IT'S ISN'T AS BASS BAD AS IT SHOULD BE. 78 GENERATED FOR THE REPORT, ONE OF AREAS IDENTIFIED BY YOUR COMMUNITY. OKAY. SO WHAT HAPPENED HERE WAS THAT EIGHT OF THE CENTERS ABLE TO PROVIDE TWINE 0-10, 5 BETWEEN 11-20, THREE BETWEEN 21-30. THERE WERE THREE MORE THAT WERE MUCH MORE THAN THAT. ANY QUESTIONS ABOUT THIS? KELLY? PLEASE IDENTIFY YOURSELF. >>> KELLY BUCKHAM, DIRECTOR. I JUST HAD A QUESTION. BACK A FEW SLIDES, THE ONE ON SYSTEMS ADVOCACY. >> Bob Michaels: YEAH. >> Audience Member: SO GIVEN THE FACT THAT 24% OF THE CONSUMERS GOT INVOLVED IN THE SYSTEMS ADVOCACY ACTIVITY, WHAT WOULD YOU THINK IS AN APPROPRIATE NUMBER? I KIND OF LOOKED AT THAT AND THOUGHT, LET'S SAY JULIA'S NUMBER FOR EXAMPLE, 200-SOME CONSUMERS, AND 24% MEANS THAT BASICALLY 50 PEOPLE GOT INVOLVED. TO ME THAT, ACTUALLY IS A PRETTY HIGH NUMBER. THERE'S A LOT OF PEOPLE THAT AREN'T GOING TO CHOOSE TO DO THAT. >> Bob Michaels: RIGHT, RIGHT, WE KNOW THAT. IT'S HOW WE LOOKED AT THAT. SOME PEOPLE WOULD BE SATISFIED WITH 28%, AT LEAST THAT'S 25-28, AND SOME CENTERS WOULDN'T. IT'S UP TO THE CENTERS TO DECIDE WHAT STANDARDS THEY WANT TO SET FOR THEMSELVES. ON HERE, WHEN YOU'RE REVIEWING YOUR DIG, YOU'LL SEE 28, YOU MIGHT SET EIGHT. OUR GOAL DOESN'T NEED TO CHANGE FOR THAT. AT SOME POINT IN THE FUTURE YOU 79 COULD PROVIDE 50%, AND THAT'S DOING THE MANAGEMENT MORE THAN THE DATA. IT'S WHAT CAN YOU DO TO CHANGE THAT. MAUREEN. >> Audience Member: MAYBE I'M NOT UNDERSTANDING THIS CORRECTLY. UNDER THE OUTCOME FINDINGS FOR INDICATOR NO. 9, THERE WERE ONLY EIGHT CILs THAT REPORTED. >> THERE WERE EIGHT THAT REPORTED AND A SEATTLE OF 19 IS RECORDED. >> THAT'S RECORDED BECAUSE ONLY 43% OF THEM HAD AN ADVOCACY PLAN OUT OF THE 27 TOTAL? IS THAT HOW THAT -- >> Mike Hendricks: ACTUALLY, ONE OF THE THINGS WE FOUND YOU'RE JUST NOW POINTING OUT, THAT IS NOT EVERY CIL COULD ANSWER EVERY QUESTION. SOME OF THEM SUL SAID, DARN GOOD QUESTIONS, WE HAVE NO WAY TO KNOW. THAT ALSO IS GOOD INFORMATION. THIS IS THE CASE WITH NINE OF THE 28 SELL CILs COULDN'T TELL US BOB BEEN AND JUST COULDN'T RECORD THE INFORMATION. >> Audience Member: WASN'T ONE OF THE QUESTIONS, WE DON'T DO SYSTEMS ADVOCACY WORK. I WOULD BE INTERESTED IN KNOWING THE NUMBER OF CILs IN THE WHOLE HARRIS GROUP THAT FLAT-OUT SAY THEY DON'T DO IT. >> Bob Michaels: INNURE 10, THE NUMBER AND PERCENT OF CONSUMERS SERVED BY THE CENTER WITHIN THE PAST CALENDAR YEAR, WHO MOVED OUT OF AN INSTITUTION AND INTO A SELF-DIRECTED COMMUNITY-BASED SETTING. OKAY. I THINK THE IMPORTANT THING IS THE LITTLE BLOCKS ON THE LEFT, AS I RECALL. 80 70% OF CONSUMERS, THAT WE SURVEY WHEN THEY MOVE OUT. 70%. OKAY. NOW, ONE PROBLEM WE FOUND OUT LOOKING AT THIS IS THAT DIFFERENT CENTERS DO DIFFERENT THINGS. SOME CENTERS IDENTIFY EVERYBODY THAT FORTUNE THOUGHT ME MIGHT WANT TO MOVE ON IN AN INSTITUTION. OTHER CENTERS DON'T BRING SUSPECT INTO A SYSTEM INDIANAPOLIS THINK SAY, THEY WANT TO MOVE OUT. YOU MIGHT HAVE A CENTER THAT HAS THREE PEOPLE MOVE OUT, AND THREE OUT OF THREE. THEN YOU MIGHT HAVE ANOTHER CENTER THAT HAS 100 BUT 20 MOVED OUT, YOU ONLY HAVE 20% THAN UNPERCENTED TAKE A LOOK AT THIS DATA AND MAKE SURE YOU UNDERSTAND WHAT YOU NEED TO DO TO DO A BETTER JOB. OKAY? AND THE LAST ONE, THE NUMBER AND PERCENT OF CONSUMERS SERVED BY THE CIL WITHIN THE PAST CALENDAR YEAR WHO REMAINED IN A SELF-DIRECTED COMMUNAL BASIS SETTING TWICE APPROXIMATELY AT RISK OF MOVING INTO AN INSTITUTION. AND AGAIN, IT WAS 85 PERSONS, 85% OF PEOPLE WERE NAMED IN THE -- TO MAKE COMMUNITY USING THE INSTITUTION. THIS WAS A TOUGH ONE. THAT WAS PROBABLY THE HARDEST ONE OF ALL OF THEM TO DECIDE WHO WAS AT RISK AND WHO WASN'T. ONE OF THE THINGS WE DID AFTER A WHILE IS THAT WE SAID, LET'S MAKE THE DEFINITION REFLECT WHAT THEIR COMMUNITIES STANDARDS ARE. IF A COMMUNITY HAS SOMETHING WHERE THEY SAY A DEFINITION, WE'RE AT RISK, THEN YOU CAN USE 81 THAT AS THE STANDARD. LIKE IN ARIZONA WE HAVE A -- BEFORE THE CENTER GETS THE CONSUME TORE WORK WITH, THAT CONSUMER IS IDENTIFIED AS TO WHETHER OR NOT THEY ARE A RISK. PRIOR TO TO THE TIME WE GET THEM. A NUMBER OF STATES HAVE A SYSTEM LIKE THAT. IF NOT, YEAH, USE IT FOR A DEFINITION IF YOUR STATE DIDN'T HAVE ONE. ANY QUESTIONS? WE ALSO GAVE EACH CENTER A CONFIDENTIAL REPORT SHOWING HOW THAT CENTER COMPARES WITH ALL OF THE OTHERS COMBINED. THE REPORT LOOKS LIKE THIS AND YOU CAN SEE IT IN YOUR FOLDER. WHAT IT D YOU CAN COMPARED ALL 28 CENTERS IN ONE COLUMN, PRETTY SURE OF THE STANDARDS. AND THEN THE SECOND CALL-DOWN WAS FOR THAT FIRE AND RESCUE SO, THEY COULD SEE HOW THEY COMPARED AGAINST ALL THE OTHERS. OKAY? QUESTIONS YEAH. >> Audience Member: DAN RES LER, ALABAMA. WOULD A PREMATURE -- WOULD IT BE PREMATURE TO SAY THESE WOULD BE BENCHMARKS, FOR CILs, THESE NUMBERS HERE ON THIS FORM? >> Bob Michaels: I THINK IT IS BECAUSE I THINK THE BENCHMARK WOULD BE CENTER BY CENTER, WOULDN'T IT? YOU DEALT YOUR OWN BENCHMARKS? I DON'T KNOW. >> Mike Hendricks: IF BY BENCHMARK YOU MIGHT BE -- YOU MEAN BASELINE MAYBE? SORT OF WHERE WE ARE RIGHT TODAY? >> Audience Member: YEAH, COMPARED TO THE REST OF THE COUNTRY. >> Mike Hendricks: THEN YOU 82 WOULD SAY IT'S THE BEST WE'VE GOT SO FAR. I THINK THAT'S A FAIR STATEMENT, DON'T YOU? IT'S THE BEST WE'VE GOT SO FAR. IS IT WHERE WE WANT TO BE IN FIVE YEARS? PROBABLY NOT. DOES IT COVER THE RIGHT QUESTIONS, DOES IT COVER IT THE WAY WE WANT TO, PROBABLY NOT, BUT IT'S THE BEST WE HAVE RIGHT NOW. WE PROMISED EVERY CIL FROM THE VERY BEGINNING THAT WE WOULD NOT EVER USE THIS DATA TO COMPARE THEM AGAINST EACH OTHER. THAT WAS REAL IMPORTANT, BECAUSE WE TALKED YESTERDAY ABOUT DEGREE OF DIFFICULTY. YOU KNOW, NONE OF US NOW HOW HARD YOUR CONSUMERS HAVE IT AT YOUR CIL, WHETHER THEY ARE IN WORTH SHAKE THAN EVERYBODY ELSE'S. WE SAID WE ARE NOT GOING TO PUT YOU IN THE AWKWARD POSITION OF RANK ORDERING THE 28 CILS FROM BEST TO WORK ON THE BASIS OF OUTCOMES. THAT IS A VERY REAL FEAR THAT SOMEBODY MIGHT ONE DAY. THERE'S A FEAR OF NOT USING OUTCOMES TO RANK-ORDER CILs. THERE IS A DEGREE OF DIFFICULTY THAT'S REALLY, REALLY IMPORTANT. WE SAID WE DON'T DO THAT. WHAT WE DID SAY WAS THAT WE WOULD GIVE YOU BACK INFORMATION ON HOW YOU'RE DOING, COMPARED TO EVERYBODY ELSE. THAT'S WHAT THIS WAS. ONLY THE CIL GOES THIS. THE CILs THAT HAVE PRETTY MUCH THE REALLY TEAM TO HAVE LIKE THIS REPORT. IT GAVE THEM SOME PERSPECTIVE AT LEAST ON HOW THEY WERE DOING COMPARED TO ALL THE OTHER CILs. 83 I COULD SEE ON A NATIONAL LEVEL SOMETHING LIKE THIS COMPARED TO THE NATIONAL STATS COULD BE GREAT. BUT THERE'S THIS ISSUE, THIS CONUNDRUM OF HOW TO PROTECT AND MAKE SURE YOU DON'T HURT CILs BY PROVIDING OUTCOME INFORMATION BUT HELPING. >> Bob Michaels: RIGHT. RICHARD. >> Audience Member: THANK YOU. THIS IS NOT A QUESTION, IT'S JUST A TECHNICAL UPDATE. AND AS MUCH FOR THOSE PEOPLE WHO ARE PARTICIPATING BY THE WEB. RICHARD PETTY, ILRU. THIS MORNING WE HAD SEVERAL DROPOUTS AND WE'VE SEEN NOTES THAT CAROL HAS SEAFOOD I HAVE RECEIVED E-MAILS ABOUT THEM. THERE IS A WEB PROBLEM WITHIN THE HOTEL'S NETWORK, AND THEY, AS I UNDERSTAND IT, ARE WORKING TO RESOLVE IT. THERE HAVE BEEN INTERMITTENT DROPOUTS. WE THINK THEY ARE RESOLVED AND THOUGHT THEY WERE SEVERAL TIMES, BUT PERHAPS THEY ARE NOW RESOLVED FULLY. THOSE PARTICIPATING BY WEB SHOULD HAVE BETTER WEB COVERAGE FOR THE REST OF THE DAY. THANK YOU. >> Audience Member: ALSO, WE SENT AN E-MAIL TO OUR ONLINE AUDIENCE ASKING THEM TO SUBMIT QUESTIONS AND COMMENTS. SO I'LL BE WATCHING FOR THEM. >> Bob Michaels: ANYTHING NOW? >> Audience Member: NOTHING RIGHT NOW. >> Audience Member: THIS IS RICHARD PETTY AGAIN, BOB AND MIKE, I KNOW THERE WERE A COUPLE OF WEB QUESTIONS BY THE END OF THE DAY YESTERDAY. ONE OF THEM WORKS BEGAN TO ADDRESS THIS WARNING THAT HAD TO 84 DO WITH, WELL -- WE'LL COME BACK TO IT. BUT AVERAGE LUNCH PERHAPS YOU GUYS CAN COME BACK TO THE QUESTIONS AND REREVISIT THEM AND MAKE SURE WE'VE GOT THEM NAILED DOWN. >> Mike Hendricks: RICHARD, YOU MIGHT NOT HAVE BEEN IN THE ROOM WHEN WE SAID TOMORROW OR LAST SESSION IS 75 MINUTES, COMPLETELY UNSTRUCTURED RIGHT NOW, FOR WHATEVER'S ON PEOPLE'S MINDS. WE'RE DEFINITELY GOING TO ADDRESS A WHOLE BUNCH OF THOSE THEN IF NOT BEFORE. >> Audience Member: I MAY HAVE MISSED THAT, THANK YOU VERY MUCH. >> Mike Hendricks: I THINK ONE OF THE THINGS BOB AND I WERE HOPING FOR WOULD BE TO STIMULATE A DISCUSSION OF DO YOU THINK THIS KIND OF INFORMATION WOULD BE USEFUL TO YOU? IF WE HAD IT AS A FIELD, WOULD IT BE HELPFUL? WOULD IT NOT BE HERSELF? WHAT MORE DO WE NEED? SOMEONE SAID WE NEED SOME MONEY INFORMATION. THAT WOULD BE A GOOD IDEA. YOU'RE THE FIRST ONES SEEING THIS, GIVE US SOME FEEDBACK, PRO AND COM, WHATEVER. >> Audience Member: I'D LIKE TO SEE WHAT KINDS OF, BASED UPON THE INFORMATION CIL HAS GOT BACK ON THIS, WHAT KIND OF CHANGES HAPPEN IN THE PROGRAMS, IF ANY OF THEM SHARE THAT WITH YOU. OR IS IT JUST TOO EARLY TO TELL? >> Bob Michaels: YEAH, WE PROVIDED THIS INFORMATION OVER A COUPLE YEARS NOW. IT'S REALLY INTERESTING, THE KIND OF THINGS WE HEARD BACK FROM PEOPLE. ONE NOR WAS TELLING US, WHEN I 85 LOOKED OVER THE DATA, I STARTED TO REALIZE OUR ABILITY TO CALL PEOPLE BACK ON LINE IN ART WAS -- ONE STAFF MEMBER WAS REALLY PARTICULAR ABOUT MAKING SURE SHE HAD GOTTEN ALL THE INFORMATION AND THE OTHER WASN'T. EVERY TIME ONE PUT TOGETHER A LIST, THEY HAD A LIST OF -- THAT JUST CAME FROM ONE STAFF MEMBER. SO THE DIFFERENCE BETWEEN THE FIRST YEAR AND THE SECOND YEAR IS THAT REALLY GOT DOWN ON YOUR BACK AND MADE SURE THAT THEY RESPONDED. THERE WERE A LOT OF LITTLE THINGS LIKE THAT. PEOPLE SAID, YOU KNOW, WE DON'T HAVE A SYSTEMS APP AS PLAN. BUT WE WILL NEXT YEAR. OR IT WAS ALL IN MY HEAD, AND I HAVEN'T WRITTEN IT DOWN. AND THEN A LOT OF CENTERS REALLY AREN'T IDENTIFYING WHAT THEY THOUGHT THEY KNEW WHAT THEIR COMMUNITY WANT. SO THERE'S A LOT OF RESISTANCE TO THAT. WE JUST KIND OF STUCK WITH THAT, THOUGHT THE TASK FORCE WAS REALLY IMPORTANT TO DO IT. SO WE SUSPECT THAT THE ADVOCACY PLANS WILL BECOME MORE AND MORE EVIDENCE AS THE YEARS GO BY. >>> ANN WEEKS FROM WEST VIRGINIA. I THINK THERE ARE SEVERAL THINGS WHEN. DAN WAS ASKING B IS THIS A BENCHMARK FOR CENTERS. BUT WHEN MIKE WAS TALKING ABOUT THAT YOU ALL WERE PROMISING THE CENTERS IN THE STUDY THAT YOU WERE GOING TO PROTECT THEM, THERE THERE WASN'T GOING TO BE THAT COMPARE ON. IS THAT A BENCHMARK? IF HE LOOK AT THE CENTERS AS A WHOLE, THAT IT COULD BE A DANGER THAT THEY START LOOKING AT. 86 THERE ARE TOO MANY AREA, WHAT COMMUNITY SERVICES ARE AVAILABLE AND WHAT SERVICES AREN'T. SO I THINK THAT COMPARISON COULD BE REALLY TRICKY. BUT I THINK SOMETHING LIKE, WHAT DID THE CENTERS DO WITH THE INFORMATION ONCE THEY GOT IT WOULD BE SOMETHING THAT WOULD BE VERY INTERESTING, THAT COULD BE MORE READILY AVAILABLE ON A NATIONAL LEVEL WITHOUT ENDANGERING THE CENTER. BECAUSE THIS WOULD INTERESTING. >> Bob Michaels: MIKE GUSTYING TO TALK ABOUT HOW DO YOU USE THIS INFORMATION? HE HAD THE VERY FIRST SESSION, OUTCOME MEASURES AND OUTCOME MANAGEMENT. REALLY TAKING THE INFORMATION YOUR GATHER EITHER. >> I'VE ALWAYS BEEN SCARED TO OFFSET THE PROCESS. WE ARE TRYING TO BE AWARE OF HOW THIS INFORMATION CAN BE USED AND HOW CENTERS CAN BE COMPARED AGAINST EACH OTHERS. THE BAD THINGS WE'RE AFRAID OF, THAT WOULD GET AWAY FROM THE END. THAT'S I THINK ONE OF THE PROBLEMS WE HAVE WITH THE THREE AREAS IS THAT THEY HAVE ALREADY CHOSEN, WELL, THIS SOUNDS REALLY GOOD BECAUSE THEY HAD THREE UNDER HERE AND SUDDENLY THE CENTER IS NOT GOOD, WELL, THAT ISN'T IMPORTANT TO THEM. EVER SINCE FROM THE FIRST TIME WE GOT TOGETHER AND STARTED TO TALK ABOUT WHAT CENTERS WORK -- YOU KNOW, THIS IS 20 YEARS AGO -- ONE OF THE THINGS WE TRIED TO GET FORWARD AND GOT INCORPORATED WAS THAT THE BOARD OF DIRECTORS DECIDES WHAT IT IS THE CENTER WILL DO. THEY DECIDE WHAT THE PRIORITIES ARE. 87 YOU CAN'T COMPARE ONE AGAINST ANOTHER BECAUSE THE BOARD OF DIRECTORS RESPONDED TO THE COMMUNITY. IT'S GOING DOWN AS ONE OF THE FIVE THINGS THAT MAY NOT BE IMPORTANT TO YOU AND PEOPLE WHO COUNT NUMBERS. I'M SORRY. >> Audience Member: MIKE WALLACE. I GUESS AS NEW DIRECTOR AS OF JANUARY, ONE OF THE THINGS THAT WOULD REALLY BE NICE, THOUGH, IS SOME SENSE OF HOW MUCH I OUGHT TO EXPECT FROM MY STAFF. I DON'T KNOW -- I UNDERSTAND THE DIFFICULTY WITH COMPARISON. BUT IF YOU'RE IN THE V.R. SYSTEM, YOU KNOW HOW MANY CASES A COUNSELOR CAN HANDLE. YOU KNOW SOME SORT OF BENCHMARKING, ET CETERA. I KNOW IT'S DIFFICULT BECAUSE THERE ARE SO MANY CHOICES THAT YOU SAY THE BOARD OF DIRECTORS CAN MAKE. CERTAINLY FOR A FIRST-YEAR CENTER LIKE OURS THERE WAS A LOT OF PUBLIC CONTACTING AS OPPOSED TO NECESSARILY WORKING WITH INDIVIDUAL CONSUMERS. BUT AT THE SAME TIME I GUESS THERE'S ALSO THE SENSE WHERE I FEEL LIKE THAT WE HAVE SOME RESPONSIBILITY TO SAY TO THE TAXPAYERS, HERE WHAT'S WE DID AND HERE'S THE NUMBER OF CHANGES WE MADE IN PEOPLE'S LIVES, ET SET. I LOVE THESE OUTCOME MEASURES, THEY ARE GOING HELP US PLAIN WHY WE EXIST. >> Audience Member: WHERE HAVE YOU FROM? >> Audience Member: O I'M SORRY, BALTIMORE, MARYLAND. >>> IF I SAY BLESS YOUR HEART, YOU'LL UNDERSTAND WHAT I MEAN BY THAT. 88 I THINK THAT YOU HAVE SAID A VERY IMPORTANT THING THAT, THERE ARE SEVERAL OF US IN THE ROOM WHO HAVE BEEN CENTER DIRECTORS SINCE MAYBE BEFORE YOU WERE BORN. >> Audience Member: I'M A REALLY YOUNG GUY. >> Audience Member: AND I WOULD REALLY LIKE FOR A GROUP OF CENTER DIRECTORS WHO ARE HERE THAT HAVE BEEN AROUND DOING THIS FOREVER TO HAVE A CONVERSATION WITH YOU, BECAUSE YOU'VE HIT ON A VERY, VERY IMPORTANT THING THAT WE STRUGGLE WITH, WITH RSA, WITH YOU ARE OUR DSU, WE DON'T WALK AND TALK LIKE THEM. SO BECAUSE WE ARE DIFFERENT, WE CAN'T CONTROL HOW MANY PEOPLE MEET US OR WANT US. IT'S ONLY WHAT WE CAN DO WITH IT. I WOULD LIKE TO CHANNEL INTO SOME OF US, AND I KNOW I WOULD BE MORE THAN WILLING TO, AND I CAN IMAGINE SEVERAL OF THE OTHERS OF US TO MEET WITH YOU BEFORE YOU LEAVE, TO HAVE THAT KIND OF CONVERSATION. BECAUSE WE'RE NOT ABOUT NUMBERS. WE'RE ABOUT WHAT PEOPLE NEED AND MEETING THEIR NEEDS, NOT HEAVEN WE THINK THEY NEED CONCLUSION LEADS ME TO A THOUGHT ABOUT, LET'S NOT LOSE SIGHT OF THE FACT THAT WE'RE ALREADY BEING EVALUATED ON THINGS THAT MAKE NO SENSE. WE'RE ALREADY DOING A 704 RECORD. SO BETTER THAT WE ARE IN CHARGE OF OUR OWN CRAZINESS THAN HAVING OTHER PEOPLE TELLING US WHAT WE'RE SUPPOSED TO BE ALLOWED. >> Bob Michaels: JULIA, THAT'S WHAT THIS WHOLE THING IS. >> Audience Member: EXACTLY. I WOULD RATHER WITH DISCUSS AMONG OURSELVES THE VALIDITY OF 89 WHAT WE HAVE HERE, AND REPORT ON THIS KIND OF THING, GOOD, BAD OR INDIFFERENT, THAN REPORT HOME PEOPLE WE CAN A RESTORATION ON, WHICH IS WHAT THE 704 ASKS, WHICH NEVER HAS MADE ANY SENSE. >> Bob Michaels: THAT'S REALLY WHAT THE WHOLE THING IS ALL ABOUT. I APPRECIATE THERE A. YOU GUYS DECIDE WHAT IT IS THAT YOU WANT TO DO. AS THE YEARS GO BIKERS WE'LL BE GONE. WE HOPE WE'VE GIVEN YOU SOMETHING TO START WITH, AND THIS IS A GOOD -- THIS IS A GOOD PROJECT IF YOU WANT TO BUY RIGHT INTO IT. IT'S BEEN DONE FOR YOU. IF THERE ARE NEW THINGS YOU WANT TO DO, THAT'S GREAT. BUT YOU DECIDE HOW MUCH YOU WANT TO SHARE WITH EACH OTHER NOW. WE DON'T WANT TO -- WE DON'T WANT TO CREATE A SYSTEM THAT WAS GOING TO ABUSE YOU. AND THAT WAS -- SO EVERY TIME WE TALKED ABOUT ANYTHING WE WANTED TO DO, WE ALWAYS TALKED IN TERMS OF WE DON'T WANT TO EMBARRASS CENTERS. WE DON'T WANT CENTERS TO BE COMPARED AGAINST EACH OTHER BY SOME OUTSIDE SOURCE THAT DOESN'T UNDERSTAND INDEPENDENT LIVING. PAT? >> Audience Member: WELL, I JUST WANT TO APPLAUD THE WORK AND THE GUINEA PIGS, THE 28 CENTERS THAT SAID THEY WOULD PARTICIPATE. THIS WAS A TIME AND LABOR-INTENSIVE THING. ANYBODY IN THE ROOM, MY HAT'S OFF TO YOU, IF I HAD A HAT. THEN I ALSO HAVE A TINY LITTLE TECHNICAL QUESTION. I READ THROUGH THE MANUAL LAST NIGHT AND WE WERE TALKING AT OUR TABLE ABOUT SOME OF THE FOLKS 90 THAT WE ENCOUNTER ARE REAL -- WELL, THEY HAVE SOME COGNITIVE ISSUES OR THEY ARE OLD AND CAN'T REMEMBER THINGS, LIKE I AM. SO ONE OF THE THINGS I WAS WONDERING ABOUT, WHEN YOU DID THE CALL-BACKS OF THE INTERVIEWS AND PEOPLE COULD NOT REMEMBER HAVING HAD AN ENCOUNTER WITH YOUR CENTER OF INDEPENDENT LIVING, WERE YOU ALLOWED TO PROMPT THAT PERSON BY SAYING, YOU REMEMBER WHEN YOU CALLED AND DEBBIE ANSWERED THE PHONE AND YOU AND DEBBIE TALKED ABOUT -- COULD YOU DO THOSE SORTS OF PROMPTS TO REMIND? >> Bob Michaels: THERE WERE SOME PROMPTS WE WERE ALLOWED, AND WE TRIED TO BE VERY CAREFUL WHEN WE DID THAT. I THINK THE QUESTIONNAIRES HAVE MOST OF THIS, AS I RECALL. IF YOU LOOK AT A QUESTIONNAIRE YOU'LL JUST SEE SOME OF THE PROMPTS THAT WE WERE ALLOWED. YOU'LL SEE SOME OF THE STUFF, -- IT WAS THAT KIND OF THING. SOMETIMES WE SET IT UP, BUT WE DIDN'T WANT TO GIVE THEM THE ANSWER. THAT WAS ALWAYS THE TRICK FOR US. >> Mike Hendricks: RIGHT. THAT WAS THE THING, WE WANTED TO PROMPT AS LITTLE AS POSSIBLE, BUT ONLY AS MUCH AS NECESSARY. WE CERTAINLY DIDN'T WANT TO LEAD THEM IN ANY WAY. AND THAT'S A TRICKY THING, THAT'S A TRICKY THING. LET ME MENTION ONE THING, IF I CAN, ABOUT THE RESPONSE OF I DON'T REMEMBER. THAT'S SOMETHING WE LEARNED. THAT IS CLEAR LEARNING. FIRST YEAR-ROUND WE DIDN'T HAVE I DON'T REMEMBER AS AN OPTION. WE HAD YES OR NO. AND IT'S JUST DUH, ONE OF THOSE 91 REASONS YOU DO THE FIELD TEST. WE LEARNED AS BOB GAVE YOU THE DATA, SOME OF THE PEOPLE SAID I DON'T REMEMBER. THE NEXT YEAR-ROUND WE LEARNED AND WE HAD A SPECIFIC POSSIBLE ANSWER AS I DON'T REMEMBER. THAT'S ONE OF THE REASONS YOU DO THE FIELD TEST. >> Audience Member: ANOTHER QUESTION KIND OF ALONG THOSE LINES. DID YOU GATHER INFORMATION FROM THE CILs THAT PARTICIPATED? AND WHO ASKED THE QUESTIONS? WAS IT STAFF THAT HAD WORKED WITH THEM OR VOLUNTEERS OR A HODGEPODGE OF IT? >> Bob Michaels: IT WAS BOTH. SOMETIMES STAFF, SOMETIMES IT WAS VOLUNTEER, SOMETIMES BOARD MEMBERS. JUST WHOEVER THEY SENT. WE GAVE THE CENTER A VERY SPECIFIC SET, WE HAD A PRIMARY PERSON FROM EACH CENTER THAT WAS RESPONSIBLE. THEN THEY WOULD PICK SOMEBODY, TO THE FIRST YEAR. BUT THEN WE ALLOWED UP TO FOUR THE SECOND YEAR. WE REQUIRED THAT EVERYBODY HAS TO ATTEND THE TWO SESSIONS, SO WE PUT SOME STIPULATIONS IN THERE THAT REQUIRED THAT THEY BE TRAINED. BUT IT WAS ALL KINDS OF PEOPLE, SOMETIMES VOLUNTEER, SOMETIMES NOT, SOMETIMES A STAFF MEMBER. IT VARIED. FRED? >>> FRED JOHNSON, RALEIGH, NORTH CAROLINA. JUST FROM A STAFF PERSPECTIVE, ONE OF THINGS WE TALK ABOUT REGULARLY THROUGHOUT THE YEAR, IS WHY PEOPLE ARE COMING THROUGH OUR DOORS TO US. WE LOOK AT THE NUMBERS, WORK WITH PEOPLE DAY IN AND DAY OUT 92 AND SEE THAT NUMBERS OF WOMEN, MORE CONSUMERS WHO ARE WOMEN. WE LOOK AT THESE AND WE WONDER WHY THAT IS. WE DON'T HAVE A WAY TO EVALUATE THAT. WHAT YOU'RE GIVING US IS A WAY IN WHICH TO POSSIBLY FIND SOME ANSWERS AS TO WHY THAT IS. IT TIES INTO MAYBE A CERTAIN SENSE OF EVENTUALLY MARKETING PROGRAMS. THE STAFF MEMBERS ALWAYS HAVE THESE ININQUISITIVE QUESTIONS BECAUSE WE'RE WORKING WITH PEOPLE DIRECTLY, SO MANY VARIOUS NEEDS. WE'RE WONDERING WHY THEY FOUND THIS, HOW THEY FOUND US, WHY THE NUMBERS ARE THE WAY THEY ARE AND SO FORTH. THIS WILL HELP US A LOT AS A STAFF TO UNDERSTAND THE CENTER AND OUR RELATIONSHIP WITH THE COMMUNITY. >> WE HAD THAT. WHEN WE TOOK A LOOK, THERE WASN'T ANYTHING DIFFERENT BETWEEN AND RESPONSES. THOSE KINDS OF THINGS, ONE DISABILITY OVER ANOTHER, THINGS LIKE THAT, WE THEREFORE DIDN'T LOOK AT -- DIDN'T LOOK FURTHER AT THE STATISTICS BUT WE CLEARLY HAD MORE WOMEN THAN MEN. WE DID THAT. AT SOME POINT WE'RE GOING TO TRY TO COMPARE THAT. WHERE WE GET THAT INFORMATION FROM, I DON'T KNOW, BECAUSE IT'S NEVER GATHERED, SAME WAY WITH OUR CENSUS FIGURES OR THE 704 OR WHEREVER, IT JUST DOESN'T WORK FOR US. SOMEWHERE DOWN THE ROAD WE'LL IDENTIFY. IT'S NOT A BIG PROBLEM BUT IT'S THERE. >> Audience Member: IT'S A QUESTION. 93 >> Bob Michaels: WELL, IT'S ALMOST TIME. ONE LAST QUESTION? ANYBODY ELSE? IS LYNNAE HERE? THERE YOU ARE! >> Mike Hendricks: I'VE LEARNED THAT OUR GUEST SPEAKER IS GOING TO BE SPEAKING AT 12:50, WHICH GIVES US 20, 25 MINUTES TO GET LUNCH, HAVE IT NOT BE QUITE SO NOISY, THAT KIND OF THING. SO B WHENEVER YOU WANT TO BREAK. OUR GUEST SPEAKER FOR THOSE ONLINE, WILL BE SPEAKING AT 12:50. LET ME JUST DOUBLE-CHECK THAT. THE LUNCH SPEECH WILL BE ONLINE, CORRECT? >> Audience Member: CORRECT. >> Mike Hendricks: FOR THOSE ONLINE, 12:50 PACIFIC TIME. >> Bob Michaels: WE'LL GO AHEAD AND BREAK AND SEE YOU FOR DINNER RIGHT AWAY.