ROUGH DRAFT 9-14-11, Outcome Measures for Centers for Independent Living – An IL NET Resource Presented by ILRU MIKE MICRO, WELL, MY FRIENDS FRIENDS, MY CLOCK SAYS IT'S 10:45, AND OUR FRIENDS ONLINE ARE SAYING IT'S 10 WILL BE 45, AND IT'S TIME TO BEGIN THE SESSION ON STORING OUTCOME INFORMATION. SO LET'S DO THAT. BEFORE WE START, WE'VE HAD SOME REALLY INTERESTING STUFF COME 43 ONLINE, SOME INTERESTING QUESTIONS, WHICH WE'VE TRIED TO FIT IN. WE'VE HAD SOME INTERESTING SUGGESTIONS OF THINGS TO DO DIFFERENT THAT WE'RE TRYING TO DO. AND WE'VE ACTUALLY HAD A PERSON WE FAMILIAR WITH PORTLAND WHO HAS SENT IN, BELIEVE IT OR NOT, THEY SENT IN ONLINE THEIR SUGGESTION OF WHERE WE ALL REALLY NEED TO GO TONIGHT. I THINK THIS IS GREAT. THANK YOU SO MUCH OUT THERE, IN ONLINE WORLD. CAROL EUBANKS, I DON'T KNOW HOW MUCH YOU ALL KNOW CAROL. SHE'S ONE OF THOSE PEOPLE, WITHOUT CAROL, YOU KNOW, THIS SESSION NEVER, EVER WOULD HAVE HAPPENED. LET ME TELL YOU, SHE HAS BEEN INCREDIBLY IMPORTANT. IT'S BEEN BEHIND THE SCENES, SO WE'LL GET HER OUT IN THE SCENES FOR A MOMENT. SHE'S BEEN FANTASTIC AND ALSO IN HANDLING THE ONLINE STUFF. SO SHE HAS THIS GOOD INFORMATION FOR US. >> YOU SAID DINNER. I DON'T THINK IT'S DINNER. >> NO, JUST TONIGHT. THIS PERSON SAID THAT WE MUST GO TO VOODOO DOUGHNUTS. WE WENT AND FOUND OUT THAT VOODOO, IT'S NOT VERY CLOSE BY BUT IT IS IN FARELESS SQUARE AND YOU CAN TAKE THE TRAIN, IT'S A NICE 11-BLOCK WALK. BUT HE SAID -- LET ME LOOK AGAIN TO SEE EXACTLY WHAT THIS PERSON SAID -- HE SAID THE BACON STRIPS ON A MAPLE BAR ARE AWESOME. LET'S JUST KEEP THOSE COMMENTS ROLLING IN FROM THE VIRTUAL AUDIENCE. THANKS. >> Mike Hendricks: THIS ACTUALLY IS A VERY, VERY FAMOUS LOCAL 44 PLACE, YOU REALLY OUGHT TO TRY TO BE THERE. IT'S BEST AT 2:00 A.M., THE CROWD IS MOST INTERESTING THEN. IF YOU CAN'T MAKE THAT, GET THERE WHEN YOU CAN. IT'S AT 3rd AND ANKENY. AND CAROL HAS MAP TO HELP YOU FIGURE IT OUT. THAT'S A DARN GOOD SUGGESTION. >> ARE YOU TALKING ABOUT THE DOUGHNUT PLACE? >> YES, VOODOO DOUGHNUTS. A COUPLE OF THOSE AND COUPLE OF COFFEES AND YOU WILL NOT SLEEP A BIT TONIGHT. >>> LET'S JUMP AHEAD -- LET'S NOT JUMP AHEAD, LET'S KEEP MOVING AHEAD. BECAUSE FRANKLY NOW I'M EXCITED ABOUT HAVING BOB SHOW US THE RESULTS OF WHAT WE LEARNED THIS YEAR. BEFORE THAT WE HAVE TO DO ONE STEP ON THE YELLOW BRICK ROAD AND THAT'S THE STEP, SCORING OUTCOME INFORMATION. IT MAY SEEM SIMPLE, NOTHING TO THINK ABOUT. BUT THERE'S ACTUALLY MORE THERE THAN MEETS THE EYE. LET'S COVER A FEW THINGS ABOUT IT. FIRST OF ALL, WHY DOES IT MATTER: LET'S JUST ASSUME IN YOUR CIL YOU'VE DECIDED THAT YOU'RE GOING TO CALL 25 CONSUMERS. AND THAT YOU'RE GOING TO CALL 25 I&R CALLERS. AND THAT YOU'RE GOING TO LOOK IN YOUR CIL FILES AND INTERVIEW YOUR COLLEAGUES AND GATHER SOME INFORMATION THAT WAY. I'M NOT SAYING YOU SHOULD OR YOU'RE GOING TO, BUT LET'S JUST ASSUME YOU ARE. THEN WHY DOES IT MARRY HOW YOU STORE THE INFORMATION? ACTUALLY IT MATTER AS LOT. 45 FIRST OF ALL, YOU DON'T WANT TO LOSE IT. AND STUFF DOES GET LOST. YOU DON'T WANT TO LOSE ALL OF IT OR EVEN PARTS OF IT. YOU ALSO NEED TO KEEP IT CONFIDENTIAL. ONE OF THE THINGS WE PROMISED PEOPLE WHEN WE MADE THESE PHONE CALLS THAT IS THEIR INFORMATION WILL BE KEPT CONFIDENTIAL. WHAT THAT MEANS IS AS FEW PEOPLE AS POSSIBLE GET TO SEE IT, AND THOSE WHO DO GET TO SEE IT CERTAINLY DON'T SAY ANYTHING ABOUT IT TO ANYBODY. YOU NEED TO BE ABLE TO ANALYZE IT NOW. YOU NEED TO HAVE IT IN SOME KIND OF A FORMAT THAT YOU CAN ACTUALLY WORK WITH AND GET INFORMATION OUT OF IT. AND ALSO, HERE'S ONE YOU MAY NOT HAVE THOUGHT OF: YOU MAY FIND IN SIX MONTHS YOU'VE COME ACROSS ANOTHER PIECE OF INFORMATION, MAYBE ABOUT THESE PEOPLE, MAYBE IT'S SOME OTHER -- I DON'T KNOW WHAT IT MIGHT BE. AND YOU WANT TO MERGE THAT IN WITH YOUR DATA AND YOU WANT TO ANALYZE THEM TOGETHER. FOR INSTANCE, I DON'T KNOW, MAYBE THE NUMBER OF PEOPLE WHO HAVE RECEIVED SOME OTHER KINDS OF SERVICE OR SOMETHING. I REALLY CAN'T IMAGINE. YOU CAN, I'M SURE. BUT YOU MAY WANT TO MERGE IT. IF YOU DON'T HAVE IT, THEN YOU CAN'T MERGE IT. HERE'S THE OPTIONS YOU'VE GOT. DEPENDS ON HOW YOU GATHER THE INFORMATION. WE YOUR PAPER FORMS. BOB SHOWED THESE TWO. WE HAD PEOPLE ON THE TELEPHONE FILLING OUT THIS PAPER FORM WHILE THEY WERE TALKING TO A PERSON ON THE PHONE. 46 YOU MIGHT WHY, WHY DIDN'T YOU JUST HAVE THEM ENTER IT INTO THE COMPUTER LIKE WITH A HEADSET? WHY DID YOU HAVE TO HAVE THE PIECE OF PAPER IN BETWEEN? AND THAT'S BECAUSE A SURVEY RESEARCH EXPERT ADVISED US TO DO THAT. THEY SAID WHAT HAPPENS IS, IF YOU GOT A PERSON FOCUSING ON TYPING IT IN, THEY ARE NOT FOCUSING ENOUGH ON WHAT THE OTHER PERSON IS SAYING AND NOT HEARING THE STORY WELL ENOUGH. YOU KNOW, YOU COULD DISAGREE BUT THIS WAS THEIR ADVICE TO THE TASK FORCE THAT TOOK IT SERIOUSLY. THEY SAID BETTER TO SEPARATE THE TASKS. ONE, FOCUS ON WHAT THE PERSON'S SAYING, MAKE SURE YOU GET IT, AND THEN HAVE YOU ENTERING THE DATA. FOR BETTER OR WORSE OR BOTH, WE USED PAPER FORM. IF YOU USE PAPER FORMS, FIRST OF ALL, LOCK THEM IN A SAFE PLACE. I MEAN LOCK THEM IN A SAFE PLACE, FOR BOTH PROTECTION AND CONFIDENTIALITY. THEN ENTER THE INFORMATION INTO A COMPUTER, EITHER AN OPTICAL SCANNER OR MORE LIKELY, SINCE WE HAVE OPEN-ENDED QUESTIONS WHERE WE WROTE WORDS, MORE LIKELY THEN SOME KIND OF MANUAL ENTRY LIKE WE DID. MAYBE CONVERT THE OPEN-ENDED ANSWERS. SOME OF THESE QUESTIONS, HERE'S A NO/YES. PRETTY EASY TO ENTER, RIGHT? HERE'S ANOTHER ONE, IF YOU DO SYSTEMS ADVOCACY WORK, PLEASE PROVIDE EXAMPLES OF SOME POSITIVE CHANGES ACHIEVED OR NEGATIVE CHANGES PRESENTED IN THE ABOVE IDENTIFIED AREAS AS A RESULT OF YOUR EFFORTS. 47 THAT'S WORDS, THAT'S NOT A YES OR NO, THAT'S WORDS. SOMEBODY HAS TO GO EVENTUALLY LATER ON AND READ THOSE WORDS AND MAKE SENSE OF THEM AND CODE THEM AND PUT THEM IN THE COMMUTER. BOB AND I ACTUALLY DID THAT THIS YEAR. WE WERE THE HUMAN COMPUTERS FOR THOSE, IF YOU WILL. OR MAYBE ENTER THE WORDS DIRECTLY. YOU CAN JUST ENTER THE WORDS AND LET THE COMPUTER DO THE ANALYSIS. WE DECIDED NOT TO GO THAT WAY. WE DECIDED TO ANALYZE THEM OURSELVES. OR YOU COULD, IF YOU WANT TO, YOU COULD SKIP THE PAPER FORMS COMPLETELY, IT'S YOUR OPTION. YOU COULD JUST HAVE THE PERSON SITING THERE AT COMPUTER, AND TYPING THE ANSWERS AS THEY ARE GIVEN. YOU COULD, IF YOU DECIDED THAT YOU WANTED TO TRY IT, JUST PUT THIS WHOLE THING INTO THE COMPUTER AND HAVE THE PERSON RESPOND BY GIVING AN ANSWER TO IT. THERE ARE SOME CHALLENGES THERE, BUT YOU JUST FILL IT OUT AND SEND IT BACK. THERE ARE WEB SURVEYS LIKE SURVEY MONKEY, AND FRANKLY, YOU CAN PROBABLY THINK OF OTHER WAYS WE CAN'T TO STORE THIS INFORMATION. LET'S HEAR WHAT WE DID IN THE FIELD TEST. >> Bob Michaels: OKAY. SO NOW WE'VE GATHERED THE INFORMATION WITH COMPLETING THE THREE QUESTIONNAIRES, AND WE'VE GOT TO ENTER THE INFORMATION INTO SOME SORT OF A DATABASE. WE USED SURVEY MONKEY. WE JUDGED IT, WE HEARD THEY WERE 48 WHISPERED TO BE GOOD. WE TOOK A LOOK AT IT. THEY HAVE A FREEBIE, IF YOU WANT, YOU CAN GET IN AND USE IT. BUT THE NUMBER OF PEOPLE THAT RECORDED AND MONITORED INFORMATION FROM THEM, THE ASSISTANCE YOU GET FROM THEM, ALL THE OTHER THINGS, IT SEEMED WORTH IT FOR US. WE PAID THEM AN EXTRA $200 A YEAR TO GO AHEAD AND RECORD THIS INFORMATION. $200 WASN'T -- ISN'T A LOT OF MONEY, BUT FOR SOME CENTERS IT WOULD BE A LOT OF MONEY. SO YOU HAVE A DECISION TO MAKE ABOUT THAT. ANYHOW, THAT'S WHAT WE D WE USED SURVEY MONKEY. SO LET ME TALK A LITTLE BIT ABOUT IT. WE RECORD THE INFORMATION ON PAPER FORMS. . STAFF ENTERS THE INFORMATION FROM EACH PAPER FORM IN NUMBERS AND WORDS INTO A COMPUTER DATABASE, WHICH WAS SURVEY MONKEY. SO WHAT IS SURVEY MONKEY? WELL, IT'S A PRIVATE AMERICAN COMPANY THAT I THOUGHT WAS IMPORTANT, YOU'RE NOT SENDING INFORMATION ABOUT YOUR CONSUMERS TO BANGLADESH, YOU KNOW, AND IT ENABLES USERS TO CREATE THEIR OWN WEB SURVEY. YOU CREATE IT YOURSELF. BASICALLY WHAT I DID WAS ENTERS THE QUESTIONNAIRES AND ENTERED THEM PRETTY MUCH INTO SURVEY MONKEY. YOU CAN USE FREE OR ENHANCED PAID PRODUCTS WITH THE SERVICES. WHY USE SURVEY MONKEY? WELL, IT WAS RELATIVELY INEXPENSIVE. IT'S EASY, EXCELLENT CUSTOMER SERVICE. 49 THEY GO -- THEY GET RIGHT BACK TO YOU FOR ANY E-MAIL YOU SEND THEM. IT'S EASY TO DESIGN A SURVEY. THEY GIVE YOU OPTIONS. AND THEY HAVE SIMPLE COLLECTION METHODS AND SYSTEMS. SO IT'S VERY EASY FOR YOU TO GET THAT INFORMATION OUT TO PEOPLE THAT NEED THE DATA, AND WAYS TO PUT THAT, IT'S REAL STRAIGHTFORWARD ANALYSIS OF THE DATA THEN, AS WELL. MAUREEN? >> Audience Member: DID YOU RUN INTO ANYBODY ADDRESSING ANY ACCESSIBILITY ISSUES USING SURVEY MONKEY? >> Bob Michaels: NO, NO PROBLEMS. WE HAD A COUPLE OF PEOPLE ASK US QUESTIONS AND SAID, YOU KNOW, IT'S ACCESSIBLE AND ALL THAT. BUT PRETTY MUCH IT WASN'T A PROBLEM. BY AND LARGE. RICHARD? >> Audience Member: [INAUDIBLE] THERE IT IS. ABOUT ACCESSIBILITY AND SURVEY MONKEY, IN THE PAST SURVEY MONKEY HAS NOT BEEN ESPECIALLY ACCESSIBLE. IN FACT, IT'S BEEN PRETTY INACCESSIBLE. OVER THE PAST COUPLE OF YEARS IT'S BECOME QUITE ACCESSIBLE. WE HAVE BEGUN TO USE IT AT ILRU FOR SOME PROJECTS. I RECENTLY PERSONALLY TOOK AN ONLINE SURVEY FROM THE UNIVERSITY OF PENNSYLVANIA SCHOOL OF VETERINARY SERVICES, ABOUT DOG GUIDE USE. SO THE ENTIRE AUDIENCE OF THAT SURVEY WAS PEOPLE WITH VISUAL IMPAIRMENTS. AND IT WAS -- IT'S LEVEL OF ACCESS WAS EXEMPLARY. THERE ARE SOME TRICKS AND TRAPS 50 THAT YOU NEED TO BE AWARE OF AS YOU DESIGN, BUT FOR SIMPLE SURVEYS OUT OF THE BOX IS FINE. FOR MORE COMPLICATED SURVEYS THERE ARE SOME THINGS YOU NEED TO DO THAT JUST HAVE TO DO WITH GOOD WEB DESIGN. BUT SURVEY MONKEY HAS BECOME A GOOD OPTION FOR SURVEYING. >> Bob Michaels: THANKS, RICHARD. ONE OF THE REASONS I'VE ALWAYS ENJOYED WORKING WITH ILRU IS THEY HAVE REAL HIGH STANDARDS IN TERMS OF ACCESSIBILITY. IF IT'S ACCESSIBLE TO EVERYBODY, IT DOESN'T HAPPEN IF IT'S NOT ACCESSIBLE TO EVERYBODY. SO I TRUST WHEN SOMETHING COMES FROM THEM THAT IT MEETS ALL THE ACCESSIBILITY REQUIREMENTS. SO WHAT ARE THE FEATURES OF SURVEY MONKEY? WELL, THEY HAVE A WIDE VARIETY OF QUESTIONS, MULTIPLE CHOICE, COMMENTS, DEMOGRAPHICS, AND SO FORTH. THEY HAVE REQUIREMENTS, YOU CAN ASK QUESTIONS OR YOU HAVE TO ANSWER CERTAIN QUESTIONS BUT NOT OTHERS. YOU CAN MARK IT, IT'S VERY TOES DO THAT. THEY HAVE CORRECTION FEATURES. IF THERE IS A PROBLEM AND SOMEBODY ENTERS SOME BAD INFORMATION INTO IT, I COULD MAKE THE CHANGES. THEY JUST HAVE TO GIVE ME RING. THERE'S FILTERING, CROSSTABBING, AND DOWNLOADING RESPONSES. LOTS OF DIFFERENT WAYS TO LOOK AT INFORMATION. THEY ALLOW YOU TO DO -- LOOK AT IT IN TERMS OF IN A CERTAIN WAY, ALL THAT. THEN THEY HAVE SKIP LOGIC. FOR THOSE WHO DON'T KNOW WHAT SKIP LOGIC IS, IT'S THE SYSTEM IN THERE THAT WOULD ALLOW TO 51 YOU, LIKE, FOR INSTANCE, IF YOU SAID YES OR YOU SAID NO TO A QUESTION, YOU MIGHT SKIP FROM NO. 6 TO NO. 8. BUT IF YOU SAID YES, YOU GO TO NO. 7. SO THAT IS A FEATURE THAT NOT EVERYBODY HAS BUT THEY DO. SO HERE'S AN EXAMPLE OF IT FROM THE INTERVIEW FORM. IT WOULD SAY, SOMETIMES WE'RE ABLE TO HELP PEOPLE LEARN NEW SKILLS, NEW KNOWLEDGE, OR NEW RESOURCES, AND SOMETIMES WE'RE NOT. FOR YOU PERSONALLY, DO YOU THINK THAT YOU'VE LEARNED ANY NEW SKILLS, KNOWLEDGE AND RESOURCES THAT YOU DIDN'T HAVE WHEN YOU FIRST STARTED WORKING WITH THE CENTER? AND THEN DON'T KNOW, NO, AND YES. SO IF YOU ANSWER YES YOU GO TO QUESTION NO. 6 BLOW. IF YOU SAID NO OR DON'T KNOW, YOU WOULD GO RIGHT TO NO. 7. THEN IN SURVEY MONKEY, WE JUST SEE IT'S VERY CLOSE. I TOOK A COPY RIGHT OFF SURVEY MONKEY. IT BASICALLY IS THE SAME THING. SOMETIMES WE'RE ABLE TO HELP PEOPLE LEARN NEW SKILLS, KNOWLEDGE, OR NEW RESOURCES, AND SOMETIMES WE'RE NOT. FOR YOU PERSONALLY, DID YOU LEARN, DID YOU WHEN YOU FIRST CAME TO THE CENTER, DON'T KNOW, NO, AND YES. IT'S ALMOST EXACTLY WHAT IT IS THAT YOU SAID. HERE'S ANOTHER EXAMPLE OF IT, HOW IT WOULD LOOK ON THERE. WHAT SPECIFIC SKILLS, KNOWLEDGE OR RESOURCES DID YOU LEARN ABOUT THAT YOU DIDN'T KNOW BEFORE? IT SAYS HERE, WRITE ON THE FIRST LINE BELOW THE FIRST THING THE CONSUMER MENTIONS. 52 IF THE CONSUMER MENTIONS A SECOND THING, WRITE IT ON THE SECOND LINE. THAT WAS IN THE QUESTIONNAIRE. AND AGAIN, IT'S EXACTLY THE SAME. WHEN THEY PUT IT IN SURVEY MONKEY, IT'S EXACTLY THE SAME THING THAT YOU ADD IN THE QUESTIONNAIRE. SO I GUESS THE MESSAGE I'M TRYING TO GET ACROSS TO YOU IS THAT SURVEY MONKEY WAS REALLY NICE TO WORK WITH. THEY WERE FAIRLY FLEXIBLE, YOU COULD PUT THE EXACT SAME THING, THEY ASK YOU A QUESTION RIGHT AT THE BEGINNING. WHAT KIND OF QUESTION DO YOU WANT TO ASK? DO YOU WANT A MULTIPLE CHOICE WITH ONE ANSWER, MULTIPLE CHOICE WITH SEVERAL ANSWERS? DO YOU WANT TO REQUIRE PEOPLE TO ANSWER THIS QUESTION? YES OR NO. AND THERE WERE ALL KINDS OF THINGS YOU CAN DO, AND THEY ENDED UP BEING A VERY, VERY FLEXIBLE SYSTEM. NOW THEY HAVE COME UP WITH SOMETHING NEW THAT THEY ARE ADDING THIS YEAR. SO YOU CAN GO BEYOND THE $200 IF YOU WANT. IT HAS IN THERE, YOU IDENTIFY, SAY, MAYBE A LIST OF 23 THINGS AND YOU IDENTIFY THE TOP FIVE. AND THEN NEXT QUESTION WOULD BE, OF THESE FIVE, HOW MANY OF THEM DO YOU RECEIVE. IT WOULD AUTOMATICALLY DO THAT FOR YOU. IT WOULD KNOCK OUT THE ONES THAT WEREN'T CHECKED AND JUST HAVE THE ONES IN THERE THAT YOU WERE TALKING ABOUT. SO IT'S A LOT OF FLEXIBILITY. WE WERE CONCERNED BECAUSE I KNOW A NUMBER OF YOU, EVERYBODY HERE 53 USES A DIFFERENT -- USES SOME SORT OF DATA MANAGEMENT SYSTEM. AND SO WE WENT TO THE SYSTEMS AND WE WROTE THEM, WE ASKED EACH ONE OF THEM A SERIES OF QUESTIONS. BECAUSE WE WANTED TO FIND OUT HOW THE Q-90 AND OTHERS ANSWERED THE QUESTIONS THAT YOU HAD, WHETHER OR NOT THEY WERE FLEXIBLE, WE WANTED TO USE THEM. WE ASKED ABOUT THEIR GATHERING OF DEMOGRAPHIC INFORMATION, THE ABILITY TO RANDOMLY SAMPLE, THE PROCESSES, SO OUTCOME INFORMATION, THEY ALREADY GATHER, AND WHETHER OR NOT THEY WILL BE ABLE TO DO IT IN THE FUTURE. THE OUTCOMES UNIQUE TO A CENTER FOR INDEPENDENT LIVING, ABILITY TO ANALYZE OUTCOME DATA, AND THE ABILITY TO GENERATE REPORTS. WE SENT THIS OUT TO EVERYBODY THAT WAS LISTED IN THE ILRU WEBSITE ON SERVICES. THERE WERE ABOUT 8 OR 9 DIFFERENT ASSISTANTS FOR GATHERING INFORMATION. CFAL, G-90 AND INDEPENDENTS ONLINE WERE THE ONLY ONES THAT ANSWERED US INITIALLY. WE THEN GOT SOMETHING FROM ANOTHER PROVIDER THAT WAS NOT ABLE TO ANSWER IMMEDIATELY. THEY ALL WERE ABLE AND WILLING TO DO ALL THOSE NINGS, TO GATHER NEW INFORMATION IF YOU HAVE A NEW DATABASE, IF YOU WANTED NEW OUTCOME INFORMATION THAT YOU WANTED INTO THE COMPUTER, THEY WILL ADAPT IT TO MEET YOUR NEED. THEY WILL GIVE YOU REPORTS, THEY WILL HELP YOU ANALYZE OUTCOME INFORMATION. THEY ALREADY HAVE GATHERED DEMOGRAPHIC INFORMATION. BUT THEY ALL WILL DO A RANDOM SAMPLING FOR YOU, WHICH IS SOMETHING I DIDN'T EXPECT THEM 54 TO BE ABLE TO DO. BUT EVERYBODY, INCLUDING SURVEY MONKEY, SAID YES. ANY QUESTIONS ABOUT THAT? INDEPENDENTS ONLINE IS A SYSTEM OUT OF FLORIDA, AND THEY PREFER -- A LOT OF CENTERS DOWN THERE USE IT. YEAH. >> Audience Member: THIS ISN'T A QUESTION SO MUCH AS CLARIFICATION. WHEN YOU SAY Q-90 YOU MEAN CELL MANAGEMENT, CORRECT? >> Bob Michaels: YEAH, RIGHT. >> Audience Member: CORRECT. [CIL MANAGEMENT] >> Audience Member: MICHAEL IS FROM CIL MANAGEMENT SUITES AND I WANTED TO POINT OUT HE'S HERE AND WE REALLY ENJOYED TALKING WITH HIM AND JUST HIS INTERESTS IN I THINK BUILDING A SYSTEM THAT HELPS COLLECT DAY FOR BASE OUTCOME INFORMATION. >> Mike Hendricks: IF I COULD ADD MY COMMENT TO THIS, AND I'M NOT PUSHING Q-90 OR IO OR CFAL OR THE OTHER ONE, BECAUSE I DON'T EVEN KNOW WHAT THEY MEAN, I WOULD LIEU TOBACCO COMPANY SUGGEST THIS. WE CHOSE SURVEY MONKEY AS OPPOSED TO ONE OF THESE FOR OUR FIELD TEST BECAUSE WE KNEW WE WERE DOING A FIELD TEST, JUST THAT. WE DIDN'T KNOW WHAT WE WERE GOING TO RUN INTO, WE DIDN'T KNOW HOW MUCH WE WERE GOING TO HAVE TO CHANGE IT. BOB BECAME THE SURVEY MONKEY EXPERT, NOT ME. HE'S OUR MAN ON THIS, SO TALK TO HIM ABOUT STUFF, NOT ME. WE CHOSE THAT BECAUSE IT WAS FIELD TEST. IF I WERE IN YOUR SHOES, MAY I SAY, IF I WERE IN YOUR SHOES FROM A METHODOLOGY POINT OF 55 VIEW, MAYBE I MIGHT USE SURVEY MONKEY FOR THE FIELD TEST BECAUSE IT WAS VERY HELPFUL, WASN'T IT? >> Bob Michaels: YEAH. >> Mike Hendricks: IT WAS REALLY GOOD AND FLEXIBLE. BUT LONG-TERM I WOULD NOT BE THINKING OF SURVEY MONKEY. LONG-TERM I'D SAY YOU WANT TO GO THIS WAY, AND I THINK YOU AGREE. YOU WANT TO INTEGRATE YOUR OUTCOME INFORMATION ALONG WITH ALL THE OTHER INFORMATION YOU'VE GOT. YOU DON'T WANT TO HAVE OUTCOME STUFF STANDING OVER HERE IN THIS OTHER DATA PILE WHEN YOU'VE GOT YOUR OTHER DATA OVER HERE. YOU WANT TO GET THEM TOGETHER. DO YOU AGREE, BOB? >> Bob Michaels: RIGHT. >> Mike Hendricks: I DON'T KNOW WHICH WOULD BE THE BEST WAY TO GO, YOU'D HAVE TO THINK ABOUT IT. THAT'S WHY WE WERE SO PLEASED THERE WERE SO MANY YESES UP HERE. SOUNDS LIKE THESE FOLKS ARE WELL AWARE OF THE IMPORTANCE OF OUTCOMES. THEY KNOW ONE DAY THEIR SYSTEMS ARE GOING TO HAVE OUTCOME DATA IN THEM. >> Bob Michaels: COMMENTS? >> Audience Member: DAN KESSLER, ALABAMA. DO THESE COMPANIES SAY THEIR SYSTEMS CURRENTLY CAN DO THIS NOW? OR THAT IT'S CAPABLE OF MAKING THE CHANGES? >> Bob Michaels: NO. WE ASKED THEM IF THEY WERE DOING OUTCOME INFORMATION, ALL OF THEM SAID THEY WERE ALREADY. OF THE SAME KIND OF OUTCOME INFORMATION WE WOULD BE ASKING, FOR BUT THEY ALSO SAID WE ARE 56 WILLING TO ADAPT AND DO WHAT WE NEED TO TO MEET YOUR NEEDS. OTHER COMMENTS? NOW ON TO THE GOOD STUFF. >> Mike Hendricks: NO, NOT QUITE. IT'S THEIR TURN. YES. ALL RIGHT. YOU MAY NOT HAVE THE ANSWERS TO THESE RIGHT NOW BUT SPEND A FEW MINUTES AT YOUR TABLE MULLING OVER SOME OF THIS STUFF WITH YOUR COLLEAGUES. SEE WHAT YOU'RE THINKING ABOUT STORING OUTCOME INFORMATION BACK AT YOUR OWN CIL. >> Audience Member: MIKE, THIS IS RICHARD. >> Mike Hendricks: OKAY, RICHARD. WE WOULD ASK THE PEOPLE WHO ARE PARTICIPATING BY WEB TO DO THIS AND WE WILL BE ASKING FOR THEIR FEEDBACK. >> Mike Hendricks: , YES, THE PEOPLE ONLINE, PLEASE DO THIS SAME THING AND WE'LL ASK FOR YOUR FEEDBACK. GOOD JOB, RICHARD.