PAULA MCELWEE: You see how it begins to all come together, pull in that information from different sources and that information informs where you are going to go next and how you are going to measure what you accomplish and how you are going to please the people, the customers, right? So wow, is that something. I think that was fantastic. Anybody have any thoughts about this whole consumer satisfaction piece? What is yours, go ahead. Yes. Then I will ask what I was going to ask. AUDIENCE MEMBER: Well, one thing that we implemented that seems to be working out very well is the consumer advisory board that meets with the executive director once a month and in there they discuss what is going on with the center, what is going on with the instructions and some of the things they may want to add to instructions or take out so this is a good way to continually get information. PAULA MCELWEE: Continually get information. It's so important, isn't it? A lot of consumer satisfaction processes that I have seen, I hate to tell you, are dry, low return, not very useful pieces of information, and what Anne just described to you was very vital, very alive, if we can use that term, because it's moving, it is give and take, constant information that constantly affects how they deliver services and I think that's very exciting to see. Anybody else have a comment or a question around the consumer satisfaction piece? Yes. Over here. AUDIENCE MEMBER: I am just delighted with what I have been hearing. Especially, the whole thing, but with regard to the consumer satisfaction. I have a question on best practices, on being able to gather that data. We don't have a lot of people that actually come into our CIL, just by the virtue of the nature of the stuff that we do but I do like the idea of having comment cards and I am wondering if it would be appropriate for comment cards to be handed out after every engagement because then there is a lack of, you know, maybe it could go in a comment box or something like that because I do like that kind of data and I am just wondering if other people do things like that. PAULA MCELWEE: Yes. Let's hear from everybody, but I can tell you that I do see, you know, comment Drop boxes sometimes at centers, where, and cards just, in places where people can just pick them up, but actually passing them out every time you have any event or any gathering of people makes good sense, makes real good sense. Also for the intake, whoever does the intake, to hand somebody a card, a comment card at the end of the process and say, I hope you will fill this out and drop it in this box or maybe you have an address on it so they can drop it in the mail or whatever. Anybody have any experiences they would like to share about what you do related to that? What are some of your practices? Or other consumer satisfaction input practices? AUDIENCE MEMBER: I do have another question, is we have just been implementing, closed files, send-out letters and things like that but the question is do we send to everybody that is currently open or do we just send to closed and unfortunately this is a new practice just from a year and a half when I started with the organization and so we have been selecting a percentage but I am really feeling the need to survey everybody. PAULA MCELWEE: Well, you know, certainly getting a survey in everyone's hands gives you a little better feel for the total picture of what is going on. What we are finding, what we are finding out in the field, and I don't know if this varies a lot from community to community but we are finding that there are two barriers to getting information once a year, which is a lot of times the, you know, the once a year you look at how many closed cases or once a year you send out a satisfaction survey. One of them is people move a lot more than we thought that they did or more than people used to move, and so you send something in the mail, and you get back, you know, no forwarding address piece, that they still may even be dropping in but they didn't give you their address change and you may not see them again for another five or six months and so you can't get them because you have lost track. The other thing is TracPhones or disposable phones. If their number is not changing, sometimes that is the case, then they run out of minutes at the end of the month or they don't want the answer the call if it's somebody they don't have to hear from because they have limited plans because we work with a lot of people who have very tight financial situations, and we are finding it's not always easy to reach people by phone, either. Sometimes they don't want to talk to us. Yes, back here. AUDIENCE MEMBER: I think another issue with once-a-year surveys is you will oftentimes only capture something that recently happened with a individual, good or bad, so a more ongoing conversation with that person quarterly or whatever, but once a year is sometimes a skewed perception. PAULA MCELWEE: Good point. So ongoing and reaching as many people as you can gives you better feedback typically. Those would be some tips. Other comments or questions around this issue, consumer satisfaction? PAULA MCELWEE: Anybody want to share what they feel like they will be doing when they get home with the information you have learned so far today. Any aha moments, any things that you would like to share that are going to affect how you go forward in this? AUDIENCE MEMBER: I think I will incorporate all these good ideas and also involve more of the consumers in the center. Right now they are involved in volunteering and learning the process of the CIL, center there, but I think I am going to involve them more in being more creative with ideas on how we can better our center. PAULA MCELWEE: Those ideas that people have about things that they would like to do. AUDIENCE MEMBER: We have the same, another one of the same issues, consumers that will accomplish something and then want to give back. We don't always do well with that. We don't always give them nice opportunities to do that. I think we have a lot of things we can think through around that. We do have a few people who are peer associates and they help with tutoring, or help, they assist with classes and stuff. I think we can do better. I think we are going to make more of an effort to engage people in systems advocacy. I think we've slipped on that. So those are the things we are thinking about now. PAULA MCELWEE: Good. What other ideas do you have that you are going to take home with you? AUDIENCE MEMBER: Sorry, this kind of stuff geeks me. I like it. I really like the conversation about how to uncover the underserved populations. And start to take a look at how we can do a better job at that. And we are also in the middle of writing our strategic plan and so I just want to commend the folks that were just up and did a fantastic job. I am going to be embedding a few more ideas from this program into our strategic plan while it's still wet paint. PAULA MCELWEE: Good. And we've mentioned this several times but you are required to have a 3-year program and financial plan, so that's one of the requirements if you are getting federal money, is to have that 3-year plan. It can be longer and it can be as detailed as you want it to be, but that's part of what you report on in your 704 report, what did you do on the plan for the last year, what are your plans for the coming year, and so they are looking all the time on that but also it is something they are looking at when they go on site for a review. So you want your planning process to be in place. A lot of times, though, it has not been as vital as living as, as living, breathing, moving as what we have seen today and I, during this week. I think it's really been exciting to see how some of the folks are adapting that or looking at that. Other comments or questions? Back here. AUDIENCE MEMBER: I think just kind of building off of that, working on our 3-year plan that's in place and we are about halfway through that plan right now. It is not as detailed, it has the basics of what you need and what is required, but being able to take some of these things to be able to have more of those clearly defined measures and opportunities to do the best with that, instead of just focusing on just the numbers or the percentage or what we are trying to grow, but looking at more at, making sure we are on all on the same page as to how we can do this and here are thoughts and ideas of how we can make sure we get the feedback. I have been working on that the last few months and I feel like this gives me a lot of ideas on how to actually incorporate those things a little bit more. And I think with all of the different formats out of the consumer feedback we really have been, we are a rural area in Minnesota and have four offices now and are just starting to try to do more things in group settings when possible because people are looking for opportunities to learn some of those skills and if we, if it works to do a class and more of that peer support and learning from each other, it works really great. Rather than just an independent living specialist and so I think we have been doing some surveys and asking people and getting some feedback about groups and classes and I think this gives us a lot of ideas about how to get more feedback because one person says I want this kind of support group. And another person, I want it but evening, and you get two or three pieces of feedback that don't go together but the more information we get, the better we can serve. PAULA MCELWEE: You get the input, you can continue to adjust, right? Excellent. You know, one of the things that we haven't talked about too much and I want to mention as you close the loop on this is how do you get back to the consumers regarding what their input taught you. How do you get back to consumers related to whether or not you accomplished the goals that you set? What is that feedback loop like? How do we involve all of the people that are a part of our community in each step of this process? Not just in getting input but also in telling how your input was used. I don't know if somebody has a comment or will tell us how you do that or are doing that. I think that might be helpful. Maybe Anne can jump in on that a little bit, too. I don't know how you do that, but ANNE WEEKS: Well, one of the things we always do is we share raw data with the consumer, like we have a consumer forum and we will share that data but then we also, at the next year's consumer forum say, you know, this is what we did, this is what we've accomplished, this is not what we accomplished, and take a look at from their perspective, well, is it still important this year, or do we need to move on to something else? PAULA MCELWEE: Okay. Good. All right. Other comments or questions for the good of the group?