Page 1 Independent Living Research Utilization We create opportunities for independence for people with disabilities through research, education, and consultation. ilru logo in block red letters with blue eyebrow swoosh above and below Independent Living Research utilization. www.ilru.org. Page 2 Independent Living Research Utilization High-Performance Measurement Framework Webinar Series for SILCs Webinar 2: How to Select and Capture the Best Measures to Quantify your Impact and Value Presenters: Sheri Chaney Jones Jeremy Morris August 7, 2019 Page 3 Evaluation Survey & Presenter Contact Information Your feedback on this webinar is important to us. At the end of the presentation you will have the opportunity to complete a brief evaluation survey. Sheri Chaney Jones – scjones@measurementresourcesco.com Jeremy Morris - jmorris@ohiosilc.org Paula McElwee – paulamcelwee.ilru@gmail.com Page 4 What You Will Learn… How to create high-performance measurement cultures The importance of a SILC logic model for success How to attract funders with data How to create the perfect customized statewide measurement framework Impactful data-driven communication strategies using data Page 5 Five Strategies for Turning Data into Dollars Diagram with five boxes and arrows pointing between them: Culture, Clarify, Capture, Communicate, Change/Celebrate Page 6 2. Clarify your Vision Looking through a lens outside that brings color into focus Page 7 Clarifying Tool: Five Whys Exercise Why do you do what you do? Why, so that… Why, so that… Why, so that… Why, so that… Why, so that… Why do your funders/ partners want to partner with you? Why do your participants/ clients participate? What do they hope to accomplish? Page 8 Client Example Before We are a public agency that blah, blah, through blah, blah, blah aims to improve blah, blah, blah, for Ohio’s blah, blah, blah……. After “Promoting choice, independence, and quality of life for all Aging Ohioans.” Page 9 Clarify Tool: Logic Models Screenshot of a logic model that lists Inputs, Human Capital, Policies and Plans, Funding, Activities/Outputs (Core Services, Community-Based Services) Short-term outcomes Medium-term outcomes Long-term outcomes/impacts Page 10 Stop Counting Hamburgers, Start Measuring Your True Profit (i.e. Outcomes) 33,000,000 burgers daily 350,000 burgers daily Page 11 What is the Difference Between Short, Intermediate, and Long-Term Outcomes? Three arrows: Short-Term - Happens immediately after contact with client Intermediate Term - The social change that occurs when individuals go out into the world and apply short-term outcomes Long-Term - What happens when efforts impact many people who engage in social change over time Page 12 Learning from Leaders Silhouette of two people climbing a mountain and this quote: "All you need is the plan, the roadmap, and the courage to press on to your designation." Earl Nightingale Graphic of Ohio SILC - Ohio Statewide Independent Living Council. Jeremy Morris, Executive Director Ohio Statewide Independent Living Council Page 13 High-Performance Measurement Clarify Questions How did/does having a logic model help the OSILC in terms of its clarity? Page 14 3. Capture Your Results Dilbert cartoon - The MBA Guy I put together a spreadsheet that might interest you. Ow! Ow! It's so boring, it hurts my head! My brain is trying to escape through my ear. I get this a lot. Page 15 Defining the Best Success Measures Measuring tape with word SUCCESS Page 16 Reflect on both the stakeholders, as well as your own needs Your Whys Two intersection circles 1) Your Funder's Whys 2) Your Participant Whys Page 17 OSILC Global Measures Two columns 1) Desired Performance Measure and 2) Description Improve quality and effectiveness of services - Effectiveness of services reflects the extent the IL Network is improving the capacity of service providers and meeting the needs of the consumers. It includes the percent of consumers meeting their goals as well as how long it takes a consumer to achieve goals. This measure also examines perception of quality and cost per successful outcomes. Increase elements of a meaningful quality of life - Elements of a meaningful quality of life include constructs that allow an individual to fully live the life they desire with the needed supports that allows them to life this life. This includes achieving personal health and wellbeing, overcoming barriers to self-fulfillment, exercising control over decisions and making choices based on personal preferences. Increase community participation - Community participation is defined as a consumer successfully participating in meaningful adult life. Activities can include participation in recreation activities, volunteering, civic engagement and employment. Community participation is defined as a consumer successfully participating in meaningful adult life. Activities can include participation in recreation activities, volunteering, civic engagement and employment. - Increase diversion/transition from more restrictive settings - This measures the extent the IL Network are successful at keeping people out of and/or transitioning consumers from nursing homes. Increase awareness of independent living and independent living services/alignment of public perception This measure monitors the level of awareness in the general public about independent living and services. It also monitors how well the general perception of independent living and services align with the IL Network. Advance policies and practices for independent living and equity This long-term measuring is defined as the number of practices and policies at the organizational and public policy level that move forward because of IL Network efforts. Page 18 What to Capture Demographic data on participants Activities/ Services received Start and end date Costs/ Revenues Participant and stakeholder feedback (satisfaction data) OUTCOMES (your true profit) Page 19 Rules for Selecting Good Measures Provides an agreed upon basis for decision making Are understandable Applies broadly May be interpreted uniformly A way to measure it exists Has face validity Economical to apply Page 20 12 Outcome Measures 1. Increased knowledge and learning 2. Changed attitudes 3. Increased readiness 4. Reduction of undesirable behavior 5. Increased desirable behavior 6. Maintenance of new behavior 7. Increased social status 8. Increased economic conditions 9. Increased health conditions 10. Reduction in administrative costs 11. Increased economic development 12. Participant, employee, and/ or stakeholder satisfaction Page 21 Measures Test Are these measures related to important policy and practice issues?  Are these measures relevant and apply to practitioners? Can these measures be communicated in a way that will influence outcomes? Are incentives built into the system for collecting and acting on these data? Are the measures linked to the stakeholders interests? Page 22 Developing Objective Measures “Garbage in, garbage out” Stay neutral, avoid leading questions Ask one item at a time Test survey questions first Keep it short – only ask the most important questions Page 23 Success Measures: How to Capture Existing data sources/ Literature review Use the tools you have Use free/ Low cost tools Seek qualified vendors, but not until you have defined success Page 24 OSILC Measurement Framework Table with 3 columns 1. What was done by the IL Network? # of capacity building meetings convened  # of information, advocacy/policy activities  # of community engagement efforts  # of people served/engaged  # of contact hours/units of service provided  # of referrals made 2. How well was it done? % of target population reached  % of consumers/partners who report positive experience and satisfaction with services  % of partners who report increased capacity  % of consumers who completed goals on the Independent Living Plans  % increase in awareness and alignment of IL and IL services  Cost per success 3. How are people better off? % of eligible consumers successfully transitioned/diverted to appropriately less restrictive environments  % of consumers participating in meaningful adult life  # of policy changes at the state, local and organizational levels  % of consumers who report increase in independence  % of consumers who report improved quality of life   Page 25 Learning from Leaders Silhouette of two people climbing a mountain and this quote: "All you need is the plan, the roadmap, and the courage to press on to your desgination." Earl Nightingale Page 26 High-Performance Measurement Capture Questions Where is Ohio’s IL Network in the implementation of your measurement framework? What obstacles have you had to overcome? How has having this measurement framework helped you? Page 27 Webinar 2: Action Items Activity 4: Five Whys Activity 5: Logic Model Activity 6: Measurement Selection Page 28 Final Questions and Evaluation Survey Any final questions? Directly following the webinar, you will see a short evaluation survey to complete on your screen. We appreciate your feedback! https://usu.co1.qualtrics.com/jfe/form/SV_6rQkTJFOfPyaY2V Page 29 SILC-NET Attribution This project is supported by grant number 90ISTA0001 from the U.S. Administration for Community Living, Department of Health and Human Services, Washington, D.C. 20201. Grantees undertaking projects under government sponsorship are encouraged to express freely their findings and conclusions. Points of view or opinions do not, therefore, necessarily represent official Administration for Community Living policy.