Designing and Conducting Focus Group Interviews 
Presenter Dr. Richard Krueger 
Date: December 5, 2003

LAUREL:  Good afternoon.  This is Laurel Richards with ILRU
in Houston.  Welcome to today's Web cast and today we are
going to focus on focus groups with the subtitle asking
questions that yield powerful information. 

And we were pleased with today's topic, primarily because we
know that with folks running independent living centers.  What
we need to be doing in order to best meet the needs of our
consumers has been critical to people working at centers since
before 1972 when the first ones got started.  The responsibilities
that people running centers field feel about reaching out to new
populations to under served populations to folks living in rural
areas has created an increased need for doing effective
assessment of consumer needs. 

To top it all off, since 19 -- I guess the amendments to the
Rehab Act in 1990, was it the '92 amendments or the '98
amendments when the statewide I. O. councils were formed. 
They were given the express charge of doing statewide
consumer needs in order to develop the plan for independent
living which would be in effect for three years.  A lot of
creative methods were used from survey instruments to doing
dialing in 800 numbers to get the needs of consumers, but it is
never been quite according to the reports we have gotten back,
it is never been quite satisfactory and there has always been a
search for more effective means to obtain consumer needs. 
The whole issue of evaluation and how do we know whether or
not our services are making a difference and it is been
suggested that perhaps focus groups are the means to doing
both.  I think probably that question will come up in today's
presentation and if not as part of the presentation as certainly as
some of the questions that follow up.  So we were really
pleased to see today's topics. 

With today's presenters I was telling a colleague of ours at the
National Rehab Hospital center for disability research that we
were -- this was today's topic and we had this guy, Dick
Krueger is going to present.  And she said, oh, we just bought
his book.  I thought, my goodness, we have got a hot one today. 
Dick, it is a real pleasure that we have you presenting today and
we also are doing this presentation in collaboration with our
partners at the University of Iowa College of Law, the
technology for independence project, and we are working with
these folks on just a real important project related to looking at
research and especially applying research to the needs of people
working in the independent living field.  We are working
specifically today with Helen Schartz who will take over as
moderator and leading the discussion. Helen, I would like to
turn it over to you now, please. 

HELEN:   Okay, thank you, Laurel.  Thank you for that great
introduction to the topic and to independent living centers and
how important focus group research may be to them.  Hello
everyone my name is Helen Schartz and I am here at the Law
Health Policy and Disability Center at the University of Iowa
and this Web cast is the fourth in a series, on disability research
and assistive technology.  The Web casts are funded by a grant
from the National Institute on Disability and Rehabilitation
Research to promote technology for independence of persons
with disabilities, and we are very delighted this afternoon to
have with us Dr. Richard Krueger who will be using focus
group research.  He is truly an expert in the area as you may
have read any of his articles or a number of his books and he
has extensive experience both conducting focus group research
and writing articles and publications on this methodology.  His
most recent text is an invaluable resource for any of you who
are planning on doing focus group research.  He is a professor
at the University of Minnesota where he teaches courses in
program evaluation and research methods and he is also
president of the American Evaluation Association. Dr. Krueger,
welcome. 

RICHARD:   Thank you.  It is very good to be with you today. 
I feel a little out of place because I am sitting here at my office
at the University of Minnesota with a little headset on and a
microphone in front of me and I feel like I am talking only to
myself.  I know there are many people out there.  It is a very
pleasant day here where I am sitting.  There is a little bit of
snow on the ground and up here in this part of the country
where I am at; we prefer not to have too much snow on the
ground. 

Let me review the plan of what I am proposing today of how
we go about our topic.  I would like to take about 40 minutes
and give a brief overview of some key elements of focus groups
particularly talking about things like moderating, asking
questions, analysis, recruitment, and also getting volunteers
involved. 

Most of the things that I will be talking about are also
illustrated in a handout I think is on your Web site.  It is called
designing and conducting focus group interviews and as we go
along I will be referring to certain pages in there that are in the
upper right-hand corner of items and topics and so I am going
to be having that in front of me as we go through today and that
will take place for about 40 minutes. 

After 40 minutes, what I would like to do is we are going to
pause and start taking questions.  And my understanding is that
if anyone has questions, to go ahead and just submit those
questions and we will take them at about 40 minutes and get
through as many of them as we can. 

If it happens that we have more than we can handle, I will
respond to them in writing a little bit later and we can put them
on the Web cast. 

Let me start off with a little brief overview of focus groups and
where they've come from and what they are because part of
knowing about how to use focus groups goes back to how they
were developed and what they were developed for. 

A lot of this goes back to the work of an eminent sociologist
named Robert Merton.  Just before World War II he was asked
by the military to examine the morale of soldiers.  He knew this
was not an easy task and we did not have instruments. We did
not have tests to do to use to measure this, but Merton
discovered that if he put these soldiers in groups and asked
them questions in a certain way and created a permissive non-
threatening environment, that these soldiers would tell a great
deal about their experiences.  After the war, he wrote up his
work and he was the first one to actually call this a focused
interview and one of the things, if I had a sketch in front of me,
I would draw a picture right now of a funnel, and this funnel
depicts sort of the nature of the questions that are used in the
focus group because at the top of the funnel it is wider and at
the beginning of the focus group we have much more general
kinds of questions that are asked.  But then as the group
proceeds, the questions more specific, more targeted and more
to what we call the key areas of concern.  And waits this way of
structuring questions that Merton found to be helpful. 

You know an interesting thing happens when you ask people
questions.  Because a mistake that interviewers have done for
years is they ask good questions but in the wrong sequence. 
And what Merton discovered is that sequence, the way those
questions are positioned in the set of questions makes a big
difference. 

He also discovered that it was very important to create this non-
threatening environment, to create an environment where it was
okay to disagree with each other, where people felt
comfortable. 

In the military he had to be very careful to have people of the
same rank together so they would feel comfortable.  If he mixed
people across ranks, there would be power differentials and that
would not work.  He had to be sure that they felt free to
disagree with each other.  And when he created that
environment, he found it was a very interesting, exciting
conversation. 

Now, to me, part of the fascinating part of this story was that
among academics, focus groups was not embraced either -- in
spite of the fact that Merton was a giant in the academic field,
but it was not pretty much into the 1970's, late 70's and 80's that
academics came back to focus groups.  What was interesting is
academics by and large were rather nervous about focus groups
and how they were operating and there were a couple of
interesting reasons for this. 

One of those was that what they were worried about is people
influencing each other.  For example, when I was taught how to
interview some years ago in my graduate program, I was taught
that we were supposed to go out and interview one person at a
time.  First, one person, listens to them, take notes, and then
goes to a second and listens, take notes, go to a third; but never
telling the person what -- the individual you are talking to what
the person before had said. 

To do that would be called contamination.  And when
researchers looked at the focus groups early on, they saw
contamination, and that was the worst thing you could say
about someone else's work is that it was contaminated because
people were influencing each other. 

You know, what is interesting about that is that now a days we
do not use the word contamination to describe what is taking
place.  Instead, we call it real life because, in fact, that is what
happens in real life.  When you think about it, and you think
about decisions that you make over the course of time, very
likely those decisions are a product of conversations with other
people.  And one of the things that we sometimes overlook is
that our individual decisions are often a result of conversations,
discussions, interactions that we have with other people.  And
so partly to understand how those ideas are developed, we put
people in groups, like focus groups, and listen to them as they
talk about it.  And we find that out of that conversation, new
linkages begin to develop and a new understanding of how
people see issues. 

I will come back to that topic in just a minute, but there was one
other thing that academics was concerned about and it was a
very intriguing thing with focus groups.  It was the fact that
people would change their mind in the course of the interview. 
Now, think about what that might mean. If you are interviewing
someone and halfway through they talk differently about the
topic.  What is fascinating is we have done hundreds of
individual interviews.  We have done a lot of them on the
phone. We have done them in person, and I have never yet seen
a person change their mind in an individual interview or on the
telephone. 

However, it happens with some regularity in a focus group
where people will listen to others and they wll find that other
people are persuasive and they will start talking differently
about the topic because the arguments are convincing to them. 
Now, what is fascinating about that, that that phenomenon
exists is how do you analyze that?  Because what happens is we
are discovering that opinions are dynamic and changeable. 
They are not static and stable; and much of the research that is
done on opinions, you know, and various things assumes that
opinions stay the same.  And, in fact, sometimes they do, but
sometimes they do not.  Sometimes they change.  And so one of
the things to -- one of the unique qualities of the focus group is
that it is been able to tap into that when you have a topic that is
fluid and flexible. 

What we found, for example, was that when we are talking --
and this is where focus groups really took off, is during the
1950's, and particularly in market research topics.  And it was
the American business community that really started making
use of them in the 1950's, 1960's, and 1970's and still today. 
And what they were interested in was this decision-making of
consumers, how people went about making a decision about a
product.  What was it that made for a satisfied customer?  How
did people think about a product and what could we do to
position our product or to make it as appealing as possible. 
And that is where the early work in focus groups took place. 

Let me give you a quick example of a way that focus groups
were used, and this goes back some years where early on
researchers were concerned about why people select certain soft
drinks, certain sodas to drink.  Was it flavor?  Was it thirst? 
What was it that was the drive factor? And in interviews with
people, individual interviews, the people who they talked to say
things like flavor and thirst, these were key factors.  And then
an interesting thing happened when they put people in focus
groups and they started talking about drinking sodas and
drinking beverages and what was happening is people would
start talking about holding the beverage, the can or the bottle or
the glass in their hand, and how they felt as they hold it in their
hand.  And they would talk about things like being comfortable
around other people.  They would use phrases like things go
better when I am having it in my hand.  And what they would
talk about is their interaction with other people.  And as the
people who were listening to this, the people who were making
the soft drinks said, you know, we could emphasize those
things. We could tell people that our soft drink -- or we could
imply that our soft drink will do certain sociable things.  

So it is no surprise when you are on television or in the media see
pictures of soft drinks, that people figure prominently here. 
There are people around.  And the person who is holding the
beverage is often the center of attention and that you will be on
mountain tops, singing songs with people and holding hands or
you might be at a beach or at a pool and everyone is looking
admiringly at the person who is holding the beverage.  You
know, those are not accidental images.  They are very
deliberate, and they go back to -- they have their foundations in
some things that people would talk. 

Now, who would have thought that that is how you sell soft
drinks.  Now, those same techniques that were used in that
environment have been used in other kinds of things.  
For example, later -- and I am jumping ahead now several
decades, but in some of the early focus groups we are dealing
with teenagers and smoking and the concern was how do you
get kids -- or what is persuasive?  How do we create an
environment where teenagers or preteens even will not smoke? 

And what they found was that people would give messages,
reasons for not smoking, but they were adult messages.  They
weren't messages that kids would respond to. For example,
when I was in high school, we had an antismoking campaign
and I was a recipient of it.  I remember being 15 years old and
sitting in class and it was our football coach who one day gave
us a little talk about what would happen if we smoked
cigarettes.  And the answer was that we would die of
emphysema at age 65.  Now, the coach was 62 years old and he
was a smoker, and he truly believed he was going to die in a
few more years from emphysema.  And he was deadly serious
as he gave this talk, and we were not -- and we were taking this
very seriously, too, because we loved this coach.  He was a
good person.  But you know, in all honesty, underneath the
table we were counting on our fingers how many years we
could smoke before we die of emphysema, and the message he
was trying to get across of you are going to die really was not
the message we received.  The message we received is we have
got 50 years that we can smoke before we die.  And that seemed
like a long time when you are 15. 

Well, that has been one of the problems to develop effective
messages often means going to the groups, the people that you
are trying to reach, and listen to what they have to say.  What is
it that is scary about smoking to teenagers?  What is it that gets
them to think carefully about doing it or dissuades them from
doing it?  And some of you may have seen some of the more
effective campaigns recently that have worked.  One of those
that actually comes out of some work in Florida creates the
impression that major tobacco companies are out to deceive you
and you need to be aware of this in order to prevent this sort of
deception happening. And so it gets teenagers sort of their right
just indignation to the front about I am not going to be deceived
by some major corporation.  And they stopped smoking, not
because of emphysema, not necessarily because it smells bad,
but because I do not want to be deceived and it works. 

Prevention campaigns have been one of those major uses of
focus groups.  Now I am going to move kind of quickly.  What
happened over a number of years is that there have been across
over from the business sector to the public nonprofit area of
using focus groups where we find educational, religious
organizations, foundations, public health groups, starting to do
focus groups because they see the value of listening to people.
And on topics of customer satisfaction, on topics that deal with
needs that people have, on topics of prevention programs, on
topics that relate to educational efforts and how to create an
educational environment that is meaningful for people. 

Out of all this work, there was sort a set of characteristics that
was descriptive of focus groups and I will jump to page one in
your handout right now that talks about characteristics of focus
group interviews and here are some of those things that make a
focus group distinctive from other kinds of experiences, and
basically it involves four things:  The participants, the
environment, the moderator, and then analysis and reporting.
And these things each are critical factors of a focus group.  The
participants, for example, these are people who are carefully
recruited.  There were a limited number of people.  We were
finding that between six and eight is a good size to conduct a
focus group. These people have some things in common.  They
have homogeneity, and the moderator often tells them what that
is, they have experienced something, and they have that thing in
common and finally we do these groups in a series.  And we
repeat, not with the same people, but with similar people,
because what we find is that doing only one focus group can be
risky.  We often try to do a series of three or sometimes four,
maybe more in some cases, and we look for a pattern that cuts
across those groups. 

The environment is a place where people feel comfortable,
where people can see each other. Usually they are seated around
a table where, they are comfortable.  They know they are in a
place where perhaps they have been before.  Or is it tape-
recorded.  Market researchers often use one way mirrors and
special rooms, and we found, frankly, that when you start
working on topics such as what many of you are working on
that are sensitive to people, that deal with people's concerns
about health and their livelihood, one way mirrors do not work
very well.  And we have often discouraged people from using
those because people sometimes are concerned about who is
watching and who is behind the mirror and hearing what I have
to say. 

We also encourage tape recording.  I am going to come back to
that in a minute.  It is a way of capturing the data in the group. 
The moderator has certain sets of skills, of introducing the
group, leading the discussion, using a set of questions and
creating the environment.  I am going to come back and talk
about that in a minute. 

Finally we have the analysis.  We use words like systematic and
verifiable.  These are critical words that help describe how
analysis takes place. Now, I am going to jump to Page 2 where
we talk about moderator skills, and here are some things that
the moderator -- this is the person who leads the group -- needs
to think about.  And by the way, these are groups that will range
in length between about 60 minutes and two hours.  There is no
magic number in terms of how long a group should be.  Two
hours is about the maximum anyone can sit in one place
comfortably, and it takes usually 45 or 50 minutes for the
questions to start working because you need to get people
feeling comfortable with it. The most successful groups are
often about 90 minutes in length. 

But this moderator is a person who goes through a little
introduction at the beginning.  As a matter of fact, if you will
look on Pages 4 and also on Page 3, there is a script and an
outline of an introduction to a focus group. And this is sort of a
generic way of introducing the group, giving people sort of a
welcome, glad you are here.  Here is what the topic is going to
be, and also telling them why they were invited to this group,
how they were selected.  Sometimes people are curious about,
did I do something or why are you inviting me into this group? 
So we describe what that is.  We have often talked about the
social usefulness of this, why is this study valuable and who
will benefit from this.  And oftentimes we also talk about whom
the sponsor is. 

Now, by the way, this is a distinction here between focus
groups in the commercial area and the kind that many of you
will be doing.  If you go to a market research firm and
participate in a focus group, they will usually never tell you
who the sponsor is and how they are going to use those focus
groups, partly because they -- they are concerned about whether
you will say something simply by knowing who the sponsor is. 
What we find is when you are working in the public area on
topics of concern, people will not talk to you unless you let
them know who is asking for it and that that group or that
organization has a legitimate need for that information.  And
then in the outline we also talk a little bit about the guidelines,
some ground rules that will guide the conversation and then we
ask the first question. 

This first question, by the way, is around the table, around robin
question that usually is one that is non-threatening, that does
not denote status or levels, but they answer very briefly and it
sort of breaks the ice getting people to talk. 

And then the moderator continues on with the sets of questions,
using some of the skills on Page 2.  I am going to just highlight
a couple of those on Page 2. 

One of those skills that are important is to be able to control the
length and how much people say. And this is not always an
easy task.  Some people will talk at great length and with great
enthusiasm.  And some won't talk very much at all. And so one
of the strategies is to be able to somehow control this a bit; and
this is not a perfect rocket science by any means, but the
strategies often used is by using things like eye contact, for
example, by looking at people and keeping looking at them
often people -- individuals will continue to talk to you. 
However, if once a person gets off a topic, is moving on to
something different than what you are looking for or it is not
relevant to the study, oftentimes that is when the moderator
stops looking at them. If the person continues, it may be even
that that is when the moderator actually interrupts the person in
a gentle way, in a kind way, in a non-offensive way, thanking
them for their comments, but bringing it back to the topic and
letting people continue on with the topic. 

One of the things that are important in a focus group is for that
moderator to exercise that little hit of control to keep people on
the topic. As you will find some who will wander off and talk
about other issues and concerns, and the moderator needs to
bring them back and refocus on the question. 

There is a few other things and we can handle some of those as
we get through to questions in the group.  One of those that is
important to think about and another technique that works well
in groups is what we call the pause and the probe. The pause is
a very effective tool and it is interesting to watch veteran
moderators use it because it is often used like I might ask a
question and look at someone, and maybe that person is
thinking.  It takes a few moments to answer a question.  I will
just wait.  Now, some people get very uncomfortable with a
five second pause.  I mean, some moderators, they feel like it is
on the radio.  I have to talk constantly, but when you are
moderating a focus group, sometimes the most effective thing
you can do is ask a question and then just wait for an answer. 
It is interesting.  Try this sometime when ire in a group.  Ask a
question and just wait for the answer.  What you will find is that
some people just need more time to think, and you sometimes
will get wonderful answers coming out.  Sometimes it will only
take five, six, seven seconds, but that little bit of time can be
very effective. 

The probe is something where you come back with further --
you want more information, things like could you tell me more
about that?  Could you give me an example?  Could you explain
further?  People sometimes will give you very superficial
answers to questions, and a good moderator will practice sort of
looking perplexed.  You know, try it.  Look in the mirror and
try looking perplexed and then use that in the group.  Or some
people, you know, they can raise an eyebrow or they do things,
and each of us will show this in differing ways, but sometimes
you just look at them and say tell me more.  And that is a signal
that person needs to go in more depth. 

One of the problems with focus groups is sometimes people
stay on superficial, trivial things.  They never get below the
surface and get into the richness, and so a good moderator
keeps pushing people, knowing that some people can't go there,
but some can.  And that richness can be very helpful to you in
the study. 

Now, I want to skip to questions.  There are some questions
used in a focus group and some of this is on Pages 6 and 7 and
8 in your handout. And there is some techniques of asking
questions that have been have effective in a focus group. And
some of these go back to things I said earlier, having sort of a
funnel approach.  In a nutshell, there are a couple of things I
would stress:  One is do not have too many questions.  A typical
focus group of about 90 minutes in length.  We might have ten,
11 or 12 questions, but do not have 20.  I once saw one that
someone had 34 questions they were asking a group of people,
and I forgot.  It was about an hour and a half focus group.  And
you know what will happen when you do that.  You will get
trivial answers.  No one will go in any depth and you will have
just very superficial answers.  What is important is to give
people -- honor people by giving them enough time to give you
an answer to the question so you want not too many, and you
want them sequenced so that your -- what we call key questions
-- your most important questions are asked toward the end of
the focus group.  You know, that is sort of at the bottom of the
funnel now. And this is sort of the secrets of writing questions
that we do not often talk about.  When we write questions, we
actually write them in a reverse order.  In other words, we start
with the last question, the key questions, usually there are
several of them that are really most important to us.  And then
we work backwards. 

Because it is far easier to write questions backwards and have a
sequence so it fits in to where you want to go.  For example --
for example, let's suppose we were working for Coca-Cola and
we wanted questions about Coke.  There are probably certain
questions, but before that we probably want them to talk about
Colas, and before that, maybe sodas in general.  And maybe
before that just beverages, and so we have this kind of a
sequence going in so that people become comfortable with the
topics and now they are zeroing in on specifically what we
want. 

Think about your questions, try to put them in that kind of a
targeted funnel where you might spend 10, 15, 20 minutes even
talking, leading up to it and then after 20 minutes, maybe after
30 minutes, you come -- you start with your truly most
important questions. 

The difficulty is if you start first with your most important
questions, people are not ready for you.  It takes awhile for
people to feel -- even if they know the other people in the
group, it takes awhile for people to be comfortable with what is
happening, what is taking place in the group, how the entire
environment might work.  And so by having that sequence, that
sort of really helps a lot in getting at the targeted information. 
And then there is a few other strategies like -- you know, look
at your questions and if you can answer them in one word, if
you can answer it by saying yes, no or one word answer, go
back and change that question.  These questions should be the
kind that -- what we call open-ended.  They beg for some
discussion, for a sentence, a couple of sentences to explain what
is happening. 

Be careful with things that are -- where you ask why.  Now this
sounds strange I know, because that is why you are doing focus
groups is you want to know why.  But you know when you ask
why or, you know, it is okay to do this maybe once in awhile,
but I've seen questions where almost every one is why.  Why
this, why this, and you will find that the group tends to get very
defensive about how they go about answering.  And instead you
can ask it differently.  You can say tell me about this
experience, what brought it about?  What prompted it?  What
led up to it?  What caused it to take place?  And people will
give you very helpful information. 

And sometimes why leads you astray because people will try to
invent an answer.  Let me give you an illustration.  I am not a
good grocery shopper, and when I go to buy groceries, I might
have a little list with me, but I add things along the way.  I am
going up and down the aisles and I put things in my cart and
when I get to the check out, I have a lot more than what is on
my list, and if you were to stop me at the check out counter and
you were to pick up one of my items and you were to say to me,
Dick, why did you buy this?  You know what I would do?  I
would look at the label and I would look at the price and I
would tell you how good of a bargain this was and it had all the
right, healthy kind of things in it, and you know what, I am
making all this up on the spur of the moment.  That really was
not what got me to put it in.  It was more of an impulse that as I
was going by these things, I have not had these things for a
while, and it looked so appealing, I threw it in my cart.  This
was not higher order thinking going on at all.  This was
reaction, and you'd have to ask that question different to find
out truly how this took -- maybe if you said to me, how did that
get into your cart?  Or tell me about your experience and you
can ask the question differently, and I probably would give you
a different answer, but asking why may not get you there.  So
just be careful of things like that. 

I am going to jump now to the topic of analysis and there are
several items on here.  This is one of the hardest parts of doing
focus groups is the analytical part.  And this takes place actually
in several places in your notes.  On Page 9 it talks a little bit
about note taking and on Page 10, systematic analysis and 111
are some tips, and 12, and 13.  Later you might want to look
over some of these, but for now, I just want to emphasize two
words to you in analysis, and these are words that we look for. 
Those words are systematic and verifiable.  These are key
words that are important.  Systematic -- what that means is that
you have a system.  You have a strategy, a set of steps.  On
Page 10 on your notes, it shows an example of one process of
being systematic.  The reason that being systematic is important
is that it tells other people around you that you are just not
pulling this out of the air.  You are not inventing the results;
you have a way of capturing this information, a process of how
you handle the data.  For example, there are things you do well
here in the group and there are things you do immediately after
the group is over and then later there are some additional steps. 
This is all part of a system you use.  You are a researcher here
following this system. 

The word verifiable means that you have a trail of bread
crumbs.  This process   you have kept records.  You have kept
this process available so that someone else could take a look at
how you went about capturing it through things like field notes,
through things like tape recordings and other kinds of ways,
keeping a track of what is taking place. 

One of the big questions to think about is how much time you
are going to spend on analysis because in some cases you will
find situations where you should be very rigorous and very
careful.  Where you should literally spend weeks going through,
transcribing carefully everything that is said, using a careful
analytic process, and you are doing that careful process because
maybe you want to publish results, maybe it is a highly
skeptical audience and you need to be very cautious. 

But others of you in other times will find that you do not have
the time to do that.  You do not have 40 hours or 80 hours to do
analysis.  Maybe you have a very short amount of time.  And
there are some things you could do if you have just a very brief
amount of time.  Let me give you an example.  Before you
leave the focus group, and let's suppose that you are on a very
tight time line here, that you have to have results within hours
after the focus group is done.  And keep in mind now that with
speed always comes some risk here error, of things that can go
wrong.  The more careful you are, the more thorough you are,
the more rigorous you are, the more time it will take, but there
are times where you are not able to do it. 

So one of the strategies is before you adjourn the focus group,
you might actually give a little oral summary right there to the
group.  In other words, it could be the moderator or it could be
one of the associates that is working with you on your team. 
And you take two minutes and you say here are the key points
that we have talked about.  Tell me if I've got them right.  And
then you go through these key points.  Maybe you use an
overhead -- not an overhead, a flip chart, or you go through
these key points, and let them give you feedback, yes, you have
heard it right.  Or maybe change this or maybe change that.  But
that is a way of speeding up the analytic process by having
them involved, the participants involved. 

Now, I am going to jump to recruitment, page 15 of your notes. 
This is another big challenging thing because you cannot do a
focus group if people do not show up.  And this was one of the
major factors early on that gave us a great deal of concern
because we knew how to moderate, ask questions and do
analysis, but we did not know how to get people to show up for
the focus group.  Now, that sounds like a real simple thing, but
it was not.  What we found was that there were a couple of
secrets that we sort of discovered and we learned them by
watching the people doing market research.  And the answer --
the secrets are on the top and the bottom of page 15.  On the top
of the page it talks about systematic notification procedures. 
And what this is a step-by-step process of how you invite
someone, and the secret here is that you personalize the
invitation, and you are repetitive in that invitation.  We will
invite people three times.  The first contact will be usually in
person or on the telephone, and we make it as personal as we
can be.  We will tell that individual about why we want them to
join with us. It is certain things they have or certain
experiences.  We do not say to people, you are one of a
thousand people.  No, not ever.  You are special. You are
someone who has wisdom about a topic we would like you to
join us for a conversation. Would you be able to do that?  And
we try to be very honoring and up beat in this first contact and
the second pointed of contact is in writing.  We send them a
letter of what it is, who is sponsoring and provide the details. 
The third point of contact is sort of a reminder phone call.  We
call it the dentist reminder call where the day before you get a
call reminding them of the group.  We have had people tell us
you must really want me here because you have invited me
three times.  We say, yeah, that is right. 

And then, at the bottom of page 15, the other thing to think
about is the incentive.  You know, sometimes we use money. 
Sometimes we can't use money or we do not have money
available.  In market research, that is the predominant incentive
is they will pay people 20, 50, $100 or more, to come to the
focus group; but in nonprofit and public areas, we are not
always able to do that.  And sometimes there are other things
that are as effective or more effective than money.  It might be
that people -- you are doing a study to help other people.  Let
me give you an example. 

We were doing a study with veterans who had certain
disabilities and what we found was that if we would tell the
veterans that this study would help other veterans with
disabilities, we found that it greatly increased the likelihood
they would show up because veterans care a lot about other
veterans.  That is magic to them.  You say this might help
others and if you can build a case, they will come, because they
are doing something that serves others.  There is that brother
hood of veterans.  It was a powerful factor.  That is a powerful
incentive for coming.  For other groups it could be things like
how you invite them or where it is or it might be food. 
Sometimes a small gift of some kind, but think about what it is
that will get people to actually show. 

Okay, let me talk a bit about working with a team of volunteers
in doing focus groups because this may be a case for some of
you.  And one of the issues to think about is, okay, who should
do this study?  Is this a place where you could turn this over to
volunteers, to other staff, to colleagues to help you?  The
answer is yes; but you need to be careful and thoughtful about
this.  We have done a lot of studies where many of the studies
that I have been involved in lately have been ones where we go
in and I have gone in and trained a group of people in a
community and they, in turn, go out and do focus groups with
other people in the community.  And what we found is that it is
very effective.  It is very effective for a couple of reasons, that
these people who are the -- and I am going to use the word
volunteers, however, they may be staff or they may have other
capacities, but basically they have been willing to share with
you some time and help you with the study. 

We find they are highly committed.  We find that they have a
lot of energy and they have contacts.  They know of people that
you or I might not know, and they can get to places and open
doors and do things that you or I might not be able to do.  They
may know languages that we do not know. They may know
things about culture or traditions that you or eye may not know
and they can be very effective in doing this.  And they might be
the kind that can go into a place and be immediately trusted
because they come from an area or they have an experience.  So
they have great potential. 

But now here is the challenge:  If you are working with a team
of volunteers, it is critical that they march together, that they do
things in common.  If you have a team of five or six or ten and
everyone does their own study, you can't pull that together. 
That is impossible.  So you have to have a structure, a way to
sort of being sure they all ask the same questions so that you
have compare ability, and that they go through some training
kind of a process so they are very familiar with what roll they
are going to play because some volunteers will very naturally
be very good at certain parts and some are good at other things. 

One of the things we have learned with volunteers is to do not
expect every volunteer to be a moderator.  You know, there are
certain tasks that some people are better at.  Some people are
blessed with a natural ability to lead a small group discussion,
and some people are far better at other things like recruiting, for
example.  I mean, if you are doing focus groups, it is very
helpful to have a volunteer who is willing to -- on the phone or
in person -- contact people and invite people to come to the
group.  What we will often do is assemble a team of people and
then look at the different responsibilities that we have.  We will
give people chances to practice each of the different skills, and
then we will try to make an assignment or put people in places
where they have some strength.  Some might moderate the
group. Some might help us with recruiting.  Some might sit in
the groups and maybe take notes.  Some of them might be
skillful at analysis and being willing to transcribe or even to go
over the content of material and to share -- and prepare a report. 
And there may be others who are -- who have gifts in terms of
communicating the results when you are all done.  That is one
of the blessings of having a team of people as they come in with
different skills.  And your challenge as the team leader is to
how to deploy those skills and how to get them all working
together. 

Just a couple of tips to help you in this: One of them is when
you are working with a team of people, give them lots of
chance to practice the focus groups and give them feedback. 
When we prepare them, we often will -- we do it in differing
ways, but typically it is about somewhere between 12 and 16
hours of instruction, and it is about 50 percent practice.  If they
are teenagers, we do even more practice, but we will actually sit
down and do focus groups and we will stop and we will give
feedback and we will practice another group and we will
demonstrate and we will give them coaching and feedback, and
we find that, you know, it does not work just to tell people how
to do it.  They need to practice it and get feedback and then they
become quite good at it. 

And then another tip is to watch them perform for you before
you make assignments.  Watch them lead a group, and then
decide if they should do that with others.  Watch them do
recruiting. Practice recruiting each other and then see if you
want them to do it. 

Also we have them work in teams when they go out to do
things.  They usually have a partner that works with them, and
we find that team to be very effective because also if something
were to happen, you have a teammate that can help you. 
Another tip is to over sample.  Do more focus groups than you
need because your quality will vary.  You do not know where
this is going to occur at the beginning but at the end, you will
look back and you will say these were superb, wonderful. 
Some of the best I have ever seen, and then there are some over
here that the quality isn't that good, but if you have over
sampled, you have got enough of the quality level that will help
you through the process. 

And finally, just be respectful of the time these people are
giving you because oftentimes they've got other things they
could do with their time as well.  So -- now you should all be
congratulated because you have gone through about three
credits worth of material here in about 40 minutes and I am
serious about that.  When I teach a class, we do a lot of practice
as we do it, but this is sort of the key element of what is
involved.  In looking back, there are a couple of parts that look
easy that are really hard and some things that are hard that look
easy.  For example, analysis -- analysis is a difficult task and if
you have had -- you know, if you are doing it for the first time,
it is going to be a struggle and you will wonder whether you
really got into the right thing or not.  A lot of people worry
about moderating, but we find that that is one of the easiest
things to teach, putting people through experiences and letting
them do it a few times.  There are some people that come so
naturally to moderating -- for example, I have seen some
teenagers who are so good at this, that, you know, I would like
to think that I helped them, but you know, they came with these
skills already and all I did was point them in the right direction
and give them an outlet and they developed those skills, but
there are some people who are just marvelous.  And part of the
secret of moderating is not talking.  You know, people come up
to us and they say, well, I do not think I should moderate
because I do not like talking in front of a group.  Those are the
people I really like as moderators because I do not like to see
people who talk a lot in front of a group moderating because the
moderator's job is to listen.  The moderator's job is to create an
environment where other people talk and if you have someone
who loves to hear themselves talk, I wouldn't let them moderate
a group.  I would put them in a position where they need to
listen because the more the moderator is talking; the more you
are losing information.  The moderator's job is to get everybody
else in the group to be talking and to be answering the
questions. 
Well, let's stop now and probably there are some questions that
we have along the way that we need to attend to.  And so let's
see, was it Mark, Duff the questions or who asked the
questions? 

DAWN:  This is Dawn at ILRU and we do not have any E-
mail questions that have come in just yet. 

RICHARD:   Okay, folks, start typing up those questions so we
have some because it will make it a lot more fun if we can deal
with your questions.  But does anyone else have a question that
they would like to ask? 

LAUREL:  Well, I do, Dick.  This is Laurel Richards.  And one
is when -- in doing an evaluation -- let's say of a program, the
quality of a program such as a service delivery program, is a
focus group a good means of obtaining information on how you
are doing? 

RICHARD:   Yes.  Well, the answer is yes, but let me give a
little explanation.  You know, one of the principles that we
always try to follow in good research and good evaluation is to
use multiple methods because what we have learned is that the
human experience is so complex that there is no one way that
we can discern what is happening.  And the challenge I think, if
it is something like (Inaudible) -- which can be complicated is
to think about when and how you might use differing kinds of
methods.  For example, a very simple answer is to use some
methods that deal more with qualitative kinds of responses such
as focus groups or interviews, and then to move that into a
quantitative where you might use some kind of a survey, where
you have people rate or rank differing things. Oftentimes we
use focus groups first because it helps give us an understanding
of vocabulary, language, big concepts that people use,
motivating factors, barriers and other kinds of issues; but the
limitation of focus groups is you do not know how many people
feel that way.  All you know is that somebody or maybe even
with passion they feel that way, and so in order to determine,
how many or what percentage of people, you need to use some
kind of a survey strategy where you do a sampling with more of
a random sampling across a population. And so part of the
answer is that I would use focus groups, but I would also use
other means as well, and having them working sort of together
is going to give you some powerful results when you are done. 
The challenge is going to be how you do all that on a limited
budget and that is often the practical difficulty that people have
is you have a limited amount of time or a limited budget and so
how do you get the best results for that limited amount of time. 
And so what you will have to think about is, you know, maybe
one part or the other you do on a very limited scale.  In other
words, you might do focus groups fairly quickly or a survey
perhaps with a more limited sample, but you try to keep the
costs low, but keep the other technique is, keep that one at the
higher level of quality. 

LAUREL:  Thank you.  It is problematic in part because -- for
instance, in an activity such as web-related activities, the
funding agencies always want to know, you know, a qualitative
assessment of it and when anonymous people drop in and out, it
is complicated.  And yet we are expected to have precise
outcomes for any activity that we -- in which we become
engaged.  And one solution was suggested was focus groups,
but I think you are sort of saying that you need to mix different
mediums. 

RICHARD:   Let me talk a bit about strategy that people are
starting to use and in this -- it is so much in the developmental
stages that hasn't settled down a lot, but one of the things we are
beginning to see are people using the computer and electronic
communications together with focus groups.  And one of the
simple ways, for example, is to have people on the telephone. 
It is like a telephone conference call, only you are doing a focus
group on the telephone.  And when that happens, we typically
have fewer people and a shorter amount of time.  And it might
be five or six people, and usually it is 60 minutes or less and
you can have people across the country all hooked together on a
telephone and have a nice conversation on a topic.  And that
works. Now, you could do the same thing on the Internet where
people have like a chat room where you can go in and as a
moderator you have people sign in at a certain time.  You put to
them certain questions and they can respond on their keyboards.

Or another variation that is being used is where people will
volunteer to be -- to spend 15 minutes each day for five days
and sometime during the day they volunteer to log in on a Web
site and on the Web site is a question that is being asked.  And
they are supposed to read the question and then to type in their
answer.  And then the second day, you can call it up again and
you will see all the answers of everybody else and then the
person who is leading it will give you a question that builds on
the first question.  It sort of takes what was said yesterday and it
helps you sort of think about the question of the day.  And so
then you kind of look over the comments of yesterday and then
you write your answer for today.  It happens a third day and the
fourth day and so you have that kind of a focused discussion. 
After about the fifth day, what you are finding is that people are
getting really into this and you are building on the conversation. 
What people like is they can sign in at any time during the day
as long as they put in about 15 minutes and read the questions
and give an answer.  But people are starting to do things like
that where they are taking it from a person to person and
actually doing it on a keyboard or a telephone, and you know,
in a few years what is going to happen is we will have these
cameras on our computers and we will be able to look at each
other and he will have people connected and be able to see their
faces on your computer.  I mean that is not very far a way and
you will be able to do a focus group right in your office looking
at your computer screen. 

LAUREL:  (Laughter).  We are getting closer and closer. 

RICHARD:   Yeah, it is.  And the technology is there.  It is just
that the price of it hasn't come down yet to the place where it is
sort of commercially available.  The professionals can do it.  As
a matter of fact, one of the things that is sort of the big technical
thing that is used in a lot of focus groups are -- and this is more
in the market research environment -- that they will have a one
way mirror and behind the one way mirror they will have a
video camera.  And they will tell people in the group that you
are being recorded, but they won't tell you exactly how you are
being recorded.  You know, you do not know.  Is it video, audio
or what?  But they will -- there are people listening and we are
recording, but behind the one-way mirror they may have a
video camera and they may be broadcasting that actually back
via telephone line to headquarters.  So this might be happening
in Minneapolis, but back in Detroit, back in New York, back in
Los Angeles, there are people watching you on a big screen T.
V. in real-time and, you know, and what they are doing is they
are the people who have designed the product, whatever it is,
and they are watching you talk and if they have a question they
might write it down on a piece of paper, send it by fax to the
moderator or actually comes in to the back room.  Someone
walks into the focus group and hands the moderator the
question and the moderator asks this next question and it
actually came from Los Angeles, Atlanta or where ever and the
participants were really not aware that they are being watched
by people at some distance a way. 

LAUREL:  Good heavens. 

RICHARD:   Yeah. 

HELEN:   I am wondering if you can talk a little bit about
recording.  I know that some people are -- people who are new
to focus group research are often a little squeamish about
recording and audio vs. video. 


RICHARD:   I would like to because one of the things that we
have become passionate about is we have become very big fans
of audio recording but not video recording.  Let me tell you
why.  What we often use and what we encourage is a remote
microphone, one that is sort of unobtrusive that can lay flat on
the table and then we run a little cord off the table to a tape
recorder.  Good old-fashioned tape recorder and we tell people -
- we always tell people we are recording and we give them the
reason why.  And we tell them that we can't write down things
fast enough and people will say things so quickly that we want
to be sure we get them.  And we try to convey that it is an
honoring thing.  We are doing it because people have this
wisdom, these insights and, you know, we just can't capture it
fast enough.  That is true. My memory is fading, but my
memory and my notes are not good enough and when I go back
and listen to that audiotape, I pick up things that I missed
completely. And so we have become very fond of the
audiotape.  Now, there are certain times and places where you
can't do it, but if at all possible, I would encourage you to try
doing it.  It does not mean you have to transcribe it.  I mean,
that is another decision you might make later and that is a costly
decision.  It could cost you 150 bucks for example, maybe
more, maybe less, if you are going to hire someone to transcribe
that focus group interview. But it might be that you just want to
listen to it or listen to parts of it instead of the entire thing. A
video recording, on the other hand, we find that it bothers some
people.  If we use a video recorder, we try to always put it out
where people can see it.  We find that what works in market
research does not work very well in the kinds of focus groups
we are doing, and we found that people are more concerned
about the video recording. One of the possibilities, though, that
could be considered is to have someone sit in the focus group,
not right at the table, but back a little bit from the table with a
laptop computer where this person, if they are fast on the
keyboard, they might actually be able to get close to a complete
transcript in real-time and we have people who are pretty fast,
70 words per minute, 65, 70 words per minute and depending
on the speed at which people talk in a group, they can almost
get a full transcript in the actual time and the real-time that that
focus group is going on.  And if the laptop is quiet, it may not
bother people, and people are becoming more comfortable. 
That becomes another possibility. And certainly field notes, you
know, just good old fashioned notes being taken of the
comments and the quotes can be very helpful, but if you are
moderating, it is just too difficult to take notes and moderate at
the same time.  When I moderate, I take very sketchy notes
because I want to look at people when they are talking.  I want
to take notes that will help guide me in that overall
conversation.  So I will have a colleague work with me whose
job it is to sit in that room and to help just take notes in that
room. So the note taking -- the way you are capturing
information is kind of like a stairway now.  At the lowest level
of the stairway is what we might call memory where -- and this
is also the most fragile -- because it is true that you might be
blessed with a good memory and you can remember much of
what is said if you are not too distracted at tend of the group. 
But memory is fleeting and it is imperfect, and so field notes
become sort of the next level, the next level of being more
rigorous. And then beyond that, beyond field notes it might be a
laptop or it might be a tape recording, and so on, and so what
you have is a series of steps. Each step that you take may take
greater investment of time and energy and cost, but each step
also tends to improve the quality of the data that you have. I
guess I would just caution that in literature on interviewing
there is a concept called selective perception and it is an
interesting idea.  And what it talks about -- what this concept is,
is that as human beings, one of the limits that we all have is our
way of perceiving and understanding things.  That we are in a --
for example, listening to a focus group and people are talking to
us about a topic, and they start talking about something that we
do not understand, and what happens is we stop taking notes
and we are trying to figure out -- because you are wondering
what should I write down.  I do not know what they are talking
about now.  It is confusing to me.  So I am thinking for a
minute as this person is talking and they are going on for 20 or
30 seconds and I am still trying to figure this out.  I do not
know what they are saying.  And I keep thinking now if I wait
longs enough it will come to me.  They will say will something
and I will be able to write something down and then pretty soon
this person stops and someone else starts talking and now you
understand this other person; but you have lost this part that you
did not take notes on.  Your perception limited you here and it
was selective in the sense that we write down things that make
sense to us and when it does not make sense, we do not record
it.  We are not able to perceive it.  That is the limit we have and
that inadvertently we write down things we like and we agree
with.  We may not always write down things that we do not
understand or that we do not feel comfortable with.  The tape
recorder, though, is Ruthless.  It records everything.  And there
is value of having that extra bit of care.  That is why also it is
good to have two people in the focus group so that if one person
sort of perceives something especially as you are working in
areas of disabilities or people who have different experiences in
life, it is good to have someone who is close to that experience
who can make sense out of that.  Because someone coming in to
it fresh or new is not going to be able to make sense of some of
those unless they've had enough experience with it. 

HELEN:   That raises an interesting point for me in that part of
what this project does is to help people better understand the
idea of participatory action research, the idea of having
participants from the community involved in all aspects of the
research.  And so when you are coming to choose moderators
for your group, how important is it to choose a moderator that
has the characteristics of the individuals that are going to be in
your group? 

RICHARD:   You know, that is a great question because it is
one of those where if you would have asked me ten years ago or
20 years ago, I would give you a different answer and probably
ten years from now I will give you a still different answer, but
let me start with ten years ago I would have said that it is very
important to have a person who has had an intensive, deep
experience with that because things will happen, and they need
to know how to ask and how to follow up and how to get
information.  And there is still truth in that. There is truth in that
answer, but I've also seen more studies that we have done
recently where sometimes an outsider -- someone who does not
have the experience, whatever it might be, is able to get people
to talk about an experience in ways that an insider may not be
able to. Let me give you an example.  We were doing some
focus groups on diabetes prevention up on an Indian reservation
in northern Minnesota, and one of the issues that we were
concerned about was exercise and how to get people from the
reservation involved in some kind of an exercise program.  And
when we had -- in this particular study we were doing a series
of focus groups and we had some moderators and we tried to
emphasize this was a key part, which were from the reservation
themselves.  We trained them.  They were local residents.  They
were Indians, they led the groups and by and large they did a
great job.  And then we had some who were Caucasians and
they came in and led the group but they were off reservation
people.  And we found an interesting thing when we looked at
the results. When the people from the area asked questions, they
did not push people.  They did not ask, for example, people
would talk about barriers to exercise and people would say --
one of the answers would be the dog.  The dogs.  And the
person if it was an Indian who was asking would just move on
to the next issue, but when the white person asked the question,
and they said dogs, the person raised their eyebrow and said I
do not understand.  What does dogs have to do with exercise? 
And then they went on to talk about leash laws and on the
reservation they do not have a way of leashing the dogs and the
police will not pick up dogs and we have some ferocious dogs
out here that if you are walking or your children are walking to
school, you very likely could have some very fierce dog
following you that would be very frightening.  And the reason
we do not exercise outside is there are dogs out there and they
are scary.  Now, it was when someone actually sat down and
explained that like we were a child, but that is what we needed
for us to come a way with information.  You see the Indians
who were doing the group knew that.  This was all a parent to
them; but we as outsiders were not aware of what that is.  So
one of the things to think about is getting people to explain their
experiences in their life and sometimes an outsider can get
people to tell you more than an insider can.  And I am not
saying that is true in every case, but we have seen some places
where there is an advantage to having someone from the outside
who, in a very sincere and a trusting way, says to people help
me understand how does this work, tell me what is taking place.
And we find that people will often open up and tell us great
insights that they may not tell someone who has lived there for
a long time. 

HELEN:   I can see that. I can see that everyone makes
assumptions and so if the moderator is too integrated into the
community, the participants may assume that the moderator
already knows things that they do not need to voice then. 

RICHARD:   Yeah, you said that really well.  I should write
that down.  You said it better than I did.  Exactly, that is the
case. 

HELEN:   One of the things I have thought about in having a
moderator that is part of the community is that that is one of
those components that may assist in creating that non
threatening environment so that individuals feel more at ease
with this person.  This person they believe has shared some of
their experiences and can maybe understand them Bert.  Are
there other suggestions that you can give for creating that non-
threatening environment for people who are new to this kind of
research? 

RICHARD:   Yes.  This will apply to only some people and not
to others, but humor, humor can work, but you have got to be
careful with humor here. Because some people -- some of those
people listening now today are good at it and some of you are
not good at it.  And if you are not good humor, if you are not
comfortable, it is best not to use it.  Do not ever use humor at
anyone's expense, but I have seen some moderators in focus
groups who will use a gentle kind of a humor that will relax
people, make people feel comfortable in the group, and will
make it kind of fun.  And it will lighten it up.  And sometimes it
does not even take a lot of humor.  Sometimes it takes a smile. 
You will be surprised -- we have watched people go out in
groups and they will take this very seriously.  You know, I am
going out on a research study now and I've got all my materials
and they go into the group and when you look at them, these are
now members of our team who are going to be moderating, they
look very serious.  I mean, they do not smile.  They look like
this is going to be a painful experience and one of the things we
-- we push our people to do is to smile.  I mean, one of my
favorite shows on T. V, it may be showing my age, but I like to
watch Lawrence Welk, and you know what is fun about
Lawrence Welk, watch those people smile.  I do not know what
they do, but they force them to smile and it is sort of like that
kind of a smile, you know, that I am happy to be here.  I am
happy that you were willing to come and share information. 
And when you are in a group and someone who is asking
questions and leading you -- it isn't a phony smile, it is a sincere
smile.  And even if it is a serious topic, to let people know that
we are pleased that they are able to be there and to share and to
not have that scary, straight faced look like is threatening to
people.  So I think those kinds of things can help. What we
have seen is that every moderator has his or her own set of
skills.  And it is a mistake to ask each person to be a carbon
copy of another person and when we work with a team of
people, we tell each person, you have got to be yourself.  And
each of you do different things to establish trust and rapport
with people.  And you have got to think about what will work
for you and some of you can use humor.  Some of you can -- it
is a smile.  Some of you will do other things. Another thing that
often is important is what we call -- you know it sounds kind of
corny, but it is what we call small talk.  You go in to do a focus
group in a neighborhood or a community or some location, and
not everyone shows up at the same time.  Some come early and
you have got a few minutes before it starts.  It is quite important
that the moderator stops talking to people.  And you can talk
about the weather and traffic and you can talk about sports. 
You know, try to avoid controversial topics, but make people
feel welcome and get them used to sort of talking to each other.
And that sort of breaks the ice and so when you then actually
start the group, people have already practiced a little bit.  We
send out a lot of graduate student who take this like real
seriously and they have forgotten thousand do this little chitchat
small talk, and so we actually practice with them ahead of time. 
Now, okay, here is what you say.  Ask them about this or talk
about this. Talk about road construction, talk about the weather. 
Talk about the football, basketball, baseball game that took
place over the weekend, and what some professional
moderators do, interestingly enough, is one of the first thing
they do when they arrive in a small community is they will get
the local newspaper to find out what is happening in the
community.  And that will give them some insights about what
to do with small talk. 

HELEN:   That is an interesting approach. I had never thought
of getting the local newspaper in order to figure out what small
talk to do, but a very good point.  I think those are all good
points about creating that non-threatening environment so
people feel comfortable to voice their opinions.  I do not want
to Monday open lies you.  So, dawn, do we have any E-mail
questions? 

DAWN:    We have got seven that came in. 

HELEN:   Fire away. 

DAWN:    First one, how would be the best way to structure
questions when you are needing input on what to put in the next
state plan for independent living? 

RICHARD:   Well, you know, off the top of my head, it strikes
me that what you are looking for are some ideas of what might
work.  I would be -- just kind of rambling now a bit, but I
would be tempted to get people to talk about what really has
been effective in the past.  In other words, what -- in looking at
past plans or past efforts, last few years, what has really worked
well and needs to be continued?  And I would probably ask a
question about what is a good idea that we have not been able
to do very well and what are those things.  And what are the
barriers that get in the way that we need to be fixed?  And so I
think what I would do is start talking about what is worked well
in the past and what is been problems in the past and what can
we do about those as we look into the future?  What do we need
to adapt and to change?  But what I would be careful about is --
I do not know how your plan works, but I would think right a
way about different levels of people and I would keep them
separate.  I think you have got some professionals who know
about plans and I would ask them the questions and I would
keep them separate from perhaps the client tell or the people
who are the recipients of the plan or the consumers, if you will. 
And I would ask them questions, but you know, the groups
would be separate.  And maybe there are still other groups as
well.  You know, a way of doing plans is also asking people in
other places who are doing plans what is worked for them and
what have you found to be successful?  So you can go on in a
number of different levels but I would look at what is worked
well, what is not worked well and let them give you some
advice on those things.  What is next? 

DAWN:    I think you have already touched on this
subjected so I am going to read two questions kind of together. 
The first one is please expand a little bit more on how to keep
participants on track, and also have you ever had to provide
significant amounts of education to participants before
conducting the focus groups? 

RICHARD:   Let me start with the second one.  Yes, we have
had certain topics that are very confusing and complex.  In fact,
we just did some on insurance and why people do not have
insurance and we had to go back in and review some basics of
insurance.  And so when that happens, we will often in one of a
variety of ways have a little educational thing at the beginning. 
It might be that we take five or ten minutes and give people a
little overview of the topic, or we might have an expert there
who gives a little description and is available to answer
questions, but this expert knows that they are not to give
opinions, but just to answer questions on the topic.  And this
expert maybe remains in the group as a resource to anyone who
has a question about that complex topic. Sometimes we will
have a video that we share with people ahead of time to kind of
bring them up to speed or we will have handouts or materials.
Sometimes even sending it to them in advance, but quite often
there will be some body of knowledge that we want people to
see and that will take a little bit of time. Now, what you have to
be careful of is you do not have much time to do a lot of
educational efforts.  So you will have to think about what is
really critical here for them to see and what you have time to
put in. Now, keeping people on track this can be a problem. 

There are some groups where everybody is, you know,
excellent.  And there are others times where we have what we
call these people whose minds go where -- what is the line from
Star Trek -- go where no one has gone before?  Their minds
wander. They get off topic easily.  And when that happens, part
of the moderator's job is to bring them back. I have learned this
though in watching a lot of focus groups are that people are
somewhat predictable.  That if they wander on you on one
question, they are likely to do it again.  And there are people, if
they stay on topic and they are crisp and clear, they will
probably continue to do that.  But some of the techniques I've
used for example is -- sometimes I do not do this normally, but
I might even write the question on a flip chart so everybody
sees the question and remembers the question.  Sometimes what
happens is people forget the question; and so if you have a
group of people who might wander on you or if I am seeing
this, I might just jump quickly on the flip chart and write the
question down so that everybody can stay focused on the
question. Or I will just repeat the question.  I will say, you
know, here is the question we are talking about, and so
everybody stays on topic.  You can't enforce it 100 percent, but
you can do those little things to be sure that people are kind of
sticking with you. I have about five minutes to go. Can we take
one more question?  Is that okay? 

LAUREL:  Yes, that would be great. 

DAWN:    Well, two more that could maybe go as a couple. 
Would you discuss the use of software programs for analysis,
pros and cons and also have you ever used qualitative research
software to analyze focus group data?  If so, what software
would you recommend? 

RICHARD:   Well, I commend your -- whoever is asking this
are some very sophisticated people out there.  Because most
people do not even know this software is out there, but there are
some excellent software's out there.  The most common are
things called Nvivo, and it is an out growth of another program
called Nudist, with an asterisk and another program is called the
ethno graph, and these programs, what they are designed to do
is you take a transcript from a focus group or an interview, and
you bring them in and then you do what we call coding.  You
assign labels to various sections and then later on your
computer you can call up those codes very quickly across a
variety of focus groups. Now, here is my take of this:  These
programs, they are excellent programs, but they are expensive. 
They will cost hundreds of dollars, and they are -- they have a
steep learning curve.  They are not intuitive; they are not easy
to pick up.  I would tend to use those if you are doing a series of
studies where rigor is very important, where you are going to
have a lot of -- a large number of studies, and I would also try
to get someone who is very skillful or has had some experience
with this software to help you with it because if you are starting
off doing focus groups and learning this software at the same
time, it is just too much to handle. What I would suggest is
taking a look at those pages that talk about the long-table
approach, which is -- in fact, it is a simplified way of doing
what the computer is doing.  And one of the things that is
happening in the field is people are starting to use other kinds
of programs like Microsoft word or Microsoft excel or access. 
These are not designed as qualitative research tools, but people
are using those kinds of software and they are developing their
own coding schemes and analyzing results using those
preexisting kinds of software.  It is an interesting possibility.  If
you were to ask me do I use it?  The answer is I sometimes will
use nudist or Nvivo or the ethno graph.  I do not normally do it
because it still takes too much time to go through the entire
process.  At some point in the future, this will be solved
because we will have instant voice transcriptions on computers,
but we are not there at this point right now. So we have covered
a lot of material, and if there are some additional questions,
what I propose, if you want to send them to me by E-mail, I will
put an answer down and if a question comes up and you want to
send me a question, I will do my best.  I cannot guarantee I will
get back immediately, but feel free to drop me a note.  I would
be happy to help in whatever way I can. 

DAWN:    Yeah, there are two more questions here that I
will just forward them to you. 

RICHARD:   That is great.  Well, thank you very much.  I very
much enjoyed being with you and I am looking forward to
hearing more about the kinds of projects you are involved in. 

HELEN:   Thank you, Dr. Krueger, you have done a great job
of giving us practical information for conducting focus groups
and I will let Laurel take care of ending us off. 

LAUREL:  Just a wonderful presentation. I am sitting here
taking notes on key issues and one is this thing on the
development of the questions and how important that is, and
I've been processing that.  This has just been -- just terrific. 
And, Dick, I think you may get the award for the guy with the
best voice on all the Web casts that we have done. 

RICHARD:   Thank you. 

LAUREL:  Excellent.  And I suspect our captioner will tell you
have a wonderful pace as well. This Web cast is part of
NIDRR's series of initiatives to promote the use of research by
those of us working in the disability fields, independent living
and rehabilitation fields.  A lot of us are stakeholders in the
outcomes of research initiatives and are beneficiaries, and we
do not always have an opportunity to hear from the researchers
outcomes of the findings or like today the mechanisms for
actually conducting the research.  And I think this one in
particular is going to serve a lot of the needs of those of us in
the independent living fields who are concerned with capturing
the needs and ideas and interests of the folks with whom we
work and serve. This is also a project, which we are conducting
in partnership with our colleagues at the law and health policy
and disability center, the folks at the University of Iowa College
of Law.  The project is called the technology for independence,
and it is a very good program, which is looking at research and
how to craft studies and examine problems and issues that folks
working in independent living experience, but taking a research
perspective of that, very practical and useful. In fact, Helen, I
believe you all have a new Web site, which is extremely
attractive.  It is a repository of information, not just about the
project but also other useful information on assistive
technology. This presentation will be archived along with other
Web casts that we conduct.  We have done as, Helen said, I
guess about four other Web casts on research-related topics and
we invite you to go to the Web cast page and you will click on
the one of archives of previous Web casts and there you will
see, in a couple of days there will be this one, and you will click
on the subject area and it will take you to the web page where it
will have the same information that is here, the handouts, the
about the presenter, about Dick Krueger, and it will have a link
to this audio presentation which you can listen to again and it
will also have a link to the transcript which is, as you can see on
the right-hand side of your screen, is being done as -- during
throughout this presentation. So we invite you to check that out
and the other presentations that are there.  We will also have a
discussion forum.  Dick, we can set up this discussion forum so
that the questions that are sent to you, we can post on the
discussion forum along with your response and, therefore, folks
can drop in and get the information that others may have
requested. 

RICHARD:   That would be great. 

LAUREL:  We will take the initiative in capturing your
response and putting it back on there.  And then, finally, to
acknowledge the support also of the research information for
independent living project which helps to support this initiative. 
And we at ILRU have a series of people who have been
extremely helpful to us in working on this.  And, Helen, I think
I mispronounced your last name before, it is Schartz. It is not
Schwartz.  Helen and her colleagues and Peter Blanck and
James Schmeling and Heather Richey are our partners on the
independence project and here atILRU, we have the input of
Marj Gordon, Sharon Finney, Dawn Heinsohn, Rachel Kosoy
and Mark Richards to make these Web casts happen as well as
our technical team of Rob Dickehuth, who somehow makes the
audio portion, which is being inputted via telephone line, to
broadcast over the Web site so that you all can pick it up from
your own computer there where you are.  And Marie Bryant
who is doing the real-time captioning.  Thanks to all of you. 
Thanks to NIDRR, and we look forward to you joining us for
our next presentation, and, again, Dick it is just been terrific. 
Thank you very much. 

RICHARD:   Thank you.  God bless. 

LAUREL:  Good afternoon now from those of us in Houston
and Iowa and in Minnesota and guess who is the warmest? 
Good afternoon now.