Principles Supporting Qualitative Research

Qualitative Research Defined

What is it?

Qualitative research is used to describe a broad range of research strategies with roots in the social sciences, especially anthropology and sociology. There are various approaches to qualitative research.

Approach	Key Questions
Case Study	What are the characteristics of this particular entity, phenomenon, person, setting?
Ethnography	What are the cultural patterns and perspectives of this group in its natural setting?
Ethology	How do the origins, characteristics, and culture of different societies compare to one another?
Ethnomethodology	How do people make sense of their every day activities in order to behave in socially acceptable ways?
Grounded Theory	How is an inductively derived theory about a phenomenon grounded in the data in a particular setting?
Phenomenology	What is the experience of an activity or concept from the perspective of there particular participants?
Symbolic Interaction	How do people construct meaning and shared perspectives by interacting with others?

Beliefs Connected to Qualitative Research

∑ The importance of Voice - - the qualitative researcher wants the reader to understand that the researcher is a human being, not a disembodied distraction, and believes that writing in this more personal style is more “honest” and better able to convey the richness and depth of human experience.

∑ The inclusion of Multiple Perspectives - -the qualitative researcher believes there are multiple ways in which the world can be known.

∑ The Construction of Knowledge - - human knowledge is constructed from experience and therefore is a reflection of the mind as well as nature. Knowledge is made, not simply discovered.

∑ The importance of Form - - the forms through which humans represent their conception of the world influences what they are able to say and how they interpret it. Form also influences experience. (Example - - a sociologist conducting a study will be influenced by the content of their experience which will in turn influence future experience. The form of representing knowledge in sociology is linguistic which will influence how representations of knowledge are presented. Sociology tends to embrace particular epistemologies which influence methods of investigation and influences potential explanations).

∑ The primacy of Experience - - neither science nor art can exist outside of experience. Observation and making meaning of events is influenced by life experience. This applies both to the researcher and to the research participants.

∑ Understanding as Holistic - - the researcher focuses on the wholeness of experience rather than on objects or parts. Experience and behavior are integrated and inseparable relationships of subjects and objects, of parts and of the whole. The whole is greater than the sum of the parts.

Features or Characteristics of Qualitative Studies

∑ Field Focused/Natural Context - -Qualitative studies occur in natural settings. This includes places where people interact (e.g. in the classroom, at the board meeting, in the park, etc.) and to the study of inanimate objects (e.g. how textbooks are written, how buildings are designed, how classrooms are organized, etc.). Qualitative studies are non-manipulative, that is they tend to study situations and objects intact - - the researcher observes, interviews, records, describes, interprets, and appraises settings as they are. They focus on naturally occurring processes.

∑ Self as Instrument /Personal Contact- - The researcher engages in the situation and attempts to make sense of it. Data are collected through human observation and are interpreted through perceptions. Sorting into significance is done through the schema held by the researcher. The instrument (researcher) is recognized as subjective. The researcher has direct contact with and gets close to the people, situation, phenomenon. The researcher’s own experiences and insights are an important part of the inquiry.

∑ Interpretive Character -- The researcher attempts to explain why something is taking place. There is a focus on answering “why” and “how” something occurs. There is a focus on meaning rather than specific behaviors. The focus is not on outcomes, but on process. Rather than statistical analysis, qualitative research seeks to generate conjectures and patterns. Their focus is on identifying and interpreting patterns of human responses as a result of their own knowledge, experiences and theoretical orientations. Questions are about how things occur, how people interact with each other, how certain kinds of questions are answered, what meanings people give to words or actions, how attitudes are translate into actions, etc.

∑ Use of Expressive Language - - the presence of voice in text and the use of expressive language to create empathy (connecting readers to the text) is a goal. Good qualitative research helps connect the reader to the experience using detailed, thick descriptions and using direct quotations to capture experiences and perspectives.

∑ Depth of Perspective/Attention to Particulars - - Intent is to retain the flavor of a particular event, individual, situation or object. Attention is paid to aesthetic features of the case. The researcher is aware of the distinctiveness of the setting/context and conveys these to the reader. There is also a focus on depth (rather than breadth) of understanding. Hence the typical focus on a small number of cases or participants studied in depth over time.

∑ Inductive Analysis - - The researcher is immersed in the details and specifics of the data to discover categories, dimensions and interrelationships. The research begins with open-ended questions rather than by attempting to test theoretically derived hypotheses (deductive approach).

∑ Dynamic Systems - - There is attention to process and an assumption that change is constant and ongoing whether the focus of the study is an individual or a culture.

∑ Unique Case Orientation - - assumes that each case is special and unique and that the first level of inquiry is to respect and capture the details of the individual cases being studied. Cross-case analysis follows from and depends on the quality of individual case descriptions.

∑ Context Sensitivity - - Findings are placed in a social, historical, and temporal context limiting generalizations across settings, time and space.

∑ Empathic Neutrality - - complete objectivity is impossible, but pure subjectivity undermines credibility. The researcher is attempting to understand (not prove) something in all its complexity. The goal is not to advocate, not to advance personal agendas, but to understand. Personal experiences and empathic insight is included as part of the relevant data while taking a neutral non-judgmental stance toward whatever content may emerge.

∑ Design Flexibility - - the research is open to adaptation as understanding deepens and/or situations change.

∑ Use of “Word” data - - kinds of data collected include interview transcripts, field notes, photographs, audio recordings, videotapes, diaries, personal comments, memos, official records, textbook passages, and anything else that can convey actual words or actions of people. Data are not reduced to numbers.

∑ Persuasiveness - - Rather than focusing on statistical evidence, qualitative studies persuade by reason, recognizing the social nature and the influence that perspective has on the ways people “know.” The reader is persuaded by the coherence of the case, the cogency of the interpretation - - whether the interpretations “seem right.” There are no tests of statistical significance, and in the end, what counts is a matter of judgment. Success of a qualitative study is based on coherence, insight, instrumental utility, believability, trustworthiness, and logic of interpretation.

∑ Focus on emic perspectives - - Qualitative researchers write from the perspective of the participants (emic perspective) rather than from the researcher’s own perspective or that of an external observer (etic perspective). Their research is focused on the perspectives of the subjects in the study - what they are thinking, why they are thinking it, the participants’ assumptions, motives, reasons, goals, values.

Limitations of Qualitative Studies

∑ Subjectivity is inherent - - The researcher has substantial control over both the design and the analysis and the study is influenced by the researcher’s perceptions.

∑ Labor intensive - - collection and analysis occur simultaneously, the amount of data gathered is enormous, and few computer programs are available to assist with management and analysis. Much of the work is done by hand.

∑ Misunderstanding of Novices - - people think they can engage in qualitative inquiry with limited or sometimes no training. Quality and accuracy of studies are then compromised.

∑ Time Intensive - - the purpose of qualitative research is to seek deeper understanding of a specific phenomena and understanding generally develops over time. Depending on the topic, it often takes a year or more to conduct a qualitative study.

∑ Limited Generalizability - - the unique features of the setting or individuals studied makes it difficult to generalize. Qualitative researchers argue that all social structures are unique and therefore generalizability is an inappropriate standard by which to judge qualitative research.

Types of Questions Addressed in Qualitative Research

What is happening in a specific setting?

What do specific events mean to the people involved?

Are events organized into patterns and principles that guide everyday life?

How are events in a particular setting related to events at other system levels inside and outside the setting?

How does the manner in which life in this setting is organized compare with other ways of organizing life in other places/times?

What are the meanings and perspectives of participants in the setting?

What are the relationships regarding the structure, occurrence, and distribution of events over time?

Qualitative research is useful for describing or answering questions about particular, localized occurrences or contexts and the perspectives of particular groups toward events, beliefs, practices. It is also useful for exploring a complex research area about which little is known or for beginning to understand a group or phenomenon.

Qualitative questions:

∑ Have the potential to illuminate everyday life, to make the familiar strange and better understood

∑ Can provide the specific, concrete details to guide an understanding of a particular setting

∑ Can provide interpretation of local meanings that activities and practices have for the group engaged in them and illuminate differences across settings (e.g. how one teacher implements whole language might be considerably different than another teacher, but both would indicate they were using whole language).

Warning: Choice of a research topic should not be based on very personal issues, as individual passion may undermine objectivity. Do not set out to “prove” something you already know. The question or problem to be addressed determines the appropriate research approach. Method follows (not precedes) the topic of investigation.

Comparison of Qualitative and Quantitative Paradigms

Quantitative research emphasizes

1. Starting with specific hypotheses or questions derived from theory/previous research

2. Selecting a large, random sample representative of the population

3. Using objective instruments (e.g. achievement tests, attitude scales, etc.)

4. Presenting results using statistics and making inferences to the population.

5. “Distance” between researcher and subjects and emphasis on following the research plan

Qualitative research emphasizes

1. Starting with general research problems and not formulating hypotheses (although hypotheses may emerge from the data analysis and may shift during the course of the study).

2. Selecting a small, purposive sample (not random) which may or may not be representative of the larger population.

3. Using relatively unstructured instruments (e.g. interviews, observations, etc.) and “intense” data collection (e.g. over extended periods of time).

4. Presenting results mainly or exclusively in words and de-emphasizing generalizations to the population

5. Researcher awareness of their own orientations/biases/experiences and personal interaction in the context with an emphasis on flexibility in the research.

	Quantitative Research	Qualitative Research
Approach	aims and methods of social sciences are not different from other sciences and strives for testable and confirmable theories to explain phenomena using value-free (objective), focused, and outcome-oriented methods	subject matter of social sciences is fundamentally different and requires a different goal for inquiry and a different set of methods of investigation that is value-laden (subjective), holistic, and process oriented
Assumptions	data can be objectively and reliably drawn from a sample and accurately reflect reality, data can be manipulated mathematically to objectively quantify differences or relationships; research is the discovery of social “facts” devoid of subjective perceptions and divorced from context.	social phenomena are complex and interactive, no single number of group of numbers can account for variability in social interaction, reality is constructed through perceptions which are not subject to simple appraisal and numeric summary; behavior is bound to the context in which it occurs and social reality cannot be reduced to variables in the same manner as physical reality, it must be understood from the “insiders” perspective and involves the subjectivity of both the researcher and the participants
Data	Numeric, operationalized variables, quantifiable coding, statistical analysis, counts, measures	Narrative, descriptive, people’s own words, personal documents, field notes, artifacts, official documents, audio and video tapes, transcripts
Data Collection	Non-participant structured observations, structured and formal interviews, administration of tests and questionnaires, experiments, survey research, quasi-experiments, existing data sets	Document collection, participant observation, unstructured and informal interviews, taking of extensive and detailed field notes, review of artifacts, focus groups
Data Analysis	deductive, statistical, yields primarily numeric data which are susceptible to statistical analysis - parametric and non-parametric, descriptive and inferential . Raw data are numbers and analysis is performed at the end of the study	ongoing, inductive, models, theories, concepts, constant-comparative methods; typically yields verbal descriptions largely derived from interviews and observational notes; notes are analyzed for themes and patterns which are described and illustrated with examples, including quotations and excerpts; coding of data and verbal synthesis; raw data are words and analysis is ongoing and involves synthesis.
Data Interpretation	Conclusions and generalizations are formulated at the end of the study, stated with predetermined degrees of certainty	Conclusions are tentative, reviewed on an ongoing basis, and generalizations are speculative or nonexistent
Design	structured, predetermined, initial hypotheses, formal, inflexible, specified in detail in advance of the study; involves intervention, manipulation and control	evolves over time, flexible, developing hypotheses, general, negotiated, a hunch as to how to proceed, specified only in general terms in advance of the study; involves non-intervention and minimal disturbance
Generalizability	Assumes researcher can discover “laws” that add to reliable predictions and control of phenomena; searches for regularities in samples of individuals/groups; statistical analysis reveals trends in behavior and trends are sufficiently strong to have practical value.	Assumes each individual, culture, setting is unique and it is important to appreciate uniqueness, therefore, generalizations are context dependent.
Goal	Explain causes of phenomena through objective measurement and numerical analysis	Understand social phenomena through holistic pictures and depth of understanding
Hypotheses	Tests hypotheses; hypotheses are specific, testable, and stated prior to the study	Generates hypotheses; hypotheses tentative, evolving, based on a particular study
Key concepts	variables, operationalize, reliability, validity, statistical significance, replication, prediction	contextualization, process, field notes, triangulation, insider/outsider perspective, meaning making, making judgments
Measurement	Goal of measurement is objectivity, meaning the scoring and collection of data is not influenced by the researcher’s values, biases, and perceptions; heavy reliance on tests, scales, structured questionnaires with specified procedures for scoring, standardized tests; collection of numeric data; analysis occurs at the end	researcher is the key measurement tool, procedures are subjective, observe and interact with humans/environment, non-standardized; narrative data; analysis ongoing
Methods	a priori decision rules (p<.05), control of variable, descriptive, correlational, causal-comparative, experimental	evolving methods, naturalistic, unobtrusive, ethnographic, case studies, focus groups, historical
Model of Explanation	Relies on hypothetical-deductive model of explanation with inquiry beginning with theory from which hypotheses are drawn and tested using predetermined procedures	Believes the search for generalizations is misguided because human behavior is always bound to the context and must be interpreted case by case
Problems	control of extraneous variables, validity, approach obtrusiveness, validity/reliability issues	time consuming, data reduction is difficult, reliability, generalizability, non-standardized procedures
Reasoning	Deductive reasoning - deduce from theory what should be observed	Inductive reasoning - make observations and then draw conclusions
Research Purpose	prediction, produce causal statements, establish relationships, generalize findings	gain insight, describe events and patterns, increase understanding of ideas, feelings, beliefs and motives taking context into account
Research Setting	Controlled to the greatest degree possible	Naturalistic (as is) to the greatest degree possible
Review of Literature	Extensive, significantly affects design of particular study	Limited, does not significantly affect particular study
Role of Researcher	researcher strives to be objective, impartial, detached. Investigators goal is objectivity; seeks to keep personal views, beliefs, biases from influencing data collection and analysis process; involves minimal interaction and if interaction is necessary (i.e. interview) then try to standardize the process. The sample’s role in the study is passive.	researcher acknowledges an interest in the outcomes, personal involvement, and that there is subjectivity in selection of the question, methods chosen, and analytic procedure; researcher deliberately interacts in a personal way; data collection procedures are open to modification depending on the situation; researcher is free to use intuition and judgment as a basis for deciding how to frame questions or how to make observations; individuals studied may be given opportunities to volunteer ideas and perceptions and even participate in data analysis
Sample	randomized, controlled for extraneous variables, size important, intent to select “large”, representative sample in order to generalize results to a population; stratified, control groups, precise control of extraneous variables	non-representative, small, purposive, intent to select “small”, not necessarily representative sample in order to acquire in-depth understanding; theoretical sampling, based on the context
Soundness Criteria	Internal validity - how truthful are the findings; external validity - how applicable are the findings to other settings; objectivity - how sure is the researcher that the findings are reflective of the subjects rather than the product of researcher biases; reliability - how sure that the findings could be replicated	Credibility - inquiry conducted in such a manner to ensure that the subject was adequately identified and described; transferability - burden of demonstrating the applicability of findings rests with the investigator who would make that transfer rather than with the original researcher; dependability - accounting for changing conditions in the phenomenon; confirmability - can the findings be confirmed by another researcher
Study of Context	Seeks to understand complex phenomena by analyzing component parts (variables); each investigation examines only a few of the possible variables that could be studied; the situation context is either ignored or controlled.; data are collected at a few intervals and focus is on precise measurement	Seeks to understand a complex phenomenon by examining it in its totality in context; may not know what to focus on until study is underway; identifies relevant themes and patterns (emergent) which then focus the study; data collection is continuous and intensive
Techniques	experiments, standardized instruments, structured interviews, structured observations	observation, open-ended interviews, review of documents, participant observation, researcher as instrument
Values	Admits inquirers values may play a role in what problem is investigated, but investigation itself should be value free with specific procedures designed to isolate and remove all subjective elements and leave objective “facts”	Argues that inquiry is always value bound and inquirers must be explicit about the roles that values play in any given study. Says values are inherent in the choice of problem, methods, ways to interpret, and selection of the context where the study takes place. Personal values are to be explicit and attempts to expose values embedded in the context.
View Values of the World	The world is relatively stable, uniform, and coherent, therefore it can be measured, understood and classified	The world is neither stable, coherent, nor uniform and therefore “truth” cannot be obtained because perspectives and understandings differ from group to group
Words to describe	experimental, hard data, empirical, positivist, statistical, objective	Ethnographic, fieldwork, naturalistic, descriptive, participant observation, soft data, subjective

Some Qualitative Approaches/Designs

Case Study Research

In a case study, one case is selected of particular interest. This case is observed on a regular basis over a period of time. The focus of the research is on process. The question is focused on what can be learned from this particular case. The case could be one individual, or one organization, or one group, etc. An intrinsic case study is undertaken because the researcher wants a better understanding of a particular case, not because it represents other cases or illustrates a particular trait. Instrumental case studies involve examination of a case in order to provide insight into an issue or refine a theory. Collective case studies involve study of a number of cases in order to understand a broader phenomenon or a general condition. Uses multiple data sources.

Grounded Theory

One of the most widely used interpretive strategies used in qualitative research. The focus in grounded theory is to unravel elements of experience and use interrelationships to build theory that enables the researcher to understand the nature and meaning of an experience for a particular group of people in a particular setting. Researcher constantly reviews field notes, etc. and attempts to provide explanations to guide future observations. These observations then confirm or disconfirm the explanation. There is a constant shift from observing and collecting data to analyzing. The researcher stresses open processes in conducting the research rather than fixed methods and procedures. Focus is on process questions - experience over time or change that may include stages or phases. Rooted in sociology. Primary data collection strategy is recorded interviews. Other data sources may include observation, journaling, diary, memos.

BiographyAutobiography/Life Stories

This method seeks to report on and document the history of a person’s life. These can be objective, historical, artistic, narrative, personal, collective, institutional. Many researchers study problems anchored in their personal biographies.

Participative Inquiry/Action Research

This method attempts to make qualitative research more humanistic, holistic, and relevant to the lives of humans. It views participants as co-creating their reality through participation, experience, and action. There are four phases of action research: the co-researchers agree on an area of inquiry, ideas and procedures are applied in everyday work/life., co-researchers become fully immersed in the activity/experience, co-researchers reconsider the original research problem. The goal is to produce knowledge and action directly useful to a group of people. A second aim is to empower people through the process of constructing their own knowledge. Methods include unstructured observations, journaling, surveying, and reviewing documents/records.

Hermeneutics

The focus in hermeneutics is on consciousness and experience, on the inner worlds of the participants as related to texts. It includes a historical component to add to the meaning of the experience by reflecting on political and other activities of potential influence. Hermeneutics involves the art of reading a text so that the intention and meaning behind appearances are fully understood.

Ethnography

Combines participant observation and many of the characteristics of non-participant observation. Attempts to obtain a holistic picture of a group, setting, or situation. The emphasis is on documenting or portraying the every day experiences of individuals. Key tools are in-depth interviewing and continuous observations. There is seldom an initial hypothesis. The research is sustained over time. The goal is to paint a picture that as thorough, accurate, and vivid. Example of an ethnographic question: What is life like in an inner-city high school? Strengths: provides a comprehensive perspective. Provides a deep and rich understanding of behavior. Is useful for topics that are not easily quantified. Nuances that may be missed with other methods are detected. Appropriate for behaviors that are best understood in their natural setting. Limitations: highly dependent on particular researcher’s observations, few ways to check validity of conclusions, observer bias is difficult to eliminate, generalizability is almost nonexistent., variables may remain unclear. Questions are descriptive, focused on understanding the values, beliefs, and practices of a cultural group. The method is rooted in anthropology. Primary data collection methods are unstructured interviews, participant observations, and field notes. Other data sources may include documents, records, photographs, maps, genealogies, social network diagrams.

Phenomenology/Ethnomethodology

the aim in phenomenological research is to determine what an experience means for the persons who have had the experience and are able to provide a comprehensive description of it. From these individual description, general meanings are derived. It involves interpreting the originally given descriptions of a phenomenon using reflective analysis and interpretation of the participants’ accounts. The intent is to understand meaningful concrete relations implicit in an original description of experience in the context of that particular situation. Questions are focused on meaning, eliciting the essence of experience. The method is rooted in philosophy. Primary methods of data collection are audiotaped conversations, written anecdotes of personal experience. Other data sources may include reflections, poetry, art.

Historical

Historical studies are past, rather than present, oriented and thus use different data collection techniques than other qualitative approaches. The purpose of historical research is to gain insights or reach conclusions about past persons or occurrences as well as current events. It is more than compiling factual information. It requires interpretations. The interpretive emphasis and the use of non-numeric data place it in the qualitative domain.

Historical studies may focus on particular individuals, particular social issues, links between old and new (e.g. comparing textbook changes in the last three decades). Some historical research seeks to reinterpret prior historical works (called revisionist history).

Data for historical studies include newspapers, legislative documents, court testimony, diaries, committee meeting records, yearbooks, memos, relics, photographs, reports, and sometimes interviews with living reporters of past events. These are categorized into primary and secondary sources. Primary sources are first-hand information sources - - eyewitness reports and original documents. Secondary sources include second-hand information - a textbook author’s description of a theory, someone who heard about the event but did not experience it firsthand. A major problem with much historical research is over-reliance on secondary sources.

Historical data should not be accepted at face value. Diaries, memoirs, reports, testimonies reflect the interpretation of the author and may be biased. Sources should be examined for authenticity and truthfulness using external and internal criticism.

Portraiture

Portraiture is a relatively new method of approaching qualitative studies. Sarah Lawrence Lightfoot describes the process in her book The Art and Science of Portraiture and examples are provided in two of her works, Respect and The Good School. The goal is to paint a vivid portrait or story that reflects meaning from the perspective of both participants and researcher. The researcher’s experiences become part of the portrait. Methods include in-depth interviewing and observations over a period of time. A personal relationship between the researcher and the participants is developed.

Semiotics/Discourse Analysis

This is the study of linguistic units. It looks at the relationship between words and their meanings stressing the system of relations between words as a source of meaning. Views language as a social construction. Words are viewed as signs that bring together a concept and an image, derive their meaning from their place within an articulated system, are arbitrary (e.g. different languages use different terms for the same concepts), can be put together in combinations/patterns (syntagmatic relationships), and their use excludes other choices (express paradigmatic oppositions). Questions focus on verbal interaction and dialogue. Primary data collection strategy is recorded dialogue (text-based, or audio/video recorded).

Qualitative Data

Techniques for Acquiring Qualitative Data

Data collection in qualitative studies occurs in a particular context - commonly called fieldwork. Collection occurs simultaneously with analysis and interpretation and occurs throughout the study. Think of it as collecting “waves” of data, each wave providing additional information that further focuses the nature of the study. The final product is a rich description or narrative with quotations typically used to illustrate the voice and understandings of the participants. The focus is to use language to paint a rich picture of the setting and its participants.

Selecting a Setting

Do not choose place where you have been integrally involved for you may see only what you expect to see. The ideal site is:

One where you have access

One where there is a high probability that you will find a combination of people, programs, and interactions related to your question

One where you will be able to maintain a presence over a period of time

Determine why a particular site should be selected.

How will permission be obtained for accessing the site?

What will be done at the site?

How will the researcher avoid disrupting the normal routine?

How participative will the researcher be?

What will be the duration and frequency of observations?

Interviews

Interviewing is used to gather information in the subjects own words from which insights on their interpretations can be obtained. Qualitative researchers generally use open-ended informal interview techniques (and generally do NOT use fixed-response questionnaires or surveys). Subjects are encouraged to talk about experiences, feelings, beliefs. The most important characteristic of a good qualitative interviewer is to be a good listener (i.e. don’t talk!).

Interviews can vary from informal conversations to open-ended interviews to in-depth discussions with key informants. Sometimes structured interviews are conducted to compare views across different subjects, groups, or settings. Interviews also vary in that there may be one-time interviews, multiple interviews with the same participant, multiple interviews with various participants, or group interviews.

Types of Interviews

Unstructured - - exploratory, only an area of interest in chosen beforehand, interviewer “follows his/her nose” in formulating and ordering questions. Also includes impromptu conversations that occur during observations.

Partially Structured - - Area of interest is chosen and questions are formulated, but order is up to the interviewer. Interviewer may add questions or modify them as deemed appropriate, questions are open-ended and responses are recorded nearly verbatim, possibly taped.

Semi-Structured - - questions and order of presentation are determined. Questions are open-ended, interviewer records the essence of each response.

Structured - - questions and order are pre-determined, responses are coded by interviewer as they are given.

Totally structured - - questions, order, and coding are predetermined and the respondent is presented with alternatives for each question so that phrasing of responses is structured. Questions are self-coding in that each choice is pre-assigned a code.

General guidelines for interviewing:

Listen more, talk less.

Follow up on what participants say and ask questions when you don’t understand.

Avoid leading questions, ask open ended questions.

Don’t interrupt. Learn how to wait.

Keep participants focused and ask for concrete details.

Tolerate silence.

Don’t be judgmental about participants’ beliefs or views. You are there to learn about their perspectives whether you agree or not.

Don’t debate with participants over their responses. You are a recorder, not a debater.

Differences in qualitative and quantitative interviews

Quantitative interviews are similar to “survey research” in that there are mainly fixed-choice questions and generally random samples. Interviews in qualitative studies are generally open-ended and generally use small, purposively selected samples.

Focus Groups

Focus groups is a type of interview where multiple participants are involved and responses can build on one another. A focus group is particularly useful in obtaining a variety of views or opinions about a topic or issue. Focus groups are used to obtain information of qualitative nature from a predetermined and limited number of people.

A semi-structured group session, moderated by a group leader, held in an informal setting, with the purpose of collecting information on a selected topic. A carefully planed discussion designed to obtain perceptions on a defined area of interest in a permissive non-threatening environment, appropriate when the goal is to explain how people regard an experience, idea, or event, and conducted with a small number of people by a skilled interviewer. The focus group:

• Allows for group interaction that may bring out information that would not come out either with observation or in response to preconceived questions

• Allows for greater insights into "why" opinions are held

• Allows the researcher to interview a larger number of people in less time

• Allows collection of "rich" data at a reasonable cost

• Is useful in improving planning/design of programs or evaluation of programs

• Advantages: Socially oriented, Flexible format allows probing, High face validity, Low cost, Increases sample size of qualitative studies

• Disadvantages: Less control than in individual interviews, Data more difficult to analyze, Requires trained interviewers, groups vary considerably, Groups are difficult to assemble, Must be conducted in an environment conducive to conversation

• Useful When -

Insights are needed in exploratory or preliminary studies

There is communication or understanding gap between groups

The purpose if to uncover factors relating to complex behaviors

The is a desire for ideas to emerge fro the group

There is a need for additional information to prepare for a larger study

Client/audience places high value on capturing the open-ended comments of target audience

• Not Useful When -

The environment is emotionally charged

Statistical projections are needed

Other methodologies can produce better quality information

Confidentiality cannot be ensured

The researcher has lost control over critical aspects of the study (participant selection, question development, analysis protocol) and the study is prone to bias

Procedures

• Duration 1-2 hours. General rule is to plan for less time than you tell participants.

• Number Groups 3-6 different groups should be used.

• Size 4-12 with certain characteristics in common (over invite by 20% to cover for no shows); over 12 NOT recommended; IDEAL size 6-8.

• Composition participants alike in some way (not in opinions). Subgroups may be necessary if there are differences in background or role-based perspectives. General rule is to keep groups homogenous in terms of prestige or status.

• Sample systematically selected (purposive sampling). In organizations, include groups with different roles.

• Method - non-directive, nurture different points of view, identify trends and patterns in perceptions.

Types of Questions

• Introductory Round robin question that everyone answers

Background - to locate people in relation to other people

• Transition Move the conversation into the key questions

Experience/behavior - to elicit descriptions of behaviors, actions, activities

Knowledge - to find out what respondents consider factual information

• Key Questions that drive the study (2-5 questions)

Opinion/value - to find out what people think

Feeling - to understand emotional response of people to an experience

• Ending Bring closure to discussion

Asking the Questions

• Memorize the questions (and the introduction) and try not to refer to notes

• Start with a general question so the discussion is not restricted

• Questions should be asked in a natural progression

• Only 3-5 broad questions should be introduced in a focus group. The facilitator may have some sub questions in mind, but should not dominate the discussion.

• Facilitator introduces a question followed by unstructured group discussion until the facilitator introduces a second question. Time criterion may be used.

• Before going on to the next question, the facilitator summarizes for the group to check out accuracy of perceptions. This allows participants to clarify and correct.

• The pause - - learn to be silent longer than the participants

• The probe (use sparingly) - - ask - - Explain further, provide an example, Describe what you mean

Guidelines for Focus Groups

• Do not explore too many topics

• Limit discussion to 2-5 key questions

• Use open-ended questions (avoid dichotomous questions and why questions)

• Use structured guide (or topical guide) to aid in analyzing across groups

• Build prior team consensus on what questions to ask and at what level of detail

• Word questions in familiar language, use words that make sense to participants

• Ask uncued questions first, then follow-up with cued questions if needed

• Standardized strategies may be used (sentence completion, conceptual mapping)

• Serendipitous questions are best asked at the end of the session

Necessary Skills for Facilitators

Timing

Curiosity for the topic

Respect for participants

General background knowledge

Ability to communicate clearly

Self-discipline

Friendliness and humor (smile)

Ability to listen, not talk

Ability to avoid expressing personal views

Familiarity with questions

Ability to listen and think simultaneously

Ability to remember what was discussed (past/present/future perspective)

Maintain low level of facilitator involvement

Take care not to lead the discussion

Remain neutral and aware

Observations

Participant Observer - engages fully in the activities being studied but is known to the participants as a researcher. Example: a researcher gets permission from a teacher to sit in a class and make observations over a semester. Participant observation is a common qualitative method. Goal is to immerse researcher in the setting so he/she can see, hear, feel, experience, subject’s’ daily life. Sometimes referred to as “privileged observation.”

Unobtrusive or Non-participant Observer - researcher watches but does not participate in group activities. Example: researcher conducts a number of interviews with teachers in a school.

Naturalistic observation - observing individuals in their natural settings, making no effort to manipulate variables or control activities, but simply to observe and record. Example: observation of students at an athletic event.

Covert Observer - researcher disguises identity from other participants (ethical issues). Example: a researcher becomes a certified teacher and takes a position in a school for the purpose of conducting research and without telling anyone of this intent.

Simulations - asking subjects to act out certain situations or roles. May be individual or group role playing. The subjects are told what to do, but not how to do it. Example: individuals are asked to portray two administrators discussing a problem teacher. There is no guarantee that what occurs would normally occur in real-life.

Observer Role

There are degrees of observer involvement in the setting. A complete participant is not known to other participants as a researcher. The researcher interacts in the group as naturally as possible and for all intents and purposes is one of the group. The participant-observer participates fully in the activities of the group but also makes it clear he/she is doing research. The observer-as-participant identifies self as a researcher but makes no pretense of being part of the group. The complete observer observes the group without becoming in any way a participant in the group. The subjects may not realize they are being observed. There are advantages and disadvantages to each role. The complete participant is most likely to get the truest picture, but there are ethical questions. The complete observer is least likely to affect the actions of the group. The participant observer may have some effect on what the group does. The participant observer and the observer-as-participant are both likely to focus attention on the activities of the research rather than their normal routine and thus affect typicality.

Potential Problems with Observation

Observer effect - refers to the fat that the presence of an observer can have a considerable effect on the behavior of subjects and hence on the outcomes of the study. The data reported may reflect more the biases of the observer. Observer bias refers to the possibility that certain characteristics or ideas of observers may bias what they “see.” This is probably true no matter how impartial the researcher attempts to be. NO one can be totally objective; all are influenced by past experience. Observer expectation is when the researchers know they are observing participants with certain characteristics (e.g. high IQ) and then may expect a certain type of behavior. Use of audiotapes and videotapes helps in some respects to guard against selective observations.

Differences in qualitative and quantitative observations

Quantitative observations use checklists with preliminary work done prior to the observation. Tally marks are generally used to identify behaviors. Qualitative observations use words to describe behaviors and attempt to describe what is happening in a context.

Textual/Content Analysis (artifacts, documents, records)

Documents and artifacts can include materials produced by the subject (writing portfolios), personal documents (diaries, letters), records (memoranda, meeting minutes, newsletters), memorabilia (yearbooks, newspapers), documents/photographs from historical archives.

Includes analysis of written records such as textbooks, novels, newspapers, music, speech transcripts, photographs, birth and death certificates, curricula, diaries, letters and well as analysis of non-written records such as audio tapes, video tapes, computer files.

Differences in qualitative and quantitative content analysis

Quantitative content analysis uses counting in terms of researcher identified categories. The categories are sufficiently precise to enable different coders to arrive at the same results. Qualitative content analysis develops categories after reviewing the content in an attempt to understand participants’ categories.

Personal Experience (journaling and other methods)

This method involves directing participants to recall personal experience through a variety of techniques. Participants may be asked to tell family stories, to journal about particular topics, to tell life stories in response to a set of questions, to construct a chronicle of their lives, to recall memories in response to specific artifacts (e.g. a photograph, grandmother’s quilt, etc.), to explain their thinking. Methods used in data collection include think-aloud techniques, stimulated recall, and key event reporting. The researcher works with the participants to create the data.

Field Notes

Written descriptions of people, events, objects, places, activities, conversations, etc. These notes may supplement information from official documents and interviews or may comprise the main research data. Field notes should also include the researchers’ reactions, reflections, and tentative assumptions.

Two basic aspects: descriptive and reflective.

Descriptive aspects includes: verbal portraits, reconstruction of dialogues, complete descriptions of physical settings, accounts of particular events (who was involved, how, what was done), details about activities, researchers’ own behavior, maps, date/time/place, characteristics of those present, documentation of gestures, direct quotes.

Reflective aspects include: speculations about the data analysis and emerging patterns/themes, comments on the research method (accomplishments, problems, decisions), records of ethical dilemmas and conflicts, analysis of the researchers’ frames of minds, points of clarification, hunches, confusions.

Condensed field notes are taken on site using abbreviations and key words and phrases. More detailed field notes are then developed as soon as possible (the more time that elapses, the more likely the researcher is to forget important details).

Hints for Field Notes

Write on only one side of the paper (less confusing, especially if you photocopy or cut them up later). Big margins for coding and afterthoughts. Use a shorthand system (your own). Do not discuss observations until after you have written the field notes as conversation can alter perceptions. Read through notes to clarify and fill in. Be careful to distinguish between what actually happened and what your perceptions are of the event. Keep the descriptive and reflective sections separate (one suggestion is to divide the paper down the middle). Try to recognize and dismiss your own assumptions and biases and remain open to what you see - trying to see things through the participants’ perspectives. Number the lines in the field notes to aid in finding particular sections. Finally, do not assume that you know what you are looking for until you “experience” the setting and participants for awhile.

The concept of field notes has been expanded to include use of photographic and audio and video recordings that can be used to document physical layout, artifacts, details that are difficult to document verbally.

Recording Information

Advantages: reduce the need for taking complete field notes. Can be retrieved later and used as a guide for constructing extensive field notes. Useful in assembling exact quotes and gathering information about such things as tone of voice and gesture. They provide a more comprehensive description of events. They contribute to reliability as they are permanent products that contribute to the audit trail.

Disadvantages: presence may make participants more sensitive about what they say or do, over-reliance may lead to situations where the technology does not work and no other data is collected or situations where alternative plans have not been developed.

“Sampling” in Qualitative Research

Sampling in qualitative studies is generally small and non-random. The nature and intensity of data collection in qualitative studies limits the number of participants or cases that can be included. Qualitative researchers should provide detailed information about how the participants were selected and should provide detailed descriptions of the participants, including the size of the sample and relevant characteristics (e.g. age, socioeconomic status, gender, ability-level, etc.).

How many participants/cases are enough? The answer is “It Depends!” There are no hard and fast number rules to represent the correct number of participants in a qualitative study. Resources (time, money, participant availability, participant interest, etc.) will influence the number of participants. There are two general indicators of adequate participants. First is the extent to which the participants represent the range of potential participants in the setting. The second indicator is redundancy of information (when the researcher begins to hear the same thoughts, perspectives, and responses from participants, little more can be learned). This is referred to as data saturation.

Types of qualitative sampling include:

1. Extreme or Deviant Case - selecting cases that are unusual or special

2. Intensity Sample - selecting participants who permit study of different levels of the research topic (e.g. some high-achieving students and some low-achieving students, some experienced teachers and some inexperienced teachers).

3. Typical Case - selecting typical cases to study

4. Maximum Variation - selecting participants based on diverse variations in order to identify common patterns

5. Stratified Purposeful - selection based on subgroups, several cases at each of several levels of variation of the phenomenon

6. Homogeneous Sample - selecting participants who are very similar in experiences, perspectives, or outlook to produce a narrow sample and make collection and analysis simpler.

7. Critical Case - selecting a single case that provides a crucial test of a theory, program or phenomenon

8. Snowball or Chain Sample - selecting a few people who can identify other people who can identify other people who would be good participants for the study. This is useful when a study is carried on in a setting in which possible participants are scattered or not found in clusters.

9. Criterion Sample - selecting all cases that meet some criteria or have particular characteristics (e.g. students who have been retained for two consecutive years, female administrators with more than 20 years of experience).

10. Theory-based or operational Construct sample - sampling by choosing examples of a theoretical construct

11. Confirming and disconfirming case sample - selection of participants to look for variation or exceptions; confirming cases are selected to confirm patterns, themes or meanings found in previous cases; disconfirming cases are selected because they are believed likely to disconfirm previous findings

12. Purposeful random - selecting by random means participants who were purposively selected and who are too numerous to include all in the study (e.g. randomly selecting 10 students from the senior class of 50). This approach lends credibility to the sample, but is still based on an initial sample that was purposively selected.

13. Politically important sample - selection of participants who are well known and would create wide interest in the study

14. Convenience sample - selection of participants because they are available and easy to study (this strategy should be avoided).

15. Opportunistic sample - selecting participants based on taking advantage of unexpected situations

“Validity and Reliability” Concepts in Qualitative Research

Researcher Bias

One potential threat to validity in qualitative studies is “researcher bias” resulting from selective observations, selective recording of information, selective reporting of information, allowing personal views to affect data interpretation. Strategies to enhance bias-free research

∑ Reflexivity - - means that the researcher actively engages in critical self-reflection about potential for bias (self-awareness and methods to control bias). Qualitative researchers often include a section in their reports that discusses researcher bias, sometimes called autobiography in qualitative dissertations.

∑ Researcher Journaling - - the researcher documents his/her thinking during the research process.

Descriptive Validity

Factual accuracy of the account (Did what was reported as taking place actually happen? Did the researcher accurately report what was seen and heard? ) Descriptive validity increases credibility and defensibility of research. Also adds to confirmability (ability of others to document findings). Strategies to enhance Descriptive Validity:

∑ Investigator triangulation - - use of multiple observers to record and describe the context and participant behavior and to interpret the data. Cross-checking among multiple observers helps ensure that investigators agree on what took place. Corroboration of observations across multiple investigators decreases chance of external reviewers questioning the research.

∑ Audit trail - - documentation and maintenance of records to allow others to verify the description. Also used for interpretive validity as researchers’ thinking is documented as is the process used to analyze data and create coding.

Interpretive Validity

Accurately representing participant reality, accurately portraying the meaning attached by participants to what is being studied. The degree to which the researcher understands the participants’ views, thoughts, feelings, intentions, experiences and portrays them in the research report. Strategies to enhance Interpretive Validity:

∑ Participant Feedback/Member Checking - - sharing interpretations with the participants in order to clear up any misunderstandings. Asking participants whether they agree with what you have said about them.

∑ Low Inference Descriptors/ “Thick, Rich Description” - - Verbatim information, including the actual language, dialect, and personal meanings of the participants (e.g. direct quotations). Verbatim reporting allows the reader to experience the participants’ perspectives.

Theoretical Validity (Plausibility)

The degree to which theoretical explanations developed from the research study fits the data collected. Theory refers to how and why a phenomenon operates//occurs and is more abstract than descriptions. Involves using theoretical constructs to explain findings. Strategies to enhance Theoretical Validity:

∑ Extended Fieldwork /Long-term Observation- - collecting data in the field over an extended period of time. This increases researcher’s confidence in the patterns observed. More time in the field tends to increase the theoretical detail.

∑ Theory Triangulation - - the use of multiple theories and perspectives to interpret and explain data.

∑ Interdisciplinary Triangulation - - the use of other disciplines (art, sociology, history, dance, architecture, anthropology, etc.) to inform the research process and to understand the findings.

∑ Pattern Matching - - predicting a series of results that form a “pattern” and then determining the degree to which actual results fit the pattern.

∑ Negative Case Sampling - -intentionally searching for cases that do not fit your explanation so that you do not bias the data to support your theory. The final explanation should reflect the majority of the people in the study.

∑ Peer Review - - Discussion of interpretations and conclusions with other people, generally “disinterested peers” or other researchers not involved directly in the study. The peer plays the role of “devil’s advocate” challenging the researcher to provide solid evidence for interpretations and conclusions. Discussions with peers who are familiar with the research can also provide useful insights.

Internal Validity (Credibility)

The degree to which a researcher is justified in concluding that an observed relationship is causal. Qualitative research is particularly useful in determining how phenomena operate (i.e. processes) and developing preliminary causal hypotheses and theories. Strategies for enhancing Internal Validity:

∑ Researcher as Detective - - Characterizes the qualitative researcher as he/she searches for evidence of cause and effect. The researcher carefully considers potential causes and effects by systematically eliminating “rival” explanations until the “case” is made “beyond a reasonable doubt.” This may include the use of “hypothetical control groups” - - the researcher thinks about what would have happened if the causal factor had not occurred based on own and others’ expertise and published research studies. Generally, the researcher as detective makes a list of rival explanations (e.g. confounding extraneous variables) that are possible or plausible explanations for the relationship. The researcher must take the role of skeptic. Rival explanations are then checked against the data collected and/or sometimes require additional data collection.

∑ Method Triangulation - - the researcher uses more than one method of research in a single study (e.g. survey, ethnography, experimental) or uses different types of data collection procedures (e.g. interviews, questionnaires, observations, focus groups). The logic is to combine different methods that have different weaknesses and strengths.

∑ Data Triangulation - - the use of multiple data sources in a single study. This does not refer to multiple methods of data collection but the use of multiple sources using the same method of collection (e.g. interviewing multiple students, observing multiple classrooms, etc.). Data triangulation can involve collecting data at different times, in different places, or with different people. For example, if a researcher wanted to understand student apathy data triangulation might include (a) interviewing parents, teachers, students identified as apathetic, and the students’ peers) or (b) interviewing apathetic students in different class periods during the day and in different types of classes.

External Validity (Transferability)

External validity is the extent to which you can generalize from the research to other people, settings, or times. Strategies to enhance External Validity:

∑ Rich Context Description /Contextual Completeness- - researcher should provide information on the number and kinds of people in the study, how they were selected, the nature of the relationship between the researcher and participants, a description of the context, information about informants who provided information, the methods of data collection used and the data analysis techniques used.

∑ Replication Logic - - the more times a research finding is shown to be true with different sets of people or in different contexts, the more confidence one can place in generalizing. This means repeating the study with a different group, or in a different setting.

Evidence of Reliability in Qualitative Studies

Researchers’ methods are detailed so adequacy and logic can be determined

Evidence of researcher qualifications is provided

Researchers assumptions are made clear

Research questions are stated

Researchers were present in the context for an adequate period of time

Data were collected from multiple sources

Researchers saved data for re-analysis

Evidence of trustworthiness in qualitative studies

Researcher acknowledges, shows sensitivity about, and maintains an ethical stance toward the participants

Researchers’ work and analyses are fully documented, logic of data categorizations are evident, and relationships among concepts seem accurate within the identified theory

Descriptions are factual and provide evidence of minimal distortion due to omission or commission and includes cases or situations that might challenge the conclusions

Data were collected from more than one source and there is evidence confirming the accuracy of the respondents’ accounts.

Researcher is tolerant of ambiguity, has searched for alternative explanations through multiple sources of data, and has devised ways to check the data quality.

Evidence of formulating and reformulating interpretations and analyses of data including comparisons of data and checks of emerging hypotheses against new data.

Researcher is self-analytical and recognizes limits of subjectivity; shows evidence of guarding against value judgments in analysis

Results are presented in a manner such that others might be able to use them

Study is linked to a larger context

Researcher acknowledges limitations of the study as far as generalizing to other settings/contexts.

Qualitative Analysis and Reporting

Understanding Qualitative Information

Preparing to Analyze

Data analysis in qualitative studies is inductive. The researcher looks for patterns that emerge from the data. There are no pre-defined variables to focus on for analysis. The variables are identified through review of the data. There are no agreed upon approaches to analyzing qualitative data.

Analysis of qualitative data is a difficult task. Generally you have piles of field notes, piles of transcripts, piles of documents - - all of which you must sort through and determine what is important. Unlike quantitative data which are analyzed in routine ways, you must find your own idiosyncratic method of analysis. You must systematically (1) search - becoming familiar with the data and identifying main themes, (2) describe - examining the data in depth to provide detailed descriptions of the setting, participants, activities, etc. (3) classify - categorize and code data by physically grouping data into themes, (4) integrate and interpret - synthesize the organized data into general conclusions or understandings. The process is lengthy and time consuming. The process is iterative - - the researcher will go through these stages more than once. And the process tends to lose structure and become unpredictable - i.e. interpretation may lead to classifying, describing may lead to interpretation, etc.

In qualitative analysis, the researcher is the data interpreter and the quality of the interpretation depends heavily on the intellectual qualities he/she bring to the research.. The researcher must be able to integrate, synthesize, and interpret. The researcher must really “know” the data - - not just on paper.

Some authors classify qualitative analysis in this way:

Descriptive analysis - - researcher gives an account of a place or process, helping to visualize the situation as a means of understanding it.

Interpretive analysis - - the researcher explains or creates generalizations to help develop new concepts or elaborate on existing ones with a goal to provide insights.

Verification analysis - the researcher verifies assumptions, theories, and generalizations.

Evaluative analysis - the researcher provides judgments about policies, practices, and attempts to answer questions such as “How was a process implemented?”, “What was the process like?”, “How has it worked, for whom, and are there exceptions?”

Organizing the Data

Qualitative data consist primarily of verbal descriptions which need to be analyzed for themes and patterns which the researcher then describes and illustrates with examples that include quotes and excerpts. The process of data collection and analysis occurs simultaneously.

The first step is to determine a system for managing the data. Make sure the data (field notes, transcripts, observer comments, memos, reflections) are dated, organized, and sequenced. You can also organize computer files and create separate folders for different types of data and stages of analysis. Copy all data so that you can mark up or underline important parts, cross out unneeded parts, even literally cut sections out without losing your original data. Make computer backup files as well.

Computerized processing - there are some programs available for coding, searching and sorting qualitative date (Ethnograph, Nudist, etc.). Other technologies can also assist in qualitative research - e.g. word processing programs can help manage field notes, spreadsheets for tracking participants and setting characteristics, graphics packages for creating diagrams, etc.

Searching/Reading/Memoing

The first step after data are organized is to read the data to get a sense of it. This means reading everything - including comments in the margins, memos, etc. As you read, make additional notes in the margins, underline sections or issues that seem important. Document themes or common threads that reoccur. Generally you repeat this step (more than once) before moving to the next step.

Description

The purpose of this analysis component is to provide a picture of the setting, people, and events so that the reader will have an understanding of the context. This focuses on painting a verbal picture (thick, rich description) that describes the world as viewed from the participants’ perspective (emic). The description can be a chronological ordering of events, description of a “typical day”, emphasize key contextual episodes, or illuminate different participant perspectives. Description also leads to the separation and grouping of pieces of data related to different aspects of the setting, events, and participants.

Classifying/Coding Data

Qualitative analysis is a process of breaking data into smaller units, determining the import of these units, and putting them back together in an interpreted form. Breaking down the data is a process of classifying or coding. A category is a classification of ideas or concepts. Concepts in the data are examined and compared and connections are made, thus forming categories. Lower-level categories can be organized into higher and more abstract conceptual categories.

The categories one researcher uses to organize qualitative data may not be the same categories another researcher would use to organize the same data. In addition to researcher bias, interest, and style another reason for this is the difference in topics being researched. The topic of the study should have an effect on the categories chosen.

Different authors recommend different ways of defining categories. Some suggestions include: participant acts, activities, participant meanings, relationships among participants, settings, perspectives of participants, participants ways of thinking, regularly occurring activities, infrequently occurring activities, methods, group acts, individual acts, etc. Each of these categories may be further broken down into sub-categories. Relying on pre-defined categories may increase the likelihood that the researcher will miss other important categories in the data. Categories should always be provisional and the researcher should not become too attached to initial categories.

Begin qualitative analysis by coding - review transcripts and documents and identify categories as they emerge. A coding system is developed and refined to facilitate sorting and reviewing data. Then organize by category - - categories result from reading and re-reading field notes, etc. and noting regularities related to ideas, activities, setting, structures, etc. Data are sorted into the categories and resorted and re-categorized as additional data are collected and analyzed.

The researcher may end up with 100 categories which must be collapsed into a manageable number - perhaps 10 to 20. Categories are then merged into patterns. Patterns are links among categories that further integrate the data and are used as the basis for reporting. In the end, the researcher has a pyramid of data - - data pieces, categories, and patterns, with each level representing more integration and abstraction. Of course there is no guarantee that patterns can be induced from the categories.

Data Analysis

Analysis of qualitative data involves organization, classification, categorization, a search for patterns, and synthesis. Analysis is recursive. There are no guidelines for determining how much data and analysis are needed to support conclusions. Interpretation focuses on plausible support. How long should analysis go on? The answer is “it’s hard to say!” Analysis depends on the nature of the study, the amount of data, and the analytic abilities of the researcher. But typically, it takes longer than analysis for quantitative studies. There are three key strategies used for qualitative analysis: constant comparison, negative case analysis, and analytic induction.

Constant -comparison - comparing identified topics and concepts to determine distinctive characteristics so they can be placed in the appropriate categories. As a new topic or idea is identified, it is compared to the existing categories. The researcher asks “Is this topic or concept similar to or different from existing categories?” Categories are then added or modified to fit the new data then tested with additional data. This method is similar to the analytic approach except that data are collected from several cases before initial hypothesis development. Data are examined for categories, patterns, consistencies and inconsistencies. Steps in this approach include: collect data from several cases, identify important issues or recurring events and use them to create categories, collect additional data to provide more examples for each category and to elaborate on dimensions within the categories, describe how the categories account for documented events then reformulate some categories and delete others as data dictate, identify patterns and relationships, develop a theory by continuing to collect and compare data and refining categories and relationships.

Negative case and discrepant data analysis - searching for data that are negative or discrepant from the main data. Information that contradicts an emerging category or pattern is a negative case. Data that provide a variant perspective is a discrepant case. It is too easy for a researcher to adopt or cling to an initial hunch and fail to examine counter evidence. This method provides a counterbalance to the tendency to stick with first impressions.

Analytic Inductive Analysis - concerned with the development and test of a theory and is aimed at generalizability. This method is suggested if the researcher has a specific problem, question or issue in mind and wants to develop a theory or hypothesis about it. There are 7 steps : define the event of study, develop a tentative hypothesis, evaluate another case to see if it supports the hypothesis, reconsider and restate the hypothesis, evaluate cases that may disprove the hypothesis, reformulate the hypothesis based on these observations, continue until the hypothesis is confirmed.

Corroboration - Refers to the use of various methods of triangulation to support findings. Triangulation could include information from various sources, various participants, various settings, various observers, etc. Data are analyzed for corroboration of ideas or themes.

Data Interpretation

Interpreting is the reflective, integrative and explanatory part of dealing with the data. The researcher must explain the meaning of the data to others. Interpretation is based on the connections, common aspects and linkages among the data, especially the identified categories and patterns. Interpretation requires more conceptual and integrative thinking than data analysis and involves abstracting important understandings from the detailed and complex data. The focus is on what is important in the data, why is it important, and what can be learned from it. Interpretation is personal. There are no hard and fast rules for interpreting meaning from qualitative data. It is dependent upon the perspective and abilities of the researcher.

Every piece of data is not incorporated into the interpretation. The type of study will influence interpretation. For example, an ethnography will focus on cultural patterns and perspectives, a case study will focus on characteristics of a single person or group, phenomenology will focus on how individuals experience a phenomenon.

Some general strategies can help in data interpretation. First, pay attention to your research focus, it should guide the researcher in selecting portions of data for interpretation. Second, look at categories that contain large amounts of data as they are likely to identify important concepts or practices for interpretation and are likely to contain links or sequences. Third, focus on the patterns, what do the interrelationships between categories suggest? Fourth examine existing studies related to your topic to help in identifying interpretations. Fourth, talk with peers about the data and discuss interpretations. Fifth, step back from the data every now and then and take time to reflect.

Note, while analysis and interpretation of qualitative data rely heavily on the preferences and abilities of the researcher, qualitative researchers do not have carte blanche to rely strictly on feelings or preferences. If qualitative research were based solely on producing unsubstantiated opinions which ignored contradictory data and biases, it would be of little value. Six questions can help check data quality: Are data based on actual observations or hearsay? Is there corroboration by others? In what circumstances was the observation made? How reliable are those providing data? What motivations might have influenced a participant’s report? What biases might have influenced how an observation was made or reported?

Computer-assisted analysis

There are computer programs that can assist with qualitative data analysis. They tend to provide for: storing and organizing data files and text; inputting data and diagrams from word processors; creating an index system for data; searching for words, phrases, or categories; organizing data by categories or patterns; producing counts of words and phrases; constructing data diagrams; displaying information about aspects of the study. The programs, however, do not help with the interpretive process. One drawback is that once the computer organizes the data, there may be a tendency for the researcher to skip re-examination. Use of computer programs does not eliminate the researcher’s need to interpret and make decisions about both the data and what to write about the data.

Reporting Qualitative Results

Qualitative data are reported using “words” rather than numbers, although newer forms are incorporating visual representations (computer generated graphics, pictures, video, audio) as well. The goal is to provide “thick, rich description.” The reader should get a picture of the context, “see” the participants, and “hear” their voices. Reports are generally heavy with quotes.

Characteristics of the Qualitative Research Report

Qualitative reports often have features analogous to a story - - - a setting in which data are collected, characters who are informants, and a plot in the form of social action in which the characters are engaged. Keeping the story coherent and logical may mean eliminating data that are not central to the plot. General guidelines for qualitative report writing: engage the readers’ interest through description and dramatization. Trace the evolution of the account. Develop overall coherence. Select key themes. Use simple language. Make concepts and connections explicit. First person voice is acceptable.

Unlike quantitative research reporting, there is no single accepted outline for a qualitative research proposal or report. However, some components seem to appear regularly in qualitative reports.

∑ Introduction - - often the report begins with something interesting (a quote, a story) to capture the reader’s interest. The introduction introduces a question (or puzzlement) as well as describes how the researcher became interested in the topic and how the question evolved. The introduction should explain the need for the study, citing relevant literature, and explain how the study will be useful. Often this is the only literature presented outside of the literature incorporated into the findings section. In other words, qualitative reports often do not include a separate literature review section.

∑ Research paradigm - -while this section was nearly always present in qualitative studies a short time ago, there is now less need for it. Quantitative studies seldom justify their paradigm. However, there are still readers who are not familiar with qualitative research paradigms who may need the information. This section then describes the research paradigm (the epistemological and conceptual foundation for the study), describes the philosophical correlates of the paradigm (e.g. phenomenology, hermeneutics, etc.), cites authors in the area, explains the assumptions of the paradigm (e.g. the nature of knowledge and reality, task differences from conventional research, relationship of the researcher to the participants), and suggests appropriate criteria for evaluating the research that are consistent with the paradigm.

∑ Methods - - This section generally describes your research method (e.g. ethnographic field study, single case study etc.) and your procedures (e.g. interviews, observations, etc.). Often, major authors who have described these methods are cited. The researcher should describe in detail what will be done (or has been done) in the study, including how informants were selected, how entry was gained into the research setting, procedures to protect the rights of informants, the type of data to be collected (e.g. field notes, audio tapes, video tapes, existing documents, etc.), the intended data collection procedures (e.g. sample interview questions to be used, equipment needed, etc.), the procedures used to track the research process (audit trail ), intended analysis procedures (coding, sorting, data reduction, transcription process, categorization, etc.), how the design evolved over time, and how data will be organized and presented. The researcher also should explain how “reliability” and “validity” will be addressed (e.g. triangulation, member checking, peer debriefing, auditing, etc.). In addition, qualitative studies generally include in the methods a detailed description of the context of the study and descriptions of the participants.

∑ Preliminary Biases/Suppositions - - the researcher should summarize the literature reviewed to date and describe how the review has influenced the approach to the research. Previous experience with the topic (personal and professional) should be explained as well as how this has influenced the researcher’s perspectives. Anticipated hunches should be disclosed. The procedures used to control bias (help the researcher remain “open”) should also be explained (e.g. peer debriefing, multiple observers, etc.). Limitations of the study should be discussed.

∑ Presentation of Data - - there are multiple ways that qualitative researchers use to present the data. All involve the use of quotations and/or excerpts. Some choose to present the data by data collection method, each chapter presenting the results from one method (e.g. data from surveys, data from faculty interviews, data from student interviews, data from documents, etc.). Others choose to present a series of cases with each chapter presenting a unique description (e.g. “Carol”, “Tom”, Brigid” - - or - - School 1, School 2, School 3, etc.) Others present the data by key theme with each chapter representing a different theme (e.g. Structural Obstacles, The Student-Teacher Relationship, Power and Authority, etc.). Data or findings are generally integrated or related to the literature and may include interpretive statements from the researcher. This is unique to qualitative research. Quantitative research does not reference literature in the findings section (generally there is a separate literature review and connections are made in the concluding chapter). Quantitative research also does not include researcher perspectives or interpretations in the findings section. These are reported only in the concluding section. In qualitative studies these are woven throughout the findings section.

∑ Conclusion/Discussion/Interpretation - - Both qualitative and quantitative research includes final comments that attempt to “make sense” of the data. This is generally the final chapter of a dissertation or report. In qualitative studies, this may be where integrating theories are proposed. Recommendations for practice and suggestions for future research are also generally included in this section in both styles of research.

Common Criteria for Judging Qualitative Research Reports

∑ Methods are explicated in detail so reader can judge whether methods were adequate and make sense. A rationale for using qualitative methods is provided. Methods for attaining entry, managing the researcher role, data collection, recording, analysis, ethics and exit are described. There is an audit trail and description of how the site and sample were selected. Data collection and analysis are public.

∑ Assumptions are states, biases expressed, and researcher does self-analysis for personal biases and analysis of theoretical biases.

∑ Researcher guards against value judgments in data collection and analysis.

∑ There is abundant evidence from raw data to demonstrate connection between presented findings and real world and data are presented in readable, accessible form, perhaps aided by graphics, models, charts, and figures.

∑ Research questions are stated and study answers those questions.

∑ Relationship between the study and previous studies is explicit. Definitions of phenomena are provided, but research foes beyond previously established frameworks.

∑ Study is reported in a manner accessible to other researchers, practitioners, and policy makers with adequate translation of findings so others can use them.

∑ Evidence is presented showing researcher was tolerant of ambiguity, searched for alternative explanations, checked out negative instances, and used a variety of methods to check findings (triangulation).

∑ Report acknowledges limitations of generalizability, while assisting readers in seeing transferability.

∑ It is clear that there was a phase in which problem focus was generated from the field not from library research. The study is an exploration, not merely a study to find data to verify old theories.

∑ Observations are made of a full range of activities over a full cycle.

∑ Data are preserved and available for re-analysis.

∑ Methods are devised for checking data quality.

∑ In-field work analysis is documented.

∑ Meaning is elicited from cross-cultural perspectives.

∑ Researcher maintains ethical standards.

∑ Data collection strategies are the most adequate and efficient available. Evidence that the researcher fine-tuned the research instrument and recognizes when he/she is getting subjective.

∑ The study is tied into “the big picture” and provides a holistic view.

∑ The researcher traces the historical context.

“Generalizability” Concepts in Qualitative Research (Transferability)

In quantitative research, generalizability in research is described as the ability to use a statistical process and a randomly selected sample from a population to determine that whatever is true of the sample will also be true (within certain limitations of probability) of the population from which the sample was drawn. Generalizing in qualitative research may be thought of as going beyond the information given and transferring what has been learned from one situation to another (thus the qualitative term “transferability” rather than generalizability). A person must recognize the similarity from one situation to the next and then make the appropriate inference.

What generalizes in qualitative research can be regarded as skills, images, and ideas. In qualitative research, learning to see and learning to write about what is seen are generalizable skills. The ability to generate a vivid and concrete image, a portrait of the context provides a conceptual device for generalizations. Ideas that generalize help us develop theories and explanations.

People generalize through formal inference, attribute analysis, and image matching. Formal inference is what occurs in quantitative studies. Attribute analysis involves using specific attributes or characteristics that mark a particular class of objects or processes to identify their presence. Image matching focuses on the gestalt, matching a pattern.

Typically, qualitative studies do not have generalizability as a major purpose. The people and settings in qualitative research are generally not randomly selected and thus these studies are weak in population validity (generalizability from the sample to the population or generalization across populations).

Qualitative researchers are often more interested in documenting particularistic findings rather than universalistic findings - the goal is to show what is unique about a certain group of people or a certain setting or event rather than broadly apply findings. Many qualitative researchers do not believe in “universal laws” (things that apply to everyone) or “general laws (things that apply to many people). Qualitative studies are considered weak on ecological validity (generalizing across settings) and temporal validity (generalizing across times).

Some researchers argue that qualitative studies can be generalized to other people, settings, and times to the degree that they are similar to the people, settings, and times in the original study. Sometimes referred to as “naturalistic generalization” - - generalizations based on similarity.

Two features of qualitative research are important in terms of generalizing. (1) Metaphor and literary narrative can be used to convey things literal language cannot represent as well. (2) Attention to the particular can be used to say something about the general.

In quantitative studies, generalizations are made by the researcher. However, readers should understand that many generalizations are made in quantitative research from samples that were non-random. In qualitative studies, it is the readers choice to determine whether the research findings fir their particular situation. Qualitative researchers believe in recognizing the conditional qualities of life and that contexts are specific (what works at one time may not work at another, what works with one group may not work with another, what is appropriate in one setting may not be in another).

Quantitative researchers point to the limited ability of qualitative research to contribute to the accumulation of knowledge as a weakness. Since situations are unique and methods are dependent on the individual researcher, accumulation is thought to be difficult if not impossible. Qualitative researchers argue that quantitative knowledge also has problems with accumulation - - variables are not consistently defined, interpretations depend on perspectives (e.g. whether the researcher’s perspectives are Piagetian, Eriksonian, Skinnerian, etc.). Can one accumulate findings of research conducted using different theories, different core concepts, different instrumentation, different key variables? Qualitative researchers argue that knowledge accumulation is horizontal rather than vertical, ideas contribute to the development or refinement of conceptual frameworks, perspectives, or metaphors through which the world is viewed. It is not clear that Erikson’s perspectives can be integrated with Skinner’s but both may help us understand aspects of human development. In this context, qualitative researchers argue, qualitative research does contribute to knowledge accumulation. They view as mistaken the idea that there is one theory. Connections are made by the reader.