Transformative research and evaluation

Mertens, Donna M. (2009). New York, NY: Guilford.

Introduction (1-7)
Mertens earlier noted four dominant paradigms in educational and psychological research: positivism, constructivism, pragmatism and emancipatory. In this text, he renamed emancipatory to transformative to emphasize working together with people for personal and social transformation. In this context, he discussed the transformative paradigm as "an overarching metaphysical framework" (3) to cover various approaches labeled critical theory, feminist theory, participatory, inclusive, culturally responsive, etc. that aim to furthering social justice through research and evaluation. [Here Mertens mentions research and evaluation together. I wonder what is difference. Perhaps it is the use of terms. Research is often referred to activities of seeking knowledge for the sake of knowledge while evaluation as more practical, but knowledge-for the sake of knowledge is not practical in particular in the domain of the transformative paradigm. In a sense, research is part of evaluation.]

1. Resilience, resistance, and complexities that challenge (9-42)

Mertens wrote for the foci of the transformative paradigm:

"Rather, the paradigm focuses on culturally appropriate strategies to facilitate understandings that will create sustainable social change. Understanding the dynamics of power and privilege and how they can be challenged in the status quo is also a priory" (10)

In this light, she emphasized participatory approaches to research and evaluation and stressed the potential role of the researcher or evaluator as an "instrument of social change" (11).

Mertens wrote, "Participatory action research is one example of an approach that is compatible with the transformative paradigm when it is applied to the goal of social justice" (21).

Mertens discussed ethical impetus. Looking at the Belmont report from the U.S. national commission for the Protection of Human Subjects of Biomedical and Behavioral Research (1979), she wrote that the landscape of ethics is useful, and yet, issues outside of the norms and principles also arise in the conduct of research: "some ethical issues will surface differently or not at all, depending on the researchers' or evaluators' paradigmatic stance" (35)

Chilisa (2005, Educational research within postcolonial Africa. International Journal of Qualitative Studies in Education, 18, 659-684) suggests that ethics narrowly defined as protection fails to protect the researched as it fails to recognize ethics in light of respect for the integrity of the researched community.

Mertens also looked into AEA's Guiding Principles over a period of time and other codes, and suggested that revisions of professional association code recently made indicate a "greater awareness of the need to consciously incorporate principles of cultural competence as a salient dimension of their ethical codes" (36): APA, AERA, America Evaluation Association, American Sociological Association, American Anthropological Association, and the United Nations.

[Find out the examples, and look for some examples in Canadian contexts]

Mertens read through Kirkhart (1995, 2005) and Lincoln (1995) for the discussion of research validity. In particular, Kirkhart (2005) proposed "multicultural validity" to indicate "correctness or authenticity of understanding across multiple, intersecting cultural contexts" (p.22) and suggested five justifications for multicultural validity: theoretical, experiential, consequential, interpersonal, methodological.

Lincoln (1995, Standards for qualitative research. Paper presented at the annual meeting of the AERA, San Francisco) suggested the followings for the criteria for quality in research and evaluation: positionality or standpoint epitemology (researchers should acknowledge all texts are incomplete and limited by the position of the researchers), community (know well enough of the community), attention to voice (researchers should seek out those who are silent), critical reflexivity (high level of awareness to acknowledge the psychologocal state of others and themselves), reciprocity (should demonstrate that a method was used to develop a sense of trust and mutuality with the participants), sharing the perquisites of privilege. (as cited 39-40)

[Checking the quality of the embedded research (PV), it demonstrates sustained quality]

2. Basic beliefs and commensurate theories (43-69)

Some philosophical statements of the transformative paradigm:

Axiology (assumptions about ethics): "Ethical considerations include respect for cultural norms of interaction; beneficence is defined in terms of the promotion of human rights and increase in social justice)

Ontology (assumptions about what exists; what is reality): "Rejects cultural relativism and recognizes the influence of privilege in determining what is real ad the consequence of accepting one version of reality over another; multiple realities are shaped by social, political, cultural, economic, ethnic, gender, disability, and other values" (p. 49)

Epistemology (assumptions about the nature of knowledge and the relationship between the researcher/evaluator and the stakeholders needed to achieve accurate knowledge): Recongnizes an interactive link between researcher/evaluator and participants/co-researchers/evaluator; knowledge is seen as socially and historically situated; issues of power and privilege are explicitly addressed; development of a trusting relationship is seen as critical

Methodology (assumptions about appropriate methods of systematic inquiry): "Inclusion of qualitative methods (dialogical) is seen as critical; quantitative and mixed methods can be used; interactive link recognized between the researcher/evaluator and participants in the definition of the focus and questions; methods are adjusted to accommodate cultural complexity; and contextual and historical factors are acknowledged. especially as they relate to discrimination and oppression. (p. 49)

About Validity
Gaventa and Cornwall (2006, Power and knoweldge. In Reason and Bradbury (Eds.) Handbook of action research (71-82)) argued for the need to evolve new concepts of validity in participatory research and the quality of knowledge. [So, WHAT IS THEIR CONCEPT?]

Mertens stressed that researchers should be sensitive to cultural diversity because "The embedded biases of researchers and evaluators toward culturally diverse people threaten validity. In contrast, validity is enhanced by cultural responsiveness" (62). [Mertens stressed cultural responsiveness as criteria of validity]

In this regard, Mertens disregarded triangulation and called for crystallization: she wrote that triangulation was used to validate findings in traditional qualitative research through different methods. But she implied that triangulation does not convey the multifaceted nature of knowledge, subject to cultural refraction (oh, nice word that I coined), and supported crystallization indicating multidimentionalities, angles of approaches, transmutations, borrowing from Richardson and St. Pierre (2005, Writing: A method of inquiry, in Denzin & Lincoln (eds.) The sage handbook of qualitative research (3rd, 959-978)).


6. Quantitative, qualitative, and mixed methods (164-198)

Mertens suggests that specific quantitative and qualitative approaches have the potential to be used as a mixed-methods approach as long as they are used in culturally appropriate context.

In particular in the context of participatory action research, Mertens quotes Greenwood and Levine (2007, Introduction to action research, Sage), who argued that AP is not limited to group meetings and qualitative methods. They regarded action research as a research strategy that "generates knowledge claims for the express purpose of taking action to promote social analysis and democratic social change" (p. 5, cited in 183). They summarized, "Surveys, statistical analyses, interviews, focus groups, ethnographies, and life histories are all acceptable, if the reason for deploying them has been agreed upon by the AP collaborators and if they are used in a way that does not oppress the participants" (p. 6).

[GREENWOOD and LEVINE is good to quote to support the mixed method used in my study.]

Qualitative research & evaluation methods

(3rd ed.). Patton, Michael Quinn. (2002). Thousand Oaks, CA: Sage.

1. The nature of qualitative inquiry (3-29)
*Three kinds of qualitative data: interview, observation, and document

The purposes of research and evaluation are different, but research can be evaluative: "When one examines and judges accomplishments and effectiveness, one is engaged in evaluation. When this examination of effectiveness is conducted systematically and empirically through careful data collection and thoughtful analysis, one is engaged in evaluation research" (p. 10)

*Criteria to judge the quality of the findings:
evaluation standards: utility, feasibility, propriety, accuracy
nontraditional criteria: trustworthiness, diversity of perspectives, clarity of voice, credibility of the inquirer to primary users of the findings.

Patton noted, "In qualitative inquiry, the researcher is the instrument. The credibility of qualitative methods, therefore, hinges to a great extent on the skills, competence, and rigor of the person doing fieldwork--as well as things going on in a person's life that might prove a distraction" (p. 14)

Patton discussed the purposes of open-ended responses and participant observation for evaluative research: "the open-ended responses permit one to understand the world as seen by the respondents. The purpose of gathering responses to open-ended questions is to enable the researcher to understand and capture the points of view of other people without predetermining those points of view through prior selection of questionnaire categories. . . . The task for the qualitative researcher is to provide a framework within which people can respond in a way that represents accurately and thoroughly their points of view about the world..." (p. 21)

In order to overcome the limitations pertaining to what people say, Patton called for participant observation for evaluation research. "The purpose of observational analysis is to take the reader into the setting that was observed" (p. 23).

[In my evaluation research, I utilized participant observation, written out in depth in chapter 3, and made available direct quotes from the observed to provide readers with their points of view.]

2. Strategic themes in qualitative inquiry (37-73)
* Patton presented some analysis strategies: unique case orientation (capturing the details of the individual case being studied), inductive analysis and creative synthesis, holistic perspective (focusing on complex interdependence and providing it as a complex system), context sensitivity (attentive to a social, historical, and temporal context), voice/reflexivity (reflective on my own voice)

With regard to reflexivity, Patton wrote: "Reflexivity has entered the qualitative lexicon as a way of emphasizing the importance of self-awareness, political/cultural consciousness, and ownership of one's perspective" (p. 64) He suggested using the first-person, active voice in this light.

Patton seems to be skeptical about paradigmatic division. He argued that a paradigmatic strategy indeed can be a hindrance for evaluation research. He argued:

"Such prejudices and paradigmatic blinders limit methodological choices, flexibility, and creativity. Adherence to a methodological paradigm can lock researchers into unconscious patterns of perception and behavior that disguise the biased, predetermined nature of their methods "decision." " (p. 71)

Further, he argued that "the qualitative-quantitative debate oversimplified and often confused methodological and philosophical issues" (p. 71), as in reality, there are the cases that notably consistent methods and philosophy in a certain paradigm have proven problematic.

3. Variety in qualitative inquiry: Theoretical orientations (75-142)
Patton introduces 16 qualitative inquiries, including phenomenology and grounded theory, which I think are the most relevant to my dissertation research. So I will take a close look at them.

Phenomenology
Foundational question: what is the meaning, structure, and essence of the lived experience of this phenomenon for this person or group of people?

Phenomenon can be emotion, a relationship, a program, or a culture. The focus is on describing the essence of people's experience in a certain phenomenon and developing a worldview. It usually involves participant observation and in-depth interviews. For example, the essence of being a participant in a particular program can be a kind of phenomenological study. [My chapter 4 is a definitely phenomenological study in that sense.]

Grounded theory
Foundational question: What theory emerges from systematic comparative analysis and is grounded in fieldwork so as to explain what has been and is observed?

Popular in social sciences and methodology with emphasis on generating, instead of testing, theory. Patton asserts that all the approaches are grounded in the empirical world, but the conceptualizations of what to ask vary from one inquiry to another. "While the phrase "ground theory" is often used as a general reference to inductive, qualitative analysis, as an identifiable approach to qualitative inquiry, it consists of quite specific methods and systematic procedure" (p. 126). Thus, he argued:

"As a matter of philosophical distinctness, then, grounded theory is best understood as fundamentally realist and objectivist in orientation, emphasizing disciplined and procedural ways of getting the researcher's biases out of the way but adding healthy doses of creativity to the analytic process." (p. 128-129).

[To the extent that I compared the experience of the research participants with that of others in other programs in Chapter 4, it has a systematic comparison emphasized in grounded theory, and yet, this analysis is not so much grounded theory as phenomenology. On the other hand, Chapter 5 is more linked to grounded theory as I intend to provide some methodological tools to researchers. But I also doubt that it can be called grounded theory as I do not develop theory out of the fieldwork. ]

4. Particularly appropriate qualitative applications (143-205)
Patton holds pragmatism in his approach to methodology: "Pragmatic and utilitarian frameworks can guide qualitative inquiry on their practical and applied underpinnings without having to be attached to or derived from a theoretical tradition." (p. 145)

Patton suggests various evaluation models:

Goal-free evaluation
"doing fieldwork and gathering data on a broad array of actual effects or outcomes, then comparing the observed outcomes with the actual needs of program participants" (p. 169). In general, it purposefully suspends the researcher's judgment.

[Then, I can take up goal-free evaluation for the embedded study. The participants want to find out ways to help increase retention rate, in the end, they suggested a way. There is no way to check the actual effects, but from the participants' perspective, the research brought about some solutions they looked for. ]

Transaction models: responsive and illuminative evaluation

The transaction model of evaluation that Ernie House (1978) called concentrates on case studies and "places prime emphasis on perception and knowing as a transactional process between researcher and research participant" (p. 172). Its variations include illuminative evaluation, developed in educational evaluation, and Stake's (1975) responsive approach to evaluation, which requires to have "face-to-face contact with people in the program and learn about diverse stakeholder's perspectives, experiences and concerns. This responsive evaluation has been integrated into Guba and Lincoln's (1989) "responsive constructivist evaluation" known as the fourth generation evaluation, which is based on naturalistic inquiry.
Connoisseurship studies
It places the evaluator's perceptions and expertise at the center of the evaluation process, "drawing heavily on his or her own judgments about what constitutes excellence" (172).

Interactive and participatory applications:

Placing a greater emphasis on human interaction in designing evaluation strategies, Patton wrote:

Qualitative methods may also be perceived as more personal because of their inductive nature. This means that, again, rather than imposing on people or a program some predetermined model or hypotheses, the results unfold in a way that takes into account idiosyncrasies, uniqueness, and complex dynamics. Finally, qualitative methods may be perceived as more humanistic and personal simply by avoiding numbers. (p. 176)

Action research, action learning, reflective practice
i.e. adult educators interviewing students as case studies, then through cross-case analysis they deepened their sensitivity to new immigrant students and developed their professional skills.
Participatory research and evaluation
Film: Awakening (1990)--a real story of participatory research in a mental hospital

Patton provided an overview of a growing popularity of participatory research:
"Interest in participatory research has exploded in recent years, especially as an element of larger community change efforts (Stoecker 1999). The principal researcher trains the coresearchers to observe, interview, reflect, and/or keep careful records or diaries. Those involved come together periodically to share in the data analysis process. The purpose of such shared inquiry is typically to elucidate and improve the nature of practice in some arena of action" (183).

In particular, participatory evaluation emphasizes "working with people" (183), thus, Patton wrote: "The process is facilitated by the researcher, but is controlled by the people in the program or community. They undertake a formal, reflective process for their own development and empowerment" (183).

Patton noted that participatory evaluation has been used with great success by a number of NGOs and private voluntary organization in the Third World.

Patton also drew attention to various connotations of participatory inquiry, saying that there is no definitive definition for participatory evaluation. Patton, in particular, emphasized teaching inquiry logic and skills as the purpose of participatory evaluation, as "the process of participatory self-evaluation, in and of itself, provided useful learning experience for participants" (184).

[Here Patton suggests participatory evaluation/inquiry as a pedagogical form.]

Finally, he noted, "these participatory approaches often employ qualitative methods because those methods are understandable, teachable, and usable by people without extensive research training." (185)

[Patton regards participatory paradigm from the perspective of pragmatism and links qualitative methods as strong candidates for that paradigmatic inquiry.]

8. Qualitative analysis and interpretation (431-534)

Patton discussed about summative vs. formative evaluation: Summative evaluations are usually accompanied by a full report with data, analysis, and recommendations, whereas formative evaluation is conducted for program improvement and may not generate a written report. Findings are often presented orally to the staff or funders. Patton wrote:
"The methods are qualitative, the purpose is practical, and the analysis is done throughout fieldwork; no written report is expected beyond a final outline of observations and implications. Academic theory takes second place to understanding the program's theory of action as actually practices and implemented" (435)

[The VT research can be viewed as formative evaluation to give insights into the OP program, which is not academic theory oriented and rather concentrates on providing critical insights from the insiders' perspective.]

Theory-based analysis approaches: Patton discusses phenomenological analysis and ground theory analysis in great detail, along with qualitative comparative analysis and analytic induction.

Analytic induction seems to be the most relevant to my approach. It originally emphasized discovering universal causal generalizations, but gradually moved away to qualitative inquiry and comparative case analysis including examination of preconceived hypotheses. In this approach, "researchers actively seek to disconfirm emerging hypotheses through negative case analysis, that is, analysis of cases that hold promise for disconfirming emerging hypotheses and that add variability to the sample. In this way, the originators of the method sought to examine enough cases to assure the development of universal hypothesis." (Jane Gilgun 1995:268-269, We shared something special: the moral discourse of incest perpetrators, Journal of marriage and the family, 57, 265-81). Analytic induction consists of re-examinations.

9. Enhancing the quality and credibility of qualitative analysis (541-597)

In traditional scientific research, among the criteria for judging the credibility are objectivity, validity, systematic rigor of fieldwork procedures, triangulation, etc. The common aim is "to use qualitative methods to describe and explain phenomena as accurately and completely as possible so that their descriptions and explanations correspond as closely as possible to the way the world is and actually operates" (p. 546)

Social construction and constructivist criteria: Lincoln and Guba (1986) proposed credibility (for internal validity), transferability, dependability and confirmability. In combination, they viewed these criteria as addressing trustworthiness (in lieu of rigor). Fundamentally, they are "suspicious of causal explanations and empirical generalizations applied to complex human interactions and cultural system" (546)

Critical change criteria: another stream of qualitative inquiry aims at social and political change. Thus consequential validity matters in addressing the credibility of the inquiry.

[For the analysis of my dissertation study, what criteria should be applied? The set of criteria should stem from the hypotheses of participatory (video) research. What does participatory research claim to be able to do? Does my study prove or disprove it? How do I make judgment? --My judgment should be grounded in constructivist perspective, the fourth generation evaluation. Then, what I need to verify is the claims of participatory research. ]

Visual Methods in Social Research

Reviewed by Mark Turin

BANKS, MARCUS. Visual methods in social research. xvi, 201 pp., tables, illus., bibliogr. London, Delhi: Sage Publications, 2001.

In this helpful introductory text, Banks offers advice to social scientists interested in using visual imagery in their work, and discusses the benefits of doing so. The author's approach is neither dogmatic nor evangelical, combining instead quiet confidence with cautionary good sense. The result is a very solid book which will be particularly useful for students in sociology and anthropology hoping to engage with issues of visual representation.

Visual methods in social research covers empirical approaches to image creation as well as image analysis. Drawing on wide-ranging examples, from holiday postcards to vintage film, Banks illustrates how methodology relating to visual form in the social sciences is '[either] scattered or confined to quite specific areas' (p. 2). The author makes effective use of rhetorical questions and discrete subsections (such as 'unnatural vision' and 'reading narratives'), thereby encouraging the reader to reconsider often-unquestioned assumptions about visual culture. Banks is well versed both topically and technologically, and jumps nimbly from caste in India to the Kayapo of Brazil, and from Polaroid to webcams. Moreover, by choosing explicitly contemporary reference points, such as Jorg Haider or Quentin Tarantino's now-cult film, Pulp fiction, he succeeds in engaging younger readers on their own visual territory.

A number of sections are particularly strong and worthy of special attention. Discussing the presence of foreign objects and non-indigenous bodies in ethnographic films, Banks points out that in future generations, the 'sight of "imported" artefacts would not be taken as mournful tokens of lost innocence, but celebrated as cultural appropriations' (p. 144). About the aptly named HADDON web-based meta-catalogue of archival ethnographic film and footage, the author's own creation, he is characteristically modest. The existence of this online catalogue and of a supporting website for the book under discussion, , embodies the hands-on approach which Banks advocates. While largely photo-centric, Visual methods also offers useful pointers for working with computer-based digital media, one advantage of which is the potential for 'research parity between sound, text and image' (p. 162). Increasingly, as academic publishing becomes a not-for-profit enterprise and publishers attempt to curtail their production costs, a well-conceived website of video clips and images can provide access to a supporting corpus of rich visual data which can help to illustrate a written text. Focusing on the Yanomamo interactive CD-ROM. Banks demonstrates how the use of multimedia 'opens up a non-linear space within which the detail can be absorbed at the user's own pace, and the arguments which rest upon that detail can be fully explored' (p. 165). This is the power of bits and bytes: the social researcher can 'usefully employ multimedia to state her own case but also to outline the alternative interpretations and provide access to the raw materials to allow the user to test them all' (p. 164). Banks concludes his otherwise cautious narrative with an unexpected, albeit entirely justified, sting in the tail. Having offered the disclaimer that his book contains little explicit theory, he takes a well-aimed swipe at scholars who advocate nothing other than 'swooping god-like into other people's lives and gathering "data" ... according to a predetermined theoretical agenda [which] strikes me not simply as morally dubious but intellectually flawed' (p. 179).

Students would be well served to read this book alongside James Monaco's now-classic How to read a film (currently in its third edition). In tandem, these background handbooks are amply sufficient to prepare researchers in social science with enough practical know-how, together with a healthy dose of reflexive understanding, to ensure the responsible use of visual imagery in their work. Banks is at his best when he stresses the material nature of visual media and the cultural nature of vision (bringing to mind John Berger's work), and in demonstrating how visual objects are entangled in social relations. In his introduction, Banks suggests that 'good visual research rests upon a judicious reading of both internal and external narratives' (p. 12). His own approach--critical yet mindful--does just this.

An alternative paradigm dialog

Egon C. Guba (1990). In E.G. Guba (Ed.), The paradigm dialog (pp. 17-30). Newbury Park, CA: Sage.

In this chapter, Guba proposed as the generic definition of paradigm "a basic set of belief that guides action" (p. 17), in particular, disciplined inquiry as the action in our concern. Such paradigms respond to three questions: ontological, epistemological, and methodological questions.

1) Ontological: What is the nature of reality?
2) Epistemological: What is the nature of the relationship between the knower (researcher) and the known (researched)?
3) Methodological: what is the process of finding out knowledge or research?

Positivism
The positivist paradigm is rooted in a realist ontology, the belief that "there exists a reality out there driven by immutable natural laws" (p. 19). The purpose of science is to discover the true nature of reality and how it truly works.

From this ground, positivists practice an objectivist epistemology, in which they put questions directly to nature and wait for its answer without altering it. Since the inquirer can be positioned far away from nature, research outcomes are value free. Within this paradigm, experimental/manipulative methodology is the most appropriate in such a way that questions and hypotheses are stated in advance and subjected to empirical tests under carefully controlled conditions.

Postpositivism
It is a modified/less naïve version of positivism. It is based on critical realism: postpositivists believe that reality exists out there, yet because of human sensory/intellectual limitations, it is impossible for humans truly to perceive it. Thus, they emphasize the need for being critical about what researchers find.
Epistemologically, postpositivists recognize the absurdity of the separation of inquirers from the inquired; thus they take on a modified objectivist epistemology in the sense that objectivity is a regulatory ideal to adhere, yet in reality, can be achieved reasonably closely (approximated) only by cleaning up one's preposition, relying on the tradition of the field to report the inquiry, and seeking judgment from peers in the disciplinary community. Guba criticized that the latter two requirements virtually make is impossible for new paradigm to assert themselves.
Methodologically, it put forth critical multiplism (modified experimentalism/manipulatism), which was also articulated as triangulation by Denzin (1978). Because of limited human sensory, it seeks as many sources as possible to draw findings. They often calls for more qualitative methods, depending more on grounded theory, and reintroducing discovery into the inquiry process.

Critical Theory
Guba proposed the label of "ideologically oriented inquiry" instead of critical theory, which encompasses feminism, neo-Marxism, Frerieism, participatory inquiry, and critical theory. The shared quality in this family is rejecting the claim of "value freedom" made by positivism (and postpositivism to an extent). They believe that values do enter into every inquiry; inquiry thereby becomes a political act.

Guba criticizes an ontological juncture that critical theorists (ideologists) adhere to:

But, for whatever reason, critical theorists (ideologists) have elected to believe in an objective reality--as the phrase commonly used by them, "false consciousness," readily demonstrates (because it implies that there is a "true consciousness" somewhere "out there," or, more likely, possessed by the inquirer or some better informed elite). The task of inquiry is, by definition, to raise people (the oppressed) to a level of "true consciousness." Once they appreciate how oppressed they are, they can act to transform the world. The close parallel between transforming the world and predicting and controlling it should not be lost. (p. 24)

Critical theorists undertake subjectivist epistemology, regarding inquiry acts as a process intimately related to the inquirer's values. Yet, Guba points out a logical disjunction a "realist ontology coupled with a subjectivist epistemology" (p. 24).

Guba asserted that critical theorists' methodology is more consistent and framed it as transformative and dialogic one. It seeks to eliminate false consciousness and to facilitate transformation.

Constructivism
Constructivist calls for a completely different paradigm. It acknowledge value-laden facts and furthermore, the interaction of inquirer and inquired into that shapes outcomes of an inquiry. Therefore, constructivists state that:

It [constructivism] renders the distinction between ontology and epistemology obsolete; what can be known and the individual who comes to know it are fused into a coherent whole. Further, it makes the findings of an inquiry not a report of what is "out there" but the residue of a process that literally creates them. Finally, it depicts knowledge as the outcome or consequence of human activity; knowledge is a human construction, never certifiable as ultimately true but problematic and ever changing. (p. 26)

From this perspective, ontologically, it takes a position of relativism in that "realities are multiple, and they exist in people's minds" (p. 26). Guba argued that relativism is "the key to openness and the continuing search for ever more informed and sophisticated constructions" (p. 26).

Epistemologically, the constructivist takes a subjectivist position, not because of the limited human conditions but because of its ontological assumptions: if realities exist only in people's minds, subjectivist interaction is the only way to accessing them.

Methodologically, the constructivist takes hermeneutics and dialectics: it attempts to depict individual constructions as accurately as possible, and simultaneously, contrast and compare dialectically to generate informed constructions on which there is substantial consensus.

Knowing poverty

:Critical reflections on participatory research and policy. Brock, Karen & McGee, Rosemary (Eds.). (2002). London: Earthscan Publications.

Introduction (Karen Brock, 1-13)
Brock observed that PR as a poverty assessment method has been accepted in dominant institution, yet methodological credibility (validity), representation, and applicability have been in debate. Alongside, PR has been criticized within. Brock focused on the process of "decontextualizing local people's knowledge"(p. 9), which questions which voices are included and excluded as local knowledge. Pointing out Chambers's chapter, Brock also noted the dilemmas and choices a researcher faces in analyzing data. Drawing on methodological, epistemological, and ethical dilemmas, Chambers argued for the necessity of reflexivity: "the process of reflecting critically on the part one plays as a researcher in the formation, framing and construction of knowledge" (p. 9).

Brock's epistemological stance is interesting to note: Knowledge in PR is inseparable from the concept of action. She wrote:

"In participatory research, knowledge is not produced for its own sake, but is seen as being embedded in processes of social change or problem solving, inseparable from the idea of action." (p. 9)

Brock sharply noted that PR is a "process of reflection, learning, and action, producing the development of a critical consciousness" (p. 10). [This model of cycle is different from that of action research--plan, action, observe...--It is more internal and fundamental, happening at the level of consciousness. Thus, I do think that the discourse of action research is distinguished from that of participatory research. Here, action is not immediate, more about influencing policy.]

Brock introduces other topics: how policy-makers learn and change? What is a local perspective on consciousness? What are the policy implications of participatory research?

Power, knowledge and policy influence: Reflections on an experience (Robert Chambers, 133-165)

In his experience, Chambers drew attention to power and knowledge. He stated that power distorts realities and frames the realities of others. In particular, he wrote:

"Researchers had power to determine how open or closed the process was. Knowledge was formed iteratively through interactions of researchers' and participants' priorities, concepts, frames, realities and patterns of analysis and representation. Academic values and practices in part conflicted with the ethics of policy influence. (p. 136)

Because of this very reason, Chambers argued for researchers' reflexivity.

The action research dissertation

: A guide for students and faculty. Herr, Kathryn & Anderson, Gary L. (2005). Thousand Oaks, London and Delhi: Sage.

1. Introduction (1-7)

The authors note that action research, despite its similarities with QUAL (even with QUAN), is different in that "research participants themselves are either in control of the research or are participants in the design and methodology of the research" (p. 1) and action is central to research. They stated as the most important feature the shifting locus of control from professional researchers to research subjects. They noted several terms indicating research in collaboration with practitioners or community members: PAR, community-based PAR, participatory rural appraisal, feminist action research, emancipatory praxis... The authors choose action research as an umbrella term of these.

The authors defined action research as (1) inquiry done "by or with insiders to an organization or community" (p. 3) (2) action-oriented, (3) a reflective process, meaning "deliberately and systematically undertaken and generally requires that some form of evidence be presented to support assertions."(p. 3). Interestingly, the authors argued that "what constitutes evidence or, in more traditional terms, data is still being debated" (p. 3). (4) It is value laden. The authors rightly pointed out that although action research hopes for solving pressing problems or improving situations, what constitutes improvement or a solution is not self-evident. That reflects conflicting values and an unequal distribution of power and resources. Here they stress the importance of reflexivity: "[T]he notion of reflexivity is crucial because action researchers must interrogate received notions of improvement or solutions in terms of who ultimately benefits from the actions undertaken"(p. 4)

The authors observe a researcher's double burden: action (improvement of practice or social change) and research (valid knowledge production), which sets up a conflict between the rigor and the relevance of the research. Similarly, action research demands a form of intervention which is not honored by traditional researchers. The authors pointed out such interventions constitutes a spiral of action cycles--plan-act-observe-reflect (kemmis 1982, p.7)

The authors also observe that "there is more writing about action research than documentation of actual research studies" (p. 6) They suggest that "a dissertation may represent the documentation of a successful collaboration and be used as a case study of not only the process but also the product of the collaboration" (p. 6).
2. Action research traditions and knowledge interests (8-28)

The authors noted two bands of action research: business-oriented administration and emancipatory practice linked to anticapitalist, grassroots movement. They asserted that "action research is working with an oppressed community ti identify what Freire(1970) called generative themes, or issues that the community agreed had highest priority" (p. 9) Participatoy action research (PAR) emphasizes the aspect of this internal/participatory process from insiders. The authors noted, "Participatory action research (PAR) tempers this expert knowledge with the expertise of locals about their own problems and solutions" (p. 10).

The authors defined participatory research by and large in the legacy of Freire (1970 Pedagogy of the Oppressed). He viewed research as a form of social action in which generative themes are identified and used as a basis of literacy instruction through collaboration. Focusing on Freirean-inspired participatory research, Brown and Tandon(1983) argued that traditional action research concentrated on an individual or group level of analysis, whereas emancipatory participatory research tends to focus on a broader societal analysis. Such PR often creates a resistance to the research.

On the other hand, coining the term co-operative inquiry, Heron (1996) undertook experiential approaches to action research, emphasizing intersubjectivity among researchers and participants: "[T]here is the possibility of reciprocal participative knowing" (p. 11)

The authors remarked that both the trends (liberation or humanistic psychology) are concerned with using PR for humanization.

The authors pointed out the danger of co-optation in doing action research. They argued that the nature of social institutions is keeping its status quo, thus, if action research lacks a critical spirit, it runs the risk of legimitizing the current status quo.

They noted emerging critical approaches to action research from feminist and critical race theorists to a lesser extent.

Epistemologically, the authors trace back to Jurgen Habermas (1971, Knowledge and Human Interests), who argued knowledge production is never neutral and always pursued with some interest. He argued three types of interest: technical interest (controlling over the natural and social realm), practical interest (gaining understanding through interpretation), and emancipatory interest (the release of human potential and the investigation of ideology and power relation). Depending on types of interest, the authors argue that research aims vary. They see action research in tune with emancipatory knowledge interest, which aims at critical reflection on power constraint imposed on our understanding. However, they stated:

"[W]hile certain methodologies tend to be associated with certain knowledge interest, we don't see a conflict in using statistical methods toward emancipatory ends, as long as it forms part of a cycle of ongoing critical reflection" (p. 28).

[Here, again, the authors place action (participatory) research as a paradigm distinguished from qualitative or qualitative methodology, while slightly emphasizing their position in the emancipatory stream. HOWEVER, when participants utilize or emulate certain methods, which are considered dehumanizing, would it be considered emancipatory? The positionality of participants as insiders would make a difference, while outside researchers conduct surveys, for instance...But I am not quite sure right now.]

3. The continuum of positionality in action research (29-48)

The authors explore different implications of positionality between insiders and outsiders. Here PAR is defined as either reciprocal collaboration or outsider(s) collaborating with insider(s). They assert that although reciprocal collaboration is ideal, many different positinalities are possible and valuable as long as researchers are aware of them and honest about them. The issue of positionalities is also important to the authors because it affects the criteria for quality or trustworthiness of the study they conduct.

4. Quality criteria for action research (49-68)
The authors reject validity (QUAN) and trustworthiness (QUAL/naturalistic researchers) as criteria for action research because the terms do not acknowledge action-oriented outcomes. Instead, they propose five validity criteria which are associated with the goals of action research.

Goal1: the generation of new knowledge-->dialogic and process validity
Goal2: the achievement of action-oriented outcomes-->outcome validity
Goal3: the education of both researcher and participants-->catalytic validity
Goal4: Results that are relevant tot eh local setting-->Democratic validity
Goal5: A sound and appropriate research methodology-->process validity

Process validity: the manner in which research is conducted, including triangulation
Outcome validity: whether action occurs [What is action? In many emancipatory participatory research, action may be intrinsic and they may lack agency to implement the research results but their research has implications of policy change. Thus, actions that may be qualified in more conventional action research may not be appropriate here.]
Catalytic validity: does research lead its participants and researchers to know the reality and transform it? This validity has to do with transformative potential.
Democratic validity: is the research done in collaboration with all parties? In particular, it draws on locality (Cunningham, 1983): does problems emerge from a local context and the solutions are appropriate to that context?
Dialogic validity: peer review.

[I WONDER whether it is appropriate to apply these criteria to both business-oriented and emancipatory action research.]

Reason and Bradbury (2001, Handbook of action research) defined action research as "an emerging, evolutionary and educational process of engaging with self, persons and communities that needs to be sustained for a significant period of time" (p. 12, as cited p. 58).

In short, the authors depart from current QUAN and QUAL validity criteria for action research.

About GENERALIZATION.
The authors take on Robert Stake's (1986) position on naturalistic generalization to illustrate their own. He explained naturalistic generalizations as a way of leading audiences to vicarious experience and helping them think of their own problems in a new way:

A person would be described as an individual, with uniqueness not just in deviance scores, but as a key to understanding the interactions. A reader senses the experience of teamwork in this particular situation. It is a unique situation in some respects, but ordinary in other respects. Readers recognize similarities with situations of their own. Perhaps they are stimulated to think of old problems in a new way (p.99)

In writing dissertation, the authors emphasize that students clarify the evolution, documenting the decisions made. They acknowledge that in naturalistic inquiry, methodologies evolve, yet it should be documented. Secondly, they stress that the students think through their multiple roles in this research as researcher, as outsider, as facilitator, etc.


////
Brown & Tandon
doi: 10.1177/002188638301900306 Journal of Applied Behavioral Science September 1983 vol. 19 no. 3 277-294

Heron, J. (1996). Co-operative inquiry: Research into the human condition. London: Sage.

Cunningham, J. B. (1983). Gathering data in a changing organization. Human relations, 36 (5), 403-420

Stake, R. (1986). An evolutionary view of educational improvement. In E.R. House (Ed.), New directions in educational evaluation (pp. 89-102). London: Falmer.

The Practice of Qualitative Research

Sharlene Nagy Hesse-Biber & Patricia Leavy. (2006). Sage Publications.

Chapter 4. In-Depth Interview (pp. 119-145)

[I chose this chapter to discuss among the VTW research group members. It is written in a plain yet engaging manner with some examples to illustrate the authors' points.]

The authors introduce the significance of in-depth or intensive interview a common method for qualitative research. In comparison with quantitative interview, they argue, qualitative interviews, which are often involved with in-depth interview, should be "knowledge-producing conversation" (p. 128) based upon reciprocal relationships between interviewees and interviewers as well as a sense of shared authority, which regards interviewees as experts.

Similarly, defining the qualitative interview as "co-creation of meaning" (p. 134), the authors call for reflexivity among researchers. In their term:

"The heart of the qualitative interview requires much reflexivity, that is, sensitivity to the important 'situational' dynamics between the researcher and researched that can impact the creation of knowledge" (p. 135)

In the discussion of dealing with difference between researchers and researched, the authors talk about the notions of insiders and outsiders. They argue that one's status as insider/outsider is fluid and prone to change in the course of research. In general, they agree, insiders can bring much in-depth data, yet state the advantage of being an outsider:

"By not belong to a specific group, you may be viewed as more unbiased by your respondent. Likewise, you may be more likely to ask things that you would otherwise take for granted as "shared knowledge" and in fact learn that your participation have their own way of viewing a given issue." (p. 140)

The authors suggest to be mindful of the importance of difference between researchers and researched.

In the discussion of analysis and interpretation of interview data, the authors emphasize the importance of negative cases that do not fit the overall problems, as they are often the most informative. As a way of triangulating data, they suggest to try hard to disprove a researcher's presumed ideas.

The Methodological Dilemma

Gallagher, Kathleen. (Ed.). (2008). London and New York: Routledge.

Introduction (pp. 1-8)
Gallagher takes a constructionist perspective, writing, "Through our research and the choices we make, we imagine the world in ever more complex and interesting ways and re-commit ourselves to our intellectual and political projects. But we also risk our comfortable norms and truth claims each time we seek understanding" (pp. 1-2).

In the same vein, she argues that "researchers can mobilize theory [sic], in creative ways, to think through the 'pragmatics' of method" (p. 2). Thus she calls for unfettering our imagination for creative methods.

The Art of Methodology: A Collaborative Science (pp 67-81)

Gallagher proposes participatory research methods as a dialectical exchange between science and art. She argues that such exchange is indispensable in social sciences and humanities research, insinuating a potential for research that is politically engaged as well as pedagogically useful.

Interdisciplinary agendas in visual research: re-situating visual anthropology

Pink, Sarah. (2003). Visual Studies, 18 (2), 179-192.

In her review of recent literature around visual research methods, which owe much to visual anthropology, Pink notes as common interests among visual researcher reflexivity, collaboration, ethics, and the relationship between the content, social context and materiality of images.
(p. 9)

Looking through the history of visual anthropology and contemporary visual culture studies, Pink concludes that across disciplines, critical approaches to the interpretation of images has departed from positivist truth-seeking.

Pink refers to reflexivity as a key concern among visual researchers, yet calls for clarification of distinct claims to and uses of reflexivity. Calling reflexivity "a kind of autobiography" (p. 130), Rose (2001, Visual Methodologies) writes certain discourses refuse to be reflexive. Foucauldian discourse analysis is one of them, for it analyzes power subconsciously embedded in texts. Pink agrees with Rose to the extent that these methodologies cannot be critical visual ones because they lack reflexivity.

For Pink, reflexivity in ethnography means a researcher's awareness of "how they look and understand when doing ethnography as well as when they look at images" (p. 188).

Pink further traces approaches to reflexivity in research: Whereas visual anthropologists such as Ruby regards reflexivity as an ethical issue, Emmison and Smith (2000, Researching the Visual) interpret reflexivity as a question of validity. For MacDougall, another visual anthropologist, a 'deep' reflexivity requires revealing "the position of the author in the very construction of the work" (1998:89, Transcultural Cinema; cited on the page 188). Critiquing Holliday (2000), who refered to reflexivity as "a mere buzz word" (p. 505), Pink writes Holliday's argument lack historical understanding of issues and debates around visual anthropology in the past couple of decades.

In sum up, Pink writes:

"Across the different approaches, reflexivity has commonly been coined as a need for understanding ‘‘where the researcher is coming from’’ and how this impacts on the knowledge
produced. Some leave this at a question of validity and research quality control. However, most visual anthropologists take a quite different tack. They argue that reflexivity should be integrated fully into processes of fieldwork and visual or written representation in ways that do not simply explain the researcher’s approach but reveal the very processes by which the positionality of researcher and informant were constituted and through which knowledge was produced during the fieldwork" (p. 189)

On "Reflexivity" in Qualitative Research: Two Readings, and a Third

Macbeth, Douglas. (2001). Qualitative Inquiry, 7 (1), 35-68.

According to Macbeth, reflexivity in qualitative research is a "deconstructive exercise for locating
the intersections of author, other, text, and world, and for penetrating the representational
exercise itself" (p. 35).

He finds contemporary expressions of reflexivity in cultural and radical relativism, which problematizes meaning and values as well as representation. Thus, he conceives relativism and reflexivity, intertwined in the crisis of objectivism and totalizing discourses, as a feature of a postmodern condition.

Macbeth categorizes reflexivity into three domains: positional, textual, and constitutive.

By positional reflexivity, he means self-referential analytic exercise that attempts to deconstruct the duality of power and antipower, hegemony and resistance, and insider and outsider. Macbeth notes that positional reflexivity becomes the very methods of qualitative methodology in particular in educational research, thus it "becomes an obliged topic, if not method", for qualitative researchers.

Macbeth finds the root of textual reflexivity in anthropology that threw skepticism on the correspondence between representation (ethnographic reports) and object (view and beliefs). In Macbeth’s treatment, textual reflexivity is a mode of writing that disrupts both the realist assumption of representation in text and the text itself by various devices and experiments. [Jay Ruby delves into this kind of reflexivity]

Lastly, Macbeth looks for reflexivity in an ethnomethodological sense. Harold Garfinkel, in his seminal work "Studies in Ethnomethodology" (1967), defined the term ethnomethodology as "the investigation of the rational properties of indexical expressions and other practical actions as contingent ongoing accomplishments of organized artful practices of everyday life" (p. 11). In other words, ethnomethodology is a study of ways in which people make sense of their everyday activities. Ethnomethodological reflexivity is an ordinary activity that constitutes "world making" (p. 49) and points to "the organization of ordinary sense and meaning...reflexively made of the social technologies for producing and detecting them" (p. 49). Macbeth states that for EM, reflexivity is far from professional or methodological senses. Rather it is the organization of routine grounds of everyday life.

For instance, in reviewing a videotape in a classroom, Macbeth recognizes that students become analysts in their choices in class and understanding of the sequential organizations through reflexive activities. They make sense of their environment and cohort it through reflexivity. Here reflexivity is inherent in people's action. Thus Macbeth writes about the reflexivity in EM mode: "A constitutive reflexivity thus has no use for nonreflexive worlds and makes no sense as a demarcation exercise" (p. 55).

Qualitative Researching

Mason, Jennifer. (1996). Qualitative Researching. London, Thousand Oaks, and New Delhi: Sage.



Generating Qualitative Data: Interviewing (35-59) [research method training for VT Summer Research Group].

Generating Qualitative Data: Observation, Documents and Visual Data (60-82)

Doing Qualitative Research Differently

: Free Association, Narrative and the Interview Method. Hollway, Wendy, & Jefferson, Tony. (2000). London, Thousand Oak, and New Delhi: Sage.

Introduction: the Need to do Research Differently (pp.1-6)

The authors question the veracity of accounts that a research subject tell during interview. For them, one of the reasons for believing is democratic and related to power differentials between researcher and researched. Feminists, according to the author, have been conscious of this issue of power differentials and advocated the principles of giving voice to voiceless women. However, the authors claim that researchers cannot give voice to others, quoting a statement by Reissman (1993, Narrative Analysis, p. 3):
"we cannot give voice" since we "do not have direct access to another's experience. We deal with ambiguous representations of it--talk, text, interaction and interpretation"

[This argument is not logical. Giving voice does not mean directly accessing someone's experience.]

From the standpoint of critical realism (Bunge, 1993; Watkins, 1994-5), the authors argue that researchers interpret their subject's experience only through ambiguous representations of it, yet there is a relationship between people's representations and their experiences. Here the emphasis is given to that the relationship, tracked only by the researcher, depends on the researcher's particular view.

Producing Data with defended subjects (26-54)
From their pilot studies, the authors delineate four distinct approaches to interview: traditional, feminist, narrative, and clinical case-study. While traditional approaches focus on interview technique, feminists are conscious of power balance between interviewer and interviewee. In narrative approaches, an interviewee becomes a story-teller, thus the interview agenda is open to development, depending on the interviewee's experiences. Clinical case-study emphasizes a researcher's reflexivity on their own involvement in the interview process, such as anxiety, personal experience, etc.

The authors propose biographical-interpretative method. It emulates a biographer's role to listen to someone's story and make a coherent account out of the story-telling, Gestalt, explaining, "a whole which is more than the sum of its parts, an order or hidden agenda" (p. 34). The authors propose four principles of this method: (1) the more open the better, (2) elicit stories (i.e. Tell me about a time when you feel unsafe.), (3) avoid why questions to obtain concrete meanings from the interviewee. Why questions usually induce empty, cliché-ridden discussion, the authors assert. (4) Follow up using respondents' ordering and phrasing by taking a note.

The authors emphasize free association that comes unconsciously during interview.

More visualizing. more methodologies

: on video, reflexivity and qualitative research. Sarah Pink. The Sociological Review. 2001. 586-599.

Pink regards reflexivity as a key theme in the development of visual anthropology and explores the potential of video for reflexive approach to qualitative research in sociology and anthropology. She cites that in some recent work, video methods and visual methods in general has been situated as part of a reflexive methodology. For instance, for Ruby a reflexive approach to visual anthropology is ethical such that the researcher, as the producer of visual materials, reveals the constructedness of his or her argument by showing his or her own particular way of generating and processing the data.

Pink expands her idea of reflexivity to giving agency to the subject as see from Michaels' study on the impact of television on Aboriginal communities. Similarly, the project where mothers with HIV recorded their messages on video for their children demonstrates a reflexive approach: The producers, mothers, project their own images through negotiation of self-image and performance. The images produced as a result bring spontaneous and vivid information to the researchers. Another example of a reflexive approach to video in the production of ethnographic knowledge is mentioned in Ferrándiz's work, where he recorded video in collaboration with his informants. Through the video work, Ferrández not only learned ritual practices performed by his informants but also the way in which a camera create a visual truth by catalyzing certain aspects of the rituals. In short, such reflexive approaches deepened our understanding of the subject under study.

Picturing Culture: Exploration of Film and Anthropology

Ruby, Jay. (2000). Chicago: The University of Chicago Press.

Exposing Yourself: Reflexivity, Anthropology and Film (pp. 151-180).

Ruby states that reflexivity is a multifaceted concept that has been used in many ways for different purposes and to be reflexive in a work of anthropology is "to insist that anthropologists systematically and rigorously reveal their methods and themselves as the instrument of data generation and reflect upon how the medium through which they transmit their work predisposes readers/viewers to construct the meaning of the work in certain ways" (p. 152). Thus, to him, reflexivity means primarily revealing producer and process, whereby is explored strictly from an anthrologist's point of view.

While exploring reflexivity in anthropological cinema, Ruby points out Chronicle of a Summer (Jean Rouch, 1961) and A Man with a Movie Camera (Vertov) as examples of anthropological cinema in that Chronicle combined personal and participatory concerns with revealing the process, which was an emphasis of A Man with a Movie Camera.

Reflexivity in Ruby's own term is a anthropologist's research practice framed in a new paradigm that explores knowledge production process. This perspective becomes explicit in his experience with Hubert Smith, a documentary filmmaker who attempted to make a reflexive documentary about Mayan culture. Ruby insisted on depicting the director's conversations with the anthropologist involved in the project as well as meetings with the consulting panel list.

Criticizing both art and science in that art (in this case, film) is perceived as an interpretation of reality and science as a mirror of reality, Ruby asks for a new theory of pictorial communication suitable for a visual anthrology. (This new theory could involve the researched's participation...I think.)

Ruby (Picturing Culture, p. 186): “Postmodern critics like George Marcus and James Clifford wish to add another goal—to make viewers aware of the constructed and tentative nature of anthropological knowledge. This admonition logically leads us into a discussion of reflexivity…Martinez (1992) has argued that these two goals should be linked. If a film is reflexively open, less authoritative, and multivocal, viewers may be more able to overcome their ethnocentric tendencies and gain some empathetic feelings for the people portrayed in the film.” –In this argument, an anthropoligist’s reflexivity reflected in an ethnographic film should achieve an ultimate goal of anthropology, which is overcoming ethnocentric tendencies among audiences and gaining empathy for people in the people.

Concerning cooperatively produced and subject-generated films, Ruby writes, “From an anthropological perspective, they offer the possibility of perceiving the world from the viewpoint of people who offer the possibility of perceiving the world from the viewpoint of people who lead lives that are different from those traditionally in control of the means of imaging the world. Subject-generated films are a tool used by some disenfranchised people in their efforts to negotiate a new cultural identity. For other indigenous and minority producers, among movies and television is a way into the profits and power of the established order. These films challenge some basic assumptions about the authorship of ethnographic films and potentially offer insight into the importance of culture in the construction of a film. This shift in authority has the potential to permanently alter ethnographic film practice” (p.196)

As examples of this practice, Ruby points out Robert Flaherty’s Nanook of the North and the cine-train project of Alexander Medvedkin, Dziga Vertov and others who produced films about local problems with the assistance of local political powers in the twenties and the early thirties in the Soviet Union. Another example mentioned by Ruby is the British film Housing Problems (1935, Edgar Anstey and Arthur Elton), in which people speak for themselves on the screen. However, Ruby states that cooperative and subject-generated films were generally ignored until the 1960s.
As a limitation of cooperative filmmaking, Ruby argues that while a multivocal approach to the documentary empower subjects, the filmmaker still holds the ethical and intellectual responsibility for the film for reasons—lack of subjects’ editorial authorship and lack of an understanding of potential problems, etc.

As Ruby points out, a shared authority in cooperative, collaborative and community-based films brings the subject’s view of themselves to the surface and alienates the journalistic objectivity based on positivism. However, he takes a note from Sol Worth, who disputed the Challenge for Change program while writing an unpublished version of a review of You Are on Indian Land. Worth (1972, A Review of You Are on Indian Land, Draft of a review later published in a revised form in the American Anthropologist, 74 (4), 1029-31) writes, “Teaching others to make and use film was essentially teaching white middle-class values of how information films brought about gradual and democratic change. You Are on Indian Land is a perfect example of a professional white liberal film made in “consultation” with Indians.”—I AM NOT SURE AT THIS POINT ABOUT WHO MADE THE DOCUMENTARY.

Ruby asserts that sharing authority in filmmaking is difficult because collaboration requires the participants to have certain technical, intellectual and cultural parity. Thus he raises the question: if the participants are knowledgeable enough, why would they need outsiders to collaborate?

Doing educational research

Kenneth Tobin & Joe Kincheloe (Eds.) (2006). Rotterdam, The Netherlands: Sense Publishers.

Qualitative Research in Classrooms (Tobin, pp.15-58)
Criteria for judging quality of ethnography include whether the research involving human subjects benefit the subjects and the researchers ensure that the subjects know that the benefits will be realized only in the future and improved once the research is enacted (p.25). In interpretive research, it is important to ask two main questions, what's going on and why is it happening, as soon as returning from the field. While formulating answers, the methodology takes researchers to search for patterns of coherence and contradictions as a whole--because culture consists of thin coherence dialectically interconnected contradictions (cited on p. 30; Sewell, 1999). Those initial findings are described in vignettes, little stories, then accumulate multiple data sources, beyond interviews, to support the patterns and contradictions. (He recorded video, audio, and photographs.) According to Tobin, intellectual efforts lie in connecting the theoretical framework that underpins a study to the empirical and theoretical works undertaken in the study (p.54). To produce a high quality research, in the planning stage, set a boundary and reexamine what has been done has all relevant meanings to the research once the study is unfolded.

Preparing to do Educational Research (Joyce & Tutela, pp. 59-83)
Each author is introduced within the context of their family and educational background. Being women who tend to be more attentive to what others say (cited on p.64; Gilligan, 1994), they choose a dialogue to uncover their voice in educational research. Dialogues are also a vehicle for understanding the world from their own experience and reality while staying conscious of the world (cited on p 65; Freire 1990). [The beginning part of this article is rather provocative in that the authors, the researchers, read themselves as the researched. In other words, they situate themselves in a research context and lead to the dialogue, which constitutes the main content of this article. Clever!] In the dialogue, Joyce's graduate study is mainly traced. She brought in journals that she kept while working in a high school for five years, which led her to study minority underachievement. While taking courses, she grounded up theoretical approaches and kept annotation in various ways, such as indexing, sticky notes, questioning texts and so forth. The rest of the questions include general study strategies and dissertation theories and methodologies she took.

Critical Cultural Studies Research (Steinberg, pp.117-137)
For critical cultural studies through reading popular media, Steinberg introduces various disciplines she employs for bricolage research. Levi-Strauss (1966) defines it as a form of taking research strategies from various disciplines and traditions to unfold research in context (p.119): Critical ethnography, based on the poststructualism, proposes dialogues between researcher and researched with the intention of breaking the hierarchial relation embedded in traditional ethnography between them (Atkinson & Hammersley, 94; cited on p. 121); Poststructuralist feminism challenges tradional patriarchal exclusions and connects class problems with female oppression; Critical poststructuralist theory emphasizes applying a theory to a local phenomenon to "guarantee individuals and community the capacity to make meaning and act independently."; Critical hermeneutics is concerned about the interrelations between identity/subjectivity/consciousness construction and cultural production.
hermeneutics is back to original difficulty and “an appreciation of the complexity and ambiguity of human life” (p.129)


Philosophical and Historical Research (Thayer-Bacon & Moyer, pp.139-156)
Library of Congress American Memory Collection (http://memory.loc.gov/ammem)
Harvard University's OASIS (Online Archival Search Information System)
ArchivesUSA
US National Archives (http://www.archives.gov/)
Archives Canada (http://www.archivescanada.ca/)

Participatory Activist Research/Action Research (Martin & McLaren, pp.157-190)
Action Research for social change and social justice develops Freire's notion of praxis--dialogic relationship between practice and theory--and is based on shared participation for problem solving process and awareness of power structure within the context. Unlike traditional research about education, which intends to conform individuals to the existing social structure and values--in Foucault's term, domiciled body through institutional disciplinary practices--critical traditions emphasize the relationship between power and knowledge. According to McLaren, the production and transmission of knowledge is a mechanism of marginalizing the knowledge of the Other (cited on p.177). The authors note that the characteristics of this research are that it is participatory and collaborative, focusing on change, based on self-critical communities, embodying praxis, aiming group theorizing, and exlicitly political, which is distinctive from all research (p.179). Still limitation exists as the authors phrase 'blind or manipulative power dressed up as emancipation or empowerment.' Issues and problems are frames by researchers initially and research moves along with the priviledged position of the researcher.

Historical Research in Education (Villaverde, Kincheloe, Helyar, p.311-345)
Historiography is by definition the study of history, historical methods and its writing and employs a meta-analysis of the writing of the past by using various methods such as deconstruction, discourse analysis, textual analysis, hermeneutics, semiotics and ideological contextualization) for interpretation (p.346). Villaverde writes that while philosophy guides to how to frame a problem and theorize the solutions, methodology leads researcher to actions in the study, and asserts critical interpretation, meta-analysis and asking unique questions as the integral methodologies in historiography. Kincheloe emphasizes critical theory in historiography as an aparatus of gaining new lenses for new interpretation of educational phenomena. Thus, groudning in social theory, he says critical historians should ask questions about power relations: who are advantaged and disadvantaged by particular educational plans? By addressing constructivism of historiography, he contends, "understanding the ways that the construction of a historical narrative is in part a ctrative act, a feat of the imagination does not take away from the usefulness of historical scholarshop. Indeed, it provides us with a more accurate picture of the historiographical process and how history may be either distorted or used in a socially beneficent manner"(p. 322). I also note his further insight into the intensified power of 'capital-driven global colonialism': this global structures 'co-opt' education insiduously to maintain its regulation on people around the world and research should explore this phenomenon in particular locales within the context of such intrusion and resistance. Finally he stresses a postformalistic notion of simultaneous multiperspectivism, where multiple perspectives, even contradictorily, are presented simultaneously. According to him, such efforts constitute a form of historiographical cubism and widen a horizen of research.