Saturday, October 2, 2010

Knowledge Shared: Participatory Evaluation in Development Cooperation

Jackson, Edward T., & Kassam, Yusuf. (1998). West Hartford, CT: Kumarian.

The authors define participatory evaluation as a strategic too for development projects. They stated that the essence of participatory evaluation lies in shared knowledge between local community and specialists (or professionals). Through shared knowledge, they argued that (1) participatory evaluation better serves the interests of both the local beneficiaries and development agencies; (2) it democratizes and enriches the assessment of development; (3) it enhances the benefits for the stakeholders.

Historically examining various participatory evaluation projects, the authors proposed the definition of participatory evaluation:

"Participatory evaluation is a process of self-assessment, collective knowledge production, and cooperative action in which the stakeholders in a development intervention participate substantively in the identification of the evaluation issues, the design of the evaluation, the collection and analysis of data, and the action taken as a result of the evaluation findings. . . . This approach evaluation employs a wide range of data collection and analysis techniques, both qualitative and quantitative, involving fieldwork, workshops, and movement building. " (p. 3)

With respect to fourth generation evaluation that Guba and Lincoln proposed, the authors argued that this approach replaced the concept of participation with that of stakeholder interaction; it separates the participation from the rhetoric concern of "empowerment" in the 1970s and 1980s, thus making the participatory evaluation methodologically more achievable. Yet, the authors argued that in participatory evaluation, all the stakeholders are not equal and thus, there should be a bias in favor of the least powerful in the process of participatory evaluation. This perspective makes the authors argument distinct from other collaborative forms of assessment. [Here, participatory evaluation is written in the context of macro development projects, namely to criticize co-opted participatory evaluation as a tool kit, thus the direct comparison with fourth generation evaluation does not sound feasible.]

Among the issues related to participatory evaluation, the authors empathized the importance of facilitation skills to make a safe space in which every stakeholder can freely speak.

Friday, October 1, 2010

Fourth generation evaluation

Egon Guba & Yvona Lincoln. (1989). Newbury Park, London, and New Delhi: Sage.

Forward (7-19)
The authors criticized that pervasively positivistic approaches to evaluation treat evaluation as a scientific process that misses its social, political, value-oriented nature. Instead, they called for a constructivist approach, arguing that evaluation outcomes "represent meaningful constructions that individual actors or groups of actors form to 'make sense' of the situations in which they find themselves" (p. 8). Evaluation, therefore, is a process of negotiation. They called this evaluation paradigm fourth generation evaluation, empathizing the nature of construction and negotiation among stakeholders. In this light, they noted that such evaluation should be action-oriented. Here, the role of evaluator is defined as the orchestrator of the negotiation process:

"If there is to be a course of action with which most stakeholders can agree, it can only be arrived at through negotiation [sic] . . . If that goal is to be achieved, the evaluator must play a larger role by far than simply that of technician-gathering-information; instead he or she must be the orchestrator of the negotiation process, which in the final analysis is the guts of the evaluation." (p. 10)

The authors also pointed out that the ethics involved in research should go beyond informed consent, harm, and confidentiality. To make research more ethical, they argued that it should allow full participative involvement among its stakeholders and others in the evaluation process such that they partake in the research as equal partners in the aspects of design, implementation, interpretation, and evaluation. [Here, the authors are arguing for participatory research and empathized that they should participate in the evaluation process.]

Besides constructivist methodology, they discuss "responsive focusing" as one of the two elements of fourth generation evaluation. Developing the concept of responsive evaluation by Robert Stake, they argued that the participants of the evaluation, stakeholders, "have the right to place their claims, concerns, and issues on the table for consideration (response), irrespective of the value system to which they adhere" (p. 12). With these claims, concerns, and issues, responsive focusing is progressed to resolve them.

The Coming of Age of Evaluation (21-49)

Looking back the history of evaluation for more than a century, the authors defined the first generation evaluation as measurement; the second as description; the third as judgment. Here the evaluator assumes the role of judge while retaining the technical and descriptive functions as well. One of the authors criticisms on the first three generations is "overcommitment to the scientific paradigm of inquiry"(p. 35). Against this background, the authors develop their evaluation generation based on the concept of responsive evaluation, first proposed by Stake (1975, Evaluating the arts of education, Columbus, OH: Merrill): It "determines parameters and boundaries through an interactive, negotiated process that involves stakeholders and that consumes a considerable portion of the time and resources available." Thus, constructivism is a natural methodology for them to pursue this new emerging paradigm of evaluation.

Similar to the research methodology that the authors argued for, they proposed the following criteria for judging the adequacy of fourth generation evaluation:

A. Trustworthiness
(in lieu of validity, reliability and objectivity that stem from the positivist paradigm). Instead, they put forward credibility, dependability, and confirmability.
1) Credibility focuses on "establishing the match between the constructed realities of respondents (of stakeholders) and those realities as represented by the evaluator and attributed to various stakeholders" (p. 237). Here, the objective neutral reality is refuted. Some of the techniques to enhance credibility are prolonged engagement, persistent observation, peer debriefing, negative case analysis, progressive subjectivity ("monitoring the evaluator's own privilege, for instance, by recording his or her developing construction. In other words, watching out what the evaluator discovers more than what they were looking for. Here the authors write, "If the inquirer 'finds' only what he or she expected to find, initially, or seems to become 'stuck' or 'frozen' on some intermediate construction, credibility suffers.), and member checking.

2) Dependability (in lieu of reliability)
The authors assert that reliability is concerned with the stability of the data over time, but argued that methodological changes and shifts in construction inevitably result in different research products, thus important is not to bring about stable data but to make such changes and shifts trackable ("publicly inspectable" -p. 242). This enables an inquiry audit. The authors assert that this audit should be accompanied by the concern of confirmability.

3) Confirmability (in lieu of objectivity)
Whereas objectivity in the conventional paradigm is rooted in method, confirmability is rooted in the data themselves, argued the authors: "This means that data (constructions, assertaions, facts, and so on) can be tracked to their sources, and that the logic used to assemble the interpretations into structurally coherent and corroborating wholes is both explicit and implicit in the narrative of a case study" (p. 243)

B. The hermeneutic process of its own quality control
The authors empathized the hermeneutic and dialectic process for quality control. A simple way of doing is to analyze any type of feedback immediately on receipt and incorporate into the emerging joint as the process continues. In so doing, the authors argued that the possibility of the evaluator's prejudice can shape the evaluation results is virtually zero as long as their own construction is "laid on the table along with all the others and made to withstand the same barrage of challenge, criticism, and counterexample as any others..."(p. 244).

C. The authenticity criteria
1) Ontological authenticity: Through the provision of various experience, the authors argued, the evaluation process should enable the respondents to improve their consciousness of the world.
2) Educative authenticity: It represents " the extent to which individual respondents' understanding of and appreciation for the constructions of others [sic] outside their stakeholding group are enhanced" (p. 248)
3) Catalytic authenticity: It is related to the purpose of evaluation. It asks whether the evaluation triggers some forms of action or decision making. The authors argued that evaluation should take into consideration the call for "getting theory into action" (p. 249). This process has to do with the ownership of the participants. They put it:

"When action is jointed negotiated, it should follow that action is "owned" by participants and, therefore, as the research has shown, more willingly carried out." (p. 250)

4) Tactical authenticity: This concept compensates certain conditions that prohibit such actions. It refers to "the degree to which stakeholders and participants are empowered to act" in the process of evaluation (p. 250) How can it be demonstrated? The authors suggested soliciting testimony, following up and rendering judgment. They suggested asking the following questions: "Was it participatory? [Here fourth genetaion evaluation is the very same concept with participatory evaluation] Have all stakeholders felt that they or their representatives have had a siginificant role in the process? Are all participants more skilled than previously in understanding and utilizing power and negotiation techniques? If the answers to those questions are uniformed yes, then tactical authenticity has probably been achieved" (p. 250)

Tuesday, September 28, 2010

Radical adult education and learning

Griff Foley. (2001). International Journal of Lifelong Education, 20 (1/2), 71-88.

"Radical adult educators are those who work for emancipatory social change and whose work engages with the learning dimension of social life." (p. 72)

Foley acknowledged the democratic imperatives of Carl Rogers's approach to humanistic education, in which meaningful experiential learning was emphasized; yet, Rogers, unlike critical and radical educators, ignored social analysis and political commitment. More, Foley finds the root of the confusion in defining power in Rogers's approach to non-directive teaching. In Foley's definition, empowerment in education takes place when educators use their power to create educational situation in which learners can exercise power. For him, it is neither something that educators can do for the learners nor something that can be obtained by abandoning the educator's power. Foley suggests the educator's active role in empowering the learners. In this regard, he put it:

This will involve educators in recognizing the invasiveness of their work and struggling with learners to build a different sort of relationship, one that is based on a notion of solidarity rather than on a patronizing notion of service. (p. 75)

In an triumphal capitalism, Foley argued, radical adult education needs to be more strategic. One of the strategies is the action research method. He said that it is "ordinary yet radical... because it uncovers, and acts on, what is generally hidden and untouched. (pp. 82-83)

"We learn as we act, and our learning is both tacit and explicit" (p. 86). Foley emphasizes the understanding of the complex learning process that takes place in places where we work and live. [How is this statement related to the participatory research?]

Power in practice

: adult education and the struggle for knowledge and power in society. Ronald Cervero & Arther Wilson (Eds.). (2001). San Francisco, CA: Jossey-Bass.

Preface (xv-xxi)
The authors pointed out that the literature of adult education lacked an examination of power relations manifested in adult education, although adults became more engaged in learning activity and the issues of knowledge and power are pervasive. Within this context, they ask, "who benefits from and who should benefit from the programs, practices, and policies of adult education?" (p. xv)

The authors empathized that adult education is concerned not only about knowledge distribution but also social, cultural, and economic power, which they called 'material effects' (p. 2). Holding this position, they reviewed three different conceptions of the political in adult education practice:
1) "The political is personal" (articulated by Freire's partner, Knowles's andragogue, Mezirow's emancipatory educator, Daloz's mentor, Belenky's mid-wife, and Brookfield's skillful teacher): the authors critiqued this learner-centered view romanticizes learners. In this view, they argued, "the highest professional and moral principle for adult educators, then, to involve learners in identifying their needs" (p. 5). They further described:

"Key metaphors are those of building on the experience of these potential adult learners; hearing their "voice", which has often been silenced; or giving them access to learning opportunities that historically have been denied" (p. 5)

The authors argued that lack of social structural reference in this view only addressed universal purposes and failed to approach the political in the context of adult education.

2) "The political is practical": This view is concerned primarily about getting things done from the perspective of educators. Thu, the authors argued, this perspective "lacks an analysis of adult education's relationship to the wider systems of power in society, this view of politics does not usually have an explicit social vision of whose interests should be served by adult education" (pp. 7-8).

3) "The political is structural": This view is not concerned about generic adult learners but learners who are oppressed by socially structured power. As a pitfall of this approach, the authors pointed out that it failed to provide concrete direction for micro practice. There exists a structural framework, but it lacks "applied, middle-range work" (Apple, 1988, p, 200, "Teachers and text", as cited p. 10). [My research, by probing this approach at work through participatory research practice, will contribute to suggesting such applied knowledge to address power relations in the context of adult education.]

Within these strands of the political, the authors finally proposed three propositions:

1) There is a reciprocal relationship between power and adult education: They argued, "power relationships both provide the grounds for action in adult education and are also always acted on by adult education" (p. 11). In other words, the power relations structuring our lives are played out in adult education; at the same time, adult education contributes to redistributing the power.

2) Adult education is a site of struggle for knowledge and power:

[Foley (1993, The neighborhood house: site of struggle, site of learning, British Journal of Sociology of Education, 14(1), 21-37, p. 20) wrote, "A major problem, though, with much supposedly critical adult education theory is that it is often very abstract; it has generally failed to develop analyses of adult learning in particular social contexts" (p. 20).]

[A thought: my research allows looking at adult learning at a micro level.]

The authors empathized that the struggle for knowledge and power define the politics of adult education and stated that this standpoint recognizes multiple interests at stake in adult education activity and requires negotiations of these conflicts, which leads to the question,"who benefit?"

3) All adult educators are social activists in practice (regardless of his or her particular perspective on society). In this light, the authors refuted adult educators as neutral knowledge facilitator, but knowledge-power brokers, who constantly question who (should) benefit from this education.

/// epilogue

The authors took the stance that adult education is a critical site for the struggle of knowledge and power, yet noted the limit of the current research after compiling the empirical studies:

"We share, however, a continuing concern that our actual practices in distributing knowledge, reshaping structures, and changing who benefits remain idiosyncratic and undeveloped" (pp. 268-69)

[Can this limitation overcome through participatory research as a form of adult education?]

Sunday, September 26, 2010

The art of case study research

Robert E. Stake. (1995). Thousand Oaks, London and Delhi: Sage.

Stake divided case study into intrinsic and instrumental: intrinsic case study is carried out when the case holds unique intrinsic interest and researchers often do not have a choice for the case. Instrumental case study, on the other hand, is often performed to obtain a general understanding of a phenomenon or relationships within it beyond the particular case. Stake, however, acknowledge that case study is not opt for generalization, yet empathized that it allows refinement of understanding, labeling this "petite generalization" as opposed to "grand generalization" (p. 7). Thus, he argued, "The real business of case study is particularization, not generalization. . . . There is emphasis on uniqueness, . . . understanding the case itself" (p. 8).

Stake stated that the case itself matters to intrinsic case study whereas an issue matters more to instrumental case study. Stating issues appears to me stating a central research question by Creswell. Stake wrote that the issue can be stated as simple as "instructional role of the principal", but urged it to be more descriptive, perhaps in a causal way, such as "Is the accountability of the central office diminish the primary role of a principal in a local school?"

This issue statement leads to research design and conceptualization. He, then, suggested to come up with information and evaluative questions.

As id Clifford Geertz (1973) emphasized "thick description" in his ethnographic research, Stake stressed the importance of a researcher's interpretation as an integral part of qualitative research. Similar to thick description or multiple realities, he called experiential learning "natualistic generalization" (Sake & Trumbull, 1982). In the context of research, it implies a researcher's role, which is "not veridical representation so much as stimulation of further reflection, optimizing readers' opportunity to learn" (p. 42). As opposed to explicated generalizations, Stake's definition of naturalistic generalization stems from information recipients' perspective:

Naturalistic generalizations are conclusions arrived at through personal engagement in life's affairs or by vicarious experience so well constructed that the person feels as if it happened to themselves. (p. 85)

The importance of naturalistic generalization, Stake stated, is that it is embedded in the experience of the reader. [this process of generalization is empathetic such that readers come to understand the described issue by empathetical feeling. Then, does this mean that naturalistic generalization may exclude some readers who cannot have a similar background or experience?]

In order to help readers make naturalistic generalization, Stake argued, researchers need to provide them with "vicarious experience, " empathizing time, space, and person in the researchers' account. Specifically,

Our accounts need to be personal, describing the things of our sensory experiences, not failing to attend to the matters that personal curiosity dictates. A narrative account, a story, a chronological presentation, personalistic description, emphasis on time and place provide rich ingredients for vicatious experience. (pp. 86-87)

Subjectivity is not seen as a pitfall of qualitative research by Stake. Although qualitative researchers have routines for triangulation to validate their observations, Stake noted, such routines are used often to approximate those in quantitative research, yet there are no definite protocols to set subjective misunderstanding for a test.

Stake defined the roles of case researchers--as teacher, advocate, evaluator, biographer, interpreter, and theorist who uses "the uniqueness of the case to illustrate ways in which each case is likely to be complex, not what cases have in common" (p. 105).

Triangulation (107-120)
The protocols identified by Denzin in his book, The Research Act, Stake summarized data source triangulation, investigator triangulation, theory triangulation, and methodological triangulation. In addition to the triangulation, Stake empathized member checking, research subjects' feedback on research report drafts. With this respect, Stake (p. 116) wrote:

In long use of member checking, I typically get little back from the actor--not very satisfying byt entirely necessary. I often do not have all my facts straight and I need help. I impose upon the actors' time to do some of my work. The most frequent response of the actors to whom I have sent drafts is not to acknowledge that I have sent anything. Often, the account is routine, apparently not deserving a response. But sometimes I get a thorough reading, a mutually respectful argument, and suggestions for improvement. I think I can say that all my reports have been improved by member checking.

Writing the report (121-131)
Stake refuted story-telling as a form of a case study report. He also argued that traditional research report--statement of the problem, literature review, design, data gathering, analysis and conclusion is ill-fitting for a case study report, either. One thing he suggested here is using a vignette to serve the issue (problem) and assertion. (Read Harry Wolcott's (1990) Writing up qualitative research)

Synthesizing Gramsci and Freire: possibilities for a theory of radical adult education

Peter Mayo. (1994). Int'l journal of lifelong education, 13(2), 125-48.

Mayo proposed a foundation of a radical theory of adult education grounded in the concepts and practices of Gramsci and Freire. Here he suggested:

1) "[R]adical adult education initiatives should be rooted in a commitment to confronting oppression" (p. 140) and it should aim at emancipation from oppression.
2) Radical adult education should recognize the importance of cultural activity as a site of struggle by problematizing the taken-for-granted reality and exposing the underlying ideologies.
3) Radical adult education should be seen in the context of social movement against different forms of oppression across gender, class, race, and ethnicity.
4) The aspect of culture is important to radical adult education. Mayo argued that for Freire, "the dominant language be taught by radical adult educators in a problematizing manner, whereas Gramsci thought "subordinated people should learn the dominant language in order to survive in the power struggle" (p. 143) [This interpretation is not so convincing.] Anyhow, Mayo argued that radical adult educators should critically incorporate the dominant culture in order to empower the learners.

As a limitation of their perspectives, Mayo pointed out (1) the failure to take account of the complexity of the nature of oppression. In particular, Freire has been criticized by feminist scholars for his lack of sustained analysis of race, gender and sexuality. In particular, Weiler (1991, Freire and a feminist pedagogy of difference. Harvard Educational Review, Vol. 61, No. 4,
pp. 449-474.) criticized the simplicity of the dichotomy between oppressor and oppressed overlooked the multiplicity of subjectivities involved in learning. Thus, Mayo suggested, radical adult education should include feminist and anti-racist education.

Lastly Mayo pointed out global capitalism, in which business climate takes over democracy (Ross and Trachte's (1990), cited 144). According to him, it can devastate adult education by limiting public funds to social programs. Given that situation, he argued that the locus of power should be given to international forces beyond the dynamic of national-popular and called for international alliance.

As Coben rightly pointed out, Mayo does not clearly state the purpose of the education in this essay. He talked about emancipation from oppression, but doesn't talk about whether the oppression originate and which end we go at the end... Here, emancipation is no more than a romanticized idealistic myth.

Saturday, September 25, 2010

Demystifying literacy: reading, writing and the struggle for liberation

By: James, Michael D., Convergence, Vol. 23, Issue 1, 14-25.

James argued that the prevailing tendency in literacy programs was remedial and they aimed at adaptation in spite of the illusory notion of empowering people. Unlike the prevailing assumption, he stressed that "knowledge itself is not power. Power occurs only in material relationships" (p. 18) and "Literacy by itself does not and cannot change oppressive conditions". He argued "Literacy is a part of liberation only when it is consciously linked with the process of "rupturing colonial domination, recovering democratic life, or revolutionary triumph."(p. 24)

This article makes me think of the OP. It does not declare "adaptation" as its goal, but "liberation". Yet, does it envision material relationship or "an instrument of struggle" (p. 24)?Doesn't it enlarge the mystification that freeing mind would free one from material constraints?

Then, how this educational approach would be contrasted with my approach to education through participatory research?