Showing posts with label empowerment. Show all posts
Showing posts with label empowerment. Show all posts

Friday, October 1, 2010

Fourth generation evaluation

Egon Guba & Yvona Lincoln. (1989). Newbury Park, London, and New Delhi: Sage.

Forward (7-19)
The authors criticized that pervasively positivistic approaches to evaluation treat evaluation as a scientific process that misses its social, political, value-oriented nature. Instead, they called for a constructivist approach, arguing that evaluation outcomes "represent meaningful constructions that individual actors or groups of actors form to 'make sense' of the situations in which they find themselves" (p. 8). Evaluation, therefore, is a process of negotiation. They called this evaluation paradigm fourth generation evaluation, empathizing the nature of construction and negotiation among stakeholders. In this light, they noted that such evaluation should be action-oriented. Here, the role of evaluator is defined as the orchestrator of the negotiation process:

"If there is to be a course of action with which most stakeholders can agree, it can only be arrived at through negotiation [sic] . . . If that goal is to be achieved, the evaluator must play a larger role by far than simply that of technician-gathering-information; instead he or she must be the orchestrator of the negotiation process, which in the final analysis is the guts of the evaluation." (p. 10)

The authors also pointed out that the ethics involved in research should go beyond informed consent, harm, and confidentiality. To make research more ethical, they argued that it should allow full participative involvement among its stakeholders and others in the evaluation process such that they partake in the research as equal partners in the aspects of design, implementation, interpretation, and evaluation. [Here, the authors are arguing for participatory research and empathized that they should participate in the evaluation process.]

Besides constructivist methodology, they discuss "responsive focusing" as one of the two elements of fourth generation evaluation. Developing the concept of responsive evaluation by Robert Stake, they argued that the participants of the evaluation, stakeholders, "have the right to place their claims, concerns, and issues on the table for consideration (response), irrespective of the value system to which they adhere" (p. 12). With these claims, concerns, and issues, responsive focusing is progressed to resolve them.

The Coming of Age of Evaluation (21-49)

Looking back the history of evaluation for more than a century, the authors defined the first generation evaluation as measurement; the second as description; the third as judgment. Here the evaluator assumes the role of judge while retaining the technical and descriptive functions as well. One of the authors criticisms on the first three generations is "overcommitment to the scientific paradigm of inquiry"(p. 35). Against this background, the authors develop their evaluation generation based on the concept of responsive evaluation, first proposed by Stake (1975, Evaluating the arts of education, Columbus, OH: Merrill): It "determines parameters and boundaries through an interactive, negotiated process that involves stakeholders and that consumes a considerable portion of the time and resources available." Thus, constructivism is a natural methodology for them to pursue this new emerging paradigm of evaluation.

Similar to the research methodology that the authors argued for, they proposed the following criteria for judging the adequacy of fourth generation evaluation:

A. Trustworthiness
(in lieu of validity, reliability and objectivity that stem from the positivist paradigm). Instead, they put forward credibility, dependability, and confirmability.
1) Credibility focuses on "establishing the match between the constructed realities of respondents (of stakeholders) and those realities as represented by the evaluator and attributed to various stakeholders" (p. 237). Here, the objective neutral reality is refuted. Some of the techniques to enhance credibility are prolonged engagement, persistent observation, peer debriefing, negative case analysis, progressive subjectivity ("monitoring the evaluator's own privilege, for instance, by recording his or her developing construction. In other words, watching out what the evaluator discovers more than what they were looking for. Here the authors write, "If the inquirer 'finds' only what he or she expected to find, initially, or seems to become 'stuck' or 'frozen' on some intermediate construction, credibility suffers.), and member checking.

2) Dependability (in lieu of reliability)
The authors assert that reliability is concerned with the stability of the data over time, but argued that methodological changes and shifts in construction inevitably result in different research products, thus important is not to bring about stable data but to make such changes and shifts trackable ("publicly inspectable" -p. 242). This enables an inquiry audit. The authors assert that this audit should be accompanied by the concern of confirmability.

3) Confirmability (in lieu of objectivity)
Whereas objectivity in the conventional paradigm is rooted in method, confirmability is rooted in the data themselves, argued the authors: "This means that data (constructions, assertaions, facts, and so on) can be tracked to their sources, and that the logic used to assemble the interpretations into structurally coherent and corroborating wholes is both explicit and implicit in the narrative of a case study" (p. 243)

B. The hermeneutic process of its own quality control
The authors empathized the hermeneutic and dialectic process for quality control. A simple way of doing is to analyze any type of feedback immediately on receipt and incorporate into the emerging joint as the process continues. In so doing, the authors argued that the possibility of the evaluator's prejudice can shape the evaluation results is virtually zero as long as their own construction is "laid on the table along with all the others and made to withstand the same barrage of challenge, criticism, and counterexample as any others..."(p. 244).

C. The authenticity criteria
1) Ontological authenticity: Through the provision of various experience, the authors argued, the evaluation process should enable the respondents to improve their consciousness of the world.
2) Educative authenticity: It represents " the extent to which individual respondents' understanding of and appreciation for the constructions of others [sic] outside their stakeholding group are enhanced" (p. 248)
3) Catalytic authenticity: It is related to the purpose of evaluation. It asks whether the evaluation triggers some forms of action or decision making. The authors argued that evaluation should take into consideration the call for "getting theory into action" (p. 249). This process has to do with the ownership of the participants. They put it:

"When action is jointed negotiated, it should follow that action is "owned" by participants and, therefore, as the research has shown, more willingly carried out." (p. 250)

4) Tactical authenticity: This concept compensates certain conditions that prohibit such actions. It refers to "the degree to which stakeholders and participants are empowered to act" in the process of evaluation (p. 250) How can it be demonstrated? The authors suggested soliciting testimony, following up and rendering judgment. They suggested asking the following questions: "Was it participatory? [Here fourth genetaion evaluation is the very same concept with participatory evaluation] Have all stakeholders felt that they or their representatives have had a siginificant role in the process? Are all participants more skilled than previously in understanding and utilizing power and negotiation techniques? If the answers to those questions are uniformed yes, then tactical authenticity has probably been achieved" (p. 250)

Sunday, November 8, 2009

Action and Knowledge

: Breaking the monopoly with participatory action-research. Orlando Fals-Borda and Mohammad Anisur Rahman (Eds.) (1991). NY: The Apex Press.



Some basic ingredients (O. Fals-Borda, 3-12)

Fals-Borda (and Rahman) does not distinguish participatory research (PR) and PAR strictly and rather specify the action component to emphasize action-research grounding in participation and participatory research tied with action.

Fals-Borda finds practical uses of PAR in a research method, adult education and sociopolitical action in that PAR encompasses the three aspects as an experiential methodology. He explains that experiential methodology is a process of personal and collective behavior and implies knowledge acquisition upon for power construction and social movements for the oppressed.

Fals-Borda lays out as the philosophical ground experience, which allows one to apprehend the essence intuitively, feel and understand reality, and thus place oneself in a wider context. He states that the experience, which contributes to understanding knowledge for change, stems from two different strands: one is Cartesian and academic and the other is experiential and practical. He proposes practical commitment, a form of praxis, as the only resolution to the dialectical tension between the two knowledge strands. He thereby writes,

"Therefore academic knowledge combined with popular knowledge and wisdom may result in total scientific knowledge of a revolutionary nature which destroys the previous unjust class monopoly."(p.4)

His approach to epistemology is grounded in a revolution and change and calls for commitment and praxis. In this regard, he argues that such commitment and praxis leads to a rejection of the asymmetry of the subject/object relationship that characterizes academic research and daily life.
Participatory theory, on the other hand, demands the destruction of the asymmetry and the transformation to the subject/subject relationship through participation, Fals-Borda writes. For him, the essence of participation is voluntary separation from the submission and dependence implicit in the subject/object relationship through experience. He adds that the authentic participation is resplendent with feelings and attitudes of an altruistic, cooperative, communal, thus democratic nature. He emphasizes that the process of participation thereby leads to a rejection of vertical authoritarian structure and exploitation and turns all the involved into organic intellectuals of the working class, who can continue the transformation process without external agents.

Fals-Borda succinctly says that PAR generally proposes a resolution of contradictions in a given region through recourse to endogenous elements and thereby may mobilize people from bottom to up and from periphery to center.

Besides a PAR's implication in a sociopolitical context, Fals-Borda looks into a phenomenological aspect of PAR from the experiences of those involved in PAR. Fals-Borda argues that they learn to "know and recognize" themselves as a means of empowerment by realizing that the science is a cultural production of purposeful and relative knowledge, which serve those who create and control it, thus it is equally possible that people's science is created through an informal and endogenous process. In this way Fals-Borda refuses the universality of knowledge and emphasizes the multiplicity of knowledge and its constructivism.

He suggests four techniques of PAR that may cultivate people's countervailing power: 1) collective research, 2)critical recover of history (selective discovery through collective memory of past elements useful in defense of the interests of the oppressed), 3) valuing and applying folk culture (to mobilize people, it is essential to develop techniques based on the folk culture, such as art, music, drama, myths, story-telling, etc.) 4) production and diffusion of new knowledge (for better efficient communication). Fals-Borda elaborate the fourth technique within the concept of knowledge ownership. He argues that there should a systematic way of returning the knowledge to the community and organization in order for them to maintain the ownership.

The Theoretical Standpoint of PAR (M.A. Rahman, 13-23)
Rahman delineates the basic ideology of PAR that "a self-conscious people, those who are currently poor and oppressed, will progressively transform their environment by their own praxis. In this process others may a catalytic and supportive role but will not dominate" (p.13).

Rahman precisely points out domination of masses by elites in controlling material production and means of knowledge production and the validation of knowledge. He asserts that scientific knowledge depends on consensus as to the method of verification, thus there exist different epistemological paradigms with different verification systems. From this perspective, he argues that the mass can ground their scientific knowledge based upon their own verification system, and maintains that an immediate objective of PAR is to bring to the people the legitimacy of such knowledge and the right to use it in order to act upon it.

Rahman finds a reason for growing application of PAR in the crisis of the right and the left, which the wisdom of "vanguards" is challenged, particularly in Eastern Europe. From this angle, Rahman regards PAR as a promotion of people's empowerment for changing their immediate environment at a micro level intervention via-a-vis states' imposing ideas of development. This empowerment, in his analysis, comes from two elements: autonomous/democratic people's organization and the promotion of popular knowledge. These two elements are either side of one coin to Rahman. He stresses that the process of organization is preceded and accompanied by a process of "concientization"--Rahman defines this term as a process of self-awareness raising through collective self-inquiry and reflection, which takes different forms such as dialogues, investigations and knowledge generation. Thus, to him, the transformation of the relations of knowledge is central to the entire task of empowerment.

Rahman also looks at PAR as a macro-level social transformation at a national level and concludes that PAR as the macro-social transformation is characterized as a cultural movement, independent of political movements. He finds the reason for the rise of such cultural movements in the crisis of left that assumed "advanced consciousness" (p.19).

Similar to the argument made by Fals-Borda, Rahman maintains there coexist intellectual/professional knowledge and popular/organic knowledge without dominance of one over another. By noting that the macro-level movements of PAR have been led by those coming from intellectual/professional tradition, Rahman points out negative forces implicated within such practices. Besides the assumed superiority of consciousness of the leaders, he sharply points out the fragility/instability of commitment for change:

"[C]ommitment us subject to transformation with the evolution of one's social existence. It often collapses from defeat, but may also diffuse or degenerate through the attainment of success that brings glory and power, altogether new experiences in one's existence. And commitment has also sometimes shown its fragility in the face of attractive temptations...In any case, there is no assured method of transferring commitment to succeeding generations who have not lived through the struggles from which commitment is socio-historically born." (p.21)

Rahman stresses the possibility that PAR sustains the macro level movements benefiting the initiatives of the people:

"Leaders may emerge to become a part of macro social-transformative efforts and thereby influence their course of progress; the culture of PAR may se a standard for working with the poeple and challenge macro vanguards to demonstrate teir commitment; the deepening of popular awareness, if PAR is successful, may work as a countervailing force against attemps at domination; aand the social formations (aware and assertive people's organizations and popular movements) promoted by PAR may be expected to support the best macro leadership that exists or that may emerge in the society" (p.22-23)

Despite the hopeful speculations of PAR, Rahman emphasizes the implementation of permanent vigilance and self-criticism within PAR. In particular in the face of instability, he stresses the recognition of possible degeneration and open popular analysis of such phenomenon.

A self-review of PAR (M.A. Rahman and O. Fals-Borda, 24-34)
Rahman and Fals-Borda reflect upon PAR for the past twenty years in the early 1990. They pays attention to the World Symposium on Action Research and Scientific Analysis in Columbia in 1977, where Antonio Gramsci became a prominent figure and the notion of participation was revisited. During this period, they summarize that PAR was regarded as 1) a research methodology that emphasizes the subject/subject relationship evolving in symmetrical and non-exploitative patterns of life and 2) social activism that is characterized with commitment to a promotion of people's collective and systematic praxis, which they argue became a primary object of PAR.

Since the early 1980s, they note, PAR advanced from micro and local community issues to complex, urban and regional dimensions while simultaneously becoming subject to cooptation. Examples of cooptation include: widening academic careers in the subject, a growing number of academic discussions and forums, appointments by governments, UN agencies' recognition of PAR as a viable alternative, and NGO's participatory approach.

With this circumstance, the authors emphasize the philosophy and practical results of PAR as a means of countering faulty assimilation. According to them, PAR validates and stimulates popular knowledge--"knowledge existing as local or indigenous science and wisdom, to be advanced by the people's self-inquiry" (p.31)--with their own verification system and its results should be seen in real life. To Rahman and Fals-Borda, PAR is a philosophy of life as much as a method in that PAR as a method for rigorous knowledge search is an open-ended process toward a structural transformation of society and culture, which requires continued commitment, an ethical stand and self-critique.

Rahman and Fals-Borda state that the purpose of PAR is to break up the monopoly of science and culture led by elites and to support people's movements for progress and facilitate their transition to the political dimension. The reason is found in the authors' standpoint of knowledge: they see that unequal relation of knowledge is a critical factor that perpetuates class/elite domination over the people. What is the most important to them with respect to popular action for socio-political change is to break the monopoly of knowledge and stimulate popular knowledge, which they call "a knowledge of life".

Toward a Knowledge Democracy: viewpoints on participatory research in North America (John Gaventa, 121-131)

Gaventa reviews participatory research within the context of democracy in a knowledge society in North America (He notes that participatory research has been explored much in the Third World within the context of adult education and development.) He finds three domains where PR is observed: 1) for the purpose of attaining knowledge of their own, 2) for the purpose of community-based, labor and minority education, and 3) New Left/environmental movements.

Gaventa sheds light on three strategies that are used in popular PR:
1. The reappropriation of knowledge: it is concerned about public access to knowledge/information, which is produced by the dominant knowledge system and shielded from the ordinary by secrecy, privatization and professionalization. The process of people's gaining control over knowledge is an empowering one because it allows people to realize the power structure that exists behind the knowledge control. (Gaventa's assumption is that power has been exercised through the control of knowledge.) An example is found in a research project done in a mining town to see whether chemicals in their water or work place are destroying their health. The participants were taught a basic literacy skill to read medical textbooks and in fact investigated the case. He label the process of people's viewing themselves as researchers and developing unique ways of gaining information "guerilla research". The reappropriation of knowledge contributes to challenging the fallibility of scientific knowledge and grounding action upon the unmasking of power structure, however, it is limited in the sense that it has little to do with creation of knowledge.

2. Development of people's knowledge: Gaventa develops this idea from the notions of Fals-Borda's popular science and Gramsci's intellectual and organic knowledge. Popular science means popular wisdom or knowledge based on empirical and common sense that belongs to the grassroots people and constitutes their culture (Fals-Borda, 1982, Participatory Research and Rural Social Change, Journal of Rural Co-Operation, X(1), 25-40). Fals-Borda's approach to popular knowledge resonates in Gramscian's tradition, which regards everyone as intellectual who can develop a popular and organic knowledge that turns the common sense into "good sense". Gaventa also notes that Robert Chambers in his book Rural Development: Putting the Last First (1983) demonstrates that such local knowledge of the ordinary proved to be more useful to them. Gaventa argues that the common approaches to PR are also useful in industrialized western societies in that "Within the Western world, popular knowledge is constantly being created in the daily experience of work and community life. The legitimacy of such knowledge, too, is constantly being devalued and suppressed by the dominant science" (p.128). The limitation of this strategy lies in the fact that the base of popular knowledge stems from people's experiences. What if knowledge required do not come from experience? Do all people need to be experts to participate in the process?

3. Popular participation in social production of knowledge: In the above case, Gaventa seconds a division of labor in a way that a group of people are involved in a decision making of knowledge production through popularly controlled research centers. An example of this approach is found in some research groups in Denmark and Sweden, where "reference groups" of those affected by research are involved with professional researchers in carrying out the projects.

By and large, Gaventa holds the creation of an alternative organization of science and knowledge that "is not only for the people but is created with them and by them as well" (italic from the original text, p. 131) toward genuine democracy.

Remaking Knowledge (O. Fals-Borda, 146-164)
Fals-Borda defines PAR as an autonomous collective investigation different from typical research methods. It is characterized as a "communal enterprise" in which the knowledge is validated socially through people's own verification system. Fals-Borda summarizes three theoretical elements that enrich the process of PAR:

1. the ontological possibility of a popular science: Popular science, which is emancipatory and collective by nature, becomes a tool in the quest for justice and answer to epistemological questions such as knowledge for whom and knowledge for what.

2. the existential possibility of transforming the researcher/researched relationship: Fals-Borda redefine the concept of participation to create a subject/subject relationship. For him, participation goes beyond representative democracy. It means "breaking up intentionally by means of vivencias ("experience") the asymmetrical subject/object relationships of submission, dependence, exploitation and oppression that exist between persons, groups and social classes" (p.152). He stresses that despite the recognition of their otherness researcher and researched interact and collaborate to create an enriched knowledge and achieve social progress, for instance, by outsiders' providing technical expertise, situational analysis or acting as intermediaries while insiders provide specific local knowledge and know-how and act as critics.

3. Autonomy and identity in collective research: this is essential to constructing knowledge democracy and build people's countervailing knowledge and power, civic movement and self-reliance promoting organizations vis-à-vis the state, central power, monopolies and military complex.

The third element seems more important to Fals-Borda. Within this context, he stresses praxis as the technical foundation of PAR for critical recovery of history and culture. Following Hegel and Marx, Fals-Borda emphasizes the practicality of praxis and defines as a "dialectical unit formed by theory and action in which action is cyclically determinant" (p.156).

Among the theoretical convergences between PAR and other schools of thought noted are critical pedagogy and some ethnologists who approach indigenous and local culture with a participatory philosophy. He also points out among sociologists Alain Touraine's method of "sociological intervention" (1978 La voix et le regard), which attempted to bridge the gap between research and action by asking researchers to work with social movements as mediators. Fals-Borda differentiates Touraine's social intervention from PAR in that Touraine warned deeper involvement or commitment although he advocated undertaking collective research. It is notable that Fals-Borda finds the nominal root of PAR in Kurt Lewin (1946, Action-Research and Minority Problems, Journal of Social Issues, 2, 34-46). He criticizes that despite the similarity to PAR, his insights were reduced to small group processes (industrial management) and clinical approaches (veteran's rehabilitation) after his death, however, transcended to Organization-Development school of action research in community work, educational systems and organizational change in the 1970s.

Fals-Borda states as PAR's general trends holistic philosophy, relativist knowledge, interdisciplinary descriptive methods, intuition, daily life phenomena and the human scale, and suggests PAR in terms of a new paradigm in that it calls for general epistemic change in its nature. He connects PAR to Foucault's idea of knowledge rather than Kuhnian principles and intends to develop a system of subjugated knowledge through PAR.