Action Research - Section 7

Chapter 4 Research Methodology

4.1 - Quantitative and qualitative research

David Nunan writes:

... research is a process of formulating questions, problems, or hypotheses; collecting data or evidence relevant to these questions/problems/hypotheses; and analysing or interpreting these data. ( Nunan 1992, p 3)

These are the fundamental elements of every piece of research: hypotheses/questions, data collection, data analysis/interpretation. The choice of the method mainly depends on the goals of the research and also on the role of the researcher. I am a teacher, a teacher-researcher in this case, and my goal consists of analysing and evaluating the effectiveness of my work by observing the outcomes of my students. The deeper aim is obviously the improvement of the teaching/learning process in which I am involved together with the environment where it occurs. At the same time all my research projects have been aimed at enriching my theoretical knowledge about educational processes and improving my professional skills, points that are strictly connected with the former goals.

As Johnson puts it:

The goals of teacher-researchers are multiple and controversial. One goal is professional development. Inquiry can be extremely valuable to individual teachers who wish to reflect and improve on their own teaching and thus grow in personal professionalism ... projects can also result in school improvement. For example, they can produce evidence that may convince school administrators of the value of certain approaches to learning. (Johnson 1992, p 224- 225)

The first choice to be made is between quantitative and qualitative research, and this is easy because:

Quantitative research is designed to give numerical results which can be reported in tables, graphs and charts stating the number of something, the proportion of something, or what the trends are. Qualitative research has a different goal: it answers the question ‘What is going on here?’ ... from the perspective of those who are in the situation being researched (Bouma, 2000 p171)

Qualitative research is the natural choice as what I want to do is to probe what ‘goes on’ during my ICT-based lessons and see when and how they result in students’ language improvement, create a motivating climate and interaction. What I want to do is observation and analysis of what I think should be a ‘good practice’ but that I have not evaluated before according to a systematic method.

Gary Bouma goes on saying:

The aim of qualitative research is often to describe in detail what is happening in a group, in a conversation or in a community – who spoke to whom, with what message, with what feelings, with what effect. (Bouma 2000, p 173)

The boundaries of my research will be my ‘group’ and my ‘community’. The focus of my piece of research will be concerned with the context where I work, the students I work with, the activities I have devised, the resources I have been using. The reason why I want to find limits to my research is the awareness that the only context I can fully control, manage and analyse is the only one I know, both from the human and from the organisational points of view.

4.2 - Case study

Inside the framework of the qualitative analysis, the most suitable type of research in my situation is the case study, since, as Johnson says:

The purpose of a case study is to describe the case in its context...the unit of analysis (i.e. the case) might also be a teacher, a classroom, a school, an agency, an institution, or a community. (Johnson 1992, p 76).

What is a case study?

A case study is defined in terms of the unit of analysis. That is, a case study is a a study of one case. A case-study researcher focuses attention on a single entity, usually as it exists in its naturally occurring environment, . (Johnson 1992, p 75)

In my research the case will be limited within the precise boundaries of a class where some language activities are carried out by using ICT in the precise context of my school, since I too am a part of this context. This will be the “bounded system” I will observe and analyse and I’ll be a component of this system . My case study will essentially have a descriptive purpose, it will be defined as a descriptive case study, whose aim is to observe in depth what goes on during these lessons, and in which I will play the role of the participant researcher.

Stake, reporting Louis Smith’s words, states:

Louis Smith, one of the first educational ethnographers, helped define the case as a ‘bounded system’, drawing attention to it as an object rather than a process. (Stake 1995, p 2)

My class and I, inside the school context where we work will be the object of my research and so also my “bounded system”.

It could be classified also as a piece of “teacher research”, a “school case study”, according to the typologies devised by Stenhouse and shortly reported by David Nunan:

Stenhouse (1983) develops a typology of case studies... neo-ethnographic, which is the in-depth investigation of a single case by a participant observer... evaluative, which is ‘a single case or group of cases studied at such depth as the evaluation of policy or practice will allow (usually condensed fieldwork)’. In contrast with these first two, the multi-site case study consists of ‘condensed fieldwork undertaken by a team of workers on a number of sites and possibly offering an alternative approach to research to that based on sampling and statistical inference’. Such research approaches ethnography... The final type ... is teacher research. This is ‘classroom action research or school case studies undertaken by teachers who use their participants status as a basis on which to build skills of observation and analysis’ (Stenhouse 1983: 21) (Nunan 1992, p 77)

As I said before I will be a “participant observer”, a part of the system, and inevitably, just like in any other system, I will interact with the other components, I will influence them and be influenced by them. This can represent an advantage, because it means I deeply know the context and I can catch and comprehend any minimal details of what will occur in the situation, on the other hand this can be a disadvantage, as I can be influenced by my knowledge of the context and interpret more than observe, or see what I want to see and neglect what I do not want to see.

As Cohen and Manion put it:

The accounts that typically emerge from participant observations are often described as subjective, biased, impressionistic, idiosyncratic, and lacking in the precise quantifiable measures that are the hallmark of survey research and experimentation. (Cohen and Manion 1994, p 110)

I do not think this can represent a problem in my research case, as what I am interested in is just the unique interaction occurring between me and my class in our specific school environment, not in an aseptic experimental field, where everything could be certainly more objective, but inevitably false, meaning false the opposite of real.

A frequent criticism of case study research is that it is not possible to draw a generalisation from the analysis of a single case. The answer is again in the purpose of the researcher and the type of research he/she decides to carry out. Stake says:

Case study research is not sampling research. We do not study a case primarily to understand other cases. (Stake 1995, p 4)

And he goes on by affirming:

The real business of case study is particularization, not generalization. (Stake 1995, p 8)

Nonetheless generalisation – ‘ ...an inference that goes beyond the data.’ (CARE, UEA 1994, p 77) - is an important issue of research and is related to its validity, both internal, as far as reproducibility is concerned, and external, as far as generalisation is concerned.

Can a case study have as its results the production of judgments upon a typology of situations, can a case study be generalised?

Cohen and Manion state:

...the case study researcher typically observes the characteristics of an individual unit – a child, a clique, a class, a school or a community. The purpose of such observation is to probe deeply and to analyse intensively the multifarious phenomena that constitute the life cycle of the unit with a view to establishing generalizations about the wider population to which that unit belongs. (Cohen and Manion 1994, p 106)

So, according to them, a generalisation could be possible but only to the other population to which the observed unit belongs. The problem is to define the population to which my unit belongs: only my other classes? My school? Schools in general? The other classes where similar activities are carried out?

It has to be added that in a case study we are in the field of ‘naturalistic enquiry’, whose

... meaning in use now, in educational research and evaluation, emphasises the idiosyncrasy and intentionality of human action, the importance of biography, history and particular circumstance. (CARE, UEA 1994, p 21)

Consequently a type of generalisation, defined as naturalistic, can be achieved comprehending other people/situations that feel a basic identity with the studied case:

...the emerging theory of naturalistic generalisation ... suggests that on the basis of a study or studies accounts should be constructed that have the potential to resonate with the reader’s experience. The reader is invited to underwrite the account by appealing to her tacit knowledge of human situations. Thus to generalise is to resonate with prior experience or to see common features among empirically different but conceptually equivalent human experiences. (CARE, UEA 1994, p 79)

4.3 - Collecting data

A wide range of techniques can be used to collect information for case studies... data collection can be entirely naturalistic, in which researchers collect information from the natural setting as unobtrusively as possible, or it may involve eliciting language or other information. Very often both naturalistic data and elicited information are gathered to provide different perspectives on the same issue. (Johnson 1992, p 86)

According to Johnson, the techniques for collecting data and the perspectives from which data are drawn and/or elicited have to be various so that a triangulation coming from the different sources and showing different points of view can allow the researcher to be accurate in the analysis and to show the validity of the research.

My next step is to think of the various sources of data that can answer my research questions most directly and from whom they could be collected. The immediate response is that I can collect data by direct observation and, at the same time, elicit some more in-depth considerations and personal ideas from the protagonists of the situation. In a few words I am interested in analysing and comparing the points of view of both the students and the teachers, both naturalistically and by directly requesting interpretations through interviews and document analysis.

Observation

Since I want to observe what goes on in class in terms of language acquisition and interaction, ‘naturalistic observation’ of class activities is my first and principal source of data.

David Nunan says:

... questions ... concerned with investigating behaviour in context, suggest descriptive and interpretative research. ( Nunan 1992, p 3)

To sum up, my observation is naturalistic, descriptive and interpretative.

Naturalistic, because natural is the environment in which it takes place and natural are the roles of the protagonists, even if my role is a double one, teacher and researcher. The former is natural and the latter unusual but not totally unnatural because teachers always tend to ‘observe’ what goes on in their classroom in order to control and manage the situation; this time the observation’s purposes have a further facet to develop.

Descriptive, because description is the goal of my piece of qualitative research carried out in the boundaries of the environment in which I work, represented by the case of the class I have chosen and the specific activities I want to analyse.

Interpretative, because the analysis and the triangulation of the data collected will show trends and will allow us to interfere interpretations from them that can be valid for the situation described.

Of course I am a participant-observer and this implies arguments in favour and against:

. ...in participant observation the observer becomes part of that which is observed. Participant observers use their position in a group and their own experience of a process in order to gain information about it. (Bouma 2000, p 179)

On one hand having the researcher as an active participant offers some unusual opportunities for collecting data, because of his/her deep knowledge of the situation and the normality of his/her presence in the context, but could bring some problems as well, obviously related to the possible lack of impartiality of the observation’s accounts.

For some researchers this contingency of observer and observed poses an ever-present threat to the truth that could be told, and must be contained through constant vigilance and technical virtuosity in designing observation protocols. (CARE, UEA 1994, p 111)

Is there a solution to this problem? I think it is only up to the intellectual honesty of the participant-researcher, in this case the teacher, who has to clearly describe the setting/field of the research and his/her role in it, and, when reporting data collected, clearly say when s/he is describing or narrating events and when s/he is interpreting them.

Some ‘insulation’ of observer and observed may be attempted though, through injunctions about the language of observation: e.g. aim for description rather than judgement; distinguish evidence from interpretation. (CARE, UEA 1994, p 111)

During the observation, the qualitative case study researcher keeps a good record of events to provide a relatively incontestable description for further analysis and ultimate reporting. He or she lets the occasion tell its story, the situation, the problem, resolution, or irresolution of the problem. (CARE, UEA 1994, p 62)

How do I carry on my classroom observation? The methods are many and all well known: audio-recording and video-recording are the most commonly used and each of them presents advantages and disadvantages. I exclude video-recording because it is the most difficult to carry out and the most intrusive and ‘unnatural’, and I opt for audio-recording because the students do not feel embarrassed by the use of the cassette recorder as it is one of the teaching aids normally used in English lessons. The main advantage of this method is obviously the possibility to faithfully catch all the verbal exchanges, pauses and interactions during the lessons, the main disadvantage is represented by the amount of time required to transcribe the entire recordings, as it is necessary to have them turned into text for ‘...the purpose of summarization, analysis and presentation.’ (Bouma 2000, p 183).

However, in my case study it is fundamental to have the precise description of the classroom events as the basis of the analysis, so I think it is worth spending the necessary time in handling the recorded information.

Personal diary

For many researchers, the most important thing is to have a personal diary or log in which everything is kept: calendar, telephone numbers, observation notes, expenses. (CARE, UEA 1994, p 55)

I have always been used to write informally my impressions, ideas, hopes, disappointments, etc., in my personal diary, even when I did not know it could be considered a useful tool for a teacher. I thought I liked to do this instead of formally reporting data, because I am an incredibly untidy and “non-systematic” person, but I have always noticed that the information contained in my diary has always been extremely useful for my work. So, now I go on using my diary, as usual, with a particular attention during the period ‘under observation’. It is a sort of informal account of the events I want to describe where I can freely express my point of view, my considerations, anxiety or satisfaction on the part of the teacher. The teacher-researcher will consider it, later, as one of the sources to be analysed together with the others.

Questionnaires and Inventories for the students

I gather data from the students in a systematic way with a questionnaire/inventory addressed to the class at the end of each lesson observed, in order to know their impressions, observations, interpretations after each experience.

The advantages of inventories or of closed questions is that they are a fast way to collect a lot of information because they require a short time to be answered and to be ‘read’. On the other hand, they represent a limitation in the possibility to fully express impressions, opinions and relevant issues because the scope and coverage of the questions are totally controlled by the person who creates them, and important points could be totally neglected only for the reason that the questionnaire’s author does not consider them important. Another disadvantage concerns the long time that closed questions need to be clearly devised and the attention the writer has to pay to writing very clear instructions on how to answer the questions.

For this reason, to have a clearer and more complete representation of the opinions of the students, I use also a different method in the end of the observation period: I give them an open questionnaire in which the students are free to put in evidence both negative and positive aspects of the experience.

As to the open questions, it has to be said that they are easy to devise, more comprehensible to the people who have to answer them, they produce more interesting, and often unexpected data, but because of this greater freedom the responses are difficult to analyse and organise.

All the questionnaires are written and answered in Italian, so that the students, who are at an elementary level of English language competence, can correctly understand what they are asked to do, they feel free to completely express their ideas and do not perceive the questions as a kind of English class-work.

Documents

The documents I evaluate are the plans of the email module and of the content-based module carried out in co-presence with the subject matter teacher of Mathematics, materials used in the lessons, and some written work produced by the students during the activities observed. They are useful to clarify the purposes of the experience, the way the teachers have decided to carry it out, and show some results, either good or bad.

Always in the perspective of triangulation of data, the documents have the function to be compared with the evidence from other sources and can help the researchers to make inferences about the events observed.

Research ethics

Ethical issues are inevitably present also in a classroom case study carried out by a teacher in one of her classes, inside her school environment, since inevitably there are some unusual things happening in the classroom during the period of observation. For this reason the research participants – students and teachers of the class involved - and the school Headteacher, as responsible for everything happens in the school, are informed of the nature of the research that is carried out, of its purpose and the procedures to be followed. The information is provided in appropriate and separate meetings with the class, the subject-matter teachers involved and the Headteacher, preceding the period of observation. The information is only given orally and no written documents to be signed are used, as it would sound too formal and unnatural in a situation concerning individuals who know one another so well. It has to be added that this is the second time that I am conducting a research in some of my classes, so for the Headtecher and my colleagues this does not represent a novelty any more. The use of too formal procedures and documents might be counterproductive on the part of both students and teachers, as it would generate anxiety and an artificial behaviour during the lessons observed. The class observation, on the contrary, should be considered as a procedure aimed at improving the work of all the learning community: the language teacher, who can see negative and positive aspects of her work, and the students than can receive the benefits of the teacher’s research results.

After informing the students, the Headteacher and the other teachers involved, about the features and the purpose of my investigation, they are reassured that the treatment of the data collected will be strictly confidential: the questionnaires are anonymous, and their names will not be mentioned in the documents I enclose for the dissertation.

Finally, all participants are reassured that no changes in the school organisation are to be expected, in fact the time schedule of the class and the curricula planning will be respected; the changes that might occur will be negotiated with the participants before starting the research as well as the procedures that will be followed in the lesson observation.

All the activities connected to data collection and lesson observation take place in the usual environments of the classroom or the multimedia lab, at the usual time the students have their English lessons. The questionnaires and the inventories are completed in the last minutes of the lessons in anonymous form, both for privacy reasons and in order to make them freely express their thoughts.

I start the class observation after all of them have accepted the proposal and have shown real interest to co-operate for this project. The period of observation is approximately one month and the lessons observed are six: four lessons about email exchange and/or forum messages exchange, and two lessons of the content-based module of English and Maths.

Top of the page