Pre-Use Evaluation of ‘ English Unlimited’ B1, Unit 4
Materials evaluation is as important as writing materials and should be based on the same principles, bearing in mind people using them. It also important to have a clear distinction between analysis and evaluation.
Analysis is more objective and analysis questions usually have “yes” and “no” answers, whereas evaluation is subjective and often can be answered using “very likely” , “very unlikely” and etc. (Tomlinson, 2013).
There are different types of evaluations and they vary depending on the purpose, timing and etc. :
Pre-Use evaluation, consists of two stages:
Stage 1. It is a flicking through stage. Checking topic choices, attractiveness, visuals, and etc. It is impressionistic.
Stage 2. It is an evaluation of a small number of materials and involves a checklist which makes the evaluation criteria explicit, the checklist has to be up to date, “local”: there are many published checklist but they are general and contradict the evaluation aim.
Whilst-use evaluation
Post –use evaluation
There are many published checklist, however Tomlinson(2013) suggest that most of the lists are subjective and proposes a way of developing criteria for material evaluation:
1. Brainstorming a set of universal criteria
Based on principles of SLA and the results of classroom observation.
2. Subdividing some criteria.
3. Monitoring and revising the list of universal question.
4. Categorizing the list.
5. Developing media-specific criteria: if the voices are easily distinguished and etc.
6. Developing content-specific criteria
7. Developing age-specific criteria
8. Developing local criteria. ”Different criteria will apply in different circumstances “(Cunningham 1995)
9. Developing other criteria: culture-specific
10. Trial criteria
11. Conducting the evaluation.
If to put it in a nutshell: evaluation criteria consists of three key notions: Psychological validity(learners’ needs, goals and etc); pedagogical validity(teacher’s skills, abilities, beliefs, theories); process validity(materials’ writer presentation of the content and approach to teaching and learning),Rubdy(2003).
Why has an evaluation to be principled and systematic?
1. Materials play an important role in language learning process, F. Mishan and I. Timmis(2015) .
2. Materials represent professional, financial and political investment Sheldon,(1988). E.g. providing course books for the state school system.
3. Academic reason: it can serve as a powerful professional development activity, Sheldon(1998).
We have based our pre-evaluation process on comparing the principles and analysis found in reading B.Tomlinson(2013), I. McGrath(2013) and F.Mishan and Timmis) to our principles, beliefs, and expectations and used the following stages of evaluation:
1. Analysis.
-Is the course book attractive
-age category
-if the course book has CDs, Teacher’s book, and etc
2. Content-related criteria.
-If the topics are engaging and politically and culturally sensitive, have an element of authenticity and etc.
3. Language related criteria.
-grammar rules and presentation, vocabulary.
4. Student related criteria.
-if the materials meet students’ needs, learning styles, if the interactions are easy to follow.
5. Teacher-related criteria.
-If the materials easy to adapt, supplement and follow.
6. Format.
Conducting this evaluation we followed the principles we had selected at the beginning of the evaluation process and we came up with a total score of 68%. It was interesting that other groups, which used different evaluation checklist, come up with approximately the same figure.
LINK FOR A PRESENTATION: https://prezi.com/jizdkkih56k3/pre-use-evaluation-of-coursebook/
Here is an example How I used our evaluation on the material I have designed for my online student studying Hotel Management and Hospitality. For this evaluation, I applied the evaluation criteria from our presentation.
The aim was to create a suitable material involving visual example and having an education purpose; we had to learn the correct procedure of checking-in into the hotel following the worldwide standards. It wasn’t an easy task to come up with a suitable authentic video matching the student’s level and needs. After a long search, I was lucky to come across an authentic video, taken by American girls practicing the check-in pricedure.
https://www.youtube.com/watch?v=QcRqBBAiC4o
The original video was too long and had a lot of extra unnecessary information. I’ve edited the video and managed to shorten it to 1min 30sec.
If to follow our criteria, I have ticked off the most of the points:
- the topic is authentic
- it’s culturally and politically sensitive
- it’s relevant to the student
- the task meets the real purpose
- and it’s engaging
- if it promotes cultural awareness; I think yes, it shows the rules of hospitality in English spoken countries: smile, be friendly, ask some impersonal questions.
- the vocabulary and grammar match the student’s level
- it balances all four skills
- it promotes and encourages speaking( we practiced the check-in procedure, the right, and the wrong way, tried to spot the mistaken in check-in procedure)
- corrected the pronunciation when needed
- easy and clear instructions
- the materials are students centered
- a variety of activities helping to achieve the aim of the lesson( drill the check-in procedure, the worksheet three allows free practice using different situations).
- The materials are easy to adapt. They can be easily used for other learners
- they are easy to supplement
- allows for recycling the language and grammar(question forms, polite forms)
These worksheets and video were used not only for acquiring the language but also meeting the purpose of student’s academic study. This material allowed us to activate schemata, we spoke about her experience of staying in the hotels, how she sees herself in this role, what are her expectations and vision of luxury hotels, we brushed up the essential vocabulary(hotel facilities), and she was very pleased with the video as it served as a clear example of what she should do and allowed plenty of room for practicing.
References:
Tomlinson, B. (2013) Materials evaluation. In: Tomlinson, B. (ed). Developing Materials for Language Teaching. London: Bloomsbury.
Mishan, F. & Timmis, I. (2015) Materials Development for TESOL. Edinburgh: Edinburgh University Press. chapter 4: Materials evaluation and adaptation.
McGrath, I. (2013) Teaching Materials and the Roles of EFL/ESL Teachers: Practice and Theory. London: Bloomsbury. (chapter 3 The professional literature and chapter 5 How teachers evaluate coursebooks.)
Johnson, K., et al. (2008) A step forward: investigating expertise in materials evaluation. ELT Journal 62 (2).