The pre-seminar task for week three involved organising ourselves into groups of three, choosing one of the selected readings and summarising it for the rest of the group. Based on our own experiences and the reading, we created a draft framework for materials evaluation.
Our first port of call was Thomlinson’s state-of-the-art paper that was set as a key reading at the start of the course and an article by Roberts (1996). From this introduction, I gathered a few points to keep in mind when deciding on our framework such as:
• What is the context? Should we evaluate the coursebook for use in multilingual, general English classes in Brighton or as a global coursebook? We decided on the former. Tomlinson (2012) distinguishes between universal and local criteria. While universal criteria help evaluate materials for any learner anywhere, local criteria are specific to context.
• How many ‘ticks’ will mean the coursebook is ‘good’?
• Tomlinson (2012) distinguishes between material evaluation and analysis. Does the book have extra materials? Are there any teacher’s notes? Is a vocabulary list included? Are all analysis questions. We decided not to include analysis questions.
• Will all the principles we looked at in the second seminar translate to an evaluation of a coursebook? For example, one of our criteria was that materials should be authentic. Do authentic materials refer to ungraded texts or do you believe, like Tomlinson, that authentic materials refer to material where the texts are graded but the tasks are authentic? Tomlinson believes an authentic task is when learners communicate in order to achieve a non-linguistic outcome and do not necessarily refer to real-life tasks.

Tomlinson (2012) has a checklist for evaluation questions:
Is each question an evaluating question?
Does each question only ask one question?
Is each question answerable?
Is each question free of dogma?
Is each question reliable? (Would it be interpreted in the same way by different evaluators?)

I then looked at McGrath (2013) and the format of evaluation, where he talks about arranging criteria into categories such as:

• Students/ Teacher/ Syllabus/ Examinations.

• Programme and course/ Skills/ Exercises and activities/ Practical concerns. (Such as price, durability, and availability)

• programme factors/ teacher/ learner/ content/ pedagogical factors. (Such as method, design of activities and exercise types)

As a group, our chosen categories were teacher/ student/ content/ design,  and the format of our criteria would be quantifiable, closed, yes/no questions. We also discussed if individual criteria would be weighted using another scale to allow for distinctions to be made between criteria felt to be more/less important.

I then looked at McGrath (2013) again and while the whole chapter looked at retrospective evaluation, he lists a lot of criteria gleaned from various studies where teachers and students have answered questionnaires regarding textbooks. For example, one study looked at New English File and Opportunities and the criticisms of those two books such as; insufficient support for independent learning, pronunciation practice and vocabulary review. Also, because both books are intended for the global market, the interviewees felt that the target language culture was always presented, expressing the ubiquitous idea that global textbooks can’t meet local needs.

So far our framework looks like this:

TEACHER

Is the coursebook easy to adapt?

Does the coursebook have opportunities for personalisation?

Does the coursebook cater for teachers with different teaching styles?

Does the coursebook introduce teachers to new techniques?

Is the decision-making weighted towards the teacher?

Is there guidance on using material and an answer keys available for the teacher?

STUDENT

Will students find the tasks and topics engaging/ motivating?

Does the coursebook cater for students with different learning styles?

Does the coursebook encourage communication?

Is the coursebook relevant to learner’s needs?

Does the book encourage learner autonomy?

Does the coursebook integrate soft skills?

Does the coursebook use technology to facilitate language learning?

Can learners supply their own ideas?

CONTENT

Does the coursebook contain some authentic texts and tasks?

Is the coursebook culturally sensitive? /Does it help build intercultural awareness?

Is the coursebook related to real life?

Does the coursebook reflect the nature of language learning?

Is the content primarily form or meaning-oriented?

Does the coursebook present grammar in context?

Does the coursebook contain interesting/ up to date topics?

Does the coursebook provide opportunities for recycling?

Will students find the coursebook suitably challenging?

DESIGN

Does the coursebook have a clear and logical layout?

Does the coursebook have clear instructions?

Does the coursebook have catchy visuals?

Is there continuity and/or a route through the coursebook?

Do the illustrations inspire students to be creative?