In seminar 4 we addressed how, when and why we adapt and supplement materials with a focus once again on ELT coursebooks as we acknowledged they form the backbone (if not the staple diet) of the the majority of language courses.  Even after selecting coursebooks for the ‘best fit’ for our curriculum, our students and ourselves, we also universally agreed during this seminar that adaptations were both necessary and routinely  applied. In my first post I referred to Tomlinson’s summary of Islam and Mares’ (2003) guidance on evaluation criteria pertaining to adaptation objectives which were:

adding real choice, catering for all learner styles, providing for learner autonomy, developing high level cognitive skills, and making the input both more accessible and more engaging.

I reflected there that this is is one of the key areas I’m looking to gain insight into so that I can more confidently more away from the coursebook to meet the specific needs of students in our materials rich, post-method era.

In our seminar we looked at how we adapted.  It seems we did it both consciously in a planned way but also unconsciously and spontaneously during a lesson. We shared some of the ways we did this, such as changing the order of tasks and their pace, leaving bits out, adding extra dimensions or substituting items.

We also discussed when we did it, which was often at the course and lesson planning stage but also during lessons in response to students’ reactions to the material.  Unconscious adaptation, such as adding extra student relevant examples also occurred and spontaneous changes such as altering the pace of an activity that seemed wrong or shortening/extending activities to increase the engagement of students or to add challenge and to differentiate.  In this way, we, as experienced teachers, reflect Madsen and Bowens’ (1978) description of ‘good teachers’ who as Tomlinson (2012) summarises:

are always adapting the materials they are using to the context in which they are using them in order to achieve the optimal congruence between materials, methodology, learners, objectives, the target language and the teacher’s personality and teaching style.

Madsen and Bowen’s key rationales for adaptation were: to personalise, individualise, localise and modernise materials.  These ideas regarding ‘why’ we adapt run through the literature and are represented in the encompassing notion of  adapting for ‘context’.  Although publishers create and we select materials for a target market, adaptation allows us to add value to what is offered there, so it can become even more suitable for the precise needs of the students in our context.  As McGrath (2002) states,  – to make them a ” better match for a specific learning context.”

Here below are the key reasons for adaptation according to McGrath  – as taken from the seminar slides. It can be seen that some stem from a perceived shortfall in the material itself (e.g. it’s not up-to-date)  and some from the need to fine tune to students’ particular needs (e.g. simplification/complexification). What is not included below is the adaptation for teacher styles and personality that would complete the picture as both Madsen and Bowen describe above and Maley (1998) refers to as the “complex trade-off between (these) three major elements in the equation…”.  I am sure I’m not alone in having a preferred teaching style with preferred procedures. I’m constantly looking at ways to make my classrooms active and creative, and to reduce or eliminate passive behaviour. Yes, this is for and on behalf of the students, but undeniably part of my preferred style too.  In a note to myself however, I acknowledge what Mishan and Timmis point out, that too much eclectic adaptation in pursuit of lively and engaging classes can lead to loss of coherence and focus. Principled adaptation should always put the goals of the students first.

The importance attached to  addressing the ‘contextualising’  issue at the coursebook design stage has been highlighted by writers such as Saraceni (2003), who, as Tomlinson (2012) writes, proposes that “materials should actually be written with learner adaptation in mind, aiming to be learner centred, flexible, open ended, relevant, universal and authentic, and giving choices to learners.”  This approach to materials writing is the ‘fundamental change’ that is needed in coursebook design in order to provide “greater flexibility in decisions about content, order, pace and procedure” according to Prabhu as reported in Maley (1998).

 Prabu’s proposals developed by Maley in the same article suggest ‘flexi-materials’ that seek to lessen the constraining influence of coursebook use.  Coursebooks that take this approach are certainly welcomed by teachers, offering different pathways through the same raw materials – encompassing different procedures, levels of difficulty and language content and alternative texts.  They have the potential to make adaptation more time efficient for teachers and in Saraceni’s model a collaborative process between teacher and students.   Essentially and crucially however, the responsibility remains with the teacher to filter, enhance and orientate to context, whether that be in one ‘flexi-materials’ package or not.  

In many educational institutions the interactive white board allows adaptations in enhanced ways – making alternative content (additions, extensions, reordering and branching in Maley’s (2011) words) more accessible for both teacher and students.  In this way adaptation has evolved to encompass this resource. While not available in the institution where I currently teach, I can see that the interactive white board serves the adaptation and supplementation process well.  And since adaptation is intrinsic to what we do, anything that enhances that process is welcomed: white boards, flexi-materials, but most importantly, a principled, responsible approach.

References

Tomlinson, B. (2012) Materials Development for Language Learning and Teaching. Language Teaching 45 (02): pp 151-152

Maley, A.  Squaring the Circle – reconciling materials as constraint with materials as empowerment. In Tomlinson, B. (ed). (1998) Materials Development in Language Teaching pp279 -294.

McGrath, I. (2002) Materials Evaluation and Design for Language Teaching, chapter 4.

Mishan, F & Timmis, I (2015) Materials Development for TESO, chapter 4.

For the 2nd part of the seminar in week three the coursebook writer Theresa Clementson, who had written’English Unlimited’ (the coursebook we had just evaluated) shared her insights and approaches from her professional perspective.

She told us that she is usually invited to participate in new coursebook writing when the publishers ‘spot a gap in the market’ and commission accordingly. Cambridge for example, have a 10 year plan. Ultimately of course, materials need to be popular and  have universal appeal in order to be able to sell well and make a profit for the publisher.

The perceived ‘gap in the market’ or ‘need’ is translated into a brief from the publisher. For  ‘English Unlimited’ that brief was for:

• an adult orientated coursebook
• with an international flavour (no special focus on UK culture)
• using real English based on corpus – using English as lingua franca
• recognising teachability

We asked if she used a principled approach when setting about her writing task and how she implemented this and she observed that her approach to writing a book is similar to that of writing a lesson plan.  Her chief concern is for all her material to be interesting, and in relation to ‘English Unlimited’, interesting and cerebrally engaging for adult learners.

Further principles for her, based on her knowledge and experience in TESOL were:

• Using a text based approach based on real language in us
• Facilitating noticing
• Using proper communicative tasks
• Using lexical chunks
• Facilitating learner autonomy
• Ensuring cognitive challenge

It was interesting to compare her objectives to our own evaluative criteria and findings based on our principles.   We divided our criteria into teacher, student, language, and content categories.  The criteria she mentioned were mostly content and language related.  For instance, we gave her book a positive high rating of 5/6 for ‘helping students to notice grammar in use’  – which was one of our criteria and one of her stated objectives when writing.  We also  positively rated her use of ‘authentic texts’ in unit 4, for the stories and pictures based on real life ‘extreme weather events’.  Although time did not allow for her to go into all the principles she used and at no point did she refer to using any kind of checklist, it was clear that strongly held universal principles informed her writing and that because of her considerable experience these had become more implicit than explicit – in the way Hadfield (2014) had suggested (discussed in my previous blog entry ‘Principles and Frameworks for ELT design’).

On the subject of process, she commented that writing was both a collaborative and synthesising process whereby ongoing evaluation and redesigning takes place to produce the finished product which often looks nothing like its original conception.  And here, from the literature, Jolly and Bolitho’s (2011) ‘principled, practical and dynamic framework’ of a materials design process is reflected.  With reference to process she made a few comments about digital learning materials. While they are cheap to produce and cheap to adapt and update, her words of warning were that producing materials quickly affects their quality.  And from her experience the process is necessarily lengthy and highly evaluative. My own point of view on this is that, it doesn’t have to be that way.  Digital materials are quicker to produce and revise but if subject to the same rigorous principled design and evaluation criteria as printed material, there is no reason for them to be sub standard.  I’m thinking here of an analogy with TV advertising.  Some do the job impressively and effectively with regard for underlying ‘principles’.  Some are utter rubbish.  As teachers, one key role is to be highly discerning when using digital language learning materials.  

Finally, I was interested in her perspective on post-use evaluation.  When writing 2nd editions she explained that publishers seek and receive direct feedback from many sources – although she didn’t give details here due to time constraints, she encouraged us all to critically assess the coursebooks we use and contribute to the feedback loop using the platforms offered by publishers. I very mush appreciated listening to her perspective.

This was a busy week culminating in a very engaging seminar in two parts.  Following on from our work on principles for materials design we were given the following task to research and present in the 1st half of the seminar along with 2 other groups.

• What framework most clearly reflects your approach to materials evaluation?
• Arrive at a materials framework and apply that to the coursebook ‘English Unlimited B1+ unit 4.

Remembering the discussions and reading from the previous week we were minded to recall that an effective design framework, for a ‘constituency bigger than just me an my learners today’, needs to capture the most salient ‘universal’ principles and allow for ‘contextualisation’ – or adaptation to local issues.

In Tomlinson’s view, (State-of-the-Art Article 2012: 153):

The ongoing evaluation of the developing materials should be driven by a set of agreed principles, both universal principles applicable to any learning context and local criteria specific to the target learning context(s).

We used skype, email and two face-to-face meetings to organise and develop the project. We used  ‘prezi’ to present our findings (a new tool for me!)  Our initial brief was to research one author each to share with each other and inform our project.  I read McGrath and I’ve summarised it here:

Notes from McGrath (2013) Teaching Material and the role of EFL/ESL Teachers.

Chapter 3: The professional literature.

Rationale for evaluating materials – principally coursebooks:

Teachers have two broad responsibilities in relation to materials: to evaluate them and to redesign them. These processes are interrelated. They therefore have a “critical and creative stance in relation to materials”.

The role of materials selection is (or should be) a ‘stage’ in the course design process. Material selection should not dictate syllabus design. The design of courses and thus the selection of materials is governed by principles.

(Tomlinson, (2013): ( not from this paper) refers to 6 principles of language acquisition and 4 principals of language teaching combining to produce 30 principles of materials writing and materials writing is closely allied to material evaluation – sharing a concern for the same principles.)

Types/stages of evaluation

Predictive: Most widely used and involving selecting the coursebook prior to use. When there are many options involved it can be done in 2 stages.

• Stage 1 is a flick stage, to filter out the least appropriate options. It is concerned with general attractiveness; the blurb; topic choices; visuals; layout. It’s impressionistic. May include price and availability.

• Stage 2 is a much closer scrutiny of one or a small number of materials that have passed the flick stage. This involves a checklist which makes the evaluation criteria explicit therefore “providing a common framework for decision making …. (in which) … “systematic attention is paid to all aspects considered to be important.”

(The other types are ‘In use’ and ‘Retrospective’ evaluation which are based on the effects of using the material, but these are not relevant to this task).

There are many published checklists. Which one should one choose?

Most of them have main categories under which a series of sub categories are listed. These main categories are related to: –

• How the materials fit with the syllabus/curriculum aims.
• How they fit with the students needs
• How they fit with the teacher (teacher suited)
• How they fit with specific areas of language and skills

Pitfalls in selecting published checklists:

• Context: Different criteria will apply in different circumstances” (Cunningham 1995). Local factors are very important. An example of a local factor might be, the availability of the target language outside the classroom, or the exam needs of the curriculum.

Roberts (1996) notes that because of the importance of context, checklist comparison is merely academic, so their main benefit is to promote careful consideration about how to evaluate and what system to adopt.

(Tomlinson provides help with this).

• When the checklist was produced: being up –to- date is important because of the changing landscape of SLA research and methodology practices.

Processes of evaluation:

Analysis comes first – of both the materials and the context. This is descriptive, answering the question ‘What is there?’ and is therefore mostly objective.

There are elements such as “what do users need to do? – the tasks and exercises, and ‘what are the aims and principles behind the sequencing and selection of language items and topics?’. With regard to context, ‘what is the syllabus fit, the language policy fit, the fit with the institution’s aims and exam needs?

Evaluation follows that, based on criteria set out in a checklist. It should be carefully designed and systematically implemented.

(More on this in chapter 5 below.)

Format comes last – This is concerned with ‘technical’ issues aimed at arriving at “a design which is fit for purpose”. It is concerned with issues such as: using open or closed questions; using a weighting for more important items; using a numeric or other rating scale; whether it’s a 1^st or 2^nd stage checklist; how convenient and time efficient is it to use?

The chapter goes on to report on issues to do with adaptation, supplementation and pre/in lesson evaluation and concludes thus:

“Careful selection procedures will not eliminate the need for adaptation, but they can reduce it. The same holds true for supplementation.”

Chapter 5: How teachers evaluate coursebooks – some studies.

This chapter reported on a range of evaluation studies in different locations. After the introductory section 1:

Section 2 reported on a few retrospective evaluation studies.

Section 3 reported on how selection procedures differ greatly across different contexts and found that while some were systematic, most were not.

Section 4 reported on how experienced teachers developed their own criteria. In this section it was interesting to note that these criteria were mainly the same criteria as those occurring in published checklists. An example is provided of a teacher generated checklist, asking:

Does the coursebook……

• Cater for all four skills?
• Introduce real life topics?
• Match the vocabulary needs and grammar level of the students?
• Have clear page layout
• Have a CD Rom for homework?
• Have a teacher’s book with information and resources?
• Give an introduction into the culture of the foreign language?
• Have appropriate content for the time available?

Section 5 discusses the relevance of in-use and post-use evaluations and notes that published studies of this type are very limited.

The chapter concludes that from looking at these studies, a systematic approach to coursebook evaluation for selection purposes “is the exception rather than the rule.”

Action points for task based on my interpretation of the reading:

• Look at some recently published checklists as a starting point (a flick test for checklists! ) and pick one or two for closer scrutiny
• Match the criteria found in them to our beliefs as teachers as to what should be included based on our principles (formed from teaching experience and knowledge of methodologies and SLA theory).
• Select the criteria for the ‘best fit’ to the above in both broad and sub categories – add in missing criteria or delete criteria that don’t fit our beliefs – (i.e. edit them)
• Consider the context – who is this course book designed for and how will it be used?
• Consider the technical ‘format’ issues for the framework – for maximum efficiency,ease of use, clarity, time available.
• Finalise the design; trial it.
• Apply the checklist to the selected coursebook and specific unit, after an introductory ‘stage one’ first impressions and general factors (e.g. price/availability).
• Evaluate the process – what would we change/refine/ recommend?

In comparing the process (above) with the one Tomlinson proposes (below) we found an integrated process that we used to devise our framework.

1. Brainstorm beliefs
2. Decide on shared beliefs
3. Convert the shared beliefs into universal criteria
4. Write a profile of the target learning context for the materials
5. Develop local criteria from the profile
6. Evaluate and revise the universal and local criteria
7. Conduct the evaluation

(Tomlinson 2013: 44)

Here is the link to our presentation which contains a framework based on our shared principles:

http://prezi.com/jizdkkih56k3/pre-use-evaluation-of-coursebook/

Concluding remarks:

Comparing our presentation to those of the other groups was interesting as we seemed to approach the task in slightly different ways. Our criteria were grouped into categories where others weren’t; we used statements to agree or disagree with, where others used questions.  I think though that the key message we all took away from completing this task and listening to the other presenters was that coursebook evaluation should follow a process.  That is, it should be systematic. Equally important it should be contextualised – fit for purpose. Furthermore, the criteria for the checklist that forms the backbone of any evaluative framework also needs to be evaluated. 

Checking your criteria:

a) Is each question an evaluation question?

b) Does each question only ask one question?

c) Is each question answerable?

d) Is each question free of dogma?

e) Is each question reliable in the sense that other evaluators would interpret it in the same way?

(Tomlinson & Masuhara , 2004: 7)  – as presented on the seminar slide for week 3 #35.

References:

McGrath, I. (2013) Teaching Materials and the Roles of EFL/ESL Teachers: Practice and Theory

Tomlinson, B. (2013) Materials Evaluation. In; Tomlinson, B. (ed). Developing Materials for Language Teaching (2nd ed).

Mishan, F. & Timmis, I. (22015) Materials Development for TESOL, chapter 4.

Seminar number two was primarily a task-based experience; we were charged with the task of coming up with some principles of ELT practice that we believed to be central to our beliefs about how languages are learned (Hall 1995 in Tomlinson 2012) and upon which we would base our (theoretical) framework for materials design. Individually we produced our lists and then shared these in small groups and compared our findings. They were not wildly different but neither were they exactly the same. To complicate matters, a further layer of published ‘design principles from the literature’ was added to the mix and we were asked to eliminate the duplicates and arrive at a list of 16. We were polite with each other, listened to each others’ points of view, made our cases for different elements and compromised in places. It wasn’t easy and certainly provoked much discussion.

The process we experienced seemed to aptly reflect the essential nature of trying to find ‘universal principles’. We all had different backgrounds – in English language teaching; in our experience of this course and our reading so far for this module; in our own educational and language learning experiences; and in our ages and countries of origin; (to name just some of the key variables). In other words, we all brought different ‘contextual’ baggage with us to the table of principles selection. This was the fundamental idea I took away from this seminar, that yes, universal principles based on second language acquisition theory and pedagogic theories can be devised, honed and prioritised, but as teachers we all have different ideas about those priorities and different teaching situations in which we see ourselves applying them. An effective design framework, for a ‘constituency bigger than just me an my learners today’ – and I think these were Paul’s words, needs to capture the most salient ‘universal’ principles and allow for ‘contextualisation’ – or adaptation to local issues. In Tomlinson’s view, (State-of-the-Art Article 2012: 153):

“The ongoing evaluation of the developing materials should be driven by a set of agreed principles, both universal principles applicable to any learning context and local criteria specific to the target learning context(s).”

In that same publication he reviews how some authors set out principles on which to develop materials. Flores, (1995) used 5 principles, Penaflorida (1995) 6, while Tomlinson (1998b and 2011b) suggested 15 principles based on SLA theory and his experience of teaching. To what extent though, have the majority of materials developers used design frameworks based on these or other principles? Disappointingly, not much, according to the literature. Historically, it seems, the main ‘tools’ for materials design have been to replicate or adapt previous materials, to rely on creative inspiration during the writing process, and to refer to individual repertoires of ‘what worked for them’. Our class discussion on this issue left some of us wondering if they used principles at all, in the absence of any explicit reference to them. However, perhaps giving them the benefit of the doubt, we concluded that, as expert material writers who were also (or had at some point been) teachers, their principles had become internalised or implicit. This conclusion was supported with subsequent reference to Hadfield (2014; 320-359) who examined and reported on her own and other materials writers’ experiences, finding that just because these principles remain unarticulated does not mean they don’t exist. Furthermore, while the expert writer might not use an explicit set of principles,  novice and developing writers would find having them invaluable.

Writing on task design, Johnson, K. (2003) draws from specific research to identify what makes an ‘expert’ materials writer, identifying the following key behaviors:

• Concrete visualisation of activity for student and teacher
• Ability to abandon what’s not right
• Does things one at a time – and spends time analysing the problem and reviewing it
• Highlights important considerations (principles)
• Looks at breadth first, then depth in design
• Makes higher level decisions before lower ones
• Identifies consequences
• Constantly reviews, with a view to reject, fix or maintain
• Is sensitive to learner context
• Explores post task types
• Uses repertoire
• Creates choice possibilities
• Complexifies task beyond necessity

From his analysis we can see that expert writers approach design in a way that draws on their implicit knowledge and results in a patten of behaviors with which they tackle design problems. Some of them also reflect principles more overtly – such as being sensitive to learner context.

Tomlinson’s key principles based on SLA research findings and taken from Tomlinson (2008b) include:

• the language experience needs to be contextualised and comprehensible
• the learners need to be motivated, relaxed and positively engaged
• the language and discourse features available ….need to be salient, meaningful and frequently encountered
• the learners need to achieve deep and multi-dimensional processing of the language

In Tomlinson (2013) these were listed:

• meaningful exposure to language
• effective and cognitive engagement
• mental resources used in L1
• noticing how L2 is used
• contextualised and purposeful communication
• interaction encouraged
• focus on meaning

Coming into this seminar I had just read the papers in which these principles were discussed. I had recently completed the second language acquisition and methodology modules of this course and I have 5 years of my own teaching experience to draw from. This all fed into the task we were set in class as I produced my own list and reviewing the ‘design principles’ supplied.  I’m not sure which aspect influenced me the most – but then that’s the nature of schema.  A bit of this and that is added to what’s already there to build a new set of understandings.  And this is one reason why no two individuals’ ‘sets of principles’ will ever be exactly the same. So, not surprisingly, my group’s (eventually agreed) list of  principles was different to the other groups’ lists  and theirs to each others. Here for the record, is my group’s list which was just 15 in the end: 

•  have engaging content
• be authentic and relevant to real life
• challenge students
• be flexible
• recognise the need for long-term goals
• well presented
• Student centred
• integrate skills work
• Meaningful and accessible
• take into account that learners differ in affective attitudes
• require and facilitate learner self investment
•  give multiple examples
• direct learners’ attention to linguistic features of input
• provide opportunities for outcome feedback
• reflect the complex nature of the learning task and made manageable

Where to from here?  Given the unlikelihood of finding a comprehensive list of principles that suits all circumstances, it is perhaps equally important to focus on the process of deciding upon those principles suited to the time, place and purpose of use.  This is the essential idea behind a framework for materials writing, and Jolly and Bolitho (2011) set out a ‘principled, practical and dynamic framework‘ as a guide to this process.  This starts with the identification and then the exploration of needs in which the objectives of the materials need to be clearly defined; it continues with the contextual and pedagogic realisation of materials, such as cultural appropriateness and how the learners might use the materials; followed by production and student use, covering issues of trialling and piloting; and finally (but that’s not actually where it finishes) evaluation against objectives.  There isn’t really a ‘finally’ because once the evaluation stage is reached the process starts over again from the beginning – it’s iterative.  Our seminar slides flesh out this process  (week 2 in class presentation slides 45-61:) as does the literature in which it was reported (Tomlinson 2013).  In week three we deal more thoroughly with frameworks and process as they relate to EVALUATION.  To prepare for that we were put into new groups and given the following question and task:

•  What framework most clearly reflects your approach to materials evaluation?
• Arrive at a materials framework and apply that to a coursebook.

References:

Johnson, K (2003) Designing language teaching tasks.

Tomlinson, B. (2012) Materials development in language learning and teaching. Language Teaching 45.2.

Tomlinson, B. (2013) Developing principled frameworks for materials development, in Tomlinson, B (ed)  Materials for language teaching (2nd ed).

Hadfield, J. (2014) Chaosmos: spontaneity and order in the materials design process., in Harwood, N. (ed) English language teaching textbooks: content, consumption, production.