Throughout this blog the idea of evaluating tasks has been central to the on-going evolution of material design. Ellis (1998) states that evaluation can be used to determine the pedagogical merits of one activity or it can be used to facilitate the process of curriculum development. This is what Ellis calls micro-, and macro-evaluations.

Macro-evaluation can be defined as evaluation that seeks to answer one or both of the following questions:

1. To what extent was the programme/project effective and efficient in meeting its goals?
(Accountability evaluation)

2. In what ways can the programme/project be improved?
(Development evaluation)

Macro-evaluations are less likely to have much concern for the teachers in the classroom, whose focus is on the learning and development of the students. They are less likely to look at a programme as a whole, and will focus on whether specific activities and techniques appear to ‘work’ in the context of a particular lesson. A teacher’s macro-evaluation is probably going to manifest from a whole host of micro-evaluations, taking place throughout the set period of time. Micro-evaluations are characterised by a narrow-focus on specific aspects. In terms of materials, we might ask whether a particular activity is effective or efficient in achieving learning goals. Although, to determine these two states as satisfactory is difficult to do without a systemic and principled approach to evaluation.

As mentioned in my previous post, re: evaluations of course books and materials. The evaluation can be conducted at one or at all three stages: Pre-use, during use, and post-use. Similarly, tasks can be evaluated at different stages of planning, use and reflection. Ellis (1989) suggests that you can start with an external evaluation (pre-use) examining the tasks claims about: intended audience, proficiency level, the context in which writers of materials intend them to be use, the way the language has been organised into teachable units, and the writer’s’ views on language and methodology. This is can be followed by an internal evaluation (in-depth) investigation of the aspects of presentation of the materials, the grading and sequences, the kind of texts used, and the relationship between exercises and tests.

With some many considerations and the many different reasons for evaluating a task, a balance needs to be found based on the purpose and context. When looking at using authentic texts for example, there is a positive aspect of naturally occurring language input, but this may have an equal weighting of the negative considerations for vocabulary overload. The overarching theme (throughout the blog) being that it must be the needs of the learners that are at the core of these considerations.

There are seven dimensions of evaluation put forward by Ellis (1989) that are applicable to both micro- and macro-approaches:

I. Approach
II. Purpose
III. Focus
IV. Scope
V. The evaluators
VI. Timing
VII. Types of information

Taking these seven dimensions into account, I will look at how they would apply to a micro-evaluation of a task.

The approach is about looking at the method or beliefs that the task is created for. Approaches are not polarised but are part of a large spectrum. In an attempt to simplify this Ellis (1998) bookends the spectrum with two generalised models: focused and unfocused tasks i.e. more behaviourist and objective moving toward communicative and sociocultural perspectives. The approach needs to classify:

• The type of input (e.g. verbal or nonverbal) and
• The procedure i.e. activities the learners are to perform (experiential or elicitative language use).
• The language skills the students are going to be using (receptive or productive).
• The intended outcome of the task, that is, what the learners will have done on completion of the task.

The approach informs the second dimension, which is to reflect on the purpose of the task. This is also connected to the timing dimension. The purpose is trying to measure and give feedback on learning, or to develop and reflect on the task (accountability and development). A full description of the purpose requires objectives of the task, ‘real’ world or pedagogic to be clear. Real world is asking learners to approximate in class what is required of them outside, whereas pedagogic requires some language activity which is not found in the real world but does facilitate language acquisition. In the context of micro-evaluation it is difficult to establish whether the learners acquired new linguistic knowledge or improved fluency.  Therefore it may be necessary to evaluate whether the hypothesised methods effects on language acquisition have been achieved instead of defined learning objectives. Commonly, timing looks at whether the task is going to be formative or summative in its feedback. Micro-evaluations can do both depending on when the task is completed within the curriculum. The task’s timing will go some way to  aiding the data that determines its focus.

Focus is looking at the effectiveness and efficiency of the task. This appears to be a difficult dimension to evaluate. Firstly, it may be necessary to judge what and how well something has been learned upon the task’s completion. If the learner shows gains in those aspects of language proficiency, which were the starting objective, then the task can be considered ‘effective’. If something is learnt is hard to clarify and requires scope. To glean if something is efficient it would need to be compared to other tasks that have similar structure and objectives, which is very difficult to do. It must also use the same types of information. This data can be drawn from traditional objective models such as test scores. While, a more holistic approach could be considered, where the evaluation is directed a greater variety of information such as: documentary information of student work (coursework), self-reflection and discussion of progress and development, and observational feedback from the classroom.

To recognise which of these types of information is needed, one must address the scope. This tries to value if the task meets its ‘stated’ goals, or whether to examine the goals themselves (are the goals appropriate for the learning question). The internal scope uncritically accepts the goals of the programme, while an external scope submits the goals for critical scrutiny. External evaluation of scope will inquire if unpredicted learning has taken place. Two types of information collection should be considered: information relating to the outcome of the task that is the focus of the evaluation. Secondly, the data relating to whether, the learners can perform the task (or similar ones) without pedagogic support?

Finally, the evaluators themselves need to be considered. Who is conducting the evaluation? Are they within the institution or and external candidate? Internal evaluations may be considered more effective as it needs involvement from ‘all stakeholders’ who have investment. However, outside ones may offer a more objective, ‘fresh’ perspective and may give more credibility. Another option is for an advocacy evaluation, this asks various parties to argue their case for their own bias and stances. It can be argued that this is more likely to foster development, but only if that is the goal of the evaluation (Ellis, 1998). An evaluation needs clear distinctions between its conclusion and recommendation. In the cyclical nature of materials design, recommendations should be given based on the conclusions drawn from the seven dimensions all being considered.

Evaluations are very difficult for me to comprehend out of context. It is important to remember that it is the clarity of what is being evaluated that must come first. The systemic and principle approach can only work if it is guided by its purpose. I feel that the more I engage with task creation and evaluate the approach and scope of my tasks. There are going to be suitable for my students, and the curriculum aims and are more likely to satisfy the focus of macro-evaluations both internally and externally. Evaluation will also allow for more reflection and development of my teaching beliefs, style and effectiveness.

References

Ellis, R. (1998) The evaluation of communicative tasks. In: Tomlinson, B. (ed). Materials development in language Teaching. (2nd ed) Cambrdige: Cambridge University Press. Pp212-235.

Ellis, R. (2011) The evaluation of communicative tasks. In: Tomlinson, B. (ed). Materials development in language Teaching. (2nd ed) Cambrdige: Cambridge University Press. Pp217-238.

This seminar (Week 7) looked at what a task is. At the start of this course a material was defined as anything that aids the learning or acquisition of language. Therefore, what is a task? Ur (1988) describes a task as something that activates engagement with the materials being used. It should have a clear objective, language system or skill usage. To stimulate engagement the task must demand the student to actively participate receptively or productively (or both).

Ellis (1998) describes the different components of a task:

• An activity of some kind
• It has specified outcomes
• Language comprehension or production or both
• Learners focus their attention principally on meaning rather than form

There is clear overlap here with Ur’s description. However, Ellis excludes the ‘typical’ grammar exercises found in course books taking a Present, Practice, Produce (P.P.P) approach to language learning. This is due to their requirement for a focus on form. However, it could be argued that language awareness activities, in general, could also fall under this descriptor and may not be ‘seen’ as a task if the learners are attempting to discover linguistic phenomena.
Shalvelson & Stern (1981) as cited in Nunan (1989:47) offer another theory of what a task is composed of:

1. Content – subject matter to be taught
2. Materials – the things that the learners observe or manipulate
3. Activities – the things which the learners and teachers will be doing in the lesson
4. Goals – the teacher’s general aim of the task
5. Students – their abilities, need and interests
6. Social community – the class as a whole and its sense of ‘groupness’

This may seem more comprehensive but it does appear to cover a lot of the same themes. If a material is anything that aids the acquisition and knowledge of language, then a task is the specific description of how and who will be interacting with the material and what they are expected to do with it. In some ways a material is a stick and task is a fishing rod (as the old analogy goes).

Another dimension that Ellis (2012) has added to a task descriptor is the idea of them being focused or unfocused. The former affords ‘discrete’ or ‘controlled’ language features to be practiced through communication, while the latter aims to have communication as the foundation for language to be acquired at the point of need. This notion brings into focus two different approaches to material design and teaching. Task-supported Language Teaching (TSLT) is traditionally found in the pages of course books, ones with a structural syllabus using PPP. Materials are divided into units that are sub-divided into skills, language, and functional tasks, as a means of scaffolding knowledge in preparation for a final activity. Task-based Language Teaching (TBLT) supports the idea of communication with unfocused activities and tasks that create a ‘need’ for certain language functions.

The TBLT supports a socio-cultural perspective that aligns itself more with my own beliefs of ELT. However, this is may not be as simple as it first appears. The courses that I teach on are academic in content and are judged on coursework receptive and productive tasks that are scored using a uniformed marking description. This means that pressures are put on both students and staff to try and reach certain targets. With that in mind more rigid and focused approaches need to be taken in order to meet deadlines. Having observed the teachers that I work with and by reflecting on my own practice we find ourselves in a situation of a more eclectic and mixed method approach using both TSLT and TBLT. This suggests to me that the needs of the students and situation come first and leaves less room for rigidity of approach. The coursework tasks are uniformed and are marked with marking descriptors. Yet it is still the teacher who has control (in my situation) prior to the assessment and they can create and adapt tasks accordingly. Therefore, teachers can set the purpose and scope for their task evaluations as means of development and reflection.

Task types and taxonomies

There are various types of tasks that an ELT teacher encounters through published materials and peer created tasks. Different task types lend themselves to more focused or unfocused approaches and this is quite apparent when flicking through any published course book. Loosely speaking they could be set along a Kline (see below). There are of course more task types to consider. As mentioned above several tasks will make up a single lesson and this may require a mixed approach of focused and unfocused activities.

Gap-fill – Matching – Compare and Contrast – Reformulate – Project work

Focused Unfocused

Over the years I have encountered Bloom’s taxonomy of learning and teaching. During an in-house training session at Bellerbys College I was given this helpful description of applying Bloom’s Taxonomy to task design.

Blooms taxonomy and task design HO

Each level/stage moves from the more focused mastery tasks to the more open and unfocused developmental ones. For me it perfectly sums up the process and the type of tasks I create and adapt for my students throughout a term. It also makes it clear that the tasks are not more communicative or more difficult problems to solve. It is describing how learner becomes more autonomous by using the skills they have mastered and applying them to the unfocused and more cognitively demanding tasks. This is, in many respects, similar to the Zone of Proximal Development [put forward by Vykotsky) that asks learners to push themselves beyond their current level to acquire new knowledge.

Now having looked at what a task is, it is important to address the issues and considerations of evaluating a task.

References

Ellis, R. (1998) The evaluation of communicative tasks. In: Tomlinson, B. (ed). Materials development in language Teaching. (2nd ed) Cambrdige: Cambridge University Press. Pp212-235.

Ellis, R. (2011) The evaluation of communicative tasks. In: Tomlinson, B. (ed). Materials development in language Teaching. (2nd ed) Cambrdige: Cambridge University Press. Pp217-238.

Nunan, D. (1989) Designing Tasks for the Communicative Classroom. Cambridge University Press.

Ur, P. (2009) Grammar Practice Activities: A practical guide for teachers. (2nd ed) Cambridge: Cambrdige University Press. (See section 3 – Activties pp. 11-26)