Visuals and text: the power of multimodality

Do visuals play an important role in ELT? Why?  And how can we make the most of them to exploit their power?

Pictures have the power to convey information, ideas and feelings.  They have been around far longer than the written word. In ELT they are extensively used for a number of important reasons summarised succinctly in Duchastel’s taxonomy for illustrations in instructional texts as follows:

  • affective:provided to enhance interest and motivation
  • attentive: intended to attract and direct attention
  • didactic: intended to facilitate learning by showing something difficult to convey in words
  • supportive: provided for less able learners
  • retentional: provided to facilitate memorisation
29 Sep 1932 --- Construction workers eat their lunches atop a steel beam 800 feet above ground, at the building site of the RCA Building in Rockefeller Center. --- Image by © Bettmann/CORBIS

29 Sep 1932 — Construction workers eat their lunches atop a steel beam 800 feet above ground, at the building site of the RCA Building in Rockefeller Center. — Image by © Bettmann/CORBI

Here’s a picture I’ve used in the past to help with an understanding of  ‘head for heights’.  If your knees feel wobbly looking at this, you probably don’t have one! I used it for didactic reasons as ‘head for heights’ is difficult to explain in words.

 

 

We use images because we know that ‘meaning making’ is a complex business and that “language  is but one of the communicative resources through which meaning is (re)made” (Early, M. et al. 2015).  Modes work together to provide meaning, validate it, and deepen understanding.  An image exploits the visual mode and images can include static photographs like this one; graphic images and cartoons; and moving images, namely video.  To elicit the same idea of ‘head for heights’ and its opposite ‘fear of heights’ I found this YouTube video which serves the same purpose  – to illustrate an abstract idea.

It’s interesting from a practical point of view to note here how I found both of these.  The first I simply googled on google images as I did when I used it the first time. However, I didn’t check if it was copyright protected at the time.  If I had wanted to reproduce or alter it in any way I might have breached copyright. Walker and White (2013) have some useful advice here in recommending the Creative Commons website which filters the images searched for, by those with permission to be used for non-commercial purposes. Not only that, they suggest that it generates more diverse images than a google search and that this could be useful in illustrating abstract ideas – such as the one I was attempting to convey.  This is what it says on their home page.

“Creative Commons is a non-profit organisation that assists authors and creators who want to voluntarily share their work, by providing free copyright licences and tools, so that others may take full and legal advantage of the Internet’s unprecedented wealth of science, knowledge and culture.” (https://creativecommons.org). 

I used it to run my search again and that’s how I found the YouTube video above. On cross checking this process however, I found it just as easy to use the creative commons filter under ‘features’ directly in YouTube. The same applied to google images under their drop down tabs ‘search tools’ and ‘usage rights’.  In all cases it changed the images I was offered but as you can see below, and with the video, that doesn’t matter, as they do the job just as well. This was a useful lesson on sourcing images for me.

Ein waghalsiger Pressephotograph ! Gefährlicher Stand eines Pressephotographen an der äussersten Kante des 300 m hohen Deutschlandsenders in Königswusterhausen bei der Herstellung einer Zeitungsaufnahme.

Ein waghalsiger Pressephotograph !
Gefährlicher Stand eines Pressephotographen an der äussersten Kante des 300 m hohen Deutschlandsenders in Königswusterhausen bei der Herstellung einer Zeitungsaufnahme.

 

 

 

images class floor

Returning to the consideration of why images are important in ELT, it is necessary to understand the role of multi-modality.  Communicative resources or modes, are said to include “image, gaze, gesture, music, movement, speech and sound-effects” (Jewitt & Kress, 2003 p1, cited in Early, M. et al 2015).  And as Walker and White (2013: p84) point out, “All learning and teaching involves the use of a number of modes, speaking, writing, images, and movement, which give learners the opportunity to understand and combine information represented in different ways.” (More on this in my next post  – Sound and Vision).

This idea, particularly as it relates to images, is  supported by the work of Paivao (2006), who proposed a theory of ‘dual coding’ in which he claimed that “words and images are processed and stored in different ways by the brain,….and that images that gave the same messages as words could help learning and memorisation” (Walker and White, 2013: p80). In other words they function ‘retentionally’.  This research however has been both supported and contested, so Walker and White advise that assumptions about the positive effects of combining written texts and images should not automatically be made, although combining images and speech is more fully supported. Visuals then, might either enhance or distract understanding when used with written text, so it’s important as teachers to use them correctly. In an article that pre-dates Paivao’s work but covers similar research and associated arguments, Canning (1998) is reported thus:

Pictures help individual learners predict information, infer information, deduce information, analyze today’s world so that it can be brought into today’s classroom and offer social settings which can immerse or expose the learner to new ideas or further promote an already created setting. If a visual is used in a testing or teaching situation it can enhance clarity and give meaning to the text or to the message being communicated. 

(Canning-Wilson, C. (2001), in ELT Newsletter 48).

And in the same article, this author refers to learner preferences for visuals to be: in colour, related to a story, related to experiences, associated with events, people, places, and things that are familiar in some way.  Going on to highlight some of the ways pictures are used incorrectly, thus making them ineffective as learning aids, the following list is cited from the author’s previous work:

  1. the use of violent scenes
  2. too many distractors
  3. too crowded or cause an overwhelming effect of information
  4. too small or not clearly defined
  5. stereotyped images
  6. poor reproduction
  7. not related to text
  8. irrelevant captioning
  9. offers too much information
  10. unclear picture which doesn’t compliment the text
  11. poorly scaled illustration
  12. cluttered composition that is not aesthetically meaningful

In our seminar session on this topic, colleagues did presentations on different aspects of the relationship between visuals and ELT:

  • on how info graphics and pictocharts can be exploited.
  • on how to evaluate photos, images and graphics using Duchastel’s taxonomy and considering some of the potential problems mentioned in the list above.  
  • and on how they have been used in coursebooks  – using the above criteria again but also considering  whether the visuals have just a functional role or whether they provoke a mental and linguistic reaction.

Hill, D.A. (2013), looked into this latter element and reported some interesting findings. He found, in his examination of 3 intermediate student courseooks, that over 50% of their pictures were purely decorative, which he describes as a “great waste of effort” (Hill, D.A. 2013, p163, in Tomlinson, B. (ed).  Even among those that performed a useful function – to define meanings or facilitate tasks for example, he found that they were useful only at a relatively low level, in that they failed to stimulate students “to use the language at their disposal creatively, starting from the pictures” (Hill, D.A. 2013,p162).  Hill refers to Corder, P (1966) to make the distinction between ‘talking about’ and ‘talking with’ a picture.  While the former is factual and descriptive, useful in some ways: it is limited. The latter could, he says, flow from the former, thus extending the usefulness of a single picture and allowing “learners to bring their own reality to a lesson” (Hill, D.A. 2013,p165).  In this way the visuals are a starting point, rather than and a means to 1 specific end.  They can create, as Keddie, (2009) points out, a space that can be filled by language.

Referring back to one of my pictures: I would choose the one of  the small boy in the glass skyscraper because it is colourful, sharp and uncluttered. Also, because it has no distracting and irrelevant text attached. 

images class floorI can still use it didactically to help make meaning.

I  could also use it descriptively to elicit and recycle associated vocabulary – skyscrapers,floors, lifts, view, clouds, city, for example.

 

Additionally however, I could  use it as Hill and others suggest, to allow  students to talk ‘with the picture’ – to bring their reality to it.  And to do that, I wrote a few simple prompts to accompany my new ‘legal’ picture.

  • Have you ever been up a very tall building or to a very high place?
  • Where and when was it? Who were you with?
  • What was it like and how did you feel?
  • Imagine you are this little boy. What’s the story of your day? How does this experience affect your life?

Having evaluated the choice of visual, decided how I might use it to elicit meaning, decided how I might uses it descriptively to talk about the picture, and written some prompts to enable students to talk with the visual, I have got an almost useable activity for a lesson.  I don’t have a context in which to use it though – no student need exists at present, and would have to convert it into classroom ready material and evaluate and amend it accordingly.  But it’s a start!

References:

Canning-Wilson, C (2001)Visuals & Language Learning: Is There A Connection? In ELT Newsletter 48, February 2001.

Early, M. et al. (2015) Multimodality: out from the margins of english language teaching. TESOL quarterly 49 (3).

Hill, D. A. (2013), The Visual Element in EFL coursebooks in Tomlinson,B (ed) Developing Materials for Language Teaching (2nd edn).

Walker, A and White, G (2013) Technology Enhanced Language Learning: Connecting Theory and Practice. Chapter 6.

Leave a Reply

Your email address will not be published. Required fields are marked *