Assessment

11 Diagnostic Tasks

John M. Levis

This chapter offers some suggestions for diagnosing pronunciation errors for individual students. For diagnosing groups of students or learners with less advanced reading skills, see the chapter written by Levis & Echelberger (2022). The examples included in the chapter are suggestions for how to diagnose particular features of speech. Not all sections will be important for all situations. For example, if you are only teaching vowels, you can just do the word reading. If you want an overall impression of reading intelligibility, just use the reading and/or a free speech sample.

Overall: This section gives an overall impression of the student’s abilities in relatively free speech. Reading aloud can sometimes give a different impression of a student’s abilities than free speech, especially with fluency. Transcribing each answer in full gives a strong view of how intelligible a speaker is.

Dictation: The listening diagnostic is a dictation. It tries to get at the perceptual abilities of the student. What are they actually hearing in normal speech? Are they missing function or content words? Do they not hear grammatical endings accurately? Most pronunciation diagnostics do not test listening, though it is a critically important area.

Reading: This reading passage is a general diagnostic. The one we use is from Celce-Murcia et al (2010). We use it to listen for specific areas only, because it is overwhelming to use it to listen for every possible target: consonants, word stress, intonation, and grammatical endings. Vowels, linking, rhythm and prominence are tested elsewhere.

Words (Vowels): I call this the “I dare you!” way to diagnose vowel issues. I find that in longer texts, vowel issues are hard to hear. With minimal pairs, the problems are more obvious to everyone. Students see the pairs and know there is a difference. They make the difference if they can. Teachers also can hear pronunciation problems more easily. It’s completely non-contextualized, but I’ve not found a better, more painless way to diagnose vowels. There’s nothing hidden in the task.

Phrases (Linking): This section allows the teacher to listen for linking with a couple of sentences for each linking issue. Each phrase is read two times, with the second one counting. (On short phrases, there’s little chance to get a rhythm going consistently with a single reading, so reading it twice ensures students get a better chance at producing the phrase smoothly.)

Sentences (Rhythm): This section tests phrase rhythm. It involves two separate issues. First, it is intended to help listen for the presence of a reduced vowel in function words. Second, it is intended to assist in listening for the lengthening of stressed syllables in content words.

Dialogues: These dialogues focus on prominence (sentence stress) in new information and in contrasts. Prominence can be hard to diagnose since it can depend on speaker intent, but these dialogues highlight obvious uses of prominence that are easy to hear.

Free Speech: In this section, students talk about their favorite holiday celebration. By transcribing or listening to the first minute of their speech, it is possible to notice spontaneous speech errors.

Free Speech (Retelling a Video Story): In this section, students narrate the content of a short video story. We usually transcribe the first minute of their speech (or have students transcribe it) to complement other diagnoses.

 

Task: Overall impressions of speaking and pronunciation abilities

In an interview format, the student answers the questions as you record. Either rate the student as they speak, or transcribe what the student said in normal spelling. The answers give the teacher a chance to describe global impression of the student’s pronunciation and speaking abilities in free speech. We suggest listing any speaking or pronunciation patterns that seem odd or wrong.

  1. What is your name and where are you from?
  2. Tell me something interesting about where you are from, about your hometown.
  3. What are you studying? Can you tell me a little about it?
  4. Why do you want to improve your pronunciation?
  5. If you could travel anywhere in the world, where would you go? Why?

Listen and rate the student’s speech according to intelligibility, comprehensibility, and accentedness (Munro & Derwing, 1995)

Intelligibility

How you understand the words that are spoken, i.e., how many words/phrases can you not understand?

Number of words/phrases not understood ____

Comprehensibility

How much effort do you have to put forth to understand the student? Use a 1 – 9 scale:

Student is always easy to listen to   1  2  3  4  5  6  7  8  9   Student is frequently hard to listen to

Accentedness

How pronounced is the student’s accent relative to a local accent? Use a 1 – 9 scale:

Student has barely noticeable accent  1  2  3  4  5  6  7  8  9  Student has a strong foreign accent

Comments about the learner’s speech:

 

 

Task: Dictation

With two native speakers, record this dialogue at normal speed, then play it for the students two times. The first should be played through fully while the students listen. The second time, pause after each turn so the students can write what they hear.

You can analyze the errors they make in their dictation to decide what kinds of language features they had difficulty hearing accurately. Common types of listening errors (mostly unstressed function words) are underlined.

A: Excuse me, could you give me a hand?

B: I’m not sure. What did you need?

A: I’d like some help moving this table. It’s very heavy.

B: Where’d you want to move it?

A: I wanted to move it up to the next floor.

B: That’ll be too much for just two of us. We’ll need more people.

Original Dialogue Recording:


There are two example dictations in this section, one by a Spanish speaker and the other by a Chinese speaker. In both cases, the student dictations show consistent trouble with hearing unstressed function words and contractions. For example, both learners did not hear the word “I’m” in the phrase “I’m not sure”. Speaker B’s second turn, “Where’d you want to move it?” showed the contraction of “did” was missed by one learner and heard as “would” by the second.

Student Example (Spanish) Student Example (Chinese)
A: Excuse me, could you give me a hand? A: Excuse me, could you give me a hand?
B: ___ not sure. What is your need? B: ___ not sure. What did you need?
A: I’ll like some help moving this table. It’s very heavy. A: I would like some help move this table. It’s very heavy.
B: Where ___ you want to move it? B: Where would you want to move it?
A: I want ___ to move it up to the next floor. A: I want__ to move that __ to ___ next floor.
B: That’ll be too much just for the two of us. We ___ need more people. B: That would be too much for just two of us. We___ need more people.

Comments about learner’s listening abilities:

 

Task: Reading

This passage is from Teaching Pronunciation (Celce-Murcia, Brinton & Goodwin, 2010). We use it for four types of possible errors: consonants (not highlighted), grammatical endings -ed and -s (underlined), word stress (possible words in bold), and final intonation (with arrows at the ends of sentences). It can be used more generally for all types of errors, but it is hard to listen to more than one feature at a time.

  1. Is English your native language?
  2. If not, your foreign accent may show people that you come from another country.
  3. Why is it difficult to speak a foreign language without an accent?
  4. There are a couple of answers to this question.
  5. First, age is an important factor in learning to pronounce.
  6. We know that young children can learn a second language with perfect pronunciation.
  7. We also know that older learners usually have an accent, though some older individuals also have learned to speak without an accent
  8. Another factor that influences your pronunciation is your first language.
  9. English speakers can, for example, recognize people from France by their French accents.
  10. They can also identify Spanish or Arabic speakers over the telephone, just by listening carefully to them.
  11. Does this mean that accents can’t be changed?
  12. Not at all!
  13. But you can’t change your pronunciation without a lot of hard work.
  14. In the end, improving appears to be a combination of three things: concentrated hard work, a good ear, and a strong ambition to sound like a native speaker.
  15. You also need accurate information about English sounds, effective strategies for practice, lots of exposure to spoken English, and patience.
  16. Will you make progress, or will you give up?
  17. Only time will tell, I’m afraid.
  18. But it’s your decision.
  19. You can improve!
  20. Good luck, and don’t forget to work hard

Sentences 1-7 (Consonant listening, Korean speaker)

A transcript of the audio passage with the text marked to denote where pronunciation needs some work.
The first seven sentences of the reading passage are annotated to show the errors in the pronunciation of consonants. A longer description of the image is provided below.

In sentence 1, “Is English your native language?”, “native” is pronounced as “lative” and “language” is pronounced as “lang uh gee”. In sentence 2, “If not, your foreign accent may show people that you come from another country,” the word “foreign” is pronounced as “pour in” and “that” is pronounced “dat”. In sentence 3, “Why is it difficult to speak a foreign language without an accent”, “foreign” is again pronounced “pour in” with a [p] rather than [f]. “Language” is again pronounced the same as in sentence 1, as “lang uh gee.” In sentence 4, “There are a couple of answers to this question, ”answers” is pronounced “answer.” In sentence 5, “First, age is an important factor in learning to pronounce,” “first” is pronounced “first uh” and “learning” starts with an [r] sound instead of an [l] sound. Sentence 6 says, “We know that young children can learn a second language with perfect pronunciation.” The word “that” is pronounced without a final [t] sound and “perfect” is pronounced “purr pecked.” In addition, “language” is pronounced “lang edge.””. Finally, sentence 7 says, “We also know that older learners usually have an accent, though some older individuals also have learned to speak without an accent. The word “that” is pronounced “dat,””, the [r] sounds in older” and learners” are not pronounced, “usually” is pronounced “use a Lee” and the word “learned” has two missing consonant sounds, [r] and [d].

 

In Sentences 8-13, two diagnostic listening tasks are illustrated. The first is the pronunciation of -s and -ed endings, and the second is word stress.

Sentences 8-13 (-ed and -s endings, Spanish speaker)

A transcript of the audio passage with the text marked to denote where pronunciation needs some work.
The next few sentences of the reading passage are annotated to show the errors in the pronunciation of consonants. A longer description of the image is provided below.

In sentence 8, “Another factor that influences your pronunciation is your first language,” the speaker pronounces the -s ending of “influences” correctly. In sentence  9, “English speakers can, for example, recognize people from France by their French accents,” the plural -s of “speakers” and “accents” is not pronounced. In sentence 10, “They can also identify Spanish or Arabic speakers over the telephone, just by listening carefully to them,” the final -s of “speakers” is not pronounced. In sentence 11, “Does this mean that accents can’t be changed?,” the final -s of accents is not pronounced, but the final -ed of “changed” is correctly pronounced. In sentences 12, 13, and 14, “Not at all! But you can’t change your pronunciation without a lot of hard work. In the end, improving appears to be a combination of three things: concentrated hard work, a good ear, and a strong ambition to sound like a native speaker,” all -s and -ed endings (in the words concentrated, appears, things) are pronounced correctly.

Sentences 8-13 (Word stress, Thai speaker)

The same sentences as in the last example are transcribed, now highlighting areas where word stress could be improved.
Sentences 8 through 13 are annotated to show word stress errors in pronunciation. A longer commentary of the notes in the image is provided below.

The recording is from a speaker of Thai. The targeted pronunciation features for this section is word stress, using a selection of multisyllabic words in the section. In sentence 8, “Another factor that influences your pronunciation is your first language,” the word influences is stressed on the third syllable rather than the second. The word “pronunciation” has the correct word stress. In sentence 9, “English speakers can, for example, recognize people from France by their French accents,” the word “recognize” is mispronounced with stress on the second syllable. Sentence 10, “They can also identify Spanish or Arabic speakers over the telephone, just by listening carefully to them,” has four words targeted. The speaker mis-stresses “identify” (4th syllable, not 2nd), “telephone” (3rd syllable not 1st), and “carefully” (2nd syllable, not 1st). The word “Arabic” is stressed correctly on the 1st syllable. Sentence 14 also has four targeted words: improving, combination, concentrated, and ambition. The sentence reads “In the end, improving appears to be a combination of three things: concentrated hard work, a good ear, and a strong ambition to sound like a native speaker.” The word “improving “ is wrong because all syllables are equally stressed, “combination” is correctly stressed, “concentrated has stress on the 3rd syllable not the 1st, and “ambition” is stressed on the 1st syllable rather than the 2nd.

Sentences 14-17 (Intonation,  Japanese speaker)

A transcript of the sentences read in the audio file above, with rising or falling intonation noted via arrows.
The reading for sentences 15-20 is annotated to illustrate intonation. This is commented on further in the text below.

 Sentence 14 presents a list of several items. In lists, all phrases except the last are pronounced with rising intonation. The last is pronounced with falling intonation. Instead of rising, rising, rising, falling, the speaker pronounces falling, falling, level, falling. Sentence 16 is a choice question or alternative question “Will you make progress, or will you give up?”), usually pronounced with falling on the last choice and rising on all other choices. The speaker uses rising on both the 1st and 2nd choices. In sentence 19 “Only time will tell, I’m afraid,” the parenthetical “I’m afraid” should be pronounced with level intonation. The speaker uses falling intonation. Finally, sentence 20 should end with falling intonation. The speaker pronounces this correctly.

 

Task: Words (vowels)

This section tests minimal pairs of vowels that are often difficult for speakers of other languages in English. Each set of four word pairs includes a common vowel difficulty that causes difficulties for learners from different languages.

(Vowels, Chinese speaker)

Column 1

/iy/ — /ɪ/

beat—bit

deed—did

seek—sick

leap—lip

/ɪ/ — /ɛ/

bid—bed

miss—mess

lit—let

lift—left

/ɜr/ — /ʌ/

bird—bud

girl—gull

shirt—shut

lurk—luck

 


Column 2

/ɛ/ — /æ/

met—mat

bed—bad

men—man

guess—gas

/uw/ — /ʊ/

boot—book

Luke—look

pool—pull

fool—full

/ey/ — /ɛ/

late—let

mate—met

laid—led

raid—red


Column 3

/a-ɔ/ — /ow/

caught—coat

bought—boat

law—low

raw—row

/a-ɔ/ — /ʌ/

cot—cut

lock—luck

cob—cub

mod—mud

/ay/ — /aw/ — /ɔy/

nice—noise

ties—toys

kind—count

high—how

Task: Phrases (Linking)

This section tests four types of linking to help decide if the linking is appropriate in the marked cases. I recommend that learners read or record each phrase two times. Then I listen to the second one, and circle if the linking is wrong (#1-2 C__V; #3-4 V__V; #5-6 C__Different C; #7-8 C__Same C)

(Phrases, Chinese speaker)

  1. Come_over to my house.
  2. A bowl_of ice cream
  3. Play_it again.
  4. It was so_easy.
  5. Bob_tried_to help.
  6. Don’t_walk_behind_me.
  7. In_November.
  8. About_2:00.

 

Task: Sentences (Rhythm)

This section tests phrase rhythm. Rhythm includes two features: Reduced vowels in function words (underlined in the first group) and lengthened stressed syllables (Underlined in the second group). I have students read each sentence twice, then I listen for each feature separately. Multiple errors give a good clue about whether it’s important to teach a particular aspect of rhythm, such as weak forms or lengthening of stressed syllables.

Sentences (Rhythm, Persian speaker)

Sentences (Rhythm, Chinese speaker)

  1. Do you like bread and(/ən/) butter?
  2. I tried out for(/ər/) the team.
  3. Have you been to(/ə/) New York?
  4. I had to(/ə/) either stay or(/ər/) go.
  5. I used to(/ə/) work on(/ə/) a farm.
  6. Have you ever flown on(/ə/) a plane?
  7. He said that(/ə/) he was(/ə/) there.
  8. I used to(/ə/) drive everywhere in(/ə/ɪ/) a truck.

Stressed words

  1. Do you like bread and butter?
  2. I tried out for the team.
  3. Have you been to New York?
  4. I had to either stay or go.
  5. I used to work on a farm.
  6. Have you ever flown on a plane?
  7. He said that he was there.
  8. I used to drive everywhere in a truck.

 

Task: Dialogues

This section examines the prominence of new information and contrasts. Normal prominence patterns are in bold. (When asking students to read, remove the bolding.) If the student puts prominence on the correct word but also on another word, it is a mistake that indicates that prominence is not likely to be noticed.

Dialogue 1

Excuse me, I lost a book.

What kind of book?

It was a chemistry book. A lab book.

For Chemistry 101?

No, Chemistry 201.

Dialogue 2

We’re having a party for Ellen on Monday. Can you come?

Does she know about it?

No, it’s a surprise party. Why?

She hates surprise parties.

Dialogue 3

I just got a new computer.

What type? A desktop?

No, it’s a laptop. It’s fast, really fast. Not like my old one.

How old was it?

Six years. No, that’s wrong. It was seven years old.

Dialogue 4

I heard Jane makes clothes.

Actually, I think she sells them. She works at the mall.

Which mall?

The new one.

 

Task: Free Speech

Listen to the first minute of the student’s narration of their favorite holiday. Then rate the student’s speech according to intelligibility, comprehensibility, and accentedness (Munro and Derwing, 1995)

Free speech (Spanish speaker, from Celce-Murcia, Brinton & Goodwin, 1996)

 

Intelligibility

How you understand the words that are spoken, i.e., how many words/phrases can you not understand?

Number of words/phrases not understood ____

Comprehensibility

How much effort do you have to put forth to understand the student? Use a 1 – 9 scale:

Student is always easy to listen to   1  2  3  4  5  6  7  8  9   Student is frequently hard to listen to

Accentedness

How pronounced is the student’s accent relative to a local accent? Use a 1 – 9 scale:

Student has barely noticeable accent  1  2  3  4  5  6  7  8  9  Student has a strong foreign accent

 

Comments about the learner’s speech:

 

Task: Free Speech (Retelling a Video Story)

Listen to the first minute of the student’s narration of their video narration. Then rate the student’s speech according to intelligibility, comprehensibility, and accentedness (Munro and Derwing, 1995). The video story being retold is from the following video:

Retelling a video story (Vietnamese speaker)

Intelligibility

How you understand the words that are spoken, i.e., how many words/phrases can you not understand?

Number of words/phrases not understood ____

Comprehensibility

How much effort do you have to put forth to understand the student? Use a 1 – 9 scale:

Student is always easy to listen to   1  2  3  4  5  6  7  8  9   Student is frequently hard to listen to

Accentedness

How pronounced is the student’s accent relative to a local accent? Use a 1 – 9 scale:

Student has barely noticeable accent  1  2  3  4  5  6  7  8  9  Student has a strong foreign accent

Comments about the learner’s speech:

 


References

Celce-Murcia, M., Brinton, D., & Goodwin, J. M. (1996). Teaching pronunciation: A reference for teachers of English to speakers of other languages. Cambridge University Press.

Levis, J. M., & Echelberger, A. (2022). Integrating Pronunciation into Language Instruction. In J. Levis, T. Derwing, & S. Sonsaat-Hegelheimer (Eds.), Second Language Pronunciation: Bridging the Gap Between Research and Teaching (pp. 19-41). Wiley Blackwell. https://doi.org/10.1002/9781394259663.ch2

 

Worksheet

This downloadable worksheet can be used to summarize errors when doing a full diagnostic for an individual student. It can also be used to help remind a teacher of the types of suprasegmental errors that may be important for a class:

Pronunciation Diagnostic Worksheet [DOC]

License

Icon for the Creative Commons Attribution 4.0 International License

Teaching Pronunciation with Confidence Copyright © 2024 by John M. Levis is licensed under a Creative Commons Attribution 4.0 International License, except where otherwise noted.