Abstract

Purpose: Evaluating speaking skill is an exceedingly difficult and intricate subject. Two methods of testing oral proficiency are usually used: holistic and analytic scoring. To this end, this study aimed to compare analytic and holistic techniques for scoring in evaluating oral proficiency skills.

Methodology: The participants of this study were 70 second-grade university EFL students who were studying English Language Teaching at a university in Iran. The participants in this research were both male (n=40) and female (n=30) with the age range from24 to 25.

Main Findings: The results showed a statistically considerable diversity between analytic and holistic methods of evaluation considering that the p-value was estimated at 0.002 (P < 0.05).

Applications: The findings confirmed that employing these two scoring techniques in the procedure of evaluation may be considered proper as they seem to complement each other, one augments the other and lead to more inclusive evaluation.

Novelty/Originality: This study is new since it is the first time in an Iranian university context; oral proficiency skill was evaluated through holistic and analytic scoring.

References

  1. Abedi, P., Keshmirshekan, M. H., &Namaziandost, E. (2019). The comparative effect of flipped classroom instruction versus traditional instruction on Iranian intermediate EFL learners' English composition writing. Journal of Applied Linguistics and Language Research, 6(4), 43-56. http://www.jallr.com/index.php/JALLR/article/view/1036
  2. Al-Amri, M. (2010). Direct spoken English testing is still a real challenge to be worth bothering about. English Language Teaching, 3 (1), 113-117. https://pdfs.semanticscholar.org/c5a7/99 0d5eb6da7e2762e4c86717b 61766823510.pdf. https://doi.org/10.5539/elt.v3n1p113
  3. Anderson, J. (2015). A guide to the practice of English language teaching for teachers and trainee teachers. Nairobi: East African Educational Publishers Ltd.
  4. Azadi, G., Biria, R., &Nasri, M. (2018). Operationalising the Concept of Mediation in L2
  5. Teacher Education. Journal of Language Teaching and Research, 9(1), 132-140.http://www.academypublicati on.com/ojs/index.php/jltr/article/view/jltr0901132140. https://doi.org/10.17507/jltr.0901.17
  6. Baryla, E., Shelley, G., & Trainor, W. (2012). Transforming rubrics using factor analysis: Practical assessment. Practical Assessment Research & Evaluation, 17 (4). 1-17. https://pareonline.net/getvn.asp?v=17&n=4
  7. Bilá, M. (2010). Perception and production of a second language and the concept of a foreign accent. In S. Pokrivčáková et al. (Eds.) Modernization of teaching foreign languages: CLIL, inclusive and intercultural education, pp. 123-143. Brno: Masaryk University.
  8. Cambridge English: Understanding Results Guide. (2014). http://www.gml.cz/prof/zajickova/Cambri dge%20exams_information/Understanding%20results%20guide.pdf
  9. Chen, G. (2016). Developing a Model of analytic rating scales to assess college students’ l2 Chinese oral performance. International Journal of Language Testing, 6 (2), 50-71.http://ijlte-1.ov2.com/content/ijlte_1_ov2_ com/wp-content_138/uploads/2019/07/443-2016-6-2.pdf
  10. Chuang, Y. (2009). Foreign language speaking assessment: Taiwanese college English teachers‟ scoring performance in the holistic and analytic rating methods. The Asian EFL Journal, 11 (1), 150-173. https://www.asian-efl-journal.com/main-journals/foreign-language-speaking-assessment-chinese-taiwanese-college-english-teachers-scoring-performance-in-the-holistic-and-analytic-rating-methods/
  11. Council of Europe. (2001). Common European framework of reference for languages: Learning, teaching, assessment. Cambridge: Press Syndicate of the University of Cambridge.
  12. Davies, A. (1999). Dictionary of language testing. Cambridge: Cambridge University Press.
  13. Finson, K., Ormsbee, C., & Jensen, M. (2011). Differentiating science instruction and assessment for learners with special needs, k-8. Thousand Oaks: Corwin, A SAGE Company. https://doi.org/10.4135/9781483387529
  14. Goh, C. & Burns, A. (2012). Teaching speaking: A holistic approach. New York: Cambridge University Press.
  15. Göktürk, N. (2016). Examining the Effectiveness of Digital Video recordings on Oral Performance of EFL Learners. Teaching English with Technology, 16 (2), 71-96. https://eric.ed.gov/?id=EJ1135917
  16. Gondová, D. (2014). Taking first steps in teaching English: Assessing learners. Žilina: EDIS.
  17. Green, A. (2014). Exploring language assessment and testing: Language in action. New York: Routledge. https://doi.org/10.4324/9781315889627
  18. Hashemifardnia, A., Namaziandost, E., &Rahimi Esfahani, F. (2018a). The Effect of Teaching Picture-books on ElementaryEFL Learners’ Vocabulary Learning.JELTL (Journal of English Language Teaching and Linguistics), 3(3), 247-258.https://jeltl.org/index.php/jeltl/article/view/151. https://doi.org/10.21462/jeltl.v3i3.151
  19. Hashemifardnia, A., Namaziandost, E., &Sepehri, M. (2018). The effectiveness of giving grade, corrective feedback, and corrective feedback-plus-giving grade on grammatical accuracy. International Journal of Research Studies in Language Learning, 8 (1), 15-27.http://consortiacademia.org/10-5861-ijrsll-2019-3012/. https://doi.org/10.5861/ijrsll.2019.3012
  20. Hashemifardnia, A., Namaziandost, E., Shafiee, S. (2018).The Effect of Implementing Flipped Classrooms on Iranian Junior High School Students' Reading Comprehension. Theory and Practice in Language Studies, 8(6), 665-673.http://www.academypublication.com/ojs/index.php/tpls/article/view/tpls0806665673. https://doi.org/10.17507/tpls.0806.17
  21. Hosseini, E. Z., Nasri, M., &Afghari, A. (2017). Looking beyond teachers’ classroom behavior: novice and experienced EFL teachers’ practice of pedagogical knowledge to improve learners’ motivational strategies. Journal of Applied Linguistics and Language Research, 4(8), 183-200.http://www.jallr.com/index.php/JALLR/article/view/729
  22. Jankowska, A. &Zielińska, U. (2015). Designing a self-assessment instrument for developing the speaking skill at the advanced level. In M. Pawlak and E. WaniekKlimczak (Eds.) Issues in teaching, learning and testing speaking in a second language. Heidelberg: Springer. https://doi.org/10.1007/978-3-642-38339-7_16
  23. Jonnson, A. &Svingby, G. (2007). The use of scoring rubrics: Reliability, validity and educational consequences. Educational Research Review, 2 (2), 130-144. https://ila.onlinelibrary.wiley.com/doi/abs/10.1002/jaal.525. https://doi.org/10.1016/j.edurev.2007.05.002
  24. Keshmirshekan, M. H., Namaziandost, E., &Pournorouz, M. (2019). The Impacts of Creative Writing Activities on Iranian EFL Learners’ Progress in Writing, Writing Dispositions: Focus on Attitude to English Course. EPH - International Journal of Educational Research, 3(9), 12-22.https://ephjournal.com/index.php/er/article/view/1552/911
  25. Khamkhien, A. (2010). Teaching English speaking and English-speaking tests in the Thai context: a reflection from Thai perspective. English Language Teaching, 3 (1), 184-190. http://www.ccsenet.org/ journal/index.p hp/elt /article/view/5253. https://doi.org/10.5539/elt.v3n1p184
  26. Kondo-Brown, K. (2002). A FACETS analysis of rater bias in measuring Japanese second language writing performance. Language Testing, 19 (1), 3-31.https://journals.sagepub.c om/doi/abs/10.1191 /0265532202lt 218oa?journalCode=ltja. https://doi.org/10.1191/0265532202lt218oa
  27. Llach, M. (2011). Lexical errors and accuracy in foreign language writing. New York: Multilingual Matters. https://doi.org/10.21832/9781847694188
  28. Luoma, S. (2004). Assessing speaking. Cambridge: Cambridge University Press. https://doi.org/10.1017/CBO9780511733017
  29. Metruk, R. (2017). Pronunciation of English dental fricatives by Slovak University EFL Students. International Journal of English Linguistics, 7 (3), 11-16. http://www.ccsenet.org/journal/index.php/ijel/article/view/64987. https://doi.org/10.5539/ijel.v7n3p11
  30. Mirshekaran, R., Namaziandost, E., & Nazari, M. (2018). The Effects of Topic Interest and L2 Proficiency on Writing Skill among Iranian EFL Learners. Journal of Language Teaching and Research, 9(6), 1270-1276.http://www.academypublication.com/ojs/index.php/jltr/article/view/jltr090612701276. https://doi.org/10.17507/jltr.0906.16
  31. Murugaiah, P. (2016). Pecha Kucha style PowerPoint presentation: An innovative CALL approach to developing oral presentation skills of tertiary students. Teaching English with Technology, 16 (1), 88-104. https://eric.ed.gov/?id=EJ1135930
  32. Namaziandost E., &Nasri, M. (2019a).A meticulous look at Long’s (1981) interaction hypothesis: does it have any effect on speaking skill?Journal of Applied Linguistics and Language Research, 6(2), 218-230.http://www.jallr.com/index.php/JALLR/article/view/1023
  33. Namaziandost E., &Nasri, M. (2019b).The impact of social media on EFL learners’ speaking skill: A survey study involving EFL teachers and students.Journal of Applied Linguistics and Language Research, 6(3), 199-215.http://www.jallr.com/index.php/JALLR/article/view/1031
  34. Namaziandost E., &Nasri, M. (2019c).Innovative practices in L2 writing materials in the EFL
  35. classroom: Effect on writing enhancement and attitude to English course. Asian Journal of Research in Social Sciences and Humanities, 9(9), 1-12. http://www.academypublication.com/ojs/index.php/jltr/article/view/jltr09018089
  36. Namaziandost, E., &Nasri, M., Rahimi Esfahani, F., &Keshmirshekan M. H. (2019). The impacts of spaced and massed distribution instruction on EFL learners’ vocabulary learning. Cogent Education, 6: 1661131. https://doi.org/10.1080/2331186X.2019.1661131.
  37. Namaziandost, E., Abedi, P., &Nasri, M. (2019). The Role of Gender in the Accuracy and Fluency of Iranian Upper-intermediate EFL Learners’ L2 Oral Productions. Journal of Applied Linguistics and Language Research, 6(3),110-123. http://www.jallr.com/index.php/JALLR/article/view/1018
  38. Namaziandost, E., Nasri, M., &Keshmirshekan, M. H. (2019). Cohesive conjunctions in applied linguistics research articles among Iranian and non-Iranian researchers: A comparative corpus-based study. Journal of English Language Studies, 4(2), 101 – 119.http://jurnal.untirta.ac.id/index.php/JELS/article/view/5431
  39. Namaziandost, E., Nasri, M., & Rahimi Esfahani, F. (2019a). Pedagogical Efficacy of Experience-Based Learning (EBL) Strategies for Improving the Speaking Fluency of Upper-intermediate Male and Female Iranian EFL Students.International Journal of Research in English Education, 4(2), 29-41.http://ijreeonline.com/article-1-160-en.html. https://doi.org/10.29252/ijree.4.2.29
  40. Namaziandost, E., Nasri, M., & Rahimi Esfahani, F. (2019b).Texts with Various Levels of Hardness, Reading Comprehension and Reading Motivation: I+1 Versus I-1. ELT Forum: Journal of English Language Teaching, 8(1), 60-77. https://journal.unnes.ac.id/sju/index.php/elt/article/view/30992
  41. Namaziandost, E., Rahimi Esfahani, F., Nasri, M., &Mirshekaran, R. (2018). The Effect of Gallery Walk Technique on Pre-intermediate EFL Learners’ Speaking Skill. Language Teaching Research Quarterly, 8, 1–15.https://www.researchgate.net/publication/328449149_The_Effect_of_Gallery_Walk_Technique_on_Pre-intermediate_EFL_Learners%27_Speaking_Skill. https://doi.org/10.17507/tpls.0812.16
  42. Nasri, M. &Biria, R. (2017). Integrating multiple and focused strategies for improving reading comprehension and l2 lexical development of Iranian intermediate EFL learners.International Journal of Applied Linguistics & English Literature, 6(1), 311-321.https://journals.aiac.org.au/index.php/IJALEL/article/view/2803.https://doi.org/10.757 5/aiac.ijalel.v.6n.1p.311
  43. Nasri, M., Biria, R., & Karimi, M. (2018). Projecting Gender Identity in Argumentative Written Discourse. International Journal of Applied Linguistics & English Literature, 7(3), 201-205.https://journals.aiac.org.au/index.php/IJALEL/article/view/4289. https://doi.org/10.7575/aiac.ijalel.v.7n.3p.201
  44. Nasri, M., Namaziandost, E., & Akbari, S. (2019). Impact of pictorial cues on speaking fluency and accuracy among Iranian pre-intermediate EF learners. International Journal of English Language and Literature Studies, 8(3), 99-109. https://ideas.repec.org/a/asi/ijells/2019p99-109.html. https://doi.org/10.18488/journal.23.2019.83.99.109
  45. O’ Sullivan, B. (2012). Assessing speaking. In C. Coombe, P. Davidson, B. O’ Sullivan, and S. Stoynoff (Eds.) the Cambridge guide to second language assessment, pp. 234- 246. New York: Cambridge University Press.
  46. Pan, M. (2016). Nonverbal delivery in speaking assessment. From an argument to a rating scale formulation and validation. Singapore: Springer. https://doi.org/10.1007/978-981-10-0170-3
  47. Pokrivčáková, S. (2010). Modern teacher of English. Nitra: ASPA.
  48. Razali, K. &Isra, M. (2016). Male and female teachers’ roles in assessment of speaking skill: Gender equality. International Journal of Child and Gender Studies, 2 (1), 1-10.https://jurnal.ar-raniry.ac.id/index.php/equality/article/view/1448/0
  49. Ruammai, P. (2014). Constructing scoring instrument for writing assessment and fostering critical thinking. In H. Lee (Ed.) The international conference on language and communication innovative inquiries and emerging paradigms in language, media and communication, pp. 127-137.
  50. Saritha, K. (2016). Rubric for English language teaching research. Research Journal of English Language and Literature, 4 (2), 725-731.https://scholar.google.com/citations?user=yiMY04wAAAAJ&hl=en
  51. Sarwar, M., Alam, M., Hussain, S., Shah, A., &Jabeen, M. (2014). Assessing English speaking skills of prospective teachers at entry and graduation level in teacher education program. Language Testing in Asia, 4 (5),18-31. https://languagetestingasia.springeropen.com/articles/10.1186/2229-0443-4-5. https://doi.org/10.1186/2229-0443-4-5
  52. Shakibaei, G., Shahamat, F., &Namaziandost, E. (2019). The effect of using authentic texts on Iranian EFL learners' incidental vocabulary learning: The case of English newspaper. International Journal of Linguistics, Literature and Translation (IJLLT), 2(5), 422-432. http://www.ijllt.org/the-effect-of-using-authentic-texts-on-iranian-efl-learners-incidental-vocabulary-learning-the-case-of-english-newspaper/
  53. Taylor, L. &Galaczi, E. (2011). Scoring validity. In L. Taylor (Ed.), M. Milanovic & C. Weir (Series Eds.) Studies in language testing 30. examining speaking. research and practice in assessing second language speaking, pp. 171-233. Cambridge: Cambridge University Press.
  54. Thornbury, S. (2005). How to teach speaking. Harlow: Pearson Education Limited.
  55. Topkaraoğlu, M. &Dilman, H. (2014). Effects of studying vocabulary enhancement activities on students’ vocabulary production levels. Procedia - Social and Behavioral Sciences, 152, 931-936. https://doi.org/10.1016/j.sbspro.2014.09.345
  56. Tuan, L. (2012). Teaching and assessing speaking performance through analytic scoring approach. Theory and Practice in Language Studies, 2 (4), 673-679. doi:10.4304/tpls.2.4.673-679. https://doi.org/10.4304/tpls.2.4.673-679
  57. Ur, P. (2012). A course in English language teaching. Cambridge: Cambridge University Press.
  58. Ziafar M., &Namaziandost, E. (2019). Linguistics, SLA and lexicon as the unit of language. International Journal of Linguistics, Literature and Translation (IJLLT), 2(5), 245-250. http://www.ijllt.org/linguistics-sla-and-lexicon-as-the-unit-of-language/