Washback of english proficiency test in classroom activities at national university of arts education

VNU Journal of Science: Education Research, Vol. 36, No. 1 (2020) 1-15<br /> <br /> <br /> <br /> <br /> Original Article<br /> Washback of English Proficiency Test in Classroom Activities<br /> at National University of Arts Education<br /> <br /> Dinh Thi Phuong Hoa*<br /> Dean of Foreign Languages, Hanoi Law University,<br /> 87 Nguyen Chi Thanh, Dong Da, Hanoi, Vietnam<br /> <br /> Received 05 November 2019<br /> Revised 10 February 2020; Accepted 17 February 2020<br /> <br /> Abstract: A study in Vietnam concerning the effects of The Vietnam Six-levels of Foreign<br /> Language Proficiency Framework, specially English Proficiency Tests for graduates, on classroom<br /> teaching and learning activities are reported. The study explores the phenomenon of washback or<br /> backwash, the influences of testing on 9 teachers and 679 non-English major students. It is cited as<br /> the only known research investigating washback in language education through classroom<br /> observation. The study was conducted at National University of Art Education, and combined<br /> classroom observations with data from interview, questionaire responses and document analysis to<br /> determine whether washback exist, to what degree it operates, and whether it is a positive or<br /> negative force in this educational context. The insights from the findings indicate that washback of<br /> English Proficiency Tests for graduates occurred in both positive and negative forms, to some degree, in<br /> teaching and learing content, methods and styles. Evidence of washback, both positive and negative, on<br /> the way teachers design tests was also found. This should help Vietnamese educators to prepare<br /> favourable conditions for enhancing the benificial washback of EPT. The findings have contributed to<br /> the knowledge of a nature of washback and consequently opened a new understanding to recognize the<br /> dissimilar levels of washback. further research is recommended.<br /> Keywords: Washback, English Proficiency Tests, classrooms activities.<br /> <br /> <br /> 1. Introduction * 757) stated that, “the importance of English has<br /> flashed an increasing concentration in the<br /> Today, English has become a global development of English language teaching in<br /> language that offers the chances to integrate numerous countries” [1]. In Vietnam, English<br /> into all the professions. Khamkhien (2010, p. has been instructed nationwide as a compulsory<br /> subject at both lower, upper secondary level and<br /> _______ tertiary level; and as an elective subject at<br /> * Corresponding author.<br /> E-mail address: dinhphuonghoa.ecas@gmail.com primary level from 1980s to present (Nguyen,<br /> https://doi.org/10.25073/2588-1159/vnuer.4335<br /> 1997, p.5) [2]. Notwithstanding its impact,<br /> 1<br /> 2 D.T.P. Hoa / VNU Journal of Science: Education Research, Vol. 36, No. 1 (2020) 1-15<br /> <br /> <br /> <br /> English language teaching and learning for non- National University of Art Education<br /> major learners in Vietnam are contradictory to (henceforth NUAE) is Undergraduate<br /> all expectations as the language skills of institution that is not specialized in foreign<br /> listening, speaking, reading and writing have languages; the new language-training program<br /> not been appreciated. Furthermore, most of the must require a language proficiency of<br /> teachers have still taught English with VNFLPF level 3 upon graduation. However,<br /> traditional techniques, as teacher-centered or because of limitation of training time and<br /> the grammar-translation method for many English in mixed-big sized classes (from 55 to<br /> years. Conversely, for fulfilling the needs of a over 65 students) in a large room where is 105<br /> modern society in the globalization epoch, square meters in area, no microphone, and thus,<br /> Vietnamese Prime Minister issued Decision No some students could not listen to all lessons<br /> 1400/QD-Ttg of September 30, 2008, clearly. 679 students were from 18 to 22 years<br /> approving the scheme “Foreign Language of age. They were from different Northern areas<br /> Teaching and Learning in the national of Vietnam. Although, they had 3 years of<br /> education system during 2008 - 2020” and now learning English at high schools, their English<br /> this scheme is extended to 2025 (National proficiency was at beginner level (A0),<br /> Foreign Languages Project for short) [3]. The therefore, the Rector of NUAE decided to apply<br /> scheme aims at implementing an educational for English proficiency of VNFLPF level 2<br /> innovation and evaluation of foreign language (A2) upon graduation. As a result, EPT.2 (A2)<br /> teaching and learning at all levels in the of VNFLPF is a compulsoty requirement for<br /> national education system. Accordingly, NUAE graduation from 2016 and EPT.3 (B1)<br /> Minister of Education and Training issued the will start being used from 2021.<br /> Circular N0 01/2014/TT-BGDĐT of January On the basic of the background of the<br /> 24, 2014, approving The Vietnam Six-levels of Vietnam educational innovation context,<br /> Foreign Language Proficiency Framework particularly the context at NUAE, the study<br /> (henceforth VNFLPF). This framework consists attemped to address these issues:<br /> of six levels that are compatible with the 1) Whether English Proficiency Test will<br /> Common European Framework of Reference positively influence the English language<br /> for Languages [4] (CEFR for short) and other teaching process at National University of Art<br /> common international language proficiency Education, Vietnam.<br /> levels and used as reference when writing 2) Whether the changes in the teaching<br /> curriculums and teaching plans. According to process will beneficially affect teaching<br /> National Foreign Languages Project, with strategies, which will lead to changes in<br /> undergraduate institutions that are not learning style at National University of Art<br /> specialized in foreign languages, the new Education, Vietnam.<br /> language-training program must require a<br /> language proficiency of VNFLPF level 3 upon 2. Literature review<br /> graduation. Based on this framework, English<br /> Proficiency Test from level 2 to level 5 2.1. The definition of washback in this study<br /> (henceforth EPT.2 and EPT.3-5) is conducted The term “washback” is predominant in<br /> and issued. Among these, EPT.2 is compatible language teaching and testing literature as well<br /> with A2 of CEFR and EPT.3-5 is compatible as general education. However, the term<br /> with B1, C1, and C2 of CEFR. It thus became a “washback” has been defined and<br /> very high-stakes test with serious consequences interchangeably by many researchers and<br /> for non-English major students. organizations worldwide.<br />  D.T.P. Hoa / VNU Journal of Science: Education Research, Vol. 36, No. 1 (2020) 1-15 3<br /> <br /> <br /> In applied linguistics, the term “washback” In short, for the purpose of this paper, the<br /> or backwash is defined as the influence or term “washback/backwash” is understood to be<br /> impact of tests on curriculum/syllabus design, the influences that tests have on teachers and<br /> language teaching and language testing [5]. students in terms of the methods/activities they<br /> Accordingly, tests can influence teachers and use in their classrooms to teach/study English<br /> learners, and thus influence teaching and as Foreign Language.<br /> learning activities. The influences may be either<br /> positive or negative, depending on various 2.2. The Vietnam Six-levels of foreign language<br /> facets not yet defined. Nevertherless, whether a proficiency framework<br /> seperate and distinguishable phenomenon of<br /> washback exists is still open to debate; and The CEFR provides a detailed description<br /> there appear to be very few emprical studies of learner level by skills, in a language-neutral<br /> directly investigating this phenomenon [6]. format. Therefore, the CEFR is used for many<br /> In the educational evaluation literature, dissimilar practical purposes because its<br /> washback is considered the influences of testing influence goes beyond merely describing<br /> on teaching and learning practices. Therefore, language proficiency of learners, they are:<br /> tests can drive teaching and learning that is also teacher training programs, developing<br /> mentioned as measurement-driven instruction syllabuses, creating tests/exams, marking<br /> [7]. Fitz-Gibbon (1996) defined impact as any exams, evaluating language learning needs,<br /> effect of the service [or of an event or initiative] designing courses, developing learning<br /> on an individual or group [8]. This definition materials and describing language policies<br /> accepts that the impact can be positive or continuous/self-assessment.<br /> negative and may be intended or accidental. Accordingly, VNFLPF is designed based<br /> When holding this definition, measuring impact on CEFR in the Vietnam educational context.<br /> is about identifying and evaluating change [9]. This framework consists of six levels and its<br /> Messick (1989) expanded the concept of Can-do descriptors that are compatible with<br /> consequential validity, changing the previous CEFR and other common international<br /> notions about score interpretation and test use. language proficiency levels. Therefore,<br /> The concept of washback in test validity VNFLPF is used as reference when writing<br /> research is primarily associated with Messick’s curriculums teaching plans, assessement and<br /> concept of consequential validity. Therefore, designing test.<br /> washback is defined as an “instance of the VNFLPF describes foreign language<br /> consequential aspect of construct validity and a proficiency at three broad bands with six main<br /> focal point of validity research” [10], which levels: level 1 and level 2, level 3 and level 4,<br /> covers components of test use, the impact of level 5 and level 6. The scale starts at level 1<br /> testing on test-takers and educators, the and finishes at level 6 that is compatial with<br /> interpretation of results by decision-makers, CEFR from A1 to C2 as the following:<br /> and any possible misuses, abuses, and For the purpose of this paper, the usage of<br /> unintentional effects of tests. The influences of VNFLPF helps to define clearly certain<br /> tests on teachers, students, institutions, and society requirements for competency, capacity in<br /> are accordingly considered one type of validity listening, speaking, reading and writing, and<br /> evidence. Many other researchers have also thus English level 2 (A2) of VNFLPF learner is<br /> emphasized the meaning of justifying test use and actived in the performance of the four main<br /> exploring its consequences ([11, 12]). Therefore, language activities, including listening,<br /> washback also plays a key role in the process of speaking (spoken interaction), reading,<br /> educational innovation and assessment in writing (written production) in the public, the<br /> language teaching and learning [13]. personal, the educational and the occupational<br /> domains with some types of text and questions.<br /> 4 D.T.P. Hoa / VNU Journal of Science: Education Research, Vol. 36, No. 1 (2020) 1-15<br /> <br /> <br /> d<br /> <br /> <br /> <br /> <br /> Table 1. The 6 levels of the VNFLPF<br /> <br /> VNFLPF (Level) General Descriptions<br /> Level 1 Can communicate in basic English with help<br /> A - Basic user<br /> (A1 - Breakthrough) from the listener<br /> Level 2 Can communicate in English within a limited<br /> (A2- Way-stage) range of contexts<br /> B - Independent user Level 3 (B1 - Threshold) Can communicate essential points<br /> Can use English effectively, with some fluency,<br /> Level 4 (B2 - Vantage)<br /> in a range of contexts<br /> Level 5 (C1 - Effective Can use English fluently and flexibly in a wide<br /> C - Proficient user<br /> Operational Proficiency) range of contexts.<br /> Level 6 (C2 -Mastery or Can use English, very fluently, precisely and<br /> Highly proficient) sensitively, in most contexts<br /> D<br /> 2.3. Some washback studies of Communicative Orientation of Language<br /> Teaching [18] and interview methods to<br /> Studies on washback reveal varied and compare “teachers’ perceptions toward both old<br /> sometimes different findings. The following and new HKCEE”. Cheng (2004) based on a<br /> section discusses the washback influences on combined research framework that employed<br /> teaching or learning activities in classroom. multiple approaches to explore both the macro<br /> The field of washback has been investigated level (including the main parties within the<br /> by many researchers around the world. Among HongKong educational context) and the micro<br /> these, the washback model of Alderson and level in schools (concerning different aspects of<br /> Wall (1993) is considered a classic and English teaching and learning) to recognize the<br /> landmark study. Alderson and Wall (1993) used washback phenomena by using English<br /> obsevation method to carry out their Srilanka study questionaires that consisted of three parts. Part<br /> on investigating the washback existing of English 1 discovered the general information of teacher.<br /> teaching and learning activities in classroom. Part 2 with 5-point Likert scale of agreement<br /> Alderson and Wall (1993, p. 120-121) developed discovered teacher’s perceptions and 5-point<br /> the fifteen hypotheses (WHs for short) that Likert scale of frequency of Part 3 discovered<br /> combined different possible aspects of washback, teacher’s reactions to the new HKCEE through<br /> including the effect on what to teach/learn, how to their classroom teaching and learning activities.<br /> teach/learn, the rate and sequence of Regarding the washback of CEFR, Pan and<br /> teaching/learning, the degree and depth of Newfields (2012) worked on discovering how<br /> teaching/learning and the attitudes to content, English proficiency graduation requirements<br /> method, etc. of teaching/learning [14]. Alderson have impacted 17 tertiary educational<br /> and Hamp-Lyons’s model (1996, p. 296) used institutions in Taiwan by using extensive<br /> interviews and one-week-classroom questionnaire and interview data [19]. Among<br /> observations of teachers to review and correct them, the survey contained two types of<br /> WHs of Alderson and Wall (1993) that “tests questions: multiple-choice questions with<br /> will have different amounts and types of categorical responses and 5-point Likert scale<br /> washback on some teachers and some learners questions with pseudo-ordinal responses. Since<br /> than other teachers and learners” [15]. 2003, Taiwan’s Ministry of Education (TME)<br /> The studies of Cheng (1999 and 2004) established a list of recommended tests to set<br /> focused on old and new HongKong Certificate English thresholds for graduates to generate a<br /> Examination in English (HKCEE) ([16, 17]). level of English proficiency, which were<br /> Cheng (1999) used classroom observation that modified according to the CEFR B1 or A2<br /> combined her data of baseline study and Part A levels. They included two local tests: the<br />  D.T.P. Hoa / VNU Journal of Science: Education Research, Vol. 36, No. 1 (2020) 1-15 5<br /> <br /> <br /> General English Proficiency Test (GEPT) and the Taiwanese tertiary educational framework<br /> the College Student English Proficiency Test from the perspectives of students.<br /> (CSEPT). The GEPT was a 5-level, four-skill Comparatively little research of Pan and<br /> general English proficiency examination Newfields was conducted regarding the effects<br /> commissioned by TME in 1999. The CSEPT of tests on the learning processes, in contrast to<br /> was 2-level, listening-reading-grammar test for the significant number of studies on the effects<br /> university-level students in Taiwan. of tests on teaching. Their study confirmed the<br /> Accordingly, Pan and Newfields (2012) argument of other washback studies that<br /> conducted their study after the inception of standardized tests were not a panacea that<br /> English certification graduation requirements in always succeeded in changing students’ study<br /> Taiwan; so a comparison of the baseline and a habits. Therefore, their study also made it clear<br /> follow-up study to determine the consequences that the test requirements did not lead to a<br /> brought about by the tests was not viable. notable amount of “studying for the test” a<br /> Therefore, a comparison of the differences phenomenon often reported in examination-<br /> between the schools with graduation oriented societies. However, the study of Pan<br /> requirements and those without graduation and Newfields had three noteworthy<br /> requirements will be used to reveal test effects.<br /> limitations. Firstly, the study of Pan and<br /> In short, this part focuses on some<br /> Newfields was conducted at a period when<br /> washback studies published between 1993 and<br /> many institutions in Taiwan were eager to adopt<br /> 2012. The first part also reviews how these<br /> the government’s EFL graduation exam policy<br /> studies have investigated washback. All studies<br /> and thus, washback appeared to be ineffective.<br /> cited here explore different aspects of washback<br /> Secondly, their study has relied on self-reported<br /> and use various instruments. Alderson and Wall<br /> student data and thus, such information was<br /> investigated evident of both beneficial and<br /> easily prone to expectancy bias. Subsequent<br /> harmful washback on the content of teaching<br /> investigation should include more classroom<br /> and on ways of assessing, but not on teaching<br /> observational data and seek to corroborate<br /> methodology. Alderson and Hamp-Lyons<br /> student data with other data sources from<br /> ascertained the influence of the TOEFL on class<br /> teachers and school administrators. This should<br /> teaching and TOELF affected both what and<br /> allow their study to get a more accurate and<br /> how teachers teach, but the effects differed<br /> dynamic picture of how washback patterns are<br /> from teacher to teacher. However, the study of<br /> perceived by different test stakeholders. Lastly,<br /> Alderson and Hamp-Lyons had three significant<br /> one goal of introducing graduation<br /> limitations. Firstly, they did not include<br /> requirements was to improve the ability of<br /> questionaires. Secondly, they choiced<br /> graduates to communicate effectively in<br /> participants and lastly, they dealt with<br /> English in the office that would be very<br /> washback primarily from perspectives of<br /> difficult for the researchers to measure, further<br /> teachers, hardly addressing students’ opinions.<br /> research should pay more attention to this<br /> Cheng contributed to the few washback studies<br /> aspect of washback.<br /> by using both quantitative and qualitative<br /> All of the reviewed studies have been<br /> methods. Cheng’s study was useful because her conducted in primary and secondary schools or<br /> study attemped to evaluate the effects of the tertiary educational institutions in Srilanka,<br /> new examination, however, a longitudinal HongKong, Taiwan. The methods were used<br /> research with a longer timeframe than the one involved either written questionaires or<br /> used by Cheng might shed better light on the interview/observations. They found evidence of<br /> influences of the new HKCEE. Pan and washback influences on teachers’ behaviours or<br /> Newfields aimed to discover the test effects learning. Accordingly, there has no previous<br /> brought about by graduation requirements in research into washback effects arising from<br /> 6 D.T.P. Hoa / VNU Journal of Science: Education Research, Vol. 36, No. 1 (2020) 1-15<br /> <br /> <br /> <br /> EPT.2 or standardised tests in Vietnam National teachers of English) and 679 non-English<br /> University of Art Education. Thus, further major students of NUAE.<br /> research into this area is still needed.<br /> Drawing on some washback models and 3.2. Conducting the document analysis<br /> some empirical studies on language teaching or<br /> learning activities in classroom of Alderson and The researcher collected all institutional<br /> Hamp-Lyon (1996), Cheng (1999 and 2004) policy documents on innovating methods of<br /> and Pan and Newfields (2012), this study will assessment, syllabus, and supplementary<br /> be designed to investigate “Washback of materials according to VNFLPF and EPT.2 for<br /> English Proficiency Test in Classroom getting the data because such artifacts of<br /> Activities at NUAE”. The study concentrated on everyday experience can provide information<br /> an exploration of the effects of washback on about what has been encouraged or<br /> teachers/teaching process may offer insights discouraged; about what has happened or will<br /> about how VNFLPF and EPT.2 influence happen ... etc. [21]. Therefore, such documents<br /> language teaching or learning activities in are particular useful for educational research.<br /> classroom at the educational innovation of<br /> NUAE context. Furthermore, evidences from 3.3. Conducting the questionaire<br /> various sources of this study also helped to<br /> consider how the teachers and students benefit The survey of this study was carried out<br /> from the innovation. within from December 25, 2017 to January 12,<br /> 2018. Simple random sampling was employed<br /> in this study. For comparing the correct<br /> 3. Methodology and data responses given by each group, Teacher<br /> Questionnaire and Student Questionnaire<br /> This work was conducted between January consisted of four parts and the same contents<br /> 2014 and November 2018, aiming to capture that were modified and adapted to Cheng<br /> the changes when VNFLPF was introduced<br /> (2004). All items of Questionnaires were<br /> into teaching in 2014 until the first cohort of<br /> designed according to the results of VNFLPF<br /> NUAE’s students took the EPT.2 graduation<br /> examination in 2017. and EPT.2 analysis. The same contents of<br /> For ensuring the validity and reliability of Teacher Questionnaire and Student<br /> the questionnaire items, qualitative input and Questionnaire was designed to check who<br /> piloting procedures were carried out that lead to remember or who tell the truth and thus,<br /> ensure the content validity and thus its determine what happens in classrooms activities<br /> consequential validity [20]. This study collected and how washback operates if it occurs.<br /> data of three kinds: (1) documents analysis, (2) Due to the length of this study, Teacher<br /> focus group interview, (3) questionaires and (4) Questionnaire and Student Questionnaire were<br /> classroom observations. However, the data described shortly as the following (Table 2).<br /> from (1), (2) and (3) are considered backdrop to<br /> the discussion (4) because of the extent of the 3.4. Conducting the observations and instruments<br /> data and space limitation.<br /> After receiving the permission of all<br /> 3.1. Subjects of the study participants, 10 classes (English level A2) of<br /> ten teachers were chosen for observing. The<br /> The subjects of the project were Rector of researcher conducted the observations to<br /> NUAE, Head of Training Department, 12 obsever what happens in the English classroom<br /> teachers of English at NUAE (02 Vice and thus, determine what and how teacher teach<br /> Directors of Foreign Language Central and 9/12 or what and how students learn.<br />  D.T.P. Hoa / VNU Journal of Science: Education Research, Vol. 36, No. 1 (2020) 1-15 7<br /> <br /> <br /> Table 2. Teacher Questionnaire and Student Questionnaire<br /> <br /> Numerical order Concepts Variables Scales<br /> A: Personal details<br /> Part 1 Engling proficiency, ages 2/4 Nominal Scale<br /> B: EFL teaching and learning activities<br /> <br /> Contents and communicative method of teaching EFL 372<br /> Part 2<br /> (including listening, speaking, reading, writing skills)<br /> <br /> Topics 52<br /> While-lesson<br /> activities<br /> <br /> <br /> <br /> <br /> Texts 72<br /> <br /> Question types 32<br /> 5-point Likert<br /> Activities 28 scale<br /> of frequency<br /> Topics 52<br /> (Homework)<br /> After school<br /> <br /> <br /> <br /> <br /> Texts 72<br /> <br /> Question types 32<br /> <br /> Activities 28<br /> Post-lesson activities Correct and Comment 4<br /> Part 3 Materials 13 Nominal Scale<br /> <br /> Part 4 Assessment (including listening, speaking, reading, writing tests) 10 Nominal Scale<br /> t<br /> The observation happened from January to examination ans Round 2 was observed before<br /> March 2018. Classes are scheduled one day per the semeter examination to explore the<br /> week with substantial uninterrupted work differences of influences of VNFLPF and<br /> periods and the teaching session lasted EPT.2 between two Rounds. 50 minutes of<br /> approximately 200 minutes (4 periods) per day every observation was the length of each lesson<br /> every morning or afternoon. There are 55 period and the teaching session lasted<br /> periods of English level A2 from December 25, approximately 200 minutes (4 periods) per day<br /> 2017 to March 23, 2018. Therefore, the every morning or afternoon. The observation<br /> observation process was divided into 2 rounds, process was divided into 2 rounds, they were<br /> they were Round 1 and Round 2. Round 1 was Round 1 and Round 2 as the following<br /> took palace that far from the semeter (Table 3).<br /> Table 3. Observation timeline<br /> <br /> Duration: Spring semester, 2018<br /> Round 1 The length of classroom observation period Time<br /> 10 English lessons 50 minutes for each observation of one English lesson from January 5 to March 23, 2018<br /> Round 2 The length of classroom observation periods Time<br /> 30 English lessons<br /> 7<br /> 150 minutes for each observation of 3 English lessons from March 26 to March 30, 2018<br /> 8 D.T.P. Hoa / VNU Journal of Science: Education Research, Vol. 36, No. 1 (2020) 1-15<br /> <br /> <br /> <br /> For getting the exact information, thick using Excel and IBM Statistical Product and<br /> descriptions and the responds of teachers and Services Solutions software. The survey<br /> students in a natural manner, teachers and explored the differences between findings of<br /> students were explained about the observation. two groups (Teacher and Student). The<br /> The observation scheme was designed and differences were tested for determining<br /> adapted according to Cheng (1999) that statistical significance by using the Lavene’s<br /> combined the data of mentioned questionaires test and the independent sample T-test. The<br /> and Part A of COLT. These descriptive data Levene’s test for Equality of Variances was<br /> would be assessed according to their common used to clarify the equal distribution in each<br /> outcomes. Therefore, the researcher also subgroup. The Independent Samples Test<br /> discovered how VNFLPF and EPT.2 compares the mean scores of two groups on all<br /> influences teachers and students. given variables. A probability of less than .05<br /> was taken as statistically significant for the<br /> 3.4. Conducting the interviews survey (p < 0.05). If it is not significant, the<br /> value is greater than .05 (p > 0.05), the two<br /> After observations, the focus-group variances are not significantly different; that is,<br /> interviews were held because the participants the two variances are approximately equal. If<br /> had a few experiences of teaching and learning the Levene's test is not significant, the second<br /> English by that time. The open-ended questions assumption should be met. The possibility of<br /> were designed to attain the best feasible quality error could increases with the number of T-tests<br /> of responses from the members because the being carried out. Accordingly, a method<br /> open-ended questions were used to add the triangulation with a complementary multiple-<br /> depth of the data via participants’ individual method design were used in this study to ensure<br /> experiences [21]. Moreover, these open-ended against errors arising from the data collection<br /> questions were applied in both individual and and analysis. The present study was designed<br /> focus group interviews. This combination after the beginning of English graduation<br /> helped to focus on getting the specific requirements at NUAE; so a comparison of the<br /> information that would be comparable across baseline and a follow-up study to define the<br /> the group of participants. consequences brought about by VNFLPF and<br /> The researcher carried out at least 05 EPT.2 were not viable. Therefore, a comparison<br /> minutes of some focus group interviews for of the differences between teachers and students<br /> triangulation after observation. Among these, will be used to reveal test effects in the<br /> the researcher took note the attitudes of the classroom activities as the following findings<br /> teachers and students and the discussion and discussion.<br /> between the participants when taking tasks were<br /> allocated in order to discover what teachers<br /> used and taught, and how students responded. 4. Findings and discussion<br /> After receiving the permission of some<br /> participants, some formal focus group 4.1. Results of document analysis<br /> interviews were audiotaped and transcribed in<br /> short, and thus translated precisely. As stated in the methodology, document<br /> analysis involved institutional policies on<br /> 3.5. The analysis procedures curriculum, the official course documents,<br /> The analysis of involved a calculation of the methods of assessment and supplementary<br /> amount of time/times was applied to the materials used by teachers. Relevant details of<br /> observation data and Part 1, 3 and 4 of Teacher the analyses are given below.<br /> Questionnaire and Student Questionnaire by - Curriculum and methods of assessment<br />  D.T.P. Hoa / VNU Journal of Science: Education Research, Vol. 36, No. 1 (2020) 1-15 9<br /> <br /> <br /> (a) Teaching contents and methods of for listening/speaking/reading and writing<br /> assessment have been changed. Table 4. skills. The practices are designed by teachers<br /> Illustrates the changes in teaching contents and were short answer questions,<br /> methods of assessment. gap-filling/identifications sentences/paragraphs,<br /> (b) Teachers of English are encouraged to etc. that are identical to EPT.2 of VNFLPF or<br /> use texts taken from journals, books and news practice tests at A1 and A2 level.<br /> Table 4.1. The changes in teaching contents and methods of assessment<br /> <br /> Learning<br /> Summative<br /> Teaching Teaching Teaching outcomes<br /> Teaching Contents Formative assessment<br /> Year hours of hours of Contents of of<br /> of semester 2 assessment (achieveme<br /> semester 1 semester 2 semester 1 University<br /> nt test)<br /> graduation<br /> Questions<br /> and Writing<br /> From Unit 1<br /> From Unit 1 to Answers or Test<br /> to Unit 14 of<br /> Unit 6 of Lifeline Writing (Grammar<br /> 2013 80 periods 55 periods Lifeline<br /> textbook Test and<br /> textbook<br /> (Pre-intermediate) (Grammar Reading<br /> (Elementary)<br /> or Reading exercise)<br /> exercise)<br /> Speaking<br /> Four skills Test/ Writing<br /> Four skills and<br /> and Reading Test<br /> grammar/ EPT.2 of<br /> grammar/ Test/ (Objective<br /> 2017 80 periods 55 periods vocabulary of KNLNNV<br /> vocabulary of Listening test and<br /> KNLNNVN level N<br /> KNLNNVN Test or Writing<br /> 2<br /> level 1 Writing test)<br /> Test<br /> v<br /> The analysis of the official course limitation and mixed-big size class, one of four<br /> documents indicated the official course sub-tests (listening/speaking/reading/writing<br /> documents were set before 2013 for semester 1 test) is applied for both formative assessment<br /> and 2 were not EPT.2 of VNFLPF or practice and semester examinations at NUAE. The<br /> tests at A1 and A2 level. This shows that the analysis indicated that a part of the semester<br /> impact of those on the teaching before 2013. examination focused on testing the mastery of<br /> Since 2014, a new trend has been seen: grammar structures and vocabulary and that<br /> Teachers of English have been encouraged to type of English test had stayed unchanged.<br /> use a variety of authentic materials besides the There have been changes in the nature of the<br /> official course documents. Thus, caution must examination and the changes in question look<br /> be taken when interpreting the official course undifferentiated to EPT.2 and CESOL item<br /> documents. This is also an issue that was types and content. Hence, the interpretation<br /> mentioned in the interview with leaders must be that the semester examinations were<br /> and teachers. shaped on the EPT.2 and CESOL examinations<br /> - Supplementary materials used by teachers in the four sub-tests<br /> Leaders claimed that the formative (listening/speaking/reading/writing test) as far<br /> assessment and semester examinations of as item types and content are concerned.<br /> English are similar to EPT.2 and CESOL tests In short, results of the analysis of the<br /> (apart from the sub-writing of semester supplementary materials practiced by teachers<br /> examinations). However, because of time of English and students indicated they used<br /> 10 D.T.P. Hoa / VNU Journal of Science: Education Research, Vol. 36, No. 1 (2020) 1-15<br /> <br /> <br /> <br /> various authentic materials that including teachers showed that the contents of their<br /> commercial publications, journals, books and teaching focused on four (skills<br /> news for listening/speaking/reading and writing listening/speaking/reading and writing<br /> skills. They covered most Cambridge ESOL activities). The differences respons of Teachers<br /> materials [23] (CESOL for short) that were and Students on teaching and learning activities<br /> available in Vietnam. Teachers and students did in the classroom as the following Table 4.2.1,<br /> not use other kinds of materials (This is dealt Table 4.2.2, Table 4.2.3 and Table 4.2.4.<br /> with in the results of the interview and The survey data in Table 4.2.1 suggested<br /> observation). The effects of CESOL tests were that teachers and students varied little in terms<br /> seen in the official course documents, but these of listening writing practices for English study.<br /> materials were chosen after 2013. Teachers of Only 2 (C2.1.5.1 of Topic and C2.2.B.9.1 of<br /> English tended to use materials from CESOL Participant organization) of the 46 survey items<br /> sources to prepare students for semester had statistically significant differences (p