LANGUAGE TEACHING EVALUATION MID TERM PROJECT

Analyzing senior high school Mid term test

By: SYILVIA WENNY J. 1103757

ENGLISH EDUCATION POST GRADUATED PROGRAM PADANG STATE UNIVERSITY PADANG 2012

A. Introduction

Education aims to develop students ability, knowledge, skills, etc. In order to reach those, education needs curriculum that contains a set of plan and role of content, material and teaching learning method. Curriculum is used as a basis of teaching learning process. It also contains a tool that is used to know how far students reach the goal. It is called evaluation. In evaluating students, teacher needs some tests that have some purposes, such as for placement, for continuing to the higher level of education, etc. Test consists of written test and spoken test. Written test is usually used to know students ability in mastering writing skill, grammar, etc. While spoken is sometimes used to know students ability in speaking and responding the listener. Students prefer in doing written test to spoken test. For them, written test does not make them more nervous than spoken test. Then, when they do not know how to answer written test, they may not answer although the grades will be decreased. It does not happen when they meet spoken test, they feel hard to say I dont know in front of their teacher. For teachers, written test has some weaknesses. Firstly, they have to have much money in order to copy the question and answer sheet. Then, they have to have much time in correcting the answers made by students. And, sometimes some students are not appropriate with the grade they got as a result of the test to the process their learning process. It does not happen when teachers have the spoken test. Based on the issues above, the paper focuses on written test. In designing test, especially written test, the teacher needs some considerations in order to fulfill their needs in grading students. The test must be shown as clear as possible in order to avoid misunderstanding of student. It is one of the considerations. And, the principles in designing test are practicality, reliability, validity, authenticity and washback. In this paper, the writer will analyze a test that is used for mid term test of Private SMA Padang for English subject. The analyzing will show whether the test has accommodated all aspects of English language learning and assessment that have been mentioned before.

B. Review of the Related Literature

Syilvia Wenny J (1103757). MID TERM TEST OF LTE

1. Definition of Test Test is usually used when learning process has been done. It is used to know whether students have mastered the lessons that have been taught or not. Brown (2010) states that test are prepared administrative procedures that occur at identifiable times in a curriculum when learners master all their faculties to offer peak performance. There are some components of a test, first, method is a set of techniques, procedures, or items that requires performance on the part of test-taker. The method must be explicit and structured in order to make teachers are easy to analyze the result of test. And students are easy to understand what must be answered. Then, test must measure as a process of quantifying a test-takers performance according to explicit procedures or rules. Test can measure the general ability of students, and also measure specific competencies or specific objectives. For example, when students want to face the last examination in order to continue the higher level, they will be tested their general ability on each subject. If the subjects of test is English, they will be tested their ability in listening, speaking, reading and writing. It also happens to others subject on different ability. Test can measure the specific competencies or objectives, such as writing test. After having the test, the results will be analyzed whether they have the coherent, unity, and other aspects that must be measured in writing test. Moreover, a test measures an individuals ability, knowledge or performance. Before measuring the students, teachers must be sure that the students have been taught the knowledge and ability. If the teachers do not know what have been taught, it could be harmful in scoring the students, because students have their own perception of knowledge, and it will not have synchronize between the knowledge given to knowledge that students have before. In testing students performance, teachers must keep attention to their performance. Teachers must be aware students words choice, students grammar, etc. Those can be a consideration in scoring them. Test measures a given domain. Test has to have the particular section to measure which particular criteria. For example, a vocabulary test may focus on only

Syilvia Wenny J (1103757). MID TERM TEST OF LTE

the set of words covered in a particular lesson or unit. Another example is writing test that focus only particular genre of a text. It will help students to be focused in studying. 2. Types and Purposes of the Test Before designing test, it is better for teachers to determine the purpose and types of the test. The impact of doing this is making teachers easy in making decisions for the students. Bachman (1990) states that since the basic purpose of tests in educational programs is to provide information for making decisions, the various specific uses of tests can be understood by considering the types of decision to be made. Brown and Abeywickrama (2010) classify the types and the purposes of the test include: a. Achievement Test Achievement test is measuring students ability within a classroom lesson unit, or even total curriculum. This test is used to determine whether course objectives have mastered at the end of a given period in instruction. This is a reason why this test is often summative. The achievement test should be determined by the objective of the lesson or course, the relative importance assigned to each objective, the tasks employed in classroom lessons during the unit of time, the time frame for the test and turn around time and its potential for formative feedback. b. Diagnostic Test Diagnostic test is used to diagnose aspects of a language that a student needs to develop or that a course should include. For example in vocabulary test, teachers may diagnose the choice of words that the students made in sentence. Diagnostic test seems similar to the achievement test. In fact, achievement test analyze the extent to which students have acquired language features that have already been taught, while diagnostic test should elicit information on what students need to work in the future.

c. Placement Test

Syilvia Wenny J (1103757). MID TERM TEST OF LTE

Placement is to place student into a certain level or section of a language curriculum or school. Bachman (1990) states that in many language programs students are grouped homogeneously according to factors such as level of language ability, language aptitude, language use needs, and professional or academic specialization. Decisions regarding the placement of students into appropriate groups must be made. d. Proficiency Test Proficiency test is to know students global competence in a language. It tests overall ability. It is always summative and norm-referenced. e. Aptitude Test Aptitude test is designed to measure capacity or general ability to learn a foreign language a priori (before taking a course) and ultimate predicted success in that undertaking. 3. Principle of Language Assessment Before administering the test to the students, teachers have to determine the principles of language assessment. It is done to know how effective, appropriate or useful for the students and teachers itself. Those principles are (Brown, 2010): a. Practicality Practicality refers to the logistical, down to earth, administrative issues involved in making, giving, and scoring an assessment instrument. These include costs, the amount of time it takes to construct and to administer, ease of scoring and ease of interpreting/reporting the results. There are some examples of impractical test. First, if the test needs students to have five or more hours in doing that, it can be said impractical. Then, when the result of test can be analyzed by the computer only, it can be said impractical. Moreover, if teachers need much money in designing test, it also can be said impractical. And there are still many more examples that show impractical test.

b. Reliability
Syilvia Wenny J (1103757). MID TERM TEST OF LTE

A reliable test is consistent and dependable. If teachers give students the same test in different time, and the result is similar, it can be said that the test is reliable. Some factors regarding reliability are the student, the scoring, the test administration and the test itself. c. Validity Validity is the extent to inference made from assessment results is appropriate, meaningful and useful in terms of the purpose of the assessment. A valid test measures exactly what it purposes to measure. It does not measure irrelevant or contaminating variables. It relies as much as possible on empirical evidence. It involves performance that samples the tests criterion. It offers useful, meaningful information about a test-takers ability. And it is supported by a theoretical rationale or argument. There are some evidences in measuring validity test; content related evidence, criterionrelated evidence, construct related evidence, consequential validity and face validity. d. Authenticity Authenticity is the degree of correspondence of the characteristics of a given language test task to the features of a target language task and then suggested an agenda for identifying those target language tasks and for transforming them into valid test items. Authentic test contains language that is as natural as possible. It has items that are contextualized rather than isolated. It includes meaningful, relevant, interesting topic. It also provides some thematic organization to items, such as through a story line or episode. Then it offers tasks that replicate real world tasks. e. Wash back Washback is the effect of testing on teaching and learning. Washback influences what and how teachers teach. It influences what and how learners learn. Then, it offers learners a chance to adequately prepare. It also gives learners feedback that enhances their language development. Moreover, it is more formative in nature than summative. And, it provides conditions for peak performance by the students.

Syilvia Wenny J (1103757). MID TERM TEST OF LTE

4. Standard Based Assessment Standard based assessment refers to procedures that have been specifically designed to test such competencies. Indonesia has the standard based education. It makes all school have to follow those standards that is shown on curriculum. The test will be designed based on the curriculum so that teachers and students are easy to master certain objective or competencies. Standardized test offers high levels of practicality and reliability and are often supported by impressive construct validation studies. It is capable of accurately placing hundreds of students onto a normreferenced scale with high reliability ratios. 5. Design Classroom Language Test When deigning, administering and making maximum tests in the classroom, teachers need to considers some considerations. Those considerations are formed into several questions, which are: a. What is the purpose? b. What are the objectives of the test? c. How will the test specifications reflect both the purpose and the objectives? d. How will the test item types (tasks) be selected and the separate items arranged? e. In administering the test, what details should teacher attend to in order to help students achieve optimal performance? f. What kind of scoring, grading and/or feedback is expected? There some considerations in designing the communicative test. Weirr (1990) mentions that there are at least seven items that should be considered significantly in design of communicative test. a. Realistic context the test tasks should be regarded as appropriate to the candidates situation. b. Relevant information gapcandidates should have to process new information as they might in the real life situation c. Intersubjectivitytasks should involve candidate both as language receiver and language producers. In addition, the language produced by the candidate

Syilvia Wenny J (1103757). MID TERM TEST OF LTE

should be modified in accordance with what their expectations of the addressee are perceived to be. d. Scope for development of activity by the candidatesthe tasks should give candidates the chance to assert their community independence and allowance should also be made for the creative unpredictability of communication in the tasks set and the marketing schemes that are applied. e. Allowance for self monitoring by candidatesthe tasks should allow candidates to use their discourse processing strategies to evaluate their communicative effectiveness and make any necessary adjustment in the course of an event. f. Processing of appropriately sized inputthe size and scope of the activities should be such that they are processing the kinds of input they would normally be expected to. g. Normal time constraint operativethe tasks should be accomplished under time constraints. Besides, Weir (1990) also gives suggestions about the test task dimensions which may be useful into consideration by the teachers or test administer: a. Size of textthe length of text, receptive or productive, that involved in the event. b. Grammatical complexity and range of cohesion devices required in the event. c. Functional rangethe degree of variety of illocutionary acts involved in the event d. Referential rangethe breadth and depth of lexical knowledge required to handle activities in the event 6. Communicative Language Ability Bachman (1990) states that communicative language ability (CLA) can be described as consisting of both knowledge or competence, and the capacity for implementing or executing that competence in appropriate, contextualized language use. In designing test, teachers need to complete the communicative language. CLA

Syilvia Wenny J (1103757). MID TERM TEST OF LTE

consists of language competence, strategic competence and psycho-physiological mechanisms. Language competence includes organizational competence, which consists of grammatical competence and textual competence, and pragmatic competence, which consists of illocutionary and sociolinguistic competence. Strategic competence is seen as the capacity that relates language competence, or knowledge of language, to the language users knowledge structures and the features of the context in which communication takes place. Strategic competence performs assessment, planning, and execution functions in determining the most effective means of achieving a communicative goal.

Syilvia Wenny J (1103757). MID TERM TEST OF LTE

C. Test Analysis 1. Purpose and Types of the Test

The Heading Form of Mid Semester Test of Senior High School

The writer will analyze the mid semester test of senior high school. The test is for the second grade students. The test consists of listening test; match what students hear to the appropriate options in question sheets and reading test. There is no writing and speaking test. The kind of this test is achievement test. The function of this test is to measure students achievement and ability. Moreover, this test is used to know whether the students have reached what they have been taught. 2. Principles of Language Test Assessment Before designing the test, teacher must consider the six principles of test that has been explained above. They are practicality, reliability, validity, authenticity, wash back and efficiency. The test will be analyzed based on those principles from Brown and Abeywickrama (2010). a. Practicality Practicality is regarded by teachers (and the students) time constraints, costs, and administrative details and to some extent by what occurs before and after the test. In determining whether the test is practicality, teachers need to consider the following considerations: Are administrative details all carefully attended to before the test? Can students complete the test reasonably within the set time frame? Can the test be administered smoothly, without procedural glitches? Are all printed materials accounted for?
Syilvia Wenny J (1103757). MID TERM TEST OF LTE

Has equipment been pre-tested?... Is the cost of the test within the budgeted limits? Is the scoring/evaluation system feasible in the teachers time frame? Are methods for reporting results determined in advance? Based on those considerations, administrative details have been attended

before the test. It makes the time is not wasteful for the students in completing the test. In order words, students also can complete the test within the set time frame. This test provides 75 minutes for 40 items. Therefore, the students have approximately 2 minutes in doing each item. The test is administered well, and there is no any problem in procedural. After that, all materials have been accounted smoothly. School budgets the cost test well, so there would not be out of budgeted spent. Also, teachers are feasible in scoring the test. On the other hand, the test did not pre-test, but the test was designed based on the curriculum. And for reporting result, the analyzer did not find any information. Based on those considerations, it can be concluded that the test is generally practical. As an additional information, this test is written in two columns, the font used is times new roman, and the font size is 11 and use A4 paper. b. Reliability Reliability applies to the students, the test administration, the test itself, and the teacher. The guidance in determining whether the test is reliable is: Does every student have a cleanly photocopied test sheet? Is sound amplification clearly audible to everyone in the room? Is video input clearly and uniformly visible to all? Are lighting, temperature, extraneous noise, and other classroom conditions equal (and optimal) for all students? For closed-ended responses, do scoring procedures leave little debate about correctness of an answer? The test is photocopied cleanly and clearly. It makes students easy to read the test. Based on the teacher who teaches this subject, the sound for listening is not clear enough but students can still understand the audio. The test does not use video, so this question is not used. The test uses multiple choices only, so

Syilvia Wenny J (1103757). MID TERM TEST OF LTE

there is no debate about the correctness of an answer that is usually derived from close-ended response. To sum up, this test is reliability. c. Validity Validity is extent to which the assessment requires students to perform tasks that was included in the previous classroom lessons and that directly represent the objectives of the unit on which the assessment is based. Several steps that determine the validity, they are: Are unit objectives clearly identified? Are unit objectives represented in the form of test specifications? Do the test specifications include tasks that have already been performed as part of the course procedures? Do the test specifications include tasks that represent all (or most) of the objectives for the unit? Do those tasks involve actual performance of the target task? This test clearly identifies the objectives. Those objectives understand the narrative text, note, announcement, private letter and job application. The specification of the task also shows in the test. Then, the test generally is valid to measure the students competence. The strategies on how to answers are also shown in the first form and it is written in Indonesian. To sum up, the test is valid. d. Authenticity The considerations in determining the authenticity of a test are: Is the language in the test as natural as possible? Are items as contextualized as possible rather than isolated? Are topics and situations interesting, enjoyable, and/or humorous? Is some thematic organization provided, such as through a story line or episode? Do tasks represent, or closely approximate, real-world tasks?

This test uses natural language in telling the narrative text, note, announcement, private letter and job application. Those kinds of text are

Syilvia Wenny J (1103757). MID TERM TEST OF LTE

commonly used in the real life. Then, the topics that is written is enjoyable so that students can read it well while relate them into the students imagination. By those explanations, it can be said that this test has the authenticity. e. Washback Washback provides on how tests influence both teaching and learning. Washback usually can be done after having the result test. Here are some questions in analyzing the washback of a test: Is the test designed in such a way that you can give feedback that will be relevant to the objectives of the unit being tasted? Have the test-maker given students sufficient pre-test opportunities to review the subject matter of the test? In the written feedback to each student, do you include comments that will contribute to students formative development? After returning tests, do you spend class time going over the test and offering advice on what students should focus on in the future? After returning tests, do you encourage questions from students? If time and circumstances permit, do you offer students a chance to discuss results in an office hour? All of the questions above are not available for this test. Based on the teachers information, this test is designed by teachers forum. It makes a hard to get the washback that needs interaction between the test-maker and students. The information is only got if students feel unsatisfied in completing the test. In addition teacher sometimes reads the test and asks the students what are the problems that found on the test. When the problems have been collected, teachers just give some comments to the forum, and sometimes the forum considers the problem and sometimes not all problems are considered.

3. Designing Classroom Language Test

Syilvia Wenny J (1103757). MID TERM TEST OF LTE

Brown and Abeywickrama (2010) state that for the purpose of making practical applications, test-makers need to consider four scenarios as they proceed through the six steps explained before for designing an assessment. Those scenarios are reading quiz, grammar unit test, midterm test and listening/speaking final exam. According to Brown and Abeywickrama, there are some considerations, which are: Provide clear directions. Write a prompt on either a narrative or a descriptive essay. The prompt must be on familiar topic that students will, with a reasonable level of confidence, be able to write coherently about. Assign an expectation of a full page, handwritten, and no more than two full pages. Students will be allowed a 90 minute time limit. In the prompt, include evaluation criteria: content, organization, rhetorical discourse and grammar/mechanics. The final grade is to include four sub scores: content, organization, rhetorical discourse and grammar/mechanics. Based on those specifications, this test has clear directions that use Indonesian so that students would understand the directions. The writer doesnt write directly the prompt on either narrative or description, but students can identify the text because they have been taught before. The text is familiar for them. The time is allocated for 75 minutes. This test can be said that it is having been fulfilled to be a good midterm test. The content of this test are made in the form of reading test. The test maker used a multiple-choice. The test consists of 40 questions and shows five choices of answer (a, b, c, d and e). The test includes two skills of language, listening and reading. There is no information how to score the result of this test made by students. By analyzing this test, we may make assumption that they final score will be 100, so that the score of each item is 2.5.

Syilvia Wenny J (1103757). MID TERM TEST OF LTE

For standard based assessment, we analyze the test based on KTSP (Kurikulum Tingkat Satuan Pendidikan). The competence standards on this test are short functional text. The test-maker made this test based on those competence standards. We can see each items relate to all materials which can be seen in curriculum and competence standards.

Syilvia Wenny J (1103757). MID TERM TEST OF LTE

D. Conclusion and Suggestion After analyzing the test of mid term test for the second grade of senior high school, it can be concluded that: 1. The test is an achievement test that measures the ability and capacity of the students. 2. There are components that relate to principle of language assessment; practicality, reliability, validity, authenticity and washback. The test almost has all the components. But the washback does not complete those principles. 3. The design of language test is appropriate so that the test can be distributed. 4. The test is based on KTSP. 5. The test-maker made the test appropriate with communicative language test.

Syilvia Wenny J (1103757). MID TERM TEST OF LTE

REFERENCES

Bachman, Lyle F. 1990. Fundamental Consideration in Language Testing. Oxford: Oxford University Press Brown, H. Douglas and Abeywickrama. 2010. Language Assessment Principles and Classroom Practices. Pearson Education.

Weir. Cyril J. 1990. Communicative Language Testing. Prantice Hall

Syilvia Wenny J (1103757). MID TERM TEST OF LTE