ASSESSING WRITING
A. Genres of Written Language
1. Academic Writing
Papers and general subject reports
Essays, compositions
Academically focused journals
Short answer test responses
Technical reports (e.g./ lab reports)
Theses, dissertations
2. Job-related Writing
Messages (e.g., phone messages)
Letters/emails
Memos (e.g., interoffice)
Reports (e.g., job evaluations, project reports)
Schedules, labels, signs
Adverisements, announcements
Manuals
3. Personal writing
Letters, emails, greeting cards, invitations
Messages, notes
Calendar, entries, shopping lists, reminders
Financial documents (e.g., tax forms, checks, loan applications)
Forms questionnaires, medical reports, immigration documents
Diaries, personal journals
Fiction (e.g., short stories, poetry)
B. Types of Writing performance
1. Imitative
To produce written language, the learner must attains skills in the fundamental, basic tasks of writing letters, words, punctuations and very brief sentences. This category includes ability to spell correctly and to perceive phoneme-grapheme correspondences in the English spelling system. It is a level at which learners are trying to master the mechanics of writing. At this stage, form is primary if not exclusive focus, while context and meaning are of secondary concern.
2. Intensive (Controlled)
Beyond the fundamentals of imitative writing are skills in producing appropriate vocabulry withing a context, collocations and idioms and correct grammatical features up to the length of a sentence. Meaning and context are of some importance in determining corrrectness and appropriateness but in assessment tasks are more concernd with a focus on form, and are rather strictly controlled by the test design.
3. Responsive
Here, assessment tasks require learners to perform at a limited discourse level, connecting sentences into a paragraph and creating a logically connected sequence of two or three paragraphs. Tasks respond to pedagogical directives, lists of criteria, outliness and other guidelines. Genres of writing include brief narrative and descriptions, shorts reports, lab reports, summaries, brief responses to reading and interpretations of charts or graphs. Under specified conditions, the writer begins to exercise some freedom of choice among alternative forms of expression of ideas. The writer has mastered the fundamentals of sentences-level grammar and is more focused on the discourse conventions that will achieve the objectives of the written text. Form focused attention is mostly at the discourse level, with a strong emphasis on context and meaning.
4. Extensive
Extensive writing implies successful management of all the processes and strategies of writing for all purposes, up to the length of an essay, a term paper, a major research project report, or even a thesis. Writers focus on achieving a purpose, organizing and developing ideas logically, using detalis to support or illustrate ideas, demonstrating syntactic and lexical variety, and in many cases, engaging in the process of multiple draft to achieve a final product. Focus on grammatical form is limited to occasional editing, or proofreading of a draft.
C. Micro and Macro Skills of Writing
1. Microskills
Microskills apply more appropriately to imitative and intensive types of writing task.
a) Produce graphemes and orthograpic patterns of English.
b) Produce writing at an effecient rate of speed to suit the purpose.
c) Produce an acceptable core of words and use apprpriate word order patterns.
d) Use acceptable grammatical system (e.g., tenses, agreement, pluralization, patterns and rules).
e) Express a particular meaning in different grammatical forms.
f) Use cohesive device in written discourse.
2. Macroskills
Macroskills are essential for the successful mastery of responsive and extensive writing.
g) Use the rethorical forms and conventions of written discourse.
h) Appropriately accomplish the communicative functions of written text according to form and purpose.
i) Convey links and connections between events and communicate such relations as main idea, supprting idea, new informations, given informations, generalization, and exemplification.
j) Distiguish between literal and implied meanings when writing.
k) Correctly convey culturally specific references in the context of the written text.
l) Develop and use a battery of writing strategies, such as accurately accessing the audience’s interpretation, using prewriting devices, writing with fluency in the first draft, using paraphrases and synonyms and soliciting peer and instructor feedback, and using feedback for revising and editing.
D. Designing Assessment Tasks
1. Imitative Writing
Imitative writing is used for the beginning level English learner which needs basic training in and assessment of imitative writing: the rudiments of forming letters, words, and simple sentences. We examine this level of writing first.
a) Task in (Hand) writing letters, words, and punctuation
· Copying
· Listening cloze selection task
· Picture-cued task
· Form comletion task
· Converting numbers and abbrebiations to words
b) Spelling task and detecting phoneme-grapheme correspondences
· Spelling test
· Picture cued-task
· Multiple choices techniques
· Matching phonetics symbols
2. Intensive (Controlled) Writing
This next level of writing is what second language teacher training manuals have for decades called controlled writing. It may also be thought of as form focused witing, grammar writing, or simply guided writing. A good deal of writing at this level is display writing as opposed to real writing: students produce language to display their competence in grammar, vocabulary, or sentence formation, and not necessarily to convey meaning for an authentic purpose. The traditional grammar/vocabulary test has plenty of display writing in it, since the response mode demonstrates only the test-taker’s ability to combine or use words correctly. No new information is passed on form one person to the other.
a) Dictation and Dicto-Comp
b) Grammatical transformation tasks
c) Picture cued tasks
· Short sentences
· Picture description
· Picture sequence description
d) Vocabulary assessment tasks
e) Ordering tasks
f) Short answer and Sentence completion tasks
3. Responsive and Extensive
In this section we consider both responsive and extensive writing tasks. They will be regarded here as a continuum of possibilities ranging from lower-end tasks whose complexity exceeds those in the previous category of intensive or controlled writing, through more open-ended tasks such as writing short reports, essays, summaries, and responses, up to texts of several pages or more.
a) Paraphrasing
b) Guided question and answer
c) Paragraph construction tasks
· Topic sentence writing
· Topis development within a paragraph
· Development of main and supporting ideas across paragraphs.
d) Strategic options
· Attending to task
· Attending to genre
E. Test of Written English (TWE)
One of the number of internationally available standarized tests of writing ability is the Test of Written English (TWE). Established in 1986, the TWE has gained a reputation as a well-respected measure of written English, and a number of research articles supports its validity ( Frase et al., 1999; Hale et al., 1996; Myford et al., 1996). In 1998, a computer-delivered version of the TWE was incorporated into the standard computer-based TOEFL and simply labeled as the “writing” section of the TOEFL. The TWE is still offered as a separate test especially where only the paper based TOEFL is available. Conrrelations between the TWE and TOEFL scores (before TWE became a standard part of TOEFL) were consistently high, ranging from 57 to 69 over 10 test administrations from 1993 to 1995.
The TWE is in the category of a timed impromptu test-takers are under a 30 minute time limit and are not able to prepare ahed of time for the topic that will appear. Topics are prepared by a panel of experts following specifications for topics that represent commonly used discourse and thought patterns at the university level.
Test preparation manuals such as Deborah Phillips’s Longman Introductory Course for the TOEFL test (2001) advice TWE test takers to follow six steps to maximize success on the test:
a) Carefully identify the topic.
b) Plan your supprting ideas.
c) In the introductory paragraphn, restate the topic and state the organizational plan of the essay.
d) Write effective supporting paragraphs (show transitions, include a topic sentence, specify details).
e) Restate your position and summarize in the concluding paragraph.
f) Edit sentence structure and retorical expression,
The scoring guide for the TWE follows a widely accepted set of specifications for a holistic evaluation of an essay. Each point on the scoring system is defined by a set of statements that address topic, organization and development, supporting ideas, facility (fluency, naturalness, appropriateness) in writing, and grammatical and lexical correctness and choice.
F. Scoring Methods For Responsive and Extensive Writing
1. Holistic Scoring
The TWE scoring scale is a prime example of holistic scoring. Each point on a holistic scale is given a systemtic set of descriptors, and the reader-evaluator matches on overall impression with the descriptors.
Holistic scoring is a method by which trained readers evaluate a piece of writing for its overall quality. The holistic scoring used in Florida requires readers to evaluate the work as a whole, while considering four elements: focus, organization, support, and conventions. This method is sometimes called focused holistic scoring. In this type of scoring, readers are trained not to become overly concerned with any one aspect of writing but to look at a response as a whole.
· Focus
Focus refers to how clearly the paper presents and maintains a main idea, theme, or unifying point. Papers representing the higher end of the point scale demonstrate a consistent awareness of the topic and do not contain extraneous information.
· Organization
Organization refers to the structure or plan of development (beginning, middle, and end) and whether the points logically relate to one another. Organization refers to (1) the use of transitional devices to signal the relationship of the supporting ideas to the main idea, theme, or unifying point and (2) the evidence of a connection between sentences. Papers representing the higher end of the point scale use transitions to signal the plan or text structure and end with summary or concluding statements.
· Support
Support refers to the quality of the details used to explain, clarify, or define. The quality of support depends on word choice, specificity, depth, credibility, and thoroughness. Papers representing the higher end of the point scale provide fully developed examples and illustrations in which the relationship between the supporting ideas and the topic is clear.
· Conventions
Conventions refer to punctuation, capitalization, spelling, and variation in sentence used in the paper. These conventions are basic writing skills included in Florida's Minimum Student Performance Standards and the Uniform Student Performance Standards for Language Arts. Papers representing the higher end of the scale follow, with few exceptions, the conventions of punctuation, capitalization, and spelling and use a variety of sentence structures to present ideas.[1]
- Development
Developments are all major ideas are set off by paragraphs which have clearly stated or implied topics; the main idea and all major topics are supported by concrete, specific evidence.
- Style
Style is sentences relate to each other and to the paragraph topic and are subordinate to the topic; word and phrase choice is felicitous; tone is consistent and appropriate.
- Correctness
Correctness there are no major mechanical errors (e.g., agreement) and only a few minor errors (e.g., spelling).
- References
References are source material is incorporated logically, insightfully and elegantly; sources are documented accurately, elegantly and emphatically.[2]
Advantages of holistic scoring include:
· Fast evaluation
· Relatively high inter-rater reliability
· The fact that scores represent “standards” that are easily interpreted by lay persons
· The facts that scores tend to emphasize the writer’s strengths
· Applicability to writing across many different disciplines
It’s disadvantegs must also be weighed into a decision on whether to use holistic scoring:
· One scores masks differences across the subskills within each score
· No diagnostic information is available (no washback potential)
· The scale may not apply equally well to all genres of writing
· Raters need to be extensively trained to use the scale accurately
2. Primary trait scoring
A second method of scoring, primaty trait, focuses on “how well students can write within a narrowly defined range of discourse”. This type of scoring emphasizes the task at hand and assigns a score based on the effectiveness of the text’s achieving that one goal. In summary, a primary trait score would assess:
· The accuracy of the account of the original (summary)
· The clarity of the steps of the procedure and the final result (lab report)
· The description of the main features of the graph (graph description), and
· The expression of the writer’s opinion (response to an article)
3. Analytic Scoring
For classroom instruction, holistic scoring provideslittle washback into the writer’s further stage of learning. Primary trait scoring focuses on the principal function of the text and therefore offers some feedback potential, but no washback for any of the aspects of the written production that enhance the ultimate accomplishment of the purpose. Classroom evaluation of learning is best served through analytic scoring, in which as many as six major elements of writing are scored, thus enabling learners to home in on weakness and capitalized on strengths.
Analytic scoring may be more appropriately called analytic assessment in order to capture its closer association with classroom language instruction than with formal testing. Brown and Bailey (1984) designed an analitical scoring scale that specified five major categories and a description of five different level in each category, ranging from “unacceptable” to “excellent”.Here are the five categories:
1. Organizations (Introduction, body and conclusion)
2. Logical Development of Ideas (Content)
3. Grammar
4. Punctuation, Spelling and Mechanics
5. Style and Quality of Expression
G. Assessing Initial Stages of The Process of Composing
1. Focus your efforts primarily on meaning, main idea, and organization.
2. Comment on the introductory paragraph.
3. Make general comments about the clarity of the main idea and logic or appropriateness of the organization.
4. As a rule of thumb, ignore minor (local) grammatical and lexical errors.
5. Indicate what appear to be major (global) errors.
6. Do not rewrite questionable, ungrammatical, or awkward sentences; rather. Probe with a question about meaning.
7. Comment on features that appear to be irrelevant to the topic.
H. Assessing Later stagess of The Process of Composing
1. Comment on the specific clarity and strength of all main ideas and supporting ideas, and on argument and logic.
2. Call attention to minor (“local”) grammatical and mechanical (spelling, punctuation) errors, but direct the writer to self-correct.
3. Comment on any further word choices and expressions that may not be awkward but are not as clear or direct as they could be.
4. Point out any problems with cohesive devices within and accross paragraphs.
5. If appropriate, comment on documentation, citation of sources, evidence, and other support.
6. Comment on the adequacy and strength of the conclusion.
Assessing Speaking
From a pragmatic view of language performance, listening and speaking are almost always closely interrelated. Speaking is a productive skill that can be directly and empirically observed, those observations are invariably colored by the accuracy of effectiveness of a test-taker’s listening skill, which necessarily compromises the reliability and validity of an oral production test. The interaction of speaking and listening challenges the designer of an oral production test to tease apart, as much as possible, the factor accounted aural intake.
Another challenge is the design of elicitation techniques. Because speaking is product of creative construction of linguistic strings, the speaker make choices of lexicon, structure, and discourses. If your goal is to have test-taker’s demonstrate certain spoken grammatical categories. In receptive performance, the elicitation stimulus can be structured to anticipate predetermined responses and only those responses. In productive performance, the oral or written stimulus must be specific enough to elicit output within an excepted range performance such that scoring or rating procedures apply appropriately. For example, in a pictures-series task, the objective of which is to elicit output within a story in a sequence of events, test-takers could opt for a variety of plausible ways to tell the story, all of which might be equally accurate.
1. Imitative speaking
At one end of a continuum of types of speaking performance is the ability to simple parrot back imitate a word or phrase or possibly a sentence. This is a purely phonetic of level of oral production, a number of prosodic, lexical, and grammatically properties of language may be included in the criterion performance. Test takers are assessed if they have the ability to imitate a word while pronouncing. However conveying the meaning is not the purpose. The role of listening here is the short term storage.
Phonepass Test
A popular test that use imitative production task is phonepass, a widely used, commercially available speaking test in many countries. Among a number speaking task on the test, repetition of sentences occupies a prominent role. It is remarkable that research on the phonepass test has supported the construct validity of its repetition tasks not just for test-takers phonological ability but also for discourse and overall oral production ability (Townshend et al., 1998;Bernstein et al,. 2000; Cascallar & Bernstein, 2000)., The phonepass findings could signal an increase in the future use of repetition and read aloud procedures for the assessment of oral production. Because a test-takers output is completely controlled, scoring using speech recognition technology become achievable and practical.
2. Intensive Speaking
A second type of speaking frequently employed in assessment context is the production of short stretches of oral language designed to demonstrate competence in a narrow band of grammatical, phrasal, lexical, or phonological relationship. The intensive level, teat- takers are prompted to produce short stretches of discourse(no more than one sentence) through which they demonstrate linguistic ability at a specified level of language.
Direct response Tasks
The test administrator elicits a particular grammatical form or a transformation of sentence. Such tasks are clearly mechanical and not communicative, but they do require mini al processing of meaning in order to produce the correct grammatical output.
Read-Aloud Tasks
Intensive reading-aloud tasks include reading beyond the sentence level up to a paragraph or two. This technique is easily administered by selecting passage that incorporate test specs and by recording test-takers output; the scoring relatively easy because all of test-takers oral production is controlled.
Sentence/ Dialogue Completion Tasks and Oral Questionnaires
Test-takers are first given time to read through the dialogue to get its gist and to think about appropriate lines to fill in.
Picture-Cued Tasks
One of the more popular ways to elicit oral language performance at both intensive and extensive levels is a picture-cued stimulus that requires a description from the test-takers. Pictures may be very simple, designed to elicit a word or a phrase; somewhat more elaborated and “busy” ; or composed of a series that tells story or incident.
Translation
Translation is a part of our tradition in language teaching that we tend to discount or disdain, if only because of our current pedagogical stance plays down its importance. Translation method of teaching are certainly passé in an era of direct approaches to creating communicative classrooms. We should remember that in where English is not the native or prevailing language, translation is a meaningful communicative device in context where the English user is called on to be an interpreter.
3. Responsive Speaking
Responsive assessment tasks include interaction and test comprehension but at the somewhat limited level of very short conversations, standard greeting and small talk, simple requests and the like. It helps English language instructors assess students’ ability to engage in discussion with one or more interlocutors. More creativity on the students’ part is required.
Question and Answer
Question and answer task can consist of one or two questions from an interviewer, or they can make up a portion of a whole battery of questions and prompts in an oral interview. This form of assessment requires one person, an interlocutor, to ask the student questions. Students’ answers can be 1 sentence responses, or, can entail a more detailed answer depending upon the question asked.
Giving Instructions and Directions
This type of assessment entails a student asking the teacher questions. Essentially, the student assumes control over the conversation. This type of assessment requires students to be able to give directions given a particular situation. Topics should be relevant and of .immediate interest to students.
Paraphrasing
This type of assessment requires students to listen to a 4 to 6 sentence paragraph. Upon finishing the paragraph, instructors prompt students to give a 1 to 2 sentence summary of what they just heard.
4. Interactive Speaking
Interactive speaking is extremely similar to responsive speaking, however varying in that an interactive speaking session is much more involved, may include multiple interlocutors, and is commonly found in the English speaking world. Interactive speaking assessments are important because they allow instructors to evaluate students’ ability in producing fluid, detailed, and in-depth discussions with one or more interlocutors.
Interview
‘oral production assessment’ is mentioned the first things that comes to mind is oral interview : a test administrator and a test-taker sit down in a direct face to face exchange and proceed through a protocol of questions and directives. Interviews provide students with chances to use what they have learned in an authentic situation; they give students the chance to engage in meaningful dialogue.
Role Play
Role playing is a popular pedagogical activity in communicative language-teaching classes. Within constrains sets forth by guidelines, it frees students to be somewhat creative in their linguistic output. Role play opens some windows of opportunity for test-takers to use discourse that might otherwise be difficult to elicit. Role playing provides students with a chance to have fun with the English language. Moreover, it provides students with mock situations that mimic real-world situations.
Discussion and Conversation
Discussion and conversation with and among are difficult to specify and even more difficult to score. Assessing the performance of participants through scores or checklist course, discussion is an integrative task, and so it is also advisable to give some cognizance to comprehension performance in evaluating learners.
Games
Games are usually the easiest way to get students engaged in learning English. One type of interactive game assessment is information gap grids. Information Gap Grids In this game, students interview each other, in English, to determine the answers to various questions.
5. Extensive Speaking
Extensive speaking tasks involve complex, relatively lengthy stretches of discourse. They are frequently variations on monologues, usually with minimal verbal interaction. Extensive speaking is extreme important; students are left by themselves to produce clear and intelligible speech. There usually is some type of audience present; there is no dialogue between presenter and audience members. Only the person presenting speaks during this time.
Oral Presentations
Oral presentations would not be uncommon to be called on to present a report, a paper, a marketing plan, a sales idea, a design of new product, or method in academic a professional arenas. Oral presentations give students a chance to use what they have learned in English class by culminating everything into one strong and concise presentation.
Picture Cued Story telling
Picture cued story telling is one of the most techniques for eliciting oral production is through visual picture, photographs, diagrams, and charts. The purpose of picture-cued story telling is to provide students with examples of how chronology is used in discussions. Also, it is used to illustrate situations.
Retelling Story
This is type of task in which test-takers hear or read a story or news event that they are asked to retell. In this form of extensive speaking assessment, students re-tell a story they heard. The story can be modified from its original form. We have already looked at this elicitation device for intensive tasks, but at this level we consider a picture or a series of picture as a stimulus for a longer story or description. If we are eliciting specific grammatical or discourse features, you might add to the directions like “tell the picture that these pictures describe. Using the past tense of verb” .
Translation
Translation of words, phrases, or short sentences was mentioned under the category of intensive speaking. The longer texts are presented for the test-takers to read in native language and translate into English
