Naplan Writing Marking Criteria For Assignment

Early May would be incomplete without some NAPLAN controversy. This year’s comes from the announcement last week that the national exam sat by students across the country in Years 3, 5, 7 and 9 is to be marked by computers in 2017.

Part of the argument for moving to online marking is that it will decrease turnaround time from months to just weeks. While this is uncontroversial for multiple-choice-style tests, which have a correct answer, it is much more problematic when applied to creative writing.

Can computers mark creative writing?

The NAPLAN written task is usually a narrative or persuasive task and is an extended piece of prose. The marking criteria include audience, text structure, cohesion, vocabulary, paragraphing, sentence structure, punctuation and spelling.

When writing persuasive texts, the guide explains that:

students are required to write their opinion and to draw on personal knowledge and experience when responding to test topics.

The guide also explains that for narrative texts, there should be a:

growing understanding that the middle of the story needs to involve a problem or complication that introduces conflict, danger or tension that must be resolved. It is this uncertainty that draws the reader in and builds suspense.

The question is whether computers can appropriately mark students’ creative writing with this level of sophistication.

According to the Australian Curriculum, Assessment and Reporting Authority (ACARA), they can.

The approach being taken is one that uses supervised machine learning, where sample tests marked by humans are fed into an algorithm that learns how to recognise quality responses by reverse-engineering scoring decisions. Trials conducted by ACARA have demonstrated that:

artificial intelligence solutions perform as well, or even better, than the teachers involved.

One argument is that computer marking has less variability than human markers, although these claims to marker reliability are contested.

For example, what would happen if a student were to submit a nonsense piece that happened to meet the expectations of the algorithm?

Automated marking is not a new thing. It has been particularly visible since the rise of MOOCs and the search for a cheap alternative for marking student papers.

The research literature provides a mixed picture of potential benefits and pitfalls, yet there has been vocal opposition to computer marking from academics and educationalists.

The rise of algorithms can be seen in many places, including chess-playing computers, self-driving cars, metadata analysis to predict behaviour, online advertising, speech-recognition software and auto-completing search engines. It seems only logical that algorithms would enter our classrooms.

What actually matters in education?

One thing that strikes me as ironic is that we would be using computers, which can’t actually read or write, to test the reading and writing of our students. Is the next step to replace our teachers with robot instructors who can provide standardised, objective and completely emotionless feedback in the classroom?

How can a computer assess creativity and flair? How would it recognise irony, wit and humour? What about writers who use unconventional approaches for effect?

While algorithms can easily process literal meaning, what happens with inferential meaning or drawing on rich contexts, background knowledge, prior learning, cultural and social discourses? These are all part of the complex tapestry of human meaning-making in reading and writing.

As one example, the NAPLAN marking guide refers to the use of classical rhetorical discourse in persuasive writing, including:

Pathos - appeal to emotion

Ethos - appeal to values

Logos - appeal to reason.

I have not yet come across a computer except in science-fiction films that has emotions or values that could be appealed to in any persuasive sense.

There are serious concerns that computer marking of the NAPLAN writing task will have unintended effects on teaching and learning, including online reading and writing strategies different to those of traditional print-based comprehension and composition.

A further concern is that computer marking will have a reductive effect on student writing, with “teaching to the test” becoming more of a problem than it already is.

Maybe it isn’t that far-fetched to imagine computers marking assignments and robots teaching classrooms. After all, there are predictions that we will reach the singularity, the point at which artificial intelligence overtakes humans, in 2029.

Wouldn’t ACARA be better off putting the money into something that has an impact on the quality of learning of students in Australian schools rather than conducting this particular experiment? To be focusing on test scoring that is faster and cheaper seems to be at odds with what actually matters in education.

Until we reach the singularity, perhaps we should focus on improving equity and access for students who are most disadvantaged in our education system, and leave the robots out of it.

Marking Scheme for Writing Assignments

This scheme is currently used on the writing courses and some other courses at the Athabasca University. It should be used as a guide to writing expectations. Some instructors may use different marking schemes.

Content—refers to the following elements:

  • A clear understanding and complete analysis of the topic (given the length/scope of the assignment)
  • An awareness of audience and purpose
  • The use of appropriate quotations (where relevant)
  • Originality of ideas and expression
  • Appropriate evidence of reading and research (where relevant)

10

Outstanding

  • Original ideas well developed, relevant, and thoroughly supported
  • Analysis complete
  • Ideas and expressions original
  • Evidence of reading and research apparent (where appropriate)
  • Perceptive insights
  • Text interesting

9

Excellent

  • Topic coverage complete
  • Appropriate elements achieved to a high degree
  • Many ideas and expressions original
  • Some evidence of research (where appropriate)
  • Text interesting and shows promise

8

Very good

  • Topic coverage mainly complete
  • Most elements completed well

7

Good

  • Topic coverage nearly complete—minor omissions only
  • Analysis weak in places

6

Satisfactory

  • Topic coverage basic
  • Evidence of some analysis

5

Sufficient—improvement needed

  • Topic coverage just adequate
  • Other elements present at a basic level
  • Minor omissions in some elements

4

Insufficient—remediation suggested

  • Topic coverage inadequate
  • Analysis lacking
  • Text uninteresting
  • Omissions in several elements

3

Unsatisfactory—remedial work needed

  • Intent of the writing difficult to understand
  • Omissions in most elements

2

  • Text unfocussed and confusing
  • Major omissions in all elements

1

  • Off-topic
  • Complete lack of audience awareness
  • Text unfocussed and confusing

Top ^

Organization—refers to the following elements:

  • A clear thesis statement
  • A variety of effective transitions to make the writing ‘flow’
  • Appropriate and logical structure both within the assignment as a whole and within the paragraph
  • Good main ideas at the paragraph level
  • Maintenance of ‘purpose’ of the writing
  • An introduction, development and conclusion (paragraphs at the essay level; sentences at the paragraph level
  • Effective sentence variety
  • An awareness of audience

10

Outstanding

  • Arguments thoroughly developed
  • Strong links between sentences and paragraphs making the text logical
  • Appropriate introduction, development and conclusion
  • Mastery of the organizational elements

9

Excellent

  • Appropriate elements achieved to a high degree
  • Structure logical and readily discernible

8

Very good

  • Structure apparent
  • Effective transitions
  • Most elements completed well

7

Good

  • Some minor omissions so that ‘flow’ is not well maintained
  • Structure mainly discernible

6

Satisfactory

  • Structure apparent but at a basic level
  • Omissions in some elements cause ‘flow’ problems

5

Sufficient—improvement needed

  • Structure just adequate
  • Other elements present at a basic level
  • Problems with some elements cause lack of ‘flow’

4

Insufficient—remediation suggested

  • Structure inadequate
  • Lack of logical connection between parts of writing
  • Omissions in several elements

3

Unsatisfactory—remedial work needed

  • Structure and ‘flow’ problems cause confusion
  • No clear purpose to the writing
  • Omissions generalized

2

  • Structure unfocussed and confusing
  • Shift(s) of purpose
  • Major omissions in elements

1

  • Purpose unsupported by structure
  • Complete lack of audience awareness
  • Shift of focus and purpose
  • Major omissions generalized

Top ^

Mechanics—refers to the following elements:

  • Spelling, correct and consistent in usage
  • Punctuation, correct, consistent and with appropriate variety
  • Capitalization
  • Legibility, particularly of handwritten assignments

10

Outstanding

  • Mastery of all elements
  • No errors

9

Excellent

  • All elements achieved to high degree
  • One or two minor errors only

8

Very good

  • Most elements completed well
  • Minor errors only, not affecting meaning

7

Good

  • Minor errors in at least three elements
  • Errors not affecting meaning

6

Satisfactory

  • Errors in all elements
  • Errors distract reader and interfere with understanding

5

Sufficient—improvement needed

  • Errors in all elements
  • Errors affect meaning
  • Use of elements is only basic

4

Insufficient—remediation suggested

  • Major errors in more than one element
  • Inconsistency of usage
  • Errors cause some comprehension problems

3

Unsatisfactory—remedial work needed

  • Major errors in most elements

2

  • Major errors in all elements
  • Errors cause comprehension problems

1

  • Complete, or almost complete lack of elements
  • Errors cause serious comprehension problems

Top ^

Grammar—refers to the following elements:

  • Sentence formation; clauses and phrases appropriately formed and connected
  • Word order and form
  • Verb tense, form, voice (active or passive), and mood (indicative, imperative, subjunctive)
  • Subject-verb agreement
  • Pronoun case forms and pronoun agreement with antecedent
  • Appropriate adjective and adverb form
  • Parallelism
  • Appropriate use of modifiers
  • Direct and indirect speech

10

Outstanding

  • Correction of text not required
  • A variety of complex grammatical structures used
  • Evidence of mastery of advanced and complex structures

9

Excellent

  • Text is almost perfect
  • Evidence of near mastery of advanced and complex structures
  • All appropriate elements achieved at high level of competence

8

Very good

  • Most elements completed well; only a few minor errors
  • High level achievement of most elements

7

Good

  • Minor errors in more than one type of structure
  • Meaning and comprehension not affected by errors
  • Variety of complex structures is used

6

Satisfactory

  • Minor errors in several types of structure
  • Errors distracting but no interference with comprehension

5

Sufficient—improvement needed

  • Some major errors apparent and several minor ones
  • Errors cause some problems with clarity or cause minor confusion

4

Insufficient—remediation suggested

  • Variety of major, global errors
  • Errors distract reader, impeding meaning and comprehension

3

Unsatisfactory—remedial work needed

  • Pervasive and major errors
  • Errors present serious impediment to meaning and comprehension

2

  • Errors basic and pervasive in nature
  • Comprehension difficult

1

  • Numerous errors, even basic ones
  • Text incomprehensible

Top ^

Style—refers to the following elements:

  • Evidence of stylistic control
  • Writing at the appropriate language level (informal, general, formal)
  • Writing appropriate to content, subject, purpose, and audience
  • Demonstration of effective tone and appropriate vocabulary
  • Evidence of creativity
  • Length and complexity of sentences
  • Maintenance of consistent style

Common indicators of stylistic problems include:

  • Shift of focus
  • Monotonous repetition of one or two syntactical patterns
  • Change in level or tone
  • Pretension (attempt at outward show of ability that appears to be false or inaccurate)
  • Use of slang expressions and clichés
  • Choppiness (short, unconnected sentences)

10

Outstanding

  • Evidence of mastery of all appropriate elements
  • Style perceptive and consistent

9

Excellent

  • All appropriate elements achieved to high degree

8

Very good

  • Most elements completed well
  • No significantly detraction from writing from minor omission

7

Good

  • Some omissions in several categories
  • Omissions begin to detract from writing

6

Satisfactory

  • Inconsistent application of style rules
  • Elements present at basic level only

5

Sufficient—improvement needed

  • Most elements present at basic level
  • Inconsistencies and omissions detract from the writing

4

Insufficient—remediation suggested

  • Some basic elements missing
  • Inconsistencies and omissions a serious distraction

3

Unsatisfactory—remedial work needed

  • Most skills insufficient for assignment
  • Omissions generalized

2

  • Text unfocussed and confusing
  • Major omissions in elements

1

  • Text unstructured and incoherent
  • Lack of all required skills

Top ^

Updated September 10 2014 by Student & Academic Services

Categories: 1

0 Replies to “Naplan Writing Marking Criteria For Assignment”

Leave a comment

L'indirizzo email non verrà pubblicato. I campi obbligatori sono contrassegnati *