A Look at the OGT
Constructing Standards-Based Assessment Questions
by Carol Brown Dodson
As an Ohio educator, you probably wonder at times how the questions for the Ohio
Graduation Test are developed. You've likely written many such questions for your
classes, but like most teachers, you probably struggle to find four plausible choices
for multiple-choice questions. You may find, after students respond to the questions
on your tests, that they totally misunderstood at least one of the questions you
asked. Other times, you might find that a correct response to a question doesn't
necessarily mean that a student has achieved mastery of a benchmark or grade-level
indicator.
How do you develop questions that indicate whether or not students achieve a benchmark?
Each year, following the spring administration of the OGT, the Ohio Department of
Education posts the test on its website. The
reading OGT , including passages and questions as well as a scoring guide
and results, is available as a PDF file from the department. A close review of previous
OGT questions is useful. Learning to write standards-based assessment questions,
however, requires more than just looking at questions. By deconstructing test questions,
we can begin to understand how the questions are constructed.
Let's look at some of the passages and questions from the Spring 2006 Ohio Graduation
Test. The nonfiction passage "Wheelchair Flying" by Carrie Dearborn provides an
opportunity for measuring students' achievement in reaching the goals set by the
informational text standard. Students are required to read the paragraph quoted
below:
Wheelchair flying takes place on an asphalt path around a pond. I quite startle
people as I zoom by them at full-out 7 m.p.h. Part of the high of this sport is
that people get a new slant on wheelchairs and wheelchair users. One time, a little
kid pointed at my chair and said, "Look, Mom, can you get me one?"
Ohio Graduation Test, March 2006, Ohio Department of Education
Students are expected to refer to the paragraph to answer a question that assesses
them on informational text Benchmark B. The benchmark reads this way:
Identify examples of rhetorical devices and valid and invalid inferences, and explain
how authors use these devices to achieve their purposes and reach their intended
audiences.
Ohio Academic Content Standards, English Language Arts, Ohio Department of Education
The question (below) requires students to infer meaning and to understand why the
author uses a quote from a small boy.
4. Dearborn includes in the passage the quotation from the little boy in
order to show
- that people of all ages can wheelchair fly.
- how people's attitudes can change.
- that people can sometimes be rude or impolite.
- how dangerous people think wheelchair flying can be.
Ohio Graduation Test, March 2006, Ohio Department of Education
Answer B was correctly chosen by 72 percent of the students who responded to this
question. They understood that the boy's plea for a "flying wheelchair" suggests
a different view of wheelchair users than is commonly held by children. The boy
might not have said this if he had watched someone going by slowly in an electric
wheelchair, but seeing the author fly by at about 7 miles per hour changed his attitude
from what might have revealed negative feelings toward a disability to a desire
to have the same experience that the disabled person has.
Let's look at the stem for the above question.
Dearborn includes in the passage the quotation from the little boy in order to show...
Notice that the question requires students to think not only about the meaning of
the quotation, but also about the author's purpose for including the quotation from
the little boy. As a result, students must look beyond the text itself and consider
why a writer includes certain information and details. That addition to the question
forces it into a higher level of questioning than it might have reached if the question
simply asked why the boy wanted a wheelchair. But even more importantly, the question
assesses the benchmark expectation that students should be able to explain how authors
use rhetorical devices and inferences to achieve their purpose.
View the results
Another consideration in constructing standards-based assessment questions is to
recognize the kind of information that is available about students who select the
incorrect answer. Ohio's tenth grade students generally do extremely well when answering
questions that test reading process, but only 48 percent of the students responded
with the correct answer for the question given below. After reading the passage
"The Grandfather" by Gary Soto, students were asked to select the correct answer
to the following question:
33. Why did the grandfather view the avocado tree as a symbol of hope and
promise?
- For the grandfather, the avocado tree is a reminder of his prosperous
life in Mexico.
- The grandfather had begun growing the avocado tree from a small seed
in a jam jar.
- The avocado tree needs more love and care than the lemon, orange and
apricot trees.
- For the grandfather, life is like an avocado tree, which bears fruit
slowly and patiently.
Ohio Graduation Test, March 2006, Ohio Department of Education
The above question tests students on their achievement of the reading process standard,
Benchmark B:
Demonstrate comprehension of print and electronic text by responding to questions
(e.g., literal, inferential, evaluative and synthesizing).
Ohio Academic Content Standards, English Language Arts, Ohio Department of Education
Literal-level questions are easy for the majority of students, but when the question
causes students to infer, evaluate, or synthesize to get the correct answer, the
students frequently select the incorrect answer. In the case of this question about
how the grandfather viewed the avocado tree, students had to infer what the text
merely implied. Many tenth graders selected the wrong response, with a full 33 percent
of them choosing response B, "The grandfather had begun growing the avocado tree
from a small seed in a jam jar."
Notice that it's not just the response, but the overall development of both the
stem and the possible answers, that leads to information about students' ability
to respond to inferential questions. The benchmark drives the question and is always
the focus of the question.
Some questions cause students to read critically and fully
Another item that assesses students on Benchmark B in the reading process standard
is the following short-answer question about the "Wheelchair Flying" passage.
In your Answer Document, explain what the author means when she says, "Wheelchair
flying gives me the freedom to, well, stretch my 'legs,'" and give a detail or example
from the passage to support your idea. Write your answer in the Answer Document.
(2 points)
Ohio Graduation Test, March 2006, Ohio Department of Education
To receive 2 points for their answer to the question, students had to provide an
explanation and support the explanation by giving a single detail or example from
the passage. Yet only 39 percent of the students responding to the question received
2 points. Let's look at some student responses to gain insight into the question
and its intent. The following answer received a score of 2.
According to the scoring guide:
The response provides a reasonable explanation for what the author means by the
expression (it gives her a chance to do something challenging ... to make it seem
that she can do as much as someone who isn't in a wheelchair) and supports this
example with an example from the text (Now I indulge in a sport few people have
tried―flooring my electric wheelchair).
Ohio Graduation Test, March 2006, Ohio Department of Education
Another student received a score of 0 because this student failed to look at the
expression beyond the literal meaning. Notice the answer given by the student.
This time the scoring guide states that "interpreting the author's legs as her wheelchair
is a literal understanding of the expression and does not adequately describe the
author's intended meaning." The use of the phrase "intended meaning" rather than
a literal understanding is an important distinction being made by the question.
The student is not being tested on getting the exact meaning of the words, but rather
is being tested on what is being conveyed by the entire phrase. The student must
interpret or infer the meaning based on an understanding of the overall passage.
How can I tell if they've achieved the benchmark?
An extended-response question for "The Grandfather" assesses students on Benchmark
E of the literary text standard. Here is Benchmark E:
Analyze the use of a genre to express a theme or topic.
Ohio Academic Content Standards, English Language Arts, Ohio Department of Education
The question requires students to have an understanding of theme and its relationship
to the structure of the story. Notice that the question does not ask students to
identify the theme. Instead, it asks them to explain its development and to support
their answer with specific details or examples from the story.
35. Explain how the concept of growth is developed in the story. Use three details
or examples from the story to support your answer in the Answer Document.
(4 points)
Ohio Graduation Test, March 2006, Ohio Department of Education
A review of the percentage of students at each score point reveals that a majority
of students have not yet achieved the benchmark. A mere 28 percent of students who
responded to the question received a score of 4.
In order to receive a score of 4, students had to give a plausible explanation of
the theme's development and then provide three details or examples to support the
explanation. The following answer earned 4 points.
Ohio Graduation Test, March 2006, Ohio Department of Education
According to the detailed description of the above response in the scoring guide:
The response provides a plausible explanation (the tree grew over the years as the
boy did and as the grandfather grew older) and supports it with three details from
the passage (the boy was able to jump over it at first, then it was as tall as he
was, finally ... all the men on top of each other weren't as tall).
Ohio Graduation Test, March 2006, Ohio Department of Education
The student who wrote this response seems to have an understanding of theme and
a grasp of how an author uses details to develop the theme. The responses to the
4-point question reveal clearly whether or not students have achieved the benchmark.
What are some elements of questions that assess benchmarks?
The questions that effectively assess benchmarks are stated in such a way that students
must go beyond simple recall of details and information. Students are asked to explain
why authors use some details or rhetorical devices. The same is true when students
are tested on literary elements and techniques.
By following some of these same patterns for test development in your classroom,
you can determine how well your students are doing in meeting standards and benchmarks.
You've seen a few examples here, but be sure to go to the
Ohio Department of Education's website, where you can deconstruct other
OGT questions and analyze students' responses to them.
The Ohio Resource Center collection includes released OGT items which can be accessed
by browsing Ohio Standards or by conducting an advanced search for assessment items,
grade 10, reading. Also included in the ORC collection are questions from the National
Assessment of Educational Progress (NAEP). The NAEP questions are extremely useful
in assessing benchmarks, whether the questions are used as they were written or
as models of good standards-based assessment questions.
Some other
ORC resources may be used to help you as you develop standards-based questions
and rubrics.
Authentic Assessment Toolbox
The Authentic Assessment Toolbox is a how-to hypertext about creating authentic
tasks, rubrics, and standards for measuring and improving student learning. Author
Jon Mueller includes detailed, practical application chapters: (1) What is it? (2)
Why do it? (3) How do you do it? (4) Standards, (5) Tasks, (6) Rubrics, (7) Portfolios,
(8) Examples, (9) Glossary, and (10) Workshops. The author discusses rubrics, both
analytic and holistic, and portfolio assessment in detail at this professional development
site, and examples draw from all areas of the curriculum. (author/bcbrown)
Assessing to Inform Instruction
The March/April 2005 issue of Adolescent Literacy In Perspective,
"Assessing to Inform Instruction," focuses on assessing literacy in the middle and
high school classroom. In the feature article, "Living with Hope, Rather Than Doubt:
Classroom Assessment of Adolescent Literacy Learning," author Gwynne Ellen Ash poses
and then answers specific questions about literacy assessment: (1) Classroom assessment
of literacy learning: What do I want to measure? (2) How do I assess these elements
and still have time to teach? (3) Students and teachers as partners, and (4) Assessment
stress: Remembering your purpose. Additionally, two classroom vignettes support
the practical application of assessment. The first vignette, "'Will This Be on the
Test?' Literacy and Assessment in High School English," provides author Colleen
Ruggieri's insights and strategies for assessment: Positive Results with Pre-Assessment,
K-W-L-R Charts, Anticipation Guides, Making Meaning by Marking the Text, Double-Entry
Diaries, and From Study Guides to Comprehension Construction. The second vignette,
by authors Darla Wagner, Joe Paris, Judy Mikita, Jay Falls, and Pat Tallman, looks
at how to tell if our students are learning the material. The focus in this vignette
concentrates on addressing and responding to questions the authors deem significant
in literacy assessment: (1) Are students aware of the daily learning goal? (2) Do
students understand the connection between the class activities and assignments
and the learning goal? and (3) When students leave the room, how do we know which
ones "got it"? This month's "Look at the OGT" focuses on improving students' writing,
and author Carol Brown Dodson looks at specific ways to improve students' writing
through "Writing Suggestions." (author/bcbrown)
Effective Reader Scoring Guide
Developed by educators at the Northwest Regional Education Lab (NWREL), this reading
assessment provides a thorough, yet practical, way to evaluate students' oral and
written responses to texts. Understanding of the text is assessed based on six observable
traits of effective readers: conventions, comprehension, context, interpretation,
synthesis, and evaluation. The scoring guide may be used to outline strengths and
weaknesses of students and to set clear performance goals. For additional information
about this scoring guide, refer to
The Traits of an Effective Reader. (author/ncl)
Carol Brown Dodson is the outreach specialist for the Ohio Resource Center. Dodson
was an English language arts consultant for the Ohio Department of Education and
is past president of OCTELA (Ohio Council of Teachers of English Language Arts).
Dodson, formerly a high school English teacher, department chair, and supervisor
of English language arts in Columbus Public Schools, serves on the Ohio Graduation
Test Reading Content Committee.
Return to top
|