"Depth-Of-Knowledge Levels for Four Content Areas - Norman L. Webb"

ADVERTISEMENT
ADVERTISEMENT

Download "Depth-Of-Knowledge Levels for Four Content Areas - Norman L. Webb"

451 times
Rate (4.5 / 5) 27 votes
Depth-of-Knowledge Levels for Four Content Areas
Norman L. Webb
March 28, 2002
Language Arts Levels of Depth of Knowledge
Interpreting and assigning depth-of-knowledge levels to both objectives within
standards and assessment items is an essential requirement of alignment analysis. Four
levels of depth of knowledge are used for this analysis.
Reading (based on Wixson, 1999)
Level 1
Level 1 requires students to receive or recite facts or to use simple skills or abilities. Oral
reading that does not include analysis of the text as well as basic comprehension of a text
is included. Items require only a shallow understanding of text presented and often
consist of verbatim recall from text or simple understanding of a single word or phrase.
Some examples that represent but do not constitute all of Level 1 performance are:
• Support ideas by reference to details in the text.
• Use a dictionary to find the meaning of words.
• Identify figurative language in a reading passage.
Level 2
Level 2 includes the engagement of some mental processing beyond recalling or
reproducing a response; it requires both comprehension and subsequent processing of text
or portions of text. Intersentence analysis of inference is required. Some important
concepts are covered but not in a complex way. Standards and items at this level may
include words such as summarize, interpret, infer, classify, organize, collect, display,
compare, and determine whether fact or opinion. Literal main ideas are stressed. A Level
2 assessment item may require students to apply some of the skills and concepts that are
covered in Level 1. Some examples that represent but do not constitute all of Level 2
performance are:
• Use context cues to identify the meaning of unfamiliar words.
• Predict a logical outcome based on information in a reading selection.
• Identify and summarize the major events in a narrative.
Level 3
Deep knowledge becomes more of a focus at Level 3. Students are encouraged to go
beyond the text; however, they are still required to show understanding of the ideas in the
text. Students may be encouraged to explain, generalize, or connect ideas. Standards and
items at Level 3 involve reasoning and planning. Students must be able to support their
thinking. Items may involve abstract theme identification, inference across an entire
passage, or students’ application of prior knowledge. Items may also involve more
superficial connections between texts. Some examples that represent but do not constitute
all of Level 3 performance are:
Depth-of-Knowledge Levels for Four Content Areas
Norman L. Webb
March 28, 2002
Language Arts Levels of Depth of Knowledge
Interpreting and assigning depth-of-knowledge levels to both objectives within
standards and assessment items is an essential requirement of alignment analysis. Four
levels of depth of knowledge are used for this analysis.
Reading (based on Wixson, 1999)
Level 1
Level 1 requires students to receive or recite facts or to use simple skills or abilities. Oral
reading that does not include analysis of the text as well as basic comprehension of a text
is included. Items require only a shallow understanding of text presented and often
consist of verbatim recall from text or simple understanding of a single word or phrase.
Some examples that represent but do not constitute all of Level 1 performance are:
• Support ideas by reference to details in the text.
• Use a dictionary to find the meaning of words.
• Identify figurative language in a reading passage.
Level 2
Level 2 includes the engagement of some mental processing beyond recalling or
reproducing a response; it requires both comprehension and subsequent processing of text
or portions of text. Intersentence analysis of inference is required. Some important
concepts are covered but not in a complex way. Standards and items at this level may
include words such as summarize, interpret, infer, classify, organize, collect, display,
compare, and determine whether fact or opinion. Literal main ideas are stressed. A Level
2 assessment item may require students to apply some of the skills and concepts that are
covered in Level 1. Some examples that represent but do not constitute all of Level 2
performance are:
• Use context cues to identify the meaning of unfamiliar words.
• Predict a logical outcome based on information in a reading selection.
• Identify and summarize the major events in a narrative.
Level 3
Deep knowledge becomes more of a focus at Level 3. Students are encouraged to go
beyond the text; however, they are still required to show understanding of the ideas in the
text. Students may be encouraged to explain, generalize, or connect ideas. Standards and
items at Level 3 involve reasoning and planning. Students must be able to support their
thinking. Items may involve abstract theme identification, inference across an entire
passage, or students’ application of prior knowledge. Items may also involve more
superficial connections between texts. Some examples that represent but do not constitute
all of Level 3 performance are:
• Determine the author’s purpose and describe how it affects the interpretation
of a reading selection.
• Summarize information from multiple sources to address a specific topic.
• Analyze and describe the characteristics of various types of literature.
Level 4
Higher order thinking is central and knowledge is deep at Level 4. The standard
or assessment item at this level will probably be an extended activity, with extended time
provided. The extended time period is not a distinguishing factor if the required work is
only repetitive and does not require applying significant conceptual understanding and
higher-order thinking. Students take information from at least one passage and are asked
to apply this information to a new task. They may also be asked to develop hypotheses
and perform complex analyses of the connections among texts. Some examples that
represent but do not constitute all of Level 4 performance are:
• Analyze and synthesize information from multiple sources.
• Examine and explain alternative perspectives across a variety of sources.
• Describe and illustrate how common themes are found across texts from different
cultures.
Writing
Level 1
Level 1 requires the student to write or recite simple facts. This writing or
recitation does not include complex synthesis or analysis but basic ideas. The students
are engaged in listing ideas or words as in a brainstorming activity prior to written
composition, are engaged in a simple spelling or vocabulary assessment or are asked to
write simple sentences. Students are expected to write and speak using Standard English
conventions. This includes using appropriate grammar, punctuation, capitalization and
spelling. Some examples that represent but do not constitute all of Level 1 performance
are:
• Use punctuation marks correctly.
• Identify Standard English grammatical structures and refer to resources for
correction.
Level 2
Level 2 requires some mental processing. At this level students are engaged in
first draft writing or brief extemporaneous speaking for a limited number of purposes and
audiences. Students are beginning to connect ideas using a simple organizational
structure. For example, students may be engaged in note-taking, outlining or simple
summaries. Text may be limited to one paragraph. Students demonstrate a basic
understanding and appropriate use of such reference materials as a dictionary, thesaurus,
or web site. Some examples that represent but do not constitute all of Level 2
performance are:
• Construct compound sentences.
• Use simple organizational strategies to structure written work.
• Write summaries that contain the main idea of the reading selection and pertinent
details.
Level 3
Level 3 requires some higher level mental processing. Students are engaged in
developing compositions that include multiple paragraphs. These compositions may
include complex sentence structure and may demonstrate some synthesis and analysis.
Students show awareness of their audience and purpose through focus, organization and
the use of appropriate compositional elements. The use of appropriate compositional
elements includes such things as addressing chronological order in a narrative or
including supporting facts and details in an informational report. At this stage students
are engaged in editing and revising to improve the quality of the composition. Some
examples that represent but do not constitute all of Level 3 performance are:
• Support ideas with details and examples.
• Use voice appropriate to the purpose and audience.
• Edit writing to produce a logical progression of ideas.
Level 4
Higher-level thinking is central to Level 4. The standard at this level is a multi-
paragraph composition that demonstrates synthesis and analysis of complex ideas or
themes. There is evidence of a deep awareness of purpose and audience. For example,
informational papers include hypotheses and supporting evidence. Students are expected
to create compositions that demonstrate a distinct voice and that stimulate the reader or
listener to consider new perspectives on the addressed ideas and themes. An example that
represents but does not constitute all of Level 4 performance is:
• Write an analysis of two selections, identifying the common theme and generating a
purpose that is appropriate for both.
Source of Challenge Criterion
The Source of Challenge criterion is only used to identify items where the major
cognitive demand is inadvertently placed and is other than the targeted language arts
skill, concept, or application. Cultural bias or specialized knowledge could be reasons for
an item to have a source of challenge problem. Such items characteristics may cause
some students to not answer an assessment item or answer an assessment item incorrectly
or at a lower level even though they have the understanding and skills being assessed.
Mathematics depth-of-knowledge levels
Level 1 (Recall) includes the recall of information such as a fact, definition, term, or a
simple procedure, as well as performing a simple algorithm or applying a formula. That
is, in mathematics a one-step, well-defined, and straight algorithmic procedure should be
included at this lowest level. Other key words that signify a Level 1 include “identify,”
“recall,” “recognize,” “use,” and “measure.” Verbs such as “describe” and “explain”
could be classified at different levels depending on what is to be described and explained.
Level 2 (Skill/Concept) includes the engagement of some mental processing beyond a
habitual response. A Level 2 assessment item requires students to make some decisions
as to how to approach the problem or activity, whereas Level 1 requires students to
demonstrate a rote response, perform a well-known algorithm, follow a set procedure
(like a recipe), or perform a clearly defined series of steps. Keywords that generally
distinguish a Level 2 item include “classify,” “organize,” ”estimate,” “make
observations,” “collect and display data,” and “compare data.” These actions imply more
than one step. For example, to compare data requires first identifying characteristics of
the objects or phenomenon and then grouping or ordering the objects. Some action verbs,
such as “explain,” “describe,” or “interpret” could be classified at different levels
depending on the object of the action. For example, if an item required students to
explain how light affects mass by indicating there is a relationship between light and
heat, this is considered a Level 2. Interpreting information from a simple graph, requiring
reading information from the graph, also is a Level 2. Interpreting information from a
complex graph that requires some decisions on what features of the graph need to be
considered and how information from the graph can be aggregated is a Level 3. Caution
is warranted in interpreting Level 2 as only skills because some reviewers will interpret
skills very narrowly, as primarily numerical skills, and such interpretation excludes from
this level other skills such as visualization skills and probability skills, which may be
more complex simply because they are less common. Other Level 2 activities include
explaining the purpose and use of experimental procedures; carrying out experimental
procedures; making observations and collecting data; classifying, organizing, and
comparing data; and organizing and displaying data in tables, graphs, and charts.
Level 3 (Strategic Thinking) requires reasoning, planning, using evidence, and a higher
level of thinking than the previous two levels. In most instances, requiring students to
explain their thinking is a Level 3. Activities that require students to make conjectures are
also at this level. The cognitive demands at Level 3 are complex and abstract. The
complexity does not result from the fact that there are multiple answers, a possibility for
both Levels 1 and 2, but because the task requires more demanding reasoning. An
activity, however, that has more than one possible answer and requires students to justify
the response they give would most likely be a Level 3. Other Level 3 activities include
drawing conclusions from observations; citing evidence and developing a logical
argument for concepts; explaining phenomena in terms of concepts; and using concepts
to solve problems.
Level 4 (Extended Thinking) requires complex reasoning, planning, developing, and
thinking most likely over an extended period of time. The extended time period is not a
distinguishing factor if the required work is only repetitive and does not require applying
significant conceptual understanding and higher-order thinking. For example, if a student
has to take the water temperature from a river each day for a month and then construct a
graph, this would be classified as a Level 2. However, if the student is to conduct a river
study that requires taking into consideration a number of variables, this would be a Level
4. At Level 4, the cognitive demands of the task should be high and the work should be
very complex. Students should be required to make several connections—relate ideas
within the content area or among content areas—and have to select one approach among
many alternatives on how the situation should be solved, in order to be at this highest
level. Level 4 activities include designing and conducting experiments; making
connections between a finding and related concepts and phenomena; combining and
synthesizing ideas into new concepts; and critiquing experimental designs.
Science Levels of Depth-of-Knowledge
Interpreting and assigning depth-of-knowledge levels to objectives both within standards
and assessment items is an essential requirement of alignment analysis. Four levels of
depth of knowledge are used for this analysis. Because the highest (fourth) DOK level is
rare or even absent in most standardized assessments, reviewers usually will be making
distinctions among DOK levels 1, 2 and 3. Please note that, in science, “knowledge” can
refer both to content knowledge and knowledge of science processes. This meaning of
knowledge is consistent with the National Science Education Standards (NSES), which
terms “Science as Inquiry” as its first Content Standard.
Level 1. Recall and Reproduction
Level 1 is the recall of information such as a fact, definition, term, or a simple procedure,
as well as performing a simple science process or procedure. Level 1 only requires
students to demonstrate a rote response, use a well-known formula, follow a set
procedure (like a recipe), or perform a clearly defined series of steps. A “simple”
procedure is well-defined and typically involves only one-step. Verbs such as “identify,”
“recall,” “recognize,” “use,” “calculate,” and “measure” generally represent cognitive
work at the recall and reproduction level. Simple word problems that can be directly
translated into and solved by a formula are considered Level 1. Verbs such as “describe”
and “explain” could be classified at different DOK levels, depending on the complexity
of what is to be described and explained.
A student answering a Level 1 item either knows the answer or does not: that is, the
answer does not need to be “figured out” or “solved.” In other words, if the knowledge
necessary to answer an item automatically provides the answer to the item, then the item
is at Level 1. If the knowledge necessary to answer the item does not automatically
provide the answer, the item is at least at Level 2. Some examples that represent but do
not constitute all of Level 1 performance are:
• Recall or recognize a fact, term, or property.
• Represent in words or diagrams a scientific concept or relationship.
• Provide or recognize a standard scientific representation for simple phenomenon.
• Perform a routine procedure such as measuring length.
Level 2. Skills and Concepts
Page of 9