Building a WIDA Assessment

tabletop with charts and people working at a laptop

Discover what it takes to make a high quality assessment

WIDA assessments are built on a strong foundation of research. We partner with top experts in the field to design and maintain our tests. We also work with educators from around the world to ensure our tests are informed by best practices in the day-to-day education of multilingual learners.

Continue reading below about:

  • key activities in the test development process
  • educator involvement in test design
  • milestones in the test item development cycle
  • the role of the WIDA English Language Development Standards Framework in test design

Test Development Process

Test development work varies from test to test and year to year. These are some key steps that typically occur between an initial concept and the final assessment you use.

line drawing of hand with pencil and checkboxes

Write test items and create artwork

Item writers work with our partners at the Center for Applied Linguistics (CAL) to create new test content aligned to the WIDA English Language Development Standards Framework and based on educator-recommended topics and themes. CAL’s design team creates unique graphics to support these new items and WIDA staff reviews, refines and approves the new content.

line drawing of person holding booklet

Review by WIDA community

To ensure new test content is appropriately vetted, WIDA coordinates several review activities and events. These activities and events can include such things as reviews of test content and field testing.

line drawing of four types of data charts

Analyze data

Psychometricians, who are trained experts in measuring the validity and reliability of an assessment, investigate how the new test content performed during field testing.

line drawing of booklet with magnifying glass

Finalize the assessment

WIDA assessment experts determine which field-tested items continue as scored pieces of an assessment. Before testing begins, WIDA staff thoroughly check the final version of the assessment and hear input from consortium member leaders before the assessment is administered to students.

Educator Involvement

Educator involvement varies from test to test and year to year. These are some of the key activities that educators, from state-level administrators to classroom teachers, participate in to help WIDA make the tests and teaching tools students and teachers around the world rely on.

For the duration of the COVID-19 pandemic, all events are virtual and participation is by invitation only.

Theme generation

Educators brainstorm general topics for test items. This step ensures future test content accurately reflects current classroom cultures and practices. This activity typically occurs annually in spring or early summer.

ELD Standards Framework expert review

Educators review test items for authenticity and grade-level appropriateness. Reviews suggest revisions that better align the test content to the WIDA ELS Standards Framework. This activity typically occurs annually in summer.

Content reviews

Reviewers ensure test items are grade-level appropriate and factual. This activity for the Writing domain typically occurs annually in fall. This activity for the Listening, Speaking and Reading domains typically occurs annually in spring.

Bias and sensitivity reviews

Reviewers ensure test items are free of material that might favor any subgroup of students over another on the basis of gender, race or ethnicity, disability, home language, religion, culture, region, or socio-economic status. This activity for the Writing domain typically occurs annually in fall. This activity for the Listening, Speaking and Reading domains typically occurs annually in spring.

Field tests, pilot tests, tryouts

Educators and students interact with test content in development in the context of a full assessment experience. The in-development content doesn’t result in or contribute to student scores. These activities help WIDA staff understand whether the new test content is accurately assessing English language proficiency. These activities are conducted as needed, but all new content goes through this step as it is developed.

Forms reviews

State-level administrators go through an entire test, item by item, to identify errors or inconsistencies introduced during final assembly of the test content. Critical errors are corrected immediately, and additional feedback guides future test development. This activity typically occurs annually in summer.

Classroom observations

WIDA staff visit educators and students to stay up to date on the teaching and learning happening in today’s classrooms. These experiences inform the instructional and assessment resources WIDA develops.

Cognitive labs

WIDA staff observe students and educators as they complete test content in development. Observers then interview participants to learn how they reacted to and though through the items and tasks. This activity helps WIDA staff refine test questions and clarify test administration guidance. This activity is conducted as needed.

two speech bubbles

What participants said...

“The discussion with other practitioners was fantastic, and hearing about how the test is viewed and administered across the country and in different settings. I gained valuable insights into how the test is scored and what the expectations are for students and why.”
– 2020 ACCESS for ELLs Content Review participant

“I came away from the experience feeling like my voice and ideas were valued, which was a great feeling.”
– 2020 ACCESS for ELLs Content Review participant

“I think most educators do not realize the ‘behind the scenes’ that takes place before students are expected to perform.”
– 2020 ACCESS for ELLs Bias and Sensitivity Review participant

test development timeline in stages over four yearsTest Item Development Cycle

Creating new content for WIDA assessments is an intensive, multi-year process. Consider a typical ACCESS for ELLs development cycle. Initial item writing begins in the summer and continues through the end of the calendar year. Item reviews typically take place in the spring.

By fall, the new items have been incorporated into the assessment as field test items. They appear to students just like regular test content, but that do not contribute to the test scores. 

Items that don’t work well are saved for revision and repurposing. Items that do work well are incorporated into the test and become operational. In other words, the next time a student sees the item, it will contribute to the test score. 

A test item developed in calendar year 1 will help provide a score that an educator sees in summer of calendar year 4. 

Test Item Refreshment

The steps of the test development process happen every year. However, not every WIDA assessment is rebuilt from scratch every year. To accurately calculate scores and compare student performance, some test items and tasks in the current year need to be the same as those in the previous year. It’s also a big project to create new test content – so we keep what works!

Test developers aim to replace items that are older and that could be improved, according to established guidelines on how much test content should change. For example, about one third of the ACCESS for ELLs Online Listening and Reading content is refreshed each year.

 

 

row of five icons representing the wida standardsStandards-Referenced Test Design

WIDA assessments are standards-referenced tests, meaning that student performance is compared to the WIDA ELD Standards Framework. Any student can achieve any score, and students are not ranked against each other or against the expected performance of monolingual English speakers.

Each folder on a Listening or Reading test and each task on a Speaking or Writing test is aligned with one or more of the WIDA English Language Development Standards Statements.