Ändra sökning
RefereraExporteraLänk till posten
Permanent länk

Direktlänk
Referera
Referensformat
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Annat format
Fler format
Språk
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Annat språk
Fler språk
Utmatningsformat
  • html
  • text
  • asciidoc
  • rtf
Optimal Item Calibration for Computerized Achievement Tests
Stockholms universitet, Samhällsvetenskapliga fakulteten, Statistiska institutionen.ORCID-id: 0000-0003-2889-0263
Stockholms universitet, Samhällsvetenskapliga fakulteten, Statistiska institutionen.ORCID-id: 0000-0003-4161-7851
2019 (Engelska)Ingår i: Psychometrika, ISSN 0033-3123, E-ISSN 1860-0980, Vol. 84, nr 4, s. 1101-1128Artikel i tidskrift (Refereegranskat) Published
Abstract [en]

Item calibration is a technique to estimate characteristics of questions (called items) for achievement tests. In computerized tests, item calibration is an important tool for maintaining, updating and developing new items for an item bank. To efficiently sample examinees with specific ability levels for this calibration, we use optimal design theory assuming that the probability to answer correctly follows an item response model. Locally optimal unrestricted designs have usually a few design points for ability. In practice, it is hard to sample examinees from a population with these specific ability levels due to unavailability or limited availability of examinees. To counter this problem, we use the concept of optimal restricted designs and show that this concept naturally fits to item calibration. We prove an equivalence theorem needed to verify optimality of a design. Locally optimal restricted designs provide intervals of ability levels for optimal calibration of an item. When assuming a two-parameter logistic model, several scenarios with D-optimal restricted designs are presented for calibration of a single item and simultaneous calibration of several items. These scenarios show that the naive way to sample examinees around unrestricted design points is not optimal.

Ort, förlag, år, upplaga, sidor
2019. Vol. 84, nr 4, s. 1101-1128
Nyckelord [en]
achievement tests, computerized tests, item calibration, optimal restricted design, two-parameter logistic model
Nationell ämneskategori
Sannolikhetsteori och statistik
Forskningsämne
statistik
Identifikatorer
URN: urn:nbn:se:su:diva-169646DOI: 10.1007/s11336-019-09673-6ISI: 000492593800010OAI: oai:DiVA.org:su-169646DiVA, id: diva2:1323669
Tillgänglig från: 2019-06-12 Skapad: 2019-06-12 Senast uppdaterad: 2022-02-26Bibliografiskt granskad
Ingår i avhandling
1. Achievement tests and optimal design for pretesting of questions
Öppna denna publikation i ny flik eller fönster >>Achievement tests and optimal design for pretesting of questions
2019 (Engelska)Doktorsavhandling, sammanläggning (Övrigt vetenskapligt)
Abstract [en]

Achievement tests are used to measure the students' proficiency in a particular knowledge. Computerized achievement tests (e.g. GRE and SAT) are usually based on questions available in an item bank to measure the proficiency of students. An item bank is a large collection of items with known characteristics (e.g. difficulty). Item banks are continuously updated and revised with new items in place of obsolete, overexposed or flawed items over time. This thesis is devoted to updating and maintaining the item bank with high-quality questions and better estimations of item parameters (item calibration). 

The thesis contains four manuscripts. One paper investigates the impact of student ability dimensionality on the estimated parameters and the other three deal with item calibration.

In the first paper, we investigate how the ability dimensionality influences the estimates of the item-parameters. By a case and simulation study, we found that a multidimensional model better discriminates among the students.

The second paper describes a method for optimal item calibration by efficiently selecting the examinees based on their ability levels. We develop an algorithm which selects intervals for the students' ability levels for optimal calibration of the items. We also develop an equivalence theorem for item calibration to verify the optimal design.  

The algorithm developed in Paper II becomes complicated with the increase of number of calibrated items. So, in Paper III we develop a new exchange algorithm based on the equivalence theorem developed in Paper II.

Finally, the fourth paper generalizes the exchange algorithm described in Paper III by assuming that the students have multidimensional abilities to answer the questions.

Ort, förlag, år, upplaga, sidor
Department of Statistics, Stockholm University, 2019. s. 26
Nyckelord
Achievement test, Equivalence theorem, Exchange algorithm, Item calibration, Item response theory model, Optimal experimental design
Nationell ämneskategori
Sannolikhetsteori och statistik
Forskningsämne
statistik
Identifikatorer
urn:nbn:se:su:diva-174079 (URN)978-91-7797-879-4 (ISBN)978-91-7797-880-0 (ISBN)
Disputation
2019-11-15, William-Olssonsalen, Geovetenskapens hus, Svante Arrhenius väg 14, floor 1, Stockholm, 10:00 (Engelska)
Opponent
Handledare
Anmärkning

At the time of the doctoral defense, the following papers were unpublished and had a status as follows: Paper 1: Manuscript. Paper 3: Manuscript. Paper 4: Manuscript.

Tillgänglig från: 2019-10-23 Skapad: 2019-10-02 Senast uppdaterad: 2022-02-26Bibliografiskt granskad

Open Access i DiVA

fulltext(2204 kB)262 nedladdningar
Filinformation
Filnamn FULLTEXT01.pdfFilstorlek 2204 kBChecksumma SHA-512
6a2339362ffe20a0d8579e25aac0d89aba725a58962b8fd950970fb00959120d0b6f6e505a80b50bc78e91e8e4bf1493990c178145977dfb268e207d7dbde670
Typ fulltextMimetyp application/pdf

Övriga länkar

Förlagets fulltext

Person

Ul Hassan, MahmoodMiller, Frank

Sök vidare i DiVA

Av författaren/redaktören
Ul Hassan, MahmoodMiller, Frank
Av organisationen
Statistiska institutionen
I samma tidskrift
Psychometrika
Sannolikhetsteori och statistik

Sök vidare utanför DiVA

GoogleGoogle Scholar
Totalt: 263 nedladdningar
Antalet nedladdningar är summan av nedladdningar för alla fulltexter. Det kan inkludera t.ex tidigare versioner som nu inte längre är tillgängliga.

doi
urn-nbn

Altmetricpoäng

doi
urn-nbn
Totalt: 386 träffar
RefereraExporteraLänk till posten
Permanent länk

Direktlänk
Referera
Referensformat
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Annat format
Fler format
Språk
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Annat språk
Fler språk
Utmatningsformat
  • html
  • text
  • asciidoc
  • rtf