Change search
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
Optimal Calibration of Items for Multidimensional Achievement Tests
Stockholm University, Faculty of Social Sciences, Department of Statistics. Karolinska Institute, Sweden.ORCID iD: 0000-0003-2889-0263
Stockholm University, Faculty of Social Sciences, Department of Statistics. Linköping University, Sweden.ORCID iD: 0000-0003-4161-7851
2024 (English)In: Journal of educational measurement, ISSN 0022-0655, E-ISSN 1745-3984, Vol. 61, no 2, p. 274-302Article in journal (Refereed) Published
Abstract [en]

Multidimensional achievement tests are recently gaining more importance in educational and psychological measurements. For example, multidimensional diagnostic tests can help students to determine which particular domain of knowledge they need to improve for better performance. To estimate the characteristics of candidate items (calibration) for future multidimensional achievement tests, we use optimal design theory. We generalize a previously developed exchange algorithm for optimal design computation to the multidimensional setting. We also develop an asymptotic theorem saying which item should be calibrated by examinees with extreme abilities. For several examples, we compute the optimal design numerically with the exchange algorithm. We see clear structures in these results and explain them using the asymptotic theorem. Moreover, we investigate the performance of the optimal design in a simulation study. 

Place, publisher, year, edition, pages
2024. Vol. 61, no 2, p. 274-302
Keywords [en]
Achievement tests, exchange algorithm, item calibration, multidimensional item response model, optimal restricted design
National Category
Probability Theory and Statistics
Research subject
Statistics
Identifiers
URN: urn:nbn:se:su:diva-174077DOI: 10.1111/jedm.12386ISI: 001184813000001Scopus ID: 2-s2.0-85187667942OAI: oai:DiVA.org:su-174077DiVA, id: diva2:1357029
Available from: 2019-10-02 Created: 2019-10-02 Last updated: 2024-09-05Bibliographically approved
In thesis
1. Achievement tests and optimal design for pretesting of questions
Open this publication in new window or tab >>Achievement tests and optimal design for pretesting of questions
2019 (English)Doctoral thesis, comprehensive summary (Other academic)
Abstract [en]

Achievement tests are used to measure the students' proficiency in a particular knowledge. Computerized achievement tests (e.g. GRE and SAT) are usually based on questions available in an item bank to measure the proficiency of students. An item bank is a large collection of items with known characteristics (e.g. difficulty). Item banks are continuously updated and revised with new items in place of obsolete, overexposed or flawed items over time. This thesis is devoted to updating and maintaining the item bank with high-quality questions and better estimations of item parameters (item calibration). 

The thesis contains four manuscripts. One paper investigates the impact of student ability dimensionality on the estimated parameters and the other three deal with item calibration.

In the first paper, we investigate how the ability dimensionality influences the estimates of the item-parameters. By a case and simulation study, we found that a multidimensional model better discriminates among the students.

The second paper describes a method for optimal item calibration by efficiently selecting the examinees based on their ability levels. We develop an algorithm which selects intervals for the students' ability levels for optimal calibration of the items. We also develop an equivalence theorem for item calibration to verify the optimal design.  

The algorithm developed in Paper II becomes complicated with the increase of number of calibrated items. So, in Paper III we develop a new exchange algorithm based on the equivalence theorem developed in Paper II.

Finally, the fourth paper generalizes the exchange algorithm described in Paper III by assuming that the students have multidimensional abilities to answer the questions.

Place, publisher, year, edition, pages
Department of Statistics, Stockholm University, 2019. p. 26
Keywords
Achievement test, Equivalence theorem, Exchange algorithm, Item calibration, Item response theory model, Optimal experimental design
National Category
Probability Theory and Statistics
Research subject
Statistics
Identifiers
urn:nbn:se:su:diva-174079 (URN)978-91-7797-879-4 (ISBN)978-91-7797-880-0 (ISBN)
Public defence
2019-11-15, William-Olssonsalen, Geovetenskapens hus, Svante Arrhenius väg 14, floor 1, Stockholm, 10:00 (English)
Opponent
Supervisors
Note

At the time of the doctoral defense, the following papers were unpublished and had a status as follows: Paper 1: Manuscript. Paper 3: Manuscript. Paper 4: Manuscript.

Available from: 2019-10-23 Created: 2019-10-02 Last updated: 2022-02-26Bibliographically approved

Open Access in DiVA

No full text in DiVA

Other links

Publisher's full textScopus

Authority records

Ul Hassan, MahmoodMiller, Frank

Search in DiVA

By author/editor
Ul Hassan, MahmoodMiller, Frank
By organisation
Department of Statistics
In the same journal
Journal of educational measurement
Probability Theory and Statistics

Search outside of DiVA

GoogleGoogle Scholar

doi
urn-nbn

Altmetric score

doi
urn-nbn
Total: 281 hits
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf