The design of an achievement test is crucial for many reasons. This article focuses on a population’s ability growth between school grades. We define design as the allocating of test items concerning the difficulties. The objective is to present an optimal test design method for estimating the mean and percentile ability growth with good precision. We use the asymptotic expression of the variance in terms of the test information. With that criterion for optimization, we propose to use particle swarm optimization to find the optimal design. The results show that the allocation of the item difficulties depends on item discrimination and the magnitude of the ability growth. The optimization function depends on the examinees’ abilities, hence, the value of the unknown mean ability growth. Therefore, we will also use an optimum in-average design and conclude that it is robust to uncertainty in the mean ability growth. A test is, in practice, assembled from items stored in an item pool with calibrated item parameters. Hence, we also perform a discrete optimization using simulated annealing and compare the results to the particle swarm optimization. 

Before items can be implemented in a test, the item characteristics need to be calibrated through pretesting. To achieve high-quality tests, it's crucial to maximize the precision of estimates obtained during item calibration. Higher precision can be attained if calibration items are allocated to examinees based on their individual abilities. Methods from optimal experimental design can be used to derive an optimal ability-matched calibration design. However, such an optimal design assumes known abilities of the examinees. In practice, the abilities are unknown and estimated based on a limited number of operational items. We develop the theory for handling the uncertainty in abilities in a proper way and show how the optimal calibration design can be derived when taking account of this uncertainty. We demonstrate that the derived designs are more robust when the uncertainty in abilities is acknowledged. Additionally, the method has been implemented in the R-package optical.

Objective: Cognitive impairment is a key element in most mental disorders. Its objective assessment at initial patient contact in primary care can lead to better adjusted and timely care with personalised treatment and recovery. To enable this, we designed the Mindmore self-administrative cognitive screening battery. What is presented here is normative data for the Mindmore battery for the Swedish population. Method: A total of 720 healthy adults (17 to 93 years) completed the Mindmore screening battery, which consists of 14 individual tests across five cognitive domains: attention and processing speed, memory, language, visuospatial functions and executive functions. Regression-based normative data were established for 42 test result measures, investigating linear, non-linear and interaction effects between age, education and sex. Results: The test results were most affected by age and to a lesser extent by education and sex. All but one test displayed either linear or accelerated age-related decline, or a U-shaped association with age. All but two tests showed beneficial effects of education, either linear or subsiding after 12 years of educational attainment. Sex affected tests in the memory and executive domains. In three tests, an interaction between age and education revealed an increased benefit of education later in life. Conclusion: This study provides normative models for 14 traditional cognitive tests adapted for self-administration through a digital platform. The models will enable more accurate interpretation of test results, hopefully leading to improved clinical decision making and better care for patients with cognitive impairment.

In recent years, the interest in measuring growth in student ability in various subjects between different grades in school has increased. Therefore, good precision in the estimated growth is of importance. This paper aims to compare estimation methods and test designs when it comes to precision and bias of the estimated growth of mean ability between two groups of students that differ substantially. This is performed by a simulation study. One- and two-parameter item response models are assumed and the estimated abilities are vertically scaled using the non-equivalent anchor test design by estimating the abilities in one single run, so-called concurrent calibration. The connection between the test design and the Fisher information is also discussed. The results indicate that the expected a posteriori estimation method is preferred when estimating differences in mean ability between groups. Results also indicate that a test design with common items of medium difficulty leads to better precision, which coincides with previous results from horizontal equating.

In this thesis, we examine two topics in the area of educational measurement. The first topic studies how to best design two achievement tests with common items such that a population mean-ability growth is measured as precisely as possible. The second examines how to calibrate newly developed test items optimally. These topics are two optimal design problems in achievement testing. Paper I consist of a simulation study where different item difficulty allocations are compared regarding the precision of mean ability growth when controlling for estimation method and item difficulty span. We take a more theoretical approach on how to allocate the item difficulties in Paper II. We use particle swarm optimization on a multi-objective weighted sum to determine an exact design of the two tests with common items. The outcome relies on asymptotic results of the test information function. The general conclusion of both papers is that we should allocate the common items in the middle of the difficulty span, with the two separate test items on different sides. When we decrease the difference in mean ability between the groups, the ranges of the common and test items coincide more.

In the second part, we examine how to apply an existing optimal calibration method and algorithm using data from the Swedish Scholastic Aptitude Test (SweSAT). We further develop it to consider uncertainty in the examinees' ability estimates. Paper III compares the optimal calibration method with random allocation of items to examinees in a simulation study using different measures. In most cases, the optimal design method estimates the calibration items more efficiently. Also, we can identify for what kind of items the method works worse.

The method applied in Paper III assumes that the estimated abilities are the true ones. In Paper IV, we further develop the method to handle uncertainty in the ability estimates which are based on an operational test. We examine the asymptotic result and compare it to the case of known abilities. The optimal design using estimates approaches the optimal design assuming true abilities for increasing information from the operational test.

Large scale achievement tests require the existence of item banks with items for use in future tests. Before an item is included into the bank, it's characteristics need to be estimated. The process of estimating the item characteristics is called item calibration. For the quality of the future achievement tests, it is important to perform this calibration well and it is desirable to estimate the item characteristics as efficiently as possible. Methods of optimal design have been developed to allocate calibration items to examinees with the most suited ability. Theoretical evidence shows advantages with using ability-dependent allocation of calibration items. However, it is not clear whether these theoretical results hold also in a real testing situation. In this paper, we investigate the performance of an optimal ability-dependent allocation in the context of the Swedish Scholastic Aptitude Test (SweSAT) and quantify the gain from using the optimal allocation. On average over all items, we see an improved precision of calibration. While this average improvement is moderate, we are able to identify for what kind of items the method works well. This enables targeting specific item types for optimal calibration. We also discuss possibilities for improvements of the method.

In computerized adaptive tests, some newly developed items are often added for pretesting purposes. In this pretesting, item characteristics are estimated which is called calibration. It is promising to allocate calibration items to examinees based on their abilities and methods from optimal experimental design have been used for that. However, the abilities of the examinees have usually been assumed to be known for this allocation. In practice, the abilities are estimates based on a limited number of operational items. We develop the theory for handling the uncertainty in abilities in a proper way and show how optimal calibration design can be derived in this situation. The method has been implemented in an R package. We see that the derived optimal calibration designs are more robust if this uncertainty in abilities is acknowledged.