This open access book presents a person-centered exploration of student profiles, using variables related to motivation to do school mathematics derived from the IEA’s Trends in International Mathematics and Science Study (TIMSS) data. Statistical cluster analysis is used to identify groups of students with similar motivational profiles, across grades and over time, for multiple participating countries. While motivational variables systematically relate to school outcomes, linear relationships can obscure the diverse makeup of student subgroups, each with varying combinations of motivation, emotions, and attitudes. In this book, a person-centered analysis of distinct and meaningful motivational profiles and their differences on sociodemographic variables and mathematics performance broadens understanding about the role that motivation characteristics play in learning and achievement in mathematics. Exploiting the richness of IEA’s TIMSS data from many countries, extracted clusters reveal consistent, as well as certain nuanced patterns that are systematically linked to sociodemographic and achievement measures. Student clusters with inconsistent motivational profiles were found in all countries; mathematics self-confidence then emerged as the variable more closely associated with average achievement. The findings demonstrate that teachers, researchers, and policymakers need to take into account differential student profiles, prioritizing techniques that target skill and competence in mathematics, in educational efforts to develop student motivation.

This open access book presents a person-centered exploration of student profiles, using variables related to motivation to do school mathematics derived from the IEA’s Trends in International Mathematics and Science Study (TIMSS) data. Statistical cluster analysis is used to identify groups of students with similar motivational profiles, across grades and over time, for multiple participating countries. While motivational variables systematically relate to school outcomes, linear relationships can obscure the diverse makeup of student subgroups, each with varying combinations of motivation, emotions, and attitudes. In this book, a person-centered analysis of distinct and meaningful motivational profiles and their differences on sociodemographic variables and mathematics performance broadens understanding about the role that motivation characteristics play in learning and achievement in mathematics. Exploiting the richness of IEA’s TIMSS data from many countries, extracted clusters reveal consistent, as well as certain nuanced patterns that are systematically linked to sociodemographic and achievement measures. Student clusters with inconsistent motivational profiles were found in all countries; mathematics self-confidence then emerged as the variable more closely associated with average achievement. The findings demonstrate that teachers, researchers, and policymakers need to take into account differential student profiles, prioritizing techniques that target skill and competence in mathematics, in educational efforts to develop student motivation.

Consistent behavior is a desirable characteristic that common items are expected to have when administered to different groups. Findings from the literature have established that items do not always behave in consistent ways; item indices and IRT item parameter estimates of the same items differ when obtained from different administrations. Content effects, such as discrepancies in instructional emphasis, and context effects, such as changes in the presentation, format, and positioning of the item, may result in differential item difficulty for different groups. When common items are differentially difficult for two groups, using them to generate an equating transformation is questionable. The delta-plot method is a simple, graphical procedure that identifies such items by examining their classical test theory difficulty values. After inspection, such items are likely to drop to a non-common-item status. Two studies are described in this report. Study 1 investigates the influence of common items that behave inconsistently across two administrations on equated score summaries. Study 2 applies an alternative to the delta-plot method for flagging common items for differential behavior across administrations. The first study examines the effects of retaining versus discarding the common items flagged as outliers by the delta-plot method on equated score summary statistics. For four statewide assessments that were administered in two consecutive years under the common-item nonequivalent groups design, the equating functions that transform the Year-2 to the Year-1 scale are estimated using four different IRT equating methods (Stocking & Lord, Haebara, mean/sigma, mean/mean) under two IRT models--the three- and the one-parameter logistic models for the dichotomous items with Samejima's (1969) graded response model for polytomous items. The changes in the Year-2 equated mean scores, mean gains or declines from Year 1 to Year 2, and proportions above a cut-off point are examined when all the common items are used in the equating process versus when the delta-plot outliers are excluded from the common-item pool. Results under the four equating methods were more consistent when a one-parameter rather than when a three-parameter logistic model was fitted. In two of the four assessments, the treatment of outlying common items had an impact on aggregate statistics: equated mean scores, mean gains and proportions above a cut-off differed considerably. Factors such as the number of outlying items, their type (dichotomously or polytomously scored), their level of difficulty, the direction and the amount of their change from Year 1 to Year 2, and the IRT model and equating transformation fitted to the data are discussed with regards to their influence on equated summary statistics. The differential behavior of common items can be considered as a special case of Differential Item Functioning (DIF); the two different groups that respond to a common item can be regarded as the focal and reference groups, and their performance can be compared for DIF. Study 2 applies the Mantel-Haenszel statistic (Mantel & Haenszel, 1959), which is widely used for DIF analysis, on one statewide assessment that was administered to two consecutive annual cohorts of students. Sixty-nine common items, including nine polytomous items, are analyzed first with the delta-plot method and then with the Mantel-Haenszel procedure. A scheme for flagging dichotomous items for negligible, intermediate, or large DIF takes into account both the significance of the Mantel-Haenszel statistic and the effect size of the log-odds ratio; an alternative scheme developed specifically for polytomous items utilizes Mantel's chi-square statistic (Mantel, 1963) and the Standardized Mean Difference (e.g., Dorans & Schmitt, 1991/1993). The Mantel-Haenszel procedure flagged three common items, including one polytomous, for intermediate DIF. The delta-plot identified two dichotomous items only; one of which was flagged by both procedures. Assumptions are examined and it is argued that the Mantel-Haenszel procedure is more appropriate for comparing the performance of two groups because differences in the distributions of ability of the two cohorts are taken into account. The availability of schemes that classify items according to the amount of DIF they exhibit can be informative for the judgmental decision on how to deal with flagged items. However, some caveats relating to test construction and implementation of the equating design are noted if the proposed procedures are to be applied effectively. The same common items and an adequately large number of them must be presented in corresponding forms across administrations. This is pertinent especially for assessments employing a matrix-sampling design, where the common-items are spread among many forms. The following are appended: (1) ST Output--Equating Transformations; (2) Mantel-Haenszel Procedure and ETS Scheme for Flagging; and (3) SPSS Output for Principal Components of Responses to Common. (Contains 12 notes.).

Due to the high stakes that my be attached to placement decisions, it is imperative that the placement process be as solid and defensible as possible. An integral part of the placement process is the identification and use of cut scores, the point(s) on the score scale that classify students into adjacent categories for placement decisions. This paper is geared toward helping college administrators make valid decisions regarding setting cut scores, focusing particularly on selecting a method, but also discussing issues such as defining performance levels and validating the process.