High-quality measures of instructional practice are essential for research and evaluation of innovative instructional policies and programs, as well as for providing feedback to teachers and administrators. In this study, the authors examined whether using anchoring vignettes in web-based surveys improved the validity of teachers' self-assessments of their mathematics instruction. To investigate validity, they compared correlations between teachers' self-ratings and other measures of teaching including teachers' value-added scores, student surveys, and observation ratings of instruction before and after calibration to examine whether calibration improves the correlation between teachers' self-ratings and other teaching measures. Data came from 61 mathematics teachers in grades 4-9 participating in the Bill & Melinda Gates Foundation's Measures of Effective Teaching Extension project. The sample was roughly evenly distributed between elementary (grades 4-5) and middle school (grades 6-8) teachers, with only three percent teaching 9th grade. The authors used both non-parametric and parametric methods to calibrate teachers' self-ratings. Preliminary findings suggest that anchoring vignettes represent a promising innovation for measuring teachers' instruction through survey self-reports. Specifically, the authors found: (1) Teachers' survey responses that are calibrated through the use of anchoring vignettes have increased variation compared to teachers' raw survey responses, particularly for the cognitive challenge dimension; (2) Teachers' calibrated survey responses regarding mathematical vocabulary and cognitively challenging tasks are more strongly correlated with the composite measure of teacher performance compared to raw survey responses; and (3) If teachers gave their self-rating after rating the vignettes, rather than before, the entire collection of calibrated self-ratings are significantly correlated with the composite performance measure (p

Research on whether teachers can give accurate self-reports in surveys about their mathematics instruction is fairly mixed. Some of that research indicates that teachers can provide some general approximation of their mathematics instruction in survey self-reports, while other studies find very little correlation between teachers' survey responses and their mathematics practices. This diversity of findings points to elements within school district and program implementation context that influence teachers' understanding of mathematics instruction and, thus, the accuracy of their reports about that instruction. While some research provides evidence that teachers' understanding drives the accuracy of their survey responses, no research to date has provided evidence about what aspects of district context impact the accuracy of teachers' self-reports and whether that accuracy can change over time. In this paper, the authors consider the accuracy of teachers' survey reports about their mathematics instruction over a two-year period in two urban school districts. Their work suggests that several elements of district context matter for the accuracy of teachers' self reports, including their mathematics learning opportunities and the presence of other big instructional initiatives within the district. These findings, drawn from in-depth quantitative and qualitative data gathered in two localized education settings, provide key hypotheses to guide future survey research and controlled studies on teachers' understanding of their mathematics instructional practices. In the present study, the authors find that the same district with higher-quality instruction also had many more teachers who provided accurate self-reports of that instruction, while teachers in the other district often overestimated the quality of their instruction in surveys. Using additional survey and interview data, they present some individual and district factors that explain the accuracy of teachers' responses.