Assessment has been an education hot button since the day the first one room schoolhouse sent home the first report card, and probably even before then. And while questions about what is measured, how we do it what it means will persist, assessment as we know it today probably won’t.
Already, the very meaning of assessment is moving away from the counting of right and wrong answers on a common, standardized test. It’s moving, experts say, to be increasingly driven by computer and Artificial Intelligence systems that can make assessment more adaptive to each student, includes more means of measurement than just testing, and makes assessment both more accurate and more efficient – increasing information while cutting the time to administer and grade a test.
Those experts–the ones with that vision for the future for education assessment–will meet on June 10-13 in Minneapolis as part of the International Association for Computerized Adaptive Testing (IACAT) 2019 Conference. When they do, it may be the most important meeting about assessment in a generation.
Sponsored in part by big names such as Pearson, ETS and Duolingo, the upcoming IACAT conference will play host to hundreds of professionals and specialists in fields such as psychology, psychometrics, AI, and testing development. They will not only discuss the future of assessment but debut new products, techniques, and technologies that will be in classrooms and exam rooms almost immediately. If education testing has a learning lab or a World’s Fair, IACAT is it.
Dr. David Weiss, who is generally regarded as the grandfather of Computerized Adaptive Testing, is the featured keynote speaker at IACAT 2019, being held June 10-13 in Minneapolis.
Perhaps chief among those experts at IACAT will be Dr. David Weiss, who is generally regarded as the grandfather of Computerized Adaptive Testing – the cornerstone testing practice by which a computer picks a test-taker’s next question based on how they answered the previous one. Dr. Weiss gave what is believed to be the very first presentation on adaptive testing in 1969 to the American Psychological Association Conference. And although the idea of CAT, as it’s known, has been around since then, it is still a relatively uncommon practice in academics, where less efficient, less accurate tests of standard length and pre-set questions are the norm.
Dr. Weiss, who taught at the University of Minnesota and co-founded the company Assessment Systems, which is hosting the IACAT conference, is expected to address the use of CAT in both practice and frequency, in a rare keynote address. Among the topics Dr. Weiss is expected to address is how even standardized tests should not have time limits and how future test designers should ensure a focus on people instead of numbers.
“Assessment,” Dr. Weiss said in previewing his speech, “has become the domain of mathematicians and an exercise in statistics, people who don’t necessarily have the background understanding of psychology. And we can do that too much and lose sight of the effect on people and forget that the end result should be to do a good job measuring everyone.”
It’s impossible to know whether Dr. Weiss’s words will resonate with the interested audience of test designers and practitioners. But he won’t be the only influential or experienced voice at IACAT hoping to shape the very nature of academic and career assessment.
Source: International Association of Computerized Adaptive Testing
The International Association for Computerized Adaptive Testing (IACAT) Conference June 10-13 on the University of Minnesota campus features academics, practitioners, test developers, school leaders, and testing companies coming together to share and learn about the best ways to assess learning.
Additional conference sessions include thought-provoking concepts such as ways to assess nonverbal IQ traits and learning gains in autistic students, identifying enemy question items that sabotage test validity, multistage testing, using artificial intelligence in assessment, on-the-fly machine learning, item response theory, rapid guessing behavior, vertical scaling, and more.
Perhaps the most interesting, even controversial, future for assessment in education isn’t even AI or machine grading – it’s something called performative assessment, or performance assessment. And according to those assembling at for IACAT next month, it could bring about the demise of the standardized test entirely, which would be big news in education.
Performative assessment is assessment that includes everything a student does – even day-to-day tasks such as reading, note-taking, class participation, and homework – as evidence of performance and learning. In practice, performative assessment can provide thousands of data points related to subject comprehension and mastery as opposed to just select, mid-level questions a standardized test may offer. It proposes to make assessment an ongoing, evolving practice instead of a designed, timed, proscribed testing experience.
Many of the experts who will be in Minnesota in June agree that this new breed of assessment is, and will be, more accurate than even computer-assisted and adaptive testing, which is already more accurate and efficient than the old fill-in-the-bubble, multiple choice tests. There’s a reason, the experts say, that good teachers know which students will do well on a test and which won’t – before the test is even given. And that reason is that teachers collect weeks and months of data for each student before any test, making the test itself a confirming reality instead of a genuine insight. There’s no reason, the assessment experts say, to leave all that information out of the picture.
“Performance testing will merge in the education environment as instruction becomes more personalized and computerized,” said Dr. Weiss. “If instruction is by computer, everything a student does should be used as assessment data – so testing and instruction merge, allowing all information to be used continuously for assessment.”
Even if that’s coming, Dr. Weiss cautioned it won’t happen overnight. Given how long it’s taken even computer-assisted and adaptive testing to take hold in classrooms, he thinks true performance assessment will take at least 20 years to be widely used in education, regardless of how much more efficient and more accurate these assessments would be.
Not only are the results of assessments used to measure student achievement, but student achievements are used to measure the successes and shortcomings of teachers, so these tests matter greatly to schools and entire education models. How they are designed, by whom and what they aim to measure will be the looking glass through which we see all future education attainment. As such, it’s not hyperbole to suggest that what happens in Minnesota in June may end up being the most consequential education conference to come along in quite some time.
For more information about the IACAT conference or to register, click here.
