Proven in large-scale educational assessment
The New South Wales Education Standards Agency (NESA) uses Concerto to administer computer-adaptive literacy and numeracy testing to 7,000 16-18 year old high school students per year. In 2014, The University of Cambridge Psychometrics Centre worked with NESA (formerly BOSTES, the Board of Studies Teaching and Educational Standards) to trial a computer-adaptive version of its RoSA literacy and numeracy tests in 3 schools across New South Wales. The trial was conducted using Concerto and data collected from around 200 students.
We showed that the length of the original RoSA test could be significantly reduced using a computer-adaptive question selection algorithm, instantiated in Concerto. In fact, the test was reduced from 120 questions, taking 120 minutes to complete, to an average of 47 questions taking only 40 minutes to complete. Not only did Concerto make the test 40% more efficient, it also brought additional benefits such as students being more motivated to answer all the questions and less frustrated by those which were too easy or too difficult for their ability level.
On-demand adaptive testing system with instant feedback
In the following year, the Psychometrics Centre worked with NESA to roll out the test to all students taking the RoSA literacy and numeracy test. NESA already had existing test items, so they were added to the platform. Additionally, instead of two testing sessions at fixed points in the year, the Concerto test was provided on an on-demand basis to be taken by students at any point during the school year. Teachers recommend a student to NESA, and then the student is given an account to take the test at any point during the next weeks.
Concerto calculates scores and students receive immediate provisional results on the screen at the end of the test. Concerto was also integrated with the NESA student administration system, so that students then receive their final scores by email from NESA within 24 hours.
Concerto supports ongoing assessment innovation
NESA continued to use Concerto for the RoSA tests in subsequent years. In 2016 NESA started a pilot of a new literacy and numeracy gamified test for primary school children. The gamified test has a fun graphical interface that runs on tablets or PC, but still has the power of Concerto’s advanced testing and feedback calculation algorithms behind the scenes.
