There’s been a lot of controversy surrounding this year’s SATs tests for Year 6 pupils – with some students reportedly left in tears over how hard they were.
So, online learning platform Atom Learning asked ChatGPT to solve the same arithmetic and reasoning questions the students had to navigate, which the government recently made public.
How did the AI fare?
Out of 36 arithmetic questions for a total of 40 points, the AI managed to solve 32, totalling 34 points, which corresponds to 85% of the test.
Some of the questions it missed the mark on included:
1. ⅔ + 2⅓ =
2. 7,306 − 1,847 =
3. ⅛ ÷ 2 =
4. 3,066 ÷ 73 =
For the reasoning questions – which one teacher described as “the hardest I have ever seen in 20 years of teaching” – the AI cracked 18 out of 25 questions, totalling 24 out of 35 points, corresponding to 68.6%.
Some of the questions it didn’t get right, included:
1. William has four parcels. 1.4kg, 1,500g, 2kg, 300g. Write the masses in order, starting with the heaviest.
2. This sign shows the number of empty spaces on each level of a car park at 10am. Level 2: 511. Level 1: 268. In this car park, each level has 800 spaces.
What is the total number of cars parked in the car park at 10am.
3. The circumference of a circle is 60 cm. Two points on the edge of the circle are called A and B. The angle between point A and point B is 90 degrees. What is the distance around the edge of the circle from A to B?
4. Angle X is in the bottom left of the triangle. The angle at the top of the triangle is 30 degrees. Angle Y is on the outside of the bottom right of the triangle between the rectangle and the triangle. (Each point of the triangle is touching the rectangle). Calculate angle x and y.
‘Interesting and worrying’
A spokesperson for Atom Learning said: “It’s interesting and worrying at the same time that 10-to 11-year-olds were handed a test that not even an Artificial Intelligence has managed to complete to its fullest.”
They acknowledged that while the likes of ChatGPT and other AI programmes are “not infallible” it’s important to remember that these questions were supposed to be tailored for Year 6 students.
“In a situation in which not even an AI can find answers to what is supposed to be basic math, it’s hard to imagine how young students felt when these same questions were put in front of them on one of the most important days of their lives as school children,” they continued.
“Therefore, it’s now easier to understand the reactions that many of them had, which is to be added to the very little time they had to complete the tests.”
The first maths paper (arithmetic) had to be completed in 30 minutes, while the second maths paper (reasoning) had to be done in 40 minutes.
And it wasn’t just maths that proved to stump a lot of pupils this year, with a SATs reading paper causing anguish too.
James Bowen, assistant general secretary at the school leaders’ union the NAHT, said of the reading paper: “We have had clear feedback from school leaders that this year’s paper was not pitched appropriately for a large proportion of pupils and even highly proficient readers struggled with it.
“It is essential that test papers are accessible for the large majority of pupils. We need to remember that these are 10- and 11-year-olds and the last thing we need are papers that leave them feeling demotivated and dejected.”
Schools Minister Nick Gibb said the government would look into the test following complaints from schools.
Meanwhile Gillian Hillier, chief executive of the Standards and Testing Agency, which developed the tests, said they’d been trialled on thousands of pupils.