What the ChatGPT learning study really tells us

In this issue: unpacking claimed ChatGPT benefits for student learning; France and New Zealand plan to ban social media for teens; new research on how parents' own screen habits affect children.

I thought we could take a close look together this week at a new study claiming ChatGPT boosts student learning. I want to summarize the findings but also to help us all develop the healthy skepticism we need as parents when we see research claiming to know what's best for our kids.

The study - published in Nature Humanities and Social Sciences Communications - claims that ChatGPT has a positive effect on students' learning performance, learning perception, and higher-order thinking. This meta-analysis, combining findings from 51 separate research studies, was authored by two Chinese researchers and published in a peer-reviewed journal that's part of the Nature portfolio.

Some key research findings, before sharing my critique:

• The study suggests that using ChatGPT as a learning tool has a significant positive impact on improving students' learning performance

• The largest benefit appears when ChatGPT is used to build practical skills like coding or problem-solving, with the lowest gains in STEM subjects

• The study looked at interventions ranging from less than one week to more than eight weeks, with the largest impact seen in the 4-8 week range

Overall, the study shows a positive pattern across the board, suggesting that ChatGPT interventions support learning performance. Students using ChatGPT outperformed control groups who didn't use it, leading the researchers to recommend that ChatGPT should be incorporated into classrooms.

Now let's look at the limitations that aren't making the headlines:

1. None of these studies is longitudinal, meaning they don't look at what happens to kids over time. The value of longitudinal studies is immense. If you had looked at the impact of social media on children just when it was launched - and only for eight weeks - you might have thought it had only positive effects. But over time, longitudinal studies have shown that social media's actual impact on children can be seriously negative.

2. Kids were not just given freedom to use the tool however they wanted - all these learning studies were run by scientists and educators. ChatGPT was used in an organized, managed way, controlled by educators - which is key to the positive outcomes which were observed.

3. None of the studies asks the obvious question: does swapping teacher for a chatbot erode the human connections schools are meant to build? By ignoring whether AI use erodes real-world conversation, teamwork, and teacher-student bonds, the research overlooks the socializing core of schooling. Yes, ChatGPT may help with improving grades in the short term, but if it pushes kids deeper behind screens, the long-term costs can be weaker empathy, poorer collaboration skills, and serious social-development problems.

4. The generalizability of the outcomes is limited because 75% of the studies are in college settings. Only five are in secondary school and just one in primary school, which the authors also see as a gap. I don’t see major reasons not to use AI in college settings - it is a perfect time to explore these tools. But these studies seem to suggest that any time is the right time for AI adoption, and we've seen with screen time and social media that the later you introduce them in a child's life, the better it tends to be for their development.

5. Finally, the study looks at short-term improvements under supervised conditions, not the messy reality of everyday, unsupervised use of ChatGPT. I think the researchers completely failed to look at a very important aspect - how often the model hallucinated / provided incorrect information and whether teachers corrected those mistakes.

In conclusion, this study doesn't change my thinking about when and how to introduce AI tools to children. As a parent you should try to control the AI FOMO you might have. Your kids will have plenty of time to understand and learn to use the latest tech when they actually need it. The later you introduce these tools in their life, the better it will be for them because they'll have time to develop foundational skills: critical thinking, social intelligence, emotional regulation, and creative problem-solving.

Check this out: things worth your attention

📝France’s AI minister calls for a Europe-wide ban on social media for children under 15

France’s digital affairs and artificial intelligence (AI) minister is on a mission to unify Europe to force social media companies to put on age verification markers, adding that there should be no social networks for those under the age of 15. Clara Chappaz said that France has three months to “mobilise its European partners” for an agreement that obliges social media platforms to verify the age of their users, and failing to do so would result in sanctions.

📝New Zealand’s prime minister proposes social media ban for under-16s

In an effort to protect young people from harms such as exposure to violent content and cyberbullying, the draft bill is modelled on Australian laws and would force digital platforms to verify the age of users or face heavy fines.

📝Research finds parents’ screen time may hinder child development

Parents’ use of technology in their child’s presence was associated with poorer cognition and social behaviour outcomes and increased screen time among young children. Around 7 out of 10 parents use their phone during play or mealtimes and nearly 9 out of 10 admit to checking their phone once or more in front of their children every day. […] Kids might feel ignored or like they’re competing for their parents’ attention. This can disrupt the natural flow of interactions, which is particularly important as young children are acquiring critical social and language skills. Our goal isn’t to make parents feel guilty, but to raise awareness of how everyday device use might affect moments of connection, and how small, intentional changes can make a meaningful difference.

📝Smartphone use, wellbeing, and their association in children

This study shows the increasingly problematic use of smartphones and its negative association with the overall declining quality of life of children over the last seven years. Problematic smartphone use in children and adolescents has increased in the last seven years, while quality of life has decreased. Problematic smartphone use is associated with decreased quality of life, and this association was especially strong in 2022 and 2023. Restricting children’s smartphone use might help to improve or maintain a higher quality of life.

📝‘The crux of all evil’: what happened to the first city that tried to ban smartphones for under-14s?

As well as celebrating the fall in smartphone ownership in year 6 from 75% to 12%, the head teacher of Cunningham Hill school is delighted that the number of parents buying children smartphones has also dropped in lower years. In December 2023, 30% of year 5 students had smartphones; a year later this had dropped to 4.8% – just three children. He thinks the movement will expand organically. “It will take a few years to really show the impact.” He has also noticed fewer parents ignoring their children at pickup time in favour of their mobiles.

📝TikTok Fails to Address Risks to Children and Young People’s Mental Health Despite Past Warnings

In an investigation using accounts to simulate 13-year-olds online, Amnesty International found that within 20 minutes of starting a new account and signaling an interest in mental health, more than half of the videos in TikTok’s ‘For You’ feed related to mental health struggles. Multiple of these recommended videos in a single hour romanticized, normalized or encouraged suicide.

📝How to improve children's screen time and strengthen family bonds

The most important thing to remember is to be mindful. You can follow some popular sites like Common Sense Media or Protect Young Eyes, which provide general information about media, what to be aware of and even some of the advantages of consuming media. I also want people to be aware of how much media children are consuming when they are with their grandparents, and to plan in advance when they know their grandchild will be staying with them. If you know you will be tired at night, plan non-media-related activities during the day so they can watch TV or a movie when you are tired.

📝Optimizing energy balance in youth: reducing sugar-sweetened beverages and screen time

We report strong evidence that both sugar-sweetened beverage consumption and screen time influence childhood obesity directly, as well as through interactions with sleep patterns. In summary, we suggest that decreasing sugar-sweetened beverage consumption, reducing electronic screen time, and improving sleep quality would, independently and through interactions with each other, prevent and treat obesity.

🔈‘We Have to Really Rethink the Purpose of Education’ | The Ezra Klein Show

One thing to try this week

Do a phone use audit - for one day, keep track (maybe with the help of your partner) of how many times you check your phone when with your kids.

I want to hear from you

What questions about kids and technology keep you up at night? Would you like to contribute to a future newsletter? Add a comment or send me an email at svoinea@gmail.com.

This newsletter will evolve based on what matters to you. Reach out with your thoughts and questions, and if you know other parents who might find this helpful please send them the newsletter.