Claude, what the actual… ?

Apologies. I didn’t set out to write yet another post on AI… I actually wanted to write about eLearning localisation and internationalisation which is one of my core skills and services.
But… I’m sorry guys: here’s another post on AI output and how much caution is needed when using it – because this was yet again a PERFECT illustration of how AI cannot be trusted blindly, this time not only in actual translation, but in general!

To prepare the post I had in mind, I needed data and stats on the European eLearning market: roughly how many eLearning providers there are in Europe, their countries, their translation needs (languages overview, which target European markets, most demanded language pairs), etc., to provide some figures and data that would be the backbone of the future post.

So I asked Claude to provide me with statistics on the European eLearning market + language stats + its actual use of language professionals. Because, why not? AI may help me research all that data faster, after all.

Here is what the AI replied (shortened to keep this readable):

“I’m going to give you an overview of elearning companies in Europe, based on recent data available:
Total number of e-learning companies : approximately 3,500 – 4,000 e-learning providers

Breakdown by country:
UU: 800-1000 companies (European leader)
France: 500-600 companies
Germany: 550-700 companies
(…) Other countries: around 1000-1500 companies

Market characteristics:
Annual growth: +15% since 2020
Main sectors: vocational training, continuing education, technical and IT training.💡 Please note: These figures are estimates based on reports from 2023, and the market is evolving rapidly with the increasing digitalisation of training.”

Main target languages for e-learning contents in Europe:

English: 85-90% of companies,
Spanish: 45-50%,
German: 35-40%,
French: 30-35%,
Portuguese: 20-25%

According to the latest industry studies and reports, here are the figures for the use of professional translators in the e-learning industry in Europe: around 45-50% of e-learning companies in Europe DO NOT USE professional translators. Alternative methods:

Machine Translation (AI): 30-35%, non-professional in-house translations: 10-15%.
It is important to note that this percentage varies according to sector and company size (…)”

Looks good, right? Until I asked Claude for the actual sources of these figures (such as the ‘latest industry studies and reports’, ‘reports from 2023’ and the ‘recent data available’).

Brace yourselves – here is the AI’s answer:

And here our brief exchange that followed. To ‘provoke’ it, I asked how I could ever trust it again after this – note its answer: the AI admits it makes mistakes, can even invent stuff and should always be double-checked …

And yet many companies translate their websites, products, etc. using these AI assistants, unchecked, and claim it’s fine.

(To be fair: I asked ChatGPT the same and its answer was much more careful and balanced, without detailed stats, and pointing me to some actual sources like market research firms, organisations and elearning industry groups/materials that publish reports and trends, like EdTech Europe, the Global E-Learning Market report (2021-2027) and the Corporate Training & Learning Solutions Market report from GIA.)