Is AI Making Us Bird-Brained?

I recently took to the stage at Codegarden, a technology festival in Denmark run by Umbraco for over 20 years, to talk about AI, trust and sustainability.

Everyone’s talking about AI at the moment, but in the spirit of the talk I delivered last year - Is ChatGPT killing the penguins? - I wanted to take a slightly different look at things. Not least because I’m concerned!

What am I concerned about?

The world has plenty of talks and articles about the wonders of AI. What I’ve observed is that it probably doesn’t have enough talks that balance the excitement with thinking critically about how, where, and when to use this technology.

There’s a delicate balance to tread here, as any perceived criticism of generative AI is often met with accusations of ‘AI doomerism’ or being stuck in the past - but this stuff is important, hence me getting on stage and talking about it.

What did I cover?

My talk started with the intrinsic (and important) link between understanding and trust. My primary argument is that you should not trust the output of LLMs unless you understand a) how they work and b) the subject matter you’re asking about.

The reason why is obvious, but lost on so many - because the outputs are not guaranteed to be correct. I see so many people using this technology to create outputs that contain incorrect information, result in incorrect behaviours and so on. Hence the motivation for the talk!

I then spent some time talking about the varied and exciting ways LLMs behave, which should make us all think more critically about them: hallucinations, data poisoning, prompt injection and on it goes… I spent some time talking about the lethal trifecta, and gave lots of examples of clever people making themselves look silly.

One of my favourites - South Africa’s exciting new national AI Policy turned out to contain a bunch of hallucinated references that didn’t exist. Whoops.

I also discussed how hallucinations can be seen as a problem from a sustainability point of view - all that compute spent ending up being wasted on incorrect information or output.

In an effort to make everyone think more critically about the how, why and when of using AI in their day-to-day, I then introduced a concept which I’ve coined.

The AI Consequences Continuum

I touched on this briefly in my last Substack post. The concept here is that some tasks don’t require a great deal of accuracy, fact-checking, or well-defined behaviour.

One example would be ‘vibe coding’ a prototype of something for your own use. Doesn’t work exactly as it should? Hard to maintain? Not a huge problem.

The further you go along the continuum, however, the more dire the potential consequences of incorrect / misleading information or functionality becomes. Writing a conference talk? You probably don’t want to be quoting things that don’t exist. Building a new e-commerce website for a charity client? You need to ensure it’s secure and works correctly.

The consequences of not applying the right level of due diligence to some of these tasks can range from reputational damage, financial consequences, all the way up to damage to human beings. The example I gave was a therapeutic medical mobile app we’ve built recently for one of our clients. Dispensing misleading information in that context could be seriously dangerous to mental and physical health.

It’s obvious, isn’t it?

Yes, it should be, but it’s clearly not. People across all sorts of industries are overly focusing on the speed and convenience of this technology. It feels fast. It is fast! It’s amazing.

But, it’s also fallible. Yes, you can get AI agents to write your code, test your code, fix your code. There are some situations where that makes sense. But there are also plenty where it doesn’t.

So I’d implore you all to think about the AI continuum of consequences. In fact, ask yourself three questions related to using AI to do something for you.

• Could you?
• Should you?
• And what’s the worst that could happen if you do?