blog.dataforce.org.uk

Slightly with which I make new posts, it has been my most recent post for a while. And I think it deserves an update.

When that post was made:

• Opus 4 was the best of the Claude models
• Claude-Code had only been GA for about a month.
• OpenAI were also in their 4 series, GPT-4o, GPT-4.1 and o3/o4-mini.
• Codex CLI had also only existed for a couple of months
• Google Gemini was on the 2.5 series

I think my initial feelings were fair and accurate for these models. In addition, Opus was more-expensive to use, so I was more likely to be using Sonnet for things unless I thought it really needed something.

As mentioned in that post, I had a new-and-fresh claude-code subscription. I had only intended to have it for a couple of months… Turns out I’ve never stopped paying for it, and have been using it for various levels of things since.

It’s crazy how fast the state of this has all changed, and how much my use and experience has changed…

The TL;DR here is that I am no longer sitting close to “Hype” and I now sit much closer to “Incredibly useful Tool”, but read on for a more fleshed out version of this!

AI in coding, Hype or genuinely helpful? I’ve always leaned towards ‘hype’ but a recent dive into using it for some real-world tasks started to shift my perspective… slightly.

I’ve long been generally skeptical of “AI” in all its forms. The idea behind it is fun, the technology is fun (if ethically problematic), but overall I’ve been rather underwhelmed.

I’ve pretty much had access to ChatGPT from the beginning. I’ve asked it questions and felt underwhelmed by the replies (I know it has gotten better). I’ve seen the hallucinations on things I know about, so I feel like can’t really trust it on things I don’t. It talks confidently, it sounds convincing, it’s probably entirely hallucinated. I have however also then seen friends using it to analyse games of online werewolf and summarise that day’s activities and chat to help them keep up when they’re too busy. It’s usually at least able to follow and accurately report on 50% of what is happening (it can’t read people though yet, so ymmv).

On the image side, I’ve used Stable Diffusion to generate images. I’ve also used image generation tools like Midjourney since the early days and seen it go from strength to strength. I’ve used it for quick throwaway joke images, but never really managed to get it to do quite what I had in mind (though usually good enough for a joke). I’ve also used and seen friends using Bing Image Creator to great success to generate meme-images for in-jokes among friend groups.

And while these tools are usually “good enough”, I’ve also then seen them generate 6-fingered people or obviously fake images. I’ve also laughed at the Will Smith eating spaghetti test (and then been impressed at the later improvements to this) or the fact LLMs can’t spell strawberry.

And I know other people have more success with it. I’m constantly being bombarded on social media by posts from people saying how AI is going to put people out of jobs, and that if you’re not spending 50% of your day in ChatGPT you’re wasting your time. But despite this I’ve just not succumbed to the hype, all my experience to date was very firmly at the level of “Meh, It’s ‘ok’. Not good. Certainly not a tool I expect to use all day every day”.

But earlier this month I finally dipped my first toe into the LLM/AI Coding pool…