Slightly with which I make new posts, it has been my most recent post for a while. And I think it deserves
an update.
When that post was made:
Opus 4 was the best of the Claude models
Claude-Code had only been GA for about a month.
OpenAI were also in their 4 series, GPT-4o, GPT-4.1 and o3/o4-mini.
Codex CLI had also only existed for a couple of months
Google Gemini was on the 2.5 series
I think my initial feelings were fair and accurate for these models. In addition, Opus was more-expensive to
use, so I was more likely to be using Sonnet for things unless I thought it really needed something.
As mentioned in that post, I had a new-and-fresh claude-code subscription. I had only intended to have it
for a couple of months… Turns out I’ve never stopped paying for it, and have been using it for various levels
of things since.
It’s crazy how fast the state of this has all changed, and how much my use and experience has changed…
The TL;DR here is that I am no longer sitting close to “Hype” and I now sit much closer to “Incredibly
useful Tool”, but read on for a more fleshed out version of this!
AI in coding, Hype or genuinely helpful? I’ve always leaned towards ‘hype’ but a recent dive into using it
for some real-world tasks started to shift my perspective… slightly.
I’ve long been generally skeptical of “AI” in all its forms. The idea behind it is fun, the technology is
fun (if ethically problematic), but overall I’ve been rather underwhelmed.
I’ve pretty much had access to ChatGPT from the beginning. I’ve asked it questions and felt underwhelmed by
the replies (I know it has gotten better). I’ve seen the hallucinations on things I know about, so I
feel like can’t really trust it on things I don’t. It talks confidently, it sounds convincing, it’s probably
entirely hallucinated. I have however also then seen friends using it to analyse games of online werewolf and summarise that day’s activities and chat to
help them keep up when they’re too busy. It’s usually at least able to follow and accurately report on 50% of
what is happening (it can’t read people though yet, so ymmv).
On the image side, I’ve used Stable Diffusion to generate images. I’ve also used image generation tools like
Midjourney since the early days and seen it go from strength to
strength. I’ve used it for quick throwaway joke images, but never really managed to get it to do quite
what I had in mind (though usually good enough for a joke). I’ve also used and seen friends using Bing Image Creator to great success to generate meme-images for
in-jokes among friend groups.
And while these tools are usually “good enough”, I’ve also then seen them generate 6-fingered people or
obviously fake images. I’ve also laughed at the Will Smith eating spaghetti test (and then
been impressed at the later improvements to this) or the fact LLMs can’t spell strawberry.
And I know other people have more success with it. I’m constantly being bombarded on social media by posts
from people saying how AI is going to put people out of jobs, and that if you’re not spending 50% of your day
in ChatGPT you’re wasting your time. But despite this I’ve just not succumbed to the hype, all my experience to
date was very firmly at the level of “Meh, It’s ‘ok’. Not good. Certainly not a tool I expect to use all day
every day”.
But earlier this month I finally dipped my first toe into the LLM/AI Coding pool…
What if I told you I’ve tracked every single night of sleep for the past 14 years using an app that’s
basically been dead for over a decade?
If you actually know me, then it probably comes as no surprise at all. Anyone who has spent any considerable
amount of time chatting with me will know a few things about me:
I don’t like change.
I like graphs and data.
I do not always have the best relationship with sleep.
So when I say “I’ve been tracking my sleep and graphing” the response is likely to be one, or both, of “of
course you have…” or “but, why?”
The app in question is SleepBot which is one of the
early arrivals on the sleep tracking scene and the initial premise was fairly simple. You press a button to say
you went sleep, and another when you finished. And then you can rate the sleep and write some notes about it.
This button can be in the app itself or a widget on your homescreen. Importantly - it also drew
graphs.
Eventually you could also put the app on your bed next to you and it would record any unusual sounds you
made, or motion you made. And it had a smart alarm feature, and it had web-sync capabilities. I never really
used any of these features other than the basic tracking.
I’ve blogged before about Advent of Code and how I’ve been doing it each year since it began in 2015 and
wrote a benchmarking tool for it - AoCBench.
A few years ago when I first wrote AoCBench, there was limited formal guidelines from Eric with regards to
repo content, specifically, the closest thing to “offical” guidelines for commiting inputs to repos was a
couple of social media posts that essentially stated that it was fine, but just don’t go collecting them
all:
In general I ask people not to publish their inputs, just to make it harder for someone to try to steal
the whole site. The answer is probably fine, but also probably not very interesting since they vary per
person.
I don’t mind having a few of the inputs posted, please don’t go on a quest to collect many or all of the
inputs for every puzzle. Doing so makes it that much easier for someone to clone and steal the whole site. I
put tons of time and money into Advent of Code, and the many inputs are one way I prevent people from copying
the content.
And also other comments
where it wasn’t actively discouraged, but preferred not to.
So AoCBench was designed around all the participants including their own inputs in the repo and this allowed
nice things like testing of solutions against different inputs to ensure validity (something you otherwise
can’t really do.)
However sometime last year shortly after the start of the month this policy changed (Before / After) and an explicit request
was added to the site not to:
If you’re posting a code repository somewhere, please don’t include parts of Advent of Code like the
puzzle text or your inputs.
This policy was also codified on the official subreddit and
moderators (and other users) started actively (and often aggressively) checking the repos of anyone who posted
and insisting they immediately remove any inputs (and purge them from git history) and sometimes resulting in
users being banned for not complying. This also hurt a bit with debugging solutions where sometimes different
inputs have different properties that may trip some people up.
I don’t want to get too much into any legal technicalities around this and if the inputs are or aren’t
copyrightable, or how it has been handled and the negatives around it. I just want to continue enjoying Advent
of Code each year with my friends and our benchmarking tool, and if I can respect the request then I’ll do that
as well.
For the first year (last year) we continued as-is, but this year people using AoCBench felt stronger about
not including inputs in the repos. So how can we do this?
I have been a long-time user of proxmox for my virtualisation
needs, starting back in the days of Proxmox 3 (prior to that I was running Ubuntu with OpenVZ rolled in by
hand). Back then I only had a single server and resources were tight, so I deployed a lot of my workloads as
OpenVZ containers.
As time went on, proxmox switched to LXC, and I dutifully converted all my containers to LXC and kept on
going.
More time went on and I added more servers, and ended up clustering them for ease of management. Then
eventually replacing them all with more powerful nodes so now resources were no longer a concern. I also
eventually added Ceph into the mix (using proxmox’s built-in support for ceph) and 10G Networking so that I had
shared storage for VMs and could the start doing VM migrations between nodes quickly.
But, LXC Conainers have flaws - live migration only works for full-VMs. LXC Containers have to do a reboot
to actually move onto and start running on the new node. For some workloads this isn’t really noticable or a
problem - but for others this is quite bad.
Also, as more and more of what I run involves Docker, it’s a lot easier/nicer/safer to run these workloads
in actual VMs rather than LXC containers.
But installing and configuring full-VMs was a chore. With LXC Containers you could be up and running in
minutes by just deploying a template. Full-VMs required a full installation to an empty disk. This could be
automated using kickstart/preseeding etc (And I wrote a
tool to help manage a pxe-boot environments for this purpose). But over time, this has now become trivial
as well - cloud-init is now supported directly within proxmox and all the major OSes provide cloud-init
compatible disk images, so getting a fresh VM is a matter of cloning a template, updating some cloud-init
settings and starting the VM.
Due to all of this, almost all of the VMs I create these days are full VMs. Anything new - it’s a VM, which
gives me all-the-good-stuff™.
But I still have a lot of legacy LXC containers. These all end up suffering any time I do hardware/software
maintenance on the host nodes, or if I have any problems that require putting a host into maintenance mode.
I have recently added some iscsi-backed storage to my proxmox-based server environment, primarily as an off-server location to
store backup data.
For a multitude of reasons, such as the sensitive nature of the data, the fact that the physical storage
lies outside of my control, and just good security hygiene - I wanted to ensure that the data is all encrypted
at rest.
I wanted to be able to use this iscsi as a storage target for proxmox allowing me to just add the volumes to
VMs allowing HA, and I didn’t want to have to do encryption inside every VM incase I accidentally forgot to
enable it for one of the VMs (remember, the storage is hosted external to me so I have no control over the
physical access to it) so to do this I have made use of LUKS encryption on the iscsi block device that I am
presented with and then I run LVM over the top of this. (LVM-on-LUKS as-opposed
to LUKS-on-LVM)
Unlike the previous theme, this one is actually one I mostly ended up designing myself rather than just
finding one that I mostly liked and running with it, and given that, I figured I’d talk a little bit about the
thoughts behind it and how it came to be and what else I’ve done behind the scenes.
Since my previousposts about running docker-swarm with ceph, I’ve been using this
fairly extensively in production and made some changes to the setup that follows on from the previous
posts.
In $DayJob we make use of Dell S4048-ON Switches for 10G Top-of-Rack (ToR) switching and also
sometimes 10G Aggregation/Core for smaller deployments. They’re fairly flexible devices with a high number of
10G ports, some 40Gs and they can do L3 ports and L2 ports. You can also run them either Stacked or in VLT mode
for redundancy purposes.
In addition these things use ONIE (Open Network Install Environment) and can run different firmware images -
though we almost exclusively run these with DNOS 9 which is the Force10 FTOS code that Dell acquired some time
ago rather than DNOS 10.
One evening, I was tasked with an “emergency” build request. We had some kit being shipped to a remote PoP
the following day and the intended routers were delayed, so we needed to get something quickly and temporarily
in place to take a BGP Transit Feed and deliver VRRP to the rest of the kit. A spare S4048 we had lying around
would do the job sufficiently for the time period needed. I figured it wouldn’t take too long to get the base
config needed and get it ready to be shipped with the rest of the kit.
So I got the Datacenter to rack/cable/console it so that I could begin configuration then set aside some
time in the evening to do the work.
As I was watching the switch boot up I noticed something odd. Turns out the last engineer who had used this
device had chosen to install the OpenSwitch OPX ONIE firmware on it instead of the usual DNOS9 firmware. So
much for my quick and easy config.
At this point, I could have just reloaded the device into the ONIE installer environment and installed DNOS9
and been done with it all. But, I had a fairly open evening, and I’d not yet really played about much with any
of the alternative ONIE OSes, so armed with my Yak Sheers, I thought I’d have a look around.
(After all this, I then re-imaged the device onto our standard deployment image of DNOS9 and completed the
required config work that I was supposed to be doing.)
In $DayJob we make fairly extensive use of MPLS ATOM Pseudowires (XConnects) between our
various datacenter locations to enable services in different sites to talk to each other at layer2.
The way I describe this to customers is that in essence these act as a “long cable” from Point-A to Point-B.
The customer gets a cable at each side to connect to their kit, but in the middle of it there is magic that
routes the packets over our network rather than an actual long-cable. Packets that enter 1 side will be pushed
out the other side, and vice-versa. We don’t need to know or care what these packets are, we are just
transparently transporting them.
