blog.dataforce.org.uk

AI in coding, Hype or genuinely helpful? I’ve always leaned towards ‘hype’ but a recent dive into using it for some real-world tasks started to shift my perspective… slightly.

I’ve long been generally skeptical of “AI” in all its forms. The idea behind it is fun, the technology is fun (if ethically problematic), but overall I’ve been rather underwhelmed.

I’ve pretty much had access to ChatGPT from the beginning. I’ve asked it questions and felt underwhelmed by the replies (I know it has gotten better). I’ve seen the hallucinations on things I know about, so I feel like can’t really trust it on things I don’t. It talks confidently, it sounds convincing, it’s probably entirely hallucinated. I have however also then seen friends using it to analyse games of online werewolf and summarise that day’s activities and chat to help them keep up when they’re too busy. It’s usually at least able to follow and accurately report on 50% of what is happening (it can’t read people though yet, so ymmv).

On the image side, I’ve used Stable Diffusion to generate images. I’ve also used image generation tools like Midjourney since the early days and seen it go from strength to strength. I’ve used it for quick throwaway joke images, but never really managed to get it to do quite what I had in mind (though usually good enough for a joke). I’ve also used and seen friends using Bing Image Creator to great success to generate meme-images for in-jokes among friend groups.

And while these tools are usually “good enough”, I’ve also then seen them generate 6-fingered people or obviously fake images. I’ve also laughed at the Will Smith eating spaghetti test (and then been impressed at the later improvements to this) or the fact LLMs can’t spell strawberry.

And I know other people have more success with it. I’m constantly being bombarded on social media by posts from people saying how AI is going to put people out of jobs, and that if you’re not spending 50% of your day in ChatGPT you’re wasting your time. But despite this I’ve just not succumbed to the hype, all my experience to date was very firmly at the level of “Meh, It’s ‘ok’. Not good. Certainly not a tool I expect to use all day every day”.

But earlier this month I finally dipped my first toe into the LLM/AI Coding pool…

What if I told you I’ve tracked every single night of sleep for the past 14 years using an app that’s basically been dead for over a decade?

If you actually know me, then it probably comes as no surprise at all. Anyone who has spent any considerable amount of time chatting with me will know a few things about me:

1. I don’t like change.
2. I like graphs and data.
3. I do not always have the best relationship with sleep.

So when I say “I’ve been tracking my sleep and graphing” the response is likely to be one, or both, of “of course you have…” or “but, why?”

The app in question is SleepBot which is one of the early arrivals on the sleep tracking scene and the initial premise was fairly simple. You press a button to say you went sleep, and another when you finished. And then you can rate the sleep and write some notes about it. This button can be in the app itself or a widget on your homescreen. Importantly - it also drew graphs.

Eventually you could also put the app on your bed next to you and it would record any unusual sounds you made, or motion you made. And it had a smart alarm feature, and it had web-sync capabilities. I never really used any of these features other than the basic tracking.

I’ve blogged before about Advent of Code and how I’ve been doing it each year since it began in 2015 and wrote a benchmarking tool for it - AoCBench.

A few years ago when I first wrote AoCBench, there was limited formal guidelines from Eric with regards to repo content, specifically, the closest thing to “offical” guidelines for commiting inputs to repos was a couple of social media posts that essentially stated that it was fine, but just don’t go collecting them all:

In general I ask people not to publish their inputs, just to make it harder for someone to try to steal the whole site. The answer is probably fine, but also probably not very interesting since they vary per person.

– https://x.com/ericwastl/status/1465805354214830081

I don’t mind having a few of the inputs posted, please don’t go on a quest to collect many or all of the inputs for every puzzle. Doing so makes it that much easier for someone to clone and steal the whole site. I put tons of time and money into Advent of Code, and the many inputs are one way I prevent people from copying the content.

– https://www.reddit.com/r/adventofcode/comments/7lesj5/comment/drlt9am/

And also other comments where it wasn’t actively discouraged, but preferred not to.

So AoCBench was designed around all the participants including their own inputs in the repo and this allowed nice things like testing of solutions against different inputs to ensure validity (something you otherwise can’t really do.)

However sometime last year shortly after the start of the month this policy changed (Before / After) and an explicit request was added to the site not to:

If you’re posting a code repository somewhere, please don’t include parts of Advent of Code like the puzzle text or your inputs.

This policy was also codified on the official subreddit and moderators (and other users) started actively (and often aggressively) checking the repos of anyone who posted and insisting they immediately remove any inputs (and purge them from git history) and sometimes resulting in users being banned for not complying. This also hurt a bit with debugging solutions where sometimes different inputs have different properties that may trip some people up.

I don’t want to get too much into any legal technicalities around this and if the inputs are or aren’t copyrightable, or how it has been handled and the negatives around it. I just want to continue enjoying Advent of Code each year with my friends and our benchmarking tool, and if I can respect the request then I’ll do that as well.

For the first year (last year) we continued as-is, but this year people using AoCBench felt stronger about not including inputs in the repos. So how can we do this?

This post is part of a series.

1. Docker Swarm with Ceph for cross-server files
2. Upgrading Ceph in Docker Swarm
3. Docker Swarm Cluster Improvements (This Post)

Since my previous posts about running docker-swarm with ceph, I’ve been using this fairly extensively in production and made some changes to the setup that follows on from the previous posts.

This post is part of a series.

1. Docker Swarm with Ceph for cross-server files
2. Upgrading Ceph in Docker Swarm (This Post)
3. Docker Swarm Cluster Improvements

This post is a followup to an earlier blog bost regarding setting up a docker-swarm cluster with ceph.

I’ve been running this cluster for a while now quite happily however since setting it up, a new version of ceph has been released - nautilus - so now it’s time for some upgrades.

Note: This post is out of date now.

I would suggest looking at this post and using the docker-compose based upgrade workflow instead, up to the housekeeping part.

I’ve mostly followed https://docs.ceph.com/docs/master/releases/nautilus/#upgrading-from-mimic-or-luminous but adapted it for the fact we’re running everything in docker. I recommend that you have a read though this yourself first to have an idea of what we are doing and why.

(It’s worth noting at this point that this guide was mostly written after the fact based on command history so I may have missed something. It’s always a good idea to do this on a test cluster first, or in a maintenance window!)

This all started with a comment I overheard at work from a colleague talking about a 2FA implementation on a service they were using.

“It works fine on everything except Google Authenticator on iPhone.”

… What? This comment alone immediately piqued my interest, I stopped what I was doing, turned round, and asked him to explain.

He explained that a service he was using provided 2FA support using TOTP codes. As is normal, they provided a QR Code, you scanned it with your TOTP application (Google Authenticator or Authy or so), then you typed in the verification code - and it worked for both Google Authenticator and Authy on his Android phone, but only with Authy and not Google Authenticator on another colleagues iPhone.

This totally nerd sniped me, and I just had to take a look.

This post is part of a series.

1. Docker Swarm with Ceph for cross-server files (This Post)
2. Upgrading Ceph in Docker Swarm
3. Docker Swarm Cluster Improvements

I’ve been wanting to play with Docker Swarm for a while now for hosting containers, and finally sat down this weekend to do it.

Something that has always stopped me before now was that I wanted to have some kind of cross-site storage but I don’t have any kind of SAN storage available to me just standalone hosts. I’ve been able to work around this using ceph on the nodes.

Note: I’ve never used ceph before, I don’t really know what I’m doing with ceph, so this is all a bit of guesswork. I used Funky Penguin’s Geek Cookbook as a basis for some of this, though some things have changed since then, and I’m using base-centOS not AtomicHost (I tried AtomicHost, but wanted a newer-version of docker so switched away).

All my physical servers run Proxmox, and this is no exception. On 3 of these host nodes I created a new VM (1 per node) to be part of the cluster. These all have 3 disks, 1 for the base OS, 1 for Ceph, 1 for cloud-init (The non-cloud-init disks are all SCSI with individual iothreads).

For a few years now I’ve been enjoying Eric Wastl’s Advent of Code. For those unaware, each year since 2015 Advent of Code provides a 2-part coding challenge every day from December 1st to December 25th.

In previous years, Myself and Chris have been fairly informally trying to see who was able to produce the fastest code (Me in PHP, Chris in Python). In the final week of last year to assist with this, we both made our repos run in Docker and produce time output for each day.

This allowed us to run each other’s code locally to compare fairly without needing to install the other’s dev environment, and made the testing a bit fairer as it was no longer dependant on who had the faster CPU when running their own solution. For the rest of the year this was fine and we carried on as normal. As we got to the end I remarked it would be fun to have a web interface that automatically dealt with it and showed us the scores, but there was obviously no point in doing that once the year was over. Maybe in a future year…

Fast forward to this year. Myself and Chris (and ChrisN) coded up our Day 1 solutions as normal and then some other friends started doing it for the first time. I remembered my plans from the previous year and suggested everyone should also docker-ify their repos… and so they agreed…

This post is part of a series.

1. DNS Hosting - Part 1: History
2. DNS Hosting - Part 2: The rewrite
3. DNS Hosting - Part 3: Putting it all together (This Post)

In my previous posts I discussed the history leading up to, and the eventual rewrite of my DNS hosting solution. So this post will (finally) talk briefly about how it all runs in production on MyDNSHost.

Shortly before the whole rewrite I’d found myself playing around a bit with Docker for another project, so I decided early on that I was going to make use of Docker for the main bulk of the setup to allow me to not need to worry about incompatibilities between different parts of the stack that needed different versions of things, and to update different bits at different times.

The system is split up into a number of containers (and could probably be split up into more).

This post is part of a series.

1. DNS Hosting - Part 1: History
2. DNS Hosting - Part 2: The rewrite (This Post)
3. DNS Hosting - Part 3: Putting it all together

In my previous post about DNS Hosting I discussed the history leading up to when I decided I needed a better personal DNS hosting solution. I decided to code one myself to replace what I had been using previously.

I decided there was a few things that were needed:

• Fully-Featured API
  • I wanted full control over the zone data programmatically, everything should be possible via the API.
  • The API should be fully documented.
• Fully-Featured default web interface.
  • There should be a web interface that fully implements the API. Just because there is an API shouldn’t mean it has to be used to get full functionality.
  • There should exist nothing that only the default web ui can do that can’t be done via the API as well.
• Multi-User support
  • I also host some DNS for people who aren’t me, they should be able to manage their own DNS.
• Domains should be shareable between from users
  • Every user should be able to have their own account
  • User accounts should be able to be granted access to domains that they need to be able to access
    • Different users should have different access levels:
      • Some just need to see the zone data
      • Some need to be able to edit it
      • Some need to be able to grant other users access
• Backend Agnostic
  • The authoritative data for the zones should be stored independently from the software used to serve it to allow changing it easily in future

These were the basic criteria and what I started off with when I designed MyDNSHost.