blog.dataforce.org.uk

This post is part of a series.

1. DNS Hosting - Part 1: History
2. DNS Hosting - Part 2: The rewrite
3. DNS Hosting - Part 3: Putting it all together (This Post)

In my previous posts I discussed the history leading up to, and the eventual rewrite of my DNS hosting solution. So this post will (finally) talk briefly about how it all runs in production on MyDNSHost.

Shortly before the whole rewrite I’d found myself playing around a bit with Docker for another project, so I decided early on that I was going to make use of Docker for the main bulk of the setup to allow me to not need to worry about incompatibilities between different parts of the stack that needed different versions of things, and to update different bits at different times.

The system is split up into a number of containers (and could probably be split up into more).

This post is part of a series.

1. DNS Hosting - Part 1: History
2. DNS Hosting - Part 2: The rewrite (This Post)
3. DNS Hosting - Part 3: Putting it all together

In my previous post about DNS Hosting I discussed the history leading up to when I decided I needed a better personal DNS hosting solution. I decided to code one myself to replace what I had been using previously.

I decided there was a few things that were needed:

• Fully-Featured API
  • I wanted full control over the zone data programmatically, everything should be possible via the API.
  • The API should be fully documented.
• Fully-Featured default web interface.
  • There should be a web interface that fully implements the API. Just because there is an API shouldn’t mean it has to be used to get full functionality.
  • There should exist nothing that only the default web ui can do that can’t be done via the API as well.
• Multi-User support
  • I also host some DNS for people who aren’t me, they should be able to manage their own DNS.
• Domains should be shareable between from users
  • Every user should be able to have their own account
  • User accounts should be able to be granted access to domains that they need to be able to access
    • Different users should have different access levels:
      • Some just need to see the zone data
      • Some need to be able to edit it
      • Some need to be able to grant other users access
• Backend Agnostic
  • The authoritative data for the zones should be stored independently from the software used to serve it to allow changing it easily in future

These were the basic criteria and what I started off with when I designed MyDNSHost.

I run ubuntu on my servers, and since moving to Hugo, I wanted to make sure I was using the latest version available.

The ubuntu repos currently contain hugo version 0.15 in Xenial, and 0.25.1 in artful (And the next version, bionic only contains 0.26). The latest version of hugo (as of today) is currently 0.32.2 - so the main repos are quite a bit out of date.

So to work around this, I’ve setup an apt repo that tracks the latest release for hugo, which can be installed and used like so:

sudo wget http://packages.dataforce.org.uk/packages.dataforce.org.uk_hugo.list -O /etc/apt/sources.list.d/packages.dataforce.org.uk_hugo.list
wget -qO- http://packages.dataforce.org.uk/pubkey.gpg | sudo apt-key add -
sudo apt-get update
sudo apt-get install hugo

This repo tracks the latest hugo debs in all 4 of the architectures supported: amd64, i386, armhf and arm64 and should stay automatically up to date with the latest version.

This post is part of a series.

1. DNS Hosting - Part 1: History (This Post)
2. DNS Hosting - Part 2: The rewrite
3. DNS Hosting - Part 3: Putting it all together

For as long as I can remember I’ve hosted my own DNS. Originally this was via cpanel on a single server that I owned and then after a while I moved to a new server away from cpanel and moved to doing everything myself (web, email, dns) - hand-editing zone files and serving them via BIND.

This was great for learning, and this worked well for a while but eventually I ended up with more servers, and a lot more domains. Manually editing DNS Zone files and reloading BIND was somewhat of a chore any time I was developing things or spinning up new services or moving things between servers - I wanted something web-based.

There wasn’t many free/cheap providers that did what I wanted (This was long before Cloudflare or Route 53), so around 2007 I did what any (in)sane person would do - I wrote a custom control panel for managing domains. and email. and it was multi-user. and it had billing support. and a half-baked ticket-system… ok, so I went a bit overboard with the plans for it. But mainly it controlled DNS and throughout it’s lifetime that was the only bit that was “completed” and fully functional.

For a while now I’ve been thinking of moving this blog to a statically generated site rather than using wordpress.

There are a number of reasons for this:

1. I can version-control the content rather than relying on wordpress database backups.
2. It renders quicker
3. I don’t actually use any of the wordpress features, so it’s just bloat, and a potential security hole.

I’ve seen Chris successfully use Hugo for his site and it seems to do exactly what I want, So I took the opportunity during the Christmas break to spend some time converting my blog from Wordpress to Hugo.

Actually doing this was a multi-stage process.

It did not help.

It’s been a while since I updated this. Not through a lack of wanting to, more a combination of things - lack of time, lack of anything worth writing, hating the old blog theme…

So I’ve replaced the theme!

Maybe that’ll help.

It probably won’t.

I recently had the misfortune of having a server I am responsible for used as a target for DNS Amplification, and thought I’d share how I countered this. (Whilst this was effective for me, your mileage may vary, but if this actually helps someone then it’s worth posting about.)

This particular server was the main recursor for the site that it was located at (And this was correctly limited not to allow open recursion), but was also authoritative for a small selection of domains. (Yes I know mixing recursors and resolvers is bad.)

The problem only came about when I needed to relocate the server to another site. In order to ensure continuity of service whilst the nameserver IP change propagated, I added some port-forwards at the old site that redirected DNS traffic to the new site. This however meant that all DNS traffic going towards the server came from an IP that was trusted for recursion. Oops.

Over the past few weekends I’ve been (slowly) working on moving my websites around a bit so that things are all in once place, and in the case of this blog, no longer hosted on my home ADSL connection.

At the moment all non-existent pages on http://home.dataforce.org.uk/, http://dataforce.org.uk/ and http://shanemcc.co.uk/ will now redirect to the equivalent link on http://blog.dataforce.org.uk/, over time I will work on moving all public content from these sites over to here (There isn’t much, they’ve mostly been used as dumping grounds!)

After this http://home.dataforce.org.uk/ will be primarily for private things and http://dataforce.org.uk/ and http://shanemcc.co.uk/ will simply redirect here. Eventually I may look to transition this blog over to one of the raw domains (probably dataforce.org.uk). Ultimately I’m trying to do this without breaking any links that may exist to files/etc on these domains.

So if stuff breaks over the next few weeks, that’s why. Feel free to leave a comment if you notice anything or something goes missing.

A while ago I posted a python script that allowed automatically adding labels to GMail messages based on contact groups.

Unfortunately, a side effect of this script was that Google occasionally would lock an account out for “suspicious activity”, and for this reason I stopped using the script.

However recently I looked at Google Apps Scripts to see if this would allow me to recreate this using Google-Approved APIs, and the good news is, yes it does.

The following script implements the same behaviour as the old python script. It checks every thread from the past 2 dates (so today, and yesterday) and then for each message in the thread gets the list of groups the sender is in (if the sender is a contact, and in any groups) and then checks to see if there are labels that match the same name, if so it applies them to the message.

To get this running, create a new project on the Google Apps script page, then paste the code in.

Modify scheduledProcessInbox and processInboxAll to include a label prefix if desired (eg contacts/) and then enable the desired schedule (click on the clock icon in the toolbar). Once this has been scheduled you can run an initial pass over the inbox using processInboxAll() - however this is limited to the last 500 threads.

The code can now be found here on github

Any questions/comments/bugs please leave them here or on github.