Simple SQLite in Express

I don’t have experience with SQLite and want to shift one of my apps over from Mongoose since apparently SQLite is much more capable than I imagined. My usual tactic when trying something new is to try and get a minimal project working on it, so what follows is the simplest possible node/express REST API to demo SQLite.

The simplest possible Express app is going to look something like this. Of course we would have gone to the terminal with npm i express first so this could run.

Using LXC templates in Proxmox

I wrote a couple of weeks ago about a standard workflow I use to spin up a web service in an LXC container to add to my self-hosted collection of services. It went a bit like: do this, and then this, then this other thing. Whenever you find yourself repeating a set of steps like this, it’s usually a sign that you should be automating it. Not just to save time (although this is a key benefit) but also to improve repeatability and to avoid introducing errors.

Practice your restore strategy

My homelab set up is a production node, (pve-prod1) a backup production node (pve-prod2) and a development machine (pve-dev1). They are all G2 800 minis, but pve-prod1 has a i7 6700T and 32GB RAM, where as the other two are i5 6500T with 16GB. My thinking is that the older two can easily share the workload of the main production machine for disaster recovery. Everything is virtualised on top of Proxmox, so sharing up the VM’s and containers is trivial.

Gogs, Gitea, Forgejo

I’ve been really pleased with Gogs - it’s lightweight, was simple to spin up, and has worked perfectly. But then this morning on Mastodon, there’s a post from @Codeberg.org describing a security vulnerability in their Git hosting project Forgejo. This issue also apparently affects Gitea and Gogs - what’s up with that?

I actually already did spend a bit of time comparing Gogs and Gitea before deciding on Gogs, since I’d heard of people running Gitea over the past year or so, but only seen that Gogs seemed to be popular with self-hosters in a Lemmy post I’d read. My first impression was that Gitea was more focused on CI/CD and seemed to have a more complicated install process.

Git - pushing to two remotes

I am loving running a local Gogs instance - it’s nice pushing my git repos to a totally private hub that I know is backed up with all my other self-hosted infrastructure.

Of course, there’s good reasons to have code in GitHub as well - my build-in-public philosophy, the vague possibility that some of it might be useful to someone, my contribution to our future AI overlords, and when I need to make some code linkable - for example from one of these posts. And of course there’s this bit of social-engineering which I assume was inspired by the bathroom decor in Veronica Mars .

Concurrency and channels in Go

In the long ago times, I’d done several years of commercial programming before I ever had to worry about dealing with multiple things happening at the same time. Perhaps because of the rarity of this problem, doing it in traditional languages was not always elegant.

In the modern world of everything happening on the network, and systems being build out of micro-services and APIs, the beginning programmer probably has to deal with this stuff in Programming 102. Luckily, modern languages have these considerations built in, and one language with a particular reputation for that is Go.

Date formatting in Go is quirky

When I’m working in an unfamiliar language, I find its quicker to just ask ChatGPT to write samples of anything I need than to look it up. For instance, last night I needed to format a date in Go, and rather than Google that and pick one of the results and scroll past the ads to read something, I just asked ChatGPT to give me a code example of formatting a date I gave it to DDMMYYYY.

Gogs - your own tiny GitHub

(edit: - I’ve had a rethink about my source hosting)

Once you’re familiar with coding tools, like the excellent VS Code , and git , it’s immediately apparent that these tools can be applicable for other purposes. A great example is that I now do my financial accounting in plain text (using beancount ). I have a python script that converts by bank account data in to the beancount format text files, I edit them in VS Code with a plugin that does the syntax highlighting and checks everything balances.

New Self-Hosted Service Workflow

I’ve developed a bit of a workflow for setting up a new service of some type on the homelab. Installing it is the obvious thing, but I also have a few quality of life things I do to make it a full production-quality part of my installation. I thought it might be helpful to run through those things using a recent example of adding audiobookshelf .

audiobookshelf

audiobookshelf is a web based system for viewing, playing, downloading and/or generally managing your audio books. I’ve been an Audible user/subscriber, but recently got grumpy at them about something - I think I had paused my subscription, and my downloaded books were still available on my phone. I was halfway through one, upgraded the app, and then wasn’t able to play the book without re-subscribing. That might not be exactly right, but it was some type of frustrating carry on like that.

Ansible - Importing a Playbook

Ansible is a system for automating server tasks, and these tasks are written in a special yaml file called a playbook. I had need to call one playbook from another today and learned a couple of things.

Plays vs Tasks

In Ansible we run tasks. A group of tasks run against one particular sets of hosts is called a play. Here is a playbook with one play, and two tasks:

ViewTube

Whenever I encounter one of those “What are you self-hosting?” threads, I know I’m about to waste an hour looking at, and often trying out, software I probably don’t really need, and that was the case with this post on the lemmy.world Selfhosted community.

The basic idea of ViewTube is that it’s a self-hosted front end for YouTube, which just happens to strip out all the advertising and tracking. You can create your own local accounts which allows you to subscribe to channels and which keeps your progress so you don’t start over if you go back to a video - although I couldn’t see a history list. Forgetting your history might be a feature in an app designed to prevent tracking.

Adding Front Matter To mdserver

The very first issue I opened on mdserver - my server project that serves HTML from markdown files - was that the title of the page (which shows in the browser tab, and is used for browser bookmarks) needed to be set inside the markdown file, rather than generated from the file name. I didn’t invent this idea - I’ve seen this sort of metadata in the top of Jekyll and Hugo markdown. Here’s an example from the Jekyll website :

Building Docker images for multiple architectures

My little mdserver app has been a good way for me to start experimenting with the the devops side of things, especially building for Docker. Since I wanted to make the Docker image available for ARM Linux & x86 Linux I had a janky shell script that looked like this:

#!/bin/bash

# Extract the version number from package.json using jq
VERSION=$(jq -r .version package.json)

docker build --platform linux/amd64 -t iankulin/mdserver:$VERSION -t iankulin/mdserver:latest .
docker build --platform linux/arm64 -t iankulin/mdserver:arm64-$VERSION -t iankulin/mdserver:arm64-latest .

docker push iankulin/mdserver:arm64-$VERSION 
docker push iankulin/mdserver:arm64-latest 

docker push iankulin/mdserver:$VERSION
docker push iankulin/mdserver:latest 

So I’d build two different versions, and use the tags to separate them. In the registry it’d look like this:

Docker volume backup is more complicated than it should be

When I set up my first Docker container (I think for Uptime Kuma ), I had read around and understood there were two choices for persistent; bind mounts (where the data inside the container is effectively a symlink to a location on the local file system) or name volumes where Docker abstracted that away a bit, so you didn’t have to worry where it was - I sort of understood Docker ‘managed’ it.

Bruno asserts

I mentioned Bruno the other day. Although it’s still very much under development, it is shaping up as a great Postman/Insomnia replacement.

One of the aspects I’ve been using today is asserts. As part of a request, you can add some asserts - so when you’re hitting an endpoint it will check what status should it be returning, or given the data you’re passing in, what should be in the response body.

Displaying markdown as HTML

In the spirit of over-complicating things, when I wanted to collect all the links to the services on my homelab into one place, I decided I needed to write them in markdown, and have them converted on the fly into HTML by a server. Then when I couldn’t find exactly what I was after (Harp was closest) of course, I decided to write it.

Markdown

Markdown has definitely been having it’s moment over the last couple of years. It’s a simple open format mark-up language that is quite readable in it’s source form. Although it’s now very fashionable as an input for static site generators, most people will have run in to it when adding simple formatting to forum comments or on instant messaging platforms.

Ansible playbook to start Proxmox hosts

In my last post , I talked about tagging guests in a Proxmox node so I could easily see which VMs and LXCs I needed to manually start before I ran an Ansible script to run all my apt updates. It would have been reasonable to wonder why I didn’t just add things to my playbook to magically do that.

The answer would be, I haven’t gotten around to it yet, so here goes:

Proxmox tags to solve a problem

Each weekend I run an Ansible script that updates all my apt based VMs and containers. For the production machines, that’s everything, but my dev Proxmox is full of half-finished projects. Some of these have IP addresses reserved and are in the Ansible hosts file (because whatever service they are running is almost ready to move to the production server) others do not.

Long story short, the dev server has some containers and VM’s that need turned on before I run the updates, and some that don’t. I could just start them all up, for the ten minutes the updates usually take, but that seems wasteful somehow. If there was only some way to mark the ones I need to turn on in the Proxmox webgui! Well, there is. We can add tags to machines in Proxmox.

apt update - BADSIG 871920D1991BC93C

I have an ansible script that runs each weekend which basically does an apt update && apt upgrade -Y on every Debian based instance. This weekend it failed on one Ubuntu host. When I went it to try it manually, this was the output:

Hit:1 http://au.archive.ubuntu.com/ubuntu jammy InRelease
Hit:2 https://download.docker.com/linux/ubuntu jammy InRelease                  
Hit:3 http://au.archive.ubuntu.com/ubuntu jammy-backports InRelease             
Hit:4 http://au.archive.ubuntu.com/ubuntu jammy-security InRelease              
Get:5 http://au.archive.ubuntu.com/ubuntu jammy-updates InRelease [119 kB]      
Err:5 http://au.archive.ubuntu.com/ubuntu jammy-updates InRelease                          
  The following signatures were invalid: BADSIG 871920D1991BC93C Ubuntu Archive Automatic Signing Key (2018) <ftpmaster@ubuntu.com>
Get:6 https://pkgs.tailscale.com/stable/ubuntu jammy InRelease
Fetched 125 kB in 1s (125 kB/s)
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
11 packages can be upgraded. Run 'apt list --upgradable' to see them.
W: An error occurred during the signature verification. The repository is not updated and the previous index files will be used. GPG error: http://au.archive.ubuntu.com/ubuntu jammy-updates InRelease: The following signatures were invalid: BADSIG 871920D1991BC93C Ubuntu Archive Automatic Signing Key (2018) <ftpmaster@ubuntu.com>
W: Failed to fetch http://au.archive.ubuntu.com/ubuntu/dists/jammy-updates/InRelease  The following signatures were invalid: BADSIG 871920D1991BC93C Ubuntu Archive Automatic Signing Key (2018) <ftpmaster@ubuntu.com>
W: Some index files failed to download. They have been ignored, or old ones used instead.

Solved

The first google result mentions apt-cache - which I also run , so a first level debug step is to delete the /etc/apt/apt.conf.d/00aptproxy file that redirects apt requests to the cache I run in an LXC container. After that, if I re-run the apt update it works perfectly. Seems like a problem with the cache then. I’m not sure why it would only affect this host though - I have other Ubuntu VM’s in the fleet that are not getting the original error.

We need to talk about Bruno

I’ve mentioned before that I was using Insomnia as a tool to check my REST APIs as I was developing them, and that I was avoiding Postman (which I guess is more widely used since it’s worth USD5.6 billion ) because

The only reason I’m using Insomnia instead of Postman is that when I tried Postman, it straight away wanted some of my data to make it work. Insomnia hasn’t forced me to do that yet.