Web-Dev on dev.endevour

State of AI tooling (for me)

Mon, 07 Jul 2025 00:00:00 +0000

I’ve been meaning to write this for a couple of weeks, so let’s get to it - things are moving to fast to reflect too long; which is it’s own risk.

In March, I wrote about how I was using AI in coding , which was Codeium (now Windsurf) in VS Code for completions, and ChatGPT and Claude online for architecture questions and code gen that was more than half a function.

End to end testing - Cypress basics

Mon, 12 May 2025 00:00:00 +0000

When you’ve made a change to your web-app, do you run it then click around the new bits to check it works? Good start, but instead of doing that yourself, do it in a faster, more comprehensive and automated way with an end-to-end (E2E) testing setup using Cypress . Here’s how.

E2E

End to End testing is testing your app as a user might - by clicking links, entering data, looking at the screen and checking everything is okay, but it’s scripted like a unit test and the results are checked with assertions. Like unit testing this allows you to build up a collection of comprehensive tests that easily detect for unexpected behaviours - not just in the results of functions in your app, but in the user experience of the app.

Functional Javascript array methods

Mon, 14 Apr 2025 00:00:00 +0000

I’ve been whipping up a little mock-database unit that has a few access functions but actually stores the data as arrays for a demo project for a post I’m writing. In the process I wrote this gem:

export function dbOrdersAdd(order) {
 const orderCopy = { ...order };
 // since id is a stringified number, finding the max is a bit of a mess
 const maxId = orders.reduce((max, o) => Math.max(max, parseInt(o.id)), 0);
 orderCopy.id = String(maxId + 1);
 orders.push(orderCopy);
 return { ...orderCopy };
}

In the comment I’m claiming the code is a bit of a mess (and from a readability point that’s true) but actually I love the elegance of using the reduce() method here.

Node.js built in test runner

Mon, 17 Mar 2025 00:00:00 +0000

For the longest time, I’ve been using Mocha (test runner) and Chai (assertion library) for my JS testing. They are reliable old friends.

One of the effects of the existence of Bun and Deno has been to spur Node onto adding some new features, so after appearing as an experimental feature in 18, the Node test runner dropped in Node 20.

I’m not sure if the familiar unit test layout of Mocha and Node is inherited from Jest, or comes from older testing frameworks of which JUnit and NUnit were the first ones I’d ever used. Before that I just used to write tests as lumps of assertions in regular code - which worked but wasn’t as pleasant to use as a proper unit test setup. Regardless, the system of bundling a few tests together and having them all run and spit out green ticks is not a new one.

Where I'm up to with AI for coding

Mon, 03 Mar 2025 00:00:00 +0000

There’s still plenty of controversy about LLMs for coding, and not without reason. But I thought I’d run through what I’ve tried, and where I’ve landed for using AI. Also what the pitfalls are, where it’s useful and how it’s changed my practice.

Issues

Training data

The training data for large language models generally is problematic. There’s no doubt that they have been trained on copyright material. With code it’s slightly less murky since there is a high availability of good quality open source data with attached licenses to train models on. No doubt this include code written by people who don’t approve of it being used by AI, but I think the popular reading of most open source licenses is that using it for training is fine.

A bit of web-scraping with Cheerio

Mon, 17 Feb 2025 00:00:00 +0000

I had an idea for a little holiday project that required a list of episodes from The Rest Is History podcast. On their ‘Episodes’ page, they have a player, and a list of post entries for the most recent eighteen podcasts. There is a ‘show all’ button, but it doesn’t work.

The player does contain the full list of episodes (about 600) including a number of duplicates, so I expected if I inspected the network calls that I’d see a JSON package arriving with what I wanted. This is what I almost always find these days so I’ve had very little call to do any real web scraping - it’s normally just a matter of locating the endpoint and perhaps extracting an API key from a header.

Perils of Benchmarking

Mon, 06 Jan 2025 00:00:00 +0000

I’ve been containerising my websites, with their servers to make deployment simple and robust, and to move to a CI/CD workflow. Since an install of a production web server is large, I would be running about ten of these containers, and there’s already a good server facing the net and doing the reverse-proxying (NGINX Proxy Manager), I chose to bundle the Busy-Box httpd server with my sites inside the Docker containers.

Moving a domain from Wordpress

Mon, 30 Dec 2024 00:00:00 +0000

I love the convenience of a hosted blog on wordpress.com, but one of the justifications for my ‘investment’ in homelab hardware and learning time was that I’d reduce my spend on hosted platforms by self-hosting them. I’ve already quit Evernote and dropped back to the free plan on Dropbox by building systems to replace them for less money and more data sovereignty. And now, the recent Wordpress drama has made me uneasy about Matt having control of domains I’ve got registered with wordpress.

Updating a deployment on fly.io

Mon, 16 Dec 2024 00:00:00 +0000

I’ve had my external UptimeKuma chugging away on fly.io , for free, for months now, and the container image it was based on was a bit out of date, so I wanted to update it. I hadn’t looked at fly.io for months, and couldn’t really recall what I’d done to create it.

The way this works is that that you create a fly.toml file that sets out the details of your app. From memory I think I used the one from the docs and gave it a unique name, the name of the Docker image, the port, the datacentre location, and the directory for the persisted data. The you run fly deploy from the directory with the toml file (having already installed the CLI tool and logged in) and you’re in business.

Fixing TLS for wget in BusyBox

Mon, 25 Nov 2024 00:00:00 +0000

I’ve been containerising my static websites with BusyBox (because it’s small), and in an earlier post showed how to even get the container to update parts of the site by reaching out with wget to download resources from elsewhere and saving them inside the container where we are serving the ‘static’ site from. I’d done this by including a bash script in the container with the wget in a loop with a sleep. Then started the script and the httpd server in the CMD line of the dockerfile.

Fancier Website in a Docker Container

Mon, 18 Nov 2024 00:00:00 +0000

The previous post went over how to bundle a static website into a Docker container. That’s a neat little trick - keeping the entire website and making it trivial to install on a VPS behind Nginx Proxy Manager. It worked great for most of my little websites.

But…

A couple of my websites had very minor ‘dynamic’ content. One was pulling down the local temperature from OpenWeather, then exposing a cut-down version of that as a REST endpoint so all my servers could grab it without me being rate-limited by OpenWeather for abusing my free API key. Another one re-hosted an image that changes a couple of times a day from an unreliable service.

Website in a Docker Container

Mon, 11 Nov 2024 00:00:00 +0000

Having figured out how to use the GitHub package registry, I was a bit inspired by this blog post from Florin Lipan to deliver all my little static websites as Docker containers. I’m not as focused as he is about making them tiny, but I did steal the idea of using BusyBox httpd for serving them, resulting in about 4MB containers. That’s small enough for me, and since they are all very similar, there’s a fair bit of layer reuse going on.

npm ERR! Exit handler never called!

Mon, 21 Oct 2024 00:00:00 +0000

I quite like GitHub scanning all my code and sending me security advisories. Here’s today’s:

With these, and my dependabot alerts, fixing them is usually just a matter of pulling down the project, running an npm update, building any artifacts, then pushing it back up. But today, not so:

package-lock.json

It’s probably worth revisiting what the package-lock.json does. It contains all the versions of any packages you’ve imported, and their dependencies. The idea is that this will make the build reproducible. We don’t commit the node_modules folder (that actually contains all that package code), but npm can reproduce it exactly by using the version information in the package-lock.json file. Here’s a snippet where you can see all those versions:

Code reuse by publishing to NPM

Mon, 14 Oct 2024 00:00:00 +0000

If you find yourself copying over a source file from one Node project to another because it’s a handy utility you wrote and are used to using, you’re only doing it half right. A better way to do this is to publish your utility to the Node Package Manager (NPM). That way you can just import your utility where ever you need it, it will live in the node_modules of any project that uses it, and most importantly, updates are sorted out automatically - because that’s what package managers are good at.

Uploading files to a web app with Node

Mon, 02 Sep 2024 00:00:00 +0000

My default approach to web apps at the moment is Node/Express SSR. I needed to have users be able to upload files this week, and as usual there’s an express middleware that makes it trivial. This post just steps through using multer to make it simple to enable file uploads on your website.

Express & middleware

Before we look at file uploading, it’s worth just explaining how it fits with the other tools we’re using:

Authentication basics for Node apps

Mon, 19 Aug 2024 00:00:00 +0000

Pretty much every serious web app needs to include a way for users to log in securely and to be served their content. Since there’s a lot of complexity in this, it’s highly advisable to use good libraries to support this. In a future post we’re going to use those libraries, but first I want to explain what’s happening at the lower level and tease out some of the concepts as we build a secure system from the ground up.

Using LLMs for coding

Mon, 01 Jul 2024 00:00:00 +0000

This post looks at the context for some of my thinking about AI for supporting software development, and where I’ve landed on it for the time being.

The landscape

I briefly wrote about ChatGPT’s coding ability at the end of 2022. The wide availability of this tool marked the beginning of what I think can fairly be described as a revolution. The controversies that have crystalised since have not dampened my amazement of this step forward in what compute can do, especially around natural language processing.

Peek inside a Docker image

Mon, 29 Apr 2024 00:00:00 +0000

A ‘dockerfile’ contains all the instructions to build a Docker image. Here’s my first draft for a project I’m working on:

FROM node:20
WORKDIR /usr/src/app
COPY package*.json ./
RUN npm install
COPY . .
EXPOSE 3000
CMD ["node", "server.js"]

COPY . . is copying all of the files in my project into the working directory of the image so they can be run. Of course we don’t need them all for the app - for example the node_modules directory will be created when we npm install so no need to copy that, and I don’t need all my dot files in the container.

NGINX Proxy Manager

Mon, 15 Apr 2024 00:00:00 +0000

I’ve mentioned using NGINX as an interface between the internet and a service a while ago. This works by all incoming traffic coming to NGINX, and NGINX determining which service that traffic should go (from the NGINX config files) then acting as a middleman. This functionality is generally referred to as a ‘reverse proxy’.

This is nice for a few reasons:

Deploying a Node app in Docker

Sun, 31 Mar 2024 00:00:00 +0000

When I wrote the install instructions for mdserver (little Markdown server Node app) on it’s github page it was something like:

Have node.js installed and working
Clone the repo
Start with npm start

Which is great if you know how to do those things (they are bread and butter to a web dev) but not if you’re a self-hoster who just wants a web server that converts markdown to HTML on the fly. For any situation where you just want to use the app, what you probably want is a Docker image of the app.

Certbot - removing a domain

Mon, 18 Mar 2024 00:00:00 +0000

I had a number of domains all running on one host when I first set them up with certbot. One started to be serious, so I moved it to another host and ran certbot there. That all worked perfectly, but of course, the old domain is still part of the original certificate, so when I went to renew it, it came up with some errors.

Here’s a few commands that are going to help navigate this situation if you’ve found yourself in the same spot:

Quick & Dirty auth with nginx & Node

Fri, 23 Feb 2024 00:00:00 +0000

One of the basic requirements for any serious web app is a proper users/roles/authentication system - but if you’re just throwing up a utility of some kind on a public IP for testing, and you don’t want it to be abused, then this could be an option. There’s a few components:

Your app. In this demo it’s going to be Node, but it could be Go or whatever your server-side poison is. The app is listening for connections on a non-web port (ie not on 80 or 443), I’m going to use the traditional 3000.
A firewall. That port (in my example 3000) must not be accessible from the internet. It has to be blocked by a firewall.
A web server (I’m using nginx) that enforces basic auth.

I briefly discussed web server basic auth earlier - it’s a system built into the web server that requires a log in for a route, and authenticates it against the credentials in a password file (usually named .htpasswrd) and only serves the content if authenticated.

Beginning Node App Security

Fri, 16 Feb 2024 00:00:00 +0000

Since I’m using Tailscale to painlessly manage all my networking on the homeserver here and my remotes, I’ve had the luxury of being a bit casual about the security of my internal apps and self hosted dev tools. I’m currently iterating on a web app that requires public access, and is therefore up on a VPS and exposed to all the evils of the open internet.

I am in no way a security expert, but here’s a few of the (reasonably simple) steps I’ve taken to secure my node app.

User Sessions & Cookies in Node

Fri, 09 Feb 2024 00:00:00 +0000

When you are learning app development, you can create all sorts of apps that work for you, but for any serious app, it’s going to need to authenticate users and persist sessions across visits. So much so, that as a professional developer, you’ll probably build that out first - it becomes a sort of boiler plate you always drop in.

In this post, focusing on the server side, using node, express, and particularly express-session, I’ll try and build up from nothing to a reasonable usable user login system explaining the increasing complexity and reasons for it. To follow along you’ll need basic familiarity with node and express.

Web Development Overview

Mon, 05 Feb 2024 00:00:00 +0000

I don’t often just link to someone else’s content, but I was really impressed with Brad Traversy ’s “Web Development In 2024 - A Practical Guide” video. Apparently he does these every year - it’s just a really comprehensive overview of Web Development pitched at beginners.

Fly.io, Uptime Kuma & scraping a status page

Fri, 02 Feb 2024 00:00:00 +0000

I’ve been aware since I set up Uptime Kuma for my monitoring, that having an instance on my local network monitoring my VPS websites wasn’t ideal. The main reason being that the flakiest part of my infrastructure is my 4G home internet, so if that goes down I have no website monitoring, and even if I did, the notifications couldn’t get out.

Of course, it would also be a simple matter to run an instance on the VPS that I host the sites on, but that has a similar problem in that if the VPS goes down, so does my monitoring of the VPS. What I really need is a third, independent space to run an instance.

How to Have Cooler File Icons in VS Code

Mon, 29 Jan 2024 00:00:00 +0000

I watch a lot of programming demos on Youtube, and it’s been low key bugging me for a while that everyone has cooler little icons in the explorer view of their VS Code than I do. For example, they have the HTML 5 shield logo next to their index.html, but I have the little fragment tag <>. Really, there was no point spending two hours customising my OhMyZSH! terminal if I’m just going to let myself down with disappointing VS Code file icons.

Getting Your Vite React App to Work on Github Pages

Fri, 26 Jan 2024 00:00:00 +0000

One of the many cool things about GitHub is GitHub Pages - the free web hosting Microsoft gives you while they vacuum up your code for CoPilot training. Each repository you keep there can have pages at <your-github-username>.github.io/<repo-name>

GitHub

To enable this, you need to go into the settings for the repository - look down the left for “Pages”.

It’s possible to have it based on a complicated GitHub action (where your build step happens on GitHub when you push your code), but the easiest thing is just to have it deployed from a branch. To do this you choose which branch (usually main) and whereabouts in the main branch your HTML is. The choices are in the root of your project, or in the /docs directory. I’ve chosen the /docs directory in the screenshot above, since my messy React project is in the root.

React Expense Tracker App

Mon, 22 Jan 2024 00:00:00 +0000

I’m focused on React frontend skills these holidays, and working through Mosh’s React 18 course. The exercise today (which I think I nailed, although I spent more than the recommended hour on) was a small app to track expenses. Like most of Mosh’s exercises it was great because it exercised all the understandings up to that point - so it’s a good starting React project. It used Zod for the form validation which is completely new to me, but looks great.

What's unfinished in your Udemy?

Fri, 19 Jan 2024 00:00:00 +0000

If you work or study in tech, I always feel a good getting-to-know-you question is “what courses or tutorials did you start, but not finish?”

My Udemy doesn’t look too bad:

The ZTM course was good, but I got stuck on an AI API exercise. I think it’s a common sticking point for students since Andrei includes a little rant about how it definitely does work - but I downloaded his repo with the solution and it was having the same errors I was and I gave up in frustration. I probably should have just skipped that one.

Copying Objects in JS

Mon, 15 Jan 2024 00:00:00 +0000

I’ve paid for a month of Mosh to do his React 18 course , and one of the things he makes a big deal about is not to go too deep with nested objects for your state. As soon as you start to update them it becomes apparent why.

Because of the way state works in React, if we need to update part of an object it has to be deep copied, the changes applied to this copy, then that new copy passed back to React to replace the previous version. So, how we copy objects becomes a matter of particular interest.

CSS for React Components

Fri, 12 Jan 2024 00:00:00 +0000

Subscribe to my UX design course 😉

If you think back to HTML as being a document with headings and paragraphs and other semantic bits, it made a lot of sense to have the styles (expressed as CSS) separate to the document. This allows us to change the styles without touching the document - perhaps the user wanted a dark theme, needed the text bigger for accessibility, or perhaps the document was being consumed in some other way - for example a screen reader - so the styles were superfluous.

React - a To Do Example

Mon, 08 Jan 2024 00:00:00 +0000

Since I’m on a roll making different versions of the To Do app, this might be a good time to talk about React . React is one of the giants of front end libraries. It’s based on a few big ideas - and to work effectively in React you need to wrap your head around these.

Overview

Components - when you are developing in React, the starting point of your build is to decompose the user interface in to logical pieces. These components (comprising a mixture of HTML and Javascript) will be the building blocks of your app. In a good composable architecture components are reusable, and that is true for React (there are several sources of components you can pull in). For example, if you created some sort of special slider for your app, it is possible to reuse that quite easily.

htmx - A To Do Example

Fri, 05 Jan 2024 00:00:00 +0000

HTMX is an interesting project to me, and I’ve used it a bit in my large collection of 70% completed side projects, but haven’t really discussed it here. The plan for this post is to talk briefly about what it is exactly, then convert a simple ‘conventional’ (HTML/CSS/Javascript) app to htmx and think about some the differences.

htmx

You could (I recommend you do) read the book about the concepts behind htmx . Carson Gross (the man behind htmx) calls it a book, but its quite the treatise, it could fairly be called a manifesto.

Testing Node.js apps - Mocha, Chai, and Supertest

Mon, 01 Jan 2024 00:00:00 +0000

Bruno is a great open source Postman/Insomnia replacement, and I’ve been using it for basic tests of my node servers using the built in asserts and loving it. This is pretty great, and I gather it’s also possible to go beyond this and write tests in JS in Bruno . I believe it also has the hooks needed to build it into your CI/CD systems.

Any large project is probably going to benefit from a more comprehensive suit of testing tools, and while I’ll still be using Bruno, my serious tests will be managed with these other tools.

Simple SQLite in Express

Thu, 28 Dec 2023 00:00:00 +0000

I don’t have experience with SQLite and want to shift one of my apps over from Mongoose since apparently SQLite is much more capable than I imagined. My usual tactic when trying something new is to try and get a minimal project working on it, so what follows is the simplest possible node/express REST API to demo SQLite.

The simplest possible Express app is going to look something like this. Of course we would have gone to the terminal with npm i express first so this could run.

Gogs, Gitea, Forgejo

Mon, 18 Dec 2023 00:00:00 +0000

I’ve been really pleased with Gogs - it’s lightweight, was simple to spin up, and has worked perfectly. But then this morning on Mastodon, there’s a post from @Codeberg.org describing a security vulnerability in their Git hosting project Forgejo. This issue also apparently affects Gitea and Gogs - what’s up with that?

I actually already did spend a bit of time comparing Gogs and Gitea before deciding on Gogs, since I’d heard of people running Gitea over the past year or so, but only seen that Gogs seemed to be popular with self-hosters in a Lemmy post I’d read. My first impression was that Gitea was more focused on CI/CD and seemed to have a more complicated install process.

Concurrency and channels in Go

Tue, 12 Dec 2023 00:00:00 +0000

In the long ago times, I’d done several years of commercial programming before I ever had to worry about dealing with multiple things happening at the same time. Perhaps because of the rarity of this problem, doing it in traditional languages was not always elegant.

In the modern world of everything happening on the network, and systems being build out of micro-services and APIs, the beginning programmer probably has to deal with this stuff in Programming 102. Luckily, modern languages have these considerations built in, and one language with a particular reputation for that is Go.

Date formatting in Go is quirky

Sat, 09 Dec 2023 00:00:00 +0000

When I’m working in an unfamiliar language, I find its quicker to just ask ChatGPT to write samples of anything I need than to look it up. For instance, last night I needed to format a date in Go, and rather than Google that and pick one of the results and scroll past the ads to read something, I just asked ChatGPT to give me a code example of formatting a date I gave it to DDMMYYYY.

Adding Front Matter To mdserver

Fri, 24 Nov 2023 00:00:00 +0000

The very first issue I opened on mdserver - my server project that serves HTML from markdown files - was that the title of the page (which shows in the browser tab, and is used for browser bookmarks) needed to be set inside the markdown file, rather than generated from the file name. I didn’t invent this idea - I’ve seen this sort of metadata in the top of Jekyll and Hugo markdown. Here’s an example from the Jekyll website :

Bruno asserts

Sat, 11 Nov 2023 00:00:00 +0000

I mentioned Bruno the other day. Although it’s still very much under development, it is shaping up as a great Postman/Insomnia replacement.

One of the aspects I’ve been using today is asserts. As part of a request, you can add some asserts - so when you’re hitting an endpoint it will check what status should it be returning, or given the data you’re passing in, what should be in the response body.

Displaying markdown as HTML

Wed, 08 Nov 2023 00:00:00 +0000

In the spirit of over-complicating things, when I wanted to collect all the links to the services on my homelab into one place, I decided I needed to write them in markdown, and have them converted on the fly into HTML by a server. Then when I couldn’t find exactly what I was after (Harp was closest) of course, I decided to write it.

Markdown

Markdown has definitely been having it’s moment over the last couple of years. It’s a simple open format mark-up language that is quite readable in it’s source form. Although it’s now very fashionable as an input for static site generators, most people will have run in to it when adding simple formatting to forum comments or on instant messaging platforms.

We need to talk about Bruno

Fri, 27 Oct 2023 00:00:00 +0000

I’ve mentioned before that I was using Insomnia as a tool to check my REST APIs as I was developing them, and that I was avoiding Postman (which I guess is more widely used since it’s worth USD5.6 billion ) because

The only reason I’m using Insomnia instead of Postman is that when I tried Postman, it straight away wanted some of my data to make it work. Insomnia hasn’t forced me to do that yet.

New Project Routine

Sat, 21 Oct 2023 00:00:00 +0000

I have a sort of muscle memory for starting little web projects now. I seem to have landed on node/express SSR apps with HTMX sprinkles. So it goes a bit like this:

Create a working directory - all lower case with a simple, but unlikely to be duplicated by me, name.
Open the directory in vscode
npm init in the directory to create the package.json
create a public sub directory, and drop htmx.min.js in there, and create a styles.css there. I’m always conflicted about what to do about this htmx dependency. I’d rather host it rather than use their CDN because reasons . But I also feel bad about committing it on Github. I could .gitignore it, but then when I clone the project on the production server I’d need to add another step to download it. HTMX is only 44K, and Microsoft can afford the bandwidth, so for the moment I commit them, but I need a better solution for the future.
using the git tools in vscode, add .DS_Store to .gitignore (which also creates it), then edit it to also ignore node_modules
npm install express
npm install ejs
create a server.js, and add the hello world code
create a readme.md
commit these files as “initial”
Create the repo on github with the same name - no readme and no licence. I do it this way for a couple of reasons - I want to find out at this point if I’ve already used this repo name, and I want it to give me the cut and paste commands to push the repository.

Simple API endpoint in Go

Wed, 27 Sep 2023 00:00:00 +0000

I’d like a small, quick, low load endpoint on all my nodes and VM’s that exposes a text keyword indicating if that machine is okay for RAM and disk space. I’m currently using Uptime Kuma to monitor if these machines are pingable, but I’d love a tiny bit more information from them so I’d get a Ntfy buzz on my phone if a machine is in trouble.

I mentioned a couple of weeks ago that the benefit of doing it in C rather than Node.js was probably not worth the trouble, but then being a fickle developer, decided to write it in Go.

Use VS Code to work on remote files

Thu, 21 Sep 2023 00:00:00 +0000

If you’ve got a script, or some code to work on, and it’s on a VM somewhere, you can always ssh in and use nano or vim to make your edits. Like a caveman. With an archaic editor, no intellisense, and no spell checking.

Or….

This magic - of editing a files on a remote server over SSH is achieved by using a Microsoft plugin for VS Code - “Remote - SSH ”

Lightweight Web Servers

Fri, 15 Sep 2023 00:00:00 +0000

I’ve been using the excellent Uptime Kuma for my monitoring, but a couple of recent incidents - an external USB mount disappeared on a remote machine, an NVME drive filled up on a different node and stopped backups working because of a configuration error - have made me start to think about more robust monitoring.

The are many great tools for this - Nagios , Prometheus etc. but they are pretty substantial time investments for the excellent power. They can save time series data and display them beautifully. However, all I really want is to add some extra ability to Uptime Kuma.

Cookies, Sessions & Tokens

Tue, 12 Sep 2023 00:00:00 +0000

I’m up to the point in a web app where it needs to come off my lan and into the hands of a couple of users for alpha feedback. Before that happens, I have to add some sort of login/authentication system since it I want to use real, sensitive data. There’s lots of detailed blog posts and videos of how to implement this in an Express app with passport, but what I was missing was the big picture of what actually needs to happen.

Sorting out Node package dependencies when cloning old repos

Wed, 06 Sep 2023 00:00:00 +0000

If you clone an old node project and npm install it, you’ll most likely get a bunch of errors and warning messages. If you just decide to yolo it and run the project, you’ll get a bunch more.

I’ve been doing this exact thing. I want to add some auth to my app, and I’ve been following WebDevSimplified ’s video about using passport . I was building into my app without really understanding what I was doing, ran into problems and decided just to clone his repo and integrate the code into my app. The repo is four years old.

Hide 'Problems' for a file in VS Code

Fri, 25 Aug 2023 00:00:00 +0000

I’m interested in trying out Pico CSS - a lightweight CSS library, but when I tossed it into my project, the linter found and reported 29 problems. One of my processes is to just keep that problems tab clear as I work, so I’d like that to go away.

It’s possible, but only by ’excluding’ the file from your project - it won’t show up in the file view either. That’s fine with me, I never want to deal with the file so we’ll do that, although it might confuse me in seven years if I come back to this project, so I’ll drop a link in my .git_ignore as a clue for future me (excluding the file in VS Code doesn’t affect git finding it).

Installing a Node app on a server

Tue, 22 Aug 2023 00:00:00 +0000

Before I write a fancy Ansible playbook to automatically set up the Nginx/Node combo on my web servers, it might be worth going through how to deploy a Node app so it can run on a server without you being logged in.

Until now, I’ve been running my tests on my laptop, or in a server logged in as myself - sometimes detaching from tmux. But we need a bit more professional set up than that. The process will look something like this:

nginx in Front of a node.js app

Fri, 04 Aug 2023 00:00:00 +0000

NGINX is a great webserver and reverse proxy - as in it can hand off requests to other web-servers. That’s the situation I want to have set up on my VPS. I want NGINX to handle incoming requests - some of them will just be sorted out by returning static HTML, others (like the weather api I’ve been playing with) need to be handed off to other services to respond to.

Unlearning Relational DB

Sun, 09 Jul 2023 00:00:00 +0000

Some of my first university programming was writing CICS COBOL transactions against IBM’s DB2 relational database. My commercial work after uni was mostly written in Clipper which was a sort of compiled version of dBase and used the same data file format. The minimal web work I did in the before times relied on MySQL .

All of which is to say, I’m very comfortable designing relational database schema, and I understand what’s going on at the disk level when they are being accessed and written to.

How to deploy a Node.js app

Wed, 05 Jul 2023 00:00:00 +0000

This is one of those things that is simple once you know it. I had my tiny Node service working on my MacBook, but how do I run it on the server?

Native or Container

Obviously I need Node.js installed on the server, should I have it in a Docker container, or native on the machine. There’s no clear answer here - in a container set up with Docker Compose might be more in line with my ideology of treating machines as disposable, but a native install is simpler, and I probably want to make life simpler at this stage when I’m learning everything.

Using Node.js to return a static file

Sun, 02 Jul 2023 00:00:00 +0000

As mentioned in the previous post , stage one is just to return the same static text file, but from the Node server, rather than NGINX. That’s non-trivial to a rank beginner since I need to figure out 1) how to serve a static file from Node, and 2) how to configure NGINX to hand off calls to the API to Node. This post will look at both of those, but it’s first probably worth just setting out what each of the puzzle pieces are.

Complicating the Temperature API

Wed, 28 Jun 2023 00:00:00 +0000

I’ve been slammed with other work, so my web dev learning has fallen well behind. Luckily, the YouTube procrastination algorithm noticed this and suggested I watch a video from CodeWithCon titled Learn Backend in 10 MINUTES .

Since I was watching a video of a guy learning to land a C152 at St Baths (a skill I do not need) at the time, it was hard to argue with myself that I didn’t have ten minutes to learn all of backend programming.

Installing SSL Certificates with Nginx on Docker

Sat, 29 Apr 2023 00:00:00 +0000

When you’ve successfully got Nginx running in a Docker container, AND got your domain correctly pointing at your nascent website, you’re then going to want to set it up for encrypted, and therefore trusted, browsing with SSL.

Certificates

A couple of posts ago, I mentioned that it was simpler to let Porkbun be the authoritative nameserver for a domain. Part of the reason for that is that if we do that, Porkbun had a button you can press which connects to LetsEncrypt and generates the certificates for you. This usually takes an hour or so, then you’ll be able to download the bundle from that same page.

Your own Aussie server on BinaryLane

Sun, 05 Feb 2023 00:00:00 +0000

Listening to podcasts, I’ve been jealous of US developers who seem to have masses of $5/month VPS (Virtual Private Server) options. When I looked for similar Australian offerings a few months ago, they all seem to start at around $35 which is outside of my ‘have a play with something’ budget range.

I could of course use one of the international options, but one of the main apps on my app ideas list needs to be hosted in Australia and work under Australian data privacy rules. That might be the case for Digital Ocean (or other US companies) if you select an AU server, but I’m not a lawyer. For the imaginary clients of my imaginary app, me being able to say that the hosting is with an Australian company in Australia would be a plus.

Expired Packages Part II

Tue, 31 Jan 2023 00:00:00 +0000

Following on from the previous post…

I went the nuclear route - deleted the node_modules folder, package-lock.json and installed the packages from packages.json. I still had some errors, but the react app at least ran correctly. Also, the messages are a bit more intelligible, and all of them cascade from this one.

# npm audit report

nth-check <2.0.1
Severity: high
Inefficient Regular Expression Complexity in nth-check - https://github.com/advisories/GHSA-rp65-9cf3-cjxr
fix available via `npm audit fix --force`
Will install react-scripts@2.1.3, which is a breaking change
node_modules/svgo/node_modules/nth-check

From my, admittedly ignorant, viewpoint, there’s a couple of weird things going on here.

Expired packages

Mon, 30 Jan 2023 00:00:00 +0000

At several points in the Complete Web Developer course, deprecated packages have been used, with the slide before the video explaining what’s happening, and giving a work around, or sometimes - as is the case for the bit I’m just starting - exhorting the benefits of dropping you into a non-working mess and having you figure it out yourself.

While this argument can be reasonably made - that figuring things out on your own is a valuable skill - it’s also a useful fig leaf to cover up the fact that they haven’t bothered to fix the course to make it work out of the box.

CodePen

Sun, 29 Jan 2023 00:00:00 +0000

I think I’ve written about CodePen before, its a site that allows users to quickly put together HTML, CSS & JS and see the results as they edit. Users ‘pens’ are public and can be tagged, so it also serves as a repository of examples.

It’s possible to host incredibly complex artefacts, such as this 3D Sony Walkman , but what I mostly use it for is to work out simple things - like how to collapse a row of text into a column with a media query.

Using the Community

Fri, 27 Jan 2023 00:00:00 +0000

You can’t always successfully google problems when you’re starting out - usually because you don’t know the correct terminology for the issue or solution. Often you might still get a newbie StackOverflow hit, but when there’s not even that, you need a human to help out.

One of the things ZTM do with their courses is to have a Discord based community, then set tasks to encourage it’s use - for example one of the exercises I’ve already had was to go there and answer a question. Earlier ones were to introduce yourself and to find a partner to work with - both of which would have forced anyone not used to Discord to figure it out.

Openlayers & Vite

Thu, 26 Jan 2023 00:00:00 +0000

In Randy Pausch’s last lecture he talks about the benefit of brick walls in our lives - they tell us how much we really want something. Software development is full of these brick walls - things we want to do, but there’s a barrier to achieving it. Will we persevere and accomplish the thing, give up, or some other compromise.

APIs - http & https Mixed Content error

Tue, 24 Jan 2023 00:00:00 +0000

<img src="/images/screen-shot-2023-01-16-at-4.45.53-pm.jpg alt=“Mixed Content: The page at ‘’ was loaded over HTTPS, but requested an insecure resource '

Ran into a little bump today - I was calling a cool API that gives the current location of the International Space Station. In a classic case of “it worked on my machine” it worked perfectly in the Live server in VS Code on my laptop, but when I pushed it up to my GitHub space, it didn’t work - throwing the error:

React code is not HTML

Sun, 22 Jan 2023 00:00:00 +0000

I was looking at this ugly code in a React app:

<div style={{overflow: 'scroll', border: '1px solid black', height: '600px' }}>
 { props.children }
</div>

Since I don’t need any of those CSS properties to change at any stage, I could just convert it to pure HTML/CSS right? Well no:

The newbie trap I’ve fallen for here is that although that <div style= tag looks like HTML, it’s actually not. It’s not a template that will be filled out in the build step, it’s React code that will be used to mutate the virtual DOM.

De-structuring objects in JS

Fri, 20 Jan 2023 00:00:00 +0000

I’ve worked through my first React tutorial app, and obviously that’s a lot - I’m struct by how messy mixing HTML, JS and React is.

One language feature that’s being used quite a bit, and that is apparently a JS ability I’d never seen is ‘destructuring’ object properties. It’s very cool and obviously useful. It’s a way of extracting just the properties you need from an object and then using them without accesing them via the object. An example will make it clearer.

Digital Color Meter

Wed, 18 Jan 2023 00:00:00 +0000

For the Calculator project, I needed to know the exact RGB values for the colours on the iOS calculator buttons so I could reproduce them. Assuming a tool for reading colours from the screen exisited, I googled it, and was surprised to find this exact tool is already installed by default on MacOS.

It’s called Digital Color Meter and just shows the RGB values for anything on the screen under the cursor.

Calculator

Mon, 16 Jan 2023 00:00:00 +0000

I’ve been doing a bit of driving during the holidays, which means a lot of podcast listening. An episode of JavaScript Jabber about JS features you should never use sparked my interest in [eval()](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/eval). eval() takes whatever you pass it in a string and executes it in the JS engine. This is a crazy concept if you’ve come from complied languages, and has obvious security implications. As with dynamic typing, I’m trying to force myself out of my comfort zone to embrace JS’s unique talents so I was keen to try eval().

CWD - 185 - Problem solving

Sat, 14 Jan 2023 00:00:00 +0000

/* 
Question 1: Clean the room function: given an input of [1,2,4,591,392,391,2,5,10,2,1,1,1,20,20], 
make a function that organizes these into individual array that is ordered. For example 
answer(ArrayFromAbove) should return: [[1,1,1,1],[2,2,2], 4,5,10,[20,20], 391, 392,591]. 
*/

function ctrFunction1(inputArray) {
 // copy the array since we're mutating it
 const array = [...inputArray];
 array.sort();

 const numberObject = {};
 for (const number of array) {
 if (numberObject[number] === undefined) {
 // this property does not exist, so add it
 numberObject[number] = [];
 }
 numberObject[number].push(number);
 }

 // object now contains arrays for each number, but the ones with a
 // single element need degloved
 for (property in numberObject) {
 if (numberObject[property].length === 1) {
 numberObject[property] = numberObject[property][0];
 }
 }

 // now turn back to array 
 return Object.values(numberObject);
}

const array1 = [1, 2, 4, 591, 392, 391, 2, 5, 10, '2', 1, 1, 1, 20, 20];
const transformedArray1 = ctrFunction1(array1);
console.log(transformedArray1);
// [1, 1, 1, 1], [2, 2, '2'], 4, 5, 10, [20, 20], 391, 392, 591]

Line 10

When I’m looking at a function, I’d prefer not to also have to hold global state in my head - so I’m all for functional programming as far as that goes. I’m less concerned about side effects, so I wouldn’t always bother to copy a parameter like this, but the argument is stronger for an array than an object since in other languages an array might be a value type.

Types of Concern

Fri, 13 Jan 2023 00:00:00 +0000

I am still struggling with the dynamic typing of JS. I guess the benefit of such an approach is needing less characters - which makes sense in a scripting language like bash, but in real programming it opens us up to a whole class of avoidable errors.

To program defensively in JS would mean loading the start of function with a series of type checks. I don’t see much of that in other people’s code, so I assume we just, don’t?

Lost in Translation

Wed, 11 Jan 2023 00:00:00 +0000

We’re in a pretty good place now (compared to a few years ago) in terms of being able to rely on JavaScript behaving the same on different platforms. There’s still some differences (mostly in when things are implemented) but overall, not to bad once you decide to no longer support Internet Explorer.

In times past, it was a lot more painful . A few of approaches to deal with this arose. One is to let a library, such as jQuery or a polyfill deal with it, and the other is use a translation utility such as Babel to down convert (transpile) your modern JavaScript to something that will run in more browsers.

Functions in JavaScript

Mon, 09 Jan 2023 00:00:00 +0000

As with other languages, functions are a little lumps of code with their own scope. They can optionally take some arguments, and optionally return a value.

In JavaScript they often have names, can be passed around as types and have a condensed form suitable for functional programming.

function addNums(a, b) {
 return a+b;
}

console.log(addNums(3,4)) // 7

Scope

Arguments are passed in by value so they have local scope only in the function body.

CodePen.io

Thu, 05 Jan 2023 00:00:00 +0000

I quite often leave a link to a GitHub repo to share my source in these posts, and on a few recent ones, a link to a live version of a page on my github.io. In a recent installment of CWD , Andrei shared some previous students’ solutions, and some were hosted on CodePen.io which I hadn’t seen before.

It’s a cute concept, you can enter HTML, CSS & JS and see a live view of the page below. It looks super extensible - there’s mentions of SCSS, Typescript and preprocessors for JS in the settings.

Step Ahead

Wed, 04 Jan 2023 00:00:00 +0000

I was a bit pleased with myself when I started the next content element in the Complete Web Developer course to find that one and a half of the extensions I’d made to the tutorial app for my own fun were specified as the next task.

In my previous post, I’d talked about using a class to denote if an item was completed, and using a style to indicate this by crossing it out. What I haven’t discussed was that I’d captured right click events on the list items to make this delete them. I wasn’t entirely happy with that for a couple of reasons:

Web Reference

Tue, 03 Jan 2023 00:00:00 +0000

There is no shortage of places to reference material to help when developing web apps, including a gazillion tutorials and blog posts (like mine) of various quality, and more importantly - based on the state of play at the time they were written, which could be any time in the last twenty years. I keep bumping up against this - great, clear explanations addressing whatever I was googling, but which turn out to use out of date bits.

Document Object Model - ToDo

Mon, 02 Jan 2023 00:00:00 +0000

I’m up to Section 12 of the Complete Web Developer course “DOM Manipulation” and it feels like we’re finally at the stage of pulling everything (HTML, CSS & JavaScript) together to make minimal web apps. Since the course is light on building challenges, I’ve set myself one - to make a simple todo list (the classic step up from “hello world”).

The Document Object Model is an entity representing the HTML with attached CSS for a page. The magic is that we can access this in JavaScript, and therefore change it, including hooking into events on it - such as a user pressing a button.

Are you okay JavaScript arrays?

Sat, 31 Dec 2022 00:00:00 +0000

As a visitor from sensible type-safe land, this makes me uncomfortable:

As I keep learning, I’m interested to find out if JavaScript objects turn out to just be arrays. To get from here to there, you’d just need to be able to use some sort of self[2] notation to access properties from inside the functions.

Curse of Backwards Compatibility

Thu, 29 Dec 2022 00:00:00 +0000

I was listening to a JavaScript podcast today (JavaScript Jabber ) and in one of the discussions a point was made about how HTML, CSS and JavaScript have all had to maintain considerable legacy behaviors that compile-able languages do not have to. For instance, when Swift underwent some substantial changes from Swift 2 to Swift 3 - some code broke for developers and needed reworking because things had changed or been removed. Nothing broke for users - they could either still use their previously compiled applications, or they were delivered new ones from the app store.

Running Javascript in VS Code

Tue, 27 Dec 2022 00:00:00 +0000

I’ve been using the Live Server plugin to see HTML & CSS updated as I edit, and that will also be useful when I start using Javascript for web development, but as you can see above, I’m not quite up to that. It seemed there should be a way to run JS in VS Code, and it turns out it’s easy.

You just need something installed that can run Javascript. Node.js is the obvious choice, and you’re going to need it later in your development journey. Just install Node.js then the first time you try to run some JS in VS code, it will ask you what to use, select Node and you’re in business.

99 CSS Layout challenge

Fri, 23 Dec 2022 00:00:00 +0000

In the Zero To Mastery Complete Web Developer course, I’m up to the first practical challenge - to use CSS to layout a reasonably standard looking web page using flex-box and grid to make it responsive.

Frustratingly, both for writing this, and while I was trying to build the page, I’m unable to screenshot the example of the page I was supposed to be building, and instead had to keep opening the video and seeking the two second flash of the completed project, and eventually being reduced to photographing my laptop screen like a boomer relative sending me a meme:

HTML 001

Fri, 16 Dec 2022 00:00:00 +0000

A HTML file is a text file that can be displayed in a web browser. It is marked up in the sense that tags are applied to the text to signify the purpose of that text in the structure of the document. For example:

<h1>Greetings</h1>
Hello Earthlings

The <h1> tag tells the browser that Greetings is a heading. The heading tag is paired. There’s an opening tag <h1> and closing tag </h1> that let the browser know where the heading starts and ends. Most tags are paired, but there are some unpaired tags such as
which inserts a line break.

CSS for Beginners

Thu, 15 Dec 2022 00:00:00 +0000

I mentioned a couple of days ago that the ZTM webdev course was skipping forwards too quick and that it would need to be supplemented. For CSS, I think the supplement for me is going to be this series from Dave Gray.

Who is Emmet?

Wed, 14 Dec 2022 00:00:00 +0000

I knew there was some magical way of entering all the the boilerplate in Visual Studio Code as I’d seen it happen in several videos, and assumed is was some sort of macro expansion thing in the editor. Fast forward a few blog post readings and youtube viewings and I keep seeing tangential references to someone called Emmet. Turns out they’re the same thing, and it’s pretty cool.

It’s not a new idea to have functionality in code editors to insert snippets of code. Emmet goes a bit further than that - and like many tools made by programmers for programmers it goes way to technical to the point where you need to memorise ridiculous amounts of combos to to some awesome stuff (I’m looking at you whoever made it possible to use vi commands in VS Code). Nevertheless, Emmet is extremely handy even at my n00b level.

ZTM - Complete Web Developer

Tue, 13 Dec 2022 00:00:00 +0000

I started my first Udemy a few days ago. I was watching one of those “How I’d learn to code if I started over ” YouTubes, mainly because I’d like to know enough JavaScript to write little REST API’s on Node.js, but also because I’m starting to think web development makes more sense for a couple of the applications I’ve got on my (ever growing) list of app ideas.