A while ago I updated my site to Eleventy.js. This improved my blogging process considerably, as Markdown is quick to write, and especially code samples are easy to copy-paste without any escaping.

I put all my posts into one Github repository. Using Visual Studio Code, I get nice realtime preview, and once I'm done, I just git commit, git push on my computer, and git pull on the server, and regenerate the site with Eleventy.

Now only issue is that one usually finds 5-10 things to change after every post, and while VS Code has great git support built in, even the simple update process gets a bit tedious with commands being run on SSH side. So I started wondering if I could use the Github webhooks to automate the regeneration. Turns out: YES.

Simple Script to Pull Markdown from Github and Regenerate

First component is of course a small shell script to automate the git pull and regeneration. One could do a more elaborate one, but this worked for me server-side:

#!/bin/sh

cd ~/myblog_post # here are the .md files
git pull
cd ~/myblog_eleventy # here's the 11ty site generator
npx @11ty/eleventy --input=. --output=../apps/myblog_static # hosting dir

Node.js Mini-Server for Github Webhooks

Next, I needed to set up a web server that would get the HTTP POST from Github whenever I push changes. Here your configuration will depend on hosting you have, but Opalstack for example has simple installation of a Node.js application. I usually disable the automatic restarting (crontab -e etc.), use ./stop script and run my server manually for a while to see everything works, before restoring the crontab.

If you choose to forego Github webhook security mechanisms, the code is really simple, but in that case, anyone knowing the addess of your server can flood you with fake push requests. So let's take the high road and use this gist to verify Github hooks! I chose to use Polka so I needed to modify the headers part of the code just a bit:

const { exec } = require("child_process");
const polka = require('polka');
const { json } = require('body-parser');
const crypto = require('crypto')

const port = 12345;
const secret = 'ohreally :)';
const sigHeaderName = 'x-hub-signature-256';
const sigHashAlg = 'sha256';

// Middleware to verify Github "signed" POST request
function verifyPostData(req, res, next) {
  console.log('Verifying signature', req.headers[sigHeaderName]);

  if (!req.rawBody) {
    console.log('Request body empty');
    return next('Request body empty');
  }
  
  const sig = Buffer.from(req.headers[sigHeaderName] || '', 'utf8');
  const hmac = crypto.createHmac(sigHashAlg, secret);
  const digest = Buffer.from(sigHashAlg + '=' +
    hmac.update(req.rawBody).digest('hex'), 'utf8');

  if (sig.length !== digest.length || !crypto.timingSafeEqual(digest, sig)) {
    console.log('Got request with invalid body digest');
    return next(`Request body digest (${digest}) did not match
      ${sigHeaderName} (${sig})`);
  }
  
  console.log('Verification done.');
  return next()
}

polka()
  .use(json({ verify: (req, res, buf, encoding) => {
    // Store raw body data in req rawBody variable
    if(buf && buf.length) req.rawBody = buf.toString(encoding || 'utf8');
  }
  }))
  .get('/', (req, res) => { res.end('Hello, polka!'); }) // just for testing
  .post('/myblog', verifyPostData, (req, res) => {
    console.log('Article repo updated, generating site...');
    exec('~/blog_eleventy/gen.sh', (error, stdout, stderr) => {
      if(error) console.log(`error: ${error.message}`);
      if(stderr) console.log(`stderr: ${stderr}`);
      console.log(`stdout: ${stdout}`);
    });
    res.end(`Hello, Github!`);
  })
  .listen(port, () => {
    console.log(`> Running on localhost:${port}`);
  });

For some reason, my particular configuration had the Github authorization header in req.headers['x-hub-signature-256'] instead of the capitalized X-Hub-Signature-256. Go figure.

Read post

I've had an iPad Pro 12.9" for some time, and been happily using it for sheet music when playing the piano. However, having to interrupt your playing to swipe to the next page does get annoying. You can get a $100 commercial AirTurn pedal, but since one can get a microcontroller from Ebay/Aliexpress for $4 and a simple foot pedal switch for $2, I thought it would be a fun one evening hacking project. It turned out quite nice:

Getting started: ESP32 devkit

The sheet music applications on iPad (and Android) usually have bluetooth keyboard support, turning the page when user presses an arrow key or spacebar. So the minimum viable product is just a bluetooth-enabled microcontroller that can pair with the iPad and send a single key upon request.

The ESP32 chip has both WiFi and Bluetooth, and I chose it for this project, as it is readily available in a compact form factor, and it's easy to program with Arduino. Searching [AliExpress](https://www.aliexpress.com for ESP32 should give you plenty of options.

I had a ESP32 board labelled with "ESP32 DEVKITV1" in my parts box, and it was quite easy to set up with this ESP32 tutorial:

1. Install the driver for USB-UART bridge
2. Add source URLs for ESP32 to Arduino and install ESP32 support
3. Select "DOIT ESP32 DEVKIT" from the board menu
4. Hold down "Boot" button on the board while selecting "Upload", release when the console says "Connecting..."

Before you proceed with the tutorial, check that you can get the lights blinking or flash some other example code successfully for the board. There are plenty of resources around if you hit into any issues! I had to google the step 4 myself, although it would have sufficed to read the linked tutorial carefully...

Read post

Continuing from my Project Euler Problem 1, here are the next two Euler problems and some reflection!

Project Euler Problem 2

The second problem asks to find the sum of even Fibonacci numbers numbers below four million. For procedural languages, this is quite walk in the park, but for me a concise method to produce the series in Clojure proved a bit of a brain nugget!

Python 3

For Python, I decided to make a simple generator that yields the Fibonacci numbers below a given treshold N. Rest is just list comprehension:

def fib(N):
    i, j = 0, 1
    while j < N:
        yield(j)
        i, j = j, i+j

print(sum(i for i in fib(4e6) if i%2==0))

C++

In C++ it's natural to just loop with a conditional addition. This time I decided to drop using namespace std; to save a few characters. Without i, j = j, i+j syntax, the temporary variable ij is a bit ugly.

#include <iostream>

int main() {
    int i=0, j=1, s=0;

    while(j<4000000) {
        if(j%2==0) s += j;
        int ij = i + j; i = j; j = ij;
    }
    std::cout << s << std::endl;
}
}

Read post

I've long thought about learning Lisp. Some time ago I ran across Clojure and the map, set and vector implementations felt like a modern addition. It's running on top of JVM which is nice in a sense, but a major pain in the ass when installing. Well, I swallowed my wow to never install JDK again and got Clojure running quite painlessly.

To learn the ropes, I thought it would be a fun idea to rewrite Project Euler problems 1-10 with Clojure to see how it would compare against my language of choice for these types of algorithmic problems, Python.

Just to make it more fun, I decided to implement them in my second favorite language, Javascript as well. Since ECMAScript 9, it's actually a pretty powerful and concise language, and coupling it with Node.js unleashes some serious usage options outside just web pages. Npm dependency hell sucks, but that's a topic for another post. For more bare metal, I'm also doing C++ (basically sticking to C functionality until maps and sets make themselves useful). Let's see how the languages stack up!

Project Euler Problem 1

The first problem in Project Euler is simple, add up numbers below 1000 that are divisible with either 3 or 5 (or both). Modulo operator % can be used to get the remainder of a number when divided by 3 and 5, and basically we add all numbers that have either modulo as zero.

Python 3

With Python, the most concise implementation is to use the sum(iterable) and a list comprehension, netting us a nice one-liner:

print(sum(i for i in range(1000) if i%3==0 or i%5==0))

Read post

After Angular, React.js and Vue, svelte is the new and cool kid on the block. I have to admit, the compile-to-plain-javascript philosophy is nice, and what is especially cool is the fact that with Rollup.js you can easily bundle a svelte app into easy-to-embed component.

I wanted to make a simple embeddable Svelte component for a friend of mine, that would be as simple as possible to include in a blog post, web site or similar with just:

<script src="https://mydomain.com/myscript.js"></script>

The script should ideally just put the component in the same place. Turns out this is really easy with Svelte. Let's dive in!

Setting up Svelte

You may want to read more about rollup-plugin-svelte and Making an app from excellent Svelte tutorial to get more familiar with the tools used here, as I'll only cover the bare essentials.

First you need to install the necessary plugins. We'll start with Rollup and install it globally so you can just use rollup command to do things. If you want, you can also install rollup locally.

user@server:~$ sudo npm install --global rollup
user@server:~$ rollup -v
rollup v2.40.0

I initially got an error saying something about imports and after a bit of googling found out that my Ubuntu version of node was very old (node -v outputted 8.x.x when 10+ was needed) – if you encounter the same issue, upgrade to a newer node (nvm is a great too to do this).

Next make a product directory and install svelte and rollup-plugin-svelte:

user@server:~$ mkdir embed-svelte
user@server:~$ cd embed-svelte/
user@server:~/embed-svelte$ npm init

[answering to npm prompts]

user@server:~/embed-svelte$ npm install --save-dev svelte
user@server:~/embed-svelte$ npm install --save-dev rollup-plugin-svelte
user@server:~/embed-svelte$ npm install --save-dev @rollup/plugin-node-resolve

Simple Svelte component

We'll go really barebones with this tutorial, just a simple heading and a paragraph. Create a Svelte component Embed.svelte:

<script>
export let name; // from props
</script>

<h1>Hello, {name}!</h1>

<p>This component was brought to you by <a href="https://codeandlife.com/">Code
  &amp; Life</a>.</p>

Then we'll need a simple .js script to import that our mini-component and inject it to the DOM of whatever page has included the resulting script. Let's call this embed.js:

Read post

So, I started writing a file database and toolset called fileson to take advantage of AWS S3 Glacier Deep Archive (let's just call it GDA from now on). With 1 €/mo/TB storage cost, it is essentially a dirt cheap option to store very infrequently accessed data like offsite backups.

Why not just use rclone? Well, I disliked the fact that all tools do a ton of (paid) queries against S3 when syncing. I thought a simple JSON file database should work to keep track what to copy and delete. Well, that work is progressing, but as a part of that...

Encrypting on the fly with Python and Pycrypto(dome)

I started thinking that client side encryption would be useful as well. AES is tried and tested, and it's easy to find sample code to do it. But it seems wasteful to first create encrypted files on your hard drive, then upload them to AWS and finally delete everything.

Luckily, the Python AWS SDK boto3 has a great example on how to upload a file to S3 with upload_fileobj that accepts "a readable file-like object". What does that mean? Let's find out!

Read post