Split MIDI Files with Python

In my previous post I showed how to use Raspberry Pi to automatically record MIDI files from digital piano whenever you turn it on. However, if you sit down and play for an hour, and get one big MIDI file, it is not very useful. So what to do?

Pianoteq has a feature that splits your playing session based on breaks you take between playing notes. So I decided to mimick this feature with a simple Python script that:

  1. Keeps tab on keys and pedals pressed
  2. Whenever X seconds elapse without any keys or pedals down, a new MIDI file is started
  3. Additionally, if user presses and releases sustain pedal several times in the end, filename for that MIDI is altered to “highlight” that file

I first thought to learn enough of MIDI file format to do everything from scratch, but there’s quite a bit of small details to handle, so in the end I decided to use an external toolkit from Craig Stuart Sapp called midifile to do the heavy lifting. You should be able to just clone the Git repo and make it with Raspberry Pi:

pi@raspberrypi:~ $ git clone https://github.com/craigsapp/midifile
pi@raspberrypi:~ $ cd midifile
pi@raspberrypi:~/midifile $ make

You should now have two useful commands in ~/midifile/bin: toascii to read a MIDI file and dump an ASCII (text) version of it, and tobinary to do the reverse. With Python’s Popen and smart piping, we can read and write the binary MIDI files as they were in this text format. You can check out how the format looks like with some MIDI file you have:

~/midifile/bin/toascii somemidi.mid | less

Here’s a sample of a MIDI file recorded by arecordmidi (I added the note in square brackets):

"MThd"
4'6
2'0
2'1
2'384

;;; TRACK 0 ----------------------------------
"MTrk"
4'50044
v0      ff 51 v3 t120
v0      ff 58 v4 '4 '2 '24 '8
v7147   90 '58 '29
v88     b0 '64 '5
v1      b0 '64 '7

[LOTS OF MIDI EVENTS]

v0      ff 2f v0

There’s some header data in the beginning, and each MIDI event is comprised of a deltatime field starting with ‘v’, and then fairly standard MIDI events in straightforward syntax. I hardcoded my program to expect single track which starts with tempo and speed data (ff 51 and ff 58 lines) which I use to calculate how many deltatime units is one second. I copy this header part to start of every MIDI file, and append the final “ff 2f v0” (END TRACK) event to the end. Here’s the full code:
Continue reading Split MIDI Files with Python

Using Raspberry Pi as an automatic MIDI logger

During my summer holidays I got an interesting idea: Pianoteq has a very nice feature of “always on MIDI logging” that saves everything you play on your keyboard while Pianoteq was on. I’ve previously made some MIDI projects and had a great idea:

How about building a small device that records everything I play on my piano, and save it as MIDI files?

This would enable me to later grab a good performance, and eliminate the “recording anxiety” I get if I know I’m recording and should definitely not do any mistakes during the next 1000+ notes. Furthermore, even with easy MIDI recording to USB stick, it’s still several manual steps plugging the memory stick in, starting recording, stopping it, lugging it to a computer, etc.

My first idea was to use some WLAN-enabled embedded device, but MIDI IN would require optoisolators and some custom electronics, and more modern digital pianos often come with only USB MIDI, so it could easily become an exercise in communication protocols. Fast forward a couple of minutes to my next revelation:

Raspberry Pi Model 0 W already has USB and WLAN, and it’s small. Why not use that?

Turns out using a RaspPi as fully automated MIDI logger is really easy. Read on for instructions!

Update: Also check out my follow-up post to split the recorded MIDI files automatically!

Recording MIDI with Raspbian

Turns out recording MIDI from a USB MIDI enabled device is really easy. When I plug in my Kawai CS-11 (sorry for the unsolicited link, I love my CS11 :) to the Pi (or just turn it on when it’s plugged in), dmesg shows that the Pi automatically notices the new MIDI device:

[  587.887059] usb 1-1.5: new full-speed USB device number 4 using dwc_otg
[  588.022788] usb 1-1.5: New USB device found, idVendor=0f54, idProduct=0101
[  588.022800] usb 1-1.5: New USB device strings: Mfr=0, Product=2, SerialNumber=0
[  588.022807] usb 1-1.5: Product: USB-MIDI
[  588.074579] usbcore: registered new interface driver snd-usb-audio

Once the USB MIDI device is found, you can use arecordmidi -l to list available MIDI ports:

pi@raspberrypi:~ $ arecordmidi -l
 Port    Client name                      Port name
 14:0    Midi Through                     Midi Through Port-0
 20:0    USB-MIDI                         USB-MIDI MIDI 1

Continue reading Using Raspberry Pi as an automatic MIDI logger

Picoscope 2208B MSO Review

There are few tools that are essential for an electronics hobbyist. When I started, I had a soldering iron, a multimeter and some components, and that was about it. That got me quite far because you can do simple debugging even with a multimeter, but once you start to do any communications, you will either work in the dark or get a signal analyzer, oscilloscope, or both. I reached that point about 9 months into my hobby, and eventually decided to get an entry-level PicoScope from Picotech. You can read the whole story from my PicoScope 2204 review from four years ago.

Long story short, I was extremely happy with my Picoscope, and I’ve been using Picotech’s products ever since in various projects. In the past years, I’ve also been collaborating with Picotech, so I’ve had the chance to use also their higher end models, including the frighteningly powerful 4-channel, 200 MHz, 16 bit PicoScope 5444B, which is really great but maybe even too hefty for my use. So when I was offered the chance to try out Picotech’s latest generation of their entry-level 2000 series published just a month ago, I was immediately in.

Without further ado, let’s get reviewing!

PicoScope 2000 series overview

The new PicoScope 2000 series is divided into roughly two groups of equipment: The entry models 2204 and 2205 range in price from 139€ for the 10 MHz 2-channel 2204A to 419€ 2205A and 2405A which are 25 MHz and have MSO (mixed-signal oscilloscope, i.e. it has 16 channel digital part as well) capability and 4-channels, respectively. Don’t let the low bandwith confuse you, even these models have sampling rates ranging from 100 MS/s to 500 MS/s, so you will get quite a lot of measuring power out of them.

Biggest limitation with 2204 and 2205 models is the buffer size, which ranges from 8 kS to 48 kS, so for longer captures than a few waveforms, only option is the continuous capture over USB which worked at a steady rate of 1 MS/s the last time I used it. So you can do unlimited capturing of signals around 100 kHz, but above that it’s the normal oscilloscope triggering business — that’s the way scopes have always worked from their beginnings, so it gets the job done as well.

  2204 2205 2206 2207 2208
Bandwith 10 MHz 20 MHz 50 MHz 70 MHz 100 MHz
Sample rate 100 MS/s 200 MS/s 500 MS/s 1000 MS/s 1000 MS/s
Resolution * 8 bit 8 bit 8 bit 8 bit 8 bit
Memory 8 kS 16 kS (48 kS w. MSO/4ch) 32 MS 64 MS 128 MS
Price (2015-22-05) 139 € 209 € 319 € 459 € 629 €
Options MSO or 4ch MSO or 4ch MSO or 4ch MSO or 4ch

*) Resolution for repeating signals can be increased to 12 bit with multiple samples
Continue reading Picoscope 2208B MSO Review

New Picotech 2208B Unboxing Video!

Picotech launched a new set of very compact but powerful 2000 series oscilloscopes just a few weeks ago, and with my long-term collaboration with them (they really rock :) they were kind enough to send me a unit for reviewing! The review will hopefully come quite soon, but meanwhile, enjoy this amazing unboxing video. And with “amazing”, I mean “I did not completely fail the lighting and achieved 90 % intelligible pronunciation”.

The unit in question is the high-spec 100 MHz 2208B with 16 channel logic analyzer unit built in. I knew that this replacement line to the old light blue 2000 series scopes was small, but I was still amazed by the compactness of this beast. Having had a top end 5000 series Picoscope for a few years, I’m seriously considering the “downgrade”, as I mostly use just one or two channels of the scope, and even those are in many cases digital signals. But I’ll return to this in the review later.

Let me know what you think, either here on in the Youtube comments section!

BeagleBone Black GPIO Benchmark

Look what the mailman brought: It’s a shiny (or maybe matte?) BeagleBone Black, freshly arrived (actually it’s been over a month, but time sure flies…) from Newark element14! I’ve been doing Raspberry Pi related hacking for a while, but especially when the Pi was still fresh and new, I did from time to time consider if the grass would be greener on other side of the fence. Or blacker, in this case, as I mean BeagleBone Black.

BeagleBone was long very much more powerful than Raspberry Pi, but now that Pi2 has come out, price and specification-wise they are closer than ever. A quick personal comparison chart:

  BeagleBone Black Raspberry Pi 2 (B)
Price 46 € (Element14) 32 € (Element14)
Processor 1GHz single-core Cortex-A8 0.9GHz quad-core Cortex-A7
Memory 512MB DDR3 1GB
Connections USB host, USB device, micro-HDMI 4x USB, HDMI, 3.5mm Audio/analog video
GPIO 2x 46 pin headers (65 digital I/O) 40 GPIO pins (26 digital I/O)
Other 4GB integrated flash, works as USB device camera and display interface on board

When Pi1 was out, the BeagleBone Black with the more modern Cortex-A8 chip and higher clockrate was definitely the more powerful, but now with 4-core Pi2, the tables have somewhat turned. Still, the clockrate is higher and there’s more GPIO. And speaking of GPIO, my Raspberry Pi vs. Pi2 GPIO benchmark has gotten a lot of interest, so I thought the best way to take this black beauty for a test drive would be to benchmark BeagleBone Black GPIO in a similar way.

Test setup

Test bench

The test subject is the most recent revision C of BeagleBone Black. I followed the (a bit lacking in detail and readability) Getting Started guide and downloaded the latest Debian Jessie image (8.3, 2016-01-24), flashed it to card and ran apt-get update and apt-get dist-upgrade (2016-04-14).
Continue reading BeagleBone Black GPIO Benchmark

Topre Realforce vs. Matias Mini Shootout

Topre Realforce vs. Matias Mini

It’s been ten months since I got and reviewed the Topre Realforce 88UB. I’ve been very satisfied with the keyboard, the only real issue being that for some reason my brain is having strange difficulties with adjusting to the new microlayout since my time with HHKB — for some reason I still hit adjacent keys quite a lot when coding, getting # instead of % or ‘k’ instead of ‘j’ (doubly frustrating with Vim!).

However, the thirst for new experiences never really leaves you, so I decided to try out the Matias switches that have caused quite a lot of discussion at Geekhack. Since I’ve liked the low thud of Topre keys and loved the compact layout of RF 88UB, I decided I’d see how the Matias Mini Quiet Pro compares to my Realforce.

After using the Matias keyboard extensively for several weeks, I think I have enough experience to write a bit about this new entrant to the rather established Cherry/Topre/Unicomp triopoly of mechanical niche keyboards. To make it more interesting, it’ll be a shootout against the reigning king, Topre Realforce 88UB. Fight is on!

Warm-up Round: Specifications and Price

thekeyboardco
I got my Realforce through The Keyboard Company in UK, and they were kind enough to provide the Matias Mini review unit for this shootout. As a thanks I’m including their banner here, and based on several years of personal experience I can really recommend them, especially if you’re within EU as there will be no customs fees.

Specification-wise the Realforce and Matias are quite similar with reduced tenkeyless layout sporting function and cursor keys. The Topre has standard pageup/pagedown etc. column whereas Matias a bit more compact but requires a function key to access insert, home and end and the more esoteric print screen, scroll lock and pause/break keys. Price-wise, the Matias sits in the not-quite-inexpensive $160 price range, but the Topre almost doubles this with its hefty $295 price tag.

  Topre Realforce 88UB Matias Mini Quiet Pro
Price ca. $295 ca. $160
Layout Tenkeyless Compact with cursors
Mechanism Topre 45g Matias quiet
Weight Heavy (1200g) Sturdy (950g)
Connectivity USB USB
Keys Dye sublimation (black on black) Laser etched (white on black)
Extras Yellow WASD keys and keycap tool 3 USB 2.0 ports and two cables (long & short)

Continue reading Topre Realforce vs. Matias Mini Shootout

Raspberry Pi 2 vs. 1 GPIO Benchmark

Raspberry Pi 2 GPIO Benchmark

It’s battle time! Some of you may have heard that Raspberry Pi 2 is out with more punch than ever. Just how much more? Well, apt-get dist-upgrade went about 5 times faster with the new Pi. With 1 GB of RAM and four cores, this will definitely be a boost for my home SSH box ergonomics over the previous version.

But what about hacking? There has been a lot of interest in getting GPIO benchmarks for the Pi 2 similar to my earlier Raspberry Pi GPIO benchmark. Well here it is! Please refer to the earlier article for source code and nice screenshots of square waves, as I’ll concentrate on the performance difference only here. You can also get the code from Github:

https://github.com/jokkebk/rpi-gpio-benchmark

Summary of results

All the Pi 1 benchmarks were ran 14th and 15th February 2015 using latest versions of the libraries as stated in my updated benchmark post. Pi 2 benchmarks were all run 25th and 26th March 2015 with the latest versions. If you get significantly different results at a later date, please let me know and I’ll update the table!

Language Library Pi 1 Pi 2 Change
Shell /proc/mem access 2.8 kHz 7.0 kHz 2,5x
Shell / wiringPi WiringPi gpio utility 40 Hz 95 Hz 2,4x
Python RPi.GPIO 70 kHz 243 kHz 2,5x
Python wiringpi2 bindings 28 kHz 103 kHz 3,7x
Ruby wiringpi bindings 21 kHz N/A 3,7x
C Native library 22 MHz 41.7 MHz 1,9x
C BCM2835 5.4 MHz 7.2 MHz 1,3x
C WiringPi normal GPIO wiringPiSetup() 4.1 MHz 9.3 MHz 2,3x
C WiringPi GPIO wiringPiSetupGpio() 4.6 MHz 9.4 MHz 2x
C WiringPi sys wiringPiSetupSys() 120 kHz 185 kHz 1.5x
Perl BCM2835 48 kHz 154 kHz 3.2x

Continue reading Raspberry Pi 2 vs. 1 GPIO Benchmark

Topre Realforce 88UB – A Happier Hacking Keyboard

Topre Realforce 88UB review

This is my review of the Topre Realforce 88UB mechanical keyboard with evenly weighted 45g switches and UK layout. This is the ISO layout sister model of the Topre Realforce 87U (87UB / 87UW) so everything in this review will apply for that model, except a few layout details regarding the extra keys. Before diving into details, a short background of my previous journey:

Two years ago I got really interested in high quality mechanical keyboards. After all, the keyboard is the instrument I use several hours every day, so if I am spending close to 2000€ on a workstation and display, why should I compromise with a cheap 30€ keyboard, especially since the keyboard will likely last several workstation lifespans?

After heavy research I settled on Happy Hacking Keyboard 2 Professional due to its compact “hacker” layout which seemed great for vim, and Topre key switches, which many consider to have the very best tactile feel. See my review of the HHKB2 keyboard for details.

However, in two years of typing with the HHKB, I never fully learned to use function key bindings for cursor keys and Home/End fluently. Coding with Vim somewhat alleviated the problem, but every time with a command prompt or a shell this nagged me. Also, playing games required a separate keyboard, as most games make use of function keys, and many make use of both WASD and cursor keys. It was time for something else. Something better.

Topre Realforce 88UB – Best of both worlds?

After my experience with HHKB, I had the critical components for my dream keyboard well thought out:

  1. Tenkeyless model with dedicated cursor and function keys. I’ve also used CM Quickfire TK and I can say from experience it is a terrible idea.
  2. ISO layout, so I get the \ and | right next to my right pinky, and an additional key next to z to get < and > in Finnish keyboard layout (it doubles as another \| in US layout)
  3. Topre switches for the ultimate typing comfort. Having used rubber dome, scissor mechanisms and Cherry MX blues, and trying out reds, blacks and browns, there really is no competitor. Buckling spring is just too heavy and noisy for my taste.

The items 1 and 2 are quite easy, but the third one essentially meant that the Topre Realforce line would be the primary candidate to fill all these needs. I’ve used The Keyboard Company before and they have a superb selection of Topre models, so I quickly zoomed in on the black 88-key model with UK layout:

Topre Realforce 88UB UK layout
Continue reading Topre Realforce 88UB – A Happier Hacking Keyboard

Mounting / Fixing Raspbian SD Card from Raspberry Pi

Shut down my Pi today and thought to make a copy of files in its SD card. This is what mount /dev/sdf2 /mnt had to say:

mount: wrong fs type, bad option, bad superblock on /dev/sdf2,
       missing codepage or helper program, or other error
       In some cases useful info is found in syslog - try
       dmesg | tail  or so

Great. After trying parted and fsck, it became apparent that for some reason, the root partition is marked as being 1 block longer than the physical card. Must be a bug with Raspbian partition expansion or something.

Thankfully, I found this gold nugget which suggested using resize2fs to fix it. Turns out I had to run e2fsck first (and say “y” a couple of times):

sudo e2fsck /dev/sdf2
sudo resize2fs /dev/sdf2
mount /dev/sdf2 /mnt

Voilá! Fully functioning filesystem again.

Raspberry Pi GPIO Benchmark Updated!

main2015

The new Raspberry Pi model 2 is out and the Pi world seems more popular than ever. My 2012 benchmark of different RaspPi GPIO access methods has been getting a lot of hits, so I thought to revisit it, and have now updated all the benchmarks with latest versions of firmware and GPIO libraries. I’ve also upgraded my oscilloscope to PicoScope 5444B, so the scope bandwith limitations I had earlier are now gone. :)

Because the benchmark has been linked from many other sites, I’ve just updated the old post to keep links pointing to right places.

Read the updated Raspberry Pi GPIO Speed Benchmark!