SQLAlchemy Results to JSON – the Easy Way

Another short web coding tidbit for today. I’ve been working on a Bottle.py+SQLAlchemy based backend for a personal project, and found very little information on how to convert a SQLAlchemy Core resultset into JSON.

Here is a glimpse into what was most commonly suggested. Tons of code. But wait, did you know you can cast a row into a dict? No? Well, it makes a world of difference:

import json

# somewhere here, accounts table is defined with SQLAlchemy syntax

def example():
    res = conn.execute(select([accounts]))

    # return all rows as a JSON array of objects
    return json.dumps([dict(r) for r in res])

Now, which do you prefer, the last line or the longer 17-lines in StackOverflow? There is a slight problem though, namely date and numeric(x,y) fields, as they will turn into datetime.date and decimal.Decimal types. Thankfully, Python’s JSON library can be provided with a function to handle classes that the library itself doesn’t handle:

import decimal, datetime

def alchemyencoder(obj):
    """JSON encoder function for SQLAlchemy special classes."""
    if isinstance(obj, datetime.date):
        return obj.isoformat()
    elif isinstance(obj, decimal.Decimal):
        return float(obj)

def example():
    res = conn.execute(select([accounts]))

    # use special handler for dates and decimals
    return json.dumps([dict(r) for r in res], default=alchemyencoder)

Simple X Audio Switch (Double-Y)

I’ve had a small problem over a year: I have two headphone amplifiers, a transistor-based O2 headphone amp, and a tube-based Little Dot MKII. I also have two sets of headsets. Because switching the cables all the time is somewhat tedious, I haven’t really used the other amp and headphone set.

There are several Y switches that you can buy, but most seemed either expensive or mostly tailored towards electric guitar enthusiasts. And I would still need to combine two Y switches to toggle between the two amps and headphones. As the construction is really simple, I decided to solder my own. Here’s a short tutorial on how to make one yourself. You’ll need:

  1. 4 audio connectors (I chose 3.5mm over RCA), two to each end
  2. 2 switches with at least 3 connectors (left, right, ground) – I chose two 4 way switches
  3. A box where you can drill holes for the connectors and switches
  4. Small length of wire, some solder and soldering iron

1. The Schematic

Audio X switch schematic

The design of the switch is extremely simple. In both ends, three wires come from each of the two connectors to a switch, and the switches are connected to each other. I chose not to combine any ground lines to avoid ground loops. You need also to select a switch which breaks the connection on one side before connecting the other so the amplifiers will not feed into each other.

2. Soldering the wires

Despite the simple construction, there is a lot of soldering involved. Each of the four connectors will have ground plus left and right channels, and all have the switch on the other end. That makes 24 solder points (4x3x2), and there’s six more between the switches for a total of 30 points. The 3.5mm connectors had small “L” and “R” markings to tell which channel is left and which is right. I used different colors of wires for each type to avoid mixing L, R and ground.

solder
Continue reading

Changing template tags in Bottle.py SimpleTemplate

A web coding tip for a change: After a long coding hiatus, I decided to try my hand at recoding my web-based budget software with AngularJS on the client side and Bottle.py handling the backend. Superb and compact combination, by the way!

Bottle.py comes with a great minimalist templating engine called SimpleTemplate, which uses {{ var }} syntax for inline variables. This does not mix well with client side AngularJS which uses the exactly same delimiters. There is an easy way to change Angular’s syntax with $interpolateProvider, but guess what? Many AngularJS additions, like the datagrid component ui-grid (previously ng-grid) don’t respect this setting, and just plain break with custom delimiters. Not nice

Changing Bottle.py template tags

So, what if you want to change SimpleTemplate syntax? Turns out there is very little documentation for it, even if it’s a single line change. Just locate this line in bottle.py:

default_syntax = '<% %> % {{ }}'

…and change it to:

default_syntax = '<% %> % [[ ]]'

And that’s it! Actually, not quite, because bottle default error page template is hardcoded for curly braces. So locate ERROR_PAGE_TEMPLATE in the same file, and change every {{var}} to [[var]].

There is elegant code in bottle.py that seems to enable replacing the default syntax, but unfortunately the only tip for using it seemed somewhat complicated. So I opted for this simple hack. I’ll sure regret it when I next time update my bottle.py, but, well, that’s not in this month, right?

Giving Raspberry Pi Camera Nearsight with Reading Glasses

Raspberry Pi with Reading Glasses

Back again! The summer holidays gave me some time to write after a long hiatus, and this time it’s a Raspberry Pi related article. I’ve had the excellent opportunity to play around with a Raspberry Pi Camera Module for a few days. Or actually modules, as I got both the normal and NoIR without IR filter (more about that later) from Farnell / Element 14. They also stock an excellent selection of Pi accessories, so be sure to check those out, too.

But without further ado, let’s get onward. I’m still thinking up cool projects to do with the camera, so if you have nice ideas, please feel free to share them in the comments section!

Unboxing and First Impressions

Raspberry Pi IR camera and a normal one

The camera modules arrived in simple boxes, branded with element14 logo and URL. A nice additional touch was an included instruction sheet outlining the installation procedure, as well as a link to www.element14.com/picamera with further info.

Both the IR-filtered (the one showing normal visible light) and the NoIR (the one without the filter, and thus showing both normal light AND infrared) have exact same outward appearance. The installation was quite easy, but the flat cable offers less positioning and flexing freedom that your standard webcam – obviously the Pi camera is meant for more integrated installations.

Raspberry Pi camera electronics
Continue reading

Picoscope Beta for Linux

Picoscope for Linux

Tux logo by Larry Ewing, Simon Budig, Anja Gerwinski

I’ve been a big fan of Picotech’s USB connected PC oscilloscopes ever since I purchased my first PicoScope 2204 almost two years ago. I liked the compact form factor on my desk a lot, and the powerful Picoscope software for Windows – Picotech makes only one version of this software so you get the same functionality with a £159 ($260) 2204 scope as a £5,995 ($9,900) PicoScope 6407 user would – although of course the scope features would be wildly different.

Now Picoscope is a great piece of Windows software and as I generally use Win7 to avoid reboots every time I want to use Photoshop or play a session of Mass Effect, it’s been perfect for me. However, quite a few of electronics enthusiasts are also big advocates of open software movement, and while Picotech has had drivers and SDK for Linux for a while to implement things like my realtime composite decoder, the fact that there is no Linux version of the oscilloscope software has been unfortunate. So when I noticed in the latest Picotech newsletter that there is now a beta of Picoscope for Linux, I knew I had to take it for a spin.

Installing Picoscope on Linux Mint 15

Picotech’s installation process is built on apt packaging system, so a Debian-based Linux distro is the easiest installation target. This includes the wildly popular Ubuntu and Linux Mint distros, which means mainstream Linux users are well catered to. In Picotech forums, the beta thread had at least one user who extracted necessary stuff from the .deb packages and installed the software for Fedora, too.

In Debian-based Mint, the installation went without any hiccups just by following the instructions at Picotech’s Linux Drivers page. Note that sudo apt-get install picoscope also installs all drivers so you don’t need to install your model-specific driver separately. Essentially the installation is just:

sudo echo deb http://labs.picotech.com/debian picoscope main >> /etc/apt/sources.list
wget -O - http://labs.picotech.com/debian/dists/picoscope/Release.gpg.key | sudo apt-key add -
sudo apt-get update
sudo apt-get install picoscope

Continue reading

Unzip Library for C

Christmas holidays are a wonderful time to invent new projects. I decided I’d do some desktop coding for a change, and try to code an optimized image viewer for my old zipped pocket camera photos. First task of course was to read a zip file.

To my surprise, there wasn’t a “GNU standard library” available for this task like there is zlib for general compression, or libjpeg and libpng for images. Best match for my simple needs seemed to be Minizip, but at 7378 lines of code, and 2125 for just unzip.c (utilizing zlib so basically just file handling), I was not convinced, especially because I knew I had some very specific requirements to cater for (namely uncompressing all JPEGs to memory for fast rendering and thumbnail generation).

Zip File Structure – The Essentials

The ZIP file format turned out to be surprisingly simple, especially since I decided I would be sticking to bare essentials and skipping zip64 support, encryption, multifile zips, and all other compression methods than “store” (no compression) and “deflate” (easily decompressed with zlib, see below). Even with barebones setup, my zip routines would handle about 99.9 % of zips out there just fine.

Drawing on excellent ZIP format documentation from InfoZip’s latest appnote, the file structure I needed to parse seemed to have the following structure:

  • Local file header 1
  • File data 1
  • Data descriptor 1
  • Local file header N
  • File data N
  • Data descriptor N
  • … optional decryption / extra stuff …
  • Central directory
  • … zip64-specific extra stuff …
  • End of central directory record

Continue reading

Turning PC On with a Knock Using ATtiny45 and a Piezoelectric Sensor

PS/2 with ATtiny45

Today’s post is something I’ve prepared for a long time. Hardware-wise it’s a simple thing – ATtiny45 emulating a PS/2 device, sending a keypress when three knocks are detected in the attached piezoelectric sensor (or piezo buzzer as they are also called). But if your computer can boot on PS/2 keyboard input and you have your computer stowed somewhere hard to reach (or just want to impress your friends), it’s a pretty neat little gadget! Here’s a video of it in action:

My PC takes a few seconds to put anything on display, but if you look at the bottom right corner, you can see the blue power LEDs light up immediately after the knocks.

What You’ll Need

Components
Hardware-wise this hack is super simple. You’ll need less than $10 in parts and many probably already have these lying around:

  • ATtiny45. Actually, any ATtiny or ATmega with 4kB or more flash, A/D converter and two timers will work with small adjustments, and with -Os -DMINIMAL compiler flags also 2kB MCUs (ATtiny2313 doesn’t have a A/D but you can either work around it or use a button)
  • Piezo buzzer and 1 Mohm resistor to act as knock sensor
  • PS/2 connector, or alternatively a passive USB-PS/2 adapter (I have half a dozen from old keyboards and mice) and USB cable (like the one I used in my V-USB tutorial)
  • Breadboard and wire. Alternatively you can solder it on a simple PCB like I eventually did.
  • Optionally, a 4k7 ohm pullup resistor for RESET line, and a LED and 330 ohm resistor to indicate state

The Schematic and Breadboard Setup

Schematic

The PS/2 part as discussed in my minimal PS/2 keyboard post doesn’t require any other hardware than the ATtiny. The piezo element uses a 1 Mohm resistor like in the Arduino Knock Sensor tutorial, providing a path for voltage level to get back to zero over time. The LED is connected to PB4.

The PS/2 connector also provides power to the device. Instead of soldering a custom PS/2 connector for the project, I took a passive USB-PS/2 adapter I had lying around and used a multimeter to find out which USB pins correspond to the PS/2 ones. Not surprisingly, PS/2 GND and VCC are connected to USB GND and VCC. In my adapters, PS/2 clock was connected to D+ and data to D-. You can see the mnemonic printout I made on that one below, as well as one possible breadboard configuration.
Continue reading

HHKB Professional 2 Keyboard Review

HHKB Professional 2

The keyboard is something that I use daily, and whether I’m writing e-mails or coding, I’ll likely do several hours of typing a day. Last summer when I switched to US layout in coding and started using Vim, I started thinking that maybe I should upgrade my seven year old Logitech keyboard so something hopefully better. And when I get such a project, I did what I always do: Went totally overkill with research and ended up spending a few hundred euros once I had made up my mind on the “most optimal choice” for me. :)

Keyboards: 101

My worldview after 2000 was essentially that laptop type flat keyboards are the way of the future, and keyboard choice mainly depends on whether you buy a Logitech or Microsoft one, and do you get the top of the line model or an OEM version for 15 euros. Enter Geekhack and some interesting discussions at Stack exchange, and it quickly became apparent that there is more to it.

First choice one needs to make is the layout of the keyboard. Kinesis makes some weird looking ones that some people swear on, and there are matrix-type layouts, I decided I would continue to risk carpal tunnel syndrome with a “normal” layout for the time being, as I don’t want to optimize my brain for a keyboard type that would only be available at home.
Continue reading

Tutorial: State Machines with C Callbacks

State machine

Many electronics projects involve the device transitioning from one state to another. On a high level, it could be that your project is initially in a state where it awaits input, and once it receives it, it goes to another state where it performs a series of actions, eventually returning back to the initial state. On a lower level, many communications protocols are often simple or increasingly complex state machines. In many cases, an elegantly implemented state machine can simplify your code and make it easier to manage.

There are several methods to implement state machines programmatically starting from simple if-conditions to state variables and switch structures. In this tutorial I’ll cover a slightly more advanced method of using callbacks or “function pointers” as they are implemented in C. This has some performance benefits, makes up for some clean code, and you’ll learn a bit on the way!

State Machines with Conditionals

First, to introduce the idea of a state machine, lets take a simple example: Blinking LEDs. Imagine you want the LED to turn on for a second, and then turn off. We could do it with a simple 2-state machine:

enum states { 
  LED_ON,
  LED_OFF
};

enum states state = LED_OFF;

while(1) {
  if(state == LED_OFF) {
    led_on();
    state = LED_ON;
  } else {
    led_off();
    state = LED_OFF;
  }
  sleep(1); // sleep for a second
}

If you can understand the code above, you have pretty much grasped the fundamentals of state machines. We have some processing specific to given state, and when we want to go to another state, we use a variable (in this example it’s called state) to do that. And by the way, if you haven’t encountered enum before, you should check out Enumerated type in Wikipedia. It’s a very handful tool to add to your C coding arsenal.
Continue reading

Vim Colorschemes with Putty aka. GUI vs. xterm-color256

256 color Vim in Putty

I made a big step in coding geekdom this summer by upgrading the most low-level part of my programming workflow. It started when I got frustrated with Mac keyboard shortcuts on Scandinavian keyboard layout (they Just Don’t Work for most apps), and switched to US layout in coding. Once I made that transition, I started thinking that maybe I could improve my coding speed a bit more, and see what all the fuzz is about Vim.

The greatness of Vim in coding comes from the fact that Vim has separate modes for editing text, and navigating around. While not editing, all normal keys become powerful commands, and you can do text manipulation like duplicating lines, indenting sections etc. without ever leaving this “normal mode”.

Well, Vim is great, but an additional bonus to its power is the fact that almost every *nix system has it preinstalled. So even if I’m not on my own computer, I can just launch an SSH client and use Vim to edit the piece of code I’m working on. No need to compromise. Except color schemes, which I just couldn’t get working over Putty. Today I solved that puzzle after one and half hours of googling, and thought to share the findings, maybe someone will find this the next time they face the problem.
Continue reading

arrow Older posts