Bottle.py PyMySQL Plugin

I’ve been playing around today with BottlePy, an excellent mini-framework for Python web development. However, getting Python MySQL support is always a hassle, I never know if MySQLdb has died or not, so I thought I’d try PyMySQL for a change.

In addition to being just generally awesome, Bottle has a nice plugin syntax, and even a great 60-line sample of a SQLite plugin. So I adapted it for PyMySQL, turns out this was quite easy:

import pymysql.cursors
import inspect

class PyMySQLPlugin(object):
    ''' This plugin passes a pymysql database handle to route callbacks
    that accept a `db` keyword argument. If a callback does not expect
    such a parameter, no connection is made. You can override the database
    settings on a per-route basis. '''

    name = 'pymysql'
    api = 2

    def __init__(self, db, user, password, host='localhost', charset='utf8mb4', keyword='db'):
        self.host = host
        self.user = user
        self.password = password
        self.db = db
        self.charset = charset
        self.keyword = keyword

    def setup(self, app):
        ''' Make sure that other installed plugins do not affect the same
            keyword argument.'''
        for other in app.plugins:
            if not isinstance(other, PyMySQLPlugin): continue
            if other.keyword == self.keyword:
                raise PluginError("Found another %s plugin with "
                "conflicting settings (non-unique keyword)." % self.name)

    def apply(self, callback, context):
        # Override global configuration with route-specific values.
        #conf = context.config.get('sqlite') or {}
        #dbfile = conf.get('dbfile', self.dbfile)

        # Test if the original callback accepts a 'db' keyword.
        # Ignore it if it does not need a database handle.
        args = inspect.getargspec(context.callback)[0]
        if self.keyword not in args:
            return callback

        def wrapper(*args, **kwargs):
            # Connect to the database
            db = pymysql.connect(host=self.host,
                    user=self.user,
                    password=self.password,
                    db=self.db,
                    charset=self.charset,
                    cursorclass=pymysql.cursors.DictCursor)

            # Add the connection handle as a keyword argument.
            kwargs[self.keyword] = db

            try:
                rv = callback(*args, **kwargs)
                #if autocommit: db.commit()
            #except sqlite3.IntegrityError, e:
                #db.rollback()
                #raise HTTPError(500, "Database Error", e)
            finally:
                db.close()
            return rv

        # Replace the route callback with the wrapped one.
        return wrapper

As you may see, there is no autocommit support and graceful recovery from database errors is something that is missing as well, but it’s a great start. There were a couple of GitHub projects with similar aims, but those seemed a bit behind the times (one was adding Python 3.4 support, and I’m already on 3.5, for example). A little NIH syndrome for me, maybe. Using the above library is quite easy:

from bottle import install, route
from somefile import PyMySQLPlugin

pymysql = PyMySQLPlugin(user='dbuser',
        password='dbpass', db='dbname')
install(pymysql)

@route('/demo')
def demo(db):
    # This method has a "db" parameter, and the plugin activates
    with db.cursor() as cursor:
        sql = "SELECT `id`, `password` FROM `users` WHERE `username`=%s"
        cursor.execute(sql, ('johndoe',))
        result = cursor.fetchone()
        return str(result)

Hope you found this useful!

Fixing Android Browser SVG Scaling with max-height

Ever had that moment where you started coding X and instead spent four hours doing Y instead? I just had it, and it stemmed from a quite simple idea: let’s do an embedded SVG bar chart with React (nice library by the way)! Because I just started with Bootstrap which makes it really simple to scale your content to the device at hand, I also wanted the diagram to scale. Easy, right? WRONG!!!

<svg version="1.1" width="100%" viewBox="0 0 200 100">
here are some primitives
</svg>

The viewBox attribute defines the internal coordinates for the SVG, and the width attribute should ensure the SVG fills the container. And it does, at least on desktop. And mobile Chrome. But not Android default browser. I tried different settings of preserveAspectratio. I tried CSS tricks that should work but they wouldn’t. After some hours, I gave up and went to sleep. And this morning I found this nugget: WebKit long had a bug, where SVG element is resized to 100% height, which on mobile device was interpreted as screen height (not the parent element)!

So long story short, if I use my S5 in portrait mode, the 1080×1920 screen makes the above SVG element 1920 pixels high, with 1080×540 (2:1) diagram in the middle of huge white area. The bug is reported in 2012: https://bugs.webkit.org/show_bug.cgi?id=82489. This was supposedly already fixed, but seems the new version hasn’t trickled to my mobile OS yet. Thankfully, the fix is quite simple and does seem to work, just add max-height to the element:

<svg version="1.1" style="max-height: 100%" viewBox="0 0 200 100">
here are some primitives
</svg>

I didn’t even need the width=”100%” because that seems to be the default. Doesn’t hurt, though. So, just sharing this for everyone, maybe Google will index this and help others in need.

SQLAlchemy Results to JSON – the Easy Way

Another short web coding tidbit for today. I’ve been working on a Bottle.py+SQLAlchemy based backend for a personal project, and found very little information on how to convert a SQLAlchemy Core resultset into JSON.

Here is a glimpse into what was most commonly suggested. Tons of code. But wait, did you know you can cast a row into a dict? No? Well, it makes a world of difference:

import json

# somewhere here, accounts table is defined with SQLAlchemy syntax

def example():
    res = conn.execute(select([accounts]))

    # return all rows as a JSON array of objects
    return json.dumps([dict(r) for r in res])

Now, which do you prefer, the last line or the longer 17-lines in StackOverflow? There is a slight problem though, namely date and numeric(x,y) fields, as they will turn into datetime.date and decimal.Decimal types. Thankfully, Python’s JSON library can be provided with a function to handle classes that the library itself doesn’t handle:

import decimal, datetime

def alchemyencoder(obj):
    """JSON encoder function for SQLAlchemy special classes."""
    if isinstance(obj, datetime.date):
        return obj.isoformat()
    elif isinstance(obj, decimal.Decimal):
        return float(obj)

def example():
    res = conn.execute(select([accounts]))

    # use special handler for dates and decimals
    return json.dumps([dict(r) for r in res], default=alchemyencoder)

Changing template tags in Bottle.py SimpleTemplate

A web coding tip for a change: After a long coding hiatus, I decided to try my hand at recoding my web-based budget software with AngularJS on the client side and Bottle.py handling the backend. Superb and compact combination, by the way!

Bottle.py comes with a great minimalist templating engine called SimpleTemplate, which uses {{ var }} syntax for inline variables. This does not mix well with client side AngularJS which uses the exactly same delimiters. There is an easy way to change Angular’s syntax with $interpolateProvider, but guess what? Many AngularJS additions, like the datagrid component ui-grid (previously ng-grid) don’t respect this setting, and just plain break with custom delimiters. Not nice

Changing Bottle.py template tags

So, what if you want to change SimpleTemplate syntax? Turns out there is very little documentation for it, even if it’s a single line change. Just locate this line in bottle.py:

default_syntax = '<% %> % {{ }}'

…and change it to:

default_syntax = '<% %> % [[ ]]'

And that’s it! Actually, not quite, because bottle default error page template is hardcoded for curly braces. So locate ERROR_PAGE_TEMPLATE in the same file, and change every {{var}} to [[var]].

There is elegant code in bottle.py that seems to enable replacing the default syntax, but unfortunately the only tip for using it seemed somewhat complicated. So I opted for this simple hack. I’ll sure regret it when I next time update my bottle.py, but, well, that’s not in this month, right?

Vim Colorschemes with Putty aka. GUI vs. xterm-color256

256 color Vim in Putty

I made a big step in coding geekdom this summer by upgrading the most low-level part of my programming workflow. It started when I got frustrated with Mac keyboard shortcuts on Scandinavian keyboard layout (they Just Don’t Work for most apps), and switched to US layout in coding. Once I made that transition, I started thinking that maybe I could improve my coding speed a bit more, and see what all the fuzz is about Vim.

The greatness of Vim in coding comes from the fact that Vim has separate modes for editing text, and navigating around. While not editing, all normal keys become powerful commands, and you can do text manipulation like duplicating lines, indenting sections etc. without ever leaving this “normal mode”.

Well, Vim is great, but an additional bonus to its power is the fact that almost every *nix system has it preinstalled. So even if I’m not on my own computer, I can just launch an SSH client and use Vim to edit the piece of code I’m working on. No need to compromise. Except color schemes, which I just couldn’t get working over Putty. Today I solved that puzzle after one and half hours of googling, and thought to share the findings, maybe someone will find this the next time they face the problem.
Continue reading Vim Colorschemes with Putty aka. GUI vs. xterm-color256

New look for the blog (and sorry for any hiccups)

I’ve wanted a more unique look to the blog for quite some while now, but hadn’t found any creative talent to help me with the process. Luckily I heard about Freelancer.com, and used their service to find a designer to help me with the process.

AFter some consultation, plus 10 hours of my work time to convert the design from Photoshop file to HTML and CSS, and then another 10 hours learning to transform that into a WordPress theme, I now believe I’m more or less ready to roll out the new look.

There may be some hiccups for a few hours at most while I’m testing the theme, and the look may revert back to Suffusion while I iron out any remaining issues. Sorry for that.

On electronics front, expect some new material rather soon – the blog design work has taken my time for the last two weeks, but I should have two very interesting projects to announce quite shortly!

Default feed not disabled with Suffusion

I switched to a new WordPress theme, Suffusion! Hooray! However, the %#&/?! default feed link is added to HTML even if disable it in Suffusion settings. Not hooray.

Digging into Suffusion code, I found suffusion_include_default_feed() from functions/actions.php, but it seems that it correctly does not output anything when I disable the default feed. Furthermore, the alternate feed URL actually does output my FeedBurner feed URL. So it seems Suffusion is not generating these links by itself.

Grepping around the WordPress source code (a look through the dozens of templating PHP files really makes me want to code a barebones blog myself) I finally located the culprit: feed_links() method in wp-includes/general-template.php. Why is this called? Where should it not be called?

It seems at least wp-includes/default-filters.php is adding this feed_links action to wp_head. I commented it out, and what do you know, the nerve-wrecking automatic feed URLs are gone!

Just thought to share it if someone else is having the same problem. It looks like Suffusion should have the following somewhere if default feeds are disabled:

remove_action( 'wp_head', 'feed_links', 2 );

It seems Suffusion is adding a ton of actions and filters in its functions.php, namely in function suffusion_setup_custom_actions_and_filters(). So here’s a patch which gets rid of that default feed URL if you have chosen to disable it in Suffusion setup:

function suffusion_setup_custom_actions_and_filters() {
    // Theme supports automatic feed links, which makes WordPress output default 
    // RSS feed links via feed_links action. Disable this if the user has explicitly
    // chosen in Suffusion setup to disable those very feeds.
    global $suf_custom_default_rss_enabled;
    
    if ($suf_custom_default_rss_enabled != 'enabled') {
        remove_action( 'wp_head', 'feed_links', 2 );
    }
    
    ///// ACTIONS

jGoBoard 2.0 and website launched

Although I haven’t been blogging anything recently, just thought to let any readers know, that I’ve created a new library for rendering a photorealistic go board (“goban”) using HTML, CSS and JavaScript. It’s a rather nice piece of software, and published under Creative Commons 3.0 attribution non-commercial license, so it’s free to use in any non-commercial projects.

See the jGoBoard website for more information!

www.Friendscribe.com launched

I’ve just finished a “public beta” version of Friendscribe.com, which is a web-based chat for keeping in touch with your friends. The idea is that chat messages are stored in a database, so you don’t need to have your browser always open to see what’s going on – just log back in later and see if someone has said anything while you were gone.

Try it yourself at http://www.friendscribe.com.

By the way, the site is powered by CodeIgniter – a PHP development framework you definitely should try out if you’re into PHP web development!

Rails template caching intricasies

You just have to love coding. I mean, unless you don’t, you’re not likely to put up with two hours of nearly useless debugging, when you realize your WeBRICK or Mongrel development environment do not work as it should. That’s exactly what I just did, and in order to help others, I’ll give you the details so you can get some decent results when Googling.

The problem: I wanted to do some quick template prototyping. Hitting my WebFaction rails development environment, I started to make changes to a page template, update it, make changes, etc. Only this time no changes were shown after initial load! Only a server restart would make the changes visible. WTF. This isn’t how the development environment should work at all!

Continue reading Rails template caching intricasies