Unzip Library for C

Christmas holidays are a wonderful time to invent new projects. I decided I’d do some desktop coding for a change, and try to code an optimized image viewer for my old zipped pocket camera photos. First task of course was to read a zip file.

To my surprise, there wasn’t a “GNU standard library” available for this task like there is zlib for general compression, or libjpeg and libpng for images. Best match for my simple needs seemed to be Minizip, but at 7378 lines of code, and 2125 for just unzip.c (utilizing zlib so basically just file handling), I was not convinced, especially because I knew I had some very specific requirements to cater for (namely uncompressing all JPEGs to memory for fast rendering and thumbnail generation).

Zip File Structure – The Essentials

The ZIP file format turned out to be surprisingly simple, especially since I decided I would be sticking to bare essentials and skipping zip64 support, encryption, multifile zips, and all other compression methods than “store” (no compression) and “deflate” (easily decompressed with zlib, see below). Even with barebones setup, my zip routines would handle about 99.9 % of zips out there just fine.

Drawing on excellent ZIP format documentation from InfoZip’s latest appnote, the file structure I needed to parse seemed to have the following structure:

  • Local file header 1
  • File data 1
  • Data descriptor 1
  • Local file header N
  • File data N
  • Data descriptor N
  • … optional decryption / extra stuff …
  • Central directory
  • … zip64-specific extra stuff …
  • End of central directory record

Continue reading Unzip Library for C

PiSerial Arduino Communication Library

RGB LED demo

As a continuation to my Raspberry Pi and Arduino communication post, I thought I’d do the same but opposite way. This time, I thought it would be nice to make a proper Arduino library to make the process more streamlined.

To make communication more robust, I decided to implement a more formal communication over serial to enable applications that have some idea if a command sent to Arduino was successfully carried out or not, and also introduce basic error recovery if invalid commands or wrong number of parameters are received.

If you haven’t worked with Arduino libraries before, I suggest you to familiarize yourself with this basic tutorial or the other one at Arduino Playground. If you’re not familiar with AnalogWrite(), a quick peek to Arduino PWM Tutorial may also be in order.

Serial communication protocol

I wanted to make a simple formal protocol to send commands and hex, byte or word-sized parameters related to that command over serial line to the Arduino. For the protocol, I had these requirements:

  1. Text-based, so communication can be emulated with Arduino IDE’s “Serial Monitor” tool
  2. Size-efficient to speed up communications with slow baud rates
  3. Support for at least a few dozen commands and free number of parameters
  4. Success and error messages from Arduino
  5. Capability to return to known state after invalid commands, invalid number of arguments, communication errors or breakdowns

After some consideration, I chose to select non-overlapping sets of symbols from the ASCII character set for commands, parameters, control characters and success/error messages:

  • Small letters (a-z) for commands (26 should be enough for everyone, right?)
  • Numbers 0-9 and capital letters A-F to send parameters as hex-encoded values
  • Newline to mark end of command and parameters (either ‘\r’ + ‘\n’ or just ‘\n’)
  • Capital letters O, K, and R for success (OK) and error (RR) indications

With non-overlapping characters used for different things, detecting errors in sent commands becomes easier, as any “non-expected” character marks an error situation and as status messages OK and RR do not appear in commands, implementing two-way communication (commands to both directions) is easy later on.
Continue reading PiSerial Arduino Communication Library

Simple FAT and SD Tutorial Part 2

In the last week’s part 1 of my FAT and SD tutorial, we got as far as reading the file entries in root directory, and peeking into a file with hex editor. Now we’ll cover the file allocation table itself to enable reading longer files, and adapt the code into a small footprint FAT16 library!

File allocation table in FAT16

In the previous part, we learned that the data on a FAT-formatted disk is stored in clusters. In our test image, the cluster size was 32 sectors, i.e. 16 kiB (16 384 bytes). Let’s imagine a newly formatted disk with 100 free clusters, with the clusters numbered from 2 (the first cluster at the beginning of data area) to 101 (the very last cluster). Now let’s copy some files there:

File operation File allocation
Copy HAMLET.TXT (193 082 bytes) to disk Clusters 2-13 allocated for the file (196 608 bytes)
Copy README.TXT (353 bytes) to disk Cluster 14 allocated for the file
Create SUBDIR on disk Cluster 15 allocated for the directory file entries
Create 1.TXT in SUBDIR Cluster 16 allocated for the file, a new file entry written to SUBDIR (beginning of cluster 15)

Continue reading Simple FAT and SD Tutorial Part 2