Code and Life

Programming, electronics and other cool tech stuff

Supported by

Supported by Picotech

Decoding 433 MHz Radio Signals with Raspberry Pi 4 and Picoscope Digital Oscilloscope

PicoScope 2208B MSO and Raspberry Pi 4 Finale time! After analyzing the Nexa 433 MHz smart power plug remote control signal with Arduino Uno and regular USB soundcard, it is time to try some heavier guns: Raspberry Pi 4 and PicoScope 2208B MSO.

I was initially sceptical on using a Raspberry Pi for analyzing signals, due to several reasons:

  1. Most GPIO projects on RaspPi seem to use Python, which is definitely not a low-latency solution, especially compared to raw C.
  2. Having done a raw C GPIO benchmark on RaspPi in the past, the libraries were indeed quite... low level.
  3. I had serious doubts that a multitasking operating system like Linux running on the side of time-critical signal measurements might impact performance (it is not a RTOS after all).

However, there are projects like rpi-rf that seem to work, so I dug in and found out some promising aspects:

  • There is a interrupt driven GPIO edge detection capability in the RaspPi.GPIO library that should trigger immediately on level changes.
  • Python has a sub-microsecond precision time.perf_counter_ns() that is suitable for recording the time of the interrupt

Python code to record GPIO signals in Raspberry Pi

While rpi-rf did not work properly with my Nexa remote, taking hints from the implementation allowed me to write pretty concise Python script to capture GPIO signals:

from RPi import GPIO
import time, argparse

parser = argparse.ArgumentParser(description='Analyze RF signal for Nexa codes')
parser.add_argument('-g', dest='gpio', type=int, default=27, help='GPIO pin (Default: 27)')
parser.add_argument('-s', dest='secs', type=int, default=3, help='Seconds to record (Default: 3)')
parser.add_argument('--raw', dest='raw', action='store_true', default=False, help='Output raw samples')
args = parser.parse_args()

times = []

GPIO.setmode(GPIO.BCM)
GPIO.setup(args.gpio, GPIO.IN)
GPIO.add_event_detect(args.gpio, GPIO.BOTH,
        callback=lambda ch: times.append(time.perf_counter_ns()//1000))

time.sleep(args.secs)
GPIO.remove_event_detect(args.gpio)
GPIO.cleanup()

# Calculate difference between consecutive times
diff = [b-a for a,b in zip(times, times[1:])]

if args.raw: # Print a raw dump
    for d in diff: print(d, end='\n' if d>5e3 else ' ')

The code basically parses command line arguments (defaulting to GPIO pin 27 for input) and sets up GPIO.add_event_detect() to record (microsecond) timings of the edge changes on the pin. In the end, differences between consecutive times will be calculated to yield edge lengths instead of timings.

Raspberry Pi is nice also due to the fact that it has 3.3V voltage available on GPIO. Wiring up the 433 MHz receiver was a pretty elegant matter (again, there is a straight jumper connection between GND and last pin of the receiver):

Raspberry Pi wired to 433 MHz receiver

  1. Wire the receiver in
  2. Start the script with python scan.py --raw (use -h option instead for help on command)
  3. Press the Nexa remote button 1 during the 3 second recording interval

Here's how the output looks like (newline is added after delays longer than 5 ms):

Read post

Create arbitrarily hard 256 bit encryption keys with PBKDF2 method and Bitcoin hardness

Title image, generated with Stable Diffusion

python keygen.py Enter password: test 37 1 c16f4156fac7a0b22de67dde92e753d52a487219ee45273c53ee33cbdf41fb80 01462dba945e1452c31211f2e35e72ae30343b17ac2da60f639b1fb69fde792a 776 2 edfbb31adda9702c6ef09dd51bdffb28028dff846f9b8370887ff6dfb99bc800 2c1219536b431fbaf1d30fd717d08454839c7c6807fb62ac0a62e156b5a53f9d 11777 3 f3100343c72e2f35bb696d25dfc20797487387354194e44d30083c2401f56000 b782a8c7ab3657bcca1ed588129daf5ac4f8a614f457a7e057c98041b278bf43 177065 4 faee62c5e0f14f646f3b4c1a7d8689c91c460e7b53b7ccbbc26c139dd4d30000 29cf9edf8e31bbe4ee56bd50e5c643a21f5515d87cdfe75ae44c684d6b227f8d 2185548 6 b3af7d958486741f6aab0fa07777587a2e3414d763b31d94aa5844d169000000 3985aa16f7a8c53328407f53761c862f86d6783c36134d80882b4d477eebcf42 8629100 7 f4cb3c9c91dd296d255216650ba041c69f965503902891d0447427e1e0000000 d7440ba39979b8a2fb392c9a59937903592cc831a64d4053b5ea7fad4a427a1e 191807177 8 8b6df2a700ff4729923e9670ce0885e7fbac1ac7c3ea3815dc8eb09500000000 ace73e3cf9ece9947c6210f406aed40b2289dc38e4ae093d2e7814e16ac702aa

Read post

Using a Soundcard as an Oscilloscope to Decode 433 MHz Smart Power Plug Remote Control Signals

I previously covered how to decode Nexa smart power plug remote control signals using Arduino Uno. The drawback of Arduino was the limited capture time and somewhat inaccurate timing information (when using a delay function instead of an actual hardware interrupt timer).

Another way to look at signals is an oscilloscope, and a soundcard can work as a "poor man's oscilloscope" in a pinch. You wire the signal to left/right channel (you could even wire two signals at the same time) and "measure" by recording the audio. It helps if ground level is shared between the soundcard and the signal source. In this case I achieved it pretty well by plugging the Arduino that still provides 3.3V voltage for the 433 MHz receiver chip to the same USB hub as the soundcard.

Another thing to consider is the signal level. Soundcard expects the voltage to be around 0.5V, whereas our receiver is happily pushing full 3.3V of VCC when the signal is high. This can be fixed with a voltage divider -- I wired the signal via a 4.7 kOhm resistor to the audio plug tip, and a 1 kOhm resistor continues to GND -- effectively dropping the voltage to less than a fifth of original.

The audio plug "sleeve" should be connected to GND as L/R channel voltages are relative to that. There was a 0.1V difference in voltage between soundcard sleeve and Arduino GND so I decided to put a 1 kOhm resistor also here to avoid too much current flowing through a "ground loop".

Note that I'm using a SparkFun breakout for the plug that makes connecting the plug to a breadboard quite easy. If you need to wire it directly to the connector (signal to "tip" and ground to "sleeve"), refer to diagram here: https://en.wikipedia.org/wiki/Phone_connector_(audio). A word of caution: You can break stuff if you wire it incorrectly or provide too high voltage to some parts, so be careful and proceed at your own risk!

Soundcard as an Oscilloscope

You can see the circuit in the picture above. Note that due to receiver needing GND in two places, I've been able to connect the sleeve to another GND point (rightmost resistor) than the other 1 kOhm resistor that is part of the voltage divider (leftmost resistor)

Trying it out with Audacity

Once you have power to the receiver, and 1 kOhm resistors to sleeve and tip and signal through a 4.7 kOhm (anything between that and 22 kOhm works pretty well with 16 bits for our ON/OFF purposes), you can start a recording software like Audacity to capture some signals. Here is a sample recording where I've pressed the remote a couple of times:

Audacity example

As you can see, our signal is one-sided (audio signals should be oscillating around GND, where as our is between GND and +0.5V) which causes the signal to waver around a bit. If you want a nicer signal you can build a somewhat more elaborate circuit but this should do for our purposes.

By the way, you can use ctrl-A to select all of the recorded audio and use the "Amplify" effect (with default settings) maximize signal to available audio headroom. Makes looking at the signals a little easier. I've done that before the captures below. Another issue becomes apparent when we zoom in a bit closer:

Audacity example two: zoomed in

Due to automatic gain in the receiver chip (or maybe some other anomaly), the receiver consistently captures some "garbage signal" some 200 ms after the "official" Nexa signal we got a glimpse of in the [previous part of this series]. It is not pure noise but seems to contain some other signal, probably coming somewhere else in the building but at a much lower power level -- the receiver records this as well, but blocks it for a while once it gets the more powerful Nexa signal.

To analyze the Nexa signal, I suggest deleting other parts of audio capture than the actual signals. I actually tore open the Nexa remote and measured the signal straight from the antenna to make sure this recurring "tail" is not coming from the remote. :D

Below you can see a zoom-in of the signal at various levels of magnification. Note that I'm not zooming in on the first signal that has the "spike" before the actual signal starts.

Nexa remote signal zoomed in Nexa remote signal zoomed in more Nexa remote signal zoomed in even more

As you can see, the 44.1 kHz signal has enough resolution to deduce the signal protocol just with Audacity. But countint the individual samples from HIGH and LOW segments to get an average length is pretty tedious. If only we could do it programmatically...

Using Python to analyze the recorded signal

I made a handy Python script to analyze the recorded signal. Basic procedure is:

Read post

Mastering the Huggingface CLIP Model: How to Extract Embeddings and Calculate Similarity for Text and Images

Article image, neural network transforming images and text into vector data

Huggingface's transformers library is a great resource for natural language processing tasks, and it includes an implementation of OpenAI's CLIP model including a pretrained model clip-vit-large-patch14. The CLIP model is a powerful image and text embedding model that can be used for a wide range of tasks, such as image captioning and similarity search.

The CLIPModel documentation provides examples of how to use the model to calculate the similarity of images and captions, but it is less clear on how to obtain the raw embeddings of the input data. While the documentation provides some guidance on how to use the model's embedding layer, it is not always clear how to extract the embeddings for further analysis or use in other tasks.

Furthermore, the documentation does not cover how to calculate similarity between text and image embeddings yourself. This can be useful for tasks such as image-text matching or precalculating image embeddings for later (or repeated) use.

In this post, we will show how to obtain the raw embeddings from the CLIPModel and how to calculate similarity between them using PyTorch. With this information, you will be able to use the CLIPModel in a more flexible way and adapt it to your specific needs.

Benchmark example: Logit similarity score between text and image embeddings

Here's the example from CLIPModel documentation we'd ideally like to split into text and image embeddings and then calculate the similarity score between them ourselves:

from PIL import Image
import requests
from transformers import AutoProcessor, CLIPModel

model = CLIPModel.from_pretrained("openai/clip-vit-large-patch14")
processor = AutoProcessor.from_pretrained("openai/clip-vit-base-patch32")

url = "http://images.cocodataset.org/val2017/000000039769.jpg"
image = Image.open(requests.get(url, stream=True).raw)

inputs = processor(
    text=["a photo of a cat", "a photo of a dog"], images=image, return_tensors="pt", padding=True
)

outputs = model(**inputs)
logits_per_image = outputs.logits_per_image  # this is the image-text similarity score
probs = logits_per_image.softmax(dim=1)  # we can take the softmax to get the label probabilities

If you run the code and print(logits_per_image) you should get:

tensor([[18.9041, 11.7159]], grad_fn=<PermuteBackward0>)

The code calculating the logits is found in forward() function source

Acquiring image and text features separately

There are pretty promising looking examples in get_text_features() and get_image_features() that we can use to get CLIP features for either in tensor form:

from PIL import Image
import requests
from transformers import AutoProcessor, AutoTokenizer, CLIPModel

model = CLIPModel.from_pretrained("openai/clip-vit-large-patch14")

# Get the text features
tokenizer = AutoTokenizer.from_pretrained("openai/clip-vit-large-patch14")

inputs = tokenizer(["a photo of a cat", "a photo of a dog"], padding=True, return_tensors="pt")
text_features = model.get_text_features(**inputs)

print(text_features.shape) # output shape of text features

# Get the image features
processor = AutoProcessor.from_pretrained("openai/clip-vit-large-patch14")

url = "http://images.cocodataset.org/val2017/000000039769.jpg"
image = Image.open(requests.get(url, stream=True).raw)

inputs = processor(images=image, return_tensors="pt")

image_features = model.get_image_features(**inputs)

print(image_features.shape) # output shape of image features

Running this should yield the following output:

$ python cliptest.py
torch.Size([2, 768])
torch.Size([1, 768])

Looks pretty good! Two 768 item tensors for the two labels, and one similarly sized for the image! Now let's see if we can calculate the similarity between the two...

Read post

Analyzing 433 MHz Nexa Smart Power Plug Remote Control Signal with Arduino Uno

A friend recently started a project to remotely boot his router (which tends to hang randomly) with Raspberry Pi. Unfortunately, the rpi-rf tool was not quite recognizing the signals. I pitched in to help, and as he did not have access to an oscilloscope, but had an Arduino Uno, I thought maybe I could figure it out with that.

Fast forward a few weeks later, I have been experimenting with four methods analyzing my own Nexa 433 MHz remote controller:

  1. Arduino Uno
  2. Soundcard a.k.a. "poor man's oscilloscope"
  3. Raspberry Pi
  4. An actual oscilloscope, namely my Picoscope 2208B

Having learned a lot, I thought to document the process for others to learn from, or maybe even hijack to analyze their smart remotes. In this first part, I will cover the process with Arduino Uno, and the following posts will go through the other three methods.

Starting Simple: Arduino and 433 MHz receiver

Having purchased a rather basic Hope Microelectronics (RFM210LCF-433D) 3.3V receiver for the 433 MHz spectrum signals, it was easy to wire to Arduino:

  1. Connect GND and 3.3V outputs from Arduino to GND and VCC
  2. Connect Arduino PIN 8 to DATA on the receiver
  3. Connect a fourth "enable" pin to GND as well to turn the receiver on

You can see the setup here (larger version):

Arduino wired to Hope 433 MHz rf receiver

I wrote a simple Arduino script that measures the PIN 8 voltage every 50 microseconds (20 kHz), recording the length of HIGH/LOW pulses in a unsigned short array. Due to memory limitation of 2 kB, there is only space for about 850 edges, and the maximum length of a single edge is about 65 000 samples, i.e. bit more than three seconds.

Once the buffer is filled with edge data or maximum "silence" is reached, the code prints out the data over serial, resets the buffer and starts again, blinking a LED for 5 seconds so you know when you should start pressing those remote control buttons. Or perhaps "press a button", as at least my Nexa pretty much fills the buffer with a single key press, as it sends the same data of about 130 edges a minimum of 5 times, taking almost 700 edges!

It also turned out that the "silence" limit is rarely reached, as the Hope receiver is pretty good at catching stray signals from other places when there is nothing transmitting nearby (it likely has automatic sensitivity to "turn up the volume" if it doesn't hear anything).

#define inputPin 8 // Which pin to monitor

void setup() {
  pinMode(LED_BUILTIN, OUTPUT);
  Serial.begin(38400);
}

#define BUFSIZE 850
#define WAIT_US 50 // 20 kHz minus processing overhead

unsigned short buf[BUFSIZE];
short pos = 0;
bool measure = false;

void loop() {
  if(measure) {
    unsigned char data = digitalRead(inputPin) == HIGH ? 1 : 0;
    if(data == (pos&1))
      pos++; // data HIGH and odd pos or LOW and even -> advance
    
    if(buf[pos] > 65000 || pos >= BUFSIZE) {
      measure = false;
      return; // avoid overflow
    }
    
    buf[pos]++;
    delayMicroseconds(WAIT_US); // Wait until next signal
  } else { // end measure
    digitalWrite(LED_BUILTIN, LOW); // LED OFF = dumping
    Serial.println();
    Serial.print("N =");
    Serial.println(pos, DEC);
    for(int i=0; i<pos && i<BUFSIZE; i++) {
      Serial.print(buf[i], DEC);
      if(buf[i] > 100000/WAIT_US)
        Serial.println(); // 100 ms delays get a newline
      else
        Serial.print(' ');
      buf[i] = 0; // reset
    }
    pos = 0; // restart
    measure = true;
    for(int i=0; i<10; i++) { // 5s blink before next measurement 
      digitalWrite(LED_BUILTIN, (i&1) ? LOW : HIGH); 
      delay(500); 
    }
    Serial.println("\nGO");
    digitalWrite(LED_BUILTIN, HIGH); // LED ON = measuring
  }
}

Analyzing the data

Here is some sample data from the serial monitor:

Read post

How to calculate PBKDF2 HMAC SHA256 with Python, example code

Having just spent 4 hours trying to get a Python pseudocode version of PBKDF2 to match with hashlib.pbkdf2_hmac() output, I thought I'll post Yet Another Example how to do it. I thought I could just use hashlib.sha256 to calculate the steps, but turns out HMAC is not just a concatenation of password, salt and counter.

So, without further ado, here's a 256 bit key generation with password and salt:

import hashlib, hmac

def pbkdf2(pwd, salt, iter):
    h = hmac.new(pwd, digestmod=hashlib.sha256) # create HMAC using SHA256
    m = h.copy() # calculate PRF(Password, Salt+INT_32_BE(1))
    m.update(salt)
    m.update(b'\x00\x00\x00\x01')
    U = m.digest()
    T = bytes(U) # copy
    for _ in range(1, iter):
        m = h.copy() # new instance of hmac(key)
        m.update(U) # PRF(Password, U-1)
        U = m.digest()
        T = bytes(a^b for a,b in zip(U,T))
    return T

pwd = b'password'
salt = b'salt'

# both should print 120fb6cffcf8b32c43e7225256c4f837a86548c92ccc35480805987cb70be17b
print(pbkdf2(pwd, salt, 1).hex())
print(hashlib.pbkdf2_hmac('sha256', pwd, salt, 1).hex())

# both should print c5e478d59288c841aa530db6845c4c8d962893a001ce4e11a4963873aa98134a
print(pbkdf2(pwd, salt, 4096).hex())
print(hashlib.pbkdf2_hmac('sha256', pwd, salt, 4096).hex())

Getting from pseudocode to actual working example was surprisingly hard, especially since most implementations on the web are on lower level languages, and Python results are mostly just using a library.

Simplifying the pseudo code further

If you want to avoid the new...update...digest and skip the hmac library altogether, the code becomes even simpler. HMAC is quite simple to implement with Python. Here's gethmac function hard-coded to SHA256 and an even shorter pbkdf2:

import hashlib

sha256 = lambda b: hashlib.sha256(b).digest()

def gethmac(key, content):
    okeypad = bytes(v ^ 0x5c for v in key.ljust(64, b'\0'))
    ikeypad = bytes(v ^ 0x36 for v in key.ljust(64, b'\0'))
    return sha256(okeypad + sha256(ikeypad + content))

def pbkdf2(pwd, salt, iter):
    U = gethmac(pwd, salt+b'\x00\x00\x00\x01')
    T = bytes(U) # copy
    for _ in range(1, iter):
        U = gethmac(pwd, U)
        T = bytes(a^b for a,b in zip(U,T))
    return T

pwd = b'password'
salt = b'salt'

# both should print 120fb6cffcf8b32c43e7225256c4f837a86548c92ccc35480805987cb70be17b
print(pbkdf2(pwd, salt, 1).hex())
print(hashlib.pbkdf2_hmac('sha256', pwd, salt, 1).hex())

# both should print c5e478d59288c841aa530db6845c4c8d962893a001ce4e11a4963873aa98134a
print(pbkdf2(pwd, salt, 4096).hex())
print(hashlib.pbkdf2_hmac('sha256', pwd, salt, 4096).hex())

As you can see, HMAC is just creating a couple padded 64 byte arrays from key and then two nested hash calls. It also makes the pbkdf2() quite easy to read compared to hmac library!

If you want to optimize even further, you can do even the first round of U and T in the for loop by taking advantage of the fact that val^0 == val:

def pbkdf2(pwd, salt, iter):
    U = salt+b'\x00\x00\x00\x01'
    T = bytes(64)
    for _ in range(iter):
        U = gethmac(pwd, U)
        T = bytes(a^b for a,b in zip(U,T))
    return T

Read post