After a bit of AI hiatus, I noticed that llama 3.0 models were released and wanted to try the models. Sure enough, after a week the weights we re available at the official site. However, the Docker image hasn't been used in a while and I wanted to upgrade it without losing the models.

There was almost no information on this available online yet, and even the ollama docker documentation is quite non-existent — maybe for seasoned Docker users it is obvious what needs to be done? But not for me, so let's see if I can manage it.

Upgrading the docker image

First, let's just upgrade the ollama/ollama image:

$ sudo docker pull ollama/ollama

This is nice, but the currently running container is still the old one. Let's stop it:

$ sudo docker stop ollama

Checking the location of the files

I remember I set a custom directory to store the models. Let's check where it is:

$ sudo docker inspect ollama | grep -i volume
            "VolumeDriver": "",
            "VolumesFrom": null,
                "Type": "volume",
                "Source": "/mnt/scratch/docker/volumes/ollama/_data",
            "Volumes": null,

As can be seen, the models are stored in /mnt/scratch/docker/volumes/ollama/_data. Let's make a hard-linked copy of the files into another folder, to make sure we don't lose them:

$ sudo bash
$ cd /mnt/scratch
$ cp -al docker/volumes/ollama/_data ollama_backup

Now that we have the models backed up, let's remove the old container:

Read post

Having just spent 4 hours trying to get a Python pseudocode version of PBKDF2 to match with hashlib.pbkdf2_hmac() output, I thought I'll post Yet Another Example how to do it. I thought I could just use hashlib.sha256 to calculate the steps, but turns out HMAC is not just a concatenation of password, salt and counter.

So, without further ado, here's a 256 bit key generation with password and salt:

import hashlib, hmac

def pbkdf2(pwd, salt, iter):
    h = hmac.new(pwd, digestmod=hashlib.sha256) # create HMAC using SHA256
    m = h.copy() # calculate PRF(Password, Salt+INT_32_BE(1))
    m.update(salt)
    m.update(b'\x00\x00\x00\x01')
    U = m.digest()
    T = bytes(U) # copy
    for _ in range(1, iter):
        m = h.copy() # new instance of hmac(key)
        m.update(U) # PRF(Password, U-1)
        U = m.digest()
        T = bytes(a^b for a,b in zip(U,T))
    return T

pwd = b'password'
salt = b'salt'

# both should print 120fb6cffcf8b32c43e7225256c4f837a86548c92ccc35480805987cb70be17b
print(pbkdf2(pwd, salt, 1).hex())
print(hashlib.pbkdf2_hmac('sha256', pwd, salt, 1).hex())

# both should print c5e478d59288c841aa530db6845c4c8d962893a001ce4e11a4963873aa98134a
print(pbkdf2(pwd, salt, 4096).hex())
print(hashlib.pbkdf2_hmac('sha256', pwd, salt, 4096).hex())

Getting from pseudocode to actual working example was surprisingly hard, especially since most implementations on the web are on lower level languages, and Python results are mostly just using a library.

Simplifying the pseudo code further

If you want to avoid the new...update...digest and skip the hmac library altogether, the code becomes even simpler. HMAC is quite simple to implement with Python. Here's gethmac function hard-coded to SHA256 and an even shorter pbkdf2:

Read post

Continuing the awesome and not so unique stream of ideas on what to do with ChatGPT, here's a bit modified take to my previous post on self-running ChatGPT generated Python code. This time, let's do a shell script that takes a description of what you want as a shell command, and returns just that command. Here's how it will work:

$ shai find latest 3 files
46 total tokens
ls -lt | head -n 3
$ ls -lt | head -n 3

total 1233
-rwxrwxrwx 1 root root  5505 huhti   4  2023 python-chatgpt-ai.md
-rwxrwxrwx 1 root root 10416 maalis 26  2023 golang-sveltekit-chatgpt.md

As seen, this time I'm not running the command automatically, but just returning the command. This is a bit safer, as you can inspect the command before running it. The script is quite simple:

#!/usr/bin/env python

import sys, os
from openai import OpenAI
from configparser import ConfigParser

# Read the config file openai.ini from same directory as this script
path = os.path.dirname(os.path.realpath(__file__))
config = ConfigParser()
config.read(path + '/openai.ini')

client = OpenAI(api_key=config['openai']['api_key'])
prompt = ' '.join(sys.argv[1:])
role = ('You are Ubuntu linux shell helper. Given a question, '
        'answer with just a shell command, nothing else.')

chat_completion = client.chat.completions.create(
    messages=[ { "role": "system", "content": role },
              { "role": "user", "content": prompt } ],
    model = config['shai']['model']
)

print(chat_completion.usage.total_tokens, 'tokens:')
print(chat_completion.choices[0].message.content)

I decided GPT 3.5 Turbo is a good model for this, as it should be able to handle shell commands quite well. You also need to have a openai.ini file in the same directory as the script, with the following content:

[openai]
api_key = sk-xxx

[shai]
model = gpt-3.5-turbo

To use it, just install the OpenAI Python package with pip install openai, and then you can use the script like this:

$ chmod +x shai
$ ./shai find latest 3 files

And putting the script in your path, you can use it like any other shell command. Enjoy!

Read post

In recent years, the use of graphics processing units (GPUs) has led to the adoption of methods like PBKDF2 (Password-Based Key Derivation Function 2) for secure password storage. PBKDF2 is a key derivation function that is designed to be computationally expensive in order to slow down dictionary attacks and other brute force attacks on passwords. With the increase in processing power that GPUs provide, PBKDF2 has become a popular choice for password storage.

As the development of processing power continues to advance, it has become necessary to increase the number of iterations used in PBKDF2 in order to maintain a high level of security. With more iterations, it becomes even more difficult for an attacker to crack a password using brute force methods.

Recently, I had an idea. What if it were possible to run PBKDF2 arbitrarily long and print out points that match certain criteria? This could potentially provide an even higher level of security for password storage, as the number of iterations could be increased to levels that would make brute force attacks infeasible. It's an idea worth exploring and I'm excited to see what the future holds for PBKDF2 and other password security measures.

Bitcoin difficulty

One of the key features of the Bitcoin network is its use of difficulty to scale the hardness of block signing based on the number of computers that are currently mining. In other words, as more computers join the network and begin trying to solve the cryptographic puzzles required to add new blocks to the blockchain, the difficulty of these puzzles increases in order to maintain a consistent rate of block creation. This ensures that the network remains secure and resistant to attacks, even as the number of miners grows over time.

The basic idea behind this technique is fairly simple: by requiring that a certain number of zeros be added to the block hash, the complexity of the puzzle increases in powers of two. Every hash is essentially random, and modifying the hashed data by the tiniest bit results in a new hash. Every other hash ends in zero, and every other in one. With two zero bits, it's every 4th. To zero a full byte (8 bits) you already need 256 (2^8) tries. With three bytes, it's already close to 17 million.

Printing out PBKDF2 steps at deterministic points

Combining the two ideas is one way to deterministically create encryption keys of increasing difficulty:

Read post

I previously covered how to run your own ChatGPT script with Python and Golang. But what if you want to create a script that automatically runs the ChatGPT generated code? That is what I will cover in this post. The idea is really simple:

1. Create a script that asks for user input
2. Pass the input to ChatGPT
3. Run the ChatGPT generated code

NOTE: Please read the caveats at the end of the post before using this!

Calling The OpenAI API

First step is simple, just call the OpenAI API with the user input. If "-v" is given as the first argument, print the raw answer and usage statistics as well.

#!/usr/bin/python3

import openai
import sys

openai.api_key = 'sk-xxx'

verbose = sys.argv[1] == '-v'
prompt = ' '.join(sys.argv[2 if verbose else 1:])

resp = openai.ChatCompletion.create(
  model="gpt-3.5-turbo",
  messages=[
        {"role": "system", "content": "You are Python code generator. Answer with just the Python code."},
        {"role": "user", "content": prompt},
    ]
)

data = resp['choices'][0]['message']['content']

if verbose: print(data, 'Usage was:', resp['usage'], sep='\n')

Parsing the Code From Response

A rather simplistic implementation looks for set of start tokens and end tokens and returns the code between them. This is not perfect, but it works as long as the ChatGPT response does not contain multiple code blocks.

# Extract lines of code from the response tagged with ``` or ```python
lines = []
ok_start = ['```', '```python', '```py', '```python3', '```py3']
started = False
for line in data.splitlines():
    if not started:
        if line.strip() in ok_start:
            started = True
    else:
        if line.strip() == '```': break
        lines.append(line)

Running The Code

The code is then written to a temporary file and executed. After running, the file is deleted.

import os
import tempfile

# Create a temporary file
with tempfile.NamedTemporaryFile(mode='w', delete=False) as tf:
    tf.write("#!/usr/bin/env python3\n")

    # Write lines to the file
    for line in lines:
        tf.write(line + '\n')

# Make the file executable
os.chmod(tf.name, 0o755)

# Run the file with python3
os.system(tf.name)

# Delete the file
os.unlink(tf.name)

Trying It Out

The whole script is available here for your convenience. If you are on Linux, you can chmod +x the script and run it directly. Otherwise, you can run it with python3 pyai.py:

$ pyai -v simple hello world app
print("Hello, world!")
Usage was:
{
  "completion_tokens": 6,
  "prompt_tokens": 30,
  "total_tokens": 36
}
Hello, world!

Omitting the -v argument will only run the generated code:

$ pyai print out 10 first primes
2
3
5
7
11
13
17
19
23
29

$ pyai calculate sha1 checksum of file pyai
SHA1 checksum of pyai: db5ab334ca2af7508460d5ade8dac7fedd48bcf0
$ sha1sum pyai
db5ab334ca2af7508460d5ade8dac7fedd48bcf0  pyai

Python has pretty powerful libraries, so you can access the web as well:

$ pyai print out longest h1 tag content text from https://codeandlife.com
Using a Soundcard as an Oscilloscope to Decode 433 MHz Smart Power Plug Remote Control Signals
$ # If you want to know what the code (and usage) was, use -v in the beginning
$ pyai -v print out longest h1 tag content text from https://codeandlife.com
```python
import requests
from bs4 import BeautifulSoup

url = "https://codeandlife.com"
response = requests.get(url)
soup = BeautifulSoup(response.content, "html.parser")

h1_list = soup.find_all("h1")
longest_h1 = max(h1_list, key=lambda x: len(x.text))

print(longest_h1.text.strip())
```
Usage was:
{
  "completion_tokens": 76,
  "prompt_tokens": 41,
  "total_tokens": 117
}
Using a Soundcard as an Oscilloscope to Decode 433 MHz Smart Power Plug Remote Control Signals

Yes you can read files too:

$ pyai print out contents of /etc/passwd
# And the contents of /etc/passwd come here :D

$ pyai read /etc/passwd, split by colon and print the largest 3rd entry value
65534

Conclusions and Caveats

While the code is simple and has many limitations, it is staggeringly powerful. You can do pretty much anything you can do with Python with this. But the downside is it's like the "I'm feeling lucky" version of scripting. You don't know what the code is going to do. You can't even see the code before running it. So, use this with caution! Potential drawbacks may include, but are not limited to:

1. Racking up your OpenAI API usage and costing you money
2. ChatGPT getting it wrong and the code not working
3. Code working but doing something you didn't expect
4. Overwriting (important) files
5. Getting really malicious code that does something bad

Currently code injection to LLMs is pretty hard, but it is not unthinkable that someone will find a way to do it. So, please be careful when using this. I am not responsible for any damage caused by this script!

Read post

OpenAI came out with ChatGPT, and wow, that is quite something. What is also remarkable is the load the ChatGPT client is under, and how often it is "experiencing high demand". Or just requires you to prove you are human and log in again.

You can get ChatGPT Plus for $20 a month, but hey, you can also get chat experience for $0.002 per 1000 tokens. To hit that monthly fee, you need to use 10 M tokens, which is not that far from 10 M words. That is pretty heavy use...

Using OpenAI ChatGPT (gpt-3.5-turbo) through Python API

To use the ChatGPT API, at its simplest form with Python3 you just pip install openai and create a short script:

#!/usr/bin/python3

import openai
import sys

openai.api_key = 'sk-yourkeyhere'

if len(sys.argv) < 2:
    prompt = input('Prompt: ')
else:
    prompt = ' '.join(sys.argv[1:])

resp = openai.ChatCompletion.create(
  model="gpt-3.5-turbo",
  messages=[
        {"role": "system", "content": "You are a programming expert giving advice to a colleague."},
        {"role": "user", "content": prompt}
    ]
)

print(resp['choices'][0]['message']['content'])

print('Usage was', resp['usage'])

You need to create credentials at OpenAI platform, enter your credit card and set a warning and hard treshold for monthly billing (I set mine to $4 and $10, respectively). But after filling your API key to the script, you can just run it:

$ python3 chat.py What is the capital of Alaska
The capital of Alaska is Juneau. However, I believe you were looking for programming advice. What specifically are you working on and what kind of advice are you seeking?
Usage was {
  "completion_tokens": 34,
  "prompt_tokens": 30,
  "total_tokens": 64
}

Now that is pretty nice, but we can do better!

Golang client with SvelteKit frontend

In my previous Golang+SvelteKit GUI post I explored how to create a Go application acting as a web server and making a user interface with SvelteKit:

1. Golang has high performance and excellent set of libraries to accomplish many tasks
2. Cross-platform support out of the box with compact executables
3. SvelteKit is fast to develop as a frontend, requiring very low amount of code for rich interactive UIs

OpenAI does not produce it's own Go library, but that API as well documented and shabaronov has made an excellent Golang OpenAI API library that makes calling the API simple. It even supports GPT4, so if you have access to that, you can create a GPT4 chat client as well.

Without further ado, here's the Github repository for my GoChatGPT client. You can basically git clone://github.com/jokkebk/gochatgpt and follow the instructions in README.md to get it running, it's about 4 commands all in all.

Let's look a bit what the code does!

Golang ChatGPT Backend

Main method of the backend is nothing too complex:

1. Serve the SvelteKit GUI from static folder (including index.html when user requests /).
2. Have a chat endpoint at /chat that takes a JSON object with chat messages and passes it to OpenAI API.
3. Return the OpenAI [ChatGPT] response as a string to calling client.

func main() {
	// Get the API key from the environment variable OPENAI_API_KEY
	apiKey := os.Getenv("OPENAI_API_KEY")
	client := openai.NewClient(apiKey)

	http.Handle("/", http.FileServer(http.Dir("static")))
	http.HandleFunc("/chat", func(w http.ResponseWriter, r *http.Request) {
		var messages []openai.ChatCompletionMessage
		err := json.NewDecoder(r.Body).Decode(&messages)
		if err != nil {
			http.Error(w, err.Error(), http.StatusBadRequest)
			return
		}

		ans, shouldReturn := chatGPT(client, messages)
		if shouldReturn {
			http.Error(w, "ChatGPT error", http.StatusBadRequest)
			return
		}
		// Serialize the response as JSON and write it back to the client
		w.Header().Set("Content-Type", "text/plain")
		w.Write([]byte(ans))
	})

	address := "localhost:1234"
	log.Printf("Starting server, go to http://%s/ to try it out!", address)
	http.ListenAndServe(address, nil)
}

With the rather straightforward OpenAI, the chatGPT() function is nothing to write home about either – we get the chat messages, put them into ChatCompletion object, pass to API and (hopefully) return the top answer (or an empty error if it failed):

Read post