After a bit of AI hiatus, I noticed that llama 3.0 models were released and wanted to try the models. Sure enough, after a week the weights we re available at the official site. However, the Docker image hasn't been used in a while and I wanted to upgrade it without losing the models.

There was almost no information on this available online yet, and even the ollama docker documentation is quite non-existent — maybe for seasoned Docker users it is obvious what needs to be done? But not for me, so let's see if I can manage it.

Upgrading the docker image

First, let's just upgrade the ollama/ollama image:

$ sudo docker pull ollama/ollama

This is nice, but the currently running container is still the old one. Let's stop it:

$ sudo docker stop ollama

Checking the location of the files

I remember I set a custom directory to store the models. Let's check where it is:

$ sudo docker inspect ollama | grep -i volume
            "VolumeDriver": "",
            "VolumesFrom": null,
                "Type": "volume",
                "Source": "/mnt/scratch/docker/volumes/ollama/_data",
            "Volumes": null,

As can be seen, the models are stored in /mnt/scratch/docker/volumes/ollama/_data. Let's make a hard-linked copy of the files into another folder, to make sure we don't lose them:

$ sudo bash
$ cd /mnt/scratch
$ cp -al docker/volumes/ollama/_data ollama_backup

Now that we have the models backed up, let's remove the old container:

$ sudo docker rm ollama

And then recreate the container with the new image:

$ sudo docker run -d --gpus=all -v ollama:/root/.ollama -p 11434:11434 --name ollama ollama/ollama

Let's check out if it was able to "resume" from the previous docker ollama volume:

$ sudo docker exec -it ollama bash
root@6926fda0d22c:/# cd ~/.ollama
root@6926fda0d22c:~/.ollama# ls
history  id_ed25519  id_ed25519.pub  models
root@6926fda0d22c:~/.ollama# du -s * 
8	history
4	id_ed25519
4	id_ed25519.pub
41930268	models

Great! Looks like we still have 42 GB of models in the new container. Continuing the previous bash session, let's try out mixtral:

root@6926fda0d22c:~/.ollama# ollama run mixtral
>>> Write me a poem  
 Of course, I'd be happy to write a poem for you. Here it goes:

In the quiet of the night, beneath the starry sky,
I sit and ponder, as the world rushes by.
The moon casts down her gentle light,
And fills my heart with peace and delight.

[...]

Conclusion

I was a bit worried about container deletion and recreation losing the models, but it seems that the models are stored in a volume and are not lost when the container is deleted. This is great news, as it makes upgrading the container a breeze. Essentially it should be enough to just:

$ sudo docker pull ollama/ollama
$ sudo docker stop ollama
$ sudo docker rm ollama
$ sudo docker run -d --gpus=all -v ollama:/root/.ollama -p 11434:11434 --name ollama ollama/ollama

But if you are worried about having to redownload tens of gigs of models, make sure to back it up before deleting the container, just to be safe.