Ollama and WebUI

What it is good for? Absolutely ... something, maybe?

Seeing a bunch of stuff on Twitter about running Large Language Models locally, and I had a few hours this afternoon to dabble, so I figured hey sure let’s do it.

Found an interesting video on YouTube by Dan Vega:

Ollama is local software for Mac and Windows(!) that takes care a lot of the overhead of running models, but is strictly command line. Here's how to get it started:

ollama run llama3.1

Super easy. You type your queries much like using ChatGPT on the web or an iPhone app, except it’s in a terminal window and it spits out the answers right there, without the pretty user interface. It's all plain text.

WebUI gives a nice, pretty wrapper for that and it’s clearly copied on the font and colors of ChatGPT. Which is a bummer as I really prefer Anthropic’s Claude interface, especially their “artifacts” feature, which opens a sidebar with a mini-window with nice presentation of code and the like.

WebUI runs in Docker and since I like to have things neat I downloaded the Docker Desktop app and it’s a lovely UI for that program.

Still, per Dan Vega, you go to the WebUI docs page and run this command:

docker run -d -p 3000:8080 --add-host=host.docker.internal:host-gateway -v open-webui:/app/backend/data --name open-webui --restart always ghcr.io/open-webui/open-webui:main

Fine. But because I seem to have negative energy when it comes to code, the first time I ran WebUI it asked for a user and password (all local to the Docker image but it's set up out-of-the-box for group useage). I let Apple generate a strong password — which failed to save anywhere, so then I was locked out. MoTH#RF*$ER!!

Killed the Docker image, trashed it, trashed the local resources that I just downloaded, and then Googled to figure out how to change the user environment variable according to the WebUI docs: “If you want to disable login for a single-user setup, set WEBUI_AUTH to False. This will bypass the login page.” Basically I have to add

--env=WEBUI_AUTH=False

to the “docker run” command that downloads everything and sets up the local web page UI.

... or so I thought. Did not work at all. There’s a way to do it in the Docker Desktop app somewhere but it needs to be done before the “docker run” command sets it all up and I simply cannot figure out how to do that. For now, I’ll just create a user with a fake name and password or something like that and see if I can get Chrome to save it for me.