When attempting to install ROCm on Fedora 42 Kinoite (I’ll be referring to this as F42K from now on), I encountered several issues:

1. The installation guide for ROCm does not include a Fedora version.
2. Attempting to install the RHEL version leads to an error message stating it does not support the current distribution.
3. Fedora packages does have a rocm meta package, however, it is currently only supported on Rawhide or EPEL 10.1 (which isn’t recommended on Fedora systems).
4. Rebasing from F42K to Fedora Kinoite Rawhide is not feasible due to some layered packages having dependency issues due to missing packages in the Rawhide package repository.

Therefore, we need a workaround for installing ROCm: containers!

Method 1: Using Ollama with Alpaca

For users of Fedora Immutable, this method will come most naturally; we’re going to use Toolbx (which is preinstalled on Fedora Immutable variants) to get ROCm installed.

Installing ROCm in a Toolbx container can be done as follows:

1. Create your toolbox with toolbox create <name>
2. Enter your toolbox with toolbox enter <name>
3. Install the RHEL version of the AMDGPU installer within the toolbox with sudo dnf install https://repo.radeon.com/amdgpu-install/6.4.1/rhel/9.6/amdgpu-install-6.4.60401-1.el9.noarch.rpm (or follow AMD’s official instructions here)
4. Run amdgpu-install --no-dkms --usecase=rocm -y

Now, installing Ollama can be done as follows:

1. Either download the latest Ollama version from here or use curl -L https://ollama.com/download/ollama-linux-amd64.tgz -o ollama-linux-amd64.tgz
2. Extract the archive somewhere.
3. Add the bin directory inside into your PATH.

Running Ollama can now be done with ollama serve within your toolbox. It should properly detect and use AMD GPUs.

For the frontend, I like to use Alpaca with this method, as the user interface is pleasant to use. Within Alpaca, you just need to go to the “Manage Instances” page and add Ollama as an instance. Select the one that is just “Ollama”, not “Ollama (Managed)”. The default settings should work just fine.

However, there are some downsides to using Alpaca, which is that thinking models generally don’t seem to actually think (only Phi 4 Reasoning seemed to actually think from my testing). Additionally, web search seems to be broken as it does so with solely DuckDuckGo, which constantly gives me a rate limit error when I attempt to use it.

Well, what if I want a slightly more versatile user interface and a more seamless setup?

Method 2: Using Ollama and Open WebUI (with Podman Compose)

There exists another very popular user interface for running LLMs known as Open WebUI. However, unlike Alpaca, it requires a server to be run, usually in the form of a Podman/Docker container. For this method, I’ll be using Podman as it’s preinstalled with F42K, but using Docker with the same approach should be doable.

It should also be noted that if we use Podman directly, we can utilize the ollama:rocm image, allowing us to skip the steps needed to install ROCm ourselves.

To make things even simpler to set up, I’ll be using Podman Compose, which allows the creation of containers using a YAML file, making things more deterministic.

My compose.yml file is as follows:

version: "3.8"

services:
  ollama:
    image: docker.io/ollama/ollama:rocm
    container_name: ollama
    restart: unless-stopped
    volumes:
      - ollama:/root/.ollama:Z
    networks:
      - common
    devices:
      - /dev/kfd:/dev/kfd
      - /dev/dri:/dev/dri
    security_opt:
      - label=type:container_runtime_t
            
  openwebui:
    image: ghcr.io/open-webui/open-webui:main
    container_name: openwebui
    restart: unless-stopped
    ports:
      - 3000:8080
    depends_on:
      - ollama
    volumes:
      - openwebui_data:/app/backend/data:Z
    networks:
      - common
    environment:
      - OLLAMA_BASE_URL=http://ollama:11434

networks:
  common:
volumes:
  ollama:
  openwebui_data:

(Special thanks to this comment for solving the issue regarding device access.)

To use it:

1. Put the compose.yml file somewhere.
2. In the same directory, run podman compose up -d
3. If prompted to select a repository to download the images from, always select the docker one.
4. Now, if you go to localhost:3000, you should be able to access Open WebUI.
5. Create an admin account with an email address and password.
6. To add a model, simply go to the model selection area of the chat (should be on the top left), and in the “Search a model” field, type a model name from Ollama’s models. An option showing “Pull <model> from Ollama.com” should appear, and if you gave a valid model name, it should download it upon clicking.

That should be all that’s needed to get up and running with a model.

Something to note is that you can allow other devices you own to access Open WebUI by using Tailscale. However, one issue with this is that you are required to spin up the compose file every time on your computer before being able to access it from your other devices.

What if we want to start the containers at boot? Doing so with Podman Compose is quite difficult, but there does exist an alternative that integrates more closely with systemd, enabling startup at boot: Podman Quadlets.

Method 3: Starting Ollama and Open WebUI at Boot (with Podman Quadlets)

Podman Quadlets is like Podman Compose but managed by systemd. This allows our containers to be linked to systemd services, allowing them to be run at login time/boot (We’ll get back to this later).

First, create the following files within the ~/.config/containers/systemd/ directory.

1. ollama-webui.pod

[Pod]
PodName=ollama-webui
PublishPort=3000:8080

[Install]
WantedBy=multi-user.target default.target

2. ollama-webui-ollama.container

[Container]
AddDevice=/dev/kfd:/dev/kfd
AddDevice=/dev/dri:/dev/dri
ContainerName=ollama
Image=docker.io/ollama/ollama:rocm
Pod=ollama-webui.pod
Volume=ollama.volume:/root/.ollama:Z
SecurityLabelType=container_runtime_t

[Service]
Restart=always

3. ollama-webui-openwebui.container

[Unit]
Requires=ollama-webui-ollama.service
After=ollama-webui-ollama.service

[Container]
ContainerName=openwebui
Environment=OLLAMA_BASE_URL=http://ollama:11434
Image=ghcr.io/open-webui/open-webui:main
Pod=ollama-webui.pod
Volume=openwebui.volume:/app/backend/data:Z

[Service]
Restart=always

4. ollama.volume

[Volume]

5. openwebui.volume

[Volume]

Your ~/.config/containers/systemd/ directory should now look like the following:

~/.config/containers/systemd
├── ollama.volume
├── ollama-webui-ollama.container
├── ollama-webui-openwebui.container
├── ollama-webui.pod
└── openwebui.volume

Next, run:

1. systemctl --user daemon-reload
2. systemctl --user start ollama-webui-pod.service

Open WebUI should now be accessible from localhost:3000. However, there is still one slight problem. The containers will only spin up at login time, not boot. To fix this, we need to run:

loginctl enable-linger <your user>

Now, Ollama and Open WebUI should launch at boot time, and combined with Tailscale, you may now use Open WebUI across devices in your Tailnet right after powering on your system.

Edit: Do note that your computer (that’s running Ollama + Open WebUI) needs to be able to have network access at boot, which may not be trivial especially when using WiFi. In my case, using an ethernet connection seems to work just fine.

So, the problem is that Alpine Linux uses the musl C library instead of glibc. This causes a compatibility problem when running Rust programs that were compiled on the main system within the Alpine Linux container. In addition, the PATH environment variable is inherited from the host system due to how Toolbx works.

Here’s the workaround I found that seems to work:

• Inside the Alpine Linux container:
  • Enable the testing repository
  • Install both starship and nushell using the apk package manager.
  • In the file ~/.cache/starship/init.nu, replace all occurrences of ^/home/<user>/.cargo/bin/starship to just starship, and make sure that the starship executable is in the system path.
• On the host system:
  • Find the cargo env file located at ~/.cargo/env
  • Edit the line with export PATH such that it appends the ~/.cargo/bin directory to the path instead of prepending it.
    • As the container will inherit the PATH from the host system, the nu and starship executables within the ~/.cargo/bin directory will be deprioritized, allowing the system packages to take priority.

However, one problem with this workaround is that when using toolbox enter <alpine linux container name> from nushell on the host system, it will try continuing to use the same nushell executable that’s currently being used by the host system within the container. To work around this, you should first create a bash session with bash, then entering your toolbx container, and then using nu to start up nushell.

P.S. Since this is me trying to backtrack what I did, I may have left out some things. If it doesn’t work out for you, please make a comment.

Attempt 1: Wordpress

Link to Wordpress.

When one usually looks to create a website as a complete newbie to the web development space, the first results usually tend to be either website services like Squarespace or Wix, or Wordpress (the open source web content management system, not the hosting service). Squarespace and Wix felt a little bit restrictive in terms of what type of content I could create with it, which made me attempt to work with Wordpress.

I eventually gave up on Wordpress due to the following reasons:

1. I spent way too much time configuring the site, since there are simply an overwhelming number of plugins and options.
2. Wordpress itself isn’t very secure, and due to most websites being built on Wordpress, there are many malicious bots out there trying to crack the admin account to gain access to the site. I ended up spending days researching and implementing all sorts of security measures such as 2FA, relocating the admin login URL, lots of hardening plugins, and even setting up the Wordpress SMTP mailing service in order to get emails whenever suspicious activity was spotted. Suffice to say, I felt like I was living in fear of my site being breached, despite there not being any content on the site.
3. When I actually got around to building the website, I discovered the default page builder offered by Wordpress to be very restrictive, and when I went to look for alternative options, I found out that some types of content blocks were simply paywalled. It then became a game of “finding the best page builder”, which eventually led to some more analysis paralysis on my end.
4. I eventually realized that Wordpress is sorely overkill for the purpose I needed it to serve. I just wanted to create some content, yet this platform is geared up for creating e-commerce websites or other fancy applications.

After going through the endless cases of analysis paralysis from all the options Wordpress had to offer, it was safe to say that my motivation was completely drained. Until…

Attempt 2: Ghost

Link to Ghost.

A couple months later, I watched a video from Ali Abdaal about how to build a website, where one of his recommendations was Ghost. At first glance, it appeared to be a more batteries-included version of Wordpress, which was great to hear as it removed all the hardships I had with Wordpress.

After setting up an instance of Ghost myself and starting to write content, I discovered a few things that really bugged me:

1. There was no way to implement 2FA, and I really did not want my admin account to be secured solely by a password.
2. The page builder felt quite weird. it was not Markdown, but more like a locked-down version of Notion’s editor. This annoyed me as it did not really offer the features I wanted for easily getting my ideas in text. There was a markdown block available, but writing in markdown in another editor just felt weird.

It didn’t help that back then, I also struggled really badly with getting my thoughts and ideas down in text, meaning that when it came to writing content, I always just blanked out.

Attempt 3: Hugo

Link to Hugo.

I caught wind of GitHub Pages from a classmate of mine, which prompted me to look into static site generators further. It is basically perfect security-wise, since static sites means that the entire site is just basic HTML with some very light JavaScript. In other words, there is no database behind the scenes that could be hacked.

Since all the content pages are written in Markdown, I found out that I could also use Obsidian to edit the files, making my writing experience significantly better.

On the hosting end, I decided to use Cloudflare Pages over GitHub Pages due to Cloudflare Pages’ more generous limits.

As for the theme, I initially tried to use the Stack theme, but I found it quite difficult to configure and understand. Later, I tried the PaperMod theme, but the complete lack of sidebars (table of contents could only be placed at the top of each page) made it feel too restrictive in my opinion. I eventually settled on a fork of the PaperMod theme, PaperModX, which checked all my boxes. To add a little personal touch, I switched up the color scheme to the Nord theme, hence my personal fork, PaperModX-nordish.

As for the comment system, I’m using Utterances, which uses GitHub issues to store the comments, allowing me to get away with not using a VPS. However, I hope to use Remark42 eventually, as it allows for anonymous comments. But until I have more reasons to get an internet-connected VPS, I will be relying on Utterances.

And that’s where we are now!