Learn how to use Flux CD and KitOps together to create repeatable, shareable, and scalable ML deployment workflows using GitOps principles for production-ready AI/ML applications.

KitOps and Flux

Transforming ML prototypes into production-ready products can get messy. Constantly changing datasets, fiddly model parameters, scattered artifacts, and a growing web of dependencies make it challenging to move these into production with confidence. This challenge has three key components:

1. Existing packaging tools weren't designed for the large files and interconnected artifacts required by AI/ML projects.
2. Most packages lack tamper-proof guarantees—even without malicious intent, the rapid pace of data science work and AI evolution means unintentional changes can compromise the security and provenance of production-bound models.
3. Traditional deployment pipelines weren't built for this complexity, especially when teams are running quickly.

To solve these problems, you need two complementary tools:

1. KitOps for packaging and tamper-proof artifacts—this open source project makes it easy to package even complex AI/ML projects into tamper-proof OCI artifacts that are compatible with all existing tools.
2. Flux CD for deployment—it brings GitOps principles to the ML world, automating deployments, maintaining synchronization, and providing your team with a clear, versioned view of what's running where.

In this tutorial, you'll learn how to use Flux CD and KitOps together to create a workflow that's repeatable, shareable, and truly scalable—designed specifically for how ML teams work today.

Understanding FluxCD

Flux CD is a declarative GitOps continuous delivery tool for Kubernetes that automatically keeps your cluster synchronized with your Git repositories. Unlike traditional CI/CD tools, Flux follows a "pull" model where the deployment system continuously monitors Git repositories for changes and automatically applies them to your Kubernetes cluster.

While initially more complex to implement, a pull-based CD tool like Flux (which resides inside the Kubernetes cluster and pulls from Git as its source of truth) offers several advantages over the more common push-based deployment model (where an external pipeline pushes changes to various clusters):

• Security: No cluster credentials leave the cluster; no inbound access needed
• Drift Detection: Continuous reconciliation automatically corrects out-of-band changes
• Scalability: Environments self-manage deployments independently
• Environment Control: Easy to scope access and behavior per cluster via Git

Key features of Flux CD include:

• Declarative Configuration: All deployment configurations are stored in Git repositories or OCI registries
• Automated Synchronization: Flux continuously monitors repositories and automatically keeps the cluster in-sync
• Multi-Source Support: Works with Git repositories, Helm repositories, and OCI artifacts
• Security-First Approach: Uses SSH keys or tokens for secure Git access
• Rollback Capabilities: Easy rollback to previous versions through Git history
• Multi-Tenancy Support: Isolate deployments across different teams and environments

FluxCD consists of several controllers:

• Source Controller: Manages Git repositories, Helm repositories, and OCI artifacts
• Kustomize Controller: Applies Kustomize configurations
• Helm Controller: Manages Helm releases
• Notification Controller: Handles alerts and notifications

Prerequisites

Before you install Flux CD, make sure you have the following set up:

• Docker for building and running containers
• A running local Kubernetes cluster (like Minikube or Kind)
• The kubectl CLI to interact with your cluster
• Git CLI installed to interact with that repo (e.g., pushing changes)

Understanding the Architecture: KitOps + Docker + Flux CD

Before diving into this tutorial, it's important to understand the distinct responsibilities of KitOps and Docker, as they solve different aspects of the ML deployment challenge.

• KitOps handles model packaging. It bundles your trained model, code, datasets, dependencies, and documentation into a versioned, tamper-proof artifact called a ModelKit. Think of it as a specialized archive designed specifically for ML, stored in a container registry like Jozu Hub (which is also available self-hosted or on-premises) or any other OCI-compatible registry.

• Docker handles model execution. It builds a container image that knows how to serve predictions. During the Docker build process, you use kit unpack with filters to extract only what you need from the ModelKit into the runnable container.

  NOTE: If you prefer to skip manual building and unpacking, Jozu Hub provides a streamlined approach using built-in Docker support. The Hub provides ready-to-run Docker images. For example:

  docker run -it --rm jozu.ml/<user>/<model>/basic:<version>

  This command starts the container immediately.

• FluxCD orchestrates deployments using a Git repository as the single source of truth. You can store the Kitfile alongside your Kubernetes manifests and Dockerfile in the same repository. While the actual model artifacts reside in the container registry, Flux coordinates deployments entirely from Git, ensuring a consistent, auditable deployment history.

This separation of concerns provides several benefits:

• Data scientists can update models without modifying container builds
• AI/ML teams can manage ML assets independently from deployment logic
• Organizations gain better version control, traceability, and repeatability in production

Below is an architecture overview:

In more advanced setups, you can avoid rebuilding containers when models change. Containers can dynamically fetch the latest ModelKit at runtime, or your CI/CD pipeline can trigger updates based on new artifact versions. This decoupling is a key advantage of adopting artifact registries in ML pipelines.

Download the Model

For this tutorial, we'll download distilbert-base-uncased from Hugging Face using the kit import command:

kit import https://huggingface.co/distilbert/distilbert-base-uncased

Once downloaded, unpack the ModelKit into a folder called my-ml-project:

\\`shell
mkdir -p my-ml-project && kit unpack distilbert/distilbert-base-uncased:latest -d ./my-ml-project
\\`

This command extracts the model into the specified folder.

Now, navigate into the project folder

\\`shell
cd my-ml-project
\\`

Next, create a requirements.txt file in your project root folder with the following dependencies:

\\`python
# requirements.txt

transformers==4.35.2

torch

scikit-learn==1.3.0

numpy==1.24.3

uvicorn

fastapi==0.104.1

pydantic==2.5.0
\\`
Optionally, you can create a README.md file at the root folder level with project documentation, or use this Quickstart README from GitHub Gist as a template.

Now, navigate into the project folder and create two directories—code and datasets:

\\`shell
cd my-ml-project

mkdir -p {code,datasets}
\\`

The code folder will contain two files: inference.py for loading the trained model and making predictions, and train.py for training and saving the model.

First, create inference.py inside the code folder:

\\`python
from fastapi import FastAPI

from pydantic import BaseModel

from transformers import pipeline

app \= FastAPI()

classifier \= pipeline("sentiment-analysis", model="./fine_tuned_model")

class Input(BaseModel):

text: str

@app.get("/")

def health():

return {"status": "ok"}

@app.post("/predict")

def predict(input: Input):

return classifier(input.text)

if __name__ \== "__main__":

import uvicorn

uvicorn.run(app, host="0.0.0.0", port=8000)
\\`

This FastAPI application serves the model using the transformers library. It loads a fine-tuned model from ./fine_tuned_model and exposes two endpoints: a health check at / and a /predict endpoint that accepts text input and returns sentiment predictions. Input validation is handled by Pydantic, and the application runs on port 8000.

Next, create train.py in the code directory:

\\`python
from transformers import DistilBertTokenizer, DistilBertForSequenceClassification, Trainer, TrainingArguments

from sklearn.model_selection import train_test_split

import torch, json

class Dataset(torch.utils.data.Dataset):

def __init__(self, texts, labels, tokenizer):

   self.texts \= texts  

   self.labels \= labels  

   self.tokenizer \= tokenizer

def __len__(self): return len(self.texts)

def __getitem__(self, idx):

   enc \= self.tokenizer(self.texts\[idx\], padding="max\_length", truncation=True, max\_length=128, return\_tensors="pt")  

   return {  

       'input\_ids': enc\['input\_ids'\].squeeze(0),  

       'attention\_mask': enc\['attention\_mask'\].squeeze(0),  

       'labels': torch.tensor(self.labels\[idx\])  

   }

def load_data():

with open("datasets/sample_data.json") as f:

   data \= json.load(f)\['samples'\]  

texts \= [d['text'] for d in data]

labels \= [d['label'] for d in data]

return texts, labels

def train():

texts, labels \= load_data()

tokenizer \= DistilBertTokenizer.from_pretrained("./")

model \= DistilBertForSequenceClassification.from_pretrained("./", num_labels=2)

X_train, X_val, y_train, y_val \= train_test_split(texts, labels, test_size=0.2)

train_dataset \= Dataset(X_train, y_train, tokenizer)

val_dataset \= Dataset(X_val, y_val, tokenizer)

args \= TrainingArguments(

   output\_dir="./results",  

   evaluation\_strategy="epoch",  

   save\_strategy="epoch",  

   save\_total\_limit=1,             

   per\_device\_train\_batch\_size=4,  

   num\_train\_epochs=3,  

   logging\_dir="./logs",  

   logging\_steps=10,               

   load\_best\_model\_at\_end=True,  

   no\_cuda=True,                   

   report\_to="none",               

   seed=42                         

)

trainer \= Trainer(model=model, args=args, train_dataset=train_dataset, eval_dataset=val_dataset)

trainer.train()

model.save_pretrained("./fine_tuned_model")

tokenizer.save_pretrained("./fine_tuned_model")

if __name__ \== "__main__":

train()
\\`
This code sets up a training pipeline for fine-tuning a DistilBERT model for binary text classification using PyTorch and Hugging Face Transformers. After training completes, the model is saved locally to a directory named fine_tuned_model.

Finally, create sample_data.json in the datasets folder:

\\`json
{

"samples": [

{ "text": "I love this product, it's amazing!", "label": 1 },

{ "text": "This is terrible, I hate it.", "label": 0 },

{ "text": "The weather is nice today.", "label": 1 },

{ "text": "Poor quality, very disappointed.", "label": 0 }

]

}
\\`

This file contains sample datasets for training the model.

Train the Model

To train your model, first create a virtual environment.

Navigate to the root of your project.

For Windows users:

# Create the virtual environment

python -m venv venv

# Activate it (Command Prompt)

venv\Scripts\activate

# Or for PowerShell:

venv\Scripts\Activate.ps1

For Mac and Linux users:

# Create the virtual environment

python3 -m venv venv

# Activate it

source venv/bin/activate

After activating the virtual environment, install the dependencies and run the training script:

pip install -r requirements.txt

python train.py

After training completes, you should see the saved model in the fine_tuned_model directory.

Edit the Kitfile

Open the Kitfile and define all the files needed for the project. This tells KitOps what to track when packaging your ModelKit, preserving the structure and workflows of your local development environment.

For this tutorial, configure the Kitfile as follows (replace \<> with your name below):

\\`yaml
manifestVersion: v1.0.0

package:

name: distilbert-sentiment

version: 1.0.6

description: DistilBERT model for binary sentiment classification

authors: \<\<Author Name>>

model:

name: distilbert-base-uncased

framework: transformers

version: 1.0.6

path: ./fine_tuned_model

description: Fine-tuned DistilBERT model

code:

- path: ./code

description: Inference and training scripts

- path: ./requirements.txt

description: Python dependencies

datasets:

- path: ./datasets/sample_data.json

description: Sample sentiment dataset

docs:

- path: ./README.md

description: Project documentation
\\`

Package the Model with KitOps

Use the kit pack command to create the ModelKit. Ensure that the ModelKit in your local registry matches the naming structure of your remote registry:

kit pack . -t jozu.ml/your-user/distilbert-sentiment:1.0.0

Replace your-user with your Jozu Hub username.

This command scans your project folder using the Kitfile as a manifest and collects:

• The trained model from ./fine_tuned_model
• The code in ./code
• Your requirements.txt
• Any datasets or documentation files listed

It then packages everything into a versioned, tamper-proof ModelKit and tags it with the specified URI. This artifact gets stored in your OCI registry and will later be pulled into your Docker build.

After running this command, you should see a confirmation message:

Next, use the kit push command to upload the newly built ModelKit from your local repository to the Jozu Hub remote repository:

kit push jozu.ml/your-user/distilbert-sentiment:1.0.0

Once the command completes, you'll see a message displaying "[INFO] pushed" along with the newly built ModelKit digest, confirming successful upload.

After the push completes, log into Jozu Hub at jozu.ml and click "My Repositories" on the menu bar. You should see your new repository.

Create a Docker Image and Startup Script

The Dockerfile handles the runtime environment inside the container, ensuring everything is properly configured when the container is deployed, eliminating the need for manual setup.

Create a Dockerfile with the following content:

\\`shell
FROM python:3.10-slim

WORKDIR /app

# Install system dependencies

RUN apt update && apt install -y curl && rm -rf /var/lib/apt/lists/*

# Install kitops

RUN curl -sSL https://github.com/kitops-ml/kitops/releases/latest/download/kitops-linux-x86\_64.tar.gz \

| tar -xvz --directory /usr/local/bin

# Pull ModelKit at build time and extract everything

RUN kit unpack jozu.ml/oyedeletemitope76/distilbert-sentiment:1.0.0 -d /app

# Install Python dependencies at build time

RUN pip install -r requirements.txt

# Copy and prepare the startup script

COPY run_model.sh .

RUN chmod +x run_model.sh

EXPOSE 8000

# Set Python path

ENV PYTHONPATH=/app

ENTRYPOINT ["/app/run_model.sh"]
\\`

Here's what each section of the Dockerfile does:

• Base image: Starts from python:3.10-slim, a minimal Python 3.10 image to keep the container size small

• WORKDIR: Sets the working directory inside the container to /app

• System dependencies: Updates apt and installs curl, then cleans up the cache to reduce image size

• KitOps installation: Downloads and extracts the KitOps CLI tool from GitHub into /usr/local/bin, making it globally accessible

• Model unpacking: Uses KitOps to pull the prepackaged model (distilbert-sentiment:1.0.0) from jozu.ml and extracts it to /app

• Python dependencies: Installs all packages listed in the model's requirements.txt

• Startup script: Copies and makes executable the run_model.sh script

• Port exposure: Exposes port 8000 for the FastAPI application

• Python path: Sets PYTHONPATH to /app so imports work correctly

• Entrypoint: Specifies that the container should run run_model.sh when it starts

Next, create the startup script run_model.sh:

\\`shell
#!/bin/bash

set -e

echo "[+] ModelKit already unpacked during build"

echo "[+] Dependencies already installed during build"

echo "[+] Starting inference server..."

# Start the FastAPI server with uvicorn

python code/inference.py
\\`

Build and Push the Image

To create the Docker image with your model and application fully integrated, run:

docker build -t your-username/distilbert-runtime:1.0.0 .

Then push the image to Docker Hub:

docker push your-username/distilbert-runtime:1.0.0

Replace your-username with your DockerHub username.

Install Flux CD

For macOS and Linux users, install Flux by running:

curl -s https://fluxcd.io/install.sh | sudo bash

You can also install using Homebrew:

brew install fluxcd/tap/flux

For Windows, Open PowerShell as Administrator, then run:

iwr -useb https://fluxcd.io/install.ps1 | iex

For a complete list of options and detailed instructions, refer to the official guide here

Before proceeding, ensure that Flux is correctly installed and that your Kubernetes cluster meets all requirements by running the following command:

\\`
flux check --pre
\\`

You should see output indicating that all prerequisites have passed, for example:

\\`
► checking prerequisites
✔ Kubernetes 1.33.0 >=1.31.0-0
✔ prerequisites checks passed
\\`

This confirms that Flux is ready to operate on your cluster.

Bootstrap Flux in Your GitHub Repository

For this tutorial, you'll need a GitHub repository to store your Kubernetes manifests. Log into GitHub and create a new repository called ml-infra.

You'll also need to create a classic personal access token (PAT) from your GitHub account. This is required because the flux bootstrap github command interacts with the GitHub API in ways that fine-grained tokens don't yet fully support.

Make sure that your GitHub Personal Access Token (PAT) has the necessary permissions to create repositories, as Flux will automatically create a repository during the bootstrap process.

Next, run this command to bootstrap Flux:

\\`shell
flux bootstrap github \

--token=$GITHUB_TOKEN \

--owner=\<your-github-username> \

--repository=ml-infra \

--branch=master \

--path=clusters/dev
\\`

Replace the placeholders with your information:

• --token: Your classic personal access token
• --owner: Your GitHub username or organization
• --repository: The repository name where your Kubernetes configs will reside
• --branch: The branch Flux should monitor (usually main or master)
• --path: The folder in the repository where Flux will write cluster configurations

This command performs several actions:

• Installs core Flux components in your Kubernetes cluster
• Creates a deploy key in your GitHub repository for secure access
• Sets up the required directory structure
• Commits initial Flux configuration files to your repository

It effectively bootstraps a complete GitOps workflow in a single step.

Once complete, clone the repository:

git clone https://github.com/<your-username>/ml-infra.git

cd ml-infra

Next, create the following directory structure:

• Create an apps folder
• Inside apps, create a distilbert subfolder
• This folder will contain two manifest files: deployment.yaml and kustomization.yaml

Deploying with Flux CD

Create deployment.yaml inside the distilbert folder:

\\`yaml
apiVersion: apps/v1

kind: Deployment

metadata:

name: distilbert-sentiment-classifier

namespace: default

labels:

app: distilbert-sentiment-classifier

spec:

replicas: 1

selector:

matchLabels:

 app: distilbert-sentiment-classifier  

template:

metadata:

 labels:  

   app: distilbert-sentiment-classifier  

spec:

 containers:  

   \- name: model-runtime  

     image: koded001/distilbert-runtime:1.0.0  

     ports:  

       \- containerPort: 8000  

     env:  

       \- name: MODEL\_VERSION  

         value: "1.0.2"  

       \- name: JOZU\_REGISTRY  

         value: "jozu.ml/oyedeletemitope76"  

     readinessProbe:  

       httpGet:  

         path: /  

         port: 8000  

       initialDelaySeconds: 10  

       periodSeconds: 5  

     livenessProbe:  

       httpGet:  

         path: /  

         port: 8000  

       initialDelaySeconds: 30  

       periodSeconds: 10  

     resources:  

       requests:  

         cpu: "250m"  

         memory: "512Mi"  

       limits:  

         cpu: "500m"  

         memory: "1Gi"  ‘  

\\`

This deployment manifest:

• Runs the model as a container using the koded001/distilbert-runtime:1.0.0 image
• Configures environment variables
• Exposes port 8000
• Includes readiness and liveness probes to ensure reliable operation
• Defines CPU and memory resource limits

Next, create kustomization.yaml:

\\`yaml
apiVersion: kustomize.config.k8s.io/v1beta1

kind: Kustomization

resources:

- deployment.yaml
\\`

This tells Kubernetes to apply the resources defined in deployment.yaml. Kustomize helps you manage and customize Kubernetes configurations cleanly, especially across different environments.

Now, tell Flux to apply your application configuration. In the clusters/dev folder, create distilbert-app.yaml:

\\`yaml
apiVersion: kustomize.toolkit.fluxcd.io/v1

kind: Kustomization

metadata:

name: distilbert-app

namespace: flux-system

spec:

interval: 5m

path: "./flux-config/apps/distilbert"

prune: true

sourceRef:

kind: GitRepository

name: flux-system
\\`

This configuration tells Flux to automatically deploy and synchronize everything under ./apps/distilbert from your Git repository every 5 minutes.

Commit and push your changes:

\\`
git add .

git commit -m "Add distilbert deployment"

git push origin master
\\`

Flux will automatically detect these changes and deploy the API within a minute.

To monitor Flux synchronization:

flux get kustomizations -A

Verify that your pods are running in Kubernetes:

kubectl get pods

To test your sentiment analysis API locally, forward the port:

kubectl port-forward <pod-name> 8000:8000

Replace <pod-name> with your actual pod name from the previous command.

Updating Your Model Workflow with Flux CD

When you update your model (for example, retraining for better accuracy), Flux CD automatically detects and deploys the changes—no manual kubectl apply commands needed.

Step 1: Update the Container Image

After training and packaging the new model, rebuild the Docker image and update the image version in your deployment.yaml file:

\\`yaml
containers:

- name: model-runtime

image: koded001/distilbert-runtime:1.0.9  \# previously 1.0.0    

\\`

Step 2: Commit and Push the Change

Use Git to commit and push your updated deployment file:

\\`shell
git add apps/distilbert/deployment.yaml

git commit -m "Update model runtime to v1.0.9"

git push origin master
\\`

Step 3: Let Flux Handle the Deployment

Flux CD automatically detects the change, fetches the latest state from your Git repository, and applies the updated configuration to your Kubernetes cluster.

To watch Flux in action:

flux get kustomizations -A -w

To observe the pod rollout in real-time:

kubectl get pods -w

You'll see output similar to the following, showing the old pod terminating while a new one is created:

NAME	READY	STATUS	RESTARTS	AGE
distilbert-sentiment-classifier-5fc9997478	1/1	Terminating	0	30m
distilbert-sentiment-classifier-7d8b9c4a12	0/1	ContainerCreating	0	5s
distilbert-sentiment-classifier-7d8b9c4a12	1/1	Running	0	45s

Step 4: Verify the Deployment

Confirm that the new container is running:

kubectl get pods

This demonstrates the power of GitOps: once you commit changes, Flux automatically applies them to your cluster in a controlled and auditable manner—no manual commands or direct cluster interaction required.

Conclusion

The GitOps approach with Flux CD provides a robust foundation for ML operations, ensuring that your models are deployed consistently, securely, and with full traceability. When combined with KitOps for artifact management, you have a complete solution for modern ML deployment workflows.

This tutorial has shown you how to:

• Package ML models into tamper-proof artifacts with KitOps
• Build containerized model servers with Docker
• Implement GitOps-based deployments with Flux CD
• Create automated, auditable deployment pipelines for ML models

If you're interested in learning more about KitOps or connecting with our team, please join our community Discord channel.