Frequently Asked Questions: A Deep Dive into Jozu and KitOps
We had an incredible time connecting with the Kubernetes and cloud-native community at KubeCon India! The energy and enthusiasm from attendees who stopped by the Jozu booth was truly inspiring. Throughout the event, we had engaging conversations about how organizations are tackling AI/ML deployment challenges and exploring solutions for managing models at scale.
Many of you asked thoughtful questions about Jozu's architecture, deployment options, and how it fits into existing workflows. We wanted to take a moment to address some of the most common questions we received. Whether you visited our booth or couldn't make it to the event, we hope these answers provide clarity on how Jozu can help streamline your AI/ML operations.
Deployment and Infrastructure
What are the prerequisites for on-premises deployment?
For organizations looking to deploy Jozu on-premises, you'll need a few key components. First, a Kubernetes cluster where you'll deploy the Jozu Hub. Within this environment, you'll need a Postgres database to maintain Jozu Hub state, an OCI-compliant container registry that our Hub Bot can access with read/write permissions, and an OIDC authentication provider for secure access management. Additionally, you'll need access to the Kubernetes clusters where your AI models will be deployed so we can install the in-cluster caching mechanism that optimizes model distribution.
What does Jozu use for the backend and database?
Jozu leverages the infrastructure components mentioned in our prerequisites. The backend runs on Kubernetes, uses Postgres for state management, and integrates with your existing OCI-compliant container registry for artifact storage. This approach ensures Jozu fits seamlessly into your existing infrastructure without requiring specialized or proprietary systems.
How does versioning work in on-premises deployments?
Versioning in Jozu is straightforward and leverages familiar tools. Your ModelKits are stored in your existing OCI container registry, maintaining full control over your artifacts. Each ModelKit is built and versioned using our open source KitOps tools, available through both CLI and Python SDK. This means you can integrate versioning into your existing CI/CD pipelines and maintain consistency with your current versioning strategies.
What happens when Jozu upgrades? What do clients need to adapt?
We've designed upgrades to be as seamless as possible. When deployed on-premises, Jozu is distributed as a Helm chart and managed entirely by your team. Updates are made available through a separate, security-controlled repository. Upgrading is accomplished using standard Helm commands that your operations team is already familiar with. We provide detailed release notes and migration guides when necessary, ensuring you're never caught off guard by changes.
Security and Access Control
How do you handle RBAC and granular access to datasets between different users?
Role-based access control is fundamental to enterprise deployments. Jozu integrates with your existing OIDC provider, allowing you to leverage your current identity and access management policies. This extends to dataset access, where you can define granular permissions based on user roles and responsibilities.
What if datasets contain sensitive information that shouldn't be accessible to everyone?
This is a critical concern we hear frequently. Jozu's architecture allows you to implement access controls at multiple levels. You can restrict access to entire ModelKits or specific components within them. By leveraging your existing authentication provider and our built-in access control mechanisms, you can ensure sensitive data remains protected while still enabling collaboration where appropriate.
What approach does Jozu take for model scanning?
Security is paramount in AI deployments. Currently, model scanning is handled through integration with the open source ModelScan project, providing vulnerability detection and security analysis for your models. We're actively expanding our security capabilities and planning to add LLM evaluations and runtime security checks in the future, ensuring comprehensive protection throughout the model lifecycle.
What about observability, scanning, and auditing?
Observability is built into Jozu's architecture. Beyond the ModelScan integration for security scanning, Jozu provides comprehensive logging and monitoring capabilities. Audit trails track all actions within the system, from model deployments to access requests, ensuring you maintain compliance and can investigate any issues that arise.
Integration and Workflow
How should organizations add Jozu to their existing workflow?
Jozu is designed to complement, not replace, your existing tools. We ship with a built-in workflow engine based on Argo Workflows, a popular choice in the Kubernetes ecosystem. The beauty of this approach is flexibility – each workflow can be edited to match your needs, and you can create entirely new workflows at any time. This means you can start with our defaults and gradually customize them as you better understand your requirements.
Is there integration with OpenFGA?
Not at this time, but we're always evaluating new integrations based on community feedback. If OpenFGA integration is important for your use case, we'd love to hear more about your requirements.
How does Jozu handle multiple nodes and clusters deployed for different use cases?
Multi-cluster deployments are a reality for many organizations, and Jozu is built with this in mind. Jozu Hub can manage ModelKits across multiple clusters, with each cluster potentially serving different purposes or teams. The in-cluster caching mechanism can be deployed to each cluster, ensuring optimal performance regardless of where your models are running. This distributed approach maintains performance while giving you the flexibility to organize your infrastructure according to your needs.
Technical Comparisons
How does KitOps compare to Buildpacks?
This is an interesting comparison that highlights the unique value of KitOps. Buildpacks, like containers, excel at simplifying the portable and repeatable running of applications. However, they're specifically designed for runtime artifacts – things that are actively used when the application is running.
KitOps takes a different approach. It's designed to package, version, and secure all the artifacts needed to share and reproduce an AI project, not just the runtime components. This includes training data, model weights, configuration files, notebooks, and documentation. The KitOps CLI and Python SDK make it simple to generate these comprehensive packages from your code or integrate packaging into your CI/CD systems. Think of it as creating a complete, versioned snapshot of your AI project that anyone can use to understand, reproduce, or build upon your work.
Looking Forward
The questions and conversations from KubeCon India have given us valuable insights into the challenges organizations face when operationalizing AI/ML workloads. We're committed to addressing these challenges and continuing to evolve Jozu based on your feedback.
If you have additional questions or want to dive deeper into any of these topics, we encourage you to reach out to our team or explore our documentation. For those interested in trying Jozu, we offer proof-of-value engagements that let you experience the platform in your own environment with your own models.
Thank you to everyone who stopped by our booth at KubeCon India. Your enthusiasm and thoughtful questions energize our mission to make AI/ML deployment simpler, more secure, and more scalable. We look forward to continuing these conversations and seeing how you leverage Jozu and KitOps in your organizations.
Have more questions? Want to share your use case? Connect with us on GitHub or reach out to our team directly. We're always eager to learn from the community and help solve real-world AI/ML deployment challenges.