Over the past year, there has been a culmination of hype and excitement around Generative AI (GenAI). Most organizations initiated proof-of-concept projects for GenAI, eager to reap the technology’s benefits, which range from improved operational efficiency to cost reductions. According to recent research, 88% of organizations are in the midst of actively investigating GenAI, transcending other AI applications. However, the vast majority of organizations have yet to surpass this initial proof-of-concept stage and graduate GenAI applications into production. As we move into 2025, more organizations will begin to formalize their GenAI strategies, creating and deploying a host of new GenAI applications across their infrastructure.
Creating GenAI Applications with Kubernetes
As organizations build out GenAI applications, they will leverage many different GenAI models. To optimize, and derive the most value and accuracy from their GenAI applications, enterprises will utilize proprietary data to create these models, primarily through a Retrieval-Augmented Generation (RAG) architecture. A RAG architecture enables organizations to customize models based on company data, so that GenAI applications are personalized to an enterprise and their specific use cases. Most GenAI applications will contain proprietary company data as a result of this approach, creating many security concerns for organizations.
Consequently, some organizations will opt to deploy GenAI applications in their data center, an existing hub for sensitive enterprise data. Most organizations, however, want the flexibility to deploy GenAI applications across both cloud environments and on-premises in their data center. With flexibility at the forefront, Kubernetes is quickly becoming the de facto platform on which GenAI applications are being deployed.
Kubernetes provides organizations with the means to seamlessly deploy GenAI applications across cloud and on-premises environments, while also boasting other benefits including observability, workload scheduling, automation and networking–attributes that are advantageous to developers working to create and deploy such applications. Organizations can run Kubernetes for GenAI across various workloads including virtual machines (VMs), containers, or bare metal servers — or a mixture of all three.
Kubernetes Security Becomes Paramount
While most organizations already actively deploy and run various types of applications on Kubernetes, security continues to remain an afterthought for many. As Kubernetes becomes the orchestrator of GenAI applications, securing them is now paramount. GenAI applications, unlike any other existing applications, present increased security risk, especially when it comes to data privacy, integrity, and security. Built using sensitive data sources from inside an enterprise, once an organization deploys such applications, their attack surface increases greatly.
Given this changing dynamic, there will be a heightened focus on Kubernetes security in 2025 and beyond. There are several key areas that are vital to provide comprehensive security for GenAI applications being deployed on Kubernetes.
1. Implementing Network Security Access Controls
First and foremost, organizations will be required to implement strong network security access controls. As organizations will have multiple applications accessing multiple GenAI models and data sources, controlling this is a combinatorial problem. Network security is a critical aspect of any Kubernetes deployment, ensuring that data transmitted within clusters is protected against unauthorized access, interception, or modification. microsegmentation in particular is crucial to enhancing network security within Kubernetes environments. This technique divides networks into smaller, isolated segments, allowing for granular control over traffic flow and significantly bolsters security posture.
An organization’s network access controls must be able to support across cloud environments and data centers, in addition to supporting containers, VMs and bare metals. Network security mechanisms must also be deployed at the security edge, as much of the GenAI inference, the process of running data through a trained AI model to conduct a task, may be located at the edge closer to users.
2. Proactively Managing Vulnerabilities
Organizations must also prioritize vulnerability management. While a container image is the core building block of a Kubernetes workload, many organizations use insufficiently secure container images. Organizations must implement continuous monitoring, image scanning and policy enforcement processes to detect vulnerabilities, malware, and unsafe configurations across all Kubernetes clusters. By implementing vulnerability management practices, organizations can proactively identify and address vulnerabilities within container images before they are deployed into production.
3. Protecting Against Known and Unknown Threats
Runtime security is another crucial element to securing Kubernetes, protecting against known and zero-day attacks, whether they are network or container-based. This is crucial for GenAI applications as any breach could pose an existential threat to an organization given how much proprietary and sensitive company data resides within such applications. Organizations should invest in mechanisms that can instantly detect, block, and mitigate risks across their environment and automatically quarantine infected Kubernetes workloads the moment threats are detected.
4. Preventing & Addressing Misconfigurations
Misconfigurations are one of the most common and detrimental security risks for organizations utilizing Kubernetes. In the context of GenAI, misconfigurations can leave an organization’s private information dangerously exposed, hence the need for careful management and monitoring. This process involves continuously monitoring images, workloads, and Kubernetes infrastructure configuration against common configuration security standards and referencing CIS benchmarks when configuring Kubernetes.
5. Maintaining Observability
Finally, organizations must maintain a real-time view of traffic flows within and outside Kubernetes clusters to understand workload communications and connections, service dependencies, and policy enforcement. This will enable organizations to proactively identify and resolve security gaps and policy violations.
2025 will be the year that many organizations officially deploy GenAI applications across their infrastructure. With Kubernetes set to serve as the core platform for deploying and running these applications, there is an inherent need for organizations to step up their security in this domain. Implementing these five elements will be crucial for proactively addressing Kubernetes security risks before they can be exploited by an attacker and will also help organizations foster an enhanced Kubernetes security posture.
Want to learn about Calico features for container networking and security for GenAI workloads? Schedule a demo.
