The Generative Artificial Intelligence (GenAI) innovations and advancements over the past 1.5 years have been unmatched. Gartner predicts that by 2026, more than 80% of enterprises will have deployed GenAI-enabled applications in production environments and/or used GenAI application programming interfaces or models. This is up from less than 5% in 2023.
But GenAI application development and deployment is heavily laden with security risks, and security is not keeping pace with the speed of innovation. This is part of a broader trend when it comes to AI security: 82% percent of respondents to a recent IBM Institute for Business Value study acknowledged that secure and trustworthy AI is essential to the success of their business, but 69% of those surveyed still said that innovation precedes security.
How can organizations derive value from GenAI application development and deployment without compromising on security?
Risks and Opportunities Building GenAI Applications
Organizations are able to derive the most value by customizing AI models with proprietary data; taking a generic model and using it off-the-shelf adds limited value. Gartner predicts that by 2027, more than 50% of the GenAI models that enterprises use will be specific to either an industry or business function – up from about 1% in 2023.
The design many organizations use for their GenAI applications is a Retrieval Augmented Generation (RAG) architecture. RAG architecture supplies an organization’s proprietary data to a Large Language Model (LLM), helping to train such models, as well as create prompts for the LLM to generate accurate, desired outputs. This approach enables teams to develop an application that is specific to their organization and its unique needs.
But this creates the risk of confidential data flowing out of the organization and even being used to train other models. Proprietary data is supplied in the LLM in the form of a prompt, which can then be subject to attack.
The risk of sensitive or confidential data exfiltration is top of mind when using these models, but it doesn’t have to be. Setting governance and security controls at both the ingress and egress level helps address top security issues. Fine-grain controls to regulate the data and traffic coming in from the Internet and external sources, and for the traffic leaving the GenAI application, is key. The main consideration there is that specific data does not leave the enterprise in an unregulated way.
Tension Builds Between Developers and Security Teams
Organizations looking to develop and deploy GenAI applications must empower their developers and give them the freedom to experiment. But the platform engineers and security teams tasked with avoiding and mitigating security risks want to establish as many controls as possible. This leads to tension between these two equally important groups.
Security must be wired in at every level of the GenAI development and deployment cycle to avoid potentially devastating consequences. One of the biggest challenges around security for the GenAI applications is from a governance and security guardrails standpoint. As mentioned, platform and/or security engineers must be able to set granular controls at the GenAI application level, both on the ingress and egress side, to regulate what can come in and what can go out. Without the guardrails, it’s impossible to empower and enable developers to securely experiment and innovate.
To drive mutually-beneficial relationships, security leaders should include developers in their security efforts, allowing them to participate in setting their own security controls, and providing the proper tools to accomplish their goals.
Where Does Open Source Fit In?
Open source creates further opportunities and risks. Presuming organizations have multiple GenAI applications – and each application is composed of multiple services – they are likely using a healthy dose of open source, both in their applications, and some off-the-shelf open source models.
While users can pull something off-the-shelf and leverage it, there are risks in these models, as it’s impossible to guarantee that these models have not been breached. Enforcing some level of multi-tenancy, or some level of isolation between the applications, is critical. That way, if one application has been compromised, the boundaries are in place to ensure that the rest of the applications do not get compromised. Being able to enforce those controls, to establish isolation at the application level – or even at the namespace level – becomes a must-have. These security guardrails are used as a preventative measure to ensure that if something does get compromised, the user can contain the breach with a limited blast radius.
Industry-Wide Prioritization of Security
There is an incredible amount of excitement and energy around GenAI, but the cost of innovation cannot be security. Ten years ago, we saw an implosion of SaaS companies, and we’re starting to witness something similar: thousands of GenAI companies are being established to tackle different problems and take on niche categories. But they themselves have to establish security controls and guardrails to prevent security incidents, and to protect data so that enterprises using their services don’t run into security issues. Now is the time for everyone to start prioritizing security. Prioritizing security is critical to the future success of secure GenAI application development and deployment.
Want to learn about Calico features for container networking and security for GenAI workloads? Schedule a demo.
