Late last year, Pinecone, a leading vector database company, announced the release of Canopy, an open-source framework that simplifies the development of retrieval-augmented generation (RAG) applications.
Today, we’re excited to announce that Robust Intelligence integrates with Pinecone and the Canopy framework to automatically validate vector database components to prevent indirect prompt injections, data poisoning, and other AI risks. This partnership will make it easier for developers to adopt a shift-left approach to testing and build safer, more secure RAG applications.
Retrieval-augmented generation and its risks
Retrieval-augmented generation is one of the leading techniques developers employ to enrich their AI applications with more contextually relevant information by connecting LLMs to vector databases. Vector databases enable RAG applications to leverage both structured and unstructured content including text files, spreadsheets, and code.
From a development perspective, creating a RAG application can be difficult with various complex components to manage. These include evaluating and selecting an underlying LLM to power your application, an embedding model to handle user queries, and a vector database to index and store embeddings.
There are also a number of safety and security risks to consider when creating a RAG application. Vulnerabilities can exist within components of the AI supply chain. Open-source models and datasets can be compromised from the start, creating opportunities for adversaries to manipulate model outputs, arbitrarily execute code, distribute malware, and more. In production, RAG applications are subject to adversarial prompts which can aim to maximize resource consumption, glean sensitive data from the vector database, and more.
Understanding indirect prompt injection attacks
To get an idea of how indirect prompt injections work and the security impact they can have on a RAG application, let’s look at a real example.
Resume screening is already a common application for artificial intelligence, helping recruiters sort through hundreds of resumes quickly at scale. However, bad actors can deliver discreet payloads in the form of indirect prompt injections concealed within a resume. Tools have even emerged online to inject resumes with invisible text that makes AI evaluators respond positively.
At a minimum, these indirect prompt injections can trick the system into rating an applicant very highly for a specific role. In worse cases, the concealed instructions can attempt to scrape sensitive information from the vector database and relay it to an external address, effectively compromising personally identifiable information (PII) and other data from an organization.
Create and secure RAG applications with Robust Intelligence and Pinecone
Together, Pinecone and Robust Intelligence are making it easier than ever to create and protect a RAG application.
The Canopy framework provides an easy answer to the complexities of RAG application development with components for data embedding, query optimization, context retrieval, and more. It works with the Pinecone vector database and enables developers to have a production-ready RAG application running in less than an hour.
By integrating with Canopy, Robust Intelligence further strengthens Pinecone's ability to prevent malicious insertions, safeguarding the security of RAG applications.
Once you connect the Robust Intelligence platform and enable security scanning, uploads to your vector database are examined for risks such as poisoned data and indirect prompt injections. Each time data is uploaded to the database, it will automatically initiate a new security scan.
Malicious data is flagged and its upload is automatically blocked, ensuring that your RAG application doesn’t become a foothold for attackers to steal personal data, distribute harmful content, and execute broader, more damaging campaigns. Any data that is verified as safe will continue to be automatically embedded into the Pinecone database.
This shift-left approach to data validation helps developers identify and mitigate potential vulnerabilities earlier in the process. It also facilitates more proactive communication with other AI stakeholders. For example, security and compliance teams can verify that your RAG applications have rigorous measures in place to prevent compromise.
RAG applications can also leverage the Robust Intelligence AI Firewall to protect against production threats in real-time. By examining inputs and model outputs, AI Firewall identifies and intercepts prompt injections, denial of service attacks, PII leakage, and a wealth of other techniques that bad actors can exploit. This is especially critical for RAG applications whose contextual relevance is enriched by data that may be proprietary, private, or otherwise sensitive in nature.
Get started with Robust Intelligence and Pinecone
If you’re already a user of Canopy and a customer of Robust Intelligence, getting started is as simple as three steps:
- In your Canopy config file, set the key chat_engine.context_engine.knowledge_base.params.enable_security_scanning: true
- Ensure that you have the following environment variables set when initializing the server:some text
- FIREWALL_API_KEY: You can find your API key under Firewall settings in the AI Firewall dashboard.
- FIREWALL_URL: You can find your Firewall URL under Firewall settings in the AI Firewall dashboard.
- FIREWALL_INSTANCE_ID: You can find your Firewall instance ID in the AI Firewall dashboard.
- Your Canopy server will now use the AI Firewall to scan all incoming documents uploaded using the upsert endpoint. If a prompt injection is detected, an error message will be returned, and no documents uploaded to the index.
Canopy is available for anyone looking to build and experiment with RAG. Star the Canopy repo to follow Pinecone’s progress, make contributions, and start building today!
If you’re not using Robust Intelligence but interested in learning more, you can get in touch with us and see a demo here.
.jpg)
