The arrival of generative AI has opened many exciting possibilities. From workflow optimization to customer interaction, generative AI offers all sorts of options that could transform an almost limitless number of businesses.

But if you’ve ever spent much time trying to put generative AI to work, you’ve probably realized that it’s not quite so simple – and that it has its limitations.

And as with any technological innovation, there are important security considerations you need to take into account if you’re going to keep your system and your users safe.

For one thing, generative AI applications are limited to their training data, meaning they can’t give up-to-date answers, and they often don’t properly understand the context of a question.

What’s worse, these limitations can lead AI applications to ‘hallucinate’, giving responses to questions they couldn’t possibly answer. Because the application’s knowledge base is so vast, it can be easy to take these hallucinations at face value – and that could be a dangerous mistake.

But these issues don’t have to truly restrict what you can do with generative AI applications thanks to retrieval augmented generation (RAG). RAG gives generative AI applications supplementary information, which can be up to date and include detailed information from a specific field, allowing the application to provide accurate answers and fully grasp the meaning of the users’ questions.

What exactly is RAG?

You could think of large language models as being something like a person with a massive general knowledge. They’re going to be able to answer lots of questions, but if you start asking them something specific, they’re probably going to run into trouble.

RAG is like a subject-specific textbook, on hand during an open book exam. It provides very specific, up-to-date information on a specialist area. The LLM combines this with its massive general knowledge to deliver fast, accurate, context-relevant responses.

Why do I need to worry about security?

RAG unlocks many potentially game-changing uses of generative AI.

It can be used to search area-specific databases, like legal libraries to check for precedents, or it can give patient-specific medical advice.

It can even be used to give real-time advice on writing and coding, with the user's live actions serving as a supplementary source of information.

But it’s not hard to see how using RAG in these ways could mean a lot of sensitive or personal information flowing into your generative AI application.

Generally, only public data is used by generative AI applications. This is because the LLMs that power them are trained on public data, i.e. the internet, but also because the companies behind them are increasingly wary of being accused of circumventing privacy regulations.

The additional, sometimes user-specific, information provided by a RAG system changes this. To abide by current regulations, you must tell your users what information might be collected about them while using the application, how it will be used, and provide a mechanism for them to opt-out or request data deletion.

As well as keeping users informed, you need to make sure you work with platforms that will keep their data secure and will keep you in line with the most up-to-date data governance.

How do I keep it secure?

Firstly, make sure you pick a platform that is secure and has all the correct data governance capabilities. That means all the processes, policies, measures and controls around the lifecycle of the data. The platform should have end-to-end encryption, be secure by default, and have achieved compliance at the highest levels.

You should also consider a cloud-based platform, as many of the leading cloud tech providers are also AI innovators. This will help you take advantage of new innovations as they emerge with minimal fuss.

Next, you should look for a platform that can isolate vector workload infrastructure from other database infrastructure, so that different workloads can run at the same time without competing for resources.

Finally, make sure you choose a platform that has been proven at scale. You don’t want to be a test case. Look for a platform that has a proven track record working with global, enterprise level customers.