Cybersecurity and AI: Enabling Security While Managing Risk

GenAI is just the latest in a series of technologies that have promised huge boosts in productivity fueled by automation of tasks. Past attempts to fully automate complex security activities have rarely been successful and can be a wasteful distraction.

Although there are benefits to using GenAI models and third-party large language models (LLMs), there are also unique user risks that require new security practices. These fall into three categories:

Content anomaly detection

• Hallucinations or inaccurate, illegal, copyright-infringing and unwanted or unintended outputs that compromise decision making or cause brand damage

• Unacceptable or malicious use

• Unmanaged enterprise content transmitted through prompts, compromising confidential data inputs

Data protection

• Data leakage, integrity and compromised confidentiality of content and user data in hosted vendor environment

• Inability to govern privacy and data protection policies in externally hosted environments

• Difficulty conducting privacy impact assessments and complying with regional regulations due to the black-box nature of third-party models

• The fact that once raw data is put into a model, there is no easy way of removing it, short of rebuilding the model, which is not practical and extremely expensive

Ironically, the conversational interface that brought GenAI into the spotlight is the culprit behind GenAI’s heightened risk of failure for internal enterprise use cases. There are two reasons for this increased risk of failure:

Usage and ROI. ROI depends on your employees using and benefiting from the technology. There’s no guarantee that employees will use a conversational interface. Without a clear value proposition, employees will not integrate GenAI into their daily workflows. Demonstrating benefit is an uphill battle as all you can track is usage; the rest is left to the user.

Prompts and hallucinations. Different users ask the same question in different ways and may receive just as many different answers. The quality of their prompts affects the quality of the answers, which may be hallucinations. Organisations must train employees on prompting, support them when they receive errant answers and invest in prompt preprocessing and response postprocessing to mitigate these issues.

To combat these roadblocks, support cybersecurity practitioners through prebuilt prompts based on their observed activity in a specific task, instead of expecting them to stop what they are doing to ask a question.