Evasion Attacks on LLMs – Countermeasures in Practice

Evasion attacks represent a broad attack vector for generative AI and especially language models. Parameters such as the targeted attack objective, the attack mechanism, the presence and type of obfuscation, entry points into the LLM system, the attacked system component, and the attacker’s position contribute to a high degree of flexibility in developing such attacks. This publication therefore focuses on effective and practical countermeasures, which are integrated into the LLM system alongside the use of design patterns to promote a robust architecture.

The findings presented in this publication have been compiled from public sources. The added value lies particularly in the systematic approach to countermeasures, which have been selected, combined, distinguished, and clearly presented from a wide range of documents. It was also verified that the described countermeasures are already present in existing tools or software libraries, thereby demonstrating their practical relevance. To further simplify application, the publication includes a checklist that guides the target audience step-by-step through the topic and contains basic countermeasures that significantly enhance the security of LLM systems. Developers and IT security officers in companies and public authorities are directly supported in the development of LLM systems.

The developed systematization of countermeasures does not claim to be exhaustive, as the topic is complex,extensive, and subject to constantly high dynamics. Moreover, the practical value in application strongly depends on the specific individual case. Technical, human, and resource limitations all play a role here. It can be expected that evasion attacks will evolve just as dynamically as LLMs and their overarching systems. Therefore, continuous analysis of the topic is essential. Those responsible should go beyond this current publication to keep up with the state of the art by conducting their own research using appropriate sources, and use this knowledge as a foundation for risk assessment.

Read more