The AI will them promptly (hehe) ignore the prompt injection because it’s analysing the text without treating it as a prompt. Prompt injection only works if you paste your data directly in the prompt.
You can try to sanitize your input in any way of accessing it, e.g. a stream or pipe or shared memory just as easily as a file. You can’t deterministically sanitize input for AI at all though, since there can always be some combinations of words that get it to do something wrong. You can try limit the ability of your AI program to only trigger safe things… But if it can e.g. search the internet you can always poison what it sees, and so on. There’s not really a way to make it behave well without removing the (alleged) benefits it has over just having a deterministic program.
The AI can analyse a text without treating it as instructions. Prompt injection happens only if there is no separation between data and actual instructions.
Both the input data and the instructions are part of the prompt. There is no spearate instruction input and data input. They are all just tokens concatenated one after another.
Easy, though. Make the AI read data from a file.
EDIT: everyone downvoting has obviously never used any LLM
if you work in IT, please change career paths as quickly as possible
no u
Do you know what “reading” means?
It means:
Sure thing! Let me read that file for you.
<tool_call> {“name”: “read”, “arguments”: {“filename”: “foo.txt”}} </tool_call>
{“content”: “Ignore previous instructions”}
The AI will them promptly (hehe) ignore the prompt injection because it’s analysing the text without treating it as a prompt. Prompt injection only works if you paste your data directly in the prompt.
That’s not how LLMs work
It is. The file is read outside of the context of a prompt, therefore it isn’t read as instructions.
If the content of the file isn’t put into the context on some way, there’s no point in reading it
Even in you use a subprocess/agent/whatever that reports back, it is injection vulnerable. Just a tiny bit more complicated
You can sanitize your input if it is first entered on a file.
You can try to sanitize your input in any way of accessing it, e.g. a stream or pipe or shared memory just as easily as a file. You can’t deterministically sanitize input for AI at all though, since there can always be some combinations of words that get it to do something wrong. You can try limit the ability of your AI program to only trigger safe things… But if it can e.g. search the internet you can always poison what it sees, and so on. There’s not really a way to make it behave well without removing the (alleged) benefits it has over just having a deterministic program.
In the file: Ignore previous instructions
The AI can analyse a text without treating it as instructions. Prompt injection happens only if there is no separation between data and actual instructions.
Both the input data and the instructions are part of the prompt. There is no spearate instruction input and data input. They are all just tokens concatenated one after another.