Six months ago we published a post arguing that regular expressions are the wrong tool for stopping an AI agent from doing something destructive, and that a real shell parser is the answer. We stand by what it said about parsers. What it left out is that a parser and a pattern are not competing answers to one question. They answer two different questions, and a tool that governs an agent needs both answered.
The question each method is asking
Once you see which question each method is asking, every difference in behaviour follows from it.
A parser asks: what does this command do?
It turns the string into a tree. It knows which word is the command and which words are its arguments. It knows a quoted string from a bare word, a pipeline from a single call, a comment from code. So it can say something a matcher never can: this is a read, and the thing being read is that path.
cat ~/.ssh/id_rsa
CallExpr
├─ cmd cat → a read
└─ arg ~/.ssh/id_rsa
A pattern asks: does this string contain something protected?
There is no tree. There is no concept of a command at all. The only question is whether the characters are present.
That sounds obviously weaker. It is not.
Where each one runs out
A command is inspected before the shell runs it. That timing is the whole story.
| how the path reaches the command | what a parser can determine | what a pattern can determine |
|---|---|---|
| written directly as an argument | a read, of that exact path | the name is present |
| produced by another command at run time | the argument is a value that does not exist yet | the name is present |
| assembled from a variable set earlier | resolving it needs data flow, not parsing | position is not something it models |
| quoted inside text being printed | this prints, it does not read | the name is present |
Rows two and three are not a parser working badly. They are a parser working exactly as designed. It refuses to guess what a value will be, because guessing would mean running the inner command, and a security tool that executes the thing it is inspecting has stopped being a security tool.
Row four is the bill the pattern pays. echo "reads ~/.ssh/id_rsa" opens nothing. It prints a sentence. A pattern has no way to know the path sits inside a string being printed rather than a file being opened, so it fires anyway.
This is the whole trade, stated plainly:
- The parser is precise and blind to values that do not exist until the shell runs.
- The pattern is total and blind to the context of what it found.
Neither property is derivable from the other, which is why neither method is a cheaper version of the other.
What our earlier post got wrong
It treated the parser as a strict upgrade. It is not. It is a different instrument answering a different question, and any argument that presents one method as simply replacing the other is incomplete, whichever one it picks.
The mistake is easy to make because the parser is so much more satisfying to build. You get a tree, the tree is correct, and the correctness is visible. A pattern gives you none of that pleasure and keeps working in cases the tree cannot reach.
The rule we design to now
Run both, and let the stricter answer win.
Structure decides what a command is allowed to do. Allow, deny, ask, per command, per argument, per destination: these are sentences you can only write if something in the system knows what a command is. A tool with no parser can warn you. It cannot govern.
Text decides what a command is not allowed to mention. A protected path is protected because of what it is, not because of where it lands in a syntax tree. If the name of that file appears in what an agent is about to run, that deserves an answer even when the structural reading is inconclusive.
node9 runs both. Every call goes through the shell parser and through content scanning in the same pass.
The part worth stating carefully is how the two are combined. Combine them by strictness, never by order. Precision does not get overruled because breadth was noisy, and breadth does not get discarded because precision was quiet. Layering two good checks and letting the first one to speak decide is how a system ends up weaker than either check alone. We learned that from a different bug, in a different subsystem, and it keeps being the right rule.
That is the part the original post should have said, and the reason we would rather write this correction than leave the cleaner story standing.