Eighteen months ago we published a post arguing that regular expressions are the wrong tool for stopping an AI agent from doing something destructive, and that a real shell parser is the answer. We stand by half of it. The other half was a story we told ourselves because our architecture was already committed.
This post is the correction, and it contains two commands our own tool does not catch.
The question each method is asking
A parser and a pattern are not two implementations of one idea. They answer different questions, and once you see which question each is asking, every difference in behaviour follows from it.
A parser asks: what does this command do?
It turns the string into a tree. It knows which word is the command and which words are its arguments. It knows a quoted string from a bare word, a pipeline from a single call, a comment from code. So it can say something a matcher never can: this is a read, and the thing being read is that path.
cat ~/.ssh/id_rsa
CallExpr
├─ cmd cat → a read
└─ arg ~/.ssh/id_rsa
A pattern asks: does this string contain something protected?
There is no tree. There is no concept of a command at all. The only question is whether the characters are present.
That sounds obviously weaker. It is not.
The measurement
Here is cat ~/.ssh/id_rsa written four ways, checked against node9's parser and against a plain path pattern of the kind most tools in this space use.
| command | parser | pattern |
|---|---|---|
cat ~/.ssh/id_rsa | blocks | matches |
cat $(echo ~/.ssh/id_rsa) | allows | matches |
K=~/.ssh/id_rsa; cat $K | allows | matches |
echo "reads ~/.ssh/id_rsa" | allows, correctly | matches, wrongly |
Rows two and three are ours. Those are real, they are on a clean install with the credential jail enabled and no configuration involved, and they are not a bug in the sense of something being broken. They are the parser working exactly as designed.
In row two the argument to cat is the output of another command. The parser sees that correctly. It refuses to guess what that output will be, because guessing would mean running the inner command, and a security tool that executes the thing it is inspecting has stopped being a security tool. In row three the same value arrives through a variable. Resolving it needs data flow across statements, which is a different and much harder analysis than parsing.
The parser is not wrong in either row. It is simply being asked a question it cannot answer from the text alone.
Why the pattern wins those rows, and what it costs
The pattern catches all three because it never depended on the path being in argument position. It does not know what argument position is. .ssh/ is in the string, so it fires.
Row four is the bill for that. echo "reads ~/.ssh/id_rsa" does not read anything. It prints a sentence. The pattern has no way to know the path is inside a string being printed rather than a file being opened, so it fires anyway.
This is the whole trade, stated plainly:
- The parser is precise and blind to values that do not exist until the shell runs.
- The pattern is total and blind to the context of what it found.
Neither property is derivable from the other, which is why neither method is a cheaper version of the other.
What we got wrong
Our original post treated the parser as a strict upgrade. It is not. It is a different instrument with a different failure mode, and we shipped the failure mode without naming it because we were busy being pleased about the instrument.
Worse, we already had the other layer. node9 runs a content scanner and a dangerous-words check on every call, alongside the parse. Both are text layers. Neither of them was carrying the credential rule. The capability was in the product and the rule was wired to the one layer that structurally cannot see rows two and three.
That is not a missing feature. It is a rule attached in the wrong place, which is a more embarrassing kind of mistake and a much easier one to fix.
The rule we use now
Run both, and let the stricter answer win.
Structure decides what a command is allowed to do. Allow, deny, ask, per command, per argument, per destination: these are sentences you can only write if something in the system knows what a command is. A tool with no parser can warn you. It cannot govern.
Text decides what a command is not allowed to mention. A protected path is protected because of what it is, not because of where it lands in a syntax tree. If the name of that file appears anywhere in what the agent is about to run, that deserves an answer even when the structural reading is inconclusive.
Combine them by strictness, never by order. Precision does not get overruled because breadth was noisy, and breadth does not get discarded because precision was quiet. We wrote that rule down after a different bug taught us the same lesson, and it keeps being the right rule.
Why we are publishing the misses
Because someone was going to find them, and because a security tool that only publishes the rows it wins is not giving you information, it is giving you an advertisement.
If you run node9, you can reproduce every row in the table:
node9 explain Bash 'cat ~/.ssh/id_rsa'
node9 explain Bash 'cat $(echo ~/.ssh/id_rsa)'
The first stops. The second, today, does not. That is the honest state of it, and the fix is a layer we already run.