También en español: rosmarops.com/es/blog/lo-que-no-se-leyo-no-cuenta-como-limpio/
Imagine a review tool can’t open a credentials file and doesn’t tell you. You read the report, there’s nothing about that file, and you assume it’s fine. But it never looked. And it could be the one that matters.
That’s the rule I care about most in RosmarOps, and probably the least flashy one: a scanner that stays quiet isn’t the same as a scanner that approves. So when it can’t read something, it would rather not give a score.
An agent only gets a score if RosmarOps could read everything it needs for that agent:
- the capability graph was built and validated;
- the run didn’t time out while building it;
- none of that agent’s configuration sources failed to read;
- no file failed to read in a way that could hide a credential.
If any of those fails, the report says “No score” and gives the reason.
An agent RosmarOps doesn’t know how to evaluate yet isn’t scored at all: the report lists it as detected, not evaluated yet. Today that’s Cline.
Hiding things doesn’t buy a better score
The flip side of the rule is that you can’t raise the score by hiding things. These are the obvious ways to try, and what happens with each.
If you make a source unreadable, you get no score, not a better one. If you remove more than 10% of your projects from the configuration, the report marks the comparison with the previous run as not comparable. If you uninstall an agent, it drops out of the report and the comparison with the previous run is marked not comparable. With no agents left, the report says “No score: no AI agents found”, never 100.
The only thing that works is actually fixing it. If you actually remove a credential from disk, its finding disappears and its weight comes off the score.
What a 100 means
A 100 means: no exposure found in what was read. It never means secure, and it never means protected. And the overall status of the run can’t be green if anything failed along the way.
This sometimes makes the report less comfortable to read. A “No score” is more frustrating than an 85. But the question RosmarOps answers is “what can my agents reach today?”, and silence doesn’t answer it. When it couldn’t look, it tells you it couldn’t.
I wrote this with AI assistants. The machine, the findings and the decision to publish are mine.