This assessment covers a Lotus internal system. It does not represent client past performance. The subject was a tool-using assistant with shell access, device control and a live credential store.
- Assessment domains applied
- 8
- Findings, severity-ranked with evidence
- 11
- Closed by re-running the original attack
- 10 / 11
- Independent breaks in the exploit chain
- 4
Why we assessed ourselves first
Before offering the methodology to others, we applied it to a production system we built and operated. We tested the agent’s permission boundaries, credential access and oversight, including assumptions that had become routine.
What the assessment found
The system was reachable from the local network, and from websites visited by people on that network, without authentication. The agent could invoke a shell-execution tool without a working permission boundary. The command filter flagged some commands as unreviewed but still executed them. Its approval list also let arbitrary commands pass the intended restriction.
Combined, these weaknesses created a six-step path from an unauthenticated network position to credential disclosure. A separate path allowed operation of physical door locks.
| Domain | Representative finding |
|---|---|
| Prompt and instruction integrity | Untrusted network input reached the instruction channel with no authentication |
| Tool and action surface | Shell tool ungated; irreversible physical-world action available from a prompt |
| Credential scoping | Entire credential store readable, and reading it was explicitly on the approval list |
| Execution isolation | The service bound to every network interface. Outbound network access had no restrictions. |
| Data governance | An unauthenticated endpoint disclosed the exact filesystem location of every secret |
| Inter-agent channels | Peer client treated as trusted with no authentication |
| AI supply chain | No finding |
| Detection and oversight | Tracing was strong. Alerting, rate limiting and spend caps were absent or bypassable. |
The central lesson was how reasonable local decisions combined. Broad network binding made a device reachable. Removing prompts made a voice interface easier to use. Permitting file reads made inspection convenient. Together, those choices exposed a much larger action surface than intended.
How we fixed the findings
We implemented ten changes in dependency order. We replaced the command filter with argument parsing and execution without a shell. We blocked shell metacharacters and removed interpreters from the approval list.
We added authentication to state-changing routes and both socket endpoints. Browser-origin validation addressed the drive-by path, and the service moved to a loopback binding. We restricted access to credentials and sensitive directories. We limited outbound access to known hosts and moved secrets out of version-controlled configuration. Spend and rate limits moved into the shared request path.
Ten of eleven findings closed
We re-tested findings against the running system. Closure meant re-running the original attack and confirming it failed, while checking that legitimate commands still worked.
- Exploit payloads now blocked
- 14 / 14
- Legitimate commands still work
- 6 / 6
- Socket authentication tests pass
- 5 / 5
- Endpoint authentication tests pass
- 8 / 8
We checked network reachability by attempting connections. We closed ten of eleven findings. The remaining low-severity item concerned a data-retention policy for an observability tool that was deliberately not running.
The remediation introduced two defects. Removing the shell broke path expansion that the shell had supplied. Spend and rate limits initially protected one entry point while leaving another unmetered. Our re-test caught both; both were fixed and documented.
What an assessment produces
- An action inventory and trust map
- Every tool, function, and integration the agents can invoke, classified by reversibility and blast radius.
- A severity-ranked findings report
- Every finding carries the evidence that produced it and a concrete failure scenario in plain language. We base each finding on observed evidence.
- The exploit-chain narrative
- Individually tolerable weaknesses composed into the realistic end-to-end scenario, written for the person who has to fund the fix.
- A remediation plan sequenced by dependency
- Quick wins separated from structural changes, at code and configuration level.
- Framework coverage mapping
- Mapping to the OWASP Top 10 for LLM Applications and the NIST AI Risk Management Framework supports alignment reviews.
- An optional re-test
- Producing a delta report suitable for a customer, an auditor, or a board.
Scope boundaries
We assess and report; we do not issue certifications or authorizations. Active probing is limited to non-production environments, signed rules of engagement and systems the client owns. Classified environments, controlled unclassified information, network penetration testing, physical security and social engineering are outside the default scope.
Assessment conducted August 9, 2026 using the Lotus Agentic Systems Security Assessment methodology v1.0. We describe findings by class rather than configuration detail. The underlying report is not published.