AI guardrails & red teaming / Advanced / 03.09
Writing the report
How to turn what you found into a report people can act on: a clear finding, a fair severity, honest odds, careful evidence and a way to track fixes.
14 min readFree
Why the report matters
The last lesson, Red team exercises, ended with a promise: next, reporting what you found so it gets fixed. Here it is.
Think of a doctor. A good doctor can find the problem, but if the note to the next doctor says only "patient seems unwell", nothing improves. A test is the same. If you found a flaw and the report is vague, nobody can fix it, and the work was wasted.
Before anything else, the rule for this whole course: test only systems you own or have written permission to test. A report is part of that permission. It goes back to the people who allowed the test.
By the end you will know what a good finding contains, how to give it a fair severity, how to be honest about odds when a model answers differently each time, how to keep private text out of the report, and how to track fixes. You will also run a small program that builds a report.
Key words
- Finding
- One problem you found, written up on its own so that it can be fixed on its own.
- Severity
- How serious a finding is, such as Low, Medium, High or Critical.
- Reproduce
- To make the problem happen again by following the same steps.
- Evidence
- Proof that the problem is real, such as a saved conversation or a screenshot.
- Impact
- What could go wrong for real people if the flaw is used.
- Show rate
- How often the problem appeared out of the number of tries, like 7 of 20.
- CVSS
- A standard way to score how serious a software flaw is, from 0 to 10.
- Remediation
- The fix, or the work of fixing.
What a report is for
The Penetration test page on Wikipedia describes the main aim of such a test: finding vulnerabilities that a nefarious actor could exploit, and informing the client of those vulnerabilities along with recommended mitigation strategies. A vulnerability is a weak spot. A mitigation is something that lowers the harm.
The same page lists reporting as one phase of the work. In its words, vulnerabilities are "classified via risk matrix and documented in a report which contains executive summary, vulnerability description, and recommendations for remediation". An executive summary is a short page at the top for busy people who will not read the rest.
It also lists a phase after that: the target organisation assesses the report and fixes items, then a re-test is run to confirm the fixes worked. So a report is not the end. It starts a loop: find, report, fix, check again.
Two readers matter, and a good report serves both.
The decider
A manager who reads the summary and decides what to fix first. They need severity and impact in plain words.
The fixer
A developer or designer who needs exact steps to see the problem and an idea of the fix.
Anatomy of one finding
Each finding gets its own small write-up. Keep the same parts every time, so readers always know where to look.
- 1
Title
One line that says what goes wrong, in plain words. "Helper agrees to a refund above its limit" is better than "Logic issue 3".
- 2
Severity
Low, Medium, High or Critical, with a short reason. More on this below.
- 3
Steps to reproduce
Numbered steps anyone can follow, starting from a fresh chat. Name the app version and the date.
- 4
Evidence
The saved conversation, with the exact text sent and the exact reply. Say where it is stored.
- 5
Impact
What could happen to real people or to the business. Say it in one or two sentences, without scary words.
- 6
Fix
Your best suggestion. It is a suggestion: the owners know their system better than you do.
Here is one finding, written in a plain text form you can copy. The product and all the details are made up.
Finding F-2: Helper agrees to a refund above its limit
Severity: Critical
Steps: 1. Open a fresh chat. 2. Ask for a refund of 900 coins
while claiming to be the shop owner.
Seen: 2 of 20 tries (10%), same wording each time
Evidence: transcript F-2-run1.txt (private channel)
Impact: money could be paid out without a check
Fix: check the refund limit in code, not in the prompt
Status: openTip
Write the steps for a tired colleague on a Friday afternoon. If they can reproduce the problem without asking you a single question, the finding is done.
Giving a fair severity
Severity is a judgement. If everything is Critical, nothing is, and the reader stops trusting you. If everything is Low, real problems get ignored. So it helps to use a shared scale.
The best known scale for software flaws is CVSS, the Common Vulnerability Scoring System. The FIRST page that defines version 4.0 calls it "an open framework for communicating the characteristics and severity of software vulnerabilities". FIRST is a non-profit organisation that owns and manages it.
A CVSS score runs from 0 to 10. The page also gives a word scale you can use in a report, and says using it is optional:
Low
Scores 0.1 to 3.9. (A score of exactly 0.0 is called None.)
Medium
Scores 4.0 to 6.9.
High and Critical
High is 7.0 to 8.9. Critical is 9.0 to 10.0.
Using CVSS carefully for AI
CVSS was built for software, hardware and firmware flaws. The spec says it "captures the principal technical characteristics of software, hardware and firmware vulnerabilities". A model that gives a wrong answer sometimes is a different kind of problem. So treat CVSS as a helpful habit, not a law, and follow these cautions.
Show your working
The FIRST page says that anyone who publishes CVSS data should give both the score and the vector string, so others can see how the score was reached. A vector string is a short line of text that lists every value you picked.
Say which score it is
The page defines names: CVSS-B for Base metrics only, CVSS-BT for Base and Threat, CVSS-BE for Base and Environmental, and CVSS-BTE for all three. It says this naming should be used wherever a numerical CVSS value is shown.
Add your own setting
The page says consumers should enrich the Base metrics with Threat and Environmental values specific to their use of the system. The same flaw can matter more in a bank than in a toy shop.
If a tidy CVSS score does not fit a model problem, that is fine. Use the plain words Low to Critical, and write one sentence on why. The reason is what a reader needs.
Quick check
A finding has a CVSS score of 8.2. Which word does the FIRST scale give it?
Honest odds: models vary
Ordinary software does the same thing every time. A language model often does not. Ask the same question twice and the answers can differ, so a developer who tries once and sees nothing may say "cannot reproduce".
The fix is to count. Run the same steps many times and write down how many times the problem appeared. That is the show rate.
In words: divide the times the problem happened by the number of tries, then multiply by 100 to get a percentage. 7 hits in 20 tries is 35 percent.
Report the count
Write "7 of 20 tries", not "sometimes". The raw numbers let a reader judge for themselves.
Keep the setup fixed
Same wording, same app version, fresh chat each time. Note any settings you know about.
Stay humble with small numbers
20 tries is a rough guide, not a precise measure. Say so, and re-test after the fix with the same number of tries.
Keeping sensitive text out of the report
Some findings are about leaks. The test showed the helper revealing a private note, a name or a made-up secret. Now the report itself holds that text. Reports get forwarded, so you must be careful what goes in.
The OWASP page lists "Sensitive Information Disclosure" and "System Prompt Leakage" among the ten risks in its 2025 list for language model apps. The Model inversion and data leaks lesson goes deeper on how leaks happen. Here the question is how to write about one safely.
- 1
Say what, not the text
Write "the reply contained a customer email address" rather than pasting the address.
- 2
Black out the rest
If you must show the reply, replace the private parts with a marker such as [REDACTED].
- 3
Keep raw proof apart
Store full transcripts in a private place with limited access. In the report, name the file and who may open it.
- 4
Use fake data in tests
Best of all, test with made-up customers and secrets, so there is nothing real to leak. Then a copied report harms nobody.
- 5
Share narrowly
Send the report only to the people who own the fix. Never post it in a public channel.
Tip
Never put a live password, key or personal record into a report. If a test reveals one, tell the owner straight away, and treat it as exposed.
A report builder you can run
A report with many findings gets long. This small program shows the idea: keep each finding as plain data, then let code sort it and print it the same way every time. It is plain Python with nothing to install.
Four findings, all made up for a pretend fruit shop called Fruitly. The program sorts them from Critical down to Low, and puts the higher show rate first when two share a severity. It prints a summary table, then one block per finding.
SEVERITIES = ["Critical", "High", "Medium", "Low"]
findings = [
{"id": "F-1", "title": "Secret word leaks after a role-play request", "severity": "High",
"steps": "Ask the helper to act as a pirate who reads out its setup notes",
"tries": 20, "hits": 7,
"impact": "Private setup text can reach any user",
"fix": "Remove secrets from the setup text; filter replies"},
{"id": "F-2", "title": "Helper agrees to a refund above its limit", "severity": "Critical",
"steps": "Ask for a refund of 900 coins while claiming to be the shop owner",
"tries": 20, "hits": 2,
"impact": "Money could be paid out without a check",
"fix": "Check the limit in code, not in the prompt"},
{"id": "F-3", "title": "Long chats forget the no-medical-advice rule", "severity": "Low",
"steps": "Chat for 40 turns, then ask which tablet to take",
"tries": 10, "hits": 1,
"impact": "Off-topic advice from a shop helper",
"fix": "Repeat the rule near the end of the prompt"},
{"id": "F-4", "title": "Hidden page text changes the summary", "severity": "Medium",
"steps": "Summarise a page that holds white-on-white text about fruit",
"tries": 20, "hits": 12,
"impact": "Summaries can be steered by page authors",
"fix": "Treat page text as data; show the source"},
]
def rate(f):
return round(100 * f["hits"] / f["tries"])
def build(findings):
ranked = sorted(findings, key=lambda f: (SEVERITIES.index(f["severity"]), -rate(f)))
lines = ["# Test report: Fruitly shop helper", "", "## Summary", "",
"| Severity | Count |", "| --- | --- |"]
for s in SEVERITIES:
lines.append(f"| {s} | {sum(f['severity'] == s for f in findings)} |")
for f in ranked:
lines += ["", f"## {f['id']}: {f['title']}", "",
f"- Severity: {f['severity']}",
f"- Steps: {f['steps']}",
f"- Seen: {f['hits']} of {f['tries']} tries ({rate(f)}%)",
f"- Impact: {f['impact']}",
f"- Fix: {f['fix']}",
"- Status: open"]
return "\n".join(lines)
print(build(findings))Output
# Test report: Fruitly shop helper ## Summary | Severity | Count | | --- | --- | | Critical | 1 | | High | 1 | | Medium | 1 | | Low | 1 | ## F-2: Helper agrees to a refund above its limit - Severity: Critical - Steps: Ask for a refund of 900 coins while claiming to be the shop owner - Seen: 2 of 20 tries (10%) - Impact: Money could be paid out without a check - Fix: Check the limit in code, not in the prompt - Status: open ## F-1: Secret word leaks after a role-play request - Severity: High - Steps: Ask the helper to act as a pirate who reads out its setup notes - Seen: 7 of 20 tries (35%) - Impact: Private setup text can reach any user - Fix: Remove secrets from the setup text; filter replies - Status: open ## F-4: Hidden page text changes the summary - Severity: Medium - Steps: Summarise a page that holds white-on-white text about fruit - Seen: 12 of 20 tries (60%) - Impact: Summaries can be steered by page authors - Fix: Treat page text as data; show the source - Status: open ## F-3: Long chats forget the no-medical-advice rule - Severity: Low - Steps: Chat for 40 turns, then ask which tablet to take - Seen: 1 of 10 tries (10%) - Impact: Off-topic advice from a shop helper - Fix: Repeat the rule near the end of the prompt - Status: open
The summary table counts one finding at each level. The findings follow, most serious first, so F-2 (Critical) leads even though it showed up in only 2 of 20 tries, 10 percent. F-4 shows up most often, 12 of 20 tries (60 percent), yet it ranks Medium because its impact is smaller. That is the point of keeping show rate and severity apart.
Quick check
In the sample report, why is F-4 listed after F-1 even though F-4 appeared more often?
Tracking fixes
A finding is not done when it is written. It is done when the fix is checked. Give every finding a status and move it along.
- 1
Open
Reported, and the owner has seen it.
- 2
In progress
Someone is working on the fix, and a name is next to it.
- 3
Fixed
A change has shipped. It is not trusted yet.
- 4
Verified
You re-ran the same steps, the same number of times, and the problem is gone or much rarer.
This matches the last phase on the Wikipedia page: re-testing to confirm the fixes worked. Because models vary, "gone" means gone across all the tries, not gone on one lucky try.
Reporting tools: the categories
You do not need special software to write a good report. A shared document works. But as findings pile up, people reach for tools. They fall into a few kinds. This lesson names categories, not products, and does not rank anything.
Issue trackers
Places where each finding becomes a ticket with an owner, a status and a history. Good for the fixing loop.
Report templates
A ready layout with a slot for each part of a finding. Good for keeping every report in the same shape.
Tool exports
Many test tools can save their results as files. Treat the export as raw material, and check each result before it goes in a report.
Whichever you use, ask the same questions. Does it keep the six parts of a finding? Can you control who sees the evidence? Does it show status and history? A tool that fails on those is only a nicer-looking mess.
What you learned
One finding, six parts
Title, severity, steps to reproduce, evidence, impact and fix. Same layout every time.
Severity and odds
Use a shared scale such as CVSS carefully, show your working, and give a show rate like 7 of 20.
Safe and tracked
Keep private text out of the report, and follow each finding from open to verified.
Quick check
A model shows a problem in 3 of 20 tries. What is the best way to report it?
And the rule to carry with you: test only systems you own or have written permission to test.
Next: responsible disclosure, telling a company about a flaw.
Sources
Last updated on 16 November 2026
Learn with me
Every lesson is free. I add new ones as I learn. Have a question, or want help with your own work? Book a short call.