prompts

Prompt Analyzer: Debug & Improve System Prompts

Analyze system prompts for issues, visualize structure with interactive TOC, detect semantic duplicates, and export for AI-powered fixing

Do you know that feeling when your AI agent starts misbehaving, and after an hour of debugging you discover there was a silly typo buried somewhere in your 15,000-token system prompt? I know that feeling. And I never want to repeat it.

So I built this little tool to quickly analyze any of my prompts — whether from observability logs or my raw textprompts files — and catch those issues before they bite me.

How It Works

  1. Paste your prompt — Raw text, LogFire JSON trace, or textprompts format with TOML headers
  2. Click "Analyze" — The tool runs several checks in parallel (formatting, consistency, tool definitions, duplicates)
  3. Review & Filter — Use the checkboxes to show/hide issues by severity (High/Med/Low) and toggle duplicate detection
  4. Copy for LLM Fix — Click the button and paste into your favorite coding agent or AI app to have it fix everything automatically!

The Secret Sauce: Visual Structure

Beyond auto-fixing, you might find it super helpful to have a quick table of contents to get a feel for your prompt. Each section shows its size as a percentage of the total, plus a breakdown of what it contains:

→ Agent Identity [12%] (R:45% I:30%)
→ Brand Voice [8%] (C:60% I:25%)
→ Tools [35%] (I:80% O:15%)
→ Examples [20%] (E:90%)

We also color-code each row by its category — Role, Task, Context, Instructions, Output Format, or Examples. This makes it dead easy to spot inconsistencies and gaps at a glance. Is your "Context" section accidentally full of instructions? You'll see it immediately.

Category Legend

R — Role
T — Task
C — Context
I — Instructions
O — Output Format
E — Examples

Basically, this tool gives you a different lens on your prompts to detect issues quickly!

What Gets Detected

  • Formatting & Grammar — Typos, inconsistent punctuation, weird spacing
  • Logical Inconsistencies — Contradictions, confusing instructions, duplicated content
  • Tool Issues — Inconsistent tool names, argument mismatches, missing definitions
  • Near-Duplicates — Semantic similarity detection to find redundant sections

Supported Input Formats

1. Raw System Prompt

Just paste any system prompt text. Works great with markdown headers and XML tags for structure detection.

2. AI SDK Traces (LogFire JSON)

Paste the JSON trace from your observability tool. The analyzer extracts the prompt from attributes.prompt and concatenates all system messages.

3. Textprompts Format

Speaking of textprompts — it's one of my favorite prompt management libraries! Besides supporting the three languages I write AI in (Python, TypeScript, and Julia), it has useful metadata fields like description and version right inside the file. My AI agents can mark up key insights and bump versions — super easy to manage with Git!

---
title = "Shopping Assistant"
version = "2.1.0"
description = "Updated brand voice section"
---
You are a helpful shopping assistant...

Note on Placeholders

Patterns like {variable} are recognized as runtime placeholders and won't be flagged as issues. This supports textprompts and other templating formats.

Next time your agent starts hallucinating product prices or forgetting tool arguments, maybe — just maybe — it's a typo buried on line 847. At least now you'll find it in seconds instead of hours.

I'm still adding new checks as I discover more ways prompts break in production. If you have a horror story (or a feature request), I'm all ears!