Spam Fighter 3000
Meet Jan2, aka Spam Fighter 3000: my AI digital twin that answers your phone, fights spam calls, and only patches through what matters with smart briefings
"We've been trying to reach you about your car's extended warranty..."
Sound familiar? Last Saturday, instead of just blocking another spam number, I decided to fight back. At the AI Tinkerers hackathon, I built Spam Fighter 3000 – and apparently the judges liked it enough to award me the ElevenLabs voice-agent track prize! 🏆
Meet Jan2: My AI Bodyguard
Here's the pitch: What if you had a digital twin that could answer your phone, politely tell spammers to delete your data (GDPR-style), and only patch through the calls that matter – with a smart briefing sent to your phone so you know exactly who's calling and why? I have that now!
Want to try it right now?
- 📞 Call my assistant Jan2: +31 97010250205 (will move to my UK number once approved)
- 🥷 Or battle-test your own spam-fighting skills in the dojo: spam-fighter-3000.svilupp.deno.dev (no login required!)
- 🤖 Or switch to "App" tab and try Jan2 directly!
The Secret Sauce: Teaching Your Twin
The magic isn't just in blocking spam – it's in the training. The system has an interviewer agent that probes and learns your preferences through conversations. How would you handle different call scenarios? What makes a call important to you? All this context gets stored and used for real-time decision making.
Then comes the fun part: The Dojo. It's essentially a training room where you can try to scam your own digital twin. Call it, pretend to be a telemarketer, test edge cases – and immediately see if your defenses hold. It's oddly satisfying trying to social-engineer yourself.
The Dojo - Training environment for testing your AI twin's defenses
The Stack: A Love Letter to Distributed Chaos
Here's where it gets fun for the system design nerds:
The full stack - because why use one service when you can use six? 🎮
The stack is deliberately distributed for maximum iteration speed:
- Frontend: React Router app hosted on Deno (because who has time for build configs?) – for training and alignment
- AI Logic: Pydantic AI running on Modal as an MCP server
- Phone Number: Twilio for the actual phone line
- Voice Agent Platform: ElevenLabs handling the conversational interface
- Notifications: Pushover for real-time briefings when calls get patched through
- LLM Providers: xAI, Anthropic, and Google powering the AI logic
Here's the call flow: All calls hit the Twilio phone number → ElevenLabs voice agent platform → AI logic (back and forth) → If it's for Jan, Pushover notification fires and ElevenLabs transfers to Jan's secret number. The frontend app is where the magic happens for training and aligning Jan2's responses.
How Jan2 Decides Your Fate
Every call goes through Jan2's decision matrix. Think of it like a bouncer at an exclusive club, but the bouncer happens to be three different AIs wearing a trench coat:
Jan2's call classification flow - from *ring* to action in milliseconds 📞
- Gemini 2.5 Flash Lite for lightning-fast first impressions (spam gets the boot instantly)
- Grok 4 Fast for the tricky middle ground (when things get interesting)
- Claude Sonnet 4 as the final boss for complex calls (the heavy hitter)
But here's the clever part: there's a bunch of orchestration logic and sequencing on top of these models. Each layer only passes a filtered subset of information to the next – never the raw inputs. So even if a spammer somehow figured out one layer, they'd still be missing crucial context that the other layers rely on.
Most spam calls get filtered out in milliseconds, while legitimate calls get the red-carpet treatment. The multi-layered approach also makes it harder for spammers to reverse-engineer the system – good luck figuring out which AI just hung up on you when you don't even know what information made it to each decision point!
The Synchronization Challenge
One painful lesson: When your prompt lives in ElevenLabs' UI, your tools are defined in TypeScript, and your logic sits in a Python MCP server, keeping everything synchronized is a nightmare.
My hack? A single Markdown file in my repo with all the prompts, tool definitions, and configurations. One source of truth, copy-paste to deploy. Not elegant, but when you're racing against the clock, it works.
Things That Surprised Me
Getting a phone number is ridiculously hard!
- UK numbers require a "regulatory bundle" approval (still waiting...)
- Twilio's "trial" requires identity verification AND a £20 minimum charge
- Dutch (+31) and Finnish (+358) numbers are the path of least resistance
Modal continues to amaze me.
The live development loop with remote execution and log tailing is exactly what rapid prototyping needs. No dev-prod divide, just ship it.
ElevenLabs is powerful but...
Finding code examples for programmatic agent creation (like OpenAI's Agents SDK) was surprisingly difficult. The UI is decent, but I'd rather drive everything from code. Hence the MCP server pattern – I didn't want to wrestle with their tool UI.
What's Next?
- Actually use this for a few weeks and see what breaks
- Get that UK number approved (bureaucracy willing)
- Figure out number porting so this becomes my primary call handler
- Get myself a new private number that spammers don't have yet
- Maybe... just maybe... make it work for more than one user?
The Bottom Line
Is it perfect? No. Is it production-ready? Definitely not. But in a world where we're drowning in spam calls, having a tireless AI assistant that remembers to ask telemarketers to delete your data while you sip your coffee? That feels like the future we deserve.
Huge thanks to the hackathon organizers AI Tinkerers and sponsors: Anthropic, ElevenLabs, Vibe Kanban, and Dedalus Labs for making this possible.