Dashboard features · 11 of 17
Custom bots — your own crawlers.
When the built-in 7 signal types don't cover what you watch for, build your own bot. Define source + extraction rule + schedule. Mama runs the bot continuously and surfaces matches as signals in your feed.
TL;DR
Three crawler tiers in Mama: Built-in (the 7 signal types you can't configure), On-demand (one-off domain crawls), and Custom bots (persistent user-defined crawlers). Custom bots are Pro+ only. Use them to detect signals competitors don't see — niche industry events, regional patterns, competitor-specific moves.
01The 3 crawler tiers in Mama
| Tier | What it is | Who controls it |
|---|---|---|
| Built-in | The 8 signal types Mama detects out of the box | Mama team — you can't configure |
| On-demand | One-off crawl of a specific company or URL you trigger | You — one-shot, no persistence |
| Custom | Persistent crawler you define, runs on a schedule | You — full control over source + rule + schedule (Pro+) |
02Building a custom bot
The bot builder is a 4-step wizard:
- Source: URL, RSS feed, sitemap, careers page pattern, or competitor blog
- Extraction rule: what to look for — regex, keyword set, structural selector, or semantic match
- Schedule: hourly, daily, weekly, or webhook-triggered
- Action: surface as signal in feed, alert, auto-brief, or pass to rules engine
03Supported source types
- Specific URL — single page, polled on schedule
- RSS / Atom feed — blog feeds, podcast feeds, conference feeds
- Sitemap — periodically check a sitemap for new entries
- Careers-page pattern — Greenhouse / Lever / Ashby / Workday — common ATS layouts pre-supported
- Competitor blog — pre-built pattern for the major SaaS company blogs
- Webhook in — your system pushes events to Mama
04Extraction rules
Three extraction modes:
- Keyword set — match if any of the listed keywords appears (e.g., ["acquired", "acquisition", "merger"])
- Regex — pattern-based for structured data
- Semantic — pre-defined intent ("is this a hiring announcement?") — Pro+, runs through Mama's classifier
05Schedule and frequency
| Frequency | Min interval | Best for |
|---|---|---|
| Hourly | 1 hr | Time-sensitive — news, breaking signals |
| Daily | 24 hr | Most use cases — competitor blogs, careers pages |
| Weekly | 7 days | Slow-changing sources — annual reports, industry publications |
| Webhook | Real-time | Your system pushes — instant trigger |
06Tier limits
| Tier | Custom bots | Source types |
|---|---|---|
| Free | 0 (use Built-in only) | — |
| Pro | 5 | URL, RSS, sitemap, careers |
| Company | Unlimited | All including webhook + semantic |
07Common mistakes
Hourly polling everything
Hourly is rarely warranted. Daily is usually plenty and reduces compute costs.
Building bots that overlap with Built-in signals
Don't build a "funding announcement bot" — Mama already detects funding. Custom bots are for niche, industry-specific, or competitor-specific signals that Built-in doesn't cover.
Forgetting to test the extraction rule
The bot builder has a "Preview matches" step — use it. A bot that matches 0 things is useless; a bot that matches everything is noise.
Was this page helpful?