The CRM Was Already There: A Sales-Ops Bot That Reads Gmail Labels

The Inbox Was the Business

This is the project log for a bot we built at New Idea Machine, the software consultancy I subcontract to through JustinTime Systems Limited.

NIM is small enough that one person carries most of the client-facing load. Sales, scheduling, follow-ups, contracts, meeting notes: all of it arrives as email, and all of it lands in a single inbox. That's a volume problem no amount of diligence solves.

The obvious move is to buy a CRM. The obvious move is wrong, for a boring reason: a CRM is a second place to maintain. When one inbox is already absorbing a full day, adding a pipeline tool beside it doesn't split the work, it doubles the surface. Any system that demands new discipline on top of a full workload gets abandoned in three weeks, and then you have a stale CRM lying to you on top of the original problem.

So we built a bot instead. It's called Basecamp (no relation to the project-management product), it lives in Telegram, and it answers questions like "who haven't we heard back from" and "what's the story with Contoso."

Every company and person named in this post is fabricated. The label shapes are real; the names in them are not.

It Had to Be Private, Which Decided Everything Else

Start with the constraint that shaped the whole build: this bot reads NIM's client email. Contracts, rates, prospect names, things people said in confidence about their own companies. Those are NIM's clients, people who never agreed to anything involving me, let alone anything involving a model.

That rules out the easy version. We could have had a far more capable bot in an afternoon by pointing an API key at a frontier model, and it would have shipped a consultancy's client correspondence to a third party as a side effect of building a to-do list. "Their terms of service say they don't train on it" is not a promise any of us had standing to make on those clients' behalf.

So inference is local: Gemma 4 12B on Ollama, on hardware I own, shared across my whole bot fleet. It's the model that passed my tool-calling harness while leaving enough VRAM for two concurrent sequences alongside a whisper server. The bot still talks to Google's API, because that's where the mail already is. What it doesn't do is hand that inbox to a model nobody in this arrangement controls.

Choosing a 12B model means accepting that it is not going to gracefully infer what you meant. Everything downstream in this post (the exact taxonomy rules, the named anti-patterns, the hard tool-call caps) is the cost of that decision. It's a real cost, and it's the right one, because the alternative was a privacy posture none of us could defend to the person whose mail it is.

Two Wrong Sources of Truth

The first design gave the bot its own little CRM: a memory vault with verbs for writing and searching lead records. It was clean, structured, and completely empty, because nothing ever wrote to it. And a model asked to list N leads from an empty source under prompt pressure to be helpful will produce N leads. The morning briefs started arriving with people in them who did not exist. That story, and the test harness I built so it couldn't happen again, is its own post.

The fix in that post was to stop pointing at the empty vault and start pointing at Gmail, where threads had been labeled by hand for years. Correct instinct. But I wrote the queries from imagination:

gmail-search "label:lead" 50      # the funnel
gmail-search "label:client" 50    # active clients

Those returned zero results. Not sometimes. Always, for weeks. There is no label called lead. I'd invented a tidy schema and assumed the real inbox matched it, which is the same mistake as the memory vault wearing a better outfit: I swapped a source that was empty for a source that didn't exist.

And it looked like a fix the entire time. It named a real tool, queried a real system, and was syntactically valid. The only thing wrong with it was that I had never once run it against the actual mailbox.

What Was Actually in There

Eventually I did the thing I should have done first, which took one command:

google-api --as <user> gmail-labels

The real taxonomy is not flat, and it is considerably better than what I'd invented. Funnel stage is encoded by an underscore-prefixed parent label, with each contact as a nested child:

_LEADS/Contoso Foods | Alice Example    → active lead (company + person)
_LEADS/Bob Example                      → active lead (person only)
_LEADS - NOT READY/Fabrikam Systems     → captured, not ready to pursue
_LEADS - LOST/Carol Example             → dead
_CLIENTS                                → converted

Sit with what that means. The label name is the record. _LEADS/Contoso Foods | Alice Example carries the company, the person, and the stage, without reading a single email. The entire roster is one API call and a jq filter:

# every active lead, one "Company | Person" per line
google-api --as <user> gmail-labels \
  | jq -r '.[].name | select(startswith("_LEADS/"))'

Swap the prefix for other stages. Zero email reads. Zero tokens spent on message bodies. A workflow I'd been designing as "search fifty threads and infer" is a string operation on a list of label names, which is exactly the kind of work a 12B model does reliably.

Nobody designed this. It accreted by hand, daily, for years before we showed up with a bot, maintained by someone with every reason to get it right. The best data model in the system was the one already in production, and I hadn't thought to look for it.

The Trap That Made It Loop

One Gmail behavior cost me an embarrassing amount of debugging. Both of these return zero:

gmail-search "label:lead"      → 0   (no such label)
gmail-search "label:_LEADS"    → 0   (parent labels hold no messages)

The second is the nasty one. _LEADS exists, it's spelled correctly, and it still returns nothing, because a Gmail parent label holds no messages directly. Every actual message sits on a child like _LEADS/Contoso Foods | Alice Example.

From the model's side, a correct-looking query returning empty is an invitation to try variations: label:_LEADS/, then label:"_LEADS", then in:_LEADS, burning the whole tool budget on syntax roulette. So that failure is now written into the prompt as a named anti-pattern, both queries spelled out, with an explicit "do not retry these in other forms."

Documenting the wrong query turned out to matter as much as documenting the right one. A small model doesn't need to be told what works nearly as badly as it needs to be told which plausible thing to stop trying.

Read-Only, On Purpose

Basecamp reports on this taxonomy and never touches it. No creating labels, no moving threads, no lead-to-client promotion verb. The tool that would allow it, gmail-modify, is deliberately absent from the lead workflow.

This looks like a missed opportunity and isn't. The taxonomy is hand-curated and encodes judgment that exists nowhere else. A bot writing to it is a bot corrupting the system of record the business runs on, gradually enough that nobody notices until the funnel is wrong.

Which means the prompt also has to teach it to tolerate variation instead of tidying it. Any taxonomy that grows by hand over years accumulates the same few shapes:

• a singular parent alongside the plural one (_LEAD/ and _LEADS/)
• nesting in the opposite order
• two spellings of the same stage coexisting
• the same contact as a plain label and under a stage parent

Match by name, report the exact label text you found, never normalize silently. A model that "helpfully" reports _LEAD/Dave Example as _LEADS/Dave Example has destroyed the evidence that a variant exists, and that variant is real information about how the taxonomy actually grew.

The Email Half: Who's Waiting on a Reply

Leads were the first workflow. The one that gets used every morning is staleness, defined precisely, because "stale" is exactly the kind of word a small model improvises around. A thread is stale when both hold:

1. it's older than seven days (older_than:7d), and
2. the last message was not from the inbox owner. Someone is waiting.

Condition one is a search. Condition two requires reading the thread, because only the thread response carries per-message from and date fields. Search results are IDs and snippets; they cannot tell you who spoke last.

google-api --as <user> gmail-search \
  "in:inbox category:primary older_than:7d -from:noreply -from:notifications" 50
google-api --as <user> gmail-thread <threadId>    # check messages[-1].from

Both filters in that query earn their place. category:primary drops promotions, social, and updates. Most of what makes an inbox feel unmanageable was never a follow-up. The -from:noreply pair catches automated senders Gmail miscategorizes. Filtering at search time is one tool call; filtering by reading is dozens.

And then the rule I'd now put in every agent prompt I write:

Hard cap: one gmail-search plus at most ten gmail-thread calls per triage request. After ten thread reads, answer with what you found, even if there are more candidates.

Listing five real stale threads beats getting cut off mid-sweep with nothing. The prompt supplies a priority order for spending that budget (unread first, then real human senders, then most recent), so the truncation is deliberate instead of landing wherever the context ran out. An agent that dies halfway through has produced zero; one that stops at ten and offers a deeper sweep has produced a morning brief.

Stale threads get shown, not filed into the task tracker. "Someone is waiting on a reply" is state that belongs in Gmail: a star, a label, an archive. Same principle as not building the CRM: don't create a second place where truth has to be maintained.

Reaching Older Mail

The third workflow is history: "what's the story with this client," "did we ever send that quote." Default search returns the most recent N, which is fine until the answer is from March. Two tools, and the prompt is explicit about which question each answers. A date window when you roughly know when, page tokens when the question is "walk all of it," capped at three to five pages before summarizing.

google-api --as <user> gmail-search \
  "from:contact@x.com after:2026/01/01 before:2026/04/01" 50

The rules that mattered were behavioral, not syntactic. Try a window before claiming something doesn't exist: you checked the most recent N, which is not the same as checking. Don't bounce the question back with "what date range?" before attempting a single search. If from: is empty, vary the field before varying the date. And when reasonable windows genuinely come back dry, say so.

That last one is the leads rule in a different costume. Nearly every prompt fix in this system reduces to giving the model an honest empty answer it is permitted to give.

7:00 AM Is the Whole Product

Those three workflows are ingredients. Nobody wakes up wanting to query a lead taxonomy. What justifies the build is one Telegram message that arrives before the day starts:

hermes cron create '0 7 * * *' --name daily-itinerary \
  --prompt "Build the morning itinerary: calendar today, open beads,
            overdue follow-ups, meeting prep from recent transcripts." \
  --deliver telegram:<chat-id>

Calendar, open tasks, who's waiting, and what came out of yesterday's meeting transcripts, composed unprompted, every weekday. Two properties of that command are why the fleet runs on Hermes Agent now.

--deliver is a target, not a hope. Under the previous runtime, getting a scheduled job's output into a chat meant prompt-engineering the model into calling a message tool at the end of its run, reinforcing it, and watching it forget anyway. Delivery stopped being a behavior I had to elicit and became a parameter.

Cron runs in an isolated session, with no prior conversation context. A scheduled job executing inside the live chat inherits whatever was said yesterday and leaves its own output in the window, so the 7am brief slowly poisons the thread it's delivered into. Isolation makes it reproducible: same inputs, same shape, regardless of what anyone typed at the bot on Tuesday.

Reminders Don't Go Through the Model

The itinerary surfaces overdue things, and here the model is disqualified. Small local models are bad at time. Hand a 12B model a list of timestamps, ask which are past due, and you'll get a confident answer that differs on each run.

So it never makes that comparison. The model's only job is to write a reminder to a file. A separate job (--no-agent --script, no LLM in the loop at all) reads that file every thirty minutes and decides what's due:

out=$(heartbeat-check 2>/dev/null || true)
if [ -n "$out" ] && [ "$out" != "HEARTBEAT_OK" ]; then
  printf '%s\n' "$out"
fi

Empty stdout means nothing is sent, so a quiet half-hour is genuinely silent instead of forty-eight "HEARTBEAT_OK" messages a day. The generative part writes; deterministic code decides. Any step that has to be correct rather than plausible should be looking for a way out of the model.

The Bot Never Sees a Credential

Reading someone else's mailbox means holding a Google service account key with domain-wide delegation. That key had no business being anywhere near a process whose inputs are arbitrary Telegram messages and arbitrary email content. Prompt injection against a mail-reading agent isn't hypothetical, it's the obvious attack, and anyone can send that inbox an email.

So the key lives in a different Unix user entirely.

graph LR
    subgraph "Telegram"
    U[User]
    end
    subgraph "basecamp user"
    H[Hermes gateway] --> G[google-api CLI]
    end
    subgraph "google-proxy user"
    P[API proxy] --> K[(SA key)]
    end
    U --> H
    G -->|group-restricted unix socket| P
    P --> W[Google Workspace]
  

The google-proxy service runs as its own system user, owns the agenix-encrypted service account JSON, and exposes a Unix socket only the basecamp group can connect to. The agent shells out to a google-api CLI that talks to that socket. It receives API data, never a key, never a bearer token. If someone talks the model into exfiltrating its credentials, there are none to find.

Three constraints on top of that:

• Impersonation is whitelisted server-side. The --as flag accepts a short fixed list of addresses and rejects everything else at the proxy. The model cannot widen its own reach by being clever, because the check runs on the side it can't reach. Domain-wide delegation is a loaded gun; the whitelist is the trigger lock.
• No general-purpose network tools. The bot has web access through a search backend and a fetch wrapper, both of which already know what they're allowed to reach. What it doesn't have is a socket it can point anywhere.
• Secrets never touch the state directory. They arrive via systemd EnvironmentFile only. The runtime's own .env merging would persist the Telegram token into the bot's state directory, and from there into every backup archive.

One honest caveat, since posts like this usually skip them. Moving to this runtime cost me a layer: the previous one had a deny-by-default exec allowlist, and this one approves commands by default with pattern detection and owner approval on top. I took that trade knowingly for the scheduling primitives above, and it moved the containment boundary from the command list down to the process isolation underneath it.

That's the part worth generalizing. When you adopt a tool for one capability, work out which of your existing defenses it retires, because the answer is rarely none. The layer you stop thinking about is the one that was doing the work.

The Part That Generalizes

I spent two design iterations building the bot a place to keep its knowledge, and both times it filled that place with fiction. The version that works owns no state at all. It reads a taxonomy a human maintains, in a tool they already use, and it's forbidden from writing to it.

The reflex when a model confabulates is to add rules: be accurate, don't guess, only report verified information. I wrote all of them and they did nothing, because the bot was being asked to describe an empty room in a building that didn't exist. That's a testing problem once you know to look for it. But first it's a research problem, and the research is unglamorous: go look at the actual data before you design the schema.

The prompt modules behind all of this are parameterized Nix expressions, so putting lead tracking on a second bot is one import line. That part I'd do the same way again. The part I'd change is the couple of weeks I spent writing careful instructions about a label that was never there.

Header photo by Oskar Kadaksoo on Unsplash.