A case study · Kaariz Hussain
Somebody promised something.
Nobody followed up.
How I built Open Loops, a tool that reads Slack for the promises nobody is chasing, your own included, and writes the follow-up. This is the thinking, the wrong turns, and what each one taught me.
The problem
Every promise is made in passing.
Assistants live in channels like this one. Three people make three promises in a minute. Nobody writes them down, and nothing in Slack remembers that a sentence was a commitment.
Five days later · Today, highest priority
5 days late.
One message in your own DM each evening. Tap a line to see what it caught. Every item quotes the sentence it came from.
Screens re-created from a real run on a test workspace. Every line of text is verbatim; the draft and the closed promise come from the details posted in its thread.
Where it came from
It started as a portfolio piece for a job application.
The first version took a day: an explainer page and a demo inbox. The application didn't work out, but the problem was a real one, so I kept going. It became a longer exercise in building something that works on real messages, testing it honestly, and fixing what broke.
First version: an explainer page with a demo inbox.
The same detector, running on a schedule.
Moved onto Slack. Real messages broke it within ten minutes.
Packaged as an installable skill, with setup written down step by step.
Seed test with two test accounts and 13 scripted messages.
Launch video from a real run.
Adversarial review. Twelve fixes, each with a regression test.
The founding decision
Rules-based detection. Follow-ups stay in your hands.
An assistant can't act on a list they can't trust. So the detector is regular expressions and plain comparisons. A model only fetches messages and posts the result. It drafts the nudge but never sends it. If it flags something, it can show you exactly why.
Reproducible results
The same messages, settings, tracking history and date always produce the same digest. No model judges anything, so every bug can be reproduced and gets a test that fails without the fix.
Uses what you already have
Reads Slack through the connector you've already approved. Nothing new to install in your workspace.
Quotes its source
Every item carries the sentence, who said it, where, and why it's dated.
Try it yourself
Promise, or not?
These are real sentences the rules got wrong at some point. Each mistake became a rule and a test. Make your call.
What the data said
items per 100 messages.
I ran it over 3,725 real emails from the public Enron corpus. More than one message in three produced something, so a real mailbox came out as a list of 178. The corpus has no labels, so that's a rate, not an accuracy. But it showed a problem accuracy alone wouldn't fix: a list that long doesn't get read. Choosing what goes at the top mattered as much as detecting.
What moved the top eight items
I tried four ways of reading intent from the words. One field the mailbox already kept did more than all four.
Excluding over-broad phrases· worth 1.4%Filtering distribution lists· almost nothingHand-written lists of message types· leaked three timesGrammar test for a named object· made the top eight worse- ✓Respecting mail the reader had already deleted. A quarter of the items came from it.
Lesson: the overall rate and the quality of the first eight items move independently. Only the second one gets read.
Before building more
Is an "open loop" even a well-defined thing?
Two labellers graded the same 79 items blind, against a written rubric. They agreed far more than chance would explain (Cohen's kappa 0.76), which says the idea is worth pursuing, not that the detector is right. The interesting part was where they disagreed.
Both labellers were language models, so they may share blind spots. Two working assistants are the real test.
Items one kept that the other dropped
Every disagreement ran the same way.
That hints at a difference in strictness more than in definition. It's a lead, not a finding. It was enough to steer the design toward letting the reader tune it: items they reject teach it which phrasings to stop flagging.
Method
Real messages find what reading the code doesn't.
The first time it ran against a real Slack workspace, four bugs surfaced in ten minutes. None of them were in the detection logic.
Messages arrived newest first, sorted to the minute, so a promise could land after its own delivery and never close.
Every app-sent message carried the same "Sent using" line, so every message looked related to every other.
Thread replies aren't in a channel read at all. Anything promised in a thread was invisible.
A thread read returns a different, undocumented format. The adapter would have quietly parsed nothing.
The worst bugs are silent
A false alarm is annoying. A missing item is invisible.
A bare "Attached." used to close every open promise in the thread. Things left the list without being done, which is exactly what the tool exists to prevent.
Illustrative thread. The rule now: anywhere it can't tell, it says so. A list that quietly drops things is worse than one that admits what it missed.
The seed test
A channel is not one relationship.
Every workspace it had read had one member, so three rules borrowed from email never got tested. Two test accounts posting 13 scripted messages broke all three at once.
No name beats a wrong name.
If it can't tell who a promise is owed to, it says nobody rather than guessing. Who it's owed to, whose move it is and which conversation it's in are now decided per message, never per channel.
Seed test, 13 items
Right on the first run, then right after the three rules were rewritten. Each rule has a test that guards it in both directions.
Designing the digest
What the detector needs to know isn't what a person needs to read.
A brief, then the details
The DM shows today's three items and a short "also open" list. Everything else (every item, drafts, evidence, what was read) goes in the thread underneath.
Urgency fades differently
Sam's overdue ask slid from #1 to #6 as it aged. The fix: when someone is waiting on you, their item never ages out of Today. Your own overdue items still do.
Reply to fix it
Type "3 7" to say items 3 and 7 aren't real. Each digest also shows a few messages it found nothing in: "miss b" flags one it walked past, and a bare "miss" says none of them held a commitment. Rejections count as evidence about phrasing, not just about one item.
What I got wrong
The mistakes became the rules.
A fix checked in one direction
A fix for delegated tasks was tested on outgoing messages only. It quietly hid every incoming promise with a named subject, which is the main use case.
Now: every fix is checked both ways, incoming and outgoing.
Results kept in a temp folder
Early runs, including my first recall reading, were saved in temporary folders and lost.
Now: one persistent working folder, and a ledger that survives.
Two setups, one DM
A second setup posted its own numbered list into the same DM, so my reply meant for one was applied to the other.
Now: one writer per ledger, written into the repo's rules.
October · Asking others to break it
Then I had it attacked on purpose.
An independent review (Codex) went after the deployed code looking for ways it fails. Every finding was reproduced on the live version before anything changed, then worked in isolation, one area at a time: message parsing, the ledger, corrections, the calendar and tooling. Twelve fixes shipped the same day, each with a test that fails without it. Several were silent: the digest looked fine and was quietly wrong.
A thread root with nothing in it no longer hides its replies.
Real messages are not discarded for looking like connector furniture.
Each digest's spot check is counted once, and only a real answer counts.
Calendar times and the digest's date are the reader's, not the provider's or UTC's.
Commit titles, verbatim. The next evening's real run turned up one more: an empty channel reported as unreadable. Fixed with a test the same night.
Where it stands
A working prototype. Not yet proven.
It runs every evening on my own workspace and posts a real digest. It hasn't been tested by independent users, so every claim here is about how the code behaves, not about how much it helps.
- Finding a practical problem: promises made in passing that nobody tracks.
- Making design decisions and holding to them: plain rules, no app, every item quoting its source.
- Testing against real messages and independently reproducing review findings.
- Investigating failures until I understood them, then writing the rule and the test that keep them fixed.
commits, each one explaining why
test suites, run on every push
Open Loops
Finds the things nobody is chasing.
A working prototype for Slack, built as a portfolio project.
Kaariz Hussain · me.kaarizmatic.com