AI in Context, Ep 01: Delegating to agents
Agents should be doing the easy part that you're repeating all the time. This one is about handing that part over properly.
Watch the session
- 1:41“It's not about the tooling. It's really about the methodology.”
- 7:32“This is just like a model loop that has a task, that has a process, that has an outcome, that has a way of determining whether or not it has achieved that outcome.”
- 20:34“Because I'm trying something new, I have to assume I'm going to get it wrong.”
- 43:20“I think time is a much better judge than a better reasoning model.”
- 48:47“If I do everything in the same agent, how the heck am I going to know what's broken?”
- 0:00 Why AI in Context
- 1:41 It's not about the tooling
- 4:47 What is the outcome I want to achieve?
- 5:37 I want to be more active on social media
- 7:22 This is actually an agent
- 9:48 720 agent flows happening right now for my business
- 10:45 How do you do this?
- 13:51 GitHub is just a ledger
- 17:12 One agent to a repo, or all your agents in a single repo?
- 20:02 The first version is going to be the worst version of the agent
- 21:10 The easy path is managed agents
- 25:40 The hard path is owning the infrastructure
- 27:26 An eval set that determines whether the agent is doing its job
- 30:45 What is a PR?
- 36:20 I call these blood taxes
- 40:37 Can we do a dry run?
- 42:01 The model family takes precedence over the reasoning level
- 48:33 You should not be adding on to an agent
- 51:11 Some form of database that lives atop all of your agents
- 55:47 Keep it in the same chat
- 59:44 At the end of every working session, I commit
- 1:01:30 What's next
What we covered
- What to hand off, and what stays with you
Patience work is the stuff anyone could answer if they had the time. Judgment work is the stuff only you should answer. Agents get the first kind.
- How to write the handoff
Agents need SOPs and reliable sources. What does the business need, what are the sources it can use, and what can it do with them.
- One real task, delegated live
I take something from my own week and hand it to an agent on screen, start to finish, including the parts that go wrong.
- How to check the work
Evals, in plain words: decide what good looks like before the agent runs, so you know whether it worked.
- What I'm running right now
Several agents working at once, overnight, and what that actually looks like day to day.
Agent building is glorified SOPs.
Agents are only as good as the content that goes into it.
The delegation brief
One page to fill in before you hand anything to an agent. If you can't fill it in, the agent can't do the job either. We'll fill one in together live.
The job in one sentence
If you can't say it in one sentence, it's two jobs. Split it.
What done looks like
Describe the finished thing. A file, a draft in the inbox, a filled row. Something you can look at.
Sources it may use
The folders, docs, inboxes and sites it's allowed to read. Anything not listed is off limits.
What it must never do
Send, delete, pay, promise, or talk to a client on your behalf. Write down your own list.
When to stop and ask
The moments it should hand back to you instead of guessing: missing info, money, anything upset.
How I'll check it
What you'll look at to know it worked, and how often. This is your eval.