The Ingredients of Intelligence: Why "Smart" AI Starts with You
Heather HenryWithin R&D Operations at Toast, we’re constantly looking for ways to "cook up" efficiency. In our previous post, we shared how we’re using custom AI agents to automate PRDs and release notes, giving our Technical Product Managers (PM-Ts) superpowers to match our engineers.
But as we lean further into this frontier, we’ve learned a critical lesson: AI is only as smart as the person directing it. To get the most out of these tools, it isn’t enough to just "use AI." You have to use it with intent. Whether you’re a developer writing code or a product manager analyzing data, being "AI-smart" comes down to two things: precision of scope and an understanding of the underlying architecture.
To illustrate why, let’s step out of the office and into the kitchen.
The $120 Toaster: Why Context is Currency
Imagine you’re hungry for breakfast, but you can't find your toaster. You decide to hire a helper named Claude to find it for you. You’re paying Claude $20 an hour. You give him your address, say "find my toaster," and wait.
Claude is diligent. He starts in the attic. He checks the guest bathroom. He spends an hour in the basement and another in the garage. Six hours later, he finds the toaster exactly where it should be: in the back of a kitchen cupboard.
The Cost: $120.
The Result: You have your toaster, but you’ve long since missed the window for breakfast.
Now, imagine a smarter request. You tell Claude: "Find my toaster, but only look in the kitchen." Claude finds it in 15 minutes. It costs you $5, and you’re eating avocado toast before your first meeting.
The AI Connection: Large Language Models (LLMs) operate on a similar economy. Instead of an hourly wage, we pay in tokens. Every piece of irrelevant information the AI processes, and every "room" it wanders into because you didn't define the boundaries, costs time and resources.
We often assume that because AI is "smart," it should know to look in the kitchen first. But AI doesn't have your perspective unless you provide it. To be efficient, you have to be explicit. It’s like the classic "PB&J Challenge" from grade school: if you don’t tell the AI to open the jar first, it’ll try to spread peanut butter through the glass. Before you hit "enter," ask yourself: Am I pointing the AI at the whole house, or just the kitchen?
The "Security" Scare: The Danger of Technically Correct Data
Efficiency is about cost; accuracy is about scope.
Let’s say you’re so impressed with Claude finding the toaster that you hire him for a bigger job. You’re going on vacation and want a security audit. You tell Claude: "Check every door in this house and tell me if the locks work."
Claude returns with a terrifying report: 60% of your doors failed the security check. Panicked, you cancel your flight and call an expensive locksmith. But when you look at Claude’s notes, you realize the "failures" were your bedroom door, your pantry, and your linen closet. Claude wasn't wrong, those doors don't have locks, but they aren't exterior risks. Technically, the data was 100% accurate, but contextually, it was useless. Your exterior doors were perfectly safe all along.
The Business Impact: In a data-driven environment, this happens more often than you’d think. If you ask an AI to analyze "customer churn" without defining whether that includes trial accounts, canceled duplicates, or actual lost revenue, you might get a "technically correct" number that leads to a disastrous business decision.
Understanding the data architecture, knowing what lives in which "room" of your database, is the only way to ensure your AI isn't inflating the results with "closet doors."
How R&D Operations is putting this into practice
Both of these lessons, efficiency and accuracy, came to life for us through our own AI-assisted tooling. We were pushing good data hygiene in our product planning system, so teams started looking into ways to surface data quality gaps. One of the checks they wanted to run was identifying records missing a critical planning field. On paper, it seemed simple: search all records, flag the ones where the field is blank. Easy.
The result? The AI came back reporting that a staggering percentage of our records were missing this key data point. Alarm bells went off. Were we really that far behind on data hygiene? When we dug in, it was a classic closet door problem. More than half of the flagged records belonged to teams and projects that don't use that field at all. The AI wasn't wrong - yes, those records had a blank field - but contextually, the result was meaningless. We had asked Claude to check every door in the house, and he dutifully flagged every interior door without a deadbolt. Technically accurate. Completely misleading.
This is exactly why we're investing in purpose-built AI skills for our processes. Rather than allowing open-ended queries across our entire dataset, the tools we're building encode that contextual knowledge. They understand which teams and projects use which fields, and they automatically scope the query to only the records where a missing value is actually meaningful. The intelligence is baked in, so even a user who doesn't know the underlying data architecture gets a trustworthy answer, every time. Less noise, fewer tokens, and results you can actually act on.
Building a Sharper Appetite for AI
We aren't just building agents; we’re building AI literacy. We’re teaching our teams that the "magic" of AI is actually a partnership.
Define the Room: Limit your AI’s search area to save on "token" costs and time.
Audit the Scope: Ensure your prompts distinguish between the "closet doors" and the "front doors" of your data.
By being mindful of how we make requests, we ensure that our tools don't just give us more information, but the right information. We’re moving beyond just doing things faster—we’re doing them smarter, ensuring that when we use AI to "cook," the results are exactly what we ordered.
Stay tuned as we continue to refine our recipes for a more efficient, AI-driven future.
____________________________
This content is for informational purposes only and not as a binding commitment. Please do not rely on this information in making any purchasing or investment decisions. The development, release and timing of any products, features or functionality remain at the sole discretion of Toast, and are subject to change. Toast assumes no obligation to update any forward-looking statements contained in this document as a result of new information, future events or otherwise. Because roadmap items can change at any time, make your purchasing decisions based on currently available goods, services, and technology. Toast does not warrant the accuracy or completeness of any information, text, graphics, links, or other items contained within this content. Toast does not guarantee you will achieve any specific results if you follow any advice herein. It may be advisable for you to consult with a professional such as a lawyer, accountant, or business advisor for advice specific to your situation.