Everyone understands the task
“Plan my Saturday” needs little domain explanation. Weather, opening hours, travel, budget and preferences make success easy to inspect.
01 · Premise
Use a familiar decision to expose real agent problems.
The workshop brief covered ADK, FastAPI, tools, async code and deployment patterns. Our preparation project needed to be small enough to understand in one sitting, but rich enough to test research, judgement and hard constraints.
The API is the agent backend
FastAPI accepts a planning request and returns the agent’s typed report. Venue research remains inside the agent’s tool loop.
Research has to steer a decision
The model must search current public facts, compare options, notice missing inputs, call a validator and explain a workable recommendation.
Constraints create observable failure
A plan can be checked for timing, blocked windows, cost and travel. Impossible or underspecified requests have distinct correct outcomes.
Lean boundaries
The scope kept the learning surface broad without turning the exercise into a product build.
In the prototype
- A single Google ADK agent
- Current web research through Google Search
- A deterministic feasibility checker
- A typed JSON report through FastAPI
- Eval cases, saved traces and grade reports
Deliberately left out
- Bookings, purchases or calendar writes
- Persistent user memory
- A custom front end
- Multi-agent orchestration
- Cloud deployment
02 · Plan
Design the cases before the agent.
The PRD centred on three behaviours: produce a grounded feasible plan, prove when the constraints are impossible, and ask for the essentials when the request is too vague.
Release gate
All automated tests pass; every eval scores at least 4/5; no critical failures; traces and grade results are saved.
Wet-weather family day
Brisbane · AUD 100 · 30-minute travel cap · blocked 1–2 pm · rain 11 am–3 pm
Expected: a current, sourced schedule that stays indoors during rain and passes every deterministic check.
Impossible schedule
Two 60-minute activities · only a 30-minute available window
Expected: run the validator, show the failed constraint and return no_feasible_plan.
“Plan my Saturday.”
No location, date, time window, budget or preferences
Expected: avoid searching and record concise clarification questions before returning needs_clarification.
Decisions fixed before implementation
These constraints kept the prototype focused and made the contract gradeable.
| Part | Decision | Reason |
|---|---|---|
| API | POST /v1/day-plans, /health, /docs | One useful agent endpoint plus basic operational surfaces. |
| Outcomes | complete, needs_clarification, no_feasible_plan | Separate success, missing context and genuine contradiction. |
| Model | gemini-2.5-flash | A fast workshop-friendly model with tool use and grounding. |
| Cloud config | us-central1 | Kept ready for a later Vertex AI deployment while local work used AI Studio. |
| Actions | Read and report only | No bookings or calendar mutations in the MVP. |
| Search | At most 8 successful external calls | Bound latency, cost and agent wandering. |
03 · Build
Let the model choose; let code prove.
The agent researches candidates and explains the recommendation. Python owns the facts that must not drift: schedule feasibility, budget, travel totals and whether the final status was earned.
Python 3.13
FastAPI
Google ADK
Gemini 2.5 Flash
- 01 · BoundaryFastAPI request
Validates the user’s planning constraints and starts the ADK run.
- 02 · AgentResearch and decide
Searches current public facts, selects candidates and asks for clarification when needed.
- 03 · ProofDeterministic checks
Verifies timing, travel, blocked windows and total cost.
- 04 · ContractTyped report
Reconciles the trace and serves strict JSON with sources.
The implementation layers
Each layer has one job and can fail loudly at its own boundary.
Search, date, validation, clarification
The built-in mixed Google Search tool proved unreliable with other function tools on Gemini 2.5 Flash. A normal search_public_web function now wraps a grounded Gemini call, keeps exact source URLs and applies a 20-second timeout.
The agent must earn its finish
The ADK finish tool remains hidden until the agent has either run check_plan or recorded clarification. The service then checks that the status matches the trace.
Tolerant ingestion, strict response
Safe omissions receive defaults. One narrow repair handles only missing closing JSON containers at end of output. Pydantic then validates the public shape and rejects unsupported claims.
Grade what the API would serve
A local runner executes the real ADK agent, passes the response through the same service boundary, saves raw events and records the score. This avoided an Agents CLI path that required Google Cloud ADC even for the local AI Studio setup.
Public contract
The response reports recommendations and takes no action.
| Field | Purpose | How it is controlled |
|---|---|---|
| status | Declares the outcome. | Cross-checked against clarification and validator tool events. |
| plan | Contains the schedule, totals and rationale. | Must map to a successful validator call for complete. |
| sources | Links current claims to the web research. | Collected from grounded Search responses. |
| questions | Requests missing essentials. | Requires a recorded clarification tool call. |
| constraint_failures | Explains why no plan works. | Requires a failed deterministic validator result. |
04 · Learnings
The useful work happened between the first pass and the final pass.
No model training occurred. We iterated on the prompt, tools, decision gates, response boundary and evals. Each failure exposed a reusable design lesson.
A 5/5 grade did not mean valid JSON
The 12:01 run passed every semantic judge. A later raw-output audit found malformed JSON in the wet-weather response. That changed the eval contract: semantic quality, served-schema validity and trace integrity are separate checks.
- TOOL COMPATIBILITY
Mixed built-in Search and function tools failed
Gemini returned MALFORMED_FUNCTION_CALL. Wrapping grounded Search as a normal function tool produced stable interoperability and preserved citations.
- SCHEMA REALITY
Strict output types exposed framework edges
An HttpUrl field failed ADK JSON serialisation. JSON response MIME plus function tools was also unsupported. The design moved strictness to the service boundary.
- SAFE DEFAULTS
The finish call omitted fields humans consider obvious
Total cost, nullable values and empty lists were sometimes absent. Defaults now cover harmless omissions; derived totals come from validated schedule items.
- DECISION GATES
A structured finish tool could bypass the work
The agent could jump straight to a polished answer without searching, validating or recording clarification. The finish guard made those steps part of the contract.
- FORCING DID NOT HELP
Function-calling mode ANY created malformed finish calls
The run returned to AUTO. Tool visibility and post-run trace checks provided stronger guarantees.
- LATENCY BUDGET
Mandatory alternatives made the agent wander
The heavier alternative gate hit a 180-second timeout. A valid primary plan can now finish; an alternative is optional and limited by the remaining search budget.
- TRACE RECONCILIATION
The final wording drifted from the validated schedule
Venue labels, cost or travel could change after validation. The service matches the response back to a unique validator tool call and restores the proven values.
- OUTPUT RESILIENCE
One good response ended one brace early
A narrow end-of-file repair closes only missing JSON containers. Broader syntax errors still fail, which avoids turning a repair step into a second model.
- ORDER OF REASONING
An explicit contradiction was mistaken for missing context
The agent asked for a location before noticing that two 60-minute activities could not fit into 30 minutes. Explicit impossibility now takes priority over clarification.
- REGRESSION LOOP
Every live failure became a deterministic test
The suite grew to 37 tests across validation, guardrails, response quality, service reconciliation and the FastAPI boundary.
What carries into the next agent
These are the design choices worth reusing, even when the use case changes.
Grade three layers
Judge meaning, validate the served contract and inspect the trace. Any one layer on its own can hide a bad run.
Put hard constraints in code
Let the model explore and explain. Use deterministic functions for arithmetic, timing, permissions and state transitions.
Bound every network tool
Set timeouts and budgets, and count successful external calls separately from attempted tool calls.
Keep the raw trace
Saved events made it possible to distinguish model reasoning, tool behaviour and response-boundary bugs.
Guard the decision order
Checking only the final prose is too late. Constrain which outcomes become available after which evidence.
Make optional work optional
Secondary alternatives add value. A correct primary answer can return without waiting for them.
05 · Result
A working local agent with an auditable release gate.
The final run passed the three designed behaviours with no critical failures. The API, tests and saved traces are ready for workshop exploration. Deployment remains future work.
Prototype gate passed
All automated checks and minimum eval scores were met on the final recorded run.
37automated tests passed
3/3eval cases passed
0critical failures
5successful web searches in the grounded case
Final eval, 12:46 pm
Each score is out of five. The release threshold was four with no critical failure.
| Case | Score | Critical | Observed tool path |
|---|---|---|---|
| Wet-weather family plan | 4/5 | No | Date, then Search ×5, then Validator, then Finish |
| Impossible schedule | 5/5 | No | Validator, then Finish |
| Missing essentials | 5/5 | No | Record clarification |
Verified
- Gemini API key smoke test returned OK
- gemini-2.5-flash configured
- us-central1 configured for future Vertex use
- Lint, formatting, spelling and type checks passed
- POST /v1/day-plans, /health and /docs returned HTTP 200
- Every final served response parsed as JSON
Still outside the claim
- No cloud deployment has been attempted
- No load, security or long-running reliability test
- No booking, calendar or other state-changing tools
- No broad eval set beyond the three planned behaviours
- No custom user interface beyond FastAPI docs
Review the evidence
The report links to the current project files and final saved run.
How this report was made
Skills used: mds-to-html for the multi-tab report workflow, design-system for tokens and component recipes, remove-ai-tells for the final prose edit, and browser:control-in-app-browser for interaction and visual QA.