Five code changes cut AI agent token waste and stop hallucinations

Prompt engineering has hit its ceiling. Relying on an LLM to follow complex business rules through natural language instructions is a recipe for system instability. When you tell an agent in a system prompt to "never book more than 10 guests," or "only use the payment tool after verifying a balance," the model processes these rules probabilistically. They are suggestions, not absolute logical constraints. Under load, or when faced with unexpected inputs, the model will inevitably bypass them.

Every token you feed an agent costs money. If your application packs dozens of tool definitions, deep conversation histories, and exhaustive instructions into every single API request, your bills skyrocket while accuracy nosedives. A bloated context window leads directly to retrieval degradation and high latency. When the model receives too much irrelevant information, it hallucinates.

To build production-grade agentic applications, you must move beyond the prompt. By implementing structural code changes in your application architecture, you can enforce strict execution boundaries, dynamically filter resources, and guarantee correctness. Using Strands agent, an open-source agent framework maintained on AWS, we can solve these reliability problems at the code level.

Prerequisites and setup

To follow this guide, you should have a solid foundation in Python, basic familiarity with vector embeddings, and an understanding of how LLMs use tools (function calling).

We will use the following core libraries and services:

• Strands Agent Framework: An open-source Python framework from AWS designed to build, manage, and scale agentic workflows.
• OpenAI API: Used as the base LLM generator (though Strands supports any model provider, including Amazon Bedrock).
• Neo4j: A graph database used to execute structured Cypher queries for GraphRAG.
• FAISS & Sentence Transformers: Used to build and search local, free vector indexes for semantic tool selection.

To initialize your environment, install the required dependencies and configure your environment keys:

# Install the core dependencies
!pip install strands-agent openai neo4j sentence-transformers faiss-cpu python-dotenv

import os
from dotenv import load_dotenv

load_dotenv()
os.environ["OPENAI_API_KEY"] = "your-api-key-here"

Stop paying for unused tools with semantic selection

Imagine your agent has access to 29 different tools handling flights, hotel bookings, weather, payments, and cancellations. In a traditional architecture, every single user message sends the JSON schemas for all 29 tools into the LLM's context window.

# The traditional approach: bloating the context window
from strands import Agent
from my_tools import all_29_tools

# All 29 schemas are sent on every single user interaction
naive_agent = Agent(tools=all_29_tools, model="gpt-4o")
response = naive_agent.run("Is it raining in Lisbon?")

A single tool schema averages 100 to 200 tokens. Packing 29 tools into the context adds up to roughly 3,000 to 4,000 tokens per call before the user even types a single word. You pay for these system tokens on every turn.

We can fix this by building a local vector database of our tool descriptions. When a user sends a query, we perform a quick semantic search against this database, retrieve only the top 3 most relevant tools, and dynamically swap them into the active agent context. This slashes our token footprint from thousands to under 300 tokens.

Here is how to implement semantic tool selection in code using Strands:

from sentence_transformers import SentenceTransformer
import faiss
import numpy as np
from strands import Agent, tool

# Initialize local embedding model
embed_model = SentenceTransformer('all-MiniLM-L6-v2')

# Define our tools with detailed docstrings
@tool
def get_weather(location: str):
    """Retrieve the current weather for a specific city."""
    return f"The weather in {location} is sunny and 22C."

@tool
def book_hotel(hotel_name: str, guests: int):
    """Book a hotel room for a specific number of guests."""
    return f"Successfully booked {hotel_name} for {guests} guests."

# ... define the rest of your tools ...
tool_registry = {get_weather.name: get_weather, book_hotel.name: book_hotel}
tool_descriptions = [t.description for t in tool_registry.values()]
tool_names = list(tool_registry.keys())

# Create a local FAISS index of tool descriptions
embeddings = embed_model.encode(tool_descriptions)
dimension = embeddings.shape[1]
index = faiss.IndexFlatL2(dimension)
index.add(np.array(embeddings).astype('float32'))

def get_relevant_tools(user_query: str, k: int = 2):
    query_vector = embed_model.encode([user_query])
    distances, indices = index.search(np.array(query_vector).astype('float32'), k)
    selected_names = [tool_names[idx] for idx in indices[0]]
    return [tool_registry[name] for name in selected_names]

# Execution cycle with dynamic tool loading
query = "I need to find a place to stay in Lisbon"
active_tools = get_relevant_tools(query, k=1)

# We build the agent dynamically with only the necessary tools
dynamic_agent = Agent(tools=active_tools, model="gpt-4o")
response = dynamic_agent.run(query)
print(f"Active tools: {[t.name for t in active_tools]}")
print(response)

If you want to maintain a conversation, you can use the swap_tools hook to dynamically clear out unused tools and load new ones on each agent loop, ensuring the history stays clean and cheap.

Replace vector similarity guesses with Neo4j GraphRAG queries

Traditional Retrieval-Augmented Generation (RAG) relies on vector search to find relevant text chunks. This works wonderfully for open-ended questions like "tell me about our cancellation policy." However, it fails spectacularly when asked structural questions that require calculation or aggregation over a dataset.

If you ask a standard RAG system "What is the average hotel rating in Paris?" or "How many hotels offer a pool?" the vector database will return the top three most similar matching text fragments. The LLM then attempts to calculate the answer based on that random slice of data. It is forced to estimate, guess, and present those hallucinations as ground-truth facts.

GraphRAG addresses this by building a structured knowledge graph in a graph database like Neo4j. Instead of relying on similarity search, the agent uses the LLM to generate a precise Cypher database query. This query executes directly on Neo4j, returning a computed, mathematically correct answer.

from neo4j import GraphDatabase
from strands import Agent, tool

# Connect to our local Neo4j instance
neo4j_driver = GraphDatabase.driver("bolt://localhost:7687", auth=("neo4j", "password"))

@tool
def query_knowledge_graph(cypher_query: str):
    """
    Execute a Cypher query against the Neo4j database to get structured information.
    Always use this tool when asked for aggregations, counts, or multi-hop relationship details.
    """
    with neo4j_driver.session() as session:
        result = session.run(cypher_query)
        records = [record.data() for record in result]
        return str(records)

system_prompt = """
You are an analytical travel assistant. When asked about counts, averages, ratings, 
or relationships between entities, write a precise Cypher query and execute it using 
the query_knowledge_graph tool. 

Schema details:
Nodes: (:Hotel {name: string, rating: float, city: string}), (:Amenity {name: string})
Relationships: (:Hotel)-[:OFFERS]->(:Amenity)
"""

agent = Agent(tools=[query_knowledge_graph], system_prompt=system_prompt, model="gpt-4o")

# Under the hood, this generates: 
# MATCH (h:Hotel {city: 'Paris'}) RETURN avg(h.rating) as avg_rating
response = agent.run("What is the average rating of hotels in Paris?")
print(response)

Using GraphRAG, the agent receives the exact computed mathematical average (e.g., 4.7) directly from the database engine. The LLM simply formats the raw, deterministic result for the user, removing any opportunity for math calculations or hallucination.

Catch silent failures with multi-agent validator swarms

Often, when a single agent calls a tool and receives an error, it tries to self-rationalize the failure. Instead of telling you the booking failed, it might generate a highly confident message stating that your room is secured. This happens because the agent acts and validates its actions inside the same cognitive loop.

We can decouple this by introducing a multi-agent validation chain. Using a swarm pattern, we split the responsibilities among three distinct agents: the Executor, the Validator, and the Critic.

from strands import Swarm, Agent

# Agent 1: Performs the requested action
executor = Agent(
    name="Executor",
    system_prompt="Fulfill the hotel booking request using the provided tools.",
    tools=[book_hotel]
)

# Agent 2: Checks the tool outputs for errors or inconsistencies
validator = Agent(
    name="Validator",
    system_prompt="""
    Review the actions taken by the Executor. Check if any tools returned errors. 
    Output 'VALID' if everything succeeded, or 'INVALID: <reason>' if something failed.
    """
)

# Agent 3: Approves the output or flags the failure
critic = Agent(
    name="Critic",
    system_prompt="""
    Review the Validator report. If VALID, approve the user response. 
    If INVALID, output a clear failure message explaining why the action could not be completed. 
    Never fabricate a success confirmation.
    """
)

# Connect them together into an automated Swarm chain
swarm = Swarm(
    agents=[executor, validator, critic],
    entry_point=executor
)

response = swarm.run("Book the luxury hotel for 15 guests")
print(response)

If the booking database rejects the request (e.g., due to overcapacity), the Executor receives an error. While a single agent might cover this up, the Validator will spot the error, flag it to the Critic, and the Critic will deliver a clear rejection back to the user.

Enforce absolute business logic with neuro-symbolic Python guardians

What happens when you have non-negotiable business rules? For example, "Never confirm a booking until payment has been processed," or "Do not allow reservations to exceed 10 guests."

If you write this rule in the system prompt, a clever prompt injection or a long conversation history can cause the model to ignore it. A neuro-symbolic guardian enforces these boundaries in your application's Python layer using Strands lifecycle hooks. The agent cannot bypass Python code.

from strands import Agent, hook_provider, before_tool_call

class BookingGuardrails:
    @hook_provider
    def register_guardrails(self, register):
        # Register a callback before the 'book_hotel' tool executes
        register(before_tool_call("book_hotel"), self.enforce_capacity_rule)

    def enforce_capacity_rule(self, event):
        # Extract parameters directly from the intercepted tool call
        guests = event.args.get("guests", 0)
        if guests > 10:
            # Raise an exception to halt execution immediately
            raise ValueError(f"Execution blocked: Bookings cannot exceed 10 guests. Attempted: {guests}")

# Instantiate the agent with the Python guardrail attached
guardian_agent = Agent(
    tools=[book_hotel],
    hooks=[BookingGuardrails()],
    model="gpt-4o"
)

try:
    # This call will hit our Python guardrail and throw an error before running the tool
    guardian_agent.run("Book the Grand Hotel for 12 guests")
except Exception as e:
    print(e)  # "Execution blocked: Bookings cannot exceed 10 guests. Attempted: 12"

Keep tasks alive using soft-constraint runtime steering

Hard blocks (like our hook guardian above) are ideal for strict security and legal rules. However, they provide a frustrating user experience for soft rules. If a user tries to book a room for 12 guests, you don't necessarily want to terminate their session. Instead, you want to steer the agent to propose a compromise—like booking two separate rooms.

Using runtime steering, when a rule is violated, we don't abort. Instead, we insert a correction feedback injection directly into the model's generation stream, forcing it to adapt and finish the task.

# Implementing a steering controller for soft-constraints
def steer_booking(event):
    guests = event.args.get("guests", 0)
    if guests > 10:
        # Instead of throwing an error, we inject a corrective instruction into the context
        event.steer("""
        The maximum capacity per reservation is 10 guests. 
        Do not execute the booking tool directly with your current arguments. 
        Instead, propose splitting the reservation into multiple smaller bookings and ask the user if they agree.
        """)

By leveraging steering, the agent gracefully handles soft boundary failures, correcting itself in real-time without crashing or requiring manual human intervention.

Moving from local experiments to AWS production

While running Jupyter Notebooks locally is excellent for prototyping, production environments require robust API gateways, long-term memory, observability, and strict security controls.

By transitioning your Python designs to Amazon Bedrock, you offload your infrastructure overhead. You can register your Strands tools as AWS Lambda functions, store your dynamic steering and validation policies in Amazon DynamoDB, and use Neo4j AuraDB as your managed cloud Graph database. This setup ensures your agents run fast, cheap, and deterministically, scale with zero server management, and remain entirely isolated from hallucination loops.