Configured and deployed an AI customer support agent exploring the complete InteractiveAI platform lifecycle: Build context, Connect tools, Deploy via Interactive Agent, Govern with traces, and Improve with datasets.
InteractiveAI is the operating platform for building, deploying, and managing production AI systems. It provides end-to-end infrastructure for the complete agentic AI lifecycle: from tracing LLM interactions and managing prompts to evaluating output quality and monitoring performance across development and production environments.
This project was designed as a focused, hands-on exercise to learn the platform's architecture by building something real: a fictional SaaS customer support agent that can onboard users, answer technical questions, and handle mock ticket data โ all while enforcing brand rules.
InteractiveAI's architecture fundamentally separates what the agent does (routines) from how it behaves (policies). The Build > Context section is where you define all four components that shape the agent's personality, knowledge, and workflows.
Condition โ action rules matched against every turn. Encodes safety, compliance, tone, and business rules that apply across all routines (e.g. "never guess if the info isn't provided.").
Graphs of chat, tool, and fork nodes connected by transitions. These define the branching logic for intents like billing, technical support, or onboarding.
Injects domain vocabulary into every turn. Used to define product-specific jargon so the agent understands the exact terminology used by customers.
Text blocks interpolated into routine chat nodes via ${macro-id}. Used for personalized greetings or standardized disclosure texts.
# policy: support-guardrails
name: support-guardrails
description: Core behavioral rules for the SaaS support agent
conditions:
- type: always # applies to every turn
actions:
- type: instruct
content: |
You are a helpful, technically-precise SaaS support agent.
- Always be professional and empathetic.
- NEVER guess or fabricate answers. If information is not
available in your context, say so clearly.
- Cite the specific product feature or documentation
when answering technical questions.
- If the user appears frustrated, acknowledge their
frustration before providing a solution.
# routine: customer-support-flow
name: customer-support-flow
description: Main support routine with intent-based branching
nodes:
- id: greeting
type: chat
content: "${welcome-greeting}"
transitions:
- target: billing-help
condition: "user asks about billing, pricing, or invoices"
- target: technical-help
condition: "user asks a technical question"
- target: onboarding
condition: "user is new or asks about getting started"
- id: billing-help
type: chat
content: |
I'd be happy to help with your billing question.
Let me pull up your account details.
transitions:
- target: resolve
condition: "user confirms issue is resolved"
- id: technical-help
type: chat
content: |
Let me look into that technical issue for you.
Can you share any error messages or steps to reproduce?
transitions:
- target: resolve
condition: "user confirms issue is resolved"
- target: escalate
condition: "issue requires engineering escalation"
- id: onboarding
type: chat
content: |
Welcome! I'll walk you through getting started
with our platform. Let's begin with setup.
- id: escalate
type: tool
tool: support-tools:create_ticket
content: "Creating a support ticket for the engineering team."
- id: resolve
type: chat
content: "Glad I could help! Is there anything else?"
The Connect section handles everything the agent needs to interact with the outside world: tool servers (via MCP), a knowledge base for retrieval grounding, and a secrets manager for API keys and credentials.
Tools are functions the agent calls at runtime, served by your MCP servers. The platform allows connecting external APIs for fetching user profiles or creating support tickets in Zendesk/Jira.
Project-scoped secrets store API keys and credentials securely. Secrets are injected as environment variables at deployment time โ the agent code never sees raw keys.
Retrieval grounding via pgvector or an external HTTP search endpoint. The agent can search this on every turn to ground answers in actual documentation rather than hallucinating.
The agent uses two independent model lanes โ chat (customer-facing) and evaluation (internal decisions) โ each with its own primary and fallback. The lanes never cross.
# Mock tool server declaration in the agent manifest
mcps:
- name: support-tools
url: https://my-mcp-server.example.com
api_key:
env_ref: SUPPORT_TOOLS_API_KEY # resolves from Secrets Manager
tools:
- name: get_user
description: "Fetch user profile by email or ID"
- name: create_ticket
description: "Create a support ticket for engineering escalation"
The Interactive Agent deployment is where agent configurations become running services. The platform's deployment wizard walks through five steps: Identity, Endpoint, Runtime, Schedule, and Review.
Agent configuration name, instance name (locked once live), container image, and image version from the operator catalog.
Toggle public URL on/off. When enabled, the agent gets a publicly reachable hostname โ ours is test-agent-1xmcc8.interactive.ai.
Environment variables, Router API Key (outbound LLM billing), Agent API Key (inbound auth), and project secrets attachment.
Optional uptime windows with cron-like expressions (e.g., Mon-Fri 09:00-18:00) and IANA timezone. Scale to zero outside the window to save cost.
router-<instance-name> containing the Router API Key, Agent API Key, and platform credentials. This bundle is attached automatically even if you skip the Secrets step โ a great reliability pattern.
The agent is deployed and running at its public endpoint. Interact with it below โ it's the same agent served by InteractiveAI's built-in chat UI:
Live InteractiveAI agent โ sign in with the Agent API Key to start a conversation.
The Govern section is where enterprise AI platforms shine โ and it's a critical interview topic. Every agent interaction generates a hierarchical trace that you can inspect in detail.
Each conversation is a session. Navigate to Govern > Sessions to see every user interaction, with timestamps, message counts, and status.
Hierarchical call traces show the full execution flow. Open a trace to see which tool was called, how long each step took (latency), and how many tokens were consumed.
Individual steps within a trace. Each observation captures the inputs, outputs, model used, duration, and token usage โ the atomic unit of debugging.
Attach quality scores (manual or automated) to traces and observations. Enables trend tracking: is the agent getting better or worse over time?
# Using the InteractiveAI SDK to retrieve traces programmatically
from interactive_ai import InteractiveAI
client = InteractiveAI()
# List recent traces for debugging
traces = client.traces.list(
project_id="my-project",
limit=10,
order_by="created_at",
order="desc"
)
for trace in traces:
print(f"Trace: {trace.id}")
print(f" Duration: {trace.duration_ms}ms")
print(f" Tokens: {trace.total_tokens}")
print(f" Status: {trace.status}")
# Drill into observations
for obs in trace.observations:
print(f" [{obs.type}] {obs.name}: {obs.duration_ms}ms")
The Improve section closes the loop: take real production traces, turn them into test cases, and run evaluations before shipping changes. This is how you ensure zero regressions when updating prompts or context.
Navigate to Improve > Datasets and create a test suite. Map user inputs to expected agent outputs. Each dataset item is a regression test for your agent.
Go to a successful production trace in Govern, click "Add to Dataset", and map the user's input + agent's output into your test suite. Real conversations become regression tests.
Run your dataset against different agent configurations. Compare results side-by-side to validate that a new policy or routine doesn't break existing behavior.
Add human annotations to traces and observations. Comments on specific agent responses create an audit trail and inform future prompt tuning.
Integrating third-party enterprise platforms into a portfolio site introduces real-world security challenges. I designed this implementation to bypass modern browser cookie restrictions while maintaining a seamless user experience.
Modern browsers (Chrome, Safari) block cross-origin cookies in <iframe> embeds (SameSite=Lax/Strict). This breaks authentication when embedding the InteractiveAI test portal directly inside a standard web page widget.
Instead of a fragile iframe or a complex reverse proxy, I engineered a Sized Popup Widget. Clicking the red floating bubble spawns a new OS-level window sized exactly like a widget. The browser treats it as a top-level secure context, allowing cookies to flow perfectly while preserving the "widget" feel.