๐Ÿค– InteractiveAI Platform ยท Full Lifecycle

Support Agent: End-to-End on InteractiveAI

Configured and deployed an AI customer support agent exploring the complete InteractiveAI platform lifecycle: Build context, Connect tools, Deploy via Interactive Agent, Govern with traces, and Improve with datasets.

InteractiveAI YAML Policies Routines Glossary MCP Tools Traces Datasets
๐Ÿ—๏ธ
Build
๐Ÿ”—
Connect
๐Ÿš€
Deploy
๐Ÿ‘๏ธ
Govern
๐Ÿ“ˆ
Improve
01: Context & Objective

A hands-on learning project for the InteractiveAI platform

InteractiveAI is the operating platform for building, deploying, and managing production AI systems. It provides end-to-end infrastructure for the complete agentic AI lifecycle: from tracing LLM interactions and managing prompts to evaluating output quality and monitoring performance across development and production environments.

This project was designed as a focused, hands-on exercise to learn the platform's architecture by building something real: a fictional SaaS customer support agent that can onboard users, answer technical questions, and handle mock ticket data โ€” all while enforcing brand rules.

Why this project matters The InteractiveAI platform is structured around five core lifecycle stages โ€” Build, Connect, Deploy, Govern, Improve. This project touches every single one, giving end-to-end experience with the platform's architecture in a single focused build.
๐Ÿ—๏ธ
Build
Define agent behaviour
๐Ÿ”—
Connect
Integrations & secrets
๐Ÿš€
Deploy
Production infrastructure
๐Ÿ‘๏ธ
Govern
Observability & tracing
๐Ÿ“ˆ
Improve
Evaluate & iterate
5
Platform lifecycle stages covered
4
Context components built
~3h
End-to-end build time
02: Build โ€” Defining Agent Context

Decoupling application logic from behavioral context

InteractiveAI's architecture fundamentally separates what the agent does (routines) from how it behaves (policies). The Build > Context section is where you define all four components that shape the agent's personality, knowledge, and workflows.

YAML

Policy โ€” Behavior Rulebook

Condition โ†’ action rules matched against every turn. Encodes safety, compliance, tone, and business rules that apply across all routines (e.g. "never guess if the info isn't provided.").

YAML

Routine โ€” Workflow Graph

Graphs of chat, tool, and fork nodes connected by transitions. These define the branching logic for intents like billing, technical support, or onboarding.

JSON

Glossary โ€” Domain Terms

Injects domain vocabulary into every turn. Used to define product-specific jargon so the agent understands the exact terminology used by customers.

Markdown

Macro โ€” Reusable Templates

Text blocks interpolated into routine chat nodes via ${macro-id}. Used for personalized greetings or standardized disclosure texts.

Sample: Policy YAML

# policy: support-guardrails
name: support-guardrails
description: Core behavioral rules for the SaaS support agent

conditions:
  - type: always  # applies to every turn

actions:
  - type: instruct
    content: |
      You are a helpful, technically-precise SaaS support agent.
      - Always be professional and empathetic.
      - NEVER guess or fabricate answers. If information is not
        available in your context, say so clearly.
      - Cite the specific product feature or documentation
        when answering technical questions.
      - If the user appears frustrated, acknowledge their
        frustration before providing a solution.

Sample: Routine YAML with branching logic

# routine: customer-support-flow
name: customer-support-flow
description: Main support routine with intent-based branching

nodes:
  - id: greeting
    type: chat
    content: "${welcome-greeting}"
    transitions:
      - target: billing-help
        condition: "user asks about billing, pricing, or invoices"
      - target: technical-help
        condition: "user asks a technical question"
      - target: onboarding
        condition: "user is new or asks about getting started"

  - id: billing-help
    type: chat
    content: |
      I'd be happy to help with your billing question.
      Let me pull up your account details.
    transitions:
      - target: resolve
        condition: "user confirms issue is resolved"

  - id: technical-help
    type: chat
    content: |
      Let me look into that technical issue for you.
      Can you share any error messages or steps to reproduce?
    transitions:
      - target: resolve
        condition: "user confirms issue is resolved"
      - target: escalate
        condition: "issue requires engineering escalation"

  - id: onboarding
    type: chat
    content: |
      Welcome! I'll walk you through getting started
      with our platform. Let's begin with setup.

  - id: escalate
    type: tool
    tool: support-tools:create_ticket
    content: "Creating a support ticket for the engineering team."

  - id: resolve
    type: chat
    content: "Glad I could help! Is there anything else?"
Architecture insight The platform renders routines as interactive visual diagrams, making it easy to trace conversation paths and debug unexpected agent behavior. Policies sit on top of routines โ€” they fire on every turn regardless of which routine node is active, acting as guardrails across the entire agent.
03: Connect โ€” Tools & Secrets

External integrations and secure credential management

The Connect section handles everything the agent needs to interact with the outside world: tool servers (via MCP), a knowledge base for retrieval grounding, and a secrets manager for API keys and credentials.

๐Ÿ”ง MCP Tool Servers

Tools are functions the agent calls at runtime, served by your MCP servers. The platform allows connecting external APIs for fetching user profiles or creating support tickets in Zendesk/Jira.

๐Ÿ” Secrets Manager

Project-scoped secrets store API keys and credentials securely. Secrets are injected as environment variables at deployment time โ€” the agent code never sees raw keys.

๐Ÿ“š Knowledge Base

Retrieval grounding via pgvector or an external HTTP search endpoint. The agent can search this on every turn to ground answers in actual documentation rather than hallucinating.

๐Ÿค– LLM Router

The agent uses two independent model lanes โ€” chat (customer-facing) and evaluation (internal decisions) โ€” each with its own primary and fallback. The lanes never cross.

Security detail worth mentioning API keys and secrets are scoped to the Project level, not the Organization level. This means different projects within the same org have fully isolated credential namespaces โ€” a significant enterprise security pattern. The Secrets Manager uses bundle secrets that are auto-attached during first deployment.
# Mock tool server declaration in the agent manifest
mcps:
  - name: support-tools
    url: https://my-mcp-server.example.com
    api_key:
      env_ref: SUPPORT_TOOLS_API_KEY   # resolves from Secrets Manager
    tools:
      - name: get_user
        description: "Fetch user profile by email or ID"
      - name: create_ticket
        description: "Create a support ticket for engineering escalation"
04: Deploy โ€” Interactive Agent

From configuration to running service in 5 steps

The Interactive Agent deployment is where agent configurations become running services. The platform's deployment wizard walks through five steps: Identity, Endpoint, Runtime, Schedule, and Review.

๐Ÿ“‹ Step 1: Identity

Agent configuration name, instance name (locked once live), container image, and image version from the operator catalog.

๐ŸŒ Step 2: Endpoint

Toggle public URL on/off. When enabled, the agent gets a publicly reachable hostname โ€” ours is test-agent-1xmcc8.interactive.ai.

โš™๏ธ Step 3: Runtime

Environment variables, Router API Key (outbound LLM billing), Agent API Key (inbound auth), and project secrets attachment.

๐Ÿ“… Step 4: Schedule

Optional uptime windows with cron-like expressions (e.g., Mon-Fri 09:00-18:00) and IANA timezone. Scale to zero outside the window to save cost.

First deploy secret On first deploy, the platform auto-creates a bundle secret named router-<instance-name> containing the Router API Key, Agent API Key, and platform credentials. This bundle is attached automatically even if you skip the Secrets step โ€” a great reliability pattern.

Try the live agent

The agent is deployed and running at its public endpoint. Interact with it below โ€” it's the same agent served by InteractiveAI's built-in chat UI:

test-agent-1xmcc8.interactive.ai

Live InteractiveAI agent โ€” sign in with the Agent API Key to start a conversation.

05: Govern โ€” Observability & Tracing

Complete visibility into what the AI system is doing

The Govern section is where enterprise AI platforms shine โ€” and it's a critical interview topic. Every agent interaction generates a hierarchical trace that you can inspect in detail.

๐Ÿ” Sessions

Each conversation is a session. Navigate to Govern > Sessions to see every user interaction, with timestamps, message counts, and status.

๐Ÿ“Š Traces

Hierarchical call traces show the full execution flow. Open a trace to see which tool was called, how long each step took (latency), and how many tokens were consumed.

๐Ÿ”ฌ Observations

Individual steps within a trace. Each observation captures the inputs, outputs, model used, duration, and token usage โ€” the atomic unit of debugging.

โญ Scores

Attach quality scores (manual or automated) to traces and observations. Enables trend tracking: is the agent getting better or worse over time?

Debugging workflow When something goes wrong: (1) Find the session in Govern โ†’ Sessions. (2) Open the trace to see the full execution flow. (3) Drill into specific observations to see exact tool inputs/outputs. (4) Check token counts and latency to identify bottlenecks. This is the exact flow an FDE would walk a customer through.
# Using the InteractiveAI SDK to retrieve traces programmatically
from interactive_ai import InteractiveAI

client = InteractiveAI()

# List recent traces for debugging
traces = client.traces.list(
    project_id="my-project",
    limit=10,
    order_by="created_at",
    order="desc"
)

for trace in traces:
    print(f"Trace: {trace.id}")
    print(f"  Duration: {trace.duration_ms}ms")
    print(f"  Tokens: {trace.total_tokens}")
    print(f"  Status: {trace.status}")
    
    # Drill into observations
    for obs in trace.observations:
        print(f"    [{obs.type}] {obs.name}: {obs.duration_ms}ms")
06: Improve โ€” Datasets & Evaluation

Systematic quality evaluation and regression testing

The Improve section closes the loop: take real production traces, turn them into test cases, and run evaluations before shipping changes. This is how you ensure zero regressions when updating prompts or context.

๐Ÿ“‹ Datasets

Navigate to Improve > Datasets and create a test suite. Map user inputs to expected agent outputs. Each dataset item is a regression test for your agent.

๐Ÿ”„ Trace-to-Dataset Pipeline

Go to a successful production trace in Govern, click "Add to Dataset", and map the user's input + agent's output into your test suite. Real conversations become regression tests.

๐Ÿงช Experiments

Run your dataset against different agent configurations. Compare results side-by-side to validate that a new policy or routine doesn't break existing behavior.

๐Ÿ“ Annotations

Add human annotations to traces and observations. Comments on specific agent responses create an audit trail and inform future prompt tuning.

The production feedback loop The killer feature: production traces feed directly into test datasets. When you update a policy or add a routine, you run the dataset to verify every known-good conversation still works. This is the enterprise answer to "how do you test prompt changes?" โ€” and it's a key differentiator to articulate in an interview.
07: Architecture & Security

Frontend Integration & Cross-Origin Security

Integrating third-party enterprise platforms into a portfolio site introduces real-world security challenges. I designed this implementation to bypass modern browser cookie restrictions while maintaining a seamless user experience.

๐Ÿ”’ The Third-Party Cookie Problem

Modern browsers (Chrome, Safari) block cross-origin cookies in <iframe> embeds (SameSite=Lax/Strict). This breaks authentication when embedding the InteractiveAI test portal directly inside a standard web page widget.

๐Ÿ›ก๏ธ The Sized Popup Solution

Instead of a fragile iframe or a complex reverse proxy, I engineered a Sized Popup Widget. Clicking the red floating bubble spawns a new OS-level window sized exactly like a widget. The browser treats it as a top-level secure context, allowing cookies to flow perfectly while preserving the "widget" feel.

Integration Architecture Diagram

sequenceDiagram autonumber actor User participant Portfolio as cv.manuelmezo.com participant Widget as Sized Popup Window participant IAI as test-agent-1xmcc8 participant Router as InteractiveAI Router User->>Portfolio: Clicks red floating bubble Portfolio->>Widget: window.open(url, width=400, height=600) Note over Widget: Top-Level Secure Context Widget->>IAI: GET /chat/ IAI-->>Widget: Returns React App + Login UI User->>Widget: Enters Agent API Key Widget->>IAI: POST /login (Sets First-Party Cookie) IAI-->>Widget: 200 OK + Session Token Widget->>Router: WebSocket / Streaming Chat Router-->>Widget: AI Response
Why not a Reverse Proxy? I initially prototyped a Firebase Cloud Function reverse proxy to trick the browser into thinking the API was first-party. However, since the InteractiveAI portal is a compiled React app that uses absolute URLs for its internal API calls, the proxy broke client-side routing. The sized popup is the most robust, enterprise-ready fallback for external developer portals.
Resources ๐Ÿ“– InteractiveAI Documentation  ยท  ๐Ÿค– Live Agent Endpoint  ยท  ๐Ÿ—๏ธ Quickstart Guide

โ† Back to Portfolio