AI Agent Platform Case study

Agent.Press

The agent platform the engagement is built on: an LLM agent framework with an admin console, an extensible tool system and retrieval pipelines for running production assistants.

AI Agent Platform

Agent.Press Agent.Press

Senior Full-Stack AI Engineer · Architecture & Platform

System map · Agent runtime

Production scope

  • One runtime, several products ERP automation, reporting workflows and an internal assistant
  • Packages, not forks One fix reaches every client system

Agent request path

  1. Request

    Console or embedded client

  2. Context

    Conversation state assembled

  3. Agent runtime

    Provider-agnostic model turn

  4. Capability

    Retrieval

    Tools

    MCP

  5. Validation

    Results checked against a schema

  6. Result

    Returned into the conversation

Another turn while the answer is incomplete

Shared platform

  • Published packages, not per-client forks
  • Provider-agnostic model layer
  • Console and embedded clients
  • Request tracing

Engineering

TypeScript · Express · PostgreSQL · Vector retrieval · MCP · OpenTelemetry

Public product · Proprietary implementation omitted

Role
Senior Full-Stack AI Engineer · Architecture & Platform
Disclosure
Public product · Proprietary implementation omitted

The problem

What the system had to solve.

Every engagement needed the same foundation underneath it: retrieval, tool execution, conversation state and a console to configure and watch an agent from. Rebuilding that per client is where reliability goes to die.

What I owned

The responsibility behind the work.

Architected platform capabilities as a shared foundation and delivered client systems on top of them, so the same agent runtime carries several production products.

The Sage Intacct automation for Reis Contracting ships against the Agent.Press API, library and interface packages, so platform work and product work moved together rather than drifting apart.

Engineering approach

How it was built.

Worked across the agent runtime and its surfaces: configuration and conversation state, an extensible tool system, retrieval pipelines over a vector-enabled Postgres store, and the console and drop-in chat clients above the API.

Agent request path

  1. Request
  2. Context
  3. Tools
  4. Validation
  5. Result

The same foundation, every time

What every agent engagement rebuilds.

Every production assistant needs the same substrate before it can do anything client-specific: somewhere to keep conversation state, a way to give the model tools and validate what comes back, retrieval over documents the client owns, and a console someone non-technical can configure and watch it from.

Built per engagement, that substrate is where reliability goes to die. Four clients means four subtly different retry behaviours, four tool contracts, four ways of handling a model that returns something unexpected, and four codebases to fix when one of them turns out to be wrong.

Two ways to run several client agents

Deliberately avoided

A fork per client.

  1. Client A fork
  2. Client B fork
  3. Divergent tool contracts
  4. A fix applied four times, or once
  5. No shared upgrade path

Platform packages

One runtime, versioned and published.

  1. Shared agent runtime
  2. Versioned package releases
  3. Client systems depend on the packages
  4. One fix reaches every client
  5. Upgrades are deliberate, per client
The right-hand shape is the one the platform exists to make possible.

Published packages, not a shared folder

How platform work and client work stay together.

The platform is a monorepo of applications and libraries with an orchestrated build. What makes it a platform rather than a large project is that the libraries are versioned and released, so a client system takes a dependency on a published version rather than on whatever is currently on the main branch.

What ships from the monorepo

Applications

  • Agent API
  • Admin console
  • Drop-in chat client
  • Customer portal
  • Documentation site

Published libraries

  • Agent runtime and API
  • Shared client library
  • Component library
  • Command-line tooling

A client system pinning a platform version is the difference between an upgrade being a decision and an upgrade being an accident.

The agent request path

What happens between a question and an answer.

A request is not a model call. It is context assembly, a model turn that may ask for tools, tool execution against real systems, validation of what comes back, and a decision about whether to loop again or answer. The runtime owns that loop so no client system has to implement it.

Request loop

  1. Request With conversation state
  2. Context assembly History and retrieved knowledge
  3. Model turn May request tool calls
  4. Tool execution Against real systems
  5. Validation Structured results checked
  6. Answer or another turn
The loop is the product. Tool results re-enter the model turn, and the cycle continues until the agent has an answer it can defend.

Answers stream to the client over a websocket rather than arriving as one response, because a multi-turn agent that has to call three tools is not something a user should watch a spinner through.

Provider-agnostic model layer

Not betting the platform on one vendor.

The runtime talks to models through one abstraction with several provider implementations behind it. OpenAI, Anthropic, Google and xAI are all reachable through the same interface, so the choice of model is configuration rather than architecture.

Model access

  1. Agent runtime One model interface
    1. OpenAI
    1. Anthropic
    1. Google
    1. xAI
One interface, several providers. A client with a data-residency or vendor constraint is a configuration change, not a rewrite.

Model choice moves faster than platforms do. Anything that hard-codes one vendor is committing to rewriting the part of the system that changes most often.

The tool system

How an agent is given something to do.

Tools are the boundary between a model and real systems, and therefore the place where an agent platform is either safe or dangerous. Each tool declares its input schema, is executed by the runtime rather than by the model, and returns a structured result that is validated before it re-enters the conversation.

Tool execution

  1. Model requests a tool with arguments
  2. Arguments validated against a schema
  3. Runtime executes the tool
  4. Result validated
  5. Result returned into the conversation
The model asks; the runtime decides and executes. A malformed or unexpected result is caught before it becomes part of an answer.

What the tool layer provides

  • Declared input schemas
  • Runtime-side execution
  • Structured, validated results
  • Per-agent tool configuration
  • Default tools including knowledge search

Model Context Protocol

Tools the platform did not have to write.

Alongside its own tools, the runtime speaks Model Context Protocol, so an agent can be given capabilities from MCP servers without those capabilities being built into the platform. A client that already exposes its systems over MCP does not need bespoke platform work to make an agent useful against them.

Two sources of capability

Platform tools

  • Written and versioned in the platform
  • Shared across client systems
  • Knowledge search and built-in capabilities

MCP servers

  • Capabilities the platform never had to implement
  • Client systems exposed on their own terms
  • Added by configuration
Both arrive at the model through the same validated tool interface.

Retrieval and the knowledge base

Grounding answers in what the client actually owns.

An agent that answers from the model alone is a liability in a business context. Retrieval grounds answers in the client’s own material, held as embeddings in the same PostgreSQL database as the rest of the platform’s state rather than in a separate vector service.

What the knowledge base holds

  • Freeform text
  • Structured tables
  • Documents, including PDFs
  • Crawled web content

Keeping embeddings in PostgreSQL means retrieval joins the same transactional store as everything else: one database to operate, one backup, one consistency story.

Retrieval path

  1. Agent decides it needs knowledge
  2. Knowledge-search tool invoked
  3. Embedding similarity over the knowledge base
  4. Relevant context returned
  5. Answer grounded in retrieved material
Search is exposed to the model as a tool, so the agent decides when it needs to look something up rather than every prompt carrying everything.

Getting knowledge in, and keeping it current

A knowledge base is a pipeline, not an upload form.

Material arrives as uploaded documents, as text entered directly, as structured tables, and as web pages that have to be fetched and extracted. Each path ends in the same place: chunked, embedded and searchable. Documents land in object storage; the extracted content is what gets embedded.

Ingestion sources

  1. Knowledge base One embedded representation
    1. Uploaded documents Including PDF extraction
    1. Freeform text
    1. Structured tables
    1. Crawled web pages
Four entry points, one representation. What differs is extraction, not what happens afterwards.

Web content goes stale. Scheduled refresh tasks re-fetch source URLs on a cron expression, so a knowledge base built from a client’s public site does not quietly become a record of what that site said last year.

Surfaces above the API

Who configures an agent, and who talks to it.

The API is the platform, but almost nobody uses an API directly. The console is where an agent is configured, its knowledge managed and its conversations inspected. The drop-in chat client is a small embeddable build that puts an agent into an existing product without that product taking on the platform.

Client surfaces

Console

  • Agent configuration
  • Knowledge management
  • Conversation inspection
  • Virtualised long lists

Embedded

  • Drop-in chat widget
  • Small standalone build
  • Streaming responses

The console is a single-page application with typed routing, server-state caching and schema-validated forms. Long conversation and knowledge lists are virtualised, because an agent that has been running for months has more history than a browser will render.

Access and API hardening

The unglamorous half of a platform.

A platform carrying several clients’ systems has to be defensible before it is clever. Sessions and OAuth are handled by a dedicated auth layer against the same database, and the API carries the ordinary protections a public surface needs.

Baseline protections

  • Session-based authentication
  • OAuth provider sign-in
  • Password hashing
  • Rate limiting
  • Security headers
  • Explicit CORS policy
  • Presigned URLs for file access

File access is granted through short-lived presigned URLs rather than by proxying downloads through the API or making a bucket readable.

Watching an agent in production

Why a non-deterministic system needs tracing.

A deterministic service can be debugged from logs and a stack trace. An agent cannot: the same question can take a different path through tools and turns on two consecutive days, and "it gave a bad answer" is not a stack trace.

The runtime is instrumented with OpenTelemetry, exporting traces and logs over OTLP, so a single conversation can be followed across its turns, its tool calls and its retrieval — which is the only way to answer why a particular answer came out the way it did.

What a trace covers

  1. Request received
  2. Context and retrieval
  3. Each model turn
  4. Each tool call and its result
  5. Response returned
Structured logging alongside tracing, so an incident starts from evidence rather than from reproduction.

Engineering scope

What I worked on.

  1. Agent runtime: conversation state, tool execution and the model loop
  2. Provider-agnostic model layer across OpenAI, Anthropic, Google and xAI
  3. Model Context Protocol support alongside built-in tools
  4. Retrieval and knowledge ingestion over a vector-enabled Postgres store
  5. Console, drop-in chat and portal surfaces above the API
  6. Versioned package releases client systems depend on
  7. OpenTelemetry tracing across the agent loop

Why these decisions mattered

The trade-offs behind the platform.

  • Versioned packages over a fork per client

    Decision
    Client systems depend on published platform releases.
    Why
    One reliability fix reaches every client, and each client still chooses when to take it.
  • Provider-agnostic model layer

    Decision
    One interface with several model providers behind it.
    Why
    Model vendors change faster than platforms. Hard-coding one guarantees a rewrite of the fastest-moving part.
  • Embeddings in PostgreSQL

    Decision
    Vector retrieval lives in the primary database.
    Why
    One store to operate, back up and reason about, instead of a second system that can silently fall out of step.
  • Tools executed by the runtime

    Decision
    The model requests; the runtime validates and executes.
    Why
    It is the only place to enforce schemas and keep an unexpected model output from reaching a real system.
  • MCP alongside platform tools

    Decision
    Speak an open protocol as well as providing built-in tools.
    Why
    Client capabilities can be added without platform work, through the same validated interface.
  • Tracing from the start

    Decision
    OpenTelemetry across the agent loop.
    Why
    A non-deterministic system cannot be debugged by reproduction; the evidence has to already exist.

Impact

The verified result.

Client systems ship against the platform packages rather than a private fork, so the ERP automation, the reporting workflows and the internal assistant share one runtime, one tool contract and one upgrade path.

Technology

The technical surface.

Runtime

  • TypeScript
  • Express
  • WebSockets

Models

  • OpenAI
  • Anthropic
  • Google
  • xAI
  • MCP

Retrieval

  • Embeddings
  • PostgreSQL vectors
  • PDF extraction
  • Web crawling

Data

  • PostgreSQL
  • Drizzle ORM
  • Object storage

Clients

  • React
  • TanStack Router
  • TanStack Query
  • Tailwind

Operations

  • OpenTelemetry
  • Structured logging
  • Turborepo
  • Versioned releases

Next: Local Falcon

Back to all work Start a conversation