AOS Documentation

Purpose & Vision

AI Orchestrator Studio (AOS) is a full-stack, self-hosted AI platform that lets organisations build, deploy, and manage autonomous AI agents capable of performing real infrastructure tasks — not just answering questions.

Traditional AI chatbots are limited to text conversations. AOS agents go further: they SSH into servers, execute SQL queries, scan documents with OCR, call REST APIs, manage Docker containers, and delegate work to other agents. Each agent is a configurable unit with its own system prompt, skill set, LLM provider, and credential bindings.

AOS is designed for enterprise IT teams, MSPs, and DevOps organisations that need AI to automate complex, multi-step operational workflows — not just generate text. The platform covers 9 enterprise domains: Infrastructure, Database Administration, Security & Compliance, Networking, DevOps, Cloud / OpenStack, Project Management, Customer Experience, and Microsoft 365 / Copilot — with 95+ built-in skills across 37 categories.

Architecture Deep-Dive

AOS follows a three-tier architecture designed for production deployment, horizontal scaling, and clear separation of concerns. Every tier can run on a separate server.

Tier 1 — Application Server

Tier 2 — Database & Cache Layer

Tier 3 — Async Workers & LLM

Agent Execution Flow

When a user sends a message, it passes through the Enterprise Orchestrator for deterministic 3-path routing (no LLM in the router), then into the UniversalAgentExecutor for the full agent pipeline.

Enterprise Orchestrator — 3-Path Deterministic Routing

UniversalAgentExecutor — Full Pipeline

1. 1. Agent Loading Load agent config from PostgreSQL: name, system_prompt, skills, parameters, LLM routing config.
2. 2. Skill Separation Loaded skills are split into two groups:
   • Internal skills (type=internal) → prompt-injected as context enrichment, NOT exposed as callable tools.
   • Callable skills (type=ssh, http, python, sql, ansible, etc.) → converted to OpenAI function-calling tool definitions.
3. 3. Credential Discovery (3 tiers) Tier 1: Agent config (agent.extra_config.credentials).
   Tier 2: RBAC bindings (credential_bindings table).
   Tier 3: Auto-discovery (scan skills → match by credential type/name).
4. 4. RAG Injection If agent has linked documents: vector search user query against DocumentChunks in ChromaDB → top-K results injected into system prompt as ### Reference Context ###. This is where .skill package reference files become searchable.
5. 5. System Prompt Assembly Final prompt is built in order: [1] Agent's base system_prompt + [2] Internal skill prompts + [3] RAG reference chunks + [4] Deep Planning instructions (if enabled) + [5] ReAct instructions (if provider doesn't support native function calling).
6. 6. ModelRouter Selection ModelRouter.select(agent_config, task_type, message):
   • Task routing map (e.g., code_gen → connection_id_for_codellama)
   • Primary connection (primary_connection_id)
   • System default (system_settings.default_llm_connection_id)
7. 7. LLM Inference LLMClient auto-detects provider from URL, probes candidate endpoints, sends prompt. For vLLM/Custom providers: tools payload is skipped (ReAct text fallback handles tool calling).
8. 8. 4-Tier Tool Call Parsing (max 10 rounds) Tier 1: native tool_calls[] format → execute directly.
   Tier 2: parse ACTION / ACTION_INPUT blocks from text (ReAct).
   Tier 3: false-completion detection + forceful re-prompt.
   Tier 4: keyword scoring against skill descriptions → auto-invoke best match.
9. 9. Skill Execution + Delegation Matched skill handler runs (SSH, SQL, HTTP, Docker, Ansible, etc.). If skill is agent_delegation, spawns a sub-agent with recursion depth guards (max 3 levels). Results feed back as OBSERVATION.
10. 10. Final Answer When LLM produces no more tool calls (FINAL ANSWER), the synthesised response is returned to the user and stored in chat history (Redis).

Multi-LLM Routing Engine

AOS doesn't lock you to a single LLM. The ModelRouter (model_router.py) implements per-agent, task-aware model selection with automatic fallback chains across 10+ LLM providers.

Routing Decision Flow

Task Classification

The ModelRouter classifies each user message into a task type using keyword scoring. Each task type can be routed to a different LLM connection.

Per-Agent Routing Configuration

Each agent’s llm_routing config supports:

Supported LLM Providers

The LLMClient (llm_client.py) auto-detects the provider from the connection URL and adapts its behaviour accordingly:

Fallback Chain Behaviour

The call_with_fallback() method in ModelRouter automatically retries with the next connection in the chain when the current one fails. This ensures agents stay operational even when individual LLM endpoints go down.

Deep Agent Planning Loop

AOS implements a Deep Agent execution model that forces LLMs to plan before they act. When enabled, the system injects a planning prompt into the agent’s system message, requiring the LLM to produce a numbered execution plan with dependencies and risk checks before invoking any tools.

4-Tier Tool Execution

The execution loop uses a 4-tier fallback strategy to guarantee tool execution works with any LLM — whether it supports native function calling or not:

False-Completion Detection

A critical challenge with LLMs is hallucinated action — the model claims “done!” without calling any tool. AOS detects this with a dual-layer heuristic:

• Future-tense laziness: Phrases like “I would…”, “here’s how…”, “you should…” (20+ patterns)
• Past-tense hallucination: Phrases like “has been created”, “successfully configured”, “done!” (50+ patterns)
When detected, the system switches to ReAct mode and re-prompts: "STOP. You did NOT execute anything. DO IT NOW." If the model still fails to emit structured ACTION blocks, Tier 4 intent auto-dispatch attempts safe tool recovery.

Configuration

.skill Package System

AOS agents can be extended with .skill packages — portable ZIP archives that bundle a skill definition with reference knowledge files. When imported, the skill is registered and all reference documents are automatically ingested into the RAG vector store.

.skill File Format

A .skill file is a renamed ZIP archive with this structure:

SKILL.md Structure

The YAML frontmatter (name + description) is parsed as skill metadata. The markdown body becomes the skill’s system_prompt. Files in references/ are automatically chunked, embedded, and stored in the agent’s ChromaDB collection as RAG documents.

Skill Types

Import API

Pre-Built Template Agents

AOS ships with 4 ready-to-import .skill packages in the repository root. Each contains a full skill definition plus reference knowledge files that are auto-ingested as RAG documents.

OpenStack / HCS Skills

AOS ships with 15 built-in OpenStack/HCS skills covering all core services. Agents can manage compute, networking, storage, identity, orchestration, and object storage through natural-language instructions.

6-Stage RAG Pipeline

AOS implements a production-grade, 6-stage RAG pipeline that goes far beyond basic chunking and keyword search. Each stage is independently configurable, wrapped in try/except fallbacks, and wired to the RAGOptimizer policy engine for dynamic tuning.

Stage ① — Semantic Chunking

Instead of fixed character-count splitting, AOS uses the ChunkingPolicy from the RAG Optimizer to split text at paragraph boundaries, heading patterns (Markdown, uppercase, CHAPTER/SECTION), and sentence endings. Small fragments are auto-merged with neighbors. Overlap is applied for cross-boundary context. Falls back to fixed 4000-char chunking if semantic splitting fails.

Each chunk is stored with enriched metadata: doc_id, doc_type, source_file, upload_date, page_number, chunk_index, and chunking_method — enabling downstream metadata filtering.

Stage ② — Ingest-Time Embedding

Immediately after chunking, the DocumentProcessor calls VectorStoreService.embed_document() to embed all chunks using sentence-transformers/all-MiniLM-L6-v2 (384-dim) and store them in ChromaDB with enriched metadata. This means documents are queryable the instant processing completes — no separate "embed" step needed. Non-fatal: if embedding fails, the document is still text-searchable.

Stage ③ — Hybrid Search with Reciprocal Rank Fusion

At query time, AOS runs two parallel searches: keyword (BM25-style scoring with structural detection) and vector (cosine similarity via ChromaDB). Results are merged using Reciprocal Rank Fusion (RRF):

Stage ④ — Metadata Pre-Filtering

The retrieve_for_agent() method now accepts optional filter parameters: doc_type (e.g. "pdf", "docx"), source_file (substring match), upload_date_from / upload_date_to (ISO date range). Filters are applied at both the ChromaDB where-clause level (for vector search) and as post-filters (for keyword results). All filters are fully optional — no filter = search everything.

Stage ⑤ — Cross-Encoder Re-Ranking

After hybrid search, the top candidates are re-ranked using cross-encoder/ms-marco-MiniLM-L-6-v2 — a dedicated re-ranking model that scores each (query, chunk) pair locally. This is the PRIMARY re-ranker (fast, no API cost). If the cross-encoder is unavailable, the system falls back to the original LLM API re-ranking (sends chunks to the LLM for 0.0–1.0 scoring).

Combined scoring: 30% keyword score + 70% cross-encoder score. The final top-K results are returned to the agent's context window.

Stage ⑥ — Semantic Query Cache

A new RAGSemanticCache service uses Redis to cache RAG results. On each query, the cache embeds the query and checks for any cached entry with cosine similarity > 0.95. Cache hit → return cached results instantly. Cache miss → run the full pipeline → store results with a 24-hour TTL.

OCR Worker

The Celery-based OCR worker applies an 8-step image preprocessing pipeline (deskew, denoise, threshold, contrast, DPI scaling, border removal, rotation correction, binarisation) before running Tesseract 5. If the confidence score falls below the threshold, the page is automatically sent to an LLM Vision model for re-extraction. Successfully extracted text is auto-promoted into the 6-stage RAG pipeline — no manual step required.

Enhanced RAG Pipeline — Self-Healing

The 6-stage pipeline above describes what RAG does. The v2 enhancements below describe how the pipeline survives real production conditions: air-gapped servers, sqlite-old RHEL hosts, transient Ollama hiccups, connection-pool storms during bulk reindex, and stale stats reporting.

Embedding Provider Chain (3 tiers + retry)

VectorStoreService._embed_via_api() resolves the embedding endpoint in this order, then commits to it for the rest of the process via a sticky _api_endpoint_available flag — so a transient failure can never silently downgrade to an offline local model.

Retry Policy

Vector Store Backend Fallback

VectorStoreService.__init__() wraps Chroma initialisation in except Exception — not just ImportError — so RHEL9 hosts whose sqlite3 < 3.35.0 trigger Chroma's RuntimeError still get a working RAG via the Postgres-backed _SQLiteVecBackend.

Tunable Chunk Sizing

Defaults were lowered from 4000/600 to 1500/300 for better recall on small/medium corpora. Override per-deployment via env:

Bulk Operations API

Sandbox & On-Spot Skills

Operators can author and attach a Python skill to any agent in a single API call — without restarting the backend, without editing files, and without giving the LLM unrestricted code execution. Inline Python skills run inside a hardened executor by default.

Create & Attach in One Call

Sandbox Guarantees

Skill Types

Agent Academy

The Agent Academy turns a fresh universal agent into a domain expert in four steps — using only the building blocks already in AOS (.skill packages, the 6-stage RAG pipeline, sandbox skills, and the bulk reindex API).

Tracks

The Four Steps

• 1 · Enrol Create a universal agent (or pick one). Optionally bind credentials it will need at graduation.
• 2 · Ingest References Import a .skill pack — its references/ folder is auto-uploaded as documents and linked. RAG kicks in via /api/documents/embed-pending. Verify with /vector-store/stats: total_vectors should grow.
• 3 · Self-Quiz An on-spot sandbox skill (academy_quiz, python_inline) generates 10 questions from chunked references and asks the agent to answer them. Wrong answers are logged with the missing chunk IDs — feeding the next iteration.
• 4 · Graduate The agent is flagged academy_status = graduated in metadata. Channel connectors (Teams, Slack, REST) can now publish it. Re-enrolment re-runs steps 2–3 if references change.

Bootstrap Script (sketch)

OCR / RAG Architecture — Dual-Path Design

AOS processes every uploaded document through two independent paths that converge into a single RAG vector store. Understanding both paths is essential for production tuning and troubleshooting.

Database Schema

Path A — App-Server (Synchronous)

Runs inside the FastAPI process on POST /api/documents/upload. Best for native-text PDFs, DOCX, TXT, Markdown — any format that already contains extractable text. Completes in seconds.

• 1 · File receipt & metadata Save to uploads/, insert row into documents table (status = processing).
• 2 · Text extraction PyPDF2 first, fallback to PyMuPDF (fitz). If extracted text < MIN_USEFUL_TEXT (50 chars), OCR is triggered inline via Tesseract → easyocr → LLM Vision cascade.
• 3 · Semantic chunking Split into overlapping chunks (default 1 000 tokens, 200 overlap) using sentence-boundary-aware splitter.
• 4 · Embedding & storage Each chunk → all-MiniLM-L6-v2 (384-dim, L2-normalised to 768-dim) → upserted into agent-scoped ChromaDB collection.
• 5 · Status update Set document status to completed. RAG is immediately available.

Path B — Celery Worker (Asynchronous)

Triggered via POST /api/documents/batch-scan or the admin “Scan NFS” button. Designed for scanned PDFs, images (TIFF/PNG/JPEG), and bulk directory ingestion. Runs on a separate worker node (or the same host) via Redis-brokered Celery.

• 1 · Directory scan Walk the NFS/local path, discover PDF/image files, create documents rows (status = queued).
• 2 · Celery task dispatch One Celery task per file. Redis broker distributes across available workers.
• 3 · PDF → image pdf2image (poppler) converts each page to a PIL image at configured DPI.
• 4 · 8-step preprocessing Deskew → denoise → threshold → contrast → DPI scale → border removal → rotation correction → binarisation.
• 5 · OCR cascade Tesseract 5 first. If confidence < threshold → easyocr. If still low → LLM Vision (vLLM multimodal endpoint).
• 6 · Auto-promote to RAG Extracted text is chunked and embedded exactly like Path A (same chunker, same model) → upserted into ChromaDB.
• 7 · Status callback Celery result backend updates document status to completed or failed with error details.

Path A vs Path B — Comparison

Batch-Scan Flow

Document → Agent Linking

Documents are linked to agents via the document_agent_links join table. A single document can be linked to multiple agents — each agent gets its own copy of the chunks in its scoped ChromaDB collection.

Query-Time RAG Flow

When a user sends a message to an agent with RAG enabled, the query goes through the 6-stage pipeline to retrieve the most relevant context from that agent’s document collection.

• 1 · Semantic cache check Hash the query → check Redis semantic cache. If hit (cosine > 0.95), return cached answer immediately.
• 2 · Hybrid search Run both dense (embedding similarity) and sparse (BM25 keyword) search on the agent’s ChromaDB collection.
• 3 · RRF fusion Reciprocal Rank Fusion merges both result lists into a single ranked list (k=60).
• 4 · Cross-encoder re-ranking cross-encoder/ms-marco-MiniLM-L-6-v2 scores each candidate for precise relevance. Top-N (default 5) survive.
• 5 · LLM generation Surviving chunks are injected into the system prompt as context. The LLM generates a grounded answer with source citations.

Embedding Pipeline

Security Model

Data Governance & Regulatory References

AOS includes a built-in Data Governance engine that enforces enterprise policies for data classification, PII detection, retention rules, and access controls. Each policy is enriched with regulatory references mapping to international standards.

Semantic Query Cache Architecture

The RAGSemanticCache in rag_cache.py is a standalone service class with four public methods. It can be integrated at any point in the retrieval pipeline.

Deployment Options

Quick Start (Single Server)

Production (Multi-Server)

System Requirements

Infrastructure Admin Panel

The Infrastructure tab on the Admin Panel gives operators a single pane-of-glass to view and edit the backend’s .env configuration and to verify that every external service is reachable — without SSH access.

Server Configuration

Fields are grouped into four colour-coded sections:

Password fields are always displayed as •••••••• and are only written back to .env if the user actually changes them. An amber dot appears next to any field that has been modified but not yet saved. On save the backend writes the new values into the .env file and clears the settings cache (get_settings.cache_clear()) so the next request picks up changes immediately.

Service Health Check

The Service Health panel tests real connectivity to five infrastructure components:

Each service shows a • green / • amber / • grey status dot plus a detail string so operators can diagnose connectivity issues at a glance.

API Reference (Key Endpoints)