Multi-Agent Orchestration Platform
Exploring agentic AI patterns and autonomous workflow coordination
Updated Jan 2026Design Notes
I built Agentic-PR to work through multi-agent orchestration patterns end-to-end, including the business layer around them. The technical problem was the substantive one: coordinating autonomous AI agents, managing state across complex workflows, building production-grade systems for unpredictable LLM behavior.
The architecture had to survive LLM output that doesn't behave like a normal API response — retryable failures, partial completions, and variance that a typed backend can't assume away. Solo-building the full stack (backend, AI service layer, web and mobile clients) forced every one of those decisions to be made explicitly rather than deferred to a team's division of labor.
Result: a working demonstration of multi-agent coordination at production scale — ~8,000 lines of backend, a hybrid model-routing layer, and a real-time A/B testing system — that remains operational. The system design below is what shipped.
System Architecture
What I Personally Built
Complete Backend Architecture (~8,000 lines)
Designed and implemented the entire NestJS backend from scratch: modular service architecture, MongoDB/PostgreSQL data layer, Bull job queues with Redis, and WebSocket real-time communication layer.
AI Service Layer with Hybrid Model Routing
Built the complete LLM integration layer: template-based prompt engineering, structured output parsing, intelligent model routing (DistilBERT for speed, GPT-4 for complexity), and comprehensive error handling with retries.
A/B Testing Infrastructure
Implemented the complete A/B testing system: variant generation, random assignment, open/click tracking, statistical significance calculation, and real-time dashboard updates via WebSocket.
Full-Stack Web & Mobile Applications (~7,000 lines)
Built the Next.js web application and React Native mobile apps (iOS/Android) with shared business logic, Chakra UI components, and real-time WebSocket integration for live updates.
Infrastructure & DevOps
Set up the complete deployment pipeline: Docker containerization, Terraform infrastructure as code, nginx reverse proxy, OAuth 2.0 integration (Google, LinkedIn, Twitter), and JWT session management.
Results & Outcomes
Technical Achievements
- Production-grade LLM reliability with exponential backoff and fallbacks
- Real-time WebSocket updates for live dashboard feedback
- Hybrid model routing reduced API costs by ~60% vs GPT-4 only
- Statistically rigorous A/B testing with significance tracking
Repository Structure
Monorepo architecture with shared types and utilities
agentic-pr/
├── apps/
│ ├── api/ # NestJS Backend (~8,000 lines)
│ │ ├── src/
│ │ │ ├── ai/
│ │ │ │ ├── ai.service.ts # LLM integration
│ │ │ │ ├── ai-queue.producer.ts # Job queue producer
│ │ │ │ ├── ai-queue.consumer.ts # Job queue consumer
│ │ │ │ └── prompts/ # Template prompts
│ │ │ ├── pr/
│ │ │ │ ├── pr.service.ts # PR pitch management
│ │ │ │ ├── pr.controller.ts # REST endpoints
│ │ │ │ └── dto/ # Data transfer objects
│ │ │ ├── sentiment/
│ │ │ │ ├── sentiment.service.ts # Hybrid analysis
│ │ │ │ ├── distilbert.service.ts # Fast model
│ │ │ │ └── alerts.service.ts # Change detection
│ │ │ ├── websocket/
│ │ │ │ ├── websocket.gateway.ts # Socket.io gateway
│ │ │ │ └── websocket.service.ts # Real-time updates
│ │ │ ├── auth/
│ │ │ │ ├── oauth.strategy.ts # Google/LinkedIn/Twitter
│ │ │ │ └── jwt.strategy.ts # Session management
│ │ │ └── common/
│ │ │ ├── filters/ # Error handling
│ │ │ └── interceptors/ # Logging, transforms
│ │ └── test/ # Integration tests
│ │
│ ├── web/ # Next.js Web App (~4,000 lines)
│ │ ├── src/
│ │ │ ├── pages/
│ │ │ ├── components/
│ │ │ │ ├── dashboard/ # Real-time dashboards
│ │ │ │ ├── ab-testing/ # A/B test UI
│ │ │ │ └── sentiment/ # Sentiment charts
│ │ │ ├── hooks/
│ │ │ │ └── useWebSocket.ts # Real-time hook
│ │ │ └── lib/
│ │ │ └── api.ts # API client
│ │ └── public/
│ │
│ └── mobile/ # React Native (~3,000 lines)
│ ├── src/
│ │ ├── screens/
│ │ ├── components/
│ │ └── services/
│ ├── ios/
│ └── android/
│
├── packages/
│ ├── shared-types/ # TypeScript interfaces
│ └── shared-utils/ # Common utilities
│
├── infrastructure/
│ ├── terraform/
│ │ ├── main.tf
│ │ ├── variables.tf
│ │ └── outputs.tf
│ └── docker/
│ ├── Dockerfile.api
│ ├── Dockerfile.web
│ └── docker-compose.yml
│
└── docs/
├── architecture.md
├── api-reference.md
└── deployment.mdThe Technical Challenge
Traditional automation requires explicit programming of every step—if-then logic that breaks when conditions change. Agentic AI enables autonomous agents to make decisions and execute tasks, but introduces new challenges around coordination and reliability.
The core question: How do you orchestrate multiple autonomous AI agents reliably in production?
LLM Integration at Scale
Production LLM systems need more than API calls—they need prompt engineering, response validation, cost optimization, and graceful handling of model failures and rate limits.
Async Workflow Coordination
AI tasks are inherently async and unpredictable. The system needs message queues, job prioritization, retry logic with exponential backoff, and real-time status updates.
Multi-Model Orchestration
Different tasks benefit from different models—DistilBERT for fast sentiment analysis, GPT-4 for complex content generation. The system routes tasks to optimal models based on requirements.
What I Built
A complete production platform with ~15,000+ lines of TypeScript across backend, frontend, and mobile applications. The system handles AI-powered content generation, A/B testing, sentiment analysis, and real-time workflow coordination.
AI Service Layer (~2,000 lines)
Comprehensive LLM integration with template-based prompt engineering, structured output parsing, and fallback handling for API failures.
1@Injectable()
2export class AiService {
3 private readonly logger = new Logger(AiService.name);
4 private openai: OpenAIApi;
5
6 constructor(private configService: ConfigService) {
7 const apiKey = this.configService.get<string>('OPENAI_API_KEY');
8 const configuration = new Configuration({ apiKey });
9 this.openai = new OpenAIApi(configuration);
10 }
11
12 /**
13 * Generate a completion using the OpenAI API
14 */
15 async generateCompletion(prompt: string, params: Record<string, any> = {}): Promise<string> {
16 // Replace template variables in the prompt
17 let processedPrompt = prompt;
18 for (const [key, value] of Object.entries(params)) {
19 const placeholder = `{{${key}}}`;
20 if (typeof value === 'string') {
21 processedPrompt = processedPrompt.replace(new RegExp(placeholder, 'g'), value);
22 } else if (Array.isArray(value)) {
23 processedPrompt = processedPrompt.replace(new RegExp(placeholder, 'g'), value.join(', '));
24 }
25 }
26
27 const response = await this.openai.createCompletion({
28 model: 'gpt-4',
29 prompt: processedPrompt,
30 max_tokens: 2000,
31 temperature: 0.7,
32 });
33
34 return response.data.choices[0].text.trim();
35 }
36}PR Service with A/B Testing (~770 lines)
Automated A/B testing for AI-generated content with real-time tracking, statistical analysis, and WebSocket-based dashboard updates.
1@Injectable()
2export class PRService {
3 constructor(
4 @InjectModel(PRPitch.name) private prPitchModel: Model<PRPitchDocument>,
5 private aiService: AiService,
6 private journalistService: JournalistService,
7 private webSocketService: WebSocketService,
8 ) {}
9
10 /**
11 * Send a PR pitch with A/B testing
12 */
13 async sendPitch(id: number): Promise<PRPitchDocument> {
14 const pitch = await this.findOne(id);
15 const journalist = await this.journalistService.findOne(pitch.journalistId);
16
17 // Randomly select variant A or B for this recipient
18 const variant = Math.random() > 0.5 ? 'A' : 'B';
19 const subjectLine = variant === 'A' ? pitch.subjectLineA : pitch.subjectLineB;
20 const content = variant === 'A' ? pitch.contentA : pitch.contentB;
21
22 // Update pitch status and send real-time update via WebSocket
23 pitch.status = 'sent';
24 const savedPitch = await pitch.save();
25
26 this.webSocketService.sendPRUpdate({
27 action: 'pitch_sent',
28 pitch: { id: savedPitch._id, variant, timestamp: new Date().toISOString() }
29 });
30
31 return savedPitch;
32 }
33
34 /**
35 * Track email open for A/B test analysis
36 */
37 async trackOpen(id: number, variant: 'A' | 'B'): Promise<PRPitchDocument> {
38 const pitch = await this.findOne(id);
39 variant === 'A' ? pitch.opensA += 1 : pitch.opensB += 1;
40
41 // Real-time dashboard update
42 this.webSocketService.sendABTestUpdate({
43 action: 'variant_opened',
44 pitch: { id, variant, opensA: pitch.opensA, opensB: pitch.opensB }
45 });
46
47 return pitch.save();
48 }
49}Hybrid Sentiment Analysis (~940 lines)
Multi-model sentiment analysis combining DistilBERT for speed with GPT-4 for accuracy, plus automated alerting when sentiment changes significantly.
1@Injectable()
2export class SentimentService {
3 private distilBertModel: any;
4 private isModelLoaded: boolean = false;
5
6 constructor(
7 @InjectModel(SentimentAnalysis.name) private sentimentAnalysisModel: Model<SentimentAnalysisDocument>,
8 private aiService: AiService,
9 private userSettingsService: UserSettingsService,
10 ) {
11 this.initializeModels();
12 }
13
14 /**
15 * Analyze text sentiment using preferred model (DistilBERT or GPT-4)
16 */
17 async analyzeSentiment(userId: string, dto: CreateSentimentAnalysisDto): Promise<SentimentAnalysisResponseDto> {
18 const userSettings = await this.userSettingsService.getUserSettings(userId);
19 const preferredModel = dto.preferredModel || userSettings?.preferredSentimentModel || SentimentModel.DISTILBERT;
20
21 const sentimentAnalysis = new this.sentimentAnalysisModel({
22 userId,
23 content: dto.content,
24 modelUsed: preferredModel,
25 isProcessed: false,
26 });
27
28 const savedAnalysis = await sentimentAnalysis.save();
29
30 // Process asynchronously - hybrid DistilBERT + GPT-4 approach
31 this.processSentimentAnalysis(savedAnalysis._id, preferredModel);
32
33 return this.mapToResponseDto(savedAnalysis);
34 }
35
36 /**
37 * Check if sentiment change warrants an alert
38 */
39 private async checkForSentimentAlert(analysis: SentimentAnalysisDocument): Promise<void> {
40 const previousAnalyses = await this.sentimentAnalysisModel
41 .find({ userId: analysis.userId, source: analysis.source })
42 .sort({ createdAt: -1 }).limit(5);
43
44 const avgPreviousScore = previousAnalyses.reduce((sum, a) => sum + a.score, 0) / previousAnalyses.length;
45 const changeMagnitude = Math.abs(analysis.score - avgPreviousScore);
46
47 if (changeMagnitude >= this.alertThreshold) {
48 await this.createSentimentAlert(analysis.userId, {
49 title: 'Significant sentiment change detected',
50 severity: changeMagnitude >= 0.7 ? AlertSeverity.CRITICAL : AlertSeverity.MEDIUM,
51 recommendedActions: this.generateRecommendedActions(analysis, changeMagnitude)
52 });
53 }
54 }
55}Async Queue System
Production job queue with priority scheduling, retry logic, and exponential backoff for handling AI workloads reliably.
1// AI Queue Producer - Async job processing
2@Injectable()
3export class AiQueueProducer {
4 constructor(@InjectQueue('ai-queue') private queue: Queue) {}
5
6 async generateSocialMediaContent(params: SocialMediaContentParams, priority: number = 2) {
7 const job = await this.queue.add(
8 AiJobType.CONTENT_GENERATION,
9 { params },
10 {
11 priority,
12 attempts: 3,
13 backoff: { type: 'exponential', delay: 5000 },
14 },
15 );
16 return job.id;
17 }
18}
19
20// AI Queue Consumer - Process jobs with retry logic
21@Processor('ai-queue')
22export class AiQueueConsumer {
23 constructor(private readonly aiService: AiService) {}
24
25 @Process(AiJobType.CONTENT_GENERATION)
26 async processContentGeneration(job: Job) {
27 this.logger.log(`Processing content generation job ${job.id}`);
28 try {
29 return await this.aiService.generateSocialMediaContent(job.data.params);
30 } catch (error) {
31 this.logger.error(`Error in job ${job.id}: ${error.message}`);
32 throw error; // Triggers retry with exponential backoff
33 }
34 }
35}Technical Stack
Backend
- • NestJS (TypeScript) - Modular architecture
- • MongoDB - Document storage for workflows
- • PostgreSQL - Relational data
- • Redis - Caching & queue backend
- • Bull - Job queue with priorities
- • Socket.io - Real-time WebSocket
AI/ML
- • OpenAI GPT-4 - Content generation
- • DistilBERT - Fast sentiment analysis
- • Template prompts - Structured generation
- • A/B testing - Variant optimization
- • Hybrid routing - Model selection
Frontend
- • Next.js (React) - Web application
- • React Native - iOS/Android mobile
- • Chakra UI - Component library
- • WebSocket - Real-time updates
- • TypeScript - Type safety
Infrastructure
- • Docker - Containerization
- • Terraform - Infrastructure as code
- • nginx - Reverse proxy
- • OAuth 2.0 - Google, LinkedIn, Twitter
- • JWT - Session management
Technical Challenges Solved
Production LLM Reliability
LLM APIs fail, rate limit, and produce unexpected outputs. Built comprehensive error handling with retries, fallbacks, and output validation. The queue system ensures no work is lost during failures.
Real-time Coordination
AI tasks run asynchronously but users need immediate feedback. Implemented WebSocket-based real-time updates so dashboards show live status of A/B tests, sentiment changes, and content generation.
Cost Optimization
GPT-4 API calls are expensive at scale. Implemented intelligent model routing—using DistilBERT for fast sentiment analysis, reserving GPT-4 for complex content generation. Added caching and response truncation where appropriate.
Multi-Platform Deployment
Single codebase serving web, iOS, and Android with shared business logic. React Native mobile apps connect to the same backend APIs with real-time WebSocket support.
Mathematical Formulation
Key algorithms underlying the agentic orchestration platform.
A/B Testing Statistical Analysis
Conversion Rate Estimation:
where = opens/clicks, = emails sent per variant.
Z-Test for Significance:
where is the pooled proportion. Reject null hypothesis if (95% confidence).
Sample Size for Power:
Minimum samples needed to detect effect size with 80% power.
Sentiment Analysis & Alerting
Sentiment Score (DistilBERT):
where is the pooled representation from DistilBERT, = sigmoid activation.
Change Detection (Moving Average):
Alert triggered when (default ).
Alert Severity Classification:
Job Queue Priority & Retry Logic
Exponential Backoff:
where = retry attempt, ms base delay, = jitter to prevent thundering herd.
Priority Queue Ordering:
Jobs ordered by priority (lower = higher priority), then by timestamp (FIFO within priority).
Key Technical Learnings
"Production LLM systems are 20% prompt engineering and 80% infrastructure—error handling, retries, cost management, and observability. The happy path is easy; reliability is hard."
"Hybrid model architectures work well in practice. Fast models (DistilBERT) for high-volume tasks, powerful models (GPT-4) for complex generation. Route intelligently based on task requirements."
"Real-time feedback transforms user experience with AI systems. WebSocket updates showing live progress make async AI tasks feel responsive and trustworthy."
Production Insights
- • Exponential backoff for API failures
- • Output validation before storage
- • Cost tracking per API call
- • Structured logging for debugging
Architecture Patterns
- • Event-driven async processing
- • Priority queues for job scheduling
- • WebSocket for real-time updates
- • Hybrid model routing
Scope
Full-stack, solo build: backend architecture, the AI service layer, the real-time coordination system, and both the web and mobile clients. The business layer (customer development, sales, marketing) ran alongside it but isn't what's documented here — this page covers the system design.