AI Engineer, Agent Analytics & Optimization

Conviva is the experience analytics platform for AI agents, apps, and streaming. Conviva pioneered quality of experience measurement for streaming video and has supported live digital experiences across 20 Super Bowls, 10 Olympic Games, and five FIFA World Cup tournaments. With full-census, real-time, stateful analytics, Conviva shows what consumers actually experience, not just whether systems are running, and connects that experience to revenue, retention, and risk. The Opportunity AI agents are becoming a new digital surface alongside mobile and web. This role builds the systems that let AI agents understand, query, and act on Conviva's behavioral data — MCP servers, agentic workflows, and the AI connectors that tie the platform into any client or downstream system. This is a builder role end-to-end: design and ship agentic systems from tool interface through deployment, observability, and iteration — working closely with product and customers. This is not a research role, and it is not prompt engineering; the team builds production systems that use AI to solve real problems at scale. How This Role Fits This is an engineering role, distinct from Conviva's Product Builder roles on the same team: - Product Builders validate direction through customer discovery and prototypes — deciding what agent analytics should become. - This role turns that direction into production-grade systems that run reliably at scale — owning the build and the runtime, not the roadmap. What Success Will Look Like: Build & Ship - Design and build MCP servers that expose Conviva's data and analytics as discoverable, reliable tools for AI agents, with proper auth, scoping, and access controls - Architect and ship agentic workflows: multi-step reasoning, tool use, orchestration patterns, and human-in-the-loop where needed - Develop AI connectors integrating Conviva's intelligence with external agents, clients, and downstream systems - Own the full lifecycle — API/tool design, deployment, monitoring, production hardening Reliability & Iteration - Use evals to validate agent behavior before shipping, catch regressions, and debug production divergence - Build observability and tracing to make agent behavior inspectable and auditable - Tune for latency, cost, and reliability as models and data drift Execution & Collaboration - Partner with Product Builders, design, and customers to ship working software (not prototypes) - Build internal SDKs, reusable patterns, and shared context that raise the floor for the whole team - Stay close to the fast-moving MCP/agent ecosystem and bring back what's relevant Who You Are and What You've Done: - 3–7+ years of software engineering experience, with production systems shipped end-to-end - Strong Python backend fundamentals; comfortable owning services from design through deployment (fullstack a plus) - Hands-on experience building and operating LLM-powered agents in production — with concrete stories of what broke, how you fixed it, and what you'd do differently - Practical familiarity with agentic patterns: tool use, structured outputs, prompt chaining, multi-agent coordination, and knowing when a deterministic workflow is the right call - Experience designing/building MCP servers or equivalent tool-use / AI connector integrations - Have used evals to validate and debug agent behavior - Comfortable with cloud deployment (GCP/AWS), containerized services, and async Python - Strong product instinct — thinks about what users actually need and pushes back when a spec doesn't add up Bonus Points - Experience with agentic frameworks (LangGraph, Anthropic SDK, Pydantic-AI, CrewAI, or similar) - Agent observability/tracing (Langfuse, LangSmith, OpenTelemetry, or homegrown) - Background in digital experience analytics, observability, or APM - Familiarity with agent evaluation or LLM-ops tooling - Published writing, talks, or open-source work at the intersection of engineering and AI What We Offer - A front-row seat to a category being created in real time — agent analytics is where web analytics was in 2005 - A team that values builders — we use AI every day to move faster, not just talk about it - A hybrid role shaping both the "what" and the "how" — strategy meets craft - Competitive compensation, equity, and benefits The expected salary range for this full-time position is $180,000–$250,000 + equity + benefits. Actual level and compensation are determined by qualifications, professional background, and relevant experience. Learn more at conviva.ai and follow the company on LinkedIn and YouTube.