Full Time Product Manager, LLM Router Jobs, in Filigrain - Maukerja

About the role

A single AI application may need a reasoning model for one request, a faster model for the next, and a fallback when a provider goes down. Our LLM router gives developers and enterprises a single API that selects models based on task quality, cost, latency, reliability, and customer policies.

We're hiring a Product Manager to own the LLM routing product and lead the engineering team that builds and operates it. You'll set the roadmap, lead a small team of senior engineers through day-to-day execution, and be accountable for what ships and how it performs in production.

What you'll do

• Lead the engineering team and own delivery. Set priorities, clarify ownership, and agree on delivery milestones with senior engineers. Review progress, give actionable feedback, resolve blockers, and make decisions when scope, timelines, and technical constraints collide. Work with the team to balance feature delivery, reliability, and technical debt.
• Own the routing roadmap. Define how the router selects a model for each request. Decide which signals should guide routing, including prompt characteristics, historical model performance, real-time latency, and customer policies.
• Drive evaluation strategy. Partner with engineers to design benchmarks for coding, reasoning, summarization, tool use, and creative tasks. Use offline evaluations and production data to decide which routing changes to ship.
• Build for developers and enterprise teams. Make it easy for individual developers to get started with a single API and sensible defaults. Support enterprise requirements such as custom routing policies, governance, audit logs, private deployments, and SSO.
• Shape the developer experience. Define how developers integrate, monitor, and troubleshoot the router through its APIs, SDKs, dashboards, and observability tools.
• Improve cost and reliability. Prioritize budget controls, per-route cost caps, automatic fallbacks, retries, caching, and service-level commitments.
• Manage the model portfolio. Decide which models and providers to add, promote, or retire. Work with model providers on integration requirements and pricing.
• Stay close to customers. Speak regularly with independent developers and enterprise platform teams. Learn how they build, debug, and deploy LLM applications, and use what you learn to shape the roadmap.
• Measure product outcomes. Define and track metrics such as routing quality, cost per successful request, p50 and p95 latency, reliability, and retention. Use them to prioritize work and assess the impact of releases.

What we're looking for

• 5+ years of product management experience, including 2+ years working on developer-facing, API-first, or infrastructure products.
• Experience leading a software engineering team and owning its delivery. You can point to a product or release where you set priorities, clarified responsibilities, handled a difficult technical or delivery trade-off, and took responsibility for the outcome. Experience leading senior engineers is especially relevant.
• Hands-on use of AI tools. You regularly use tools such as Claude or Cursor to research problems, prototype ideas, draft specs, or build internal tools.
• A practical understanding of LLM fundamentals, including prompts, context windows, sampling parameters, tool use, and model evaluation. You can explain why different models perform better on different tasks.
• A track record of shipping technical products that require trade-offs among quality, latency, cost, and reliability.
• Curiosity about the model ecosystem. You follow model releases, understand the trade-offs between proprietary and open-weight models, and can spot when a benchmark gives a misleading picture.
• Strong analytical skills and comfort using SQL, experiments, and evaluation data to make product decisions.
• Clear written communication. You can write a focused PRD, document technical decisions, and explain releases to developers.
• A track record of making decisions with incomplete information, shipping improvements, and adjusting based on results.

Bonus points

• An engineering background or hands-on experience building production systems with LLM APIs.
• Experience building LLM-powered applications such as agents, RAG systems, or copilots.
• A working understanding of inference economics, including token pricing, batching, prompt caching, and model serving.
• Experience selling to or supporting enterprise customers through security reviews, procurement, or deployment.
• Experience working with model providers as partners or vendors.

What success looks like

• After 3 months, the engineering team has clear priorities, owners, and milestones. You've shipped a measurable routing improvement and built relationships with early customers across both developer and enterprise segments.
• After 6 months, releases have improved routing quality and reduced customer cost per successful request. You've shipped features that address concrete needs for both self-serve developers and enterprise platform teams.
• After 12 months, more customers rely on the router as their default integration for production LLM workloads. You can show which product decisions drove adoption and how the team delivered them.

Why this role

LLM routing is still taking shape. Models change quickly, and decisions about which model to use, when to fall back, and what developers can control have a direct impact on production applications. You'll own those decisions and lead the team that turns them into a product developers and enterprises can rely on.