Full-Stack Engineer, Agent Evaluation & Reliability (In-Person) at Idelsoft

Job description

We're hiring a backend-leaning Full-Stack Engineer to build the infrastructure that makes enterprise AI agents reliable. The platform evaluates how agents perform in production, finds what went wrong, and keeps improving both the agents and the knowledge they rely on. Most of the hard problems are in the backend: data modeling, queues, execution pipelines, integrations, observability and debugging distributed workflows. You'll also take features through the full stack and ship the product that makes the system useful. You'll work directly with the founders, the engineering lead and enterprise customers, with real ownership of architecture and product decisions from day one. What you'll do • Build and own the backend systems behind an agent evaluation and continuous-improvement platform • Design data models for evaluation datasets, experiment runs, agent traces, knowledge versions, findings and customer-defined quality criteria • Own enterprise integrations end to end: authentication, OAuth, webhooks, incremental syncs, backfills, rate limits, schema changes, retries, monitoring and production support • Take features from technical design through deployment, monitoring, customer feedback and iteration • Work directly with customers to understand their systems, diagnose failures and turn one-off requests into reusable product features You may be a fit if • You have 2 to 4 years of building and running production software, including time at a fast-paced startup • You have strong backend skills in production TypeScript and Node.js (or Bun) and can ship across the full stack • You know PostgreSQL well and have hands-on experience with Redis, job queues, background workers or workflow systems • You've owned projects and features end to end and pick things up quickly • You understand why evaluating nondeterministic agent systems is different from regular unit testing • You can work in person in San Francisco Nice to have • Founding or early engineer experience at a YC or other early-stage startup • APIs, integrations, webhooks and data pipelines for multi-tenant enterprise software • A background in enterprise AI, agent infrastructure, observability or distributed systems • Experience at a high-growth AI startup What we offer • Competitive salary and meaningful equity, with flexibility on the mix • Visa sponsorship available • Direct work with the founders and enterprise customers

Requirements

Must-haves

Build and own the backend systems behind an agent evaluation and continuous-improvement platform
Design data models for evaluation datasets, experiment runs, agent traces, knowledge versions, findings and customer-defined quality criteria
Own enterprise integrations end to end: authentication, OAuth, webhooks, incremental syncs, backfills, rate limits, schema changes, retries, monitoring and production support
Take features from technical design through deployment, monitoring, customer feedback and iteration
Work directly with customers to understand their systems, diagnose failures and turn one-off requests into reusable product features
You may be a fit if
You have 2 to 4 years of building and running production software, including time at a fast-paced startup
You have strong backend skills in production TypeScript and Node.js (or Bun) and can ship across the full stack
You know PostgreSQL well and have hands-on experience with Redis, job queues, background workers or workflow systems
You've owned projects and features end to end and pick things up quickly
You understand why evaluating nondeterministic agent systems is different from regular unit testing
You can work in person in San Francisco

Nice-to-haves

Founding or early engineer experience at a YC or other early-stage startup
APIs, integrations, webhooks and data pipelines for multi-tenant enterprise software
A background in enterprise AI, agent infrastructure, observability or distributed systems
Experience at a high-growth AI startup

Application prep

Use the role’s language and application details while tailoring your materials.

Keywords to mirror

aitypescriptnode.js

Platform: manatal
Effort: medium