Senior AI Engineer

Senior AI Engineer at HermeticRemote · Full-timeAbout HermeticRestaurants and event venues lose bookings every day to slow replies. A guest asks about a rehearsal dinner on a Tuesday night, and by the time someone answers on Thursday, they’ve booked somewhere else.Hermetic fixes that with Mia, an AI assistant that picks up inquiries within seconds, answers questions about rooms, menus, and availability, follows up when a guest goes quiet, and hands the conversation to a coordinator at exactly the right moment.The RoleYou’ll own Mia’s conversations end to end: the agent that writes each reply, the models that decide whether a reply can go out on its own or should wait for a human, and the measurement that tells us whether a change actually made Mia better. The work sits across LLM systems, applied ML, and production engineering.What You’ll Work OnThe conversation agent: Mia’s replies, routing, tool calls for availability and pricing, handoff detection, and guardrails that keep things accurate and on-brand.Autonomy: Improve the models and per-location thresholds that predict which replies need an edit, so more can go out safely on their own.Evaluation that drives decisions: Build offline eval sets from real conversations, regression tests for prompts and graph changes, and online metrics tied to what customers care about: edit rate, handoff rate, response time, and bookings.Voice: Help take Voice Mia from early adopters to every customer, with per-customer rollout flags, safe call transfer, and instant rollback when something sounds off.Running it for real: Tracing and cost tracking on every LLM call, prompt versioning, latency budgets, and being the calm person in the room when a model provider has a bad day.Raising the bar: Write the design docs others build from, review code thoughtfully, and help engineers new to LLM systems get good at them fast.What You Bring6+ years building production software, including 2+ years shipping LLM-powered features that real users depend onStrong Python, and comfort reading and changing TypeScriptReal experience with agent frameworks or tool-using LLM workflows, and opinions about where they breakExperience building evaluation for a model or LLM system and using it to make actual ship or don’t-ship callsPractical ML skills: you can train, calibrate, and threshold a classifier, and explain the precision-versus-coverage trade-off to someone who isn’t an ML personSolid production instincts around async queues, idempotency, retries, observability, and costClear writing: your design docs and PR descriptions make it easy for others to actNice to HaveVoice AI or real-time conversational systemsExperience in hospitality, events, booking, or conversational commerceHuman-in-the-loop systems, or learning from human editsHands-on time with SageMaker, Langfuse, or LangSmithOur StackServices: Python, TypeScript, ReactData: PostgreSQL, Redis, SQSInfrastructure: AWS, Grafana, PostHog (analytics and feature flags)Integrations: Twilio, plus the CRMs and booking systems our venues use, such as Salesforce and TripleseatHow We WorkWe’re a small team with a lot of ownership, and you’ll ship to production every week. AI coding tools are part of how we build. We expect you to use them well, and to know when not to trust them.