Software Engineer

The Build Agent is ServiceNow's AI coding assistant, purpose-built for the platform's metadata-driven substrate - operating natively across ServiceNow's scoped applications, tables, and metadata types. The framework is critical to Build Agent success — it's already driven measurable wins: recovering Build Agent correctness, cutting latency and inference cost ahead of a major release, and catching high-severity defects that manual testing missed before they reached production. Key areas of focus include: Evaluation Infrastructure: Golden prompt sets, Pass@1 and functional scoring, failure categorization (plumbing vs. metadata-creation failures), and coverage across ServiceNow metadata types and UI workflows. Model Support & Benchmarking: Structured evaluation of candidate foundation models against the production default, with failure-consistency analysis to separate scaffold/tuning issues from genuine capability gaps. Telemetry: Token usage, cache efficiency, and inference cost tracked alongside correctness as first-class release signals. What you get to do in this role: Eval orchestration at scale, Agent observability and tracing.