Deepgram
Software Test Engineer
See how this job matches your profile
Sign in for an AI-powered fit score, breakdown, and a tailored resume.
Job Description
Deepgram is seeking a Software Test Engineer to design, build, and maintain automated test frameworks and exploratory test suites across their products, models, APIs, and data platforms. The role involves identifying system weaknesses, testing edge cases and adversarial inputs, and automating validation to quickly detect regressions. The engineer will translate product requirements and model metrics into automated regression tests, evaluation pipelines, data-quality gates, load tests, and release criteria. Collaboration with QA, Research, Product, Data, and Engineering teams is key for planning testing, executing evaluations, supporting user acceptance testing, and communicating risks. The position requires providing precise reproduction steps for any issues found. The core excitement of this role lies in building scalable automation that ensures the reliability of Deepgram's products, models, and data workflows for customers.
Qualifications
Required Qualifications
- BS, MS, or PhD in Computer Science, AI, Applied Math, or a related field, or equivalent experience.
- 5+ years of professional software or QA engineering experience, with a track record of shipping test infrastructure or evaluation systems (senior candidates with significantly deeper experience welcome).
- Solid backend/scripting experience in a language such as Python, Rust, Go, or similar.
- Experience designing and building automated test pipelines, evaluation frameworks, or data-processing systems.
- Strong analytical skills and comfort reasoning about metrics, thresholds, and statistical variation in results — able to distinguish real regressions from noise.
- Ability to take charge of ambiguous technical challenges and communicate effectively across research, engineering, and product teams.
Preferred Qualifications
- Hands-on experience testing or evaluating modern AI systems such as LLMs, RAG pipelines, agents, or multimodal models, including analyzing model behavior and failure modes.
- Experience with voice, audio, speech recognition, or real-time systems, and familiarity with metrics such as WER, MOS, latency, and time-to-first-byte.
- Experience building or improving test, evaluation, benchmarking, or ML infrastructure used by multiple teams or external users.
- A strong appreciation for test and evaluation quality, including correctness, reproducibility, determinism, and consistency across environments.
- Experience building test tooling for React Native, mobile applications, or other cross-platform environments that extends validation beyond the desktop.
- Familiarity with cloud infrastructure, containers, ephemeral test environments, CI/CD systems, and monitoring tools such as Grafana, canaries, and anomaly detection.
- Experience serving as a technical bridge across teams or platforms—including product, QA, evaluation, training, inference, data, or agent frameworks—with the communication skills to build alignment and influence decisions.
- Prior involvement in open-source projects through contributions, reviews, maintenance, or community engagement.
- Experience with voice, audio, speech recognition, or real-time systems, and familiarity with metrics like WER, MOS, or latency/TTFB.
- Prior involvement in open-source projects, through contributions, reviews, maintenance, or community engagement.
- Experience acting as a technical bridge across teams or platforms (evaluation, training, inference, agent frameworks), combining architectural understanding with clear communication and influence.
- Familiarity with cloud infrastructure, containerized/ephemeral environments, and monitoring tooling (e.g. Grafana, canaries, anomaly detection).
Skills & Technologies
About the Company
Deepgram
View company profile →
Interested in this role?
Sign in or create a free account to see how this job matches your skills, apply with one click, and let our AI tailor your resume.
Sign in to applyJob Details
Employment Type
Full Time
Experience Level
Senior
Salary Range
$150,000 – $220,000
Location
Remote, CA
Posted
Today
Country
US