Quality Associate - AI/ML
Location: In-office (Hyderabad)
Position Type: Full-Time
Joining: Immediate
About Deccan AI
Deccan AI is a GenAI data company that runs the post-training and production layer for the world's leading AI labs and enterprises — from expert feedback and RL environments to managed AI workforces and evaluation infrastructure. Backed by A91 Partners, SIG (Susquehanna International Group), and Prosus Ventures, and having grown 10x last year, Deccan works with a majority of the Magnificent 7 and the top frontier labs globally. Its global network of 1 million+ domain experts powers Helix, its hybrid eval suite, and EnterpriseOS, its scaled ops automation platform. Headquartered in Mountain View, CA.
About the Role
We are hiring a Quality Associate - AI/ML who understands the quality of model training datasets from an ML perspective - its distribution, its hard cases, its hidden weaknesses - and who designs the pipelines that surface those signals which precedes human-in-the-loop quality. Your work will shape how quality is measured, reported, and improved across every project we ship to our frontier-lab clients.
Roles & Responsibilities
- Analyze model training datasets to enhance quality.
- Translate ML-relevant observations into quality criteria for human-in-the-loop interventions.
- Build judges and automated signal pipelines that pre-screen every dataset before human audit.
- Continuously calibrate these systems against human gold sets so they track real quality, not just measurable proxies.
- Decide where automation is sufficient and where human judgment is needed; design the triage logic between the two.
- Establish the statistical foundations for how quality is measured - sampling, confidence intervals, defect-rate methodology.
- Produce per-batch reports and dataset-level insight packs that characterize what each dataset is, not just whether it passed.
Requirements
Must-have
- Obsessive attention to granular details and specificity in reviewing data samples
- Strong ML literacy - ability to provide ML signals for data quality
- Solid programming proficiency in Python; comfortable building pipelines, working with LLM APIs, and writing prompt-engineering at scale.
- Hands-on experience training or fine-tuning ML models with different annotation types and modalities
- Statistical foundation - sampling theory, confidence intervals, hypothesis testing.
- Sharp attention to detail and critical thinking - able to spot patterns, anomalies, and edge cases that others miss in large datasets.
- Analytical communication: able to translate ML-driven findings into conclusions humans across the company can act on.
Good-to-have
- Prior work with LLM-as-Judge systems or automated evaluation frameworks.
- Familiarity with annotation tooling or eval frameworks.
- Background in quality systems, process excellence, or adjacent QA disciplines (Six Sigma, ISO, clinical or manufacturing QA).
Benefits
- Opportunity to work on cutting-edge AI/ML projects with frontier-lab clients.
- Collaborative and inclusive work environment.
- Career growth and professional development opportunities.
- A culture that values innovation, creativity, and operational excellence.