Associate Lead - Testing

While technology is the heart of our business, a global and diverse culture is the heart of our success. We love our people and we take pride in catering them to a culture built on transparency, diversity, integrity, learning and growth. If working in an environment that encourages you to innovate and excel, not just in professional but personal life, interests you- you would enjoy your career with Quantiphi! ****Role:** Lead/Associate Lead – QA + MLOps & Generative AI** ****Experience:** 10+ years** ****Location:** Mumbai/Bangalore (Hybrid)** ****Key Responsibilities:**** ****AI/ML & GenAI Testing Strategy (AWS Ecosystem)**** **Define testing approaches for AI systems built on AWS services such as:** * **Amazon SageMaker** * **Amazon Bedrock** * **AWS Lambda** * **Amazon API Gateway** * **Amazon Kinesis** * **AWS Glue** * **Amazon S3** * **Amazon CloudWatch** **Design validation frameworks covering:** * **Model accuracy & performance validation** * **Data drift & concept drift detection** * **Hallucination detection for LLMs** * **Prompt robustness testing** * **RAG validation (retrieval accuracy + grounding)** * **Bias & fairness validation** * **Safety & toxicity testing** ****MLOps Quality Engineering (AWS-Centric)**** **Validate the end-to-end ML lifecycle including:** * **Data ingestion & feature pipelines** * **Model training & hyperparameter tuning** * **Model versioning & registry** * **Deployment validation** * **Canary & blue/green release validation** **Work with AWS-native services such as:** * **SageMaker Pipelines** * **SageMaker Model Monitor** * **SageMaker Feature Store** * **Bedrock model evaluation workflows** * **CloudWatch-based observability** **Implement CI/CD quality gates for ML pipelines integrated with AWS DevOps tools.** ****GenAI & Agentic AI Testing**** **Define quality engineering approaches for:** * **LLM-based applications using Amazon Bedrock** * **Prompt engineering validation** * **Multi-agent orchestration testing** * **Chatbot & Voice bot conversational testing** * **Intent classification validation** * **Conversation drift & fallback validation** * **API contract validation for LLM integrations** **Build reusable evaluation harnesses for:** * **BLEU / ROUGE scoring** * **Embedding similarity scoring** * **Response consistency** * **Safety scoring frameworks** ****Framework & Capability Development**** * **Design reusable AI testing accelerators** * **Create AWS-aligned AI test automation frameworks (Python-first)** * **Develop synthetic data generation strategies** * **Establish AI quality scorecards** * **Build an internal AI QA Center of Excellence** ****Client Engagement & Leadership**** * **Lead AI/ML quality strategy workshops** * **Perform AI risk & readiness assessments** * **Present quality architecture to CXOs** * **Drive QA transformation programs** * **Mentor QA teams on AWS-based AI testing** * **Own delivery for AI testing engagements end-to-end** **Must have skills:** ****Testing Expertise**** * **8–12+ years in Quality Engineering** * **Strong test strategy, automation & governance experience** * **Experience leading QA transformation initiatives** * **Experience building frameworks from scratch** ****AI/ML & GenAI Expertise**** * **Deep understanding of ML lifecycle** * **Experience testing ML models (NLP preferred)** * **Hands-on experience validating LLM applications** * **Strong understanding of:** * **Prompt engineering** * **RAG architecture** * **Embeddings** * **Bias & explainability** ****AWS AI/ML Expertise**** * **Hands-on experience with:** * **Amazon SageMaker (training, deployment, monitoring)** * **Amazon Bedrock (LLM integration & evaluation)** * **S3-based data pipelines** * **AWS IAM (security validation)** * **CloudWatch monitoring** * **Lambda & API Gateway integrations** * **AWS CI/CD (CodePipeline / CodeBuild preferred)** ****Understanding of:**** * **Infrastructure as Code (Terraform / CloudFormation)** * **Observability in AI systems** * **Cost monitoring for ML workloads** ****Technical Skills**** * **Python (mandatory)** * **Experience with ML libraries (Scikit-learn, TensorFlow, PyTorch)** * **Experience with LLM frameworks (LangChain, etc.)** * **API & automation testing frameworks** * **Git-based workflows** * **Leadership & Communication** * **Strong client-facing communication** * **Experience leading QA teams** * **Ability to create strategy decks & solution proposals** * **Strong stakeholder management** _If you like wild growth and working with happy, enthusiastic over-achievers, you'll enjoy your career with us__!_