Senior Test Engineer - AI ML

While technology is the heart of our business, a global and diverse culture is the heart of our success. We love our people and we take pride in catering them to a culture built on transparency, diversity, integrity, learning and growth. If working in an environment that encourages you to innovate and excel, not just in professional but personal life, interests you- you would enjoy your career with Quantiphi! ****Role:** Senior Test Engineer - AI/ML** ****Experience:** 3-6 years** ****Location:** Mumbai/Bangalore (Hybrid)** ****Key Responsibilities:**** * Design, develop, and maintain UI and API automation test suites using modern automation frameworks. * Independently own end-to-end testing activities including test planning, automation development, execution, defect management, and reporting. * Validate AI/ML applications, Generative AI solutions, LLM-powered applications, and intelligent workflows. * Design and execute functional, integration, regression, and end-to-end test automation for traditional and AI-enabled applications. * Develop and maintain reusable automation frameworks, libraries, and utilities. * Automate API testing using REST Assured (Java) or Python Requests. * Develop UI automation using Playwright. * Validate backend data using SQL and perform database verification. * Develop automated validations for AI model outputs, prompt responses, RAG pipelines, and chatbot conversations. * Validate AI responses for correctness, consistency, relevance, grounding, safety, hallucinations, and bias. * Collaborate with ML Engineers, Software Engineers, and Product Owners to define AI quality validation strategies. * Integrate automated test suites into CI/CD pipelines and implement quality gates. * Leverage GenAI tools to accelerate test case generation, automation development, debugging, documentation, and productivity. * Contribute to AI testing frameworks, accelerators, and Quality Engineering best practices. * Mentor junior engineers and promote automation-first and AI-first Quality Engineering practices. **Must have skills:** * Strong understanding of Software Testing Life Cycle (STLC), SDLC, Agile, and Scrum methodologies. * Hands-on experience in designing test strategies, test scenarios, test cases, and test execution. * Strong expertise in UI automation using Playwright and API automation using REST Assured (Java) or Python Requests. * Hands-on experience with Postman for API testing and validation. * Strong SQL skills with hands-on experience in database testing and backend data validation. * Experience integrating automation suites with CI/CD pipelines using Jenkins, GitHub Actions, or similar. * Proficiency with Git and version control best practices. * Hands-on experience testing AI/ML or Generative AI applications. * Understanding of Large Language Models (LLMs), Retrieval-Augmented Generation (RAG), chatbots, copilots, or AI agents. * Experience validating AI responses for accuracy, relevance, consistency, hallucinations, toxicity, bias, and safety. * Experience designing test scenarios for prompt engineering and AI workflow validation. * Understanding of AI evaluation techniques and automated output validation. * Familiarity with AI evaluation frameworks or libraries (e.g., DeepEval, Ragas, LangSmith, Promptfoo, MLflow, or similar). * Experience using GenAI tools such as ChatGPT, GitHub Copilot, Kiro, Claude, or similar to accelerate test design, automation development, debugging, test data generation, and documentation. * Strong debugging, root cause analysis, and analytical problem-solving skills. * Ability to independently own end-to-end testing activities with minimal supervision. * Excellent communication, collaboration, and stakeholder management skills. * Ability to mentor junior engineers and promote automation and Quality Engineering best practices. * Strong attention to detail with a quality-first and automation-first mindset. ****Nice to have skills**** * **Knowledge of AWS services such as Amazon Bedrock, SageMaker, EC2, Lambda, API Gateway, S3, RDS, DynamoDB, CloudWatch, ECS/EKS, and IAM.** * **Experience testing cloud-native applications deployed on AWS.** * **Experience in Data Warehouse, ETL, or Data Pipeline testing.** * **Experience with performance testing tools such as JMeter or k6.** * **Exposure to Docker, Kubernetes, and containerized application testing.** * **Relevant certifications in Playwright, AWS, Quality Engineering, or Agile are an added advantage.** _If you like wild growth and working with happy, enthusiastic over-achievers, you'll enjoy your career with us__!_