Job description

We are seeking a Machine Learning, AI Chatbot Test Engineering Analyst to our client's team to drive innovation in AI-powered division leveraging Chatbots within their proprietary platform for external information. This is a great opportunity to work with cutting-edge AI and automation technologies. We are seeking a Test Engineer/SDET with 5+ years of experience to join our team and drive innovation in AI-powered test automation. This role focuses on building and maintaining intelligent test automation infrastructure on AWS and CloudBees, with emphasis on self-healing scripts and AI-driven testing solutions. Key Responsibilities - Design, develop, and maintain automated test scripts and frameworks - Deploy and configure reliable automation assets in CloudBees CI/CD pipelines - Troubleshoot and resolve technical issues with automation infrastructure - Create AI-based automation solutions using AWS Bedrock and frameworks like LangChain - Integrate AI agents and solutions with existing automation platforms - Implement self-healing test scripts and Intelligent Test Automation (ITA) strategies - Apply context engineering to ensure AI-driven testing relevance and coverage This position is a hybrid opportunity (1-2x onsite per week in Maryland and/or Wash DC office).- Required Skills Required Technical Skills - Programming Languages: Python, JavaScript/TypeScript, Java, SQL, Unix Shell - AI/ML Frameworks: AWS Bedrock, LangChain, prompt engineering - Test Automation: Selenium, Playwright, API testing frameworks - Cloud & DevOps: AWS services, Docker, Kubernetes, Git - CI/CD: CloudBees, Jenkins, pipeline automation - Infrastructure: Test environment setup and maintenance Key Qualifications - Bachelor's degree in Computer Science, IT, or related field - 5+ years of experience in software testing and test automation - Proven experience with AI tools for test generation, refactoring, and self-healing - Strong understanding of DevOps practices and CI/CD pipelines - Experience building and scaling testing agents - Knowledge of context engineering for AI-driven testing Preferred Skills - Experience testing or supporting AI/ML and Generative AI solutions, including LLMs, RAG architectures, prompt engineering, and AI model validation within enterprise applications. - AWS certifications - Experience with modern testing frameworks beyond Selenium - Background in machine learning or AI applications - Experience with microservices and containerized applications - Knowledge of infrastructure as code (Terraform, CloudFormation)

Requirements

Must-haves

Design, develop, and maintain automated test scripts and frameworks
Deploy and configure reliable automation assets in CloudBees CI/CD pipelines
Troubleshoot and resolve technical issues with automation infrastructure
Create AI-based automation solutions using AWS Bedrock and frameworks like LangChain
Integrate AI agents and solutions with existing automation platforms
Implement self-healing test scripts and Intelligent Test Automation (ITA) strategies
Apply context engineering to ensure AI-driven testing relevance and coverage
This position is a hybrid opportunity (1-2x onsite per week in Maryland and/or Wash DC office).-
Required Technical Skills
Programming Languages: Python, JavaScript/TypeScript, Java, SQL, Unix Shell
AI/ML Frameworks: AWS Bedrock, LangChain, prompt engineering
Test Automation: Selenium, Playwright, API testing frameworks
Cloud & DevOps: AWS services, Docker, Kubernetes, Git
CI/CD: CloudBees, Jenkins, pipeline automation
Infrastructure: Test environment setup and maintenance
Key Qualifications
Bachelor's degree in Computer Science, IT, or related field
5+ years of experience in software testing and test automation

Nice-to-haves

Experience testing or supporting AI/ML and Generative AI solutions, including LLMs, RAG architectures, prompt engineering, and AI model validation within enterprise applications.
AWS certifications
Experience with modern testing frameworks beyond Selenium
Background in machine learning or AI applications
Experience with microservices and containerized applications
Knowledge of infrastructure as code (Terraform, CloudFormation)

Application prep

Use the role’s language and application details while tailoring your materials.

Keywords to mirror

machine learningaiawsci/cdpythonjavascripttypescriptjavasqlmlapidockerkubernetes

Platform: careers-html
Effort: medium

Benefits

Benefits detected in the listing.

Vision