Job description

Position: Senior QA Automation Engineer (AI-First) Location: Remote Contract Type: Full-time vendor (Direct contract with Inallmedia.com) Time Zone Alignment: Central Time (CT) ±2 hours 🧭 About Inallmedia.com Inallmedia.com is a global technology and design firm focused on building impactful digital solutions through remote, distributed teams across LATAM. We partner with international clients across industries, providing long-term technical expertise, product innovation, and team augmentation. 🚀 Project Overview You will join a forward-thinking engineering team building an innovative data center power management platform designed to optimize energy consumption and relieve grid pressure. As a Senior QA Automation Engineer (AI-First) , your primary focus will be designing, building, and enforcing a robust automated testing strategy across distributed systems, Python microservices, event pipelines, and non-deterministic agentic AI workflows. 🔍 Key Responsibilities • Comprehensive Test Strategy: Own the automated test strategy across unit, integration, contract, end-to-end, and edge-deployment validation. • UI & Workflows: Build and maintain end-to-end test suites in Playwright for operational UIs and human-in-the-loop approval workflows. • Backend & Pipeline Coverage: Build API, contract, and integration testing for Python microservices and Kafka event pipelines , including replay and failure-injection scenarios. • CI/CD Quality Enforcement: Integrate all test suites into CI as enforced quality gates with strict thresholds rather than advisory reports. • Suite Health & Root-Cause Triage: Monitor execution time, flake rate, and critical path coverage; perform root-cause analysis to permanently fix test flakiness rather than relying on simple retries. • AI & Agentic Evaluation: Build evaluation suites for agentic AI workflows (validating tool selection, arguments, escalation behavior, and operating envelope limits). • Non-Deterministic Testing: Define regression testing for non-deterministic components, including golden-set evaluations, tolerance thresholds, and drift detection post-prompt/model changes. • Edge & Boundary Verification: Automate test suites for edge deployments (offline/reconnection scenarios) and verify human-in-the-loop security boundaries under adversarial conditions. • Process & Mentorship: Convert operational SOPs and incomplete specs into executable acceptance criteria alongside Technical Business Analysts, and coach engineers on test ownership. 💡 Must-Have Skills • Experience Level: 5+ years of hands-on experience owning test automation for distributed systems. • Core Automation Stack: Deep production experience with Playwright (fixtures, parallelization, trace analysis, CI integration) and strong proficiency in both Python and TypeScript . • Backend & Messaging: Proven experience in API, contract, and integration testing, service virtualization/mocking, and event-driven architectures like Kafka . • CI/CD & Infrastructure: Practical experience setting up CI quality gates in GitHub Actions or Azure DevOps , with basic Kubernetes literacy for containerized testing environments. • AI & Problem-Solving: Hands-on experience with AI-assisted testing methodologies, critical judgment of AI-generated code/tests, and strong root-cause debugging skills. • English Proficiency: Fluent English for daily written and verbal communication. 🌟 Nice-to-Have Skills • AI Agents & Automation: Experience with Playwright MCP, AI coding agents running non-interactively in CI, or the Claude Agent SDK . • LLM Product Testing: Prior experience testing LLM or agent-based products in production, non-deterministic systems, or building MCP servers for test observability. • Resilience & Security Testing: Experience with performance/load testing for high-volume telemetry, chaos/fault-injection, or security testing in CI (dependency scanning/dynamic analysis). • Domain Knowledge: Background in energy systems, smart grid technology, industrial operations, or data center environments. 🌐 Time Zone & Collaboration The role requires collaboration with teams aligned to Central Time (CT). Flexibility to overlap with North America business hours is expected. 💬 Language All interviews, documentation, and daily communication will be in English. #LI

Requirements

Must-haves

Position: Senior QA Automation Engineer (AI-First)
Location: Remote
Contract Type: Full-time vendor (Direct contract with Inallmedia.com)
Time Zone Alignment: Central Time (CT) ±2 hours
Comprehensive Test Strategy: Own the automated test strategy across unit, integration, contract, end-to-end, and edge-deployment validation.
UI & Workflows: Build and maintain end-to-end test suites in Playwright for operational UIs and human-in-the-loop approval workflows.
Backend & Pipeline Coverage: Build API, contract, and integration testing for Python microservices and Kafka event pipelines , including replay and failure-injection scenarios.
CI/CD Quality Enforcement: Integrate all test suites into CI as enforced quality gates with strict thresholds rather than advisory reports.
Suite Health & Root-Cause Triage: Monitor execution time, flake rate, and critical path coverage; perform root-cause analysis to permanently fix test flakiness rather than relying on simple retries.
AI & Agentic Evaluation: Build evaluation suites for agentic AI workflows (validating tool selection, arguments, escalation behavior, and operating envelope limits).
Non-Deterministic Testing: Define regression testing for non-deterministic components, including golden-set evaluations, tolerance thresholds, and drift detection post-prompt/model changes.
Edge & Boundary Verification: Automate test suites for edge deployments (offline/reconnection scenarios) and verify human-in-the-loop security boundaries under adversarial conditions.
Process & Mentorship: Convert operational SOPs and incomplete specs into executable acceptance criteria alongside Technical Business Analysts, and coach engineers on test ownership.
Must-Have Skills
Experience Level: 5+ years of hands-on experience owning test automation for distributed systems.
Core Automation Stack: Deep production experience with Playwright (fixtures, parallelization, trace analysis, CI integration) and strong proficiency in both Python and TypeScript .
Backend & Messaging: Proven experience in API, contract, and integration testing, service virtualization/mocking, and event-driven architectures like Kafka .
CI/CD & Infrastructure: Practical experience setting up CI quality gates in GitHub Actions or Azure DevOps , with basic Kubernetes literacy for containerized testing environments.
AI & Problem-Solving: Hands-on experience with AI-assisted testing methodologies, critical judgment of AI-generated code/tests, and strong root-cause debugging skills.
English Proficiency: Fluent English for daily written and verbal communication.
AI Agents & Automation: Experience with Playwright MCP, AI coding agents running non-interactively in CI, or the Claude Agent SDK .
LLM Product Testing: Prior experience testing LLM or agent-based products in production, non-deterministic systems, or building MCP servers for test observability.
Resilience & Security Testing: Experience with performance/load testing for high-volume telemetry, chaos/fault-injection, or security testing in CI (dependency scanning/dynamic analysis).

Nice-to-haves

Nice-to-Have Skills

Application prep

Use the role’s language and application details while tailoring your materials.

Keywords to mirror

aipythonapici/cdtypescriptazurekubernetesllm

Platform: breezy
Effort: medium