Job description

We're currently hiring for remote team members in the following states: DC, DE, GA, MA, MD, NC, NH, NJ, OH, PA, SC, UT, and VA. We're AWeber, a remote-first company that helps small businesses around the world form meaningful connections with their customers and grow their businesses using our marketing communication software. We are fanatical about small businesses and we hope you are too! At AWeber, agentic development is core to how we build. Our engineers spend most of their time designing solutions for agents to implement and reviewing agent-generated work for quality and correctness. On the QA team, you build the agents that do that work. As a QA Engineer II, you will spend far less time hand-writing test cases than you would elsewhere, and far more time building and improving the agents that generate our coverage, diagnose our failures, and watch our systems, then proving that what they produce can be trusted. We deploy continuously, many times a day, across a platform that small businesses run their livelihoods on. That makes your judgment the thing that protects them. You own quality end to end: the test strategy, the automated coverage, the exploratory testing that finds what automation never will, real-time monitoring of live systems, and the call on whether something is ready to ship. Your mission is to make our quality signals trustworthy, so that a green build actually means something and the team can move fast on top of it. When you find a gap in the process, we expect you to fix it and tell people, not just file it. On any given day, you could be found writing the test plan for a feature before a line of it exists, reviewing an agent-generated test suite and rejecting it because the assertions do not prove the behavior they claim to, exploring a new release by hand and finding the edge case nobody specified, tightening an agent's harness so it stops producing flaky waits, diagnosing why a pipeline job fails only in staging, or writing up the cause of a failure so the next person does not have to rediscover it. You'll thrive as a QA Engineer II if you Treat a passing test suite as a claim to be verified rather than a result to be trusted, especially when an agent wrote it. Take a project from start to finish on your own, resolve what you can independently, and escalate early on what you cannot. Are comfortable being the person who says a release is not ready, and can make that case with evidence instead of instinct. Investigate root causes rather than just rerunning a test, and write down what you learned so it compounds for the team. Get genuinely curious about how a system breaks, and go looking for the edge cases nobody wrote a ticket for. Default to proactive communication and understanding the value that comes from working in the open. See a broken process and fix it, then tell people it changed, instead of routing around it quietly. Stay steady when several priorities compete and a release is in flight, and recover fast when something gets through. Embrace emerging technology while staying skeptical enough to ask what it costs, how it fails, and how we would know. Required skills and qualifications 4+ years in QA or test automation in an Agile environment, including hands-on experience building test automation frameworks and not only writing scripts inside someone else's. API testing with Python, using pytest, unittest, or a comparable framework. UI testing with Cypress and TypeScript, including the discipline of stable selectors and reliable page objects. Demonstrated use of AI development tools and coding assistants to produce, review, or maintain tests, and the judgment to evaluate what they generate rather than accept it. Solid understanding of CI/CD pipelines, ideally GitHub Actions, with the ability to diagnose a failing job rather than just report it. Exploratory testing skill: finding meaningful defects in a running product without a script to follow. Experience with Docker and containerized environments, Git and modern version control workflows, and API tooling such as Postman or curl. Working experience with AWS services in a testing or deployment context. Ability to apply proven QA techniques including requirement traceability, boundary value analysis, and application stability and usability testing. Strong written communication. You will document test plans, failures, and sign-offs for an audience that includes both engineers and agents. Proven ability to work self-directed in a deadline-driven environment, and the willingness to help more junior QA engineers on testing platforms and procedures. Preferred qualifications Experience building agents and not only using them: writing and iterating on prompts, wiring up tool use, and getting a multi-step agent loop to run reliably against a real system. Experience evaluating agent or LLM output systematically, whether through eval harnesses, scoring rubrics, or regression detection. Experience testing in a high-volume agentic or AI-assisted codebase, including the verification and review challenges that come with it. Experience with real-time monitoring and observability tooling, and using production signals to find quality problems before customers report them. Experience with Kubernetes, or testing services deployed across multiple environments with differing states. Familiarity with performance, load, or security testing. Experience in a remote-first, asynchronous engineering culture. Background in one of our domains: email or marketing SaaS, web platforms, or backend services at scale. Working knowledge of the Atlassian stack (Jira, Confluence). AWeber Benefits 100% Remote - we're not going back to the office and here's why . Strong culture that supports flexibility, entrepreneurialism, and collaboration. Think we should be doing something different? Share your feedback! We’re looking for you to add your AWesomeness to our culture. We have some other pretty cool benefits to make your life easier: 100% Company Paid PPO medical, dental, vision insurance (spousal and domestic partner benefits available). 4-7 weeks of paid time off and holidays (based on tenure). 4 week paid sabbatical (based on tenure). 401K retirement plan with 4% company match. Company Profit Share. Home office equipment and internet stipend. Tuition reimbursement, conferences, and learning opportunities. Gym Memberships Reimbursement. Company Paid Short Term Disability Insurance. Company Paid Life Insurance. Additional Supplemental Benefits (Long Term Disability, Critical Illness, and Additional Life Insurance).

Requirements

Must-haves

We're currently hiring for remote team members in the following states: DC, DE, GA, MA, MD, NC, NH, NJ, OH, PA, SC, UT, and VA.
You'll thrive as a QA Engineer II if you
Treat a passing test suite as a claim to be verified rather than a result to be trusted, especially when an agent wrote it.
Take a project from start to finish on your own, resolve what you can independently, and escalate early on what you cannot.
Are comfortable being the person who says a release is not ready, and can make that case with evidence instead of instinct.
Investigate root causes rather than just rerunning a test, and write down what you learned so it compounds for the team.
Get genuinely curious about how a system breaks, and go looking for the edge cases nobody wrote a ticket for.
Default to proactive communication and understanding the value that comes from working in the open.
See a broken process and fix it, then tell people it changed, instead of routing around it quietly.
Stay steady when several priorities compete and a release is in flight, and recover fast when something gets through.
Embrace emerging technology while staying skeptical enough to ask what it costs, how it fails, and how we would know.
Required skills and qualifications

Nice-to-haves

Experience building agents and not only using them: writing and iterating on prompts, wiring up tool use, and getting a multi-step agent loop to run reliably against a real system.
Experience evaluating agent or LLM output systematically, whether through eval harnesses, scoring rubrics, or regression detection.
Experience testing in a high-volume agentic or AI-assisted codebase, including the verification and review challenges that come with it.
Experience with real-time monitoring and observability tooling, and using production signals to find quality problems before customers report them.
Experience with Kubernetes, or testing services deployed across multiple environments with differing states.
Familiarity with performance, load, or security testing.
Experience in a remote-first, asynchronous engineering culture.
Background in one of our domains: email or marketing SaaS, web platforms, or backend services at scale.
Working knowledge of the Atlassian stack (Jira, Confluence).
AWeber Benefits
100% Remote - we're not going back to the office and here's why .
Strong culture that supports flexibility, entrepreneurialism, and collaboration. Think we should be doing something different? Share your feedback! We’re looking for you to add your AWesomeness to our culture.

Application prep

Use the role’s language and application details while tailoring your materials.

Keywords to mirror

goagileapipythontypescriptaici/cddockerawsllmkubernetesjira

Platform: greenhouse
Effort: quick

Benefits

Benefits detected in the listing.

Dental

Vision

401k

Paid Time Off