Job description

Must Have Technical/Functional Skills Associates with 7-10 years' experience in data/ETL testing with hands-on PySpark and exposure to big data tools. Ability to perform below responsibilities: 1. Design scalable PySpark-based test architectures for ETL/data pipelines, including modular frameworks for batch processing. 2. Architect end-to-end data validation systems in Hadoop environment for lineage, schema evolution 3. Lead system design for Hadoop/Hive test environments, including YARN resource management, dynamic partitioning. 4. Exposure to Zephyr-Jira-ServiceNow integrated test management systems with experience on API-driven automation. 5. Design CI/CD test pipelines for PySpark/Hadoop jobs, incorporating artifact management, parallel execution, and blue-green deployments. 6. Create data quality system designs using PySpark integrated with Hive metadata services. 7. Design testing platforms, test data generators 8. Mentor juniors on PySpark testing basics, contribute to testing strategy discussions Salary Range: $100,000 to $125,000 per year

Requirements

Must-haves

Must Have Technical/Functional Skills
Associates with 7-10 years' experience in data/ETL testing with hands-on PySpark and exposure to big data tools.
Design scalable PySpark-based test architectures for ETL/data pipelines, including modular frameworks for batch processing.
Architect end-to-end data validation systems in Hadoop environment for lineage, schema evolution
Lead system design for Hadoop/Hive test environments, including YARN resource management, dynamic partitioning.
Exposure to Zephyr-Jira-ServiceNow integrated test management systems with experience on API-driven automation.
Design CI/CD test pipelines for PySpark/Hadoop jobs, incorporating artifact management, parallel execution, and blue-green deployments.
Create data quality system designs using PySpark integrated with Hive metadata services.
Design testing platforms, test data generators
Mentor juniors on PySpark testing basics, contribute to testing strategy discussions

Nice-to-haves

None flagged.

Application prep

Use the role’s language and application details while tailoring your materials.

Keywords to mirror

jiraapici/cd

Platform: careers-page
Effort: medium