Job description

Big Data Engineer Join a dynamic team optimizing a modern, Lakehouse-focused analytics platform. This position blends strategic business insight with hands-on technical execution. You will collaborate with cross-functional partners to gather requirements, design pragmatic architectures, and deploy scalable streaming and ingestion pipelines end-to-end. Responsibilities: • Design, build, and maintain production ETL/ELT pipelines using Databricks, Apache Spark (PySpark/Spark SQL), and Delta Lake. • Implement Delta Live Tables (DLT) for streaming/batch workflows and set up data governance via Unity Catalog. • Integrate high-volume event streams using Apache Kafka and leverage AWS cloud infrastructure (S3, IAM). • Cleanse, transform, and curate large structured and semi-structured datasets to support BI reporting and machine learning needs. • Enforce data quality, operational observability, and engineering best practices in an Agile environment. Qualifications: • 5+ years in Data Engineering; strong proficiency in SQL, Spark, and Databricks. • Hands-on experience with Delta Lake, Delta Live Tables, Unity Catalog, Kafka, and AWS analytics services. • Background in Agile delivery and translating complex requirements into reliable models. • Preferred: Exposure to Snowflake, NoSQL databases (e.g., Cassandra), or enterprise messaging systems. Location & Work Schedule: This is a 6+ month auto-renewing contract. Open to 100% remote work for non-local candidates willing to work EST hours. Atlanta-based candidates will work a hybrid schedule (3 days in-office). Through Visionaire Partners, full-time W2 contractors gain access to a full benefits package (for contractors, spouses, domestic partners, and dependents), including a 401(k) with up to 4% match, medical, dental, vision, life insurance, short/long-term disability, critical illness, hospital indemnity, accident coverage, and Medical/Dependent Care FSAs. #LI-Hybrid

Requirements

Must-haves

Big Data Engineer
Design, build, and maintain production ETL/ELT pipelines using Databricks, Apache Spark (PySpark/Spark SQL), and Delta Lake.
Implement Delta Live Tables (DLT) for streaming/batch workflows and set up data governance via Unity Catalog.
Integrate high-volume event streams using Apache Kafka and leverage AWS cloud infrastructure (S3, IAM).
Cleanse, transform, and curate large structured and semi-structured datasets to support BI reporting and machine learning needs.
Enforce data quality, operational observability, and engineering best practices in an Agile environment.
5+ years in Data Engineering; strong proficiency in SQL, Spark, and Databricks.
Hands-on experience with Delta Lake, Delta Live Tables, Unity Catalog, Kafka, and AWS analytics services.
Background in Agile delivery and translating complex requirements into reliable models.
This is a 6+ month auto-renewing contract. Open to 100% remote work for non-local candidates willing to work EST hours. Atlanta-based candidates will work a hybrid schedule (3 days in-office).

Nice-to-haves

Preferred: Exposure to Snowflake, NoSQL databases (e.g., Cassandra), or enterprise messaging systems.

Application prep

Use the role’s language and application details while tailoring your materials.

Keywords to mirror

sqlawsmachine learningagile

Platform: careers-page
Effort: medium

Benefits

Benefits detected in the listing.

Dental

Vision

401(K)