Requirement ID:
93954
Job Title:
AWS PySpark Databricks Developer
Job Type:
Contract
Duration:
6 - 9 months
Location:
REMOTE-EST HOURS
Job Description:
Role Descriptions: Terradata to Databricks migration
Essential Skills: Terradata to Databricks migration
Desirable Skills:
Keyword:
Skills: Digital : Python~Digital : Databricks~Digital : PySpark
Experience Required: 8 + Years
CANDIDATE MUST HAVE HANDSON EXP IN DATABRICKS, PYTHON AND PYSPARK
Job Title AWS PySpark Databricks Developer
Relevant Experience
(in Yrs) 5-6 years+
Technical/Functional Skills • 5–6 years of total experience in data engineering or big data development. • 2–3 years hands-on experience with Databricks and Apache Spark. • Proficient in AWS cloud services (S3, Glue, Lambda, EMR, Redshift, CloudWatch, IAM). • Strong programming skills in PySpark, Python, and optionally Scala. • Solid understanding of data lakes, lakehouses, and Delta Lake concepts. • Experience in SQL development and performance tuning. • Familiarity with Airflow, dbt, or similar orchestration tools is a plus. • Experience in CI/CD tools like Jenkins, GitHub Actions, or Code Pipeline. • Knowledge of data security, governance, and compliance frameworks.
Experience Required 5-6 years+
Roles & Responsibilities • Develop and maintain scalable data pipelines using Apache Spark on Databricks. • Build end-to-end ETL/ELT pipelines on AWS using services like S3, Glue, Lambda, EMR, and Step Functions. • Collaborate with data scientists, analysts, and business stakeholders to deliver high-quality data solutions. • Design and implement data models, schemas, and Lakehouse architecture in Databricks. • Optimize and tune Spark jobs for performance and cost-efficiency. • Integrate data from multiple structured and unstructured data sources. • Monitor and manage data workflows, ensuring data quality, consistency, and security. • Follow best practices in CI/CD, code versioning (Git), and DevOps practices for data applications. • Write clean, reusable, well-documented code using Python / PySpark / Scala.
Apply Now
Copyright Apex 2000 INC. All Rights Reserved.