Data Engineer

• Must Have Technical/Functional Skills
• Must have atleast 8+ years of extensive Data engineering experience in Databricks, ETL/ELT using data pipelines.
• SQL/Procesurr experience.
• Must have 8 years of hands-on development proficiency in implementing Databricks solutions using scala and spark with data frames and notebooks/sql
• Must have 8+ years of hands-on development and performance tuning/ enhancement experience in scala/pyspark and sql
• Must have strong experience with Databricks, including developing and optimizing spark jobs, data transformations, and data processing workflows
• Must have experience and a good understanding on the above topics
• Databricks delta lake storage
• Unity catalog
• Lakehouse architecture
• Medallion architecture
• Databricks serverless options
• Dataset and dataframe
• Databricks clusters configuration and sizing
• Databricks job orchestration
• Spark structured streaming
• Roles & Responsibilities
• Must have experience with complex and large volume data streaming and data transformations using Azure Databricks
• Must have extensive experience in datawarehouse / data lakehouse implementation
• Must have experience with monitoring, debugging, and resolving issues in kive Databricks jobs
• Must have strong hands on expertise in troubleshooting devops pipelines, azure , and AKS services
• The candidate shd be able to provide data ingestion, streaming and transformations solutions and do development work
• Experience with apache big data implementation with programming proficiency like pyspark, R or Java prior to
• Databricks is advantageous

None flagged.