Job description

Summary: We are seeking an experienced professional with 6-10 years of relevant experience in Azure Data Engineering to join our dynamic Data & Analytics Team. The position is open to any NTT Data location with temporary remote work, and the shift timing is from 02:00 PM IST to 11:00 PM IST. The ideal candidate will have strong hands-on experience with Microsoft Azure data services, Azure Synapse Analytics, and/or Microsoft Fabric, focusing on data engineering, integration, transformation, and enterprise data platforms. Responsibilities: • Design, develop, and maintain enterprise data solutions using Microsoft Azure. • Build scalable data ingestion, transformation, and processing solutions. • Develop and optimize data pipelines using Azure Data Factory, Azure Synapse Analytics, and/or Microsoft Fabric. • Design solutions that integrate data from databases, APIs, files, SaaS platforms, cloud applications, and enterprise systems. • Develop reusable data engineering frameworks and components. • Implement batch and near-real-time data processing solutions. • Monitor, troubleshoot, and optimize Azure data workloads. • Support modernization and migration of legacy data platforms to Azure. • Design and develop data solutions using Azure Synapse Analytics and/or Microsoft Fabric. • Build enterprise data warehouses and Lakehouse solutions. • Develop data pipelines and orchestration workflows. • Work with Synapse Pipelines, Fabric Data Factory, and Azure Data Factory. • Develop notebooks using Python, PySpark, and Spark SQL. • Build and optimize data transformation processes. • Support migration and modernization initiatives between traditional Azure architectures, Azure Synapse, and Microsoft Fabric. • Evaluate appropriate technologies based on performance, scalability, cost, security, and business requirements. • Design and develop robust ETL and ELT pipelines. • Ingest structured, semi-structured, and unstructured data from multiple enterprise sources. • Integrate data from databases, APIs, files, SaaS applications, and cloud platforms. • Develop reusable ingestion and transformation frameworks. • Implement incremental data loads and Change Data Capture (CDC) strategies. • Develop data validation, reconciliation, and error-handling processes. • Troubleshoot pipeline failures and data integration issues. • Optimize pipelines for performance, scalability, reliability, and cost. • Design and implement modern Data Lake, Lakehouse, and Data Warehouse architectures. • Work with Azure Data Lake Storage Gen2 and Microsoft OneLake. • Implement Bronze, Silver, and Gold data layers. • Design dimensional data models using Star and Snowflake schemas. • Develop fact and dimension tables. • Implement Slowly Changing Dimensions (SCD). • Work with Delta Lake and related technologies. • Implement partitioning and performance optimization strategies. • Support enterprise data warehouse modernization and migration initiatives. • Work with Microsoft Azure technologies including Azure Data Factory, Azure Synapse Analytics, Azure Data Lake Storage Gen2, Azure SQL Database, Azure SQL Managed Instance, Azure Functions, Azure Key Vault, Azure Event Hubs, Azure Storage, Microsoft Entra ID, and Microsoft Fabric. • Develop integrations between Azure services and enterprise applications. • Implement secure and scalable cloud data architectures. • Support cloud migration and modernization initiatives. • Monitor Azure resources and optimize platform performance and cost. • Apply cloud architecture and engineering best practices across data solutions. • Develop solutions using Fabric Lakehouse and Data Warehouse. • Build pipelines using Fabric Data Factory. • Develop Dataflows Gen2. • Work with OneLake. • Develop Fabric Notebooks using PySpark, Python, and Spark SQL. • Implement Delta Lake-based data architectures. • Support Direct Lake and Power BI integration. • Manage Fabric workspaces and deployment processes. • Monitor and optimize Fabric capacity utilization. • Develop scalable data transformations using Python, PySpark, and Spark SQL. • Process large datasets using distributed computing. Requirements: • Minimum 6 years of relevant experience in Azure Data Engineering components - Pyspark, SQL, Azure Data Factory, and Azure data services. • Must have experience with Azure Synapse Analytics or Microsoft Fabric (minimum 2 years). • Ability to work independently with minimal supervision and contribute to the team with hands-on development. Required Skills: • Strong hands-on experience with Microsoft Azure data services, Azure Synapse Analytics, and/or Microsoft Fabric. • Experience in data engineering, data integration, data transformation, and enterprise data platforms. • Proficiency in designing, developing, implementing, and optimizing scalable data pipelines, data warehouses, Lakehouses, and enterprise data solutions using the Microsoft Azure data ecosystem. • Significant experience working with Azure Synapse Analytics or Microsoft Fabric. Preferred Skills: • Experience across both Azure Synapse Analytics and Microsoft Fabric platforms is highly desirable. Benefits: We offer a competitive salary, comprehensive benefits package, and opportunities for professional growth and development within our organization.

Requirements

Must-haves

Design, develop, and maintain enterprise data solutions using Microsoft Azure.
Build scalable data ingestion, transformation, and processing solutions.
Develop and optimize data pipelines using Azure Data Factory, Azure Synapse Analytics, and/or Microsoft Fabric.
Design solutions that integrate data from databases, APIs, files, SaaS platforms, cloud applications, and enterprise systems.
Develop reusable data engineering frameworks and components.
Implement batch and near-real-time data processing solutions.
Monitor, troubleshoot, and optimize Azure data workloads.
Support modernization and migration of legacy data platforms to Azure.
Design and develop data solutions using Azure Synapse Analytics and/or Microsoft Fabric.
Build enterprise data warehouses and Lakehouse solutions.
Develop data pipelines and orchestration workflows.
Work with Synapse Pipelines, Fabric Data Factory, and Azure Data Factory.
Develop notebooks using Python, PySpark, and Spark SQL.
Build and optimize data transformation processes.
Support migration and modernization initiatives between traditional Azure architectures, Azure Synapse, and Microsoft Fabric.
Evaluate appropriate technologies based on performance, scalability, cost, security, and business requirements.
Design and develop robust ETL and ELT pipelines.
Ingest structured, semi-structured, and unstructured data from multiple enterprise sources.
Integrate data from databases, APIs, files, SaaS applications, and cloud platforms.
Develop reusable ingestion and transformation frameworks.
Implement incremental data loads and Change Data Capture (CDC) strategies.
Develop data validation, reconciliation, and error-handling processes.
Troubleshoot pipeline failures and data integration issues.

Nice-to-haves

Experience across both Azure Synapse Analytics and Microsoft Fabric platforms is highly desirable.

Application prep

Use the role’s language and application details while tailoring your materials.

Keywords to mirror

azurepythonsql

Platform: jobdiva
Effort: medium

Benefits

Benefits detected in the listing.

Vision