Job description

Must Have Technical/Functional Skills Must Have Skills: • Databricks (Notebooks, Delta Lake, Delta Live Tables, Unity Catalog) • Python and PySpark • Spark SQL and Apache Spark • ETL/Data Pipeline Development • Big Data technologies: Hadoop, Spark, Kafka, Hive, HDFS, Sqoop • Strong SQL querying and data modeling • Data ingestion, transformation, cleansing, and validation • Cloud platforms (AWS/GCP) • Data warehouse and large-scale data processing experience • Experience with streaming data using Kafka and Spark Streaming • Agile methodologies (Scrum, Kanban, SAFe) Preferred Skills: • Scala / Scala-Spark • IBM DataStage, Tibco, or similar ETL tools • AWS services: S3, Glue, Lambda, Kinesis/MSK • BI Tools: Tableau, Power BI, Spotfire, OAC • Databases: Teradata, DB2, BigQuery, Mainframe • Git version control • Data governance and retention modeling Roles & Responsibilities • Design, develop, and maintain scalable data pipelines and ETL solutions. • Build and optimize data ingestion frameworks for structured and unstructured data. • Implement data quality, validation, standardization, and governance processes. • Develop data infrastructure integrating multiple heterogeneous data sources. • Support analytics, reporting, machine learning, and business intelligence initiatives. • Work with Data Scientists and BI teams to prepare and curate data for analytics. • Develop scalable solutions for Big Data processing and streaming workloads. • Analyze data and provide actionable insights and recommendations to business stakeholders. • Gather requirements from business teams and translate them into technical solutions. • Lead and mentor junior data engineers. • Collaborate across multiple projects as the Data Engineering SME. • Drive adoption of Data Engineering best practices and standards. Salary Range-$100,000-$135,000 a year #LI-KR3 TCS Employee Benefits Summary: Discretionary Annual Incentive. Comprehensive Medical Coverage: Medical & Health, Dental & Vision, Disability Planning & Insurance, Pet Insurance Plans. Family Support: Maternal & Parental Leaves. Insurance Options: Auto & Home Insurance, Identity Theft Protection. Convenience & Professional Growth: Commuter Benefits & Certification & Training Reimbursement Time Off: Vacation, Time Off, Sick Leave & Holidays. Legal & Financial Assistance: Legal Assistance, 401K Plan, Performance Bonus, College Fund, Student Loan Refinancing.

Requirements

Must-haves

Must Have Technical/Functional Skills
Databricks (Notebooks, Delta Lake, Delta Live Tables, Unity Catalog)
Python and PySpark
Spark SQL and Apache Spark
ETL/Data Pipeline Development
Big Data technologies: Hadoop, Spark, Kafka, Hive, HDFS, Sqoop
Strong SQL querying and data modeling
Data ingestion, transformation, cleansing, and validation
Cloud platforms (AWS/GCP)
Data warehouse and large-scale data processing experience
Experience with streaming data using Kafka and Spark Streaming
Agile methodologies (Scrum, Kanban, SAFe)

Nice-to-haves

Scala / Scala-Spark
IBM DataStage, Tibco, or similar ETL tools
AWS services: S3, Glue, Lambda, Kinesis/MSK
BI Tools: Tableau, Power BI, Spotfire, OAC
Databases: Teradata, DB2, BigQuery, Mainframe
Git version control
Data governance and retention modeling
Roles & Responsibilities
Design, develop, and maintain scalable data pipelines and ETL solutions.
Build and optimize data ingestion frameworks for structured and unstructured data.
Implement data quality, validation, standardization, and governance processes.
Develop data infrastructure integrating multiple heterogeneous data sources.

Application prep

Use the role’s language and application details while tailoring your materials.

Keywords to mirror

pythonsqlawsgcpagilescrummachine learning

Platform: careers-page
Effort: medium

Benefits

Benefits detected in the listing.

Dental

Vision

401k

Parental Leave