Job description

ABOUT THE ROLE: As a Data Engineer, you will play a key role in designing and building scalable, real-time and batch data solutions that power analytics and operational platforms across NBCUniversal Operations & Technology. This role will focus on building and maintaining API connectors, setting up event-driven data pipelines, managing data flow into an event house or lakehouse, and ensuring data quality and reliability throughout. You will collaborate with cross-functional teams to enable the efficient movement and transformation of data for enterprise use. • API Connector Development: Build, maintain, and document scalable API integrations with internal and external platforms to automate data ingestion and synchronization. • Event-Driven Architecture: Design and implement data pipelines using event-driven frameworks and messaging systems (e.g., Kafka, Kinesis) to support real-time analytics and operational use cases. • Data Lakehouse Engineering: Manage data ingestion and transformation pipelines feeding into an event house or lakehouse architecture, ensuring consistency and scalability across datasets. • Data Transformation & Orchestration: Use ETL/ELT tools and scripting to clean, transform, and enrich data from various sources into usable formats for analysts and stakeholders. • Data Quality & Validation: Establish automated validation rules, monitor data integrity, and proactively resolve quality issues to ensure trust in data systems. • Collaboration & Enablement: Partner with analytics, business, and engineering teams to understand data requirements and deliver scalable solutions that meet evolving needs. REQUIREMENTS: • Bachelor’s degree in Computer Science, Engineering, Data Science, or a related field, or equivalent experience. • 3-6 years of experience as a data engineer or in a similar backend data role. • Experience building and maintaining API integrations and event-driven data pipelines. • Proficiency in SQL and data pipeline scripting (e.g., Python, PySpark). • Experience with cloud platforms such as AWS, Azure, or Google Cloud (e.g., S3, Redshift, Snowflake, or Databricks). • Familiarity with event streaming platforms such as Apache Kafka, Kinesis, or similar. • Strong attention to detail with a commitment to data accuracy and quality assurance. • Ability to thrive in a fast-paced, collaborative environment. PREFERRED QUALIFICATIONS: • Experience with CI/CD pipelines and infrastructure-as-code tools (e.g., Terraform, GitHub Actions). • Familiarity with lakehouse architecture principles and tools like Delta Lake or Apache Iceberg. • Experience building robust logging, error handling, and monitoring for data pipelines. • Understanding of data governance, privacy, and compliance principles. • Strong documentation and communication skills, with the ability to explain technical solutions to non-technical stakeholders. • Background in media, entertainment, or consumer technology environments.

Requirements

Must-haves

API Connector Development: Build, maintain, and document scalable API integrations with internal and external platforms to automate data ingestion and synchronization.
Event-Driven Architecture: Design and implement data pipelines using event-driven frameworks and messaging systems (e.g., Kafka, Kinesis) to support real-time analytics and operational use cases.
Data Lakehouse Engineering: Manage data ingestion and transformation pipelines feeding into an event house or lakehouse architecture, ensuring consistency and scalability across datasets.
Data Transformation & Orchestration: Use ETL/ELT tools and scripting to clean, transform, and enrich data from various sources into usable formats for analysts and stakeholders.
Data Quality & Validation: Establish automated validation rules, monitor data integrity, and proactively resolve quality issues to ensure trust in data systems.
Collaboration & Enablement: Partner with analytics, business, and engineering teams to understand data requirements and deliver scalable solutions that meet evolving needs.
Bachelor’s degree in Computer Science, Engineering, Data Science, or a related field, or equivalent experience.
3-6 years of experience as a data engineer or in a similar backend data role.
Experience building and maintaining API integrations and event-driven data pipelines.
Proficiency in SQL and data pipeline scripting (e.g., Python, PySpark).
Experience with cloud platforms such as AWS, Azure, or Google Cloud (e.g., S3, Redshift, Snowflake, or Databricks).
Familiarity with event streaming platforms such as Apache Kafka, Kinesis, or similar.
Strong attention to detail with a commitment to data accuracy and quality assurance.
Ability to thrive in a fast-paced, collaborative environment.

Nice-to-haves

Experience with CI/CD pipelines and infrastructure-as-code tools (e.g., Terraform, GitHub Actions).
Familiarity with lakehouse architecture principles and tools like Delta Lake or Apache Iceberg.
Experience building robust logging, error handling, and monitoring for data pipelines.
Understanding of data governance, privacy, and compliance principles.
Strong documentation and communication skills, with the ability to explain technical solutions to non-technical stakeholders.
Background in media, entertainment, or consumer technology environments.

Application prep

Use the role’s language and application details while tailoring your materials.

Keywords to mirror

apisqlpythonawsazureci/cd

Platform: careers-browser
Effort: medium