Senior Data Engineer at Chicory

Job description

About the Company Chicory is the leading contextual advertising platform for CPG & grocery, transforming food content into dynamic commerce experiences that take consumers from inspiration to checkout. Our platform reaches 150 million grocery shoppers each month across a network of 7,000+ websites, food blogs, and apps, including The Kitchn, Food Network, MyFitnessPal, and powers 70+ leading retailers through direct integrations and retail media partnerships. By combining engaging media with contextually relevant, brand-safe content, we help CPG brands and retailers connect with consumers in an active mindset during high-intent moments—whether they’re discovering new recipes, planning meals, or building grocery lists. From curiosity to conversion, Chicory enables brands to show up in the moments that matter most, delivering full-funnel outcomes with ease. Recognized as a 2025 Best Workplace by Inc. Magazine and as a 2023 Fastest-Growing Private Company in America by Inc. 5000, we believe that what sets Chicory apart is our diverse experiences and skillsets coupled with our shared values. Learn more at www.chicory.co . About The Role Chicory is hiring a Senior Data Engineer to own and expand the data platform behind our advertising, campaign, product, and financial reporting. You’ll work with high-volume, real-time ad events, partner APIs and files, and internal business systems using GCP, BigQuery, Dataflow, Cloud Composer, Python, SQL, and Superset. The role covers the full data stack, from ingestion and orchestration through warehouse design, data marts, and BI, giving you the opportunity to see how data moves from our advertising platform into reports used across the company. As Chicory’s Senior Data Engineer, you will have the autonomy to define the next version of our data lake and warehouse architecture and lead its implementation. You’ll remain hands-on, building pipelines and data models, establishing clear metric definitions, and improving how teams access and use data. You will work closely with backend and cloud engineers, as well as Campaign Management, Finance, Product, and leadership. Your work will directly support campaign performance, revenue reporting, publisher payments, and product decisions. This role offers broad technical ownership and the opportunity to establish the engineering practices that will support Chicory’s data systems and future data team. Responsibilities Map how ad requests, bids, impressions, clicks, campaign metadata, spend, revenue, publisher payments, partner files, and product events move from their source into BigQuery and Superset Choose and implement the new BigQuery structure, including raw, staging, core, and reporting datasets; table grain and keys; naming; partitioning and clustering; incremental loads; and ownership Build and maintain batch and streaming ETL and ELT pipelines with Python, SQL, Apache Beam, and Dataflow using data from Pub/Sub, Cloud Storage, APIs, databases, and files Build and maintain Cloud Composer and Apache Airflow DAGs with clear dependencies, useful retries and alerts, and tasks that can be rerun safely Choose and implement a version-controlled SQL transformation workflow using Dataform, dbt, or an equivalent tool, then move important calculations out of scheduled queries and Superset dashboards Build the BigQuery tables and data marts used for campaign delivery, spend and revenue reconciliation, publisher payments, partner reporting, product analytics, and leadership reporting Investigate mismatches between source systems, partner reports, BigQuery, and Superset; correct the logic, repair affected data, and document why the numbers changed Add tests and alerts for late or missing data, duplicate records, unexpected schema changes, broken joins, and totals that do not reconcile Maintain the Superset datasets that power business reporting, review expensive or confusing queries, and make sure shared metrics come from the same maintained model Review BigQuery usage and cost, then improve the tables, queries, schedules, storage, or retention rules responsible for avoidable spend Manage data access with GCP IAM, limit access to sensitive fields, and document retention or deletion rules where they apply Use Git, automated tests, code review, deployment pipelines, and infrastructure as code so changes can be reviewed, repeated, and rolled back Work directly with Engineering, Campaign Management, Finance, Product, Sales, and leadership to define calculations, verify new tables, and decide when old reports can be retired When a pipeline fails or a report looks wrong, lead the investigation and recovery. Leave runbooks, diagrams, model descriptions, and metric definitions that another engineer can follow Build upon team and company culture and act as a champion of the Chicory Principles Qualifications Demonstrated success designing, rearchitecting, and operating a production analytical data platform that includes a data lake or durable raw layer, a cloud data warehouse, governed data models, and business-facing data marts Deep hands-on knowledge of BigQuery, including schema and table design, partitioning and clustering, query plans and performance, workload patterns, permissions, retention, and cost management Strong Python and advanced SQL skills, including the ability to write maintainable, tested production code and review work across ingestion, transformation, orchestration, and data-serving layers Hands-on production experience building batch and streaming systems with Apache Beam and Dataflow and with Apache Airflow and Cloud Composer, including retries, idempotency, dependency management, duplicate and late-arriving events, schema evolution, replay, backfills, monitoring, and failure recovery Strong data-modeling judgment across dimensional, event, and domain models, with experience creating stable facts, dimensions, slowly changing dimensions, semantic definitions, and reusable analytical datasets Track record of establishing automated data quality, reconciliation, observability, lineage, alerting, service-level objectives, incident response, and operational runbooks for critical data products Experience enabling BI and self-service analytics through Superset or a comparable platform, including curated datasets, governed metrics, access patterns, dashboard performance, and stakeholder education Applied understanding of data governance, least-privilege access, privacy and sensitive-data handling, auditability, retention, and the controls required for financial or revenue-impacting data Experience with software-engineering practices for data systems, including Git, automated testing, CI/CD, code review, infrastructure as code such as Terraform, and clear technical documentation Demonstrated ability to operate as an autonomous senior technical owner: assess an unfamiliar environment, make architectural decisions, sequence migrations, resolve ambiguity, lead incidents, and communicate tradeoffs with both engineers and non-technical stakeholders The base salary range for this position is $160,000 - $185,000 annually. Location: New York/Hybrid or Remote Benefits & Perks Focusing on your health and wellbeing: Comprehensive healthcare benefits, including dental + vision plans Flexible paid time off policy for vacation, sick days, mental health days, and more! 14 paid holidays + flexibility to take paid time off for any other holidays you observe Employer-paid membership to One Medical Free ClassPass membership Creating a culture in which you can thrive: Annual learning & development stipend Flexible remote and hybrid work style arrangements Fun and friendly office in NYC stocked with free snacks Virtual and in-person outings and fun activities with our amazing team Dog-friendly workplace Supporting you financially: Competitive compensation package, including base salary + stock options Flexible Spending Account (FSA) + Health Spending Account (HSA) 401k retirement plan Commuter benefits Discounted pet insurance Our Philosophy At Chicory, we know that we are strongest when every person is valued, respected and treated with dignity. We actively put aside assumptions and judgment and work to make space for all points of view. We do serious work but don’t take ourselves too seriously. We consider our playful spirit a competitive advantage to building better relationships and finding the smartest solutions. We produce the best results by hiring smart people and giving them the space, safety and the resources to do their work. We are building a company people love working with and working at. Our Commitment Statement to Diversity, Equity, Inclusion, & Belonging At Chicory, diversity, equity, inclusion, and belonging are key ingredients in our recipe for success. Our diverse experiences, backgrounds, identities, and ways of thinking elevate the level at which we operate; this allows us to innovate in old-school industries by looking not at how things have been done, but how they should be done. We are building a workplace built on merit and equity where every one of us has a voice at the table. Our Principles Be Good People: The success of our company is totally dependent on having good, smart people. Own It: We take pride in our work. When we see a problem, we step up. We hold ourselves, and each other, to the highest standard. We learn from mistakes, admit when we are wrong, and take full responsibility for delivering results, not just for our role, but for the mission. Separate Your Ego From Your Position: We speak up with conviction, but we stay open to being wrong. Leadership isn’t about winning arguments or defending turf; it’s about serving the mission. We challenge ideas without taking it personally, and we invite feedback without getting defensive. Our goal is the best outcome, not to “be right.” Find Reality: We pursue truth over comfort. We seek to understand before insisting on being understood. We dig beneath symptoms to uncover root causes, and we challenge assumptions, even our own. Clarity comes from curiosity, not certainty. We don’t settle for surface-level answers. We seek reality, even when it’s inconvenient. Have a Beginner’s Mind: Just because something has been done a certain way doesn’t mean it’s the best way. We stay curious, challenge assumptions, and question the status quo. We stay innovative by tackling problems from a fresh perspective. Bias for Action: Speed matters in business. We don’t wait for permission, perfect clarity, or endless alignment. We cut through noise, complexity, and politics to drive progress. When in doubt, we act, learn, and adapt because action beats indecision every time. Customer Obsessed: Our customers are our North Star. We listen closely, dig deep to understand their needs, and deliver 5-star service at every touchpoint. We build products that actually work: simple, intuitive, and designed to solve real problems. Above and Beyond: We don’t settle for “good enough,” we aim for exceptional. We go above and beyond what’s expected for our team, our customers, and ourselves. We stay curious and hungry. Every day is an opportunity to get better.

Requirements

Must-haves

Map how ad requests, bids, impressions, clicks, campaign metadata, spend, revenue, publisher payments, partner files, and product events move from their source into BigQuery and Superset
Choose and implement the new BigQuery structure, including raw, staging, core, and reporting datasets; table grain and keys; naming; partitioning and clustering; incremental loads; and ownership
Build and maintain batch and streaming ETL and ELT pipelines with Python, SQL, Apache Beam, and Dataflow using data from Pub/Sub, Cloud Storage, APIs, databases, and files
Build and maintain Cloud Composer and Apache Airflow DAGs with clear dependencies, useful retries and alerts, and tasks that can be rerun safely
Choose and implement a version-controlled SQL transformation workflow using Dataform, dbt, or an equivalent tool, then move important calculations out of scheduled queries and Superset dashboards
Build the BigQuery tables and data marts used for campaign delivery, spend and revenue reconciliation, publisher payments, partner reporting, product analytics, and leadership reporting
Investigate mismatches between source systems, partner reports, BigQuery, and Superset; correct the logic, repair affected data, and document why the numbers changed
Add tests and alerts for late or missing data, duplicate records, unexpected schema changes, broken joins, and totals that do not reconcile
Maintain the Superset datasets that power business reporting, review expensive or confusing queries, and make sure shared metrics come from the same maintained model
Review BigQuery usage and cost, then improve the tables, queries, schedules, storage, or retention rules responsible for avoidable spend
Manage data access with GCP IAM, limit access to sensitive fields, and document retention or deletion rules where they apply
Use Git, automated tests, code review, deployment pipelines, and infrastructure as code so changes can be reviewed, repeated, and rolled back
Work directly with Engineering, Campaign Management, Finance, Product, Sales, and leadership to define calculations, verify new tables, and decide when old reports can be retired
When a pipeline fails or a report looks wrong, lead the investigation and recovery. Leave runbooks, diagrams, model descriptions, and metric definitions that another engineer can follow
Build upon team and company culture and act as a champion of the Chicory Principles
Deep hands-on knowledge of BigQuery, including schema and table design, partitioning and clustering, query plans and performance, workload patterns, permissions, retention, and cost management
Strong Python and advanced SQL skills, including the ability to write maintainable, tested production code and review work across ingestion, transformation, orchestration, and data-serving layers
Strong data-modeling judgment across dimensional, event, and domain models, with experience creating stable facts, dimensions, slowly changing dimensions, semantic definitions, and reusable analytical datasets
Track record of establishing automated data quality, reconciliation, observability, lineage, alerting, service-level objectives, incident response, and operational runbooks for critical data products
Experience enabling BI and self-service analytics through Superset or a comparable platform, including curated datasets, governed metrics, access patterns, dashboard performance, and stakeholder education
Applied understanding of data governance, least-privilege access, privacy and sensitive-data handling, auditability, retention, and the controls required for financial or revenue-impacting data
Experience with software-engineering practices for data systems, including Git, automated testing, CI/CD, code review, infrastructure as code such as Terraform, and clear technical documentation
Comprehensive healthcare benefits, including dental + vision plans
Employer-paid membership to One Medical

Nice-to-haves

None flagged.

Application prep

Use the role’s language and application details while tailoring your materials.

Keywords to mirror

gcppythonsqlci/cdgo

Platform: greenhouse
Effort: quick

Benefits

Benefits detected in the listing.

Dental

Vision

401k

Pto

Paid Time Off