Senior Data Engineer at Clera

Job description

ABOUT THE ROLE Join an early-stage data infrastructure company building a platform that transforms information from multiple sources into reliable, current company intelligence. You will own the data platform end to end, from architecture and ingestion through production operations, helping ensure trusted data powers downstream products. WHAT YOU'LL DO - Design and build a multi-source platform for concurrent ingestion, event-driven processing, and orchestrated batch workflows. - Develop resilient intake from provider APIs, webhooks, files, and web scraping, including rate limits and failure handling. - Orchestrate scheduled workflows and historical backfills with tools such as Airflow, Dagster, or Prefect. - Build event-driven processing with messaging or task queues, including retries, dead-letter queues, and controlled replay. - Store original payloads in Google Cloud Storage and use dbt to transform data into raw, standardized, and curated layers. - Implement idempotent upserts, data contracts, automated validation, pipeline tests, and monitoring for freshness and failures. - Design and optimize PostgreSQL schemas and loading patterns for downstream use. - Integrate provider enrichment and LLM-based structured extraction into the platform. WHAT WE'RE LOOKING FOR - At least 5 years building and operating production data platforms, with ownership from architecture through deployment and ongoing support; 8 or more years is a plus. - Production experience with workflow orchestration tools such as Airflow, Dagster, Prefect, or Temporal, including dependencies, retries, and recovery. - Hands-on production experience with dbt and Google Cloud Platform. - Strong Python skills, including asynchronous HTTP and concurrent processing with tools such as asyncio, aiohttp, or HTTPX. - Strong SQL and PostgreSQL experience in data modeling, query optimization, and incremental loading. - Experience building event-driven systems and implementing data contracts, retention policies, validation, testing, and monitoring. - Web scraping, entity resolution, data enrichment, and LLM-based extraction experience are helpful. - Comfort working with broad ownership and changing priorities in a small team; BigQuery and Dataform familiarity is a plus. COMPENSATION & BENEFITS Base salary: $120,000 to $160,000 USD annually. Visa sponsorship is not available. LOCATION Remote within the United States. New York-based candidates are welcome, but New York residence is not required.

Requirements

Must-haves

Design and build a multi-source platform for concurrent ingestion, event-driven processing, and orchestrated batch workflows.
Develop resilient intake from provider APIs, webhooks, files, and web scraping, including rate limits and failure handling.
Orchestrate scheduled workflows and historical backfills with tools such as Airflow, Dagster, or Prefect.
Build event-driven processing with messaging or task queues, including retries, dead-letter queues, and controlled replay.
Store original payloads in Google Cloud Storage and use dbt to transform data into raw, standardized, and curated layers.
Implement idempotent upserts, data contracts, automated validation, pipeline tests, and monitoring for freshness and failures.
Design and optimize PostgreSQL schemas and loading patterns for downstream use.
Integrate provider enrichment and LLM-based structured extraction into the platform.
Production experience with workflow orchestration tools such as Airflow, Dagster, Prefect, or Temporal, including dependencies, retries, and recovery.
Hands-on production experience with dbt and Google Cloud Platform.
Strong Python skills, including asynchronous HTTP and concurrent processing with tools such as asyncio, aiohttp, or HTTPX.
Strong SQL and PostgreSQL experience in data modeling, query optimization, and incremental loading.
Experience building event-driven systems and implementing data contracts, retention policies, validation, testing, and monitoring.
Web scraping, entity resolution, data enrichment, and LLM-based extraction experience are helpful.
Remote within the United States. New York-based candidates are welcome, but New York residence is not required.

Nice-to-haves

At least 5 years building and operating production data platforms, with ownership from architecture through deployment and ongoing support; 8 or more years is a plus.
Comfort working with broad ownership and changing priorities in a small team; BigQuery and Dataform familiarity is a plus.

Application prep

Use the role’s language and application details while tailoring your materials.

Keywords to mirror

llmpythonsql

Platform: ashby
Effort: quick