Job description

Database Internals Engineer — Next-Gen Parallel Compute Engine Our Company At Teradata, we believe that people thrive when empowered with better information. Teradata's Autonomous Knowledge Platform activates enterprise intelligence by unifying data, knowledge, and business context to achieve tangible outcomes. With Teradata, organizations can provide agents with full context for impact when it matters. Our solution lets businesses connect and scale on premises, in the cloud, or through a hybrid approach. Teradata delivers real business value with AI. About the Role Join the team that built the original Teradata engine, and work on what comes next. Inside the Office of the CTO, our Advanced Research Team is exploring a new generation of massively parallel, decentralized compute — engine architecture where parallelism is a first principle rather than a later refinement. That choice shapes everything that follows: the structure of plans, the expression of operators, the organization of storage, and the movement of work and data across nodes. It is the kind of systems problem that can define a career, and we are approaching it with both ambition and rigor. This is a rare opportunity to work at the frontier of database internals, learn from the people who helped define the field, and leave your mark on a new engine as it takes shape. If you want greenfield systems work backed by the people, resources, and a problem set only a company like Teradata can offer, this is the place. What You'll Do You will be responsible for building core components of a next-generation parallel compute engine from the ground up: • Build components across the engine spine: SQL front end (tokenizer, parser, binder), logical/physical plan layer, rule- and cost-based optimizer, and operators (joins, aggregates, sort, scan). • Build the storage substrate: Arrow in-memory format, slotted-page on-disk format with checksums, buffer pool, and a B-tree or LSM access method. • Implement transactions and recovery: lock/latch management, MVCC/snapshot isolation, WAL/ARIES, checkpoints, and crash recovery. • Add parallelism and distribution along the correct axis — exchange-based parallel execution for query work, consensus/replication/atomic-commit for data correctness — without conflating the two. • Write design specifications before coding (schemas; null/empty/duplicate semantics; memory budget and spill behavior; cost characteristics), write tests first, implement behind the established operator interface, verify safety then speed, and report the benchmark delta. Who You Will Work With You will work within an intentionally small Advanced Research Team inside Teradata's Office of the CTO — meaning your contributions directly shape architecture and direction rather than passing through layers of process. You will collaborate with: • The architects and engineers behind one of the most successful massively parallel databases ever shipped, who have faced the hardest problems in this space and are now focused on what comes next. • Senior experts who are available to challenge your thinking, share lessons learned from building the first generation, and provide mentorship on deep systems problems. • Peers who share a commitment to rigor, quality, and first-principles engineering — a small team where everyone's work matters and is visible. What Makes You a Qualified Candidate We are open to two complementary profiles for this role: Profile A — Distributed Query Optimizer • Deep expertise in distributed query optimization: cascading optimizers, Abstract Syntax Tree (AST) binding, logical and physical plan distribution. • Experience designing parallel execution pipelines where distribution is a first-class concern, not an afterthought. • Familiarity with systems like Apache DataFusion or similar distributed execution frameworks. Profile B — Execution Engine & Database Internals • Strong hands-on background with analytical/embedded engines (e.g., DuckDB, DataFusion) and their internals. • Deep knowledge of pipeline execution, vectorized processing, file system I/O, and open table formats (Iceberg, Delta). • Experience with transaction management, lock managers, cache management, or OS-level scheduling. What You'll Bring Technical Requirements: • Systems programming in Rust or modern C/C++ with memory-safety discipline. • Relational foundations end to end: relational algebra and the algebraic laws behind query rewrites, logical→physical lowering, and cost-based optimization. You can walk the full lifecycle — text → tokens → AST → bound logical plan → optimized plan → physical plan → batches — and reason about cost at every step. • Demonstrated depth in at least one area of the engine spine (front end, plan/optimizer, operators, storage engine, transactions/recovery, or scale-out) with working literacy across its neighbors. • Vectorized execution over a columnar (Arrow) representation: batch-at-a-time operators, the pull/iterator model, validity-bitmap-correct null handling, zero-copy buffer sharing. • Cost reasoning at the metal: cache behavior, alignment, SIMD, allocation patterns, false sharing in parallel accumulators — you can articulate why one physical implementation beats another on a given CPU. • Test- and benchmark-gated engineering: golden/sqllogictest, property and fuzz testing, deterministic simulation testing for concurrent/distributed paths, and microbenchmarks (Criterion/Google Benchmark) plus a TPC-H subset. Correctness gates before performance, always. • Sophisticated, hands-on use of AI coding agents directed against a reference-grade spec — managed as a virtual engineering team, held to the same bar as the rest of the work. Soft Skills: • Communicates clearly — can describe a system or problem at the right level of abstraction for any audience, whether a senior architect, a peer, or an AI coding agent. • Doer over theorist — moves from concept to working prototype quickly; validates by building. • Intellectually curious and self-directed — digs deep independently before asking for help; brings answers and proposals. • Collaborative with peers — receptive to input and direction while owning your area end-to-end. • Comfortable with evolving requirements — writes design documents to create clarity. • High personal bar for quality. Why We Think You Will Love Teradata Teradata's architecture has run the world's largest data workloads for over 45 years — because it was engineered right. We are applying that same discipline to the next generation of compute. This is a rare chance to work on something foundational, collaborating directly with the people who did it the first time. We prioritize a people-first culture because we know our people are at the very heart of our success. We embrace a flexible work model because we trust our people to make decisions about how, when, and where they work. We focus on well-being because we care about our people and their ability to thrive both personally and professionally. We are an anti-racist company because our dedication to Diversity, Equity, and Inclusion is more than a statement — it is a deep commitment to doing the work to foster an equitable environment that celebrates people for all of who they are.

Requirements

Must-haves

Database Internals Engineer — Next-Gen Parallel Compute Engine
Join the team that built the original Teradata engine, and work on what comes next.
You will be responsible for building core components of a next-generation parallel compute engine from the ground up:
Build components across the engine spine: SQL front end (tokenizer, parser, binder), logical/physical plan layer, rule- and cost-based optimizer, and operators (joins, aggregates, sort, scan).
Build the storage substrate: Arrow in-memory format, slotted-page on-disk format with checksums, buffer pool, and a B-tree or LSM access method.
Implement transactions and recovery: lock/latch management, MVCC/snapshot isolation, WAL/ARIES, checkpoints, and crash recovery.
Add parallelism and distribution along the correct axis — exchange-based parallel execution for query work, consensus/replication/atomic-commit for data correctness — without conflating the two.
Who You Will Work With
The architects and engineers behind one of the most successful massively parallel databases ever shipped, who have faced the hardest problems in this space and are now focused on what comes next.
Senior experts who are available to challenge your thinking, share lessons learned from building the first generation, and provide mentorship on deep systems problems.
Peers who share a commitment to rigor, quality, and first-principles engineering — a small team where everyone's work matters and is visible.
What Makes You a Qualified Candidate
We are open to two complementary profiles for this role:
Profile A — Distributed Query Optimizer
Deep expertise in distributed query optimization: cascading optimizers, Abstract Syntax Tree (AST) binding, logical and physical plan distribution.
Experience designing parallel execution pipelines where distribution is a first-class concern, not an afterthought.
Familiarity with systems like Apache DataFusion or similar distributed execution frameworks.
Profile B — Execution Engine & Database Internals
Strong hands-on background with analytical/embedded engines (e.g., DuckDB, DataFusion) and their internals.
Deep knowledge of pipeline execution, vectorized processing, file system I/O, and open table formats (Iceberg, Delta).
Experience with transaction management, lock managers, cache management, or OS-level scheduling.
Systems programming in Rust or modern C/C++ with memory-safety discipline.
Demonstrated depth in at least one area of the engine spine (front end, plan/optimizer, operators, storage engine, transactions/recovery, or scale-out) with working literacy across its neighbors.
Vectorized execution over a columnar (Arrow) representation: batch-at-a-time operators, the pull/iterator model, validity-bitmap-correct null handling, zero-copy buffer sharing.

Nice-to-haves

None flagged.

Application prep

Use the role’s language and application details while tailoring your materials.

Keywords to mirror

aisqlrustrest

Platform: gr8people
Effort: medium