Job description

Position Overview We are looking for someone with a deep and genuine obsession with hardware, regardless of where that expertise comes from. Candidates may have experience in chip architecture, SoC design, memory systems, interconnect PHYs, semiconductor manufacturing, advanced packaging, performance engineering, ML systems, HPC, datacenter infrastructure, networking, power, cooling, or large-scale cluster operations. What matters most is a first-principles understanding of how modern AI hardware works, from silicon and packaging through racks, networks, and full datacenter systems. The ideal candidate is intensely curious about why certain designs outperform others and can translate technical differences into measurable performance and economic outcomes. The role focuses on quantitative analysis of AI hardware and systems, including: • Hardware and system performance • Architecture evaluation • Silicon and system cost analysis • Power and cooling constraints • Total cost of ownership • AI infrastructure economics • Technology assessment across the full hardware stack The work will support SemiAnalysis products and research, including the Inference Simulator, InferenceX, Tokenomics Model, Accelerator & HBM Model, and AI Cloud TCO research. The scope of the role will flex toward the candidate’s strengths, with strong performers helping define their own coverage areas. The central question behind the role is: For a given model, latency target, and system configuration, how many tokens per second does each chip and architecture deliver, and what does each token cost? Responsibilities • Build first-principles performance models for LLM inference and training. • Analyze arithmetic intensity, roofline performance, prefill and decode behavior, KV cache capacity, memory bandwidth requirements, batching dynamics, parallelism strategies, and latency-throughput trade-offs. • Evaluate AI hardware across the full technology stack, including GPUs, TPUs, custom ASICs, emerging accelerators, HBM, on-chip SRAM, memory tiering, scale-up fabrics, scale-out networks, racks, pods, and datacenters. • Assess architecture and system design trade-offs, including compute versus memory bandwidth, interconnect topology, power availability, cooling requirements, and workload suitability. • Connect semiconductor manufacturing decisions to hardware performance, availability, and cost. • Analyze process-node choices, die size, reticle limits, advanced packaging, 2.5D integration, 3D stacking, HBM integration, yield, and silicon economics. • Examine new chip, system, rack, and cluster announcements using specifications, die shots, rack layouts, benchmarks, and network diagrams. • Develop independent and defensible views on real-world performance compared with vendor claims. • Validate internal models against published benchmarks and independently gathered performance data. • Investigate and explain differences between theoretical peak performance and delivered application performance. • Translate technical analysis into economic metrics, including tokens per second per watt, per dollar of capital expenditure, and per megawatt. • Contribute to TCO, tokenomics, accelerator, memory, networking, and datacenter research. • Publish detailed research on AI accelerators, systems, infrastructure, and performance. • Act as a technical authority during client calls, briefings, and discussions with engineering and investment audiences. • Collaborate with accelerator, memory, networking, semiconductor, and datacenter analysts to connect silicon-level findings with broader system and industry conclusions. Requirements • Deep technical understanding of modern compute hardware at one or more levels of the stack, including silicon, systems, networking, clusters, or datacenters. • Strong interest in learning unfamiliar parts of the hardware and infrastructure stack. • Strong quantitative reasoning skills and the ability to work from first principles using FLOPs, bytes, bandwidth, latency, joules, watts, and dollars. • Ability to build structured models that connect hardware design, workload performance, infrastructure requirements, and economic outcomes. • Working knowledge of how AI workloads stress hardware, including memory bandwidth, memory capacity, interconnect communication, collective operations, and real-world compute utilization. • Working understanding of semiconductor manufacturing and economics, including process-node trade-offs, die size, yield, die cost, packaging, and HBM integration. • Ability to explain why an architecture was designed in a particular way and how those decisions affect manufacturing cost, performance, power, and scalability. • Strong written communication skills, with the ability to explain complex technical findings clearly to both engineering and investor audiences. • Ability to take an ambiguous technical question from initial definition through analysis, validation, and publication with minimal oversight. • Self-driven, intellectually rigorous, detail-oriented, and comfortable challenging assumptions. • Genuine enthusiasm for computer hardware, semiconductor technology, AI systems, and infrastructure research. Preferred Skills • Hands-on experience with AI training or inference frameworks such as vLLM, SGLang, TensorRT-LLM, PyTorch, or JAX. • Experience with GPU programming or kernel development using CUDA, Triton, or HIP. • Understanding of transformer inference mathematics, including per-token FLOPs, memory traffic, KV cache sizing, MoE routing, and quantization effects. • Experience designing, deploying, benchmarking, or operating large GPU or accelerator clusters. • Knowledge of datacenter networking, rack-scale architecture, power distribution, cooling, or large AI infrastructure deployments. • Familiarity with training performance metrics and concepts such as model FLOPs utilization, parallelism scaling efficiency, communication overhead, and failure recovery. • Experience benchmarking heterogeneous hardware, including NVIDIA GPUs, AMD GPUs, TPUs, custom accelerators, or emerging non-NVIDIA platforms. • Hands-on semiconductor process, foundry, product engineering, yield, packaging, or test experience. • Experience with die cost modeling, semiconductor manufacturing economics, advanced packaging, or teardown analysis. • Ability to interpret die shots, packaging layouts, physical specifications, and teardown data to infer architecture and process decisions. • Prior technical publications, research papers, open-source contributions, benchmarks, hardware teardowns, or industry analysis. • Experience communicating technical findings directly to clients, executives, engineers, or institutional investors. Growth Areas The role provides opportunities to expand beyond an existing area of specialization and develop expertise across the complete AI hardware stack. Potential growth areas include: • Expanding from chip-level expertise into rack-scale and datacenter system analysis. • Developing deeper knowledge of AI inference and training workloads. • Learning first-principles performance and cost modeling. • Building expertise in accelerator, HBM, networking, power, cooling, and TCO analysis. • Connecting semiconductor manufacturing and packaging decisions to system-level performance and economics. • Developing the ability to evaluate new chips, systems, and infrastructure announcements rapidly and independently. • Strengthening technical writing and publishing skills. • Becoming a recognized technical authority with engineering, investment, and industry audiences. • Leading client briefings and translating complex hardware analysis into strategic and financial conclusions. • Collaborating across accelerator, memory, networking, semiconductor manufacturing, and datacenter research areas. • Defining and owning a research coverage area based on individual technical strengths. • Contributing directly to the development of SemiAnalysis models, simulators, datasets, and research methodologies. SemiAnalysis LLC participates in the E-Verify program to confirm the employment eligibility of all newly hired employees. For more information, visit e-verify.gov . • E-Verify Poster: everify.gov • Right to Work Poster: justice.gov

Requirements

Must-haves

Position Overview
We are looking for someone with a deep and genuine obsession with hardware, regardless of where that expertise comes from.
The role focuses on quantitative analysis of AI hardware and systems, including:
Hardware and system performance
Architecture evaluation
Silicon and system cost analysis
Power and cooling constraints
Total cost of ownership
AI infrastructure economics
Technology assessment across the full hardware stack
The work will support SemiAnalysis products and research, including the Inference Simulator, InferenceX, Tokenomics Model, Accelerator & HBM Model, and AI Cloud TCO research.
The scope of the role will flex toward the candidate’s strengths, with strong performers helping define their own coverage areas.

Nice-to-haves

Hands-on experience with AI training or inference frameworks such as vLLM, SGLang, TensorRT-LLM, PyTorch, or JAX.
Experience with GPU programming or kernel development using CUDA, Triton, or HIP.
Understanding of transformer inference mathematics, including per-token FLOPs, memory traffic, KV cache sizing, MoE routing, and quantization effects.
Experience designing, deploying, benchmarking, or operating large GPU or accelerator clusters.
Knowledge of datacenter networking, rack-scale architecture, power distribution, cooling, or large AI infrastructure deployments.
Familiarity with training performance metrics and concepts such as model FLOPs utilization, parallelism scaling efficiency, communication overhead, and failure recovery.
Experience benchmarking heterogeneous hardware, including NVIDIA GPUs, AMD GPUs, TPUs, custom accelerators, or emerging non-NVIDIA platforms.
Hands-on semiconductor process, foundry, product engineering, yield, packaging, or test experience.
Experience with die cost modeling, semiconductor manufacturing economics, advanced packaging, or teardown analysis.
Ability to interpret die shots, packaging layouts, physical specifications, and teardown data to infer architecture and process decisions.
Prior technical publications, research papers, open-source contributions, benchmarks, hardware teardowns, or industry analysis.
Experience communicating technical findings directly to clients, executives, engineers, or institutional investors.

Application prep

Use the role’s language and application details while tailoring your materials.

Keywords to mirror

mlaillm

Platform: careers-page
Effort: medium