Senior Product Manager –AI Inference Software
AMD· enterprise· Semiconductors· Jul 1, 2026 · 4mo agoPosted 118 days ago
Santa Clara, California
In office
Full-time
Senior
$129K–$184K est.
Before you apply
Ghost Signal:
Open since July 1, 2026 — 118 days, and still accepting applications.
Job description
WHAT YOU DO AT AMD CHANGES EVERYTHING
At AMD, our mission is to build great products that accelerate next-generation computing experiences—from AI and data centers, to PCs, gaming and embedded systems. Grounded in a culture of innovation and collaboration, we believe real progress comes from bold ideas, human ingenuity and a shared passion to create something extraordinary. When you join AMD, you’ll discover the real differentiator is our culture. We push the limits of innovation to solve the world’s most important challenges—striving for execution excellence, while being direct, humble, collaborative, and inclusive of diverse perspectives. Join us as we shape the future of AI and beyond. Together, we advance your career.
THE ROLE:
This role owns the framework-layer inference product strategy for ROCm, translating customer, ecosystem, and engineering signals into roadmap decisions for production AI inference on AMD Instinct hardware. As inference becomes the defining workload for production AI, you will help shape how AMD’s software ecosystem enables efficient, reliable, and competitive large-scale model deployment. You will work with engineering, strategic AI customers, ecosystem partners, and the open-source inference community to advance inference at scale.
THE PERSON:
You are a technically deep product leader with strong expertise in AI inference infrastructure and open-source software. You can reason across inference engines, serving, orchestration, memory management, and performance while understanding how these systems interact with the GPU software and hardware beneath them. You navigate complex organizations, build alignment without formal authority, and drive important work to completion. You are comfortable operating at the intersection of open-source communities and enterprise-scale customers, whether digging into a GitHub issue thread or presenting roadmap tradeoffs to a VP of Engineering at a hyperscaler.
KEY RESPONSBILITIES:
Product Strategy & Roadmap
• Own the product strategy and roadmap for ROCm’s inference frameworks software capabilities.
• Define how AMD’s framework-layer software stack for inference enables production workloads from single-GPU through rack-scale deployments, with a focus on performance, developer experience, portability, and competitive differentiation.
• Identify emerging model architectures, serving technologies, and infrastructure shifts that require changes in the inference frameworks layer and translate them into product priorities.
• Balance customer needs, ecosystem direction, technical opportunities, and engineering investment to determine where AMD should differentiate in the software stack.
Inference Software & Engineering Partnership
• Partner with engineering teams across inference engines, serving and orchestration, memory systems, GPU libraries, runtimes, kernels, and communications, with particular emphasis on the framework and orchestration layers that enable large-scale inference above the underlying GPU software stack.
• Translate customer and ecosystem needs into prioritized engineering requirements and drive execution across organizational boundaries.
• Work with engineering to identify inference performance bottlenecks and determine where framework-level software investment can have the greatest impact.
• Provide software-informed input to future GPU architecture and platform decisions relevant to inference performance at scale.
• Drive release readiness across a fast-moving open-source software ecosystem, with attention to performance, compatibility, regression risk, and adoption.
Open-Source Community Engagement
• Serve as an active AMD presence in the open-source inference community.
• Build relationships with maintainers, developers, researchers, and contributors whose work influences AI infrastructure and AMD adoption.
• Track emerging technologies and ecosystem direction through open-source projects, research, technical communities, and developer feedback.
• Communicate AMD’s roadmap and technical direction through community engagement, technical writing, and industry events.
• Represent AMD constructively in technical discussions where ecosystem and company priorities may differ.
Customer & Partner Engagement
• Work directly with sophisticated AI infrastructure customers to understand deployment requirements, performance constraints, and future needs at the framework and serving layer.
• Distinguish individual customer requests from broader technical and market signals, then translate those signals into scalable product decisions.
• Balance the needs of strategic customers with those of the broader open-source ecosystem.
• Collaborate with cloud providers, model developers, infrastructure companies, and other ecosystem partners on joint integrations and solutions.
PREFERRED EXPERIENCE:
• Deep understanding of production AI inference systems, including inference engines such as vLLM, SGLang, and AMD ATOM , and serving and orchestration systems such as llm-d , along with memory management and distributed inference.
• Experience driving complex technical initiatives across multiple engineering teams or organizations.
• Experience working with sophisticated AI infrastructure customers, open-source communities, or ecosystem partners.
• Experience in AI inference infrastructure, GPU computing, distributed systems, cloud infrastructure, high-performance computing, developer platforms, research, or technical product leadership.
• Open-source contributions, technical writing, conference talks, or sustained community engagement are strong positive signals.
• Ability to communicate technical tradeoffs clearly to engineering leaders, customer stakeholders, open-source contributors, and executive audiences.
• Product management experience in deeply technical products is valuable but not required. Candidates from engineering, research, or other technical leadership backgrounds are encouraged if they demonstrate strong product instincts and interest in product leadership.
• Familiarity with AMD Instinct™ hardware and the ROCm software ecosystem is a plus.
ACADEMIC CREDENTIALS:
• Bachelor’s degree in Computer Science, Electrical Engineering, or a related technical field. Advanced degree a plus but not required given equivalent experience.
LOCATION:
• Bay Area · Austin · Remote
This role is not eligible for visa sponsorship.
#LI-EV1
Benefits offered are described: AMD benefits at a glance .
AMD does not accept unsolicited resumes from headhunters, recruitment agencies, or fee-based recruitment services. AMD and its subsidiaries are equal opportunity, inclusive employers and will consider all applicants without regard to age, ancestry, color, marital status, medical condition, mental or physical disability, national origin, race, religion, political and/or third-party affiliation, sex, pregnancy, sexual orientation, gender identity, military or veteran status, or any other characteristic protected by law. We encourage applications from all qualified candidates and will accommodate applicants’ needs under the respective laws throughout all stages of the recruitment and selection process.
AMD may use Artificial Intelligence to help screen, assess or select applicants for this position. AMD’s “Responsible AI Policy” is available here.
This posting is for an existing vacancy.
Application prep
Use the role’s language and application details while tailoring your materials.
Keywords to mirror
aillm
- Platform
- amd
- Effort
- medium