Applied Scientist, Customer Experience and Business Trends, Customer Experience and Business Trends

• Key job responsibilities
• Develop novel scientific methodologies for evaluating autonomous, tool-using agents, advancing the state of the art in agent behavioral measurement and developer-experience research.
• Design rigorous, reproducible experiments that yield causal insight into agent decision-making, controlling for confounds and ensuring findings are grounded in observed behavior rather than self-report.
• Apply deep knowledge of cloud architecture and services to reason about the architectural decisions agents make — the service trade-offs, integration constraints, and deployment considerations that shape what gets built.
• Develop verification and synthesis methods that ensure only evidence-grounded conclusions survive, advancing techniques for reliability and faithfulness in agent evaluation.
• Design evaluation methodologies and metrics that quantify the quality of agent-produced outcomes across dimensions such as correctness, robustness, and security posture.
• Publish and present your work at internal and external scientific venues in the fields of ML.
• Collaborate with scientists, engineers, and product managers to design and implement science-focused systems.
• Proficiency in model development, validation and implementation for NLP applications.
• Excellent verbal and written communication and presentation skills, ability to convey rigorous mathematical concepts and considerations to non-experts.
• A day in the life
• About the team
• 3+ years of building models for business application experience
• PhD, or Master's degree and 4+ years of CS, CE, ML or related field experience
• Experience in patents or publications at top-tier peer-reviewed conferences or journals
• Experience programming in Java, C++, Python or related language
• Experience in any of the following areas: algorithms and data structures, parsing, numerical optimization, data mining, parallel and distributed computing, high-performance computing
• Hands-on experience (academic or industrial) building ML models for natural language processing tasks using GenAI
• Deep working knowledge of cloud architecture and services (e.g., compute, storage, databases, auth, networking) and the trade-offs involved in composing them into production systems
• Experience with agentic/tool-using systems, LLM-based agents, or the evaluation of autonomous agents
• Experience using cloud-based services and big data processing technologies such as AWS Redshift, EMR, Spark, and Sagemaker
• Prior work experience as a research scientist or applied scientist at a consumer product company
• Excellent problem-solving ability. Demonstrable track record of dealing well with ambiguity, prioritizing needs, and delivering results in a dynamic environment
• Experience in professional software development

None flagged.