Lead Machine Learning Inference Engineer (Advertising)

• The Advertising Performance group focuses on performance for all participants in the Advertising ecosystem - Advertisers, Publishers, and Roku. The systems and solutions span multiple disciplines and technologies to perform real-time multi-objective optimization across distributed systems at large scale and with low latency

• We use Machine Learning, Reinforcement Learning, AI, Control and Optimization Systems, and Auction Dynamics to solve a large set of complex problems. At the core of this is our Machine Learning and Inference Platform that powers the entire landscape

• In this role, you will architect, design, and lead the development of a SOTA Inference platform that can handle Advertising-level low latencies, scale, throughput, and availability with optimizations that span across hardware, software, and models

• We’re looking for a strong technical leader with deep experience in ML serving, high-performance computing, and industry standard frameworks - someone excited to mentor engineers, innovate at scale, and shape the future of machine learning at Roku

• Lead the design and development of a SOTA Inference platform

• Oversee the development of monitoring, observability, and other tooling to ensure system and model performance, reliability, and scalability of online inference services

• Identify and resolve system inefficiencies, performance bottlenecks, and reliability issues, ensuring optimized end-to-end performance

• Stay at the forefront of advancements in inference frameworks, ML hardware acceleration, and distributed systems, and incorporate innovations where and when they are impactful

Benefits

• Medical, wellness and financial benefits

• Free snacks and access to the company fitness center

• Unlimited paid time off policy

• Work from home opportunities- 10+ years of experience in developing and deploying large-scale, distributed systems, with at least 5 years in a leadership or technical lead role

• Proven experience optimizing performance for large-scale machine learning systems, including a deep knowledge of SOTA model optimizations, hardware-software co-design, GPU acceleration, and HPC techniques

• Experience leading teams working on high-throughput, low-latency ML serving systems

• Strong programming skills in high-performance languages

• Deep understanding of inference frameworks and ML system deployment

• Experience collaborating with and leading global, cross-functional teams

• Contributions to open-source ML or systems projects

• Excellent communication and collaboration skills

• M.S. or above in CS, ECE, or a related field