Applied Scientist, Bidding Science and Experimentation

Amazon Advertising is one of Amazon's fastest growing and most profitable businesses, responsible for defining and delivering a collection of advertising products that drive discovery and sales. Our products are strategically important to our businesses driving long term growth. We deliver billions of ad impressions and millions of clicks and break fresh ground in product and technical innovations every day! The Bidding Experimentation and Science Team is responsible for both real-time pricing and ranking decisions on the Amazon RTB (Real-Time Bidding) system as well as our experimentation infrastructure and methodology. As the owners of ranking, we have one of the most important job in the DSP which is to set the bid price on advertising opportunities. Even small incremental improvement is making a huge impact for the company, and we are responsible for developing the next generation of pricing algorithms to drive these improvements. As owners of experimentation, we work closely with not only scientists within our team but also our partners teams to develop new online experiment designs, propose new analysis methodologies to increase the statistical validity and power of our results, and build tools to enable experimenters to rapidly iterate on their ideas. Experimentation is ultimately the engine that drives our decision-making when launching new products and algorithms. The visibility of our team's work is very high and we receive high support from the management and peer teams. These systems are optimizing the price of every individual opportunity on behalf of Amazon Advertising advertisers. You will play a significant role in building and improving our production algorithms, leveraging the latest advances in ML, online optimization, optimal control, forecasting, and causal inference, to name a few. You will construct prototypes to validate your hypotheses, work with engineers to turn these prototypes into production systems, and perform offline analysis and online A/B experimentation. You will also join the Amazon science community, with opportunities to write papers, attend workshops and conferences (internal and external), participate in science events, and obtain early access to Amazon ML technologies. Key job responsibilities In this role, you will work directly with a highly motivated team of ML scientists and engineers who are passionate about applying data science and ML techniques to solve advertiser needs. You will also join the Amazon ML community, with opportunities to write papers, attend workshops and conferences (internal and external), participate in science events, and obtain early access to Amazon ML technologies. As an Applied Scientist II, you will - Be a technical leader in ML, optimization, and causal inference and drive full life-cycle projects within this team and across other teams - Build machine learning models, perform proof-of-concept, experiment, optimize, and deploy your models into production. - Develop algorithms for online optimization and control, bandits, and reinforcement learning and bring them to production. - Develop, design, and run A/B experiments, gather data, and perform statistical analysis. - Establish scalable, efficient, automated processes for monitoring and feedback. - Work closely with software engineers to assist in productionizing your work. - Research new and innovative machine learning and optimization approaches. - Recruit Applied Scientists and Data Scientists to the team and mentor scientists on the team. A day in the life - Conduct offline analyses of data to guide design decisions with the product team - Conduct A/B tests and analyze results to guide rollout, go to market, or development priority decisions - Suggest models, algorithms, designs, and/or features to sophisticate the marketing products or solutions we offer to our customers About the team Why you will love this opportunity: Amazon is investing heavily in building a world-class advertising business. This team defines and delivers a collection of advertising products that drive discovery and sales. Our solutions generate billions in revenue and drive long-term growth for Amazon’s Retail and Marketplace businesses. We deliver billions of ad impressions, millions of clicks daily, and break fresh ground to create world-class products. We are a highly motivated, collaborative, and fun-loving team with an entrepreneurial spirit - with a broad mandate to experiment and innovate. Impact and Career Growth: You will invent new experiences and influence customer-facing shopping experiences to help suppliers grow their retail business and the auction dynamics that leverage native advertising; this is your opportunity to work within the fastest-growing businesses across all of Amazon! Define a long-term science vision for our advertising business, driven from our customers' needs, translating that direction into specific plans for research and applied scientists, as well as engineering and product teams. This role combines science leadership, organizational ability, technical strength, product focus, and business understanding. Team video https://youtu.be/zD_6Lzw8raE Minimum qualifications - 3+ years of building models for business application experience - PhD, or Master's degree and 4+ years of CS, CE, ML or related field experience - Experience programming in Java, C++, Python or related language - Experience in any of the following areas: algorithms and data structures, parsing, numerical optimization, data mining, parallel and distributed computing, high-performance computing Preferred qualifications - Experience using Unix/Linux - Experience in professional software development Amazon is an equal opportunity employer and does not discriminate on the basis of protected veteran status, disability, or other legally protected status. Our inclusive culture empowers Amazonians to deliver the best results for our customers. If you have a disability and need a workplace accommodation or adjustment during the application and hiring process, including support for the interview or onboarding process, please visit https://amazon.jobs/content/en/how-we-hire/accommodations for more information. If the country/region you’re applying in isn’t listed, please contact your Recruiting Partner. The base salary range for this position is listed below. Your Amazon package will include sign-on payments and restricted stock units (RSUs). Final compensation will be determined based on factors including experience, qualifications, and location. Amazon also offers comprehensive benefits including health insurance (medical, dental, vision, prescription, Basic Life & AD&D insurance and option for Supplemental life plans, EAP, Mental Health Support, Medical Advice Line, Flexible Spending Accounts, Adoption and Surrogacy Reimbursement coverage), 401(k) matching, paid time off, and parental leave. Learn more about our benefits at https://amazon.jobs/en/benefits . USA, WA, SEATTLE - 142,800.00 - 193,200.00 USD annually