Optimise the performance of your systems with practical experiments used by engineers in the world's most competitive industries.

Experimentation for Engineers: From A/B testing to Bayesian optimization is a toolbox of techniques for evaluating new features and fine-tuning parameters. You will start with a deep dive into methods like A/B testing and then graduate to advanced techniques used to measure performance in industries such as finance and social media.

You will learn how to:

• Design, run, and analyse an A/B test
• Break the "feedback loops" caused by periodic retraining of ML models
• Increase experimentation rate with multi-armed bandits
• Tune multiple parameters experimentally with Bayesian optimisation
• Clearly define business metrics used for decision-making
• Identify and avoid the common pitfalls of experimentation

By the time you're done, you will be able to seamlessly deploy experiments in production, whilst avoiding common pitfalls.

About the technology

Does my software really work? Did my changes make things better or worse? Should I trade features for performance? Experimentation is the only way to answer questions like these. This unique book reveals sophisticated experimentation practices developed and proven in the world's most competitive industries and will help you enhance machine learning systems, software applications, and quantitative trading solutions.

About the Author

David Sweet has worked as a quantitative trader at GETCO and a machine learning engineer at Instagram. He teaches in the AI and Data Science Master's programmes at Yeshiva University.

Reviews

"Putting an 'improved' version of a system into production can be really risky. This book focuses you on what is important!"
Simone Sguazza, University of Applied Sciences and Arts of Southern Switzerland

"A must-have for anyone setting up experiments, from A/B tests to contextual bandits and Bayesian optimization."
Maxim Volgin, KLM

"Shows a non-mathematical programmer exactly what they need to write powerful mathematically-based testing algorithms."
Patrick Goetz, The University of Texas at Austin

"Gives you the tools you need to get the most out of your experiments."
Marc-Anthony Taylor, Raiffeisen Bank International

Book Information
ISBN 9781617298158
Author David Sweet
Format Hardback
Page Count 248
Imprint Manning Publications
Publisher Manning Publications
Weight(grams) 456g
Dimensions(mm) 234mm * 190mm * 15mm