Workshop: Critical Thinking About Numbers and Metrics

  • Bugs that could have been found in requirements gathering cost 1000 times more when found in production.
  • Testing costs 40% of the total development effort.
  • When the fix rate trendline crosses the failure rate trendline, it’s time to release the product.
  • The ROI of testing can be calculated by a straightforward formula.
  • Risk = Probability x Impact
  • Defect Removal Efficiency is a good way to evaluate the quality of testing.

If you’re a tester or test manager, you’ve probably seen someone pull numbers out of a hat. At the same time, critical thinking often vanishes into thin air. Numbers can have magical powers. In the right hands, they can help us to see astonishing things that might otherwise be invisible. Yet numbers may also have enormous power to dazzle, distract, and deceive. How do you know that what you seem to be seeing is real, and not just an illusion?

Wise people use critical thinking—thinking about thinking—to avoid being fooled. This half-day workshop, presented by Michael Bolton, is designed to help you learn strategies and skills—questioning, modeling, systems thinking—to help you think critically, confidently and thoughtfully about numbers.

In the workshop, we’ll learn about reliability and validity in measurement and metrics. To get started, we’ll examine some claims from tool vendors and measurement magicians. We’ll analyze some common testing metrics, and discover why they’re really only smoke and mirrors. We’ll look at how to avoid danger, and how to recognize when it might be appropriate to trust numbers. Finally, we’ll develop ways to come up with reasonable, helpful, answers when people ask for metrics, KPIs, or estimates.

Strategy Workshop
Location: Cirrus A Date: September 27, 2017 Time: 1:00 pm - 4:00 pm Michael Bolton