• •Microsoft engineer Raymond Chen highlights how selectively choosing data to match a desired conclusion leads to skewed results
• •Technical examples confirm that removing 'outlier' data points without statistical justification fundamentally invalidates software performance benchmarks
• •While the concept is well-understood in data science, it remains unclear how often developers inadvertently introduce these biases in modern AI training sets

Microsoft developer Raymond Chen explains the technical pitfalls of cherry-picking data to force software benchmarks to show improvement. This practice obscures actual system performance by ignoring inconvenient variables that do not align with the intended narrative. While common in early-stage development, the friction arises when these biased results are used to justify scaling decisions or resource allocation. If developers cannot rigorously justify the exclusion of data points, they risk building systems on faulty foundations that fail in real-world environments.