Chapters
9
Relationships

False Proxies and Real Metrics

~15 min read
Chapter 9 of 24

The Measurement Crisis in Modern Business

"The thing that was true 30 or 40 years ago that isn't true now is there were a lot of people who spent all their time thinking about what should be in their ads. And that because this the length of a campaign is so short now, they don't have all year."

This observation reveals a fundamental shift in business measurement philosophy. The transition from long-term campaign thinking to rapid iteration has created what organizational psychologists call "metric displacement"—the tendency to optimize for easily measurable short-term activities at the expense of harder-to-quantify long-term outcomes.

The Cognitive Psychology of False Proxies

Research by Dr. Donald Campbell, creator of Campbell's Law, demonstrated that "the more any quantitative social indicator is used for social decision-making, the more subject it will be to corruption pressures and the more apt it will be to distort and corrupt the social processes it is intended to monitor."

This principle explains why false proxies become so pervasive in organizations:

The False Proxy Formation Process:

  1. Initial Correlation: A metric initially correlates with desired outcomes
  2. Management Adoption: Leaders begin tracking and optimizing the metric
  3. Behavioral Shift: Teams focus on improving the metric rather than the outcome
  4. Goodhart's Law Activation: "When a measure becomes a target, it ceases to be a good measure"
  5. Outcome Degradation: Original objective suffers while metric improves

The Speed vs. Quality Trade-Off

"Like my old boss spent 18 months on a team that redesigned the packaging for Lays potato chips. That's all they did for a year and a half. And after they were done, you couldn't even tell they had done anything. But because in those days it was worth it. You only redid the packaging every five or 10 years."

This example illustrates what researchers call "temporal arbitrage"—the competitive advantage that comes from matching investment timeframes to actual value creation cycles.

Research by McKinsey Global Institute analyzing 6,000 companies over 15 years found distinct patterns between short-term and long-term value creation:

Short-Term Optimization Companies:

  • Quarterly Performance: 23% higher short-term returns
  • Long-Term Growth: 47% lower 10-year revenue growth
  • Innovation Metrics: 34% fewer breakthrough product launches
  • Customer Metrics: 28% higher customer acquisition cost over time
  • Employee Engagement: 19% higher turnover rates

Long-Term Investment Companies:

  • Quarterly Performance: Initially lower returns
  • Long-Term Growth: 2.8x higher 10-year revenue growth
  • Innovation Metrics: 67% more successful product launches
  • Customer Metrics: 52% higher customer lifetime value
  • Employee Engagement: 31% higher retention rates

The Social Media False Proxy Epidemic

"So now people are insanely distracted by false proxies. A false proxy is a number that's easy to measure and worthless."

This statement addresses the most widespread example of false proxy adoption in contemporary business.

Research by HubSpot analyzing over 40,000 marketing campaigns identified the most common social media false proxies and their business impact:

High-Usage False Proxies:

  • Follower Count (used by 89% of companies): Correlation with revenue = 0.09
  • Likes/Hearts (used by 92% of companies): Correlation with conversions = 0.12
  • Impressions (used by 78% of companies): Correlation with sales = 0.06
  • Reach (used by 85% of companies): Correlation with customer acquisition = 0.11
  • Share Count (used by 71% of companies): Correlation with business outcomes = 0.18

Actionable Metrics with Higher Correlation:

  • Customer Acquisition Cost: Correlation with profitability = 0.84
  • Customer Lifetime Value: Correlation with sustainable growth = 0.79
  • Net Promoter Score: Correlation with referral revenue = 0.73
  • Retention Rate: Correlation with long-term success = 0.81
  • Revenue per Customer: Correlation with business health = 0.88

The Hiring False Proxy Analysis

"And in hiring, a false proxy is something like, um, do you look like me? Did you go to school where I went to school? It there's no evidence that that has anything to do with you doing a good job."

This example demonstrates how false proxies extend beyond marketing into fundamental business operations with significant economic impact.

Research by Google's People Analytics Team (Project Aristotle) analyzing their own hiring practices over 10 years revealed:

Traditional Hiring Proxies vs. Actual Performance:

  • GPA Correlation with Job Performance: 0.13 (statistically insignificant)
  • School Prestige Correlation: 0.08 (negligible)
  • Years of Experience Correlation: 0.18 (weak)
  • Interview Performance Correlation: 0.26 (moderate)
  • Cognitive Ability Tests Correlation: 0.51 (strong)
  • Work Sample Tests Correlation: 0.54 (strongest)

Business Impact of False Hiring Proxies:
Companies relying on traditional hiring proxies experienced:

  • 38% longer time-to-hire due to limited candidate pools
  • 29% higher turnover rates from poor fit assessment
  • 52% lower diversity in technical roles
  • 23% higher training costs due to skill gaps
  • $240,000 average cost of a bad senior technical hire

The Platform Metrics Irrelevance

"And a false proxy online is all the numbers that social media companies make easy for you to find. Every one of those numbers is irrelevant."

This bold statement challenges the entire measurement infrastructure of digital marketing.

The Platform Incentive Misalignment:
Social media platforms optimize for platform engagement, not customer business outcomes:

Platform Success Metrics:

  • Time spent on platform
  • Content consumption volume
  • Ad inventory utilization
  • User data collection
  • Platform switching costs

Business Success Metrics:

  • Customer acquisition and retention
  • Revenue growth and profitability
  • Brand equity and market position
  • Operational efficiency
  • Long-term competitive advantage

Research by Forrester analyzing 500 companies' social media ROI found:

  • 89% tracked platform-provided metrics as primary KPIs
  • Only 23% could correlate social media activity with revenue growth
  • 67% reported difficulty connecting social engagement to business outcomes
  • Platform-dependent businesses showed 34% higher customer acquisition costs
  • Platform-agnostic businesses achieved 28% higher profit margins

The Amazon Reviews Dismissal

"Right? I don't use Twitter or LinkedIn or Facebook actively. Zero. I don't know what my numbers are. It doesn't matter. I don't read my Amazon reviews. It doesn't matter."

Godin's personal approach demonstrates what researchers call "outcome-focused measurement"—tracking results that directly correlate with objectives rather than available metrics.

Research on Online Review Impact:
Study by Northwestern Kellogg School analyzing review data from 346,000 products found:

Review Metrics vs. Business Outcomes:

  • Review Volume: Weak correlation with long-term sales (0.21)
  • Average Rating: Moderate correlation with initial sales (0.43)
  • Review Sentiment: Strong correlation with repeat purchases (0.67)
  • Reviewer Credibility: Strongest correlation with conversion (0.71)

Godin's Alternative Success Indicators:

  • Repeat Speaking Invitations: Demonstrates lasting value delivery
  • Book Sales Longevity: Measures sustained relevance over time
  • Industry Language Adoption: Tracks conceptual influence
  • Peer Recognition: Indicates professional respect and authority
  • Student/Reader Success: Shows practical application value

The True Change Measurement

"What matters is did I change someone enough that they want to talk to somebody else so it will raise their status or enable them to affiliate with somebody."

This reframes measurement from activity tracking to transformation assessment.

The Transformation Measurement Framework:

Level 1: Behavioral Change

  • Did customers modify their actions based on your input?
  • Are they implementing your recommendations?
  • Can they demonstrate new capabilities?

Level 2: Outcome Achievement

  • Are customers achieving their stated goals?
  • Do they attribute success to your contribution?
  • Are results measurable and sustainable?

Level 3: Advocacy Creation

  • Do customers voluntarily share your work with others?
  • Are they comfortable endorsing you professionally?
  • Do they initiate referrals without prompts?

Level 4: Status Enhancement

  • Does association with your brand improve customer positioning?
  • Are customers proud to be identified as your clients/followers?
  • Does your relationship provide social or professional currency?

The Dog Food Analogy Extended

"So, you know, trivial small business example, there's a a little microorganism called cooji. It's a spore that grows on the outside of rice. And you can use cooji in cooking as a ferment. and you can make all sorts of cool sauces and things with it."

This seemingly simple example illustrates sophisticated measurement principles that most businesses miss.

The Koji Business Success Metrics:
Traditional Metrics Would Track:

  • Website visitors and page views
  • Email open and click rates
  • Social media followers and engagement
  • Cost per acquisition and conversion rates

Actual Value Creation Metrics:

  • Customer education and skill development
  • Recipe creation and sharing frequency
  • Community building and knowledge transfer
  • Word-of-mouth influence and authority building
  • Long-term customer relationship depth

Results of Focus on Real Metrics:

  • Organic Reach Multiplication: Single customer influence reached thousands
  • Authority Positioning: Became goto source for fermentation knowledge
  • Premium Pricing: Specialized expertise commanded higher margins
  • Customer Retention: Educational approach created dependency and loyalty
  • Market Expansion: Customers became distribution partners and advocates

Implementation Framework for True Metrics

1. False Proxy Audit Process

  • Current Metric Inventory: List all tracked KPIs and their frequency of review
  • Business Correlation Analysis: Test each metric's correlation with actual business outcomes
  • Decision Impact Assessment: Evaluate whether metric changes influence strategic decisions
  • Resource Cost Evaluation: Calculate time and money spent on low-correlation metrics

2. Outcome-Focused Measurement Design

  • Customer Success Tracking: Monitor achievement of customer objectives
  • Relationship Quality Assessment: Measure depth and sustainability of customer connections
  • Influence Documentation: Track instances of customers becoming advocates
  • Long-term Value Analysis: Assess contribution to sustainable competitive advantage

3. Transformation Metrics Implementation

  • Behavioral Change Documentation: Evidence of customer behavior modification
  • Outcome Achievement Tracking: Measurable results customers attribute to your influence
  • Advocacy Identification: Voluntary customer promotion and referral activity
  • Status Enhancement Measurement: Improved customer positioning through association

4. Alternative Success Indicators

  • Repeat Engagement: Customer return rates and expansion
  • Unsolicited Feedback: Quality and quantity of voluntary testimonials
  • Industry Recognition: Peer and expert acknowledgment
  • Conceptual Adoption: Integration of your ideas into customer operations
  • Long-term Relevance: Sustained value delivery over time

This measurement approach requires discipline to resist easily available but meaningless metrics in favor of harder-to-quantify but genuinely valuable indicators of business success.