Skip to content

Don’t believe everything you see on the internet

This tweet was posted a few days ago:

Matt Darling reposted it but then added:

These numbers are also, fwiw, clearly wrong. 22% of Americans are not skipping meals to afford housing. That's massively above the USDA estimate of food insecurity.

This reminds me: the world is brimming with bad data from companies who use quickie surveys and clickbait toplines to get free PR. In this case, Redfin got Axios to bite, but they're hardly uniquely gullible. The Wall Street Journal publishes this junk all the time.

There's no hope of getting this to stop as long as media outlets remain eager consumers. So do yourself a favor and take a quick look at the source when you see charts like this. Reliability goes approximately in this order:

  • National statistical agencies in reliable countries (i.e., not China).
  • Well regarded international statistical agencies (OECD, World Bank, etc.)
  • Academic research (but make sure you understand it).
  • Well known pollsters (Gallup, YouGov).
  • Commercial outfits that do consistent releases (Zillow, for example, or NAR).
  • Commercial outfits that do research periodically on different subjects (McKinsey, for example, or Microsoft).
  • Companies that sporadically release clickbait survey results.
  • Dotcom companies.

The top five are mostly usable without a lot of skepticism. The bottom three should only be used if you dive deeper and decide the particular bit of research being touted seems to have been reasonably produced. One-off polls from people who aren't experienced with surveys very often suffer from poor design, bad question wording, and massive conflict of interest.

17 thoughts on “Don’t believe everything you see on the internet

    1. DianaBryan

      US Dollar 2,000 in a Single Online Day Due to its position, the United States offers a plethora of opportunities for those seeking employment. With so many options accessible, it might be difficult to know where to start. You may choose the ideal online housekeeping strategy with the help vz-04 of this post.

      Begin here>>>>>>>>>>>>>> https://accessibleidea01.blogspot.com/

  1. Joseph Harbin

    More clickbait:

    A typical home now costs $1 million and up in more than 500 U.S. cities

    There is a record number of areas in the U.S. where the “typical” home is worth $1 million or more.

    In February, the country had 550 “million-dollar” cities, or areas where the “typical” home value is $1 million or more, according to a new analysis by Zillow.

    550 U.S. cities! That sounds like a lot. Dig a little deeper and you'll find that 238 of those "million-dollar cities" are part of the three cities we know as New York, San Francisco, and Los Angeles. Amazing!

    (Very few of those million-dollar homes look anything like the one in the picture accompanying the article.)

    1. lower-case

      hard to believe the outlet that spawned maria bartiromo and rick santelli might not be completely on the up and up

      1. Joseph Harbin

        You could blame Zillow, which originally published the claim. But business & other pubs have faithfully picked up the story and passed along the hype, including:

        CNBC
        Fortune
        QZ
        Yahoo Finance
        Marketplace
        The Hill

        Even before the internet, media incentives skewed toward hyperbole. Now that everything is online, the temptations are too great to resist.

    2. Jasper_in_Boston

      Right. There have to be at least 30,000 incorporated municipalities ("cities") in the United States. That two percent of them feature million dollar home prices isn't much to write home about.

  2. QuakerInBasement

    Redfin commissioned Qualtrics to perform the survey. The study included nearly 3000 homeowners and renters.

    I see no reason to doubt the accuracy of the data. The interpretation of the data is debatable.

    Food insecurity isn't the same thing as skipping meals.

    1. bbelcourt

      Exactly. Food insecurity is about lack of access to sufficient, quality food. Skipping some meals in order to afford your mortgage has absolutely nothing to do with food insecurity. These people, presumably, have access to food, they are choosing to skip these meals for financial reasons.

    2. emh1969

      Yeah, going back to the orignal data the 22% who skipped a meal isn't based on all 3000 respondents. It's based on the respondents who reported that they struggled to pay for housing. Which was about half of the 3000 respondents. So that's a food insecurity rate of about 11% which is pretty close to the latest USDA figure (12.8% in 2022).

      Sloppy work Kevin...

      https://www.redfin.com/news/homebuying-sacrifices-survey-2024/

      1. skeptonomist

        What is the home ownership rate among the people who are considered food insecure by the USDA? I suspect very low, so those people are probably not responsible for the response about skipping meals.

        People don't always tell the truth in response to polls and they like to complain. If the questions are posed in a political context biases can be tremendous. Absolute percentages in polls may often be suspect for various reasons - trends are more likely to have meaning. It may be very difficult to pose questions that will give truthful and accurate answers. Just trusting the result of a random poll is foolish.

  3. rick_jones

    “redfin site:jabberwocking.com”
    And/or
    “redfin drum site:mitherjones.com”

    Into your search engine of choice.

  4. D_Ohrk_E1

    These numbers are also, fwiw, clearly wrong. 22% of Americans are not skipping meals to afford housing. That's massively above the USDA estimate of food insecurity.

    That may be true, but it's also the case that roughly 25% of all Americans are Medicaid recipients (85,094,448 (Dec 2023 via Medicaid) divided by 335,893,238 (Dec 2023 via Census) = 25.3%).

  5. kenalovell

    I always treasure this gem from Pew:

    ... in a February 2022 survey experiment, we asked opt-in respondents if they were licensed to operate a class SSGN (nuclear) submarine. In the opt-in survey, 12% of adults under 30 claimed this qualification, significantly higher than the share among older respondents. In reality, the share of Americans with this type of submarine license rounds to 0%.

    Pew uses this anecdote to illustrate the unreliability of opt-in polling. But they studiously avoid considering an explanation with more serious implications for all public opinion surveys: that the proportion of the population willing to respond to cold calls from unknown numbers has become so small that any polling sample is inherently unrepresentative of the population. Like the respondents who claimed they were licensed to operate a nuclear submarine, a non-negligible number may well welcome participation in a poll as an opportunity to play silly games with the pollster.

Comments are closed.