The Denver Basic Income Project is a program that gave no-strings money to homeless people in Denver. It ran through most of 2023 and included three randomly chosen groups. Group A got $1,000 per month for 12 months. Group B we don't care about. Group C got $50 per month. Here's how they did on housing after ten months:
This is the damnedest thing I've ever seen. One way of describing it is to say that the treatment group ($1,000 per month) did barely any better than the control group ($50 per month). So it turns out that giving people money has no effect.
And that's true. But the more important finding, I think, is that giving people $50 per month had a huge impact. How is that possible? This is nowhere near enough money to help someone rent an apartment, and yet rentals went up from 12% to 43%.
In another part of the study, the researchers try to calculate how much the city saved in social services due to the homeless being better off. These exercises are frequently fantasies, but let's take a look anyway:
Again, there are two ways of looking at this. One is that the total savings (about $500,000) are nowhere near what the project cost (about $5 million). The other is that giving people $50 per month had a huge impact. But how?
Another oddity is that about a third of the participants dropped out before the project ended. I get that these are homeless people, not all of whom routinely make great life decisions. Still, in Group A it's a no-strings monthly handout of $1,000! But a third of the participants dropped out partway through.
There are other bizarre findings. For example, although the money seems to have had a big impact on housing, it had almost no impact on food security. And mental health deteriorated a bit for all groups.
I don't know what to think. My main interest is that giving people $50 per month seems to have made a substantial difference on some metrics. That can't possibly be because of the money, which is too small to have an impact. Was it simply the effect of being in the project at all? That homeless people who feel cared about are more likely to get off the street?
This really demands some answers. If it holds up, it would be a cheap way of making a huge difference in homelessness, and it would pay for itself. Needless to say, I'm skeptical. But it needs to be followed up.
There is no real control, according to the data Kevin gives, assuming that all in the study are out of housing at the beginning. You need to know the normal turnover rate - how many people are in and out of housing regardless of whether they are given any money.
To put it another way, what is the probability than any homeless person (not given money) will be in housing at the end of a year?
Without having read any of the sources I assume the issue is that you have to incentivize people to participate in your study *somehow*.
True! It turns out that everyone in the three treatment groups was also given a phone. While not a true control, a fourth group given a phone (that seems to be how they asked the follow-up questions) would have provided a reasonable control group.
They could have compared to those in the Denver continuum of care. The data is there, somewhere.
They are tracked as part of homeless services. If you want to know more, just ask - it's my job to do this stuff. But every person who is homeless is counted so long as they can be found.
And so the potential flaw then becomes selection bias, which is probably why they didn't include the overall numbers to compare to because it would/could have been misleading. Unless selection into these things is truly random (which gets into ethical, practical, and regulatory issues), there can't be a true control.
Yes. Homelessness for many people is sporadic. If you select homeless people for any kind of study, a lot of them will have found their way out of homelessness within a year. (Just as if you track a group of depressed people, most of them will be un-depressed within a year, distorting the therapeutic effect of drugs or talk therapy).
And then we have the problematic group of long-term homeless people, whose problems can't be solved with money because they fundamentally are not problems with money.
My first thoughts exactly. Without knowing some base rate, these results are meaningless. It could be that with a base rate of 6% improvement in housing ... there is no effect. Its also easy, although cynical, to imagine that giving them money decreases the housing improvement rate from baseline.
Why o why can't activists come up with an actual research design?
Why o why? Making a sweeping generalization, because if they are degreed they are liberal arts majors.
So in the "Stupid or Evil?" competition you'll vote Stupid? Ignorant might be more accurate.
I veer towards Evil ... activism isn't science, they aren't interested in the entirety of the problem and just want to show their pet cause works. And control groups are fascist or something.
C’mon, man, you’re not usually given to cheap shots. As a social scientist with a fair amount of human-subjects research experience, I’d say this is a challenging area. Drawing a random sample: a subset of the homeless population has little or contact with government or charitable services. Survivor bias: not just literal; also some will go elsewhere, some will not stay in contact. And if just ‘being shown interest’ has an effect, how can you measure against a control group without contact, and an implied show of interest in the controls?
Fair - I'm unusually grumpy/cynical today. For the record, I meant no aspersions upon Rick.
In my defense, The Denver project was just SO CLOSE to actually doing this well ... three groups, random assignment, good checks on their sample, phased interventions of the three groups across 4 different cohorts to minimize chronology artifacts. And yes, its a very difficult population to work with (as in, so far from ideal lab conditions). I''ll agree that you can't get a perfect control here, and all of the odd little effects that might come with it ... but as I've commented just giving a set of phones to people would constitute a fair control, no?
Maybe I'm being far too jaded here and I'm not allowing for the organizational constraints? There are only so many funds and they ARE trying (gotta give them that).
Yeah, this.
Also, why don't we care about group B? They had the biggest change in housing! That's great.
Although TBH those of us in the sector have always known that if you want people to not be homeless, you can just literally throw money at the problem (and we're talking nominal sums, at the federal level) and it's solved. Studies like this are just about convincing people who don't seem to believe that everyone deserves a place to live.
The control, of just regular homeless people with no phone and no money is missing, as many have noted.
Without that, who knows. The sad thing Kevin apparently misses is that some homeless people suffer from such a disability that it doesn't matter how much money you give them. Which, sadly, explains no difference between $50 and $1000, which of course, to a person with some baseline level of social competence, is an incredible difference.
This is really curious. It would be great to see more follow up. I scanned the reports and it seems like this involved about 1,000 people.
Although they were homeless at least some of them had jobs so this money made their lives more secure.
From memory Oakland, CA had a similar program. Not sure who ran it. As I recall they gave $500 a month to people and the results were quite encouraging.
There is a lot to be said for human contact. I try to stop and talk to the homeless that I encounter. Some I’ve got to know well. I give them $20 bills to get something to eat or to get their meds.
It seems San Mateo county had people who would talk to the homeless and help them get connected to benefits and to get housed.
Yes, I think pretty much every city that has a significant homeless population has some agency or charitable organization working with them.
This made my mind flash back to one of my business courses in college. One section went over a study that was run in the early 1920s (IIRC). The researchers were trying to find out how to streamline work and improve productivity and solicited suggestions from the employees.
Every change they made wound up boosting productivity. So, they started undoing some of the changes. And productivity ALSO increased.
The conclusion they came to was that the employees enjoyed the attention and interaction with the researchers and managers and being able to give their opinions. I wonder if there was a similar affect from getting even small amounts of money at intervals.
eta - minor cleanup
This is where my thoughts went initially too. That the participants had to come in monthly and answer some questions, see a social worker, etc to get the money. And then without a control group the results are really dubious (is it really the money? Is it the attention? Is it the simple effect of having to keep a monthly appointment?) But no .. the money was disbursed electronically to a bank account of pre-paid card. Their documents (I skimmed...) don't seem to indicate that participants had to have any level of contact to remain in the program.
But they did give them all a phone, which I can imagine having benefits. I wish they'd had a "phone only" control group.
But would they participate if they only got a phone?
The $50/month does serve as a control, but it also self selects for people willing to participate for $50/month which could indicate that group is actively taking positive steps to help their situation.
You’re recalling the Hawthorne studies; Wikipedia has an article.
$50 had a Hawthorne effect, perhaps?
Because?
Kevin, why are you dismissing Group B? This group got $12,000 over the year, but about half in a lump sum and then $500 a month thereafter.
If Group A didn't show improvements, I might argue that $1k isn't enough for a first, last, security on an appt. But $6500 in month one? That's a huge windfall and yet it seems to have no effect (vs Group A) at 6 months.
Collectively this suggests that $ in pocket does a poor job of explaining the variance in outcomes.
I'm harsh on the design of this experiment ... but I also suspect they are gravely undercounting the financial benefits. Anyone look at the per-item cost savings? They seem woefully low.
$69 for an ambulance trip?
$199 for a hospital night?
$325 for an ER visit?
Just a guess, but those numbers are likely the Medicaid rates.
Thanks ... and wow, no wonder providers don't want to accept medicare if they don't have to
Medicare != Medicaid
Well, the last part is just wrong. It turns out, if you give as little as $50/ month, people can move off the streets.
But in general, you're misreading the outcomes because you didn't dig down into the report -- Figure 10: Percentage of Unsheltered Participants in a House or Apartment
They Rent or Own at Timepoint 1 and Timepoint 3
Group A
T1 = 0%
T3 = 43%
Group C
T1 = 0%
T3 = 28%
The level of income support delineates the type of housing that is sought.
good catch!
What the average time people are "homeless?"
There are long-term homeless issues and there are issues with housing insecurity, i.e. those couch surfing. Seasonal effects come into play too--not just cold weather, but getting somewhere to get kids in school.
"...I think, is that giving people $50 per month had a huge impact. ..."
I think this was a bizarre post. First as others have pointed out there was no control group so you don't know the impact. For many conditions likely including homelessness many people will get better on their own. Second there is something called the placebo effect in which some people get better because the treatment improves their mental attitude.