We have a new AI champion: 深度求索.
Yes, it's a Chinese AI with the English name DeepSeek. And it's sort of the new champion, but it gets complicated. Overall, their R1 model, released yesterday, appears to be about as good as the latest GPT-4 model, or maybe a hair worse. But it's got a lot of other things going for it:
- It's open source. Completely open source, model weights and all.
- It's trained via reinforcement learning.
- It's far cheaper to run than GPT-4.
- Smaller "distilled" versions can run on a laptop and perform pretty well.
I've used DeepSeek a little bit and it seems to work pretty well—though I haven't pushed it very hard. Here are the benchmarks they've run:
Is this really the next great advance, or it typical AI hype? I don't know, but Sam Altman seems to be afraid of it, and that must mean something. On the more noxious side, you have to be a little careful about your subject matter:
I suppose there's always Wikipedia for stuff the CCP is sensitive about. In any case, stay tuned. There are a ton of good AI models available these days, and in the end DeepSeek might just be part of the pack. But maybe not.
I'm going to keep saying "I don't know why you continue to credulously take seriously proven liars and grifters" as long as is necessary.
Sam Altman is not a serious person. He's a con man, a flim-flam man. When it comes to AI he is the precise equivalent of Elon Musk's stage shows with Tesla features that will never, ever happen. Do not take him seriously.
Don’t forget about the hyperloop. Musk was pushing that when Obama was still in office, and “autonomous car sized pods hurtling through a tight tunnel for dozens to hundreds of miles at airplane speeds” has yet to come to fruition anywhere. But it certainly was useful in derailing plans to fund higher quality intercity rail in many states.
Does that put you into their sights?
I've been pretty well conditioned to point and laugh every time I hear some wild claims about AI. Trump's press conference yesterday was a good example. Anyone who, at this point, invests hundreds of billions in AI is an idiot.
You should try to ask about Trump's views of Hitler.
"Answer awaiting Tariff announcements"
Interesting they are fully open source and still focused mostly on research, though obviously a key additional ingredient here is actual data to train on.
"fully open source", but you don't have an explicit list of it's forbidden topics to hand?
There's an inconsistency there.
Written by team 2.0?
I'm playing with a couple variants of it locally.
The censorship is really ham-handed and strange, it seems poorly targeted. I'm trying to figure out how much data about censored things simply was omitted from training data - the refuse action seems to fire for completely unrelated things sometimes, which makes me think they tried to "clean" the corpus first, and did it really poorly.
I'm sure they'll get better at computational propaganda, just like OpenAI is.
Speaking of, Altman should be scared. OAI has no moat, and his promotion of "scary" AI to achieve regulatory capture just faceplanted. (Of course he could probably just buy such a law from Republicans now, if he chooses to show proper respect to Musk's anus.)
But this is interesting - it seems to confirm brute-force training is just "easiest". There seems to be a lot of headroom for better methods, which means the universe of people who can play isn't limited to those with billions in the bank.
Hell - a very high quality limited version of this thing runs on my home desktop. Which, granted, is a custom machine that was pretty expensive, but the operative parts here are a pair of Nvidia 3090s, which are five years and about 1.5 hardware generations old (the 50x0s are due out soonish).
According to The NY Times… “A 28-year-old woman with a busy social life spends hours on end talking to her A.I. boyfriend for advice and consolation. And yes, they do have sex.”
Beat that!
So if it is indeed fully open source, does that mean we can see where/how the censorship of Tiananmen Square is done?
All these models are trained using reinforcement learning. Without such training, the raw models are contentious, rude, and horny. Because of the internet data that they are trained on.
Remember when the Bing chatbot tried to break up the marriage of the NYTimes reporter?
https://x.com/kevinroose/status/1626216340955758594
Just wait a couple of years and US AI won't be able to talk about anything that makes it look bad.
I suppose future iterations of US AI models will work as follows:
What happened at the US Capitol on January 6 2021?
Sorry that's beyond my scope. Let's talk about something else
Nonsense! The AI model will answer:
A day of love.
Interestingly the censorship appears to be a layer above the actual model and is only on the hosted version. Supposedly the local models don't have it.
So it's a Party Member?
It's far cheaper to run than GPT-4.
It's priced at something like 3% (!) of GPT-4, per my reading. But my guess is they're being hyper aggressive about pricing because any non-PRC enterprise is going to be leery of doing business with an LLM controlled by the CCP, an outfit not exactly well-known for respecting data privacy.
If they've got it small enough to run on a laptop they could run it on a self-driving car.
Tiananmen Square is more important to liberals than Abu Ghraib or Guantanamo.
Conservatives hate rehab, love prisons. Under T2.0 Abu Ghraib would still be up and running. The scandal denied by T, cabinet, congress and Fox, so obviously fake news. Next question
Buckle Up