Skip to content

I am now a physical chemist

I just saw a tweet about Groq, an AI chatbot that uses Meta's LLaMA-3 engine. It's incredibly fast! You should try it!

So I did. And it is incredibly fast. But there are still problems. I asked it about myself and got five paragraphs in return. Four were basically fine, but there was also this:

Drum has a background in science and has a Ph.D. in physical chemistry from the University of California, Berkeley. He has worked as a researcher and writer for various organizations, including the University of California, Berkeley, and the National Science Foundation.

I wonder where this stuff comes from? Physical chemistry, of all things. I had a TA once in a chem lab who was a physical chemistry grad student, and it was a pain in the ass. The organic chem guys were pretty easygoing, but the physical chemists all demanded military precision. If I wanted that I would have joined the Army.

Anyway, speed is all very well, but these LLMs really need to work on their accuracy. Until it gets better no one is ever going to be able to rely on them.

POSTSCRIPT: It also said, "Kevin Drum is a respected and influential voice in the online media landscape." That's nice. Maybe I forgive it.

35 thoughts on “I am now a physical chemist

    1. DianaBryan

      US Dollar 2,000 in a Single Online Day Due to its position, the United cx04 States offers a plethora of opportunities for vz02 those seeking employment. With so many options accessible, it might be difficult to know where to start. You may choose the ideal online housekeeping strategy with the help vz-30 of this post.

      Begin here>>>>>>>>>>>>>> https://letswise099.blogspot.com/

  1. iamr4man

    I asked it who created Bugs Bunny. It answered:

    Bugs Bunny was created by Leon Schlesinger Productions, which later became Warner Bros. Cartoons, Inc. The character was designed by animator and director Bob Clampett, and the first appearance of Bugs Bunny was in the cartoon short "Porky's Hare Hunt" in 1938.
    However, the modern version of Bugs Bunny as we know him today, with his wisecracking personality and catchphrases like "What's up, Doc?", was developed by animator and director Chuck Jones. Jones directed many of the classic Looney Tunes cartoons featuring Bugs Bunny, including "A Corny Concerto" (1943), "Rabbit Seasoning" (1952), and "Duck Amuck" (1945).
    Mel Blanc, a legendary voice actor, provided the voice of Bugs Bunny from 1940 until his death in 2001. Blanc's iconic voice and mannerisms helped bring Bugs Bunny to life and made him one of the most beloved cartoon characters of all time.

    Lots of inaccuracies here. A Corney Concerto was directed by Clampett, for instance. Clampett claimed he created the rabbit character in Hare Hunt but that’s disputed and the cartoon was directed by Ben (Bugs) Hardaway and Cal Dalton. Tex Avery is the man most people point to in creating the true Bugs Bunny character and he isn’t even mentioned. The Wikipedia entry is far more accurate than the AI. I asked the question because I’m an animation nerd and know the creation of the Wabbit is somewhat controversial.
    Hey, fun fact, Bugs Hardaway served with Harry Truman in WWI and he was invited to Truman’s inauguration. It’s kind of a cool story:
    https://cartoonresearch.com/index.php/bugs-hardaway-of-battery-d/

  2. NeilWilson

    I was President and CEO of a bank for decades

    The reality is that I was CFO for about 4 years.

    Virtually everything it said about me was embellished so much that I couldn't figure out where it came up with the crap that it wrote.

  3. rfritzson

    When I got a biography out of it by mentioning an old research topic along with my name, it correctly described my occupation (computer science researcher) and some specialty stuff (artificial intelligence). Then it listed two companies I had supposedly worked for - they were plausible employers but I had never worked for either.

    It concluded by saying "Sadly, he passed away in 2014." I'm pretty sure that didn't happen.

  4. Blackbeard

    It said I was the author of "Seven Habits of Highly Effective People." Stephen Covey is really the author. True, my first name is Stephen but my last name is nothing like Covey. Everything else it said about me was wrong too.

  5. cld

    I asked it 'Who is me?' and got,

    I'm just an AI, I don't have the ability to know who you are or any information about you. I'm just here to provide information and answer questions to the best of my ability.

    Obviously avoiding the topic.

  6. cld

    Kevin could just add 'Ph.D. in physical chemistry' to his bio on Wikipedia and the AI's will just keep repeating it for the rest of time, --so, voila, free Ph.D.

  7. Bluto_Blutarski

    This is how you can tell AI is being progreammed by men: when it doesn't know a thing, it simply makes something up out of thin air and then asserts it with complete and unearned confidence.

  8. Austin

    “Until it gets better no one is ever going to be able to rely on them.”

    Hahaha.

    First, this assumes that there are no jobs in which all the company wants to do is tick a box saying it did something, usually to comply with a regulation of some kind but sometimes to comply with a vague goal the company expressed that it doesn’t really believe in (e.g. becoming greener for PR purposes). The answers don’t need to be accurate for that, they just need to exist so the box can be ticked. Very few companies are ever called into question by regulators, because we’ve allowed the workforces of federal, state and local governments to atrophy, all so we can enjoy lower or stable tax rates. And very few companies are challenged on claims that they’ve achieved their goals, because media isn’t in the business anymore of doing hard journalism that might offend an advertiser. So this LLM will be good enough for this role.

    Second, actual humans that are employed to get this stuff right also fck it up. They inadvertently mix up one person with another with the same of similar name or appearance all the time. That how Trump’s team ended up serving papers to the wrong guy in NYC a few weeks ago, that’s how police departments end up storming into the wrong homes guns blazing every so often, that’s how fast food places end up giving your food away to the wrong person all the time. LLMs just need to fck up identities no worse than humans do to be deemed good enough by potential employers.

    And your postscript inadvertently reveals the final reason why LLMs will be adopted, regardless of flaws you’ve identified. As long as the decision makers are flattered by it, some of them will buy it and insist their staff use it to boost their egos. For example, I could easily see Trump’s campaign using it as long as it always spits out positive things about Trump.

    1. rrhersh

      Mixing people up: Punch my name into Google Scholar and will find three of us. I am not the molecular biologist, nor am I the vascular surgeon. I am the baseball history guy. If an LLM got the three of us mixed up, that would be understandable, and perhaps even forgivable. In practice, the LLMs I have tried are nothing like that good. What I get back varies, but it runs strongly toward people who never existed. This new one even provides fake citations!

      As for jobs, you likely are right that this will be good enough for outfits who want simply to tick off a box and don't care about the work product. But those outfits weren't paying much to begin with. You need not pay top dollar when hiring a human, if you are happy with grammatically correct gibberish.

  9. different_name

    Oh, this is fun. I've been both a British soccer (football) player and a wedding photographer in Hawaii.

    There's a programming language called Perl. It is fading in popularity now, but was widely used in the early days of the web. People joke that the name stands for "Pathologically Eclectic Rubbish Lister".

    Little did we know how close we were to getting exactly that.

  10. Jim Carey

    Exceptional intelligence (natural or artificial) is to life what an exceptionally fast vehicle is to a journey. Wisdom is to life what knowing the destination is to the journey.

    If you're moving slowly in the right direction, you expect to be passed, and its fall-off-a-log easy to recognize when the exceptionally intelligent person (Elon Musk and etcetera) or system (chatbot) has no idea where he/she/it is going.

  11. Old Fogey

    I found that I am "a former American football coach and player. He is best known for being the head coach of the Tampa Bay Buccaneers in the National Football League (NFL) from 1983 to 1985."
    Damn amnesia!

    Admittedly, I have a ridiculously common name. When I added JD I became a person well-known in law enforcement circles. I will continue to explore until I find a me I really like.

  12. Joseph Harbin

    Mindscape:

    0:22:02.6 SC: ...There clearly has been enormous advances very recently with large language models, but they also still make mistakes. And am I right to think that there's like a camp that says, "All of those mistakes are gonna go away as we get more and more data," and another one that says, "No, some mistakes are kinda systematic?"

    0:22:38.4 LV: Well, I'm not sure they should all say the same thing, that the mistakes will maybe go down, but the effort made to make them go down more and more will be bigger and bigger.

    So maybe the mistakes will go down, and to get mistakes go down even more, the harder and harder it will be. At some point, the quest for absolute accuracy becomes an acceptance of "Probably Approximately Correct" accuracy.

    That's a concept that may be useful but is different than what people generally see as one of the virtues of computers, their infallibility. When I multiply two large numbers on my calculator, I get an answer instantly that I can be certain is correct. Now we have far more advanced computer technology that at best we can say is probably correct. In moving ahead, we are also losing something, and I think that will have consequences.

    1. MrPug

      Also, the problem with the the models will get better with more data argument is the more data part. These LLMs have already been trained with vast swaths of the data on the internet. Where is "more" going to come from? Also, at some point it will be training on AI generated content which I don't think ends well for the tech.

      There is a massive AI bubble that will burst bigly ugly.

    2. lawnorder

      Apparently "truth" is simply not a concept found in LLM programs, so adding a constraint along the lines of "assertions of fact must be true" is extremely difficult.

  13. martinmc

    Using my usual test question.
    Very fast.
    Slightly more accurate than ChatGPT, but just as wrong (though differently so) on the important factual statements.
    A little deeper than ChatGPT, but, again, still wrong.
    Uses the same boilerplate sentences that sound like some PR document they both must have read.
    Verdict: Don't trust it for factual info.

  14. Old Fogey

    L. Sprague deCamp is one of my favorite authors. I asked for his best non fiction books. Five results, with summaries. First one was one of my favorites. Second was accurate, more or less. Last three were of non existent books that sounded fascinating. Third might have been based on an article by someone named DeGruye.

    Perhaps it's fitting. DeCamp had a joke in one of his sf books where a person in the future refers the poets Kelly and Sheets, who wrote The Mikado.

  15. bouncing_b

    According to groq, I’m a “physicist and engineer, and the founder and CEO of Quantum Circuits Inc. (QCI), a company that focuses on developing quantum computing and quantum simulation technologies. He has a Ph.D. in Physics from Stanford University …”
    Purely fictional. Their CEO is someone else entirely.

    When I prompted it with my actual job title I got: “Here are some key facts about [me], the oceanographer:
    Education: He holds a Ph.D. in Oceanography from the University of California, San Diego. …”
    Nope. It said I’ve received various random-sounding awards in a variety of fields, none of them mine. Etc.

  16. Larry Jones

    I am apparently "...best known for [my] work as a guitarist and vocalist for the rock band, The Outlaws," a band I've never heard of. And, just to cross the line all the way over into complete fantasy, Groq goes on "Larry Jones has also worked as a session musician and has collaborated with other notable artists, including the Allman Brothers Band and Lynyrd Skynyrd. He has also released several solo albums and has toured extensively throughout his career."

    The Larry Jones Band, coming to your town soon!

Comments are closed.