Weird stuff you get from AI assisted searches

[tomterp]

While AI searches tend to be much improved over previous generations, I still get some weird stuff. 

To the question - how many times did the Nats make the playoffs with Dave Martinez as manager, I got:

The Washington Nationals made the playoffs twice with Dave Martinez as manager: once in 2019 as a Wild Card team, and again in 2025 as a Wild Card team

So put that in your Sunshine thread.   

[dracnal]

While AI searches tend to be much improved over previous generations, I still get some weird stuff. 

To the question - how many times did the Nats make the playoffs with Dave Martinez as manager, I got:

So put that in your Sunshine thread.   

Unfortunately, it's a safe bet those searches are going to start getting worse. We've reached a point where the number of AI generated pages has massively increased, which creates a poisoned pool for it to pull from. An example is a search I just saw for famous haiku. The results showed two or three real haiku from respected links and 7 results that came from AI generated lists of famous haiku, which essentially took the first three actual ones and recombined the lines randomly to form new ones. They all kept the 5-7-5 rule, made zero sense, and came from pages with names like '10 Haiku everyone should know,' 'Beautiful Haiku,' 'Haiku to make you think.' etc.

We're rapidly approaching the point where it's not just a case of AI ingesting crappy or spurious results and making things up, but rather it eating its own poop, vomiting it up, and going back for seconds. Short of the various AI makers creating a collaborative standard or maker's mark to say 'this is AI gen content, do not use me in your models or results,' I don't know how they prevent it. Someone will figure out a method at some point and it'll improve again for a time, but this cycle is going to repeat, a lot.

[imref]

Unfortunately, it's a safe bet those searches are going to start getting worse. We've reached a point where the number of AI generated pages has massively increased, which creates a poisoned pool for it to pull from. An example is a search I just saw for famous haiku. The results showed two or three real haiku from respected links and 7 results that came from AI generated lists of famous haiku, which essentially took the first three actual ones and recombined the lines randomly to form new ones. They all kept the 5-7-5 rule, made zero sense, and came from pages with names like '10 Haiku everyone should know,' 'Beautiful Haiku,' 'Haiku to make you think.' etc.

We're rapidly approaching the point where it's not just a case of AI ingesting crappy or spurious results and making things up, but rather it eating its own poop, vomiting it up, and going back for seconds. Short of the various AI makers creating a collaborative standard or maker's mark to say 'this is AI gen content, do not use me in your models or results,' I don't know how they prevent it. Someone will figure out a method at some point and it'll improve again for a time, but this cycle is going to repeat, a lot.

spent last night reading a report on how this is killing market research. Many surveys now are using AI to provide data, which poisons future data sets.

[JCA-CrystalCity]

I am going to be interested in how the government uses AI in rule making. There's a push to use it in drafting notices. On the one hand, there's an awful lot of bad technical writers with the pen on notices, and I have had former colleagues run drafts through AI tools that make the drafts easier to follow. On the other hand, I wouldn’t want AI to actually think through a comment response given its tendency to make stuff up.

[varoadking]

I am going to be interested in how the government uses AI in rule making. There's a push to use it in drafting notices. On the one hand, there's an awful lot of bad technical writers with the pen on notices, and I have had former colleagues run drafts through AI tools that make the drafts easier to follow. On the other hand, I wouldn’t want AI to actually think through a comment response given its tendency to make stuff up.

Government has been full of artificial intelligence since time immemorial...

[skippy1999]

Government has been full of artificial intelligence since time immemorial...

[1995hoo]

We were discussing generative AI at work recently and I analogized it to the Borg. Whatever you provide gets assimilated into the collective and the AI then draws upon it in whatever way it deems appropriate in the future, whether for good or ill (in this context, "ill" need not mean "malevolent" but can also refer to the sense of "garbage results," such as the fake legal citations that keep making the news). The particular context of our discussion had to do with attorneys disclosing privileged or confidential material to AI. Unlike Picard or Seven of Nine, the privileged or confidential material is not going to be de-assimilated.

Of course, one of the people to whom I was speaking had no clue what I meant by the Borg.

[imref]

We were discussing generative AI at work recently and I analogized it to the Borg. Whatever you provide gets assimilated into the collective and the AI then draws upon it in whatever way it deems appropriate in the future, whether for good or ill (in this context, "ill" need not mean "malevolent" but can also refer to the sense of "garbage results," such as the fake legal citations that keep making the news). The particular context of our discussion had to do with attorneys disclosing privileged or confidential material to AI. Unlike Picard or Seven of Nine, the privileged or confidential material is not going to be de-assimilated.

Of course, one of the people to whom I was speaking had no clue what I meant by the Borg.

Bjorn?

[Natsinpwc]

Bjorn?

Assimilate them

https://www.youtube.com/shorts/p7z2VHjGpdY

[Count Walewski]

I've found AI quite helpful for trip planning. For best results, I sketch out an itinerary and ask ChatGPT to critique it. I often get recommendations for attractions I can see that are close to the ones I am already planning to see.

But yeah it makes up stuff. For my Switzerland trip earlier this month, I asked it what the best place to see Bernese Mountain Dogs in the Bernese Oberland is. It told me all about this dog shelter for Bernese Mountain Dogs where you can volunteer to take the dogs for walks through the Alps. It sounded amazing and unfortunately it was 100% hallucinated and does not exist IRL.

[JCA-CrystalCity]

I've found AI quite helpful for trip planning. For best results, I sketch out an itinerary and ask ChatGPT to critique it. I often get recommendations for attractions I can see that are close to the ones I am already planning to see.

But yeah it makes up stuff. For my Switzerland trip earlier this month, I asked it what the best place to see Bernese Mountain Dogs in the Bernese Oberland is. It told me all about this dog shelter for Bernese Mountain Dogs where you can volunteer to take the dogs for walks through the Alps. It sounded amazing and unfortunately it was 100% hallucinated and does not exist IRL.

for  St Bernards, there are two mountains - Le Grand St Bernard and Le Pettit St Bernard. Big on cycling race stages. Always show the dogs on TV.

[tomterp]

I asked co-pilot "what harp played on run around", and it gave me some completely fabricated story about a fully synthetic sound created electronically to do x, y and z.  I didn't copy the response.  I always thought John Popper did it.

So I asked a different question:


Question - did john popper play the harmonica on run around?

Yes — John Popper played the harmonica on Blues Traveler’s hit “Run Around.”

John Popper, the band’s lead vocalist and frontman, is a master harmonica player and a founding member of Blues Traveler. He is known for his virtuosic diatonic harmonica work, often trading between instruments depending on the song’s needs Wikipedia. On “Run Around,” which was written by Popper and became the band’s breakthrough single in 1995, he used his harmonica prominently in the arrangement Wikipedia.

The song’s harmonica parts are a signature element of its sound, blending blues-rock energy with the band’s signature improvisational style. Live performances of “Run Around” often feature Popper’s harmonica as a central feature, and archival recordings from the era confirm his role in the track’s instrumentation YouTube+1.

In fact, harmonica performances on “Run Around” are so iconic that they’ve been highlighted in Blues Traveler’s live sets and even in fan compilations of the band’s harmonica work Harmonica.com.

Sure I had caught AI in the process of making crap up again, I went back two screens to get to my original question, but it reconsidered and the whole synthetic sound thing disappeared and it came up with a similar John Popper answer.  What a croc.

[imref]

I asked co-pilot "what harp played on run around", and it gave me some completely fabricated story about a fully synthetic sound created electronically to do x, y and z.  I didn't copy the response.  I always thought John Popper did it.

So I asked a different question:


Question - did john popper play the harmonica on run around?

Sure I had caught AI in the process of making crap up again, I went back two screens to get to my original question, but it reconsidered and the whole synthetic sound thing disappeared and it came up with a similar John Popper answer.  What a croc.

GIGO. Most of the content it trained on came from Reddit.

I find AI to be incredibly useful when working with our corporate data, and a complete crapshoot when working with public data.

[Squab]

most of my work (software engineer/sre/infrastructure-type person) has now evolved into either supporting AI companies, supporting people using AI massively, or using it myself. it's a pretty miserable existence and I genuinely wish we could put this genie back in the bottle, but we can't.

undeniably a useful tool for some things though, but stick to Claude (Sonnet and up), ChatGPT (the new 5.6 stuff if you can), or Gemini (ideally the Pro model, the flash one that powers Google AI mode is kind of stupid, but still useful for simple stuff) in that order imo. Copilot is a literal joke toy and can not be trusted for anything, and Grok is a nazi lmao.

[tomterp]

Oh, for you aspiring harmonica players out there, some follow up details from our AI genius:

In Blues Traveler’s hit “Run Around”, John Popper played a standard diatonic harmonica in the 2nd position (also called “cross harp”) .

According to harmonica lesson resources for the Four album, the harmonica part for “Run Around” is in the key of G, with the C harmonica played in 2nd position. This means the player used a C diatonic harmonica, pressing the keys in the second position to produce the notes of the G major scale.

 

It's an f-ing harmonica, THERE ARE NO KEYS TO PRESS.  You inhale, or you blow, in any of 10 holes.  By playing a C harmonica to a song in the key of G, you are purposely omitting some notes from the major key of G in favor of more bendability an some notes that give you that bluesy sound and aren't available on the major key harp.  So second position is purely a choice of a different "cross" harp. 

Or to put it another way, if I know the song is in say the key of G, and I want that bluesy "cross harp" sound, I'll just grab the C harp.  And so on for any other key.  Song in Key of C?  Grab the F.  Key of A?  Grab the D.  Just simple math.