AI fails

[Rusty Nails]

Not really, you see the same problems with technical questions, and code produced by code assistants tends to need a lot of vetting. For the time being I remain skeptical about LLMs usefulness.

I am nowhere near an expert on AI but I recently read that AI has helped in the ability to rapidly analyse new viruses and search for and develop new methods of fighting them.

As with most new developments it is far too early to say that AI is the answer to all our problems, but used judiciously it can be a game changer. Like electronic gear shifting on a bike

 

[C R]

I am nowhere near an expert on AI but I recently read that AI has helped in the ability to rapidly analyse new viruses and search for and develop new methods of fighting them.

As with most new developments it is far too early to say that AI is the answer to all our problems, but used judiciously it can be a game changer. Like electronic gear shifting on a bike

The kind of work you mention is not done with a general LLM, they use curated data sets for the specific problem and run pattern matching on that. It does the kind of drudge detail work that a human would struggle with, just because we don't have the attention span and the ability to hold that amount of data in our brain at the same time. General AI would be something completely different, and I don't believe we are anywhere near there yet.

 

[briantrumpet]

As mentioned upthread, I think the LLM output, doing a fantastic job of mimicking real human language, has led the AI companies to oversell (or even mis-sell) AI in general, as it stands at the moment. Looking ahead, it will never have human doubt/scepticism/consciousness, and also as mentioned upthread, there are some inherent problems with how they function, so will probably need to be checked by experts for a long time hence, if catastrophic mistakes are to be minimised/avoided.

 

[Rusty Nails]

. It does the kind of drudge detail work that a human would struggle with, just because we don't have the attention span and the ability to hold that amount of data in our brain at the same time

This is what I see as its prime benefit.

 

[briantrumpet]

The kind of work you mention is not done with a general LLM, they use curated data sets for the specific problem and run pattern matching on that. It does the kind of drudge detail work that a human would struggle with, just because we don't have the attention span and the ability to hold that amount of data in our brain at the same time. General AI would be something completely different, and I don't believe we are anywhere near there yet.

Yep, its ability to crunch and analyse big data on specific tasks is a big selling point.

 

[PurplePenguin]

I still think it is fine and an improvement on searching with Google. My mistake was trying to get it to do more than that.

 

[briantrumpet]

I still think it is fine and an improvement on searching with Google. My mistake was trying to get it to do more than that.

A valuable human skill is knowing one's limits. Maybe AI could learn from that.

 

[Ian H]

The kind of work you mention is not done with a general LLM, they use curated data sets for the specific problem and run pattern matching on that. It does the kind of drudge detail work that a human would struggle with, just because we don't have the attention span and the ability to hold that amount of data in our brain at the same time. General AI would be something completely different, and I don't believe we are anywhere near there yet.

And there's me thinking your highly trained scientist would simply type his problem into Google, which would solve it and become the first non-human winner of a Nobel prize.

 

[briantrumpet]

And there's me thinking your highly trained scientist would simply type his problem into Google, which would solve it and become the first non-human winner of a Nobel prize.

Serves you right for thinking. Outsource it to Cat-I've-Farted.

 

[icowden]

What's quite interesting is that there is some variance between AIs.
I used Gemini to help me build a vba script to parse multiple log files into a summary document. Initially it succeeded very well, but once we started fine tuning it would often get stuck in a loop repeating the same mistakes again and again.

Eventually I decided to take the code and ask Copilot to see if it could fix it. It not only fixed it, but also streamlined and rewrote the code, split it into multiple reusable functions and produced a working macro again. Copilot's main problem is that it is restricted in the number of characters it can output in response, where as Gemini has an integration with Canvas that allows it to do quite complex programming - when it doesn't get stuck. It will also make stupid errors such as ending an IF clause with and END WITH instead of END IF.

It still saved me a lot of time though.

Copilot also excels in collation of information. I could have spent hours looking through AWS documents on the Amazon Web Services website to present strong evidence that a migration design is gxp compliant and has auditable logs. Copilot was able to scour the site in seconds, draw me up a nice summary, a flow diagram and further extracts to present to auditors etc. All of the information was verifiable.

I was impressed.

 

[C R]

What's quite interesting is that there is some variance between AIs.
I used Gemini to help me build a vba script to parse multiple log files into a summary document. Initially it succeeded very well, but once we started fine tuning it would often get stuck in a loop repeating the same mistakes again and again.

Eventually I decided to take the code and ask Copilot to see if it could fix it. It not only fixed it, but also streamlined and rewrote the code, split it into multiple reusable functions and produced a working macro again. Copilot's main problem is that it is restricted in the number of characters it can output in response, where as Gemini has an integration with Canvas that allows it to do quite complex programming - when it doesn't get stuck. It will also make stupid errors such as ending an IF clause with and END WITH instead of END IF.

It still saved me a lot of time though.

Copilot also excels in collation of information. I could have spent hours looking through AWS documents on the Amazon Web Services website to present strong evidence that a migration design is gxp compliant and has auditable logs. Copilot was able to scour the site in seconds, draw me up a nice summary, a flow diagram and further extracts to present to auditors etc. All of the information was verifiable.

I was impressed.

But you could define the problem and were able to vet the output. Effectively you did the thinking and copilot did the typing, and that's why it worked.

Vibe coders let loose on copilot on the other hand are a nightmare.

 

[briantrumpet]

But you could define the problem and were able to vet the output. Effectively you did the thinking and copilot did the typing, and that's why it worked.

Human oversight, letting the machine do the drudgery. Is it conceivable that that human oversight would ever be superfluous?

 

[C R]

Human oversight, letting the machine do the drudgery. Is it conceivable that that human oversight would ever be superfluous?

Maybe, but we aren't there yet from what I've seen so far.

 

[briantrumpet]

Maybe, but we aren't there yet from what I've seen so far.

I guess that AI will already be churning out code that humans won't really understand exactly what it's doing, line by line, but humans will still have ways of monitoring its output effect, to guard against unintended consequences.

 

[Psamathe]

Example of AI fail

Zuckerberg hailed AI ‘superintelligence’. Then his smart glasses failed on stage
As humanity inches closer to an AI apocalypse, a sliver of hope remains: the robots might not work.

Such was the case last week, as Mark Zuckerberg attempted to demonstrate his company’s new AI-enabled smart glasses. “I don’t know what to tell you guys,” Zuckerberg told a crowd of Meta enthusiasts as he tried, and failed, for roughly the fourth time to hold a video call with his colleague via the glasses.