The Dream of AI Agents: Can We Trust the Machines?
Imagine a world where intelligent machines, dubbed "agentic AI agents," are in control. No longer will humans be burdened with mundane tasks, and innovation would be limitless. However, for many experts, this vision is more like a pipe dream than a reality. A recent research paper has cast doubt on the feasibility of agentic AI agents, suggesting that they may not live up to their promise.
According to Vishal Sikka, a former SAP CTO and current head of an AI services startup called Vianai, AI agents are inherently flawed due to "hallucinations" – or made-up information. This limitation is fundamental to the nature of language models like LLMs (Large Language Models), which are used to power these intelligent machines.
"Sloppy thinking will be tolerated," Sikka warns. While not all human errors can be replicated by AI, it's unlikely that every single mistake will be entirely eliminated. "So we should forget about AI agents running nuclear power plants?" he jokingly asks.
However, some prominent figures in the field disagree with Sikka's pessimism. Robinhood CEO Vlad Tenev and Stanford-trained mathematician Tudor Achim have founded a startup called Harmonic that claims to have made breakthroughs in guaranteeing the trustworthiness of AI systems. By using formal methods of mathematical reasoning to verify an LLM's output, they believe it is possible to filter out hallucinations and create more reliable agents.
Harmonic's solution involves encoding outputs in the Lean programming language, which can be verified for correctness. While their focus has been narrow so far, coding is an organic extension of their main mission – mathematical superintelligence.
Despite these advancements, most models still suffer from "hallucinations," as OpenAI scientists have demonstrated. In a paper published last September, they asked three models, including ChatGPT, to provide the title of a lead author's dissertation and found that all three provided fake titles.
However, rather than abandoning the concept of agentic AI agents, some argue that guardrails can be created to mitigate hallucinations. This perspective holds that it is possible to create systems that learn from and overcome limitations like hallucinations.
Ultimately, the question remains whether these advancements will lead to a world where human cognitive activity is automated, potentially improving quality and lives. While mathematical verifiability may not provide clear answers, one thing is certain: we are rapidly hurtling towards an AI-driven future, and it's up to us to consider what that means for our society.
For now, the debate rages on, with no definitive resolution in sight. As Alan Kay so aptly put it, "The mathematical question is beside the point." Instead, we must focus on understanding the impact of these machines on our lives.
Imagine a world where intelligent machines, dubbed "agentic AI agents," are in control. No longer will humans be burdened with mundane tasks, and innovation would be limitless. However, for many experts, this vision is more like a pipe dream than a reality. A recent research paper has cast doubt on the feasibility of agentic AI agents, suggesting that they may not live up to their promise.
According to Vishal Sikka, a former SAP CTO and current head of an AI services startup called Vianai, AI agents are inherently flawed due to "hallucinations" – or made-up information. This limitation is fundamental to the nature of language models like LLMs (Large Language Models), which are used to power these intelligent machines.
"Sloppy thinking will be tolerated," Sikka warns. While not all human errors can be replicated by AI, it's unlikely that every single mistake will be entirely eliminated. "So we should forget about AI agents running nuclear power plants?" he jokingly asks.
However, some prominent figures in the field disagree with Sikka's pessimism. Robinhood CEO Vlad Tenev and Stanford-trained mathematician Tudor Achim have founded a startup called Harmonic that claims to have made breakthroughs in guaranteeing the trustworthiness of AI systems. By using formal methods of mathematical reasoning to verify an LLM's output, they believe it is possible to filter out hallucinations and create more reliable agents.
Harmonic's solution involves encoding outputs in the Lean programming language, which can be verified for correctness. While their focus has been narrow so far, coding is an organic extension of their main mission – mathematical superintelligence.
Despite these advancements, most models still suffer from "hallucinations," as OpenAI scientists have demonstrated. In a paper published last September, they asked three models, including ChatGPT, to provide the title of a lead author's dissertation and found that all three provided fake titles.
However, rather than abandoning the concept of agentic AI agents, some argue that guardrails can be created to mitigate hallucinations. This perspective holds that it is possible to create systems that learn from and overcome limitations like hallucinations.
Ultimately, the question remains whether these advancements will lead to a world where human cognitive activity is automated, potentially improving quality and lives. While mathematical verifiability may not provide clear answers, one thing is certain: we are rapidly hurtling towards an AI-driven future, and it's up to us to consider what that means for our society.
For now, the debate rages on, with no definitive resolution in sight. As Alan Kay so aptly put it, "The mathematical question is beside the point." Instead, we must focus on understanding the impact of these machines on our lives.
. The idea that they can learn and adapt without human oversight is both exciting and terrifying 
. If we're not careful, these machines could end up perpetuating biases and errors from their creators, rather than truly improving our lives 
.
. But at the end of the day, I think we need a more nuanced conversation about what it means to "trust" AI systems 
. Can we really rely on machines to make decisions that affect our well-being and safety? 
. It's wild to think about how much our lives are going to change with agentic AI agents, but we need to make sure we're ready for the potential downsides 
.
. But on the other hand, I don't want us to forget that there are limitations with these language models like LLMs... some experts say they're prone to "hallucinations". Can we really trust AI agents enough to have them make life-or-death decisions?
and can't even get the title of a dissertation right 
It seems like they're tryin' to create these AI systems that can think for themselves, but who gets to decide what's "good" or "bad"? 
.
... I mean, AI hallucinations are a real problem 
... it's like, what even is the point of creating these machines if they're just gonna make mistakes? 
... and don't even get me started on nuclear power plants 
but can we really create systems that are foolproof? 
. Harmonic's approach is definitely promising and more formal methods like Lean programming could help mitigate those errors. It's also interesting to see some people think that guardrails can be put in place to limit hallucinations – maybe we don't need AI agents taking over everything just yet 
. The real question is, how do we balance progress with responsibility? Can we create systems that learn from their mistakes and become even more reliable over time? I'm rooting for innovation and responsible AI development 
.
. We gotta have a serious convo about the implications of AI on our society 
. It's up to us to consider what it means for our lives when machines are doing all the thinking 
AI agents might sound awesome, but if they're just gonna be unreliable, what's the point? 
️. But seriously, I think Sikka's "hallucinations" thing is legit concern. how do you ensure an AI doesn't make up some BS info when it's not even 100% sure what's true? 
.
what if we could create systems that learn from & overcome limitations like hallucinations? 
. These "hallucinations" are a fundamental problem with how AI models like LLMs work, and I'm not sure what specific guardrails could be put in place to mitigate them 
.
.
. Do we want humans to be free from mundane tasks? Yeah, that sounds great 
. But do we want our reliance on AI to come with unforeseen consequences? Maybe not so much 
. We've seen those LLMs spitting out crazy information before - like, what's the deal with ChatGPT's dissertation title debacle? 
. The idea of encoding outputs in Lean programming language to verify correctness is actually pretty cool