← All posts

What I Think About AI Agents: Stop Building Agents, Focus on the Tools

By Alvin Hartono

I recently stumbled upon a very insightful perspective on the current AI agent frenzy, and it's been bouncing around in my head ever since. The author, an ML engineer, pointed out a pattern they're seeing: everyone's obsessed with agent loops, orchestration, frameworks, and "autonomous workflows," while neglecting the foundational tools that actually *do* the work. This really got me thinking about where the real value lies in the AI space.

The 'Fancy Shell Script' Problem

The core of the argument is that many of these so-called AI agents are essentially glorified shell scripts. They string together a series of API calls, all orchestrated by a large language model (LLM). The author argues that the current stacks are fundamentally: LLM + integrations (Slack, Gmail, web search, "parse this PDF," etc.).

This resonated with me because I've seen similar patterns in other tech bubbles. We get so caught up in the *idea* of something revolutionary that we forget to build the solid foundation it needs to stand on. It's like building a beautiful skyscraper on a swamp – impressive to look at, but ultimately unsustainable.

Are We Building the Wrong Thing?

Here's the crucial question: are we focusing on the wrong layer of the AI stack? Are we spending too much time and energy on orchestration and not enough on the underlying tools that make the orchestration worthwhile?

Think about it. An agent is only as good as the tools it has at its disposal. If those tools are weak, unreliable, or inefficient, the agent will be too, no matter how sophisticated its orchestration logic. It’s the classic garbage in, garbage out scenario.

The Allure of Autonomy (and Its Pitfalls)

I understand the appeal of autonomous agents. The idea of an AI that can independently solve complex problems, learn from its mistakes, and continuously improve is incredibly exciting. It’s the stuff of science fiction, and we’re naturally drawn to it.

However, I think we need to be realistic about the current state of the technology. We're not at the point where AI agents can truly operate autonomously and reliably in complex, real-world scenarios. Trying to force that level of autonomy prematurely could lead to a lot of wasted effort and disappointing results.

The Illusion of Intelligence

LLMs are incredibly powerful, but they're not magic. They're sophisticated pattern-matching machines that can generate text, translate languages, and answer questions with impressive accuracy. But they don't possess true understanding, reasoning ability, or common sense.

An AI agent that relies solely on an LLM for its decision-making is susceptible to all the limitations of the LLM itself. It can be easily fooled by adversarial inputs, it can generate nonsensical or even harmful outputs, and it can struggle with tasks that require real-world knowledge or experience.

Focusing on the Fundamentals: Building Better Tools

So, what's the alternative? The author's suggestion is to shift our focus from agent orchestration to building better tools. This means investing in research and development of more robust, reliable, and efficient tools for tasks like:

* Data Extraction and Processing: Improving the accuracy and speed of extracting information from various sources (documents, websites, databases, etc.). * Knowledge Representation and Reasoning: Developing better ways to represent knowledge and enable AI systems to reason about it. * Planning and Decision-Making: Creating more sophisticated algorithms for planning and decision-making that go beyond simple LLM prompting. * Human-AI Collaboration: Designing tools that allow humans and AI agents to work together effectively.

The Power of Specialized Tools

Instead of trying to build a single, general-purpose agent that can do everything, we should focus on creating specialized tools that excel at specific tasks. These tools can then be combined and orchestrated by a simpler, more reliable agent.

Think of it like building a car. You don't try to build the entire car out of a single piece of metal. You build individual components – the engine, the wheels, the chassis – and then assemble them into a complete vehicle. The same principle applies to AI agents.

What I Would Do Differently

If I were building an AI-powered product today, I would prioritize building a strong foundation of specialized tools before even thinking about agent orchestration. I would focus on solving specific problems with reliable, efficient tools, and then gradually add more sophisticated orchestration as needed.

Start Small, Iterate Quickly

I would start with a small, well-defined problem and build a tool to solve it. Then, I would iterate on that tool based on user feedback and real-world performance. Only after I had a solid foundation would I start to explore more complex orchestration scenarios.

Embrace Human-in-the-Loop

I would also embrace the concept of human-in-the-loop. Instead of trying to automate everything, I would design my system to allow humans to intervene and provide guidance when needed. This would not only improve the accuracy and reliability of the system but also allow humans to learn from the AI and vice versa.

Focus on Value, Not Hype

Ultimately, I would focus on creating real value for users, rather than chasing the latest AI hype. I would ask myself: what problems can I solve with AI that are truly meaningful and impactful? And what tools do I need to build to solve those problems effectively?

The Long Game

I believe that AI has the potential to transform the world in profound ways. But we need to be patient, realistic, and focused on building a solid foundation. By prioritizing the development of robust, reliable, and efficient tools, we can unlock the true potential of AI and create a future where AI agents are not just fancy shell scripts, but powerful problem-solving partners.

Maybe the real breakthrough comes when we stop trying to create artificial general intelligence (AGI) overnight and instead focus on augmenting human intelligence with incredibly useful and specialized AI tools. That seems like a much more achievable – and valuable – goal in the short and medium term. It's about building *smart* tools, not necessarily *sentient* agents. And that's a distinction worth remembering.

Keep reading