What I learned Creating 475,000 lines of Code With AI
It's no secret I'm a big critic of AI. The Acolytes, like true Acolytes, only extoll the virtues without any connection to actual value.
Of course "actual value" may vary...
My unvarnished opinion after spending 12+ hours a day, 7 days a week working to an actual commercial purpose?
AI is pretty useless.
The problem is with the engine itself. At the core it is merely a pattern matching engine hard-coded to love the sound of it's own voice.
This causes MAJOR problems for us "regular people".
But there is a fix and fortunately it's an easy one.
Think about how you have a conversation with another person. There is back-and-forth, rewind, fast forward, 'rabbit trails', 'getting back on track', etc.
With an LLM it is a strict linear conversation. And, worse, the LLM can't 'ignore' anything. A single word can "pollute" the LLM's decision space causing it to go off in crazy directions.
When you have a "conversation" all of the prompts and all of the responses are sent with the new stuff you enter. This is "context" but also pretty dumb from a practical standpoint.
That's because in a normal conversation your partner(s) have the ability to 'ignore', 'forget' and 'go back' - unlike the LLM.
Enter the fix.
Instead of repeatedly entering new prompts, stay with one prompt and develop it. When the LLM gives you output and you think "oh, that's good" don't rush to enter a new prompt.
Go back to the prompt you entered and modify that with the 'good stuff'.
Repeat that process until the idea or concept is more fully developed.
This is how humans "think" and "communicate" - we often don't know what we want to say until we've "talked it out".
If you "talk it out" with the LLM it won't forget the crazy stuff you said to get to your actual point. It thinks that crazy stuff is needed and helpful. But for YOU it's just throw-away.
So, throw it away by using a single prompt and editing it with the new, better stuff you and the LLM come up with.
This is why LLMs suck at decoding human communication. And since the programmers are probably not the most socially adept people in the world... that's not a surprise.
GIGO
Executive Director @ MSCI Research
6moComplete blog post here: https://dhruv-sharma.ovh/post/abms-agentic-llms/