Introducing Reality to Common AI Concerns

Since the “godfather of AI,” Geoffrey Hinton, likened AI’s growing potential to surpass human intelligence within five to ten years to raising a tiger cub that could one day kill you, I’ve seen an uptick in people asking if AI will take over the world, as if the machines are coming for us like Arnold Schwarzenegger in The Terminator. Geoffrey Hinton earned his title for pioneering work on neural networks in the 1980s and his breakthrough work with deep belief networks in the 2000s. While I have enormous respect for pioneers like Hinton, given his intimate knowledge of the mathematical basics that make these systems work, working daily with these systems at the design, development, and deployment level gives you a different perspective – both are valid, both are crucial, we just have different vantage points on the same technology. 

The most significant concerns about AI that I’ve seen tend to come from three main sources: 

• Misunderstanding of current AI capabilities 
• Confusion about AI architecture and requirements 
• Attribution of human motivations to computational systems 

Also Read: How AI Startups Can Compete Against Tech Giants in the Age of OpenAI

Understanding What “Surpassing Human Intelligence” Actually Means 

While his concerns merit serious long-term consideration for AI professionals and researchers, they also spark unnecessary fears amongst the general population that has limited insight into how generative AI works from a technical standpoint. When Geoffrey Hinton speaks of AI “surpassing human intelligence,” he’s not referring to computational speed (which machines achieved decades ago) but to a hypothetical future where AI systems might develop capabilities for independent reasoning, goal-setting, and strategic planning across domains. This is an important distinction: they don’t relate to what AI currently is; they address what AI could potentially become with significant architectural changes. So, to address it theoretically, what would current AI taking over the world even look like? Let’s think about this for a minute. AI requires prompting. Today’s AI functions as a conditional computation system, transforming inputs into outputs based on learned statistical patterns. Without an input (prompt), there’s simply no trigger to initiate the computational process. Even autonomous AI systems need some human-defined directive; otherwise, what exactly are they autonomously doing? 

The Fundamental Limitations of Current AI Systems 

“But what if we automate the prompting?” Okay, let’s follow that logic. Automate… what exactly? There are effectively infinite possible outputs an AI can generate. And with infinite possible inputs, you’d just be creating a system that outputs randomness. AI models operate in what mathematicians call a “high-dimensional latent space” – their output possibilities scale exponentially with sequence length. A GPT-style model working with a vocabulary of 50,000 tokens (units that AI language models process) could theoretically generate 50,000^n different sequences of length n, quickly exceeding the number of atoms in the observable universe. The moment you try to create a “general intelligence” that can “do anything,” you’ve actually created something that can effectively do nothing well. When you remove the specificity of purpose, you get noise, not intelligence. A constantly running program that does nothing isn’t dangerous – it’s just a waste of computing resources. 

The Self-Improvement Paradox 

“What if we develop self-improvement mechanisms for code and architecture?” I’ve created numerous tools that can autonomously modify, embed, and replicate their own code. But true self-improvement would require systems to fully understand their own architecture at a meta-level, which creates a recursive challenge. The system needs to be smarter than itself to meaningfully improve itself. In my first iterations of self-modifying systems, I created what seemed like a good idea on paper: a program that would take initial code, analyze improvements, implement them, then repeat the cycle indefinitely until the code reached perfection. While it could make incremental changes for a few iterations, the system quickly began creating incompatible modifications – layering elements on top of each other, writing text over existing content, choosing clashing colors, and creating “improvements” that conflicted with previous changes. Without explicit optimization targets, the system had no basis for determining if changes represented progress or regression. A system cannot fully predict the consequences of its own modifications without essentially running a complete simulation of itself with those changes, which is currently computationally impossible at the same level of complexity. Most critically, “improvement” isn’t an objective measure – it requires defining what “better” means, which ultimately comes back to human-defined metrics. 

Also Read: The Impact of Increased AI Investment on Organizational AI Strategies

The Challenge of Memory and Planning in AI Systems 

“What about persistent memory and long-term planning outside of active sessions?” Persistent memory isn’t just about storage capacity – it’s about meaningful organization of information. Every data scientist knows the difference between having data and having useful data – raw data contains both relevant information and irrelevant information. Long-term planning also requires a model of causality and prediction that extends beyond pattern recognition. When I’m forecasting data trends for next quarter, I’m integrating countless contextual factors about market conditions, seasonality, and business dynamics that I personally know from experiencing them. Current AI systems don’t have this integrated causal understanding; they approximate it through statistical correlations. They excel at identifying statistical patterns (if A, then B often follows) but cannot independently determine whether A causes B, B causes A, or both are caused by C. They don’t “plan” so much as they predict likely next states based on patterns they’ve seen before. Today’s AI is essentially an improved autocomplete for your search query.

It’s predicting what should come next based on patterns it’s seen. Imagine if people in the early 2000s had suggested Google’s search predictions would “take over the world.” They’d be laughed out of the room. 

The Human Element in AI Risk Scenarios 

Looking at the actual disaster scenarios published by respected organizations like the Centre for AI Safety, what stands out to me is how human-centric these problems actually are. “AI could be weaponized for creating chemical weapons.” That’s a human choosing to build weapons. “AI-generated misinformation could destabilize society.” That’s humans deploying technology for deception. “Power concentrated in fewer hands, enabling oppressive censorship.” That’s human power dynamics. “Enfeeblement where humans become dependent on AI.” That’s human laziness, not AI ambition. Notice a pattern? These aren’t AI fears. They’re human fears. They’re concerned about how other humans will use tools. The pattern isn’t “technology will destroy us” – it’s “humans will always find increasingly novel ways to cause problems, regardless of the technological landscape.” 

Addressing the Real Risks of AI Development 

Is there a risk associated with AI? Absolutely. However, the genuine risks associated with AI stem from the amplification of human intent rather than autonomous malevolence, and framing the conversation as if AI itself might independently decide to blow up a building is not just inaccurate – it’s actively harmful. It distracts us from addressing the genuine risks of how humans might deploy these technologies, from large-scale disinformation campaigns to critical infrastructure vulnerabilities from automated systems lacking safeguards. The real concern isn’t AI taking over – it’s humans using AI irresponsibly, and this effect compounds when policymakers with limited technical expertise make regulatory decisions based on public discourse dominated by these misrepresentations. With that said, what keeps me up at night isn’t AI becoming too powerful – it’s the risk that we’ll waste its potential on widespread misunderstandings of its technological structure.

[To share your insights with us, please write to psen@itechseries.com]