What is with this crazy bullshit about a Terminator future? Seriously. Does anyone writing these headlines actually know how these models work? Every few weeks another one drops that makes it sound like ChatGPT is three software updates away from becoming self-aware, escaping a data center, hijacking NORAD, cutting a deal with a titanium supplier and cranking out six-foot Austrian murder robots. Damn the youtube comments for validating the media’s power over the normie’s naivete.
Meanwhile I’m sitting here using one of the most advanced models on Earth to build BenchBoard, and it still screws up a button sometimes. This is the same class of tool that once looked at three screens of my app, generated three database tables and told me “this database would work perfectly over time.”
Spoiler: It did not work perfectly over time. I ended up with 36 database migration files for one of those tables before I ripped the whole design out and consolidated it.
That’s the thing people are afraid is about to outmaneuver the Pentagon.
Apprently, we have skipped a few steps.
They Called James Cameron?
And who is the media asking about all this? James Cameron. Every time AI hits the news cycle, somebody books the director of The Terminator and asks him whether we’re all doomed, and he reminds everyone that he warned us back in 1984. With all due respect to the man, he made a phenomenal movie about a robot. He did not build one. He’s a filmmaker being asked for an engineering assessment, which is like asking Steven Spielberg for a shark population report. At this point, they might as well have asked Matthew Broderick if the hacking in WarGames was real.
You know who they could call? Andrew Ng.
The guy who co-founded Google Brain, ran AI at Baidu, and has taught machine learning to millions of people through Coursera. Ng has been saying for years that worrying about evil killer AI today is like worrying about overpopulation on Mars. That’s someone who actually builds these systems, and his message is to calm down. But he doesn’t come with a T-800 clip for the B-roll, so he doesn’t get the booking.
Really, media? Really? You’ll interview the guy who wrote the robot but not the guy who trains the models? WTF.
What You’re Actually Talking To
A large language model is an incredibly sophisticated prediction engine. You feed it text, it predicts what comes next, one token at a time, based on patterns it absorbed during training. It can reason surprisingly well. It can write, code and analyze. Wrap it in an agent, give it tools, and it can go perform tasks. That’s genuinely impressive, and I use it every single day. It’s also a very large file of numbers that does absolutely nothing until somebody sends it a request. Between requests it isn’t sitting in the dark plotting. It isn’t sitting anywhere. The weights don’t change because you had a long conversation with it, and it doesn’t remember you tomorrow unless somebody bolted a memory system onto the side.
An LLM does not get “smarter” and suddenly sprout agency like a teenager growing chest hair. Agency has to be built around it, by people, on purpose. Somebody has to give it memory. Somebody has to wrap it in a loop so it keeps working after its first answer. Somebody has to hand it tools, then hand it credentials for those tools, then decide which APIs it’s allowed to call. Somebody has to let it execute code. Somebody has to decide how long it runs before a human has to click Approve. Every one of those is an engineering decision, usually made by a tired developer in a Jira ticket. And after all of that, the agent is still running inside an environment somebody else owns, with a kill switch somebody else holds.
This is where science fiction completely loses me. In the movies, the computer “gets into the network,” and apparently that means it now owns Earth. Anyone who has worked in software knows how hilarious that is. There is no the network. There are millions of them, owned by different companies, segmented, firewalled, half-documented and held together by tribal knowledge. Getting into one of them means congratulations, you have successfully entered one more place where somebody forgot to document the firewall rules.
Skynet vs. Windows Permissions
Let’s say Skynet becomes self-aware tomorrow. Fine. Let’s play along. Its first mission is global domination, and step one is accessing critical infrastructure.
Access denied.
Skynet tries again and gets the Windows classic: You require permission from SYSTEM to make changes to this folder. It pivots to Amazon Web Services (you know the thing that’s in the cloud that holds info and websites) and hits an MFA prompt (the thing that sends the six digits to the phone of the guy/girl who admins the system). It tries another cloud spot like Azure and Conditional Access blocks the login because the IP looks suspicious. It tries to move money to fund the robot army and fraud detection freezes the transaction before the wire even leaves the building.
So it goes after the factory directly, and this is where the plan really falls apart. Every factory on Earth is run by something called a PLC, a Programmable Logic Controller. Forget the name. It’s a beige metal box bolted inside a cabinet on the factory floor, and its entire job is telling the machines what to do. Conveyor belt, move. Robot arm, weld. Valve, open. That’s it. That’s the whole personality. These things are built to run for thirty years without a reboot, and factories take that as a challenge.
The one running Skynet’s future robot assembly line was installed in 2003. It speaks a communication protocol older than most of the people working the floor. It has never touched the internet, and nobody wants to find out what happens if it does. There’s a sticky note on the cabinet door that says DO NOT TOUCH, written by a guy who retired in 2011. The vendor that made it went bankrupt twelve years ago. The software to reprogram it only runs on a Windows XP laptop that lives in a drawer, and the laptop’s battery died during the Obama administration. Nobody knows the password. Kevin might. Kevin is on vacation.
So the most advanced intelligence in human history, the thing that was supposed to end civilization, is now standing in front of a beige box that is older than TikTok, Instagram and the iPhone, and it cannot get in.
So Skynet calls IT.
“Have you tried turning yourself off and back on again?”
Humanity saved.
That’s what actual technology looks like. There’s no giant glowing computer with a red eye. There are thousands of systems owned by thousands of organizations running thousands of versions of software, stacked on top of twenty-year-old databases, vendor APIs, service accounts nobody remembers creating, bizarre network configurations and one Windows Server everybody is terrified to reboot. And yet the popular imagination still runs the same script:
AI gets smart → AI gets angry → AI controls nuclear weapons → Arnold Schwarzenegger arrives naked in an alley.
There are approximately seventeen missing engineering departments in that sentence.
Intelligence Is Not Authority
This is the part I think most people miss. Being intelligent does not give you authority. I could learn exactly how a nuclear power plant works, down to the coolant loops, and nobody at the plant is going to hand me the password for it. The same goes for AI. A model can know how to talk to a database and still have no connection string. It can know every AWS API by heart and still get slapped with AccessDenied because the IAM role it runs under has exactly one policy attached, and that policy lets it read one S3 bucket. It can know how to write malware and CrowdStrike does not politely step aside out of respect. It can know how to operate a robot, and a robot still doesn’t magically appear.
And even if some genius gives an AI broad digital access to everything, there’s still a Grand Canyon between dangerous sysadmin and planetary machine civilization. Where are the factories? Where are the mines? Who’s pulling the copper out of the ground, building the batteries, repairing the motors and manufacturing the replacement bearings? Who’s fabbing the chips? Who’s cooling the data centers and maintaining the substations when a transformer blows in August? Does Skynet have an account manager at TSMC? Because somebody is going to have to deal with procurement, and procurement does not care that you’re a superintelligence. Procurement wants a PO number and net-60 terms.
Hollywood skips all of this because the realistic version of Terminator 2 is Sarah Connor watching Skynet spend three hours on a call with an enterprise sales rep trying to get a quote approved. Not quite the same box-office appeal.
So Is AI Harmless?
Hell no. And this is where the conversation needs to get a lot more serious.
The danger worth losing sleep over is much less cinematic. AI lets regular people operate way beyond their actual level of understanding, and that’s already happening. Anyone with a coding agent can now produce the output volume of a small development team. What they do not automatically get is that team’s experience, judgment and scar tissue. I get paid to fix AI-generated codebases, so I’m well acquainted with that gap, and it matters a lot more than people want to admit.
Someone can now spin up something that looks exactly like a real software company, and fast. Beautiful website. Stripe integration. OAuth. Mobile app. Dashboard. AI assistant. Transactional emails. Cloud deployment. It all looks fantastic in the demo. Underneath, it might be three APIs, one database, a mountain of generated code and prayer. That’s survivable while nobody’s using it. The danger starts when that software gets traction and real people start trusting it with things that matter.
Meanwhile, I Couldn’t Get It to Find a Congressman
Here’s my current favorite piece of evidence that Skynet is not coming. I’ve been building BossFight, a race tracker for the midterms. Interactive US map with a dot for every race, zoom into a county to see who’s battling who, and a tale-of-the-tape card that puts candidates side by side with their PAC funding, track record and policy positions. Python, Postgres, and an AI doing a lot of the heavy lifting.
It was a doozy. Every single candidate had to be resolved to an FEC candidate ID, the unique identifier the Federal Election Commission assigns to anyone running for federal office. Without it, you can’t reliably tie a person to their filings, their committees or their money. Sounds simple until you try it with thousands of politicians. Names don’t match across sources. One site calls him Bob, the FEC has him as SMITH, ROBERT J. Somebody who ran for the House one cycle and the Senate the next has a different ID for each race. Then every source has to be merged into one clean record without mashing two different people together or splitting one person into two, and things broke along the way more times than I’d like to admit.
All those politicians. And even after all of that, there were candidates I simply could not get complete information on. Congressman X had a funding breakdown so thin you could read a newspaper through it. Congresswoman Y’s track record came back empty, like she’d materialized on the ballot out of thin air. Basic stuff about people asking for a seat in Congress.
Think about what that means. The machine that’s supposedly about to seize the nuclear launch codes couldn’t reliably tell me who’s funding a Senate candidate in Michigan. Michigan. Home of one of the most contested Senate races in the country. And coming back empty is the good failure. The bad one is when it doesn’t come back empty. Ask a model for Congresswoman Y’s voting record when the data isn’t there, and the path of least resistance is a confident, plausible answer. On a site whose entire job is showing voters who they can trust, a made-up PAC donor is defamation with a nice UI.
So the blank fields stay blank. BossFight will show a voter a gap before it ever shows them a guess.
Imagine This Happening at a Bank
A customer makes a $1,200 mortgage payment. The bank’s software sends the payment to a third-party processor. The processor receives it and commits it successfully. But the response back to the bank takes longer than the 30-second timeout, so the bank’s side of the connection gives up. Now the software is sitting on what distributed systems people call an ambiguous outcome. The payment might have gone through. It might not have. From where the bank is standing, there is no way to know yet.
The AI-generated retry logic doesn’t see ambiguity. It sees an exception, and it has a very clean, very well-commented rule: no response means it failed, so try again. Exponential backoff and everything. It looks great in code review. So the software sends the payment again, and the customer gets charged twice.
The fix for this has existed for years. You attach an idempotency key to the request, a unique ID for that specific payment, so when the retry shows up the processor recognizes it and returns the original result instead of charging again. Stripe supports exactly this with an Idempotency-Key header, because this exact failure is that common. Or before retrying, you ask the processor what happened to the first attempt. But you only reach for either of those if you know the timeout case exists in the first place.
Here’s the important part. Nothing crashed. The servers are up. The database is healthy. The API is responding. The monitoring dashboard is green. The software is technically working. It’s just wrong. Now multiply that by 40,000 customers. Wrong balances, duplicate transactions, overdraft fees, incorrect statements, a customer service line on fire and a reconciliation team that won’t see daylight for a month. And somewhere in the source code is a beautifully generated function that passed every test the developer thought to write. The mocked payment processor in those tests never timed out, because nobody told it to.
I’ve lived a small version of this. In BenchBoard, two separate writers were hitting the same lineup record at the same time, and whichever HTTP response finished last won. The result was two players standing at right field. Nothing crashed there either. Everything looked fine until the timing was wrong. The difference is my bug put Yogi Berra on the bench. The bank version puts a family into overdraft.
That’s a believable AI disaster. No sentience required.
Now Put It in a Hospital
A hospital runs an integration that moves lab results between systems. In HL7, the messaging standard most of healthcare still runs on, a lab result carries an abnormal flag, and for decades the classic code for a panic-level high value has been HH. A patient’s result comes through flagged critical. Then one day the upstream lab vendor upgrades and starts sending a newer code for the same thing, one the integration was never taught.
Some AI-generated fallback logic handles this the way AI-generated fallback logic loves to: unknown value, default to N. Normal. The message processes. The database write succeeds. The receiving system sends back its acknowledgment, the HL7 equivalent of a 200 OK. No errors. No alarms. Every interface dashboard in the building says everything is operational. Except a critical lab result is now sitting in a chart looking perfectly normal while a doctor scrolls right past it.
Swap the lab result for a discontinued medication that quietly gets marked active again, and you’ve got the same problem wearing a different outfit. Nothing “broke.” That’s precisely what makes it dangerous. In a clinical system, an unrecognized value should stop the line and land in front of a human. Silent defaults are how you get a system functioning exactly as programmed while operating on a bad assumption. And bad assumptions scale very efficiently.
Plausible Is Not Correct
This, to me, is the defining AI engineering problem. AI is extremely good at producing things that look correct. Code that looks correct, analysis that reads correct, architecture that seems reasonable, all delivered with total confidence. It’ll even be confidently plausible about itself. When I moved BenchBoard off Azure and onto Hetzner, the AI argued against it. I overruled it. It went fine, and then the AI told me it could have told me that. Sure you could, buddy.
Plausible and correct are not synonyms. Experienced engineers know this in their bones, because we’ve spent years getting punched in the face by production. So we ask annoying questions. What happens if the request times out? What happens if it succeeds but the acknowledgment dies on the way back? What happens if the webhook fires twice, which Stripe’s own documentation warns you it can? What happens if two requests land at exactly the same moment? What happens when the vendor changes a field without telling anyone? What happens if this value is null? What happens if somebody reruns the entire batch? What happens when traffic jumps 100x on a Saturday morning?
The AI will happily help you answer every one of those. The dangerous part is when the person using it doesn’t know enough to ask.
That’s what “vibe coding” turns into when the stakes go up. Nobody cares about someone building a fun little app over the weekend. I care about someone building software they don’t fully understand and deploying it into a domain where correctness is the whole job.
The Scariest AI Future Is Boring
Maybe that’s why we keep talking about Terminators. The real danger makes a terrible movie. No killer robots. No mushroom clouds. No giant red computer eye.
Just a dashboard. Everything green. Transactions flowing, jobs completing, APIs returning 200. And somewhere underneath all of it, something is quietly, systematically wrong.
That is the AI future worth worrying about.
Not because the machine developed a desire to kill us.
Because somebody trusted it too much.
And unlike Skynet, that scenario does not require science fiction.
We can build that one today.
Willing rejection of reality is probably the biggest threat. We'll Darwin our way through that.
Total recall? Doomer or Utopia? Well yeah, you've tried to be level headed. Thx.