I Read Hex Traces Before Wireshark Existed. Here's Why I Don't Recommend Putting Your AI on a Dedicated GPU.
I lived through the Big Iron era, when mastering a system meant buying, racking and babysitting the hardware. AI has businesses buying dedicated GPUs again. Here is when local models make sense, and what I recommend for production instead.
By John Hynds · October 14, 2026
In the late 1980s I worked on networking in the Advanced Technology Group at Boeing Computer Services. There were no sniffers that did the work for you and no Wireshark. If two machines weren't talking, you read the raw packet trace in hexadecimal and worked it out yourself.
I taught myself to read those traces. Over time I could glance at a frame and spot trouble. One example: if the lowest bit of the first byte of the destination MAC address is set, the frame is multicast, and when the whole address is FF:FF:FF:FF:FF:FF, it's a broadcast that every device on the segment has to stop and process. We were on a Class B network with room for about 65,000 hosts. A broadcast storm on a network that size could bring everything to its knees. Yes, this was one flat network, before we'd broken it up into routed subnets.
Later in my career, working at several different hardware vendors, I became the expert on their particular products. That usually meant keeping one or two of their boxes at home so I could train and stay sharp. This was the "Big Iron" era, long before SaaS. If you wanted to run something, you bought the hardware, racked it, cooled it, and patched it, and the only way to truly master it was to live with it.
I thought those days were gone. Then AI arrived, and suddenly everyone is buying dedicated GPUs and loading models onto their own servers again.
I've Seen This Movie
Running your own model on your own GPU is a great way to learn. I'd encourage any technical person to try it. But history is pretty clear about what happens when you build production on hardware you have to babysit:
- The hardware ages fast. GPUs that are cutting edge today are mid-range within a couple of years, and the models keep growing.
- The models change every few months. By the time you've tuned and deployed one, a better one has shipped somewhere else.
- You own all of it. Security patches, uptime, backups, power, cooling, and the 2 a.m. outage are all yours.
- You depend on one person. Just like the Big Iron days, there's usually one person who really understands the box. When they leave, so does your system.
Where Local Models Do Make Sense
I'm not saying never. Running models locally is reasonable for training and experimenting, for highly sensitive or contract-restricted data (some ITAR or CUI work, for example) that truly can't leave your building, and for edge situations with no reliable connection. Those are real cases. They're just not most small and mid-sized businesses.
What I Recommend Instead
For production, use managed AI models that run inside a subscription you own and control, with clear rules about your data from the first day. You get the newest models without buying hardware, someone else handles uptime and security patches, and with access to multiple models, you can switch providers without being locked in. That's the approach behind Hynds AIOS™.
Then put governance around it: who can use it, on what data, and how you'll check the results. That's the part that actually protects your business, and it doesn't require a single GPU. (I go deeper on how we structure that in LLMs Think, Code Controls.)
Common Questions
Should a small business run AI on its own GPU?
For learning and experimenting, it's a great idea. For production, usually not. The hardware ages fast, models change every few months, and you own every patch, backup and outage, often with only one person who really understands the box.
When do local models make sense?
For training and experimenting, for highly sensitive or contract-restricted data (some ITAR or CUI work, for example) that truly can't leave your building, and for edge situations with no reliable connection.
What's the alternative to a dedicated GPU?
Managed AI models running inside a subscription you own and control, with clear data rules from day one and access to multiple models, so you get the newest models without buying hardware and can switch providers without lock-in.
Does AI governance require hardware?
No. Governance is deciding who can use AI, on what data, and how you'll check the results. That's what protects your business, and it doesn't require a single GPU.
Humans for purpose. AI for tasks.
If you're not sure where your company stands, start with our Free AI Readiness Check. It takes about 8 minutes and shows you where to focus first.
How AI-ready is your operation?
24 questions, about 8 minutes, instant score.