IT / Network Operations Horror Stories
Monday, February 4th, 2008
A lot of non-technical people throw around the term IT, assuming it encompasses all realms of technical employees in companies, including both network admins and programmers.
But usually, network admins and programmers report up the food chain through two distinct hierarchies. (this isn’t always the case, but it seems to be, generally)
Network folks report up through operations, usually. Programmers report up through the Development side of the org chart. They might eventually meet at the top where the CIO sits. A CTO usually indicates the head of a Development org structure.
This great anatomy of an IT horror story brought back a few memories from dealings with the network admin side of org charts in the past.
The two cases I’m thinking of are fairly opposite but both still had the same problem because of the way their orgs were structured. One was at a Fortune 50 company, the other had only 15 employees.
First a disclaimer: I understand people on the network admin side of things have a job to do. People do all kinds of stupid things if you let them run hog wild with a system: accidentally install spyware, get infected with viruses, etc.
But my problem with them was, as the article indicates in his case as well, that they often do not serve the interests of the business. Or, they believe they are the One True Gatekeeper preventing malicious activity from happening on the network & company computers.
Taking away developer admin rights is a sure-fire way to demoralize your dev team and grind productivity to a halt. That’s just the beginning, too.
Ego & Power
That’s really what the problem boils down to, imho: some network admins are addicted to the de facto power they have over the system.
OR
The company has a CYA (cover your ass) culture which justifies all kinds of behavior because of “What if…?” thinking that attempts to solve problems before they ever materialize. OR, one person does something they shouldn’t and suddenly some policies are put in place that ruin it for 5,000 other people.
Productivity Case Study: Startup vs. 15 Employee Co
I can go to LayeredTech.com and have a linux server provisioned and ready for me within 48-72 hours, for about $100 / mo. Or a VPS from VPSLink which can be provisioned for me within as little as 2-4 hours.
But at another place I worked (15 employees or so), the admin was a smart guy but it was neigh impossible to get a linux box setup.
I’ve got several dedicated servers now running throughout the net. I have root access and am used to being able to install whatever I like onto them.
So it’s a bit off-putting to go into a new company and be like, “so… you’re saying I can’t whip up this quick internal management app because there’s no linux box in the entire org?” (it was a bit of a windows shop)
That kind of boggled my mind. It took about 8 months before that admin moved on; the next guy was much cooler — he setup the linux box in an afternoon.
The Wrong Tool for the Right Job: Pointless?
Because the devteam didn’t have access to a dedicated linux box, the network admin just installed Mantis for us on one of his *nix boxes (we couldn’t get root) because he could do it in a few keystrokes using whatever distro it was. If you’ve used Trac and have seen Mantis (kinda like Bugzilla) you’ll know that Trac (and similar alternatives) are about 100x better. So we were stuck with this lousy bug tracking software, which was fine for a while at least.
But then one day the network admin revoked my Mantis admin rights. Which meant I couldn’t add categories or features or milestones or whatever it was that made that software tolerable and remotely usable to begin with.
It was all this huge debacle anyway so I just said screw it and stopped reporting bugs or tracking completed milestones.
The Lesson?
Maybe the solution is to empower your teams to take more of a hydra-based structure. Don’t let Command & Conquer rule the day.
Give teams a small budget (under $300 / mo. should do) to blow on whatever they see fit: servers, books, online subscriptions (Safari, etc), geeky gadgets (external HDs, nerf grenade launchers), etc.
Even if you’re not a startup, it probably behooves your organization to act more like one in some areas.
Could Yahoo Learn from this Lesson?
Just one example: there are thousands of engineers at Yahoo that could’ve built del.icio.us.
Even doing a covert Google-style “20% time” to get it built, wouldn’t have been a problem, imho. The problem (and I’m just spit-balling here as an outsider) would be getting through the bureaucracy and approval process to get something even as basic as del.icio.us launched there.
I hear Google can be pretty brutal about this too (these days), but at least they have a recognized pathway to get your 20% idea to see the light of day.
