Dave's Users flip the switch!

I have worked for companies making wholesale lockbox systems since 1992. Basically we build and integrate systems which process wholesale check payments between companies. Last year our system processed around $1.8 trillion in transactions, almost all of which ran through FreeBSD. But I digress.

Years ago, before I was an employee, a company I worked for built a system for a bank in Dallas using a Data General machine. This machine kept resetting itself randomly and the problem couldn't easily be identified. Over the course of around half a year, everything was analyzed, until finally our company had 3-4 technicians on site using scopes and meters on every part of this machine which they thought may be problematic, but the problem persisted.

Finally, partly out of frustration or perhaps desperation, they built a physical cage around this machine to prevent any access, intentional or incidental.

Soon after they were called in because the machine was down. It hadn't reset, it was just powered off. A bit of investigation revealed that someone had been flipping the 220v AC power off and then on to reset this machine, causing a long reboot process, which allowed employees near the end of a shift to leave a few hours early. Since access was denied by the cage, the person had flipped the 440v wall panel switch instead, not realizing that it required a key to flip it back on.

A quick check of their records revealed that the problem hadn't happened when a certain floor supervisor was on vacation, so they narrowed the problem down to him, and I believe he was let go.

Just one of many head shakers I've heard from folks who were there.