Datacenter tools?
Michael T. Halligan
michael at halligan.org
Thu Jun 3 11:40:48 PDT 2004
My ops director has a good-sized chunk of cash in our budget
marked aside for "Datacenter tools". I was wondering what people like
to keep on their datacenter for such things.
So far, we have :
- Plantronics Industrial noise-canceling phono dual-ear headset,
with connectors for the 3 different types of cell phones our ops
people use, and one portable phone we keep in a cabinet.
- Power convertors for the 3 different types of cell phones
- anti-static straps in each cabinet
- assorted sizes Velcro cable ties, as well as nylon zip ties
- 1 APC Environmental Monitoring Unit in each cabinet
- 1 Laptop in each cabinet with serial connections and adapters for
3 different types of serial connections
- Ethereal & Minicom installed, as well as a web browser
- Patch panel for laptop with 3 network connections (one for each
subnet)
- Assorted tools (punchdown, cable cutters, screwdrivers, wrenches (for
adjusting cabinets), several paper-clips (APCs have very tiny reset
buttons if they trip)
- Roll of quarters for vending machine
- One cold-spare for every classification of server, including load
balancers, switches, firewalls
- 4U chatsworth locking drawers in every cabinet to keep spare parts and
- Cold spares for anything that we could imagine will break (CPU,
memory, different sizes of hot-swap drives, raid cards, PDUs (very
helpful, just had a PDU fail on us yesterday), GBICs, we've even got a spare
1U USR tray of fans in case one of ours fails.
- Paper & pens
- 2 Label makers, with extra tape
- Post-its
- Multimeter
- Extra cables of all types (serial, scsi, cat5, cat5 crossover)
- Power tester
- Ethernet link checker
- Floor panel lifters
- Foldup nylon chair
- 2 Keyboards & Mice (less important since our datacenter provides
rolling carts)
It sounds a bit cluttered, but we have 2 cabinets dedicated for testing
environment and management tools, and production in other cabinets,
eventually we'll have 2 cabinets solely for testing, and one for
management).
We've tried to make everything as remotely manageable a possible.
Everything sits on a remote serial console, as well as a remote
reboot switch. Our datacenter has a staffed 24x7 NOC that will change
tapes, sit down at a terminal to read errors and type commands, as well
as reboot boxes.
We've also made everything redundant as possible, having everything
that's not using our clustering utilities have a hot-spare (vrrp for
network gear, LVS for servers). We've also setup automatic rebuild
scripts for all of our servers, so we can kickstart or systemimage
through a serial console.
My goal is to eliminate trips to the dc as much as possible, but making
sure we have every tool we need in case of any unforseen failure.
Some of the proposed new tools have been :
- 1 AXIS network camera per cabinet (the kind that pan, so we can
potentially look at blinkys, as well as who's accessing)
- Fluke network debuggers
Does anybody else have any other good ideas for tools to keep in the datacenter?
More information about the Baylisa
mailing list