First off, let me say I don’t usually refer to myself in the Third Person. That said, looking into a list of qualifications for a Computer Guy such as myself can easily drift off into what sounds like Greek mythology. The point of this is to highlight some of my projects/tasks in a *relatively* easy to read format.
For quick restoration, I developed and maintain a backup / restore procedure for multiple web servers. Full virtual machine Backups using Trilead VM Explorer, partial backups are also run with a script backing up key folders and MySQL databases which are then backed up to a backup server running open source BackupPC.
Web analytic are key to search engine optimization and building a website. Using tools such as Google Analytics and open source AWStats which read the actual server log files, we are able to analyze traffic and bandwidth.
Windows System updates and Security Patches
Originally, I would run Windows Updates when I noticed that little update shield by the clock. While this might work for 50-80 servers, this method is not very scalable. Thanks to setting up a Windows Update Server, it’s much easier to release updates and see which machines have patches installed, failed or needed. This is not only an incredible time saver but also great for maintaining overall security across multiple machines.
Infrastructure / System / Server Monitoring
Monitoring systems is very important to maintain a 99.9% up time. Various monitoring tools and processes allow this. Programs such as SpiceWorks, a custom website monitor and PSToolsi n addition to others allow for excellent infrastructure stability.
Built a new 2008 domain server and secondary server and thoroughly tested prior to migrating about 120 web servers from a Windows 2000 Domain with little impact to our e commerce customers other than the reboot.
Suggested this by a boss at the time, I leaped at the chance to virtualize our E commerce data center using VMWare’s ESXi 3.5. While it was a new technology to me, there is really no reason to be scared of change, especially something so promising as server virtualization. Physical to virtual (P2V) migrations drastically cut rack space and power usage while improving performance in every benchmark. For example, while a 1u server might use around 100watts, our newer 2u servers which can hold over 30 virtual machines, pulls under 300 watts total.
Storage Area Network
You can’t virtualize your server infrastructure without storage. The earliest setup from 2008 uses NFS on a 2003 server and is still used today. Given the budget constraints, I then re purposed an older server as an OpenFiler ISCSI SAN server. This is still working great, partially thanks to the multiple paths allowed with an ISCSI connection. While still limited to 1Gb cables, the multiple paths improve overall throughput over a fairly standard 1Gb network. Furthermore, even if a cable is pulled, the multiple paths keep everything up without so much as a hiccup. More recent servers to the Storage network include 2008 data center edition, RAID 10, ISCSI connections, and of course multiple paths.
You can’t run a data center without remote access. While Windows Remote desktop is OK, it is not as good when multiple people want to connect to the same machines. PCAnywhere was the solution prior to my starting, but when I saw it would be $100 per machine for a program that was just OK, I looked elsewhere. It didn’t take long to see UVNC was the way to go. 6 years later, UVNCis still the remote access of choice and I’m sure the $20,000 a year in licensing fees the company continues to save is nice too.
Central Managed Antivirus
Running a separate antivirus across hundreds of servers doesn’t make sense. With centralized antivirus such as a Trend Micro Server, Antivirus protection is significantly better while being much less time consuming and more reliable.
Installing Windows takes a long time. By keeping a current image using tools such as Acronis, VMWare Converter and Sysprep, its much quicker and takes most of the hands on portions out allowing time to work on other, possibly more meaningful projects.
Through IIS (internet information systems) optimization, noticeable improvements in site performance can be achieved. As one coworker put it “You turbocharged their site!” Even changes in browser caching alone has the affect of dropping page load times from a couple seconds to under 1. While this might seem like much, any edge your store can achieve over another will only help.
Not every server is the same, nor is every customer or site design. As such, MySQL optimization needs to be performed to keep everything running as smooth as possible. Multiple variables are modified including, but not limited to; query cache, table cache, table size, key buffer size, key_read_requests/key_reads and more. Modifying these settings in addition hardware configuration such as RAID level, number of disks, RAM, CPU, etc all go into making a solid, snappy and reliable MySQL server.
Network / Server / Website / Data Center Security
Every Network Administrator needs to stay on their toes. When managing 200 company web servers, there are exponentially more possible threats. Firewalls, PCI Scans, general network scans, security patches, and keeping up with all the ‘script kiddies’ is very important in keeping hundreds of thousands of customer records safe.
Moving to a new data center, we found ourselves in need of a reliable email server. A long term distribution of Ubuntu Linux combined with open source PostFix met the needs quite nicely.
Often referred to as the Internets phone book, the Domain Name System can be rather confusing. Whether is local DNS such as Active Directory or an internet name server, I am well versed in the different records such as A, MX, CNAME, and SPF.
Eliminate Repetitive Tasks through Automation
Managing hundreds of servers effectively can seem like a daunting task. Automation is vital in keeping a data center up to date with software patches and installs. For example, using a combination of software installing batch scripts and PSTools to push it out updates, what used to take hours or days can now be done in a matter of minutes. Another example is with AWStats. By modifying the configuration files and using a powerful editor such as Notepad++, I have cut down the install time from about 2 hours to nearly 15 minutes.
Home Theater PC Complete with DVR
Unsatisfied with offerings from the local cable company, I set out to build my own DVR. Mission accomplished thanks to Beyond TV by Snapstream! With nearly 200 hours of HD recording, anywhere access to our shows, automatic guide updates, and how easy it is to use, this ranks very well with my lovely wife.