Archive for May, 2010

VMworld voting has begun

The folks who run the VMworld tradeshow have taken a new path to deciding session content this year.  Instead of a panel making all the choices, the powers-that-be have decided to open it up to the public to vote on those topics that interest them.  I’ve submitted a proposal to speak about our soon-to-be UCS implementation and how it is going to transform our business.  If interested, please go here: and vote for my session.  My session is entitled:

Session ID: TA7081, Session Title: Case Study: vSphere on Cisco UCS – How the City of Mesa changed strategic direction



Prepping for our Cisco UCS Implementation

The purchase order has finally been sent in.  This means our implementation is really going to happen.  We’ve been told there is a three week lead time to get the product, but Cisco is looking to reduce it to two weeks.  A lot has to happen before the first package arrives.  Two logistical items of note are:

  • Stockroom prep
  • Datacenter prep

What do I mean by “Stockroom prep?”  A lot actually.  While not a large UCS implementation by many standards, we are purchasing a fair amount of equipment.  We’ve contacted Cisco for various pieces of logistical information such as box dimensions and the  number of boxes we can expect to receive.   Once it gets here, we have to store it.

Our stockroom is maybe 30×40 and houses all our non-deployed IT equipment.  It also houses all our physical layer products (think cabling) too.    A quick look at the area dedicated to servers reveals parts for servers going back almost ten years.  Yes, I have running servers that are nearly ten years old <sigh>.    Throw in generic equipment such as KVM, rackmount monitors, rackmount keyboards, etc and it adds up.   Our plan is to review our existing inventory of deployed equipment and their service histories.  We’ll then bump up that info with our stockroom inventory to see what can be sent to disposal.   Since we don’t have a lot of room, we’ll be really cutting down to the bone which introduces an element of risk.  If we plan correctly, we’ll have a minimum number of parts in our stockroom to get us through our migration.  If we are wrong and something fails, I guess we’ll be buying some really old parts off eBay…

As for prepping the data-center, it’s a bit less labor but a lot more complex.  Our data-center PDUs are almost full so we’ll be doing some re-wiring.  As a side note, the rack PDU recommended by our Cisco SE has an interesting connector to say the least.  These puppies run about $250 each.  The PDUs run over $1200 each.   Since we’ll be running two 42U racks of equipment, that equals four of each component.  That’s almost $6K in power equipment!!

As another data-center prep task, we will need to do some server shuffling.  Servers in rack A will need to move to a different rack.  No biggie, but it takes some effort to pre-cable, schedule the downtime, and then execute the move.

All-in-all, a fair amount of work to do in a short time-frame.

A Trend in Technology Provider Marketing Techniques

The recession has brought about a few major changes in sales/marketing techniques in the technology industry.  There was a time when only executive management was wined and dined and the common man was left out in the cold.  Well my friends, that time is no more.

Over the last 18 months or so, I have been invited to more lunches and activity-based events that I have in my 20+ years in the IT industry.  The two (lunches and activity based events) can be broken down into two categories of providers: those selling really expensive products and those with not-so expensive products.

Those in the really expensive product category usually are storage providers.  Since these systems can easily reach into the hundreds of thousands of dollars, the sales/marketing experience has to be equally impressive.  As such, the event most often chosen is lunch at an upscale steak restaurant such as Ruth’s Chris or Fleming’s.    The typical event consists of a small presentation (usually under 30 minutes) followed by a lunch from a scaled-down menu.  Even though the menu is scaled down, the quality of the food is not;  the reputation of the restaurant is still on display.

In the not-so expensive category, we typically find VARs and small product vendors.  The event of choice in this category is entrance to something with mass appeal such as a blockbuster movie’s opening day.   As with the lunches, the event begins with a 30 minutes presentation and then the movie begins.   This type of event has become so pervasive that I recently had three invitations to see Iron Man 2 at the same theater on the same day (all at different times).

I don’t go to the lunches very often because I feel it is disingenuous to take advantage of something so expensive for no return.   I only attend when I have a budgeted project.  I’m also careful to keep track of the “promoter”.  Some promoters are very good at setting up the presentations so that real information is imparted.  Others are there just to get butts in the seats and the presentations tend to suffer for it.  While I enjoy a good meal, I don’t want to waste my time.  However, I do partake in some of the movies since they usually take place on a Friday (my day off) and I use them to network with the VAR and other IT professionals.

Other events in the expensive category:

  • Tickets to major golf tournaments
  • Tickets to basketball games
  • Tickets to concerts

Other events in the not-so-expensive category:

  • Tickets to baseball games (many can be bought in volume for under $10 each)
  • Kart racing (fast go-karts)
  • Lunch and games at a large entertainment venue such as Dave & Busters

What else have you seen?   Anything outrageous?


Time to order and Implement Cisco UCS

Monday was a good day for us.  City Council voted to move forward with our proposed UCS purchase.  Now the fun begins.  In no particular order:

Get all parts into our service catalog, generate the order, prep facilities, attend training, schedule services, schedule server migrations, soothe ruffled feathers, order forgotten items such as PDUs, celebrate, receive equipment, install equipment, test equipment, and many more.

Implementation couldn’t occur at more inopportune time since fiscal year is coming to a close.  Talk about lots of disruption, but I think we can manage it.  I know we will make mistakes.  I know we will have successes.  It’s all part of a day in the life….

I plan on writing about our implementation so look forward to some interesting reading, hopefully, starting in June.

Categories: cisco, Hardware refresh, UCS Tags: ,

VMDK UUID Recovery

We had an incident a few weeks ago in which a technician restored a virtual server using vRanger Pro to a different name and data-store for the purpose of copying the restored server to our test lab.  No big deal, but one bad thing happened.  Unkbeknownst to the technician, the vmdk header files (the ones that point to the _flat.vmdk files) and .vmx file were still pointing to the production system.   So when he went to clean up and delete the restored virtual server, Virtual Center dutifully removed the .vmx and .vmdk header files from the REAL server.  It wasn’t able to delete the _flat.vmdk files (thankfully) because they were in use.  Also turns out we had a few disks marked Independent, Persistent so they weren’t being backed-up.

We recovered what we could but still had to manually recreate the vmdk header files.  Since I had no reference file, I took an existing one from a working virtual server, changed all the various pointers, and powered the guest up.  I got an error in regards to the vmdk UUID.  Whenever I have a disk problem, one of my “go to” tools is the VMware provided vmkfstools utility.  A quick man-page review and I found that it can create UUIDs.  Running “vmkfstools -J setuuid filename.vmdk” solved the problem.  The system now boots fine, but since I didn’t know all the details to put in the file, the drive shows up with incorrect information in Virtual Center and backups are still a problem.

A soon to be implemented solution is to create another drive with the correct parameters (via Virtual Center) and then copy everything from the bad drive into the good drive, do some drive letter changing, and Oila.  Another solution is a process change on our part.  No more deletions of this sort via Virtual Center.  Just going to remove the server from inventory and manually delete the files via the service console.

UPDATE: So what do I find in my VMware support RSS feed this morning?  An article detailing how to create a new vmdk header file to replacing a missing header file.   (Link VM KB) I like VMware’s solution better because it not only generates a new UUID (in the header file), it also adds the correct disk geometries.