Let the Migrations Begin!!

It’s been a few weeks since I last posted an update on our Cisco UCS implementation.  We’ve mostly been in a holding pattern until now.  Yes, we finally got the network integration component figured out.  Unfortunately, we had to dedicate some additional L2 switches to accommodate our desired end-goal.  If you look back a few posts, I covered the issues with connecting UCS to two disjointed L2 networks.  We followed the recommended workaround and it seems to be working.  It took us a bit to get here since my shop did not use VLANs, which turn out to be part of the workaround.

So now we have been in a test mode for a bit over a week with no additional problems found.  Now it’s time for real workloads.  We migrated a few development systems over Wednesday to test out our migration process.  Up until then, it was a paper exercise.  It worked, but required more time that we thought for VMtools and VM hardware version upgrades.  The real fun starts today when we migrate a few production workloads.  If all goes well, I’ll be very busy over the next 45 days as we move all our VMware and a number of bare metal installs to UCS.

Since we chose to migrate by moving one LUN at a time from the old hosts to the new hosts, and also upgrade to vSphere, our basic VM migrations process goes like this:

1. Power off guests that are to be migrated.  These guests should be on the same LUN.
2. Present the LUN to the new VM hosts and do an HBA rescan on the new hosts.
3. In Virtual Center, click on a guest to be migrated.  Click on the migrate link and select Host.    The migration should take seconds.
4. Repeat for all other guests on this LUN.
5. Unpresent the LUN from the old hosts.
6. Power up guests
7. Upgrade VM tools (now that we are on vSphere hosts) and reboot.
8. Power the guests down.
9. Upgrade VM hardware.
10. Power up the guests and let them Plug-n-Play the new hardware and reboot when needed.
11. Test

We chose to do steps 6 through 10 using no more than four guests at a time.  It’s easier to keep track of things this way and the process seems to be working so far.

We are lucky to be on ESX 3.5.  If we started out on ESX4, the LUN migration method would require extra steps due to the process of LUN removal from the old hosts.  To properly remove a LUN from ESX4, you will need to follow a number of convoluted steps as noted in this VMware KB.  With ESX3.5, you can just unpresent and do an HBA rescan.

I don’t know the technical reason for all these extra steps to remove a LUN in vSphere, but it sure seem like a step backwards from a customer perspective.  Maybe VMware will change it in the next version.