Enabling users with ESX 3.5.0

I've been playing admin on our shiny new ESX server and it's been a struggle trying to get the user accounts configured.  I wanted to give our QA staff enough rights to login through the VMware Web Access portal so that they could start and stop their virtual machines.  I figured I would just set up their accounts and put them in the "Virtual Machine User" role.  No of the accounts could login in.  They kept getting the dreaded "Login failed due to a bad username or password."

I dug around a bit and started searching the VMware forums.  Apparently everyone was using AD to authenticate their user accounts.  I wasn't planning using AD because we are going to be changing domains in the near future, but I figured nothing else was working, it couldn't hurt.  This message led me to a very useful post by Geert Baeke on how to integrate Active Directory with ESX 3.   There was a lot of useful stuff in that post, but the part that I needed came down to this:

esxcfg-auth --enablead --addomain=domain.com --addc=domain.com

The VMware document uses the FQDN of a domain controller for the --addc parameter, but you can use the FQDN of the domain. That way, DNS is used to find domain controllers and use one of those. The command above modifies a few files like /etc/krb5.conf and also the system-auth file in /etc/pam.d. The ESX firewall is also automatically configured to open the needed ports for AD authentication.

Before you can logon with an AD account, you need to create a console user on the ESX box that has the same name as your AD account. For example, if you have an AD account domain\esxadmin, you need to add a user to the ESX console called esxadmin. The command to use is useradd esxadmin. You can also use VI Client to create the user. You can now logon with the account and use the AD password. I tested this with ESX 3.0.1 servers against Windows 2000 and Windows 2003 domains and it worked as advertised.

I did it and it worked like a charm.  Life is good.

Fun with changing the IP address of ESX Server 3.5

We (our IT manager and myself) have been having some fun with out shiny new VMWare ESX 3.5 server.  We have had it running for about two weeks now and we decided to change it's IP address.  The ESX server was on the same subnet as our LAN.  This meant that it the virtual machines were taking IP addresses out of a pool that was needed for our physical computers.  There were some other security issues, so we decided to put it on it's own subnet.

It was fairly easy to change the IP address address via the command line (this site helped a lot), the fun started with the NFS connection.  We are using NFS to mount a folder located on a Windows file server to add some offline storage for the ESX box.  To mount with NFS, you have to create a VMKernel in the ESX networking and the VMKernel gets it's own IP address.  That IP address must be on the same subnet as the NFS server.

When we moved the ESX to it's own subnet, we put it on it's own physical network and that broke the NFS connection.  We tried a few things and then we checked the Windows box that was running the NFS server.  It had two network cards.  The second one was not enabled, but fully functional.  We enabled it and set it's IP address to the subnet of the ESX box.  I had to drop and recreate the NFS mount, but it all worked.

While testing the networking, the IT manager was running one of the virtual machines (XP 64-bit) and set the network adapter in the VM to a static IP address.   It turned out he set it to the IP address of the ESX server.  That's when the fun started.  When you connected to the VM, ESX would lose it's connection and you lost control over ESX and the VM.  After a minute or two, you could access ESX through the VMWare Infrastructure Client, but you couldn't access the VM to change it's IP address.

We racked our brains trying to figure out how to get control of the VM to reset it's IP address.  The fix turned out to be really simple.  I powered down the VM from VIC and edited it's hardware settings.  I added a second network adapter (it's all virtual) and set the first one to be disconnected.  I powered the VM back up and the new adapter had a safe IP address.  I connected to the VM's console and opened up "Network Connections" in Windows.  The first adapter was enabled, but not connected.  I opened up it's properties and set it to grab an IP through DHCP.

I powered down the VM, removed the second adapter, and reconnected the first one.  I rebooted the VM and it had a new IP address.  Peace and harmony reigned through my virtual kingdom.