VMware’s Issues and Resolution

Here!! I am trying to explain about identifying and resolving various issues that affect virtual infra performance running on VMware. Make sure about few things before start any troubleshooting in your infra such as check your latest backup for of DataBase, VMs, configuration file, and try to know about Business SLA of your current incident, so that you can set your priority and be calm during your troubleshooting activity. There are few steps as given below that i am prefer to follow during my action:–
1. Try to get VM/Apps configuration doc, if it is created during provisioning and go through it.
2. Make sure about Business impact during outage of this VM/App and nature of application that help to understand other dependency.
3. As per SLA or SAV incident nature, don’t forget to inform your team as well ask about previous incident history.
4. Now ask to Apps/VM owner if they allow to restore VM, Apps or DB from backup and also update them about RTO and RPO means data loss.
5. Check all real-time event logs for VMs/Apps at least for the day (24 hours), see URL for more info: http://pubs.vmware.com/vsphere-50/index.jsp?topic=%2Fcom.vmware.wssdk.pg.doc_50%2FPG_ChA_Diagnostics.19.3.html
6. VMware Support contact info to log a case –  001-877-486-9273 & for existing Service Request: 001-877-486-9273,choose Option 4 for Support, Option 2.
7. Before log a case with VMWare you need to sign-up on VMWare portal and check your access permission with your Manager or Concern dept.
All above mentioned practice is very generic, so it’s depend person to person as per their expertise,experience and knowledge of VMWare if you feel comfortable then do your way :).


More about Virtual Machine files : We should know about the components of virtual machines. There are multiple VMware file types that are associated with VM and make up a virtual machine. These files info are given below in the VM’s directory on a datastore.

File                        |        Example filename          |    Description
.vmx                       .vmx                          Configuration file
.vmfx                     .vmfx                         Additional configuration file
.vmtx                     .vmtx                         Template file
.nvram                   .nvram                     BIOS/EFI configuration
.vswp                     .vswp                        Swap files
.log                           vmware.log                                   Current log file
vmware-##            .log                                                Old log file entries
.vmdk                     .vmdk                        Virtual disk descriptor
-flat.vmdk              -flat.vmdk                Data disk
-rdm.vmdk            -rdm.vmdk               Raw device map file
-delta.vmdk           -delta.vmdk             Snapshot disk
.vmsd                     .vmsd                         Snapshot description data
.vmsn                      .vmsn                        Snapshot state
.vmss                     .vmss                          Suspend file

Note: It’s Depend on the VM state and configuration so probably you  not able to get all files may present in the virtual machine directory.


Day To Day used VMWare CLi 5.x:

vSphere vCLI is a separate set of CLI tools that, like vSphere PowerCLI, can be used to perform remote management of ESX and ESXi hosts. Whereas vSphere PowerCLI is very Windows-centric, vSphere vCLI has a more Linux-like “look and feel” and targets non-Windows users. VMware provides vCLI packages for installation on both Windows and Linux systems. vCLI is also packaged as part of the VMware vSphere® 5.0 Management Assistant (vMA), a Linux-based virtual appliance that packages the vCLI together with other tools to facilitate
deployment and use of the vCLI.
Whereas vSphere PowerCLI is built on top of Windows PowerShell and is composed of a collection of PowerShell cmdlets, vSphere vCLI is a combination of the following separate command-line tools:
• “esxcli”
• vmkfstools”
• “vmware-cmd”
• “resxtop”
• “vicfg-*”
1. Reading VMware ESXi logs
ESXi 5.x multiple log files have been added. The following log files are  most commonly used for troubleshooting purposes:
PATH             |        LOG FILE DESCRIPTION
/var/log/vmkernel.log            : All log entries are generated by the VMkernel.
/var/log/vmkwarning.log       : A subset of the VMkernel logs that include only warnings and sysalerts events.
/var/log/hostd.log               : Host management service (hostd = host daemon) log.
/var/log/sysboot.log         : System boot log.
/var/log/fdm.log         : VMware HA log file.
Example – 01:  To view Logs
# cat /var/log/vmkernel.log | less
# cat /var/log/vmkernel.log

Example – 02: To see VM process and reset ths Hung VM.
# “esxcli vm process list”
• “esxcli vm process kill –world-id= –type=<soft,hard,force>”
The first command provides a list of all the virtual machines currently registered on the host. The second command enables you to power off a virtual machine.
Example-03: Find all VMs currently registered to ESX host.
# vim-cmd vmsvc/getallvms
Example-04: To know VMWare ESXi version.
#vmware -v

Example-05: Check the power state of a VM.
# vim-cmd “vmsvc/power.getstate”
Example-06: Power on a VM
So having determined a VM is in a powered off state, you can power it back on with:
# vim-cmd “vmsvc/power.on”
Shutdown the virtual machine using the VMID found in Step 2 and run:
#vim-cmd vmsvc/power.shutdown VMID
Note: If the virtual machine fails to shut down, use this command:
#vim-cmd vmsvc/power.off VMID
Example – 07:
Restarting the Management agents on ESX
#  service mgmt-vmware restart
To restart all management agents on the host, run the command:
services.sh restart
Caution: Ensure Automatic Startup/Shutdown of virtual machines is disabled before running this command or you risk rebooting the virtual machines.
Example- 08: To register a VM.
# vim-cmd solo/registervm “{[datastore1] testvm/testvm.vmx}”
(# vim-cmd solo/registervm /vmfs/volumes/datastore1/testvm/testvm.vmx)
To unregister a VM.
# vim-cmd vmsvc/unregister

Example – 09: To know fiber channel, so you can get information about which adapters are used for FC.
# esxcli storage san fc list
To display FC event information:
# esxcli storage san fc events get

Example – 10: To up-grade VMTool without reboot and it’s fully tested.

  1. Right click on a VM in the vSphere Client
  2. Choose Guest
  3. Choose Install/Upgrade VMware Tools.

You will be prompted to choose how you would like the upgrade to take place, either Interactively or Automatically. Along with the Automatic option comes the ability to enter some arguments, listed as “advanced options” in the GUI, that will be passed to the install.

/s /v/qn ADDLOCAL=ALL REBOOT=ReallySuppress


Reference KB : https://www.vmware.com/support/developer/windowstoolkit/wintk40/doc/viwin_admg.pdf

in progress…

Leave a Reply

Please log in using one of these methods to post your comment:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s