HI all,

Got a problem with storage VMotion in a Cluster of 4 esx host  (vsphere 4.0.0,208167)

VMotion works ok. no errors/warning in the host's vmkernel or vmKwarning logs. Checked storage and paths no problems found.Run a test with a VM it can be storage VMotion on two out of the 4 esx hosts in the cluster.The same error is logged in the vm's vmwre.log

Host  capacity memory  77%, CPU 34%

The vm's vmware.log reports the following error:

May 06 15:40:07.831: Worker#0| DISKLIB-VMFS :CopyData '/vmfs/volumes/4ae53235-da7ef348-3f56-001e4f1dd095/ns1/ns1-flat.vmdk' : failed to move data (Cannot allocate memory:0xc0009).

Any ideas?

May 06 15:40:02.104: vmx| VMXVmdbCbVmVmxMigrate: Got SET callback for /vm/#_VMX/vmx/migrateState/cmd/##9_20cc2/op/=to

May 06 15:40:02.105: vmx| VmxMigrateGetParam: type: 2 srcIp=<10.1.2.7> dstIp=<10.1.2.7> mid=485e65f566323 uuid=44454c4c-3600-1033-8058-b9c04f323253 priority=none checksumMemory=no maxDowntime=0 encrypted=0 resumeDuringPageIn=no

May 06 15:40:02.107: vmx| VmxMigrateGetParam: type 2 unsharedSwap 1 memMinToTransfer 0 cpuMinToTransfer 0 numDisks 1

May 06 15:40:02.111: vmx| Received migrate 'to' request for mid id 1273124395311907, src ip <10.1.2.7>, dst ip <10.1.2.7>.

May 06 15:40:02.142: vmx| MigrateSetInfo: state=1 srcIp=<10.1.2.7> dstIp=<10.1.2.7> mid=1273124395311907 uuid=44454c4c-3600-1033-8058-b9c04f323253 priority=low

May 06 15:40:02.143: vmx| MigrateStateUpdate: Transitioning from state 0 to 1.

May 06 15:40:06.707: vmx| VMXVmdbCbVmVmxMigrate: Got SET callback for /vm/#_VMX/vmx/migrateState/cmd/##9_20ccc/op/=start

May 06 15:40:06.718: vmx| VmxMigrateGetStartParam: mid=485e65f566323 dstwid=165341

May 06 15:40:06.718: vmx| Received migrate 'start' request for mig id 1273124395311907, dest world id 165341.

May 06 15:40:06.720: vmx| WORKER: Creating new group with numThreads=1 (4)

May 06 15:40:06.723: vmx| CPT current = 0, requesting 1

May 06 15:40:06.724: vmx| MigrateStateUpdate: Transitioning from state 1 to 2.

May 06 15:40:06.725: vmx| Set thread 5 stack size to 1048576: Success

May 06 15:40:06.726: vcpu-0| Sync monModules(1).

May 06 15:40:06.729: vcpu-0| Done Sync monModules(1).

May 06 15:40:06.729: vmx| CPT: monitor ACKing mode 1

May 06 15:40:06.730: vmx| Destroying virtual dev for scsi0:0 vscsi=44625

May 06 15:40:06.730: vmx| VMMon_VSCSIStopVports: No such target on adapter

May 06 15:40:06.731: vmx| DISKLIB-VMFS  : "/vmfs/volumes/46f9b29c-8959b018-376f-001aa025a31a/ns1/ns1-flat.vmdk" : closed.

May 06 15:40:06.732: vmx| CPT current = 1, requesting 2

May 06 15:40:06.733: vmx| CPT: monitor ACKing mode 2

May 06 15:40:07.553: vmx| CPT current = 2, requesting 6

May 06 15:40:07.563: vmx| Checkpoint_Unstun: vm stopped for 840003 us

May 06 15:40:07.564: vcpu-0| Sync monModules(6).

May 06 15:40:07.564: vcpu-0| Done Sync monModules(6).

May 06 15:40:07.564: vmx| CPT: monitor ACKing mode 6

May 06 15:40:07.569: vmx| DISK: OPEN scsi0:0 '/vmfs/volumes/46f9b29c-8959b018-376f-001aa025a31a/ns1/ns1.vmdk' persistent R[]

May 06 15:40:07.590: vmx| DISKLIB-VMFS  : "/vmfs/volumes/46f9b29c-8959b018-376f-001aa025a31a/ns1/ns1-flat.vmdk" : open successful (10) size = 18376324096, hd = 101089547. Type 3

May 06 15:40:07.590: vmx| DISKLIB-DSCPTR: Opened : "ns1-flat.vmdk" (0xa)

May 06 15:40:07.590: vmx| DISKLIB-LINK  : Opened '/vmfs/volumes/46f9b29c-8959b018-376f-001aa025a31a/ns1/ns1.vmdk' (0xa): vmfs, 35891258 sectors / 17.1 GB.

May 06 15:40:07.590: vmx| DISKLIB-LIB   : Opened "/vmfs/volumes/46f9b29c-8959b018-376f-001aa025a31a/ns1/ns1.vmdk" (flags 0xa).

May 06 15:40:07.591: vmx| DISK: OPEN '/vmfs/volumes/46f9b29c-8959b018-376f-001aa025a31a/ns1/ns1.vmdk' Geo (2234/255/63) BIOS Geo (0/0/0)

May 06 15:40:07.592: vmx| DISKLIB-CTK   : Auto blocksize for size 35891258 is 128.

May 06 15:40:07.603: vmx| DISKLIB-CBT   : Initializing ESX kernel change tracking for fid 101089547.

May 06 15:40:07.604: vmx| DISKLIB-CBT   : Successfuly created cbt node 606810b-cbt.

May 06 15:40:07.604: vmx| DISKLIB-CBT   : Opening cbt node /vmfs/devices/cbt/606810b-cbt

May 06 15:40:07.605: vmx| Creating virtual dev for scsi0:0

May 06 15:40:07.605: vmx| DumpDiskInfo: scsi0:0 createType=11, capacity = 35891258, numLinks = 1, allocationType = 0

May 06 15:40:07.606: vmx| SCSIDiskESXPopulateVDevDesc: Using FS backend

May 06 15:40:07.653: Worker#0| DISKLIB-LIB   : CREATE: "/vmfs/volumes/4ae53235-da7ef348-3f56-001e4f1dd095/ns1/ns1.vmdk" -- vmfs capacity=0 (0 bytes) adapter=buslogic cowGran=0

May 06 15:40:07.655: Worker#0| DISKLIB-LIB   : DiskLib_CreateChildCreateParam: grain size set to : 1

May 06 15:40:07.656: Worker#0| DISKLIB-LIB   : CREATE: "/vmfs/volumes/4ae53235-da7ef348-3f56-001e4f1dd095/ns1/ns1.vmdk" -- vmfs capacity=35891258 (17.1 GB) adapter=buslogic cowGran=1

May 06 15:40:07.730: Worker#0| DISKLIB-LINK  : "/vmfs/volumes/4ae53235-da7ef348-3f56-001e4f1dd095/ns1/ns1.vmdk" : creation successful.

May 06 15:40:07.731: Worker#0| DISKLIB-VMFS  : "/vmfs/volumes/4ae53235-da7ef348-3f56-001e4f1dd095/ns1/ns1-flat.vmdk" : open successful (17) size = 18376324096, hd = 0. Type 3

May 06 15:40:07.770: Worker#0| DISKLIB-VMFS  : "/vmfs/volumes/4ae53235-da7ef348-3f56-001e4f1dd095/ns1/ns1-flat.vmdk" : closed.

May 06 15:40:07.773: Worker#0| DISKLIB-VMFS  : "/vmfs/volumes/4ae53235-da7ef348-3f56-001e4f1dd095/ns1/ns1-flat.vmdk" : open successful (528) size = 18376324096, hd = 151765263. Type 3

May 06 15:40:07.773: Worker#0| DISKLIB-DSCPTR: Opened : "ns1-flat.vmdk" (0x210)

May 06 15:40:07.774: Worker#0| DISKLIB-LINK  : Opened '/vmfs/volumes/4ae53235-da7ef348-3f56-001e4f1dd095/ns1/ns1.vmdk' (0x210): vmfs, 35891258 sectors / 17.1 GB.

May 06 15:40:07.774: Worker#0| DISKLIB-LIB   : Opened "/vmfs/volumes/4ae53235-da7ef348-3f56-001e4f1dd095/ns1/ns1.vmdk" (flags 0x210).

May 06 15:40:07.817: Worker#0| DISKLIB-DDB   : "longContentID" = "2594bb631353b27093f8f350dd11abc3" (was "329c79216ae7078d034051b559a20051")

May 06 15:40:07.831: Worker#0| DISKLIB-VMFS :CopyData '/vmfs/volumes/4ae53235-da7ef348-3f56-001e4f1dd095/ns1/ns1-flat.vmdk' : failed to move data (Cannot allocate memory:0xc0009).

May 06 15:40:07.831: Worker#0| DISKLIB-LIB   : DiskLib_Clone: Could

not perform clone using vmkernel data mover. Falling back to

non-accelerated clone. Cannot allocate memory

May 06 15:40:11.880: vcpu-0| HBACommon: First write on scsi0:0.fileName='/vmfs/volumes/46f9b29c-8959b018-376f-001aa025a31a/ns1/ns1.vmdk'

May 06 15:40:11.898: vcpu-0| DISKLIB-CHAIN : UpdateContentID: old = 0xdd11abc3, new = 0x8ae3b0a2

May 06 15:40:11.899: vcpu-0| DISKLIB-DDB   : "longContentID" = "b5392b788526fa1622fe17f58ae3b0a2" (was "2594bb631353b27093f8f350dd11abc3")

May 06 15:40:11.904: vcpu-0| DISKLIB-CHAIN : UpdateContentID: updated long content id

May 06 15:44:47.816: Worker#0| DISKLIB-LIB   : numIOs = 50000 numMergedIOs = 0 numSplitIOs = 0

May 06 15:44:47.824: Worker#0| DISKLIB-LIB   : numIOs = 50000 numMergedIOs = 0 numSplitIOs = 0

May 06 15:49:52.996: Worker#0| DISKLIB-LIB   : numIOs = 100000 numMergedIOs = 0 numSplitIOs = 0

May 06 15:49:53.031: Worker#0| DISKLIB-LIB   : numIOs = 100000 numMergedIOs = 0 numSplitIOs = 0

May 06 15:55:42.037: Worker#0| DISKLIB-LIB   : numIOs = 150000 numMergedIOs = 0 numSplitIOs = 0

May 06 15:55:42.047: Worker#0| DISKLIB-LIB   : numIOs = 150000 numMergedIOs = 0 numSplitIOs = 0

Message was edited by: richardt

Message was edited by: richardt

Hi,

I am using ESX 4.0

Hi Everyone,

I am building a new network infrastructure.  The goal is to create a fully redundant, highly available, high throughput network with EMC and VMware ESX.  I've put together a diagram of the plan.  It's also posted at EMCVMWareESXfor other user feedbacks.  Please let me know your thoughs.

Few things that I'm wondering:

1.  Would it be too intensive if I used the same NIC for HA heartbeat and Vmotion?  I have 2 for redundancy and splitting the load across.

2.  What disk RAID level should I use for the VM OS? 

3.  Will it be safe to Virtualize Web servers and SQL servers?  I plan to virtualize Windows 2008 IIS7 and Windows 2008 SQL 2008 Enterprise.

Thank you.

hi guys,

i have a vsphere machine,

i am trying to see vmkcore partition using "fdisk -lh". it is showing that "vmware vmkcore" partition was mounted to /dev/sda2 partition.

But i am not able to see that partition by using "df -h". plz see the attached picture for better understand

i hope  my question making sence.

thanks inadvance.

Hi guys,

I have installed ESX server 4.1 and have installed two virtual machines that is of "Windows XP"  and "Windows Server 2008 R2". The problem was slow mouse movement in both virtual machines, but this problem is fixed on XP virtual machine by installingVmware Tools (ver: 8.3.2). But the problem is still same on Server 2008 R2 even i have installed vmware tools on this virutal machine too. But i dont know whats wrong? cursor movement is still very slow in server 2008 R2 and i am unable to continou my work on this machine..

Any Idea?

Your help will be much appriciated. Thanks   ?:|

Hi,

I've installed ESX 4.0,  in  DMZ net work.

When I tried to add this server in Virtual center I get  "Request timed"  out error, but Im able to manage this server using vSphere client  and able to connect using Putty.

Im able to telnet 903 from VC server also from where I installed vSphere client.

any help would be highly appreciated.

Thanks,

Ajesh

Hey all,

I am running ESXi 4.1 on an Intel server and I get PSOD's whenever I perform certain disk operations. It seems to happen when I partition a disk or once in a while when writing to disk.

It happens on thin or thick provisioned disks

It ONLY happens with LSI Parallel config, not LSI SAS.

I have attached the purple screen I get. It sucks that a Linux guests disk operation can crash an entire ESXi host with 15 guests on it. The guest disk operations SHOULD be irrelevant to the host.

Any help would be appreciated.

Thanks

-J

I have a number of hosts that are configured with dvSwitches for all port groups including the service console. For reasons out of my control I need to moved them from an existing 4.0 virtual center to a new 4.1 virtual center that is being built. The new virtual center is on a new network with a completely different IP address range.

The process we are looking at using for migration seems like a PIA. Basically going to every host and creating standard vSwitches for every portgroup manually. Then moving all vm's from the port groups on the dvSwitches to the matching vSwitches. Remove the hosts from the existing virtual center server and reconnect them to the new virtual center. Finally build new dvSwitches and migrate all the vm's to them.

Anybody have suggestions on how to simplify this process?

while putting host in maintenance mode i get stuck at 2%

With the below message at 2% it get stuck.

"Waiting for all VMs to be powered off or migrated. In a DRS cluster check the Faults page on the DRS tab for troubleshooting. "

I'm having a weird issue with my WD Sharespace device.  I use a WD Sharespace to mount as an NFS device to store all of our ISO and install files.  I have one ESX 4 cluster with 5 servers, and two standalone ESX 4 servers.  The SHarespace NAS has had some custom config work done to it to allow this to work (no_root_squash added to the shares in the exports file to allow ESX NFS to work)- and it was working fine, until about a month ago.  I did an upgrade to the Sharespace NAS, and after the reboot (and reestablishing the no_root_squash in the exports file), I found I could not mount the NFS shares on the NAS to any server in the ESX cluster.  I get

"Create NAS datastore: server1.company.local: Error during the configuration of the host: Cannot open volume: /vmfs/volumes/a4e52396-dcdec3f8"

I do not get this error on the two standalone ESX servers, I was able to mount the NFS shares just fine.

I've gone through and checked the security profiles- they all seem correct, NFS client is allowed.

During troubleshooting, I traced the volume that ESX said it couldn't open.  Found out what it was, migrated all VM's from it, and then removed the data store from the system.  I get the same error, different data store.  It seems that the last data store gets a volume ID with only two sets of characters (the rest have four (like 4d3e6612-a6616638-0e9d-001b211632b5).  Not sure how it's related- I get the same kind of listing on the standalone ESX servers, and the NFS share mounted just fine.

I checked the hosts.allow file, it lists the servers in the cluster, as well as the standalone servers.  hosts.deny has one line, "portmap:ALL".

Any clues?

hi

i have added second disk (NAS) then after i am getting error "recycle bin corrupt do you want empty the recycle bin"  if i say yes then contents will not delete and geting error access denied.

tried through administrator command prompt rd /s /q D:\$Recycle.bin no luck

getting error

d:\$recycle.bin\S-1-5-~1 - Access is denied.

d:\$recycle.bin\S-1-5-~2 - Access is denied.

d:\$recycle.bin\S-1-5-~3 - Access is denied.

d:\$recycle.bin\S-3f32~1 - Access is denied.

d:\$recycle.bin\S-1-5-~4 - Access is denied.

and also i tried http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=2001320 removed the second disk and readded still my problem was not solved.

any help would be appreciated.

Regards,

Vijay Kumar

Hi all.

Vcenter essentials plus.

3 hosts, each with 2 sockets

Then I have another Datacentre within my inventory with another host running vmware vsphere 4 enterprise as a standalone host.

AFAIK - my licensing is correct.

However, I am getting a notification:

This editon of vCenter only supports 3 host(s) in the inventory. Remove some of the hosts from the inventory before performing the operation.

Any ideas?

Hi Guy’s

I currently have a VM that won’t power on!

The problem was that the DataStore went offline because the SAN had a problem with the RAID set that the DataStore was on, I was able to get this back online and the ESX servers was able to see the DataStore and see all the files\folder for the VM’s

We I went to power one of the VM’s on all I get is a black screen in the VI client

Other steps I’ve completed with no success

1. Powered on VM Server (call it VM001), failed to boot and gave a black screen on the console

2. Built a new temp 2008 server, server booted into windows, attached VM001 VMDK file to see if we could repair\get data from the file. Server was unable to read VMDK file. Device manage times out trying to detect new hardware, no new disk in disk management

3. Using the VMware VI Client migrated VM001 to a new DataStore, failed

Error: Cannot relocate virtual machine VM001

4. Using the VMware VI Client cloned VM001 server, failed

Error: Cannot clone VM001: Error caused by file [MSA1500-2-RAID-5] VM001/VM001.vmdk

5. Using the VMware VI Client copied the VMDK file for server VM001 to another folder\DataStore, failed

Error: Unexpected end of stream

Able to copy log files and other files just not the –flat.vmdk file

6. Using Veeam FastSCP copied the VMDK file for server VM001, to a local disk, failed

Error: Failed to get file.

Client Erorr: Bad address

POSIX: Failed to read data from the file

Unable to read file block

Able to copy log files and other files just not the –flat.vmdk file

7. Using the command line cloned the VMDK (vmkfstools –i) file for server VM001 to same\another DataStore, failed

Error: Failed to clone disk : Bad address (917513)

Server details: HP DL380 G7

Storage: MSA1500 running active active

ESX Details: ESX 4.1.0, 381591

At the moment I’m unable to get the server powered on or even get the VMDK off the DataStore to see if we can get any data from it

Any ideas?

Many Thanks

Jonathan

I get "Unable to connect to NFS server"  when I try to connect to FreeNAS NFS share using ESXi 5.1, but both can connect to NFS share on linux. Further linux can connect to the share on FreeNAS. I think it is some setting or protocol thing between ESXi and FreeNAS 8.2. Any help would be appreciated.

The exact error is:

Call "HostDatastoreSystem.CreateNasDatastore" for object "ha-datastoresystem" on ESXi "192.168.0.242" failed.
NFS mount 192.168.0.239:/mnt/array1 failed: Unable to connect to NFS server.

Also get similar error an ESXi 3.51 server.

IP#: esxi 3.51 192.168.0.241 / esxi 5.1 192.168.0.242 /  FreeNAS 192.168.0.239 (not virtual) / linux vm (on esxi 3.51) 192.168.0.249

I have checked the following actions

• vmkping 192.168.0.239 (freeNAS IP) and get response on both esxi 3.51 & 5.1
• I have checked ESXi5.1 firewall and made sure outgoing NFS connection is enable.
• I can connect to an NFS share on a linux box via both ESX 3.51 & 5.1 (same 192.168.0.0/24 network)
• I can also connect to the FreeNAS share from linux box and virtual linux machine (same 192.168.0.0/24 network)
• the FreeNAS can ping the esxi servers
• reverse DNS is working on FreeNAS (ie the IP of ESXi boxes gets resolved to a name)
• the ip of the esxi boxes is in the Authorized networks
• I have set "Map User" to root in FreeNAS. I dont know how to set no_root_squash in FreeNAS or BSD shell.

Here is the /etc/export file for the FreeNAS

/mnt/array1  -alldirs -maproot=root: -network 192.168.0.0/24

Hi All,

We are receiving the below alarm frequently on our ESX hosts. The alarm triggers and gets closed within19 or 20 seconds.

We don't see any issues during the alarm. We have checked hostd, vpxa log, no network drops seen from the performance chart for all the vmnics . But wondering, are these below alarms false or we are missing anything?

Our ESX build version is ESX 4.1.o build 800380

Vcenter server 4.1 build 345043

We have disabled and enabled the alarm, re-booted ESX host, nothing worked. These alarms are triggering frequently atleast once in a week for all hosts.

Message was edited by: VMfreebie

We had some issue during weekend, some one deleted few files or something happened? How to find out the root cause analysis? I have exported the logs from vcenter and I got almost 3 GB logs locally? But where to check for virtual machine related logs is not clear.  I would like to see who made changes to those VM's?  Can any one please respond? This is very urgent.

Thanks,

Ravi

I have here 4 IBM X3650 M2 (7947-52G) with a clean install ESX 4.0.0 244038

-

UEFI 1.07 D6E145FUS (upraded from 1.04)

IMM YU0057H 1.10 (upgradd from 1.05)

Diagnostics v233

-

  All 4 have the same issue. It is not possible to create a coredump

# esxcfg-dumppart -l

VM Kernel

Name                               Console

Name                                 Is Active   Is Configured

naa.600508e0000000001b3b0eef5553be07:2     

/dev/sde2                                    yes         yes

#  esxcfg-dumppart -c

naa.600508e0000000001b3b0eef5553be07:2  /dev/sde2

# esxcfg-dumppart --copy --devname /vmfs/devices/disks/naa.600508e0000000001b3b0eef5553be07:2

Single slot coredump

Error running command. Unable to copy the dump partition: Couldn't find a valid VMKernel dump file. Dump partition might be uninitialized.

I already deactivated (-d) and reactivated (-a)  and smart-activated (-S) the partition 

I also checked KB entries with no luck

Setting up a VMKcore partition to capture purple screen output

http://kb.vmware.com/selfservice/microsites/search.do?cmd=displayKC&docType=kc&externalId=1000328

Manually regenerating core dump file in VMware ESX

http://kb.vmware.com/selfservice/microsites/search.do?cmd=displayKC&docType=kc&externalId=1002769

The only hint I found was in http://communities.vmware.com/thread/217385 but I am not sure that this could be a firmware issue with a controller.

I also attached outputs from fdisk and from  vmfs disks output, esxcfg-scsidevs and lspci

Thanks for helping.

regards

Andreas

We just upgraded fro ESX3.5 to ESX 4 U1 a few weeks back. We are running on a IBM Blade Center S with HS21-7995 blades and using the SAS RAID Module for Storage.

On two occasion all the ESX hosts stopped responding/VMS were offline  and on console of each Host was displayed - SD 0:0:0:0 still retrying after 360s.

After power cycling  the Blades and  BCS Chassis, everything came back online.

Each hosts has a 20GB volume on the SAS RAID for BOOT  that is mapped to the blade.

This install was stable for over a year  on 3.5 and started this behavior after the upgrade . After the first time this occured I  subsequently reloaded all the hosts with a fresh install of ESX4 U1 after wiping the volume. We experenced this again yesterday morning.

I will likely open a support case with Vmware and I am already talking to IBM as well. Any thoughts or suggestions are appreciated.

I am having trouble migrating a VM from a datastore.  I am trying to decommission this SAN and I have been able to move all other VM's off except one.  I am not able to move it powered on on or off.  When I try to move it using vcenter I get the attached error.  If I use vmkfstools -i to clone it I get a "Function Not Implemented" error and if I try a straight cp copy of the VM I get a "read:   Function Not Implemented"  error.  The VM can be powered on and the OS and Service functions fine.

I have narrowed the problem to one of the two vmdk's the other vmdk cp's and " vmkfstools's" just fine.  I am a bit stuck on how to proceded.  I am hoping someone out there has had a problem like this and a quick fix. I would rather not rebuild the VM or restore backups to new VM if possible.  Any idea's would be a great help.  Thanks.

quick enviroment info

Current Datastore:  Windows Storage Server.

Datastore to migrate to: Dell Equallogic

Vmware 4.1 environment. 

Infrastructure:

vSphere

ESX 4

Dell R710 Server

I'm installing WIN 2003 x64 as a virtual server on ESX 4, when I'm selecting LSI logic SAS & LSI logic parallel it is not detecting hard drive, do I require driver if then from where I can download ???

For bus logic parallel(not recommended for guest OS) & Vmware paravirtual , do I need to select this for my win 2003 R2 x64 edition ?????

please help on this issue.

~Rashid