Hi,
We have Dell PowerEdge R710 on which VMware ESX 4.0 has been installed
CPU - 8CPU x 2.127 GHz
Memory - 32 GB
# vmware -v
VMware ESX 4.0.0 build-164009
I have around 10 VMs running on this and we have at a time 3 to 4 VMs powered on.
The ESX is a standalone host in a vCenter Server and not present in any cluster.
PROBLEM - The above ESX host keeps on rebooting every once in 14-15 days. Its not part of any shutdown program nor DPM comes into picture because it is not in a cluster. The rebooting has been happening since last 6 months on regular intervals.
In the hostd logs, I was able to find the below events
LOGS -
On March 27 - 4:37 the host had rebooted and below were the logs
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
[2011-03-27 04:17:37.599 F6363B90 verbose 'PropertyJournal'] ERProviderImpl<BaseT>::_GetChanges: Aggregate version Overflow ha-host summary.runtime.connectionState
[2011-03-27 04:17:37.599 F6363B90 verbose 'PropertyJournal'] ERProviderImpl<BaseT>::_GetChanges: Aggregate version Overflow ha-host summary.runtime.inMaintenanceMode
[2011-03-27 04:17:37.599 F6363B90 verbose 'PropertyJournal'] ERProviderImpl<BaseT>::_GetChanges: Aggregate version Overflow ha-host summary.runtime.powerState
[2011-03-27 04:17:37.599 F6363B90 verbose 'PropertyJournal'] ERProviderImpl<BaseT>::_GetChanges: Aggregate version Overflow ha-host tag
[2011-03-27 04:17:37.605 F6363B90 verbose 'Locale'] Default resource used for 'host.SystemIdentificationInfo.IdentifierType.AssetTag.label' expected in module 'enum'.
[2011-03-27 04:17:37.605 F6363B90 verbose 'Locale'] Default resource used for 'host.SystemIdentificationInfo.IdentifierType.AssetTag.summary' expected in module 'enum'.
[2011-03-27 04:17:37.605 F6363B90 verbose 'Locale'] Default resource used for 'host.SystemIdentificationInfo.IdentifierType.ServiceTag.label' expected in module 'enum'.
[2011-03-27 04:17:37.605 F6363B90 verbose 'Locale'] Default resource used for 'host.SystemIdentificationInfo.IdentifierType.ServiceTag.summary' expected in module 'enum'.
[2011-03-27 04:17:37.606 F6363B90 verbose 'PropertyJournal'] ERProviderImpl<BaseT>::_HasChanges: Aggregate version Overflow ha-host recentTask
[2011-03-27 04:17:37.606 F6363B90 verbose 'PropertyJournal'] ERProviderImpl<BaseT>::_GetChanges: Aggregate version Overflow ha-host configIssue
[2011-03-27 04:17:37.606 F6363B90 verbose 'PropertyJournal'] ERProviderImpl<BaseT>::_GetChanges: Aggregate version Overflow ha-host summary.managementServerIp
[2011-03-27 04:17:37.606 F6363B90 verbose 'PropertyJournal'] ERProviderImpl<BaseT>::_GetChanges: Aggregate version Overflow ha-host summary.rebootRequired
[2011-03-27 04:17:37.606 F6363B90 verbose 'PropertyJournal'] ERProviderImpl<BaseT>::_GetChanges: Aggregate version Overflow ha-host vm.length
[2011-03-27 04:17:37.621 F6363B90 verbose 'Locale'] Default resource used for 'host.SystemIdentificationInfo.IdentifierType.AssetTag.label' expected in module 'enum'.
[2011-03-27 04:17:37.621 F6363B90 verbose 'Locale'] Default resource used for 'host.SystemIdentificationInfo.IdentifierType.AssetTag.summary' expected in module 'enum'.
[2011-03-27 04:17:37.621 F6363B90 verbose 'Locale'] Default resource used for 'host.SystemIdentificationInfo.IdentifierType.ServiceTag.label' expected in module 'enum'.
[2011-03-27 04:17:37.621 F6363B90 verbose 'Locale'] Default resource used for 'host.SystemIdentificationInfo.IdentifierType.ServiceTag.summary' expected in module 'enum'.
[2011-03-27 04:17:37.628 F6363B90 verbose 'DvsTracker'] FetchDVPortgroups: added 0 items
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
However I did a search for the above line in all the hostd.log files and found numerous and after every such interval the machine used to reboot...
# cat hostd-*.log | grep -i reboot | more
Feb 17 it rebooted
[2011-03-17 12:36:49.614 F63E5B90 verbose 'PropertyJournal'] ERProviderImpl<BaseT>::_GetChanges: Aggregate version Overflow ha-host summary.rebootRequired
[2011-03-17 12:59:24.510 F5AC8B90 verbose 'PropertyJournal'] ERProviderImpl<BaseT>::_GetChanges: Aggregate version Overflow ha-host summary.rebootRequired
[2011-03-17 13:21:59.201 F6467B90 verbose 'PropertyJournal'] ERProviderImpl<BaseT>::_GetChanges: Aggregate version Overflow ha-host summary.rebootRequired
[2011-03-17 13:44:34.162 F59E3B90 verbose 'PropertyJournal'] ERProviderImpl<BaseT>::_GetChanges: Aggregate version Overflow ha-host summary.rebootRequired
[2011-03-17 14:06:26.157 F66536D0 verbose 'PropertyJournal'] ERProviderImpl<BaseT>::_GetChanges: Aggregate version Overflow ha-host summary.rebootRequired
[2011-03-17 14:22:37.456 F5931B90 verbose 'PropertyJournal'] ERProviderImpl<BaseT>::_GetChanges: Aggregate version Overflow ha-host summary.rebootRequired
[2011-03-17 14:44:48.066 F6426B90 verbose 'PropertyJournal'] ERProviderImpl<BaseT>::_GetChanges: Aggregate version Overflow ha-host summary.rebootRequired
[2011-03-17 15:07:22.759 F5AC8B90 verbose 'PropertyJournal'] ERProviderImpl<BaseT>::_GetChanges: Aggregate version Overflow ha-host summary.rebootRequired
[2011-03-17 15:28:27.211 F5AC8B90 verbose 'PropertyJournal'] ERProviderImpl<BaseT>::_GetChanges: Aggregate version Overflow ha-host summary.rebootRequired
March 27 it rebooted
[2011-03-26 23:28:34.774 F63E5B90 verbose 'PropertyJournal'] ERProviderImpl<BaseT>::_GetChanges: Aggregate version Overflow ha-host summary.rebootRequired
[2011-03-26 23:51:09.453 F59E3B90 verbose 'PropertyJournal'] ERProviderImpl<BaseT>::_GetChanges: Aggregate version Overflow ha-host summary.rebootRequired
[2011-03-27 00:13:44.435 F63A4B90 verbose 'PropertyJournal'] ERProviderImpl<BaseT>::_GetChanges: Aggregate version Overflow ha-host summary.rebootRequired
[2011-03-27 00:36:18.941 F6467B90 verbose 'PropertyJournal'] ERProviderImpl<BaseT>::_GetChanges: Aggregate version Overflow ha-host summary.rebootRequired
[2011-03-27 00:58:53.629 F5981B90 verbose 'PropertyJournal'] ERProviderImpl<BaseT>::_GetChanges: Aggregate version Overflow ha-host summary.rebootRequired
[2011-03-27 01:21:28.105 F63A4B90 verbose 'PropertyJournal'] ERProviderImpl<BaseT>::_GetChanges: Aggregate version Overflow ha-host summary.rebootRequired
[2011-03-27 01:44:02.862 F64A8B90 verbose 'PropertyJournal'] ERProviderImpl<BaseT>::_GetChanges: Aggregate version Overflow ha-host summary.rebootRequired
[2011-03-27 02:06:37.536 F6467B90 verbose 'PropertyJournal'] ERProviderImpl<BaseT>::_GetChanges: Aggregate version Overflow ha-host summary.rebootRequired
[2011-03-27 02:29:12.444 F6426B90 verbose 'PropertyJournal'] ERProviderImpl<BaseT>::_GetChanges: Aggregate version Overflow ha-host summary.rebootRequired
[2011-03-27 02:50:16.776 F66536D0 verbose 'PropertyJournal'] ERProviderImpl<BaseT>::_GetChanges: Aggregate version Overflow ha-host summary.rebootRequired
[2011-03-27 03:11:22.207 F64A8B90 verbose 'PropertyJournal'] ERProviderImpl<BaseT>::_GetChanges: Aggregate version Overflow ha-host summary.rebootRequired
[2011-03-27 03:33:57.214 F5AC8B90 verbose 'PropertyJournal'] ERProviderImpl<BaseT>::_GetChanges: Aggregate version Overflow ha-host summary.rebootRequired
[2011-03-27 03:56:32.165 F5981B90 verbose 'PropertyJournal'] ERProviderImpl<BaseT>::_GetChanges: Aggregate version Overflow ha-host summary.rebootRequired
[2011-03-27 04:17:37.606 F6363B90 verbose 'PropertyJournal'] ERProviderImpl<BaseT>::_GetChanges: Aggregate version Overflow ha-host summary.rebootRequired
April 10th it rebooted
[2011-04-09 22:52:11.709 F5A8FB90 verbose 'PropertyJournal'] ERProviderImpl<BaseT>::_GetChanges: Aggregate version Overflow ha-host summary.rebootRequired
[2011-04-09 23:14:46.162 F63EDB90 verbose 'PropertyJournal'] ERProviderImpl<BaseT>::_GetChanges: Aggregate version Overflow ha-host summary.rebootRequired
[2011-04-09 23:38:06.422 F5A8FB90 verbose 'PropertyJournal'] ERProviderImpl<BaseT>::_GetChanges: Aggregate version Overflow ha-host summary.rebootRequired
[2011-04-09 23:59:55.290 F661A6D0 verbose 'PropertyJournal'] ERProviderImpl<BaseT>::_GetChanges: Aggregate version Overflow ha-host summary.rebootRequired
[2011-04-10 00:24:00.068 F5700B90 verbose 'PropertyJournal'] ERProviderImpl<BaseT>::_GetChanges: Aggregate version Overflow ha-host summary.rebootRequired
[2011-04-10 00:48:05.281 F5A8FB90 verbose 'PropertyJournal'] ERProviderImpl<BaseT>::_GetChanges: Aggregate version Overflow ha-host summary.rebootRequired
[2011-04-10 01:10:39.892 F642EB90 verbose 'PropertyJournal'] ERProviderImpl<BaseT>::_GetChanges: Aggregate version Overflow ha-host summary.rebootRequired
[2011-04-10 01:34:44.924 F5793B90 verbose 'PropertyJournal'] ERProviderImpl<BaseT>::_GetChanges: Aggregate version Overflow ha-host summary.rebootRequired
[2011-04-10 01:57:19.457 F632AB90 verbose 'PropertyJournal'] ERProviderImpl<BaseT>::_GetChanges: Aggregate version Overflow ha-host summary.rebootRequired
[2011-04-10 02:18:23.912 F62E9B90 verbose 'PropertyJournal'] ERProviderImpl<BaseT>::_GetChanges: Aggregate version Overflow ha-host summary.rebootRequired
[2011-04-10 02:40:58.268 F5793B90 verbose 'PropertyJournal'] ERProviderImpl<BaseT>::_GetChanges: Aggregate version Overflow ha-host summary.rebootRequired
[2011-04-10 03:03:32.798 F632AB90 verbose 'PropertyJournal'] ERProviderImpl<BaseT>::_GetChanges: Aggregate version Overflow ha-host summary.rebootRequired
[2011-04-10 03:27:37.747 F63EDB90 verbose 'PropertyJournal'] ERProviderImpl<BaseT>::_GetChanges: Aggregate version Overflow ha-host summary.rebootRequired
[2011-04-10 03:50:12.876 F5793B90 verbose 'PropertyJournal'] ERProviderImpl<BaseT>::_GetChanges: Aggregate version Overflow ha-host summary.rebootRequired
[2011-04-10 04:14:17.722 F63EDB90 verbose 'PropertyJournal'] ERProviderImpl<BaseT>::_GetChanges: Aggregate version Overflow ha-host summary.rebootRequired
[2011-04-10 04:36:52.257 F661A6D0 verbose 'PropertyJournal'] ERProviderImpl<BaseT>::_GetChanges: Aggregate version Overflow ha-host summary.rebootRequired
[2011-04-10 05:00:57.062 F5700B90 verbose 'PropertyJournal'] ERProviderImpl<BaseT>::_GetChanges: Aggregate version Overflow ha-host summary.rebootRequired
[2011-04-10 05:25:02.151 F642EB90 verbose 'PropertyJournal'] ERProviderImpl<BaseT>::_GetChanges: Aggregate version Overflow ha-host summary.rebootRequired
[2011-04-10 05:47:36.552 F646FB90 verbose 'PropertyJournal'] ERProviderImpl<BaseT>::_GetChanges: Aggregate version Overflow ha-host summary.rebootRequired
[2011-04-10 06:08:49.425 F5A8FB90 verbose 'PropertyJournal'] ERProviderImpl<BaseT>::_GetChanges: Aggregate version Overflow ha-host summary.rebootRequired
[2011-04-10 06:31:17.430 F646FB90 verbose 'PropertyJournal'] ERProviderImpl<BaseT>::_GetChanges: Aggregate version Overflow ha-host summary.rebootRequired
[2011-04-10 06:55:22.354 F642EB90 verbose 'PropertyJournal'] ERProviderImpl<BaseT>::_GetChanges: Aggregate version Overflow ha-host summary.rebootRequired
[2011-04-10 07:17:56.763 F636BB90 verbose 'PropertyJournal'] ERProviderImpl<BaseT>::_GetChanges: Aggregate version Overflow ha-host summary.rebootRequired
[2011-04-10 07:39:21.941 F632AB90 verbose 'PropertyJournal'] ERProviderImpl<BaseT>::_GetChanges: Aggregate version Overflow ha-host summary.rebootRequired
[2011-04-10 08:00:05.417 F646FB90 verbose 'PropertyJournal'] ERProviderImpl<BaseT>::_GetChanges: Aggregate version Overflow ha-host summary.rebootRequired
[2011-04-10 08:24:10.317 F62E9B90 verbose 'PropertyJournal'] ERProviderImpl<BaseT>::_GetChanges: Aggregate version Overflow ha-host summary.rebootRequired
Can anyone tell what exactly is the above log depicting, I did not find any VMware KB article.
Any help is appreciated.
Thanks
Deepak