Quantcast
Channel: High Availability (Clustering) forum
Viewing all 4519 articles
Browse latest View live

Unable to add or live migrate VMs on new machine added to cluster

$
0
0

Had 1x 2012 R2 machine (call it A) set up as a cluster with another 2012 R2 machine used for shared storage. Finally got a second machine (call it B) to add to the cluster, added the machine, passed, showed up in the cluster, but I cannot live or quick migrate machines, or create VMs on A from B or B from A.

Live migrations (going from B to A) fail with:

Live migration of 'Virtual Machine TESTVM' failed.
Virtual machine migration operation for 'TESTVM' failed at migration destination 'HOST-A'. (Virtual machine ID 0D1A3F2C-7623-4D26-A5B9-71F88599BE1C)

User 'NT AUTHORITY\SYSTEM' failed to create external configuration store at '\\SHAREDSTORAGE\vms\TESTVM': General access denied error. (0x80070005)

Failed to set security info for '\\SHAREDSTORAGE\vms\TESTVM\Virtual Machines': 'General access denied error'('0x80070005').

When I try and create a VM on A from B or B from A I get:

The operation failed.

Failed to create external configuration store at '\\SHAREDSTORAGE\vms\TESTVM': General access denied error. (0x80070005)

User ‘DOMAIN\Administrator’ failed to create external configuration store at '\\SHAREDSTORAGE\vms\TESTVM': General access denied error. (0x80070005)

Went so far as to set the share permissions on \\SHAREDSTORAGE\vms to Everyone full control and Everyone full control on the NFTS permissions. No change.

If I am on A and create a VM on A in either Failover Cluster Manager or Hyper-V manager it works just fine. I can go to \\SHAREDSTORAGE\vms from either machine and create files\folders\etc no issue.


Exchange 2016 performance issues

$
0
0

We have a Typical issue wherein we have set of 16 Exchange 2016 servers with about 60k users. We have about 8 GC/DC in site,6 GC/DC out of site. Every morning for about 10-20 minutes, the users face performance issues wherin, outlook freezes,owa login is slow, sending emails is slow. When we look at the Exchange and domain controller resource performance (CPU,DISK,MEMORY), it is quite normal and under 50 %. However, the issue still happens. On the DCs, we get a lot of ATQ alerts and get deep LDAP queries for various objects, etc. We are currently unsure what is causing this. Any inputs is very much appreciated.

Error while selecting VM for High Availability in WS 2016 Hyper-V Failover Cluster

$
0
0

Dear all!

I've faced up with error while creating HA VM in WS 2016 Hyper-V Failover cluster with configuration listed below:

8-Node Hyper-V Failover Cluster on Windows Server 2016 Datacenter with all last updates.

I can create an non-HA VM on any host without problem, but when I try to make it HA, using Failover Cluster Management console and High Availability Wizard, I receive an error on step "Select Virtual Machine":



Also in "FailoverClustering-Manager" event log I've found an error: 

Failover Cluster Manager failed while managing one or more cluster. The error was 'An item with the same key has already been added.'. For more information see the Failover Cluster Manager Diagnostic channel.

In FailoverClustering-Manager Diagnostic log two errors:

Exception occurred in background operation - System.ArgumentException: An item with the same key has already been added.
   at System.ThrowHelper.ThrowArgumentException(ExceptionResource resource)
   at System.Collections.Generic.Dictionary`2.Insert(TKey key, TValue value, Boolean add)
   at System.Linq.Enumerable.ToDictionary[TSource,TKey,TElement](IEnumerable`1 source, Func`2 keySelector, Func`2 elementSelector, IEqualityComparer`1 comparer)
   at MS.Internal.ServerClusters.Configuration.HAViridianVirtualMachine.GetClusteredVirtualMachinesResourceOfType(Cluster cluster, String resourceType)
   at MS.Internal.ServerClusters.Configuration.HAViridianVirtualMachine.GetClusterableVirtualMachines(Cluster cluster)
   at MS.Internal.ServerClusters.Wizards.SelectVirtualMachinePage.LoadVirtualMachines(BackgroundOperationStatus status, Object parameter)
   at MS.Internal.ServerClusters.BackgroundOperation`2.BackgroundOperationProc(Object state)

and

An error was encountered while loading the list of available virtual machines. - System.ArgumentException: An item with the same key has already been added.
   at System.ThrowHelper.ThrowArgumentException(ExceptionResource resource)
   at System.Collections.Generic.Dictionary`2.Insert(TKey key, TValue value, Boolean add)
   at System.Linq.Enumerable.ToDictionary[TSource,TKey,TElement](IEnumerable`1 source, Func`2 keySelector, Func`2 elementSelector, IEqualityComparer`1 comparer)
   at MS.Internal.ServerClusters.Configuration.HAViridianVirtualMachine.GetClusteredVirtualMachinesResourceOfType(Cluster cluster, String resourceType)
   at MS.Internal.ServerClusters.Configuration.HAViridianVirtualMachine.GetClusterableVirtualMachines(Cluster cluster)
   at MS.Internal.ServerClusters.Wizards.SelectVirtualMachinePage.LoadVirtualMachines(BackgroundOperationStatus status, Object parameter)
   at MS.Internal.ServerClusters.BackgroundOperation`2.BackgroundOperationProc(Object state)

Could You please help me with this issue?

Thanks in advance!




2 node fail-over cluster(Active/Passive) on Windows Server 2012 R2 without RDM.(want on Shared disk)

$
0
0

Dear All,

I want build a SQL Server 2008 R2  Ent (64bit) and SQL Server 2012 Ent (64bit) 2 node fail-over cluster(Active/Passive) on Windows Server 2012 R2without RDM.(want on Shared disk)

Please share guide lines or links which can help us to Build the A/P cluster.

Thanks in advance.


SNIVAS

Cant run powershell upgrade commands

$
0
0

Hi,

I have a 3 node cluster that I have just upgraded to server 2016 using the rolling upgrade process. Everything went smoothly and I am now read to upgrade the functional level of the cluster.

However I am seeing strange errors that the required powershell commands are not recognised. See below.

PS C:\> Update-ClusterFunctionalLevel
Update-ClusterFunctionalLevel : The term 'Update-ClusterFunctionalLevel' is not recognized as the name of a cmdlet,
function, script file, or operable program. Check the spelling of the name, or if a path was included, verify that the
path is correct and try again.
At line:1 char:1
+ Update-ClusterFunctionalLevel
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : ObjectNotFound: (Update-ClusterFunctionalLevel:String) [], CommandNotFoundException
    + FullyQualifiedErrorId : CommandNotFoundException

PS C:\> Get-Cluster -Name FHMHVS00 | Update-ClusterFunctionalLevel
Get-Cluster : The term 'Get-Cluster' is not recognized as the name of a cmdlet, function, script file, or operable
program. Check the spelling of the name, or if a path was included, verify that the path is correct and try again.
At line:1 char:1
+ Get-Cluster -Name FHMHVS00 | Update-ClusterFunctionalLevel
+ ~~~~~~~~~~~
    + CategoryInfo          : ObjectNotFound: (Get-Cluster:String) [], CommandNotFoundException
    + FullyQualifiedErrorId : CommandNotFoundException

PS C:\> Get-Cluster | select ClusterFunctionalLevel
Get-Cluster : The term 'Get-Cluster' is not recognized as the name of a cmdlet, function, script file, or operable
program. Check the spelling of the name, or if a path was included, verify that the path is correct and try again.
At line:1 char:1
+ Get-Cluster | select ClusterFunctionalLevel
+ ~~~~~~~~~~~
    + CategoryInfo          : ObjectNotFound: (Get-Cluster:String) [], CommandNotFoundException
    + FullyQualifiedErrorId : CommandNotFoundException

PS C:\> Get-ClusterResource
Get-ClusterResource : The term 'Get-ClusterResource' is not recognized as the name of a cmdlet, function, script file,
or operable program. Check the spelling of the name, or if a path was included, verify that the path is correct and
try again.
At line:1 char:1

I ran Get-Command and sure enough these commands are not listed.

Any ideas would be appreciated.

Thanks



Failover Cluster volume inaccessible, showing GUID not volume

$
0
0

We have a file cluster on Server 2012 R2 (Fully updated) with a RDM passthrough disk to a Dell Compellent SAN using VMWare ESXi 5.5. Pathing is set to MRU for the RDM. 

We lost access to the cluster volume and in failover cluster manager where it would normally display the drive letter of the volume with name, it was showing the GUID and reporting 'Unknown'. Failing over to the other node resolved this. The VMWare VM/Host logs show no drops in connectivity nor does the SAN report an issue. This a fibre channel SAN, not iSCSI.

The logs at the time on the cluster node that was active say the following:

[RES] Physical Disk: Failed to open device \Device\Harddisk2\ClusterPartition1, status 0xc0000034
[RES] Physical Disk: HarddiskpIsPartitionHidden: failed to open device \Device\Harddisk2\ClusterPartition1, status 2
[RES] Physical Disk: HarddiskpIsPartitionHidden: failed to open device \Device\Harddisk2\ClusterPartition1, status 2
[RES] Physical Disk: HarddiskpIsPartitionHidden: failed to open device \Device\Harddisk2\ClusterPartition1, status 2
[RES] Physical Disk: HardDiskpGetDiskInfo: GetVolumeInformation failed for \\?\GLOBALROOT\Device\Harddisk2\ClusterPartition1\, status 3
[RES] Physical Disk: HardDiskpGetDiskInfo: failed to get partition size for \\?\GLOBALROOT\Device\Harddisk2\ClusterPartition1\, status 3

We're struggling to find reasoning why this happened. Can anyone provide possible causes or where to look that may lead us to an explanation?

Thank you in advance.

Help with failover Cluster

$
0
0

Hello,

I have a WSFC with allwayson.

2nodes are in same region and 1 in a different one.

In top of that i have a SQLServer with allwayson.

On region1 the resources can failover between the 2 nodes.

In case of DR sql server will failover (via allwayson) to region2.

In order to do some tests we have broke the communication between regions.

After, in region 2 we have forced cluster online without quorum. (In region 1 we kept everything working) 

The sql allwayon went online and all tests were done sucessefully.

After all the tests we have (in region 2) shutdown cluster and remove the vote from the DR node. We also have stopped all sql server resrouces (Avaiability groups and  listener).

Once all was down we have resume the communication between the 2 sites. NOW the problems appear.

The Cluster dns name as well as the SQL AO listener names have been removed from the primary and secondary dns.

When i try to add a listener to a new instance i get

The DNS from the cluster resrouce also has desapear.

Can anyone help?

Regards,

Arestas

Cluster Disk not visible

$
0
0

Hi!

I have created a two node windows 2012 R2 Cluster and assigned vhd disks but the disks are not visible while trying to move virtual machine storage. The cluster disks are in a windows 2008 R2 Server working as iscsi target server.

Any Suggestions?


ISCSI Target and Hyper-V on Same Server

$
0
0

Hi!

Is it possible to have ISCSi Target and Hyper-V Role on the same server and assign ISCSi drives to VMs on the same server? We need to create two node virtual storage cluster on two VMs. Can I work if everything is on the same server?

Thanks.

Windows Failover cluster across datacenter. Node showing down with network unavailable

$
0
0

hi Team,

We have 3 node windows cluster running SQL FCI. 2 node are at primary DC and 1 node is at DR . I am observing that node 3 is started showing as down in cluster manager and its networks as 'unavailable'. 2 nodes in primary are up and running with SQL FCI.

while running Cluster validation on node 3, i am observing that even IP configuration validation , detecting update level fails though node1 and node 2 are listed. Pls. find error below. not able to figure out what is going on with the cluster and showing node 3 down. Any pointers will be appreciated. Thanks

 Regards,

One Node in DR shown as Down status and network unavailable

$
0
0

Hi Team,

I have Windows Cluster with 3 nodes . Plan is to configure AG between Primary and DR datacenter. Node from DR site is showing as down status in cluster manager with all its network interfaces status as 'unavailable'. Our network team has recently configured firewalls between primary and DR sites, just wondering if this could be due to some port on firewall between site subnets?

Any pointers will be appreciated.

Regards,

Possible problems with Destroying a File Server cluster?

$
0
0

I have a 2008R2 file server cluster that is extremely critical.  It is actually running 2008R2 Storage server with SIS and Microsoft recommends re-inflating the volumes before moving to a new cluster.  The current cluster has one node that is so corrupted that we have disabled the cluster service on it, and the shared storage also has enough latency that it causes the other node to occasionally have problems when we try the sisadmin commands or sometimes just fails a datastore as offline.  We are thinking that because of these cluster issues with one bad node+bad performing storage that we are probably better to simplify everything and destroy the cluster and go back down to a single node and hopefully it will handle the storage concerns better.  Then we can focus on getting the data re-inflated and migrated to a new 2012R2 cluster.  That sounds a lot simpler than hoping the cluster becomes more stable when messing with cluster timeouts, etc.

I have never done this, but I am thinking that I just destroy the cluster, delete the cluster computer account, rename the current working node to the cluster name and use the cluster vip, and recreate the shares that it should work fine.  Most of our ownership on the datastores are set to the default 500 account and I am thinking that data should come across with all the same ACLS, etc.  Does anyone see any concerns with this plan or have any other advice?

Thanks,


Dave




add 3rd Node to Hyper-v cluster

$
0
0

I have a Windows 2012 Hyper-V Cluster with 2 Nodes. They have a connecting to SAN via FC and have 3 disk - Qourum and 2 CSV. I'm connected 3rd node to san and domain, install hyper-v role, updates, mpio. In disk management i see all 3 disk, status offline.

Whether it is necessary to online this disk or i can add node to cluster?


OPS Microsoft Server Medium: The system detected an address conflict for IP address 10.x.x.x with the system having network hardware address 00-xx-56-xx-xx-xx. Network operations on this system may be disrupted as a result.

$
0
0

Hi All,

Whenever my cluster resource move from one Node to other I am getting few error/warning message. Can anyone help me to understand why?

Cluster 2012R2
Witness File Share

Application Cluster: SQL Always On

Cluster resource 'ALXXXX_172.xx.xx.xx' of type 'IP Address' in clustered role 'ALxxxx' failed.
Based on the failure policies for the resource and role, the cluster service may try to bring the resource online on this node or move the group to another node of the cluster and then restart it.  Check the resource and group state using Failover Cluster Manager or the Get-ClusterResource Windows PowerShell cmdlet.

Cluster IP address resource 'ALxxxx_172.xx.xxx.xx' cannot be brought online because a duplicate IP address '172.xx.xx.xx' was detected on the network.  Please ensure all IP addresses are unique.

Cluster node 'Node04' was removed from the active failover cluster membership. The Cluster service on this node may have stopped. This could also be due to the node having lost communication with other active nodes in the failover cluster. Run the Validate a Configuration wizard to check your network configuration. If the condition persists, check for hardware or software errors related to the network adapters on this node. Also check for failures in any other network components to which the node is connected such as hubs, switches, or bridges.

The Cluster service is shutting down because quorum was lost. This could be due to the loss of network connectivity between some or all nodes in the cluster, or a failover of the witness disk.
Run the Validate a Configuration wizard to check your network configuration. If the condition persists, check for hardware or software errors related to the network adapter. Also check for failures in any other network components to which the node is connected such as hubs, switches, or bridges.

File share witness resource 'File Share Witness' failed to arbitrate for the file share '\\Filer006\QA_xxx_xxx'. Please ensure that file share '\\Filer006\QA_xxx_xxx' exists and is accessible by the cluster.

The cluster Resource Hosting Subsystem (RHS) process was terminated and will be restarted. This is typically associated with cluster health detection and recovery of a resource. Refer to the System event log to determine which resource and resource DLL is causing the issue.

I checked cluster logs also but not able to find anything here, My cluster is working fine even after getting this error/warning.

Quorum Disk on Passive Node.

$
0
0

Hi,

Since I am new to Windows cluster setup I have below doubts.Can someone, please provide information on below?

>>I have seen that the quorum disk is at passive and the remaining shared disks are at active node.In another server setup,we have shared disk and quorum disk on the active node.

Is that fine?

>>When we do failover I am sure that all shared disks move to the active node.I think quorum disk also moves to the active node.

Is that correct or it stays only on one node.

Thanks,


Cluster Validation - iSCSI port unreachable to UDP 3433

$
0
0

Currently I just upgrade my 4 production servers from Server 2012 to Server 2012 R2 and rebuild my Cluster with the same IP. But I encounter an issue that my cluster validation is failed because of 2 server's iscsi port is unreachable to other node. Below are the screenshot of the error. I have went through some articles and tried to solve it but still failed. Besides, I also turn of the windows firewall for 4 nodes for temporary testing.

Adding CAU role to Hyper-V cluster issue

$
0
0

Hi all,

I am trying to add CAU role to my five node Hyper-V cluster (Win2012 R2) by running following command:

Add-CauClusterRole -ClusterName HVC -Force -CauPluginName Microsoft.WindowsUpdatePlugin -MaxRetriesPerNode 3 -ConfigurationName MY_CAU -CauPluginArguments @{ 'IncludeRecommendedUpdates' = 'False' } -StartDate "29/11/2016 03:00:00" -DaysOfWeek 4 -WeeksOfMonth @(3) -VirtualComputerObjectName HV-CAU.mydomain.net -EnableFirewallRules;

and getting 

following error:

Add-CauClusterRole : The PowerShell SessionConfiguration "MY_CAU" cannot be found on cluster node "HV2".
At line:1 char:1
+ Add-CauClusterRole -ClusterName HVC -Force -CauPluginName Microsoft.WindowsUpda ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : InvalidOperation: (MS.Internal.Clu...ial credential):ClusterNodeValidationException) [Add-CauClu 
   sterRole], ClusterUpdateException
    + FullyQualifiedErrorId : EndPointNotFound,Microsoft.ClusterAwareUpdating.Commands.AddCauClusterRoleCommand
 
Add-CauClusterRole : Validation failed for adding CAU cluster role.
At line:1 char:1
+ Add-CauClusterRole -ClusterName HVC -Force -CauPluginName Microsoft.WindowsUpda ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : InvalidOperation: (:) [Add-CauClusterRole], ClusterUpdateException
    + FullyQualifiedErrorId : ClusterNodeValidationError,Microsoft.ClusterAwareUpdating.Commands.AddCauClusterRoleCommand

I have enabled PS remoting, winrm on HV2 and tried without pre-creating the HV-CAU computer account as well. I have disabledHV-CAU computer account as well.

I have give Full permission to HVC$ & HV-CAU accounts on the OU my Hyper-V hosts are residing.

What could be an issue?

Regards, Ilkin


Regards, Ilkin

Alive sanity check failed!, pending IO completed with status 1235 error in Windows Server 2008 R2 cluster node

$
0
0

Hi Guys,

 physical disk resource on the Windows Server 2008 R2 cluster node is getting failed and coming online after few second  This behavior occurs very frequently.


nic teaming live migration config

$
0
0

I have a new 2-node hyper-v cluster with the following config:

2x 10gb nic for VM/LM; dynamic LACP team; VM vlans 101-151; LM vlan 96

4x 1gb nic for VM/LM; switch independent team

1x 1gb nic for management

Connected by a 3-switch stack of Cisco Catalyst 3750-X

I am able to run VM's on the 10gb nic just fine, but I can't do live migrations over it. I have tried setting a static ip for the team and made sure they were all on vlan96. The hosts can ping the switch, but can't ping each other even though the firewall is off. I also tried forcing live migrations to happen over the 10gb nic in the failover cluster manager mmc.

My two port-channel setups are pretty basic (port-channel 1 & port-channel 2):

switchport trunk encapsulation dot1q
switchport trunk native vlan 96
switchport mode trunk
ip dhcp snooping trust

and then add the appropriate tengig interfaces to those groups.

I've checked the NIC Teaming console for both servers, and the team shows an OK status for the teams.

What am I missing to get this configuration working?

Have a redundant CA at a DR Site

$
0
0

Hi Guys

I have a Windows Server 2012 R2 Certification Authority at my production site. I need to have this role highly available with failover to my DR Site.

What is the best approach for this???

Viewing all 4519 articles
Browse latest View live


<script src="https://jsc.adskeeper.com/r/s/rssing.com.1596347.js" async> </script>