I have a handful of 2 node SQL 2008 R2 clusters. The nodes and cluster IP addresses all reside in the same subnet. Periodically we have issues where the active node in of one cluster cannot ping the active node of another cluster and cannot ping the cluster IP; but can ping the standby node IP. For example, NODE_A1, NODE_A2 are in CLUSTER_A; suppose NODE_A2 is active. NODE_B1 is the active node in CLUSTER_B; it can ping NODE_A1, but not the active NODE_A2 and not CLUSTER_A. By clearing the ARP table on NODE_B1 sometimes the ping responds to NODE_A2 and CLUSTER_A will be restored, but only temporarily. All these devices are connected to the same physical network switch.
I was wondering if having all the IP's, which can have the same MAC addresses behind them, on the same subnet may be causing issues with MAC tables not updating in a timely fashion? Is it considered a best practice to use different subnets? I'm more experienced with Exchange clusters where a dedicated heartbeat subnet is used for server to server communication, and the cluster IP would be on a different and user accessible subnet.