Having done a handful of Exadata implementations, there’s always been one piece of the configuration that’s bothered me more than anything else. In the process of ordering an Exadata, Oracle sends the customer a “Configuration Worksheet” that asks questions about how the system should be configured. It’s standard stuff: hostnames, DNS and NTP servers, UID and GID for the oracle/dba/oinstall (that’s another sore spot) accounts, and IP addresses for the various interfaces. The worksheet comes as a nifty PDF that the customer can modify to suit the needs of the Exadata system.
Unfortunately, the PDF does not allow the customer to modify the IP range used for the IB network. The only option from this form is to use the network 192.168.8.0/22 with the hosts using 192.168.10.1 – 192.168.10.22 (for a full rack). Why the /22 you might ask? Oracle recommends using a subnet of 255.255.252.0 so that multiple Exadata systems can be connected, along with an Exalogic, and whatever other products they have down the line that will connect with Exadata on the IB network. It would be nice if Oracle would allow customers to define this network range themselves, instead of sticking everybody in the 192.168.8.0/22 network. Some say that it won’t be a problem, because the interconnect is non-routable, but I disagree. Find out why after the jump