Posts Tagged ‘multipath’

Configuring ocfs2 using multipath for clustering oracle databases

Wednesday, December 16th, 2009

Here is one method for using multipath with ocfs2 on linux to create database volumes on a cluster. The reason for using multipath with ocfs2 is to bind the disks to a file system using the WWID numbers instead of allowing the operating system to assign device names. Device names can change, but WWID will not change.

  1. Make sure you have ocfs2 and multipath installed on all cluster nodes.
  2. Configure ocfs2 on the first node.
  3. Export the configuration to all nodes in the cluster. You can use ocfs2console to make life easier for your migration. You can check the nodes in the /etc/ocfs2/cluster.conf file to ensure all nodes are listed.
  4. NOTE: At this point you are finished with OCFS and be working on the local systems.
  5. Import your luns. Make sure they are shared between both or all nodes in the cluster.
  6. Set boot.multipath, multipathd, o2cb and ocfs2 to start at boot on all nodes.
  7. Configure the devices in the multipath.conf file. You can find the device names and channels using the lsscsi command. You can find the WWID with the following command: udevinfo -q symlink -n /dev/<name of device>.
  8. After the devices are configured in multipath.conf run the following: multipath -F and multipath -v2. You should see your devices under /dev/mapper/.
  9. In order for ocfs2 to see the devices they need an ocfs2 file system. You can run the following foreach loop in Kshell to accomplish this:
          cd /dev/mapper
          foreach n (`ls db*`)
          sudo mkfs -t ocfs2 -N <number of nodes>  -L $n $n
          end
  10. Create the mount points.
  11. Add the devices to /etc/fstab. Here is an example:
        /dev/mapper/<device name>   <mount point>   ocfs2  _netdev  0 0

    Note: the arguments for the file systems will vary. Oracle recommends _netdev,datavolume,noinitr for Oracle data volumes like raw, db, u and l. Regular file systems only get _netdev

  12. Run: sudo mount -a. This will mount the volumes locally and since there is an ocfs2 file system on them OCFS will pick them up.
  13. Change the permissions and ownership on the mounted volumes. Do not do this ahead of time, because the file system you created will over-mount the local directory.
  14. Copy multipath.conf to the other nodes and run multipath -F then multipath -v2. If you have any problems make sure multipathd is started.
  15. Create the mount points on each node.
  16. Add the entries to the local /etc/fstab.
  17. Run sudo mount -a.
  18. Do this for every node in the cluster.

Walla, you are finished and have your OCFS and multipath working.  If you have any comments of suggestions please post them. As always, please check out the sponsors on this site.