Chapter 19. CCD Configuration

Table of Contents
19.1. Install physical media
19.2. Configure Kernel Support
19.3. Disklabel each volume member of the CCD
19.4. Configure the CCD
19.5. Initialize the CCD device
19.6. Create a 4.4BSD/UFS filesystem on the new CCD device
19.7. Mount the filesystem

This chapter (CCD Configuration) has been contributed by Brian A. Seklecki <lavalamp@burghcom.com>

The CCD driver allows the user to "concatenate" several physical disks into one pseudo volume. CCD also lets you overcome a feature limitation in CMU RAIDFrame that does not allow you to RAID0 (file system spanning across disks) across disks of different geometry. CCD also allows for an "interleave" to improve disk performance with a gained space loss. This example will not cover that feature.

The steps required to setup a CCD are as follows:

  1. Install physical media

  2. Configure kernel support

  3. Disklabel each volume member of the CCD

  4. Configure the CCD conf file

  5. Initialize the CCD device

  6. Create a 4.4BSD/UFS filesystem on the new CCD device

  7. Mount the CDD filesystem

This example features a CCD setup on NetBSD/Sparc 1.5. The CCD will reside on 4 SCSI disks in a generic external Sun disk pack chassis connected to the external 50 ping SCSI port.

19.1. Install physical media

This step is at your own discretion, depending on your platform and the hardware at your disposal.

From my DMESG:

Disk #1:
  probe(esp0:0:0): max sync rate 10.00MB/s
  sd0 at scsibus0 target 0 lun 0: <SEAGATE, ST32430N SUN2.1G, 0444> SCSI2 0/direct fixed
  sd0: 2049 MB, 3992 cyl, 9 head, 116 sec, 512 bytes/sect x 4197405 sectors

Disk #2
  probe(esp0:1:0): max sync rate 10.00MB/s
  sd1 at scsibus0 target 1 lun 0: <SEAGATE, ST32430N SUN2.1G, 0444> SCSI2 0/direct fixed
  sd1: 2049 MB, 3992 cyl, 9 head, 116 sec, 512 bytes/sect x 4197405 sectors

Disk #3
  probe(esp0:2:0): max sync rate 10.00MB/s
  sd2 at scsibus0 target 2 lun 0: <SEAGATE, ST11200N SUN1.05, 9500> SCSI2 0/direct fixed
  sd2: 1005 MB, 1872 cyl, 15 head, 73 sec, 512 bytes/sect x 2059140 sectors

Disk #4
  probe(esp0:3:0): max sync rate 10.00MB/s
  sd3 at scsibus0 target 3 lun 0: <SEAGATE, ST11200N SUN1.05, 8808 > SCSI2 0
  sd3: 1005 MB, 1872 cyl, 15 head, 73 sec, 512 bytes/sect x 2059140 sectors    

19.2. Configure Kernel Support

The following kernel configuration directive is needed to provide CCD device support. It is enabled in the GENERIC kernel:

pseudo-device  ccd  4    # concatenated disk devices    

In my kernel config, I also hard code SCSI ID associations to /dev device entries to prevent bad things from happening:

sd0     at scsibus0 target 0 lun ?     
# SCSI disk drives
sd1     at scsibus0 target 1 lun ?     
# SCSI disk drives
sd2     at scsibus0 target 2 lun ?     
# SCSI disk drives
sd3     at scsibus0 target 3 lun ?     
# SCSI disk drives
sd4     at scsibus0 target 4 lun ?     
# SCSI disk drives
sd5     at scsibus0 target 5 lun ?     
# SCSI disk drives
sd6     at scsibus0 target 6 lun ?     
# SCSI disk drives    

19.3. Disklabel each volume member of the CCD

Each member disk of the CCD will need a special file system established. In this example, I will need to disklabel:

/dev/rsd0c
/dev/rsd1c
/dev/rsd2c
/dev/rsd3c    

Note: always remember to disklabel the character device, not the block device, in /dev/r{s,w}d*

Note: on all platforms, the cslice is symbolic of the entire NetBSD partition and is reserved.

You will probably want to remove any pre-existing disklabels on the disks in the CCD. This can be accomplished with the dd command:

# dd if=/dev/zero of=/dev/rsd0 bs=8k count=1
# dd if=/dev/zero of=/dev/rsd1 bs=8k count=1
# dd if=/dev/zero of=/dev/rsd2 bs=8k count=1
# dd if=/dev/zero of=/dev/rsd3 bs=8k count=1
    

The default disklabel for the disk will look similar to this:

# disklabel -r /dev/rsd0c
[...snip...]
bytes/sector: 512
sectors/track: 116
tracks/cylinder: 9
sectors/cylinder: 1044
cylinders: 3992
total sectors: 4197405
[..snip...]
3 partitions:
#        size   offset    fstype   [fsize bsize   cpg]
  c:  4197405       0     unused     1024  8192        # (Cyl.    0 - 4020*)    

You will need to create one "slice" on the NetBSD partition of the disk that consumes the entire partition. The slice must begin at least one cylinder offset from the beginning of the disk/partition to provide space for the special CCD disklabel. The offset should be 1x sectors/cylinder (see following note). Therefore, the "size" value should be "total sectors" minus 1x "sectors/cylinder".

Note: the offset of a slice of type "ccd" must be a multiple of the "sectors/cylinder" value.

Edit your disklabels accordingly. Be sure to specify the path to the character device, not the block device.

Note: be sure to export EDITOR=[path to your favorite editor] before editing the disklabels.

# disklabel -e /dev/rsd0c    

Note: the slice must be fstype ccd.

Because there will only be one slice on this partition, you can recycle the c slice (normally reserved for symbolic uses). Change your disklabel to the following:

3 partitions:
#        size   offset    fstype   [fsize bsize   cpg]
  c:  4196361     1044       ccd                       # (Cyl. 1 - 4020*)    

Optionally you can setup a slice other than c to use, simply adjust accordingly below:

3 partitions:
#        size   offset    fstype   [fsize bsize   cpg]
  a:  4196361     1044       ccd                       # (Cyl. 1 - 4020*)
  c:  4197405       0     unused     1024  8192        # (Cyl. 0 - 4020*)    

Be sure to write the label when you have completed. Disklabel will object to your disklabel and prompt you to re-edit if it does not pass it's sanity checks.

19.4. Configure the CCD

Once all disk are properly labeled, you will need to generate a configuration file. The configuration file resides in /etc by default. You may need to create a new one. Format:

#ccd    ileave    flags   component    devices    

Note: for the "ileave", if a value of zero is used then the disks are concatenated, but if you use a value equal to the "sectors/track" number the disks are interleaved.

Example in this case:

# more /etc/ccd.conf
ccd0  0  none /dev/sd0c /dev/sd1c /dev/sd2c /dev/sd3c    

Note: the CCD configuration file references the device file for the newly created CCD filesystems. Be sure not to reference the block device at this point, instead use the character device.

19.5. Initialize the CCD device

Once you are confident that your CCD configuration is sane, you can initialize the device using the ccdconfig command: Configure:

# ccdconfig -c -f /etc/ccd.conf    

Unconfigure:

# ccdconfig -u -f /etc/ccd.conf    

Initializing the CCD device will activate /dev entries: /dev/{,r}ccd#:

# ls -la  /dev/{,r}ccd0*
brw-r-----  1 root  operator   9, 0 Apr 28 21:35 /dev/ccd0a
brw-r-----  1 root  operator   9, 1 Apr 28 21:35 /dev/ccd0b
brw-r-----  1 root  operator   9, 2 May 12 00:10 /dev/ccd0c
brw-r-----  1 root  operator   9, 3 Apr 28 21:35 /dev/ccd0d
brw-r-----  1 root  operator   9, 4 Apr 28 21:35 /dev/ccd0e
brw-r-----  1 root  operator   9, 5 Apr 28 21:35 /dev/ccd0f
brw-r-----  1 root  operator   9, 6 Apr 28 21:35 /dev/ccd0g
brw-r-----  1 root  operator   9, 7 Apr 28 21:35 /dev/ccd0h
crw-r-----  1 root  operator  23, 0 Jun 12 20:40 /dev/rccd0a
crw-r-----  1 root  operator  23, 1 Apr 28 21:35 /dev/rccd0b
crw-r-----  1 root  operator  23, 2 Jun 12 20:58 /dev/rccd0c
crw-r-----  1 root  operator  23, 3 Apr 28 21:35 /dev/rccd0d
crw-r-----  1 root  operator  23, 4 Apr 28 21:35 /dev/rccd0e
crw-r-----  1 root  operator  23, 5 Apr 28 21:35 /dev/rccd0f
crw-r-----  1 root  operator  23, 6 Apr 28 21:35 /dev/rccd0g
crw-r-----  1 root  operator  23, 7 Apr 28 21:35 /dev/rccd0h    

19.6. Create a 4.4BSD/UFS filesystem on the new CCD device

You may now disklabel the new virtual disk device associated with your CCD. Be sure to use the character device:

# disklabel -e /dev/rccd0c    

Once again, there will be only one slice, so you may either recycle the c slice or create a separate slice for use.

# disklabel -r /dev/rccd0c
# /dev/rccd0c:
type: ccd
disk: ccd
label: default label
flags:
bytes/sector: 512
sectors/track: 2048
tracks/cylinder: 1
sectors/cylinder: 2048
cylinders: 6107
total sectors: 12508812
rpm: 3600
interleave: 1
trackskew: 0
cylinderskew: 0
headswitch: 0           # microseconds
track-to-track seek: 0  # microseconds
drivedata: 0
#        size   offset    fstype   [fsize bsize   cpg]
  c: 12508812        0    4.2BSD     1024  8192    16  # (Cyl. 0 - 6107*)    

The filesystem will then need formatted:

# newfs /dev/rccd0c
Warning: 372 sector(s) in last cylinder unallocated
/dev/rccd0c:    12508812 sectors in 6108 cylinders of 1 tracks, 2048 sectors
        6107.8MB in 382 cyl groups (16 c/g, 16.00MB/g, 3968 i/g)

super-block backups (for fsck -b #) at:
[...]    

19.7. Mount the filesystem

Once you have a created a file system on the CCD device, you can can mount the file system against an mount point on your system. Be sure to mount the slice labeled type ffs or 4.4BSD:

# mount /dev/ccd0c /mnt    

Then:

# export BLOCKSIZE=1024; df
Filesystem  1K-blocks     Used   Avail Capacity  Mounted on
/dev/sd6a      376155   320290   37057    89%    /
/dev/ccd0c    6058800       1  5755859     0%    /mnt    

Congratulations, you now have a working CCD. Please consult the rest of the manual for instructions on how to initialize the device at boot-time via RC. For more detail on any of the commands here, please see their respective man pages.