#EDIR DATA#
LANG=CRAID System Administration GuideDocument Number 007-2113-002ContributorsWritten by Susan EllisIllustrated by Dan YoungEdited by Nancy SchweigerProduction by Gloria AckleyEngineering contributions by Curtis Anderson, Jim Oliver© Copyright 1993, Silicon Graphics, Inc.name='mdash' font=symbol charset=fontspecific code=190 
			descr='[mdash]' All Rights ReservedThis document contains proprietary and confidential information of Silicon Graphics, Inc. The contents of this document may not be disclosed to third parties, copied, or duplicated in any form, in whole or in part, without the prior written permission of Silicon Graphics, Inc.Restricted Rights LegendUse, duplication, or disclosure of the technical data contained in this document by the Government is subject to restrictions as set forth in subdivision (c) (1) (ii) of the Rights in Technical Data and Computer Software clause at DFARS 52.227-7013 and/or in similar or successor clauses in the FAR, or in the DOD or NASA FAR Supplement. Unpublished rights are reserved under the Copyright Laws of the United States. Contractor/manufacturer is Silicon Graphics, Inc., 2011 N. Shoreline Blvd., Mountain View, CA 94039-7311.RAID System Administration GuideDocument Number 007-2113-002Silicon Graphics, Inc.Mountain View, CaliforniaSilicon Graphics is a registered trademark and IRIX, Onyx, and CHALLENGE are trademarks of Silicon Graphics, Inc. Post-it is a trademark of 3M Corporation.ID="41966"About This GuideThe RAID System Administration Guide describes configuration, maintenance, and error recovery procedures for Silicon Graphics'® RAID (Redundant Array of Inexpensive Disks) product. The IRIXÔ 5.0.1 version of the RAID device driver and the RAID administrative utility, raid(1M), are documented.LBL="" HELPID=""How to Use This GuideRead "
IDREF="41966" TYPE="TEXT"About This Guide
" to learn what hardware and software is documented in this guide, what the prerequisites are for using RAID, about the documentation conventions in this guide, about product support for Silicon Graphics products, and where to go for more information about RAID.Read IDREF="18849" TYPE="TITLE"Chapter 1, "Introduction to RAID,"
 to learn RAID basics: what it is, the similarities and differences between RAID and non-RAID disks, the configuration options, the RAID LEDs, and RAID failure prediction and recovery features.Turn to IDREF="90820" TYPE="TITLE"Chapter 2, "Formatting a RAID,"
 to learn how to format a new RAID and reformat an existing RAID.IDREF="83934" TYPE="TITLE"Chapter 3, "Routine Maintenance Tasks," covers a variety of tasks that system administrators occasionally may need to perform during normal operation of a RAID.IDREF="65029" TYPE="TITLE"Chapter 4, "Recovering after a Disk Warning or Failure," describes how to replace a disk drive in a RAID unit after a predictive warning or a failure. It includes instructions for rebuilding a new disk drive with the same data as the replaced disk drive.IDREF="26656" TYPE="TITLE"Chapter 5, "Error Messages," contains lists of RAID error messages and suggested recovery procedures.IDREF="17323" TYPE="TITLE"Appendix A, "Programming Hints," contains tips for application developers on getting maximum I/O performance from a RAID.Many readers will not need this entire guide. The chapters you need depend upon your role:System AdministratorSystem administrators are the primary audience of this guide. The entire guide except for IDREF="17323" TYPE="TITLE"Appendix A
 explains configuration, routine maintenance, and error recovery procedures that are likely to be performed by system administrators.System Support EngineerSystem Support Engineers can find the procedure for formatting a RAID in IDREF="90820" TYPE="TITLE"Chapter 2
.Application DeveloperApplication developers should read IDREF="18849" TYPE="TITLE"Chapter 1
 to learn about the RAID configuration options that affect the performance of their applications and IDREF="17323" TYPE="TITLE"Appendix A
 for information about optimizing I/O in applications that read and write data on RAIDs.This guide assumes that you know how to become superuser, get disk controller and unit numbers from hinv(1M), and how to make a file system on a disk with mkfs(1M).LBL="" HELPID=""Hardware and Software RequirementsRAID is supported on workstations and servers that have fast and wide SCSI buses. This includes CHALLENGEÔ L, CHALLENGE XL, POWER CHALLENGE L, POWER CHALLENGE XL, OnyxÔ Deskside, and Onyx Rack.ID="intro1"This guide applies only to RAID units manufactured by Silicon Graphics. The commands and procedures documented in this guide cannot be used when disk drives, RAID controllers, or RAID unit enclosures purchased from vendors other than Silicon Graphics are used, even when those parts appear to be identical to the ones used in RAID units purchased from Silicon Graphics.Systems with one or more RAIDs must have a non-RAID system disk.RAID requires IRIX Release 5.0.1 or later. The subsystem ID="intro2"ID="intro3"eoe1.sw.unix contains all of the files needed for RAID.LBL="" HELPID=""Documentation ConventionsTo distinguish between physical disks and what appears to IRIX to be a disk, this guide uses the following terms:diskID="intro4"one or more physical disk drives that appear to IRIX as a single diskdisk driveID="intro5"a single disk hardware unitRAIDID="intro6"the array of five disk drives that to IRIX appear as a single disk of type RAIDRAID unitID="intro7"the RAID hardware including five disk drives, a controller, a backplane, and an enclosureThree terms are used to describe the states of RAIDs and/or their disk drives:operationalID="intro8"The RAID unit is operating normally.failedID="intro9"The RAID unit or a disk drive is not responding to commands, or it is responding but is not operating properly.downID="intro10"The RAID unit or a disk drive has failed or is not operating because it has been manually shut down with the raid   ­d command.This guide uses these font conventions:ID="intro11"italicsItalics are used for command and manual page names, file names, variables, and the names of inst(1M) products and subsystems. typewriterTypewriter font is used for examples of command output that is displayed in windows on your monitor.bold typewriterBold typewriter is used for commands and text that you are to type literally.LBL="" HELPID=""Product SupportSilicon Graphics provides a comprehensive product support and maintenance program for its products. For further information in the United States and Canada, contact the Technical Assistance Center at 1-800-800-4SGI. Elsewhere, contact your local service provider.ID="intro12"LBL="" HELPID=""ID="67525"For More InformationFor more information about RAID and disk management on IRIX, see the following sources:The University of California at Berkeley report on RAID:ID="intro13"ID="intro14"D. Patterson, G. Garth, R. Katz, "A Case for Redundant Arrays of Inexpensive Disks (RAID)," University of California, Berkeley, Report No. UCB/CSD/87/391, December, 1987.The IRIX Advanced Site and Server Administration GuideIRIX manual pages on the raid administrative program and the RAID device driver:raid(1M), usraid(7)IRIX manual pages on various disk information and management tools:ID="intro15"dvhtool(1M), fx(1M), hinv(1M), mkfs(1M), mklv(1M), sar(1)LBL="1"ID="18849"Introduction to RAIDSilicon Graphics' RAID (Redundant Array of Inexpensive Disks) product is a large capacity disk that provides protection against media failure. RAID stores parity information across a group of disk drives so that if a single disk drive in the group fails, users' data can be recovered.This chapter briefly explains the characteristics of RAID and the features of the Silicon Graphics implementation of RAID. The sections in this chapter are:IDREF="42308" TYPE="TITLE"Section 1.1, "RAID Levels"IDREF="75261" TYPE="TITLE"Section 1.2, "RAID Stripe Depth"IDREF="50966" TYPE="TITLE"Section 1.3, "RAID and IRIX"IDREF="97525" TYPE="TITLE"Section 1.4, "RAID Configuration Options"IDREF="30325" TYPE="TITLE"Section 1.5, "RAID LEDs"IDREF="96898" TYPE="TITLE"Section 1.6, "RAID Failure Prediction"IDREF="37676" TYPE="TITLE"Section 1.7, "RAID Failure Recovery"LBL="1.1" HELPID=""ID="42308"RAID LevelsRAID was first defined by Patterson, Garth, and Katz of the University of California, Berkeley, in their 1987 paper, "A Case for Redundant Arrays of Inexpensive Disks (RAID)" (see "
ID="ch11"ID="ch12"IDREF="67525" TYPE="TEXT"For More Information
" in "
IDREF="41966" TYPE="TEXT"About This Guide
"). Silicon Graphics' RAID product implements these RAID levels:RAID 3, striping with a single parity disk per groupRAID 5, striping with parity spread over all disksSilicon Graphics' RAID 3 is shown in ID="ch13"ID="ch14"ID="ch15"ID="ch16"ID="ch17"ID="ch18"ID="ch19"IDREF="78700" TYPE="GRAPHIC"Figure 1-1
. It is a "4+1" implementation: data is spread across four disk drives and a fifth "check" disk drive stores parity information. A parity bit on the check disk drive is calculated by an exclusive-OR (XOR) of the corresponding bits on the four data disk drives: for each bit of data, if there is an even number of 1's on the data disk drives, the check disk drive contains a 0; if the number of 1's is odd, the check disk drive contains a 1.LBL="1-1" FILE="ch1.cgm" POSITION="INLINE" SCALE="FALSE"LBL="1-1"Figure 1-1 
ID="78700"Silicon Graphics' RAID 3 (Stripe Depth of 4)ID="ch110"If a data disk drive fails, the information on this disk drive can be reconstructed by recalculating the parity of the remaining good data disk drives and comparing that parity bit-by-bit to the parity bits on the check disk drive. For each bit, if the parities agree, the failed bit was a 0, otherwise it was a 1. If the check disk drive fails, its data can be reconstructed by recalculating parity from the data disk drives.Silicon Graphics' RAID 5, shown in ID="ch111"ID="ch112"ID="ch113"IDREF="28498" TYPE="GRAPHIC"Figure 1-2
, is similar to RAID 3, except that the parity information is distributed across all of the disk drives rather than just a single disk drive. This distribution of parity data gives better performance for multiple writes in parallel (in RAID 3, multiple writes must be sequential because all writes require a write to the check disk drive). Reconstruction of a failed RAID 5 disk drive is done in the same manner as for a RAID 3 disk drive.LBL="1-2" FILE="ch1.cgm2" POSITION="INLINE" SCALE="FALSE"LBL="1-2"Figure 1-2 
ID="28498"Silicon Graphics' RAID 5 (Stripe Depth of 4)ID="ch114"LBL="1.2" HELPID=""ID="75261"RAID Stripe DepthID="ch115"ID="ch116"ID="ch117"ID="ch118"IDREF="78700" TYPE="GRAPHIC"Figure 1-1
 and IDREF="28498" TYPE="GRAPHIC"Figure 1-2
 show the minimum "stripe depth" of 4. Each numbered slice of a disk drive is one 512-byte sector, and each disk drive in this example contains groups of four sequentially numbered sectors. IDREF="31165" TYPE="GRAPHIC"Figure 1-3
 and IDREF="75525" TYPE="GRAPHIC"Figure 1-4
 show a stripe depth of 8 for RAID 3 and RAID 5 respectively. The stripe size (the total number of sectors in each stripe) is the stripe depth times 4 (because each stripe uses four of the five disk drives for user data). The minimum stripe depth for any RAID is 4 and the maximum is 64.The stripe width, depth, and size are analogous to the width, length, and area of a sheet of paper. For Silicon Graphics RAID, the width is always 4, and the depth is user setable. Since the total stripe size is always the depth times 4, this guide and the ID="ch119"ID="ch120"ID="ch121"raid(1M) command use only the stripe depth rather than the more common "stripe size." LBL="1-3" FILE="ch1.cgm3" POSITION="INLINE" SCALE="FALSE"LBL="1-3"Figure 1-3 
ID="31165"Silicon Graphics' RAID 3 with Stripe Depth of 8ID="ch122"LBL="1-4" FILE="ch1.cgm4" POSITION="INLINE" SCALE="FALSE"LBL="1-4"Figure 1-4 
ID="75525"Silicon Graphics' RAID 5 with Stripe Depth of 8LBL="1.3" HELPID=""ID="50966"RAID and IRIXEach RAID is viewed as a single 8 GB SCSI-2 compatible disk by the IRIX operating system. The RAID appears as a single disk at the PROM level also, but the PROMs cannot determine that it is a RAID. When a RAID is installed on a workstation or server, ID="ch123"ID="ch124"hinv from IRIX displays a single line of output for each RAID. An example is:Disk drive: unit 5 on SCSI controller 0: RAIDA RAID is the same as a non-RAID disk in these ways:ID="ch125"ID="ch126"The default sector size is 512 bytes.The RAID has a volume header that is written just like any other volume header, and it uses the standard partition tables.You can use ID="ch127"mkfs(1M) and fx(1M) (IRIX fx only, not standalone fx) as usual.Filesystems on a RAID are mounted in ID="ch128"/etc/fstab.Filesystems on a RAID can be exported for NFS mounting. (As with all NFS mounting, hard mounting a RAID filesystem is more reliable than soft mounting. It is particularly recommended for RAIDs.)ID="ch129"Block and character device access is supported.The logical volume disk driver, lv(7M), supports RAIDs.The IRIS Volume Manager supports RAIDs.A RAID is different from a non-RAID disk in these ways:ID="ch130"A RAID must be formatted by the RAID administration program raid(1M).A RAID cannot be used as the system disk: it cannot contain the ID="ch131"/ (root) and /usr filesystems or the primary swap space. Third party disk drives cannot be used in a RAID.The RAID device driver, ID="ch132"usraid(7M), is used rather than other device drivers, such as dks(7M) and jag(7M).The sector size cannot be changed.Standalone fx is not supported.Add_disk(1) cannot be used.LBL="1.4" HELPID=""ID="97525"RAID Configuration OptionsRAID configuration has two components:Configuring RAIDs on SCSI channels at hardware installation timeID="ch133"Configuring a RAID at the time it is formattedTo perform this configuration, you must decide:ID="ch134"ID="ch135"The RAID levelThe stripe depthThe number of RAIDs per SCSI channelThe next three sections explain how to choose the configuration options that will maximize your applications' I/O performance. You must choose the level of each RAID and the number of RAIDs per SCSI channel before RAID units can be installed in a system.ID="ch136"LBL="1.4.1" HELPID=""ID="74222"RAID Level and Stripe DepthThe number of users using a RAID, their applications, and the I/O characteristics of the applications that access the RAID all influence the choice of RAID 3 or RAID 5 and the stripe depth. For four combinations of RAID level and stripe depth, ID="ch137"ID="ch138"ID="ch139"ID="ch140"IDREF="68114" TYPE="TABLE"Table 1-1
 shows the number of users, applications, I/O stream, and transfer size that is best suited for each combination.The best stripe depth for maximum performance of a RAID depends upon the typical read/write size of the applications that access the RAID. Optimally, each read should access just one of the five disk drives in a RAID 5. For example, if the typical read is 2 KB, a good stripe depth would be 4: four sectors of 512 bytes each is 2 KB. The entire 2 KB can be read from a single disk drive. COLUMNS="6"LEFT="0" WIDTH="32"RAID 
LevelLEFT="40" WIDTH="36"Stripe 
DepthLEFT="85" WIDTH="39"Typical 
Number 
of UsersLEFT="130" WIDTH="72"Typical 
ApplicationsLEFT="210" WIDTH="97"Typical I/O StreamLEFT="315" WIDTH="356"Typical 
Transfer SizeLEFT="0" WIDTH="32"3LEFT="40" WIDTH="36"smallLEFT="85" WIDTH="39"LEFT="130" WIDTH="72"not usefulLEFT="210" WIDTH="97"LEFT="315" WIDTH="356"LEFT="0" WIDTH="32"3LEFT="40" WIDTH="36"largeLEFT="85" WIDTH="39"lowLEFT="130" WIDTH="72"supercomputer 
applications, 
graphicsLEFT="210" WIDTH="97"single stream of large 
requests (small 
number of requests per 
second and a massive 
amount of information 
in each request)LEFT="315" WIDTH="356"1 MB or more 
is optimal; the 
range is 64 KB 
and upLEFT="0" WIDTH="32"5LEFT="40" WIDTH="36"smallLEFT="85" WIDTH="39"highLEFT="130" WIDTH="72"databases, NFS, 
file servingLEFT="210" WIDTH="97"multiple streams of 
small requests (a large 
number of requests for 
a small amount of 
information each time)LEFT="315" WIDTH="356"8 KB or less is 
optimal; the 
range is .5 KB 
to 64 KBLEFT="0" WIDTH="32"5LEFT="40" WIDTH="36"largeLEFT="85" WIDTH="39"lowLEFT="130" WIDTH="72"supercomputer 
applications, 
graphicsLEFT="210" WIDTH="97"single stream of large 
requests (small 
number of requests per 
second and a massive 
amount of information 
in each request), 
mostly reads with few 
writesLEFT="315" WIDTH="356"1 MB or more 
is optimal; the 
range is 64 KB 
and upLBL="1-1"Table 1-1 
ID="68114"RAID 3 and RAID 5 CharacteristicsID="ch141"ID="ch142"ID="ch143"ID="ch144"A rule of thumb for choosing the stripe depth is to choose a stripe depth that accommodates the typical transfer size or is slightly larger. When the typical transfer size is unknown, a stripe depth of 32 is a good choice. Larger stripe depths are better for the controller because it is more efficient to read contiguous sectors from a single disk drive in a single read operation than many reads of small numbers of sectors from several disk drives.ID="ch145"The RAID level and stripe depth are specified when formatting a RAID (see IDREF="90820" TYPE="TITLE"Chapter 2, "Formatting a RAID"
). LBL="1.4.2" HELPID=""How Many RAID Units per SCSI Channel?Because of packaging and cabling limitations, a maximum of 12 SCSI devices are supported on each SCSI channel (each RAID unit is counted as a single SCSI device). However, I/O performance can be affected by putting too many RAID units on a single SCSI channel. ID="ch146"ID="ch147"ID="ch148"ID="ch149"You can use these rules of thumb when determining how many RAID units to attach to a channel:ID="ch150"ID="ch151"ID="ch152"The rated capacity of a channel is 20 MB per second, but because of overhead, assume that the capacity is 18 MB per second.ID="ch153"A RAID 3 at maximum throughput uses most of the capacity of a channel.A RAID 5 at maximum throughput uses about one quarter of the capacity of a channel. For RAID 5, the sizes of the transfers and the number per second determine how many transfers (and therefore RAIDs) can fill the capacity of the channel.A non-RAID disk at maximum throughput uses 3 MB to 4 MB per second.Putting slow devices such as tape drives and CD-ROM drives on a channel with one or more RAID units can adversely impact the performance of the RAIDs. In general, RAID units should be put on channels with other RAID units and non-RAID disk drives only.RAID units are attached to channels at the time of hardware installation. Changing the configuration, for example, because you determine that there are too many RAID units formatted as RAID 5 on a single channel, requires a service call by your support provider.ID="ch154"The ID="ch155"ID="ch156"hinv(1M) command can be used to figure out how many RAID units are on each channel. Each controller listed in hinv output drives one channel. Counting the number of units listed for a controller tells you how many devices are on the channel for that controller. The hinv output for a controller with 3 RAID units and 5 other disk drives on its channel looks like this:Integral SCSI controller 4: Version WD33C95A
Disk drive: unit 15 on SCSI controller 4: RAID
Disk drive: unit 14 on SCSI controller 4: RAID
Disk drive: unit 13 on SCSI controller 4: RAID
Disk drive: unit 5 on SCSI controller 4
Disk drive: unit 4 on SCSI controller 4
Disk drive: unit 3 on SCSI controller 4
Disk drive: unit 2 on SCSI controller 4
Disk drive: unit 1 on SCSI controller 4LBL="1.5" HELPID=""ID="30325"RAID LEDsSeveral sets of LEDs (Light Emitting Diodes) in the RAID unit display status information. This section contains diagrams that show the locations of these LEDs and tables that list their meanings.LBL="1.5.1" HELPID=""ID="98626"RAID Controller LEDsAn array of eight LEDs on the front edge of the RAID controller (inside the RAID unit) displays status information for each of the five disk drives and for the RAID as a whole. These LEDs are shown in ID="ch157"ID="ch158"ID="ch159"ID="ch160"ID="ch161"IDREF="64824" TYPE="GRAPHIC"Figure 1-5
 and their meanings are listed in IDREF="66069" TYPE="TABLE"Table 1-2
. The table assumes that the RAID unit is powered on. The term "failed" means that the RAID unit or a disk drive is either not responding to commands or is responding but not operating properly. The term "down" means that the RAID unit or a disk drive has failed or it is not operating because it has been manually shut down with the raid   ­d command.LBL="1-5" FILE="1-5.RAID.controller.LEDs.ai" POSITION="INLINE" SCALE="FALSE"LBL="1-5"Figure 1-5 
ID="64824"RAID Controller LEDsID="ch162"COLUMNS="4"LEFT="0" WIDTH="28"LED LEFT="35" WIDTH="100"Lit MeansLEFT="140" WIDTH="111"Off MeansLEFT="260" WIDTH="356"Flashing MeansLEFT="0" WIDTH="28"1LEFT="35" WIDTH="100"The RAID is processing 
commands.LEFT="140" WIDTH="111"The RAID is not 
processing commands.LEFT="260" WIDTH="356"The RAID is processing 
commands.LEFT="0" WIDTH="28"2LEFT="35" WIDTH="100"The RAID has failed.LEFT="140" WIDTH="111"The RAID has failed.LEFT="260" WIDTH="356"The RAID is operating 
normally (note that the 
on/off cycle can be as 
long as 10 seconds).LEFT="0" WIDTH="28"3LEFT="35" WIDTH="100"A maintenance 
command such as 
rebuilding a disk drive 
or checking parity is 
active.LEFT="140" WIDTH="111"The RAID is operating 
normally.LEFT="260" WIDTH="356"A maintenance 
command such as 
rebuilding a disk drive 
or checking parity is 
active.LEFT="0" WIDTH="28"4LEFT="35" WIDTH="100"Disk drive 0 is down.LEFT="140" WIDTH="111"Disk drive 0 is operational.LEFT="260" WIDTH="356"N/ALEFT="0" WIDTH="28"5LEFT="35" WIDTH="100"Disk drive 1 is down.LEFT="140" WIDTH="111"Disk drive 1 is operational.LEFT="260" WIDTH="356"N/ALEFT="0" WIDTH="28"6LEFT="35" WIDTH="100"Disk drive 2 is down.LEFT="140" WIDTH="111"Disk drive 2 is operational.LEFT="260" WIDTH="356"N/ALEFT="0" WIDTH="28"7LEFT="35" WIDTH="100"Disk drive 3 is down.LEFT="140" WIDTH="111"Disk drive 3 is operational.LEFT="260" WIDTH="356"N/ALEFT="0" WIDTH="28"8LEFT="35" WIDTH="100"Disk drive 4 is down.LEFT="140" WIDTH="111"Disk drive 4 is operational.LEFT="260" WIDTH="356"N/ALBL="1-2"Table 1-2 
ID="66069"RAID Controller LEDsID="ch163"LBL="1.5.2" HELPID=""ID="99206"RAID Disk Drive LEDsID="ch164"ID="ch165"ID="ch166"ID="ch167"IDREF="80448" TYPE="GRAPHIC"Figure 1-6
 shows the fronts of two RAID disk drives. In the figure, they are shown rotated 90 degrees counterclockwise from their position when installed in the RAID unit. See also IDREF="17672" TYPE="GRAPHIC"Figure 4-2
.An amber LED behind the bezel of the disk drive lights when the disk drive is down (failed or shut down with raid   ­d). When this LED is lit, the LED on the edge of the RAID controller that corresponds to this disk drive will be lit also (see IDREF="98626" TYPE="TITLE"Section 1.5.1
).The green LED on the front of each disk drive is lit when it is servicing a command.Disk drives in the RAID unit are numbered 0 to 4, left to right. The number of each disk drive displayed on its front must match its drive number. The push buttons above and below the number are used to increase or decrease it.ID="ch168"ID="ch169"LBL="1-6" FILE="1-6.disk.drive.front.ai" POSITION="INLINE" SCALE="FALSE"LBL="1-6"Figure 1-6 
ID="80448"Disk Drive FrontID="ch170"LBL="1.6" HELPID=""ID="96898"RAID Failure PredictionThe 2.0 GB disk drives used in RAIDs have a sophisticated predictive failure analysis (PFA) feature. They use internal diagnostics and device information to attempt to give at least 24 hours' notice before they fail.ID="ch171"ID="ch172"ID="ch173"ID="ch174"ID="ch175"ID="ch176"PFA information is written to the system console and to the file /var/adm/SYSLOG when:A disk drive predicts that it will fail within 24 hours.The system is powered on and a disk drive has predicted its failure.The raid command is given with the ­L option.An example of a PFA message is:disk n is predicting failure, replace as soon as possible IDREF="26656" TYPE="TITLE"Chapter 5, "Error Messages," contains a complete list of PFA messages. IDREF="65029" TYPE="TITLE"Chapter 4, "Recovering after a Disk Warning or Failure,"
 explains what to do if you get a PFA message.LBL="1.7" HELPID=""ID="37676"RAID Failure RecoveryWhen a single disk drive in a RAID unit fails, the data on the failed disk drive can be reconstructed from the remaining disk drives, while IRIX and applications accessing the RAID continue to operate. However, if two disk drives fail or if a disk drive and the controller fail, the data cannot be reconstructed, and IRIX and applications can no longer access the RAID. Backup tapes should be made for data on a RAID just as on any other type of disk.ID="ch177"ID="ch178"ID="ch179"ID="ch180"With RAID, you can "hot plug" a replacement disk drive: a disk drive can be removed from the RAID unit and a replacement inserted while IRIX is running and applications are accessing the RAID. See ID="ch181"ID="ch182"IDREF="65029" TYPE="TITLE"Chapter 4, "Recovering after a Disk Warning or Failure,"
 for more information.When a disk drive has failed, removing any disk drive other than the failed disk drive will probably result in the loss of all data on the RAID.To reduce the risk of pulling the wrong disk drive when a disk has returned a PFA warning, you can use the ­d option of the raid command to mark the disk down before pulling it. This will light the amber LED in the disk drive (see IDREF="80448" TYPE="GRAPHIC"Figure 1-6
) and the LED on the controller that corresponds to the disk drive (see IDREF="64824" TYPE="GRAPHIC"Figure 1-5
) to give a clear indication of which drive must be pulled. LBL="2"ID="90820"Formatting a RAIDWhen you format a RAID, you specify the addressing order of the sectors in the RAID. Formatting is required when a RAID is first installed on a workstation or server and when you want to change the RAID level, the stripe depth, or the number of sectors of each disk drive that are used. You may want to make changes because you are doing performance tuning or because applications with different I/O characteristics are now using the RAID.ID="ch21"ID="ch22"All data on a RAID is lost when you reformat it. Be sure to make a backup of all files that are located on the RAID before reformatting it with the ID="ch23"raid(1M) command.Before formatting a RAID, be sure you know the answers to these questions:ID="ch24"Which RAID level: RAID 3 or RAID 5? (See IDREF="74222" TYPE="TITLE"Section 1.4.1, "RAID Level and Stripe Depth."
)What stripe depth? (See IDREF="74222" TYPE="TITLE"Section 1.4.1
.)How many sectors of the RAID will be used? The default is the entire usable portion of each disk drive. Choose fewer sectors (leaving some portion of each disk drive unusable) when doing demonstrations in order to reduce the formatting time.ID="ch25"Follow these steps to format or reformat a RAID:Get the controller and unit numbers of the RAID.If you are reformatting the RAID, save all data on the RAID on tape.Become superuser.If you are reformatting the RAID, unmount any filesystems on the RAID.Run ID="ch26"MAKEDEV(1M) to create the RAID device files:cd /dev   
./MAKEDEV radIf you want to, you can check the health and connectivity of the RAID with fx(1M): fx "rad(controller,unit)"fx performs a controller test and displays a menu. At the prompt, exit fx:fx> exit   ID="21101"Give this ID="ch27"ID="ch28"ID="ch29"ID="ch210"ID="ch211"raid command to format or reformat the RAID:raid -level   -s   depth   -S   sectors   /dev/rdsk/radcontrollerdunitvhwhere:levelis the RAID level: 3 or 5ID="ch212"depthis the stripe depth, and depth must be a multiple of 4 between 4 and 64sectorsis the number of sectors per RAID to format (maximum 16777216)ID="ch213"controlleris the controller number (from hinv(1M))unitis the unit number (from hinv)­s   depth and ­S   sectors are optional. If these options are not specified, the entire RAID is formatted and the stripe depth is 32. Formatting the entire RAID takes 15­20 minutes.Invoke ID="ch214"fx in expert mode and put a volume header on the RAID using these commands (shown with example output for controller = 4 and unit = 8):fx -x "rad(controller,drive)"
fx version 5.0, Apr 19, 1993
...opening rad(4,8,)
...controller test...OK
fx: Warning - invalid label on disk, ignored
Can't get drive geometry, assuming 64 sectors/track
Can't get drive geometry, assuming 8 heads
Unable to get device geometry, assuming default
Scsi drive type == ULTRA U144 RAID SGI 1.0
warning: can't read sgilabel on disk
creating new sgilabel
...creating default bootinfo
Can't get drive geometry, assuming 64 sectors/track
Can't get drive geometry, assuming 8 heads
Unable to get device geometry, assuming default
...creating default partitions
...creating default volume directory

----- please choose one (? for help, .. to quit this menu)-----
[exi]t [d]ebug/ [l]abel/ [a]uto
[b]adblock/ [exe]rcise/ [r]epartition/ [f]ormat
fx> label/create/all

...creating default bootinfo
Can't get drive geometry, assuming 64 sectors/track
Can't get drive geometry, assuming 8 heads
Unable to get device geometry, assuming default
...creating default partitions
...creating default sgiinfo
...creating default volume directory

----- please choose one (? for help, .. to quit this menu)-----
[exi]t [d]ebug/ [l]abel/ [a]uto
[b]adblock/ [exe]rcise/ [r]epartition/ [f]ormat
fx> exit

label info has changed for disk rad(4,8,). write out changes? (yes) yes   ID="86528"Confirm that the RAID is formatted properly:ID="ch215"ID="ch216"raid -p   /dev/rdsk/radcontrollerdunitvhThe output should contain this information in a single line: /dev/rdsk/radcontrollerdunitvh: RAIDlevel, sectors: sectors, stripe size: depth, OKSee IDREF="10056" TYPE="TITLE"Section 5.1.1, "Messages from Format Operations,"
 if the message doesn't end with "OK".To put one or more filesystems on the RAID, use ID="ch217"ID="ch218"mkfs(1M) to construct the filesystem(s), create the mount point(s), edit /etc/fstab, and mount the filesystem as usual. For a complete description of the procedure, see Section 4.2, "Maintaining File Systems," of the IRIX Advanced Site and Server Administration Guide. If you reformatted the RAID, restore the data from tape. ID="ch219"ID="ch220"LBL="3"ID="83934"Routine Maintenance TasksRAIDs require routine monitoring and maintenance to ensure high reliability. This chapter describes a variety of routine administration tasks that are specific to RAID or particularly important for RAID:IDREF="97939" TYPE="TITLE"Section 3.1, "Making Tape Backups"IDREF="43490" TYPE="TITLE"Section 3.2, "Monitoring /var/adm/SYSLOG"IDREF="19285" TYPE="TITLE"Section 3.3, "Getting Configuration and Status Information"IDREF="55284" TYPE="TITLE"Section 3.4, "Checking the Integrity of Parity"IDREF="51379" TYPE="TITLE"Section 3.5, "Checking for Failed Disk Drives"IDREF="39369" TYPE="TITLE"Section 3.6, "Downloading New Firmware"LBL="3.1" HELPID=""ID="97939"Making Tape BackupsEven though a RAID provides redundancy and disk drive failure prediction, it is still important to make regular tape backups of data stored on a RAID.ID="ch31"ID="ch32"The IRIX Advanced Site and Server Administration Guide provides information about tape backup tools and strategies in Chapter 5, "Backing Up and Restoring Data." LBL="3.2" HELPID=""ID="43490"Monitoring /var/adm/SYSLOGWarnings and error information for RAIDs are written to the file ID="ch33"ID="ch34"ID="ch35"ID="ch36"/var/adm/SYSLOG and the console by the raid(1M) command if the ­L option is given and by the RAID device driver. (See usraid(7M) for more information about the RAID device driver.) /var/adm/SYSLOG should be checked frequently, about every 12 hours, for messages that predict disk drive failure.An easy way to monitor /var/adm/SYSLOG automatically is to set up a cron(1M) job that checks for messages and mails the result to root or a system administrator. Follow these steps to set up a cron job:Become superuser.Write the current set of cron jobs to a temporary file:crontab -l >   tempfile   Edit the temporary file and add a single line like this (shown on multiple lines here):0      4       *       *       *       egrep '/dev/*dsk/rad' /var/adm/SYSLOG | mail -s "RAID warnings"   userid   userid is the user ID that you want the mail sent to. At 4 a.m. every day, /var/adm/SYSLOG is searched for a string included in all RAID messages and the output is mailed to userid.Add another line just like the one in the previous step, but change the 4 to 16:0      16       *       *       *       egrep '/dev/*dsk/rad' /var/adm/SYSLOG | mail -s "RAID warnings"   userid   This line performs that same check as above, but at 4 p.m. every day.Tell cron to use the new version of tempfile:crontab   tempfile   Another way to monitor messages in ID="ch37"/var/adm/SYSLOG is to use syslogd(1M). See the syslogd(1M) manual page for details.IDREF="79432" TYPE="TITLE"Section 5.1, "raid Error and Warning Messages," and IDREF="45165" TYPE="TITLE"Section 5.2, "Messages from the RAID Device Driver,"
 list the messages that can appear in /var/adm/SYSLOG and describe what to do about them.LBL="3.3" HELPID=""ID="19285"Getting Configuration and Status InformationYou can get configuration and status information about one or all RAIDs on a workstation or server by giving the ID="ch38"ID="ch39"­p option to the raid command. With this option, raid reports on disk drives that are down, RAIDs that have not been formatted, and internal configuration information in a RAID that is not consistent.To get this information, give this command as superuser (omit the /dev/rdsk argument to check all RAIDs):/etc/raid -p   /dev/rdsk/radcontrollerdunitvhcontroller is the controller number and unit is the unit number (both can be obtained from hinv(1M)) of a RAID. An example of the output from a correctly configured RAID is:/dev/rdsk/rad0d3vh: RAID3, sectors: 16777216, stripe size: 64, OKOutput is written to ID="ch310"ID="ch311"ID="ch312"/var/adm/SYSLOG and the console as well as being displayed in your shell window when you give the ­L option on the raid command line.Other messages that can follow the configuration information instead of OK are:drive n   is down   not initialized   must reformat   LBL="3.4" HELPID=""ID="55284"Checking the Integrity of ParityYou can use the ID="ch313"ID="ch314"ID="ch315"ID="ch316"raid program to check the integrity of parity and correct errors on one RAID or all RAIDs. When checking integrity, all disk drives must be operational. Checking parity at least every few days is recommended. To check parity on a RAID, give this command (omit the /dev/rdsk argument to check all RAIDs) as superuser:raid -i /dev/rdsk/radcontrollerdunitvhcontroller is the controller number and unit is the unit number (both can be obtained from hinv). ID="ch317"raid silently repairs any parity problems it finds by regenerating the parity to match the data on the disk drives. This will take 15 to 20 minutes per RAID on a lightly loaded system.To set up a ID="ch318"cron job that checks parity every night at 1 a.m., follow the procedure in IDREF="43490" TYPE="TITLE"Section 3.2
, except add this line instead (shown wrapped) to tempfile:0      1       *       *       *       raid -i -L | mail -s "RAID warnings"   userid   LBL="3.5" HELPID=""ID="51379"Checking for Failed Disk DrivesThe ID="ch319"ID="ch320"ID="ch321"raid command with the ­c option can be used to check for failed disk drives:To check for disk drives marked down (failed disks and disks marked down with ­d) on all RAIDs, give this command as superuser:raid -c   To check for disk drives marked down on just one RAID, give this command as superuser:raid -c   /dev/rdsk/radcontrollerdunitvhcontroller is the controller number and unit is the unit number (both can be obtained from hinv). This command takes a few seconds. Output is written to ID="ch322"/var/adm/SYSLOG and the console as well as being displayed in your shell window when you give the ­L option, too.When no disk drives are down, there is no output from the command. These messages are displayed if raid detects a down disk drive or other problem:raid: warning: /dev/rdsk/radcontrollerdunitvh: drive n   is down   raid: error: /dev/rdsk/radcontrollerdunitvh: transfer of config info to/from disk 2 failed: I/O errorraid: warning: /dev/rdsk/radcontrollerdunitvh: drive n   is down   raid: error: /dev/rdsk/radcontrollerdunitvh: controller has been replacedraid: error: /dev/rdsk/radcontrollerdunitvh: it doesn't match disk config info raid: error: /dev/rdsk/radcontrollerdunitvh: disk n   has been replaced, but it wasn't marked downraid: error: /dev/rdsk/radcontrollerdunitvh: the RAID should be rebuilt   Rebuilding the RAID with raid   ­r makes a down disk drive operational. For more information, see IDREF="25149" TYPE="TITLE"Section 4.1, "Replacing a Disk Drive."Each time the system is rebooted, RAIDs are checked for failed disk drives automatically with this command:ID="ch323"ID="ch324"raid -c -m -L   The ID="ch325"­m option tells the raid command to check for disk drives that were replaced while the system was off. If it finds new disks, it marks them as failed so that any data on them is not used. See IDREF="79432" TYPE="TITLE"Section 5.1, "raid Error and Warning Messages,"
 for more information.LBL="3.6" HELPID=""ID="39369"Downloading New FirmwareRAID firmware can be downloaded to flash memory in the RAID controller. When IRIX software releases include new RAID firmware (see the ID="ch326"ID="ch327"IRIX Release Notes), you may need to download the new firmware into each RAID individually to update your system.To update a particular RAID, first make sure that no applications are using the RAID, then give this command as superuser:raid -l   firmware    /dev/rdsk/radcontrollerdunitvhfirmware is the pathname of the file that contains the new firmware,   controller is the controller number, and unit is the unit number (both can be obtained from hinv). The downloading takes about a minute.After downloading is complete, reset the RAID controller by follow this procedure:Open the RAID unit as explained in steps 
IDREF="54901" TYPE="TEXT"5
 and 
IDREF="56375" TYPE="TEXT"6
 of IDREF="25149" TYPE="TITLE"Section 4.1, "Replacing a Disk Drive."Press the RAID controller Reset button shown in IDREF="64824" TYPE="GRAPHIC"Figure 1-5
.Close the RAID unit as explained in steps 
IDREF="49172" TYPE="TEXT"13
 and 
IDREF="18441" TYPE="TEXT"14
 of IDREF="25149" TYPE="TITLE"Section 4.1
.LBL="4"ID="65029"Recovering after a Disk Warning or FailureThis chapter contains several error recovery procedures:IDREF="25149" TYPE="TITLE"Section 4.1, "Replacing a Disk Drive," describes the procedure for replacing a disk drive that has failed or has predicted its failure. IDREF="30553" TYPE="TITLE"Section 4.2, "Restoring Data from Tape after Two Failures," explains the procedure to follow when two disk drives or a disk drive and the controller in a RAID have failed.IDREF="92628" TYPE="TITLE"Section 4.3, "Restarting a Hung RAID," explains how to proceed if a RAID stops responding to commands.IDREF="81072" TYPE="TITLE"Section 4.4, "Resetting a RAID to Factory Defaults," explains how to put a RAID into an uninitialized state. LBL="4.1" HELPID=""ID="25149"Replacing a Disk DriveIt is important to replace a failed or about-to-fail disk drive as soon as possible to reduce the possibility of a second disk drive's failing before the first is replaced. If a second disk drive fails, the data on the entire RAID must be recovered from tape backups.ID="ch41"Don't remove the wrong disk drive! When a disk drive has failed, removing any disk drive other than the failed disk drive will probably result in the loss of all data on the RAID. Before removing any disk drive, confirm that you are removing the correct one.Follow the procedure below to replace a disk drive in a RAID unit and rebuild the disk drive. The replacement disk drive must be the same type as the failed disk drive (contact your Silicon Graphics Sales Representative or distributor of Silicon Graphics equipment to purchase disk drives for RAID). During rebuilding, the contents of the non-failed disk drives are used to reconstruct the data that was stored on the failed disk drive.ID="ch42"ID="ch43"ID="ch44"Make a backup tape of the data on the RAID. This step is optional, but it is highly recommended.ID="ch45"Get the controller and unit of the disk drive you plan to replace from hinv(1M).If the disk drive is predicted to fail, but has not actually failed yet, telling the controller to mark the disk drive as down is recommended. Give this command as superuser:ID="ch46"ID="ch47"ID="ch48"raid -d   driveno   /dev/rdsk/radcontrollerdunitvhdriveno is the drive number of the disk you are replacing. It was in any PFA messages. This command won't mark the about-to-fail disk drive as down if there is another disk drive in the RAID already marked down, or if the RAID controller and the disk drive contain inconsistent information. The output of this command explains if it found any disk drives already marked down, and if it was able to force the disk drive down as requested. If a disk drive is already marked down, it should be replaced before replacing a disk drive that has been predicted to fail. ID="97871"Because you could lose the data on the RAID if you remove the wrong disk drive, you may want to reduce the chances of losing data. Choose and implement one of these options (least risky to most risky):Shut the system down and power it off. Before powering off the system, mark the drive to be replaced with a Post-itÔ note, since the LEDs that note its location won't be lit.Quiesce the RAID. Stop any applications that are accessing the RAID and unmount the RAID filesystem(s). Hot plug the new disk drive. No action is required.ID="ch49"ID="54901"Open the cabinet that contains the RAID unit so that you can see the front of the RAID unit. IDREF="57820" TYPE="GRAPHIC"Figure 4-1
 shows the front of one type of RAID unit enclosure; the other type of enclosure is very similar.LBL="4-1" FILE="4-1.RAID.unit.front.ai" POSITION="INLINE" SCALE="FALSE"LBL="4-1"Figure 4-1 
ID="57820"RAID Unit FrontID="ch410"ID="ch411"ID="ch412"ID="56375"Twist and pull the fasteners at the top of the unit front (see IDREF="57820" TYPE="GRAPHIC"Figure 4-1
) and lower the door into a fully open position.Note the physical location of the disk drive you plan to replace. On a failed disk drive or on a disk drive that has been marked down, its amber LED is lit (see ID="ch413"IDREF="99206" TYPE="TITLE"Section 1.5.2, "RAID Disk Drive LEDs"
) and its LED on the edge of its controller is lit (see IDREF="98626" TYPE="TITLE"Section 1.5.1, "RAID Controller LEDs"
). These LEDs are not lit on disk drives that have been predicted to fail but haven't been marked down.Push up on the drive lever on the lower left side of the disk drive you want to remove to release the disk drive module.Slide the disk drive module out (see IDREF="17672" TYPE="GRAPHIC"Figure 4-2
).LBL="4-2" FILE="4-2.rem.or.install.disk.drive.ai" POSITION="INLINE" SCALE="FALSE"LBL="4-2"Figure 4-2 
ID="17672"Removing or Installing a Disk DriveID="ch414"Use the push buttons on the front of the replacement disk drive to set its drive number to the same number as the disk drive you just removed (see IDREF="80448" TYPE="GRAPHIC"Figure 1-6
).Slide the replacement disk drive module into the opening in the RAID unit. As you slide in the module and push it to the back, its drive lever moves down.Push down on the drive lever to lock it in position. Try pulling the module out to make sure that it is locked.ID="49172"Raise the unit door and turn the fasteners to secure the front cover.The front cover must be in place during operation for proper air flow and to comply with FCC and other regulatory requirements.ID="18441"Replace any panels of the cabinet that you removed earlier.ID="15620"If you powered off the workstation or server, turn the power back on and wait one minute for the RAID disk drives to spin up.If you unmounted the RAID filesystem(s), mount it again.Rebuild the RAID by giving this command as superuser:ID="ch415"ID="ch416"/etc/raid -r   /dev/rdsk/radcontrollerdunitvhWhen the rebuilding of the RAID is complete (about 30 minutes on a lightly loaded system), the RAID is again fully functional.ID="ch417"ID="ch418"ID="ch419"LBL="4.2" HELPID=""ID="30553"Restoring Data from Tape after Two FailuresAs explained in ID="ch420"ID="ch421"ID="ch422"IDREF="37676" TYPE="TITLE"Section 1.7, "RAID Failure Recovery,"
 when two disk drives or a disk drive and the controller in a RAID fail, the data on the RAID becomes inaccessible. Examples of situations that result in two failures are:A disk drive fails, but the failure is not noticed or not repaired, and then a second disk drive fails.A disk drive fails and while attempting to hot plug a replacement for the failed disk, the wrong disk drive is accidently removed from the RAID.A disk drive fails, and then the controller fails.If two disk drives have failed, follow this procedure:Replace the failed disk drives by following steps 
IDREF="97871" TYPE="TEXT"4
 through 
IDREF="15620" TYPE="TEXT"15
 in IDREF="25149" TYPE="TITLE"Section 4.1
.Format the RAID as if it were a new RAID using the procedure in IDREF="90820" TYPE="TITLE"Chapter 2, "Formatting a RAID."Reload the files that were on the RAID from a backup tape.A service call is required to replace a failed RAID controller. ID="ch423"LBL="4.3" HELPID=""ID="92628"Restarting a Hung RAIDIf a RAID becomes hung, it doesn't respond to commands. This can occur as a result of an internal hardware fault in the controller. Two ways to attempt to restart it are:ID="ch424"Reset the RAID controller.ID="ch425"ID="ch426"Power cycle the RAID unit.ID="ch427"ID="ch428"When you perform either of these procedures, there is some risk of corruption of data on the RAID, just as there is when you press the Reset button on a workstation or server.To reset the RAID controller, follow these steps:Open the RAID unit as explained in steps 
IDREF="54901" TYPE="TEXT"5
 and 
IDREF="56375" TYPE="TEXT"6
 of IDREF="25149" TYPE="TITLE"Section 4.1
. Press the RAID controller Reset button shown in ID="ch429"IDREF="64824" TYPE="GRAPHIC"Figure 1-5
.Close the RAID unit as explained in steps 
IDREF="49172" TYPE="TEXT"13
 and 
IDREF="18441" TYPE="TEXT"14
 of IDREF="25149" TYPE="TITLE"Section 4.1
.Some RAID units have a power switch as shown in ID="ch430"ID="ch431"IDREF="57820" TYPE="GRAPHIC"Figure 4-1
. To power cycle this type of RAID unit, follow these steps:Open the cabinet that contains the RAID unit so that you can see the front of the RAID unit.Turn off power to the RAID unit (see IDREF="57820" TYPE="GRAPHIC"Figure 4-1
 for the location of the switch).Wait 15 seconds.Turn on the power.Replace the front panel of the cabinet.To power cycle RAID units that don't have their own power switches, you must power cycle the entire workstation or server.LBL="4.4" HELPID=""ID="81072"Resetting a RAID to Factory DefaultsUnder certain error conditions, for instance when configuration information in the RAID is internally inconsistent, you may need to make a last ditch effort to get the RAID into a known state. The ID="ch432"raid   ­z option puts a RAID into an uninitialized state, just as it is when it is shipped from the factory.Using the ­z option of raid makes all data on a RAID inaccessible. Do not use it if you need any of the data on the RAID.Give this command to restore the internal configuration information of a RAID to its factory default, uninitialized state:ID="ch433"ID="ch434"raid -z   /dev/rdsk/radcontrollerdunitvhcontroller is the controller number and unit is the unit number (both can be obtained from hinv). To use the RAID again, you must format it (see IDREF="90820" TYPE="TITLE"Chapter 2, "Formatting a RAID"
).LBL="5"ID="26656"Error MessagesThis chapter lists error messages that you might see on a system with one or more RAIDs installed and RAID LED error conditions that can be seen after opening the RAID unit. The causes and possible solutions of these errors are explained. This chapter contains these sections:IDREF="79432" TYPE="TITLE"Section 5.1, "raid Error and Warning Messages," lists messages from the raid(1M) command. These messages appear as standard output and optionally on the console and in /var/adm/SYSLOG.IDREF="45165" TYPE="TITLE"Section 5.2, "Messages from the RAID Device Driver," lists messages from the RAID device driver (see usraid(7M)). These messages always appear on the console and in /var/adm/SYSLOG.IDREF="17470" TYPE="TITLE"Section 5.3, "LED Error Conditions," describes error conditions that are indicated by LEDs in the RAID unit.LBL="5.1" HELPID=""ID="79432"raid Error and Warning MessagesError messages from the ID="ch51"ID="ch52"ID="ch53"ID="ch54"ID="ch55"ID="ch56"ID="ch57"ID="ch58"raid command are always displayed in your shell window. When the raid   ­L option is given, messages are also written to the console and /var/adm/SYSLOG. LBL="5.1.1" HELPID=""ID="10056"Messages from Format Operationsdrive ID="ch59"ID="ch510"n is missing, formatting failed

drive n is not the same size as the other drives: m sectors

drive n is not a supported drive type: vendor: string1, product: string2, revision: string3All disk drives must be present, working correctly, and of the same make/model./dev/rdsk/radcontrollerdunitvh: RAIDlevel, sectors: sectors, stripe size: depth, drive n   is down

/dev/rdsk/radcontrollerdunitvh: RAIDlevel, sectors: sectors, stripe size: depth, not initialized

/dev/rdsk/radcontrollerdunitvh: RAIDlevel, sectors: sectors, stripe size: depth, must reformatThe RAID is not formatted properly. Try steps 
IDREF="21101" TYPE="TEXT"7
 through 
IDREF="86528" TYPE="TEXT"9
 of IDREF="90820" TYPE="TITLE"Chapter 2
 again. If you get the error message again, there is a hardware problem. Contact your local service provider.LBL="5.1.2" HELPID=""Messages from Integrity Check Operationscannot check integrity, drive ID="ch511"ID="ch512"n is downAll disk drives must be up and functioning. There is no point in checking the parity integrity of the RAID when a disk drive is down.LBL="5.1.3" HELPID=""Messages from Check Down Operationscontroller has been replaced, it doesn't match disk config info
ID="ch513"ID="ch514"If the controller has been replaced, then any indication of down disk drives is not valid. The ID="ch515"­m option wasn't specified, so no action is taken.controller has been replaced, reprogramming to match disksIf the controller has been replaced, then any indication of down disk drives is not valid. The ­m option was specified, so the controller configuration will be forced to match that of the disk drives in the RAID.disk n has been replaced, but it wasn't marked downThis can happen if a disk drive is replaced while the system was down.disk n is being forced down, the RAID needs to be rebuiltThe ­m option was specified, so the disk drive is being marked down. It must be rebuilt using the ­r option.drive n is downThe disk drive has failed or been marked down. It it has failed, it should be replaced and rebuilt. If it was simply marked down, it can just be rebuilt.LBL="5.1.4" HELPID=""Messages from Firmware Download Operationscan't download firmware to an uninitialized RAID, re-format (-3|-5) the RAID, then try again
ID="ch516"ID="ch517"The RAID must be formatted and in good working order before new firmware can be downloaded.LBL="5.1.5" HELPID=""Messages from Force Down Operationscannot force a drive down on this RAID
ID="ch518"ID="ch519"There is some problem with the RAID. A disk drive can't be forced down at this time.the controller has been replaced. use the `-c' and `-m' arguments to reprogram the controller to match the config info on the drivesIf the controller has been replaced, then any indication of down disk drives is not valid. Using the ­c and ­m options forces the controller configuration to match that of the disk drives in the RAID. After that is done, disk drives can be marked down using the ­d option, if desired.disk n has been replaced, but it wasn't marked downThis can happen if a disk drive is replaced while the system was down. The ­c and ­m options are used at system boot time to detect this situation and mark any such disk drives down. They must be rebuilt using the ­r option.aborting: this would down a second disk in the arrayEvery attempt is made to avoid marking a second disk drive down, but the ID="ch520"­f option will forcibly mark any disk drive down. If a second disk drive is marked down, the RAID must be reformatted before it can be used for anything. All data will be lost.LBL="5.1.6" HELPID=""Messages from Rebuild Operationscannot rebuild this RAID

all disks are OK in this RAID
ID="ch521"ID="ch522"There is either a problem with the RAID and it cannot be rebuilt at this time, or there are no disk drives marked down.drive n is missing, rebuild failed

drive n is not the same size as the other drives: m sectors

drive n is not a supported drive type: vendor: string1, product: string2, revision: string3All disk drives must be present, working correctly, and of the same make/model.LBL="5.1.7" HELPID=""Messages from All Operationsmore than one disk is marked downMore than one disk drive has been marked down. This can only happen when an administrator forces down more than one disk drive. The only recovery is to reformat the RAID.drive n is missing

transfer of config info to/from disk n failedThere is configuration information stored in reserved sectors on each disk drive in a RAID. That information could not be read from the indicated disk drive. Either the disk drive has totally failed or the disk drive is not plugged in. This message will probably be accompanied by messages from the kernel.drive n's config info is unrecognizable

drive n's config info contains illegal values

disk n config info doesn't match NVRAM infoThese are all indications that something is wrong with the disk drive in the given slot number.drive n should be in slot m, raid ignoredThe configuration information from the reserved sectors of each disk drive shows that the disk drives in the RAID have been moved around.controller and drives disagree on which drive failed, controller says n failed, while drives say m failedThe configuration information from the reserved sectors of each disk drive is inconsistent. It is unclear which situation is true. Depending upon which situation is true, all the data in the RAID could be lost. The controller's NVRAM claims that one disk drive failed, and the configuration information in the reserved sectors of that disk drive match all the other disk drives, but there is a second disk drive whose configuration information does not match. Probably a double failure has happened, and the RAID must be reformatted and reloaded from a backup tape.raid corrupted, must reformatThe internal consistency of the RAID parity information has been compromised with no possibility of recovery. The RAID must be reformatted (wiping out all data on the RAID), and the contents reloaded from a backup tape.raid not initializedFormatting was not successfully completed. The RAID is not yet usable; it must be reformatted.LBL="5.2" HELPID=""ID="45165"Messages from the RAID Device DriverAll messages from the RAID device driver begin with:ID="ch523"radcontrollerdunitspartitionRAID device driver messages are written to the console and /var/adm/SYSLOG.disk n has failed, running in degraded modeThe disk drive in slot n has failed; it should be replaced as soon as possible.disk n is predicting failure, replace as soon as possibleThe disk drive in slot n has returned a PFA warning. It will probably fail within 24 hours. It should be replaced as soon as possible. You should probably mark it down before replacing it, but that is optional.not ready, initializing command requiredThe controller has not been configured, the configuration has been zeroed (with the raid   ­z option), or the controller configuration has been corrupted somehow. If the controller had previously been configured, try resetting the controller by pushing the controller Reset button (shown in IDREF="64824" TYPE="GRAPHIC"Figure 1-5
), otherwise a format operation is required.unrecovered hardware error, possibly a drive is not respondingA disk drive is not responding, but the controller cannot tell if the disk drive is dead, simply not plugged in, or there is a fault in the bus connecting the controller to the disk drive. Check all the disk drives in the RAID unit.fixing parity integrity on stripe at LBA nAn integrity check operation has detected that parity doesn't match the corresponding data for the stripe starting at the given block address. The parity is being updated to match the data automatically. If large numbers of these appear, or they recur often, contact your local service provider.recovered errorA hard error was detected on one of the disk drives in the RAID, but the controller has reconstructed the data and taken recovery actions.volume header not validThe volume header on the RAID is not valid. fx(1M) should be used to write a new one.retrying request

retries exhaustedA command has had an error and is being retried. After a certain number of retries, IRIX gives up and returns an error to the user.fatal error on maintenance command

maintenance command LBA not changing, controller may need to be RESET

unable to abort maintenance command, controller may need to be RESETA maintenance command (initialize disk drive, integrity check, or rebuild disk drive) has had an unexpected error. Retry the operation.forcing a second drive downThe raid command was used to forcibly mark down a disk drive in a RAID where there was already one disk drive down. Data may have been lost.FLASH EPROM programming error

FLASH EPROM sector protection error at page n

microcode download failed

new microcode has invalid checksum, ignored

new microcode has invalid signature, ignoredThere was an error downloading firmware to the controller. Retry the operation.auxiliary input 1 active

auxiliary input 2 activeThere are auxiliary inputs on the controller for monitoring internal conditions. Contact your local service provider.controller had internal parity error

internal target failure

unrecovered hardware error

unrecovered hardware error, 2nd fault while recovering

unrecovered hardware error, cannot find task ptr

unrecovered hardware error, recovery steps skipped

unrecovered hardware error, unknown cause

unrecovered hardware error, which stream is unknown

unspecified hardware errorThe controller has reported an internal hardware fault. Use fx to run diagnostics, then contact your local service provider.diagnostics failed on SCSI channel n: buffer test

diagnostics failed on SCSI chip channel n at ID m

diagnostics failedDiagnostics were run on the controller and a hard fault was detected. Contact your local service provider.corrupted definition page

corrupted partition page

corrupted units down infoThe RAID configuration information (the RAID level, stripe depth, and failed disk drive information) returned by the controller is not in a recognizable form. Try resetting the RAID (IDREF="92628" TYPE="TITLE"Section 4.3, "Restarting a Hung RAID"
). If that doesn't work, try reformatting (see IDREF="90820" TYPE="TITLE"Chapter 2, "Formatting a RAID"
). If that doesn't work, use the ­z option to zero out the controller configuration (see IDREF="81072" TYPE="TITLE"Section 4.4, "Resetting a RAID to Factory Defaults"
). Contact your local service provider.LBL="5.3" HELPID=""ID="17470"LED Error ConditionsRAID controller LEDs display these error conditions:ID="ch524"ID="ch525"ID="ch526"ID="ch527"ID="ch528"ID="ch529"LED 2 is not flashing.When LED 2 is not flashing (the flashing cycle can be very slow, up to 10 seconds), the RAID has failed, possibly due to an unrecoverable hardware error. Try both procedures in IDREF="92628" TYPE="TITLE"Section 4.3, "Restarting a Hung RAID,"
 to see if the RAID will start operating again. A service call by your support provider may be required to diagnose and repair the problem.Any of LEDs 4 to 8 are lit. Each lit LED indicates that a disk drive has failed. See IDREF="66069" TYPE="TABLE"Table 1-2
 and IDREF="64824" TYPE="GRAPHIC"Figure 1-5
 to figure out which one. IDREF="65029" TYPE="TITLE"Chapter 4, "Recovering after a Disk Warning or Failure,"
 explains how to fix the problem.When the amber LED in a RAID disk drive is on (see IDREF="80448" TYPE="GRAPHIC"Figure 1-6
 for the location of the amber LED), that disk drive is down. It could be down because it has failed or because it has been manually marked down with the raid   ­d command. See IDREF="65029" TYPE="TITLE"Chapter 4
 for recovery procedures.LBL="A"ID="17323"Programming HintsThis appendix contains suggestions for developers of applications that will be run on systems with RAIDs. These suggestions are designed to maximize performance of the RAIDs and the applications:ID="appa1"ID="appa2"ID="appa3"ID="appa4"ID="appa5"ID="appa6"ID="appa7"Determine the typical size of reads and use this number to optimize the stripe depth. The sar(1M) command with the ­d option reports the number of transfers to or from a block device and the number of bytes transferred.Hard mount filesystems on RAIDs that are NFS mounted rather than soft mounting them. (Hard mounting waits if the server is down, so data errors are returned after the connection times out. Soft mounting doesn't wait.)Expect different filesystem performance characteristics with RAID 3 than with a non-RAID disk. RAID 3 may not perform as you'd expect because of the many small things like inodes in a filesystem. For example, file creation and removal time may be very slow compared to data transfer times.Use raw or direct I/O instead of a filesystem to read more than 64 KB per disk access with RAID. To find out more about direct I/O, read about the F_DIOINFO command on the fcntl(2) manual page and the O_DIRECT flag on the open(2) manual page.Investigate using asynchronous I/O and direct I/O for large, data intensive applications if filesystem performance seems inadequate.When using a RAID as a raw disk, try to align I/O requests on stripe boundaries and make the sizes of requests some even multiple of the stripe depth.applications and RAIDIDREF="appa7"Programming HintsIDREF="ch141"RAID Level and Stripe Depthbackup tapesduring failure recoveryIDREF="ch45"Replacing a Disk Drivethe importance ofIDREF="ch179"RAID Failure RecoveryIDREF="ch23"Formatting a RAIDchannelcapacityIDREF="ch153"How Many RAID Units per SCSI Channel?maximum number of devicesIDREF="ch147"How Many RAID Units per SCSI Channel?recommended number of devicesIDREF="ch150"How Many RAID Units per SCSI Channel?check disk driveIDREF="ch18"RAID Levelsconfigurationat hardware installationIDREF="ch133"RAID Configuration OptionsdecisionsIDREF="ch135"RAID Configuration OptionsIDREF="ch24"Formatting a RAIDRAID levelIDREF="ch140"RAID Level and Stripe DepthIDREF="ch212"Formatting a RAIDRAID units per channelIDREF="ch151"How Many RAID Units per SCSI Channel?sectors usedIDREF="ch25"Formatting a RAIDIDREF="ch213"Formatting a RAIDstripe depthIDREF="ch118"RAID Stripe DepthIDREF="ch211"Formatting a RAIDconsole messagesIDREF="ch312"Getting Configuration and Status InformationIDREF="ch56"raid Error and Warning MessagescontrollerLEDsIDREF="ch160"RAID Controller LEDsIDREF="ch413"Replacing a Disk DriveIDREF="ch527"LED Error ConditionsReset buttonIDREF="ch162"RAID Controller LEDsIDREF="ch429"Restarting a Hung RAIDreset procedureIDREF="ch425"Restarting a Hung RAIDcron command and IRIXchecking parityIDREF="ch318"Checking the Integrity of Paritymonitoring /var/adm/SYSLOGIDREF="ch35"Monitoring /var/adm/SYSLOGdata disk driveIDREF="ch19"RAID LevelsdiskdefinedIDREF="intro4"Documentation ConventionsRAID vs. non-RAIDIDREF="ch126"RAID and IRIXIDREF="ch152"How Many RAID Units per SCSI Channel?disk drivedefinedIDREF="intro5"Documentation Conventionsdrive numberIDREF="ch166"RAID Disk Drive LEDsLEDsIDREF="ch165"RAID Disk Drive LEDspredictive failure analysis (PFA)IDREF="ch175"RAID Failure PredictionreplacingIDREF="ch44"Replacing a Disk DriveIDREF="ch418"Replacing a Disk DrivesizeIDREF="ch174"RAID Failure Predictiondown RAID unit or disk drive, definedIDREF="intro10"Documentation Conventionsdrive numberIDREF="ch47"Replacing a Disk DriveIDREF="ch169"RAID Disk Drive LEDserror messagesfrom raid commandIDREF="ch53"raid Error and Warning Messagesin /var/adm/SYSLOGIDREF="ch52"raid Error and Warning Messagesexclusive-OR parityIDREF="ch17"RAID Levelsfailed disk driveschecking forIDREF="ch321"Checking for Failed Disk DrivesreplacingIDREF="ch419"Replacing a Disk DriveIDREF="ch42"Replacing a Disk Drivefailed RAID unit or disk drive, definedIDREF="intro9"Documentation Conventionsfilesystems on RAIDIDREF="appa4"Programming HintsIDREF="ch129"RAID and IRIXIDREF="ch218"Formatting a RAIDIDREF="ch128"RAID and IRIXfont conventionsIDREF="intro11"Documentation Conventionsfx command and RAIDIDREF="ch214"Formatting a RAIDIDREF="ch127"RAID and IRIXhinv command and RAIDIDREF="ch124"RAID and IRIXIDREF="ch155"How Many RAID Units per SCSI Channel?hot pluggingIDREF="ch49"Replacing a Disk DriveIDREF="ch182"RAID Failure RecoveryI/O performance guidelinesIDREF="appa5"Programming HintsIDREF="ch148"How Many RAID Units per SCSI Channel?install RAID softwareIDREF="intro3"Hardware and Software RequirementsLED error conditionsIDREF="ch529"LED Error ConditionsLEDs in RAID unitIDREF="ch158"RAID Controller LEDslevels of RAIDIDREF="ch12"RAID LevelsMAKEDEV and RAIDIDREF="ch26"Formatting a RAIDmanual pagesIDREF="intro15"For More Informationmkfs command and RAIDIDREF="ch217"Formatting a RAIDmultiple failures in a RAID unitIDREF="ch422"Restoring Data from Tape after Two FailuresIDREF="ch180"RAID Failure Recoveryoperational RAID unit, definedIDREF="intro8"Documentation Conventionsparity bitschecking integrityIDREF="ch314"Checking the Integrity of ParityRAID 3IDREF="ch15"RAID LevelsRAID 5IDREF="ch112"RAID LevelsPatterson articleIDREF="intro14"For More Informationperformance of applicationsIDREF="appa3"Programming HintsIDREF="ch139"RAID Level and Stripe DepthPFA (predictive failure analysis)IDREF="ch176"RAID Failure Predictionpower cyclingIDREF="ch427"Restarting a Hung RAIDpower switchIDREF="ch411"Replacing a Disk DriveIDREF="ch430"Restarting a Hung RAIDprogramming hints for RAIDIDREF="appa2"Programming HintsRAIDautomatic checking for failed disk drivesIDREF="ch323"Checking for Failed Disk Driveschecking for failed disk drivesIDREF="ch320"Checking for Failed Disk DrivesIDREF="ch514"Messages from Check Down Operationschecking formattingIDREF="ch216"Formatting a RAIDchecking parity integrityIDREF="ch315"Checking the Integrity of ParityIDREF="ch512"Messages from Integrity Check Operationsconfiguration at hardware installationIDREF="ch154"How Many RAID Units per SCSI Channel?configuration decisionsIDREF="ch134"RAID Configuration Optionsconfiguration informationIDREF="ch39"Getting Configuration and Status Informationcontroller LEDsIDREF="ch525"LED Error ConditionsIDREF="ch159"RAID Controller LEDsdefinedIDREF="ch11"RAID LevelsIDREF="intro6"Documentation Conventionsdevice driverIDREF="ch132"RAID and IRIXdifferent from non-RAID diskIDREF="ch130"RAID and IRIXdisk drive LEDsIDREF="ch526"LED Error Conditionsdisk layoutsIDREF="ch14"RAID Levelsdownloading firmwareIDREF="ch517"Messages from Firmware Download OperationsIDREF="ch327"Downloading New Firmwarefailure prediction (PFA)IDREF="ch172"RAID Failure Predictionfailure recoveryIDREF="ch421"Restoring Data from Tape after Two Failuresforcing down a disk driveIDREF="ch519"Messages from Force Down OperationsIDREF="ch48"Replacing a Disk DriveformattingIDREF="ch219"Formatting a RAIDIDREF="ch510"Messages from Format Operationshinv outputIDREF="ch156"How Many RAID Units per SCSI Channel?hot plugging a replacement driveIDREF="ch181"RAID Failure Recoverylevel 3IDREF="ch13"RAID LevelsIDREF="ch142"RAID Level and Stripe Depthlevel 5IDREF="ch143"RAID Level and Stripe DepthIDREF="ch111"RAID LevelsPatterson articleIDREF="intro13"For More InformationperformanceIDREF="ch113"RAID LevelsIDREF="appa6"Programming HintsIDREF="ch136"RAID Configuration Optionsrebuilding after failureIDREF="ch522"Messages from Rebuild OperationsIDREF="ch416"Replacing a Disk DrivereformattingIDREF="ch220"Formatting a RAIDIDREF="ch22"Formatting a RAIDreplacing a disk driveIDREF="ch417"Replacing a Disk DriveIDREF="ch43"Replacing a Disk Driveresetting to factory defaultsIDREF="ch434"Resetting a RAID to Factory Defaultsroutine maintenanceIDREF="ch316"Checking the Integrity of ParityIDREF="ch36"Monitoring /var/adm/SYSLOGIDREF="ch32"Making Tape Backupssame as non-RAID diskIDREF="ch125"RAID and IRIXstripe depthIDREF="ch138"RAID Level and Stripe DepthIDREF="ch116"RAID Stripe Depthstripe sizeIDREF="ch120"RAID Stripe Depthstripe widthIDREF="ch119"RAID Stripe Depthtwo failed disk drivesIDREF="ch178"RAID Failure Recoverytypical applicationsIDREF="ch144"RAID Level and Stripe Depthraid command­level optionIDREF="ch59"Messages from Format OperationsIDREF="ch27"Formatting a RAID­c optionIDREF="ch513"Messages from Check Down Operations­d optionIDREF="ch518"Messages from Force Down OperationsIDREF="ch46"Replacing a Disk Drive­f optionIDREF="ch520"Messages from Force Down Operations­i optionIDREF="ch511"Messages from Integrity Check Operations­L optionIDREF="ch317"Checking the Integrity of ParityIDREF="ch322"Checking for Failed Disk DrivesIDREF="ch310"Getting Configuration and Status InformationIDREF="ch516"Messages from Firmware Download Operations­m optionIDREF="ch325"Checking for Failed Disk DrivesIDREF="ch515"Messages from Check Down Operations­p optionIDREF="ch215"Formatting a RAID­r optionIDREF="ch415"Replacing a Disk DriveIDREF="ch521"Messages from Rebuild Operations­S optionIDREF="ch29"Formatting a RAIDIDREF="ch28"Formatting a RAID­z optionIDREF="ch433"Resetting a RAID to Factory Defaultsat system rebootIDREF="ch324"Checking for Failed Disk Driveserror messagesIDREF="ch54"raid Error and Warning MessagesRAID controllerfailedIDREF="ch423"Restoring Data from Tape after Two FailuresLEDsIDREF="ch528"LED Error ConditionsIDREF="ch161"RAID Controller LEDsresettingIDREF="ch426"Restarting a Hung RAIDRAID unitcontroller LEDsIDREF="ch163"RAID Controller LEDsdefinedIDREF="intro7"Documentation Conventionsdisk drive LEDsIDREF="ch167"RAID Disk Drive LEDsdrawingsIDREF="ch410"Replacing a Disk DriveIDREF="ch170"RAID Disk Drive LEDsIDREF="ch414"Replacing a Disk Drivedrive numbersIDREF="ch168"RAID Disk Drive LEDspower cyclingIDREF="ch428"Restarting a Hung RAIDpower switchIDREF="ch431"Restarting a Hung RAIDIDREF="ch412"Replacing a Disk DriveSCSI devices per channelIDREF="ch149"How Many RAID Units per SCSI Channel?shell window messagesIDREF="ch57"raid Error and Warning Messagessoftware requirementsIDREF="intro2"Hardware and Software Requirementsstripe depthand disk layoutIDREF="ch122"RAID Stripe DepthIDREF="ch110"RAID LevelsIDREF="ch114"RAID LevelschoosingIDREF="ch145"RAID Level and Stripe DepthdefinitionIDREF="ch117"RAID Stripe DepthspecifyingIDREF="ch210"Formatting a RAIDstripe widthIDREF="ch121"RAID Stripe Depthsyslogd and checking SYSLOGIDREF="ch37"Monitoring /var/adm/SYSLOGsystem diskIDREF="ch131"RAID and IRIX/var/adm/SYSLOG fileIDREF="ch34"Monitoring /var/adm/SYSLOGIDREF="ch311"Getting Configuration and Status InformationIDREF="ch55"raid Error and Warning Messageswarning messagesIDREF="ch58"raid Error and Warning MessagesIDREF="ch173"RAID Failure PredictionXOR parityIDREF="ch16"RAID Levels