Convert DHCP to Static IP for IBM RSA II card

In my case: IBM System x3755 -[7163AC1]

First need to ensure the following in BIOS per IBM:
Enabling USB keyboard and mouse support:

1. Turn on or restart the server and watch the monitor screen.
2. When the message Press F1 for Configuration/Setup appears, press F1.
3. Select Advanced Setup and then select RSA II Settings.
4. In the OS USB Selection field, select Linux.
5. Select Save Values and Reboot RSA II.

Download and Install asu and ibmusbasm64-1.51-2 (custom build RPM from SRPM for Centos5 in my case).

./asu64 show all | grep RSA_DHCP

./asu64 set RSA_DHCP1 Disabled

./asu64 rebootrsa

./asu64 set RSA_HostIPAddress1 new_ip_addr
./asu64 set RSA_HostIPSubnet1 new_subnet
./asu64 set RSA_GatewayIPAddress1 new_gateway_addr

./asu64 rebootrsa

Done.

IBM BladeCenter blade SOL (LS41)

Centos5:

In /etc/inittab:
add one line:
s1:2345:respawn:/sbin/agetty -L -f /etc/issueserial 19200 ttyS1 vt100

root: init q

In /etc/securetty:
add line
ttyS1

In /etc/grub.conf:
add “console=tty0 console=ttyS1,19200″ at the end of kernel line

To access serial console:

console -T system:blade[1]

To exit:
ctrl+[
Then
ctrl+(

IBM CSM error – The saved node identifiers do not agree

Reference: http://publib.boulder.ibm.com/infocenter/clresctr/vxrx/index.jsp?topic=/com.ibm.cluster.rsct.doc/rsct_aix5l53/bl5dia0216.html

Nov 4 12:51:51 nodename_4 RMCdaemon[23836]: (Recorded using libct_ffdc.a cv 2):::Error ID: 824….5pQw8/DqU1.X8Nc/……………….:::Reference ID: :::Template ID:
0:::Details File: :::Location: RSCT,rmcd.c,1.52,211 :::RMCD_INFO_0_ST The daemon is started.
Nov 4 12:51:51 nodename_4 RMCdaemon[23836]: (Recorded using libct_ffdc.a cv 2):::Error ID: 822….5pQw8/ZxU1.X8Nc/……………….:::Reference ID: :::Template ID:
0:::Details File: :::Location: RSCT,rmcd_env.c,1.36,123 :::RMCD_2610_101_ER Internal error. Error data 1 0000001c Error data 2 00000000 Error data 3 2641-0
37 The saved node identifiers do not agree.
Nov 4 12:51:51 nodename_4 srcmstr: src_error=-9035, errno=0, module=’srchevn.c’@line:’252′, 0513-035 The ctrmc Subsystem ended abnormally. SRC will try and restart it.
Nov 4 12:51:51 nodename_4 RMCdaemon[23839]: (Recorded using libct_ffdc.a cv 2):::Error ID: 824….5pQw8/wlY1.X8Nc/……………….:::Reference ID: :::Template ID:
0:::Details File: :::Location: RSCT,rmcd.c,1.52,211 :::RMCD_INFO_0_ST The daemon is started.
Nov 4 12:51:51 nodename_4 RMCdaemon[23839]: (Recorded using libct_ffdc.a cv 2):::Error ID: 822….5pQw8/SsY1.X8Nc/……………….:::Reference ID: :::Template ID:
0:::Details File: :::Location: RSCT,rmcd_env.c,1.36,123 :::RMCD_2610_101_ER Internal error. Error data 1 0000001c Error data 2 00000000 Error data 3 2641-0
37 The saved node identifiers do not agree.
Nov 4 12:51:51 nodename_4 srcmstr: src_error=-9035, errno=0, module=’srchevn.c’@line:’252′, 0513-035 The ctrmc Subsystem ended abnormally. SRC will try and restart it.
Nov 4 12:51:51 nodename_4 RMCdaemon[23841]: (Recorded using libct_ffdc.a cv 2):::Error ID: 824….5pQw8/pVc1.X8Nc/……………….:::Reference ID: :::Template ID:
0:::Details File: :::Location: RSCT,rmcd.c,1.52,211 :::RMCD_INFO_0_ST The daemon is started.
Nov 4 12:51:51 nodename_4 RMCdaemon[23841]: (Recorded using libct_ffdc.a cv 2):::Error ID: 822….5pQw8/ncc1.X8Nc/……………….:::Reference ID: :::Template ID:
0:::Details File: :::Location: RSCT,rmcd_env.c,1.36,123 :::RMCD_2610_101_ER Internal error. Error data 1 0000001c Error data 2 00000000 Error data 3 2641-0
37 The saved node identifiers do not agree.
Nov 4 12:51:51 nodename_4 srcmstr: src_error=-9020, errno=0, module=’srchevn.c’@line:’409′, 0513-020 The ctrmc Subsystem did not end normally. The subsystem respawn li
mit has been exceeded. Check the Subsystem and restart it manually.

fcslogrpt /var/log/messages

/usr/sbin/rsct/bin/rmcdomainstatus -s ctrmc

On this node:

# cat /etc/ct_node_id
e30b82d7d977ae93

# cat /var/ct/cfg/ct_node_id
2fa704ea5403f37b

(Note: /var partition was completely corrupted, so we copied /var from another node in the cluster, to this one, thus the above 2 files are not matching each other)

# cp /var/ct/cfg/ct_node_id /var/ct/cfg/BACK_ct_node_id
# cp /etc/ct_node_id /var/ct/cfg/ct_node_id

# /etc/init.d/ctrmc status
Subsystem Group PID Status
ctrmc rsct inoperative

# /etc/init.d/ctrmc start
0513-059 The ctrmc Subsystem has been started. Subsystem PID is 23963.

# /etc/init.d/ctrmc status
Subsystem Group PID Status
ctrmc rsct 23963 active

/usr/sbin/rsct/bin/rmcdomainstatus -s ctrmc

Management Domain Status: Management Control Points
O A 0xe2ae70929d8e2b07 0001 192.168.5.1

# lssrc -a
Subsystem Group PID Status
ctrmc rsct 23963 active
IBM.CSMAgentRM rsct_rm 24017 active
IBM.ERRM rsct_rm 24020 active
IBM.AuditRM rsct_rm 24059 active
ctcas rsct inoperative
IBM.HostRM rsct_rm inoperative
IBM.ConfigRM rsct_rm inoperative
IBM.SensorRM rsct_rm inoperative
IBM.FSRM rsct_rm inoperative
IBM.LPRM rsct_rm inoperative

Hard Drive Rebuild Rate

This is one bad IBM EXP3000 hard drive:
MegaCli -ldpdinfo -a0:

PD: 3 Information
Enclosure Number: 1
Slot Number: 4
Device Id: 47
Sequence Number: 3
Media Error Count: 0
Other Error Count: 0
Predictive Failure Count: 0
Last Predictive Failure Event Seq Number: 0
Raw Size: 70006MB [0x88bb6b0 Sectors]
Non Coerced Size: 69494MB [0x87bb6b0 Sectors]
Coerced Size: 68664MB [0x861c000 Sectors]
Firmware state: Rebuild
SAS Address(0): 0×5000cca00db2de1d
SAS Address(1): 0×0
Inquiry Data: IBM-ESXSVPBA073C3ETS11 NA496JEY5BHECVBXSA496

watch “MegaCli -PDRbld -ShowProg -PhysDrv [1:4] -a0″

MegaCli -PDRbld -ShowProg -PhysDrv [1:4] -a0

Rebuild Progress on Device at Enclosure 1, Slot 4 Completed 65% in 9 Minutes.

IBM System Storage DS3000 Script Editor

The script on script editor for IBM Storage Manager can be executed via command line using SMcli command:

datanode9, enclosure 85, slot 9 has an impending drive error today, use the following command to manually fail it:

SMcli datanode9_ds2a -c “set drive[85,9] operationalState=failed;”

BEFORE REPLACING THE DRIVE:

SMcli datanode9_ds2a -c “show drive[85,9]“
Performing syntax check…

Syntax check complete.

Executing script…

Drive at Enclosure 85, Slot 9

Status: Impending failure

Mode: Assigned
Raw capacity: 136.732 GB
Usable capacity: 136.232 GB
..

AFTER REPLACING THE DRIVE:
SMcli datanode9_ds2a -c “show drive[85,9];”
Performing syntax check…

Syntax check complete.

Executing script…

Drive at Enclosure 85, Slot 9

Status: Replaced

Mode: Assigned
Raw capacity: 136.732 GB
Usable capacity: 136.232 GB
…

BEFORE REPLACING THE DRIVE:
SMcli datanode9_ds2a -c “show storagesubsystem healthstatus;”
Performing syntax check…

Syntax check complete.

Executing script…

The following failures have been found:
Impending Drive Failure – Medium Data Availability Risk
Storage Subsystem: ****
Array: 2
RAID level: 5
Enclosure: Controller/Drive enclosure
Affected drive slot(s): 9
Service action (removal) allowed: Yes
Service action LED on component: Yes
Logical Drives: 2
Status: Optimal

Logical Drive – Hot Spare In Use
Storage Subsystem: *****
Array: 2
Failed drive at: enclosure 85, slot 9
Service action (removal) allowed: Yes
Service action LED on component: Yes
Replaced by drive at: enclosure 85, slot 12
Logical Drives: 2
RAID level: 5
Status: Optimal

Script execution complete.

SMcli completed successfully.

AFTER REPLACING THE DRIVE:
SMcli datanode9_ds2a -c “show storagesubsystem healthstatus;”
Performing syntax check…

Syntax check complete.

Executing script…

Storage Subsystem health status = fixing.
The following failures have been found:
Logical Drive – Hot Spare In Use
Storage Subsystem: *****
Array: 2
Failed drive at: enclosure 85, slot 9
Service action (removal) allowed: No
Service action LED on component: Yes
Replaced by drive at: enclosure 85, slot 12
Logical Drives: 2
RAID level: 5
Status: Optimal

Script execution complete.

SMcli completed successfully.

Getting all storage ip address:
SMcli -d | sort