DS8000 Service Documentation Version 7.5

MAP2220 CEC enclosure to RPC card I2C communication failure

A view of the detail data for one of the RPCs interrupt registers is not available from one or both clusters.

About this task

Only the base LPAR in a CEC enclosure communicates with the RPC cards. The LPAR communicates with the service processor through the RTAS interface. The service processor communicates with both RPC cards through a shared I2C interface. The same I2C interface also communicates with the CEC enclosure control panel.

MAP2220 Section-1

Procedure

Display open serviceable events. Find those with the SRCs listed in Table 1. Use Table 1 to prioritize the order of repair.

Table 1. SRC and associated actions
SRC repair order	SRC definition^1, 2	Go to
BE190010	CEC enclosure 1 failed to communicate with RPC1 and RPC2.	MAP2220 Section-2 (One CEC communication failure with both RPC cards)
BE190011	CEC enclosure 2 failed to communicate with RPC1 and RPC2.
BE190012	CEC enclosure 1 and 2 failed to communicate with RPC1.	MAP2220 Section-3 (Both CECs communication failure with one RPC card)
BE190013	CEC enclosure 1 and 2 failed to communicate with RPC2.
BE190014	CEC enclosure 1 failed to communicate with RPC1.	MAP2220 Section-6 (One CEC communication failure with one RPC card)
BE190015	CEC enclosure 2 failed to communicate with RPC1.
BE190016	CEC enclosure 1 failed to communicate with RPC2.
BE190017	CEC enclosure 2 failed to communicate with RPC2.
Note 1: CEC enclosure 1 is upper, CEC enclosure 2 is lower. Note 2: See Figure 1 or Figure 2.

Figure 1. Location codes for the RPC cards (Models 941, 951)

Figure 2. Location codes for the RPC cards (Model 961)

Figure 3. Location codes for the CEC enclosure (rear view) (Models 941, 951)

Figure 4. Location codes for the CEC enclosure (rear view) (Model 961)

Location codes for the CEC enclosure (rear view) (Models 961)

Figure 5. Location codes for the PPS sequencer module

Figure 6. Location codes for the DC-UPS (Models 961, 96E) (rear)

MAP2220 Section-2 (One CEC communication failure with both RPC cards)

About this task

One CEC enclosure has a communication failure through its service processor to both RPC cards.

Procedure

Use the SRC in the serviceable event that sent you here and refer to Table 2 for an overall description of your failure. The failure could be caused by one of the following FRUs, which are listed in alphabetical order:

Models 941, 951:
- CEC enclosure control panel
- CEC enclosure service processor
- CEC enclosure I/O backplane assembly
- RPC card
- RPC card to CEC enclosure cable (I2C Y-Cable)
Model 961:
- CEC enclosure control panel
- CEC enclosure system backplane (Model 961) (contains service processor function)
- CEC enclosure I2C card
- RPC card to CEC enclosure I2C cable
- RPC card

Table 2. SRC communication-failure resources
SRC	Communication failure resources
BE190010	CEC enclosure 1 failure to communicate with RPC1 card and RPC2 card
BE190011	CEC enclosure 2 failure to communicate with RPC1 card and RPC2 card

Display open serviceable events that need repair. Is there any other serviceable event with FRUs listed in step 1?
- Yes, exit this MAP and attempt to repair that serviceable event first. If that repair does not correct this problem, return here and continue with the next step. If that repair does correct this problem, remember to also close this serviceable event.
- No, go to the next step.
Observe the CEC enclosure control panel. Does the control panel display a ball icon slowly moving clockwise around the display?
- Yes, go to MAP2220 Section-4
- No, go to MAP2220 Section-5 (One CEC communication failure to both RPC cards, continued)

MAP2220 Section-3 (Both CECs communication failure with one RPC card)

About this task

Both CEC enclosures have a communication failure through their service processors to the same RPC card. The working RPC card is not reporting any problem with the suspect RPC card. There should be no single point of failure that can cause this failure condition.

Procedure

Use the SRC in the serviceable event that sent you here to determine which RPC card the CEC failed to communicate with. Refer to Table 3.

Table 3. SRC communication interface-failure resources
SRC	Communication failure resources
BE190012	Both CEC enclosure service processors, failure to communicate with RPC1 card Models 941, 951, see Figure 1 (RPC1 is on the right, viewed from rear). Model 961 see Figure 2 (RPC1 is the upper card).
BE190013	Both CEC enclosure service processors, failure to communicate with RPC2 card Models 941, 951, see Figure 1 (RPC2 is on the left, viewed from rear). Model 961 see Figure 2 (RPC2 is the lower card).

Display serviceable events that need repair. Is there any other serviceable event listing the RPC card determined in step 1 or CEC enclosure to RPC card cable that connects to it?
- Yes, exit this MAP and attempt to repair that serviceable event first. If that repair does not correct this problem also, return here and continue at the next step. If that repair does not correct this problem, remember to also close this serviceable event.
- No, go to the next step.
At the suspect RPC card determined in step 1, ensure all the cables are properly connected before you continue. If any cables are not connected, do not connect them now. Instead go to the next step and do a psuedo repair of the RPC card. When you are directed to replace the RPC card, do not replace it but instead connect the cables.
The possible failing FRUs are the RPC card and CEC enclosure to RPC card cable.
1. Use the Exchange Parts procedure to select the suspect RPC card determined in step 1:
  1. From the navigation area, click Storage Facility Management > storage facility.
  2. From the bottom Task area, click Exchange Parts > Exchange Rack Components .... The Show Rack Enclosures window opens.
  3. Select a rack and click Show FRUs. The Show Rack FRUs window opens.
2. Select the suspect RPC Card FRU and continue the guided repair.
  1. From the Show Rack FRUs window, select RPC Card and click Exchange FRU.

MAP2220 Section-4

Procedure

The CEC enclosure control panel displays this clockwise moving icon when:
- The control panel was installed into the CEC enclosure and the service processor firmware has not logically installed the control panel, see step 2.
- The control panel is failing and needs to be replaced.
The default setting for the control panel is "installed." If the setting is "not installed" and the control panel is installed in the CEC enclosure with CEC power on, the icon moves slowly clockwise around the display. Use the ASMI Concurrent Maintenance > Control Panel menu option to ensure the control panel is logically installed. Refer to the MAP1221 ASMI menu structure.
If the control panel itself is failing, replace the control panel. Refer to MAP1215 Replace a FRU.

MAP2220 Section-5 (One CEC communication failure to both RPC cards, continued)

Procedure

Determine whether the CEC enclosure to RPC card cable is causing the problem.
- For Models 941, 951 continue at 2.
- For Model 961 continue at step 3.

For Models 941, 951, inspect the I2C Y-cable for damage or misplugging. The cable can be hot-plugged. Continue at step 4.

Table 4. Model 941, 951 I2C Y-cables
CEC enclosure	CEC enclosure connector¹	Upper RPC card connector²	Lower RPC card connector²
Upper	P1-I2C	C1-T210	C2-T210
Lower	P1-I2C	C1-T214	C2-T214
Note 1: See Figure 3. Note 2: See Figure 1.

For Model 961, inspect the I2C cable for damage or misplugging. The cable can be hot-plugged. Continue at step 4.

Table 5. Model 961 I2C cables
CEC enclosure	CEC enclosure connector¹	Upper RPC card connector²	Lower RPC card connector²
Upper	P1-C1-C1-T1	C1-J210	n/a
Upper	P1-C1-C1-T2	n/a	C2-J210
Lower	P1-C1-C1-T1	C1-J214
Lower	P1-C1-C1-T2		C2-J214
Note 1: See Figure 4. Note 2: See Figure 2.

Determine which RPC is fenced.

If an RPC card is listed in the serviceable event FRU list, then it is fenced.
If an RPC card is not listed in the serviceable event FRU list, use MAP1100 View storage facility state (end of call) to display fenced resources and then click Details to determine which RPC is fenced.

If no RPC shows as fenced, check for codes XE or XF on the status display of both DC-UPSs. Use Table 6 to select which RPC card to use for the next step.

Note: Any combination of codes may be displayed. You are only concerned with XE and XF.

Table 6. RPC card to pseudo-repair when none are fenced
DC-UPS status display	RPC card to pseudo-repair in next step
Both DC-UPSs show XF	RPC2 card (R1-C2)
Both DC-UPSs show XE	RPC1 card (R1-C1)
Any other condition, including: XE on one DC-UPS (but not both) XF on one DC-UPS (but not both)	RPC1 card (R1-C1)

Do a pseudo repair of the fenced RPC card. This resets the existing RPC card without replacing it.
- Refer to MAP2600 Pseudo FRU exchange needed to reset existing FRU.
Was the FRU verification successful?
- Yes, go to step 7.
- No, go to step 8.
Seating the cable corrected the problem. Exit this MAP and close related serviceable events.
Determine whether the control panel is causing the problem. Use the ASM interface to logically and physically remove the CEC enclosure control panel. Refer to MAP4110 Exchange the CEC enclosure control panel (concurrent).
Important: The CEC enclosure must not be power cycled or rebooted while the control panel is removed, to ensure that the load of the functional code succeeds.
Do a pseudo repair of the fenced RPC card. This resets the existing RPC card without replacing it.
- Refer to MAP2600 Pseudo FRU exchange needed to reset existing FRU.
Was the FRU verification successful?
- Yes, go to step 11.
- No, go to step 12.
The control panel is causing the failure when it is installed. Replace the control panel.
- Use MAP4110 Exchange the CEC enclosure control panel (concurrent). to install the new control panel.
- Ensure the machine type model serial/number label is moved to the new control panel.
- Exit this MAP and close all related serviceable events.
The CEC enclosure control panel is not failing. Use MAP4110 Exchange the CEC enclosure control panel (concurrent) to reinstall the existing control panel.
Replace the remaining FRUs in the serviceable event FRU list. If there are none, then the possible failing FRUs are listed in alphabetical order below. (Refer to MAP1230 Replace a FRU without using a serviceable event.)
- Models 941, 951:
  - CEC enclosure service processor
  - CEC enclosure I/O backplane assembly
  - RPC card to CEC enclosure cable (I2C Y-Cable)
  - RPC1 card (only when it is fenced; use MAP1100 to display fenced resources)
  - RPC2 card (only when it is fenced; use MAP1100 to display fenced resources)
- Model 961:
  - CEC enclosure system backplane (Model 961) (contains service processor function)
  - CEC enclosure I2C card
  - RPC card to CEC enclosure I2C cable
  - RPC1 card (only when it is fenced; use MAP1100 to display fenced resources)
  - RPC2 card (only when it is fenced; use MAP1100 to display fenced resources)

MAP2220 Section-6 (One CEC communication failure with one RPC card)

About this task

One CEC enclosure has a communication failure through the service processor to one RPC card. The RPC card is not reporting any communication failures to the service processor.

Procedure

Determine whether an LPAR is fenced (not all resources running) using MAP1100 Section-16, LPARs IMLed (CPSS).
Note: One of the two RPC cards should already be fenced.

Is an LPAR fenced?
- Yes, go to MAP2220 Section-2 (One CEC communication failure with both RPC cards) for SRC BE190010 and BE190011. Given more time, a serviceable event with either of these SRCs would be created.
- No, go to the next step.

Use the SRC in the serviceable event that sent you here to determine the failing communication interface. For Models 941, 951, use Table 7. For Model 961, use Table 8.

Models 941, 951:
1. Inspect the RPC card to CEC enclosure cable (I2C Y-Cable) for damage or misplugging. Ensure it is fully seated by pressing on each connector.
2. If a problem is found, the cable can be hot-plugged.
3. Most likely one of the RPC cards is fenced. Before this service action is complete, if that RPC card is not replaced for another reason, then that RPC card needs to go through a pseudo repair. A pseudo repair means the parts exchange process for that RPC card is used to create a reset to that card without physically replacing that RPC card. MAP1100 Section-27, Fenced Resources can be used to display fenced resources.
4. Continue at the next step.

Table 7. SRC communication interface-failure resources (Models 941, 951)
SRC	Communication failure resources	Y-Cable Connections for this SRC
BE190014	CEC enclosure 1 (upper), failure to communicate with RPC1 card (right viewed from rear)	RPC card to CEC enclosure cable Upper CEC enclosure P1-I2C (Figure 3) RPC1 card C1-T210 (Figure 2) RPC2 card C2-T210 (Figure 2) RPC card to PPSs Y-cable RPC1 card C1-T201 (Figure 2) Upper PPS E1-T1 (Figure 5) Lower PPS E2-T1 (Figure 5)
BE190016	CEC enclosure 1 (upper), failure to communicate with RPC2 card (left viewed from rear)
BE190015	CEC enclosure 2 (lower), failure to communicate with RPC1 card (right viewed from rear)	RPC card to CEC enclosure cable Lower CEC enclosure P1-I2C (Figure 3) RPC1 card C1-T214 (Figure 2) RPC2 card C2-T214 (Figure 2) RPC card to PPSs Y-cable RPC2 card C2-T201 (Figure 2) Upper PPS E1-T2 (Figure 5) Lower PPS E2-T2 (Figure 5)
BE190017	CEC enclosure 2 (lower), failure to communicate with RPC2 card (left viewed from rear)

Model 961:
1. Inspect the RPC card to CEC enclosure cable for damage or misplugging. Ensure it is fully seated by pressing on each connector.
2. If a problem is found, the cable can be hot-plugged.
3. Most likely one of the RPC cards is fenced. Before this service action is complete, if that RPC card is not replaced for another reason, then that RPC card needs to go through a pseudo repair. A pseudo repair means the parts exchange process for that RPC card is used to create a reset to that card without physically replacing that RPC card. MAP1100 Section-27, Fenced Resources can be used to display fenced resources.
4. Continue at the next step.

Table 8. SRC communication interface-failure resources (Model 961)
SRC	Communication failure resources	Y-Cable Connections for this SRC
BE190014	CEC enclosure 1 (upper), failure to communicate with RPC1 card (upper)	CEC enclosure to RPC cards Y-cable Upper CEC enclosure P1-C1-C1-T1 ( Figure 4) RPC1 card C1-J210 (Figure 2) RPC card to DC-UPS Y-cable RPC1 card C1-J201 (Figure 2) Upper UPS E1-J6 (Figure 6) Lower UPS E2-J6 (Figure 6)
BE190016	CEC enclosure 1 (upper), failure to communicate with RPC2 card (lower)	CEC enclosure to RPC cards Y-cable Upper CEC enclosure P1-C1-C1-T2 (Figure 4) RPC2 card C2-J210 (Figure 2) RPC card to DC-UPS Y-cable RPC1 card C1-J201 (Figure 2) Upper UPS E1-J6 (Figure 6) Lower UPS E2-J6 (Figure 6)
BE190015	CEC enclosure 2 (lower), failure to communicate with RPC1 card (upper)	CEC enclosure to RPC cards Y-cable Lower CEC enclosure P1-C1-C1-T1 (Figure 4) RPC1 card C1-J214 (Figure 2) RPC card to DC-UPS Y-cable RPC2 card C2-J201 (Figure 2) Upper UPS E1-J7 (Figure 6) Lower UPS E2-J7 (Figure 6)
BE190017	CEC enclosure 2 (lower), failure to communicate with RPC2 card (lower)	CEC enclosure to RPC cards Y-cable Lower CEC enclosure P1-C1-C1-T2 (Figure 4) RPC2 card C2-J214 (Figure 2) RPC card to DC-UPS Y-cable RPC2 card C2-J201 (Figure 2) Upper UPS E1-J7 (Figure 6) Lower UPS E2-J7 (Figure 6)

Display serviceable events that need repair.
Is there any other serviceable event with one or more of the same FRUs as listed in the serviceable event that sent you to this MAP?
- Yes, exit this MAP and repair the other serviceable event.
  - If the repair of the other serviceable event is successful, it closes automatically. Manually close the serviceable event that sent you here.
  - If the repair is not successful, go to step 4.
- No, go to step 4.
Replace hardware FRUs listed in the serviceable event that sent you here until the problem is repaired.
If no hardware FRUs are listed, the possible failing FRUs are listed here. To exchange a FRU that is not listed in a serviceable event, refer to MAP1230 Replace a FRU without using a serviceable event.
- Models 941, 951:
  - CEC enclosure service processor
  - CEC enclosure I/O backplane assembly
  - CEC enclosure to RPC card (I2C Y-Cable)
  - RPC1 card (only when it is fenced; use MAP1100 to display fenced resources)
  - RPC2 card (only when it is fenced; use MAP1100 to display fenced resources)
  - RPC card to primary power supply cable (to both PPS in Rack-1)
- Model 961:
  - CEC enclosure system backplane (Model 961) (contains service processor function)
  - CEC enclosure I2C card
  - RPC card to CEC enclosure I2C cable
  - RPC1 card (only when it is fenced; use MAP1100 to display fenced resources)
  - RPC2 card (only when it is fenced; use MAP1100 to display fenced resources)
  - RPC card to DC-UPS cable (in Rack-1 only)