The message: CHASSISD_PIC_HWERROR is reported in the system message file. This article documents an approach to troubleshoot it.
The message: CHASSIS_PIC_HWERROR is displayed in the syslog messages file. Following the message the FPC slot and the PIC slot in question are identified.
Additional log message entries may include: RED alarms for the PIC failure and setting of Major alarms for PIC failure
For each entry the position of the failed PIC is identified by the FPC slot and the PIC slot
Example:
Feb 6 11:53:23 lj chassisd[1458]: CHASSISD_PIC_HWERROR: PIC 1 in FPC 5 Feb 6 11:53:23 lj alarmd[1459]: Alarm set: PIC color=RED, class=CHASSIS, reason=FPC 5 PIC 1 Failure Feb 6 11:53:23 lj craftd[1460]: Major alarm set, FPC 5 PIC 1 Failure
The indicated Physical Interface Card (PIC) experienced a hardware error.
The chassis process (chassisd) did not bring the PIC online.
The port of the Flexible PIC Concentrator (FPC) experienced a hardware failure for the PIC slot indicated.
In other occurrences there are events where the FPC restarted several times during bidirectional traffic between two hyperion FPCs causing the CHASSISD_PIC_HWERROR to populate.
After an ISSU upgrade where the MIC was hot swapped out; causing the CHASSISD_PIC_HWERROR error to populate
Run the steps below, and check the PIC on-line status. If it remains off-line proceed to the next step.
1. Run the command:
request chassis pic online fpc-slot <fpc-slot #> pic-slot <pic-slot #>
Check the on-line status.
2. See Caution note. Then run the command:
restart chassis-control
Caution: This command will restart all PIC to FPC connections in the router and will disrupt traffic.
Check the on-line status.
3. Move the PIC in question to a different PIC slot of the FPC.
4. Run the command:
request chassis pic online fpc-slot <fpc-slot #> pic-slot <pic-slot #>
5. Run the command:
show chassis pic fpc slot <fpc-slot #> pic-slot <pic-slot #>
Replace the <pic-slot #> with the PIC slot number of the new position of the PIC in question. If the PIC status is off-line, please contact your technical support representative.
6. Insert a known good PIC into the slot in question. Check the on-line status. If the PIC fails to come on-line, then open a case with your technical support representative.