spike-01 throwing a Uncorrectable Machine Check Exception

[Firefishy]

Server spike-01 is occasionally throwing an Uncorrectable Machine Check Exception.

I have been unable to figure out what component has the issue. We have previously had issues with this server #89.

1346 Critical       04:07  11/05/2018 04:07  11/05/2018 0001
LOG: Uncorrectable Machine Check Exception (Board 0, Processor 1, APIC ID 0x00000010, Bank 0x00000005, Status 0xBA000000'00400405, Address 0x00000000'00000000, Misc 0x00000000'00004280)

1347 Critical       04:07  11/05/2018 04:07  11/05/2018 0001
LOG: Uncorrectable Machine Check Exception (Board 0, Processor 1, APIC ID 0x00000011, Bank 0x00000005, Status 0xBA000000'00400405, Address 0x00000000'00000000, Misc 0x00000000'00004280)

1348 Critical       09:42  12/19/2018 09:42  12/19/2018 0001
LOG: Uncorrectable Machine Check Exception (Board 0, Processor 1, APIC ID 0x00000010, Bank 0x00000005, Status 0xBA000000'00400405, Address 0x00000000'00000000, Misc 0x00000000'00000080)

1349 Critical       09:42  12/19/2018 09:42  12/19/2018 0001
LOG: Uncorrectable Machine Check Exception (Board 0, Processor 1, APIC ID 0x00000011, Bank 0x00000005, Status 0xBA000000'00400405, Address 0x00000000'00000000, Misc 0x00000000'00000080)

1350 Critical       01:53  01/16/2019 01:53  01/16/2019 0001
LOG: Uncorrectable Machine Check Exception (Board 0, Processor 1, APIC ID 0x00000010, Bank 0x00000005, Status 0xBA000000'00400405, Address 0x00000000'00000000, Misc 0x00000000'00004300)

1351 Critical       01:53  01/16/2019 01:53  01/16/2019 0001
LOG: Uncorrectable Machine Check Exception (Board 0, Processor 1, APIC ID 0x00000011, Bank 0x00000005, Status 0xBA000000'00400405, Address 0x00000000'00000000, Misc 0x00000000'00004300)

[Firefishy]

Options available:

1. Replace all RAM.
2. Replace CPUs.
3. Replace Motherboard.
4. Replace Machine.

[pnorman]

There is a related BIOS upgrade.

If that doesn't work, what makes the most sense will depend on when we plan to refresh these machines, so we need to decide that.

[Firefishy]

We are running the most recent BIOS version:

Handle 0x0000, DMI type 0, 24 bytes
BIOS Information
        Vendor: HP
        Version: P64
        Release Date: 05/21/2018

[pnorman]

We've discussed using this as another remote hands test. Let's do it - the RAM isn't that expensive and it's the next logical thing to test with them. Do you want to place the order and I'll do the portal tickets?

[pnorman]

Added @Firefishy to assignee list for placing the RAM order

[pnorman]

spike-01 is gone.