X1 carbon 7gen - broken GRUB after manual update of ECP with fwupdmgr install xxx.cab

[paloha]

Machine: X1 Carbon 7gen 20QD
OS: Ubuntu 20.04
BIOS version: 1.30
ECP version: 1.15
Secure Boot: ON
Boot Order Lock: OFF
Connected to USB-C Dock (for access to the LAN)

Yesterday, I tried to update UEFI BIOS from 1.30 to 1.33 (ECP to 1.17) using fwupdmgr (the versions were discovered by fwupdmgr), but the process got frozen at the splashscreen with the progressbar stopped in half. I had to use the reset button at the bottom of the notebook to reboot it. Fortunately the BIOS healed itself to the previous working state.

So today, I tried to manually apply the updates (to BIOS 1.34 and ECP 1.18) using the .cab files from the .zip of the BIOS Update Utility (Linux) from the official Lenovo site as documented in its txt readme.

So I extracted the .zip and run:
fwupdmgr install N2HHT35W.cab to first update the ECP from 1.15 to 1.18. It prompted to reboot so I allowed with y. The computer rebooted and the firmware installed - the progressbar went to 100% and rebooted. Then, I saw the Lenovo logo and possibility to enter BIOS. Next the screen goes blank and after a while outputs:

>>Start PXE over IPv4
    PXE -E18: Server response timeout
>>Start PXE over IPv6
    PXE-E16: No offer received. 

Then it reboots and the loop repeats. I tried to intercept by going into the BIOS. The ECV seems to be updated because the BIOS says ECV N2HHT35W (1.18).

I tried to turn off secure boot. Did not help. I tried to change the network boot to https. Also did not help. Then I reverted back to Secure boot and PXE. It rebooted maybe once or twice in the same loop and it got frozen on the >>Start PXE over IPv4 for a long time. I shut it down with long press of the power button and restarted. Again the same as last time.

Can someone please suggest what I shall do now? Thanks

[hughsie]

the process got frozen at the splashscreen with the progressbar stopped in half

Eeek, this isn't good; this normally indicates some kind of hardware error.

Start PXE over IPv4

Is the boot order set to PXE first in the BIOS for some reason?

[paloha]

I saw some discussion on the Internet also hereabout people having problems doing this update... After the bios healing I susoected it was not a HW issue, but I do not really know.

Anyways, no the PXE in bios was not set by me.
I tried to disconnect the LAN cable from the dock and now it put me into the Boot menu where I have these options:

• ubuntu
• Windows Boot Manager (wtf I do not have win)
• NVMe0
• PXE BOOT
• Linux-Firmware-Updater

If I select anything, the screen goes blank for 2 sec and the same menu reappears. If I click on PXE, the screen goes blank for 10s and again the same menu reappears.

[paloha]

Might this be a problem because I have not applied both .cab files for ECP and UEFI at the same time? So now I have ECP 1.18 while UEFI is still 1.30? If so, can I somehow apply it now?

[hughsie]

Might this be a problem because I have not applied both .cab files for ECP and UEFI at the same time

They should be independent.

If I select anything, the screen goes blank for 2 sec and the same menu reappears

I'm guessing booting a live USB media works? How was your ESP and /boot set up before? Standard grub type thing?

[paloha]

Yes, booting a live usb of ubuntu works.
Since I had to google whst ESP is I suspect all is standard. Shall I try to initiate the update of the UEFI from the live system?

[hughsie]

Shall I try to initiate the update of the UEFI from the live system

I don't think that's going to work. Worth a go, I don't think it's going to break anything else.

[paloha]

As you said, using fwupdmgr install uefi.cab did not work from the live USB.

So I created a BIOS Update bootable USB based on the BIOS Update Bootable CD and uodated the BIOS to 1.34. Unfortunately it did not resolve the problem as you suspected. I am still not able to boot to my OS. The PXE loop went away after I disconnected the Docking station and now I am using just the charger (this is what the BIOS update utility prompted me to do).

Still I am able to boot a live USB system.
Do you have any suggestions what I could try? Maybe repairing grub?

[hughsie]

Maybe repairing grub?

Yes, but before you do can you back up everything in /boot so we can compare with the new version to see what caused it to break.

[paloha]

Hurray! After fixing GRUB using boot-repair package, all seems back to normal. The UEFI and ECP are now both updated. UEFI from 1.30 to 1.34 (N2HET51P), ECP from 1.15 to 1.18 (N2HHT35W).

To sum up the procedure:
I had to create a BIOS Update Bootable USB based on the official package BIOS Update Bootable CD from Lenovo, by following these steps, thank you for the comment jaylinski. I booted from this USB and initiated UEFI BIOS update procedure (I was prompted to disconnect any extensions like Docking stations and use just a standard AC charger). The update went well, the BIOS was accessible but I was still not able to boot to my OS. So I used a Live USB with Ubuntu 20.04 to boot and from there I repaired the GRUB by following the steps in the package documentation. More precisely:

sudo add-apt-repository ppa:yannubuntu/boot-repair
sudo apt-get update
sudo apt-get install -y boot-repair && boot-repair

And clicking on the Recommended repair. The process said there was an error during the procedure, but it still works.

@hughsie thank you for your quick replies throughout the day.

... can you back up everything in /boot so we can compare with the new version to see what caused it to break.

I have stored the /boot contents before the repair along with the GRUB repair log. Do you want me to send it to you?

[hughsie]

I have stored the /boot contents before the repair along with the GRUB repair log. Do you want me to send it to you?

Please!

[paloha]

You have an email. Thanks.