I got myself a dedicated server at Hetzner from their “server auction” page.
I did set it up using their installimage. All went well, and the server was working fine, until an update to grub-efi-amd64-signed was released.
This is because the Hetzner installer seems to create an MS-DOS partition table, and uses old school MBR rather than GPT and ESP.
This is the error message:
~ # apt upgrade
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
Calculating upgrade... Done
The following packages were automatically installed and are no longer required:
linux-image-5.15.0-25-generic linux-modules-5.15.0-25-generic linux-modules-extra-5.15.0-25-generic
Use 'apt autoremove' to remove them.
The following packages have been kept back:
tzdata
0 upgraded, 0 newly installed, 0 to remove and 1 not upgraded.
1 not fully installed or removed.
After this operation, 0 B of additional disk space will be used.
Do you want to continue? [Y/n] y
Setting up grub-efi-amd64-signed (1.182~22.04.1+2.06-2ubuntu10) ...
mount: /var/lib/grub/esp: special device /dev/sda15 does not exist.
dpkg: error processing package grub-efi-amd64-signed (--configure):
installed grub-efi-amd64-signed package post-installation script subprocess returned error exit status 32
dpkg: dependency problems prevent processing triggers for shim-signed:
shim-signed depends on grub-efi-amd64-signed | grub-efi-arm64-signed; however:
Package grub-efi-amd64-signed is not configured yet.
Package grub-efi-arm64-signed is not installed.
dpkg: error processing package shim-signed (--configure):
dependency problems - leaving triggers unprocessed
Errors were encountered while processing:
grub-efi-amd64-signed
shim-signed
needrestart is being skipped since dpkg has failed
E: Sub-process /usr/bin/dpkg returned an error code (1)So, of course the package grub-efi-amd64-signed makes no sense and fails to install, since it has no ESP to install to, or so I thought.
Let’s take a look at the partition layout:
~ # lsblk -o +FSTYPE
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINTS FSTYPE
nvme0n1 259:0 0 476.9G 0 disk
├─nvme0n1p1 259:2 0 1G 0 part linux_raid_member
│ └─md0 9:0 0 1022M 0 raid1 /boot ext4
└─nvme0n1p2 259:3 0 475.9G 0 part linux_raid_member
└─md1 9:1 0 475.8G 0 raid1 crypto_LUKS
└─luks-xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx 253:0 0 475.8G 0 crypt / ext4
nvme1n1 259:1 0 476.9G 0 disk
├─nvme1n1p1 259:4 0 1G 0 part linux_raid_member
│ └─md0 9:0 0 1022M 0 raid1 /boot ext4
└─nvme1n1p2 259:5 0 475.9G 0 part linux_raid_member
└─md1 9:1 0 475.8G 0 raid1 crypto_LUKS
└─luks-xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx 253:0 0 475.8G 0 crypt / ext4Indeed, no ESP (UEFI) partition… let’s take a look at the partition table of the first SSD:
~ # gdisk /dev/nvme0n1
GPT fdisk (gdisk) version 1.0.8
Partition table scan:
MBR: MBR only
BSD: not present
APM: not present
GPT: not present
***************************************************************
Found invalid GPT and valid MBR; converting MBR to GPT format
in memory. THIS OPERATION IS POTENTIALLY DESTRUCTIVE! Exit by
typing 'q' if you don't want to convert your MBR partitions
to GPT format!
***************************************************************
Command (? for help):Indeed, just old school MS-DOS (MBR)…
However, interestingly:
~ # find /boot/efi
/boot/efi
/boot/efi/EFI
/boot/efi/EFI/BOOT
/boot/efi/EFI/BOOT/mmx64.efi
/boot/efi/EFI/BOOT/BOOTX64.EFI
/boot/efi/EFI/BOOT/fbx64.efi~ # mount | grep boot
/dev/md0 on /boot type ext4 (rw,relatime)It seems the Hetzner install image simply uses the ext4 /boot partition as UEFI boot (ESP) partition, too. (I, of course, thought, but this should be FAT32, what kind of mobo do they have?!)
So, of course, I tried to use parted, and set both boot partition to have the ESP flag, and indeed, GRUB then found the partitions to install to, and let me select them.
But still, no luck:
Setting up grub-efi-amd64-signed (1.182~22.04.1+2.06-2ubuntu10) ...
mount: /var/lib/grub/esp: unknown filesystem type 'linux_raid_member'.
dpkg: error processing package grub-efi-amd64-signed (--configure):
installed grub-efi-amd64-signed package post-installation script subprocess returned error exit status 32
dpkg: dependency problems prevent processing triggers for shim-signed:
shim-signed depends on grub-efi-amd64-signed | grub-efi-arm64-signed; however:
Package grub-efi-amd64-signed is not configured yet.
Package grub-efi-arm64-signed is not installed.
dpkg: error processing package shim-signed (--configure):
dependency problems - leaving triggers unprocessed
Errors were encountered while processing:
grub-efi-amd64-signed
shim-signed
needrestart is being skipped since dpkg has failedSo, I reverted to setting just the raid flag, rather than the esp flag.
In the end, the solution was stupid simple.
~ # efibootmgr
EFI variables are not supported on this system.Turns out, this isn’t a UEFI system at all…
/boot # apt list --installed '*grub*'
Listing... Done
grub-common/jammy,jammy,now 2.06-2ubuntu7 amd64 [installed,automatic]
grub-efi-amd64-bin/jammy-updates,jammy-updates,now 2.06-2ubuntu10 amd64 [installed]
grub-efi-amd64-signed/jammy-updates,jammy-updates,now 1.182~22.04.1+2.06-2ubuntu10 amd64 [installed]
grub-efi-amd64/jammy-updates,jammy-updates,now 2.06-2ubuntu10 amd64 [installed]
grub-pc-bin/jammy,jammy,now 2.06-2ubuntu7 amd64 [installed]
grub2-common/jammy,jammy,now 2.06-2ubuntu7 amd64 [installed,automatic]So, all the UEFI GRUB can be deleted… Note that if you run the following command on a UEFI system, it will stop booting.
apt purge shim-signed grub-efi-amd64 grub-efi-amd64-signed grub-efi-amd64-bin --allow-remove-essential
apt autoremove --purgeThis solved all my issues.
I guess the Hetzner install image might have a bug because it makes no sense to install GRUB UEFI on a BIOS system, especially with no ESP.
This cost me way too much time (the dreadful realization that you can never get that, or any time, back in your mortal life). 😵💫
Thank you for this blogpost! In the meantime, Hetzner have fixed their setup and for currently installed systems, it’s not a problem any more. But from time to time I find systems in our quite large installation which have been installed in the problematic time range and which suffer from this problem. Purging the shim stuff always solves it.
Also thank you, the decision ‘UEFI GRUB can be deleted’ solved it for me after several different trials
Coming late to this party. Got a Hetzner server via server auction in December and used the installimage to install Ubuntu 22.04.
“efibootmgr” command returns “EFI variables are not supported on this system” and lsblk shows no ESP partition on any of the 4 drives.
“find /boot/efi” gives this:
/boot/efi
/boot/efi/EFI
/boot/efi/EFI/BOOT
/boot/efi/EFI/BOOT/BOOTX64.EFI
“gdisk” shows GPT present and “apt list –installed ‘*grub*'” shows this:
grub-common/jammy-updates,jammy-updates,now 2.06-2ubuntu7.1 amd64 [installed,automatic]
grub-efi-amd64-bin/jammy-updates,jammy-updates,now 2.06-2ubuntu14 amd64 [installed]
grub-efi-amd64/jammy-updates,jammy-updates,now 2.06-2ubuntu14 amd64 [installed]
grub2-common/jammy-updates,jammy-updates,now 2.06-2ubuntu7.1 amd64 [installed,automatic]
So no “shim-signed” and no “grub-efi-amd64-signed” installed.
But I did get some odd behavior on doing an upgrade of grub-efi-amd64 via apt upgrade: an ncurses warning message that said this:
“Configuring grub-efi-amd64
You chose not to install GRUB to any devices. If you continue, the boot loader may not be properly configured, and when this computer next starts up it will use whatever was previously configured. If there is an earlier version of GRUB 2 in the EFI system partition, it may be unable to load modules or handle the current configuration file.
If you are already using a different boot loader and want to carry on doing so, or if this is a special environment where you do not need a boot loader, then you should continue anyway. Otherwise, you should install GRUB somewhere.”
It then offered me a choice between “yes” (which was assenting to continued installation) or “no” but choosing “no” put me right back to the same screen. So I hit yes.
Now everything is as indicated above. As I say I don’t have “shim-signed” or “grub-efi-amd64-signed” installed. But I do have “grub-efi-amd64-bin” and “grub-efi-amd64” installed.
Are these going to cause problems? And should I delete them in. your view?
Thanks!
Works for me.
Just wanted to say thank you!! Have had this annoying error pop up, almost since the beginning of installing Ubuntu 22.04 LTS on my Hetzner server. Couldn’t figure out what was up.
Thanks for explaining and providing a solution :) Worked wonderfully.
Thanks, it’s the solution I was looking for.
If ran in same issue and checked all. I’m little bit confused can if I delete my grub message from Author
“So, all the UEFI GRUB can be deleted… Note that if you run the following command on a UEFI system, it will stop booting.” and if i reboot my machine will it still starts normally and not crash it’s boot ?
I have the same problem as M, same output from gdisk on all 4 disks (rust, not nvme). /boot/efi is present, however efibootmgr shows the same output as yourself. As with M, I’m concerned now about trying to fix it, or indeed reboot. I’ll open a ticket but any tips you might have would be appreciated!
Hetzner came back to me very quickly and confirm that this is a known bug and that removing shim-signed and grub-efi-amd64-bin fixes the problem, and indeed I’ve done that, updated and rebooted, all good. Thanks for your post through, it convinced me that the problem was with the way Hetzner set things up, and emboldened me to open a ticked with them. :)
>>>
This is a known bug in Ubuntu 22.04. This has already been fixed in the newest version.
———–>%———–
https://bugs.launchpad.net/ubuntu/+source/grub2-signed/+bug/1971059
————%%———–
apt remove shim-signed grub-efi-amd64-bin –allow-remove-essential
————%<———-
<<<
Thanks for letting me know!
This is very informative, and thanks to Sindastra (and Adam) for detailing the cause and solution.
One note – if you try to copy/paste from Adam’s message, there must be some invisible characters (even after you reconcile the single vs double hyphen issue). The only way I was able to get the apt remove to work was to hand type in the command. _Something_ within “apt remove shim-signed grub-efi-amd64-bin –-allow-remove-essential” was not being understood by apt, as no matter where I put the allow-remove-essential flag, it would complain that it couldn’t find package “–allow-remove-essential”.
I have the same issue with a SX64 server. efibootmgr returns the same message.
Other packages however are installed luckily. Is there an official statement from Hetzner yet?
Look at the comment from Adam, there’s an official response there. 🙂
After hours of debugging and research, I just found your blog.
Thank’s a lot, you’re the only one with a working solution !
Hi, great post!
I’m also on Hetzner and having the same issue. I had come to the same conclusion – that grub-efi-amd64 were installed in error and need to be removed. Don’t really wnat to take the risk though.
I get a different result to you when running ‘sudo gdisk /dev/nvme0n1’
GPT fdisk (gdisk) version 1.0.8
Partition table scan:
MBR: protective
BSD: not present
APM: not present
GPT: present
Found valid GPT with protective MBR; using GPT.
Command (? for help):
Not sure it’s going to be safe for me to purge the efi apps you mentioned above.
Any thoughts
M
Well, this only tells me that you are using GPT instead of “MBR”. Did you try running lsblk to see if you have an ESP (UEFI partition)? What do you see in /boot/efi?
Thanks a lot, you saved me a *lot* of time :D