Grub ошибка file not found

I’m in the middle of a similar issue (incidentally also on arch)

Grub is failing to find this file and run because it is using an incorrect «prefix» in my case.

Here is what you do. You boot into grub rescue mode, then you simply figure out how to make it boot.

First you run set this will list variables, for example mine is

cmdpath=(hd0)
prefix=(hd1,msdos3)/boot/grub
root=hd1,msdos3

Now, the prefix is the variable where grub looks for the normal.mod file. In my case hd1,msdos3 is the same as /dev/sdb3 (similarly, hd0,msdos1 would be /dev/sda1) what you may want to do to see a list of valid partitions is type in ls

Now, in my case, again, grub was installed on /dev/sdb1 which was mounted as /boot in my arch partition, so the correct prefix would be (hd1,msdos1)/grub

So for me to boot, I need to do this:

set prefix=(hd1,msdos1)/grub
insmod normal
normal

In your case you will have to either remember or guess which partition you installed grub on. You can guess wrong, it won’t do any harm, the insmod command will simply fail and you can try again with another partition.

After this, grub loads as it normally should, and I can pick out of the list what I want to boot. Normally when a mess like this occurs, reinstalling grub to your mbr (using grub-install) should fix it permanently so you don’t have to do this every time you boot. I however am having a lot of difficulty finding out what to do if fixing it isn’t this easy (or I’d share what you should do).

Only if this fails (e.g. if the prefix is correct but you still can’t boot) should you resort to live or rescue cds to work around the problem (it is best to avoid that)

Kernel updates are vital for security and feature additions. But if not performed with caution, these updates can render your server unavailable or down.

In our role as Outsourced Tech Support services for web hosts, kernel updates are a commonly performed task in servers.

Today, we’ll discuss the error ‘Error 15: File not Found’ that occurs in CentOS servers when trying to boot after changes in grub.

‘Error 15 file not found’ in CentOS server – Causes

The error ‘Error 15: File not found’ happens when you try to boot a server with the updated kernel after a kernel update or patch.

Error 15: File not Found

Press any key to continue

This error is caused due to some issues with the GRUB, which is the boot loader used in CentOS servers.

The main reasons why you get this error 15 are:

1. Wrong entries in GRUB

In majority scenarios, the ‘Error 15: File not Found’ issue pops up due to an incorrect entry in the GRUB boot loader configuration file at  /boot/grub/menu.lst .

A typical entry in GRUB menu for CentOS 6 servers looks like:

title CentOS (2.6.18-238.19.1.el5PAE)
root (hd0,0)
kernel /vmlinuz-2.6.18-238.19.1.el5PAE ro root=LABEL=/
initrd /initrd-2.6.18-238.19.1.el5PAE.img
title CentOS (2.6.18-238.9.1.el5PAE)

The cause for the error 15 can be anything from an incorrect drive, wrong path or partition name, or any other mistakes in this kernel entry in the GRUB menu.

By default, the kernel images are found in /boot partition. In servers where the partition system is different, these paths have to be changed accordingly in GRUB menu.

2. Corrupt GRUB configuration file

A GRUB configuration can get corrupt during system updates. When GRUB is installed in the wrong partition, the system will not be able to boot at all.

If the required modules are missing in the expected /boot partition, system cannot boot. The MBR (Master Boot Record) can get damaged when a secondary OS is installed.

In some cases, the initrd declaration or kernel path may be missing for any listed kernels in the GRUB menu. This will prevent the server from booting into that kernel.

3. Defective hard disk

“Error 15 file not found” may also occur in CentOS servers due to change in BIOS settings, changes in hardware or errors in hard disk.

In the case of servers with multiple hard disks or OS installations, there are chances of creating multiple boot partitions and GRUB mixing up between them.

‘Error 15 file not found’ in CentOS server – Fixes

Depending on the scenario faced and the level of access to the server, we follow different methodologies to fix the error 15.

1. Fixing GRUB entries

When customers contact us saying their server fails to boot due to Error 15, we boot the server into rescue mode and access it via remote console such as IPMI or web portal.

Once the boot prompt loads, we edit the GRUB entries and fix the error. In cases where GRUB is fully corrupt, we reinstall the bootstrap code and then boot the server.

2. Test reboots

In the server where we perform kernel updates or patches, we take adequate backups. We configure GRUB with a fail-over entry to avoid a boot failure.

This fall back configuration ensures that even if a new kernel doesn’t boot due to any errors, there is a fall back working kernel entry which will boot and make the server available.

3. Hard disk inspection

To rule out ‘error 15 file not found’ in CentOS servers due to hard disk errors, we conduct periodic audits to see if there are any bad sectors or other errors.

For servers with messed up hard disks or OS installations, we backup the data and perform re-partition or re-installation as necessary, to fix the errors.

Conclusion

GRUB errors pop up mainly during kernel updates. Today we saw how our Support Engineers resolve and fix ‘Error 15 file not found’ in CentOS servers.

var google_conversion_label = «owonCMyG5nEQ0aD71QM»;

Heya! Hope you all are doing well. Me too. So, today I am gonna share something amazing and insightful. I have attended a meetup held by LinuxChix last Sunday. So after the talks, there held a chaos engineering session. Even though I hadn’t attended any session like this before, I was eager to experience these things myself. Because whenever I do something on my system, I am always very careful in case it doesn’t run into any system failures. So, this was a golden opportunity for me. In the session, my friend Neeraj and I were given a system with the error shown as:

error: file '/grub/i386-pc/normal.mod' not found.
grub rescue>

We have to resolve this error and get into the system safely. We were at the grub-rescue prompt. Basically grub-rescue prompt is somewhat more restricted as compared to grub prompt. This is because not all commands run in this shell. Only some selected commands work at the grub-rescue prompt. Very first command which we tried was “ls” command which worked and it showed the all the devices or partitions which were present in the system. So, the  output was somewhat like:

grub rescue> ls
(hd0) (hd0,msdos1) (hd0,msdos2) (hd0,msdos4) (hd0,msdos5) (hd0,msdos6)

These are the partitions which were present in the system. After that we run the “ls” command on each partition, which gives us some unexpected output. It tells if  the file-system is known or not. Grub only recognizes the “ext2” file-system. So, we were left with only three partitions to work with which were (hd0,msdos2) (hd0,msdos4) (hd0,msdos6). Now you might be wondering what does this partition name means. So, if I would explain in short, it would be like

• (hd0,msdos1) = /dev/sda1
• (hd0,msdos2) = /dev/sda2

and so on. Now, in words (hd0,msdos1) will be the 1st partition in hard-drive 0 and (hd0,msdos2) means 2nd partition in hard-drive 0. Since the system had only single hard-drive and it’s been divided into partitions, so all partition contains (hd0). Partitions except the known file systems showed the error of “Unknown file system“.

Now, we were left with the 3 partitions to look into for the required file which is “normal.mod“. Before going any further, we should understand the need for this file. Basically, normal.mod is a module which grub loads at the boot time which provides the use of normal command. Using normal command we can display the GRUB menu. Since, the grub couldn’t locate the file in its usual location, it means either the file has been displaced or it could have been deleted somehow.

Case 1: ‘normal.mod’ is displaced from its default location

So, let’s consider the first case for now ie the file has been displaced and could be in some other partitions. Grub looks for this module in the /boot/grub/i386-pc/ by default(Here i386-pc is specific, it depends on your system architecture). For that we have to check the contents of each of the three partitions in order to ensure if the file is present there or not. We can do this using ls command.

grub rescue> ls (hd0,msdos1)/boot/grub/i386-pc/

We need to do this for every partition. If we get the required module in any of the partition. Let’s say we get it in the 1st partition. So,  now we need to set the prefix variable to the partition which contains the required file. prefix variable is set to the location of grub directory at the time of grub installation. So, it is possible that grub couldn’t locate the normal.mod because the prefix variable is pointing to the wrong partition. So we can set the prefix variable using set command. Also, we need to set the root to the same partition as that of prefix . For eg

grub rescue> set root=(hd0,msdos1)
grub rescue> set prefix=(hd0,msdos1)/boot/grub

Now, since we have set the correct path to root and prefix, we can use the normal module now. So for that we have to import the module. To import it, we would use insmod command.

grub rescue> insmod normal
grub rescue> normal

Now, the grub menu would be shown and you can enter into the system safely.

Special case:

In this there can be situations like the normal.mod is not present in the exact location we are looking into. Like in our case, we are trying to locate it into “(hd0,msdos1)/grub/boot/i386-pc/”. But what if it is present in the location “(hd0,msdos1)/root/”, then pointing prefix to this location won’t be useful and also we can’t transfer it to its correct location. So, to resolve cases like this what we can do is use insmod command to import the normal module as:

grub rescue> insmod (hd0,msdos1)/root/normal.mod
grub rescue> normal

Using this you would be able to use the normal module and get into the GRUB menu. But since it isn’t placed in its correct location, this means that there were some problem while installing grub. So, if you reboot the system, you would again be thrown into the grub rescue prompt and have to follow the same procedure. It would be better if you install the grub again using the following command after entering into the system terminal:

$ sudo grub-install /dev/sda

And then you are good to go.

Case 2: ‘normal.mod’ is not present

Well, the whole procedure was for the case when the required module is misplaced. But what if it isn’t present at all in any partition and somehow got deleted. To your curiousness, this was the case with us in chaos engineering session. We tried to find the module in each known partition but couldn’t locate it anywhere. So, now we have to do the work which normal module would’ve done if it would be present. Therefore we have to find the kernel files and boot the system manually.

To boot the system we would require the kernel file which would be something like “vimlinuz-linux” and the file “initramfs-linux.img“(Sometimes, you can find it as “initrd-linux.img”). The vimlinuz-linux is actually the kernel image which mounts the root partition on the hard-disk. The initramfs-linux.img loads the futhur required modules needed for booting of the system. So, very first thing we did was to locate these files in any of the known partitions. So, we looked for these files under the /boot directory using the same ls command as:

grub rescue> ls (hd0,msdos2)/boot/
# some output
grub rescue> ls (hd0,msdos4)/boot/
# some output

We finally found the above mentioned files in the (hd0, msdos4) partition. This was the great success after searching through the internet and several wrong attempts for a long time. Now, we have to set the root to the partition which contains the kernel image using the set command so that it always points to the right location.

grub rescue> set root=(hd0,msdos4)

Next thing to be done is to run the kernel image. So for that we need to import the linux module using the insmod command and also the initramfs-linux.img file using the initrd command:

grub rescue> insmod linux
grub rescue> linux /boot/vimlinuz-linux root=/dev/sda4
grub rescue> initrd /boot/initramfs-linux.img

And finally the boot command, which would boot the system(using the above files) and allow us to get into the system

grub rescue> boot

Yay! finally  we were able to enter into the system. We were feeling like a cracker who has solved some crucial problem(just kidding!). But yet the problem existed ie the normal module isn’t present. In case if we would reboot the system, same error would’ve been shown and we have to repeat the same procedure. So to resolve it we need to do the same thing which we did for the 1st case. Do you remember it? Yeah, we need to re-install the grub to repair the damaged or deleted files using the system terminal.

$ sudo grub-install /dev/sda

As soon as the installation successfully completed, we reboot the system to check if it actually worked(we both crossed our fingers at that moment). And Hurrah! we have successfully solved the problem.

We both were appreciated by the people there. It was really an amazing experience. Cocoa was our mentor during the session and helped us in understanding things and showed us the path to solve the problem. I really look forward to attend more of these sessions. I think you should also. So, watch the LinuxChix space for more of these session.

References:

1. https://wiki.archlinux.org/index.php/GRUB#Normal_loading
2. https://askubuntu.com/questions/266429/

For now, bidding you goodbye. Meet you in the next blog. Till then…

Be curious and keep learning!