HOWTO: Figure Out What File Is Missing, and Where
One of the really maddening things about Linux (and UNIX in general) is that files are stored in random places. That is, they are not really random, but each application has its own idea of where it wants to look for stuff. You, as a user, have an inkling of what’s missing (maybe because of an error message, or because you researched on a search site). But you have not the foggiest idea of where you should look for it.
For instance, you may have installed a Python extension. Then you start python, and type:import myextension. And Python will spit out this:
Traceback (most recent call last):
File “<stdin>”, line 1, in <module>
ImportError: No module named myextension
Of course, you know that you installed myextension. So why can’t Python find it?
Well, fret no more, there is an almost simple way of figuring this out in a jiffy, using the Linux utility strace.
strace (pronounced ess-trace or strayce) and its related frenemy truss are two utilities that report back what requests an application made to the system. Graciously, the system allows users to ask for this information, so all you need to do is fire up strace, feed it your command, and there you go, you know everything the system knows (as far as calls are concerned).
Practically, you use strace as a prefix to the command you want to run. If you wanted to trace python, for instance, you would type:strace python. If you did that already (you are a fast typer!), you would see a ton of garbage on the screen, near whose end you’d find the line:
write(1, ">>> ", 4>>> ) = 4
That is the call python makes to the system to have its prompt printed out. What that means is that the system call write is invoked on channel 1 (standard output), for a length of 4 bytes. The result is 4, which is the number of bytes written. The >>> after the first 4 is the side effect of the call: since we asked to write to standard output, the prompt is written into our strace stream.
I’ll make your life easier right now and will tell you which command line options you want to give strace, with just a bare minimum of explanation. First, you want to add the -f option (follow). This tells strace to go after all the processes sparked by your initial call. You will typically need that, so just add it in automatically. Next, you really really don’t want the tons of output from strace to mix in with your application output, so you send the strace output to another file with the -o option. Finally, you want to restrict to file system calls (since we are only looking for files), which is the -e trace=file.
If we wanted to trace python to find out why it’s not finding out myextension, you would type:
strace -f -o /tmp/strace.out -e trace=file python
Now python behaves normally, albeit quite a lot more slowly so. If you type in import myextension, you will get the same error as before (since strace typically doesn’t change what the application does). When you exit, though, you will have the information you need in the file /tmp/strace.out.
In my case, the file contains 60 references to the string myextension, grouped in 10 chunks of 6 each. The first group of six looks like this:
15704 stat(“myextension”, 0x7fff1dd88ed0) = -1 ENOENT (No such file or directory)
15704 open(“myextension.x86_64-linux-gnu.so”, O_RDONLY) = -1 ENOENT (No such file or directory)
15704 open(“myextension.so”, O_RDONLY) = -1 ENOENT (No such file or directory)
15704 open(“myextensionmodule.so”, O_RDONLY) = -1 ENOENT (No such file or directory)
15704 open(“myextension.py”, O_RDONLY) = -1 ENOENT (No such file or directory)
15704 open(“myextension.pyc”, O_RDONLY) = -1 ENOENT (No such file or directory)
The number 15704 is the process ID. You can safely ignore it for now. The second component (before the parenthesis) is the system call. We have stat, which is used to get information about a file, and open, used (you guessed it) to open the file. (Hint: stat is used in the first line instead of open because it’s assumed to be a directory name).
All six calls are followed by a result code after an equal sign. In all cases, that’s -1 ENOENT (No such file or directory). ENOENT means that the file is not there, which we already knew. What we didn’t know, though, is what files the system is asked to look for.
First, there is a directory with the name we provided. It is not preceded by a location, so we are looking in the current working directory. Once the specified directory is not found in the current working directory, the system is asked to find a file named myextension. x86_64-linux-gnu.so. Then myextension.so. Then myextensionmodule.so. Etc. The order is important, as the search is typically aborted at the first file found.
The next group of 6 starts with this line:
15704 stat("/usr/lib/python2.7/myextension", 0x7fff1dd88ed0) = -1 ENOENT (No such file or directory)
You are now pros in reading this! The only difference from the first call of the first group is the location of the file (directory) we are looking for. This time, we look in /usr/lib/python2.7/. The remainder of the group of six is looking in the same directory. After that, we move on to the directory /usr/lib/python2.7/plat-x86_64-linux-gnu/
Once you exhaust the list in the strace file, you have all the locations where the file was searched, and under what name. Maybe there is a version conflict in your myextension, and it wound its way into the python2.6 location. Or maybe it’s a 32-bit version, and it’s hiding in the plat-x32 directory. No matter what: now you know where that file should be.