./executable: cannot execute binary file

Question

I have a script that works well when I ssh to the server to execute it myself, but has problems when Hudson, a continuous integration server, runs it.

I am automating tests on an embedded linux system (the target). The target is connected to Server A (RHEL 5) via serial and operated over minicom. Server B (FC 12) builds the tests that actually run on the target, and can ssh to Server A. Server C (RH) hosts Hudson, with Server B as a slave.

I've written a runscript (http://linux.die.net/man/1/runscript) script to do everything needed on the actual target; it boots the image, mounts a directory from Server B and executes the tests. A bash script on Server B invokes minicom with the runscript script along with some companion actions. I have a bash script on Server B which uses

ssh -t -t ServerA bashScript.sh

to get those tests run on the target. I am on Server C, I can get those tests run by ssh'ing to Server B and executing the script that ssh's to Server A which executes minicom with runscript. Whew. To review:

Server A: Hudson uses its slave mechanism to ssh to Server B.

Server B: kickOffTests.sh has the line ssh -t -t ServerA runTests.sh

Server A: runTests.sh calls a perl script which invokes minicom -S my.script ttyE1

Target, after booting: Mounts a directory from Server B, where the tests are, and enters that directory. It invokes yet another bash script, which runs the tests, which are compiled C executables.

Now, when I execute any of these scripts myself, they do what they should. However, when Hudson tries to do the same thing, over in the minicom session it complains about a line in the "yet another bash script" that invokes the C executable, ./executable, with ./executable: cannot execute binary file

I still have a lot to learn about linux, but I surmise this problem is a result of Hudson not connecting with a console. I don't know exactly what Hudson does to control its slave. I tried using the line export TERM=console in the configuration just before running kickOffTests.sh, but the problem remains.

Can anyone explain to me what is happening and how I can fix it? I cannot remove any of the servers from this equation. It may be possible to take minicom out of the equation but that would add an unknown amount of time to this project, so I'd much prefer a solution that uses what I already have.

Answer 1

The message cannot execute binary file has nothing to do with terminals (I wonder what led you to think that — and I recommend avoiding making such assumptions in a question, as they tend to drown your actual problem in a mess of red herrings). In fact, it's bash's way of expressing ENOEXEC (more commonly expressed as exec format error.

First, make sure you didn't accidentally try to run this executable as a script. If you wrote . ./executable, this tells bash to execute ./executable in the same environment as the calling script (as opposed to a separate process). That can't be done if the file is not a script.

Otherwise, this message means that ./executable is not in a format that the kernel recognizes. I don't have any definite guess as to what is happening though. If you can run the script on that same machine by invoking it in a different way, it can't just be a corrupt file or a file for the wrong architecture (it might be that, but there's more to it). I wonder if there could be a difference in the way the target boots (perhaps a race condition).

Here's a list of additional data that may help:

• Output of file …/executable on server B.
• Some information about the target, such as the output of uname -a if it's unix-like.
• Check that the target sees the same file contents each time: run cksum ./executable or md5sum ./executable or whatever method you have on the target just before yet-another-bash-script invokes ./executable. Check that the results are the same in the Hudson invocation, in your successful manual invocation and on server B.
• Add set -x at the top of yet-another-bash-script (just below the #!/bin/bash line). This will produce a trace of everything the script does. Compare the traces and report any difference or oddity.
• Describe how the target is booting when you run the scripts manually and when Hudson is involved. It could be that the target is booted differently and some loadable module that provides the support for the format of ./executable doesn't get loaded (or is not loaded yet) in the Hudson invocations. You might want to use set -x in other scripts to help you there, and inspect the boot logs from the target.

Answer 2

This can happen if you're missing the shebang line at the top of your script. Make sure the script starts with:

#!/bin/bash

This only showed up for me when I ran the script with sudo -u <user>