Executor - a Simple Directory/File-based Distributed Computing System
Looking for security information? Check here.
To get an overall view of how to use this software, please check the
quick start guide or
these nice images.
This software is released under the modified BSD
You'll find some example programs for the simple Java executor here.
a simple Java demo that plays "guess a number" with you
outputs hello followed by the program's parameters (if any)
hello.sh a similar program for Unix shells
salesmen.py the classic traveling salesman
program with 11 cities. Can take many (10) minutes on lower end computers.
darts, an example for the R statistics package.
This ZIP file contains both the data and the executable, but you can just
upload it as File#1 in the Executor. Using a sample from
(originally Wall Street Journal), we calculate
some summaries and indicate that in general trained experts trade a bit
better than can be gained by throwing darts on the stock index.
What does Executor do?
Executor is a small but perfectly functional Java (1.5 or higher)
JAR executable that is used to run programs in shared computer
resources like a pool of computers that you and your friends own. Moreover,
it can be used as a simple file sharing program: you can access the
files of your home computer from your office computer.
uses a shared or synchronized file system. The user creates a task in
one computer (just by uploading a file) and another computer that has
the required program will execute the task.
A class diagram of the program is
Who wrote it/uses it? What's the aim/status? Where can I get it?
I am node0, the program was written during my spare time 2013- and
I use it daily. Mail: Executorfirstname.lastname@example.org
I plan to incorporate changes and more functionality but I want to keep
it small and simple.
You can download the software from
Q: Why would I use Executor?
A: If you have a powerfull desktop computer with lots of installed
programs, and a small laptop, you can launch your tasks using the
laptop, they are executed in the desktop computer and the results
delivered in the shared directory so you can recover them with the laptop.
Q: Why wouldn't I use something better like Condor, SGE, BOINC?
A: You should. But this one is really simple to set up and use.
Q: So, can I use Microsoft Office that is in my Windows computer by
submitting a Word file by Executor in my Linux box?
A: Sadly, no. We can only manage batch type processing. Executor
cannot transfer the GUI of MSoft Office into your Linux.
Q: What about security?
A: Optional, based on GnuPG. Please see
Q: What is the protocol by which computing requests are submitted and
A: The protocol is completely file-based. When a user submits a task by
the submit GUI, Executor (1) creates a file with uniqueID.jd (jd for job
description). When Executor (2) finds a jd file, it checks if it can execute
it. If it can, it creates a file with the same uniqueID and
extension "run". Data about execution (exec host, start time..) is written
in this file.
After the execution, the exit code is written in this file, too.
Executor (3) can notify the user that the task has been executed, simply
by observing the .run files.
Q: There is no scheduling?
A: Yes, very primitive. All computers write a .info file in the shared
directory. This contains some characteristics (cpu, memory, supported
programs) of the host. When a task arrives, any computer will check if they can
run it. But if the host notices (by checking the files) that there is another
more powerfull computer available for the task, it will wait for some
time. If after the wait none has adopted the task, it will. This is
why the scheduler class is called PoliteScheduler.
Q: How about interactive tasks, tasks that ask input from the user?
A: There is a simple (file-based, again) protocol called ASK-TELL.
A program that wants input writes a question in its stdout, like this
ASK what is your name?
Executor stdout notices this question and creates a pop-up in the host
where the job was submitted. The program reads the answer in its stdin.
Please note that process management is a bit tricky. Your program must
flush the stdout instantly after writing the ASK question, otherwise
it will just wait forever. Moreover, when reading the reply, your
program must be prepared to strip flow control characters \002 \004 from
Q: What is the syntax of the job description file?
A: Some of the attribute names can sound peculiar since they
have been adopted from a more complicated program, but here they are:
clientid=xxxx.info defines the Executor who submitted the task
binaryinputfilename=filename the file that was given as File #1 by the submit GUI
workercommandline=command that the Executor will run
arguments=[optional] extra arguments from the submit GUI
datainputfilename=[optional] more input files
Q: Is there a Java API?
A: Yes, it is included in the same JAR file:
* Submit a task such that the executable is in myFilename and myCmd is
* its command line.
public String submitTask(String myFileName, String myCmd, String taskName)
* API-type method: returns hosts/nodes that can execute your task.
* @param exTension extension like "sh" or a file name like "foo.sh"
* @param coreWeigth if true, return N*H for H where N is available cores.
* @return list of info file names
public String getIdsSuitableForTask(String exTension,
Q: Is there a command line interface?
A: Yes, it is included in the same JAR file:
-H filename: list hosts that can execute filename
-C filename: list cores that can execute filename
-S filename: submit task
Q: What new functionality is planned next?
A: If you look at a job description file, it actually looks like a sequence of variables.
I have been thinking: could we support programming language constructs like FOR loops in
these files, too? However, we would need to support this by the GUI, and it can become