Getting Started with Barabash

Barabash Script Anatomy

Barabash script is a regular Python script that can have any name e.g. build.py. The script should

• import barabsh module
• define barabash operations
• invoke barabash run() function

Example:

from barabash import Files, MapOp, ReduceOp, run

# set of source code files definition
src = Files("src", ["a.c", "b.c"])
# compilation
objects = MapOp("objects", src, "%.o:%.c", "{CC} -c {in} -o {out}")
# linking
ReduceOp("program", objects, "{CC} -o {out} {in}")

run()

Line 1 contains Barabash imports.

Line 4 declares a set of files: src.

Lines 6-8 define two Barabash operations:

• objects, a map operation that compiles .c sources files into .o objects
• program, a reduce operatopm that links .o objects into program executable

Finally, line 10 starts Barabash script.

To run the script type in command line:

$ python build.py program

Operations

Operation is a base concept in Barabash. It performs an action potentially using inputs and producing outputs. Operations has a name that can be used to call the operation from command line.

Sample operation:

objects = MapOp("objects", src, "%.o:%.c", "{CC} -c {in} -o {out}")

where:

• “objects” is a name of operation
• src is an input for operation (dependency), it can be either barabash.core.Files object or some other barabash.core.Operation or list of them
• “{CC} -c {in} -o {out}” is an action

There are 3 kinds of operations in Barabash:

• map operation, defined by barabash.core.MapOp class,
• reduce operation, defined by barabash.core.ReduceOp class,
• command operation, defined by barabash.core.Command class.

All these classes are derived from barabash.core.Operation class.

Following arguments are common to all operation types.

name is a name of operation, it can be referenced in other barabash.core.Operation as dependency.

deps is a dependency or a list of dependencies. A dependecy can be a barabash.core.Files or other barabash.core.Operation. Files class provides directly list of files. In case of Operation class it also provides list of files that are produced by this operation i.e. its outputs. This is not true only for Command class which does deliver any output files. It is simply required that the Command must be always executed.

An operation is only executed by Barabash when operation’s outputs are older then inputs, i.e. outputs must be updated. Again, this is not true for Command operation which is always executed.

action is a Python function or a Bash script that will be executed in operation. More details in Actions chapter.

Actions

Action is executed:

• when outputs of an operation are outdated comparing to its inputs in case of map and reduce operations or
• always in case of command operation.

Action can be a Python function or a Bash script as a multiline string.

Both of the action types receive input file name or names and output file name. In case of Python functions all that data is passed directly as a env dictionary with a predefined keys:

• env[“in”] - a full path to input file or a list of full paths to input files,
• env[“out”] - a full path to output file.

Example:

def touch_action(env):
   os.system("touch %s" % env["out"])

and its usage in operation:

MapOp("touch", Files(["a.in"]), "%.out: %.in", touch_action)

In case of Bash script this variables can be references in script string using curly bracket notation: {in} and {out}.

Example:

touch {out}

and its usage in operation:

MapOp("touch", Files(["a.in"]), "%.out: %.in", "touch {out}")

Map Operation

Map operation is defined by barabash.core.MapOp class. This operation transforms input files to output files using given action one by one for each input file separatelly.

Map operation takes several arguments:

MapOp(name, deps, output, action, indirect_deps=[])

output is a mapping rule like in GNU make. Example:

"%.o: %.c"

which maps .c files to .o files e.g. in compilation operation or:

"%.html: %.py"

which maps .py files to .html files e.g. in pygmentize operation.

% sign is matched to whole file path.

indirect_deps is an additional set of dependent elemets (files and barabash.core.Operation) that is not treated as inputs, e.g. header files in C compilation.

Example:

# compilation
objects = MapOp("objects", src, "%.o:%.c", "{CC} -c {in} -o {out}")

Reduce Operation

Reduce operation is defined by barabash.core.ReduceOp class. This operation takes input files and transforms them in one action into one output file.

Reduce operation takes several arguments:

ReduceOp(name, deps, output, action)

output is a name of produced file, this argument can be skipped if the operation name is already a file name.

Example:

# linking
ReduceOp("program", objects, "{CC} -o {out} {in}")

Command Operation

Command operation is defined by barabash.core.Command class. This operation is always executed regardless of its dependencies. It does not produce any output files.

Command operation takes several arguments:

Command(name, deps, action)

They were already described above.

Example:

# list files in current directory
Command("ls", "ls -al")

Invoking Script

The Barabash is executed as regular Python script with expected command line arguments.

$ python ./build.py -h

output:

usage: build.py [-h] [-g] target

Barabash Build Script.

positional arguments:
  target       a target or help to get list of available targets

optional arguments:
  -h, --help   show this help message and exit
  -g, --graph  generate a graph image of all operations dependencies to
               barabash.png file

Pass help as argument to get all possible targets/operations:

$ python ./build.py help

output:

Barabash script for Barabash
Barabash Help:
upload-website
         combine-website
                 build-docs
                 render-website
                         pygmentize-examples
                 tests
build-pkg
upload-pkg

Executing specific operation – pygmentize-examples:

$ python ./build.py pygmentize-examples

output:

Barabash script for Barabash
BB(pygmentize-examples):
pygmentize -O style=friendly -f html -o /home/gof/barabash/www/ex3.html /home/gof/barabash/www/ex3.py
pygmentize -O style=friendly -f html -o /home/gof/barabash/www/ex1.html /home/gof/barabash/www/ex1.py
pygmentize -O style=friendly -f html -o /home/gof/barabash/www/ex2.html /home/gof/barabash/www/ex2.py

---

blog comments powered by Disqus