[MINC-users] Announce - qbatch 1.0 - Execute shell command lines in parallel (serial farm) on SGE/PBS clusters

[Simon Eskildsen]

Sounds great. Is this related in any way (other than the name) to Andrew's
qbatch script? https://github.com/andrewjanke/qbatch


On Mon, May 9, 2016 at 9:38 PM, Gabriel A. Devenyi <gdevenyi at gmail.com>
wrote:

> We (Gabriel A. Devenyi and Jon Pipitone) would like to announce the 1.0
> release <https://github.com/pipitone/qbatch/releases> of qbatch, a
> command-line tool for easily running a list of commands in parallel (serial
> farming) on a compute cluster. This tool takes the list of commands,
> divides them up into batches of arbitrary size, and then submits each batch
> as a separate job or as part of an array job. qbatch also gives you a
> consistent interface to use to submit commands on PBS and SGE clusters or
> locally, (support for others are planned/in testing, PRs welcome) while
> setting requirements for processors, walltime, memory and job dependencies.
> This tool can be used as a quick interface to spread work out on a cluster,
> or as the glue for for connecting a simple pipeline to a cluster (see
> https://github.com/CobraLab/antsRegistration-MAGeT for a sample
> implementation)
>
> The target audience of qbatch is two-fold: it is immediately available for
> users of PBS or SGE clusters to simplify their job construction, in
> addition, through the use of environment variables, cluster administrations
> can craft a default qbatch deployment which allows new cluster users to
> quickly submit jobs which honours the cluster’s policies.
>
> For more information, check out our github webpage here:
> http://github.com/pipitone/qbatch
> qbatch is also available in pypi via pip install qbatch
>
> —
> Gabriel A. Devenyi B.Eng. Ph.D.
> Research Computing Associate
> Computational Brain Anatomy Laboratory
> Cerebral Imaging Center
> Douglas Mental Health University Institute
> Affiliate, Department of Psychiatry
> McGill University
> t: 514.761.6131x4781
> e: gdevenyi at gmail.com
> ​
> _______________________________________________
> MINC-users at bic.mni.mcgill.ca
> http://www.bic.mni.mcgill.ca/mailman/listinfo/minc-users
>