HPC automation using Python
One of the central tenets of good research software engineering practice is to automate as much of your computational workflow as possible. Such automation can significantly speed-up the throughput of computational research as well lead to increased accuracy and ease of reproducibility.
Raphaël Démolis, a researcher at Sheffield’s Department of Civil and Structural Engineering, has been working on such an automated workflow using the Python programming language, the commercial finite element analysis package, Abaqus, and the University High Performance Computing system, iceberg. With a little help from Research Software Engineer, Mike Croucher, Raphaël recently managed to automate his entire workflow (see testimonial at http://www.walkingrandomly.com/?page_id=5122 for details)
The key software technologies used were the Python modules Paramiko and Fabric and Mike will be writing up an example of how to use them with Sheffield’s High Performance Computing system soon.