Pegasus Workflow Management System
The Pegasus project encompasses a set of technologies that help workflow-based applications execute in a number of different environments including desktops, campus clusters, clouds, and national cyberinfrastructures including grids and supercomputers. Scientific workflows allow users to easily express multi-step computations, for example retrieve data from a database, reformat the data, and run an analysis. Once an application is formalized as a workflow the Pegasus Workflow Management Service can map it onto available compute resources and execute the steps in appropriate order. Pegasus can easily handle workflows with several million computational tasks.
Kickstart is used to launch computing tasks, monitor the behavior of tasks, and report information about tasks and the hosts on which they were executed. Kickstart was originally designed to be used with Pegasus, but it can also be used separately. Kickstart uses procfs and other query mechanisms to gather basic information about the host, such as the number of CPUs and CPU cores, the amount of used and free system memory, the number of running tasks, the system uptime, and the hostname. Kickstart uses ptrace() to intercept only process creation (fork(), vfork(), clone(), exec()) and exit() events.