syconn.multi_proc¶
syconn.multi_proc.multi_proc_main¶
-
syconn.multi_proc.multi_proc_main.QSUB_script(params, name, queue=None, pe=None, n_cores=1, sge_additional_flags='', suffix='', job_name='default', script_folder=None)[source]¶ QSUB handler - takes parameter list like normal multiprocessing job and runs them on the specified cluster
IMPORTANT NOTE: the user has to make sure that queues exist and work; we suggest to generate multiple queues handling different workloads
Parameters: - params (list) – list of all parameter sets to be processed
- name (str) – name of job - specifies script with QSUB_%s % name
- queue (str or None) – queue name
- pe (str) – parallel environment name
- sge_additional_flags (str) – additional command line flags to be passed to qsub
- suffix (str) – suffix for folder names - enables the execution of multiple qsub jobs for the same function
- job_name (str) – unique name for job - or just ‘default’ which gets changed into a random name automatically
- script_folder (str or None) – directory in which the QSUB_* file is located
Returns: path_to_out – path to the output directory
Return type: str
-
syconn.multi_proc.multi_proc_main.SUBP_script(params, name, suffix='', delay=0)[source]¶ Runs multiple subprocesses on one node at the same time - no load balancing, all jobs get executed right away (or with a specified delay)
Parameters: - params (list) – list of all paramter sets to be processed
- name (str) – name of job - specifies script with QSUB_%s % name
- suffix (str) – suffix for folder names - enables the execution of multiple subp jobs for the same function
- delay (int) – delay between executions in seconds
Returns: path_to_out – path to the output directory
Return type: str