how. to stop logs going to /tmp
Hi,
Perhaps I should know how to solve this but I dont seem to be able to figure out how to get my useful log files not going to /tmp.
I run LINC like:
toil-cwl-runner --singularity --writeLogsFromAllJobs --stats --bypass-file-store --maxCores 40 --workDir /project/lotss/Software/linc/workdir/%s --outdir /project/lotss/Software/linc/outdir/%s/ --writeLogs /project/lotss/Software/linc/logs/%s/ --tmpdir-prefix /project/lotss/Software/linc/tmpdir/%s/ /project/lotss/Software/linc/LINC/workflows/HBA_target.cwl json-files/%s-target.json'%(fieldproperties['target_OBSID'],fieldproperties['target_OBSID'],fieldproperties['target_OBSID'],fieldproperties['target_OBSID'],fieldproperties['target_OBSID'])
But the logs that go to the --writelogs place do not really contain the useful information about what exactly the failure is. I.e. they are called like:
failed_CWLJob_prep.createRMh5parm.createRMh5parm_kind--CWLJob-instance--us1q3c6k_v3000.log
and contain like:
...
/var/lib/cwl/stg34688df3-25b0-4b41-b753-2fb0de8a9816/L2021897_SAP001_SB466_uv.MS
/var/lib/cwl/stg4380bf67-10c1-42b0-9837-df47882ede02/L2021897_SAP001_SB467_uv.MS
/var/lib/cwl/stg10c9e27d-da11-4e53-8910-dfe974a36882/L2021897_SAP001_SB468_uv.MS
/var/lib/cwl/stgab541edf-1cb3-4238-a9e5-c14f01dad3f4/L2021897_SAP001_SB469_uv.MS
/var/lib/cwl/stg1d73d7dc-02a7-4c67-9da3-2591ee797132/L2021897_SAP001_SB470_uv.MS
/var/lib/cwl/stga7b808b2-cef2-454d-b416-b38a5110e34b/L2021897_SAP001_SB471_uv.MS
/var/lib/cwl/stg5aac657c-a8fa-4a74-a246-793c1ca8f90e/L2021897_SAP001_SB472_uv.MS
/var/lib/cwl/stg486b1c0d-4c5e-4349-a911-b2e645f9a11a/L2021897_SAP001_SB473_uv.MS
/mKPyMU/cal_solutions.h5 > /tmp/tmpogpqi_gc/createh5parm.log 2> /tmp/tmpogpqi_gc/createh5parm_err.log
The /tmp/tmpogpqi_gc/createh5parm_err.log is the file that would contain the useful information. But /tmp/ is a temporary place on the node that is executing LINC and when the job crashes those files are removed because the job exits the node.
Is there anyway I can change this /tmp/ location to somewhere else?