Is there a limit on stack size?
There is no strict limit on the stack size. The stack and heap grow towards each other until a collision occurs. If your job terminates with an error like this:
***FAULT Encountered unhandled signal 0x00000010 (16) (SIGSTKFLT)
most likely you have used up all memory, causing the stack to write into the heap guard area.
However, in the event you would like to disable the stack guard check, set the runtime environment variable BG_STACKGUARDENABLE=0
qsub --env BG_STACKGUARDENABLE=0...
This will disable all stack-heap collision detection, so any real collision will result in corrupt data and unpredictable behavior of your program.
NOTE: For threaded programs, the child threads do not use the main process stack.
My job had empty stdout, and the stderr looks like it died immediately after it started. What happened?
<Feb 29 14:31:02.873207> BE_MPI (ERROR): The error message in the job record is as follows:
<Feb 29 14:31:02.873237> BE_MPI (ERROR): "killed with signal 6"
Check that the size of the executable is not too large to load. You can use size a.out to get this info.
Where can I find the details of a job submission?
Details of the job submission are recorded in the <jobid>.cobaltlog. This file contains the qsub command, mpirun command and environment variables. The location of this file can be controlled with the 'qsub --debuglog <path>' which defaults to the same place as the .output and .error files.