[RegCNET] CCSM divide by zero issue

[Maurice.McHugh at noaa.gov]

Colleagues,

I recently discovered a RegCM3 error produced when running CCSM data.   The model runs correctly over the east coast of the US, but blows up the same model configuration is moved over the west coast of the US.

I'd really appreciate assistance in debugging this error.

Thanks,

Maurice


[beren:29588] *** Process received signal ***
[beren:29591] *** Process received signal ***
[beren:29591] Signal: Floating point exception (8)
[beren:29591] Signal code: Floating point divide-by-zero (3)
[beren:29591] Failing at address: 0x49ca9a
[beren:29593] *** Process received signal ***
[beren:29593] Signal: Floating point exception (8)
[beren:29593] Signal code: Floating point divide-by-zero (3)
[beren:29593] Failing at address: 0x4e91fa
[beren:29588] Signal: Floating point exception (8)
[beren:29588] Signal code: Floating point divide-by-zero (3)
[beren:29588] Failing at address: 0x49ca9a
[beren:29598] *** Process received signal ***
[beren:29598] Signal: Floating point exception (8)
[beren:29598] Signal code: Floating point divide-by-zero (3)
[beren:29598] Failing at address: 0x49ca9a
[beren:29592] *** Process received signal ***
[beren:29592] Signal: Floating point exception (8)
[beren:29592] Signal code: Floating point divide-by-zero (3)
[beren:29592] Failing at address: 0x49ca9a
[beren:29598] [ 0] /lib/libpthread.so.0 [0x7f128f225990]
[beren:29593] [ 0] /lib/libpthread.so.0 [0x7fc2fb8c1990]
[beren:29592] [ 0] /lib/libpthread.so.0 [0x7f3510d76990]
[beren:29592] [ 1] regcm(interf_+0x37e) [0x49ca9a]
[beren:29592] [ 2] regcm(vecbats_+0xd4) [0x544a38]
[beren:29592] [ 3] regcm(tend_+0xcef5) [0x5266f9]
[beren:29592] [ 4] regcm(MAIN__+0x4c5) [0x506c71]
[beren:29592] [ 5] regcm(main+0x2c) [0x54e09c]
[beren:29592] [ 6] /lib/libc.so.6(__libc_start_main+0xfd) [0x7f3510a33a8d]
[beren:29592] [ 7] regcm [0x405c69]
[beren:29592] *** End of error message ***
[beren:29593] [ 1] regcm(pcp_+0xa22) [0x4e91fa]
[beren:29593] [ 2] regcm(tend_+0xca41) [0x526245]
[beren:29593] [ 3] regcm(MAIN__+0x4c5) [0x506c71]
[beren:29593] [ 4] regcm(main+0x2c) [0x54e09c]
[beren:29593] [ 5] /lib/libc.so.6(__libc_start_main+0xfd) [0x7fc2fb57ea8d]
[beren:29593] [ 6] regcm [0x405c69]
[beren:29593] *** End of error message ***
[beren:29588] [ 0] /lib/libpthread.so.0 [0x7f1a2a83a990]
[beren:29588] [ 1] regcm(interf_+0x37e) [0x49ca9a]
[beren:29588] [ 2] regcm(vecbats_+0xd4) [0x544a38]
[beren:29588] [ 3] regcm(tend_+0xcef5) [0x5266f9]
[beren:29588] [ 4] regcm(MAIN__+0x4c5) [0x506c71]
[beren:29588] [ 5] regcm(main+0x2c) [0x54e09c]
[beren:29588] [ 6] /lib/libc.so.6(__libc_start_main+0xfd) [0x7f1a2a4f7a8d]
[beren:29588] [ 7] regcm [0x405c69]
[beren:29588] *** End of error message ***
--------------------------------------------------------------------------
mpirun noticed that process rank 5 with PID 29593 on node beren exited on signal 8 (Floating point exception).
--------------------------------------------------------------------------