Advanced SSH Script sensor high failure rate

Hi,
what exact data are you pulling with your script? May you post the whole script? Additional may you try to set higher Shell and Connection Timeouts as well as a higher Scanning Interval (please try with 5min)?
Therefore switch to tab "Settings" of the sensor and go to section SSH Specific and set the Connection Timeout to 3 minutes and the Shell Timeout to 59 seconds (which is the maximum). Do you get better results then?

Hi,

may you try to configure/add

ClientAliveInterval 10
ClientAliveCountMax 6

in your sshd_config to be sure there is no keepalive-issue with the client-server-connection?

Kind regards

The addition of the ClientAliveInterval and ClientAliveCountMax certainly helped. My failure rate has dropped to about 15%, but the issue appears to still exist in the same failure spot in the code as before.

I have added some debugging notes to the scripts to track exactly where it crashes. When it fails, it will always fail when executing the sleep command the first time, no matter how short it is. I tested this by making a counter loop to output the current count in the loop and then sleep for one second (with 0 as the start before the first sleep). It always fails at 0 and thus means the session crashed at the first sleep command.

The very odd part is that this does not occur every time the script is run nor does the ssh session ever get killed by me running the script via PuTTY.

Output of a good run with results:

linux

Last login: Wed Jun  5 15:45:30 2013 from 184.173.111.37-static.reverse.softlayer.com

root@raspberrypi:~# echo PAESSHSTART;/var/prtg/scriptsxml/speedTest.sh -s 192.16 
8.128.145 -t 10;printf "\n";echo PAESSHEND
PAESSHSTART
start
line 16, check args
line 21, rm iperf.log
line 24, run iperf
line 30, start counter loop
0
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
line 36, kill iperf
line 41, read output
<prtg>
<result>
<channel>Download Speed</channel>
<value>33720</value>
<unit>custom</unit>
<customunit>kbit/s</customunit>
</result>
<result>
<channel>Upload Speed</channel>
<value>67346</value>
<unit>custom</unit>
<customunit>kbit/s</customunit>
</result>
</prtg>

PAESSHEND
root@raspberrypi:~# 

Output with killed connection on sleep:

linux

Last login: Wed Jun  5 15:56:32 2013 from 184.173.111.37-static.reverse.softlayer.com

echo PAESSHSTART;/var/prtg/scriptsxml/speedTest.sh -s 192.168.128.145 -t 10;printf "\n";echo PAESSHEND
root@raspberrypi:~# echo PAESSHSTART;/var/prtg/scriptsxml/speedTest.sh -s 192.16 
8.128.145 -t 10;printf "\n";echo PAESSHEND
PAESSHSTART
start
line 16, check args
line 21, rm iperf.log
line 24, run iperf
line 30, start counter loop
0

Any further ideas?

Hi,

unfortunately not. According to the openSSH Manpage this may be correlated with a NAT-Connection. I see from your script that you are connecting from a public IP and submit a private IP. There is at least one NAT-device in the connection that may cause these Problems.

Kind Regards

I have tested both externally and internally based on where I am working for that day. I have static NAT and routing set up for these devices so that iperf and SSH are accessible from outside the network. I can conclude now that this is a bug within PRTG and will be submitting a ticket to have this investigated by the software engineers. Thank you for your help.

Hi,

Confirmed and already fixed. At least in our next nightly canary build (see our blog for details of our branches). It will probably be released in the preview channel during next week.

If you want to check if your problem is fixed with this release do not change your productive environment to canary but install a separate trial-installation and switch to the other channel as there may be several bugs in new features that will be fixed until release.

Kind regards