The sensor uses the IP address or DNS-hostname configured in the setting "IPv4 Address/DNS Name" in the parent device. If you configure a DNS hostname, this hostname gets resolved to an IP address before doing the check.
The resolved IP-address depends on the DNS server you are using on your PRTG host. Therefore you have to make sure, the used DNS-hostname gets resolved to the internal IP address or configure this internal address in the parent device settings.
Please take a look at the sensor documentation, at the section remarks, where this is also explained: https://www.paessler.com/manuals/prtg/ip_on_dns_blacklist_sensor
Be aware that if you want to check services on both the internal and external IP addresses of this device, you have to create two devices within PRTG and configure each with the private or public address.
You can either check the name-resolution manually on the PRTG host by doing a "nslookup $device-dns-hostname" or create a new device for your DNS-Server and add an "DNS v2" sensor (https://www.paessler.com/manuals/prtg/dns_v2_sensor) with the DNS-hostname you want to resolve. You can then use a "Filter Value" to make sure it resolves to the private IP-address.
Matthias Kupfer - Team Tech Support