Are you running it on a computer in your house, connected to the same network as your main computer. If so, it would be relatively easy for someone who would have any use for said information to get it.
For example, take us.mineplex.com, the No-IP address of the Mineplex server.
If I open my handy-dandy cmd.exe, and enter the command "ping us.mineplex.com", I am given a true IP address for their server.
The Meaning of Life, the Universe, and Everything.
I will add that they can only see the external ip address (i.e the one that the dynamic dns points to), they don't know your internal ip address behind your router (assuming you have one).
Even if they had your public IP they are restricted to only only doing things on ports that are open, so only open the port that you need for your server, and you're fine for the most part, save DDoS and the like (extremely rare on a home setup, unless you really annoy someone that knows what they are doing lol)
In short YES they can see your external IP address. But that is a risk to take when hosting a server from home. But also look at it this way. If you log onto a random server the Owner of that server could see your IP so it is no real difference. The odds someone DDOS's you is very low unless you manage to get them angry.