Are you running the server on another PC or your main PC, because you should never do that because MC is very CPU bound, so it won't help if you have a game and a server running at the same time. What is your system's specs? You could have 16GB allocated to MC but alonside it you would also need a very powerful CPU in your case since you are running the game and hosting it. What is in startup script for the server BTW, since some runtime options can cause issues. But with the memory, Java uses alot of RAM, so in your case it isn't really an issue since after a while Java's garbage collector cleans it up. So only if it rises and never drops you have an issue.