a castle dont need to be perfectly symmetric neither squared shape based.
also dont need to have a a width neither a heigh limit
the bigger you make it, the more rooms, and more detailed rooms, you can place inside.
also, if you make something that big, a repeated shape all around would make it boring also, perfectly symmetric also means that if you walk through one side of it, you already know exactly what to find at the other side as it is perfectly symmetric.
still, as you can see in this image, I used symmetric patterns over an asymmetric overall
sure, this castle have nothing to do with any real castle but for its facilities and the fact it is a reinforced structure meant to least heavy attacks and to support life and battle for days/weeks, also it can provide food and mining resources.
a mine and some plantation is also a good idea for a castle, still, you need to make room for it, be it outside the castle facilities but inside castle walls or inside the castle with a lot of light and water or open area with water (anyway, water and light is needed for plantation, not for animals)