Your castle is attractive for aggressive neighbors, and because you are deaf to the Middle Ages, where the main law - it is the power, wait for the attack. To adequately meet the enemy, provide a reliable defense by strong walls and towers shooting. Do not forget about the internal buildings, they will help to maintain the army, the treasury will add money to improve and upgrade buildings.