What about in higher levels, making a way to exchange several houses for one. Example: A person has 5 housing units and can exchange those for one building without loosing any population. That way there would be more room for the decorations we get in the regattas.