Last interseasonal we all left but 3. Our task count did not change. It was as high as when we all stayed. As near as we can tell, how many tasks you get in interseasonal is dependent on how many racers participated the week before. Opting out before IS does nothing to decrease task count. Restricting will not reduce task count but will keep others from participating. And having everyone leave before race start will not work either.
If you create a new group before IS starts and everyone joins, your task count will be higher too. If you create a new group and wait to invite after IS starts, you will have the lower count.
If you all leave your home co-op, it will cease to exist. Someone has to stay to keep it alive. Having a second town on another device is very handy as a placeholder.