The first thing you need to do is clog the holes! Mice don't walk through walls, and they don't fly either, but they are incredible little climbers. Mice are capable of climbing a completely vertical surface if it's rough enough and gives their tiny claws something to grab on to. I guess what I'm trying to say is don't be surprised to find the critters hanging out in your attic. If there are holes to the outside, they may just find their way in.
Do you have cables or other pipes entering your house and the hole was never sealed properly or the seal has broken? Fix it before the fall time when the mice start hunting for a new residence. I've said it once and I'll say it again, mice can squeeze through a hole the size of your little pinkie finger. If you don't have an opportunity to pull out the caulk gun, at least take a piece of steel or copper wool and shove it in that hole. If the hole is less than 2" in diameter this is an effective solution. Holes larger than that will likely require proper building materials to completely seal off the entrance
Don't wait until it's too late and you're fighting to get rid of the mice - stop them before they ever get in! Since it may be difficult to isolate every hole in the wall, you may want to consider some other mouse repellent techniques.