We've found found that it's way easier to hide the stuff we don't actually need to than the things we do.
For instance, if we felt like swapping out a family photo with a picture of Nicolas Cage, it'll probably take months for anyone to notice. If we knock a family photo over, however, we'll somehow always leave some evidence that it happened.
Sure, some of this probably has to do with the fact that we're more panicked in the second scenario than the first, but the point is that we can hardly say that our powers of concealment have a perfect success rate.
And as we're about to see, that's also true of a lot of other people.