This is true. However one of the goals of refactoring is to get quality up. So there is a bit of a chicken and egg thing going on. By assuming you're going to be a particular level of quality, you can make it easier to get to that level of quality. But in the meantime you're doing something that temporarily makes maintenance harder.

Is that trade-off worthwhile? (My answer is, "sometimes". Apply judgment, hope it is good.)