It effects the path of slow rolling balls at the end of the break that run over the rack. Also, when removed, balls are every so slightly disturbed from where they were.
Most people don't seem to mind or care. They much, much prefer to enjoy the false sense of power and the glory of being able to consistently make a ball on the break. They want perfection in a rack, as to experience some purity in the game - but don't seem to be bothered with the imperfection and impurity of disturbing the natural path of the balls.
While I understand the rack is basically recreating and maintaining conditions for a break similar to those when the balls are brand new as well as the cloth...because the rack is on the table during and immediately after the break - it becomes part of the table. It becomes a piece of equipment that directly interfaces with the results on the table (position of the balls).
It is no longer the cloth, balls, rails and cue tip. Now there is a thin sheet of plastic film influencing the balls.