In the situation you describe, it is common practice to implement the use of the rack, and re-rack all the balls, so no, this would not be considered a foul. In other situations where the rack area has been somewhat developed, re-spotting should be observed carefully by both players, and every effort should be made to not disturb the layout. In our league it is not considered a foul when in the act of re-spotting a ball another ball is touched. However, if in the act of re-spotting, one or more balls are touched and moved from their positions, then that would be considered a -1 penalty foul and loss of turn. In such a situation it should always be the incoming player (who's turn it is at the table) that re-spots any pocketed balls.
Sent from my Nexus 5X using Tapatalk