People ask me that all the time and my answer is there is no one hardest part as it all shows up at the end. When I first started it was probally streching the cloth and finishing the pockets.
its all hard work. recovering the rails and leveling the 3 piece is the most time consuming part especially when the guy before u shoots 450 staples in each rail and the wood slate backing...
we are all from different cuts of cloth so it might take some people more time to do things than others....