I'm a 31 year old APA 4 (got into pool 2 years ago) and I want to at least reach the point of being a strong enough 6 to play Masters by age 35.
I'm hoping that's feasible, but often when I feel like I'm getting better I end up choking away a match I could have won.
In regards to your point about having fun, that's why I disagree with those who say beginners looking to get better should only do drills and not play racks. Playing actual games (whether in league or at bars) is just too fun for me to give up, and it has still taught me a decent amount.
This might be too general for what's going on with your progress, but on the one hand you have a genre that requires extreme competence and on the other hand, you want to attain that competence while having fun.
The choking you cite is probably self induced conflict. The match gets "real" so you apply rational (and unfamiliar) concerns to an incomplete skill set. Maybe it's the one ball or you can't seem to stay in line. Simply put, you're in over your head.
Back to the drawing board.
You may be lacking in patience. You want to hotrod your talent. That's still a tedious process. Skills have to be commensurate with ambition.
Back to the drawing board.
See a pattern?
Mastering pool isn't about who you beat. It's about mastering pool. The lessers will fall by their own folly.
