So much of this is going to come down to personal preference but I really think you need to head to a dealer. Although comparing a new 24 to a new 22 isn't the same as an 06 vs. 07, the new boats will still give you an idea of the relative size. You'll be amazed at how much bigger the 24 feels.
The only reason (in my mind) to not buy a 24 is if you have garage space limitations or if you trailer a long way.
I only trailer 1 mile and the bigger boats always surf better. Because of that, I wouldn't even consider a smaller boat.
Former owner of a 1987 Supra Saltare. Current owner of a Malibu 23LSV.