In regards to 2D vs 3D Mario, 3D Land and World both topped NSMB2/U. Granted Land and 2 were by a small margin, but I feel that may be telling of the NSMB longevity. People bought the DS one because of how long it had been since a 2D Mario while the Wii one offered something new with 4 player co-op. 2 and U didn't have anything new while 3D Land offered a new take on the series and World offered 4 player co-op with different characters each with their own strengths and weaknesses. I think the real key is a balance between accessibility and freshness. I love the Galaxy games, but they were hard for younger and more casual gamers to wrap their heads around as opposed to NSMBW, which is why I think that game did better. Same goes for why Sunshine is one of the lower selling Mario games. I feel if the new Mario can have a very simple yet fresh new mechanic, then it will succeed.
Besides, Mario Kart vastly outsells both 2D and 3D Mario on average so them getting one of those out quick matters more.