I end up buying new stuff based on the design of each new cake I decide to make.

(This is especially true of cake pans...seems like every time I design a cake I need *some* shape or size I don't already have.) I'm sure it's different if you're starting a business, but if you're getting into this as a hobby it's a pretty reasonable approach!
I started with almost no special tools -- just basic kitchen items like a serrated knife for carving/leveling/trimming -- and I decorated some very fun cakes with just a shortish icing spatula and a sandwich bag with a hole snipped in the corner for piping.

Of course those cakes were amateurish but they were cute and the people I served them to thought they were awesome.
Then I decided to get a few nicer tools, like a long offset icing spatula and a basic set of tips, couplers & disposable bags (normally I hate disposable products but piping bags are too hard to wash, it's just not worth it), and a turntable. I use those now on every cake, so I would include them in the top 5 for sure. I finally just got a bench scraper and I don't know what took me so long -- that belongs on the list. If I can count all the piping stuff as one item, that's four, and for #5 I'd probably go with my itty bitty offset spatula that gets nicely into the tight spaces.
After messing around with fondant a time or two I got a fondant smoother and a set of Fimo clay modeling tools (smaller and cheaper than the Wilton basic gumpaste/fondant tool set) which have come in handy. (I already owned a nice rolling pin so I'm not counting that.) But you certainly don't need those things if you're not going to work with fondant. I bought things like food coloring stamps, mini shape cutters, fondant embossing tools and so on when I had a cake design that called for them. My mom gave me a bunch of kind of random decorating stuff one year for Christmas, too, and some of it's been fun to have but hardly critical starting tools.
But you know, I think a lot of what really makes a difference is the "consumable" supplies that "real" decorators know to use: Cardboard rounds, parchment paper, Viva paper towels, dowels or straws for stacked cakes, gel food colorings, luster dusts, and so on. Those aren't tools per se, but they're more important than, say, gumpaste molds or five different sizes of star tip.
Holly