It depends upon where you live. You don't need a business license in Stafford County, however you will need the other things (kitchen inspection if you go that route, for example) to operate. As the PP mentioned, to get your tax ID number, you fill out a form on-line for the IRS. If you are working out of your home, your home owners insurance may be able to do a new policy to cover the business.
With a Sole Proprietership (assuming you don't have a business partner), this simply means that you are responsible for the bills, can be held accountable for expenses, liabilities, etc (you personally file for bankruptcy). An LLC, you would not be personally responsible, the corporation would be (for example, the corporation would declare bankruptcy...you as an individual would not). An LLC is more expensive to operate, and while it offers some tax benefits most small businesses, operating from their homes, without several employees will not benefit from it much, if at all (especially starting out). You can transition to one later. Please recognize that I am just skimming the top of these business types...there is more information out there. If you would like to really consider doing an LLC, I suggest seeing an attorney who specializes in tax law to walk you through it.
In our case, we will be a sole proprietership/partnership. The only taxes I will have to file (or dd file) will be quarterly self-employment taxes (which, while a pain in the rear...are uncomplicated). I don't believe we'll be making much money, to start, either.