A punt (also used to describe; a flat-bottomed boat and an upward kick in Rugby), with regards to wine - refers to the conical indentation found at the bottom of a wine bottle - and especially recognisable with the base of Champagne bottles. It does not seem to matter whom or in which wine drinking region or country you ask this question, there are a wide variety of explanations for its purpose.
Most agree that a bottle with a punt rests more easily on a table, because with a flat-bottomed bottle it would rock around as it would only need a small imperfection to make it unstable in the table surface. With the ring of the punt being the only surface contact, it helps the wine bottle sit much more stable.

 

In modern times, bottles are not handmade or mouth-blown as they were back in the day. They are now made in molds. So they could easily be made without punts! Many white wine bottles (example: the tall, slender Riesling bottle) are in fact made with a nearly flat bottom. However, for historical reasons, most red wine bottles are made with punts. That is because one theory of punts is that the punt helps to collect sediment into a thicker ring, so that it does not as easily slide down the inside of the bottle and out into the glass.

The more commonly cited explanations include: 
It increases the strength of the Champagne bottle, structural integrity, allowing it to hold the high pressure of sparkling wine and champagne.
The larger the punt, the larger the external size of the bottle can be made while still holding 750ml on the inside. This can give the illusion that a wine is of high quality because of the weight/size of the bottle. It is thoughts by many that the deeper the punt the more expensive the bottle and the wine inside. It is often used (along with a slight inwards taper to the bottle) as an indication of a 'better' bottle of wine.
I have poured a lot of wine, from bottles with and without punts - I typically stick to my first answer at the top of the page, but it is definitely a conversation starter.