David Cameron has rejected curbs on second home ownership, arguing restricting holiday properties would "drive out" investment in areas with high numbers of part-time residents.
The Prime Minister, who frequently holidays in the region, told the Western Morning News he believes more homes should be built to ease the housing crisis in the Westcountry.
Devon and Cornwall boast about 26,000 holiday homes and have among the highest levels of second-home ownership, which critics blame for pushing up house prices beyond the reach of local people.
In a briefing with regional newspaper journalists at the Conservative Party conference in Birmingham, Mr Cameron argued for giving communities more powers that could include ring-fencing homes for local people.
He also suggested it would be unfair to "rob" sellers of money by saying they could not sell a property as a second home.
Westcountry Liberal Democrat MPs have long pushed for planning restrictions on second homes in beauty spots.
Asked by the WMN whether he would back a cap, Mr Cameron said: "Some of the controls people have suggested will drive out the investment and the building that is required to give these communities support.
"What is necessary in some cases is that we're not building enough homes that are affordable to local people. Getting rid of some of the planning controls. Putting powers in the hands of local authorities. Allowing neighbourhoods to have local plans. I think that will help those sorts of communities.
"They can take more control of their communities, they can build homes for local people, they can make sure they have sustainable communities, they can make sure communities support local pubs and local shops and local schools.
"I'd rather go down that path than try to set restrictions, which I think you're in danger of starving investment into communities rather than allowing them to take care of themselves."
He added: "Communities can write plans about how they want to see their communities develop, that's one of the best ways of getting extra houses built for local people.
"Often the suggestion is why don't you stop people selling their homes if someone else is going to use it as a second home. I think that is quite difficult to deliver in practice. It would be very bureaucratic. And I think it would rob a lot of people of the value of their homes."
The Government wants to abolish generous tax breaks for holiday homes, giving authorities power to remove the council tax relief on second homes – which ranges from 10% to 50%.
Observers say high numbers of holiday properties turn villages into ghost communities outside of the summer season.
Cornwall tops the national second homes league table with around 14,000 holiday properties.
The South Hams in South Devon comes in at number seven, with more than 4,000 secondary properties. Around half of the properties in the village of East Portlemouth, near Salcombe, are occupied part-time.
Councils across the region are consulting on house-building targets over the next 20 years, though critics believe a free-for-all would turn rural beauty spots into a "Costa Brava" of concrete.
Dan Rogerson, Liberal Democrat MP for North Cornwall, who has campaigned for curbs to second home ownership in the region, said: "The Prime Minister appears to be out of touch with people who feel the impact of second homes.
"Perhaps he should commission research into the economic impact to see if he is right. People in my constituency would think differently."