Mr Cameron made the announcement in a statement outside Downing Street after the final result was announced.
He said he would attempt to “steady the ship” over the coming weeks and months but that “fresh leadership” was needed.
The PM had urged the country to vote Remain, warning of economic and security consequences of an exit, but the UK voted to Leave by 52% to 48%.
England and Wales voted strongly for Brexit, while London, Scotland and Northern Ireland backed staying in.
UKIP leader Nigel Farage hailed it as the UK’s “independence day” but the Remain camp called it a “catastrophe”.
The pound fell to its lowest level against the dollar since 1985 as the markets reacted to the results.
Flanked by wife Samantha, Mr Cameron said he had informed the Queen of his decision to remain in place for the short term and to then hand over to a new prime minister by the time of the Conservative conference in October.
It would be for the new prime minister to carry out negotiations with the EU and invoke Article 50 of the Lisbon Treaty, which would give the UK two years to negotiate its withdrawal, he said.
“The British people have voted to leave the European Union and their will must be respected,” said Mr Cameron. “The will of the British people is an instruction that must be delivered.”
The referendum turnout was 71.8% – with more than 30 million people voting – the highest turnout at a UK-wide vote since 1992.
Labour’s Shadow chancellor John McDonnell said the Bank of England may have to intervene to shore up the pound, which lost 3% within moments of the first result showing a strong result for Leave in Sunderland and fell as much as 6.5% against the euro.
UKIP leader Nigel Farage – who has campaigned for the past 20 years for Britain to leave the EU – told cheering supporters “this will be a victory for ordinary people, for decent people”.
Mr Farage – who predicted a Remain win at the start of the night after polls suggested that would happen – said it would “go down in history as our independence day”.
He called on Prime Minister David Cameron, who called the referendum but campaigned passionately for a Remain vote, to quit “immediately”.