George Osborne has cast doubt on whether Scotland would be able to keep the pound as its currency if it becomes independent.
A new report from the Treasury says the economic rational for a formal agreement is "not clear".
Speaking on Tuesday morning, Osborne told the BBC's Today Programme: "It's unlikely we can make it work."
His report questions whether a deal can be reached to establish a "sterling zone" between an independent Scotland and the rest of the UK.
Scottish people will vote on September 18, 2014 on whether to become an independent country or remain within the UK.
Osborne said there were "some very big questioned to be answered" about whether the UK would want to tie itself to the economy of a foreign country.
Scottish Finance Secretary John Swinney has already dismissed "insulting" claims in it that Scottish banknotes could be lost under the SNP's plans to retain the pound if the country votes to leave the UK in next year's referendum.
He said: "Instead of coming north to lecture the people of Scotland, Mr Osborne should be listening, at a time when Scotland's economy and jobs market is outperforming the rest of the UK and when the Chancellor's own financial mismanagement has been exposed by the latest international ratings downgrade."
The report - one in a series of papers from the UK Government on the impact of Scottish independence - outlines four options for currency if voters north of the border decide to leave the UK.
Scotland could continue to use the pound as part of a formal agreement with the rest of the UK, creating a sterling currency union, or could instead continue to use the pound unilaterally, with no such deal in place.
Alternatively it could join the euro or introduce a new Scottish currency.
The Scottish Government has already set out plans to retain the pound if the country becomes independent, creating a "sterling zone" with the rest of the UK.
Economics experts in the Fiscal Commission Working Group, set up by Scottish First Minister Alex Salmond, have already concluded this is both "sensible" and an attractive choice for the rest of the UK.
But the Treasury report said a formal sterling currency union "would only be possible if both an independent Scotland and the continuing UK could reach an agreement that satisfied both countries' economic interests".
It argues a formal sterling currency union would be "very different to the current arrangements and would be a profound economic change for both states".
An independent Scotland would "need to agree a negotiated set of constraints on its economic and fiscal policies", the report said, adding: "In practice this would be likely to require rigorous oversight of Scotland's economic and fiscal plans by both the new Scottish and the continuing UK authorities."
It went on: "Even with constraints in place, the economic rationale for the UK to agree to enter a formal sterling union with a separate state is not clear."
But Swinney said: "The Treasury are insulting people's intelligence if they expect them to swallow these claims.
"No-one will seriously believe that we can have our own Scottish banknotes under Westminster rule - as we have already had for so many years - and will not be able to do so as an independent country.
"It is a ridiculous claim, totally at odds with the facts, because the existing situation regarding Scottish banknotes, which is underpinned by the 2009 Banking Act, will stay in place as part of a post-independence sterling zone."
He claimed the pound was "every bit as much Scotland's currency as it is England, Wales and Northern Ireland's".
The Scottish Finance Secretary added: "If this scare story is the best George Osborne can do in advance of his visit to Scotland then it betrays the utter weakness of his case."