Almost one in four adults are unable to work out a simple pay rise, according to a poll.
Many others struggle with basic calculations such as interest on a savings account or currency exchange rates.
Figures have previously shown that around half of England's working-age population have the numeracy skills of a primary school pupil.
National Numeracy, the charity which commissioned the survey, said that with Brexit approaching, more must be done, by employers and government, to recognise the scale of the UK's numeracy problem and deal with this issue.
Overall, 23% of more than 2,000 adults questioned were unable to correctly work out the new rate of pay of someone who is paid £9 an hour and gets a five percent pay increase.
Less than a quarter (23%) knew the correct answer to a question on exchanging pounds for dollars, and less than two thirds (63%) were able to do the right calculation for a question on interest rates for a savings account.
Just one in six of those questioned (17%) got all three questions right, while 15% failed to answer any correctly.
A new report by National Numeracy and KPMG notes that a 2012 survey found that 49% of working-age adults in England have the numeracy level expected of primary school children.
It adds that the total cost to the UK economy - including the public purse, employers and individuals - of poor numeracy has been put at £20 billion a year.
An analysis by the charity suggests that the average cost to an individual with numeracy levels no higher than that of an 11-year-old is around £460 per year in wages alone.
The report warns that the importance of good numeracy skills is widely acknowledged, but the problem continues.
"Poor numeracy among individuals is often not detected and the impact on productivity within a particular workplace is not quantified," it says.
"Numeracy – the ability to solve everyday problems through the application of basic mathematical skills and understanding – is often overshadowed by concerns about literacy and subsumed in academic mathematics. And yet it is different from both."
The report adds: "The resolution of all this cannot be left to the education and skills system alone. It is not solely a matter of ensuring students leave school numerate, important though that is.
"Help must also be directed to the millions of barely numerate adults already at work, many of whom will continue to work for several decades."
National Numeracy chief executive Mike Ellicock said: "Now, more than ever, we need new solutions to this very old problem. With Brexit on the horizon, better number skills are a blatantly obvious requirement for upping the UK's game in terms of productivity.
"We know that millions of people have a real fear of numbers - but we can help everyone to overcome this and get the essentials of numeracy. For this we need people to 'just do it', employers to recognise the numeracy deficit within their own workplace and work with us to remedy it. And we want government to acknowledge the scale and urgency of the problem and support us in action to crack it."
:: The YouGov poll questioned 2,115 UK adults between June 7-8.
:: The maths calculations asked as part of the poll were:
1. Susie is paid £9.00 per hour. If Susie gets a 5% pay increase, what is her new pay per hour? (answer: £9.45)
2. Fiona inherits £8,400 and invests the full amount in a savings account which pays 2.5% interest per year. How much interest does she earn in the first year? (answer: £210)
3. For the following question, please round your answer to the nearest 10 dollars. Please imagine the current exchange rate for US dollars is £1 = 1.22 dollars. How many dollars should you get for £210? (answer: 260 dollars).
A Department for Education spokesman said: "We accept that there is much more to do to make sure people have life skills they need and this includes a basic understanding of maths.
"Those who achieve a good level in maths increase their chances of securing a job, an apprenticeship or progressing to further education. That's why we have introduced a rigorous new curriculum, a more challenging mathematics GCSE and a new core maths qualification to encourage students to continue studying maths beyond GCSE."