Banks are expected to continue cutting jobs in the next few months amid warnings that the finance sector is facing a "perfect storm".
The CBI said optimism among firms in financial services had fallen at the fastest rate since 2011.
A survey of over 100 companies identified financial market instability, competition and economic uncertainty as challenges this year.
General employment levels in financial services were expected to remain "flat" in the next three months, with increases in insurance and building societies offset by a "sharp" fall in banking.
Rain Newton-Smith, the CBI's economics director, said: "Concerns over China and a volatile start to the year for markets, alongside uncertainty about a possible Brexit, have created a perfect storm to dampen optimism in financial services."
Kevin Burrowes of PwC, which helped with the research, added: "This quarter's survey findings tell us that the cloud forming across the sector is getting darker."