The rate of rail passenger complaints jumped by 7.5% over the past year, new figures show.
Delays and cancellations in London and south-east England drove the increase, according to the Office of Rail and Road (ORR).
Some 29.4 complaints were made per 100,000 journeys on franchised operators in 2016/17, representing a total of 540,000 disgruntled passengers.
Alex Hayman, managing director of public markets at consumer group Which?, said: "It is staggering that more than half a million complaints have been made by rail passengers in the last year alone and shows how badly the sector is failing its passengers.
"We've heard from thousands of people who have to face delays, cancellations or overcrowded trains on a daily basis.
"We need to see the rail regulator given stronger powers so that it can effectively stand up for passengers when rail services fail to deliver.
"The Government must also press forward on its plans to introduce a rail ombudsman."
Jo Causon, chief executive of the Institute of Customer Service, said: "A major factor of rail customer satisfaction is service reliability, but our research identifies staff attitude and behaviour as essential drivers too.
"Customers are more concerned today about staff attitudes and behaviour than they were even five years ago and developing these skills is an essential part of improving customer service."
Punctuality on trains during the past year was the worst since 2005/06, with 12.3% of trains failing to reach their destination within five minutes for commuter services or 10 minutes for long distance trips.
The latest survey of almost 30,000 passengers by Transport Focus in autumn last year revealed that just 81% were satisfied with the railways, a figure which has not been lower since spring 2007.
Five operators in London and the South East, Greater Anglia, c2c, Govia Thameslink Railway, Southeastern and South West Trains, were responsible for 46% of all complaints in 2016/17, compared with 32% during the previous 12 months.
A spokesman for the Rail Delivery Group, representing train operators and Network Rail, said: "We know that the last year has been difficult for passengers on some parts of the railway, and the figures reflect that, but on other parts rail companies working together have raised punctuality to record highs.
"The long-term trend is of falling complaints from customers and our £50 billion-plus upgrade plan is making journeys better."
A Department for Transport spokeswoman said: "Overall, passenger complaint figures have fallen dramatically since 2001-2, but we know there is still work to do.
"Improving rail services for passengers is a priority for the Government and we've been clear that we expect to see better performance."