Priti Patel has accused the England football team of “gesture politics” for taking the knee before games.
The home secretary also refused to criticise fans who booed the players, saying it was “a choice for them”.
Labour condemned the home secretary for trying to “provoke a fight” with the team as its Euro 2020 campaign gets underway.
It came after a minority of supporters once again defied calls not to jeer the team as they took the knee before kick-off in the clash with Croatia on Sunday.
England manager Gareth Southgate has said the players will continue to do so as “a show of solidarity with the black community”.
“I think as players we’ve made it very clear that we’ll all stand together against racism and that is the reason that we’ll continue to take the knee and have done over the last few weeks,” he said.
Downing Street has also said Boris Johnson wanted people to “cheer” the England team and “not boo” during the tournament.
But in an interview with GB News on Monday, Patel criticised the players but not those who booed.
“I just don’t support, people participating in that type of gesture politics to a certain extent,” she said of taking the knee.
Asked if she supported the fans who continue to boo, Patel said: “That’s a choice for them quite frankly. I’ve not gone to a football match to even contemplate that.”
Jo Stevens, Labour’s shadow culture secretary, said: “It beggars belief that a day after the excellent win against Croatia, senior government ministers are still trying to provoke a fight with the England football team.
“Ministers should get on with their jobs and get behind the home nations.”
Labour leader Keir Starmer has said he “profoundly” believed people should not “boo your own team before kick-off”.