More than 500 people were arrested on suspicion of neglect last year for leaving a child alone.
The youngest child involved was a baby just six weeks old, the BBC learned through a Freedom of Information request.
Former MP John Hemming is now campaigning for clearer advice for parents.
In the last financial year, police arrested 510 people when they deemed a child might be at risk having been left alone.
The actual number could be much higher as only 28 of the 43 police forces in England and Wales responded to the request.
Although the oldest child deemed at risk was 15 years old, the majority of the arrests related to children under 10 years old.
There are no laws about the age a child can be left alone, but police can take action if they deem a child to be at risk.
The National Society for the Prevention of Cruelty to Children (NSPCC) has guidance on its website and suggests that babies and toddlers are never left alone and under-16s should not be left alone overnight.
The advice also says: "Children under the age of 12 are rarely mature enough to cope in an emergency and should not be left at home alone for a long period of time."
But the charity said it received 2,500 calls from adults worried about children left alone, many in the summer holidays.
On his blog, Mr Hemming, a Liberal Democrat MP for 10 years until 2015, said he was worried about the lack of clarity in the law.
He said: "The Government claim that parents are allowed to decide whether their child is mature enough to be left alone. That, however, is not true.
"What happens is that either the police or local authority social workers decide whether in their opinion the parents have neglected their child."
He added: "We need to consider what is optimal, what is ill-advised and what should be criminal or require action from the local authority."