Michael Le Vell revealed his "little dark secrets" were a string of one night stands and an affair while his wife was undergoing chemotherapy, a court has heard.
The Coronation Street agreed that he had told a friend about his "secrets" and that he had done "something unforgiveable".
But the 48-year-old denies he is a child rapist and has defended himself against 12 charges of sexual abuse.
The soap actor's private life has been laid bare in court as he admitted being an alcoholic and having one-night stands and an affair
Continuing his cross-examination at Manchester Crown Court by Eleanor Laws QC, prosecuting, Le Vell told the jury of a conversation he had with someone last year about "secrets".
He told the person: "I have had a few one-night stands."
He said he had a similar conversation four years ago but did not reveal the secrets.
Le Vell agreed also telling the same person that what he had done was "something unforgiveable".
But he denied ever saying that they would not believe "what things are inside my head".
"You said that?", asked Miss Laws.
"No," he replied.
The barrister continued: "It would be a very odd thing to have said if you were talking about one-night stands."
Le Vell said: "It would not make sense."
Miss Laws said: "I am going to suggest to you that what you were referring to was the abuse that was going on?"
"No, not at all," he replied. "There was no abuse going on."
The victim, who cannot be named for legal reasons, claims Le Vell repeatedly sexually assaulted and raped her, once while she was clutching a teddy bear.
Le Vell, whose real name is Michael Turner, denies five counts of rape, three of indecent assault, two counts of sexual activity with a child and two of causing a child to engage in sexual activity.
Miss Laws said: "I am going to suggest to you that the secret is not the one-night stands but something far more sinister."
"No," said Le Vell. "I totally disagree."
Le Vell told the court he is an alcoholic, drinking seven or eight pints every night at his local pub.
Ms Laws continued: "Because something was troubling you... to go out every night?"
Le Vell replied: "No, I went out every night because I'm working class."
The actor said his wife had allowed him to go out every night.
"So it's partly her fault?" Ms Laws said.
"I'm not blaming her for going out but she enabled me to go out," Le Vell said.
"I suggest you were finding it difficult to live with yourself," Ms Laws said, "Once this abuse started, you just could not stop yourself."
Le Vell replied: "No. Absolutely not."
"And the problem with you is you could do what you wanted to," Ms Laws continued.
"Do you think you were also cosseted in your job?"
"I was cocooned," Le Vell said.
Ms Laws: "You led quite a charmed life until September 2011?"
Le Vell agreed he led a "luxury, charmed life because of my job".
Ms Laws put it to Le Vell that he groomed the victim and gradually "pushed the limits".
"Not that you were not ashamed about this but you could not stop it?" Ms Laws said.
Le Vell replied: "It never happened."
Ms Laws: "You started to say things to her? Talk to her, say things like 'It's OK.'.This was grooming.
"It became a little bit more sinister after a while."
Le Vell replied: "No."