Both Jamna Joshi, 40, and son Moksha Jai Joshi, died in the blaze in Moscow Drive, Liverpool city, May 8.
"It was a case of if she can't have him, I can't," BBC quoted the deceased boy's father Paresh Patel as saying Thursday.
Patel, 43, hailing from Preston in Lancashire, had won custody of his son, affectionately called "Jai" two years ago. He agreed his ex-wife could meet their son during his school holidays.
Jai had been with the mother for the last two-and-a-half weeks before his death. He had not attended school.
Patel has claimed Jai's mother had "some psychological issues" and knew she had no option but to return Jai under the terms of a court order.
He said his wife's last thought would have been that "she had won", because his father was losing him.
Patel blamed the actions of police and social services for letting Joshi have her custody before Jai's death.
The police have earlier said an accelerant had been used and there was "no evidence of any third-party involvement" in the fire.
"The crux is no one can tell you of the (details) or enforceability of a court order," Patel said.
Meanwhile, the police confirmed it was aware of the existence of a court order and social service involvement with the family.
The information provided to Merseyside Police made it clear that the police did not have a power from the court to forcibly remove the child from the address, the report said.
A serious case review is under way into social services' role in the case.
Meanwhile, the police said the matter is being investigated and will be considered at both the inquest July 7 at Liverpool Coroner's Court and Lancashire County Council's Safeguarding Children Board in June.