Rehman had reportedly supplied 22 SIM cards to one Mukhtar Ahmed Sheikh -- reportedly a Kashmiri policeman who worked as an auto-rickshaw driver in Kolkata. Sheikh has also been arrested.
According to sources, one of the SIM card supplied by Rehman was used in the Mumbai terror attacks.
He will be produced before the Chief Metropolitan Magistrate Court in Kolkata in a short while.
"We are questioning them about procurement of SIM (Subscriber Identity Module) cards used in Mumbai," Jawed Shamim, the deputy commissioner of detectives in Kolkata, told news agency Reuters.
If the men are proved to be involved, it would be further evidence of Indian complicity in the three-day rampage New Delhi has blamed on Pakistan.
In February police arrested Fahim Ansari, who was carrying maps of Mumbai that highlighted several of the targets later hit in the attack.
At least 217 people were killed in the attacks last week in which 10 gunmen struck two luxury hotels and other landmarks across India's financial capital.