His progressive Muslim stand and strong opposition to Triple Talaq is what helped open the Kerala Raj Bhawan doors for him. In 2004, he joined the BJP, but quit three years later.
In his long political career, Khan has changed several parties - starting from former prime minister Charan Singh's Bharatiya Kranti Dal, Congress, Janata Dal and Bahujan Samaj Party. He has been a member of the Uttar Pradesh Legislative Assembly and as well as the Lok Sabha and served as a Union minister in the Cabinets of former prime ministers Rajiv Gandhi and V.P.Singh. Hailing from Uttar Pradesh, Khan entered politics as a student leader at the age of 26. Khan replaces P.Sathasivam, a former Chief Justice of India, who demitted office after the end of his five-year term and returned to his home town Erode in Tamil Nadu.