Reserve Bank on Friday said it has imposed a penalty of Rs 5 lakh on Bajaj Housing Finance for non- compliance with certain regulatory provisions.