The western prefecture of Hyogo saw rainfall between 80 and 100 mm by noon on Monday. About 5,000 people in Taka town were asked to leave their homes, after rising water levels in nearby rivers posed risks, Xinhua cited Japan's public broadcaster NHK as saying.
More than 70 houses were inundated in several cities, including Tamba and Nishiwaki.
In southwestern Japan, persistent heavy rainfall also forced about 150 people in Kumamoto and Kagoshima prefectures to evacuate after their homes were inundated.
The rains have already caused a havoc with many homes now under water and many people being evacuated to a safer place.The Japan Meterological Agency (JMA) has issued warnings of heavy rainfall, that could further trigger landslides and rainfall.