New York's main contract, light sweet crude for delivery in July dropped 61 cents to USD 94.77 a barrel and Brent North Sea crude for July delivery shed 39 cents to USD 102.57 in mid-morning trade.
"Inaction on the BoJ's part suggests that there may be less monetary stimulus in the future," Desmond Chua, market analyst at CMC Markets in Singapore, said.
"The market is reacting to fears that various governments will end stimulus measures. This has caused a pullback in prices," said Victor Shum, managing director at IHS Purvin and Gertz in Singapore.
Global investors were sent running after the BoJ said Tuesday the Japanese economy was "picking up", but while it warned of possible headwinds caused by weakness in key export markets it chose not to add to April's giant bond-buying splurge.
The decision reignited concerns about the role of central banks in supporting the economy, with the US Federal Reserve signalling it wanted to draw up a plan to wind down its own huge USD 85 billion-a-month stimulus scheme.
Oil prices had fallen earlier in the week after the Organization of the Petroleum Exporting Countries (OPEC) slightly lowered its 2013 global demand outlook, while disappointing economic data in China added to jitters.
In its June oil market report released Tuesday, OPEC estimated 2013 global demand at 89.65 million barrels per day, down from the prior month's forecast of 89.66 million.