The jaunts can affect cetacean behaviour and stress levels in addition to causing deaths from collisions.
But some animals are affected more than others and the long-term effects remain unclear, said scientists.
"Whale-watching is traditionally seen as green tourism. The negative is the potential for disturbance. That disturbance is a worry because we do not want to do a death by 1,000 cuts," said wildlife biologist Leslie New from the US Geological Survey in Laurel, Maryland.
The number of people joining these trips has undergone an astronomical expansion since the 1990s, from 4 million in 31 countries in 1991 to 13 million in 119 countries in 2008, the most recent year for which full data is available.
In 2008, the International Fund for Animal Welfare, an animal-protection charity in London, estimated the value of the industry at $2.1 billion, reported the journal Nature.
Although collisions with boats can hurt the animals, researchers are more concerned about effects such as animals failing to feed or using up energy swimming away from the vessels.
Marine biologist David Lusseau from University of Aberdeen in Britain has shown that the bottlenose dolphins in Doubtful Sound, New Zealand, could be driven to extinction in decades.
The large number of dolphin watching trips is driving the animals away from their preferred areas and forcing them to avoid boats instead of feeding.
The scientists presented their findings at the International Marine Conservation Congress (IMCC) in Glasgow recently.