Nine other bomb-like objects were also found in the bin in Annar Nagar this afternoon after a sweeper woman alerted them, they said, adding bomb experts were analysing the objects.
Police, who rushed to the spot, found two country bombs on the surface of the bin and the nine bomb-like objects buried under the garbage.
"We have confirmed that two objects are country bombs, and are not sure about the nine objects...the experts are analysing them," a police official said.
The spot from where the bombs were recovered is very near to an auditorium, which housed a Ram temple, where a low-intensity cycle bomb exploded on May 1, 2012, nine days ahead of the visit of top BJP leaders including L K Advani to the city for a party conference.
In 2011, a powerful pipe bomb was found and defused from under a bridge near Tirumangalam in the district, hours before Advani was to take the route on his jan chetna yatra.
The bombs recovered today were powerful and could be ignited just by causing a small fire in the dust bin, an official said.
Police said miscreants could have dumped the bombs in the dustbin as they had tightened the security and random checks were being conducted.
(With PTI inputs)