The approach will see many computer processors stacked on top of one another, cooling them with water flowing between each one. The plan is to reduce computers' energy use, rather than just to shrink them.
Dr Bruno Michel said future computer costs would hinge on green credentials rather than speed.
Michel and his colleagues have already built a prototype to demonstrate the water-cooling principle called Aquasar, it occupies a rack larger than a refrigerator. IBM estimates that Aquasar is almost 50 pc more energy-efficient than the world's leading supercomputers.
"In the past, computers were dominated by hardware costs - 50 years ago you could hold one transistor and it cost a dollar, or a franc," Michel said at IBM's Zurich labs.
The Aquasar prototype clocked up nearly half again as much, at 1.1 billion operations. Now the task is to shrink it.
"We currently have built this Aquasar system that's one rack full of processors. We plan that 10 to 15 years from now, we can collapse such a system in to one sugar cube - we're going to have a supercomputer in a sugar cube," Mark Stromberg, principal research analyst at Gartner said.