There are fans and heat sinks you can get, but unfortunately aside from that You are left with motor or gearing changes. The heat comes from current through the resistance in the windings. There is lots that goes into it but i think the major point is that If you gear down, your rpms will be higher creating more back emf and therefore less current. Or you could get a higher turn motor which will not draw as much current, at the expense of rpm potential. Supposedly brushless setups run cooler for the same output, mostly because the windings that generate most of the heat are on the outside so they can lose heat better.
