Dave, hate to say it but you're talking about things you blatantly don't understand. The reason that ally pipe is cold is because it's working as a air/air heat exchanger. No ifs, buts or maybes, it IS.
The exhaust gasses coming off that manifold will be several 100 degrees centigrade, so if that pipe is cold then you'll find that that heat had to go somewhere. Now where could it go?.. oh yeah into the intake air. The fact it was a cold day and that pipe was cool tells me the pipe was being cooled down by the air going into the engine. This isn't reckoning, this is basic physics. Secondly the exhaust manifold will get very hot well before the engine is properly warmed up.
I know someone who got an aprox 2bhp power gain by using exhaust wrap along the inlet track over the top of that putting a heat reflective sleeve. That intake route took the air over the top of the gearbox let alone over the top of the exhaust manifold which is a lot hotter (even with the bonnet open)