The pipe that runs from the exhaust manifold to the airbox is actually there for a good reason. In cold conditions it allows hot air from the manifold to enter the airbox and make engine warm up quicker. It also helps stop carburetor icing from occuring and maintains a constant temperature for the incoming air.
Inside the airbox is a thermostatic flap that is supposed to regulate incoming air temperature. When the incoming air is very cold, it closes the cold air feed to the airbox and only allows hot air drawn from the manifold to enter the carburetor. Once the engine has warmed up it then opens the cold air flap so that colder (and therefore more dense) air can enter. More dense air allows better combustion hence more power.
For about 8-9 months of the year it is not really needed, though with the English climate you'd be surprised at how much it can make a difference.
People who fit performance air filters have to remove it as aftermarket filters don't normally have a provision for a hot air inlet. However, those running carburetors often find that as soon as the air gets damp and close to freezing carburetor icing can occur.
When the temperature is warm, colder air entering the carburetor can improve responsiveness and liberate a small amount of power. But as soon as the weather gets cold it can cause problems such as icing and making the engine take longer to warm up.
Probably the best thing to do is to remove it during the summer and/ or if a performance air filter is fitted, then refit the hot air pipe with the standard airbox during the winter.
Just by removing it on a standard airbox is unlikely to show any increase in power. All it will do is make the engine take longer to warm up once the weather has cooled.