Exhaust design and especially manifold design is fairly critical to getting the power out of an engine. There is an optimum bore size for every engine and the critical factor is the cross sectional area, this must remain consistent in order to maintain the gas flow. Bends need to be properly radiused to avoid pinching which makes the loss in velocity worse. Joins in the pipe (collectors in the manifold) are also critical - the area of the individual pipes being joined should be equal to the area of the single outgoing pipe and you should never split the pipe. All of these principles were laid down by a fellow called Carlos Abarth and are adhered to knowingly or otherwise by designers of exhaust systems everywhere.
The optimum size of the individual header pipes is down to the exhaust has flow from the head. Bigger is most definitely not better unless you have as already mentioned a forced induction engine or a nitrous system fitted. Smaller isn't better either, too small restricts the gas flow *but* you do want the pipes as thin as possible. How do you get it right? Without knowing the gas flow characteristics of your engine the simple answer is experimentation, guesswork and intuition.
Increasing valve sizes is likewise guesswork unless you have some proper figures to work from, in general though the exhaust valves are of adequate size but the inlet valves are slightly restrictive so you can gain a bit of torque by fitting larger inlet valves, this in turn means fixing the rest of the induction side as the standard DMTR/DATR carburettor will run out of steam. For small performance changes you need just a rejet but beyond that you are looking at total replacement.
To make things more interesting the changes to the inlet side will also affect the exhaust side. If you were to build the perfect exhaust system without inlet valve modifications and then to change the inlet valves that exhaust system will prove inadequate. The gas volume being pumped will change and the weaknesses in the torque curve will change so you are basically back to square one. The same applies to changing the cam profile or compression ratio.
If you are going down this route all I can suggest is that you decide on a head and cam profile first, get it flow tested so you know what you've got and then worry about the exhaust.