It is, but it is also potentially quite dirty; any dust in the air ends up in the condensate. Dehumidifiers are actually quite good at collecting airborne dust and the evaporator matrix can get clogged by it; one reason why cars with A/C generally also have a pollen filter.
Just filter it through a piece of paper towel in a funnel and you'll get almost all the dust out - what's left on the paper might surprise you. If you boil the filtered water, it won't leave a residue - it's great for use in miniature steam engines.
It can also be quite heavily contaminated with bacteria; leave it for too long (weeks, not days), and you might find it develops a musty smell and starts going mouldy.
But once filtered, it's great for diluting concentrated antifreeze & screenwash, and topping up unsealed lead-acid batteries.
Another source of reasonably pure water is the runoff you get when you defrost the freezer.