After I re-fitted the connector I still had the problem. Today I didn't have the problem and all worked fine.
I have a theory what could be causing this problem:
When I came from the gym it had been raining and when I drove away I opened my window to get some air. What did I see? Drops of rain coming from the roof dripping exactly on the switchpanel and going directly inside the switch compartment:
(remember that this picture is taken from the left door since in Holland we have the steering wheel on the left. So the arrow is the window side)
It could be that the electronic circuit inside the switch gets somewhat shortcutted by the water or even the residu of it causing the signals to be distorted thus resulting in strange behaviour.
This would also explain why this problem is on the drivers side since that is the window you will most likely open if you need air or if your reaching for your burgers in the drive-in creating an open path if it's raining. Also I saw that the rainwater was mostly dripping in the switch for the driver side what could explain why the passengers side switch located on the driver side is not acting strange. Another reason for this could be the layout of tracks inside the switch element.
The fact that this problem with the window is very intermittend would only proof it more.
When I think about it I had no problem with all the warm weather that we had lately and since we started to get some rain last week I have it so now and then.
This is just a theory but for the people that have this window problem please keep an eye on it and let me know your experiences.
Thanks.