My advice would be to wait until all the nutters at the end disappear. Every year there are idiots who head home with no lights/drunk/speeding etc. Last year we had no trouble anchoring (yacht) just off Cowes. Once it's dark you're in no hurry to get home anyway so enjoy another glass of wine (your second of the evening of course) and then head home slowly making sure you keep a really good look out. It's a good evening out