It certainly seems that the weather has turned autumnal now , and having been in Arillas at this time of year in the past, I would take some warm clothes ,as the evenings in particular can be a bit chilly, and something waterproof (umbrella/cagoule etc). However, don't take the forecasts too seriously. We have found that on countless occasions, the north and/or east of Corfu has been wet, but Arillas has stayed dry!
Take the opportunity to do those walks, that it's too hot to do in the summer and discover new tavernas and restaurants.
You'll still have a great time!