No, there is no ‘but.’ Beginning your statement with that phrase does not release you from the responsibility of being sexist. (aka “I’m not trying to be sexist but…” Yes, you are.)