We are talking about the structural analysis of the dynamic between classes of people, as you know. You don't need to show that every single white person is an oppressor in order to understand that ethnic minority groups are (overall) in a disadvantageous power relationship with the white majority. It's thus perfectly reasonable that a social club for an ethnic minority should be just for that group. You don't have to admit white people on a case by case basis just because some white people aren't racist.
The fact that men can simply say they are women - and that has been enough to change language, laws, safeguarding - is enough in itself to tell you that as a class it is men that are in the group that oppresses. Individual men don't get to opt out of that analysis of power relationships by saying they aren't men anymore and must be treated as individuals.
This constant theme of yours that everybody must be assessed on an individual basis and there is nothing meaningful to be said about the dynamic between wider groups in society is just a way of engineering access to women's spaces and services by trying to make it a case by case decision thing. It's unworkable. Nobody would tolerate it for any other group except women.
