That's the whole point. Giving men access to women's spaces and services is 'harmful' for women and girls. It really isn't just about toilets and safety risk. If a disabled woman is told they can't be guaranteed a same sex carer, that's harmful. If a girl can't be guaranteed a female counsellor after being raped, that's harmful.
The same applies to male single sex services. They are there for a reason.
You have empathy only for those excluded and can't even see why we need single sex services in the first place.
Legally they are women. Like it or not.
Explain why trans men should have unfettered access to a boys changing area. Using the same "logic" you've used above.
You're only argument, to date, has been that men shouldn't have any issue's with women in such areas, so they should be allowed. Even the dangerous ones.
As for a disabled women "being denied" a same sexed carer and being harmed by not having one, you're in cloud cuckoo land. Most carer's are family these days, and usually their partner.
Or, as I suspect you will, move it to a hospital setting where the majority of those doing the caring are women anyway. If you try saying that the doctor should be a women, on/by demand, then I think you'll find most people are just happy to be seen.
Your system involves making demands for service over and above what anyone else is getting. Nearly always without thought about the cost, which isn't always monetary. Rescheduling appointments, treatments takes time. Time in which the person needing treatment could be treated. Time in which others could also be seen and treated.
As for empathy "only for the excluded", I was in that excluded group, legally, for half my life. To some extent I'm still excluded. I've never pushed for extra/special treatment, only equal treatment. Therein lies the difference.
I've agreed to go on medication whilst still a child. The long term effects have only recently been acknowledged and explored. Long term now is any time longer than six months. Things change over the years, as I've found out.
I was seen four times by a female specialist, before she decided removing part of the brain was the only course of treatment left. She seemed to delight in the detailed narrative of how the operation would be done. I later found out that I wasn't the only one who'd found her methods odd. As someone else said on a review site, she took an almost perverse delight in her explanations. Reviews on her were pulled and she's no longer on the medical register.
To you, I should have no trouble in getting that from a female. Suck it up, because women get that every day. Not bloody likely.