Wouldn't a trans man, by your logic, have been perceived as a girl/woman for years until medically transitioning, unless he socially transitioned very young? I'm trans masc non-binary and at one point thought I might be a trans man and very much know what it's like to experience sexism from folks believing I was actually a woman.
Also, even as someone who comes from a middle class background and received some of these social work services, I can thoroughly say the quality is pure dog shit the vast majority of the time, so I can't imagine what it must feel like when someone is in poverty on top of whatever else they have going on.