It took awhile for me too. I want to see it was my late teens years and early adulthood when I started being attracted to women. During childhood and early teens I would brush off bullying and teasing from girls who would sometimes call me ugly within earshot or say eww after my name, but as I got older I began to realize this was a problem for me and I never got the kind of attention other guys would. Friends and family never seemed to get it and if I complained about something like if I had a crush reject me, they'd brush me off, tell me to get over it, or that I could do better. Which is ironic considering I've never done anything anyways and they know that.
Having never had any success and then someone telling me I could do better often felt like I was so disconnected from my peers that no one could understand me.