I'm 30 and I've had this issue my whole life. I've never been taken seriously, no one respects me and I'm invisible to women, they probably assume I'm in high school. I don't have the good kind of baby face either, I look like a weird 12 year old. My face is a round blob, no jaw and chin, chubby cheeks, puffy eyes, no definition whatsoever etc.
A lot of people here think baby face means young looking and attractive (aka boyish good looks.) If you look like an adult (18+) you don't have a baby face.
I look younger than most freshmen, but luckily I can grow a disconnected goatee (van dyke) and it kind of makes me look like a teenager trying hard to look like an adult, but it's better than looking 12. I weigh almost 200 lbs and I'm not short (not tall either), but nothing can make up for my underdeveloped face.