If magic were real, everyone would be able to use it so it wouldn't be special. And if everyone COULD use it, then everyone WOULD, and you'd get a lot of people who would be turbo tryhards with it, just like with everything. Neijuan would suck the fun out of it FAST.
Also the extent to which you could use it would probably be genetic (aptitudes in general are genetic, so why would this be any different?) and since bad genes often come in packages we'd be more likely to be below average in aptitude.
In a hypothetical universe where magic were real, you'd have threads on this form (or it's parallel universe equivalent) from people who studied magic their whole lives talking about getting "magic mogged" by a chad who barely tries and threads about how they want to kill themselves because they're 30 and have been studying magic for decades but saw a video of some 10 year old asian who can do magic 1,000 times more complicated and impressive.
Magic existing would only be fun if you either had a superior genetic aptitude, or were for some reason one of the only people who could use it. Ultimately your desire for magic is just an extension of your natural desire to MOG.