do I need to worry about my base wheel quick release, it’s scratched, and maybe the QR won’t lock properly
As long as it slips in without the need of force and the security pin is fully attached, without the possibility to fall out (because of not being accurately tightened), I would say: no
when I unplug the pin, it’s a little bit hard to remove my wheel.
I wouldn’t worry. I read that a few times before.
Just don’t pull on the steering wheel, that wouldn’t be good for the shaft! Hit it lightly with your hand. Then it should actually come loose, from top to bottom or vice versa