We can actually make stronger claims about factoring. We know for certain do that it is not in NP hard, so you are correct there

And I’m not exactly a security expert, but moving away from RSA at this point makes sense. Early assumptions about the difficulty of factoring large semi-primes certainly hasn’t panned out (in particular in light of the growing risk of quantum computers.)