No. It would not be pointless at all. First of all, IP addresses are extremely vague. 75% of the time it’s extremely hard to get someone’s exact location via IP. However, your IP address can be used against you in certain types of attacks, such as a DOS/DDOS. People usually only get doxxed if they live in the middle of nowhere and their house is the only building nearby.
It also depends what VPN you are using. If you use certain types of VPN, such as Express VPN, you can switch your IP a limitless amount of times, if you switched the location of your VPN multiple times, it’s very unlikely people will be able to tell or even bother to sift through your IP addresses to differentiate your IP from your VPN ones.
Even if you slip up once, you should always continue to use measures such as VPNs to protect yourself. VPNs are just a good thing to invest in, if you ever accidentally click on a phishing link or install a virus, you’re automatically much more protected. There are people that argue that VPNs don’t hide anything, unless you’re threatening to blow up a government building, it’s extremely unlikely that anyone would ever tell the FBI to contact your VPN company to reveal your actual data.
(Also, unless you were at a personal location at the time your IP was unprotected, it doesn’t really matter anyway)