When I want to play a mod that's in 1.12.2, I need to remove to other mods in the mods folder that's in different version, but it won't let me to move them. So I copy and paste it to another folder, and try to delete it, yet it still won't let me move them to the recycle bin either, how can I remove them?
You need to close the game before doing anything - you can't move/delete files that are in use - and even if you add new mods the game will not recognize them until you restart it (this only works for resource packs and saves).
Also, if you do want multiple modded instances create a new profile (installation) with its own game directory, then each directory will have its own "mods" folder - no need to move/rename/delete files every time you switch versions (and saves as well - so you don't accidentally open a world in the wrong modded instance, or crash/error the game due to trying to load a configuration file created by a different version; this also applies to vanilla):
That's why I switched to using https://multimc.org/ as my Minecraft launcher. It does as TheMasterCaver discusses, putting each "instance" you set up in its own folder. This makes playing different modded versions of Minecraft a snap!