I recently built a gaming computer, though I'm still saving for a good graphics card, so I've got a Ryzen 3-2200G, it's integrated gpu, and 8Gbs of ram in the meantime. I'm on Windows 10 with the most recent version of 64-bit Java with 4Gbs of ram dedicated to Minecraft.
The problem I'm having is a really low fps (0-17fps: 7fps average) on low settings regardless of whether I'm playing Modded (testing with 1.7.10, 1.12.2, and 1.10.2 modpacks) or Vanilla (testing only 1.12.2). The strangeness comes in when I decided to see just how low my fps would be on max settings and my fps went up (10-35fps: 20fps average). While I liked the improvement, the fps I'm getting is still way too low as my 8 year old laptop with hardware 1/3rd of the strength get's the same fps on low settings as my PC is getting on the improved high settings.
A couple of friends decided to play the old Agrarian Skies 2 pack cause one of them doesn't have the hardware to play newer packs. I had already loaded up the Fool Craft pack and a pre-existing Skyfactory 3 save and noticed really low fps. Going back to 1.7.10, which was always a quite unstable compared to the surrounding releases, my fps was even lower capping at 3fps max before playing around with the settings and relaunching where it jump up to about 7-10fps on average with slight highs and dips to 0 (one of which lasted multiple minutes).
I have no clue what is causing this and I'm finding it very strange. All web searches come up with they typical "How to increase your fps" results which are completely unhelpful. It's a very strange issue and I know my hardware is good enough to do much better than this.
You said you have 4 GB dedicated to Minecraft, but is that also for those modpacks you mentioned? Modern modpacks need probably 5 to 6 dedicated gigs to play smoothly (or at all) and you can change the gigs under the settings of the twitch launcher.
I have an Nvidia 1050 TI but I still need to dedicate 6 gigs of RAM for most modpacks or I will get an awful experience.