I've seen people make an XP farm of them and it seems way too complicated.
Although god friggin damn it seems fun to get like a swarm of them and guide them about and throw splash potions of damage on them ^3^
I'd just disable it and leave it. You can use it later if you want to.
Silverfish don't drop anything useful as of yet (except experience). If they update it so they drop something useful try to find another stronghold. You may need to use AMIDST though seeing as eyes of ender only point towards one of the three strongholds