Dirt spawns naturally underground, you don't need to only use the dirt on the surface.
You're right. I don't know why I forgot that! I'm currently playing this seed and having a blast doing it. I found plenty of dirt. But I did have to travel to a desert for enough sand to bury the ocean monument, but a nether portal makes the trip a short one. I've expanded the island out so far that the ocean monument is now under the island, and being turned into a guardian/prismarine farm.