I would think it would be nice to see, even on Peaceful mode, some sort of half-translucent shell of a spider lying around sometimes. It's a mob, but it does absolutely nothing. Like a frozen pig. It doens't even make sound, it just gets pushed around like a weighted baloon.
It could give a slight scare to people wandering around in a cave, and make them recheck whether they had peaceful on or not >
Dammit, why is it that they keep getting more and more horrifying every time there's a discussion of them?
Because, contrary to popular belief, they don't feed on flies or other insects. In fact the only spider that actually eats what they are supposed to eat is the aptly named "bird-eating spider".
You see, what spiders actually feed on... Is FEAR...
True story. Not really.
On topic now, this is an utterly pointless idea. Spiders aren't that the scariest mob in minecraft, so if you wanted to scare people on peaceful, why not make a creeper exoskeleton.
Although any mob like this is utterly pointless and would be a waste of time.
They do, like the other guy said, but did you also know that if a spider falls from a height higher than itself (not very high at all), then it's exoskeleton could crack open and it's organs slowly leak out until it dies?
I like the concept of this. Here is how I think it might work.
When a spider despawns, there is a low chance (maybe a 1 in 50 chance) that it will leave behind an exoskeleton. An exoskeleton behaves like a boat or a minecart when not on water or rails respectively. They could be destroyed (and perhaps drop something (maybe themselves)) or they could be pushed around and used for decoration.
Another possible way these could work is that when broken, it would drop an exoskeleton. This exoskeleton would behave as the previously described decoration one. But, if a naturally spawned exoskeleton is not destroyed in two minecraft days, it hatches into 6-8 cave spiders. This could put cave spiders on the surface, or in other areas you wouldn't normally find them in.
I'm just throwing these ideas around because I like the concept of spiders leaving behind exoskeletons.