I think the biggest issue with the "came out of the ground originally" argument is that we process and concentrate stuff to the point that what was once harmless no longer is.
You've got a valid point there, and even though uranium originally came from the ground, I'm not advocating burying nuclear waste and building a playground on top of it.
I agree that there are a few categories of waste that warrant additional processing before being disposed of. I support recycling, and particularly initiatives designed to collect and properly dispose of electronic waste (due to the high concentration of lead and other toxic materials).
But for most stuff, I don't think there's anything wrong with just burying it. I get irritated at the knee-jerk objections of all the tree-hugging hippies who scream and squeal about "protecting the Earth" at the slightest suggestion that we bury diapers and Ziploc baggies. When you ask them how
it's harming the Earth to bury these things, they just say "because they take thousands of years to break down!" Ask them, "so what?" and they never have a rational answer. The best they can offer is emotion and rhetoric about "the children." But it doesn't hold up to analysis.