But sincerely, I don't feel bad about being selfish. Our household income is around $150k, and it's pretty evenly distributed between my wife and I, which means we pay an enormous amount of taxes. I'm comfortable that we're already contributing far more to society than we're taking.
You're being forced to give to charities you may not support. It's one thing to tax people to pay for defense, roads, and so forth that everyone uses. But now we give billions of tax dollars to charities through government grants. If the causes were so worthwhile, wouldn't people give on their own? Even though we give to charity, I'm not a fan of the government giving tax dollars to charities!
Honestly, this is one thing that bothers me about finance blogs. I don't understand the emphasis on "giving." Giving money away obviously has nothing to do with getting wealthy. It is a direct hindrance to getting wealthy. Giving money away may be admirable in its own right, but it is in direct contradiction to "getting rich." Giving money away can never help you get richer. If your goal is to get rich, then giving money away with no benefit can only be a detriment to that goal.
Why is giving money admirable?
The whole idea of "tithing" or otherwise supporting religious organizations that otherwise don't contribute to productivity originated in a time when priests were actually scientists and the made a very significant contribution. They figured out when to plant or when to harvest to avoid frost damage or the annual inundation, they cured the sick, and that kind of thing.
I have not seen a case where a religious leader has actually made a positive contribution along those lines in at least several centuries. Religions now basically only have entertainment value for their members. If people choose to support them, then that's their right. But I think the original need to "give" came from that role that priests played in society.
We've moved beyond that. We now have the US Naval Observatory to keep track of the calendar and doctors to heal the sick. We support those causes (whether we like it or not) through tax dollars (defense budget and medicare respectively).