David, I can empathize with a lot of what you said. I too was bullied in school, and also at home by my stepmother (you can read about her at http://www.firescribble.net/acs/betty.htm
The thing to remember about bullies is that they're the ones who are weak. They may be physically or intellectually stronger than you, but truly strong people never bully anyone else; they don't need to. If you let bullies affect your self-esteem, you're letting them win. Much of what my stepmother said to me was personally denigrating; she made me feel like a terrible person, selfish and lazy, irresponsible, thoughtless, incompetent, and downright bad. But I had a choice: I could either allow myself to be a victim and believe what she said, or I could dig my heels in and not give her the pleasure of destroying me. Once I made the right choice, everything became easier after that. Not easy, but easier.
I agree with the comments above suggesting that you seek professional help, but if for some reason you don't want to do that (I never did), you may be able to find the strength within yourself to put your past behind you and move on. Sometimes the simple act of choosing to not be a victim can make an enormous difference.