Agree with practically everything said above, so pick what works for you out of that.
Personally we use a 10 year old software program called Microsoft Money to track our expenses and forecast account balances etc. you should be able to pick up a copy for $5 or less. It can import records from your bank and once setup a lot of the recording can happen automatically.
One of the most useful tools for us and the reason we still use it is the ability to forecast account balances out to whatever period you like, so we can tell when we are likely to be either tight of flush for $$ and act accordingly.
To cut down on the manual recording rather than trying to track every individual item, we track what is important to us and have an additional category called "blow money" which is where we dump all the ice creams, coffee, cash spending etc.. The stuff we consider wasted or discretionary.
Managing our finances this way takes no more than about 1 hour per week.
A recommended previously, we pay ourselves first.