I've become a big fan of the index funds lately but I have a few questions.

How do you diversify in the indexes? I know that they provide some instant diversification but if you are in a position to open several funds is it better to have one broad spectrum index, like a total market, or is it better to buy distinct sectors of indexes?

It depends on how you want to diversify. If you are going to overweight a particular asset class (small cap, for example), then it would make sense to have a separate index fund for that. If, however, you would only weight such component indexes (small cap, large cap, blend versus value, etc.) by their market capitalization, then you would get no extra benefit over single total market index fund, since such a fund, by definition, already contains all of those component indexes weighted by their market capitalization.

I think it's worth considering an international index in addition to a total market index, because most of the total market index funds don't really include international stocks.

The first level of diversification is protection against individual stocks going bust. More is usually better. So on the surface, a Total Stock Market index looks pretty diverse. Depending on which index you pick (S&P, DJ, MSCI, etc), you'll typically see the index hold roughly ~5000 different stocks. Sounds way better than just holding 500 stocks in the S&P500? Unfortunately, most traditional indexes are market-cap weighted. That means ( share price * outstanding shares ) is what controls how much to buy in each company. This is better for tax efficiency but it also means the 10 largest companies in the United States make up 20% of the S&P500. And since the S&P500 is 85% of the Total Stock Market, Exxon alone going bust would mean a 3% drop even though you theoretically own thousands of stocks. Hence the reason why the Total Stock Market tracks the S&P500 very closely -- the 4500 non-S&P500 stocks make up such a small percentage that performance variations have little effect. The solution here is to buy the Large/Medium/Small indexes directly and decide on your own percentages to reduce the weighting of the ultra-large companies and increase the impact of the small companies.

The next level of diversification is reduction of volatility. If two indexes track each other closely, buying both indexes would be redundant (example SP500 + Total Stock). Instead, you look for categories that are not correlated with each other. The idea here is the variations between different asset classes moderate each other to form a more stable growth line without impacting performance. What classes to invest in and how much, that's a subject needing more than a paragraph or two to cover. You'll have to google up reading materials to decide your own preferences. As a starting point, I can give you the breakouts I've been able to classify for my portfolio:

Domestic Large Growth
Domestic Large Value
Domestic Small Growth
Domestic Small Value
Domestic REIT
Domestic Bonds / Short
Domestic Bonds / Medium + Long
International Developed Large Growth
International Developed Large Value
International Developed Small Growth
International Developed Small Value
International Developed REIT
International Developed Bonds / Short
International Developed Bonds / Medium + Long
International Market
International Market Value
Commodities

Some of the above categories probably will only give a small boost in return/risk ratio but I like squeezing every tiny bit of performance out of my portfolio.

Because most indices are capitalization weighted, buying a total market index fund might be the same thing as buying 97% in a large cap index fund and 3% in a small cap index fund because the total market capitalization is about 97% large cap (loosely estimated for this example). If small cap companies do better you might not notice it as much than as if you had 20% or 30% in the small cap index fund.

Adding an international stock index fund and/or a Treasury index fund wold add further diversification.