Looks like a no-brainer to me. If you're on track to pay off your mortgage in 15 years, go for it! Odds are these rates won't last too long, maybe a couple more years.
While I agree with you, mortgage rates could easily keep coming down further. The Fed is actively trying to push down long term rates and mortgages a are long term. It's not a one-to-one relationship but I would not be too surprised to see 30 yr mortgage rates at 3.5% and 15 year rates in the 2's a year or so from now. (But 5's would not surprise me either if economic conditions improve!)