When I take my foot off the gas, my car looses rpm's insanely fast, which means my car doesnt "roll" for long. Anyone know any reason why?

On Friday, when I was driving home from work, the RPMs would drop extremely slowly when trying to shift. I would push the clutch in, shift, and then I would have to wait about 4-5 seconds before the tach would drop enough to match rpms for the shift. When I pulled into my garage, I reved it up...