If you're using the SQL backend for tokens, there is an explicit API call to remove a token, but it expects you to know it already. The internal Token API doesn't currently do anything to remove expired tokens from a backing store, so a manual cleanup job to remove expired tokens for the SQL implementation would be needed.

The lack of cleanup of expired tokens in the SQL backend should be listed as a bug

I've opened a bug (1008587) related to this.