I can build around it. I can create a custom field for rating, that is filled on the fly via JS. Every time the user votes, the field is updated. Then sort by this default field.
But that would be a crutch. And it's a shame to invest time in a crutch just because the standard function is buggy. 😉