I don’t think you’re doing anything wrong. Sometimes that happens.
I would recommend saving the values you have now, and retuning the HFI and often that will fix it.
Also, for me, motor temperature has a lot to do with how well HFI works. When my motors get hot, HFI performance degrades drastically.