FAULT_CODE_OVER_TEMP_MOTOR (feat crash at 47kph)

Got this fault again. It’s definitely happening when my temperatures are up near the thresholds.

This time I caught it all on metr.

Metr records in faults page that the fault was at 14:39:30 (PST)

things to observe in the charts:

  • yes noise in the temperature readings, when running full throttle every where you see 95% duty cycle in the charts.
  • 14:39:30.486: duty cycle starts drop from 95%. ppm still high presumed temp throttling throttling. though temp is only 75c
  • 14:39:30.870: we do see a temperature read by metr at 92c !! (moment later. ) so maybe it did spike to 100C to trigger the fault? also this wildly fluctuating temp is clearly noise. it’s also very large jump compared to anything else in the charts.
  • odd thing still being the temp in the fault code is lower than the threshold start 80c, and fault code trigger of 100c, and the highest metr recorded value of 92c.
  • it actually is a slight hill climb and my thermals are going up. and I know I have some slight thermal issues climbing hills.

I gonna guess: bug in vesc code that it’s not printing the actual temp measurement that triggered i the fault.

And, I’m clearly getting sketchy noise that the hardware filtering and software filtering of the vesc is not immune to. Part of me doesn’t see any reason not to have larger window on the low pass filter in the vesc code. a large spike across 400ms seems unlikely in the real world.

Next run at this I will try to do something to physical to shield the noise, such as twisting allt he phase wires to gether or separately shielding the sensor wires or something. I’d have to learn better there.

@Pedrodemio I’m quite sure you’re right now bout the injected noise.

Fault            : FAULT_CODE_OVER_TEMP_MOTOR
Current          : 21.8
Current filtered : 19.5
Voltage          : 34.44
Duty             : 0.950
RPM              : 36554.7
Tacho            : 1647371
Cycles running   : 184274
TIM duty         : 7980
TIM val samp     : 2
TIM current samp : 4200
TIM top          : 8400
Comm step        : 0
Temperature      : 74.90
1 Like

I haven’t had any answer on vedder forum apart from the initials reply

Maybe this is something to be looked on the new branch from @DerelictRobot , it sure is dangerous if you have no idea it will happen and are under heavy acceleration or braking

4 Likes

What’s the @DerelictRobot branch?

1 Like
4 Likes

Something that is currently literally just a fork of the VESC-TOOL branch and nothing to write home about as the work that has been done hasn’t been published yet. Needs more than a few days/weeks. :wink: But one of the goals is to try to get some feedback and data on issues liek this and/or 3rd party hardware.

Currently in the process of testing the new firmware along with everyone else.

I’m considering 4.0+ to be unstable/unknown for now.

5 Likes

Well damn. that was a read. :slight_smile: with a trampa slog in the middle.

Gist of OpenRoad ( @DerelictRobot’s branch ) as i understood it. Fork vesc-tool / bldc firmware try and make an ecosystem that manages releases. identifying stable releases from community testing and keeping them changing slowly and carefully. manage dev releases so that community can test new changes.

if we consider this not dealing with noise in temp sensors a safety issue which it kinda is, the fault at 50kph could easily taken me down. I see how the OpenRoad’s focus on stability would sort of connect it to this issue.

but there’s no new process with openroad yet, so a fork is a fork.

Do you know someone that wants to hack on a fix? we can do that on any fork of bldc and submit it to vedder and openroad assuming one is found. Making the issue clear to vedder is still probably a good way to find someone who wants to fix it and has the skills. ( as hard as that may be. )

4 Likes

I’ve made it clear there, with logs and all

It’s not a difficult fix, I think even I can try to do something, but would prefer to leave to someone with more experience by fear of screwing ir more

For now I know my motor won’t overheat while riding no matter what I do, so I just set the limits way higher, but if if my new board requires the temp limit I may try something

3 Likes

yeah. I feel that way too.

2 Likes

FAAAAAAK.

Aaaand today it did. I got the fault and crashed at 47kph. fortunately I’m fine. backpack took the brunt. helmet and pads the rest.

I had every intention of disabling or altering the over temp fault settings too and I just didn’t get to it.
i never triggered the fault on the commute, and I have been watching the temps closely. but today I gunned it up an incline that I’m usually slowing down on for traffic. calling myself stoopid on this one.

observe the temperature spike is quite sharp and sudden.
some weirdness with current spikeing as well but this is metr recording dual vesc over can and combining the motor currents. could be some other part of the problem, or could be the other motor suddenly taking all the load and spiking current.

but these two observations have me questioning the “noise” angle.

full metr log:

and again, the fault records a somewhat reasonable temperature. that’s clearly a bug.

Fault            : FAULT_CODE_OVER_TEMP_MOTOR
Current          : 43.5
Current filtered : 41.2
Voltage          : 33.44
Duty             : 0.930
RPM              : 34124.4
Tacho            : 11685660
Cycles running   : 542785
TIM duty         : 7812
TIM val samp     : 2
TIM current samp : 4200
TIM top          : 8400
Comm step        : 0
Temperature      : 69.76
2 Likes

oh wait! it’s not doing that any more. thanks @rpasichnyk !!! there’s separate graphs for each motor.

both motors do the current spike. so yeah

1 Like

read that wrong. one motor drops out. with the fault. (makes sense. ) the other spikes up. so I think it’s just handling the sudden full load and trying to compensate.

again @rpasichnyk I can’t tell you how happy it makes me to have current values for both vecs separate. can we get the temperature graphs for each vesc as well?

2 Likes

Illustrating the motor temp spikes. It very clearly jumps from 77c to 85c very suddenly in this example.

1 Like

Could it be a bad temp sensor? Would explain the temp jumps, and the temp jumps would explain the fault. If it spikes quickly to 150 or something, then returns to the proper value, the ESC will throw a fault because of that value.

Just a possibility

1 Like

why can’t we have i2c bus temperature and rotor-position sensors in our motors?

so sorry @fessyfoo you’re going through this. i wonder whether there is any possibility that this is a firmware dependent error? unlikely but would you consider downgrading to the last Ack release?

2 Likes

That’s one of the reasons I’m hoping metr can record motor temps distinctly for both motors. it shows them in realtime. but records doesn’t keep the secondary ATM.

I want to see if the other motor is getting this kinda noise and jumps.

I can probably add swapping this motor out with another. ( to swap the sensors ) to my list of things to try. it is only happening on the one side.

Is it me or is your voltage sag a bit much, i see 6-7V drops throughout

I think it’s a bit much, but I haven’t had enough experience to know for sure, and have been reading things considering opening a thread to ask about it. this is a 10s5p sanyo 20700b pack.

Considering the size and cells i don’t think you should be dropping anywhere near 6-7V at 40-50A battery load

Also I just noticed in your motor temp errors your voltage is 33v which is exactly at peak sag

3 Likes

I didn’t think they were quite that high though. I see 4-6v voltage drops. what’s normal?
towards the back half of my commute resting voltage gets pretty low.

Is that just an observation or is there some known connection to lower voltage increased temp or increased temp sensor errors.

at lower voltage the current could rise to compensate, and heat could rise. but I’m in the 34v throttling so current is being limited.

:thinking: so much to think about there.