Ah ok those sound like just standard hall sensor wiring connectors.
All In One Schematic / Diagrams
you might find a good idea here.
if you can follow back to the esc from that 5 pin connector and figure out what wires are red/black that should be ~5V and the other 3 are the hall sensors. Seems like no 6th wire that is usually temp sensor, watch out for melting your motor with the VESC powa!!!