The "10ADC" input would only be used when you want to measure current in the few Amp range. Typical practice is to place the black lead in the "COM" (common) input, the red lead in the "VOma" (Volts, Ohms, Milliamp), and to always connect the black lead to the common or ground level on the device under test. I probably don't need to tell you, (but I still will) NEVER , NEVER measure a voltage source when you have current measurement or Ohms measurement selected on your meter.
The motor power leads shouldn't have to be connected to test the Hall devices, but it probably would not have an affect on the readings one way or the other.
I have no idea what your motor looks like, and I have tried to keep out of the way of folks who really know these motors, but I do know how to use test equipment, and troubleshooting techniques. I would guess that if you simply disconnect the hall sensors entirely, the controller will be in sensorless mode. But it is just a guess.