Should you still want the light under the fan, one option is to suspend it from the sides not directly above, e.g. four thin cables (one with power of course), going up at (say) 10-30 degrees from vertical (along both angles).
Having the cables at an angle like that will keep it from moving at...