You might stand the chance of a rock bouncing up into the drive belts but I have driven 6000 miles since refitting the engine with no probs.
Really should put it back on I suppose.....