Zenom said: How much can their life be peaceful without the grail.
Without the Grail, everyone's lives would have been very different to begin with, and very few of them would have ended up like this. Most of the Servants would have never known each other, and none of them would have ever met their Masters.
The Einzberns would have had no reason to invite the Magus Killer to act as their Master for HF4, after all, and that means he'd never have met Irisviel, and Illyasviel would never have been born. He'd still be doing jobs with Maiya.