Dear listena,
Looking at your configurations we can see that you are passing VRRP heart beat form the swith, and your switch is a Layer two switch,
it must have had STP enable causing your Ports to go into Listening Learning state on flap and blocking hello messages (what we call heartbeat) and hence both routes think of themselves as master, and as soon as your ports go into active state they take their actual roles,
It is a good design practice on a switch to make all the ports which are not connected with other layer two devices as STP edge ports, and then enable BPDU gaurd so that those ports cannot intrupt STP process.
so in your case, make Gi0/0/1,2 and 3 an stp Edge port and your issue will be resolved.