swallow counter

In Pulse swallow counter design, should I need to take care the latency of
prescaler? The frequency is 4GHz input for this divider.
Any usual mistake in this kind divider design?

Thanks,