What is the difference between orthogonal antenna and omni-directional antenna?

As far as I know there is no relation between orthogonal antennas and an omni-directional antenna.

I guess, orthogonal antennas refer to the antennas which have orthogonal patterns in space. And omni-directional antenna is one which have more or less a circular pattern in azimuth plane and directional pattern in elevation (example: donut-shape).

I think I know what you refer about.
For example if you are using two-cross dipoles (turnstile antenna) which are orthogonal one to each other, the antenna pattern suffer a little bit from the perfect circle omni directionality, which has for example a ground-plane antenna.

The orthogonal antennas can replace in a lot of applications a perfect circle-pattern antenna.