How to suppress the second harmonic of RF in the mixer output?

I am designing a receiver with sim14+, from mini-circuits, the IF is 500MHz~4GHz. The LO is about 5.7GHz~9.2GHz and the main RF is fixed at 5.2GHz. The RF/LO frequency of the sim14 covers 3700MHz~10GHz, and the IF is about DC~4GHz. Then I exchanged the RF and IF port. However, I found when the input frequency is about 2.6GHz+100kHz, -30dBm, with LO of 7.8GHz, +10dBm, the RF output is about -40dBm at 5.2GHz-100kHz and the second harmonic of the 2.6GHz+100kHz, i.e. 5.2GHz+200kHz is about -100dBm. It means that the second harmonic of the RF is a spur in my design, cause it would locate in the passband of the 5.2GHz bandpass filter. How can I suppress the second harmonic of the RF? Would someone tell me an effective way? No change to the 5.2GHz IF. Thanks.

All good mixers generate harmonics. Therefore it is a good idea to start with a frequency plan that includes harmonics, to see if and where to expect problems.
You can reject the harmonics at RF output by using a low-pass filter . All mixers need appropriate filters at their ports for a clean spectrum.

Also to add what was said above; some designers design filters that will terminate certain image and other 'products' output from a mixer, thereby providing a resistive, dissipative termination to unwanted energy. Not sure if your design package may account for these output facets that seen in real life ...

Jim

Differential Mixer may be an alternative...

Agree w/ Jim. Many senior engineers in my company use simple 2-arm diplexer filters after mixers, so the desired signal goes down one branch, and all the harmonics go down the other... right into a 50 ohm resistor (well-matched loads at all frequencies, keeps the mixer happy).

I know someone add some voltage to the mixer to cancel the harmonic.