Question about simulating Quadrature Mixer

I design the mixer for I and Q channel. The RF input is a single-end port but the output has 4 port ( I+ I- Q+ Q-) for quadrature structure. I have a question when simulating the NF. Do I need to combine the output into 1 port for the Noise simulation? I thought that if I compute the noise only from I or Q only the signal power will be lower than it should be.

___ I+
_____X___ I-
|
RF__|
| ___ Q+
|_____X___ Q-

X is for the Mixer

Any suggestions are very welcome.

Thanks a lot.

Added after 2 minutes:

Sorry for the image. I tried to describe that it has single-end input and two differential-end outputs.

Use a combiner and take into consideration the added insertion loss.