The problem of dynamic range!

Although I should think that you can do it in ADS, I never really bothered. There is a cute little application from Agilent called AppCAD, which is free. From the main menu choose Signals-Systems->Noise Calc. Other commercial packages exist, and many companies choose to write their own spreadsheets to do system calculations.
You are dealing with noise figure and IP3, but also metrics such as SFDR, SNR, SINAD, compression and IM3 come into play.