Why my harmonic rejection get worse?

Hi friends,
I am designing a 2.4GHz bandpass filter for approval.I try two solutions(in fact ,they are not too good for small PCB size),and get the results :modification#1 and modification#2.when i measured them alone as filter by VNA.It is obvious that modification#2 is better than modification#1 at 1.6GHz.But when i put the filter into system,i found i got the inverse result by spectrum analyzer.Any friend kindly help me find the root cause?
Thanks

Can you explain with words in more detail what you are doing. A 2.4 Ghz bandpass filter will have a harmonic at 4.8 GHz, but I do not think that is what you mean.

Am I right in thinking you have an existing bandpass filter that has a 2.4 GHz passband, and you are trying to add on a fix to give more rejection at some other frequency?

You do realize that you can not just take a bandpass and a lowpass filter and hook them up to each other and expect the rejections of the two to add linearly! There will be interaction. You need to measure the banpass WHEN HOOKED UP to the lowpass on the VNA for you to get an accurate measurement. This is because when you measure the bpf into a 50 ohm load (VNA), it acts differently than when it is hooked up to a lowpass filter (not a 50 ohm load when in the rejection band)

Your post had been better understandable, if you mentioned the frequency tripler operation explicitely.

If I understand right, both "modifications" are far from intended harmonic suppression, particularly at 1.6 GHz. So the
discussion shouldn't be about the modification effect rather than the bad filter performance in general.

I think, the basic problem is, that the filter element's coupling, including the non-ideal ground plane behaviour haven't′been
considered in the design.

I might be tempted to just add in shunt, a series connected L-C that is tuned at 1600 MHz. That way, you will have an additional rejection pole at 1600 MHz. I would try to add it as close to the antenna output connector as possible (maybe even connecting to the antenna connector ground as the ground return).

Now, adding this pole to ground will screw up the return loss a little. If you can take it, just add it in. If you can not take the degrded return loss, you will have to have a series C leading up to it, and another series C leading away from it, to match at 2.4 GHz.

It is possible to add the elliptical pole at 1600 MHz right in your bandpass filter, but then you have to worry about ground loop currents in the pc board, and it might not be as effective.

If you had a little more room in the board layout, you could have an open circuited quarterwave stub, tuned at 1600, as the elliptical element instead of lumped L-C.

Thanks ,biff44.You are always kind to me .you are right,i add an inductor in shunt with C45,should separate the lowpass filter from bandpass filter,and add some rejection poles for 1.6GHz.