cascade saw filter -80db
just out of the top of my head
possible reason 1: u r cascading 3 2-port network together, so their ABCD parameters mulitiply one by one, the final ABCD matrix probably wont give you desired performance.
possible reason 2: the two filter may have coupling between each other; that may result in additional peak of S21
just now my noob guess.
The problem is that each filter is originally designed to go between two definite impedances, usually 50 ohms purely resistive. When you replace one of these terminations by another filter the out of band impedance goes to zero or infinity reactive. This will give you a different total frequency response than the combination of the two filters alone.
The filters (assumed to be bandpass) should be a good match to 50 Ohms in the passband. Of course, outside the passband, they are highly reflective. That is why they are filters.
You have a length of coax between the two filters. Out of band, this length of coax has very high reflection terminations on both ends. This makes the length of coax into a resonator.
I think it is likely that the extra S21 peaks you are seeing are due to this coax resonator. To test this hypothesis, simply change the length of the coax between the two filters. The extra S21 peaks should move accordingly. To move the extra resonances much higher in frequency (where they might not cause problems, depending on your stop band rejection requirements), just make the coax as short as possible. If the shortest possible coax is not short enough, you need to design a single filter that will meet your requirements.
One other alternative is to put a circulator between the two filters. Then the out of band reflection from the second filter is absorbed. A circulator will increase loss, size, mass, and cost. However, if you have a circulator handy, it is quick and easy to do.
Thank you for your kindness!
The length of the coax actually influence the peak frequency.The shorter length coax get a closer peak to the passband of the filter.I saw the order is:If the length is shorter,the extra peak will be more closer to the passband and while tunning the filter's center frequency(our filters are tunnable filter),the shifting scale of the distance of the passband to the closest extra peak peak is smaller.It seems the main passband is modulate by a extra resonator peaks.
I set the coax length is 2cm,the performance isn't acceptable.The filter is made by cavity with two SMA ports.I think it's hard to get the coax length shorter.
I have thought about inserting a circulator.But it's hard to find any wideband products.
Added after 14 minutes:
First of all,I think the VSWR maybe influence the connection.The VSWR of our filters is less than 1.5.But when we simulated the case by software with an ideal VSWR of 1,it hardly get any better.I think It must be the phase of some frequency in the stopband of the filter which brings some extra peaks of S12.But it's really complicated to calculate the total ABCD matrix.