Check the exhaust manifold for leaks or loose nuts.

Spray soapy water on the manifold and flange.

If air bubbles form, you have found the culprit.

Due to heat cycles, expansion, and contraction, over time, a stock manifold will start to crack and develop your symptoms.

Is it more noticeable at Temp or under a load rather than cold?