In this paper a method for finding solutions of acoustic, vortex and entropy wave equations in a cylindrical tube with variable section area was suggested under the consideration of that the high frequency instability in a rocket engine combustion chamber is an acoustic phenomena, which Is coupled with combustion reaction. and that a combustion chamber and exhaust nozzle are usually shaped cylindrically As a consequence of that some method. which enable the mathematical analysis of the influence of entropy and vortex waves to acoustic wave. was suggested. According to the method reflection coefficients of acoustic wave on a supercritical nozzle was numerically calculated, through which it was presented that entropy or vortex waves can strengthen or weaken the reflection rate of acoustic wave.