A stochastic optimal LQR control problem under some integral quadratic (IQ) constraints is studied, with cross terms in both the cost and the constraint functionals, allowing all the control weighting matrices being indefinite. Sufficient conditions for the well-posedness of this problem are given. When these conditions are satisfied, the optimal control is explicitly derived via dual theory.