In this paper, we propose two schemes to reduce the complexity of fixed-complexity sphere decoder (FSD) algorithm in the ordering and tree-search stages, respectively, while achieving quasi-ML performance. In the ordering stage, we propose a QR-decomposition-based FSD signal ordering based on the zero-forcing criterion (FSD-ZF-SQRD) that requires only a few number of additional complex flops compared to the unsorted QRD. Also, the proposed ordering algorithm is extended using the minimum mean square error (MMSE) criterion to achieve better performance. In the tree-search stage, we introduce a threshold-based complexity reduction approach for the FSD depending on the reliability of the signal with the largest noise amplification. Numerical results show that in $8{ imes}8$ MIMO system, the proposed FSD-ZF-SQRD and FSD-MMSE-SQRD only require 19.5% and 26.3% of the computational efforts required by Hassibi’s scheme, respectively. Moreover, a third threshold vector is outlined which can be used for high order modulation schemes. In $4{ imes}4$ MIMO system using 16-QAM and 64-QAM, simulation results show that when the proposed threshold-based approach is employed, FSD requires only 62.86% and 53.67% of its full complexity, respectively.