We propose methods of linear transceiver design for two different power constraints, sum relay power constraint and per relay power constraint, which determine signal processing matrices such as base station (BS) transmitter, relay precoders and user receivers to minimize sum mean square error (SMSE) for multi-relay multi-user (MRMU) networks. However, since the formulated problem is non-convex one which is hard to be solved, we suboptimally solve the problems by defining convex subproblems with some fixed variables. We adopt iterative sequential designs of which each iteration stage corresponds to each subproblem. Karush-Kuhn-Tucker (KKT) theorem and SMSE duality are employed as specific methods to solve subproblems. The numerical results verify that the proposed methods provide comparable performance to that of a full relay cooperation bound (FRCB) method while outperforming the simple amplify-and-forward (SAF) and minimum mean square error (MMSE) relaying in terms of not only SMSE, but also the sum rate.