This paper presents new learning algorithm of dynamic Bayesian networks (DBN) by means of constrained least square (LS) estimation algorithm and gradient descent method. First, we propose constrained LS based parameter estimation for a Markov chain (MC) model given observation data sets. Next, a gradient descent optimization is utilized for online estimation of a hidden Markov model (HMM), which is bi-linearly constructed by adding an observation variable to a MC model. We achieve numerical simulations to prove its reliability and superiority in which a series of non stationary random signal is applied for the DBN models respectively.