This study presents a steel container crane movement analysis and assessment based on structural health monitoring (SHM). The accelerometers are used to monitor the dynamic crane behavior and a 3-D finite element model (FEM) was designed to express the static displacement of the crane under the different load cases. The multi-input single-output nonlinear autoregressive neural network with external input (NNARX) model is used to identify the crane dynamic displacements. The FEM analysis and the identification model are used to investigate the safety and the vibration state of the crane in both time and frequency domains. Moreover, the SHM system is used based on the FEM analysis to assess the crane behavior. The analysis results indicate that: (1) the mean relative dynamic displacement can reveal the relative static movement of structures under environmental load; (2) the environmental load conditions clearly affect the crane deformations in different load cases; (3) the crane deformations are shown within the safe limits under different loads.