Since low-rise residential buildings are the most common and vulnerable structures in coastal areas, a reliable prediction of their performance under hurricanes is necessary. The present study focuses on developing a refined finite element model that is able to more rigorously represent the load distributions or redistributions when the building behaves as a unit or any portion is overloaded. A typical 5:12 sloped low-rise residential building is chosen as the prototype and analyzed under wind pressures measured in the wind tunnel. The structural connections, including the frame-to-frame connections and sheathing-to-frame connections, are modeled extensively to represent the critical structural details that secure the load paths for the entire building system as well as the boundary conditions provided to the building envelope. The nail withdrawal, the excessive displacement of sheathing, the nail head pull-through, the sheathing in-plane shear, and the nail load-slip are found to be responsible for the building envelope damage. The uses of the nail type with a high withdrawal capacity, a thicker sheathing panel, and an optimized nail edge distance are observed to efficiently enhance the building envelope performance based on the present numerical damage predictions.