Transmission performances of direct detection-based 100-Gb/s modulation formats are investigated and compared for metro area optical networks. The effects of optical signal-to-noise ratio sensitivity, chromatic dispersion, cross-channel nonlinearity, and transmission distance on the performance of differential 8-ary phase-shift keying (D8PSK), differential phase-shift keying plus three-level amplitude-shift keying (DPSK+3ASK), and dual-carrier differential quaternary phase-shift keying (DC-DQPSK) are evaluated. The performance of coherent dual-polarization quadrature phase-shift keying (DP-QPSK) with block phase estimation and coherent DP-QPSK with digital differential detection are also presented for reference. According to our analysis, all three direct detection modulation formats could transmit a 100-Gb/s signal over several hundred kilometers of a single-mode fiber link. The results also show that DC-DQPSK outperforms D8PSK and DPSK+3ASK, and the performance of DC-DQPSK is comparable to that of coherent DP-QPSK with digital differential detection. The maximum transmission distance of DC-DQPSK is over 1,000 km, which is enough distance for metro applications.