We present new constraints on the cosmic optical background (COB) obtained from an analysis of the Pioneer 10/11 Imaging Photopolarimeter (IPP) data. After careful examination of the data quality, the usable measurements free from the zodiacal light are integrated into sky maps at the blue (${sim}0.44{mu}m$) and red (${sim}0.64{mu}m$) bands. Accurate starlight subtraction was achieved by referring to all-sky star catalogs and a Galactic stellar population synthesis model down to 32.0 mag. We find that the residual light is separated into two components: one component shows a clear correlation with the thermal $100{mu}m$ brightness, whilst the other shows a constant level in the lowest $100{mu}m$ brightness region. The presence of the second component is significant after all the uncertainties and possible residual light in the Galaxy are taken into account, thus it most likely has an extragalactic origin (i.e., the COB). The derived COB brightness is ($(1.8{pm}0.9){ imes}10^{-9}$ and $(1.2{pm}0.9){ imes}10^{-9};erg;s^{-1};cm^{-2};sr^{-1};{AA}^{-1}$ in the blue and red spectral regions, respectively, or $7.9{pm}4.0$ and $7.7{pm}5.8;nW;m^{-2};sr^{-1}$. Based on a comparison with the integrated brightness of galaxies, we conclude that the bulk of the COB is comprised of normal galaxies which have already been resolved by the current deepest observations. There seems to be little room for contributions from other populations including "first stars" at these wavelengths. On the other hand, the first component of the IPP residual light represents the diffuse Galactic light (DGL)-scattered starlight by the interstellar dust. We derive the mean DGL-to-$100{mu}m$ brightness ratios of $2.1{ imes}10^{-3}$ and $4.6{ imes}10^{-3}$ at the two bands, which are roughly consistent with previous observations toward denser dust regions. Extended red emission in the diffuse interstellar medium is also confirmed.