Among semi-quantitative or fully quantitative lateral flow assay readers, an image sensor-based instrument has been widely used because of its simple setup, cheap sensor price, and compact equipment size. For all previous approaches, monochrome CCD or CMOS cameras were used for lateral flow assay imaging in which the overall intensities of all colors were taken into consideration to estimate the analyte content, although the analyte related color information is only limited to a narrow wavelength range. In the present work, we introduced a color CCD camera as a sensor and a color decomposition method to improve the sensitivity of the quantitative biosensor system which utilizes the lateral flow assay successfully. The proposed setup and image processing method were applied to achieve the quantification of imitatively dispensed particles on the surface of a porous membrane first, and the measurement result was then compared with that using a monochrome CCD. The compensation method was proposed in different illumination conditions. Eventually, the color decomposition method was introduced to the commercially available lateral flow immunochromatographic assay for the diagnosis of myocardial infarction. The measurement sensitivity utilizing the color image sensor is significantly improved since the slopes of the linear curve fit are enhanced from 0.0026 to 0.0040 and from 0.0802 to 0.1141 for myoglobin and creatine kinase (CK)-MB detection, respectively.