The current environment of technological and competitive changes influences not only the business R&D environment but also government driven national R&D strategies. Open innovation has now become an important paradigm that is replacing the outdated paradigm of closed innovation. Many companies and nations have been increasing R&D investment because R&D has been considered a driving force for national and corporate competitive advantage. The purpose of this paper is to evaluate and compare the performance of R&D focused on open innovation according to scientific and technological outputs which is based on paper publications, patents and etc. Comparisons should not be only based on the quantity but also on the quality of the output. This paper shows that it is possible to develop DEA models that utilize the Analytical Hierarchical Process in order to transform the qualitative index into a quantitative index. Hence, the relative efficiency for R&D organizations is obtained based on both quantity and quality outputs and subsequently provides comprehensive and realistic methods for decision makers to identify levels of project efficiency.