The need for content-based retrieval from visual media, such as image and video data, is ever increasing rapidly in many applications, such as electronic art museums, internet shopping malls, internet search engines, and medical information systems. In our previous research, we proposed an architecture, called the HERMES, which is a Web Service-enabled visual media retrieval framework. In this paper, we propose the Web Service workflows that are employed in the HERMES. We describe how we designed the workflows for service registration and query processing in the framework. We especially explain how metadata and ontology can be utilized to realize more intelligent content-based retrieval on visual media data.