Connection caching technique is widely used to reduce the overhead incurred by frequency connection establishment in distributed computing environment using connection oriented protocol. We present an efficient connection caching scheme where we divide the system into several host groups and connection between the hosts in the same group is kept prior to others. In other words it makes the connection kept longer by sharing the connection use information between hosts in the same group. Every host group consists of hosts which have heavy intercommunication. And we present performance evaluation of the effect of host grouping upon connection caching and performance comparison of various grouping in several aspects includingmeanservice request interarrival time, mean service time, group size, grouping and type of group. Simulation results show that host grouping is effective in evrey performance criterion and proper grouping of hosts enhances the performance. We also present analysis results for Markov process model of our scheme which are consistent with the simulation results.