This study investigates the effect of filler content (wt%), presence of interphase and agglomerates on the effective Young’s modulus of polypropylene (PP) based nanocomposites reinforced with exfoliated graphite nanoplatelets (xGnPTM) and carbon nanotubes (CNTs). The Young’s modulus of the composites is determined using tensile testing based on ASTM D638. The reinforcement/polymer interphase is characterized in terms of width and mechanical properties using atomic force microscopy which is also used to investigate the presence and size of agglomerates. It is found that the interphase has an average width of ~ 30 nm and modulus in the range of 5 to 12 GPa. The Halpin-Tsai micromechanical model is modified to account for the effect of interphase and filler agglomerates and the model predictions for the effective modulus of the composites are compared to the experimental data. The presented results highlight the need of considering various experimentally observed filler characteristics such as agglomerate size and aspect ratio and presence and properties of interphase in the micromechanical models in order to develop better design tools to fabricate multifunctional polymer nanocomposites with engineered properties.