We study the molecular order and morphology in poly(3-hexylthiophene) (P3HT) and poly(3-hexylselenophene) (P3HS) thin films and their blends with [6,6]-phenyl-C61-butyric acid methyl ester (PCBM). We find that substitution of the sulfur atoms in the thiophene rings of P3HT by heavy selenium atoms increases the tendency of the molecules to form better ordered phase; interestingly, their overall fraction of ordered phase is much lower than that of P3HT-based films. The higher tendency of P3HS molecules to order (aggregate) is consistent with more planar chain conformation simulated. The lower fraction of ordered phase (or the higher fraction of disordered phase) in P3HS-based films is clearly identified by in-plane skeleton Raman modes under resonant excitation conditions, such as a smaller ratio of the C═C modes associated with the ordered (∼1422 cm–1) and disordered (∼1446 cm–1) phases (I1422cm–1/I1446cm–1 = 1.4 for P3HS and 0.6 for P3HS:PCBM), compared with P3HT-based films (I1449cm–1/I1470cm–1 = 2.5 for P3HT and 1.0 for P3HT:PCBM) and a larger Raman dispersion of the C═C mode: P3HS (17 cm–1) versus P3HT (6 cm–1) and P3HS:PCBM (36 cm–1) versus P3HT:PCBM films (23 cm–1). The higher fraction of disordered phase in P3HS prevents the formation of micrometer-sized PCBM aggregates in blend films during thermal annealing. Importantly, this lower fraction but better quality of ordered phase in P3HS molecules strongly influences P3HS:PCBM photovoltaic performance, producing smaller short-circuit current (Jsc) in pristine devices, but significantly larger increase in Jsc after annealing compared to P3HT:PCBM devices. Our results clarify the effects of heavy atom substitution in low band gap polymers and their impact on blend morphology and device performance. Furthermore, our study clearly demonstrates resonant Raman spectroscopy as a simple, but powerful, structural probe which provides important information about “fraction/quantity of ordered phase” of molecules, not easily accessible using traditional X-ray-based techniques.