Most stars form in star clusters and stellar associations. However, only about $\sim 1\%$ of the presently known exoplanets are found in these environments. To understand the roles of star cluster environments in shaping the dynamical evolution of planetary systems, we carry out direct $N$-body simulations of four planetary systems models in three different star cluster environments with respectively $N=2{\rm k}, 8{\rm k}$ and $32{\rm k}$ stars. In each cluster, an ensemble of initially identical planetary systems are assigned to solar-type stars with $\sim 1 M_{\odot}$ and evolved for 50~Myr. We found that following the depletion of protoplanetary disks, external perturbations and planet-planet interactions are two driving mechanisms responsible for the destabilization of planetary systems. The planet survival rate varies from $\sim 95\%$ in the $N=2$k cluster to $\sim 60\%$ in the $N=32{\rm k}$ cluster, which suggests that most planetary systems can indeed survive in low-mass clusters, except in the central regions. We also find that planet ejections through stellar encounters are cumulative processes, as only $\sim 3\%$ of encounters are strong enough to excite the eccentricity by $\Delta e \geq 0.5$. Short-period planets can be perturbed through orbit crossings with long-period planets. When taking into account planet-planet interactions, the planet ejection rate nearly doubles, and therefore multiplicity contributes to the vulnerability of planetary systems. In each ensemble, $\sim 0.2\%$ of planetary orbits become retrograde due to random directions of stellar encounters. Our results predict that young low-mass star clusters are promising sites for next-generation planet surveys, yet low planet detection rates are expected in dense globular clusters such as 47 Tuc. Nevertheless, planets in denser stellar environments are likely to have shorter orbital periods, which enhances their detectability.