If $G$ is a group and $H, K$ are normal subgroups of $G$, $H\leq K$, then $K/H$ is said to be a $G$-perfect factor if $[K/H, G] = K/H$. If $G$ is a nilpotent group, then every non-trivial factor of $G$ is not $G$-perfect. Conversely, if $G$ is finite and all non-trivial factors of $G$ are not $G$-perfect, then $G$ is nilpotent. We study (infinite) groups with no non-trivial $G$-perfect factors. We prove that if either $G$ is a locally generalized radical group with finite section rank, or $G$ has a normal nilpotent subgroup $A$ such that $G/A$ is a locally finite group with Chernikov Sylow $p$-subgroups for every prime $p$, and $G$ has no non-trivial $G$-perfect factors, then for every prime $p$ there exists a positive integer $s_p$ such that $\zeta_{s_p}(G)$, the $s_p$-term of the upper central series of $G$, contains the Sylow $p$-subgroups of $G$, and $G/Tor(G)$ is nilpotent. In particular, $G$ is hypercentral and the hypercentral length of $G$ is at most $\omega+k$, for some positive integer $k$.