On the Norms of p-Nilpotent Residuals of Subgroups in a Finite Group.

Let G be a finite group and p be a prime. We define N N p ∗ (G) to be the intersection of the normalizers of the p-nilpotent residuals of all two-generator subgroups of G whose p-nilpotent residuals are nilpotent. We show that N N p (G) = N N p ∗ (G) . Using the method in the present paper, we will be able to give an affirmative answer to an open problem in Shen et al. (Mediterr J Math 19:191, 2022), which also indicates that similar conclusions hold for many formations. It is also proved that G = N N p (G) if and only if every three-generator subgroup H of G satisfies H = N N p (H) . To this end, we introduce and investigate the IO- N N p -groups, i.e., the groups G such that G ≠ N N p (G) , but each proper subgroup and each proper quotient of G equals its p-nilpotent norm. Moreover, new results in terms of the p-nilpotent norm and the p-nilpotent hypernorm N ∞ N p (G) are given. [ABSTRACT FROM AUTHOR]