Orbital-enhanced warping effect in px,py-derived Rashba spin splitting of monatomic bismuth surface alloy

Spin-split Rashba bands have been exploited to efficiently control the spin degree of freedom of moving electrons, which possesses a great potential in frontier applications of designing spintronic devices and processing spin-based information. Given an intrinsic breaking of inversion symmetry and sizeable spin–orbit interaction, two-dimensional (2D) surface alloys formed by heavy metal elements exhibit a pronounced Rashba-type spin splitting of the surface states. Here, we have revealed the essential role of atomic orbital symmetry in the hexagonally warped Rashba spin-split surface state of the $$\sqrt 3 \times \sqrt 3 R30^0$$ 3 × 3 R 3 0 0 BiCu2 monatomic alloy by scanning tunneling spectroscopy (STS) and density functional theory (DFT). From dI/dU spectra and calculated band structures, three hole-like Rashba-split bands hybridized from distinct orbital symmetries have been identified in the unoccupied energy region. Because of the hexagonally deformed Fermi surface, quasi-particle interference (QPI) mappings have resolved scattering channels opened from interband transitions of px,py (mj = 1/2) band. In contrast to the s,pz-derived band, the hexagonal warping is predominately accompanied by substantial out-of-plane spin polarization Sz up to 24% in the dispersion of px,py (mj = 1/2) band with an in-plane orbital symmetry.