Ship identification in optical remote sensing images is essential for a wide range of civil and military applications, including maritime rescue, port management, and sea area surveillance. However, current studies focus mainly on ship detection or coarse-grained ship size identification, rather than fine-grained type identification. Moreover, interference from clouds and port facilities as well as complex conditions such as occlusion and shadows increase the difficulty of ship type identification. To address these problems, we propose a novel ship identification method by employing adjacent-branched saliency filtering and prior representation-based classification strategies, which achieves high-precision type recognition performance for large and medium-sized ships under complex environmental interference conditions. In the candidate region extraction stage, a multiscale feature aggregation structure that utilizes feature map fusion in adjacent layers and receptive field mining within the same extraction branch is presented, providing fine representation of the location and edge characteristics of ship targets in complex scenes. In the classification stage, the low-rank term describing interclass differences and the graph-based regularization term describing intraclass differences are added to the representation model as prior constraints, which can correctly classify ships in the presence of complex environmental interference such as occlusion and shadow. Experimental results on two high-quality ship datasets indicate that the proposed method realizes state-of-the-art identification performance compared with benchmark methods.