The disintegration of ice shelves, reduced sea-ice and glacier extent, and shifting ecological zones observed around Antarctica highlight the impact of recent atmospheric and oceanic warming on the cryosphere. Observations and models suggest that oceanic and atmospheric temperature variations at Antarctica's margins affect global cryosphere stability, ocean circulation, sea levels and carbon cycling. In particular, recent climate changes on the Antarctic Peninsula have been dramatic, yet the Holocene climate variability of this region is largely unknown, limiting our ability to evaluate ongoing changes within the context of historical variability and underlying forcing mechanisms. Here we show that surface ocean temperatures at the continental margin of the western Antarctic Peninsula cooled by 3-4 °C over the past 12,000 years, tracking the Holocene decline of local (65° S) spring insolation. Our results, based on TEX(86) sea surface temperature (SST) proxy evidence from a marine sediment core, indicate the importance of regional summer duration as a driver of Antarctic seasonal sea-ice fluctuations. On millennial timescales, abrupt SST fluctuations of 2-4 °C coincide with globally recognized climate variability. Similarities between our SSTs, Southern Hemisphere westerly wind reconstructions and El Niño/Southern Oscillation variability indicate that present climate teleconnections between the tropical Pacific Ocean and the western Antarctic Peninsula strengthened late in the Holocene epoch. We conclude that during the Holocene, Southern Ocean temperatures at the western Antarctic Peninsula margin were tied to changes in the position of the westerlies, which have a critical role in global carbon cycling.