The endothelial surface plays an important role in the pathogenesis of atherosclerosis and the regulation of coagulation. It has become increasingly clear that while perturbed endothelial cells generate procoagulant activity, under normal conditions they possess multiple antithrombotic and anticoagulant mechanisms, including generation of prostacyclin and plasminogen activators and synthesis of thrombomodulin as a cell surface cofactor for thrombin-catalyzed activation of protein C. In addition, anticoagulantly active heparan sulfate proteoglycans, including heparin-like molecules are apparently present on the vascular surface. Previous studies showed that homocysteine, a thromboatherogenic and atherogenic agent, inhibits an endothelial thrombomodulin-protein C anticoagulant pathway. We examined whether homocysteine might affect another endothelial anticoagulant mechanism; i.e., heparin-like glycosaminoglycan-antithrombin III interactions. Incubations of cultured endothelial cells with homocysteine reduced the amount of antithrombin III bound to the cell surface in a dose- and time-dependent fashion. In contrast with a marked reduction in the maximal antithrombin III binding capacity, the radioactivity of [35S] sulfate incorporated into heparan sulfate on the cell surface was minimally reduced. Although neither net negative charge nor proportion in total glycosaminoglycans of cell surface heparan sulfate was altered by homocysteine treatment, a substantial reduction in antithrombin III binding capacity of heparan sulfate isolated from homocysteine-treated endothelial cells was found using both affinity chromatography and dot blot assay techniques. The antithrombin III binding activity of endothelial cells decreased after preincubation with homocysteine, cysteine, or 2-mercaptoethanol, containing a sulfhydryl group; no reduction in binding activity was observed after preincubation with methionine, alanine.(ABSTRACT TRUNCATED AT 250 WORDS)