Objective: Chronic cerebrospinal venous insufficiency (CCSVI) has been extremely variable associated with multiple sclerosis in color-Doppler sonographic studies. We aimed to evaluate inter-rater agreement in color doppler sonograpy venous examination. Design: Inter-rater agreement study. Setting: First-referral multiple sclerosis centre. Participants: thirty-eight multiple sclerosis patients, and 55 age-matched (± 5 years) controls. Intervention: sonography was carried out in accordance with Zamboni's five criteria by 8 sonographers with different expertise, blinded to the status of cases and controls. Each subject was evaluated by two operators. Primary and secondary outcome measures: inter-rater agreement was measured through the Kappa statistics and the Intraclass Correlation Coefficient. Results: The agreement was no higher than chance for criterion 2-reflux in the deep cerebral veins (Kappa=-0.02) and 4-flow not Doppler-detectable in one or both the internal jugular (IJVs) or vertebral veins (VVs) (-0.09). It was substantially low for criterion 1-reflux in the IJVs and/or VVs (0.29), criterion 3-IJV stenosis or malformations (0.23), and criterion 5-absence of IJV diameter increase when passing from sitting to supine position (0.22). The Kappa value for CCSVI as a whole was 0.20 (95% confidence limits=-0.01 - 0.42). Intraclass Correlation Coefficients for the measure of cross-sectional area ranged from 0.05 to 0.25. Inter-rater agreement was low for CCSVI experts (kappa= 0.24; -0.11 to 0.59), and non-experts (0.20; -0.33 to 0.73); neurologists (0.21; -0.06 to 0.47) and non neurologists (0.18; -0.20 to 0.56); cases (0.19; -0.14 to 0.52) and controls (0.21; -0.08 to 0.49). Zamboni-trained neurosonographers ascertained CCSVI more frequently than non-trained. Conclusions: Agreement was unsatisfactory for the diagnosis of CCSVI as a whole, for each of its five criteria, and according to different subgroups. Standardization of the method is urgently needed prior to its further application in studies of patients with multiple sclerosis or other neurological diseases.