In this study, robustness index and uncertainty analysis were proposed to quantify the risk inherent in the process of climate change vulnerability assessment. The water supply vulnerability for six metropolitan cities (Busan, Daegu, Incheon, Gwangju, Daejeon, and Ulsan), except for Seoul, were prioritized using TOPSIS, a kind of multi-criteria decision making method. The robustness index was used to analyze the possibility of rank reversal and the uncertainty analysis was introduced to derive the minimum changed weights of the criteria that determine the rank reversal between any paired cities. As a result, Incheon and Daegu were found to be very vulnerable and Daegu and Busan were derived to be very sensitive. Although Daegu was relatively vulnerable against the other cities, it can be largely improved by developing and performing various climate change adaptation measures because it is more sensitive. This study can be used as a preliminary assessment for establishing and planning climate change adaptation measure.