Agricultural non‐point source pollution, common in water supply catchments worldwide, can have significant environmental and human health impacts, and its mitigation poses a challenge for policymakers. We used deliberative multi‐criteria evaluation (DMCE) to identify a mix and sequence of policy instruments (or policy design) to address agricultural non‐point source pollution using a case study of Cryptosporidium contamination in the Myponga River water supply catchment, South Australia. The major impediments to adoption of on‐farm water quality management and benefits for ecosystem services were identified using a landholder survey for use as decision criteria in DMCE. The DMCE approach involved stakeholders in policy design during two community fora held in the catchment. We developed six policy scenarios and quantified their impact on decision criteria. The relative importance of decision criteria was quantified using swing weights and consensus was reached on the preferred policy scenario. The mix, sequence, and targeting of instruments in the preferred policy scenario were refined based on information obtained through the deliberative process. Impediments to adoption included a lack of both information/knowledge and financial resources. The recommended policy scenario involved targeted information, followed by an incentive program, and finally the regulation of a mandatory code of practice for water quality management. Detailed, catchment‐specific context obtained through DMCE was critical for refining an effective mix and sequence of policy instruments. The techniques may be readily used to select and schedule policy instruments for effective mitigation of agricultural non‐point source pollution in other drinking water supply catchments elsewhere.