an approach for using the crowd to filter ideas submitted in open innovation engagements increases filtering accuracy and reduces rater time compared to conventional (5‐point Likert scale) approach key insight: crowds are better at eliminating bad ideas than identifying good ones.