OREX‐J: towards a universal software framework for the experimental analysis of optimization algorithms

The Operations Research EXperiment Framework for Java (OREX‐J) is an object‐oriented software framework that helps users to design, implement and conduct computational experiments for the analysis of optimization algorithms. As it was designed in a generic way using object‐oriented programming and design patterns, it is not limited to a specific class of optimization problems and algorithms. The purpose of the framework is to reduce the amount of manual labor required for conducting and evaluating computational experiments: OREX‐J provides a generic, extensible data model for storing detailed data on an experimental design and its results. Those data can include algorithm parameters, test instance generator settings, the instances themselves, run‐times, algorithm logs, solution properties, etc. All data are automatically saved in a relational database (MySQL, http://www.mysql.com/ ) by means of the object‐relational mapping library Hibernate ( http://www.hibernate.org/ ). This simplifies the task of analyzing computational results, as even complex analyses can be performed using comparatively simple Structured Query Language (SQL) queries. Also, OREX‐J simplifies the comparison of algorithms developed by different researchers: Instead of integrating other researchers’ algorithms into proprietary test beds, researchers could use OREX‐J as a common experiment framework. This paper describes the architecture and features of OREX‐J and exemplifies its usage in a case study. OREX‐J has already been used for experiments in three different areas: Algorithms and reformulations for mixed‐integer programming models for dynamic lot‐sizing with substitutions, a simulation‐based optimization approach for a stochastic multi‐location inventory control model, and an optimization model for software supplier selection and product portfolio planning.