Approximate Functions in a Problem of Sets Separation

In this paper, problems of mathematical diagnostics are considered. The most popular approach to these problems is based on statistical methods. In this paper, the author treats the mentioned problems by means of optimization. This approach can be useful in the case where statistical characteristics of the database are unknown or the database is not sufficiently large. In this paper, a nonsmooth model is used where it is required to separate two sets, whose convex hulls may intersect. A linear classifier is used to identify the points of two sets. The quality of identification is evaluated by the so‐called natural functional, based on the number of misclassified points. It is required to find the optimal hyperplane, which minimizes the number of misclassified points by means of the translation and rotation operations. Since the natural functional (number of misclassified points) is discontinuous, it is suggested to approximate it by some surrogate functional possessing at least the continuity property. In this paper, two surrogate functionals are introduced and studied. It is shown that one of them is subdifferentiable, and the second one is continuously differentiable. It is also demonstrated that the theory of exact penalization can be employed to reduce the given constrained optimization problems to an unconstrained one. Numerical methods are constructed, where the steepest descent directions of the surrogate functionals are used to minimize the natural one. Necessary conditions for a minimum are formulated for both surrogate functionals.