Consistency of Pairwise Comparisons and Its Role in Optimal Detection of Customers’ Preferences

Purpose: This paper analyses a problem that originates in the weighted-average model, a mathematical construct introduced by the theory of multicriteria decision-making that can be used to detect what product a customer desires. The problem occurs because the model needs to know the weight the customer assigns to each product feature, aside from the levels of all the product characteristics, in order to calculate the overall value of the product. And since by one approach the weights can be estimated by optimization, the question arises which optimization criterion to select for the procedure, as different criteria will lead to different weights and thus to different product evaluations. The paper analyses the problem in connection with the so-called consistency of pairwise comparisons, which are utilized in the optimization and describe how much the customer prefers one product feature to another. The analysis shows that the problem of which criterion to use to calculate the weights can be eliminated if the pairwise comparisons are consistent. The analysis is performed within predefined criteria and is supplemented with case studies supporting the findings. Methodology/Approach: Linear algebra, optimization techniques, case studies. Findings: The results represent a prescription customers can use if they want to avoid the pitfalls of selecting a specific optimization criterion when informing the product maker about what they want based on the weighted-average model. Research Limitation/Implication: The results are related to a specific decisionmaking model, although that model is still very general and natural. Originality/Value of paper: The problem of selecting an optimization criterion to determine decision weights is not discussed in the theory. Category: Technical paper


INTRODUCTION
In quality management, the role of customers is of paramount importance and many quality techniques are related to this concept (Zgodavova and Slimak, 2008). It has therefore been historically imperative to learn what they desire so that an appropriate product could be designed, manufactured and ultimately delivered. Since many aspects or product characteristics enter the design of such products, and they often fight each other in the sense that an improvement of one feature deteriorates another, one might think of employing a discipline that is capable of setting up these product characteristics in such a way so as to turn the product design to a reasonable compromise, when it comes to the customer's perceptions and desires. Among such disciplines, the theory of multicriteria decision-making (MDM) has had its sound position, having been tailored exactly for these kinds of problems. Generally speaking, as it doesn't have to concern only product designs, the theory tries to solve the problem of which decision to make, or which product design to choose in our context, when more decisionmaking criteria, or more product characteristics, are to be taken into account. The theory presents a variety of methods that try to pick a reasonably good decision, or product design. A class of these methods requires knowledge of weights or importance of the decision-making criteria, or product characteristics, in order to find a compromise among all the available decisions or product designs. These methods evaluate each decision, or product design, by the expression + ⋯ + , where is the level of the i-th product characteristic under the considered design and is the nonnegative weight of that characteristic. All weights sum to one. The higher the value of the summation, the more preferred the design .
One of the fundamental questions the theory tries to answer is how to select the weights, without which the calculation cannot be obviously performed. Several classes of methods exist in this regard. In one class, the methods are deterministic (Ishizaka and Nemery, 2013), whereas in another class the methods are stochastic and deal with uncertainty (Levy, 2015). The latter, based on the probability theory, is suitable for finding typical weights for an entire group of decisionmakers, or customers in our context, when the group is unknown as a whole and statistical principles must be applied to deduce the properties of that group on the basis of an analysis of its subsets. Yet another class of methods that model a different source of uncertainty in the decision-making process is the one employing the fuzzy set theory (Park et al., 2009;Wang, Kevin and Wang, 2009;Ye, 2010). Many of these methods use the so-called matrix of pairwise comparisons S which contains ratios of the unknown weights (Saaty, 2005). The ratios describe the magnitude of the weights when they are compared to one another. The higher the ratio, the more important the product characteristic represented by the weight in the numerator of the ratio, compared to the importance of the other characteristic represented by the weight in the denominator. If the ratio equals one, the two weights are the same, and the corresponding characteristics are equally important. The reason for this construct is that it is usually simpler for the customer, and thus more accurate, one would expect, to state such a ratio than to provide the weights directly. Let = = / . Each element is usually defined using Saaty's scale of grades ranging from one to nine (Saaty, 2005). The grades describe to what extent one product characteristic is preferred to another. The bottom grade of one means the two corresponding characteristics, the -th and the -th, are equally important, whereas the upper grade of nine prefers as much as possible the -th characteristic to the -th. Each grade has its specific verbal description that details the extent to which one characteristic is preferred to another. Such a description enables the customer to better quantify their perceptions about what is important and what is not. The elements are inserted in the matrix S so as to make it reciprocal: = 1/ . The main diagonal of the matrix naturally contains only ones. In the following text, it is assumed that the matrix has these properties and its size is at least 2 × 2, i.e. two or more product characteristics are involved in the analysis related to their weights.
Given the matrix of pairwise comparisons, containing information about the opinions of the customer on the importance of the product characteristics, diverse methods are subsequently applied to evaluate the matrix. The methods result in a set of weights . These methods also include approaches that seek the weights by optimizing a mathematical expression which is a function of the elements of S. Such an optimization searches for the weights that, in a sense, bring the ratios / as close as possible to the provided values . In order for this calculation to make sense, the matrix S must make sense in the first place. This is the reason why the matrix is required to be consistent. An -by-matrix is consistent if its elements satisfy the equations = for all , , = 1, 2, … , . The amount of (in)consistency is measured by the inconsistency index − / − 1 , where is the largest eigenvalue of S. If the index is smaller than 0.1, the amount of consistency is considered good enough. In the case of a (fully) consistent matrix, it can be shown (Shiraishi, Obata and Daigo, 1998) that = and the index is thus zero. Generally, the smaller the index, the better.
If the matrix S has a reasonably high consistency, the weights based on the matrix can be calculated. As has been outlined, such calculations may utilize an expression to be optimized, an optimization criterion O. The optimization occurs when the calculation provides the weights that minimize O, an expression based on differences between / and . Some of these expressions are provided later in the text. Naturally, the resulting weights generally depend on the type of the optimization criterion selected, which raises the question which optimization criterion to choose. One possibility is to employ the principles of goal programming for the set of available optimization criteria -it is possible to adopt optimization criteria , … , , the optimal values of which are , … , , respectively, if the criteria are used individually for finding the weights, and then find the weights that minimize ∑ − . This would take into account all the optimization criteria simultaneously. Such procedure, however, would not be elementary, as some of the terms in the summation can be expected to be nonlinear and even nonconvex functions of the weights, meaning that a sofwaresupported numerical procedure would be required to find only a local minimum of the expression at best, not even the global minimum. It is more convenient to select a single optimization criterion to avoid these mathematical problems. Although this may still result in a non-elementary procedure, the approach will generally be simpler. Of course, the question which optimization criterion to select still remains unanswered. This paper discusses the problem of selection of an optimization criterion, exploiting the concept of consistency of the matrix of pairwise comparisons S, because not only does this concept rationalize the matrix itself, but it also significantly contributes to solving the problem of which optimization criterion to use when finding the customer's weights. The analysis focuses on a specific class of optimization criteria, which contains many popular mathematical expressions used for these purposes, and answers the question to what extent an element of this class plays a role in the whole procedure of finding the weights in the case of full consistency of the matrix S.

EXACT WEIGHTS
Let us start by analyzing the set of linear equations with n unknown weights : If we wanted a solution to (1) to represent true weights, inequalities should also be attached to (1) -each should be nonnegative. We shall see, however, that a solution to (1), if it exists, automatically meets this condition. Let us also note that a solution to (1) trivially satisfies the equation = for = , since = 1, and given the reciprocity of S, = also holds for > . Thus, if (1) holds, then = holds for all the indices , = 1, 2, … , .
In the matrix form, equations (1) The matrix M has − 1 − 2 • … • 2 • 1 + 1 = − 1 ! + 1 rows and columns, so its rank is at most . Looking at the second, third, …, -th column of this matrix, these columns are clearly linearly independent because only the trivial linear combination of these column vectors-the combination involving zero coefficients-yields the null vector (Bican, 2000).
and so, given the last row of the matrix, 1 = 1/− + ⋯ + 1/− would have to hold, which is impossible given that 's are all positive. Therefore, the first column of M cannot be a linear combination of its other columns, in other words, the rank of M is exactly .
Let us now extend M by adding the column vector b, and write the matrix equation: for some coefficients * . The symbol "~" replaces the n-th, n+1-st, …, the second-to-last row of M. Let S be fully consistent: = , , , = 1, 2, … , . Then, in particular, = , = 1, … , − 1, > , holds. For this reason, by setting up * = 1 + + ⋯ + + , = 1, … , , we get: which is the same as (4). In other words, nonzero coefficients * were found so that (4) is satisfied, and so the matrix M and the extended matrix (M, b) have the same rank because the last column of the extended matrix, the column b, is linearly dependent on its other columns by (4). By the Frobenius theorem (Bican, 2000), this means that (1) has a solution. The solution will represent "exact weights" in the sense that their ratio (the division is possible-see the discussion below) equals exactly the corresponding element from the matrix S. What's more, the dimension of the set of all solutions to the equations Mv = 0 is known to be generally nrank(M) (Bican, 2000), which is zero now. This means that the solution to the equations Mv = b is determined uniquely (Bican, 2000).

The main result of this paragraph is that a fully consistent matrix S implies existence of a unique set of exact weights, or a unique solution to (1).
The opposite implication is also true, however (Ramík and Perzina, 2008). If (1) has a unique solution, then S is fully or perfectly consistent. Therefore, the standard matrix of pairwise comparisons S, which is reciprocal, contains positive elements only and ones on its main diagonal, is fully or perfectly consistent if and only if there exist exact weights satisfying (1). When this happens, the exact weights are determined uniquely.
When it comes to the resulting exact weights, none of the weights is zero. If a weight was zero, then all the weights would have to be zero to satisfy = , but then the weights could not sum to one, so (1) would be violated.
Further, we also know that all the weights are positive. If they were all negative, they couldn't sum to one again. If some of them were positive and others negative, = couldn't hold. The only solution is thus a set of positive numbers that represent true weights.
To give an example of this situation, let: The matrix satisfies our assumptions and is consistent-the index of inconsistency is zero, as the largest eigenvalue of the matrix is 3 (EqsQuest, 2017). Regarding the rank of M and its extension (M,b), we have by elementary transformations (Bican, 2000): Since the two matrices have the same rank of 3, the exact weights exist uniquely. Their values are = 0. 11 //// , = 0. 66 //// , ( = 0. 22 //// , and they satisfy = , , = 1, 2, 3, + + ( = 1, where is the element of S from its i-the row and j-the column.

CONSISTENCY AND CRITERIA FOR FINDING WEIGHTS
If S is not fully consistent, the exact weights do not exist, as we have seen. Another possibility then is to seek weights whose ratio is in a sense close to the corresponding elements , which reflect the customer's feelings about the ratios. Different optimization criteria were defined for this purpose-by optimizing these criteria, such an accession of the weights to the elements of S can be carried out. Standard optimization criteria include the expressions: and the weights are obtained by minimizing one of these criteria. This way, "the least squares method" (LS) and "the logarithmic least squares method" (LLS) were defined, the latter being also known as the method of geometric average, as the weights minimizing (11) can be expressed explicitly as a normalized geometric mean. From the mathematical point of view, to solve (10) or (11) means to seek the global extreme of the function of the weights (10) or (11) on the set 5 = 67: ∑ = 1 9 ∩ 67: > 0, = 1, … , 9. In the case of a fully consistent matrix S, when the exact weights exist, i.e. = / , and the weights are positive, as we have seen, so that the vector of the weights belongs to L, expressions (10) and (11) equal zero at the point represented by these exact weights. The two expressions cannot take on a lower value by their design, which means that the vector of exact weights is the point of global minimum of (10) and (11) on L. This is a rather simple but important result, because while minimizing (11) results in an explicit formula for the weights, the normalized geometric mean (Ramík and Tošenovský, 2013): minimizing (10), for instance, is much more complicated. Many scholarly texts deal with the complexity of constrained optimization of similar functions (Bonnans et al., 2006;Nocedal and Wright, 2006;Bertsekas, 2016), and there is no complete theory in this respect to this date! Regarding the global minimum of (10) and (11), the same conclusion is clearly valid not only for LS and LLS, but for any criterion of the form A , … , , the minimization of which on L is to provide the suitable weights.

Therefore, if the matrix of pairwise comparisons S is fully consistent, the global minimum of any such function g defined on L, with no other conditions imposed on this function, is at the point represented by the vector of exact weights.
Since that vector is determined uniquely, the resulting weights, the exact ones, are the same regardless of what optimization criterion g is used for their calculation. In other words, the discussion on which criterion g to choose to find the weights is pointless, as all such criteria will give us the same vector of weights in the case of consistent S! The idea is portrayed in Fig.1 which describes the case of the optimization criterion g(v1,v2) = g(x,y) for a two-weight problem with the function f(⋅) = sin(⋅) and the matrix S made up of elements = = 1, = 2, = 0.5. Figure 1 shows that it may not be elementary to find the point of global minimum of the corresponding optimization criterion. When S is fully consistent, however, it is possible to resort to a far simpler criteron, such as the one used by LLS, to get the same point of extreme, i.e. the same weights. In this example, LLS yields = D = 0.66, = E = 0.33.

Figure 1 -An Example of a More Complicated Optimization Criterion Z = G(X,Y)
From what has been said so far, it should be clear that before starting to solve an MDM problem, a working matrix of pairwise comparisons S with a high degree of consistency should be first constructed. Hardly ever will it happen that the decision-maker, or the customer in this case, will manage to set up a perfectly consistent matrix. Then, one could try to adjust its elements to a small extent to make the matrix fully consistent. Such a procedure is not going to be simple unless a systematic approach is adopted. And such an approach can be designed. To turn a highly consistent matrix to the fully consistent one, we could try to solve the auxiliary optimization problem: subject to: Viewing (13), (14), we want to change the elements of the not quite consistent matrix S to make it fully consistent, as described by (14), the change being expressed by the terms nij, but we want that change to be as small as possible, so that the change still reasonably reflects the original intentions of the decision-maker, or the customer. This procedure is not ideal in its current form, however, as it works with quite a few variables and constraints, and thus becomes cumbersome. Fortunately, it turns out it suffices if the constraints (14) are valid only for the indices i < j < k, i.e. for the elements above the main diagonal of the matrix S. If the problem (13), (14) is solved for these indices only, and the elements on the main diagonal of S are left to be equal to one, while the elements below the main diagonal are automatically changed to be reciprocal to the elements that solved (13), (14), the changed matrix will be fully consistent as is shown below. Moreover, small enough changes to the elements above the main diagonal will lead to small changes in the elements below the main diagonal because of the continuity of the reciprocal function. Thus one can focus only on altering as little as possible the elements above the main diagonal of S to secure its full consistency.
To see this, if constraints (14) are satisfied for the indices i < j < k, i.e. the changed matrix is consistent in the elements above its main diagonal, then obviously: and so the changed matrix is consistent also in the elements below its main diagonal. If i < j > k, then we have, for the case i < k < j: and, for the case k < i < j: so the changed matrix is also consistent if the property is checked with the elements both from above and below its main diagonal. Finally, to check this property with the elements from its main diagonal participating -the property is obviously valid if both the elements are from the main diagonal, i.e. they are both equal to one. If only one of them equals one, the result is also straightforward: To give an example of the procedure, we use the following matrix of pairwise comparisons with the amount of incosistency equal to 0.005. Solving (13), (14) in Matlab for the elements above the diagonal of S, we obtain: .
The highest eigenvalue of * is four, therefore the matrix is fully consistent. Still, it hasn't changed much compared to the matrix S.

CONCLUSION
This paper analyzed relations between the consistency of the matrix of pairwise comparisons and the mathematical form of an optimization criterion which, using the comparisons, determines the weights or preferences provided by the customer before the overall product value, as perceived by the customer, is calculated. The impulse for the analysis was the fact that the resulting weights depend on the optimization criterion selected if the matrix of pairwised comparisons, used for the optimization, is not fully consistent. The conclusion of the analysis is such that within a general class of optimization criteria, it does not matter which criterion is selected for the calculation of the weights as long as the utilized matrix of pairwise comparisons is fully consistent. Since the optimization criteria result in the same set of weights in such cases, it makes sense to define the pairwise comparisons consistently, so that subsequently, any optimization criterion, and preferably the one that simplifies the calculations, can be selected to calculate the weights. Since it will be almost always difficult for the customer to define the pairwise comparisons in a fully consistent way immediately, the paper incorporated a procedure that will help the customer convert the provided pairwise comparisons to fully consistent ones. The procedure is most easily implemented with a suitable software incorporating optimization algorithms.