Let’s consider an analytical review of the simplest types of expert appraisals while paying a special attention to the works where experimental investigations of expert appraisals are described. Besides, in the lecture we shall introduce a totality of definitions of the simplest types of the expert appraisals are described.
While going on to a review of the simplest types of the expert appraisals described in the scientific literature let’s note that within its frameworks we, as far as possible, shall consider a totality of definitions of the simplest types of the expert appraisals. This is necessary for forming a general terminology which, in its term, will allow to approach the formation of the system of strict definitions of the simplest types of the expert appraisals. Unfortunately, for the present it is early to speak on a formation of the system of strict definitions of the simplest types of the expert appraisals. However, while even having just a totality of definitions of the simplest types of expert appraisals we shall avoid a number of complications. So, up to the present there are no strict definitions of such simplest types as marks and spot appraisals. While noting this fact G. Tale gives the following example [87].
Let an expert gave an appraisal — "an event will happen in 2025". What prognosis is this - a point, multi-point or interval one? While solving this problem we may consider the year of 2025 either as 365 days or a time interval with a duration of a year.
The circumstance that sometimes the name of the expert appraisal type coincides with the name of a procedure with the help of which the appraisal was obtained leads also to terminological complications. In that case when there is apossibility to use various names we will use that opportunity.
Verbal appraisals (1.0)
In that case when the words or sentences of the expertise metalanguage, which is rather near to the natural language, are allowed as the expert appraisals we shall speak about verbal appraisals. The final conclusions and quantificator words are examples of such appraisals. In the work [99] under the expertise metalanguage there are understood "those special values attributed to the words of the natural language if they are used in the expertise procedure in the following virtues (below there are mentioned the main ones):
1. The names of characteristics according to which the appraisal of the objects should be made. The following judgements may be mentioned as some examples: "the colour –green" or a conclusion of an expert physician "the tumour is malignant".
2. The names of relations (quantificator words) defined for a great number of objects being studied, those of the types "better", "strongly marked", "to a considerable extent", "more similar to... than... ", etc. The following statement of an expert could serve as an example of the use of quantificator words: "During the prognostication period the growth rates of GNP of the USA will grow to an inconsiderably".
3. Let’s add such natural expert assertions as "yes", "I don’t know", "if….., then…..", "or….or….", and any logical negation.
We may find the examples of using expert appraisals of the- (1.0) type when considering the first versions of the expert systems [178].
A presentation of the knowledge in these expert systems is in particular realized in the form of a record of the type "if…., then…." assertion type. From our point of view when studying the above-mentioned type of the expert appraisals, a linguistic approach will be expedient in accordance with which not only the numbers but also the words or sentences of a natural or artificial language are allowed. According to the opinion of V.B. Kouzmin [41] the linguistic presentation of the expert appraisal possesses "three incontestable advantages:
¾ it is most similar to the natural means of expressing appraisals - to a native language and professional terminology of experts, and therefore:
¾ it allows to express an appraisal in a quite correct and unified form;
¾ in contradistinction to appraisals in a natural language linguistic presentations yield to a mathematical processing".
The linguistic variable notion introduced by L. Zade [191] and based upon the inexact multitudes theory is studied at present in numerous works [31,156]. The study of the inexact relations expressed with the quantificator words is most often related to the fuzzy set theory although another way described in the work [104] is also possible for research works.
Groupings (2.0)
This type of the expert appraisal consists in "an indication by the expert a totality of non-intersecting classes indexed with elements of a certain multitude of values of the corresponding indication" [59].
A theoretical multiple concept of a division although it is similar to the notion of grouping but does not coincide with it because some classes of the grouping may prove to be empty, and in addition many indices are not shown. In that case when the task of the experts is a division of a multitude we are saying that we are getting an expert classification. Among the classifications sometimes the so-called "free" classification is considered when the number of classes into which the expert should divide the multitude is not indicated beforehand [92].
Evidently, it is expedient to consider a case when the sub-multitudes into which the multitudes are to be divided have non-vacant intersections. In this case we shall use the term "covering"[112]. The theory of fuzzy set will be evidently useful as a mathematical means when studying the coverings. The illegible multitudes in the expert appraisals are discussed in the work of M. Pinas [167]. Similar types of the expert appraisals such as a grouping and classification are discussed in the works of B.G. Mirkin [59], Yu.N. Tyurin [92], A.V. Maamyahi [53]. The common feature of those types of the expert appraisals is the fact that the experts set an equivalence ratio for a multitude.
The structures and regularities of forming and realizing the processes of various kinds of a classification were studied in the psychologic research works of L.S. Vigotsky, G. Piage, B. Inelder, S.Z. Diachenko, R.M. Froumkina, A.V. Mikheyev, A.K. Zvonkin and many others.
The reviews of the research works on this subject were published by P. Fress and P. Piage [98] and G. Kambon [129].
Pair comparisons (3.0)
This type of the expert appraisal involves an indication of a more preferable object by the expert in every presented pair of the objects (sometimes it is also allowed to state that both of them are equivalent or incomparable). Let’s give an example of the pair comparison.
Let's an expert was proposed to compare two probable versions of the answer: A1 - a relative price of oil has a trend towards the growth; A2 - a relative price of oil has a trend towards the stability. If the expert pointed out that A1 is more preferable (probable) than A2 - we have a pair comparison.
In the literature on the expert appraisals the term "pair comparisons" relates both to a data collection procedure and to a type of the expert appraisals.
In order to distinguish them let’s leave the name "pair comparisons" for the type of appraisals, but the data collection procedure for this kind of appraisals we shall call as a "comparison by pairs".
The methods of the analysis and processing of the pair comparisons are described in detail in the monograph [140]. The pair comparison procedure was suggested for the first time in the work [143] for appraising a comparative preferability of alternatives, and afterwards it was further developed in the works [88,126,184]. The pair comparison procedure is becoming less suitable with the growth of the number of the objects (N) because of a rapid growth of the number of unitary pair comparisons - (N*(N-1)/2).
Multiple comparison (4.0)
This type of appraisals occupies an intermediate position between appraisals obtained with the pair comparison methods and with ranging in the fact that not the pairs but three, four, ..., (r)-alternatives are consecutively submitted to an expert (where r - the number of submitted ones out of the total number of alternatives - (n), r<n). The expert is ranging them by their significance.
The plural comparisons were studied, for example, in the works [161,163]. An interesting experimental and theoretical investigation of appraisals obtained with the multiple comparison method is contained in the work [94]. The appraisals in the form of the ordered threes are the most known and developed among the appraisals of this kind. The triple comparison method is described in detail in the works [85,103,131-134].
Rangings (5.0)
A regulated set of all the alternatives (objects) submitted for a consideration. The ranging is usually defined in such a way [49].
Within the ranging procedure all the submitted alternatives (objects) are being put in an order in accordance with a decrease (increase) of their preference. In addition, an indication of an equivalence of some next objects is allowed. The number which is given to every object is called its range. Usually, while taking psychological peculiarities of a man, the expert is given not more than 10 alternatives for ranging. The methods of the study of rangings are described in the literature on the multiple comparisons.
Preference vectors (6.0)
Let’s call the vector П, the i-th component of which Пi is defined as a certain number of alternatives exceeded this given one out of the entire set of alternatives A, as the preference vector П = (П1, П2,……Пn) defined for a given fixed number of alternatives (or objects) A = A1 ,A2, …. ,An[49].
In addition, the expert should not obligatorily indicate what alternatives namely are more preferable. And, what is important, every alternative is presented only once. For the first time this type of appraisals was described in the work of B.G. Litvak [49]. The information obtained from an expert using methods of pair comparisons, multiple comparisons and ranging may be presented with the help of the preference vectors.
So, for example, if the values of n-components of a preference vector are different and among them the values 0, 1, ….., n-1 have place then the expert indicated a strict ranging of the alternatives. However, it is possible to give examples of the preference vectors that are not rangings [48].
Appraisals in marks (7.0)
This type of appraisals is an intermediate one between the rangings and quantitative appraisals. In case when an expert assigns a certain value (gradation) to a parameter which characterizes an object we are speaking about marks. When generalizing an experience of using the appraisals in marks form the point of view of the expert procedures we shall consider the following defining features of such appraisals:
¾ there exist some objective criteria of assigning the mark which do not depend on an expert;
¾ every object is being appraised independently on the rest ones in the scale which is not poorer than an ordered one and is not more greatly than an interval one;
¾ the number of gradations is comparatively small.
(While considering the order ratio in a great number of scales we are guided by the work of V.S. Vysotsky [21]).
Although such a definition of marks is convenient for this classification it excludes the cases of appraising in marks in case of an absence of generally accepted standards or when an existence of an objective criterion is questionable. The appraisals in marks are described in the works [58,67,113]. Although the theory of the appraisals in marks is developed poorly they are often used in practice [67,113].
Interval appraisals (8.0)
This type of the expert appraisal is a wide-spread one. As a matter of fact an interval appraisal characterizes not a unique probable situation, but their plurality. One of the defining properties of the interval appraisal is the fact that ternary relation "BETWEEN" is fixed for a multitude.
These definitions, axiomatics, theorems and properties of this relation are described in a number of works [10,71].
During recent years the methods of an interval analysis have been widely spread especially in computations. This is a comparatively modern branch of science, but the literature on the interval analysis has already numbers, according to an opinion of Yu.I. Shokhin, over eight hundred titles [110].
The methods of using the interval analysis in the economics and in solving the management problems are already marked recently [97,98]. Already there is also considered the interval analysis with the use of a probability approach [156].
Point appraisals (9.0)
The methods of processing and analyzing the point expert appraisals are rather developed and well-known to a majority of researchers. The methods of getting quantitative data of the objects are also rather diverse ones.
In a number of spheres, in the prognostication in particular, the point appraisal or appraisals including a point one are sometimes just necessary, and although there are no strict definitions in the literature, an appraisal presented with a single real number is understood under this term.
For example, in cases when we have to make various operations with the expert appraisals, we have to solve the problem on a character of that appraisal – whether it is a point, multipoint or interval one. It depends on the type of the appraisals whether one or another operation, over these appraisals is possible. In order to solve this problem G. Tale requires – in addition to a prognostic expert appraisal - that the expert would also appraised a probability of its realization, and if it is less than 0.5 then this is a point prognosis, otherwise – it is an interval one [86].
For analogous non-prognostic appraisals it is possible to require an appraisal of the certitude degree of the expert in their correctness and subsequently to act according to the same scheme.
Multipoint appraisals (10.0)
The final totality of the point expert appraisals, interconnected as the aggregate whole defines a multipoint appraisal.
A distribution of scanty resources between a finite number of customers, estimations of values of probabilities of a group of events – all those are multipoint appraisals.
Frequently, an interdependence of appraisals manifests itself through a normalization or limitation. In the first run among multipoint appraisals let’s consider two-point ones which should be distinguished from the interval ones.
The two-point appraisal is as follows: a distribution of resources between two alternatives. There are experimental research works which confirm a rather high accuracy of results when using the two-point appraisals even in case of their normalization. In the first turn it concerns the experiments on appraisals of the values of a probability of two mutually excluding events by a man.
As concerns the conditions of a normalization then the experimental research works of Erlik showed that the appraisal of two alternatives satisfies the requirements of the normalization quite well [142]. Among the multipoint appraisals it is necessary to mention the three-point ones. Such kinds of the expert appraisals are often used in the prognostic researches works. The necessity in more informative appraisals than the point or interval one has rapen long ago.
For example,the three-point appraisals are used in some kinds of the network models of the PERT type [82] or the SEER [124], in a number of modifications of the Delphian procedures such as the Delphi II [128].
The three-point appraisal are used in the work of Yu.V. Kiselev [36]where the following assumptions concerning the character of the distribution were discussed: let an unknown quantitative characteristic as an odd value has a continuous unimodal distribution function limited by abscissa. The beta-distribution is usually chosen as a mathematical model of such characteristics. As it is known the probability density of the beta-distribution is expressed with the following formula.
If a<t<b, then f(t) = (t – a)×(b - t)/(b – a) ×B(p, q),
if t = a or t = b, then f(t) = 0,
where B(p, q) is the beta-function, p, q, a, b - parameters of the distribution; in addition, "a" and "b" define correspondingly the left and right bounds of the distribution, p and q > 0.
Because the expert evaluates just three points, but the beta-distribution is a four-parameter value then there is a liberty in the choice of the fourth parameter. If it is admitted, as it is made in the work of Yu.V. Kiselev [36] that a variance is defined just with the square of a swing and is proportionate to it: D = k× (b-a) then, while choosing the value of the coefficient k, we may obtain the value of the mathematical expectation - M, mode - MO and variance - D.
So, in the case when k = 0 we have a delta-distribution (M=MO, D = 0).
When k= 1/12 - a uniform distribution (M = (a+b)/2), D= (b – a) ×1/12).
If k = 1/36 etc. - a distribution used in the PERT-system (M = (a + 4×MO + b)/6; D = (b - a)/36 etc.
In one of the modification of the Delphi method described in the work of J. Martino and called as "multiple dating" the three-point expert appraisals are used in the following way. The experts are asked, when prognosticating the terms of occurring one or another event, to name three dates: "hardly possible", "equally possible or 50 percent one" and "actually authentic". In another version the experts are asked to indicate three dates for which a probability of the fact that a veritable answer would prove to be lower than the declared one and equal correspondingly to 10, 50 and 90 per cent (or to some other correspondingly chosen values of a probability of events which can occur) [55]. A statistical characteristic of a group answer is obtained when taking a median (or other median value) of everyone of three rows of dates.
When comparing appraisals of the "equally probable" dates (a 50-percent probability of a realization) and the dates the probability of occurring of which is considered to be equal 90 per cent it is possible to reveal a very close dependence between them that probably indicates a hidden "psychologic connection" between these two appraisals. In this case the ratio of medians, as it was found in an experimental way in the work [152] is equal to ME(0.9)/(ME(0.5) = 1.8, and a corresponding ratios of quarters are 1.6 for the lower quarters and 2.0 for the upper ones.
When using the multipoint appraisals a normalization is often required as a principle defining their interconnection. For example, in Pospelov’s method of solving matrices the expert should indicate relative weights A1, A2, .…, An of the directions of the research works meeting the normalization condition: = 100 [73].
In the analysis on the problem networks suggested by S.A. Petrovsky the expert has to give the probability estimations for several mutually incompatible events forming a complete group [3,4]. Normalization conditions are figuring here also. Certainly, the normalization represents a certain difficulty, but it may be overcome. The experiments show that the appraisals of a probability for full group alternatives not always equal to one in a sum, if only not to require this rule from the person being under the test especially [119].
According to the opinion of Sheridan and Ferrall there was always possible a certain acceptable modification of probability values ensuring an equality of their sums to one [105]. The experiments accomplished by ourselves jointly with S.A. Petrovsky confirm this fact [68,69].
Functional appraisals (11.0)
The appraisals of this kind represent a certain natural continuation of the multipoint ones. In the case when the expert gives a certain actual function f: X → R as an appraisal we are speaking about a functional appraisal.
Sometimes the realms of a definition of X and even values of the function f(X) Ì R are actually given, and an expert gives an appraisal in the form of a correspondence law.
There are attempts for developing prognoses when an input information is in the form of a probability distribution functional in order the output information should also have a form of a distribution. However such a prognostication is still at the very initial experimental stage. The development of the prognostication with the appraisals of such a kind represents, according to an opinion of E. Yanch: "one of the most important problem in the progress of the prognostic methods" [117].
The following variety of the functional appraisals is of an interest when they are presented in the form of graphs or the so-called "expert" curves. Let's remind that a sub-multitude of the product XxR consisting in the points of the kind (x, f(x)), where xÎX, is called as a graph of a real function f: X→R.
For example, let’s an expert when answering the question portray the function of two independent values in the form of a certain broken line. In the non-published work of E.M. Chetyrkin and E.Z. Demidenko there was suggested an expert procedure according to which the expert is choosing one of three types of a distribution shown in a graphic forms: a) triangular, b) trapeziform, c) uniform.
The curves plottedby the experts were discussed in the main points method even in 1969 [76]. Evidently, the classification of the "expert" curves was proposed for the first time in the work [18]. An obviousness, handy operation, possibility of interpolation, and rather simple methods of a concordance of the group of experts - all this allows to hope for a successful progress of the expert appraisals of this type. The methods of drawing and describing the expert curves are based upon the usage of notions about characteristic elements under which there are understood the peculiarities of the curves sufficient under the conditions of the problem considered in the expertise. The methods of the statistical data processing when drawing the expert curves and for recording all kinds of their transformations are discussed just in small number of works and are developed still poorly [19].
Combinations of appraisals of the first kind
The quality of decisions made upon the basis of the expert appraisals may be raised through the use of a combination of the expert appraisals of the simplest kinds. It is favored with an opportunity of using rather developed mathematical methods. As an example let’s consider some combinations of two different simplest types of the expert appraisals and two similar ones. Among the known combinations of the simplest types of the expert appraisals it is possible to indicate such a type as the grading.
In a case when every one of the objects independently on the other ones relates to one of m-ordered classes we are speaking about the grading. The grading is a particular case of a named classification in which the types of appraisals are numbered with consecutive whole numbers. This type of the expert appraisals is discussed in the work of Yu.N. Tyurin [92] and in a number of other works. The grading consists of the grouping and ranging.
It is possible to consider a combination of two equal simplest types of the expert appraisals. So, the following combination of the expert appraisals is of an interest when, side by side with the ranging according to a certain index of the objects the expert considers the differences of the characteristics of the objects and then ranges just their differences.
The expert appraisals in which, side by side with the ranging, we are considering and ranging the differences of the appraisals of the objects, are discussed in a number of works in connection with the approximately quantitative measurements [58,118]. Subsequently the ranging of the differences the appraisals was called as the ranging of the second order[58].
= 100 [73].