Image enhancement based on fuzzy aggregation techniques

Image Enhancement Based on Fuzzy Aggregation techniques zyxw zyxwvu zyxwvutsr Hamid R. Tizhoosh, Bernd Michaelis (Otto-von-Guericke-University Magdeburg; Faculty of Electrical Engineering Instituiie for Measurement Technology and Electronics D-:39016 Magdeburg, P.O. Box 4120, Germany ti;choosh/michaelis@ ipe.et.uni-magdeburg.de Abstract The human observer, however, do not regard these results as good because his judgment is subjective. This distinction between objectivity and subjectivity is the first great problem in the human-machine-interaction. Another difficulty is the fact Bhat the different people judge the image quality differenlly. This difference is also primarily due to the aforesaid human subjectivity. A good example is judgment of image quality by physicians in radiation therapy [9]. In this paper, we introduce an image enhancement system [I I] that is based on the combination of differently enhanced images. The basic idea is aggregation. The transinformation should reach a maximum (the transinformation is that part of the image information that reaches the observer, and can be perceived by h i d h e r ) . We use fuzzy measure theory, Dempster rule and fuzzy if-then rules to overcome the mentioned difficultieis. Following we describe the enhancement system. zyx zyxwvutsr zyxwvutsrqpo In many image processing applications the image quality should be improved to support the human perception. The image quality evaluation by the human observers is, however, heavily subjective in the nature. Different observers judge the image quality differently. I n many cases the relevant part of image information which is perceived by the observer should reach n maximum. In this work we present a new approach to image enhancement which is based on fusion of different algorithms. We use fuzzy measure theory to represwt the human subjectivir’y, and fuzzy integrals to aggregate this subjectivity with objective criteria. We also apply the Dempster aggregation rule to define a degree of compromise. F i n d y , we use a fuzzy rule-based approach to construct an aggregation matrix that allow us to generate enhanced images for each individual observer. As an example, we apply this approach to increme the quality of portal images that are used in radiation therapy. Introduction Image enhancement plays a fundamental role in many image processing applications where human beings (the experts) make decisions depended on the image information. But some problems arise in the inlerface between the obxrver and the mal:hine. In the image processing, we usually use some objective quality criteria to ascertain the goodness of the results (e.g. the irriage is good if it possesses a low amount of fuzziness [IO]). Proposed system Our enhancement system [ l l ] (see Fig. 1) consists of five phases: image enhanclement by different algorithms (or just one algorithm with different parameters), extraction of objective quality criteria, learning of fuzzy measure (subjective quality evaluation), aggregation (regarding to different images and different observers), and finally, inference (final quality measure for each image). After parameter determination only the algorithms of the inference and preprocessing phase are used for on-line image enhancement. Following, we give a brief description for each system phase. For simplicity, we consider the cast that three differrnt algorithms are applied to enhance the original input image. zyxwvuts zyxwvut 0-7803-5276-9/99/$10.00 0 1999 IEEE 1813 zyxwv zyxwv Fig. 1 Proposed system for improvement of image quality. If all quality criteria can be calculated subjectively, then the phase 2-4 can be carried out off-line. In this case, the inputs for phase 5 (inference) will be recalled from a data base. Phase 1 (enhancement): The first phase of the system is the enhancement of original image. Of course, the enhancement algorithms are selected regarding to the quality criterion (or criteria) which we are interested in. This quality criterion can be contrast, sharpness, noisiness, edginess, homogeneity etc., or even a combination of them. The enhanced images can be generated in two different ways; first way: different algorithms produce differently enhanced images, second way: a single algorithms produces differently enhanced images by changing its parameters. This step can be easily parallelized to accelerate the processing. Phase 2 (extraction): Regarding to the specific requirements of the applications, suitable criteria are extracted. These criteria cun serve as objective quality measures and will be aggregated with subjective measures in the forth phase via fuzzy integral. In this case, phase 24 (Fig. 1) can not be carried out off-line. Here, without loss of generality, we consider the image contrast as an objective criterion, Phase 3 (Learning of fuzzy measure): Fuzzy integration is already successfully applied in some image processing application [3,7]. The Sugeno measure (also called Afuzzy measure [5,6]) is a suitable way for representation of the subjective evaluation of image quality by the human observers. It is due to the fact that the problem is nonadditive in the nature (generally superadditive and sometimes subadditive). The construction of the Sugeno measure, on the other side, is very simple, since the subjective evaluations can be regarded as fuzzy density values G': zyxw zyxwvu zyxwvutsrqp zyxwvutsrqp 1814 The parameter g,(X) = 1: A can be computed from the condition " A + 1 = J-J(l+Ag'). i=l Phase 4 (Aggregation of criteridjudgments): The aggregation phase consists of two parts. In the first part we calculate the degree of compatibility y betweer objective criterion (in our case contrast values C*(A,)) and subjective measure (g,({ Ai}) with fuzzy integral [5,6]: zyxw In the second part, the fuzzy densities are normalized anc interpreted as basic probabilities so that the degree o compromise m1,2,....M among all M observers (experts) ca1 be recursively calculated with the Dempster rule o aggregation [ 1,4] : zyxwvutsrq zyxwvut zyxwvut zyxwvutsr zyxwvuts zyxwvutsrq m * ( A i ) = (m, @ m 2 ) ( A i ) (4) r The aggregation phase generates also two vectors = (yl ... y M )and CD = (m*(Al) mw(A2) ... m*(AN)).These vectors will be used as inputs for the inference phase. Phase 5 (Inference): The elemenis of vectors r' (degree of compatibi1it:y) and @ (degree of compromise) are fuzzified with three symmetric membership functions. The output of the inference system is a aggregation matrix (Fig. 2) quantitying the image quality and is represented by five non-symmetric membershiF1 functions. The if-then rules are formuhted heuristically as, listed in Table 1. One can also use simple fuzzy connectives (e.g. minimum or algebraic product) to aggregate the degrees of compatibility and cornpromise. Since the Dempster rule y2 T aggregate basic probabilities that have to be absolutely reliable, and since we know that the normalized fuzzy densities (scores of the observers) are not exact values, the use of an inference system is the appropriate way to overcome the inherent uncertainties in this case. The most simple way to generate images using the aggregation matrix is to build a convex Combination of individual results. Tlhis simple kind of final aggregation is selected because the contrast ,as interesting quality criterion allows such additive fusion of individual results. For more complex quality criteria, however, we need other approaches to carry lout the final aggregation (e.g. if we consider the edginess, or sharpness as quality criteria, the additive fusion can not be used!). on matrix. VERY LOW t-%-p+-q R5 I MEDIUM MEDIUM MEDIUM MEDIUM HIGH R9 IHIGH IHIGH VERY HIGH algorithms degree of compatibility observers (experts) zyxwvu degree of compromise Fig. 2.Aggregation matrix generated by if-then rules. Implementation and results The method is presented with a very simple example which gives transparent algorithms for the five Fhases. The proposed enhancement system was implemented in [email protected] images [9] were selected as input. These of images are used in radiation therapy for 1815 patient position during the treatment. Portal images are on-line images and have a poor quality (poor contrast, IOW reSOhtion and noi!;e corrupted) due to the physics of imaging devices. The image contrast was determined as quality criterion. Three fuzzy algorithms (fuzzy hYPerbolization (A), Tule-based enhancement (B) and minimization of fuzziness (C) [8,9]) were used in the first zy zy zyxwvuts zyxwvu zy phase to enhance the image contrast. The different results of these algorithms were judged by seven observers (Table 2). The parameter 1 (equations (1) and (2)) is closed to -1 because the observers' scores for algorithms A,B and C are very high (the subjective judgment here is clearly superadditive). The local contrast values were Table calculated in lox 10 neighborhoods and the most typical value was selected as global contrast value. In Fig. 3 , the aggregated image is produced for the first observer in Table 2. 2. Subjective image quality evaluation by different observers and constructed 1-fuzzy measure. (c) zyxwvutsr zyxwvu (4 Fig. 3. Example for the proposed enhancement system. Portal images a) and c) are enhanced by our overall system resulting in images b) and d), respectively. 1816 Conclusion References We developed ail enhancement system based on fuzzy measure theory iind fuzzy set theory, respectively. To integrate the human subjective evaluation within the enhancement procedure, we used !hgeno measure and Sugeno integral to define the degree of compa'tibility between objective and subjective criteria. Further, we applied the Dempster aggregation rule to define a degree of compromise among all human olbservers. Finally, we constructed a rule-based system to aggregate the both degrees. The result of our approach is an aggregation matrix that allow us to generate enhanced image for each individual observer or, as a compromise, for all observers .The final aggregation depends on the selected quality criteria. In some situations, we can not build a convex combination as the output (e.g. an additive fusion is not appropriate, if we are considering the sharpness of some image details). This aspect should be investigated in our future works. The learning of fuzzy densities is interactive in our case because we are interested in integration of human subjectivity (specially for medical applications). But, one can also use any automatic approaches for this purpose to incre,ase the image quality by fusion of different result. One of the disadvantages of measure theory is the computational complexity if the number of elements is large. Since the elements in our case are different algorithms (or different parameter sets of the same algorithm), this drawback pliiy~ no role in our system (it is not meaningful to aggregate 10 different algorithms !). The proposed system is therefore just a prototype that should be refined and extended. To achieve a higher level of image quality considering the subjective perception and opinion of the human observers, we have to overcome many difficulties. !Suitable tests, for instance, should help us to map this subjectivity into a numerical framework. Here, we need more knowledge about the psychological background of human image perception and understanding. B1:side these difficulties, the -fuzzy aggregation techniques seem to be i i powerful tool for representation and processing of the human subjectivity within image processing systems. Also fuzzy if-then rules are a sophisticated bridge between human knowledge on the one side and the numerical framework of the computers on the other side. In our next investigations, we are going to extend our system to the case that more complex image features such as edginess and homogeneity are selected. Further, we will refine the extraction and learning phase regarding to the psychological facts about the mechanisms of human perception and subjectivity. zyxw [I] Dempster, A. P. (1967): Upper and lower probabilities induced by a multivalued mapping. In: Ann. Math. Statistics, vol. 38, pp. 325-339 [2] Kandel, A., Friedman, M., Schneider, M. (1989): The use of weighted fuzzy expelcted value (WFEV) in fuzzy expert systems. Fuzzy Sets and Systems 31, 37-45 [3] Keller, J. M., Gader, P., Tahani, H., Chiang, J. H., Mohamed, M. (1994): Advances in fuzzy integration for pattern recognition. Fuizzy Sets and Systems 65, 273-283 [4] Shafer, G. (1976): A Mathematical Theory of Evidence. Princeton University Press, Princeton, New Jersey [5] Sugeno, M. (1974): Theory o f Fuzzy Integrals and Its Applications. Dissertation, Tokyo Institute of Technology, Japan [6] Sugeno, M. (1977): Fuzzy measures and fuzzy integrals: a survey. In: Fuzzy Automata and Decision Processes, NorthHolland, Amsterdam, S . 89-102 [7] Tahani, H., Keller, J. C. (1992): The fusion of information via fuzzy integration. ]In: Proc. NAFIPS'92, Puertu Vallarta, Mexico, S. 468-477 [XI Tizhoosh, H. R., Krell, G., Michaelis, B. (1997a): Locally adaptive fuzzy image enhancement. In: Reusch, B. (Ed.), Computational Intelligence, Springer-Verlag, Berlin, S. 272-276 [9] Tizhoosh, H. R., Krell, G.. Michaelis, B. (1997b): On fuzzy image enhancement of megavoltage images in radiation therapy. In: Proc. 6'h IEEE International Conference on Fuzzy Systems, Barcelona, Spanien, Band 3, S. 1399-1404 [lo] Tizhoosh, H. R. (11997): Fuzzy Image Processing (in German), Springer, Heidelberg, Germany [ l l ] Tizhoosh, H.R., Michaelis, B. (1998): Improvement of Image Quality Based Ion Subjective Evaluation and Fuzzy Aggregation techniques. EUFIT98, vol. 2 , 1998, pp. 13251329 zyx zyxwv zyxwvutsrqp zyxwvuts 1817