Double-Granule Conditional-Entropies Based on Three-Level Granular Structures

Abstract

Rough set theory is an important approach for data mining, and it refers to Shannon’s information measures for uncertainty measurements. The existing local conditional-entropies have both the second-order feature and application limitation. By improvements of hierarchical granulation, this paper establishes double-granule conditional-entropies based on three-level granular structures (i.e., micro-bottom, meso-middle, macro-top), and then investigates the relevant properties. In terms of the decision table and its decision classification, double-granule conditional-entropies are proposed at micro-bottom by the dual condition-granule system. By virtue of successive granular summation integrations, they hierarchically evolve to meso-middle and macro-top, to respectively have part and complete condition-granulations. Then, the new measures acquire their number distribution, calculation algorithm, three bounds, and granulation non-monotonicity at three corresponding levels. Finally, the hierarchical constructions and achieved properties are effectively verified by decision table examples and data set experiments. Double-granule conditional-entropies carry the second-order characteristic and hierarchical granulation to deepen both the classical entropy system and local conditional-entropies, and thus they become novel uncertainty measures for information processing and knowledge reasoning.

1. Introduction

Rough set theory can effectively implement data mining for the imprecise, inconsistent, and incomplete information [1], and it has been extensively applied in artificial intelligence and machine learning [2,3,4,5,6,7,8]. In rough set theory, attribute reduction based on decision tables is a main topic for approximate reasoning and knowledge discovery, and there are three main construction strategies: from the positive region, information measure, and a discernibility matrix  [9,10,11,12,13,14,15]. By virtue of the discernibility matrix, Wei et al. [16] proposed an incremental reduction algorithm for dynamic data; Ma et al. [17] utilized the compressed binary discernibility matrix to construct an incremental reduction algorithm for group dynamic data; moreover, Nie and Zhou [18] proposed a new discernibility matrix defined by local conditional-entropies to compute the reduction core.

Information theory originated from Shannon’s entropy system [19], and it provides an effective method for uncertainty measurement, such as in attribute reduction. Currently, information theory has been introduced into rough set theory for uncertainty analyses and information processing [20,21,22,23,24,25]. As far as attribute reduction is concerned, Miao [26] offered the informational representation of knowledge reduction and decision reduction, where entropy and mutual-information are highlighted; Wang et al. [27] conducted a comparative study on attribute reduction from the algebra and information viewpoints, where the conditional-entropy acts as a main tool; Jiang et al. [28] presented the relative decision entropy to propose a feature selection algorithm; Slezak [29] used the conditional-entropy to define approximate reducts; moreover, Qian and Shu [30] provided the mutual information criterion to evaluate candidate features in incomplete data. In general, the entropy, conditional-entropy, and mutual-information together constitute the classical information system with integrality and comprehensiveness, and they can function on rough set applications (such as attribute reduction) but may exhibit different emphases in different application scenarios. In addition, information-theoretic measures have multiple variational forms [31,32,33,34,35]. As far as conditional-entropies are concerned, they are extensively applied in rough set theory from multiple pointcuts [26,27,29,31,34,36,37,38,39], while uncertainty measurement and reduction construction still serve as two basic issues. Aiming at probabilistic rough sets, Deng and Yao [40,41] used Shannon’s entropy and conditional-entropy to interpret and determine probabilistic thresholds by an information-theoretic approach, and Ma et al. [42] considered variants of conditional-entropies to construct heuristic reduction algorithms for the probabilistic model. In particular, local conditional-entropies are put forward by adopting double condition-granules and their union locality [18], and they can distinctively determine a new discernibility matrix for reduction core computation; moreover, the information measures exhibit a novel feature of second-order expressions, especially when compared to the traditional entropy system with only single-granule descriptions [19,26,27].

Granular computing is a structural methodology of hierarchical computing and information processing [43,44], and its technology of multi-granularity and multiple levels is useful for uncertainty analyses and knowledge acquisition regarding data. In rough set theory, the information granulation is of extensive concern [45,46,47,48,49], and the granulation monotonicity plays an important role in attribute reduction [12,50,51,52]. In particular, a decision table acts as a formal background of data mining [12,53,54,55], and it involves condition/decision granules and classifications from granular structures. According to granular computing, Zhang and Miao [56] introduced three-layer granular structures of decision tables, and they further hierarchically constructed three-way informational measures based on weighted-entropies; moreover, Wang et al. [57] utilized three-layer granular structures to research three-way weighted combination-entropies. These studies adhere to three-level analyses, and the latter are directly related to granular computing [43] and three-way decisions [58], as well as their interplay. Recently, Yao [59] discussed three-way granular computing by making use of two particular types of three granules and three levels, where thinking in three levels results in an important model. Additionally, three-level analyses were extensively utilized in the location allocation and programming/optimization modeling [60,61,62].

According to [18], the new discernibility matrix is used for reduction core calculations, and its creative implementation mainly depends on local conditional-entropies. Therefore, local conditional-entropies focus on the granule-union locality rather than their underlying double-granule interaction, and the latter more essentially adheres to the second-order characteristic; moreover, they lack the condition granulation to restrict their uncertainty measurement function and information procession prospect based on knowledge. Motivated by the two issues, this paper utilizes the two-granular essence and three-hierarchical evolution to propose double-granule conditional-entropies based on three-level granular structures. Regarding the contribution, this novel type of information measures improves local conditional-entropies from both the granular interaction and hierarchical/conditional granulation, and they will achieve multiple important properties (including the integration hierarchy, number distribution, calculation algorithm, three bounds, and granulation non-monotonicity) to offer both robust measurement functions and knowledge-application prospects. Moreover, three-level granular structures here (including micro-bottom, meso-middle, macro-top) adopt only the condition part of decision table, and thus they differ from and push forward the previous ones, which include both the condition and decision parts [56].

The remainder of this paper is organized as follows. Section 2 reviews the decision table and local conditional-entropies; Section 3 proposes and studies double-granule conditional-entropies from three-level granular structures; Section 4 provides a decision table example for mechanism illustration; Section 5 makes data experiments for effectiveness verification; finally, Section 6 concludes this paper.

2. Decision Table and Its Existing Entropy Measures

Rough set theory [1] focuses on the data that are represented in an information table

$$(U,AT,\{V_a:a\in AT\},\{I_a:a\in AT\});$$

U is the universe with finite objects, $AT$ is the finite attribute set, $V_a$ is the value domain for $a\in AT$, and $I_a:U\to V_a$ is an information function to endow each object x with a value $I_a\left(x\right)=a\left(x\right)$ on attribute a. The decision table is a special type of information table with $AT=C\cup D$ and $C\cap D=\varnothing$, where C and D denote the sets of condition attribute and decision attribute, respectively, and it is simply denoted by $(U,C\cup D)$ in this paper. Furthermore, the granulation construction usually considers two parts.

(1)

The condition attribute subset $A\subseteq C$ induces an equivalence relation

$$IND\left(A\right)=\left\{\right(x,y)\in U\times U:\forall a\in A,a(x)=a(y\left)\right\},$$

and the latter provides the condition granulation or partition $U/IND\left(A\right)=\{A_i:i=1,\dots ,n\}$, where $A_i={\left[x\right]}_A^i$ represents the equivalence granule to exhibit number $|U/IND(A\left)\right|=n$.

(2)

Similarly, the decision attribute set D induces the equivalence relation $IND\left(D\right)$ and further decision classification $U/IND\left(D\right)=\{D_j:j=1,\dots ,m\}$, which consists of $|U/IND(D\left)\right|=m$ decision classes.

The decision table $(U,C\cup D)$ and its granulation from $A\subseteq C$ and D constitute the basic background for information measure construction. The probability space $(U,2^U,P)$ establishes the usual probability framework, where

$$\begin{array}{c}\hfill P:2^U\to \mathbf{Q},P\left(X\right)=\frac{\left|X\right|}{\left|U\right|},\forall X\subseteq U,\end{array}$$

(1)

and thus two usual probabilities are

$$\begin{array}{c}\hfill P\left(A_i\right)=\frac{|A_i|}{\left|U\right|},P(D_j/A_i)=\frac{|A_i\cap D_j|}{|A_i|}.\end{array}$$

(2)

Definition 1

([26,27,56]). The entropy on condition A, conditional-entropy on D given A, and mutual-information between A and D are respectively defined by

$$\begin{array}{c}\hfill \begin{array}{cc}\hfill H\left(A\right)& =-\sum _{i=1}^{n}P\left(A_i\right){\log }_{2}P\left(A_i\right),\hfill \\ \hfill H(D/A)& =-\sum _{i=1}^n\left(P\left(A_i\right)\sum _{j=1}^{m}P(D_j/A_i){\log }_{2}P(D_j/A_i)\right),\hfill \\ \hfill I(A;D)& =H\left(D\right)-H(D/A),\hfill \end{array}\end{array}$$

(3)

where

$$H\left(D\right)=-\sum _{j=1}^{m}P\left(D_j\right){\log }_{2}P\left(D_j\right).$$

Theorem 1

([26,27,56]). The entropy, conditional-entropy, and mutual-information have granulation monotonicity. Concretely,

$$\begin{array}{c}\hfill U/IND\left(A\right)⪰U/IND\left(B\right)⟹H\left(B\right)\ge H\left(A\right),H(D/B)\le H(D/A),I(B;D)\ge I(A;D).\end{array}$$

(4)

In terms of the decision table $(U,C\cup D)$, the classical system of Shannon entropies has been introduced into rough set theory, as shown by Definition 1 and Theorem 1. As three basic information measures, the entropy, conditional-entropy, and mutual-information have uncertainty semantics and granulation monotonicity, so they are extensively used in attribute reduction and heuristic algorithms [26,27,42]. The granulation relation $U/IND\left(A\right)⪰U/IND\left(B\right)$ is equivalent to $IND\left(A\right)\supseteq IND\left(B\right)$, that is,

$$\forall B_{i^{*}}\in U/IND\left(B\right),\exists A_i\in U/IND\left(A\right),s.t.,B_{i^{*}}\subseteq A_i,$$

and it is usually induced by $A\subseteq B\subseteq C$; furthermore, relevant granulation monotonicity/non-monotonicity becomes an important index to assess and apply uncertainty measures.

According to the decision table and its formal structure, Zhang and Miao [56] recently introduced three-level granular structures, i.e.,

$$\mathrm{micro-bottom} (A_i,D_j), \mathrm{meso-middle} (U/IND\left(A\right),D_j), \mathrm{macro-top} (U/IND(A),U/IND(D\left)\right),$$

and further investigated weighted-entropy constructions. As a result, the previous entropy system (Equation (3)) is actually located at macro-top and has an equivalent construction from the weighted-entropy system; at meso-middle, Zhang et al. [10] established three-way informational class-specific reducts to be compared with the algebraic class-specific reducts [9].

In particular, Nie and Zhou [18] proposed a new discernibility matrix for computing the reduction core, and they tactfully utilized a kind of novel information of so-called local conditional-entropy. As our preliminary, the relevant entropy and matrix are reviewed as follows, where let $U/IND\left(C\right)=\{C_k:k=1,..,r\}$ and the cardinality form is mainly adopted.

Definition 2

([18]). The local conditional-entropy on decision table $(U,C\cup D)$ is defined by:

$$\begin{array}{c}\hfill \begin{array}{cc}\hfill \forall C_p,C_q\in U/IND\left(C\right)& (1\le p,q\le r),\hfill \\ \hfill H_{C_p\cup C_q}\left(D/C\right)=& -\frac{|C_p|}{|C_p\cup C_q|}\sum _{j=1}^m\frac{|C_p\cap D_j|}{|C_p|}{\log }_2\frac{|C_p\cap D_j|}{|C_p|}\hfill \\ & -\frac{|C_q|}{|C_p\cup C_q|}\sum _{j=1}^m\frac{|C_q\cap D_j|}{|C_q|}{\log }_2\frac{|C_q\cap D_j|}{|C_q|}.\hfill \end{array}\end{array}$$

(5)

Definition 3

([18]). The discernibility matrix $DM={\left(r_{i^{\prime }{j}^{\prime }}\right)}_{\left|U\right|\times \left|U\right|}$ on decision table $(U,C\cup D)$ is defined by:

$$r_{i^{\prime }{j}^{\prime }}=\left\{\begin{array}{cc}\hfill c\in C,& ifmin\left(\right|d{x}_{i^{\prime }}|,|d{x}_{j^{\prime }}\left|\right)=1,c\left(x_{i^{\prime }}\right)\ne c\left(x_{j^{\prime }}\right),D\left(x_{i^{\prime }}\right)\ne D\left(x_{j^{\prime }}\right),\hfill \\ \hfill c\in C,& ifmin\left(\right|d{x}_{i^{\prime }}|,|d{x}_{j^{\prime }}\left|\right)>1,c\left(x_{i^{\prime }}\right)\ne c\left(x_{j^{\prime }}\right),D\left(x_{i^{\prime }}\right)\ne D\left(x_{j^{\prime }}\right),\hfill \\ & H_{{\left[x_{i^{\prime }}\right]}_C\cup {\left[x_{j^{\prime }}\right]}_C}(D/(C-\left\{c\right\}))>H_{{\left[x_{i^{\prime }}\right]}_C\cup {\left[x_{j^{\prime }}\right]}_C}(D/C),\hfill \\ \hfill \varnothing ,& otherwise\hfill \end{array}\right.$$

(6)

where $dx=\{D\left(y\right):y\in {\left[x\right]}_C\}$ ($\forall x,y\in U$) represents the set of decision values induced by conditional class ${\left[x\right]}_C$ while $\left|dx\right|$ means the corresponding cardinality [63]. In Equation (6), let ${\left[x_{i^{\prime }}\right]}_C=C_p$, ${\left[x_{j^{\prime }}\right]}_C=C_q$, and then

$$H_{{\left[x_{i^{\prime }}\right]}_C\cup {\left[x_{j^{\prime }}\right]}_C}(D/(C-\left\{c\right\}))=H_{C_p\cup C_q}(D/(C-\left\{c\right\}))=-\sum _{j=1}^m\frac{|(C_p\cup C_q)\cap D_j|}{|C_p\cup C_q|}{\log }_2\frac{|(C_p\cup C_q)\cap D_j|}{|C_p\cup C_q|}$$

(7)

is determined to represent the conditional-entropy of local decision table when accompanied by new universe $C_p\cup C_q$ after deleting attribute c; moreover, $H_{{\left[x_{i^{\prime }}\right]}_C\cup {\left[x_{j^{\prime }}\right]}_C}(D/C)=H_{C_p\cup C_q}\left(D/C\right)$ is clear according to Equation (5).

3. Double-Granule Conditional-Entropies Based on Three-Level Granular Structures

The local conditional-entropy in Equation (5) implements effective uncertainty descriptions to guide the in-depth discernibility matrix and core calculation [18], thus exhibiting fundamental significance. However, this basic measure has three flawed aspects, and corresponding improvements for general applications.

(1)

According to Equation (5), the locality mainly refers to less range $C_p\cup C_q$ in universe U. More essentially, we can stand on the dual granules $C_p$ and $C_q$ to propose a novel notion of double-granule conditional-entropies, and it differs from the usual entropy system with only the single-granule representation which implies a kind of first-order style. Moreover, the measure properties are lacking in [18], and we will provide in-depth properties such as restriction bounds and granulation non-monotonicity.

(2)

Regarding granular structures, all decision classes $D_j$ ($j=1,\cdots ,m$) (or decision classification $U/IND\left(D\right)$) are considered, but condition granules involve only two factors $C_p$ and $C_q$. A condition partition $U/IND\left(C\right)$) needs considering in practice to provide a system description of knowledge granulation, so we also focus on granulation $U/IND\left(C\right)$ to introduce three-level granular structures for hierarchical constructions of double-granule conditional-entropies.

(3)

Finally, the initial concept is limited to only C for expressing the discernibility matrix and reduction core, and a general subset $A\subseteq C$ has better theoretical and practical prospects, especially for the knowledge-based applications (such as attribute reduction or feature selection).

Along the above thoughts, this section mainly establishes double-granule conditional-entropies based on a universal attribute-subset $A\subseteq C$ and investigates relevant algorithms and properties, and we particularly use a kind of three-level granular structures.

From a viewpoint of only condition granulation, basic descriptions of three-level granular structures are provided in

Table 1. Three-level granular structures based on condition granulation of the decision table.

Structure Naming	Composition System	Granular Scale	Granular Level	Number of Parallel Patterns
Micro-Bottom	$(A_p,A_q)$	Micro	Bottom	$n\times n$
Meso-Middle	$(A_p,U/IND\left(A\right))$ $=(A_p,\{A_q:q=1,\cdots ,n\})$	Meso	Middle	n
Macro-Top	$(U/IND(A),U/IND(A\left)\right)$ $=(\{A_p:p=1,\cdots ,n\},\{A_q:q=1,\cdots ,n\})$	Macro	Top	1

, and relevant concepts are usually intuitionistic and descriptive according to a supporting figure with granular structures: Figure 1. Micro-bottom $(A_p,A_q)$ focuses on only two granules, meso-middle

$$(A_p,U/IND\left(A\right)=\{A_q:q=1,\cdots ,n\})$$

consists of one granule and a partition, while macro-top

$$(U/IND\left(A\right)=\{A_p:p=1,\cdots ,n\},U/IND\left(A\right)=\{A_q:q=1,\cdots ,n\})$$

considers the same partition with different construction origins. The three-level granular structures carry a kind of hierarchical integration (or decomposition) relationship, and they provide $n\times n$, n, and one parallel patterns, respectively; they will be presented in a table form with the $n\times n$ mainbody data as well as the edge statistics. Moreover, they differ from the existing three-level granular structures for decision tables, which consider not only the condition granulation (with $A_i$ and $U/IND\left(A\right)$) but also decision granulation (with $D_j$ and $U/IND\left(D\right)$) [56].

3.1. Double-Granule Conditional-Entropy at Micro-Bottom

The local conditional-entropies are actually at only micro-bottom, i.e., $(C_p,C_q)$ regarding C. As a basis of hierarchical development, this subsection improves local conditional-entropies to construct double-granule conditional-entropies at micro-bottom $(A_p,A_q)$ ($p,q\in \{1,\cdots ,n\}$), which comes from an arbitrary condition-attribute subset $A\subseteq C$. We first suppose weight coefficients

$$\begin{array}{c}\hfill \begin{array}{c}\hfill {\omega }_p=\frac{|A_p|}{|A_p|+|A_q|},{\omega }_q=\frac{|A_q|}{|A_p|+|A_q|},\end{array}\end{array}$$

(8)

where

$${\omega }_p+{\omega }_q=1.$$

Definition 4.

At micro-bottom $(A_p,A_q)$, the double-granule conditional-entropy is defined by

$$\begin{array}{cc}\hfill H_{(A_p,A_q)}\left(D/A\right)& =-{\omega }_p\sum _{j=1}^{m}P(D_j/A_p){\log }_{2}P(D_j/A_p)-{\omega }_q\sum _{j=1}^{m}P(D_j/A_q){\log }_{2}P(D_j/A_q)\hfill \\ & =-\frac{|A_p|}{|A_p|+|A_q|}\sum _{j=1}^m\frac{|A_p\cap D_j|}{|A_p|}{\log }_2\frac{|A_p\cap D_j|}{|A_p|}-\frac{|A_q|}{|A_p|+|A_q|}\sum _{j=1}^m\frac{|A_q\cap D_j|}{|A_q|}{\log }_2\frac{|A_q\cap D_j|}{|A_q|}.\hfill \end{array}$$

(9)

Proposition 1.

The double-granule conditional-entropy based on $A_p$ becomes

$$\begin{array}{c}\hfill \begin{array}{cc}\hfill H_{(A_p,A_p)}\left(D/A\right)& =-\sum _{j=1}^{m}P(D_j/A_p){\log }_{2}P(D_j/A_p)\hfill \\ & =-\sum _{j=1}^m\frac{|A_p\cap D_j|}{|A_p|}{\log }_2\frac{|A_p\cap D_j|}{|A_p|}.\hfill \end{array}\end{array}$$

(10)

By using probabilistic and cardinal forms, Definition 4 proposes the double-granule conditional-entropy at micro-bottom. In contrast to the local conditional-entropy in [18], our measure generally adopts the same essence but a different viewpoint. In other words, Equation (9) with forms $(A_p,A_p)$ and $|A_q|+|A_p|$ is equivalent to Equation (5) with styles $A_p\cup A_p$ and $|A_q\cup A_p|$ when

$$A_q\ne A_p⟹|A_q|+|A_p|=|A_q\cup A_p|,$$

but the former becomes different and coherent when

$$A_q=A_p⟹|A_q|+|A_p|=2|A_q\cup A_p|>|A_q\cup A_p|;$$

moreover, it more tends to the double-granule description rather than the granule-union locality. In Equation (9), conditional-information measures

$$-\sum _{j=1}^{m}P(D_j/A_p){\log }_{2}P(D_j/A_p),-\sum _{j=1}^{m}P(D_j/A_q){\log }_{2}P(D_j/A_q)$$

represent the uncertainty of decision classification $U/IND\left(D\right)$ regarding condition granules $A_p$ and $A_q$, respectively, and they are integrated into $H_{(A_p,A_q)}\left(D/A\right)$ by two complementary weight coefficients ${\omega }_p$ and ${\omega }_q$. As a result, $H_{(A_p,A_q)}\left(D/A\right)$ embodies a kind of information fusion of double-granule $A_p$, $A_q$ to describe decision classification $U/IND\left(D\right)$ and its uncertainty, from the perspective of conditional information. Therefore, $H_{(A_p,A_q)}\left(D/A\right)$ is naturally called the double-granule conditional-entropy, and it is actually located at micro-bottom $(A_p,A_q)$. In particular, the double-granule measures utilize the double-granule fusion to capture a new feature of second-order, because main entropy systems (such as those in Equation (3)) utilize only the single-granule description which correspondingly refers to the so-called first-order information. Proposition 1 focuses on a specific case of $A_q=A_p$, and the concrete result $H_{(A_p,A_p)}\left(D/A\right)$ degenerates into a one-order measure regarding conditional-entropy.

Proposition 2.

At micro-bottom, double-granule conditional-entropies offer $n\times n$ values, i.e.,

$$H_{(A_p,A_q)}\left(D/A\right)(p,q=1,\cdots ,n).$$

Since both $A_p$ and $A_q$ have n granules based on $p=1,\cdots ,n$ and $q=1,\cdots ,n$, $H_{(A_p,A_q)}\left(D/A\right)$ offers number $n\times n$ (Proposition 2) to correspond to $n\times n$ micro-bottoms. The $n\times n$ kinds of double-granule conditional-entropies are arranged in

Table 2. Matrix distribution of double-granule conditional-entropies at micro-bottom.

$U/IND\left(A\right)$	$A_1$	⋯	$A_q$	⋯	$A_n$
$A_1$	$H_{(A_1,A_1)}(D/A)$	⋯	$H_{(A_1,A_q)}(D/A)$	⋯	$H_{(A_1,A_n)}(D/A)$
⋮	⋮	⋱	⋮	⋱	⋮
$A_p$	$H_{(A_p,A_1)}(D/A)$	⋯	$H_{(A_p,A_q)}(D/A)$	⋯	$H_{(A_p,A_n)}(D/A)$
⋮	⋮	⋱	⋮	⋱	⋮
$A_n$	$H_{(A_n,A_1)}(D/A)$	⋯	$H_{(A_n,A_q)}(D/A)$	⋯	$H_{(A_n,A_n)}(D/A)$

, and the mainbody refers to an $n\times n$ square symmetric matrix where

$$H_{(A_p,A_q)}(D/A)=H_{(A_q,A_p)}\left(D/A\right).$$

Based on Equation (9), Algorithm 1 resorts to a “for” loop to effectively offer a double-granule conditional-entropy $H_{(A_p,A_q)}\left(D/A\right)$ for two arbitrary granules $A_p,A_q\in U/IND\left(A\right)$. Furthermore, we can achieve all $n\times n$ entropies values by adding two “for” loops regarding $p=1,\cdots ,n$ and $q=1,\cdots ,n$.

Algorithm 1: Calculation of double-granule conditional-entropy at micro-bottom

Input: Decision table $(U,C\cup D)$, target subset $A\subseteq C$, and two special indexes   $p,q\in \{1,\cdots ,n\}$; Output: Double-granule conditional-entropy $H_{(A_p,A_q)}\left(D/A\right)$ at micro-bottom $(A_p,A_q)$. 1: Compute $U/IND\left(A\right)$ to obtain two concrete granules $A_p,A_q\in U/IND\left(A\right)$, and determine ${\omega }_p,{\omega }_q$. 2: Compute $U/IND\left(D\right)$ to obtain all decision classes $D_j$ ($j=1,\cdots ,m$). 3: Let $H_p=0$, $H_q=0$. 4: for$j\in \{1,..,m\}$do 5:   $H_p\leftarrow H_p-P(D_j/A_p){\log }_{2}P(D_j/A_p)$,   $H_q\leftarrow H_q-P(D_j/A_q){\log }_{2}P(D_j/A_q)$. 6: end for 7: Obtain $H_{(A_p,A_q)}\left(D/A\right)={\omega }_p{H}_p+{\omega }_q{H}_q$. 8: return$H_{(A_p,A_q)}\left(D/A\right)$.

Theorem 2.

At micro-bottom, the double-granule conditional-entropy has lower and upper bounds. Concretely,

$${\underline{H}}_{(A_p,A_q)}\left(D/A\right)\le H_{(A_p,A_q)}\left(D/A\right)\le {\overline{H}}_{(A_p,A_q)}\left(D/A\right),$$

where

$$\begin{array}{c}\hfill \begin{array}{cc}\hfill {\underline{H}}_{(A_p,A_q)}\left(D/A\right)& =-\frac{|A_p|}{2\left|U\right|}\sum _{j=1}^{m}P(D_j/A_p){\log }_{2}P(D_j/A_p)-\frac{|A_q|}{2\left|U\right|}\sum _{j=1}^{m}P(D_j/A_q){\log }_{2}P(D_j/A_q),\hfill \\ \hfill {\overline{H}}_{(A_p,A_q)}\left(D/A\right)& =-\sum _{j=1}^{m}P(D_j/A_p){\log }_{2}P(D_j/A_p)-\sum _{j=1}^{m}P(D_j/A_q){\log }_{2}P(D_j/A_q).\hfill \end{array}\end{array}$$

(11)

Proof. 

$|A_p|,|A_q|\in [1,\left|U\right|)$ implies

$$\begin{array}{c}\hfill \begin{array}{c}\hfill {\omega }_p=\frac{|A_p|}{|A_p|+|A_q|}\in \left[\frac{|A_p|}{2\left|U\right|},1\right),\\ \hfill {\omega }_q=\frac{|A_q|}{|A_p|+|A_q|}\in \left[\frac{|A_q|}{2\left|U\right|},1\right),\end{array}\end{array}$$

(12)

so $H_{(A_p,A_q)}\left(D/A\right)\in [{\underline{H}}_{(A_p,A_q)}\left(D/A\right),{\overline{H}}_{(A_p,A_q)}\left(D/A\right)]$. □

In Theorem 2, the double bounds of $H_{(A_p,A_q)}\left(D/A\right)$ are acquired by the enlarging and reducing of weight coefficients. Regarding Equation (12),

$$\begin{array}{c}\hfill \begin{array}{c}\hfill \left(A_q\ne A_p\right)\bigvee \left(A_q=A_p\wedge |A_q|=|A_p|\le \frac{\left|U\right|}{2}\right)⟹{\omega }_p\ge \frac{|A_p|}{\left|U\right|}>\frac{|A_p|}{2\left|U\right|},{\omega }_q\ge \frac{|A_q|}{\left|U\right|}>\frac{|A_q|}{2\left|U\right|};\end{array}\end{array}$$

(13)

on the other hand,

$$\begin{array}{c}\hfill \begin{array}{c}\hfill \left(A_q=A_p\right)\bigwedge \left(|A_q|=|A_p|>\frac{\left|U\right|}{2}\right)⟹{\omega }_p=\frac{1}{2}\in \left[\frac{|A_p|}{2\left|U\right|},\frac{|A_p|}{\left|U\right|}\right),{\omega }_q=\frac{1}{2}\in \left[\frac{|A_q|}{2\left|U\right|},\frac{|A_q|}{\left|U\right|}\right).\end{array}\end{array}$$

(14)

In other words, ${\omega }_p$ and ${\omega }_q$ have theoretical lower bounds $\frac{|A_p|}{2\left|U\right|}$ and $\frac{|A_q|}{2\left|U\right|}$, respectively, but they usually have closer lower bounds $\frac{|A_p|}{\left|U\right|}$ and $\frac{|A_q|}{\left|U\right|}$, respectively. Therefore, $H_{(A_p,A_q)}\left(D/A\right)$ can theoretically achieve ${\underline{H}}_{(A_p,A_q)}\left(D/A\right)$, such as in the case

$$A_p=A_q=U⟹\left({\omega }_p=\frac{1}{2}=\frac{|A_p|}{2\left|U\right|}\right)\bigwedge \left({\omega }_q=\frac{1}{2}=\frac{|A_q|}{2\left|U\right|}\right);$$

usually, it may be practically restricted by a better measure:

$$\begin{array}{c}\hfill {\underline{H}}_{(A_p,A_q)}^{\prime }\left(D/A\right)=-\frac{|A_p|}{\left|U\right|}\sum _{j=1}^{m}P(D_j/A_p){\log }_{2}P(D_j/A_p)-\frac{|A_q|}{\left|U\right|}\sum _{j=1}^{m}P(D_j/A_q){\log }_{2}P(D_j/A_q),\end{array}$$

(15)

which offers

$$\begin{array}{c}\hfill {\underline{H}}_{(A_p,A_q)}^{\prime }\left(D/A\right)\ge {\underline{H}}_{(A_p,A_q)}\left(D/A\right).\end{array}$$

(16)

We below provide another upper bound of $H_{(A_p,A_q)}\left(D/A\right)$, which may be better than ${\overline{H}}_{(A_p,A_q)}\left(D/A\right)$ in some cases.

Theorem 3.

At micro-bottom, the double-granule conditional-entropy has an upper bound. Concretely,

$$\begin{array}{c}\hfill \begin{array}{cc}\hfill H_{(A_p,A_q)}\left(D/A\right)& \le H_{(A_p,A_q)}^{*}\left(D/A\right)\hfill \\ & =-\sum _{j=1}^m\frac{|A_p\cap D_j|+|A_q\cap D_j|}{|A_p|+|A_q|}{\log }_2\frac{|A_p\cap D_j|+|A_q\cap D_j|}{|A_p|+|A_q|}.\hfill \end{array}\end{array}$$

(17)

Proof. 

As shown in Figure 2, function $f\left(P\right)=-P{\log }_{2}P$ ($P\in [0,1]$) is convex, where $f^{ȃ}\left(P\right)=-\frac{1}{P\ln 2}<0$. Thus, let

$$P_p=P(D_j/A_p)=\frac{|A_p\cap D_j|}{|A_p|},P_q=P(D_j/A_q)=\frac{|A_q\cap D_j|}{|A_q|},$$

and then the famous “Jensen’s inequality” in mathematics could induce

$$\begin{array}{c}\hfill {\omega }_p+{\omega }_q=1⟹-{\omega }_p{P}_p{\log }_2{P}_p-{\omega }_q{P}_q{\log }_2{P}_q\le -({\omega }_p{P}_q+{\omega }_p{P}_q){\log }_2({\omega }_p{P}_q+{\omega }_p{P}_q),\end{array}$$

where

$${\omega }_p{P}_q+{\omega }_p{P}_q=\frac{|A_p|}{|A_p|+|A_q|}\frac{|A_p\cap D_j|}{|A_p|}+\frac{|A_q|}{|A_p|+|A_q|}\frac{|A_q\cap D_j|}{|A_q|}=\frac{|A_p\cap D_j|+|A_q\cap D_j|}{|A_p|+|A_q|}.$$

In other words, we can get

$$\begin{array}{c}\hfill \begin{array}{cc}\hfill H_{(A_p,A_q)}\left(D/A\right)& =\sum _{j=1}^m[-{\omega }_p{P}_p{\log }_2{P}_p-{\omega }_q{P}_q{\log }_2{P}_q]\hfill \\ & \le \sum _{j=1}^m-({\omega }_p{P}_q+{\omega }_p{P}_q){\log }_2({\omega }_p{P}_q+{\omega }_p{P}_q)\hfill \\ & =-\sum _{j=1}^m\frac{|A_p\cap D_j|+|A_q\cap D_j|}{|A_p|+|A_q|}{\log }_2\frac{|A_p\cap D_j|+|A_q\cap D_j|}{|A_p|+|A_q|}\hfill \\ & =H_{(A_p,A_q)}^{*}\left(D/A\right).\hfill \end{array}\end{array}$$

□

In Theorem 3, the convex property of information function $f\left(P\right)=-P{\log }_{2}P$ is utilized to provide a new upper bound $H_{(A_p,A_q)}^{*}\left(D/A\right)$ of central measure $H_{(A_p,A_q)}\left(D/A\right)$. When comparing Equations (7) and (17), we can surprisingly discover that $H_{(A_p,A_q)}^{*}\left(D/A\right)$ highly adheres to

$$\begin{array}{c}\hfill \begin{array}{c}\hfill H_{A_p\cup A_q}(D/(A-\left\{a\right\}))=-\sum _{j=1}^m\frac{|(A_p\cup A_q)\cap D_j|}{|A_p\cup A_q|}{\log }_2\frac{|(A_p\cup A_q)\cap D_j|}{|A_p\cup A_q|},\end{array}\end{array}$$

(18)

which naturally comes from $H_{C_p\cup C_q}(D/(C-\left\{c\right\}))$ (Equation (7)). In fact,

$$\begin{array}{c}\hfill \begin{array}{c}\hfill H_{(A_p,A_q)}^{*}\left(D/A\right)=H_{A_p\cup A_q}(D/(A-\left\{a\right\}))\end{array}\end{array}$$

(19)

when $A_p\ne A_q$; when

$$A_q=A_p⟹\frac{|A_p\cap D_j|+|A_q\cap D_j|}{|A_p|+|A_q|}=\frac{|A_p\cap D_j|}{|A_p|}\ge \frac{|A_p\cap D_j|}{2|A_p|}=\frac{|(A_p\cup A_q)\cap D_j|}{|A_p|+|A_q|},$$

where $A_p\cup A_q=A_p$, there is a difference between two measures, and we obtain

$$\begin{array}{c}\hfill H_{(A_p,A_p)}^{*}\left(D/A\right)=-\sum _{j=1}^m\frac{|A_p\cap D_j|}{|A_p|}{\log }_2\frac{|A_p\cap D_j|}{|A_p|}\ne -\sum _{j=1}^m\frac{|A_p\cap D_j|}{2|A_p|}{\log }_2\frac{|A_p\cap D_j|}{2|A_p|}=H_{A_p\cup A_q}(D/(A-\left\{a\right\})).\end{array}$$

(20)

Thus far, $H_{(A_p,A_q)}\left(D/A\right)$ has one lower bound ${\underline{H}}_{(A_p,A_q)}\left(D/A\right)$ and two upper bounds ${\overline{H}}_{(A_p,A_q)}\left(D/A\right)$, $H_{(A_p,A_q)}^{*}\left(D/A\right)$. An interesting question naturally emerges, i.e., can we necessarily determine the size relationship between ${\overline{H}}_{(A_p,A_q)}\left(D/A\right)$ and $H_{(A_p,A_q)}^{*}\left(D/A\right)$ to provide an exact bound? Unfortunately, the answer is negative, and the later example and experiment will reveal the size uncertainty. We simply provide a mechanism analysis. Let

$$P_{pq}=\frac{|A_p\cap D_j|+|A_q\cap D_j|}{|A_p|+|A_q|},$$

and its numerator/denominator be the corresponding sum of numerators/denominators of $P_p$ and $P_q$. According to [64], we can obtain

$$P_{pq}\in [min(P_p,P_q),max(P_p,P_q)]$$

but $P_{pq}$ produces an uncertainty location between $P_p$ and $P_q$. In view of the information function $f\left(P\right)=-P{\log }_{2}P$ and its maximum point $(\frac{1}{e},\frac{1}{e\ln 2})$ (Figure 2),

$$f\left(P_p\right)+f\left(P_q\right),f\left(P_{pq}\right)$$

never having the necessary size relationships, so

$${\overline{H}}_{(A_p,A_q)}\left(D/A\right)=\sum _{j=1}^m(f\left(P_p\right)+f\left(P_q\right)),H_{(A_p,A_q)}^{*}\left(D/A\right)=\sum _{j=1}^{m}f\left(P_{pq}\right)$$

also never have the necessary size relationships. In summary, ${\overline{H}}_{(A_p,A_q)}\left(D/A\right)$ and $H_{(A_p,A_q)}^{*}\left(D/A\right)$ adopt different views to become irrelevant and interactive, and they together restrict $H_{(A_p,A_q)}\left(D/A\right)$. With the addition of lower bound of ${\underline{H}}_{(A_p,A_q)}\left(D/A\right)$, there are in total three bounds to systematically emerge. Similar to $H_{(A_p,A_q)}\left(D/A\right)$ and its distributional Table 2, they can also be arranged in a table with an $n\times n$ square symmetric matrix, i.e.,

Table 3. Three bounds of double-granule conditional-entropies at micro-bottom.

$U/IND\left(A\right)$	$A_1$	⋯	$A_q$	⋯	$A_n$
$A_1$	$[{\underline{H}}_{(A_1,A_1)(D/A)}(D/A),{\overline{H}}_{(A_1,A_1)}(D/A)]$ $H_{(A_1,A_1)}^{*}\left(D/A\right)$	⋯	$[{\underline{H}}_{(A_1,A_q)}(D/A),{\overline{H}}_{(A_1,A_q)}(D/A)]$ $H_{(A_1,A_q)}^{*}\left(D/A\right)$	⋯	$[{\underline{H}}_{(A_1,A_n)}(D/A),{\overline{H}}_{(A_1,A_n)}(D/A)]$ $H_{(A_1,A_n)}^{*}\left(D/A\right)$
⋮	⋮	⋱	⋮	⋱	⋮
$A_p$	$[{\underline{H}}_{(A_p,A_1)}(D/A),{\overline{H}}_{(A_p,A_1)}(D/A)]$ $H_{(A_p,A_1)}^{*}\left(D/A\right)$	⋯	$[{\underline{H}}_{(A_p,A_q)}(D/A),{\overline{H}}_{(A_p,A_q)}(D/A)]$ $H_{(A_p,A_q)}^{*}\left(D/A\right)$	⋯	$[{\underline{H}}_{(A_p,A_n)}(D/A),{\overline{H}}_{(A_p,A_n)}(D/A)]$ $H_{(A_p,A_n)}^{*}\left(D/A\right)$
⋮	⋮	⋱	⋮	⋱	⋮
$A_n$	$[{\underline{H}}_{(A_n,A_1)}(D/A),{\overline{H}}_{(A_n,A_1)}(D/A)]$ $H_{(A_n,A_1)}^{*}\left(D/A\right)$	⋯	$[{\underline{H}}_{(A_n,A_q)}(D/A),{\overline{H}}_{(A_n,A_q)}(D/A)]$ $H_{(A_n,A_q)}^{*}\left(D/A\right)$	⋯	$[{\underline{H}}_{(A_n,A_n)}(D/A),{\overline{H}}_{(A_n,A_n)}(D/A)]$ $H_{(A_n,A_n)}^{*}\left(D/A\right)$

, and thus Table 3 correspondingly restricts Table 2.

Finally, consider relevant granulation monotonicity/non-monotonicity. In fact, micro-bottom and its double-granule conditional-entropies focus on only two condition granules and thus never consider the condition granulation and further monotonicity/non-monotonicity. Moreover, $U/IND\left(A\right)⪰U/IND\left(B\right)$ implies the granulation refining and granule decomposition from A to B; thus $A_p,A_q\in U/IND\left(A\right)$ and $B_{p^{*}},B_{q^{*}}\in U/IND\left(B\right)$ exhibit complex correspondence and uncertainty change, so we cannot mine fine relationships between $H_{(A_p,A_q)}\left(D/A\right)$ and $H_{(B_{p^{*}},B_{q^{*}})}\left(D/B\right)$.

3.2. Double-Granule Conditional-Entropy at Meso-Middle

As analyzed above, double-granule conditional-entropies at micro-bottom never consider the condition granulation to lack robust functions of uncertainty descriptions. In terms of fixed decision granulation $U/IND\left(D\right)$, $H_{(A_p,A_q)}\left(D/A\right)$ at micro-bottom $(A_p,A_q)$ involves only two condition granules $A_p,A_q$ and their interactive uncertainty information. For the function promotion, the condition granulation $U/IND\left(A\right)$ with systematic granules is worth introducing based on double-granule conditional-entropy $H_{(A_p,A_q)}\left(D/A\right)$. Thus, we will gradually strengthen the knowledge granulation $U/IND\left(A\right)$ to establish better double-granule conditional-entropies, by virtue of three-level granular structures (Table 1). This subsection discusses double-granule conditional-entropies at meso-middle

$$(A_p,U/IND\left(A\right)=\{A_1,\cdots ,A_n\})(p\in \{1,\cdots ,n\}).$$

Definition 5.

At meso-middle $(A_p,U/IND\left(A\right))$, the double-granule conditional-entropy is defined by

$$\begin{array}{cc}\hfill H_{\left(A_p\right)}\left(D/A\right)& =-\sum _{q=1}^n\left({\omega }_p\sum _{j=1}^{m}P(D_j/A_p){\log }_{2}P(D_j/A_p)\right)-\sum _{q=1}^n\left({\omega }_q\sum _{j=1}^{m}P(D_j/A_q){\log }_{2}P(D_j/A_q)\right)\hfill \\ & =-\sum _{q=1}^n\left(\frac{|A_p|}{|A_p|+|A_q|}\sum _{j=1}^m\frac{|A_p\cap D_j|}{|A_p|}{\log }_2\frac{|A_p\cap D_j|}{|A_p|}\right)-\sum _{q=1}^n\left(\frac{|A_q|}{|A_p|+|A_q|}\sum _{j=1}^m\frac{|A_q\cap D_j|}{|A_q|}{\log }_2\frac{|A_q\cap D_j|}{|A_q|}\right).\hfill \end{array}$$

(21)

Corollary 1.

At meso-middle, the double-granule conditional-entropy has an analytic expression:

$$\begin{array}{c}\hfill H_{\left(A_p\right)}\left(D/A\right)=-\left(\sum _{q=1}^n\frac{|A_p|}{|A_p|+|A_q|}\right)\left(\sum _{j=1}^{m}P(D_j/A_p){\log }_{2}P(D_j/A_p)\right)-\sum _{q=1}^n\left(\frac{|A_q|}{|A_p|+|A_q|}\sum _{j=1}^{m}P(D_j/A_q){\log }_{2}P(D_j/A_q)\right).\end{array}$$

(22)

Theorem 4.

Double-granule conditional-entropies have a hierarchical integration from micro-bottom to meso-middle, i.e.,

$$\begin{array}{c}\hfill H_{\left(A_p\right)}\left(D/A\right)=\sum _{q=1}^n{H}_{(A_p,A_q)}\left(D/A\right)=H_{(A_p,A_1)}\left(D/A\right)+\cdots +H_{(A_p,A_q)}\left(D/A\right)+\cdots +H_{(A_p,A_n)}\left(D/A\right).\end{array}$$

(23)

By Definition 5 (Corollary 1) and Theorem 4, meso-middle’s measure $H_{\left(A_p\right)}\left(D/A\right)$ (which can also be noted by $H_{(A_p,U/IND\left(A\right))}\left(D/A\right)$) hierarchically integrates double-granule conditional-entropies $H_{(A_p,A_q)}\left(D/A\right)$ by condition-granular summation on $q=1,\cdots ,n$. Thus, $H_{\left(A_p\right)}\left(D/A\right)$ inherits the features of double-granule and conditional-entropy, it considers a granule $A_p$ and condition granulation $U/IND\left(A\right)$ to be at meso-middle $(A_p,U/IND\left(A\right))$, so it is called the double-granule conditional-entropy at meso-middle. As a transition, $H_{\left(A_p\right)}\left(D/A\right)$ combines granule $A_p$ and partition $U/IND\left(A\right)$ to describe decision classification $U/IND\left(D\right)$ and its uncertainty, from the perspective of conditional information.

Similar to and based on previous discussions on $H_{(A_p,A_q)}\left(D/A\right)$ (Section 3.1), we will provide corresponding properties of $H_{\left(A_p\right)}\left(D/A\right)$, including the number distribution, calculation algorithm, three bounds, and granulation monotonicity/non-monotonicity.

Proposition 3.

At meso-middle, double-granule conditional-entropies offer n values, i.e.,

$$H_{\left(A_p\right)}\left(D/A\right)(p=1,\cdots ,n).$$

In Proposition 3, double-granule conditional-entropies naturally exhibit number n to correspond to n meso-middles. The n values can be stored in an n-dimension vector to be related to the previous distributional Table 2. By enlarging Table 2, they are represented by the marginal vector of the bottom or right in

Table 4. Marginal distribution of double-granule conditional-entropies at meso-middle and macro-top.

$U/IND\left(A\right)$	$A_1$	⋯	$A_q$	⋯	$A_n$	Meso-Middle
$A_1$	$H_{(A_1,A_1)}(D/A)$	⋯	$H_{(A_1,A_q)}(D/A)$	⋯	$H_{(A_1,A_n)}(D/A)$	$H_{\left(A_1\right)}(D/A)$
⋮	⋮	⋱	⋮	⋱	⋮	⋮
$A_p$	$H_{(A_p,A_1)}(D/A)$	⋯	$H_{(A_p,A_q)}(D/A)$	⋯	$H_{(A_p,A_n)}(D/A)$	$H_{\left(A_p\right)}(D/A)$
⋮	⋮	⋱	⋮	⋱	⋮	⋮
$A_n$	$H_{(A_n,A_1)}(D/A)$	⋯	$H_{(A_n,A_q)}(D/A)$	⋯	$H_{(A_n,A_n)}(D/A)$	$H_{\left(A_n\right)}(D/A)$
Meso-Middle	$H_{\left(A_1\right)}(D/A)$	⋯	$H_{\left(A_q\right)}(D/A)$	⋯	$H_{\left(A_n\right)}(D/A)$	Macro-Top: $H(D/A)$

, and they exactly correspond to the relevant row/column sum of micro-bottom’s information values. According to Equations (21) and (23), Algorithm 2 resorts to two “for” loops to effectively offer a double-granule conditional-entropy $H_{\left(A_p\right)}\left(D/A\right)$ for an arbitrary granule $A_p\in U/IND\left(A\right)$. In fact, the inner loop invokes Algorithm 1 to calculate an arbitrary double-granule conditional-entropy at micro-bottom, while the outer loop integrates n related bottomed measures to produce $H_{\left(A_p\right)}\left(D/A\right)$. Furthermore, we can achieve all n middle entropies values by adding a “for” loop regarding $p=1,\cdots ,n$.

Algorithm 2: Calculation of double-granule conditional-entropy at meso-middle

Input: Decision table $(U,C\cup D)$, target subset $A\subseteq C$, and a special index $p\in \{1,\cdots ,n\}$; Output: Double-granule conditional-entropy $H_{\left(A_p\right)}\left(D/A\right)$ at meso-middle $(A_p,U/IND\left(A\right))$.   1: Compute $U/IND\left(A\right)$ to obtain all condition classes $A_i$ ($i=1,\cdots ,n$) and a fixed granule $A_p\in U/IND\left(A\right)$.   2: Compute $U/IND\left(D\right)$ to obtain all decision classes $D_j$ ($j=1,\cdots ,m$).   3: Let $H_{\left(A_p\right)}\left(D/A\right)=0$.   4: for$q\in \{1,..,n\}$do   5:   Compute ${\omega }_p,{\omega }_q$.   6:   Let $H_p=0$, $H_q=0$.   7:   for $j\in \{1,..,m\}$ do   8:    $H_p\leftarrow H_p-P(D_j/A_p){\log }_{2}P(D_j/A_p)$,    $H_q\leftarrow H_q-P(D_j/A_q){\log }_{2}P(D_j/A_q)$.   9:   end for 10:   Obtain $H_{(A_p,A_q)}\left(D/A\right)={\omega }_p{H}_p+{\omega }_q{H}_q$. 11:   $H_{\left(A_p\right)}\left(D/A\right)\leftarrow H_{\left(A_p\right)}\left(D/A\right)+H_{(A_p,A_q)}\left(D/A\right)$. 12: end for 13: return$H_{\left(A_p\right)}\left(D/A\right)$.

Theorem 5.

At meso-middle, the double-granule conditional-entropy has a lower bound and two upper bounds. Concretely,

$$\begin{array}{c}\hfill \begin{array}{cc}\hfill H_{\left(A_p\right)}\left(D/A\right)& \in [{\underline{H}}_{\left(A_p\right)}\left(D/A\right),{\overline{H}}_{\left(A_p\right)}\left(D/A\right)],\hfill \\ \hfill H_{\left(A_p\right)}\left(D/A\right)& \le H_{\left(A_p\right)}^{*}\left(D/A\right),\hfill \end{array}\end{array}$$

(24)

where

$$\begin{array}{cc}\hfill {\underline{H}}_{\left(A_p\right)}\left(D/A\right)& =\sum _{q=1}^n{\underline{H}}_{(A_p,A_q)}\left(D/A\right)=-\frac{n|A_p|}{2\left|U\right|}\sum _{j=1}^{m}P(D_j/A_p){\log }_{2}P(D_j/A_p)-\sum _{q=1}^n\left(\frac{|A_q|}{2\left|U\right|}\sum _{j=1}^{m}P(D_j/A_q){\log }_{2}P(D_j/A_q)\right),\hfill \\ \hfill {\overline{H}}_{\left(A_p\right)}\left(D/A\right)& =\sum _{q=1}^n{\overline{H}}_{(A_p,A_q)}\left(D/A\right)=-n\sum _{j=1}^{m}P(D_j/A_p){\log }_{2}P(D_j/A_p)-\sum _{q=1}^n\sum _{j=1}^{m}P(D_j/A_q){\log }_{2}P(D_j/A_q),\hfill \\ \hfill H_{\left(A_p\right)}^{*}\left(D/A\right)& =\sum _{q=1}^n{H}_{(A_p,A_q)}^{*}\left(D/A\right)=-\sum _{q=1}^n\sum _{j=1}^m\frac{|A_p\cap D_j|+|A_q\cap D_j|}{|A_p|+|A_q|}{\log }_2\frac{|A_p\cap D_j|+|A_q\cap D_j|}{|A_p|+|A_q|}.\hfill \end{array}$$

(25)

Theorem 5 naturally comes from Theorems 2–4. The three bounds in Equation (25) hierarchically integrate previous three bounds at micro-bottom (Equations (11) and (17)) to correspondingly restrict $H_{\left(A_p\right)}\left(D/A\right)$. They can be supplemented into distributional Table 4, and following

Table 5. Three bounds of double-granule conditional-entropies at meso-middle and macro-top.

$U/IND\left(A\right)$	$H_{\left(A_p\right)}(D/A)$	${\underline{H}}_{\left(A_p\right)}\left(D/A\right)$	${\overline{H}}_{\left(A_p\right)}\left(D/A\right)$	$H_{\left(A_p\right)}^{*}\left(D/A\right)$
$A_1$	$H_{\left(A_1\right)}(D/A)$	${\underline{H}}_{\left(A_1\right)}(D/A)$	${\overline{H}}_{\left(A_1\right)}(D/A)$	$H_{\left(A_1\right)}^{*}(D/A)$
⋮	⋮	⋮	⋮	⋮
$A_p$	$H_{\left(A_p\right)}(D/A)$	${\underline{H}}_{\left(A_p\right)}(D/A)$	${\overline{H}}_{\left(A_p\right)}(D/A)$	$H_{\left(A_p\right)}^{*}(D/A)$
⋮	⋮	⋮	⋮	⋮
$A_n$	$H_{\left(A_n\right)}(D/A)$	${\underline{H}}_{\left(A_n\right)}(D/A)$	${\overline{H}}_{\left(A_n\right)}(D/A)$	$H_{\left(A_n\right)}^{*}(D/A)$
Macro-Top	$H(D/A)$	$\underline{H}(D/A)$	$\overline{H}(D/A)$	$H^{*}(D/A)$

provides the relevant part.

At meso-middle, $H_{\left(A_p\right)}\left(D/A\right)$ introduces the condition granulation $U/IND\left(A\right)$, but it still needs condition granule $A_p$. Thus, we cannot make a positive assertion regarding granulation monotonicity/non-monotonicity. In fact, $U/IND\left(A\right)⪰U/IND\left(B\right)$ also implies chaos between $H_{\left(A_p\right)}\left(D/A\right)$ and $H_{\left(B_{p^{*}}\right)}\left(D/B\right)$.

3.3. Double-Granule Conditional-Entropy at Macro-Top

As analyzed above, double-granule conditional-entropies at meso-middle consider the condition granulation, but in an insufficient way, and $H_{\left(A_p\right)}\left(D/A\right)$ also depends on a single condition granule $A_p$. For the thorough granulation and robust description, systematic measures $H_{\left(A_p\right)}\left(D/A\right)$ ($p=1,\cdots ,n$) can be further integrated to generate double-granule conditional-entropies at macro-top. Based on the previous thought and result in Section 3.1 and Section 3.2, this subsection further discusses double-granule conditional-entropies at macro-top

$$(U/IND\left(A\right)=\{A_p:p=1,\cdots ,n\},U/IND\left(A\right)=\{A_q:q=1,\cdots ,n\}),$$

which is given in Table 1. We will directly provide the relevant integration definition, number distribution, calculation algorithm, three bounds, and we finally uncover an important conclusion of granulation non-monotonicity.

Definition 6.

At macro-top $(U/IND(A),U/IND(A\left)\right)$, the double-granule conditional-entropy is defined by

$$\begin{array}{cc}\hfill H\left(D/A\right)& =-\sum _{p=1}^n\sum _{q=1}^n\left({\omega }_p\sum _{j=1}^{m}P(D_j/A_p){\log }_{2}P(D_j/A_p)\right)-\sum _{p=1}^n\sum _{q=1}^n\left({\omega }_q\sum _{j=1}^{m}P(D_j/A_q){\log }_{2}P(D_j/A_q)\right)\hfill \\ & =-\sum _{p=1}^n\sum _{q=1}^n\left(\frac{|A_p|}{|A_p|+|A_q|}\sum _{j=1}^m\frac{|A_p\cap D_j|}{|A_p|}{\log }_2\frac{|A_p\cap D_j|}{|A_p|}\right)-\sum _{p=1}^n\sum _{q=1}^n\left(\frac{|A_q|}{|A_p|+|A_q|}\sum _{j=1}^m\frac{|A_q\cap D_j|}{|A_q|}{\log }_2\frac{|A_q\cap D_j|}{|A_q|}\right).\hfill \end{array}$$

(26)

Corollary 2.

At macro-top, the double-granule conditional-entropy has an analytic expression:

$$\begin{array}{c}\hfill H\left(D/A\right)=-\sum _{p=1}^n\left(\sum _{q=1}^n\frac{|A_p|}{|A_p|+|A_q|}\right)\left(\sum _{j=1}^{m}P(D_j/A_p){\log }_{2}P(D_j/A_p)\right)-\sum _{p=1}^n\sum _{q=1}^n\left(\frac{|A_q|}{|A_p|+|A_q|}\sum _{j=1}^{m}P(D_j/A_q){\log }_{2}P(D_j/A_q)\right).\end{array}$$

(27)

Theorem 6.

Double-granule conditional-entropies have a hierarchical integration from micro-bottom and meso-middle to macro-top, i.e.,

$$\begin{array}{c}\hfill \begin{array}{cc}\hfill H\left(D/A\right)& =\sum _{p=1}^n{H}_{\left(A_p\right)}\left(D/A\right)=H_{\left(A_1\right)}\left(D/A\right)+\cdots +H_{\left(A_n\right)}\left(D/A\right)\hfill \\ & =\sum _{p=1}^n\sum _{q=1}^n{H}_{(A_p,A_q)}\left(D/A\right)=H_{(A_1,A_1)}\left(D/A\right)+\cdots +H_{(A_n,A_n)}\left(D/A\right).\hfill \end{array}\end{array}$$

(28)

By Definition 6 (Corollary 2) and Theorem 6, macro-top’s measure $H\left(D/A\right)$ hierarchically integrates meso-middle’s entropies $H_{\left(A_p\right)}\left(D/A\right)$ by a single summation on $p=1,\cdots ,n$, and thus it further hierarchically integrates micro-bottom’s entropies $H_{(A_p,A_q)}\left(D/A\right)$ by double summations on $p,q=1,\cdots ,n$. As a result, $H\left(D/A\right)$ inherits the features of double-granule and conditional-entropy. It considers only conditional granulation $U/IND\left(A\right)$ to be at macro-top $(U/IND(A),U/IND(A\left)\right)$, so it is called the double-granule conditional-entropy at macro-top. As an ultimate measure, $H\left(D/A\right)$ completely utilizes the $U/IND\left(A\right)$ granulation information to effectively describe decision classification $U/IND\left(D\right)$ and its uncertainty, thus holding robust measurement functions for knowledge granulation. Moreover, $H\left(D/A\right)$ can be noted by $H_{(U/IND(A),U/IND(A\left)\right)}\left(D/A\right)$).

Proposition 4.

At macro-top, the double-granule conditional-entropy offers only one value, i.e.,

$$H\left(D/A\right) \mathit{at} \mathit{macro-top} (U/IND(A),U/IND(A\left)\right).$$

In Proposition 4, the double-granule conditional-entropy naturally exhibits number 1 to correspond to the sole macro-top. In fact, the first top entropy comes from the fusion of either n middle entropies or $n\times n$ bottom entropies; thus, three-level entropies accord with three-level granular structures (Table 1) from the quantitative and structural perspective, and they embody the hierarchical integration. In particular, the sole conditional-entropy $H\left(D/A\right)$ is put into the lower-right corner of Table 4, thus corresponding to the summations of central $n\times n$ micro values and marginal n meso values. According to Equations (26) and (28), Algorithm 3 resorts to three “for” loops to effectively offer the double-granule conditional-entropy $H\left(D/A\right)$. The two inner loops invoke Algorithm 2 to calculate an arbitrary double-granule conditional-entropy at meso-middle (where the central loop invokes Algorithm 1 to construct micro-bottom’s entropies), while the outer loop integrates n related meso-middle’s information values to produce $H\left(D/A\right)$. In other words, Algorithms 1–3 exhibit a kind of hierarchical evolution based on circulation development, and thus they constitute a novel kind of three-level algorithms.

Algorithm 3: Calculation of double-granule conditional-entropy at macro-top

Input: Decision table $(U,C\cup D)$, target subset $A\subseteq C$; Output: Double-granule conditional-entropy $H\left(D/A\right)$ at Macro-Top $(U/IND(A),U/IND(A\left)\right)$.   1: Compute $U/IND\left(A\right)$ to obtain all condition classes $A_i$ ($i=1,\cdots ,n$).   2: Compute $U/IND\left(D\right)$ to obtain all decision classes $D_j$ ($j=1,\cdots ,m$).   3: Let $H\left(D/A\right)=0$.   4: for$p\in \{1,..,n\}$do   5:   Let $H_{\left(A_p\right)}\left(D/A\right)=0$.   6:   for $q\in \{1,..,n\}$ do   7:    Compute ${\omega }_p,{\omega }_q$.   8:    Let $H_p=0$, $H_q=0$.   9:    for $j\in \{1,..,m\}$ do 10:     $H_p\leftarrow H_p-P(D_j/A_p){\log }_{2}P(D_j/A_p)$,     $H_q\leftarrow H_q-P(D_j/A_q){\log }_{2}P(D_j/A_q)$. 11:    end for 12:    Obtain $H_{(A_p,A_q)}\left(D/A\right)={\omega }_p{H}_p+{\omega }_q{H}_q$. 13:    $H_{\left(A_p\right)}\left(D/A\right)\leftarrow H_{\left(A_p\right)}\left(D/A\right)+H_{(A_p,A_q)}\left(D/A\right)$. 14:   end for 15:   $H\left(D/A\right)\leftarrow H\left(D/A\right)+H_{\left(A_p\right)}\left(D/A\right)$. 16: end for 17: return$H\left(D/A\right)$.

Theorem 7.

At macro-top, the double-granule conditional-entropy has a lower bound and two upper bounds. Concretely,

$$\begin{array}{c}\hfill \begin{array}{cc}\hfill H\left(D/A\right)& \in [\underline{H}\left(D/A\right),\overline{H}\left(D/A\right)],\hfill \\ \hfill H\left(D/A\right)& \le H^{*}\left(D/A\right),\hfill \end{array}\end{array}$$

(29)

where

$$\begin{array}{cc}\hfill \underline{H}\left(D/A\right)& =\sum _{p=1}^n{\underline{H}}_{\left(A_p\right)}\left(D/A\right)=\sum _{p=1}^n\sum _{q=1}^n{\underline{H}}_{(A_p,A_q)}\left(D/A\right)\hfill \\ & =-n\sum _{p=1}^n\left(\frac{|A_p|}{2\left|U\right|}\sum _{j=1}^{m}P(D_j/A_p){\log }_{2}P(D_j/A_p)\right)-n\sum _{q=1}^n\left(\frac{|A_q|}{2\left|U\right|}\sum _{j=1}^{m}P(D_j/A_q){\log }_{2}P(D_j/A_q)\right)\hfill \\ & =-n\sum _{p=1}^n\left(\frac{|A_p|}{\left|U\right|}\sum _{j=1}^{m}P(D_j/A_p){\log }_{2}P(D_j/A_p)\right),\hfill \end{array}$$

(30)

$$\begin{array}{c}\hfill \begin{array}{cc}\hfill \overline{H}\left(D/A\right)& =\sum _{p=1}^n{\overline{H}}_{\left(A_p\right)}\left(D/A\right)=\sum _{p=1}^n\sum _{q=1}^n{\overline{H}}_{(A_p,A_q)}\left(D/A\right)\hfill \\ & =-n\sum _{p=1}^n\sum _{j=1}^{m}P(D_j/A_p){\log }_{2}P(D_j/A_p)-n\sum _{q=1}^n\sum _{j=1}^{m}P(D_j/A_q){\log }_{2}P(D_j/A_q)\hfill \\ & =-2n\sum _{p=1}^n\sum _{j=1}^{m}P(D_j/A_p){\log }_{2}P(D_j/A_p),\hfill \end{array}\end{array}$$

(31)

$$\begin{array}{c}\hfill \begin{array}{cc}\hfill H^{*}\left(D/A\right)& =\sum _{p=1}^n{H}_{\left(A_p\right)}^{*}\left(D/A\right)=\sum _{p=1}^n\sum _{q=1}^n{H}_{(A_p,A_q)}^{*}\left(D/A\right)\hfill \\ & =-\sum _{p=1}^n\sum _{q=1}^n\sum _{j=1}^m\frac{|A_p\cap D_j|+|A_q\cap D_j|}{|A_p|+|A_q|}{\log }_2\frac{|A_p\cap D_j|+|A_q\cap D_j|}{|A_p|+|A_q|}.\hfill \end{array}\end{array}$$

(32)

Theorem 7 naturally comes from Theorems 2–6. The three bounds in Equations (30)–(32) hierarchically integrate previous three bounds at meso-middle and micro-bottom, and thus they become three new uncertainty measures at macro-top $(U/IND(A),U/IND(A\left)\right)$ to correspondingly restrict $H\left(D/A\right)$. They are supplemented into the bottom in the previous bound table: Table 5.

Theorem 8.

At macro-top, the double-granule conditional-entropy has granulation non-monotonicity. That is, $U/IND\left(A\right)⪰U/IND\left(B\right)$ cannot necessarily achieve

$$\mathit{either} H\left(D/A\right)\ge H\left(D/B\right) \mathit{or} H\left(D/A\right)\le H\left(D/B\right),$$

and both cases can practically exist. In addition, the matched double bounds $\underline{H}\left(D/A\right)$ and $\overline{H}\left(D/A\right)$ (Equations (30) and (31)) also have the granulation non-monotonicity, and they cannot theoretically acquire

$$\mathit{either} \underline{H}\left(D/A\right)\ge \underline{H}\left(D/B\right) \mathit{or} \underline{H}\left(D/A\right)\le \underline{H}\left(D/B\right),$$

$$\mathit{either} \overline{H}\left(D/A\right)\ge \overline{H}\left(D/B\right) \mathit{or} \overline{H}\left(D/A\right)\le \overline{H}\left(D/B\right).$$

At macro-top, the double-granule conditional-entropy completely breaks away from the condition granule dependence to establish the condition granulation description, so it becomes a powerful type of information measure for knowledge-based uncertainty representation. In terms of condition granulation, its non-monotonicity is finally revealed in Theorem 8, and the relevant evidence will be provided in the later example and experiment. Moreover, this fundamental non-monotonicity conclusion embodies information uncertainty, and it can be induced or explained by the previous complexity mechanism at micro-bottom and meso-middle. Based on macro-top and its granulation mechanism, the related three bounds (Equations (30)–(32)) and their monotonicity/non-monotonicity can be practically observed, and thus we also obtain the granulation non-monotonicity for $\underline{H}\left(D/A\right)$ and $\overline{H}\left(D/A\right)$; however, the case of upper bound $H^{*}\left(D/A\right)$ becomes a remaining problem.

4. Decision Table Example

In this section, the above theoretical constructions and properties are illustrated by a decision table example. By extracting a part of VOTING data set (which comes from UCI database [65]), we provide a practical decision table $(U,C\cup D)$ in

Table 6. A decision table.

U	$c_1$	$c_2$	$c_3$	$c_4$	$c_5$	$c_6$	$c_7$	$c_8$	$c_9$	$c_{10}$	$c_{11}$	D
$x_1$	2	2	4	4	4	3	4	4	4	2	4	1
$x_2$	2	2	4	4	2	2	4	4	4	2	3	1
$x_3$	3	4	3	4	2	4	2	4	4	2	2	0
$x_4$	2	4	2	3	2	4	2	4	2	2	4	0
$x_5$	4	4	2	4	2	4	3	4	4	4	3	0
$x_6$	2	4	2	4	2	2	2	4	4	4	4	0
$x_7$	2	2	4	4	2	2	2	3	4	4	2	0
$x_8$	2	2	4	4	2	2	2	4	4	3	4	1

with

$$\left|U\right|=8,\left|C\right|=11,\left|D\right|=1.$$

According to this decision table,

$$U/IND\left(D\right)=\{D_1=\{x_1,x_2,x_8\},D_2=\{x_3,x_4,x_5,x_6,x_7\}\}$$

provides $m=|U/IND(D\left)\right|=2$. As an example, $A=\{c_1,c_2,c_3,c_4,c_5\}$ is chosen to generate condition granulation

$$U/IND\left(A\right)=\{A_1=\left\{x_1\right\},A_2=\{x_2,x_7,x_8\},A_3=\left\{x_3\right\},A_4=\left\{x_4\right\},A_5=\left\{x_5\right\},A_6=\left\{x_6\right\}\},$$

where $n=|U/IND(A\left)\right|=6$. By virtue of three-level granular structures (Table 1), double-granule conditional-entropies and their three bounds are calculated by relevant algorithms and definitions, and they are compactly listed in

Table 7. Information values of double-granule conditional-entropies in the example.

U	$A_1$	$A_2$	$A_3$	$A_4$	$A_5$	$A_6$	Meso-Middle
$A_1$	0	$0.6887$	0	0	0	0	$0.6887$
$A_2$	$0.6887$	$0.9183$	$0.6887$	$0.6887$	$0.6887$	$0.6887$	$4.3619$
$A_3$	0	$0.6887$	0	0	0	0	$0.6887$
$A_4$	0	$0.6887$	0	0	0	0	$0.6887$
$A_5$	0	$0.6887$	0	0	0	0	$0.6887$
$A_6$	0	$0.6887$	0	0	0	0	$0.6887$
Meso-Middle	$0.6887$	$4.3619$	$0.6887$	$0.6887$	$0.6887$	$0.6887$	Macro-Top: $7.8055$

and

Table 8. Three bounds of double-granule conditional-entropies in the example.

U	$A_1$	$A_2$	$A_3$	$A_4$	$A_5$	$A_6$	Meso-Middle
$A_1$	$[0,0]$ 0	$[0.1722,0.9183]$ $0.8113$	$[0,0]$ 1	$[0,0]$ 1	$[0,0]$ 1	$[0,0]$ 1	$[0.1722,0.9183]$ $4.8113$
$A_2$	$[0.1722,0.9183]$ $0.8113$	$[0.3444,1.8366]$ $0.9183$	$[0.1722,0.9183]$ 1	$[0.1722,0.9183]$ 1	$[0.1722,0.9183]$ 1	$[0.1722,0.9183]$ 1	$[1.2053,6.4281]$ $5.7296$
$A_3$	$[0,0]$ 1	$[0.1722,0.9183]$ 1	$[0,0]$ 0	$[0,0]$ 0	$[0,0]$ 0	$[0,0]$ 0	$[0.1722,0.9183]$ $2.0000$
$A_4$	$[0,0]$ 1	$[0.1722,0.9183]$ 1	$[0,0]$ 0	$[0,0]$ 0	$[0,0]$ 0	$[0,0]$ 0	$[0.1722,0.9183]$ $2.0000$
$A_5$	$[0,0]$ 1	$[0.1722,0.9183]$ 1	$[0,0]$ 0	$[0,0]$ 0	$[0,0]$ 0	$[0,0]$ 0	$[0.1722,0.9183]$ $2.0000$
$A_6$	$[0,0]$ 1	$[0.1722,0.9183]$ 1	$[0,0]$ 0	$[0,0]$ 0	$[0,0]$ 0	$[0,0]$ 0	$[0.1722,0.9183]$ $2.0000$
Meso-Middle	$[0.1722,0.9183]$ $4.8113$	$[1.2053,6.4281]$ $5.7296$	$[0.1722,0.9183]$ $2.0000$	$[0.1722,0.9183]$ $2.0000$	$[0.1722,0.9183]$ $2.0000$	$[0.1722,0.9183]$ $2.0000$	Macro-Top: $[2.0662,11.0196]$ $18.5049$

, respectively. The measures at micro-bottom, meso-middle, macro-top have numbers 36, 6, 1, respectively, and they correspond to the central $6\times 6$ matrix, marginal 6-dimensional vector, lower-right-corner 1 digit, respectively. In part, we provide some processes of entropy calculation as follows.

$$\begin{array}{c}\hfill \begin{array}{cc}\hfill -P(D_1/A_1){\log }_{2}P(D_1/A_1)& =0,-P(D_2/A_1){\log }_{2}P(D_2/A_1)=0,\hfill \\ \hfill -P(D_1/A_2){\log }_{2}P(D_1/A_2)& =0.3900,-P(D_2/A_2){\log }_{2}P(D_2/A_2)=0.5283,\hfill \\ \hfill H_{(A_1,A_1)}\left(D/A\right)& =\frac{1}{1+1}(0+0)+\frac{1}{1+1}(0+0)=0,\hfill \\ \hfill H_{(A_1,A_2)}\left(D/A\right)& =\frac{1}{1+3}(0+0)+\frac{3}{1+3}(0.3900+0.5283)=0.6887;\hfill \\ \hfill H_{\left(A_1\right)}\left(D/A\right)& =H_{(A_1,A_1)}\left(D/A\right)+H_{(A_1,A_2)}\left(D/A\right)+\cdots +H_{(A_1,A_6)}\left(D/A\right)\hfill \\ & =0+0.6887+0+0+0+0=0.6887;\hfill \\ \hfill H\left(D/A\right)& =H_{\left(A_1\right)}\left(D/A\right)+H_{\left(A_2\right)}\left(D/A\right)+\cdots +H_{\left(A_6\right)}\left(D/A\right)\hfill \\ & =0.6887+4.3619+0.6887+0.6887+0.6887+0.6887=7.8055.\hfill \end{array}\end{array}$$

(33)

By Table 7 and Table 8, we can make relevant verification analyses. First, entropies and bounds naturally present hierarchical integration relationships from micro-bottom to meso-middle to macro-top. Indeed, conditional-entropies are correspondingly restricted by three bounds. Moreover, the two types of upper bounds exactly have no necessary size relationships, and a part but powerful proof is provided as follows:

$$\begin{array}{c}\left\{\begin{array}{c}{\overline{H}}_{(A_1,A_2)}\left(D/A\right)=0.9183>0.8113=H_{(A_1,A_2)}^{*}\left(D/A\right),\hfill \\ {\overline{H}}_{(A_1,A_3)}\left(D/A\right)=0<1=H_{(A_1,A_3)}^{*}\left(D/A\right);\hfill \end{array}\right.\end{array}$$

$$\begin{array}{c}\left\{\begin{array}{c}{\overline{H}}_{\left(A_1\right)}\left(D/A\right)=0.9183<4.8113=H_{\left(A_1\right)}^{*}\left(D/A\right),\hfill \\ {\overline{H}}_{\left(A_2\right)}\left(D/A\right)=6.4281>5.7296=H_{\left(A_2\right)}^{*}\left(D/A\right).\hfill \end{array}\right.\end{array}$$

Finally, the granulation non-monotonicity at macro-top (Theorem 8) is verified. For this, we chose a natural attribute-addition chain:

$$\left\{c_1\right\}\subset \{c_1,c_2\}\subset \cdots \subset \{c_1,c_2,\cdots ,c_{11}\}.$$

$C{A}_k$ ($k\in \{1,2,\cdots ,11\}$) denotes the attribute subset in the chain, and its granulation is represented by

$$U/IND\left(C{A}_k\right)=\{C{A}_{k,1},\cdots ,C{A}_{k,p},\cdots ,C{A}_{k,|U/IND(C{A}_k\left)\right|}\}.$$

In other words, $C{A}_{k,p}$ corresponds to the kth chain element $C{A}_k$ to represent the pth condition granule in partition $U/IND\left(C{A}_k\right)$. According to the subset chain,

Table 9. Double-granule conditional-entropies based on an attribute-enlargement chain in the example.

Level	Measure	$C{A}_1$	$C{A}_2$	$C{A}_3$	$C{A}_4$	$C{A}_5$	$C{A}_6$	$C{A}_7$	$C{A}_8$	$C{A}_9$	$C{A}_{10}$	$C{A}_{11}$
Micro-Bottom	$H_{(C{A}_{k,1},C{A}_{k,1})}(D/C{A}_{k,})$ $H_{(C{A}_{k,1},C{A}_{k,2})}(D/C{A}_k)$ $H_{(C{A}_{k,1},C{A}_{k,3})}(D/C{A}_k)$ $H_{(C{A}_{k,2},C{A}_{k,2})}(D/C{A}_k)$ $H_{(C{A}_{k,2},C{A}_{k,3})}(D/C{A}_k)$	$1.0000$ $0.8571$ $0.8571$ 0 0	$0.8113$ $0.6490$ $0.5409$ 0 0	$0.8113$ $0.6490$ $0.5409$ 0 0	$0.8113$ $0.6490$ $0.6490$ 0 0	0 $0.6887$ 0 $0.9183$ $0.6887$	0 $0.6887$ 0 $0.9183$ $0.6887$	0 1 0 0 0	0 1 0 0 0	0 1 0 0 0	0 1 0 0 0	0 1 0 0 0
Meso-Middle	$H_{\left(C{A}_{k,1}\right)}(D/C{A}_k)$ $H_{\left(C{A}_{k,2}\right)}(D/C{A}_k)$ $H_{\left(C{A}_{k,3}\right)}(D/C{A}_k)$	$2.7143$ $0.8571$ $0.8571$	$2.6502$ $0.6490$ $0.5409$	$2.6502$ $0.6490$ $0.5409$	$3.4074$ $0.6490$ $0.6490$	$0.6887$ $4.3619$ $0.6887$	$0.6887$ $4.3619$ $0.6887$	$0.6667$ $0.6667$ $0.6667$	0 0 0	0 0 0	0 0 0	0 0 0
Macro-Top	$H(D/C{A}_k)$ $\underline{H}(D/C{A}_k)$ $\overline{H}(D/C{A}_k)$ $H^{*}(D/C{A}_k)$	$4.4286$ $2.2500$ $6.0000$ $4.9409$	$4.4891$ $1.6226$ $6.4902$ $6.6951$	$4.4891$ $1.6226$ $6.4902$ $6.6951$	$6.0035$ $2.0282$ $8.1128$ $8.5789$	$7.8055$ $2.0282$ $11.0196$ $18.5409$	$7.8055$ $2.0282$ $11.0196$ $18.5409$	$9.0000$ $1.7500$ $14.0000$ $28.0196$	0 0 0 30	0 0 0 30	0 0 0 30	0 0 0 30

provides double-granule conditional-entropies, including both part values at micro-bottom $(C{A}_{k,p},C{A}_{k,q})$, meso-middle $(C{A}_{k,p},U/IND\left(C{A}_k\right))$ and all values (as well as the three bounds) at macro-top $(U/IND\left(C{A}_k\right),U/IND\left(C{A}_k\right))$. As a supporting detail, previous Table 7 and Table 8 actually embrace the chain element $C{A}_5$ and its partition $U/IND\left(C{A}_5\right)=\{\left\{x_1\right\},\{x_2,x_7,x_8\},\left\{x_3\right\},\left\{x_4\right\},\left\{x_5\right\},\left\{x_6\right\}\}$, while double-granule conditional-entropies regarding attribute subset $C{A}_2=\{c_1,c_2\}$ and corresponding condition granulation

$$U/IND\left(C{A}_2\right)=\{C{A}_{2,1}=\{x_1,x_2,x_7,x_8\},C{A}_{2,2}=\left\{x_3\right\},C{A}_{2,3}=\{x_4,x_6\},C{A}_{2,4}=\left\{x_5\right\}\}$$

are supplemented in

Table 10. Double-granule conditional-entropies regarding $C{A}_2=\{c_1,c_2\}$ in the example.

U	$C{A}_{2,1}$	$C{A}_{2,2}$	$C{A}_{2,3}$	$C{A}_{2,4}$	Meso-Middle
$C{A}_{2,1}$	$0.8113$	$0.6490$	$0.5409$	$0.6490$	$2.6502$
$C{A}_{2,2}$	$0.6490$	0	0	0	$0.6490$
$C{A}_{2,3}$	$0.5409$	0	0	0	$0.5409$
$C{A}_{2,4}$	$0.6490$	0	0	0	$0.6490$
Meso-Middle	$2.6502$	$0.6490$	$0.5409$	$0.6490$	Macro-Top: $4.4891$

for better observation and illustration.

(1)

Since different chain subsets may have different equivalence partitions and granule numbers, the measures at micro-bottom and meso-middle consider condition granules to have a distinctive number and difficult correspondence. Table 9 focuses on the small and the same granule number, but relevant granules have different connotations. For example, the granules of the first one —$C{A}_{k,1}$ ($k=1,2,\cdots ,11$)—may be different. Thus, we cannot acquire the so-called granulation non-monotonicity assertion because of granulation incompletion, although the values at micro-bottom and meso-middle actually exhibit a kind of non-monotonic change in Table 9.

(2)

In contrast, macro-top offers the complete condition granulation, so we can effectively focus on value monotonicity/non-monotonicity for both double-granule conditional-entropies and their three bounds. Observing the bottom part of Table 9 in the enlargement chain direction, we can discover that the three types of information measures are all non-monotonic, i.e.,

$$H(D/C{A}_k),\underline{H}(D/C{A}_k),\overline{H}(D/C{A}_k)(\mathrm{except} H^{*}(D/C{A}_k)).$$

More vividly, the entropy and its three bounds regarding the chain are depicted in Figure 3, so the related granulation non-monotonicity becomes clearer. For example, the macro entropy value $H(D/C{A}_k)$ first increases and then decreases in the addition chain direction. Moreover, Table 9 and Figure 3 reflect the restriction properties of three bounds.

5. Data Experiments

In this section, the above theoretical results and their effectiveness are verified by data experiments. The new measures are mainly suitable for categorical (or nominal) data, which are usually used in the traditional rough set theory, and thus we adopt three relevant data sets from the UCI Machine Learning Repository [65], whose concrete descriptions on decision table $(U,C\cup D)$ are given in

Table 11. Three UCI data sets.

Label	Name	$\mathbf{\left|}\mathit{U}\mathbf{\right|}$	$\mathbf{\left|}\mathit{C}\mathbf{\right|}$	$\mathbf{|}\mathit{U}\mathbf{/}\mathit{IND}\mathbf{(}\mathit{C}\mathbf{\left)}\mathbf{\right|}$	$\mathbf{\left|}\mathit{D}\mathbf{\right|}$	$\mathbf{|}\mathit{U}\mathbf{/}\mathit{IND}\mathbf{(}\mathit{D}\mathbf{\left)}\mathbf{\right|}$
(1)	VOTING	435	16	342	1	2
(2)	SPECT	187	22	169	1	2
(3)	Tic-Tac-Toe	958	9	958	1	2

.

Similar to the above example, we also adopt the attribute-addition chain

$$\begin{array}{c}\hfill \begin{array}{c}\hfill C{A}_1=\left\{c_1\right\}\subset \cdots \subset C{A}_{\left|C\right|}=\{c_1,c_2,\cdots c_{\left|C\right|}\}\end{array}\end{array}$$

(34)

and its relevant symbol such as

$$U/IND\left(C{A}_k\right)=\{C{A}_{k,1},\cdots ,C{A}_{k,p},\cdots ,C{A}_{k,|U/IND(C{A}_k\left)\right|}\}.$$

Note that this attribute-subset sequence (Equation (34)) can deeply and typically probe the hierarchical knowledge-granulation within a framework of the complete lattice $(2^C,\subseteq )$. As a representative manifestation, we provide two typical results regarding the first chain element $C{A}_1=\left\{c_1\right\}$ and the last one $C{A}_{\left|C\right|}=C$.

(1)

Regarding VOTING, $\left\{c_1\right\}$ and C induce three and 342 granules, respectively, and relevant double-granule conditional-entropies and three bounds are provided in

Table 12. Double-granule conditional-entropies in the VOTING data set.

$U/IND\left(C{A}_1\right)$	$C{A}_{1,1}$	$C{A}_{1,2}$	$C{A}_{1,3}$	Meso-Middle	⋯	$U/IND\left(C{A}_{16}\right)$	$C{A}_{16,1}$	⋯	$C{A}_{16,342}$	Meso-Middle
$C{A}_{1,1}$	0.9867	0.9782	0.8369	2.8018	⋯	$C{A}_{16,1}$	0	⋯	0	0
$C{A}_{1,2}$	0.9782	0.8113	0.6578	2.4473	⋯	⋮	⋮	⋱	⋮	⋮
$C{A}_{1,3}$	0.8369	0.6578	0.6479	2.1427	⋯	$C{A}_{16,342}$	0	⋯	0	0
Meso-Middle	2.8018	2.4473	2.1427	Macro-Top: 7.3918	⋯	Meso-Middle	0	⋯	0	Macro-Top: 0

and

Table 13. Three information bounds in the VOTING data set.

$U/IND\left(C{A}_1\right)$	$C{A}_{1,1}$	$C{A}_{1,2}$	$C{A}_{1,3}$	Meso-Middle	⋯	$U/IND\left(C{A}_{16}\right)$	$C{A}_{16,1}$	⋯	$C{A}_{16,342}$	Meso-Middle
$C{A}_{1,1}$	$[1.0706,$ $1.9734]$ $0.9867$	$[0.5577,$ $1.7980]$ $0.9921$	$[0.8139,$ $1.6346]$ $0.9649$	$[2.4422,$ $5.4060]$ $2.9436$	⋯	$C{A}_{16,1}$	$[0,0]$ 0	⋯	$[0,0]$ 0	$[0,0]$ $221.3143$
$C{A}_{1,2}$	$[0.5577,$ $1.7980]$ $0.9921$	$[0.0448,$ $1.6226]$ $0.8113$	$[0.3009,$ $1.4592]$ $0.6596$	$[0.9034,$ $4.8798]$ $2.4630$	⋮	⋮	⋮	⋱	⋮	⋮
$C{A}_{1,3}$	$[0.8139,$ $1.6346]$ $0.9649$	$[0.3009,$ $1.4592]$ $0.6596$	$[0.5571,$ $1.2959]$ $0.6479$	$[1.6719,$ $4.3898]$ $2.2724$	⋯	$C{A}_{16,342}$	$[0,0]$ 0	⋯	$[0,0]$ 0	$[0,0]$ $221.3143$
Meso-Middle	$[2.4422,$ $5.4060]$ $2.9436$	$[0.9034,$ $4.8798]$ $2.4630$	$[1.6719,$ $4.3898]$ $2.2724$	Macro-Top: $[5.0174,$ $14.6755]$ $7.6790$	⋯	Meso-Middle	$[0,0]$ $221.3143$	⋯	$[0,0]$ $221.3143$	Macro-Top: $[0,0]$ 50132

, respectively.

(2)

Regarding SPECT, $\left\{c_1\right\}$ and C produce two and 169 granules, respectively, and relevant three-level measures and three bounds are provided in

Table 14. Double-granule conditional-entropies in the SPECT data set.

$U/IND\left(C{A}_1\right)$	$C{A}_{1,1}$	$C{A}_{1,2}$	Meso-Middle	⋯	$U/IND\left(C{A}_{22}\right)$	$C{A}_{22,1}$	⋯	$C{A}_{22,169}$	Meso-Middle
$C{A}_{1,1}$	0.2108	0.3815	0.5924	⋯	$C{A}_{22,1}$	0	⋯	0	1.5335
					⋮	⋮	⋱	⋮	⋮
$C{A}_{1,2}$	0.3815	0.5399	0.9215	⋯	$C{A}_{22,169}$	0	⋯	0	1.5335
Meso-Middle	0.5924	0.9215	Macro-Top: 1.5139	⋯	Meso-Middle	1.5335	⋯	1.5335	Macro-Top: 513.0879

and

Table 15. Three information bounds in the SPECT data set.

$U/IND\left(C{A}_1\right)$	$C{A}_{1,1}$	$C{A}_{1,2}$	Meso-Middle	⋯	$U/IND\left(C{A}_{22}\right)$	$C{A}_{22,1}$	⋯	$C{A}_{22,169}$	Meso-Middle
$C{A}_{1,2}$	$[0.1015,$ $0.4217]$ $0.2109$	$[0.1908,$ $0.7508]$ $0.4030$	$[0.2922,$ $1.1725]$ $0.6138$	⋯	$C{A}_{22,1}$	$[0,0]$ 0	⋯	$[0,0]$ 0	$[0.0332,$ $1.9457]$ $8.8982$
				⋮	⋮	⋮	⋱	⋮	⋮
$C{A}_{1,2}$	$[0.1908,$ $0.7508]$ $0.4030$	$[0.2801,$ $1.0799]$ $0.5400$	$[0.4708,$ $1.8306]$ $0.9429$	⋯	$C{A}_{22,169}$	$[0,0]$ 0	⋯	$[0,0]$ 0	$[0.0332,$ $1.9457]$ $161.2140$
Meso-Middle	$[0.2922,$ $1.1725]$ $0.6138$	$[0.4708,$ $1.8306]$ $0.9429$	Macro-Top: $[0.7631,$ $14.6755]$ $1.5566$	⋯	Meso-Middle	$[0.0332,$ $1.9457]$ $8.8982$	⋯	$[0.0332,$ $1.9457]$ $161.2140$	Macro-Top: $[11.2085,$ $657.6332]$ 2867

, respectively.

(3)

Regarding Tic-Tac-Toe, $\left\{c_1\right\}$ and C determine three and 958 granules, respectively, and relevant entropies and bounds are provided in

Table 16. Double-granule conditional-entropies in the Tic-Tac-Toe data set.

$U/IND\left(C{A}_1\right)$	$C{A}_{1,1}$	$C{A}_{1,2}$	$C{A}_{1,3}$	Meso-Middle	⋯	$U/IND\left(C{A}_9\right)$	$C{A}_{9,1}$	⋯	$C{A}_{9,958}$	Meso-Middle
$C{A}_{1,1}$	0.8742	0.9248	0.8794	2.6784	⋯	$C{A}_{9,1}$	0	⋯	0	0
$C{A}_{1,2}$	0.9248	0.9881	0.9509	2.8638	⋯	⋮	⋮	⋱	⋮	⋮
$C{A}_{1,3}$	0.8794	0.9509	0.8901	2.7203	⋯	$C{A}_{9,958}$	0	⋯	0	0
Meso-Middle	2.6784	2.8638	2.7203	Macro-Top: 8.2625	⋯	Meso-Middle	0	⋯	0	Macro-Top: 0

and

Table 17. Three information bounds in the Tic-Tac-Toe data set.

$U/IND\left(C{A}_1\right)$	$C{A}_{1,1}$	$C{A}_{1,2}$	$C{A}_{1,3}$	Meso-Middle	⋯	$U/IND\left(C{A}_9\right)$	$C{A}_{9,1}$	⋯	$C{A}_{9,958}$	Meso-Middle
$C{A}_{1,1}$	$[0.3814,$ $1.7483]$ $0.8741$	$[0.3635,$ $1.8622]$ $0.9404$	$[0.2859,$ $1.7642]$ $0.8796$	$[1.0308,$ $5.3748]$ $2.6940$	⋯	$C{A}_{9,1}$	$[0,0]$ 0	⋯	$[0,0]$ 0	$[0,0]$ 332
$C{A}_{1,2}$	$[0.3635,$ $1.8622]$ $0.9404$	$[0.3455,$ $1.9762]$ $0.9881$	$[0.2680,$ $1.8781]$ $0.9628$	$[0.9770,$ $5.7165]$ $2.8913$	⋮	⋮	⋮	⋱	⋮	⋮
$C{A}_{1,3}$	$[0.2859,$ $1.7642]$ $0.8796$	$[0.2680,$ $1.8781]$ $0.9628$	$[0.1905,$ $1.7801]$ $0.8900$	$[0.7444,$ $5.4224]$ $2.7324$	⋯	$C{A}_{9,958}$	$[0,0]$ 0	⋯	$[0,0]$ 0	$[0,0]$ 626
Meso-Middle	$[1.0308,$ $5.3748]$ $2.6940$	$[0.9770,$ $5.7165]$ $2.8913$	$[0.7444,$ $5.4224]$ $2.7342$	Macro-Top: $[2.7522,$ $16.5138]$ $8.3178$	⋯	Meso-Middle	$[0,0]$ 332	⋯	$[0,0]$ 626	Macro-Top: $[0,0]$ 415664

, respectively.

From the perspective of macro-top, double-granule conditional-entropies and their three information bounds based on the attribute-enlargement chain are finally summarized in Figure 4. These tables and figures can be utilized to effectively verify all previous conclusions, including the hierarchy, algorithm, restriction, and non-monotonicity. In particular, double-granule conditional-entropies are confined by three bounds, thus supporting the boundedness (Theorems 2, 3, 5 and 7); moreover, the entropies and their matched double-bounds fluctuate up and down, thus proving relevant granulation non-monotonicity (Theorem 8).

6. Conclusions

The information measures implement fundamental uncertainty measurement in rough set theory and granular computing. The local conditional-entropies have the second-order feature, but they are limited to micro-bottom for describing discernibility matrix and reduction core [18]. In this paper, double-granule conditional-entropies achieve corresponding improvements of hierarchical/conditional granulation, and thus they become broader measures with uncertainty representation and information processing. They focus more on the double-granule interaction rather than granule-union locality, which is used in local conditional-entropies [18]. This strategy directly utilizes the second-order mechanism to implement more systematic and robust uncertainty measurements, especially when compared to the current mainstream of first-order information measures. In our studies, double-granule conditional-entropies and their hierarchies, granulation, algorithms, bounds, and non-monotonicity are acquired and verified at three-level granular structures (i.e., micro-bottom, meso-middle, macro-top), and these results underlie both the efficiency in information processing and effectiveness in knowledge-based data analyses. Furthermore, their future developments and in-depth applications can be explored as follows.

(1)

In contrast to the relevant technology in [56], the hierarchical granulation of three-level granular structures focuses on the conditional granulation and relevant number, and it can be generalized for granular computing.

(2)

The double-granule conditional-entropies and their three bounds become new types of information measures with the second-order feature. In contrast to the traditional first-order entropy system, their description power and application advantage need further practical verification.

(3)

The double-granule conditional-entropies have three-restrictive bounds and granulation non-monotonicity, which have been experimentally verified by a granulation-hierarchical sequence (i.e., Equation (34)). These results are worth deeply utilizing in uncertainty measurement and data mining.

(4)

The double-granule conditional-entropies originate from the local conditional-entropies to carry a potential and distinctive advantage of discernibility matrix representation, and they also have the complete conditional granulation to have application prospects in knowledge reasoning or acquisition. Both their relationships with the discernibility matrix and their functions on attribute reduction need be deeply researched by promoting the previous studies in [18].