Concept networks of students’ knowledge of relationships between physics concepts: finding key concepts and their epistemic support

Abstract

Concept maps, which are network-like visualisations of the inter-linkages between concepts, are used in teaching and learning as representations of students’ understanding of conceptual knowledge and its relational structure. In science education, research on the uses of concept maps has focused much attention on finding methods to identify key concepts that are of the most importance either in supporting or being supported by other concepts in the network. Here we propose a method based on network analysis to examine students’ representations of the relational structure of physics concepts in the form of concept maps. We suggest how the key concepts and their epistemic support can be identified through focusing on the pathways along which the information is passed from one node to another. Towards this end, concept maps are analysed as directed and weighted networks, where nodes are concepts and links represent different types of connections between concepts, and where each link is assumed to provide epistemic support to the node it is connected to. The notion of key concept can then be operationalised through the directed flow of information from one node to another in terms of communicability between the nodes, separately for out-going and in-coming weighted links. Here we analyse a collated concept network based on a sample of 12 original concept maps constructed by university students. We show that communicability is a simple and reliable way to identify the key concepts and examine their epistemic justification within the collated network. The communicabilities of the key nodes in the collated network are compared with communicabilities averaged over the set of 12 individual concept maps. The comparison shows the collated network contains an extensive set of key concepts with good epistemic support. Every individual networks contain a sub-set of these key concepts but with a limited overlap of the sub-sets with other individual networks. The epistemically well substantiated knowledge is thus sparsely distributed over the 12 individual networks.

Introduction

Learning scientific knowledge requires learning its key concepts and their lexicon: how these concepts are related, how they can be used with other concepts and how they are connected as part of a system of other concepts. Scientific knowledge thus forms a system of networked concepts, where interconnections between the concepts also have an essential role in establishing their meaning (Rescher 1979; Kuhn 2000; Hoyningen-Huene 2013). An obvious approach to analysing scientific knowledge is to focus on terms that stand for the concepts, and on how relationships between the terms emerge on different contexts. These relationships form a networked lexicon of terms and names, where the connections between them derive from contextualised instances of how the terms are used and how situations are named (Kuhn 2000). In what follows, we refer to such lexical networks of knowledge simply as concept networks.

The assumption that concept’s meaning is related to the lexical system of terms is supported by recent advances in understanding how the meaning of ordinary concepts builds up through interlinked connections (Stella et al. 2017; Vitevich and Castro 2015). Interestingly, according to these studies difficulties and deficiencies in learning the meaning of words are directly reflected in the relational structure of the lexical networks, especially in the local and global connectivity of words in the network. The results of these studies show the importance of the relational connections between words in learning their meaning and how certain key words play a special role in learning the lexicons (Stella et al. 2017; Vitevich and Castro 2015).

The structure of the knowledge system affects how concepts are introduced in teaching scientific knowledge and how they are acquired in formal teaching and learning. In learning and teaching, too, conceptual knowledge is often approached from the viewpoint of semantic networks, because all retrieval and inference is based on traversing such networks (Chi and Ohlsson 2005). From the viewpoint of semantic networks, the concepts (or terms corresponding to them) that tend to form clusters of close connections are of particular interest because concepts within clusters share more similarities than concepts between clusters (Chi and Ohlsson 2005). In the case of lexicons of ordinary words and terms, such closeness of concepts has been successfully operationalised and measured by the closeness centrality, which is found to be indicative of key concepts (Stella et al. 2017; Vitevich and Castro 2015). This, however, may not be the case in regard to abstract, relational concepts, where long contiguous (i.e. uninterrupted directed) paths relate concepts in different clusters to each other, and when experts’ knowledge focuses on these contiguous, complex paths instead of simpler locally cohesive connections (Lachner and Nückles 2015). As well, studies on how students represent their understanding of the relatedness of physics concepts suggest that the most important concepts are the ones connected to other concepts through many contiguous paths (Koponen and Nousiainen 2013; 2018; Nousiainen 2013; Nousiainen and Koponen 2017). Such concepts are the key concepts in the students’ conceptual network and finding them in reliable way is an important and yet unresolved problem in research that attempts to understand how scientific knowledge is learned.

In learning and teaching science, from the viewpoint that important aspects of scientific knowledge are captured by the lexical structure of its terms and concepts, concept maps are obvious choices as a learning tool (Nousiainen 2013; Nesbit and Adesope 2006; Ingec 2009; Kinchin et al. 2000, 2005). Concepts maps express so-called declarative knowledge, which is explicated knowledge, written down or expressed in symbolical form, e.g. as formulas (Chi and Ohlsson 2005). In this form, concept maps reflect students’ conceptual understanding and are thus also used as tools of assessment and evaluation in learning (Ruiz-Primo and Shavelson 1996; Nicoll et al. 2001; Liu 2004). Consequently, many quantitative and qualitative techniques for the analysis of concept maps have been proposed and tested.

The quantitative methods of concept map analysis are often based on counting the hierarchical levels and the number of cross-links within a given hierarchical level. Quantitative analysis thus focuses on how concepts are interconnected within the semantic fields provided by the local connections (Ruiz-Primo and Shavelson 1996; Nicoll et al. 2001; Liu 2004; McClure et al. 1999; vanZele et al. 2004), but it fails to pay proper attention to the global structure and connectedness of concept maps. The qualitative methods of concept map evaluation attempt to correct and improve this deficiency of quantitative methods by paying attention to the global, overall visual appearance of concept maps (Kinchin et al. 2000, 2005; Liu 2004; Safayeni et al. 2005; Derbentseva et al. 2007). Such qualitative methods for analysing concept maps have revealed that visually discernible complexity in the form of spoke-, tree- or web-like features provides valuable information about the quality of knowledge represented in the concept maps (Kinchin et al. 2000, 2005). However, the validity and reliability of the qualitative methods are difficult to assess, because the criteria of being “complex” or “web-like” is based on visual inspection only. One should also be cautious in accepting the claim that the overall look of concept maps, i.e. likeness to spokes, trees or webs, has any true bearing on how individual conceptual elements are supported within the structure. The problems with qualitative analysis are thus the reverse of the ones with quantitative analysis: while the quantitative approach fails to capture the global structure, the qualitative approach fails to discern the effect of the structure on individual nodes.

Recent studies on students’ knowledge of physics concepts, investigated by using concept maps, networks and related techniques, have revealed that students’ declarative knowledge is structured on a global scale, and is web-like (Koponen and Nousiainen 2013, 2014; Nousiainen 2013) but not hierarchically organised in so simple ways as often assumed (Ruiz-Primo and Shavelson 1996; Nicoll et al. 2001; McClure et al. 1999; vanZele et al. 2004). The web-like concept networks drawn by students are tightly connected locally, having highly clustering cliques but at the same time being globally well connected through long paths that connect several concepts. The property of clustering, however, is not indicative of the key concepts but instead appears to be related to how students use auxiliary concepts. The key concepts, on the other hand, are characterised by long and contiguous paths to many other concepts in the network; the key concepts communicate with many other concepts in the network (Koponen and Nousiainen 2013, 2014, 2018; Nousiainen 2013, Nousiainen and Koponen 2017).

We approach students’ declarative knowledge and its representations as concept maps from the viewpoint of the cartography of knowledge (Börner 2015; Börner and Scharnhorst 2009; Chen et al. 2009; Shi et al. 2015), and by using network methods. The network approach pays attention to interlinked knowledge structures and thus provides a means to augment traditional methods based on local link-counting (Koponen and Nousiainen 2013; 2014). Network theory provides several operationalisations to measure such global connectivity of nodes and to find the key concepts that are globally important in the concept networks. In addition to global connectivity, we are also interested in how the given concept is connected through directed contiguous connections to other concepts in the network. Such contiguous directed connections are associated with the flow of information from one node to another, which is needed to substantiate the meaning of the concept with the network. Furthermore, the connections are weighted, where weights are associated with the epistemic strength of the information flow. These properties of interest are conveniently operationalised in terms of communicability (Estrada and Hatano 2008; Estrada et al. 2012; Estrada 2012; Benzi and Klymko 2013). Communicability allows us to study the in-coming and out-going weighted paths of the node separately, and how they contribute to the epistemic substantiation of the given node (concept) itself or provide substantiation to other nodes.

We examine 12 different concept maps constructed by individual students, where they represent their views of the relationships between concepts in electricity and magnetism. First, we discuss how these maps are constructed by students. Based on the 12 individual we construct a collated concept network where all acceptable elements (121 altogether) and the best substantiated connections (787 altogether) found in the 12 original maps are aggregated. The collated network thus contains the best available knowledge within the group of 12 students. Here, the target of analysis is this collated network. Second, we focus on the global roles of the nodes in the collated network and introduce a method based on communicability of nodes to find the key concepts that are structurally the most important. The content is discussed to the extent to determine if such concepts are also relevant from the point of view of content. Further details of the content and content analysis have been presented elsewhere (Koponen and Nousiainen 2013; Nousiainen 2013; Nousiainen and Koponen 2017). Here, we extend our previous study (Koponen and Nousiainen 2018) and provide more details of the analysis method and also explore to what extent the local connectivity of nodes determines the communicability. This is done by using an appropriate null-model, where in- and out degrees and epistemic weight distribution are preserved but links are shuffled. Third, we now compare the collated network to individual maps constructed by the students. The comparison is based on the similarity of the key concept sets and how the sets are shared. This comparison shows how much of the knowledge harboured within the group of 12 students is actually shared. The comparison provides information about how much the unshared knowledge provides the potential to improve the learning if it could be made available to group members.

Methods

The empirical sample studied here consists of 12 concept maps constructed by university students, where the students present their understanding and views of how physics concepts, in electricity and magnetism, are related. The sample of 12 concept maps contains N=121 different concepts, of which about 50-60 appear in each map, as well as the 787 most acceptable links of which about 150-200 are in a single map. The collated network analysed here aggregates all 121 concepts and 787 best links. Such a collated network represents the best available collective knowledge within the student group. The collated network is analysed as a directed, weighted network and its key concepts are identified. The identification of key concepts is based on the communicability between the nodes, separately for in-coming and out-going links. For comparisons, the ensemble of 12 networks, each of which is a substructure of the collated network, is analysed similarly and the communicabilities of all their nodes averaged over the set of 12 networks is compared with the communicabilities of nodes in the collated network.

Empirical sample: Concept maps

The empirical sample was produced during a course of seven weeks’ duration that focused on questions concerning the conceptual structure of electricity, magnetism and electromagnetism, in level of first year studies in physics. During the teaching sequence, the students first produced an initial concept network, and later, after instruction and group discussions, a final version of the network. Here, only the final versions are considered because the final stage of the students’ understanding of the relational structure of concepts is of interest. In constructing the concept maps, students were asked to concentrate on their discussions and reflections on the central concepts, laws, models and experiments they thought important in forming a well-organised and coherent picture of the topic. The construction of the concept maps was based on special kinds of concept nodes representing conceptual knowledge. The concept nodes in these networks represent: quantities, laws, models or experiments. Students were asked to pay attention on these types of knowledge and if appropriate, mention the type in the written report. Of these concept nodes, laws are either experimental laws or law-like predictions in specific situations (derived from a theory), or general laws (e.g. conservation laws) (Koponen and Nousiainen 2013; Nousiainen 2013). Students were free to introduce any concepts or conceptual elements they found necessary and were able to substantiate. However, a preliminary list of about 40 concepts was provided as a prompting, but students were asked to expand it and discard any item in the list they found not useful. The links in the network are different procedural relations between nodes (concepts), such as the following: the quantity can be changed, measured or kept constant (in experiments), or that its value is predicted or used as a parameter (e.g. models). The experiments that are central in the construction of the concept networks discussed here are the traditional teaching laboratory experiments, which are quantitative, and where a concept is operationalised, that is, made measurable through pre-existing concepts. The models involved in the construction of the concept networks are traditional textbook models used to introduce new concepts and laws. In constructing the maps, students had to give in a written report epistemic justification for the relationships between concepts, laws, models and experiments they represented in their concept maps. The concept networks and written reports thus contain only such concepts and relational connections that the students could substantiate (or justify) when they integrated new concepts as part of the networks.

An example of the 12 concept maps constructed by individual students is shown in Fig. 1. The concepts, laws, principles and models that appear most often in the maps are presented in Table 1 and numbered for later reference. In the analysis, we do not separately examine each student’s concept map in detail but instead agglomerate all 12 individual networks onto a single collated network. However, the individual concept maps are compared to the collated network. This comparison provides a window to the totality of the relational knowledge that the entire group of 12 students harbours but that none of them individually possess in its totality.

Fig. 1

An example of an individual concept map. A concept map constructed by one student, shown (redrawn for clarity) to illustrate the overall appearance of the maps. The content of the links is not essential here

Table 1 Key concepts appearing in students’ concept maps

Method of qualitative epistemic analysis

The analysis of the concept network is based on the qualitative method first to analyse the epistemic quality of the content, and on the subsequent quantitative method to analyse how that knowledge is used in the network at a global scale and how it is passed from one node to another. Qualitative analysis is used in assessing the degree of epistemic justification for the knowledge expressed in each single node contained in the maps and written reports. The qualitative analysis, based on discerning different epistemic levels, is explained in detail elsewhere (Nousiainen 2013), and only a short summary is given here. Quantitative analysis operationalises the epistemic support of each node as it receives support or is supporting other nodes in the network. In that the communicability (Estrada and Hatano 2008; Estrada et al. 2012; Estrada 2012) of the nodes is central as a measure of information flow. Consequently, the communicability of nodes is used to identify the key concepts and to find out how they are affected by the epistemic support deriving from their connection to other nodes.

The epistemic criteria of the validity of knowledge is related to the substantiation (or justification) of knowledge. Proper substantiation of knowledge can be associated with a contiguous chain of arguments which are rational and supported by evidence (Rescher 1979; Hoyningen-Huene 2013). The evaluation of the epistemic quality of such argumentation is based here on an epistemic taxonomy consisting of four levels (Nousiainen 2013): 1) Ontology, which uses ontologically correct entities and terms referring to them; 2) Factual statements such as laws, principles and relations, which are correct and have no inherent contradictions; 3) Methodological strategies, which contain possible and correct procedures of setting up experiments and developing models; 4) Justification of knowledge claims, which is systematic, coherent and logically sound. These four epistemic criteria are tiered. Each knowledge element (node) is scored so that the scoring S from 1 to 4 gives the highest tier on which the epistemic norms are fulfilled. On this basis, each node is assigned an epistemic strength s=S/S_max with S_max=4 and where s∈ [0,1]. Epistemically strong nodes have strengths 0.75<s<1, while weak nodes have values 0.0<s<0.25. The links are simply taken as they are drawn, and their content is ignored here. This is due to the fact that the linking words are not as informative as the node content.

Method of quantitative structural analysis

Following the interpretative analysis of the knowledge content, the strengths s of nodes contain the relevant information of the quality of the content of every single node. Only this information is used here to identify the key concepts. In what follows, the strength of node i is denoted by s_i, while a_ij is the link from node i to node j having a value 1 if the link exists and a value 0 if not. The set of values a_ij provides the NxN adjacency matrix a of the network. Because we are interested in how the information is passed from one node to another, we transform the directed node-weighted network to a directed link-weighted network. The weights w_ij of the directed links in the new weighted network are defined as (Koponen and Nousiainen 2018)

$$ w_{ij} =\beta \, s_{i}^{\beta} \: a_{ij} /w_{max} $$

(1)

where $w_{max}=\text {Max}\left [\left \{ \beta \, s_{i}^{\beta } \right \} \right ]$ is the maximal weight. All weights are thus normalised and w_ij∈ [0,1]. This definition of weights is motivated by the notion that the directed links w_ij pass supporting information from node i to node j in proportion to the epistemic strength of the node it originates from. The factor β allows enhanced weighting of strong links so that for β≫1 only strong links with w_ij≈1 survive while for β≪1 all links are weighted equally. The weighted network is now described through the weighted adjacency matrix W with elements [W]_ij=w_ij and allows us to perform the analysis as a weighted, directed network (Koponen and Nousiainen 2018). The collated network, which is in focus here, consists of the best substantiated connections with the highest values on w_ij as resulting from the connections in 12 individual networks. The collated network contains thus 121 nodes and 787 directed, weighted links.

The objective of the analysis is to find concepts in the network, which have the role of either feeding information to other nodes or receiving it: nodes which conceptually either support other nodes or which receive support. This calls for an analysis based on the flow of information through all contiguous paths. An obvious candidate of all centrality measures, then, is the communicability (Estrada and Hatano 2008; Estrada et al. 2012; Estrada 2012). As its name suggests, the communicability is closely connected to the idea of communicating between the connected nodes. As such, it is in many ways similar to Katz -centrality (Estrada 2012; da Costa et al. 2007). Although the analysis and its results would be very similar using Katz -centrality, here we prefer the communicability because of its convenient mathematical properties and easy interpretation (Estrada et al. 2012; Estrada 2012; Benzi and Klymko 2013).

When the networks are described using the weighted adjacency matrix W, it becomes possible to operationalise the notion of key nodes in passing the epistemic support from one node to another in terms of the communicability, which pays attention to the walks between nodes. The communicability between nodes is based on counting a weighted sum G_pq of walks between nodes p and q, where the weight is given by an inverse of a factorial of the length of the walk (Estrada and Hatano 2008; Estrada et al. 2012; Estrada 2012)

$$ G_{{pq}}= 1+\frac{\left[ \mathbf{W}^{1}\right]_{pq}}{1!}+\frac{\left[ \mathbf{W}^{2} \right]_{{pq}}}{2!}+\frac{\left[ \mathbf{W}^{3} \right]_{{pq}}}{3!}+... =\left[ e^{\mathbf{W}}\right]_{{pq}} $$

(2)

The communicabilities of a node v related to out-going (OUT) and in-coming links (IN) are then defined as the total out-communicability and in-communicability, respectively, as

$$ G_{\text{OUT}}(v)=\sum\limits_{p \neq v} G_{vp}\,, \:\:\:\: G_{\text{IN}}(v)=\sum\limits_{p \neq v} G_{pv} $$

(3)

The in- and out- communicabilities are simple and robust measures closely related to the passing of information from node to node in the network. The parameter β allows us to explore the effect of epistemic weighting on the communicabilities of the nodes and thus to assess the degree of epistemic substantiation of the knowledge in the concept networks. By increasing β, the epistemically strongest links are retained while those with low strength are effectively removed from the networks. At the other limit, where β≪1 all links receive an equal weight. Because we are studying only the relative importance of concepts, and because link weights remain normalised to the maximum value 1, limit β≪1 corresponds to a situation where all links are completely justified, i.e., maximal epistemic justification. For comparison we also calculate the in- and out-strengths D_IN and D_OUT of nodes, respectively, defined as (Estrada 2012; da Costa et al. 2007)

$$ D_{\text{OUT}}(v)=\sum\limits_{p \neq v} w_{vp}\,, \:\:\:\: D_{\text{IN}}(v)=\sum\limits_{p \neq v} w_{pv} $$

(4)

The in- and out strengths coincides with in- and out degrees of node when β≪1.

Null-model and statistical significance

The analysis of reliability and statistical significance of the results is carried out by comparing the results of the analysis to results obtained for an appropriate null-model (Estrada 2012; Kolaczyk 2009). To decide which features and which values of communicability are exceptional, and not determined simply by the in- and out-degree distribution of nodes, we define the null-model. The null model preserves number of nodes and links, the direction of links and the distribution of weights of the links, but shuffles all links. Such a null model is obtained by rewiring all in- and out-going links in the network and then assigning the weights at random but so that distribution of weights is preserved. In this study, we use 5000 rewirings to obtain an ensemble of networks to be compared with original collated networks. All rewirings are performed with IGraph software (Csardi and Nepusz 2006). For the ensemble of rewired networks, we calculate average values 〈O〉 and standard deviations σ_O of variables O∈{D_IN,D_OUT,G_IN,G_OUT}. The statistical significance of the values O is then assessed by calculating the so-called Z-scores (i.e. standardised value) of variable O defined as (Estrada 2012; Kolaczyk 2009)

$$ Z=\frac{O-\langle O \rangle}{\sigma_{O}} $$

(5)

where O is the observable value in the empirical sample. The reliability and statistical significance requires that absolute values |Z| of Z-scores are high enough, usually the value |Z|=2 taken as a limiting case. Assuming that the variables are normally distributed, Z-scores |Z|=2 and |Z|=3.0 correspond p-values 0.02 and 0.001, respectively. Here, we have chosen to use |Z|=2 as a cut-off for statistically significant deviations deserving special attention.

Similarity

The similarity comparisons of networks requires that the type of similarity in question is defined. Here we are not interested in local structural similarity, because local structure itself is not enough in deciding which concepts are the key concepts. Instead, the similarity comparisons must focus on the communicability of nodes in the network. The similarity we are interested in here is such that networks, where the same nodes have high communicabilities, are taken as similar: the more they share the same high communicability nodes, the more similar they are in all relevant aspects, irrespective of how the low values of communicabilities are distributed within the remaining nodes. When link weights are normalised, the “high” value of the communicability is taken to be those ones which exceed the limiting value 0.50. In a collated network consisting of 121 nodes, roughly 30% of the nodes have a communicability higher than 0.50 and 15% higher than 0.70. However, a direct comparison based on distributions of communicability values and standard similarity measures which pay attention to the whole distribution e.g. by using Kullback-Leibler divergence (Kolaczyk 2009) is not an option, because nodes with low values of communicability are not of interest and not relevant to the similarity.

The comparison of the similarity of two networks g and g’ is based on comparing the differences of the communicabilities of the shared set of high communicabilities. First, we select M nodes which are common to both networks and select m<M highest ranking nodes. Second, we calculate for each node v∈{m} the difference between communicabilities G(v) and G^′(v) corresponding to a node either in g or g’, respectively. Because in the similarity comparison, the in- and out-communicabilities are treated similarly, IN and OUT indices are dropped in what follows. Third, we define the relative dissimilarity of communicability as a ratio $\phantom {\dot {i}\!}r=|G(v) -G^{\prime }(v)|/(G(v)+G^{\prime }(v))$. The relative dissimilarity 0<r<1 takes into account that the effect of the difference depends on the value of G; for low communicability nodes smaller changes are relevant to deciding the dissimilarity than for nodes that have higher values of communicabilities. The total similarity $\phantom {\dot {i}\!}S_{g,g}^{\prime }$ of the networks g and g’ is then defined as

$$ S_{g,g^{\prime}}=1-\frac{1}{m}\sum\limits_{v \in {m}} \frac{|G(v) -G^{\prime}(v)|}{G(v) +G^{\prime}(v) } $$

(6)

In practice, the exact value of $\phantom {\dot {i}\!}S_{g,g^{\prime }}$ depends on the number m of the highest ranking nodes and the number M of nodes which are common, out of which m nodes is picked. However, in range 10<M<30 the results are rather stable, and we use the value that is the average of the 30% values around the most probable one when M varies from 10 to 30 and 5<m<0.9M. This selection of m corresponds roughly to relative values of communicability and strengths in range from 0.5 to 0.8, depending on epistemic weight. Of the different possible ways to define the similarity, this method proved to be the most stable and robust. The possible values of $\phantom {\dot {i}\!}S_{g,g^{\prime }}$ are from 0 to 1 and can be roughly interpreted as the relative similarity of networks g and g’.

Results

The collated concept network consisting of 12 individual concept maps is shown in Fig. 2, where node sizes are scaled according to values of the out- and in-communicabilities G_OUT(v) and G_IN(v). Results are shown for weak epistemic weighting (β≪1) and for epistemic substantiation corresponding to the original concept maps (β=0). The key nodes are numbered as explained in Table 1. Other symbols and abbreviations used throughout the text are provided in Table 2. For weak weighting, all links are equally weighted and the resulting out- and in-communicabilities correspond to the situation where all links would be maximally weighted. As Fig. 2 shows, the collated network is modular and consists of three modules. This modularity, however, emerges from the task structure. Students discussed separately the tasks of electrostatics, magnetostatics and electromagnetism.

Fig. 2

Collated concept network. The 12 individual concept networks as collated into a single network. Nodes are shown in proportion to their out- and in-communicability. In the upper row node weighting is β=10⁻² and in the lower row β=0. Note that only the part of the most importance is shown and marginal parts with auxiliary nodes are cut-off on borders

Table 2 Summary of symbols and abbreviations used in text and figures

Communicabilities

Figure 2 shows that the out-and in-communicabilities G_OUT(v) and G_IN(v) of a given node v are for many nodes essentially different. The nodes which have high out-communicability are those which feed information to other nodes and thus are (source) nodes that support the substantiation of other nodes. The nodes which have high in-communicability are ones where the information fed by other nodes ends; these are the strongly supported and substantiated (target) nodes. Nodes in these source and target groups are central to the network, and are thus the nodes associated with the key concepts. When epistemic weighting of the network is changed by changing weight parameter β as given by Eq. 2, the in- and out communicabilities are for some nodes substantially changed. Varying the weighting allows us to study the epistemic strength of the substantiation of conceptual knowledge as the collated concept network represents it.

The collated network with weighting β=10⁻² (upper panel of Fig. 2) shows the case when all links are equally weighted. This case corresponds to the situation where all links are maximally substantiated and the nodes consequently have maximal in- and out-communicabilities. Such a network corresponds to what an expert could produce (provided the set of nodes and links are similar). In this case, the well-substantiated key concepts with high in-communicability are somewhat equally distributed in all three modules, with none of the modules dominating. This is a desirable outcome, because on the basis of content there is no reason to expect that some of the modules should be more dominant than any other. On the other hand, for out-communicability the role of one cluster (electromagnetism) is less significant than the role of the two other modules (electrostatics and magnetostatics). This simply indicates that the two latter modules of electro- and magnetostatics are needed to support the knowledge in the first electromagnetism module.

The collated network with weighting β=0 (lower panel of Fig. 2) shows the case where the weighting corresponds the best substantiated nodes as collated from the original 12 concept maps. In comparison to case β=10⁻², a slightly lesser fraction of nodes have now a high in- and out-communicability. The network is now also clearly divided into modules of source and target nodes. Many of the high out-communicability source-nodes are now found in the module mediating (magnetostatics) between the other modules of source nodes (electrostatics and electromagnetism). In one of the source modules (electrostatics) only a few concepts now have the role of key concept, while in the target module the role of certain concepts as the key concepts is enhanced. In the mediating module, the changes of nodes in their roles of key concept are not very substantial and many key concepts retain their roles. The most prominent target-node module is obviously the electromagnetic module, because concepts in it need to be supported both electrostatics and magnetostatics concepts. The in- and out communicability quantifies thus quite naturally the information flows from source to target nodes.

The key concepts are supposedly those ones which receive high enough communicabilities. The histogram in Fig. 3 shows the distribution of communicability of nodes when epistemic weighting is varied. The histograms show that nodes with “high” values of communicability can be taken those ones with in- and out-communicabilities exceeding the value of 0.5. These nodes are rare, and many of them have anomalously low Z-scores. For epistemic weight logβ=2 communicability of nodes 28, 38, 74, 100 109 and 113 have Z-scores in range − 2.5<Z<−3.5, while for logβ=0 only nodes 28, 38, 74 and 109 have equally low Z-scores. Only nodes 57, 85 and 117 have statistically significant positive Z-scores, having 2.5<Z<3.5. For no epistemic weighting with logβ=−2 all key concepts have Z-scores |Z|<2. These results indicate that while many of the key nodes have communicability as expected on basis of null-model, yet several of them have a communicability not determined by local connectivity alone. Particularly clear this is for strong epistemic weighting. The conclusion that high-communicability nodes are special set of nodes is supported by examining the correlations between in- and out strengths D_IN and D_OUT and corresponding in- and out-communicabilities G_IN and G_OUT of the nodes. The ranking based Spearman ρ and Kendall τ non-parametric correlations (Corder and Foreman 2014) are summarised in Table 3. The correlations show that for β≪1 communicabilities of interest are determined by the strength of node (i.e. degree of node), but for increased demand for epistemic weighting the correlations become weaker. The deterioration of correlation shows that the communicability provides essential new information of long contiguous paths and how epistemic weighting affect them. Roughly, the message of Z-scores and correlations is that for students, it is very demanding to maintain the contiguous, logically ordered chain of connections from source nodes to target nodes, and that these chains are very often disrupted by failed or poorly established epistemic substantiation of the connections. This feature comes visible only through examination of communicabilities and is not available by focusing on local connectivity e.g. in form of node strengths.

Fig. 3

Histograms of distributions of in- and out- communicability. The histograms of values of G_IN and G_OUT for the collated network are shown for epistemic weights logβ=−2,0 and 2 as indicated in figures

Table 3 Correlation coefficients Γ[X,Y] between variables X,Y∈{D_IN,D_OUT,G_IN,G_OUT}

The key concepts

The key concepts and how they change when epistemic weighting is increased are shown in Fig. 4 as a fingerprint -map (Koponen and Nousiainen 2014). The map shows the key concepts of high in- or out-communicability as black stripes, and the lesser the communicability is the lighter is the colour of the stripe. The fingerprints reveal that there are about 23 key concepts of which, with increasing epistemic weighting, the number of strongly substantiated nodes (concepts) diminish and only about 15 remain epistemically strongly substantiated. The 23 nodes and the 15 most strongly substantiated (in boldface) are: 2, 14, 28, 27, 33 and 38 (for electrostatics, names of the concepts as in Table 1); 55, 57, 59, 63, 66, 71, 74, 82, 83, 85 and 86 (for magnetostatics); and 90, 91, 100, 109, 113, 117 and 120 (for electromagnetism). All these nodes represent concepts which are also central for the content, abstract and thus widely applicable across different contexts. Importantly, many of these concepts are general field concept of electricity and magnetism. Therefore, they are also key concepts in regard to the content.

Fig. 4

Fingerprint maps of the key concepts in the collated concept network. The key concepts having a high communicability are shown as black stripes. The weighting of links with β=10^p changes from p=−2 to p=2 as shown on the scale on the right. The communicability through out-going links (OUT) is shown in the upper panel and through in-coming links (IN) in the lower panel. Numbering of nodes is given in upper- and lower sides of the stripes. The colour-legend for relative strengths of communicability is provided on the right. Note how differently weighted networks have different sets of key nodes. The key nodes surviving after the strong epistemic weighting region are the most strongly supported/supporting ones

The set of strongly substantiated nodes for β≫1 represents that part of knowledge for which the student group is able to provide a proper epistemic justification. As is seen by comparing the results in Fig. 4 for weak (β≫1) and strong epistemic (β≪1) weighting, the key concept in the strongly weighted group is a sub-set of key concepts corresponding to the maximally substantiated (β≫1) network. It is this more limited set of key concepts that students can bring to their discussions and use in a justified manner when constructing arguments and making knowledge claims; these key nodes thus provide important information concerning what one can expect about the quality of knowledge within the student group and how it appears in discussions and argumentation within the group.

The set of key concepts and the changes in it with increasing epistemic weighting can conveniently be displayed using radar maps (or spider-web maps), where each concept is given as a corner in a polygon and the distance from the centre of the polygon is proportional to the value of the communicability. Such radar maps for source (OUT) and target (IN) key concepts are shown in Fig. 5 for 18 key concepts that appear in cases of low- and high epistemic weighting. For comparison, the radar map shows also the in- and out-strengths of the nodes. The radar maps in the left panel show the key concepts for β≪1, in which case all the links are equally weighted and the relative strengths of the communicabilities have the same values as would be obtained in a fully substantiated network where all links would receive weights 1.

Fig. 5

Radar maps of the key concepts in the collated network. The radar map representation of the 18 most important key concepts in the collated concept network. In the left column, the communicability (black polygon) is compared with the strength of nodes (red polygon) for the weighting β=10⁻². In the middle, the communicability (black) and the strenght (red) are compered for β=10⁰, and on the right for β=10². The out- and in-communicabilities are shown in the upper and lower rows, respectively

When the epistemic weighting is increased, the radar maps reveal how certain concepts gradually lose their role as key concepts and only a smaller sub-set of key concepts survives when β≫1. The in-and out strengths, however, remain largely unaffected, because they depend only on epistemic weights on nearest neighbour links. Moreover, as shown by results in Table 3 only for logβ=−2 the communicability and strength of node are strongly correlated. This difference in values of communicability and strength show that epistemic weighting of long contiguous paths depend sensitively on quality of epistemic substantiation and thus it provides more information of the epistemic substantiation than strength of node. The column at the right in Fig. 5 shows the key concepts for β=10², which is already a limiting case and further increase in β has no effect on the relative values of the communicabilities, and thus on the set of key concepts. The set of key concepts is remarkably robust, which indicates that many links contained in the collated network are sufficiently well substantiated to make the set resilient under the increased demand for epistemic substantiation.

Auxiliary concepts

The collated network contains a set of concepts which are tightly connected in terms of standard local clustering coefficient (da Costa et al. 2007; Estrada 2012). The unweighted collated network with β≪1 has nine concepts or conceptual elements with maximum local clustering coefficient of 1.00 and five from 0.70 to 0.98. Nearly all top 15 high-clustering nodes have Z-scores 2.5<Z<4.5. The average value of local clustering of collated network nodes is 0.13. None of these concepts or conceptual elements appear among nodes listed in Table 1 and none of them is indispensable for the content in general. The top conceptual elements with high local clustering coefficient are specific and context related model based derivations of certain laws, models, experiments or statement-type definitions. For example, some of the top-ranking nodes are: 121. Model exemplifying RCL-circuit and 107. Definition of mutual inductance. For collated and individual networks the 15 highest ranking concepts according to their clustering coefficient contain 12 which are derivations, models, model based definitions or experiments. Moreover, these items and ways they are reported in students’ reports closely match textbook presentations.

Similarity comparisons

The collated network which contains the best substantiated concepts of all individual 12 student-constructed concept maps is significantly more extensive and comprehensive than any of the 12 individual concept maps. The direct comparisons of individual concept maps to the collated network is not very illuminating, because the individual networks are very sparse and too few of the connections (links) contained in them are significant. Generally, while the collated network contains some 30 significant and well-substantiated concepts (see Table 1), the single individual network typically contains only from 6 to 10. Thus, there are few overlaps of single networks and the collated networks, and even fewer between the single networks themselves. An overall picture of how individual networks compare with the collated network is revealed by averaging the communicabilities of the 12 networks for all nodes and comparing these averaged values with values obtained for the collated network. In Fig. 6 are shown the communicabilities averaged over 12 individual networks for the key concepts shown in Fig. 5. The changes in the set of key concepts with increased epistemic weighting reveal substantial changes in the communicabilities, and for β≫1 only a few of the key concepts remain. Interestingly, in the set of source nodes, nodes 2 (electric charge) and 28 (empirically operationalised electric field) are dominant. In the set of target nodes, nodes 109, 117 and 120, related to magnetoelectric induction (the inverse of electromagnetic induction) are dominant. The unexpected dominance of certain key concepts in case of strong epistemic weighting is in contrast to the collated network, which turned out to be rather robust with the increased demand for epistemic substantiation. This indicates that a good and robust substantiation of the collated network actually originates only from one or two of the individual networks. Although many of the individual networks contain very few well-substantiated nodes, every network contains some which are not found in any other network. The collated network collects all these rare cases and the outcome is a well-substantiated and robust concept network, which is also highly satisfactory from the viewpoint of targeted content knowledge.

Fig. 6

Radar maps of the key concepts obtained as averages. The radar map representation of the 18 most important key concepts obtained by averaging the values of communicabilities over 12 individual concept network. The weighting of links is as denoted in upper row (note the colour annotation). The out- and in-communicabilities are shown in the upper and lower rows, respectively

A more detailed breakdown of the key concepts in individual networks as compared to the collated network is provided in Table 4, where the key concepts in individual networks are divided into three categories: I, II and III. Category I contains the concepts that appear in at least 10 networks, category II concepts appearing in at least 7 but at most 9, and III concepts that appear in exactly 6 networks only. The column COLLATED shows concepts in the collated network in order of their ranking based on the communicability. The key concepts in category I in Table 4 show that the concepts which appear in 11 or 12 networks, are not very many, but are among the key concepts in the collated network, although not the top ranking. The key concepts that appear in all 12 concept maps made by the students and that are epistemically strongly substantiated in these maps are concepts of electric charge (2) and electric field (28) as empirically operationalised. However, in the collated network the ranking of these concepts as key concepts is considerably lower, which indicates that their significant role is mostly due to their appearance in every individual concept map while most other key concepts in them are different. When the nodes and links are collated, the relative significance of 2 and 28, however, is greatly reduced. The only exception, a key concept which is important in individual networks but not in the collated network, is concept 1, electricity as recognised qualitatively. Concept 1 appears in all individual concept maps but is not epistemically strongly substantiated, and disappears from the set of key concepts when β is increased; although it is mentioned by all students in their networks, it is very weakly established key concept. Concepts in categories II and III are the most significant for the collated network and form its backbone. In these categories, however, many of the key concepts of the individual networks do not appear in the topmost set of key concepts in the collated network. The changes in the set of key concepts in categories II and III when epistemic weighting is increased are also substantial. The concepts which appear in five or fewer networks are not listed in Table 4. In this set of concepts, individual concept maps have little overlap, the concepts are poorly substantiated and do not form a robust collection under changed epistemic weighting.

Table 4 Key concepts as they appear in the collated and 12 individual concept networks

The results in Table 4 show that the differences between individual student-made concept maps are substantial, as well as the difference between any individual student map and the collated network. Here, the similarity of interest derives from the similarity of the set of key concepts and how their communicability values are distributed. A suitable similarity measure which fulfils these requirements is the similarity defined in Eq. 6. Roughly, such similarity can be interpreted as a relative, total communicability similarity between the compared networks. The similarity function S is shown in Fig. 7. The similarity of the epistemically weighted collated network to the unweighted collated (COLL) network corresponding β≪1 is shown by the uppermost curves for out-going (OUT) links at the left and for in-coming links (IN) at the right. The results show that when setting the highest demand on the epistemic weighting (β≫1), the similarity is reduced to 50% and to 60% for source and target concepts, respectively. The situation is more severe for the individual concept maps, whose similarity to the similarly weighted collated network are shown as gray dots. These similarities are for β≪1 in the best cases 50% but in the worst cases only 30%, and are reduced to very low values less than 20% for β≫1. For a comparison, the similarity of averaged (AVER) values of the communicabilities to values obtained for the collated network, corresponding to equal β, are shown, as well as the fitting function to the average over the individual network similarities. These results confirm the conclusion drawn on the basis of the radar maps in Figs. 5 and 6 and the results shown in Table 4; the individual concept maps contain only sparsely the relevant conceptual knowledge, but when they do this, it is often well substantiated. By collecting all separate well substantiated-elements, a highly satisfactory collated network results.

Fig. 7

Similarity of networks in epistemic weighting. The similarity of the epistemic weighted collated network to the unweighted collated (COLL, black circles) network with β≪1 is shown by the uppermost curves for out-going (OUT) links at the left and for in-coming links (IN) at the right. The similarity based on the averaged values of the communicabilities (AVER, gray squares) compared to the values from the collated network, with same value of epistemic weight β, is shown by the second curve from the top. The gray dots show the distribution of the similarities of the 12 individual networks. The fitting curve (the lowest curves) show the average value corresponding to the 12 individual cases

Discussion and conclusions

We have focused on finding the key concepts in how university students conceive the relational conceptual structure of electricity and magnetism. The study sample was a set of 12 concept maps and accompanying study reports made by students during a seven-week course. The 12 individual networks were agglomerated into a single collated network, which could be taken to represent the totality of concepts and their relationships which arose in the discussions and which were focused on exercises during the course. Here, we showed that the communicability of nodes in the collated network had good resolving power in revealing how different nodes (concepts) in the collated network channelled information to provide epistemic substantiation of other nodes in the network, and how the other nodes were substantiated by that information. The nodes that had a key role in transmitting or receiving the information were those with a high communicability through either the out-going or in-coming links, respectively. These nodes were the key concepts in the network.

The analysis method based on finding the key concepts on the basis of the ranking of nodes according to their communicability provided insight on the quality of the epistemic substantiation of the knowledge as it was represented in the concept network. The method introduced here for global analysis augments the traditional methods of concept network and concept map analysis, which focus on the local properties through counting links (Ruiz-Primo and Shavelson 1996; Nicoll et al. 2001; McClure et al. 1999) or on visual appearances of the patterns found in the concept maps (Kinchin et al. 2000, 2005). The method based on communicability operationalises the desired property of the conceptual contiguity of the concept network and the information flow from node to node in the network. It also provides a way to study the effect of the epistemic weighting on the emergence of the key concepts.

The results of the analysis for the collated network were compared to the results obtained by averaging the communicabilities of the nodes over all 12 individual networks. The comparison indicated that the individual networks were rather limited sub-sets of the collated network, with many of the concepts poorly substantiated. However, every single network contained some essential key concepts which the students had managed to substantiate well. Most often these concepts were found to be associated with magnetostatics. It appears that students relied on magnetostatics as mediating module of concepts which link electrostatics concepts to electrodynamics, while less information is channeled directly from electrostatics to electrodynamics. This feature may be partly related to the structure of the task, but may also emerge partly from the recency of the learned magnetostatics in comparison to electrostatics. Magnetostatics having a special role is similarly seen in a different kind of analysis, suggesting that concepts of magnetism have for students an important role in organising their knowledge of electricity and magnetism around that mediating role (Koponen and Nousiainen 2013).

In several recent studies it has been claimed that experts’ knowledge builds around interrelated set of concepts connected by their relational structure (Goldwater and Schalk 2016). This is in contrast to views, where semantic connections and feature based concepts are in focus, where the concepts (or terms corresponding them) tend to form clusters, connections are close and concepts within clusters share more similarities than concepts between clusters (Chi and Ohlsson 2005; Kemp and Tenenbaum 2008). According the relational view on concepts, structure of experts’ knowledge is characterized by distant and complex conceptual connections, while novices’ knowledge often consists of concepts closely related but remains also shallower in comparison to experts’ knowledge. On the other hand, this difference reflects not only the distinction between feature based concepts and relational concepts but also how novices and experts can access and utilize such knowledge (Lachner and Nückles 2015). Here, the key concepts with high communicability are abstract, widely applicable and independent of specific situation, for example very general field concepts. Another set of concepts is the set with high local clustering coefficients. These concepts and conceptual elements are specific derived models or specific examples, or auxiliary concepts connected with them. They are not key concepts but auxiliary and strongly context specific concepts and conceptual elements and have a role in supporting or augmenting the theoretical skeletal structure formed by the abstract theoretical concepts with high global connectivity.

Interestingly, by collecting all the best substantiated nodes and connections contained in the 12 individual concepts maps into a collated network, a highly satisfactory concept network emerges with a rich collection of key concepts and with a strong epistemic substantiation. However, individual networks have quite a lot variability in number of nodes, links and in selection of nodes. This is in agreement with findings that also individual semantic networks have large variability and aggregated network based on them may be very different from individual networks (Morais et al. 2013). On the other hand, the results suggests that in the group of 12 students, a substantial amount of relevant and well-substantiated knowledge was harboured, but with each individual student possessing only a fraction of the knowledge potentially available at the group level. If this unshared knowledge within the group could be somehow better shared, it would provide great potential for successful peer-to-peer learning.

In summary, we have shown how students’ knowledge of the relational knowledge of scientific concepts can be analysed and approached by utilising the appropriate network methods. We believe that the results presented here could not be obtained by any of the traditional methods for analysing concept maps. Moreover, the present method reduces the interpretative component of the analysis to a minimum (only the recognition and evaluation of epistemic levels is needed), overcomes the limits of local analysis, and replaces the awkward visual inspection of global analysis methods with quantitative operationalisations of the global features of interest for examining students’ declarative knowledge and the quality of its substantiation.