vol. 16 no. 3, September, 2011
vol. 16 no. 3, September, 2011 | ||||
Metadata can support multiple roles in the provision of e-learning, including the description, management, retrieval and re-usability of learning objects and learning designs, as well as the delivery of on-demand and personalised learning experiences to users. For this purpose, a variety of educational or learning object metadata standards and schemas have been developed, including the Dublin Core Education (DCEd) (Mason and Sutton 2000), the IEEE learning object metadata standard (IEEE... 2002) (hereafter, 'the IEEE standard'), and the ARIADNE metadata schema (Najjar et al. 2003). These standards and schemas cover many of the educational, technical and content-related characteristics of learning objects or learning content in general. Apart from an interest in standardisation and metadata schema development, researchers have also investigated the interoperability and technical issues associated with the implementation of learning object metadata in learning object repositories and virtual learning environments (Sampson et al. 2002) as well as the user aspects that should underpin metadata presentation in the search result interface of these systems (Balatsoukas et al. 2007; Liddy et al. 2003).
In the United Kingdom, the use of learning object metadata, such as the IEEE's standard, is an important basis for the successful implementation of e-learning systems in higher and further education. For example, the e-learning programme of the Joint Information Systems Committee has funded several UK projects on e-learning, some of which have investigated the creation and metadata annotation of learning objects in higher and further education, such as the RELOAD project, the creation of auto-generated learning object metadata, such as the RepoMMan project, and the establishment of learning object metadata creation workflows, such as the Jorum project. Research, however, is still in progress on the development of a cross-institutional and UK-wide metadata standard based on the IEEE standard (for example, the UK learning object metadata core is under development by the Special Interest Group on Metadata and Repositories at the Centre for Educational Technology and Interoperability Standards.
Although the IEEE standard has been an important component of all these efforts, little is known about the level of its uptake and use in UK higher education. For example, there are no studies investigating the level of application of metadata elements of the standard for the description of learning objects in different disciplines. It is anticipated that the investigation of differences in the application of the standard could enhance our understanding about the use of this type of metadata and stimulate further research about the reasons for these differences. Therefore, the aim of this study was to examine the effects, if any, of discipline (sciences and engineering versus humanities and social sciences) on the application of the metadata elements for the description of learning objects in the UK's Jorum learning object repository.
The paper is structured as follows. In the next section, a literature review is presented that elaborates on a definition of the concept of learning object metadata, reviews the IEEE standard and focuses on previous studies that investigated the level of application and uptake of this standard at both a national and international level. The methods developed to address the aim of this study are then presented, followed by the results of this study and the final two sections provide a discussion and some conclusions about the application of learning object metadata.
There is no formal definition of learning object metadata or metadata for learning objects. Some authors, including Karampiperis and Sampson (2003), argue that metadata for learning objects should support the retrieval of learning objects 'in an educationally efficient and effective way'. Based on Sicilia's (2005) concept of the dual role of metadata as referential and purposeful, it can be assumed that educational efficiency should guide the referential function of learning object metadata (for example, efficiency in terms of cost, time, human resources and technical infrastructure employed for the description (or referencing) of learning objects), while educational effectiveness should guide the purposeful and functional role of metadata (for example, for searching, evaluating, accessing, preserving and using learning objects). By associating this assumption with the general definition of metadata used by Greenberg (2005) as 'structured data about an object that supports functions associated with the designated object', a new definition for learning object metadata can be declared as:
Structured metadata used for the efficient description of learning objects and the effective support of educational functions related to the described learning objects.
In the context of this paper the term learning object is used to denote an electronic resource created from smaller components (such as data and information objects) and supporting one or more learning objectives (Balatsoukas et al. 2008). A learning object can be re-used in multiple educational contexts as part of larger modules and courses (Wiley 2000; Polsani 2003).
The draft IEEE standard (IEEE 1484.12.1-2002) proposes the structure and semantics of metadata elements used for the description of learning objects. The IEEE Learning Technology Standards Committee has adopted a broad definition of the kind of learning objects that the LOM standard tends to describe, as: 'any entity, digital or non-digital, that may be used for learning education and training' (IEEE Learning Technology Standards Committee 2002: 6). The standard describes various characteristics of a learning object by defining approximately eighty metadata elements, grouped under nine general categories. Table 1 presents a description of the nine categories. For example, the General metadata category includes component elements, such as Identifier, Title, Language, Description, Keywords, Coverage, Structure and Aggregation Level metadata. Some of these elements may also include further sub-component metadata elements. For example, the component element Identifier of the General metadata category, includes the Catalog and Entry sub-components. Data elements that include component data elements are named as aggregate data elements, while leaf data elements are named as simple data elements (IEEE Learning Technology Standards Committee 2002: 7). For some data elements vocabularies are defined by the standard, which are useful for assigning values to the metadata elements. All elements are optional, thus a basic core set of metadata elements is not specified. Furthermore, the standard does not provide any recommendations regarding the metadata creation process or the storage and retrieval of metadata within learning object repositories.
The syntax of metadata elements is based on the W3C XML Schema. Some of the benefits related to the use of this particular technology include:
| Metadata category | Description of contents |
|---|---|
| 1. General | Descriptive information of the learning object as a whole, such as: identifier, title, language, description, keyword, coverage, structure, aggregation level. |
| 2. Life-cycle | Elements related to the creation or revision history of the learning object as well as information about those who have contributed to the development, creation, revision of the learning object. |
| 3. Meta-metadata | Information regarding the creation of the metadata record. |
| 4. Technical | The category consists of elements that describe the technical characteristics of the learning object such as the format, size, location and technical requirements. |
| 5. Educational | The educational category aggregates elements regarding pedagogical and educational information about the use of a learning object. Such elements include: Interactivity type, Learning resource type, Interactivity level, Semantic density, Audience or Intended end user role, Context, Typical age range, Difficulty, Typical learning time, Description, Language. |
| 6. Rights | The rights category includes information regarding the intellectual property rights and conditions of use of the learning object. |
| 7. Relation | The relation category presents information about the relationship of the described learning object with other objects. |
| 8. Annotation | The annotation category provides a comment regarding the use of learning objects. |
| 9. Classification | The classification category classifies the content of the learning object based on an appropriate classification system. |
The learning object metadata standard formed the basis for the development of many nation-wide and project-specific application profiles. Factors that influenced the wide acceptance of this standard include:
Some of the most prominent application profiles include the CanCore project in Canada, the UK Learning Object Metadata Core in the UK and the SCORM application profile developed by the US Advanced Distributed Learning project. These profiles adhere to the basic semantic structure of the original standard but contextualise its use by proposing a basic core set of mandatory elements, providing guidelines for using the metadata elements as well as presenting alternative vocabularies for populating metadata elements with values.
The UK Learning Object Metadata Core (hereafter, the "UK Core") is based on a revision of the former UK Common Metadata Framework. The latest version of the Core is the draft version 0.2 (Campbell 2004a), but work is in progress for version 0.3 (Campbell 2004b). The UK Core is an application profile of the IEEE standard. It does not propose new elements but specifies a basic minimum set of mandatory elements and includes recommendations and guidelines about the use of these elements in UK higher and further education. In particular, Working draft 0.3 defines twenty-seven data elements as mandatory (Campbell 2004b). The list of mandatory elements, however, should not be perceived as a definite or ultimate list of all elements that might be useful for storing, retrieving and evaluating learning resources and learning objects.
It is worth noting that although the purpose of the UK Core is to support the description of learning objects, elements related uniquely to the structure, granularity, pedagogical context and use of a learning object have not been defined as mandatory elements. The rationale behind this decision was that these elements and their associated vocabularies have not yet been widely implemented and their use is not well understood. The rest of the metadata elements have been defined as either Recommended or Optional. These are: all metadata elements from the Education, Relation, Annotation and Classification metadata categories as well as the Coverage, Keyword, Structure and Aggregation level metadata elements from the General metadata category (see Table 1). Finally, the UK Core application profile has been used as a template for the development of other UK-based metadata profiles, such as the Jorum profile.
Although metadata standardisation progresses and many communities develop their own metadata schemas and application profiles, the level of uptake of learning object metadata has not been widely investigated. One of the early surveys that examined the level of application of learning object metadata standards in UK higher education revealed that metadata implementation has been monopolised by two standards: the Dublin Core Metadata Elements Set and the IEEE standard (Currier and Campbell 2005).
At the individual metadata element level, a survey conducted by the Centre for Educational Technology and Interoperability Standards (CETIS) (2002) in the UK and an international survey by CanCore (the Canadian learning resource metadata initiative) (Friesen and Nirhamo 2003; Friesen 2004) revealed an under-utilisation of metadata elements applied for the description of the educational, technical, structural and aggregation level characteristics of learning objects. Both surveys were focused on the use of the IEEE standard. The survey by CETIS was informal and aimed at the identification of useful case studies for further investigation. Thus the survey did not provide a clear answer about the implementation of learning object metadata in the UK. The findings of the international survey by CanCore revealed a high level of use among the elements of the General and Classification categories, such as the Title, Format, Language, Identifier, Keyword, Purpose and Taxon path. Other widely implemented elements were: the Role and Entity elements of the Contribute aggregate metadata element of the Life cycle metadata category, the Format element from the Technical category and the Learning resource type element from the Educational category. These metadata elements occurred in more than 50% of the metadata records under investigation. This survey, however, revealed an under-utilisation of metadata elements related to the educational context of a learning object (for example, Typical age range, Context, Typical learning time, Difficulty and Semantic density) as well as elements specifically used to describe the object-oriented or interactive nature of a learning object (such as Aggregation level, Structure and Interactivity level) (Friesen 2004).
The findings of other studies of the application of educational metadata conducted by Godby (2004), Qin and Godby (2003) and Sicilia et al. (2005) were similar. Godby (2004), who tried to update the findings of the CanCore survey, revealed that the majority of metadata elements in the Educational, Technical and Relation categories were among the least applied elements. Godby (2004) concluded that the focus of the description of learning objects was centred on the retrieval and discovery of metadata, ignoring the educational-learning scope of metadata standards such as the IEEE standard. Another study (Qin and Godby 2003), based on a comparison and synthetic analysis of four metadata schemas (the Ecological Metadata Language, the Gateway to Educational Materials metadata schema, IEEE Learning Object Metadata/IMS Learning Resource Metadata and the Training Exchange Definition metadata), concluded that educational metadata were underrepresented. Qin and Godby (2003) argued that there was a lack of metadata elements and vocabularies that expressed different models of learning as well as various types of learning objectives and learning contexts. For this purpose the authors proposed the creation of a learning object ontology with three main classes: learning-models, learning objectives and learning objects. Finally, the study conducted by Sicilia et al. (2005) provided the same conclusions regarding the use of educational metadata in two learning object repositories, the CAREO repository in Canada, and the MERLOT repository in United States. Sicilia et al. (2005) also highlighted the need for the provision of better quality and structured educational metadata.
Although these studies revealed an under-utilisation of Educational, Technical and Aggregation Level and Structure metadata of the IEEE standard, other researchers such as Najjar et al. (2003) provide different conclusions in the case of the ARIADNE Knowledge Pool System. According to Najjar et al., many educational elements were among the most frequently applied metadata elements in the ARIADNE Knowledge Pool System. These were: the Granularity element (91.9%), Didactical context (53.3%), Interactivity level (52.2%), Semantic density (52.4%) and Difficulty level (52.2%).
From these studies, only Sicilia et al. (2005) investigated the effects of discipline on the frequency of application of metadata elements. They compared the application of metadata elements across learning objects from five disciplines within the CAREO learning objects repository. The disciplines included were computer science, education, sciences, accounting, medical science and mathematics. The results revealed that many metadata elements were distributed homogeneously across the disciplines. These were the title, description, keywords, the meta-metadata entity element and the keyword metadata element of the classification category. However, metadata elements relating to the author of the learning object (such as Role, Entity and Date) as well as most elements from the Meta-metadata, Education and Rights categories were more frequently used for the description of learning objects from computer science, mathematics and accounting. Metadata elements from the Annotation and Technical categories were more frequently used for the description of learning objects from sciences, medical science and education.
Although Sicilia et al. (2005) provided some useful insights concerning the application of certain elements, their study is characterized by a focus on the Canadian educational sector (for example, the metadata records under examination were based on CanCore, the Canadian application profile of the IEEE standard), lack of statistical tests for the examination of differences in the application of metadata elements between disciplines, and absence of data about the application of elements between groups of disciplines (e.g., humanities and social science and sciences and engineering). The present paper addresses these gaps by focusing on a UK-based learning object repository and applying statistical tests for the identification of differences, if any, between groups of disciplines in the application of certain metadata elements. In addition, the division of disciplines into two interdisciplinary groups (humanities and social sciences versus sciences and engineering) serves the simple assumption that any differences in the application of learning object metadata elements could be made more clear in the case of groups of disciplines with a different epistemological base, rather than between individual disciplines that are similar in naorganizorganizorganizture. Furthermore, the identification of statistically significant differences at the group level could justify and provide an empirical basis for the initiation of comparative studies between disciplines at the individual- or micro-level. Finally, this study updates the one by Sicilia et al. (2005) and resets the discourse on the application of metadata elements with an emphasis on the UK higher education sector.
Other researchers such as Zhang and Jastram (2006) found differences in the application of metadata elements between professionals from different sectors, including librarians and information technology professionals, as well as between professionals from private and public organizations. In particular, they revealed statistically significant differences in the accuracy and level of application of the keyword and description metadata elements. Information technology professionals tended to apply more metadata elements than librarians. These studies, however, were focused on the metadata (or meta-tags) used in the header of html documents and not on the IEEE standard or any other type of structured learning object metadata schema. Also the researchers investigated differences between groups of professionals by sector rather than by discipline.
To address the aim of this study a survey was conducted that examined the metadata records and surrogates used for the description of learning objects in the Jorum repository. A stratified random sample of 470 records was selected. This sampling technique served the representation in the sample of metadata records of learning objects from all subjects (disciplines) covered by the Jorum repository and facilitated the random selection of cases within each stratum.
In the Jorum repository, learning objects are classified under nineteen subject headings for higher education. These subjects represent the classificatory structure of the Jorum repository. For the needs of this study the subjects were categorized into two interdisciplinary groups: 1. Sciences and engineering, and 2. Humanities and social sciences. The first group represented disciplines from natural sciences and engineering, such as physics and mathematical sciences, medicine, biology, engineering and technology. The second group included disciplines of humanistic studies as well as disciplines related to the study of society, social phenomena and relations, including economics and business administration. This categorization provides a realistic, flexible and intuitive way of grouping individual disciplines which follows the traditional categorization of academic departments in faculties. The two interdisciplinary categories were:
A. Sciences and engineering
B. Humanities and social sciences
Each of the nineteen subject headings represented a stratum (i.e. a subject area). Within each stratum a total number of thirty learning object metadata records were randomly selected for examination. It is worth mentioning, however, that a few subject headings included fewer than thirty learning objects. In these cases all metadata records in the stratum were selected for inclusion in the sample. In a typical Jorum learning object metadata record all elements are hierarchically listed following a parent-child relationship. Metadata elements are grouped in one or more categories of the IEEE standard (see Table 1). Some additional metadata categories, such as digital rights management, are also provided.
The metadata categories under investigation were: General, Life-cycle, Meta-metadata, Technical, Educational, Rights, Relation, Annotation and Classification. In addition, the following Recommended and Optional metadata elements of the IEEE standard were selected for further investigation:
The selection of the particular set of metadata elements was based on the results of previous surveys according to which these were among the least applied metadata elements (Friesen 2004; Godby 2004). Because of this it was decided to investigate further the impact of the two interdisciplinary categories on the application of these elements in the Jorum repository.
Data collected also included the frequency of application of each metadata category as well as the total number of times a specific element appeared in the sample of records. The data analysis involved the estimation of the frequencies as well as non parametric Chi-squared tests for statistical significance. Non-parametric tests were applied to investigate significant differences in the application of metadata categories and elements between disciplines (sciences & engineering versus humanities & social sciences).
The Jorum project (http://www.jorum.ac.uk/) funded by JISC under the X4L programme aimed to develop a national repository of learning objects for staff across the UK higher and further education. Users of the Jorum repository can either use or contribute learning objects and digital learning material. As part of this process, the Jorum team developed a learning object metadata creation workflow with three main roles. These include the Contributor, the Cataloguer and the Reviewer. The Contributor submits some additional learning object metadata after the upload of a learning object. Then the Cataloguer completes the learning object metadata record, and finally, the Reviewer's role is to revise or reject the metadata record. In addition to the human-generated metadata this workflow supports the creation of some automatically-generated metadata (Baird 2006).
For the standardisation of the metadata creation process the Jorum metadata application profile has been developed based on the UK Core version 0.3, which provides a basic minimum set of mandatory metadata elements as well as additional elements that are identified as Recommended or Optional. In addition, the application profile specifies some guidelines and recommendations for the application of these elements and their vocabularies.
It was anticipated that the selection of the sample of metadata records from a nation-wide learning object repository such as Jorum would reflect the cataloguing behaviour of depositors from many UK universities and various disciplines. From a practical point of view, the selection of the particular repository limited the amount of time and resources needed for researchers to obtain a large sample of metadata records.
Although the sample size of this survey consisted of 470 metadata records, 59 records were excluded from the analysis. These records were incomplete and did not provide an accurate view of the application of certain metadata categories and elements in the Jorum repository. Thus, the results reported in this section derive from the analysis of 411 completed metadata records.
Data analysis revealed consistent application of the mandatory metadata categories across the examined records. For example, the General, Life-cycle, Meta-metadata, Technical, Educational, Rights and Classification metadata categories were present in all 411 metadata. This is not the case, however, for the optional Relation and the recommended Annotation categories. In particular, the Relation metadata category was present in fifteen records and the Annotation category in 124 records.
A set of Pearson's Chi-squared tests was applied in the case of the Relation and Annotation metadata categories to investigate statistically significant differences between the two interdisciplinary categories (sciences and engineering versus humanities and social sciences) in the application of the Relation and Annotation metadata categories. The results revealed that the relation between the application of the Relation metadata category and the type of interdisciplinary category was significant: Chi-squared (1, N= 411) = 19.21, p = <0.0005. In particular, the Relation metadata category was more frequently applied in the case of metadata records describing learning objects from a science or engineering discipline. The association, however, was of weak strength (Phi-value = 0.216) thus, the type of discipline for which the metadata record was created for accounted only for 4.7% of the variance in the frequency of use of the Relation metadata element.
Similarly, the relation between the application of the Annotation metadata category and the type of discipline was significant: Chi-squared (1, N= 411) = 35.36, p = <0.0005. In this case, however, the Annotation metadata category tended to be more frequently applied in records that described learning objects from the humanities and social sciences. Again, the association was of weak strength (Phi-value = 0.293) and accounted for 8.6% of the variance in the frequency of use of the Annotation metadata category between the two interdisciplinary categories.
The contingency tables (Tables 2 and 3) present the weak but significant relation between the application of the Relation and Annotation metadata categories and the two interdisciplinary categories. The Relation metadata category was applied in fifteen records of the sciences and engineering group and never applied in humanities and social sciences. The Annotation metadata category was present in nintety-six records from the humanities and social sciences group and in only twenty-eight records from the sciences and engineering group.
| Applied | Not Applied | Row Total | |
|---|---|---|---|
| Sciences and engineering | 15 | 169 | 184 |
| Humanities and social sciences | 0 | 227 | 227 |
| Total | 15 | 396 | 411 |
| Applied | Not Applied | Row Total | |
|---|---|---|---|
| Sciences and engineering | 28 | 156 | 184 |
| Humanities and social sciences | 96 | 131 | 227 |
| Total | 124 | 287 | 411 |
The results showed that almost all Recommended elements of the Jorum metadata application profile were frequently applied to the description of learning objects (Interactivity type, Interactivity level, Learning resource type, Intended end user role, Context, Difficulty and Description) (see Table 4). Most of these elements were present in more than 400 records. This result contradicts previous studies, which revealed an under-utilisation of the particular elements in the creation of learning object records (Friesen 2004). This observation, however, does not hold for the recommended Typical age range element which was present in fewer (361) records.
While all but one of the recommended elements were frequently applied across the sample records, this is not the case for the optional elements which were not frequently applied (see Table 4). In particular, the three elements of the General metadata category (Coverage, Aggregation level and Structure) and the two elements of the Education metadata category (Semantic density and Typical learning time) were applied in less than forty and sixty records respectively. Finally, the Technical requirements element of the Technical metadata category was present in fifty-eight records.
| Recommended elements | Frequency | % |
|---|---|---|
| Interactivity type | 403 | 98.1 |
| Interactivity level | 408 | 99.3 |
| Learning resource type | 408 | 99.3 |
| Intended end user role | 408 | 99.3 |
| Context | 410 | 99.8 |
| Typical age range | 361 | 87.8 |
| Difficulty | 397 | 96.6 |
| Description | 405 | 98.5 |
| Optional elements | Frequency | % |
| Coverage | 21 | 5.1 |
| Aggregation level | 13 | 3.2 |
| Structure | 36 | 8.8 |
| Semantic density | 60 | 14.6 |
| Typical learning time | 58 | 14.1 |
| Technical requirements | 59 | 14.4 |
As shown in Table 5, all Recommended elements under investigation were more frequently applied for the description of learning objects from the humanities and social sciences. A set of Pearson's Chi-squared tests, however, were conducted to examine whether there was a significant impact of the interdisciplinary category on the application of the elements under investigation.
| Type of element | Disciplines | |||
|---|---|---|---|---|
| Humanities and social sciences | Sciences and engineering | |||
| f | % within discipline | f | % within discipline | |
| Interactivity type | 226 | 99.6 | 177 | 96.2 |
| Interactivity level | 227 | 100 | 181 | 98.4 |
| Learning resource type | 227 | 100 | 181 | 98.4 |
| Intended end user role | 226 | 99.1 | 182 | 98.9 |
| Context | 227 | 100 | 183 | 99.5 |
| Typical age range | 200 | 88.1 | 161 | 87.7 |
| Difficulty | 224 | 98.7 | 173 | 94 |
| Description (Educational) | 223 | 98.2 | 182 | 98.9 |
Table 6 presents the results of the Chi-squared tests performed in the case of each Recommended element. The results revealed that there was a relationship between the application of the Interactivity type and Difficulty elements and interdisciplinary category. These elements tended to be more frequently applied for the description of learning objects from humanities and social sciences. It is worth mentioning, however, that the associations observed were of weak strength (Phi-value < 0.2). No significant relationships were observed in the case of the remaining metadata elements.
| Metadata element by discipline type | No. of cases | Value | DF | p value |
|---|---|---|---|---|
| Interactivity type | 411 | N/A | N/A | 0.025* |
| Interactivity level | 411 | N/A | N/A | 0.093** |
| Learning resource type | 411 | N/A | N/A | 0.093** |
| Intended end user role | 411 | N/A | N/A | 0.6** |
| Context | 411 | N/A | N/A | 0.45** |
| Typical age range | 411 | 0.35 | 1 | 0.85 |
| Difficulty | 411 | 6.7 | 1 | 0.01** |
| Description | 411 | N/A | N/A | 0.7* |
| * These cases are significant at the 0.05 (5%) level
according to the Pearson's chi-squared tests. ** Fisher's exact significance (2-sided) test was selected (in each of these cases two cells had and expected count of less than 5). | ||||
Similarly, the Pearson's Chi-squared tests performed in the case of the optional elements revealed significantly higher level of application of the Coverage, Aggregation level, Structure and Semantic density elements for the description of learning objects from the humanities and social sciences (see Table 7). The strength of these relationships, however, was weak (Phi-value <0.2). No significant differences were observed in the case of the Typical learning time and Technical requirements elements. The level of application of these elements tended to be similar across the two subject categories. Finally, Figure 1 provides a graphical representation of the level of application (percentage of application within each interdisciplinary category) of all optional elements of this study between humanities and social sciences and sciences and engineering categories.
| Metadata element by discipline type | No. of cases | Value | DF | p value |
|---|---|---|---|---|
| Aggregation level | 411 | 7.5 | 1 | 0.006* |
| Coverage | 411 | 5.9 | 1 | 0.015* |
| Semantic density | 411 | 9.3 | 1 | 0.002* |
| Structure | 411 | 10.2 | 1 | 0.001* |
| Technical requirements | 411 | 2.35 | 1 | 0.13 |
| Typical learning time | 411 | 0.08 | 1 | 0.8 |
| * These cases are significant at the 0.05 (5%) level according to the Pearson's chi-squared tests. | ||||
In general, the results of this study seem to suggest that the level of obligation (Recommended or Optional) is more a critical factor in the frequency of use of learning object metadata than the disciplinary category to which the metadata elements are assigned. However, in both cases (level of obligation and interdisciplinary category) some exceptions were observed. For example, in the case of the level of obligation, the recommended Typical age range element was not consistently applied for the description of learning objects in both disciplinary categories. This can be justified by the fact that this element is characterised by the inconsistent application of vocabulary values. For example, according to the Jorum metadata application profile, the Typical age range refers to the chronological age of a user group (Stevenson 2005). In practice, however, the assignment of values for the particular element range from chronological values, such as 16+, to values associated to the educational level of the intended end-user group, such as teacher and undergraduate students.
In the case of the effects of disciplinary category on the use of learning object metadata elements, the results showed a consistent use of most Recommended and Optional metadata elements in the disciplines . These include: the General, Life-cycle, Meta-metadata, Technical, Education, Rights and Classification categories, and the Technical requirements, Interactivity level, Typical learning time, Intended end user role, Learning resource type, Context, Typical age range and Description elements. This finding supports the results of a previous study by Sicilia et al. (2005) who found no statistically significant differences in the application of metadata elements across disciplines and also that there is a general subset of metadata elements that is commonly used for the description of learning objects. However, the data analysis was conducted at the level of individual scientific disciplines (Sicilia et al. 2005) and not at the dichotomous and compound level of sciences and engineering versus humanities and social sciences (as in the present study).
Differences in the use of some metadata between the two interdisciplinary categories were observed in the following cases:
These differences represent weak, but statistically significant, associations between specific metadata elements or categories of metadata elements and interdisciplinary groups. Although there is no evidence in the literature about why this phenomenon occurs, the following assumptions can be made:
The future investigation of these assumptions could provide in-depth knowledge about the way learning object metadata elements are applied for the description of learning objects from different disciplines. However, another question arising from this discussion is whether the application (or lack of application) of certain learning object metadata elements can have an impact on the use of learning object repositories by teachers and students from different disciplines. This question is related to the concept of metadata completeness that is defined as 'the extent to which the metadata record of the learning object provides the necessary metadata to properly support a given process or activity' (Sicilia et al. 2005). Users can interact with learning object repositories at different levels, each level representing a different process or activity. These can include:
These processes or activities can facilitate several educational objectives, such as the re-use of learning objects by teachers or the facilitation of learning for students. For example, teachers can select learning objects for inclusion in lessons or whole courses based on Interactivity type or Typical age range. Similarly, students can select learning objects according to their Intended end user role or the Learning resource type. Therefore, it can be assumed that the lack of complete learning object metadata records can impede the efficiency and effectiveness of the learning experience (Morris et al. 2007). For example, Balatsoukas et al. (2007) found that students with a science background (information and computer sciences) were interested in metadata elements about the audience (Intended end user role) and Interactivity when judging the relevance of learning objects. The absence of these metadata elements could frustrate learners and increase their cognitive load.
In the case of the Jorum repository this problem could be more evident for learning objects from sciences and engineering, where metadata elements such as Interactivity type, Annotation, Intended end user role or Learning resource type were applied less frequently and inconsistently when compared to learning objects from humanities and social sciences. The reasons for the low application of these metadata elements can be attributed to their semantic ambiguity and the lack of appropriate and user-centred vocabularies (Campbell 2004b; Godby 2004).
For example, participants in the UK SearchLT project faced problems when using specific metadata elements (from an application profile) for the creation of metadata records for learning objects in engineering (Barker and Barker 2003). These included the Technical requirements element (it was difficult for cataloguers to describe the technical requirements), the Educational elements (which requires specialist knowledge on pedagogy), and the Interactivity type element (which was found confusing). Many of these elements were applied less frequently for the description of learning objects from a science or engineering discipline in the case of the Jorum repository (see Table 5). There is no strong empirical evidence, however, that differences between disciplines (such as diverse teaching and learning traditions, the nature and interactivity of learning objects, and the level of familiarisation and training with learning objects and learning object metadata) account for any inconsistencies in the application of certain learning object metadata elements. Researchers suggest the need for more research in this area (Kay and Lum 2004).
The results of this study showed that an effect of disciplinary category (sciences and engineering and humanities and social sciences) was evident in the frequency of use of only a few metadata elements and categories of metadata elements. The Annotation category tended to be used more frequently for the description of learning objects from humanities and social sciences while the Relation category was more often used in the case of disciplines from sciences and engineering . At the metadata element level, the Interactivity type, Difficulty, Aggregation level, Coverage, Structure and Semantic density elements occurred more frequently in records describing learning objects from the humanities and social sciences. However, the results showed that the majority of metadata elements and categories of metadata under investigation were applied consistently across the two disciplinary groups.
Although the results followed the execution of a sound methodological design, which involved random representation of metadata records from all disciplines included in the Jorum repository as well as objective data collection and analysis though the use of statistical tests, they should be interpreted with caution. For example, the categorization of disciplines as humanities and social sciences and sciences and engineering could have obscured the unique characteristics of individual disciplines. That is, differences in the application of metadata elements could have been observed between disciplines that occurred within the same disciplinary category. Furthermore, this research limited its focus only to certain learning object metadata elements, i.e., most of the elements of the Educational category and a selection of elements which, according to previous international studies, are less frequently used for the description of learning objects.
Setting aside these limitations, the present research (the first of its kind in the UK) provides new knowledge about the level of application and uptake of certain learning object metadata elements in the UK higher education. In addition, the findings reported in this paper can be used as a basis for further investigation. For instance, future research should examine the reasons why specific metadata elements tend to be used more frequently for the description of learning objects in the respective discpline groups. Furthermore, follow-up studies could strengthen the generalisability of the findings reported in this paper by examining the frequency of use of metadata elements across individual disciplines and larger samples of metadata records as well as across other learning object repositories in the UK.
Panos Balatsoukas is a Research Fellow in the Department of Computer and Information Sciences, University of Strathclyde, Glasgow. His research interests include information retrieval, user-centered design of information systems and relevance judgment on the web. He can be contacted at: panos@cis.strath.ac.uk
Ann O'Brien is a Lecturer in the Department of Information Science, Loughborough University, UK. Her research interests include information organization and retrieval, metadata and taxonomies. She can be contacted at: A.O-brien@lboro.ac.uk
Anne Morris is a Professor of Information Science in the Department of Information Science, Loughborough University, UK. Her research interests include e-learning, e-metrics and economic valuation of information services, electronic books, the evaluation of information services, the impact of information technology on end-users, electronic document delivery, library ergonomics including health and safety issues and the strategic management of academic libraries. She can be contacted at: A.Morris@lboro.ac.uk
| Find other papers on this subject | ||
© the authors, 2011. Last updated: 14 July, 2011 |
|
