Difference between revisions of "DiscoverEd Metadata"

From Creative Commons
Jump to: navigation, search
Line 1: Line 1:
{{lowercase}}
 
 
 
== Overview ==
 
== Overview ==
  
This document outlines some of the metadata ccLearn is utilizing for the [[DiscoverEd]] project.  These metadata are collected from embedded [[RDFa]] at crawl time.  We also look for these metadata in feeds (RSS, Atom) and OAI-PMH harvests, although these are considered as a way to bootstrap the metadata corpus.  While the metadata store may include additional information from resources, these fields are exposed by default in the search results.
+
This document outlines some of the metadata utilized by [[DiscoverEd]].  These metadata are collected from embedded [[RDFa]] at crawl time.  We also look for these metadata in feeds (RSS, Atom) and OAI-PMH harvests, although these are considered as a way to bootstrap the metadata corpus.  While the metadata store may include additional information from resources, these fields are exposed by default in the search results.
  
 
* '''Title''' (<code>[http://dublincore.org/documents/dcmi-terms/#terms-title DCT:title]</code>): A brief descriptive title for the resource.
 
* '''Title''' (<code>[http://dublincore.org/documents/dcmi-terms/#terms-title DCT:title]</code>): A brief descriptive title for the resource.

Revision as of 16:58, 28 May 2010

Overview

This document outlines some of the metadata utilized by DiscoverEd. These metadata are collected from embedded RDFa at crawl time. We also look for these metadata in feeds (RSS, Atom) and OAI-PMH harvests, although these are considered as a way to bootstrap the metadata corpus. While the metadata store may include additional information from resources, these fields are exposed by default in the search results.

  • Title (DCT:title): A brief descriptive title for the resource.
  • Summary (DCT:description): A relatively short summary or synopsis of the resource.
  • License (DCT:license, cc:license, xhtml:license): The URL of the work's license; e.g., http://creativecommons.org/licenses/by/3.0/.
  • Education level (DCT:educationLevel): What grade(s) or age-level(s) this material is suitable for.
  • Language (xml:lang, DCT:language): The language(s) of the referenced resource (not of your site).
  • Subject (DCT:subject): The subject(s) of the resource; e.g., math.

Our preferred and suggested metadata encoding/transport is XHTML+RDFa. We believe this has the broadest possible exposure for current and future software agents.

Vocabulary

Specifying Subject

The subject refers to the actual content in the resource; i.e., what is this resource about? For many resources, more than one subject will be necessary; in this case, specify multiple subject elements. We ask that you try to limit the contents of the subject to only those subjects that are objectively reflective of the entire resource. Other types of categories (opinions, metrics, etc) may have other vocabularies available which are more appropriate.

Specifying Education level

The education level should indicate all levels (student ages) for which the resource is deemed appropriate. The education level should be labeled using the DCT:educationLevel term.

Though we will accept any descriptions that seem appropriate to you, please consider using one of the following schemas:

  • primary, secondary, tertiary, adult;
  • K,1,2,3,...,20 (where the number refers to the actual grade-level).

You may include equivalent terms as well by specifying more than one value for DCT:educationLevel. For example, you might include a DCT:educationLevel for 9, 10, and secondary.

Specifying Language

When specifying the language for a resource, the value should be specified as described by RFC-4646. For example, en for English. To distinguish English (United States) from English (United Kindgom), the language would be specified as en-US and en-GB, respectively.

In an Atom 1.0 feed, the language is specified as the xml:lang attribute of the content element. Multiple languages in a single entry is not supported.

Embedding license data

The resource license should be communicated using the stable URL of the license. For example, http://creativecommons.org/licenses/by/3.0/.

See the CC with syndication formats documentation for more information on including this is a bootstrap feed.

Examples

[X]HTML + RDFa

The following is an example of how a resource at http://ocw.example.org/math/101 could be annotated with machine-readable metadata, including license and attribution information. This is our preferred manner for encoding this information as it exposes the metadata to a much wider range of clients.

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML+RDFa 1.0//EN" "http://www.w3.org/MarkUp/DTD/xhtml-rdfa-1.dtd">
<html xmlns="http://www.w3.org/1999/xhtml/"
      xmlns:dc="http://purl.org/dc/terms/"
      xmlns:cc="http://creativecommons.org/ns#">
  <head>
   <title>OER Site</title>
  </head>

  <body>
     <h1 property="dc:title">Math 101</h1>
     <h2>by <a href="http://example.org/~johnq" property="dc:author cc:attributionName" rel="cc:attributionURL">John Q. Public</a></h2>
     <p property="dc:description">Basic mathematics for 5th graders</p>
     <p>Subjects: <span property="dc:subject">Math</span></p>
     <p>Grade level: <span property="dc:educationLevel">5</span></p>
     <p>Language: <span property="dc:language" content="en">English</span></p>
     <p>License: <a href="http://creativecommons.org/by/3.0/" rel="license">Attribution 3.0</a></p>

     <p>Lorem ipsum, etc, etc.</p>

  </body>
</html>

If a site aggregates resources such that the metadata appear on a page other than the actual resource, the about attribute can be used to indicate that the metadata are about a different resource. For example, the following page could be published at http://commons.oer.example.org/math/101 and still refer to the same resource as the previous example:


<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML+RDFa 1.0//EN" "http://www.w3.org/MarkUp/DTD/xhtml-rdfa-1.dtd">
<html xmlns="http://www.w3.org/1999/xhtml/"
      xmlns:dc="http://purl.org/dc/terms/">
  <head>
   <title>OER Site</title>
  </head>

  <body>
     <div about="http://ocw.example.org/math/101">
       <h1 property="dc:title">Math 101</h1>
       <h2>by <span property="dc:author">John Q. Public</span></h2>
       <p property="dc:description">Basic mathematics for 5th graders</p>
       <p>Subjects: <span property="dc:subject">Math</span></p>
       <p>Grade level: <span property="dc:educationLevel">5</span></p>
       <p>Language: <span property="dc:language" content="en">English</span></p>
       <p>License: <a href="http://creativecommons.org/by/3.0/" rel="license">Attribution 3.0</a></p>
     </div>

     <p>Lorem ipsum, etc, etc.</p>

  </body>
</html>

Atom 1.0 Example

Here is a sample, one entry Atom 1.0 feed which implements the guidelines above. Note that inclusion of additional metadata in the feed is optional and considered inferior to inclusion with the resource using RDFa.

<feed xmlns="http://www.w3.org/2005/Atom">
  <id>http://oersite.example.org/cc/</id>
  <title>OER Aggregation Web Site</title>
  <updated>2008-01-16T12:00:00Z</updated>
  <link rel="self" href="http://oersite.example.org/cc/atom.xml" type="application/atom+xml" />
  <author>
    <name>John Q. Public</name>
    <email>webmaster@oersite.org</email>
  </author>
  <entry>
    <id>tag:ocw.org,2007-10-15:/math/101</id>
    <updated>2007-10-15T12:00:00Z</updated>
    <link href="http://ocw.example.org/math/101" />
    <title>Math 101</title>
    <summary>Basic mathematics for 5th graders</summary>
    <link rel="license" href="http://creativecommons.org/licenses/by/3.0/" />
    <category term="dc:subject:Math" />
    <category term="dc:educationLevel:5" />
    <content type="xhtml" xml:lang="en">The content</content>
  </entry>
</feed>
https://wiki.creativecommons.org/index.php?title=DiscoverEd_Metadata&diff=34870&oldid=34867
wiki.creativecommons.org
crawl=CC-MAIN-2022-33
40526
application/http; msgtype=response
sha1:TVONHA7QAZSJ4SNP5H672NZISXTR7WJO
<urn:uuid:ead5dc3a-f8b4-4d10-915e-ad410f0986b0>
2022-08-20T02:34:18Z
104.20.150.16
text/html
sha1:SH7HLF5B4HDAL37J4OMPNBMPTBYKDXSZ
<urn:uuid:698ffef2-f4da-4476-84e0-96da6df3085e>
https://wiki.creativecommons.org/index.php?title=DiscoverEd_Metadata&diff=34870&oldid=34867
response
<urn:uuid:1b7f0707-cd64-4a8d-807f-8d12e7910acc>
isPartOf: CC-MAIN-2022-33
publisher: Common Crawl
description: Wide crawl of the web for August 2022
operator: Common Crawl Admin (info@commoncrawl.org)
hostname: ip-10-67-67-222
software: Apache Nutch 1.18 (modified, https://github.com/commoncrawl/nutch/)
robots: checked via crawler-commons 1.4-SNAPSHOT (https://github.com/crawler-commons/crawler-commons)
format: WARC File Format 1.1
conformsTo: https://iipc.github.io/warc-specifications/specifications/warc-format/warc-1.1/