DART Blog » European Commission http://dart.blogs.ulcc.ac.uk Blogging about digital archives, scholarship & repositories Tue, 24 Nov 2015 10:40:45 +0000 en-GB hourly 1 http://wordpress.org/?v=4.2.5 DA visits CERN http://dart.blogs.ulcc.ac.uk/2011/06/22/da-visits-cern/ http://dart.blogs.ulcc.ac.uk/2011/06/22/da-visits-cern/#comments Wed, 22 Jun 2011 15:41:57 +0000 http://dablog.ulcc.ac.uk/?p=1540 Last week I was in a very welcoming Geneva, exactly in the European Organization for Nuclear Research (CERN) to meet other partners working on BlogForever and to have several Invenio workshops. I felt very lucky to be in the hub of such an organization and to see how many young international students are getting the […]

The post DA visits CERN appeared first on DART Blog.

]]>
CERN Computer Centre

CERN Computer Centre

CERN Computer Racks

Last week I was in a very welcoming Geneva, exactly in the European Organization for Nuclear Research (CERN) to meet other partners working on BlogForever and to have several Invenio workshops. I felt very lucky to be in the hub of such an organization and to see how many young international students are getting the opportunity to be in the forefront of high physics research.

The Globe at CERN

The Globe at CERN

CERN is home of the world’s biggest and most powerful particle accelerator, the Large Hadron Collider (LHC). This machine is installed in a 27 km circumference tunnel. The LHC records around 15 petabytes per year. All the data is stored in their vast computer centre, where open access and sharing has been the driving principle since their foundation in 1954 and an inspirational environment for the Web to be born there.

Invenio’s workshops showed us that their electronic document management system is robust and versatile, targets the management of more than 1.2 million documents and it can be used in 19 different languages. Its content is clean and complete. In just their High Energy Physics domain, they have around 700 collections and approximately 20K queries a day. As well Invenio is used for special programs like the UNESCO funded digital repos in Africa and EU funded projects like D4Science and OpenAIRE.

Silvia at CERN

Me at CERN

In the case of BlogForever and Invenio, plenty of work to be done by the Invenio Team at the User and Document Service Group. At the moment, they have more than 30 readily available Python modules that can be adapted to the case of preserving huge amount of blogs. From the point of view of my work with repositories as part of the Digital Archives and Repositories Team at ULCC, I was inspired by Invenio’s advance search engines; indexing and ranking methods.

In a more personal level, if you are ever crossing the border between France and Switzerland near Geneva, get Tram 18 and hop off at CERN to see their Microcosm and Globe exhibitions.

The post DA visits CERN appeared first on DART Blog.

]]>
http://dart.blogs.ulcc.ac.uk/2011/06/22/da-visits-cern/feed/ 0
BlogForever: Thoughts about blog data and metadata http://dart.blogs.ulcc.ac.uk/2011/04/25/thoughts-about-blog-data-and-metadata/ http://dart.blogs.ulcc.ac.uk/2011/04/25/thoughts-about-blog-data-and-metadata/#comments Mon, 25 Apr 2011 20:39:18 +0000 http://blogforever.eu/?p=487 During the ArchivePress project at ULCC, we briefly considered the data and metadata generally made available with blogs and blog posts. As ArchivePress focused on the representations of blogs in newsfeeds, we examined the metadata that is generated in common, and exposed in the newsfeeds of three of the most common blog platforms, WordPress, Blogger [...]

The post BlogForever: Thoughts about blog data and metadata appeared first on DART Blog.

]]>
From the BlogForever blog.

During the ArchivePress project at ULCC, we briefly considered the data and metadata generally made available with blogs and blog posts. As ArchivePress focused on the representations of blogs in newsfeeds, we examined the metadata that is generated in common, and exposed in the newsfeeds of three of the most common blog platforms, WordPress, Blogger and TypePad. Blogger and Typepad prefer the Atom newsfeed format; WordPress (particularly WordPress.com) prefers RSS (though it can be made to publish Atom feeds too). This analysis was done, about a year ago, things may have changed, but here is a summary of what we found.

For each Blog, the following core information is available in the feeds:

WordPress (RSS) Blogger (Atom) Typepad (Atom)
Feed Unique ID NA feed/id feed/id
Blog URL rss/channel/link feed/link@rel=”alternate” feed/link@rel=”alternate”
Blog Title rss/channel/title feed/title feed/title
Blog Description rss/channel/description feed/subtitle feed/subtitle
Date of last update rss/channel/lastBuildDate feed/updated feed/updated
Generating software rss/channel/generator feed/generator feed/generator

For each Post, we established that the following core information is available in the newsfeeds:

WordPress (RSS) Blogger (Atom) Typepad (Atom)
Post Unique ID rss/channel/item/guid@isPermaLink feed/entry/id feed/entry/id
Post Title rss/channel/item/title feed/entry/title feed/entry/title
Post Summary rss/channel/item/description NA feed/entry/summary
Post URL rss/channel/item/link feed/entry/link@rel=”alternate” feed/entry/link@rel=”alternate”
Date of publication rss/channel/item/pubDate feed/entry/published feed/entry/published
Date of last update NA feed/entry/updated feed/entry/updated
Post Author rss/channel/item/dc:creator

rss/xmlns:dc

feed/entry/author/name feed/entry/author/name
Post Category rss/channel/item/category feed/entry/category@term feed/entry/category@term
Post Content rss/channel/item/content:encoded

rss/xmlns:content

feed/entry/content

feed/entry/content

Post Comments rss/channel/item/comments feed/entry/link@rel=”replies” feed/entry/link@rel=”replies”
Post Comments Feed rss/channel/item/wfw:commentRss NA

NA

One interesting point we noted was that neither Blogger nor Typepad published a link to a Comments Feed for each post. This made our work on ArchivePress more difficult since it was predicated on being able to easily identify the Comments feed for each post, and harvest new Comments as they were published. Obviously for blogs generated other than by WordPress, this was not going to be so easy. (Our ace developer Emanuele found some workarounds, but that’s another story.)

I think this offers us an interesting overview of the core of standard, structured blog data and metadata, in three of the leading blog platforms. This is the data structure and metadata profile that is maintained in blog databases, in one of its native forms, and I’d expect it to be present in all blog platforms, since it arguably represents the essence of blogs. I hope this will be useful background when considering the core models for data and metadata handling that will be developed for BlogForever.

The post BlogForever: Thoughts about blog data and metadata appeared first on DART Blog.

]]>
http://dart.blogs.ulcc.ac.uk/2011/04/25/thoughts-about-blog-data-and-metadata/feed/ 0
BlogForever: Nominate blogs for our survey http://dart.blogs.ulcc.ac.uk/2011/04/20/nominate-blogs-for-our-survey/ http://dart.blogs.ulcc.ac.uk/2011/04/20/nominate-blogs-for-our-survey/#comments Wed, 20 Apr 2011 07:59:46 +0000 http://blogforever.eu/?p=443 Is there a particular blog or blogger you would like to see included in the BlogForever survey? We invite you to use this form to nominate them, and we will try to ensure that the blog is reviewed or the blogger contacted to participate in our survey. Tweet

The post BlogForever: Nominate blogs for our survey appeared first on DART Blog.

]]>
From the BlogForever blog.

Is there a particular blog or blogger you would like to see included in the BlogForever survey? We invite you to use this form to nominate them, and we will try to ensure that the blog is reviewed or the blogger contacted to participate in our survey.

Get a better browser!

The post BlogForever: Nominate blogs for our survey appeared first on DART Blog.

]]>
http://dart.blogs.ulcc.ac.uk/2011/04/20/nominate-blogs-for-our-survey/feed/ 0
BlogForever: Asynchronicities in blog structure http://dart.blogs.ulcc.ac.uk/2011/04/11/asynchronicities-in-blog-structure/ http://dart.blogs.ulcc.ac.uk/2011/04/11/asynchronicities-in-blog-structure/#comments Mon, 11 Apr 2011 15:56:04 +0000 http://blogforever.eu/?p=430 At an atomic level, a “blog” comprises “blog posts”, which are continually added to the blog corpus: that is the dynamic essence of a blog, and distinguishes it from old-fashioned, largely static Websites and hypertexts in which little content changed between major update iterations, which process was probably more akin to “publishing a new edition” [...]

The post BlogForever: Asynchronicities in blog structure appeared first on DART Blog.

]]>
From the BlogForever blog.

At an atomic level, a “blog” comprises “blog posts”, which are continually added to the blog corpus: that is the dynamic essence of a blog, and distinguishes it from old-fashioned, largely static Websites and hypertexts in which little content changed between major update iterations, which process was probably more akin to “publishing a new edition” in the world of non-digital publications.

The blog also displays, as part of its frame, other graphical and functional elements (sidebars, widgets, “blogrolls”, etc) which may themselves contain dynamically updated, constantly changing information. These can be added, removed, amended and rearranged at will by the blog author/editor. Blog posts that were “published” in the context of one set of framing elements, will persist through subsequent versions of that framework.

Similarly with design (layout, colours, mastheads, etc), though the persistence tends to be longer, the informal nature of blogs means that these may be easily changed by the blog editor/author, and are thus more volatile than a typical “corporate” website. Again, blog posts may persist, unchanged in themselves, through many iterations of the blog site design and layout.

blogatoms 300x223 Asynchronicities in blog structure

A simple view of blog elements and their temporal relationship

 

This very simplified visualisations suggests where we might start conceptualising key elements of a blog. It indicates that they iterate over time, but in the cases of Design, Posts and Widgets (as we’ll call them for brevity), according to independent schedules. While Posts and Comments persist in the online view of a blog, designs and widget arrangements are overwritten.

With my earlier ArchivePress project we deliberately overlooked preservation of the blog’s framing elements, and (given the much smaller scope of that project) established an acceptable rationale for doing so. The challenge for BlogForever is to find a solution to  precisely these issues. Unless we were simply to adopt the snapshot approach of Heritrix-based web archiving initiatives (e.g. Wayback/archive.org, UK Web Archive), we need to ensure the BlogForever repository supports a degree of granularity that can capture, describe and preserve atomic blog objects in a way that reflects the particular interdependencies, in order to understand and preserve them authentically, and permit the many possible authentic and valid “time slice” views and analyses that users of the archive will need.

(I appreciate, by the way that these objects themselves are compound objects, so not strictly “atomic”: but the same is also true of atoms, as our CERN colleagues can attest!)

The post BlogForever: Asynchronicities in blog structure appeared first on DART Blog.

]]>
http://dart.blogs.ulcc.ac.uk/2011/04/11/asynchronicities-in-blog-structure/feed/ 0
http://dart.blogs.ulcc.ac.uk/tag/european-commission/feed/
dart.blogs.ulcc.ac.uk
crawl=CC-MAIN-2015-48
24108
application/http; msgtype=response
sha1:EAOAIGEQQHPCVTWNSFCTNRJK57DYHJHL
<urn:uuid:4bc77e04-e46f-4a2c-8078-71e2ce46eb9b>
2015-11-27T13:33:40Z
128.86.132.238
sha1:73WQDMTPMY7ZZA5DGXJWZHGS3OXWAQXK
<urn:uuid:5bab0b44-2943-4125-8dd5-113cd9736e49>
http://dart.blogs.ulcc.ac.uk/tag/european-commission/feed/
length
response
<urn:uuid:c6682faa-1a21-4ddd-b3ee-94dc910b0699>
robots: classic
hostname: ip-10-71-132-137.ec2.internal
software: Nutch 1.6 (CC)/CC WarcExport 1.0
isPartOf: CC-MAIN-2015-48
operator: CommonCrawl Admin
description: Wide crawl of the web for Nov 2015
publisher: CommonCrawl
format: WARC File Format 1.0
conformsTo: http://bibnum.bnf.fr/WARC/WARC_ISO_28500_version1_latestdraft.pdf