Open Knowledge International Blog https://blog.okfn.org Fri, 18 Aug 2017 09:56:36 +0000 en-GB hourly 1 114721897 okfnhttps://feedburner.google.com Open Data Conference in Switzerland http://feedproxy.google.com/~r/okfn/~3/fTM37XOF-Bc/ https://blog.okfn.org/2017/08/18/open-data-conference-in-switzerland/#respond Fri, 18 Aug 2017 09:56:36 +0000 https://blog.core.okfn.org/?p=23236 This year’s Opendata.ch/2017, the Open Data Conference in Switzerland, was all about Open Smart Cities, Open Tourism & Transport Data, Open Science & Open Food Data. We learnt how Open Data can be a catalyst of digital transformation and a crucial factor for advancing data quality. We got insights into the role of open data in the daily work of journalists and learnt how open data portals make an important contribution to enable Switzerland to remain a leader and innovator in the digital world.

Over 200 people attended the conference: its’ program was composed of 8 keynotes and parallel afternoon tracks with a total of 18 workshops. A highlight of the conference was the visit of Pavel Richter, CEO of Open Knowledge International. Pavel emphasized the purpose of Open Knowledge International lying in empowering civil society organisation to use open data to improve people’s life, for instance by collaborating with human rights institutions. Recent key arguments for open data being “I can take it and put it somewhere else, in a safer place […] it works as a concept, because the data is not lost, it can be secured and re-used”. The entire Q&A with Pavel Richter and Barnaby Skinner in English is available here:

Another highlight was the closing keynote which was held by the president of the École Polytechnique Fédérale de Lausanne who spoke about “The role of “open” in digital Switzerland” and emphasized that public access to scientific data should be the norm so that the rest of the world can also profit from it. His entire talk is available in English here:

Furthermore we curated the following material for you:

]]>
https://blog.okfn.org/2017/08/18/open-data-conference-in-switzerland/feed/ 0 23236 https://blog.okfn.org/2017/08/18/open-data-conference-in-switzerland/
Open Data Handbook now available in the Nepali Language http://feedproxy.google.com/~r/okfn/~3/K4BQXmnDfqA/ https://blog.okfn.org/2017/08/17/open-data-handbook-is-now-available-in-the-nepali-language/#respond Thu, 17 Aug 2017 11:18:45 +0000 https://blog.core.okfn.org/?p=23227 On 7 August 2017 Open Knowledge Nepal launched the first version of Nepali Open Data Handbook – An introductory guidebook used by governments and civil society organizations around the world as an introduction and blueprint for open data projects. The book was launched by Mr. Krishna Hari Baskota, Chief Information Commissioner of National Information Commission, Dr. Nama Raj Budhathoki, Executive Director of Kathmandu Living Labs and Mr. Nikesh Balami, CEO of Open Knowledge Nepal at the event organized at Moods Lounge, Bluebird Mall, Kathmandu. Around 50 people working in the different sectors of open data attended the launch program.

The Open Data Handbook has been translated into more than 18 languages including Chinese, French, German, Indonesian, Italian, Japanese, Portuguese, Russian, Spanish. Now the Nepali language is also available at www.opendatahandbook.org. At the event a hard copy version of the Open Data Handbook was launched, which included the content from Global Open Data Handbook, Licensing terms from the Open Definition, some notable featured Open Data projects of Nepal and the process of asking information of the Nepal government using the Right To Information Act.

Open Knowledge Nepal believes the Nepali version of the Open Data Handbook will work as a perfect resource for government and civil society organizations (CSOs) to expand their understandings of open data and, ultimately, reap its benefits. Speaking at the event,

Mr. Nikesh Balami, CEO of Open Knowledge Nepal said “I believe that this Nepali version of the Open Data Handbook will help government policymakers, leaders, and citizens understand open data in their native language. It will also be a useful resource for CSOs to use for their own open data awareness programs, as well as data journalists who rely on data openness to report on local stories.” He thanked the volunteer who contributed on the translation, feedback, and review of the Handbook.

Mr. Krishna Hari Baskota, Chief Information Commissioner stressed the need for people in government to understand the value of open data. He also remarked that while the Nepal government is already a treasure trove of data, there is a need for more data to be created and made open. He highlighted the journey traveled by the Nepal Government in the path of open data and motivated youths to join the momentum.

Dr. Nama Raj Budhathoki, Executive Director of Kathmandu Living Labs said, “There should be an equal balance between supply and demand side of data and it’s a perfect time for Nepal to shift from Creation to Use”. Dr. Budhathoki shared his experiences of creating open data with OpenStreetMap and household surveys, and acknowledged the need for use of open data.

Open Knowledge Nepal envisions the impact of the Open Data Handbook to be mainly around the four different themes of open data: improving government, empowering citizens, creating opportunity, and solving public problems. To achieve impact within these different themes, solely having a good supply of data is not enough. We also need to ensure that the demand side is strong by increasing innovation, engagement, and reusability of published data. This Handbook will make it easier for government officials and the citizens of Nepal to learn more about open data in their native language. In doing so, it will help create a balanced environment between the supply and demand side of data, which in the long run will help promote and institutionalize transparency, accountability and citizen engagement in Nepal.

]]>
https://blog.okfn.org/2017/08/17/open-data-handbook-is-now-available-in-the-nepali-language/feed/ 0 23227 https://blog.okfn.org/2017/08/17/open-data-handbook-is-now-available-in-the-nepali-language/
OpenSpending platform update http://feedproxy.google.com/~r/okfn/~3/FVQ5YN9fvos/ https://blog.okfn.org/2017/08/16/openspending-platform-update/#respond Wed, 16 Aug 2017 10:11:13 +0000 https://blog.core.okfn.org/?p=23211 Introduction

OpenSpending is a free, open and global platform to search, visualise, and analyse fiscal data in the public sphere. This week, we soft launched an updated technical platform, with a newly designed landing page. Until now dubbed “OpenSpending Next”, this is a completely new iteration on the previous version of OpenSpending, which has been in use since 2011.

Landing page at https://openspending.org

 

At the core of the updated platform is Fiscal Data Package. This is an open specification for describing and modelling fiscal data, and has been developed in collaboration with GIFT. Fiscal Data Package affords a flexible approach to standardising fiscal data, minimising constraints on publishers and source data via a modelling concept, and enabling progressive enhancement of data description over time. We’ll discuss in more detail below.

From today:

  • Publishers can get started publishing fiscal data with the interactive Packager, and explore the possibilities of the platform’s rich API, advanced visualisations, and options for integration.
  • Hackers can work on a modern stack designed to liberate fiscal data for good! Start with the docs, chat with us, or just start hacking.
  • Civil society can access a powerful suite of visualisation and analysis tools, running on top of a huge database of open fiscal data. Discover facts, generate insights, and develop stories. Talk with us to get started.

All the work that went into this new version of OpenSpending was only made possible by our funders along the way. We want to thank Hewlett, Adessium, GIFT, and the OpenBudgets.eu consortium for helping fund this work.

As this is now completely public, replacing the old OpenSpending platform, we do expect some bugs and issues. If you see anything, please help us by opening a ticket on our issue tracker.

Features

The updated platform has been designed primarily around the concept of centralised data, decentralised views: we aim to create a large, and comprehensive, database of fiscal data, and provide various ways to access that data for others to build localised, context-specific applications on top. The major features of relevance to this approach are described below.

Fiscal Data Package

As mentioned above, Fiscal Data Package affords a flexible approach to standardising fiscal data. Fiscal Data Package is not a prescriptive standard, and imposes no strict requirements on source data files.

Instead, users “map” source data columns to “fiscal concepts”, such as amount, date, functional classification, and so on, so that systems that implement Fiscal Data Package can process a wide variety of sources without requiring change to the source data formats directly.

A minimal Fiscal Data Package only requires mapping an amount and a date concept. There are a range of additional concepts that make fiscal data usable and useful, and we encourage the mapping of these, but do not require them for a valid package.

Based on this general approach to specifying fiscal data with Fiscal Data Package, the updated OpenSpending likewise imposes no strict requirements on naming of columns, or the presence of columns, in the source data. Instead, users (of the graphical user interface, and also of the application programming interfaces) can provide any source data, and iteratively create a model on top of that data that declares the fiscal measures and dimensions.

GUIs

Packager

The Packager is the user-facing app that is used to model source data into Fiscal Data Packages. Using the Packager, users first get structural and schematic validation of the source files, ensuring that data to enter the platform is validly formed, and then they can model the fiscal concepts in the file, in order to publish the data. After initial modelling of data, users can also remodel their data sources for a progressive enhancement approach to improving data added to the platform.

Explorer

The Explorer is the user-facing app for exploration and discovery of data available on the platform.

Viewer

The Viewer is the user-facing app for building visualisations around a dataset, with a range of options, for presentation, and embedding views into 3rd party websites.

DataMine

The DataMine is a custom query interface powered by Re:dash for deep investigative work over the database. We’ve included the DataMine as part of the suite of applications as it has proved incredibly useful when working in conjunction with data journalists and domain experts, and also for doing quick prototype views on the data, without the limits of API access, as one can use SQL directly.

APIs

Datastore

The Datastore is a flat file datastore with source data stored in Fiscal Data Packages, providing direct access to the raw data. All other databases are built from this raw data storage, providing us with a clear mechanism for progressively enhancing the database as a whole, as well as building on this to provide such features directly to users.

Analytics and Search

The Analytics API provides a rich query interface for datasets, and the search API provides exploration and discovery capabilities across the entire database. At present, search only goes over metadata, but we have plans to iterate towards full search over all fiscal data lines.

Data Importers

Data Importers are based on a generic data pipelining framework developed at Open Knowledge International called Data Package Pipelines. Data Importers enable us to do automated ETL to get new data into OpenSpending, including the ability to update data from the source at specified intervals.

We see Data Importers as key functionality of the updated platform, allowing OpenSpending to grow well beyond the one thousand plus datasets that have been uploaded manually over the last five or so years, towards tens of thousands of datasets. A great example of how we’ve put Data Importers to use is in the EU Structural Funds data that is part of the Subsidy Stories project.

Iterations

It is slightly misleading to announce the launch today, when we’ve in fact been using and iterating on OpenSpending Next for almost 2 years. Some highlights from that process that have led to the platform we have today are as follows.

SubsidyStories.eu with Adessium

Adessium provided Open Knowledge International with funding towards fiscal transparency in Europe, which enabled us to build out significant parts of the technical platform, commision work with J++ on Agricultural Subsidies , and, engage in a productive collaboration with Open Knowledge Germany on what became SubsidyStories.eu, which even led to another initiative from Open Knowledge Germany called The Story Hunt.

This work directly contributed to the technical platform by providing an excellent use case for the processing of a large, messy amount of source data into a normalised database for analysis, and doing so while maintaining data provenance and the reproducibility of the process. There is much to do in streamlining this workflow, but the benefits, in terms of new use cases for the data, are extensive.

We are particularly excited by this work, and the potential to continue in this direction, by building out a deep, open database as a potential tool for investigation and telling stories with data.

OpenBudgets.eu via Horizon 2020

As part of the OpenBudgets.eu consortium, we were able to both build out parts of the technical platform, and have a live use case for the modularity of the general architecture we followed. A number of components from the core OpenSpending platform have been deployed into the OpenBudgets.eu platform with little to no modification, and the analytical API from OpenSpending was directly ported to run on top of a triple store implementation of the OpenBudgets.eu data model.

An excellent outcome of this project has been the close and fruitful work with both Open Knowledge Germany and Open Knowledge Greece on technical, community, and journalistic opportunities around OpenSpending, and we plan for continuing such collaborations in the future.

Work on Fiscal Data Package with GIFT

Over three phases of work since 2015 (the third phase is currently running), we’ve been developing Fiscal Data Package as a specification to publish fiscal data against. Over this time, we’ve done extensive testing of the specification against a wide variety of data in the wild, and we are iterating towards a v1 release of the specification later this year.

We’ve also been piloting the specification, and OpenSpending, with national governments. This has enabled extensive testing of both the manual modeling of data to the specification using the OpenSpending Packager, and automated ETL of data into the platform using the Data Package Pipelines framework.

This work has provided the opportunity for direct use by governments of a platform we initially designed with civil society and civic tech actors in mind. We’ve identified difficulties and opportunities in this arena at both the implementation and the specification level, and we look forward to continuing this work and solving use cases for users inside government.

Credits

Many people have been involved in building the updated technical platform. Work started back in 2014 with an initial architectural vision articulated by our peers Tryggvi Björgvinsson and Rufus Pollock. The initial vision was adapted and iterated on by Adam Kariv (Technical Lead) and Sam Smith (UI/X), with Levko Kravets, Vitor Baptista, and Paul Walsh. We reused and enhanced code from Friedrich Lindenberg. Lazaros Ioannidis and Steve Bennett made important contributions to the code and the specification respectively. Diana Krebs, Cecile Le Guen, Vitoria Vlad and Anna Alberts have all contributed with project management, and feature and design input.

What’s next?

There is always more work to do. In terms of technical work, we have a long list of enhancements.
However, while the work we’ve done in the last years has been very collaborative with our specific partners, and always towards identified use cases and user stories in the partnerships we’ve been engaged in, it has not, in general, been community facing. In fact, a noted lack of community engagement goes back to before we started on the new platform we are launching today. This has to change, and it will be an important focus moving forward. Please drop by at our forum for any feedback, questions, and comments.

]]>
https://blog.okfn.org/2017/08/16/openspending-platform-update/feed/ 0 23211 https://blog.okfn.org/2017/08/16/openspending-platform-update/
Using the Global Open Data Index to strengthen open data policies: Best practices from Mexico http://feedproxy.google.com/~r/okfn/~3/8cofIPCGRjA/ https://blog.okfn.org/2017/08/16/using-the-global-open-data-index-to-strengthen-open-data-policies-best-practices-from-mexico/#respond Wed, 16 Aug 2017 09:00:07 +0000 https://blog.core.okfn.org/?p=23200 This is a blog post coauthored with Enrique Zapata, of the Mexican National Digital Strategy.

As part of the last Global Open Data Index (GODI), Open Knowledge International (OKI) decided to have a dialogue phase, where we invited individuals, CSOs, and national governments to exchange different points of view, knowledge about the data and understand data publication in a more useful way.

In this process, we had a number of valuable exchanges that we tried to capture in our report about the state of open government data in 2017, as well as the records in the forum. Additionally, we decided to highlight the dialogue process between the government and civil society in Mexico and their results towards improving data publication in the executive authority, as well as funding to expand this work to other authorities and improve the GODI process. Here is what we learned from the Mexican dialogue:

The submission process

During this stage, GODI tries to directly evaluate how easy it is to find and their data quality in general. To achieve this, civil society and government actors discussed how to best submit and agreed to submit together, based on the actual data availability.

Besides creating an open space to discuss open data in Mexico and agreeing on a joint submission process, this exercise showed some room for improvement in the characteristics that GODI measured in 2016:

  • Open licenses: In Mexico and many other countries, the licenses are linked to datasets through open data platforms. This showed some discrepancies with the sources referenced by the reviewers since the data could be found in different sites where the license application was not clear.
  • Data findability: Most of the requested datasets assess in GODI are the responsibility of the federal government and are available in datos.gob.mx. Nevertheless, the titles to identify the datasets are based on technical regulation needs, which makes it difficult for data users to easily reach the data.
  • Differences of government levels and authorities: GODI assesses national governments but some of these datasets – such as land rights or national laws – are in the hands of other authorities or local governments. This meant that some datasets can’t be published by the federal government since it’s not in their jurisdiction and they can’t make publication of these data mandatory.

Open dialogue and the review process

During the review stage, taking the feedback into account, the Open Data Office of the National Digital Strategy worked on some of them. They summoned a new session with civil society, including representatives from the Open Data Charter and OKI in order to:

  • Agree on the state of the data in Mexico according to GODI characteristics;
  • Show the updates and publication of data requested by GODI;
  • Discuss paths to publish data that is not responsibility of the federal government;
  • Converse about how they could continue to strengthen the Mexican Open Data Policy.

 

The results

As a result of this dialogue, we agreed six actions that could be implemented internationally beyond just the Mexican context both by governments with centralised open data repositories and those which don’t centralise their data, as well as a way to improve the GODI methodology:

  1. Open dialogue during the GODI process: Mexico was the first country to develop a structured dialogue to agree with open data experts from civil society about submissions to GODI. The Mexican government will seek to replicate this process in future evaluations and include new groups to promote open data use in the country. OKI will take this experience into account to improve the GODI processes in the future.
  2. Open licenses by default: The Mexican government is reviewing and modifying their regulations to implement the terms of Libre Uso MX for every website, platform and online tool of the national government. This is an example of good practice which OKI have highlighted in our ongoing Open Licensing research.
  3. “GODI” data group in CKAN: Most data repositories allow users to create thematic groups. In the case of GODI, the Mexican government created the “Global Open Data Index” group in datos.gob.mx. This will allow users to access these datasets based on their specific needs.
  4. Create a link between government built visualization tools and datos.gob.mx: The visualisations and reference tools tend to be the first point of contact for citizens. For this reason, the Mexican government will have new regulations in their upcoming Open Data Policy so that any new development includes visible links to the open data they use.
  5. Multiple access points for data: In August 2018, the Mexican government will launch a new section on datos.gob.mx to provide non-technical users easy access to valuable data. These data called “‘Infraestructura de Datos Abiertos MX’ will be divided into five easy-to-explore and understand categories.
  6. Common language for data sets: Government naming conventions aren’t the easiest to understand and can make it difficult to access data. The Mexican government has agreed to change the names to use more colloquial language can help on data findability and promote their use. In case this is not possible with some datasets, the government will go for an option similar to the one established in point 5.

We hope these changes will be useful for data users as well as other governments who are looking to improve their publication policies. Got any other ideas? Share them with us on Twitter by messaging @OKFN or send us an email to index@okfn.org

 

]]>
https://blog.okfn.org/2017/08/16/using-the-global-open-data-index-to-strengthen-open-data-policies-best-practices-from-mexico/feed/ 0 23200 https://blog.okfn.org/2017/08/16/using-the-global-open-data-index-to-strengthen-open-data-policies-best-practices-from-mexico/
Data-cards – a design pattern http://feedproxy.google.com/~r/okfn/~3/ZcIMQhqLJwg/ https://blog.okfn.org/2017/08/15/data-cards-a-design-pattern/#respond Tue, 15 Aug 2017 10:31:37 +0000 https://blog.core.okfn.org/?p=23186 Cross-posted on smth.uk


It can be useful to recognise patterns in the challenges we face, and in our responses to those challenges. In doing this, we can build a library of solutions, a useful resource when similar challenges arise in the future. When working on innovative projects, as is often the case at Open Knowledge International, creating brand new challenges is inevitable. With little or no historical reference material on how best to tackle these challenges, paying attention to your own repeatable solutions becomes even more valuable.

From a user interface design point of view, these solutions come in the form of design patterns – reusable solutions to commonly occurring problems. Identifying, and using design patterns can help create familiar processes for users; and by not reinventing the wheel, you can save time in production too.

In our work on Data Packages, we are introducing a new task into the world – creating those data packages. This task can be quite simple, and it will ultimately be time saving for people working with data. That said, there is no escaping the fact that this is a task that has never before been asked of people, one that will need to be done repeatedly, and potentially, from within any number of interfaces.

It has been my task of late to design some of these interfaces; I’d like to highlight one pattern that is starting to emerge – the process of describing, or adding metadata to, the columns of a data table. I was first faced with this challenge when working on OS Packager. The objective was to present a recognisable representation of the columns, and facilitate the addition of metadata for each of those columns. The adding of data would be relatively straight forward, a few form fields. The challenge lay in helping the user to recognise those columns from the tables they originated. As anyone who works with spreadsheets on a regular basis will know, they aren’t often predictably or uniformly structured, meaning it is not always obvious what you’re looking at. Take them out of the familiar context of the application they were created in, and this problem could get worse. For this reason, just pulling a table header is probably not sufficient to identify a column. We wanted to provide a preview of the data, to give the best chance of it being recognisable. In addition to this, I felt it important to keep the layout as close as possible to that of say Excel. The simplest solution would be to take the first few rows of the table, and put a form under each column, for the user to add their metadata.

 

 

This is a good start, about as recognisable and familiar as you’re going to get. There is one obvious problem though, this could extend well beyond the edge of the users screen, leading to an awkward navigating experience. For an app aimed at desktop users, horizontal scrolling, in any of its forms, would be problematic.

So, in the spirit of the good ol’ webpage, let’s make this thing wrap. That is to say that when an element can not fit on the screen, it moves to a new “line”. When doing this we’ll need some vertical spacing where this new line occurs, to make it clear that one column is separate from the one above it. We then need horizontal spacing to prevent the false impression of grouping created by the rows.

 

 

The data-card was born. At the time of writing it is utilised in OS Packager, pretty closely resembling the above sketch.

 

 

Data Packagist is another application that creates data packages, and it faces the same challenges as described above. When I got involved in this project there was already a working prototype, I saw in this prototype data cards beginning to emerge.

It struck me that if these elements followed the same data card pattern created for OS Packager, they could benefit in two significant ways. The layout and data preview would again allow the user to more easily recognise the columns from their spreadsheet; plus the grid layout would lend itself well to drag and drop, which would mean avoiding multiple clicks (of the arrows in the screenshot above) when reordering. I incorporated this pattern into the design.

 

 

Before building this new front-end, I extracted what I believe to be the essence of the data-card from the OS Packager code, to reuse in Data Packagist, and potentially future projects. While doing so I thought about the current and potential future uses, and the other functions useful to perform at the same time as adding metadata. Many of these will be unique to each app, but there are a couple that I believe likely to be recurring:

  • Reorder the columns
  • Remove / ignore a column

These features combine with those of the previous iteration to create this stand-alone data-card project:

Time will tell how useful this code will be for future work, but as I was able to use it wholesale (changing little more than a colour variable) in the implementation of the Data Packagist front-end, it came at virtually no additional cost. More important than the code however, is having this design pattern as a template, to solve this problem when it arises again in the future.

]]>
https://blog.okfn.org/2017/08/15/data-cards-a-design-pattern/feed/ 0 23186 https://blog.okfn.org/2017/08/15/data-cards-a-design-pattern/
An approach to building open databases http://feedproxy.google.com/~r/okfn/~3/CS_HZvrfOaQ/ https://blog.okfn.org/2017/08/10/23170/#comments Thu, 10 Aug 2017 07:30:11 +0000 https://blog.core.okfn.org/?p=23170 This post has been co-authored by Adam Kariv, Vitor Baptista, and Paul Walsh.


Open Knowledge International (OKI) recently coordinated a two-day work sprint as a way to touch base with partners in the Open Data for Tax Justice project. Our initial writeup of the sprint can be found here.

Phase I of the project ended in February 2017 with the publication of What Do They Pay?, a white paper that outlines the need for a public database on the tax contributions and economic activities of multinational companies.

The overarching goal of the sprint was to start some work towards such a database, by replicating data collection processes we’ve used in other projects, and to provide a space for domain expert partners to potentially use this data for some exploratory investigative work. We had limited time, a limited budget, and we are pleased with the discussions and ideas that came out of the sprint.

One attendee, Tim Davies, criticised the approach we took in the technical stream of the sprint. The problem with the criticism is the extrapolation of one stream of activity during a two-day event to posit an entire approach to a project. We think exploration and prototyping should be part of any healthy project, and that is exactly what we did with our technical work in the two-day sprint.

Reflecting on the discussion presents a good opportunity here to look more generally at how we, as an organisation, bring technical capacity to projects such as Open Data for Tax Justice. Of course, we often bring much more than technical capacity to a project, and Open Data for Tax Justice is no different in that regard, being mostly a research project to date.

In particular, we’ll take a look at the technical approach we used for the two-day sprint. While this is not the only approach towards technical projects we employ at OKI, it has proven useful on projects driven by the creation of new databases.

An approach

Almost all projects that OKI either leads on, or participates in, have multiple partners. OKI generally participates in one of three capacities (sometimes, all three):

  • Technical design and implementation of open data platforms and apps.
  • Research and thought leadership on openness and data.
  • Dissemination and facilitating participation, often by bringing the “open data community” to interact with domain specific actors.

Only the first capacity is strictly technical, but each capacity does, more often than not, touch on technical issues around open data.

Some projects have an important component around the creation of new databases targeting a particular domain. Open Data for Tax Justice is one such project, as are OpenTrials, and the Subsidy Stories project, which itself is a part of OpenSpending.

While most projects have partners, usually domain experts, it does not mean that collaboration is consistent or equally distributed over the project life cycle. There are many reasons for this to be the case, such as the strengths and weaknesses of our team and those of our partners, priorities identified in the field, and, of course, project scope and funding.

With this as the backdrop for projects we engage in generally, we’ll focus for the rest of this post on aspects when we bring technical capacity to a project. As a team (the Product Team at OKI), we are currently iterating on an approach in such projects, based on the following concepts:

  • Replication and reuse
  • Data provenance and reproducibility
  • Centralise data, decentralise views
  • Data wrangling before data standards

While not applicable to all projects, we’ve found this approach useful when contributing to projects that involve building a database to, ultimately, unlock the potential to use data towards social change.

Replication and reuse

We highly value the replication of processes and the reuse of tooling across projects. Replication and reuse enables us to reduce technical costs, focus more on the domain at hand, and share knowledge on common patterns across open data projects. In terms of technical capacity, the Product Team is becoming quite effective at this, with a strong body of processes and tooling ready for use.

This also means that each project enables us to iterate on such processes and tooling, integrating new learnings. Many of these learnings come from interactions with partners and users, and others come from working with data.

In the recent Open Data for Tax Justice sprint, we invited various partners to share experiences working in this field and try a prototype we built to extract data from country-by-country reports to a central database. It was developed in about a week, thanks to the reuse of processes and tools from other projects and contexts.

When our partners started looking into this database, they had questions that could only be answered by looking back to the original reports. They needed to check the footnotes and other context around the data, which weren’t available in the database yet. We’ve encountered similar use cases in both OpenBudgets.eu and OpenTrials, so we can build upon these experiences to iterate towards a reusable solution for the Open Data for Tax Justice project.

By doing this enough times in different contexts, we’re able to solve common issues quickly, freeing more time to focus on the unique challenges each project brings.

Data provenance and reproducibility of views on data are absolutely essential to building databases with a long and useful futureData provenance and reproducibility

We think that data provenance, and reproducibility of views on data, is absolutely essential to building databases with a long and useful future.

What exactly is data provenance? A useful definition from wikipedia is “… (d)ata provenance documents the inputs, entities, systems, and processes that influence data of interest, in effect providing a historical record of the data and its origins”. Depending on the way provenance is implemented in a project, it can also be a powerful tool for reproducibility of the data.

Most work around open data at present does not consider data provenance and reproducibility as an essential aspect of working with open data. We think this is to the detriment of the ecosystem’s broader goals of seeing open data drive social change: the credible use of data from projects with no provenance or reproducibility built in to the creation of databases is significantly diminished in our “post truth” era.

Our current approach builds data provenance and reproducibility right into the heart of building a database. There is a clear, documented record of every action performed on data, from the extraction of source data, through to normalisation processes, and right to the creation of records in a database. The connection between source data and processed data is not lost, and, importantly, the entire data pipeline can be reproduced by others.

We acknowledge that a clear constraint of this approach, in its current form, is that it is necessarily more technical than, say, ad hoc extraction and manipulation with spreadsheets and other consumer tools used in manual data extraction processes. However, as such approaches make data provenance and reproducibility harder, because there is no history of the changes made, or where the data comes from, we are willing to accept this more technical approach and iterate on ways to reduce technical barriers.

We hope to see more actors in the open data ecosystem integrating provenance and reproducibility right into their data work. Without doing so, we greatly reduce the ability for open data to be used in an investigative capacity, and likewise, we diminish the possibility of using the outputs of open data projects in the wider establishment of facts about the world. Recent work on beneficial ownership data takes a step in this direction, leveraging the PROV-DM standard to declare data provenance facts.

Centralise data, decentralise views

In OpenSpending, OpenTrials, and our initial exploratory work on Open Data for Tax Justice, there is an overarching theme to how we have approached data work, user stories and use cases, and co-design with domain experts: “centralise data, decentralise views”.

Building a central database for open data in a given domain affords ways of interacting with such data that are extremely difficult, or impossible, by actively choosing to decentralise such data. Centralised databases make investigative work that uses the data easier, and allows for the discovery, for example, of patterns across entities and time that can be very hard to discover if data is decentralised.

Additionally, by having in place a strong approach to data provenance and reproducibility, the complete replication of a centralised database is relatively easily done, and very much encouraged. This somewhat mitigates a major concern with centralised databases, being that they imply some type of “vendor lock-in”.

Views on data are better when decentralised. By “views on data” we refer to visualisations, apps, websites – any user-facing presentation of data. While having data centralised potentially enables richer views, data almost always needs to be presented with additional context, localised, framed in a particular narrative, or otherwise presented in unique ways that will never be best served from a central point.

Further, decentralised usage of data provides a feedback mechanism for iteration on the central database. For example, providing commonly used contextual data, establishing clear use cases for enrichment and reconciliation of measures and dimensions in the data, and so on.

Data wrangling before data standards

As a team, we are interested in, engage with, and also author, open data standards. However, we are very wary of efforts to establish a data standard before working with large amounts of data that such a standard is supposed to represent.

Data standards that are developed too early are bound to make untested assumptions about the world they seek to formalise (the data itself). There is a dilemma here of describing the world “as it is”, or, “as we would like it to be”. No doubt, a “standards first” approach is valid in some situations. Often, it seems, in the realm of policy. We do not consider such an approach flawed, but rather, one with its own pros and cons.

We prefer to work with data, right from extraction and processing, through to user interaction, before working towards public standards, specifications, or any other type of formalisation of the data for a given domain.

Our process generally follows this pattern:

  • Get to know available data and establish (with domain experts) initial use cases.
  • Attempt to map what we do not know (e.g.: data that is not yet publicly accessible), as this clearly impacts both usage of the data, and formalisation of a standard.
  • Start data work by prescribing the absolute minimum data specification to use the data (i.e.: meet some or all of the identified use cases).
  • Implement data infrastructure that makes it simple to ingest large amounts of data, and also to keep the data specification reactive to change.
  • Integrate data from a wide variety of sources, and, with partners and users, work on ways to improve participation / contribution of data.
  • Repeat the above steps towards a fairly stable specification for the data.
  • Consider extracting this specification into a data standard.

Throughout this entire process, there is a constant feedback loop with domain expert partners, as well as a range of users interested in the data.

Reflections

We want to be very clear that we do not think that the above approach is the only way to work towards a database in a data-driven project.

Design (project design, technical design, interactive design, and so on) emerges from context. Design is also a sequence of choices, and each choice has an opportunity cost based on various constraints that are present in any activity.

In projects we engage in around open databases, technology is a means to other, social ends. Collaboration around data is generally facilitated by technology, but we do not think the technological basis for this collaboration should be limited to existing consumer-facing tools, especially if such tools have hidden costs on the path to other important goals, like data provenance and reproducibility. Better tools and processes for collaboration will only emerge over time if we allow exploration and experimentation.

We think it is important to understand general approaches to working with open data, and how they may manifest within a single project, or across a range of projects. Project work is not static, and definitely not reducible to snapshots of activity within a wider project life cycle.

Certain approaches emphasise different ends. We’ve tried above to highlight some pros and cons of our approach, especially around data provenance and reproducibility, and data standards.

In closing, we’d like to invite others interested in approaches to building open databases to engage in a broader discussion around these themes, as well as a discussion around short term and long term goals of such projects. From our perspective, we think there could be a great deal of value for the ecosystem around open data generally – CSOs, NGOs, governments, domain experts, funders – via a proactive discussion or series of posts with a multitude of voices. Join the discussion here if this is of interest to you.

]]>
https://blog.okfn.org/2017/08/10/23170/feed/ 1 23170 https://blog.okfn.org/2017/08/10/23170/
New research: Understanding the drivers of license proliferation http://feedproxy.google.com/~r/okfn/~3/IhYu3nwRD1I/ https://blog.okfn.org/2017/08/08/new-research-understanding-the-drivers-of-license-proliferation/#comments Tue, 08 Aug 2017 15:51:08 +0000 https://blog.core.okfn.org/?p=23162 Open licensing is still a major challenge for open data publication. In a recent blog post on the state of open licensing in 2017 Open Knowledge International identified that governments often decide to create custom licenses instead of using standard open licenses such as Creative Commons Attribution 4.0.

This so-called license proliferation is problematic for a variety of reasons. Custom licenses necessitate that data users know all legal arrangements of these licenses – a problem that standard licenses are intended to avoid by clearly and easily stating use rights. Custom licenses can also exacerbate legal compatibility issues across licenses, which makes it hard (or impossible) to combine and distribute data coming from different sources. Because of legal uncertainties and compatibility issues, license proliferation can have chilling effects on the reuse of data and in the worst case prevent data reuse entirely.

When investigating this topic further we noticed a dearth of knowledge about the drivers of license proliferation: neither academia nor grey literature seem to give systematic answers, but there are some great first analyses, as well as explanations why license proliferation is bad. Why do governments create custom licenses? Who within government decides that standard licenses are not the best solution to make data and content legally open? How do governments organise the licensing process and how can license recommendations applied across government agencies?

 

Exploring the drivers of license proliferation

In order to address these questions Open Knowledge International started a research project into license proliferation. Using the findings of the Global Open Data Index (GODI) 2016/17 as a starting point, we first mapped out how many different licenses are used in a selection of 20 countries. This includes following countries which either rank high in GODI, or where the Open Knowledge community is present:

Taiwan, Australia, Great Britain, France, Finland, Canada, Norway, New Zealand, Brazil, Denmark, Colombia, Mexico, Japan, Argentina, Belgium, Germany, Netherlands, Greece, Nepal, Singapore.

Now we want to explore how governments decide what kind of license to use for data publication. We intend to publish the results in a narrative report to inform the open data community to understand the licensing process better, to inform license stewardship, and to advocate for the use of standard licenses.

 

Get in touch!

We are planning to run interviews with government officials who are involved in licensing.  Please don’t hesitate and get in touch with us by sending an email to research@okfn.org. Feedback from government officials working on licensing is much appreciated. Also do reach out if you have background knowledge about the licensing situation in above listed countries, or if you have contacts to government.

We hope to hear from you soon!  

]]>
https://blog.okfn.org/2017/08/08/new-research-understanding-the-drivers-of-license-proliferation/feed/ 3 23162 https://blog.okfn.org/2017/08/08/new-research-understanding-the-drivers-of-license-proliferation/
OKI Agile: Kanban – the dashboard of doing http://feedproxy.google.com/~r/okfn/~3/E9KO3g7bIEw/ https://blog.okfn.org/2017/08/03/oki-agile-kanban-the-dashboard-of-doing/#respond Thu, 03 Aug 2017 09:38:12 +0000 https://blog.core.okfn.org/?p=23118 This is the fourth in a series of blogs on how we are using the Agile methodology at Open Knowledge International. Originating from software development, the Agile manifesto describes a set of principles that prioritise agility in work processes: for example through continuous development, self-organised teams with frequent interactions and quick responses to change (http://agilemanifesto.org). In this blogging series we explore the different ways Agile can be used to work better in teams and to create more efficiency in how to deliver projects. Previous posts dealt with user storiesmethodologies and the use of scrum and sprints in open data: this time we go into Kanban. 

Picking a methodology has to take into account both the team doing the work but also the project. Bigger teams call for a bigger methodology, and more critical projects call for more methodology density (higher ceremony or publicly visible correctness). Bigger methodology adds larger amount of cost to the project. So picking a big methodology, with high ceremony, for a small team and less critical projects is a waste of time, money and effort.

Scrum is an example of a relatively big methodology with high ceremony (in the agile universe). There are daily standups, sprint backlogs and commitments, product owner, scrum masters etc. Always going for Scrum may make us look like we’re organised, but we might be organised in the wrong way.

Less can lead to more

The opposite of Scrum in the agile world is perhaps Kanban. It’s designed to not interfere in any way with how the team already works. It may even be an addition to Scrum. It’s a form of what has been called Just-In-Time delivery/development.

In Scrum the team creates a sprint backlog of what the team commits to and what will be worked on during the sprint. During the sprint, which typically ranges from 2-4 weeks, nobody can touch the backlog: nothing can be added, nothing can be modified. Items in the sprint backlog can only be closed. This is done to allow the team to focus on delivery, not requirements discussions.

Kanban works on a different level. In Japanese Kan stands for card and ban means for signal. Kanban could therefore be translated as Card signals and that comes pretty close because Kanban is about using cards or some equivalent of cards to signal progress. It’s a card dashboard showing what’s being done where. There is no sprint that denotes work units, it’s just continuous work and planning when needed, or just in time.

Spreadsheet planning

In its essence, Kanban is progress visualised in a structured spreadsheet. You have columns that show different stages of progress or work. The different stages can include:

  • Incoming work
  • Brainstorming/Specifications
  • Implementation/development
  • Ready for review
  • In tests/proofreading
  • Ready for deployment
  • Deployed

This is not a be all, end all list. This is an example of what columns might be included. Kanban doesn’t tell us what columns we should use. Kanban is designed to be an addition on top of our current processes.

The rows in our spreadsheet is used to track different aspects of the project. For example, if we have different members of the team working in different areas: Designers, content writers, software developers, a row would represent each area. This could also be used to track features of the project if the team is homogeneous (all team members work together in the same area, e.g. all software developers).

The cells of the spreadsheet will then include requirements, for example user stories, that are in each stage for each feature or team. If we remember the requirements iceberg, this this is the 1-2 day tasks that we perform.

Visualising progress

As work progresses on each requirement item, it moves between the different columns. This creates the Kanban dashboard, visualising progress of the project. Kanban encourages autonomy and leadership on all levels. Every team member can come in each day, have a look at what’s needed and just do some work and then once done, move it between columns. That’s part of the just-in-time attitude. We don’t need to plan everything in beforehand, things will be done just in time.

That means for managers, who normally push schedules for work to team members, they now have to sit back and give the team autonomy over the process. Likewise, team members who are used to being told what to do, now have to take matters into their own hands and pull the work that needs to be done.

What about prioritisation you may ask? What about team members just picking low-hanging fruit or easy tasks and leave all the boring tasks until last even if they are the highest priority?

Kanban puts limits on each column. How many is up to the team, but it is not allowed to have more than a prefixed amount of requirements in each column at any given moment. So there might never be more than 8 incoming work items, 2 items being drafted, 3 being implemented etc. The only exception to this can be the last column, delivered which collects everything that has been done, but is usually not a part of the Kanban dashboard or if it is, then it’s regularly purged.

So if the team wants to implement a drafted feature but the amount of items in the implementation column has already reached the maximum they need to sort that out first, before they can take on more implementation work. The project manager or client representative will typically manage the incoming work column but that one is also limited by the maximum amount and progress.

Common sense should of course be applied to these limitations. If the team is heterogeneous, i.e. each row in the “pretend spreadsheet” is a different area instead of a different feature, the limitations apply to each of the cells in the column (e.g. we can’t have the content team limit the software developers just because they have more to do).

Principles

Kanban is really simple, it’s all about visualising the work flow and not overwhelming the team. It’s built around four principles:

  • It should work with whatever process is currently used by the team
  • It helps the team make small incremental changes towards improvement
  • It respects roles and responsibilities of each team member
  • To each her/his own, team members are autonomous and leaders over their own work

It should be really simple to pick up Kanban and add it on top of whatever you’re doing today. Kanban works really well for whiteboards but the biggest obstacle for a remote organisation is finding a good digital tool for the workflow visualisation; Trello might work if if you don’t care too much about the rows. Google spreadsheet might work if you don’t care about the extra effort of moving things around.

If you find a good way to manage a Kanban board, please share it through the comments below or on our forum!

 

]]>
https://blog.okfn.org/2017/08/03/oki-agile-kanban-the-dashboard-of-doing/feed/ 0 23118 https://blog.okfn.org/2017/08/03/oki-agile-kanban-the-dashboard-of-doing/
Why MyData 2017? http://feedproxy.google.com/~r/okfn/~3/Fa1T7ac_QXY/ https://blog.okfn.org/2017/08/02/why-mydata-2017/#respond Wed, 02 Aug 2017 09:02:09 +0000 https://blog.core.okfn.org/?p=23158

This is a guest post explaining the focus of the MyData conference in Tallinn and Helsinki later this month.

By a famous writing tip, you should always start texts with ‘why?’. Here we are taking that tip, and we actually find many ways to answer the big Why. So,

Why MyData 2017?

Did you get your data after MyData 2016 conference? No, you did not. There is lots of work to be done, and we need all the companies, governments, individuals and NGO’s on board on Aug 31-Sep 1 in Tallinn and Helsinki. When else would you meet the other over 800 friends at once?

Because no. 1: The work did not stop after MyData 2016

The organizers Fing, Aalto University, Open Knowledge Finland, and Tallinn University have been working on the topic also after the conference. Fing continues their MesInfos project, started in 2012, which goes to its second phase in 2017: implementing the MyData approach in France with a long-term pilot involving big corporations, public actors, testers and a platform. Aalto University is the home base of human-centric personal data research in Finland. Many Helsinki-based pieces of research contribute their academic skills to the conference’s Academic workshops.

Open Knowledge Finland, apart from giving the conference an organizational kick also fosters a project researching MyData implementation in Finnish public sector, of which we will hear in the conference too. Tallinn University, as the newest addition to the group of organizers, will host the conference day in Tallinn to set the base for and inspire MyData initiatives in Estonian companies, public sector, and academic domain.

In addition to the obvious ones, multiple MyData inspired companies to continue on their own. Work continues for example in Alliance meetings, and in some cases, there are people working from the bottom up and acting as change makers in their organization.

MyData 2016 went extremely well, 95 % of the feedback was positive, and the complaints were related to organizational issues like the positioning of the knives during lunch time. Total individual visitor count was 670 from 24 countries. All this was for (at the time) niche conference, organized for the first time by a team mainly of part time workers.

The key to success was the people who came in offering their insights as presenters or their talents in customer care as volunteers. MyData 2017 is, even more, community driven than the year before – again a big bunch of devoted presenters, and the volunteers have been working already since March in weekly meetings, talkoot.

Because no. 2: The Community did not stop existing – it started to grow

MyData gained momentum in 2016 – the MyData White paper is mentioned in a ‘Staff Working Document on the free flow of data and emerging issues of the European data economy’, on pages 24-25. The white paper is also now translated from Finnish to English and Portuguese. Internationally, multiple Local Hubs have been founded this year – of which you hear more about in the Global track of the conference – and a MyData Symposium was held in Japan earlier this year.

The PIMS (Personal Information Management Systems) community, who met for the fourth time during the 2016 conference, has been requesting more established community around the topic. “Building a global community and sharing ideas” is one goal of MyData 2017, and as a very concrete action, the conference organizing team and PIMS community have agreed to merge their efforts under the umbrella name of MyData. The MyData Global Network Founding Members are reviewing the Declaration of MyData Principles to be presented during MyData 2017. Next round table meeting for the MyData Global Network will be held in Aarhus in November 23.–24. 2017.

 

Open Knowledge Estonia was founded after last year’s conference. Since MyData was nurtured into its current form inside the Open Knowledge movement, where Open Knowledge Finland still plays the biggest role, MyData people feel very close to other Open Knowledge chapters. See for yourself, how nicely Rufus Pollock explains in this video from MyData 2016 how Open Data and MyData are related.

Because no. 3: Estonians are estonishing

“Why Tallinn then?” is a question we hear a lot. The closeness of the two cities, also sometimes jointly called Talsinki, makes the choice very natural to the Finns and Estonians, but might seem weird looking from outside.

Estonia holds the Presidency of the Council of the EU in the second part of 2017. In an e-Estonia, home of the infamous e-residency, MyData fits naturally in the pool of ideas to be tossed around during that period. Now, having the ‘Free movement of data’ as the fifth freedom within the European Union, in addition to goods, capital, service, and people, has been suggested by Estonians, and MyData way of thinking is a crucial part to advance that.

Estonia and Finland co-operate in developing X-road, a data exchange layer for national information systems, between the two countries. In 2017, the Nordic Institute for Interoperability (NIIS) was founded to advance the X-road in other countries as well. Finnish population registry center and their digitalized services esuomi.fi is the main partner of the conference in 2017

Estonia and Finland both as small countries are very good places to test new ideas. Both in Helsinki and Tallinn, we now have ongoing ‘MyData Alliance’ meetups for companies and public organizations who want to advance MyData in their organizations. A goal of MyData in general, “we want to make Finland the Moomin Valley of personal data” will be expanded to “we want to make Finland and Estonia the Moomin Valley of personal data”.

 

]]>
https://blog.okfn.org/2017/08/02/why-mydata-2017/feed/ 0 23158 https://blog.okfn.org/2017/08/02/why-mydata-2017/
Open Data for Tax Justice design sprint: building a pilot database of public country-by-country reporting http://feedproxy.google.com/~r/okfn/~3/BFpreGj0DOQ/ https://blog.okfn.org/2017/07/27/open-data-for-tax-justice-design-sprint-building-a-pilot-database-of-public-country-by-country-reporting/#comments Thu, 27 Jul 2017 14:36:35 +0000 https://blog.core.okfn.org/?p=23128 Tax justice advocates, global campaigners and open data specialists came together this week from across the world to work with Open Knowledge International on the first stages of creating a pilot country-by-country reporting database. Such a database may enable anyone to understand the activities of multinational corporations and uncover potential tax avoidance schemes. 

This design sprint event was part of our Open Data for Tax Justice project to create a global network of people and organisations using open data to improve advocacy, journalism and public policy around tax justice in line with our mission to empower civil society organisations to use open data to improve people’s lives. In this post my colleague Serah Rono and I share our experiences and learnings from the sprint. 

 

What is country-by-country reporting?

Image: Financial Transparency Coalition

Country-by-country reporting (CBCR) is a transparency mechanism which requires multinational corporations to publish information about their economic activities in all of the countries where they operate. This includes information on the taxes they pay, the number of people they employ and the profits they report. Publishing this information can bring to light structures or techniques multinational corporations might be using to avoid paying tax in certain jurisdictions by shifting their profits or activities elsewhere.

In February 2017, Open Knowledge International published a white paper co-authored by Alex Cobham, Jonathan Gray and Richard Murphy which examined the prospects for creating a global public database on the tax contributions and economic activities of multinational companies as measured by CBCR.

The authors found that such a public database was possible and concluded that a pilot database could be created by bringing together the best existing source of public CBCR information – disclosures made by European Union banking institutions in line with the Capital Requirements Directive IV (CRD IV) passed in 2013.  The aim of our design sprint was to take the first steps towards the creation of this pilot database.

 

What did we achieve?

From left to right: Tim Davies (Open Data Services), Jonathan Gray (University of Bath/Public Data Lab), Tommaso Faccio (University of Nottingham/BEPS Monitoring Group), Oliver Pearce (Oxfam GB), Elena Gaita (Transparency International EU), Dorcas Mensah (University of Edinburgh/Tax Justice Network – Africa) and Serah Rono (Open Knowledge International). Photo: Stephen Abbott Pugh

A design sprint is intended to be a short and sharp process bringing together a multidisciplinary team in order to quickly prototype and iterate on a technical product.

On Monday 24th and Tuesday 25th July 2017, Open Knowledge International convened a team of tax justice, advocacy, research and open data experts at Friends House in London to work alongside developers and a developer advocate from our product team. This followed three days of pre-sprint planning and work on the part of our developers. All the outputs of this event are public on Google Drive, Github and hackmd.io.

To understand more from those who had knowledge of trying to find and understand CRD IV data, we heard expert presentations from George Turner of Tax Justice Network on the scale of international tax avoidance, Jason Braganza of Tax Justice Network – Africa and Financial Transparency Coalition on why developing countries need public CBCR (see report for more details) and Oliver Pearce of Oxfam Great Britain on the lessons learned from using CRD IV data for the Opening the vaults and Following the money reports. These were followed by a presentation from Adam Kariv and Vitor Baptista of Open Knowledge International on how they would be reusing open-source tech products developed for our Open Spending and OpenTrials projects to help with Open Data for Tax Justice.

Next we discussed the problems and challenges the attendees had experienced when trying to access or use public CBCR information before proposing solutions to these issues. This lead into a conversation about the precise questions and hypotheses which attendees would like to be able to answer using either CRD IV data or public CBCR data more generally.

From left to right: Georgiana Bere (Open Knowledge International), Adam Kariv (Open Knowledge International), Vitor Baptista (Open Knowledge International). Photo: Stephen Abbott Pugh

As quickly as possible, the Open Knowledge International team wanted to give attendees the knowledge and tools they needed to be able to answer these questions. So our developers Georgiana Bere and Vitor Baptista demonstrated how anyone could take unstructured CRD IV information from tables published in the PDF version of banks’ annual reports and follow a process set out on the Github repo for the pilot database to contribute this data into a pipeline created by the Open Knowledge International team.

Datapackage-pipelines is a framework – developed as part of the Frictionless Data toolchain – for defining data processing steps to generate self-describing Data Packages. Once attendees had contributed data into the pipeline via Github issues,  Vitor demonstrated how to write queries against this data using Redash in order to get answers to the questions they had posed earlier in the day.

 

Storytelling with CRD IV data

Evidence-based, data-driven storytelling is an increasingly important mechanism used to inform and empower audiences, and encourage them to take action and push for positive change in the communities they live in. So our sprint focus on day two shifted to researching and drafting thematic stories using this data.

Discussions around data quality are commonplace in working with open data. George Turner and Oliver Pearce noticed a recurring issue in the available data: the use of hyphens to denote both nil and unrecorded values. The two spent part of the day thinking about ways to highlight the issue and guidelines that can help overcome this challenge so as to avoid incorrect interpretations.

Open data from a single source often has gaps so combining it with data from additional sources often helps with verification and to build a stronger narrative around it. In light of this, Elena Gaita, Dorcas Mensa and Jason Braganza narrowed their focus to examine a single organisation to see whether or not this bank changed its policy towards using tax havens following a 2012 investigative exposé by a British newspaper. They achieved this by comparing data from the investigation with the bank’s 2014 CRD IV disclosures. In the coming days, they hope to publish a blogpost detailing their findings on the extent to which the new transparency requirements have changed the bank’s tax behaviour.

 

Visual network showing relation between top 50 banks and financial institutions who comply with Capital Requirements Directive IV (CRD IV) and countries in which they report profits. Image: Public Data Lab

To complement these story ideas, we explored visualisation tools which could help draw insights and revelations from the assembled CRD IV data. Visualisations often help to draw attention to aspects of the data that would have otherwise gone unnoticed. Oliver Pearce and George Turner studied the exploratory visual network of CRD IV data for the EU’s top 50 banks created by our friends at Density Design and the Public Data Lab (see screengrab above) to learn where banks were recording most profits and losses. Pearce and Turner quickly realised that one bank in particular recorded losses in all but one of its jurisdictions. In just a few minutes, the finding from this visual network sparked their interest and encouraged them to ask more questions. Was the lone profit-recording jurisdiction a tax haven? How did other banks operating in the same jurisdiction fare on the profit/loss scale in the same period? We look forward to reading their findings as soon as they are published.

 

What happens next?

The Open Data for Tax Justice network team are now exploring opportunities for collaborations to collect and process all available CRD IV data via the pipeline and tools developed during our sprint. We are also examining options to resolve some of the data challenges experienced during the sprint like the perceived lack of an established codelist of tax jurisdictions and are searching for a standard exchange rate source which could be used across all recorded payments data.

In light of the European Union Parliament’s recent vote in favour of requiring all large multinational corporations to publish public CBCR information as open data, we will be working with advocacy partners to join the ongoing discussion about the “common template” and “open data format” for future public CBCR disclosures which will be mandated by the EU.

Having identified extractives industry data as another potential source of public CBCR to connect to our future database, we are also heartened to see the ongoing project between the Natural Resource Governance Institute and Publish What You Pay Canada so will liaise further with the team working on extracting data from these new disclosures.

Please email contact@datafortaxjustice.net if you’d like to be added to the project mailing list or want to join the Open Data for Tax Justice network. You can also follow the #OD4TJ hashtag on Twitter for updates.

 

Thanks to our partners at Open Data for Development, Tax Justice Network, Financial Transparency Coalition and Public Data Lab for the funding and support which made this design sprint possible.

 

             

 

]]>
https://blog.okfn.org/2017/07/27/open-data-for-tax-justice-design-sprint-building-a-pilot-database-of-public-country-by-country-reporting/feed/ 3 23128 https://blog.okfn.org/2017/07/27/open-data-for-tax-justice-design-sprint-building-a-pilot-database-of-public-country-by-country-reporting/
http://feeds.feedburner.com/okfn
feeds.feedburner.com
crawl=CC-MAIN-2017-34
146951
application/http; msgtype=response
sha1:JOJCMPHJM4UTXYYSQC3MSVQ46CZDTBO4
<urn:uuid:353d730a-4a7b-4444-987e-9440d5cf099e>
2017-08-18T12:46:13Z
172.217.7.238
application/rss+xml
sha1:7RM2QYZXFQTOSKQ5FLVN6C2CBLJADTKC
<urn:uuid:175a23ea-0a9e-4996-a804-e7c1c6e350c0>
http://feeds.feedburner.com/okfn
length
response
<urn:uuid:75374a38-7be8-407f-ac9c-333eeb00ef52>
robots: classic
hostname: ip-10-138-121-75.ec2.internal
software: Nutch 1.6 (CC)
isPartOf: CC-MAIN-2017-34
operator: Common Crawl Admin
description: Wide crawl of the web for August 2017
publisher: Common Crawl
format: WARC File Format 1.0
conformsTo: http://bibnum.bnf.fr/WARC/WARC_ISO_28500_version1_latestdraft.pdf