About Data.gov

Get Data on Data.gov

Data.gov Search & Feedback

About Data.gov

Who developed Data.gov?

Data.gov is managed and hosted by the U.S. General Services Administration, Technology Transformation Service.
Top

With what technology is Data.gov built?

Data.gov is powered by two open source applications, CKAN and WordPress, and it is developed publicly on GitHub. Learn how you can contribute to Data.gov and these larger open source projects here.
Top

What standards were used to develop the metadata displayed on Data.gov?

Data.gov follows the DCAT-US Schema v1.1 (Project Open Data Metadata Schema) – a set of required fields (Title, Description, Tags, Last Update, Publisher, Contact Name, etc.) for every data set displayed on Data.gov.
Top

What metrics are available about data on Data.gov?

On the Metrics page, users can get detailed information about the composition of the total number of datasets on Data.gov.

The total number of datasets, available on the Metrics page and on the main Data.gov and Catalog pages, is a number that changes frequently. Because the Data.gov catalog is updated on a nightly basis, the total number is subject to change as agencies add or delete datasets or Data.gov adds new agencies. Agencies may also group similar datasets into a “collection.” A “collection” counts as “1” dataset in the total, so the number of datasets may drop when agencies organize similar datasets into a collection, even though there has not been a reduction in what’s available on Data.gov. Occasionally, a technical issue could arise when the Data.gov catalog runs into a problem harvesting an agency’s metadata and also cause a temporary drop in the total number of datasets. As of June 2017, the approximately 200,000 datasets reported as the total on Data.gov represents about 10 million data resources.

The Metrics page also provides a detailed breakdown of the dataset total by type of source and by agency. While most Data.gov datasets are from federal agencies, the Metrics page also shows how many datasets are provided by state, county, and city data catalogs that make their datasets available through Data.gov.

Under each type of organization, users can see how many datasets are provided by each federal agency. For larger agencies, such as the Department of Agriculture, clicking on the + symbol will further breakdown of the USDA dataset total by each subagency.

The “Latest Entry” column on the right shows the most recent date when the Data.gov catalog detected an update to the agency’s metadata. The date listed is the most recent date when the agency added, deleted, or changed a dataset.

Top

Get Data on Data.gov

How are the datasets on Data.gov collected?

Under the terms of the 2013 Federal Open Data Policy, newly-generated government data is required to be made available in open, machine-readable formats, while continuing to ensure privacy and security.

Government data publishers looking to get their data on Data.gov should read the detailed guide: How to get your open data on Data.gov. The Data.gov team typically works with a designated open data point of contact as a liaison for each agency. Data publishers should consult with their agency point of contact to include any additional datasets on Data.gov. If you need help determining who your open data point of contact is, please contact us.

Federal CFO-Act agencies are required to:

  • Create a Single Agency Data Inventory. Agencies are required to catalog their data assets, just like they would inventory computers or desk chairs, to better manage and use these resources.
  • Publish a Public Data Listing. Agencies are required to publish a list of their data assets that are public, or could be made public. This list is made available as a data.json file hosted at the primary domain of the agency (e.g. gsa.gov/data.json)
  • Develop New Public Feedback Mechanisms. Agencies are required to set up feedback mechanisms to engage the public about where agencies should focus open data efforts, such as facilitating and prioritizing the release of datasets. Agencies are also required to identify public points of contacts for agency datasets.

Agency Public Data Listings are made available on agency websites as JSON files following the Project Open Data metadata schema (at agency.gov/data.json) and are then harvested into the central catalog for Data.gov.  Each agency is responsible for its own data.
Top

How can I add my government data to Data.gov?

Data.gov is primarily a federal open government data site. However, state, local, and tribal governments can also syndicate metadata describing their open data resources on Data.gov for greater discoverability. Data.gov does not host data directly, but rather aggregates metadata about open data resources in one centralized location. Once an open data source meets the necessary format and metadata requirements, the Data.gov team can pull directly from it as a Harvest Source, synchronizing that source’s metadata on Data.gov as often as every 24 hours.

Step 1: Organize your open data for the Data.gov Pipeline

Getting your data source ready for harvesting by the Data.gov catalog differs depending on the type of source:

  • Federal Data with Project Open DataThe most common source is the Public Data Listing as required by the Federal Open Data Policy.
  • Federal Geospatial Data: A number of federal agencies hold geospatial data which has separate requirements under different legal authorities.
  • Non-Federal DataNon-federal sources are not covered by the Federal Open Data Policy, but can be included in the Data.gov catalog voluntarily. Depending on your platform, creating this harvester might just be the push of a button or it could take a little more work, but the team will walk you through it either way. (See: https://www.data.gov/local/add )

Step 2: Coordinate with Data.gov

  • Contact the Data.gov team. Contact the Data.gov team (datagov@gsa.gov) to let them know you’d like to get started. Please include a link to your metadata in the data.json format or let us know if you have questions about how to create a data.json file from your current database along with any relevant links.
  • Connecting the pipes. The Data.gov team will create a new Harvest Source that will automatically collect information about your datasets and update Data.gov whenever changes are made on your data catalog.
  • Testing. The Data.gov team will test to ensure the harvester works properly. If anything seems wrong, the team will help you configure your data catalog so that Data.gov can collect your datasets without any errors.
  • Live within 24 hours! Once the harvester has been tested successfully, Data.gov will start automatically consuming information about your datasets and all the basic details of your datasets will be available on Data.gov with links to the source and your open data policy.

Top

Data.gov Search & Feedback

How do I find data on Data.gov?

You can search Data.gov from its catalog of government data from across the Federal Government. Once in the catalog, to find datasets you can:

  • Enter keywords in the search box.
  • Browse on the left side through types, tags, formats, groups, organization types, organizations, and categories.  Clicking on multiple items narrows your search.  You can click on the “x” to the side of any single item to remove it from the search, or “clear all” to remove all selected items in a category.
  • Search by geospatial area by drawing a boundary box on the map at the left side and clicking “Apply” to find all datasets that are tagged for that geographic area.
  • Once you find a dataset or tool of interest, click on the title and you will be taken to a page with more details on that specific dataset or tool. Some datasets are downloadable, while others are links to web sites or apps that help you access or use the data.
    Please note that by accessing datasets or tools offered on Data.gov, you agree to the Data Policy, which you should read before accessing any data. If there are additional datasets that you would like to see included on this site, please suggest more datasets here.

Top

What if I am having difficulty downloading a dataset from the catalog?

Please tell us what the problem with the dataset is so that we can correct it.
Top

Media inquiries

If you are preparing an article or creating an event, and are interested in finding information, a speaker, or other help in communicating about open data, contact the Data.gov team at datagov@gsa.gov.
Top
https://www.data.gov/about
www.data.gov
crawl=CC-MAIN-2020-05
40076
application/http; msgtype=response
sha1:6QZM6GBU5YZ5GH5MZDZQEJ5JA3OFHC3F
<urn:uuid:394b96b1-dc98-40c5-a893-58f31d6aa5fa>
2020-01-22T23:29:54Z
99.84.181.43
text/html
sha1:NA6KBZBYALRHA5MNMVKMD23FHFFCH42M
<urn:uuid:2780ef5d-6b28-4633-8ac3-5ee956a93b9e>
https://www.data.gov/about
response
<urn:uuid:7c484f22-0b17-4886-9742-dd510a2775a3>
isPartOf: CC-MAIN-2020-05
publisher: Common Crawl
description: Wide crawl of the web for January 2020
operator: Common Crawl Admin (info@commoncrawl.org)
hostname: ip-10-67-67-196.ec2.internal
software: Apache Nutch 1.16 (modified, https://github.com/commoncrawl/nutch/)
robots: checked via crawler-commons 1.1-SNAPSHOT (https://github.com/crawler-commons/crawler-commons)
format: WARC File Format 1.1
conformsTo: http://iipc.github.io/warc-specifications/specifications/warc-format/warc-1.1/