Domain specific dataset discovery on the Hugging Face hub 
COLLABORATOR
Hi! I added a link to this issue in our internal request for adding keywords/topics to the Hub, which is identical to the `topic tags` solution. The `collections` solution seems too complex (as you point out). Regarding the `domain tags` solution, we primarily focus on machine learning, so I'm not sure if it's a good idea to make our current taxonomy more complex.
2022-07-18T12:53:30Z
https://github.com/huggingface/datasets/issues/4702#issuecomment-1187342223
https://api.github.com/repos/huggingface/datasets/issues/4702
IC_kwDODunzps5GxWeP
https://api.github.com/repos/huggingface/datasets/issues/comments/1187342223/reactions
2022-07-18T12:53:30Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1187342223
https://avatars.githubusercontent.com/u/47462742?v=4
https://api.github.com/users/mariosasko/events{/privacy}
https://api.github.com/users/mariosasko/followers
https://api.github.com/users/mariosasko/following{/other_user}
https://api.github.com/users/mariosasko/gists{/gist_id}
https://github.com/mariosasko
mariosasko
MDQ6VXNlcjQ3NDYyNzQy
https://api.github.com/users/mariosasko/orgs
https://api.github.com/users/mariosasko/received_events
https://api.github.com/users/mariosasko/repos
https://api.github.com/users/mariosasko/starred{/owner}{/repo}
https://api.github.com/users/mariosasko/subscriptions
User
https://api.github.com/users/mariosasko
public
MEMBER
> Hi! I added a link to this issue in our internal request for adding keywords/topics to the Hub, which is identical to the `topic tags` solution. The `collections` solution seems too complex (as you point out). Regarding the `domain tags` solution, we primarily focus on machine learning, so I'm not sure if it's a good idea to make our current taxonomy more complex.

Thanks, for letting me know. Will you allow the topic tags to be user-generated or only chosen from a list?
2022-07-18T13:02:33Z
https://github.com/huggingface/datasets/issues/4702#issuecomment-1187363122
https://api.github.com/repos/huggingface/datasets/issues/4702
IC_kwDODunzps5Gxbky
https://api.github.com/repos/huggingface/datasets/issues/comments/1187363122/reactions
2022-07-18T13:02:33Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1187363122
https://avatars.githubusercontent.com/u/8995957?v=4
https://api.github.com/users/davanstrien/events{/privacy}
https://api.github.com/users/davanstrien/followers
https://api.github.com/users/davanstrien/following{/other_user}
https://api.github.com/users/davanstrien/gists{/gist_id}
https://github.com/davanstrien
davanstrien
MDQ6VXNlcjg5OTU5NTc=
https://api.github.com/users/davanstrien/orgs
https://api.github.com/users/davanstrien/received_events
https://api.github.com/users/davanstrien/repos
https://api.github.com/users/davanstrien/starred{/owner}{/repo}
https://api.github.com/users/davanstrien/subscriptions
User
https://api.github.com/users/davanstrien
public
MEMBER
Thanks for opening this issue @davanstrien.

As we discussed last week, the tag approach would be in principle the simpler to be implemented, either the domain tag (with closed vocabulary: more reliable but also more rigid), or the topic tag (with open vocabulary: more flexible for user needs)
2022-07-18T14:44:53Z
https://github.com/huggingface/datasets/issues/4702#issuecomment-1187590373
https://api.github.com/repos/huggingface/datasets/issues/4702
IC_kwDODunzps5GyTDl
https://api.github.com/repos/huggingface/datasets/issues/comments/1187590373/reactions
2022-07-18T14:44:53Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1187590373
https://avatars.githubusercontent.com/u/8515462?v=4
https://api.github.com/users/albertvillanova/events{/privacy}
https://api.github.com/users/albertvillanova/followers
https://api.github.com/users/albertvillanova/following{/other_user}
https://api.github.com/users/albertvillanova/gists{/gist_id}
https://github.com/albertvillanova
albertvillanova
MDQ6VXNlcjg1MTU0NjI=
https://api.github.com/users/albertvillanova/orgs
https://api.github.com/users/albertvillanova/received_events
https://api.github.com/users/albertvillanova/repos
https://api.github.com/users/albertvillanova/starred{/owner}{/repo}
https://api.github.com/users/albertvillanova/subscriptions
User
https://api.github.com/users/albertvillanova
public
MEMBER
Hi @davanstrien If i remember correctly this was also discussed inside a hf.co Discussion, would you be able to link it here too?

(where i suggested using `tags: - foo - bar` IIRC.

Thanks a ton!
2022-07-19T10:42:54Z
https://github.com/huggingface/datasets/issues/4702#issuecomment-1188897204
https://api.github.com/repos/huggingface/datasets/issues/4702
IC_kwDODunzps5G3SG0
https://api.github.com/repos/huggingface/datasets/issues/comments/1188897204/reactions
2022-07-19T10:42:54Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1188897204
https://avatars.githubusercontent.com/u/326577?v=4
https://api.github.com/users/julien-c/events{/privacy}
https://api.github.com/users/julien-c/followers
https://api.github.com/users/julien-c/following{/other_user}
https://api.github.com/users/julien-c/gists{/gist_id}
https://github.com/julien-c
julien-c
MDQ6VXNlcjMyNjU3Nw==
https://api.github.com/users/julien-c/orgs
https://api.github.com/users/julien-c/received_events
https://api.github.com/users/julien-c/repos
https://api.github.com/users/julien-c/starred{/owner}{/repo}
https://api.github.com/users/julien-c/subscriptions
User
https://api.github.com/users/julien-c
public
MEMBER
> Hi @davanstrien If i remember correctly this was also discussed inside a hf.co Discussion, would you be able to link it here too?
> 
> (where i suggested using `tags: - foo - bar` IIRC.
> 
> Thanks a ton!

This doesn't ring a bell - I did a quick search of https://discuss.huggingface.co but didn't find anything. 

The `tags: ` approach sounds like a good option for this. It would be especially nice if these could suggest existing tags, but this probably won't be easily possible through the current interface. 

2022-07-19T12:36:31Z
https://github.com/huggingface/datasets/issues/4702#issuecomment-1189000018
https://api.github.com/repos/huggingface/datasets/issues/4702
IC_kwDODunzps5G3rNS
https://api.github.com/repos/huggingface/datasets/issues/comments/1189000018/reactions
2022-07-19T12:36:39Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1189000018
https://avatars.githubusercontent.com/u/8995957?v=4
https://api.github.com/users/davanstrien/events{/privacy}
https://api.github.com/users/davanstrien/followers
https://api.github.com/users/davanstrien/following{/other_user}
https://api.github.com/users/davanstrien/gists{/gist_id}
https://github.com/davanstrien
davanstrien
MDQ6VXNlcjg5OTU5NTc=
https://api.github.com/users/davanstrien/orgs
https://api.github.com/users/davanstrien/received_events
https://api.github.com/users/davanstrien/repos
https://api.github.com/users/davanstrien/starred{/owner}{/repo}
https://api.github.com/users/davanstrien/subscriptions
User
https://api.github.com/users/davanstrien
public
MEMBER
I opened a PR to add "tags" to the YAML validator:
https://github.com/huggingface/datasets/pull/4716

I also added "tags" to the [tagging app](https://huggingface.co/spaces/huggingface/datasets-tagging), with suggestions like "bio" or "newspapers"
2022-07-19T12:39:08Z
https://github.com/huggingface/datasets/issues/4702#issuecomment-1189002499
https://api.github.com/repos/huggingface/datasets/issues/4702
IC_kwDODunzps5G3r0D
https://api.github.com/repos/huggingface/datasets/issues/comments/1189002499/reactions
2022-07-19T12:39:08Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1189002499
https://avatars.githubusercontent.com/u/42851186?v=4
https://api.github.com/users/lhoestq/events{/privacy}
https://api.github.com/users/lhoestq/followers
https://api.github.com/users/lhoestq/following{/other_user}
https://api.github.com/users/lhoestq/gists{/gist_id}
https://github.com/lhoestq
lhoestq
MDQ6VXNlcjQyODUxMTg2
https://api.github.com/users/lhoestq/orgs
https://api.github.com/users/lhoestq/received_events
https://api.github.com/users/lhoestq/repos
https://api.github.com/users/lhoestq/starred{/owner}{/repo}
https://api.github.com/users/lhoestq/subscriptions
User
https://api.github.com/users/lhoestq
public
MEMBER
Thanks @lhoestq for the initiative.
 
Just one question: are "tags" already supported on the Hub? 

I think they aren't. Thus, the Hub should support them so that they are properly displayed.
2022-07-19T12:48:32Z
https://github.com/huggingface/datasets/issues/4702#issuecomment-1189011661
https://api.github.com/repos/huggingface/datasets/issues/4702
IC_kwDODunzps5G3uDN
https://api.github.com/repos/huggingface/datasets/issues/comments/1189011661/reactions
2022-07-19T12:48:32Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1189011661
https://avatars.githubusercontent.com/u/8515462?v=4
https://api.github.com/users/albertvillanova/events{/privacy}
https://api.github.com/users/albertvillanova/followers
https://api.github.com/users/albertvillanova/following{/other_user}
https://api.github.com/users/albertvillanova/gists{/gist_id}
https://github.com/albertvillanova
albertvillanova
MDQ6VXNlcjg1MTU0NjI=
https://api.github.com/users/albertvillanova/orgs
https://api.github.com/users/albertvillanova/received_events
https://api.github.com/users/albertvillanova/repos
https://api.github.com/users/albertvillanova/starred{/owner}{/repo}
https://api.github.com/users/albertvillanova/subscriptions
User
https://api.github.com/users/albertvillanova
public
MEMBER
I think they're not displayed, but at least it should enable users to filter by tag in using `huggingface_hub` or using the appropriate query params on the website (not sure if it's possible yet though)
2022-07-19T13:06:31Z
https://github.com/huggingface/datasets/issues/4702#issuecomment-1189030260
https://api.github.com/repos/huggingface/datasets/issues/4702
IC_kwDODunzps5G3yl0
https://api.github.com/repos/huggingface/datasets/issues/comments/1189030260/reactions
2022-07-19T13:06:31Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1189030260
https://avatars.githubusercontent.com/u/42851186?v=4
https://api.github.com/users/lhoestq/events{/privacy}
https://api.github.com/users/lhoestq/followers
https://api.github.com/users/lhoestq/following{/other_user}
https://api.github.com/users/lhoestq/gists{/gist_id}
https://github.com/lhoestq
lhoestq
MDQ6VXNlcjQyODUxMTg2
https://api.github.com/users/lhoestq/orgs
https://api.github.com/users/lhoestq/received_events
https://api.github.com/users/lhoestq/repos
https://api.github.com/users/lhoestq/starred{/owner}{/repo}
https://api.github.com/users/lhoestq/subscriptions
User
https://api.github.com/users/lhoestq
public
MEMBER
> I think they're not displayed, but at least it should enable users to filter by tag in using `huggingface_hub` or using the appropriate query params on the website (not sure if it's possible yet though)

I think this would already be a helpful start. I'm happy to try this out with the datasets added to https://huggingface.co/organizations/biglam and use the `huggingface_hub` to filter those datasets using the tags. 
2022-07-19T15:18:11Z
https://github.com/huggingface/datasets/issues/4702#issuecomment-1189184750
https://api.github.com/repos/huggingface/datasets/issues/4702
IC_kwDODunzps5G4YTu
https://api.github.com/repos/huggingface/datasets/issues/comments/1189184750/reactions
2022-07-19T15:18:11Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1189184750
https://avatars.githubusercontent.com/u/8995957?v=4
https://api.github.com/users/davanstrien/events{/privacy}
https://api.github.com/users/davanstrien/followers
https://api.github.com/users/davanstrien/following{/other_user}
https://api.github.com/users/davanstrien/gists{/gist_id}
https://github.com/davanstrien
davanstrien
MDQ6VXNlcjg5OTU5NTc=
https://api.github.com/users/davanstrien/orgs
https://api.github.com/users/davanstrien/received_events
https://api.github.com/users/davanstrien/repos
https://api.github.com/users/davanstrien/starred{/owner}{/repo}
https://api.github.com/users/davanstrien/subscriptions
User
https://api.github.com/users/davanstrien
public
NONE
Is this abandoned? 
I'm looking for a transport logistics dataset; how can I find one?
2024-02-10T21:00:23Z
https://github.com/huggingface/datasets/issues/4702#issuecomment-1937132865
https://api.github.com/repos/huggingface/datasets/issues/4702
IC_kwDODunzps5zdk1B
https://api.github.com/repos/huggingface/datasets/issues/comments/1937132865/reactions
2024-02-10T21:00:23Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1937132865
https://avatars.githubusercontent.com/u/3153107?v=4
https://api.github.com/users/younes-io/events{/privacy}
https://api.github.com/users/younes-io/followers
https://api.github.com/users/younes-io/following{/other_user}
https://api.github.com/users/younes-io/gists{/gist_id}
https://github.com/younes-io
younes-io
MDQ6VXNlcjMxNTMxMDc=
https://api.github.com/users/younes-io/orgs
https://api.github.com/users/younes-io/received_events
https://api.github.com/users/younes-io/repos
https://api.github.com/users/younes-io/starred{/owner}{/repo}
https://api.github.com/users/younes-io/subscriptions
User
https://api.github.com/users/younes-io
public
MEMBER
@younes-io Full text search is probably your best bet: https://huggingface.co/search/full-text?type=dataset
2024-02-12T09:53:42Z
https://github.com/huggingface/datasets/issues/4702#issuecomment-1938346344
https://api.github.com/repos/huggingface/datasets/issues/4702
IC_kwDODunzps5ziNFo
https://api.github.com/repos/huggingface/datasets/issues/comments/1938346344/reactions
2024-02-12T09:53:42Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1938346344
https://avatars.githubusercontent.com/u/326577?v=4
https://api.github.com/users/julien-c/events{/privacy}
https://api.github.com/users/julien-c/followers
https://api.github.com/users/julien-c/following{/other_user}
https://api.github.com/users/julien-c/gists{/gist_id}
https://github.com/julien-c
julien-c
MDQ6VXNlcjMyNjU3Nw==
https://api.github.com/users/julien-c/orgs
https://api.github.com/users/julien-c/received_events
https://api.github.com/users/julien-c/repos
https://api.github.com/users/julien-c/starred{/owner}{/repo}
https://api.github.com/users/julien-c/subscriptions
User
https://api.github.com/users/julien-c
public
**Is your feature request related to a problem? Please describe.**

## The problem 

The datasets hub currently has `8,239` datasets. These datasets span a wide range of different modalities and tasks (currently with a bias towards textual data). 

There are various ways of identifying datasets that may be relevant for a particular use case:

- searching 
- various filters 

Currently, however, there isn't an easy way to identify datasets belonging to a specific domain. For example, I want to browse machine learning datasets related to 'social science' or 'climate change research'. 

The ability to identify datasets relating to a specific domain has come up in discussions around the [BigLA](https://github.com/bigscience-workshop/lam/) datasets hackathon https://github.com/bigscience-workshop/lam/discussions/31#discussioncomment-3123610. As part of the hackathon, we're currently collecting datasets related to Libraries, Archives and Museums and making them available via the hub. We currently do this under a Hugging Face organization (https://huggingface.co/biglam). However, going forward, I can see some of these datasets being migrated to sit under an organization that is the custodian of the dataset (for example, a national library the data was originally from). At this point, it becomes more difficult to quickly identify datasets from this domain without relying on search. 

This is also related to some existing issues on Github related to metadata on the hub:
- https://github.com/huggingface/datasets/issues/3625 
- https://github.com/huggingface/datasets/issues/3877

**Describe the solution you'd like**

### Some possible solutions that may help with this:

#### Enable domain tags (from a controlled vocabulary)
- This would add metadata field to the YAML for the domain a dataset relates to
- Advantages:
	- the list is controlled, allowing it to be more easily integrated into the datasets tag app (https://huggingface.co/space/huggingface/datasets-tagging) 
	- the controlled vocabulary could align with an existing controlled vocabulary 
	- this additional metadata can be used to perform filtering by domain 
- disadvantages 
	- choosing the best controlled vocab may be difficult
	- there are many datasets that are likely to fit into the 'machine learning' domain (i.e. there is a long tail of datasets that aren't in more 'generic' machine learning domain 

#### Enable topic tags (user-generated)

Enable 'free form' topic tags for datasets and models. This would be closer to GitHub's repository topics which can be chosen from a controlled list (https://github.com/topics/) but can also be more user/org specific. This could potentially be useful for organizations to also manage their own models and datasets as the number they hold in their org grows. For example, they may create 'topic tags' for a specific project, so it's clearer which datasets /models are related to that project. 


#### Collections

This solution would likely be the biggest shift and may require significant changes in the hub fronted. Collections could work in several different ways but would include:

Users can curate particular datasets, models, spaces, etc., into a collection. For example, they may create a collection of 'historic newspapers suitable for training language models'. These collections would not be mutually exclusive, i.e. a dataset can belong to zero, one or many collections. Collections can also potentially be nested under other collections.  

This is fairly common on other data reposotiores for example the following collections: 
<img width="293" alt="Screenshot 2022-07-18 at 11 50 44" src="https://user-images.githubusercontent.com/8995957/179496445-963ed122-5e26-4574-96e8-41081bce3e2b.png">

all belong under a higher level collection (https://bl.iro.bl.uk/collections/353c908d-b495-4413-b047-87236d2573e3?locale=en). 

There are different models one could use for how these collections could be created:

- only within an org
- for any dataset/model
- the owner or a dataset/model has to agree to be added to a collection
- a collection owner can have people suggest additions to their collection 
- other models....

These collections could be thematic, related to particular training approaches, curate models with particular inference properties etc. Whilst some of these features may duplicate current/or future tag filters on the hub, they offer the advantage of being flexible and not having to predict what users will want to do upfront. 

There is also potential for automating the creation of these collections based on existing metadata. For example, one could collect models trained on a collection of datasets so for example, if we had a collection of 'historic newspapers suitable for training language models' that contained 30 datasets, we could create another collection 'historic newspaper language models' that takes any model on the hub whose metadata says it used one or more of those 30 datasets. 

There is also the option of exploring ML approaches to suggest models/datasets may be relevant to a particular collection. 

This approach is likely to be quite difficult to implement well and would require significant thought. There is also likely to be a benefit in doing quite a bit of upfront work in curating useful collections to demonstrate the benefits of collections. 

 
**Describe alternatives you've considered**
A clear and concise description of any alternative solutions or features you've considered.

It is possible to collate this information externally, i.e. one could link back to the relevant models/datasets from an external platform. 

**Additional context**
Add any other context about the feature request here.

I'm cc'ing others involved in the BigLAM hackathon who may also have thoughts @cakiki @clancyoftheoverflow @albertvillanova 