Add time series data - stock market
NONE
Can I use instructions present in below link for time series dataset as well? 
https://github.com/huggingface/datasets/blob/master/ADD_NEW_DATASET.md 
2022-04-06T05:48:07Z
https://github.com/huggingface/datasets/issues/4104#issuecomment-1089847213
https://api.github.com/repos/huggingface/datasets/issues/4104
IC_kwDODunzps5A9b-t
https://api.github.com/repos/huggingface/datasets/issues/comments/1089847213/reactions
2022-04-06T05:48:07Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1089847213
https://avatars.githubusercontent.com/u/45640029?v=4
https://api.github.com/users/rozeappletree/events{/privacy}
https://api.github.com/users/rozeappletree/followers
https://api.github.com/users/rozeappletree/following{/other_user}
https://api.github.com/users/rozeappletree/gists{/gist_id}
https://github.com/rozeappletree
rozeappletree
MDQ6VXNlcjQ1NjQwMDI5
https://api.github.com/users/rozeappletree/orgs
https://api.github.com/users/rozeappletree/received_events
https://api.github.com/users/rozeappletree/repos
https://api.github.com/users/rozeappletree/starred{/owner}{/repo}
https://api.github.com/users/rozeappletree/subscriptions
User
https://api.github.com/users/rozeappletree
public
MEMBER
cc'ing @kashif and @NielsRogge for visibility!
2022-04-06T07:11:47Z
https://github.com/huggingface/datasets/issues/4104#issuecomment-1089903469
https://api.github.com/repos/huggingface/datasets/issues/4104
IC_kwDODunzps5A9ptt
https://api.github.com/repos/huggingface/datasets/issues/comments/1089903469/reactions
2022-04-06T07:11:47Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1089903469
https://avatars.githubusercontent.com/u/326577?v=4
https://api.github.com/users/julien-c/events{/privacy}
https://api.github.com/users/julien-c/followers
https://api.github.com/users/julien-c/following{/other_user}
https://api.github.com/users/julien-c/gists{/gist_id}
https://github.com/julien-c
julien-c
MDQ6VXNlcjMyNjU3Nw==
https://api.github.com/users/julien-c/orgs
https://api.github.com/users/julien-c/received_events
https://api.github.com/users/julien-c/repos
https://api.github.com/users/julien-c/starred{/owner}{/repo}
https://api.github.com/users/julien-c/subscriptions
User
https://api.github.com/users/julien-c
public
CONTRIBUTOR
@INF800 happy to add this dataset! I will try to set a PR by the end of the day... if you can kindly point me to the dataset? Also, note we have a bunch of time series datasets checked in e.g. `electricity_load_diagrams` or `monash_tsf`, and ideally this dataset could also be in a similar format. 
2022-04-06T07:21:34Z
https://github.com/huggingface/datasets/issues/4104#issuecomment-1089911307
https://api.github.com/repos/huggingface/datasets/issues/4104
IC_kwDODunzps5A9roL
https://api.github.com/repos/huggingface/datasets/issues/comments/1089911307/reactions
2022-04-06T07:28:31Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1089911307
https://avatars.githubusercontent.com/u/8100?v=4
https://api.github.com/users/kashif/events{/privacy}
https://api.github.com/users/kashif/followers
https://api.github.com/users/kashif/following{/other_user}
https://api.github.com/users/kashif/gists{/gist_id}
https://github.com/kashif
kashif
MDQ6VXNlcjgxMDA=
https://api.github.com/users/kashif/orgs
https://api.github.com/users/kashif/received_events
https://api.github.com/users/kashif/repos
https://api.github.com/users/kashif/starred{/owner}{/repo}
https://api.github.com/users/kashif/subscriptions
User
https://api.github.com/users/kashif
public
NONE
Thankyou. This is how raw data looks like before cleaning for an individual stocks:

1. https://github.com/INF800/marktech/tree/raw-data/f/data/raw
2. https://github.com/INF800/marktech/tree/raw-data/t/data/raw
3. https://github.com/INF800/marktech/tree/raw-data/rdfn/data/raw
4. https://github.com/INF800/marktech/tree/raw-data/irbt/data/raw
5. https://github.com/INF800/marktech/tree/raw-data/hll/data/raw
6. https://github.com/INF800/marktech/tree/raw-data/infy/data/raw
7. https://github.com/INF800/marktech/tree/raw-data/reli/data/raw
8. https://github.com/INF800/marktech/tree/raw-data/hdbk/data/raw

> Scraping is automated using GitHub Actions. So, everyday we will see a new file added in the above links.

I can rewrite the cleaning scripts to make sure it fits HF dataset standards. (P.S I am very much new to HF dataset)

The data set above can be converted into univariate regression / multivariate regression / sequence to sequence generation dataset etc. So, do we have some kind of transformation modules that will read the dataset as some type of dataset (`GenericTimeData`) and convert it to other possible dataset relating to a specific ML task. **By having this kind of transformation module, I only have to add data once** and use transformation module whenever necessary

Additionally, having some kind of versioning for the dataset will be really helpful because it will keep on updating - especially time series datasets 
2022-04-06T07:56:04Z
https://github.com/huggingface/datasets/issues/4104#issuecomment-1089953924
https://api.github.com/repos/huggingface/datasets/issues/4104
IC_kwDODunzps5A92CE
https://api.github.com/repos/huggingface/datasets/issues/comments/1089953924/reactions
2022-04-06T08:59:24Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1089953924
https://avatars.githubusercontent.com/u/45640029?v=4
https://api.github.com/users/rozeappletree/events{/privacy}
https://api.github.com/users/rozeappletree/followers
https://api.github.com/users/rozeappletree/following{/other_user}
https://api.github.com/users/rozeappletree/gists{/gist_id}
https://github.com/rozeappletree
rozeappletree
MDQ6VXNlcjQ1NjQwMDI5
https://api.github.com/users/rozeappletree/orgs
https://api.github.com/users/rozeappletree/received_events
https://api.github.com/users/rozeappletree/repos
https://api.github.com/users/rozeappletree/starred{/owner}{/repo}
https://api.github.com/users/rozeappletree/subscriptions
User
https://api.github.com/users/rozeappletree
public
CONTRIBUTOR
thanks @INF800 I'll have a look. I believe it should be possible to incorporate this into the time-series format.
2022-04-06T08:17:14Z
https://github.com/huggingface/datasets/issues/4104#issuecomment-1089979005
https://api.github.com/repos/huggingface/datasets/issues/4104
IC_kwDODunzps5A98J9
https://api.github.com/repos/huggingface/datasets/issues/comments/1089979005/reactions
2022-04-06T08:17:14Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1089979005
https://avatars.githubusercontent.com/u/8100?v=4
https://api.github.com/users/kashif/events{/privacy}
https://api.github.com/users/kashif/followers
https://api.github.com/users/kashif/following{/other_user}
https://api.github.com/users/kashif/gists{/gist_id}
https://github.com/kashif
kashif
MDQ6VXNlcjgxMDA=
https://api.github.com/users/kashif/orgs
https://api.github.com/users/kashif/received_events
https://api.github.com/users/kashif/repos
https://api.github.com/users/kashif/starred{/owner}{/repo}
https://api.github.com/users/kashif/subscriptions
User
https://api.github.com/users/kashif
public
NONE
Referencing https://github.com/qingsongedu/time-series-transformers-review
2022-04-09T03:21:59Z
https://github.com/huggingface/datasets/issues/4104#issuecomment-1093634969
https://api.github.com/repos/huggingface/datasets/issues/4104
IC_kwDODunzps5BL4uZ
https://api.github.com/repos/huggingface/datasets/issues/comments/1093634969/reactions
2022-04-09T03:21:59Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1093634969
https://avatars.githubusercontent.com/u/45640029?v=4
https://api.github.com/users/rozeappletree/events{/privacy}
https://api.github.com/users/rozeappletree/followers
https://api.github.com/users/rozeappletree/following{/other_user}
https://api.github.com/users/rozeappletree/gists{/gist_id}
https://github.com/rozeappletree
rozeappletree
MDQ6VXNlcjQ1NjQwMDI5
https://api.github.com/users/rozeappletree/orgs
https://api.github.com/users/rozeappletree/received_events
https://api.github.com/users/rozeappletree/repos
https://api.github.com/users/rozeappletree/starred{/owner}{/repo}
https://api.github.com/users/rozeappletree/subscriptions
User
https://api.github.com/users/rozeappletree
public
CONTRIBUTOR
@INF800 yes I am aware of the review repository and paper which is more or less a collection of abstracts etc. I am working on a unified library of implementations of these papers together with datasets to be then able to compare/contrast and build upon the research etc. but I am not ready to share them publicly just yet.

In any case regarding your dataset at the moment its seems from looking at the csv files, its mixture of textual and numerical data, sometimes in the same column etc. As you know, for time series models we would need just numeric data so I would need your help in disambiguating the dataset you have collected and also perhaps starting with just numerical data to start with... 

Do you think you can make a version with just numerical data?
2022-04-11T08:50:54Z
https://github.com/huggingface/datasets/issues/4104#issuecomment-1094735590
https://api.github.com/repos/huggingface/datasets/issues/4104
IC_kwDODunzps5BQFbm
https://api.github.com/repos/huggingface/datasets/issues/comments/1094735590/reactions
2022-04-11T08:50:54Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1094735590
https://avatars.githubusercontent.com/u/8100?v=4
https://api.github.com/users/kashif/events{/privacy}
https://api.github.com/users/kashif/followers
https://api.github.com/users/kashif/following{/other_user}
https://api.github.com/users/kashif/gists{/gist_id}
https://github.com/kashif
kashif
MDQ6VXNlcjgxMDA=
https://api.github.com/users/kashif/orgs
https://api.github.com/users/kashif/received_events
https://api.github.com/users/kashif/repos
https://api.github.com/users/kashif/starred{/owner}{/repo}
https://api.github.com/users/kashif/subscriptions
User
https://api.github.com/users/kashif
public
NONE
> @INF800 yes I am aware of the review repository and paper which is more or less a collection of abstracts etc. I am working on a unified library of implementations of these papers together with datasets to be then able to compare/contrast and build upon the research etc. but I am not ready to share them publicly just yet.
> 
> In any case regarding your dataset at the moment its seems from looking at the csv files, its mixture of textual and numerical data, sometimes in the same column etc. As you know, for time series models we would need just numeric data so I would need your help in disambiguating the dataset you have collected and also perhaps starting with just numerical data to start with...
> 
> Do you think you can make a version with just numerical data?

Will share the numeric data and conversion script within end of this week. 

I am on a business trip currently - it is in my desktop.
2022-04-11T09:06:56Z
https://github.com/huggingface/datasets/issues/4104#issuecomment-1094768378
https://api.github.com/repos/huggingface/datasets/issues/4104
IC_kwDODunzps5BQNb6
https://api.github.com/repos/huggingface/datasets/issues/comments/1094768378/reactions
2022-04-11T09:07:10Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1094768378
https://avatars.githubusercontent.com/u/45640029?v=4
https://api.github.com/users/rozeappletree/events{/privacy}
https://api.github.com/users/rozeappletree/followers
https://api.github.com/users/rozeappletree/following{/other_user}
https://api.github.com/users/rozeappletree/gists{/gist_id}
https://github.com/rozeappletree
rozeappletree
MDQ6VXNlcjQ1NjQwMDI5
https://api.github.com/users/rozeappletree/orgs
https://api.github.com/users/rozeappletree/received_events
https://api.github.com/users/rozeappletree/repos
https://api.github.com/users/rozeappletree/starred{/owner}{/repo}
https://api.github.com/users/rozeappletree/subscriptions
User
https://api.github.com/users/rozeappletree
public
CONTRIBUTOR
thanks @INF800 kashif.rasul@gmail.com should work
2024-07-21T16:14:12Z
https://github.com/huggingface/datasets/issues/4104#issuecomment-2241701256
https://api.github.com/repos/huggingface/datasets/issues/4104
IC_kwDODunzps6FnaWI
https://api.github.com/repos/huggingface/datasets/issues/comments/2241701256/reactions
2024-07-21T16:14:12Z
https://api.github.com/repos/huggingface/datasets/issues/comments/2241701256
https://avatars.githubusercontent.com/u/8100?v=4
https://api.github.com/users/kashif/events{/privacy}
https://api.github.com/users/kashif/followers
https://api.github.com/users/kashif/following{/other_user}
https://api.github.com/users/kashif/gists{/gist_id}
https://github.com/kashif
kashif
MDQ6VXNlcjgxMDA=
https://api.github.com/users/kashif/orgs
https://api.github.com/users/kashif/received_events
https://api.github.com/users/kashif/repos
https://api.github.com/users/kashif/starred{/owner}{/repo}
https://api.github.com/users/kashif/subscriptions
User
https://api.github.com/users/kashif
public
NONE
It should be in your inbox!

On Sun, 21 Jul, 2024, 9:44 pm Kashif Rasul, ***@***.***>
wrote:

> thanks @INF800 <https://github.com/INF800> ***@***.*** should
> work
>
> —
> Reply to this email directly, view it on GitHub
> <https://github.com/huggingface/datasets/issues/4104#issuecomment-2241701256>,
> or unsubscribe
> <https://github.com/notifications/unsubscribe-auth/AK4GSXLHCOGNTU5ERJ6M3ITZNPM6TAVCNFSM6AAAAABLG65FLKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDENBRG4YDCMRVGY>
> .
> You are receiving this because you were mentioned.Message ID:
> ***@***.***>
>

2024-07-21T16:18:35Z
https://github.com/huggingface/datasets/issues/4104#issuecomment-2241702426
https://api.github.com/repos/huggingface/datasets/issues/4104
IC_kwDODunzps6Fnaoa
https://api.github.com/repos/huggingface/datasets/issues/comments/2241702426/reactions
2024-07-21T16:18:35Z
https://api.github.com/repos/huggingface/datasets/issues/comments/2241702426
https://avatars.githubusercontent.com/u/45640029?v=4
https://api.github.com/users/rozeappletree/events{/privacy}
https://api.github.com/users/rozeappletree/followers
https://api.github.com/users/rozeappletree/following{/other_user}
https://api.github.com/users/rozeappletree/gists{/gist_id}
https://github.com/rozeappletree
rozeappletree
MDQ6VXNlcjQ1NjQwMDI5
https://api.github.com/users/rozeappletree/orgs
https://api.github.com/users/rozeappletree/received_events
https://api.github.com/users/rozeappletree/repos
https://api.github.com/users/rozeappletree/starred{/owner}{/repo}
https://api.github.com/users/rozeappletree/subscriptions
User
https://api.github.com/users/rozeappletree
public
## Adding a Time Series Dataset
- **Name:** 2min ticker data for stock market 
- **Description:** 8 stocks' data collected for 1month post ukraine-russia war. 4 NSE stocks and 4 NASDAQ stocks. Along with technical indicators (additional features) as shown in below image
- **Data:** Collected by myself from investing.com
- **Motivation:** Test applicability of transformer based model on stock market / time series problem

![image](https://user-images.githubusercontent.com/45640029/161904077-52fe97cb-3720-4e3f-98ee-7f6720a056e2.png)