TypeError: __init__() missing 1 required positional argument: 'scheme'
MEMBER
Hi @amirj, thanks for reporting.

At first sight, your issue seems a version incompatibility between your Elasticsearch client and your Elasticsearch server.

Feel free to have a look at Elasticsearch client docs: https://www.elastic.co/guide/en/elasticsearch/client/python-api/current/overview.html#_compatibility
> Language clients are forward compatible; meaning that clients support communicating with greater or equal minor versions of Elasticsearch. Elasticsearch language clients are only backwards compatible with default distributions and without guarantees made.
2022-03-17T12:20:20Z
https://github.com/huggingface/datasets/issues/3956#issuecomment-1070854702
https://api.github.com/repos/huggingface/datasets/issues/3956
IC_kwDODunzps4_0_Iu
https://api.github.com/repos/huggingface/datasets/issues/comments/1070854702/reactions
2022-03-17T12:20:20Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1070854702
https://avatars.githubusercontent.com/u/8515462?v=4
https://api.github.com/users/albertvillanova/events{/privacy}
https://api.github.com/users/albertvillanova/followers
https://api.github.com/users/albertvillanova/following{/other_user}
https://api.github.com/users/albertvillanova/gists{/gist_id}
https://github.com/albertvillanova
albertvillanova
MDQ6VXNlcjg1MTU0NjI=
https://api.github.com/users/albertvillanova/orgs
https://api.github.com/users/albertvillanova/received_events
https://api.github.com/users/albertvillanova/repos
https://api.github.com/users/albertvillanova/starred{/owner}{/repo}
https://api.github.com/users/albertvillanova/subscriptions
User
https://api.github.com/users/albertvillanova
public
NONE
@albertvillanova It doesn't seem a version incompatibility between the client and server, since the following code is working:

```
from elasticsearch import Elasticsearch
es_client = Elasticsearch("http://localhost:9200")
dataset.add_elasticsearch_index(column="e1", es_client=es_client, es_index_name="e1_index")
```
2022-03-17T16:59:32Z
https://github.com/huggingface/datasets/issues/3956#issuecomment-1071089881
https://api.github.com/repos/huggingface/datasets/issues/3956
IC_kwDODunzps4_14jZ
https://api.github.com/repos/huggingface/datasets/issues/comments/1071089881/reactions
2022-03-17T16:59:32Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1071089881
https://avatars.githubusercontent.com/u/1645137?v=4
https://api.github.com/users/amirj/events{/privacy}
https://api.github.com/users/amirj/followers
https://api.github.com/users/amirj/following{/other_user}
https://api.github.com/users/amirj/gists{/gist_id}
https://github.com/amirj
amirj
MDQ6VXNlcjE2NDUxMzc=
https://api.github.com/users/amirj/orgs
https://api.github.com/users/amirj/received_events
https://api.github.com/users/amirj/repos
https://api.github.com/users/amirj/starred{/owner}{/repo}
https://api.github.com/users/amirj/subscriptions
User
https://api.github.com/users/amirj
public
MEMBER
Hi @amirj, 

I really think it is a version incompatibility issue between your Elasticsearch client and server:
- Your Elasticsearch server NodeConfig expects a positional argument named 'scheme'
- Whereas your Elasticsearch client passes only keyword arguments: `NodeConfig(**options)`

Moreover:
- Looking at your stack trace, I deduce you are using Elasticsearch client **"8"** major version:
  - the Elasticsearch file "elasticsearch/_sync/client/utils.py" was created in version "8.0.0a1": https://github.com/elastic/elasticsearch-py/commit/21fa13b0f03b7b27ace9e19a1f763d40bd2e2ba4
  - you can check your Elasticsearch client version by running this Python code:
    ```python
    import elasticsearch
    print(elasticsearch.__version__)
    ```

- However, in the *Environment info*, you informed that the major version of your Eleasticsearch cluster server is **"7"** ("7.10.2-SNAPSHOT")

Could you please align the Elasticsearch client/server major versions (as pointed out in Elasticsearch docs) and check if the problem persists?
2022-03-18T07:57:36Z
https://github.com/huggingface/datasets/issues/3956#issuecomment-1072115173
https://api.github.com/repos/huggingface/datasets/issues/3956
IC_kwDODunzps4_5y3l
https://api.github.com/repos/huggingface/datasets/issues/comments/1072115173/reactions
2022-03-18T08:04:57Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1072115173
https://avatars.githubusercontent.com/u/8515462?v=4
https://api.github.com/users/albertvillanova/events{/privacy}
https://api.github.com/users/albertvillanova/followers
https://api.github.com/users/albertvillanova/following{/other_user}
https://api.github.com/users/albertvillanova/gists{/gist_id}
https://github.com/albertvillanova
albertvillanova
MDQ6VXNlcjg1MTU0NjI=
https://api.github.com/users/albertvillanova/orgs
https://api.github.com/users/albertvillanova/received_events
https://api.github.com/users/albertvillanova/repos
https://api.github.com/users/albertvillanova/starred{/owner}{/repo}
https://api.github.com/users/albertvillanova/subscriptions
User
https://api.github.com/users/albertvillanova
public
MEMBER
I'm closing this issue, @amirj.

Feel free to re-open it if the problem persists. 


2022-03-28T08:00:01Z
https://github.com/huggingface/datasets/issues/3956#issuecomment-1080320634
https://api.github.com/repos/huggingface/datasets/issues/3956
IC_kwDODunzps5AZGJ6
https://api.github.com/repos/huggingface/datasets/issues/comments/1080320634/reactions
2022-03-28T08:00:01Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1080320634
https://avatars.githubusercontent.com/u/8515462?v=4
https://api.github.com/users/albertvillanova/events{/privacy}
https://api.github.com/users/albertvillanova/followers
https://api.github.com/users/albertvillanova/following{/other_user}
https://api.github.com/users/albertvillanova/gists{/gist_id}
https://github.com/albertvillanova
albertvillanova
MDQ6VXNlcjg1MTU0NjI=
https://api.github.com/users/albertvillanova/orgs
https://api.github.com/users/albertvillanova/received_events
https://api.github.com/users/albertvillanova/repos
https://api.github.com/users/albertvillanova/starred{/owner}{/repo}
https://api.github.com/users/albertvillanova/subscriptions
User
https://api.github.com/users/albertvillanova
public
NONE
```
from elasticsearch import Elasticsearch
es = Elasticsearch([{'host': 'localhost', 'port': 9200}])
```
```
TypeError                                 Traceback (most recent call last)
<ipython-input-8-675c6ffe5293> in <module>
      1 #es = Elasticsearch([{'host':'localhost', 'port':9200}])
      2 from elasticsearch import Elasticsearch
----> 3 es = Elasticsearch([{'host': 'localhost', 'port': 9200}])

C:\ProgramData\Anaconda3\lib\site-packages\elasticsearch\_sync\client\__init__.py in __init__(self, hosts, cloud_id, api_key, basic_auth, bearer_auth, opaque_id, headers, connections_per_node, http_compress, verify_certs, ca_certs, client_cert, client_key, ssl_assert_hostname, ssl_assert_fingerprint, ssl_version, ssl_context, ssl_show_warn, transport_class, request_timeout, node_class, node_pool_class, randomize_nodes_in_pool, node_selector_class, dead_node_backoff_factor, max_dead_node_backoff, serializer, serializers, default_mimetype, max_retries, retry_on_status, retry_on_timeout, sniff_on_start, sniff_before_requests, sniff_on_node_failure, sniff_timeout, min_delay_between_sniffing, sniffed_node_callback, meta_header, timeout, randomize_hosts, host_info_callback, sniffer_timeout, sniff_on_connection_fail, http_auth, maxsize, _transport)
    310 
    311         if _transport is None:
--> 312             node_configs = client_node_configs(
    313                 hosts,
    314                 cloud_id=cloud_id,

C:\ProgramData\Anaconda3\lib\site-packages\elasticsearch\_sync\client\utils.py in client_node_configs(hosts, cloud_id, **kwargs)
     99     else:
    100         assert hosts is not None
--> 101         node_configs = hosts_to_node_configs(hosts)
    102 
    103     # Remove all values which are 'DEFAULT' to avoid overwriting actual defaults.

C:\ProgramData\Anaconda3\lib\site-packages\elasticsearch\_sync\client\utils.py in hosts_to_node_configs(hosts)
    142 
    143         elif isinstance(host, Mapping):
--> 144             node_configs.append(host_mapping_to_node_config(host))
    145         else:
    146             raise ValueError(

C:\ProgramData\Anaconda3\lib\site-packages\elasticsearch\_sync\client\utils.py in host_mapping_to_node_config(host)
    209         options["path_prefix"] = options.pop("url_prefix")
    210 
--> 211     return NodeConfig(**options)  # type: ignore
    212 
    213 

TypeError: __init__() missing 1 required positional argument: 'scheme'
```
2022-04-28T14:12:01Z
https://github.com/huggingface/datasets/issues/3956#issuecomment-1112257118
https://api.github.com/repos/huggingface/datasets/issues/3956
IC_kwDODunzps5CS7Je
https://api.github.com/repos/huggingface/datasets/issues/comments/1112257118/reactions
2022-05-04T16:34:14Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1112257118
https://avatars.githubusercontent.com/u/34819268?v=4
https://api.github.com/users/raj713335/events{/privacy}
https://api.github.com/users/raj713335/followers
https://api.github.com/users/raj713335/following{/other_user}
https://api.github.com/users/raj713335/gists{/gist_id}
https://github.com/raj713335
raj713335
MDQ6VXNlcjM0ODE5MjY4
https://api.github.com/users/raj713335/orgs
https://api.github.com/users/raj713335/received_events
https://api.github.com/users/raj713335/repos
https://api.github.com/users/raj713335/starred{/owner}{/repo}
https://api.github.com/users/raj713335/subscriptions
User
https://api.github.com/users/raj713335
public
NONE
I am facing the same issue, and version is same for the both i.e(8.1.3)
2022-04-28T14:13:01Z
https://github.com/huggingface/datasets/issues/3956#issuecomment-1112258390
https://api.github.com/repos/huggingface/datasets/issues/3956
IC_kwDODunzps5CS7dW
https://api.github.com/repos/huggingface/datasets/issues/comments/1112258390/reactions
2022-04-28T14:13:01Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1112258390
https://avatars.githubusercontent.com/u/34819268?v=4
https://api.github.com/users/raj713335/events{/privacy}
https://api.github.com/users/raj713335/followers
https://api.github.com/users/raj713335/following{/other_user}
https://api.github.com/users/raj713335/gists{/gist_id}
https://github.com/raj713335
raj713335
MDQ6VXNlcjM0ODE5MjY4
https://api.github.com/users/raj713335/orgs
https://api.github.com/users/raj713335/received_events
https://api.github.com/users/raj713335/repos
https://api.github.com/users/raj713335/starred{/owner}{/repo}
https://api.github.com/users/raj713335/subscriptions
User
https://api.github.com/users/raj713335
public
MEMBER
@raj713335, thanks for reporting.

Please note that in your code example, you are not using our `datasets` library. 

Thus, I think you should report that issue to `elasticsearch` library: https://github.com/elastic/elasticsearch-py


2022-05-04T16:37:10Z
https://github.com/huggingface/datasets/issues/3956#issuecomment-1117567206
https://api.github.com/repos/huggingface/datasets/issues/3956
IC_kwDODunzps5CnLjm
https://api.github.com/repos/huggingface/datasets/issues/comments/1117567206/reactions
2022-05-04T16:37:10Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1117567206
https://avatars.githubusercontent.com/u/8515462?v=4
https://api.github.com/users/albertvillanova/events{/privacy}
https://api.github.com/users/albertvillanova/followers
https://api.github.com/users/albertvillanova/following{/other_user}
https://api.github.com/users/albertvillanova/gists{/gist_id}
https://github.com/albertvillanova
albertvillanova
MDQ6VXNlcjg1MTU0NjI=
https://api.github.com/users/albertvillanova/orgs
https://api.github.com/users/albertvillanova/received_events
https://api.github.com/users/albertvillanova/repos
https://api.github.com/users/albertvillanova/starred{/owner}{/repo}
https://api.github.com/users/albertvillanova/subscriptions
User
https://api.github.com/users/albertvillanova
public
NONE
it is simple hack which shock you just replace    https to http in scheme

**In My Case:** ->

`es = Elasticsearch([{'host': 'localhost', 'port': 9200, "scheme": "http"}])
    if es.ping():
        print('Connected to ES!')
    else:
        print('Could not connect!')
        sys.exit()`
2023-11-21T04:25:38Z
https://github.com/huggingface/datasets/issues/3956#issuecomment-1820220479
https://api.github.com/repos/huggingface/datasets/issues/3956
IC_kwDODunzps5sflw_
https://api.github.com/repos/huggingface/datasets/issues/comments/1820220479/reactions
2023-11-21T04:26:20Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1820220479
https://avatars.githubusercontent.com/u/92209107?v=4
https://api.github.com/users/manishkr21/events{/privacy}
https://api.github.com/users/manishkr21/followers
https://api.github.com/users/manishkr21/following{/other_user}
https://api.github.com/users/manishkr21/gists{/gist_id}
https://github.com/manishkr21
manishkr21
U_kgDOBX7_0w
https://api.github.com/users/manishkr21/orgs
https://api.github.com/users/manishkr21/received_events
https://api.github.com/users/manishkr21/repos
https://api.github.com/users/manishkr21/starred{/owner}{/repo}
https://api.github.com/users/manishkr21/subscriptions
User
https://api.github.com/users/manishkr21
public
## Describe the bug
Based on [this tutorial](https://huggingface.co/docs/datasets/faiss_es#elasticsearch) the provided code should add Elasticsearch index but raised the following error, probably the new Elasticsearch version is not compatible though the tutorial doesn't provide any information about the supporting Elasticsearch version.

## Steps to reproduce the bug
```python
# Sample code to reproduce the bug
from datasets import load_dataset
squad = load_dataset('squad', split='validation')
squad.add_elasticsearch_index("context", host="localhost", port="9200")
```

## Expected results
[Creating an elastic index based on the provided tutorial](https://huggingface.co/docs/datasets/faiss_es#elasticsearch)

## Actual results
```
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-6-8fb51aa33961> in <module>
      1 from datasets import load_dataset
      2 squad = load_dataset('squad', split='validation')
----> 3 squad.add_elasticsearch_index("context", host="localhost", port="9200")

~/opt/anaconda3/lib/python3.8/site-packages/datasets/arrow_dataset.py in add_elasticsearch_index(self, column, index_name, host, port, es_client, es_index_name, es_index_config)
   3777         """
   3778         with self.formatted_as(type=None, columns=[column]):
-> 3779             super().add_elasticsearch_index(
   3780                 column=column,
   3781                 index_name=index_name,

~/opt/anaconda3/lib/python3.8/site-packages/datasets/search.py in add_elasticsearch_index(self, column, index_name, host, port, es_client, es_index_name, es_index_config)
    587         """
    588         index_name = index_name if index_name is not None else column
--> 589         es_index = ElasticSearchIndex(
    590             host=host, port=port, es_client=es_client, es_index_name=es_index_name, es_index_config=es_index_config
    591         )

~/opt/anaconda3/lib/python3.8/site-packages/datasets/search.py in __init__(self, host, port, es_client, es_index_name, es_index_config)
    123         from elasticsearch import Elasticsearch  # noqa: F811
    124 
--> 125         self.es_client = es_client if es_client is not None else Elasticsearch([{"host": host, "port": str(port)}])
    126         self.es_index_name = (
    127             es_index_name

~/opt/anaconda3/lib/python3.8/site-packages/elasticsearch/_sync/client/__init__.py in __init__(self, hosts, cloud_id, api_key, basic_auth, bearer_auth, opaque_id, headers, connections_per_node, http_compress, verify_certs, ca_certs, client_cert, client_key, ssl_assert_hostname, ssl_assert_fingerprint, ssl_version, ssl_context, ssl_show_warn, transport_class, request_timeout, node_class, node_pool_class, randomize_nodes_in_pool, node_selector_class, dead_node_backoff_factor, max_dead_node_backoff, serializer, serializers, default_mimetype, max_retries, retry_on_status, retry_on_timeout, sniff_on_start, sniff_before_requests, sniff_on_node_failure, sniff_timeout, min_delay_between_sniffing, sniffed_node_callback, meta_header, timeout, randomize_hosts, host_info_callback, sniffer_timeout, sniff_on_connection_fail, http_auth, maxsize, _transport)
    310 
    311         if _transport is None:
--> 312             node_configs = client_node_configs(
    313                 hosts,
    314                 cloud_id=cloud_id,

~/opt/anaconda3/lib/python3.8/site-packages/elasticsearch/_sync/client/utils.py in client_node_configs(hosts, cloud_id, **kwargs)
     99     else:
    100         assert hosts is not None
--> 101         node_configs = hosts_to_node_configs(hosts)
    102 
    103     # Remove all values which are 'DEFAULT' to avoid overwriting actual defaults.

~/opt/anaconda3/lib/python3.8/site-packages/elasticsearch/_sync/client/utils.py in hosts_to_node_configs(hosts)
    142 
    143         elif isinstance(host, Mapping):
--> 144             node_configs.append(host_mapping_to_node_config(host))
    145         else:
    146             raise ValueError(

~/opt/anaconda3/lib/python3.8/site-packages/elasticsearch/_sync/client/utils.py in host_mapping_to_node_config(host)
    209         options["path_prefix"] = options.pop("url_prefix")
    210 
--> 211     return NodeConfig(**options)  # type: ignore
    212 
    213 

TypeError: __init__() missing 1 required positional argument: 'scheme'
```

## Environment info
<!-- You can run the command `datasets-cli env` and copy-and-paste its output below. -->
- `datasets` version: 2.0.0
- Platform: Mac
- Python version: 3.8.0
- PyArrow version: 7.0.0
- ElaticSearch Info: 
{
  "name" : "byname",
  "cluster_name" : "elasticsearch_brew",
  "cluster_uuid" : "9xkjrltiQIG0J95ciWhqRA",
  "version" : {
    "number" : "7.10.2-SNAPSHOT",
    "build_flavor" : "oss",
    "build_type" : "tar",
    "build_hash" : "unknown",
    "build_date" : "2021-01-16T01:41:27.115673Z",
    "build_snapshot" : true,
    "lucene_version" : "8.7.0",
    "minimum_wire_compatibility_version" : "6.8.0",
    "minimum_index_compatibility_version" : "6.0.0-beta1"
  },
  "tagline" : "You Know, for Search"
}
