Multi process map did not load cache file correctly
NONE
The inconsistency may be caused by the usage of "update_fingerprint" and setting "trust_remote_code" to "True."
When the tokenizer employs "trust_remote_code," the behavior of the map function varies with each code execution. Even if the remote code of the tokenizer remains the same, the result of "asher.hexdigest()" is found to be inconsistent each time.
This may result in different processes executing multiple maps
![1698841094290](https://github.com/huggingface/datasets/assets/14285786/21fc3c65-e9fd-4a79-b12e-a1d4b9c6cf32)
![1698841117416](https://github.com/huggingface/datasets/assets/14285786/c3e5a530-54d2-4ae6-b902-ce9f85de373b)


2023-11-01T12:21:44Z
https://github.com/huggingface/datasets/issues/6369#issuecomment-1788863508
https://api.github.com/repos/huggingface/datasets/issues/6369
IC_kwDODunzps5qn-QU
https://api.github.com/repos/huggingface/datasets/issues/comments/1788863508/reactions
2023-11-01T12:21:44Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1788863508
https://avatars.githubusercontent.com/u/14285786?v=4
https://api.github.com/users/enze5088/events{/privacy}
https://api.github.com/users/enze5088/followers
https://api.github.com/users/enze5088/following{/other_user}
https://api.github.com/users/enze5088/gists{/gist_id}
https://github.com/enze5088
enze5088
MDQ6VXNlcjE0Mjg1Nzg2
https://api.github.com/users/enze5088/orgs
https://api.github.com/users/enze5088/received_events
https://api.github.com/users/enze5088/repos
https://api.github.com/users/enze5088/starred{/owner}{/repo}
https://api.github.com/users/enze5088/subscriptions
User
https://api.github.com/users/enze5088
public
NONE
The issue may be related to problems previously discussed in GitHub issues [#3847](https://github.com/huggingface/datasets/issues/3847) and [#6318](https://github.com/huggingface/datasets/pull/6318). 
This arises from the fact that tokenizer.tokens_trie._tokens is an unordered set, leading to varying hash results:
`value = hash_bytes(dumps(tokenizer.tokens_trie._tokens))`
Consequently, this results in different outcomes each time for:
`new_fingerprint = update_fingerprint(datasets._fingerprint, transform, kwargs_for_fingerprint)`

To address this issue, it's essential to make `Trie._tokens` a deterministic set while ensuring a consistent order after the final update of `_tokens`.

2023-11-01T18:33:54Z
https://github.com/huggingface/datasets/issues/6369#issuecomment-1789456749
https://api.github.com/repos/huggingface/datasets/issues/6369
IC_kwDODunzps5qqPFt
https://api.github.com/repos/huggingface/datasets/issues/comments/1789456749/reactions
2023-11-01T18:33:54Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1789456749
https://avatars.githubusercontent.com/u/14285786?v=4
https://api.github.com/users/enze5088/events{/privacy}
https://api.github.com/users/enze5088/followers
https://api.github.com/users/enze5088/following{/other_user}
https://api.github.com/users/enze5088/gists{/gist_id}
https://github.com/enze5088
enze5088
MDQ6VXNlcjE0Mjg1Nzg2
https://api.github.com/users/enze5088/orgs
https://api.github.com/users/enze5088/received_events
https://api.github.com/users/enze5088/repos
https://api.github.com/users/enze5088/starred{/owner}{/repo}
https://api.github.com/users/enze5088/subscriptions
User
https://api.github.com/users/enze5088
public
COLLABORATOR
We now sort `set` and `dict` items to make their hashes deterministic (install from `main` with `pip install git+https://github.com/huggingface/datasets` to test this). Consequently, this should also make the `tokenizer.tokens_trie`'s hash deterministic. Feel free to re-open the issue if this is not the case.
2023-11-30T16:04:45Z
https://github.com/huggingface/datasets/issues/6369#issuecomment-1834065822
https://api.github.com/repos/huggingface/datasets/issues/6369
IC_kwDODunzps5tUZ-e
https://api.github.com/repos/huggingface/datasets/issues/comments/1834065822/reactions
2023-11-30T16:04:45Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1834065822
https://avatars.githubusercontent.com/u/47462742?v=4
https://api.github.com/users/mariosasko/events{/privacy}
https://api.github.com/users/mariosasko/followers
https://api.github.com/users/mariosasko/following{/other_user}
https://api.github.com/users/mariosasko/gists{/gist_id}
https://github.com/mariosasko
mariosasko
MDQ6VXNlcjQ3NDYyNzQy
https://api.github.com/users/mariosasko/orgs
https://api.github.com/users/mariosasko/received_events
https://api.github.com/users/mariosasko/repos
https://api.github.com/users/mariosasko/starred{/owner}{/repo}
https://api.github.com/users/mariosasko/subscriptions
User
https://api.github.com/users/mariosasko
public
### Describe the bug

When I  was training model on Multiple GPUs by DDP, the dataset is tokenized multiple times after main process. 
![1698820541284](https://github.com/huggingface/datasets/assets/14285786/0b2fe054-54d8-4e00-96e6-6ca5b69e662b)

![1698820501568](https://github.com/huggingface/datasets/assets/14285786/dd62bf6f-a58f-41bf-9848-ea4fb3b62b9b)

Code is modified from  [run_clm.py](https://github.com/huggingface/transformers/blob/7d8ff3629b2725ec43ace99c1a6e87ac1978d433/examples/pytorch/language-modeling/run_clm.py#L484)


### Steps to reproduce the bug

```
block_size = data_args.block_size
IGNORE_INDEX = -100
Ignore_Input = False

def tokenize_function(examples):
    sources = []
    targets = []
    for instruction, inputs, output in zip(examples['instruction'], examples['input'], examples['output']):

        source = instruction + inputs 

        target = f"{output}{tokenizer.eos_token}"

        sources.append(source)
        targets.append(target)

    tokenized_sources = tokenizer(sources, return_attention_mask=False)

    tokenized_targets = tokenizer(targets, return_attention_mask=False,
                                  add_special_tokens=False
                                  )

    all_input_ids = []
    all_labels = []
    for s, t in zip(tokenized_sources['input_ids'], tokenized_targets['input_ids']):
        if len(s) > block_size and Ignore_Input == False:
            # print(s)
            continue
        input_ids = torch.LongTensor(s + t)[:block_size]
        if Ignore_Input:
            labels = torch.LongTensor([IGNORE_INDEX] * len(s) + t)[:block_size]
        else:
            labels = input_ids
        assert len(input_ids) == len(labels)
        all_input_ids.append(input_ids)
        all_labels.append(labels)

    results = {
        'input_ids': all_input_ids,
        'labels': all_labels,

    }
    return results



with training_args.main_process_first(desc="dataset map tokenization ", local=False):
    # print('local_rank',training_args.local_rank)
    if not data_args.streaming:
        tokenized_datasets = raw_datasets.map(
            tokenize_function,
            batched=True,
            num_proc=data_args.preprocessing_num_workers,
            remove_columns=column_names,
            load_from_cache_file=not data_args.overwrite_cache,
            desc="Running tokenizer on dataset ",
        )
    else:
        tokenized_datasets = raw_datasets.map(
            tokenize_function,
            batched=True,
            remove_columns=column_names,
            desc="Running tokenizer on dataset "
        )
```

### Expected behavior

This code should only tokenize the dataset in the main process, and the other processes load the dataset after waiting

### Environment info

transformers == 4.34.1
datasets == 2.14.5