diff --git "a/train.log" "b/train.log"
new file mode 100644--- /dev/null
+++ "b/train.log"
@@ -0,0 +1,2093 @@
+W0530 17:08:42.103000 10031 site-packages/torch/distributed/run.py:792] 
+W0530 17:08:42.103000 10031 site-packages/torch/distributed/run.py:792] *****************************************
+W0530 17:08:42.103000 10031 site-packages/torch/distributed/run.py:792] Setting OMP_NUM_THREADS environment variable for each process to be 1 in default, to avoid your system being overloaded, please further tune the variable for optimal performance in your application as needed. 
+W0530 17:08:42.103000 10031 site-packages/torch/distributed/run.py:792] *****************************************
+Trainer._get_train_sampler replaced with custom implementation.Trainer._get_train_sampler replaced with custom implementation.Trainer._get_train_sampler replaced with custom implementation.Trainer._get_train_sampler replaced with custom implementation.Trainer._get_train_sampler replaced with custom implementation.Trainer._get_train_sampler replaced with custom implementation.Trainer._get_train_sampler replaced with custom implementation.
+Trainer._get_train_sampler replaced with custom implementation.
+
+
+
+
+
+
+[2025-05-30 17:08:51,009] [INFO] [real_accelerator.py:222:get_accelerator] Setting ds_accelerator to cuda (auto detect)
+[2025-05-30 17:08:51,044] [INFO] [real_accelerator.py:222:get_accelerator] Setting ds_accelerator to cuda (auto detect)
+[2025-05-30 17:08:51,046] [INFO] [real_accelerator.py:222:get_accelerator] Setting ds_accelerator to cuda (auto detect)
+[2025-05-30 17:08:51,049] [INFO] [real_accelerator.py:222:get_accelerator] Setting ds_accelerator to cuda (auto detect)
+[2025-05-30 17:08:51,052] [INFO] [real_accelerator.py:222:get_accelerator] Setting ds_accelerator to cuda (auto detect)
+[2025-05-30 17:08:51,056] [INFO] [real_accelerator.py:222:get_accelerator] Setting ds_accelerator to cuda (auto detect)
+[2025-05-30 17:08:51,069] [INFO] [real_accelerator.py:222:get_accelerator] Setting ds_accelerator to cuda (auto detect)
+[2025-05-30 17:08:51,071] [INFO] [real_accelerator.py:222:get_accelerator] Setting ds_accelerator to cuda (auto detect)
+[2025-05-30 17:08:51,857] [INFO] [comm.py:658:init_distributed] cdb=None
+[2025-05-30 17:08:51,857] [INFO] [comm.py:658:init_distributed] cdb=None
+[2025-05-30 17:08:51,857] [INFO] [comm.py:658:init_distributed] cdb=None
+[2025-05-30 17:08:51,857] [INFO] [comm.py:658:init_distributed] cdb=None
+[2025-05-30 17:08:51,857] [INFO] [comm.py:658:init_distributed] cdb=None
+[2025-05-30 17:08:51,857] [INFO] [comm.py:658:init_distributed] cdb=None
+[2025-05-30 17:08:51,857] [INFO] [comm.py:658:init_distributed] cdb=None
+[2025-05-30 17:08:51,857] [INFO] [comm.py:658:init_distributed] cdb=None
+[2025-05-30 17:08:51,857] [INFO] [comm.py:689:init_distributed] Initializing TorchBackend in DeepSpeed with backend nccl
+You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`.
+You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`.
+You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`.
+You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`.
+You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`.
+You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`.
+You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`.
+You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`.
+
Loading checkpoint shards:   0%|          | 0/2 [00:00<?, ?it/s]
Loading checkpoint shards:   0%|          | 0/2 [00:00<?, ?it/s]
Loading checkpoint shards:   0%|          | 0/2 [00:00<?, ?it/s]
Loading checkpoint shards:   0%|          | 0/2 [00:00<?, ?it/s]
Loading checkpoint shards:   0%|          | 0/2 [00:00<?, ?it/s]
Loading checkpoint shards:   0%|          | 0/2 [00:00<?, ?it/s]
Loading checkpoint shards:   0%|          | 0/2 [00:00<?, ?it/s]
Loading checkpoint shards:   0%|          | 0/2 [00:00<?, ?it/s]
Loading checkpoint shards:  50%|█████     | 1/2 [00:00<00:00,  1.83it/s]
Loading checkpoint shards:  50%|█████     | 1/2 [00:00<00:00,  1.91it/s]
Loading checkpoint shards:  50%|█████     | 1/2 [00:00<00:00,  1.78it/s]
Loading checkpoint shards:  50%|█████     | 1/2 [00:00<00:00,  1.69it/s]
Loading checkpoint shards:  50%|█████     | 1/2 [00:00<00:00,  1.18it/s]
Loading checkpoint shards:  50%|█████     | 1/2 [00:00<00:00,  1.87it/s]
Loading checkpoint shards:  50%|█████     | 1/2 [00:00<00:00,  1.89it/s]
Loading checkpoint shards: 100%|██████████| 2/2 [00:03<00:00,  1.93s/it]
Loading checkpoint shards: 100%|██████████| 2/2 [00:03<00:00,  1.73s/it]
+Some weights of Qwen2_5_VLForConditionalGenerationWithVGGT were not initialized from the model checkpoint at /mnt/data0/zhengduo/Qwen2.5-VL-3B-Instruct/ and are newly initialized: ['merger.ln_q.weight', 'merger.mlp.0.bias', 'merger.mlp.0.weight', 'merger.mlp.2.bias', 'merger.mlp.2.weight']
+You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.
+
Loading checkpoint shards: 100%|██████████| 2/2 [00:03<00:00,  1.83s/it]
Loading checkpoint shards: 100%|██████████| 2/2 [00:03<00:00,  1.83s/it]
+
Loading checkpoint shards: 100%|██████████| 2/2 [00:04<00:00,  2.37s/it]Some weights of Qwen2_5_VLForConditionalGenerationWithVGGT were not initialized from the model checkpoint at /mnt/data0/zhengduo/Qwen2.5-VL-3B-Instruct/ and are newly initialized: ['merger.ln_q.weight', 'merger.mlp.0.bias', 'merger.mlp.0.weight', 'merger.mlp.2.bias', 'merger.mlp.2.weight']
+You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.
+
Loading checkpoint shards: 100%|██████████| 2/2 [00:04<00:00,  2.10s/it]
+
Loading checkpoint shards: 100%|██████████| 2/2 [00:04<00:00,  2.37s/it]
Loading checkpoint shards: 100%|██████████| 2/2 [00:04<00:00,  2.09s/it]
+Some weights of Qwen2_5_VLForConditionalGenerationWithVGGT were not initialized from the model checkpoint at /mnt/data0/zhengduo/Qwen2.5-VL-3B-Instruct/ and are newly initialized: ['merger.ln_q.weight', 'merger.mlp.0.bias', 'merger.mlp.0.weight', 'merger.mlp.2.bias', 'merger.mlp.2.weight']
+You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.
+Some weights of Qwen2_5_VLForConditionalGenerationWithVGGT were not initialized from the model checkpoint at /mnt/data0/zhengduo/Qwen2.5-VL-3B-Instruct/ and are newly initialized: ['merger.ln_q.weight', 'merger.mlp.0.bias', 'merger.mlp.0.weight', 'merger.mlp.2.bias', 'merger.mlp.2.weight']
+You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.
+
Loading checkpoint shards: 100%|██████████| 2/2 [00:04<00:00,  2.49s/it]
Loading checkpoint shards: 100%|██████████| 2/2 [00:04<00:00,  2.25s/it]
+Some weights of Qwen2_5_VLForConditionalGenerationWithVGGT were not initialized from the model checkpoint at /mnt/data0/zhengduo/Qwen2.5-VL-3B-Instruct/ and are newly initialized: ['merger.ln_q.weight', 'merger.mlp.0.bias', 'merger.mlp.0.weight', 'merger.mlp.2.bias', 'merger.mlp.2.weight']
+You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.
+
Loading checkpoint shards: 100%|██████████| 2/2 [00:04<00:00,  2.36s/it]
Loading checkpoint shards: 100%|██████████| 2/2 [00:04<00:00,  2.09s/it]
+
Loading checkpoint shards: 100%|██████████| 2/2 [00:04<00:00,  2.36s/it]Some weights of Qwen2_5_VLForConditionalGenerationWithVGGT were not initialized from the model checkpoint at /mnt/data0/zhengduo/Qwen2.5-VL-3B-Instruct/ and are newly initialized: ['merger.ln_q.weight', 'merger.mlp.0.bias', 'merger.mlp.0.weight', 'merger.mlp.2.bias', 'merger.mlp.2.weight']
+You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.
+
Loading checkpoint shards: 100%|██████████| 2/2 [00:04<00:00,  2.08s/it]
+
Loading checkpoint shards: 100%|██████████| 2/2 [00:04<00:00,  2.36s/it]
Loading checkpoint shards: 100%|██████████| 2/2 [00:04<00:00,  2.08s/it]Some weights of Qwen2_5_VLForConditionalGenerationWithVGGT were not initialized from the model checkpoint at /mnt/data0/zhengduo/Qwen2.5-VL-3B-Instruct/ and are newly initialized: ['merger.ln_q.weight', 'merger.mlp.0.bias', 'merger.mlp.0.weight', 'merger.mlp.2.bias', 'merger.mlp.2.weight']
+You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.
+
+Some weights of Qwen2_5_VLForConditionalGenerationWithVGGT were not initialized from the model checkpoint at /mnt/data0/zhengduo/Qwen2.5-VL-3B-Instruct/ and are newly initialized: ['merger.ln_q.weight', 'merger.mlp.0.bias', 'merger.mlp.0.weight', 'merger.mlp.2.bias', 'merger.mlp.2.weight']
+You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.
+Loading weights from local directory
+Loading weights from local directory
+Loading weights from local directory
+Loading weights from local directory
+Loading weights from local directory
+Loading weights from local directory
+Loading weights from local directory
+Loading weights from local directory
+Using a slow image processor as `use_fast` is unset and a slow processor was saved with this model. `use_fast=True` will be the default behavior in v4.50, even if the model was saved with a slow processor. This will result in minor differences in outputs. You'll still be able to use a slow processor with `use_fast=False`.
+Using a slow image processor as `use_fast` is unset and a slow processor was saved with this model. `use_fast=True` will be the default behavior in v4.50, even if the model was saved with a slow processor. This will result in minor differences in outputs. You'll still be able to use a slow processor with `use_fast=False`.
+Using a slow image processor as `use_fast` is unset and a slow processor was saved with this model. `use_fast=True` will be the default behavior in v4.50, even if the model was saved with a slow processor. This will result in minor differences in outputs. You'll still be able to use a slow processor with `use_fast=False`.
+Using a slow image processor as `use_fast` is unset and a slow processor was saved with this model. `use_fast=True` will be the default behavior in v4.50, even if the model was saved with a slow processor. This will result in minor differences in outputs. You'll still be able to use a slow processor with `use_fast=False`.
+Using a slow image processor as `use_fast` is unset and a slow processor was saved with this model. `use_fast=True` will be the default behavior in v4.50, even if the model was saved with a slow processor. This will result in minor differences in outputs. You'll still be able to use a slow processor with `use_fast=False`.
+Using a slow image processor as `use_fast` is unset and a slow processor was saved with this model. `use_fast=True` will be the default behavior in v4.50, even if the model was saved with a slow processor. This will result in minor differences in outputs. You'll still be able to use a slow processor with `use_fast=False`.
+Using a slow image processor as `use_fast` is unset and a slow processor was saved with this model. `use_fast=True` will be the default behavior in v4.50, even if the model was saved with a slow processor. This will result in minor differences in outputs. You'll still be able to use a slow processor with `use_fast=False`.
+Using a slow image processor as `use_fast` is unset and a slow processor was saved with this model. `use_fast=True` will be the default behavior in v4.50, even if the model was saved with a slow processor. This will result in minor differences in outputs. You'll still be able to use a slow processor with `use_fast=False`.
+Loading datasets: [{'annotation_path': 'data/annotation/llava_hound_255k.json', 'data_path': '/mnt/data0/zhengduo/data/', 'tag': '2d', 'sampling_rate': 0.25, 'dataset_name': 'llava_hound'}, {'annotation_path': 'data/annotation/spar_7m_train.jsonl', 'data_path': '/mnt/data0/zhengduo/data/', 'tag': '3d', 'sampling_rate': 0.03, 'dataset_name': 'spar'}]
+Loading datasets: [{'annotation_path': 'data/annotation/llava_hound_255k.json', 'data_path': '/mnt/data0/zhengduo/data/', 'tag': '2d', 'sampling_rate': 0.25, 'dataset_name': 'llava_hound'}, {'annotation_path': 'data/annotation/spar_7m_train.jsonl', 'data_path': '/mnt/data0/zhengduo/data/', 'tag': '3d', 'sampling_rate': 0.03, 'dataset_name': 'spar'}]
+Loading datasets: [{'annotation_path': 'data/annotation/llava_hound_255k.json', 'data_path': '/mnt/data0/zhengduo/data/', 'tag': '2d', 'sampling_rate': 0.25, 'dataset_name': 'llava_hound'}, {'annotation_path': 'data/annotation/spar_7m_train.jsonl', 'data_path': '/mnt/data0/zhengduo/data/', 'tag': '3d', 'sampling_rate': 0.03, 'dataset_name': 'spar'}]
+Loading datasets: [{'annotation_path': 'data/annotation/llava_hound_255k.json', 'data_path': '/mnt/data0/zhengduo/data/', 'tag': '2d', 'sampling_rate': 0.25, 'dataset_name': 'llava_hound'}, {'annotation_path': 'data/annotation/spar_7m_train.jsonl', 'data_path': '/mnt/data0/zhengduo/data/', 'tag': '3d', 'sampling_rate': 0.03, 'dataset_name': 'spar'}]
+Loading datasets: [{'annotation_path': 'data/annotation/llava_hound_255k.json', 'data_path': '/mnt/data0/zhengduo/data/', 'tag': '2d', 'sampling_rate': 0.25, 'dataset_name': 'llava_hound'}, {'annotation_path': 'data/annotation/spar_7m_train.jsonl', 'data_path': '/mnt/data0/zhengduo/data/', 'tag': '3d', 'sampling_rate': 0.03, 'dataset_name': 'spar'}]
+Vision Module - Attention Blocks:
+Trainable Block Indices: None
+Non-Trainable Block Indices: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31]
+Merger Module Trainable: False
+LLM Module - Embed Tokens Trainable: True
+LLM Module - Trainable Layer Indices: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35]
+LLM Module - Non-Trainable Layer Indices: None
+Loading datasets: [{'annotation_path': 'data/annotation/llava_hound_255k.json', 'data_path': '/mnt/data0/zhengduo/data/', 'tag': '2d', 'sampling_rate': 0.25, 'dataset_name': 'llava_hound'}, {'annotation_path': 'data/annotation/spar_7m_train.jsonl', 'data_path': '/mnt/data0/zhengduo/data/', 'tag': '3d', 'sampling_rate': 0.03, 'dataset_name': 'spar'}]
+Loading datasets: [{'annotation_path': 'data/annotation/llava_hound_255k.json', 'data_path': '/mnt/data0/zhengduo/data/', 'tag': '2d', 'sampling_rate': 0.25, 'dataset_name': 'llava_hound'}, {'annotation_path': 'data/annotation/spar_7m_train.jsonl', 'data_path': '/mnt/data0/zhengduo/data/', 'tag': '3d', 'sampling_rate': 0.03, 'dataset_name': 'spar'}]
+Loading datasets: [{'annotation_path': 'data/annotation/llava_hound_255k.json', 'data_path': '/mnt/data0/zhengduo/data/', 'tag': '2d', 'sampling_rate': 0.25, 'dataset_name': 'llava_hound'}, {'annotation_path': 'data/annotation/spar_7m_train.jsonl', 'data_path': '/mnt/data0/zhengduo/data/', 'tag': '3d', 'sampling_rate': 0.03, 'dataset_name': 'spar'}]
+sampling 63750 examples from dataset {'annotation_path': 'data/annotation/llava_hound_255k.json', 'data_path': '/mnt/data0/zhengduo/data/', 'tag': '2d', 'sampling_rate': 0.25, 'dataset_name': 'llava_hound'}
+sampling 63750 examples from dataset {'annotation_path': 'data/annotation/llava_hound_255k.json', 'data_path': '/mnt/data0/zhengduo/data/', 'tag': '2d', 'sampling_rate': 0.25, 'dataset_name': 'llava_hound'}
+sampling 63750 examples from dataset {'annotation_path': 'data/annotation/llava_hound_255k.json', 'data_path': '/mnt/data0/zhengduo/data/', 'tag': '2d', 'sampling_rate': 0.25, 'dataset_name': 'llava_hound'}
+sampling 63750 examples from dataset {'annotation_path': 'data/annotation/llava_hound_255k.json', 'data_path': '/mnt/data0/zhengduo/data/', 'tag': '2d', 'sampling_rate': 0.25, 'dataset_name': 'llava_hound'}
+sampling 63750 examples from dataset {'annotation_path': 'data/annotation/llava_hound_255k.json', 'data_path': '/mnt/data0/zhengduo/data/', 'tag': '2d', 'sampling_rate': 0.25, 'dataset_name': 'llava_hound'}
+sampling 63750 examples from dataset {'annotation_path': 'data/annotation/llava_hound_255k.json', 'data_path': '/mnt/data0/zhengduo/data/', 'tag': '2d', 'sampling_rate': 0.25, 'dataset_name': 'llava_hound'}
+sampling 63750 examples from dataset {'annotation_path': 'data/annotation/llava_hound_255k.json', 'data_path': '/mnt/data0/zhengduo/data/', 'tag': '2d', 'sampling_rate': 0.25, 'dataset_name': 'llava_hound'}
+sampling 63750 examples from dataset {'annotation_path': 'data/annotation/llava_hound_255k.json', 'data_path': '/mnt/data0/zhengduo/data/', 'tag': '2d', 'sampling_rate': 0.25, 'dataset_name': 'llava_hound'}
+sampling 234277 examples from dataset {'annotation_path': 'data/annotation/spar_7m_train.jsonl', 'data_path': '/mnt/data0/zhengduo/data/', 'tag': '3d', 'sampling_rate': 0.03, 'dataset_name': 'spar'}
+Total training samples: 298027
+Formatting inputs...Skip in lazy mode
+sampling 234277 examples from dataset {'annotation_path': 'data/annotation/spar_7m_train.jsonl', 'data_path': '/mnt/data0/zhengduo/data/', 'tag': '3d', 'sampling_rate': 0.03, 'dataset_name': 'spar'}
+sampling 234277 examples from dataset {'annotation_path': 'data/annotation/spar_7m_train.jsonl', 'data_path': '/mnt/data0/zhengduo/data/', 'tag': '3d', 'sampling_rate': 0.03, 'dataset_name': 'spar'}
+sampling 234277 examples from dataset {'annotation_path': 'data/annotation/spar_7m_train.jsonl', 'data_path': '/mnt/data0/zhengduo/data/', 'tag': '3d', 'sampling_rate': 0.03, 'dataset_name': 'spar'}
+Total training samples: 298027
+Total training samples: 298027
+Formatting inputs...Skip in lazy mode
+Total training samples: 298027
+Formatting inputs...Skip in lazy mode
+Formatting inputs...Skip in lazy mode
+sampling 234277 examples from dataset {'annotation_path': 'data/annotation/spar_7m_train.jsonl', 'data_path': '/mnt/data0/zhengduo/data/', 'tag': '3d', 'sampling_rate': 0.03, 'dataset_name': 'spar'}
+sampling 234277 examples from dataset {'annotation_path': 'data/annotation/spar_7m_train.jsonl', 'data_path': '/mnt/data0/zhengduo/data/', 'tag': '3d', 'sampling_rate': 0.03, 'dataset_name': 'spar'}
+Total training samples: 298027
+Total training samples: 298027
+Formatting inputs...Skip in lazy mode
+Formatting inputs...Skip in lazy mode
+sampling 234277 examples from dataset {'annotation_path': 'data/annotation/spar_7m_train.jsonl', 'data_path': '/mnt/data0/zhengduo/data/', 'tag': '3d', 'sampling_rate': 0.03, 'dataset_name': 'spar'}
+Total training samples: 298027
+Formatting inputs...Skip in lazy mode
+sampling 234277 examples from dataset {'annotation_path': 'data/annotation/spar_7m_train.jsonl', 'data_path': '/mnt/data0/zhengduo/data/', 'tag': '3d', 'sampling_rate': 0.03, 'dataset_name': 'spar'}
+Total training samples: 298027
+Formatting inputs...Skip in lazy mode
+
  0%|          | 0/18627 [00:00<?, ?it/s]/home/zhengduo/workspace/LLaMA-Factory/src/qwen_vl/model/modeling_qwen2_5_vl.py:1979: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
+  with torch.cuda.amp.autocast(dtype=dtype):
+/home/zhengduo/workspace/LLaMA-Factory/src/qwen_vl/model/modeling_qwen2_5_vl.py:1979: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
+  with torch.cuda.amp.autocast(dtype=dtype):
+/home/zhengduo/workspace/LLaMA-Factory/src/qwen_vl/model/modeling_qwen2_5_vl.py:1979: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
+  with torch.cuda.amp.autocast(dtype=dtype):
+/home/zhengduo/miniconda3/envs/vgllm/lib/python3.10/site-packages/torch/utils/checkpoint.py:87: UserWarning: None of the inputs have requires_grad=True. Gradients will be None
+  warnings.warn(
+/home/zhengduo/workspace/LLaMA-Factory/src/qwen_vl/model/modeling_qwen2_5_vl.py:1979: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
+  with torch.cuda.amp.autocast(dtype=dtype):
+/home/zhengduo/miniconda3/envs/vgllm/lib/python3.10/site-packages/torch/utils/checkpoint.py:87: UserWarning: None of the inputs have requires_grad=True. Gradients will be None
+  warnings.warn(
+/home/zhengduo/workspace/LLaMA-Factory/src/qwen_vl/model/modeling_qwen2_5_vl.py:1979: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
+  with torch.cuda.amp.autocast(dtype=dtype):
+/home/zhengduo/miniconda3/envs/vgllm/lib/python3.10/site-packages/torch/utils/checkpoint.py:87: UserWarning: None of the inputs have requires_grad=True. Gradients will be None
+  warnings.warn(
+/home/zhengduo/workspace/LLaMA-Factory/src/qwen_vl/model/modeling_qwen2_5_vl.py:1979: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
+  with torch.cuda.amp.autocast(dtype=dtype):
+/home/zhengduo/miniconda3/envs/vgllm/lib/python3.10/site-packages/torch/utils/checkpoint.py:87: UserWarning: None of the inputs have requires_grad=True. Gradients will be None
+  warnings.warn(
+/home/zhengduo/workspace/LLaMA-Factory/src/qwen_vl/model/modeling_qwen2_5_vl.py:1979: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
+  with torch.cuda.amp.autocast(dtype=dtype):
+/home/zhengduo/miniconda3/envs/vgllm/lib/python3.10/site-packages/torch/utils/checkpoint.py:87: UserWarning: None of the inputs have requires_grad=True. Gradients will be None
+  warnings.warn(
+/home/zhengduo/miniconda3/envs/vgllm/lib/python3.10/site-packages/torch/utils/checkpoint.py:87: UserWarning: None of the inputs have requires_grad=True. Gradients will be None
+  warnings.warn(
+/home/zhengduo/miniconda3/envs/vgllm/lib/python3.10/site-packages/torch/utils/checkpoint.py:87: UserWarning: None of the inputs have requires_grad=True. Gradients will be None
+  warnings.warn(
+/home/zhengduo/workspace/LLaMA-Factory/src/qwen_vl/model/modeling_qwen2_5_vl.py:1979: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
+  with torch.cuda.amp.autocast(dtype=dtype):
+/home/zhengduo/miniconda3/envs/vgllm/lib/python3.10/site-packages/torch/utils/checkpoint.py:87: UserWarning: None of the inputs have requires_grad=True. Gradients will be None
+  warnings.warn(
+
  0%|          | 1/18627 [00:09<51:02:05,  9.86s/it]
  0%|          | 2/18627 [00:11<24:39:42,  4.77s/it]
  0%|          | 3/18627 [00:12<16:14:02,  3.14s/it]
  0%|          | 4/18627 [00:14<14:14:16,  2.75s/it]
  0%|          | 5/18627 [00:16<13:31:38,  2.62s/it]
  0%|          | 6/18627 [00:18<12:40:11,  2.45s/it]
  0%|          | 7/18627 [00:21<12:21:19,  2.39s/it]
  0%|          | 8/18627 [00:22<10:22:48,  2.01s/it]
  0%|          | 9/18627 [00:24<10:37:55,  2.06s/it]
  0%|          | 10/18627 [00:26<10:43:41,  2.07s/it]
                                                     
{'loss': 3.5861, 'grad_norm': 45.460269927978516, 'learning_rate': 8.94454382826476e-08, 'epoch': 0.0}
+
  0%|          | 10/18627 [00:30<10:43:41,  2.07s/it]
  0%|          | 11/18627 [00:32<17:27:46,  3.38s/it]
  0%|          | 12/18627 [00:35<15:27:27,  2.99s/it]
  0%|          | 13/18627 [00:36<12:38:15,  2.44s/it]
  0%|          | 14/18627 [00:38<12:09:37,  2.35s/it]
  0%|          | 15/18627 [00:40<11:50:20,  2.29s/it]
  0%|          | 16/18627 [00:42<11:37:41,  2.25s/it]
  0%|          | 17/18627 [00:44<11:28:37,  2.22s/it]
  0%|          | 18/18627 [00:47<11:25:37,  2.21s/it]
  0%|          | 19/18627 [00:47<9:22:05,  1.81s/it] 
  0%|          | 20/18627 [00:49<8:17:23,  1.60s/it]
                                                    
{'loss': 3.7699, 'grad_norm': 24.043258666992188, 'learning_rate': 1.788908765652952e-07, 'epoch': 0.0}
+
  0%|          | 20/18627 [00:49<8:17:23,  1.60s/it]
  0%|          | 21/18627 [00:50<7:41:07,  1.49s/it]
  0%|          | 22/18627 [00:52<8:41:21,  1.68s/it]
  0%|          | 23/18627 [00:54<9:45:47,  1.89s/it]
  0%|          | 24/18627 [00:55<8:42:27,  1.69s/it]
  0%|          | 25/18627 [00:57<7:56:01,  1.54s/it]
  0%|          | 26/18627 [00:59<9:14:36,  1.79s/it]
  0%|          | 27/18627 [01:01<9:58:39,  1.93s/it]
  0%|          | 28/18627 [01:03<10:15:29,  1.99s/it]
  0%|          | 29/18627 [01:06<10:41:49,  2.07s/it]
  0%|          | 30/18627 [01:08<10:46:45,  2.09s/it]
                                                     
{'loss': 3.9194, 'grad_norm': 31.925207138061523, 'learning_rate': 2.6833631484794277e-07, 'epoch': 0.0}
+
  0%|          | 30/18627 [01:08<10:46:45,  2.09s/it]
  0%|          | 31/18627 [01:09<8:39:39,  1.68s/it] 
  0%|          | 32/18627 [01:11<9:33:21,  1.85s/it]
  0%|          | 33/18627 [01:13<10:00:06,  1.94s/it]
  0%|          | 34/18627 [01:15<10:23:51,  2.01s/it]
  0%|          | 35/18627 [01:17<10:33:05,  2.04s/it]
  0%|          | 36/18627 [01:19<10:52:13,  2.10s/it]
  0%|          | 37/18627 [01:22<10:56:01,  2.12s/it]
  0%|          | 38/18627 [01:24<10:56:54,  2.12s/it]
  0%|          | 39/18627 [01:26<10:59:25,  2.13s/it]
  0%|          | 40/18627 [01:27<9:38:20,  1.87s/it] 
                                                    
{'loss': 3.9822, 'grad_norm': 25.162328720092773, 'learning_rate': 3.577817531305904e-07, 'epoch': 0.0}
+
  0%|          | 40/18627 [01:27<9:38:20,  1.87s/it]
  0%|          | 41/18627 [01:29<10:11:07,  1.97s/it]
  0%|          | 42/18627 [01:32<10:28:40,  2.03s/it]
  0%|          | 43/18627 [01:34<10:38:55,  2.06s/it]
  0%|          | 44/18627 [01:36<10:47:54,  2.09s/it]
  0%|          | 45/18627 [01:37<9:23:26,  1.82s/it] 
  0%|          | 46/18627 [01:38<8:26:06,  1.63s/it]
  0%|          | 47/18627 [01:40<9:10:59,  1.78s/it]
  0%|          | 48/18627 [01:42<8:17:33,  1.61s/it]
  0%|          | 49/18627 [01:44<9:04:12,  1.76s/it]
  0%|          | 50/18627 [01:46<9:43:07,  1.88s/it]
                                                    
{'loss': 3.6521, 'grad_norm': 35.98725509643555, 'learning_rate': 4.47227191413238e-07, 'epoch': 0.0}
+
  0%|          | 50/18627 [01:46<9:43:07,  1.88s/it]
  0%|          | 51/18627 [01:47<8:41:25,  1.68s/it]
  0%|          | 52/18627 [01:49<9:21:00,  1.81s/it]
  0%|          | 53/18627 [01:51<9:49:13,  1.90s/it]
  0%|          | 54/18627 [01:53<10:08:01,  1.96s/it]
  0%|          | 55/18627 [01:56<10:23:43,  2.02s/it]
  0%|          | 56/18627 [01:58<10:33:05,  2.05s/it]
  0%|          | 57/18627 [02:00<10:38:03,  2.06s/it]
  0%|          | 58/18627 [02:02<10:43:33,  2.08s/it]
  0%|          | 59/18627 [02:03<9:22:06,  1.82s/it] 
  0%|          | 60/18627 [02:05<10:02:52,  1.95s/it]
                                                     
{'loss': 3.5772, 'grad_norm': 29.28506088256836, 'learning_rate': 5.366726296958855e-07, 'epoch': 0.0}
+
  0%|          | 60/18627 [02:05<10:02:52,  1.95s/it]
  0%|          | 61/18627 [02:07<10:19:38,  2.00s/it]
  0%|          | 62/18627 [02:10<10:30:50,  2.04s/it]
  0%|          | 63/18627 [02:12<10:39:36,  2.07s/it]
  0%|          | 64/18627 [02:14<10:54:53,  2.12s/it]
  0%|          | 65/18627 [02:16<11:01:39,  2.14s/it]
  0%|          | 66/18627 [02:18<11:00:51,  2.14s/it]
  0%|          | 67/18627 [02:19<9:34:08,  1.86s/it] 
  0%|          | 68/18627 [02:22<10:16:24,  1.99s/it]
  0%|          | 69/18627 [02:24<10:28:06,  2.03s/it]
  0%|          | 70/18627 [02:26<10:36:39,  2.06s/it]
                                                     
{'loss': 3.8221, 'grad_norm': 31.444040298461914, 'learning_rate': 6.26118067978533e-07, 'epoch': 0.0}
+
  0%|          | 70/18627 [02:26<10:36:39,  2.06s/it]
  0%|          | 71/18627 [02:28<10:41:01,  2.07s/it]
  0%|          | 72/18627 [02:30<10:47:26,  2.09s/it]
  0%|          | 73/18627 [02:32<10:48:01,  2.10s/it]
  0%|          | 74/18627 [02:35<10:53:07,  2.11s/it]
  0%|          | 75/18627 [02:35<8:42:07,  1.69s/it] 
  0%|          | 76/18627 [02:36<7:56:50,  1.54s/it]
  0%|          | 77/18627 [02:39<8:50:33,  1.72s/it]
  0%|          | 78/18627 [02:41<9:27:17,  1.83s/it]
  0%|          | 79/18627 [02:43<9:55:23,  1.93s/it]
  0%|          | 80/18627 [02:44<8:47:36,  1.71s/it]
                                                    
{'loss': 3.6556, 'grad_norm': 24.413436889648438, 'learning_rate': 7.155635062611808e-07, 'epoch': 0.0}
+
  0%|          | 80/18627 [02:44<8:47:36,  1.71s/it]
  0%|          | 81/18627 [02:46<9:26:34,  1.83s/it]
  0%|          | 82/18627 [02:48<9:51:59,  1.92s/it]
  0%|          | 83/18627 [02:50<10:08:41,  1.97s/it]
  0%|          | 84/18627 [02:52<10:22:28,  2.01s/it]
  0%|          | 85/18627 [02:55<10:40:03,  2.07s/it]
  0%|          | 86/18627 [02:57<10:46:23,  2.09s/it]
  0%|          | 87/18627 [02:59<10:47:56,  2.10s/it]
  0%|          | 88/18627 [03:01<10:49:27,  2.10s/it]
  0%|          | 89/18627 [03:02<8:48:40,  1.71s/it] 
  0%|          | 90/18627 [03:04<9:26:39,  1.83s/it]
                                                    
{'loss': 3.0963, 'grad_norm': 25.861753463745117, 'learning_rate': 8.050089445438284e-07, 'epoch': 0.0}
+
  0%|          | 90/18627 [03:04<9:26:39,  1.83s/it]
  0%|          | 91/18627 [03:06<9:53:04,  1.92s/it]
  0%|          | 92/18627 [03:08<10:11:24,  1.98s/it]
  0%|          | 93/18627 [03:10<10:22:29,  2.02s/it]
  1%|          | 94/18627 [03:13<10:45:48,  2.09s/it]
  1%|          | 95/18627 [03:15<10:51:22,  2.11s/it]
  1%|          | 96/18627 [03:16<9:25:50,  1.83s/it] 
  1%|          | 97/18627 [03:18<9:51:04,  1.91s/it]
  1%|          | 98/18627 [03:19<8:45:17,  1.70s/it]
  1%|          | 99/18627 [03:21<9:23:11,  1.82s/it]
  1%|          | 100/18627 [03:23<9:54:45,  1.93s/it]
                                                     
{'loss': 3.1698, 'grad_norm': 25.89274787902832, 'learning_rate': 8.94454382826476e-07, 'epoch': 0.01}
+
  1%|          | 100/18627 [03:23<9:54:45,  1.93s/it]
  1%|          | 101/18627 [03:26<10:12:01,  1.98s/it]
  1%|          | 102/18627 [03:27<9:00:31,  1.75s/it] 
  1%|          | 103/18627 [03:29<9:37:41,  1.87s/it]
  1%|          | 104/18627 [03:31<10:01:45,  1.95s/it]
  1%|          | 105/18627 [03:33<10:18:34,  2.00s/it]
  1%|          | 106/18627 [03:35<10:32:39,  2.05s/it]
  1%|          | 107/18627 [03:37<10:40:42,  2.08s/it]
  1%|          | 108/18627 [03:40<10:45:47,  2.09s/it]
  1%|          | 109/18627 [03:42<11:10:01,  2.17s/it]
  1%|          | 110/18627 [03:44<11:04:32,  2.15s/it]
                                                      
{'loss': 2.9765, 'grad_norm': 22.787872314453125, 'learning_rate': 9.838998211091236e-07, 'epoch': 0.01}
+
  1%|          | 110/18627 [03:44<11:04:32,  2.15s/it]
  1%|          | 111/18627 [03:46<11:00:18,  2.14s/it]
  1%|          | 112/18627 [03:48<10:57:08,  2.13s/it]
  1%|          | 113/18627 [03:50<11:00:47,  2.14s/it]
  1%|          | 114/18627 [03:52<9:34:10,  1.86s/it] 
  1%|          | 115/18627 [03:54<9:57:44,  1.94s/it]
  1%|          | 116/18627 [03:55<8:48:58,  1.71s/it]
  1%|          | 117/18627 [03:57<9:24:48,  1.83s/it]
  1%|          | 118/18627 [03:59<9:51:32,  1.92s/it]
  1%|          | 119/18627 [04:01<10:09:33,  1.98s/it]
  1%|          | 120/18627 [04:03<10:26:35,  2.03s/it]
                                                      
{'loss': 3.1309, 'grad_norm': 22.402860641479492, 'learning_rate': 1.073345259391771e-06, 'epoch': 0.01}
+
  1%|          | 120/18627 [04:03<10:26:35,  2.03s/it]
  1%|          | 121/18627 [04:06<10:36:14,  2.06s/it]
  1%|          | 122/18627 [04:08<10:42:18,  2.08s/it]
  1%|          | 123/18627 [04:10<10:45:32,  2.09s/it]
  1%|          | 124/18627 [04:11<9:05:59,  1.77s/it] 
  1%|          | 125/18627 [04:13<9:38:45,  1.88s/it]
  1%|          | 126/18627 [04:15<10:02:22,  1.95s/it]
  1%|          | 127/18627 [04:17<10:19:59,  2.01s/it]
  1%|          | 128/18627 [04:18<8:20:40,  1.62s/it] 
  1%|          | 129/18627 [04:20<9:07:34,  1.78s/it]
  1%|          | 130/18627 [04:22<9:38:29,  1.88s/it]
                                                     
{'loss': 3.0069, 'grad_norm': 19.699426651000977, 'learning_rate': 1.1627906976744188e-06, 'epoch': 0.01}
+
  1%|          | 130/18627 [04:22<9:38:29,  1.88s/it]
  1%|          | 131/18627 [04:24<10:00:40,  1.95s/it]
  1%|          | 132/18627 [04:27<10:18:06,  2.01s/it]
  1%|          | 133/18627 [04:29<10:30:38,  2.05s/it]
  1%|          | 134/18627 [04:31<10:37:04,  2.07s/it]
  1%|          | 135/18627 [04:33<10:42:19,  2.08s/it]
  1%|          | 136/18627 [04:34<8:33:29,  1.67s/it] 
  1%|          | 137/18627 [04:36<9:15:56,  1.80s/it]
  1%|          | 138/18627 [04:38<9:44:23,  1.90s/it]
  1%|          | 139/18627 [04:39<8:40:34,  1.69s/it]
  1%|          | 140/18627 [04:41<9:18:19,  1.81s/it]
                                                     
{'loss': 2.9675, 'grad_norm': 20.756311416625977, 'learning_rate': 1.252236135957066e-06, 'epoch': 0.01}
+
  1%|          | 140/18627 [04:41<9:18:19,  1.81s/it]
  1%|          | 141/18627 [04:43<9:47:26,  1.91s/it]
  1%|          | 142/18627 [04:45<10:05:25,  1.97s/it]
  1%|          | 143/18627 [04:47<8:54:16,  1.73s/it] 
  1%|          | 144/18627 [04:49<9:30:37,  1.85s/it]
  1%|          | 145/18627 [04:50<8:28:45,  1.65s/it]
  1%|          | 146/18627 [04:52<9:14:18,  1.80s/it]
  1%|          | 147/18627 [04:54<9:44:22,  1.90s/it]
  1%|          | 148/18627 [04:55<8:28:03,  1.65s/it]
  1%|          | 149/18627 [04:57<9:13:38,  1.80s/it]
  1%|          | 150/18627 [04:59<8:17:42,  1.62s/it]
                                                     
{'loss': 2.8854, 'grad_norm': 21.800580978393555, 'learning_rate': 1.3416815742397138e-06, 'epoch': 0.01}
+
  1%|          | 150/18627 [04:59<8:17:42,  1.62s/it]
  1%|          | 151/18627 [05:01<9:05:42,  1.77s/it]
  1%|          | 152/18627 [05:02<8:12:41,  1.60s/it]
  1%|          | 153/18627 [05:04<9:10:32,  1.79s/it]
  1%|          | 154/18627 [05:06<9:41:33,  1.89s/it]
  1%|          | 155/18627 [05:07<8:36:58,  1.68s/it]
  1%|          | 156/18627 [05:10<9:17:07,  1.81s/it]
  1%|          | 157/18627 [05:12<9:43:52,  1.90s/it]
  1%|          | 158/18627 [05:14<10:07:38,  1.97s/it]
  1%|          | 159/18627 [05:16<10:21:37,  2.02s/it]
  1%|          | 160/18627 [05:18<10:32:04,  2.05s/it]
                                                      
{'loss': 2.7602, 'grad_norm': 22.354496002197266, 'learning_rate': 1.4311270125223615e-06, 'epoch': 0.01}
+
  1%|          | 160/18627 [05:18<10:32:04,  2.05s/it]
  1%|          | 161/18627 [05:20<10:37:18,  2.07s/it]
  1%|          | 162/18627 [05:21<9:06:25,  1.78s/it] 
  1%|          | 163/18627 [05:23<9:43:38,  1.90s/it]
  1%|          | 164/18627 [05:26<10:05:38,  1.97s/it]
  1%|          | 165/18627 [05:28<10:21:00,  2.02s/it]
  1%|          | 166/18627 [05:29<9:05:11,  1.77s/it] 
  1%|          | 167/18627 [05:31<9:38:26,  1.88s/it]
  1%|          | 168/18627 [05:33<10:20:43,  2.02s/it]
  1%|          | 169/18627 [05:35<10:32:32,  2.06s/it]
  1%|          | 170/18627 [05:38<10:39:22,  2.08s/it]
                                                      
{'loss': 2.503, 'grad_norm': 18.408493041992188, 'learning_rate': 1.520572450805009e-06, 'epoch': 0.01}
+
  1%|          | 170/18627 [05:38<10:39:22,  2.08s/it]
  1%|          | 171/18627 [05:40<10:44:19,  2.09s/it]
  1%|          | 172/18627 [05:42<10:47:28,  2.11s/it]
  1%|          | 173/18627 [05:44<10:48:32,  2.11s/it]
  1%|          | 174/18627 [05:46<10:49:36,  2.11s/it]
  1%|          | 175/18627 [05:47<9:24:32,  1.84s/it] 
  1%|          | 176/18627 [05:48<8:09:37,  1.59s/it]
  1%|          | 177/18627 [05:50<9:00:05,  1.76s/it]
  1%|          | 178/18627 [05:53<9:32:18,  1.86s/it]
  1%|          | 179/18627 [05:55<9:56:30,  1.94s/it]
  1%|          | 180/18627 [05:56<8:46:49,  1.71s/it]
                                                     
{'loss': 2.4386, 'grad_norm': 17.4278507232666, 'learning_rate': 1.6100178890876567e-06, 'epoch': 0.01}
+
  1%|          | 180/18627 [05:56<8:46:49,  1.71s/it]
  1%|          | 181/18627 [05:58<9:24:34,  1.84s/it]
  1%|          | 182/18627 [06:00<9:51:24,  1.92s/it]
  1%|          | 183/18627 [06:02<10:08:20,  1.98s/it]
  1%|          | 184/18627 [06:03<8:55:24,  1.74s/it] 
  1%|          | 185/18627 [06:06<9:53:03,  1.93s/it]
  1%|          | 186/18627 [06:07<8:05:57,  1.58s/it]
  1%|          | 187/18627 [06:09<9:01:28,  1.76s/it]
  1%|          | 188/18627 [06:11<9:31:53,  1.86s/it]
  1%|          | 189/18627 [06:13<9:54:41,  1.94s/it]
  1%|          | 190/18627 [06:15<10:10:48,  1.99s/it]
                                                      
{'loss': 2.4034, 'grad_norm': 17.149494171142578, 'learning_rate': 1.6994633273703042e-06, 'epoch': 0.01}
+
  1%|          | 190/18627 [06:15<10:10:48,  1.99s/it]
  1%|          | 191/18627 [06:16<8:57:01,  1.75s/it] 
  1%|          | 192/18627 [06:18<9:30:31,  1.86s/it]
  1%|          | 193/18627 [06:20<9:52:03,  1.93s/it]
  1%|          | 194/18627 [06:23<10:08:27,  1.98s/it]
  1%|          | 195/18627 [06:25<10:20:14,  2.02s/it]
  1%|          | 196/18627 [06:27<10:29:42,  2.05s/it]
  1%|          | 197/18627 [06:29<10:40:51,  2.09s/it]
  1%|          | 198/18627 [06:31<10:47:51,  2.11s/it]
  1%|          | 199/18627 [06:33<10:52:11,  2.12s/it]
  1%|          | 200/18627 [06:34<9:26:54,  1.85s/it] 
                                                     
{'loss': 2.5062, 'grad_norm': 19.130016326904297, 'learning_rate': 1.788908765652952e-06, 'epoch': 0.01}
+
  1%|          | 200/18627 [06:34<9:26:54,  1.85s/it]
  1%|          | 201/18627 [06:37<9:55:56,  1.94s/it]
  1%|          | 202/18627 [06:39<10:16:49,  2.01s/it]
  1%|          | 203/18627 [06:40<8:23:28,  1.64s/it] 
  1%|          | 204/18627 [06:42<9:10:03,  1.79s/it]
  1%|          | 205/18627 [06:43<8:14:17,  1.61s/it]
  1%|          | 206/18627 [06:44<7:36:16,  1.49s/it]
  1%|          | 207/18627 [06:46<8:35:05,  1.68s/it]
  1%|          | 208/18627 [06:48<9:15:56,  1.81s/it]
  1%|          | 209/18627 [06:50<9:43:32,  1.90s/it]
  1%|          | 210/18627 [06:53<10:04:42,  1.97s/it]
                                                      
{'loss': 2.3067, 'grad_norm': 16.2971248626709, 'learning_rate': 1.8783542039355994e-06, 'epoch': 0.01}
+
  1%|          | 210/18627 [06:53<10:04:42,  1.97s/it]
  1%|          | 211/18627 [06:54<8:52:10,  1.73s/it] 
  1%|          | 212/18627 [06:56<9:28:15,  1.85s/it]
  1%|          | 213/18627 [06:57<8:28:25,  1.66s/it]
  1%|          | 214/18627 [06:59<9:11:18,  1.80s/it]
  1%|          | 215/18627 [07:01<9:40:04,  1.89s/it]
  1%|          | 216/18627 [07:03<8:35:08,  1.68s/it]
  1%|          | 217/18627 [07:05<9:16:58,  1.82s/it]
  1%|          | 218/18627 [07:06<8:19:22,  1.63s/it]
  1%|          | 219/18627 [07:07<7:38:36,  1.49s/it]
  1%|          | 220/18627 [07:08<7:10:35,  1.40s/it]
                                                     
{'loss': 2.7697, 'grad_norm': 18.11786460876465, 'learning_rate': 1.967799642218247e-06, 'epoch': 0.01}
+
  1%|          | 220/18627 [07:08<7:10:35,  1.40s/it]
  1%|          | 221/18627 [07:09<6:52:20,  1.34s/it]
  1%|          | 222/18627 [07:11<6:38:56,  1.30s/it]
  1%|          | 223/18627 [07:13<7:54:47,  1.55s/it]
  1%|          | 224/18627 [07:15<8:48:32,  1.72s/it]
  1%|          | 225/18627 [07:17<9:24:42,  1.84s/it]
  1%|          | 226/18627 [07:19<9:48:42,  1.92s/it]
  1%|          | 227/18627 [07:21<10:03:49,  1.97s/it]
  1%|          | 228/18627 [07:23<10:17:31,  2.01s/it]
  1%|          | 229/18627 [07:25<10:25:38,  2.04s/it]
  1%|          | 230/18627 [07:28<10:34:01,  2.07s/it]
                                                      
{'loss': 2.2275, 'grad_norm': 17.429473876953125, 'learning_rate': 2.0572450805008946e-06, 'epoch': 0.01}
+
  1%|          | 230/18627 [07:28<10:34:01,  2.07s/it]
  1%|          | 231/18627 [07:29<9:13:36,  1.81s/it] 
  1%|          | 232/18627 [07:30<8:19:20,  1.63s/it]
  1%|▏         | 233/18627 [07:32<9:06:20,  1.78s/it]
  1%|▏         | 234/18627 [07:34<9:41:43,  1.90s/it]
  1%|▏         | 235/18627 [07:36<10:06:06,  1.98s/it]
  1%|▏         | 236/18627 [07:38<8:55:55,  1.75s/it] 
  1%|▏         | 237/18627 [07:40<9:30:41,  1.86s/it]
  1%|▏         | 238/18627 [07:41<8:28:17,  1.66s/it]
  1%|▏         | 239/18627 [07:43<9:12:03,  1.80s/it]
  1%|▏         | 240/18627 [07:44<8:06:32,  1.59s/it]
                                                     
{'loss': 2.3622, 'grad_norm': 18.49774932861328, 'learning_rate': 2.146690518783542e-06, 'epoch': 0.01}
+
  1%|▏         | 240/18627 [07:44<8:06:32,  1.59s/it]
  1%|▏         | 241/18627 [07:46<8:54:45,  1.75s/it]
  1%|▏         | 242/18627 [07:48<9:30:18,  1.86s/it]
  1%|▏         | 243/18627 [07:50<8:28:03,  1.66s/it]
  1%|▏         | 244/18627 [07:52<9:10:50,  1.80s/it]
  1%|▏         | 245/18627 [07:54<9:41:14,  1.90s/it]
  1%|▏         | 246/18627 [07:55<8:21:17,  1.64s/it]
  1%|▏         | 247/18627 [07:57<9:07:44,  1.79s/it]
  1%|▏         | 248/18627 [07:59<9:38:17,  1.89s/it]
  1%|▏         | 249/18627 [08:00<8:34:41,  1.68s/it]
  1%|▏         | 250/18627 [08:02<9:15:03,  1.81s/it]
                                                     
{'loss': 2.3412, 'grad_norm': 15.055150985717773, 'learning_rate': 2.2361359570661897e-06, 'epoch': 0.01}
+
  1%|▏         | 250/18627 [08:02<9:15:03,  1.81s/it]
  1%|▏         | 251/18627 [08:05<9:43:24,  1.90s/it]
  1%|▏         | 252/18627 [08:06<8:38:00,  1.69s/it]
  1%|▏         | 253/18627 [08:06<7:06:27,  1.39s/it]
  1%|▏         | 254/18627 [08:09<8:13:50,  1.61s/it]
  1%|▏         | 255/18627 [08:11<8:59:03,  1.76s/it]
  1%|▏         | 256/18627 [08:13<9:32:44,  1.87s/it]
  1%|▏         | 257/18627 [08:15<9:54:05,  1.94s/it]
  1%|▏         | 258/18627 [08:16<8:37:38,  1.69s/it]
  1%|▏         | 259/18627 [08:17<7:51:21,  1.54s/it]
  1%|▏         | 260/18627 [08:19<8:43:28,  1.71s/it]
                                                     
{'loss': 2.2739, 'grad_norm': 16.20669937133789, 'learning_rate': 2.3255813953488376e-06, 'epoch': 0.01}
+
  1%|▏         | 260/18627 [08:19<8:43:28,  1.71s/it]
  1%|▏         | 261/18627 [08:21<7:55:38,  1.55s/it]
  1%|▏         | 262/18627 [08:23<8:49:29,  1.73s/it]
  1%|▏         | 263/18627 [08:25<9:26:59,  1.85s/it]
  1%|▏         | 264/18627 [08:27<9:53:44,  1.94s/it]
  1%|▏         | 265/18627 [08:28<8:46:20,  1.72s/it]
  1%|▏         | 266/18627 [08:29<7:58:27,  1.56s/it]
  1%|▏         | 267/18627 [08:31<8:49:51,  1.73s/it]
  1%|▏         | 268/18627 [08:34<9:24:26,  1.84s/it]
  1%|▏         | 269/18627 [08:36<9:48:47,  1.92s/it]
  1%|▏         | 270/18627 [08:38<10:05:58,  1.98s/it]
                                                      
{'loss': 2.2832, 'grad_norm': 19.289764404296875, 'learning_rate': 2.415026833631485e-06, 'epoch': 0.01}
+
  1%|▏         | 270/18627 [08:38<10:05:58,  1.98s/it]
  1%|▏         | 271/18627 [08:40<10:18:57,  2.02s/it]
  1%|▏         | 272/18627 [08:42<10:28:26,  2.05s/it]
  1%|▏         | 273/18627 [08:44<10:33:52,  2.07s/it]
  1%|▏         | 274/18627 [08:46<10:38:24,  2.09s/it]
  1%|▏         | 275/18627 [08:48<10:38:55,  2.09s/it]
  1%|▏         | 276/18627 [08:50<9:17:33,  1.82s/it] 
  1%|▏         | 277/18627 [08:52<9:44:15,  1.91s/it]
  1%|▏         | 278/18627 [08:54<10:08:55,  1.99s/it]
  1%|▏         | 279/18627 [08:56<10:21:12,  2.03s/it]
  2%|▏         | 280/18627 [08:58<10:35:46,  2.08s/it]
                                                      
{'loss': 1.9154, 'grad_norm': 16.81325340270996, 'learning_rate': 2.504472271914132e-06, 'epoch': 0.02}
+
  2%|▏         | 280/18627 [08:58<10:35:46,  2.08s/it]
  2%|▏         | 281/18627 [09:00<10:42:18,  2.10s/it]
  2%|▏         | 282/18627 [09:03<10:49:49,  2.13s/it]
  2%|▏         | 283/18627 [09:05<10:53:19,  2.14s/it]
  2%|▏         | 284/18627 [09:07<10:55:32,  2.14s/it]
  2%|▏         | 285/18627 [09:09<10:55:05,  2.14s/it]
  2%|▏         | 286/18627 [09:11<10:56:32,  2.15s/it]
  2%|▏         | 287/18627 [09:13<10:51:46,  2.13s/it]
  2%|▏         | 288/18627 [09:15<10:51:05,  2.13s/it]
  2%|▏         | 289/18627 [09:17<10:48:27,  2.12s/it]
  2%|▏         | 290/18627 [09:20<10:46:25,  2.12s/it]
                                                      
{'loss': 1.8617, 'grad_norm': 17.578046798706055, 'learning_rate': 2.59391771019678e-06, 'epoch': 0.02}
+
  2%|▏         | 290/18627 [09:20<10:46:25,  2.12s/it]
  2%|▏         | 291/18627 [09:22<10:44:25,  2.11s/it]
  2%|▏         | 292/18627 [09:24<10:44:51,  2.11s/it]
  2%|▏         | 293/18627 [09:26<10:47:03,  2.12s/it]
  2%|▏         | 294/18627 [09:28<10:49:01,  2.12s/it]
  2%|▏         | 295/18627 [09:30<10:47:54,  2.12s/it]
  2%|▏         | 296/18627 [09:31<9:22:02,  1.84s/it] 
  2%|▏         | 297/18627 [09:32<7:37:59,  1.50s/it]
  2%|▏         | 298/18627 [09:34<8:35:39,  1.69s/it]
  2%|▏         | 299/18627 [09:36<9:14:07,  1.81s/it]
  2%|▏         | 300/18627 [09:37<8:16:24,  1.63s/it]
                                                     
{'loss': 1.9192, 'grad_norm': 20.8323917388916, 'learning_rate': 2.6833631484794276e-06, 'epoch': 0.02}
+
  2%|▏         | 300/18627 [09:37<8:16:24,  1.63s/it]
  2%|▏         | 301/18627 [09:40<9:02:14,  1.78s/it]
  2%|▏         | 302/18627 [09:42<9:34:07,  1.88s/it]
  2%|▏         | 303/18627 [09:44<9:55:39,  1.95s/it]
  2%|▏         | 304/18627 [09:46<10:11:16,  2.00s/it]
  2%|▏         | 305/18627 [09:48<10:20:43,  2.03s/it]
  2%|▏         | 306/18627 [09:50<10:27:40,  2.06s/it]
  2%|▏         | 307/18627 [09:51<9:08:16,  1.80s/it] 
  2%|▏         | 308/18627 [09:53<9:36:19,  1.89s/it]
  2%|▏         | 309/18627 [09:56<9:56:47,  1.95s/it]
  2%|▏         | 310/18627 [09:58<10:12:19,  2.01s/it]
                                                      
{'loss': 1.8327, 'grad_norm': 13.06420612335205, 'learning_rate': 2.772808586762075e-06, 'epoch': 0.02}
+
  2%|▏         | 310/18627 [09:58<10:12:19,  2.01s/it]
  2%|▏         | 311/18627 [10:00<10:22:55,  2.04s/it]
  2%|▏         | 312/18627 [10:02<10:28:50,  2.06s/it]
  2%|▏         | 313/18627 [10:04<10:33:07,  2.07s/it]
  2%|▏         | 314/18627 [10:05<9:12:06,  1.81s/it] 
  2%|▏         | 315/18627 [10:06<7:29:48,  1.47s/it]
  2%|▏         | 316/18627 [10:08<8:33:38,  1.68s/it]
  2%|▏         | 317/18627 [10:10<9:15:51,  1.82s/it]
  2%|▏         | 318/18627 [10:11<8:17:53,  1.63s/it]
  2%|▏         | 319/18627 [10:14<9:01:26,  1.77s/it]
  2%|▏         | 320/18627 [10:16<9:33:40,  1.88s/it]
                                                     
{'loss': 1.8247, 'grad_norm': 14.27224349975586, 'learning_rate': 2.862254025044723e-06, 'epoch': 0.02}
+
  2%|▏         | 320/18627 [10:16<9:33:40,  1.88s/it]
  2%|▏         | 321/18627 [10:18<9:54:14,  1.95s/it]
  2%|▏         | 322/18627 [10:18<8:01:07,  1.58s/it]
  2%|▏         | 323/18627 [10:21<8:52:02,  1.74s/it]
  2%|▏         | 324/18627 [10:23<9:28:10,  1.86s/it]
  2%|▏         | 325/18627 [10:25<9:52:38,  1.94s/it]
  2%|▏         | 326/18627 [10:27<10:09:03,  2.00s/it]
  2%|▏         | 327/18627 [10:29<10:17:51,  2.03s/it]
  2%|▏         | 328/18627 [10:31<10:24:24,  2.05s/it]
  2%|▏         | 329/18627 [10:33<10:32:39,  2.07s/it]
  2%|▏         | 330/18627 [10:34<9:04:02,  1.78s/it] 
                                                     
{'loss': 1.8605, 'grad_norm': 18.729976654052734, 'learning_rate': 2.9516994633273705e-06, 'epoch': 0.02}
+
  2%|▏         | 330/18627 [10:34<9:04:02,  1.78s/it]
  2%|▏         | 331/18627 [10:37<9:33:07,  1.88s/it]
  2%|▏         | 332/18627 [10:39<9:53:41,  1.95s/it]
  2%|▏         | 333/18627 [10:41<10:08:16,  2.00s/it]
  2%|▏         | 334/18627 [10:41<8:08:54,  1.60s/it] 
  2%|▏         | 335/18627 [10:43<7:31:56,  1.48s/it]
  2%|▏         | 336/18627 [10:45<8:30:53,  1.68s/it]
  2%|▏         | 337/18627 [10:46<7:46:12,  1.53s/it]
  2%|▏         | 338/18627 [10:48<8:40:12,  1.71s/it]
  2%|▏         | 339/18627 [10:50<9:17:12,  1.83s/it]
  2%|▏         | 340/18627 [10:52<9:42:56,  1.91s/it]
                                                     
{'loss': 1.9161, 'grad_norm': 12.906477928161621, 'learning_rate': 3.041144901610018e-06, 'epoch': 0.02}
+
  2%|▏         | 340/18627 [10:52<9:42:56,  1.91s/it]
  2%|▏         | 341/18627 [10:54<10:03:12,  1.98s/it]
  2%|▏         | 342/18627 [10:57<10:17:38,  2.03s/it]
  2%|▏         | 343/18627 [10:59<10:26:15,  2.06s/it]
  2%|▏         | 344/18627 [11:01<10:32:36,  2.08s/it]
  2%|▏         | 345/18627 [11:03<10:39:49,  2.10s/it]
  2%|▏         | 346/18627 [11:04<9:16:36,  1.83s/it] 
  2%|▏         | 347/18627 [11:06<9:41:18,  1.91s/it]
  2%|▏         | 348/18627 [11:07<8:35:56,  1.69s/it]
  2%|▏         | 349/18627 [11:09<7:49:27,  1.54s/it]
  2%|▏         | 350/18627 [11:11<8:44:29,  1.72s/it]
                                                     
{'loss': 1.8474, 'grad_norm': 12.647000312805176, 'learning_rate': 3.1305903398926655e-06, 'epoch': 0.02}
+
  2%|▏         | 350/18627 [11:11<8:44:29,  1.72s/it]
  2%|▏         | 351/18627 [11:13<9:20:26,  1.84s/it]
  2%|▏         | 352/18627 [11:15<9:59:19,  1.97s/it]
  2%|▏         | 353/18627 [11:16<8:48:55,  1.74s/it]
  2%|▏         | 354/18627 [11:19<9:24:57,  1.86s/it]
  2%|▏         | 355/18627 [11:21<9:52:37,  1.95s/it]
  2%|▏         | 356/18627 [11:23<10:15:52,  2.02s/it]
  2%|▏         | 357/18627 [11:26<12:35:40,  2.48s/it]
  2%|▏         | 358/18627 [11:29<12:03:24,  2.38s/it]
  2%|▏         | 359/18627 [11:31<11:40:12,  2.30s/it]
  2%|▏         | 360/18627 [11:33<11:24:29,  2.25s/it]
                                                      
{'loss': 1.6509, 'grad_norm': 15.294533729553223, 'learning_rate': 3.2200357781753134e-06, 'epoch': 0.02}
+
  2%|▏         | 360/18627 [11:33<11:24:29,  2.25s/it]
  2%|▏         | 361/18627 [11:35<11:13:18,  2.21s/it]
  2%|▏         | 362/18627 [11:37<11:04:48,  2.18s/it]
  2%|▏         | 363/18627 [11:38<9:34:14,  1.89s/it] 
  2%|▏         | 364/18627 [11:40<9:55:50,  1.96s/it]
  2%|▏         | 365/18627 [11:42<10:12:06,  2.01s/it]
  2%|▏         | 366/18627 [11:45<10:23:25,  2.05s/it]
  2%|▏         | 367/18627 [11:47<10:30:53,  2.07s/it]
  2%|▏         | 368/18627 [11:49<10:35:27,  2.09s/it]
  2%|▏         | 369/18627 [11:50<9:14:06,  1.82s/it] 
  2%|▏         | 370/18627 [11:52<9:53:11,  1.95s/it]
                                                     
{'loss': 1.8379, 'grad_norm': 14.74196720123291, 'learning_rate': 3.309481216457961e-06, 'epoch': 0.02}
+
  2%|▏         | 370/18627 [11:52<9:53:11,  1.95s/it]
  2%|▏         | 371/18627 [11:54<10:10:11,  2.01s/it]
  2%|▏         | 372/18627 [11:57<10:21:24,  2.04s/it]
  2%|▏         | 373/18627 [11:59<10:27:51,  2.06s/it]
  2%|▏         | 374/18627 [12:01<10:32:05,  2.08s/it]
  2%|▏         | 375/18627 [12:03<10:35:22,  2.09s/it]
  2%|▏         | 376/18627 [12:05<10:36:36,  2.09s/it]
  2%|▏         | 377/18627 [12:07<10:38:47,  2.10s/it]
  2%|▏         | 378/18627 [12:09<10:39:58,  2.10s/it]
  2%|▏         | 379/18627 [12:10<9:16:14,  1.83s/it] 
  2%|▏         | 380/18627 [12:13<9:44:30,  1.92s/it]
                                                     
{'loss': 1.6068, 'grad_norm': 14.86855697631836, 'learning_rate': 3.3989266547406084e-06, 'epoch': 0.02}
+
  2%|▏         | 380/18627 [12:13<9:44:30,  1.92s/it]
  2%|▏         | 381/18627 [12:15<10:04:16,  1.99s/it]
  2%|▏         | 382/18627 [12:17<10:17:50,  2.03s/it]
  2%|▏         | 383/18627 [12:18<9:01:03,  1.78s/it] 
  2%|▏         | 384/18627 [12:20<9:36:40,  1.90s/it]
  2%|▏         | 385/18627 [12:21<8:32:14,  1.68s/it]
  2%|▏         | 386/18627 [12:24<9:17:43,  1.83s/it]
  2%|▏         | 387/18627 [12:26<9:46:52,  1.93s/it]
  2%|▏         | 388/18627 [12:28<10:08:00,  2.00s/it]
  2%|▏         | 389/18627 [12:30<10:20:23,  2.04s/it]
  2%|▏         | 390/18627 [12:32<10:29:11,  2.07s/it]
                                                      
{'loss': 1.6692, 'grad_norm': 11.408946990966797, 'learning_rate': 3.4883720930232564e-06, 'epoch': 0.02}
+
  2%|▏         | 390/18627 [12:32<10:29:11,  2.07s/it]
  2%|▏         | 391/18627 [12:34<10:32:31,  2.08s/it]
  2%|▏         | 392/18627 [12:36<10:37:31,  2.10s/it]
  2%|▏         | 393/18627 [12:38<9:13:59,  1.82s/it] 
  2%|▏         | 394/18627 [12:40<9:40:40,  1.91s/it]
  2%|▏         | 395/18627 [12:41<8:34:36,  1.69s/it]
  2%|▏         | 396/18627 [12:43<9:11:46,  1.82s/it]
  2%|▏         | 397/18627 [12:45<9:39:20,  1.91s/it]
  2%|▏         | 398/18627 [12:47<9:57:21,  1.97s/it]
  2%|▏         | 399/18627 [12:49<10:10:02,  2.01s/it]
  2%|▏         | 400/18627 [12:51<10:21:32,  2.05s/it]
                                                      
{'loss': 1.703, 'grad_norm': 14.891518592834473, 'learning_rate': 3.577817531305904e-06, 'epoch': 0.02}
+
  2%|▏         | 400/18627 [12:52<10:21:32,  2.05s/it]
  2%|▏         | 401/18627 [12:54<10:26:51,  2.06s/it]
  2%|▏         | 402/18627 [12:56<10:33:37,  2.09s/it]
  2%|▏         | 403/18627 [12:58<10:37:41,  2.10s/it]
  2%|▏         | 404/18627 [13:00<10:41:01,  2.11s/it]
  2%|▏         | 405/18627 [13:02<10:41:20,  2.11s/it]
  2%|▏         | 406/18627 [13:03<9:18:26,  1.84s/it] 
  2%|▏         | 407/18627 [13:05<9:42:54,  1.92s/it]
  2%|▏         | 408/18627 [13:08<10:04:20,  1.99s/it]
  2%|▏         | 409/18627 [13:10<10:16:54,  2.03s/it]
  2%|▏         | 410/18627 [13:12<10:26:29,  2.06s/it]
                                                      
{'loss': 1.5501, 'grad_norm': 17.497392654418945, 'learning_rate': 3.6672629695885514e-06, 'epoch': 0.02}
+
  2%|▏         | 410/18627 [13:12<10:26:29,  2.06s/it]
  2%|▏         | 411/18627 [13:14<10:32:17,  2.08s/it]
  2%|▏         | 412/18627 [13:16<10:34:21,  2.09s/it]
  2%|▏         | 413/18627 [13:18<10:38:45,  2.10s/it]
  2%|▏         | 414/18627 [13:20<10:40:27,  2.11s/it]
  2%|▏         | 415/18627 [13:22<9:16:15,  1.83s/it] 
  2%|▏         | 416/18627 [13:24<9:43:30,  1.92s/it]
  2%|▏         | 417/18627 [13:26<10:00:47,  1.98s/it]
  2%|▏         | 418/18627 [13:28<10:14:33,  2.02s/it]
  2%|▏         | 419/18627 [13:32<14:03:28,  2.78s/it]
  2%|▏         | 420/18627 [13:34<11:38:24,  2.30s/it]
                                                      
{'loss': 1.6012, 'grad_norm': 17.85817527770996, 'learning_rate': 3.756708407871199e-06, 'epoch': 0.02}
+
  2%|▏         | 420/18627 [13:34<11:38:24,  2.30s/it]
  2%|▏         | 421/18627 [13:35<9:49:15,  1.94s/it] 
  2%|▏         | 422/18627 [13:37<10:03:17,  1.99s/it]
  2%|▏         | 423/18627 [13:39<10:12:43,  2.02s/it]
  2%|▏         | 424/18627 [13:41<10:21:15,  2.05s/it]
  2%|▏         | 425/18627 [13:42<9:04:30,  1.79s/it] 
  2%|▏         | 426/18627 [13:44<9:32:20,  1.89s/it]
  2%|▏         | 427/18627 [13:46<9:52:08,  1.95s/it]
  2%|▏         | 428/18627 [13:49<10:08:47,  2.01s/it]
  2%|▏         | 429/18627 [13:51<10:19:13,  2.04s/it]
  2%|▏         | 430/18627 [13:53<10:40:23,  2.11s/it]
                                                      
{'loss': 1.6431, 'grad_norm': 14.89450740814209, 'learning_rate': 3.846153846153847e-06, 'epoch': 0.02}
+
  2%|▏         | 430/18627 [13:53<10:40:23,  2.11s/it]
  2%|▏         | 431/18627 [13:54<9:16:09,  1.83s/it] 
  2%|▏         | 432/18627 [13:56<9:41:12,  1.92s/it]
  2%|▏         | 433/18627 [13:57<8:20:39,  1.65s/it]
  2%|▏         | 434/18627 [13:59<9:02:44,  1.79s/it]
  2%|▏         | 435/18627 [14:02<9:31:23,  1.88s/it]
  2%|▏         | 436/18627 [14:04<9:51:45,  1.95s/it]
  2%|▏         | 437/18627 [14:06<10:07:38,  2.00s/it]
  2%|▏         | 438/18627 [14:07<8:53:25,  1.76s/it] 
  2%|▏         | 439/18627 [14:08<7:57:19,  1.57s/it]
  2%|▏         | 440/18627 [14:10<8:46:04,  1.74s/it]
                                                     
{'loss': 1.8257, 'grad_norm': 13.834882736206055, 'learning_rate': 3.935599284436494e-06, 'epoch': 0.02}
+
  2%|▏         | 440/18627 [14:10<8:46:04,  1.74s/it]
  2%|▏         | 441/18627 [14:12<9:20:40,  1.85s/it]
  2%|▏         | 442/18627 [14:14<9:45:12,  1.93s/it]
  2%|▏         | 443/18627 [14:17<10:02:35,  1.99s/it]
  2%|▏         | 444/18627 [14:19<10:14:08,  2.03s/it]
  2%|▏         | 445/18627 [14:21<10:23:06,  2.06s/it]
  2%|▏         | 446/18627 [14:23<10:30:43,  2.08s/it]
  2%|▏         | 447/18627 [14:24<9:09:17,  1.81s/it] 
  2%|▏         | 448/18627 [14:26<9:36:09,  1.90s/it]
  2%|▏         | 449/18627 [14:28<9:56:47,  1.97s/it]
  2%|▏         | 450/18627 [14:30<8:45:26,  1.73s/it]
                                                     
{'loss': 1.6789, 'grad_norm': 18.38890266418457, 'learning_rate': 4.025044722719142e-06, 'epoch': 0.02}
+
  2%|▏         | 450/18627 [14:30<8:45:26,  1.73s/it]
  2%|▏         | 451/18627 [14:32<9:21:11,  1.85s/it]
  2%|▏         | 452/18627 [14:34<9:44:44,  1.93s/it]
  2%|▏         | 453/18627 [14:36<10:00:59,  1.98s/it]
  2%|▏         | 454/18627 [14:38<10:12:12,  2.02s/it]
  2%|▏         | 455/18627 [14:39<8:56:45,  1.77s/it] 
  2%|▏         | 456/18627 [14:41<9:28:57,  1.88s/it]
  2%|▏         | 457/18627 [14:43<9:51:42,  1.95s/it]
  2%|▏         | 458/18627 [14:46<10:04:45,  2.00s/it]
  2%|▏         | 459/18627 [14:48<10:14:47,  2.03s/it]
  2%|▏         | 460/18627 [14:50<10:24:03,  2.06s/it]
                                                      
{'loss': 1.5056, 'grad_norm': 11.043734550476074, 'learning_rate': 4.114490161001789e-06, 'epoch': 0.02}
+
  2%|▏         | 460/18627 [14:50<10:24:03,  2.06s/it]
  2%|▏         | 461/18627 [14:51<9:05:18,  1.80s/it] 
  2%|▏         | 462/18627 [14:53<9:34:07,  1.90s/it]
  2%|▏         | 463/18627 [14:56<11:48:23,  2.34s/it]
  2%|▏         | 464/18627 [14:59<11:28:23,  2.27s/it]
  2%|▏         | 465/18627 [15:01<11:13:08,  2.22s/it]
  3%|▎         | 466/18627 [15:05<14:28:52,  2.87s/it]
  3%|▎         | 467/18627 [15:06<11:11:13,  2.22s/it]
  3%|▎         | 468/18627 [15:09<12:46:05,  2.53s/it]
  3%|▎         | 469/18627 [15:12<13:51:15,  2.75s/it]
  3%|▎         | 470/18627 [15:14<12:53:17,  2.56s/it]
                                                      
{'loss': 1.5113, 'grad_norm': 15.59850788116455, 'learning_rate': 4.203935599284437e-06, 'epoch': 0.03}
+
  3%|▎         | 470/18627 [15:14<12:53:17,  2.56s/it]
  3%|▎         | 471/18627 [15:17<12:12:11,  2.42s/it]
  3%|▎         | 472/18627 [15:19<11:44:23,  2.33s/it]
  3%|▎         | 473/18627 [15:21<11:23:31,  2.26s/it]
  3%|▎         | 474/18627 [15:24<12:50:55,  2.55s/it]
  3%|▎         | 475/18627 [15:25<10:04:09,  2.00s/it]
  3%|▎         | 476/18627 [15:26<8:50:23,  1.75s/it] 
  3%|▎         | 477/18627 [15:28<9:24:20,  1.87s/it]
  3%|▎         | 478/18627 [15:30<9:47:46,  1.94s/it]
  3%|▎         | 479/18627 [15:31<8:39:52,  1.72s/it]
  3%|▎         | 480/18627 [15:33<9:16:56,  1.84s/it]
                                                     
{'loss': 1.6656, 'grad_norm': 15.630155563354492, 'learning_rate': 4.293381037567084e-06, 'epoch': 0.03}
+
  3%|▎         | 480/18627 [15:33<9:16:56,  1.84s/it]
  3%|▎         | 481/18627 [15:36<9:45:12,  1.94s/it]
  3%|▎         | 482/18627 [15:37<8:37:20,  1.71s/it]
  3%|▎         | 483/18627 [15:38<7:51:48,  1.56s/it]
  3%|▎         | 484/18627 [15:39<7:18:24,  1.45s/it]
  3%|▎         | 485/18627 [15:40<6:54:42,  1.37s/it]
  3%|▎         | 486/18627 [15:42<8:01:33,  1.59s/it]
  3%|▎         | 487/18627 [15:44<7:25:03,  1.47s/it]
  3%|▎         | 488/18627 [15:45<7:00:13,  1.39s/it]
  3%|▎         | 489/18627 [15:47<8:09:39,  1.62s/it]
  3%|▎         | 490/18627 [15:49<8:58:06,  1.78s/it]
                                                     
{'loss': 2.1671, 'grad_norm': 12.143364906311035, 'learning_rate': 4.382826475849732e-06, 'epoch': 0.03}
+
  3%|���         | 490/18627 [15:49<8:58:06,  1.78s/it]
  3%|▎         | 491/18627 [15:50<7:59:21,  1.59s/it]
  3%|▎         | 492/18627 [15:52<8:46:26,  1.74s/it]
  3%|▎         | 493/18627 [15:54<7:56:26,  1.58s/it]
  3%|▎         | 494/18627 [15:56<8:46:14,  1.74s/it]
  3%|▎         | 495/18627 [15:58<9:22:06,  1.86s/it]
  3%|▎         | 496/18627 [16:00<9:46:04,  1.94s/it]
  3%|▎         | 497/18627 [16:01<8:37:39,  1.71s/it]
  3%|▎         | 498/18627 [16:03<9:16:33,  1.84s/it]
  3%|▎         | 499/18627 [16:05<9:41:05,  1.92s/it]
  3%|▎         | 500/18627 [16:08<9:57:44,  1.98s/it]
                                                     
{'loss': 1.7896, 'grad_norm': 12.963175773620605, 'learning_rate': 4.472271914132379e-06, 'epoch': 0.03}
+
  3%|▎         | 500/18627 [16:08<9:57:44,  1.98s/it]
  3%|▎         | 501/18627 [16:10<10:09:23,  2.02s/it]
  3%|▎         | 502/18627 [16:12<10:17:44,  2.04s/it]
  3%|▎         | 503/18627 [16:14<10:23:28,  2.06s/it]
  3%|▎         | 504/18627 [16:15<9:03:48,  1.80s/it] 
  3%|▎         | 505/18627 [16:17<9:32:33,  1.90s/it]
  3%|▎         | 506/18627 [16:19<9:53:03,  1.96s/it]
  3%|▎         | 507/18627 [16:21<10:05:59,  2.01s/it]
  3%|▎         | 508/18627 [16:24<10:16:43,  2.04s/it]
  3%|▎         | 509/18627 [16:26<10:22:48,  2.06s/it]
  3%|▎         | 510/18627 [16:28<10:30:07,  2.09s/it]
                                                      
{'loss': 1.4947, 'grad_norm': 14.984971046447754, 'learning_rate': 4.561717352415027e-06, 'epoch': 0.03}
+
  3%|▎         | 510/18627 [16:28<10:30:07,  2.09s/it]
  3%|▎         | 511/18627 [16:30<10:31:40,  2.09s/it]
  3%|▎         | 512/18627 [16:31<9:09:28,  1.82s/it] 
  3%|▎         | 513/18627 [16:33<9:37:43,  1.91s/it]
  3%|▎         | 514/18627 [16:35<9:56:28,  1.98s/it]
  3%|▎         | 515/18627 [16:36<8:45:16,  1.74s/it]
  3%|▎         | 516/18627 [16:39<9:18:49,  1.85s/it]
  3%|▎         | 517/18627 [16:41<9:43:08,  1.93s/it]
  3%|▎         | 518/18627 [16:43<9:58:15,  1.98s/it]
  3%|▎         | 519/18627 [16:45<10:11:08,  2.02s/it]
  3%|▎         | 520/18627 [16:47<10:18:26,  2.05s/it]
                                                      
{'loss': 1.6294, 'grad_norm': 15.819538116455078, 'learning_rate': 4.651162790697675e-06, 'epoch': 0.03}
+
  3%|▎         | 520/18627 [16:47<10:18:26,  2.05s/it]
  3%|▎         | 521/18627 [16:49<10:24:32,  2.07s/it]
  3%|▎         | 522/18627 [16:51<10:30:50,  2.09s/it]
  3%|▎         | 523/18627 [16:53<9:09:50,  1.82s/it] 
  3%|▎         | 524/18627 [16:55<9:37:10,  1.91s/it]
  3%|▎         | 525/18627 [16:56<8:32:16,  1.70s/it]
  3%|▎         | 526/18627 [16:58<9:16:21,  1.84s/it]
  3%|▎         | 527/18627 [17:00<9:40:51,  1.93s/it]
  3%|▎         | 528/18627 [17:01<8:33:47,  1.70s/it]
  3%|▎         | 529/18627 [17:02<7:46:37,  1.55s/it]
  3%|▎         | 530/18627 [17:05<8:37:53,  1.72s/it]
                                                     
{'loss': 1.8444, 'grad_norm': 14.638567924499512, 'learning_rate': 4.740608228980323e-06, 'epoch': 0.03}
+
  3%|▎         | 530/18627 [17:05<8:37:53,  1.72s/it]
  3%|▎         | 531/18627 [17:07<9:16:32,  1.85s/it]
  3%|▎         | 532/18627 [17:09<9:41:05,  1.93s/it]
  3%|▎         | 533/18627 [17:10<8:33:52,  1.70s/it]
  3%|▎         | 534/18627 [17:12<9:09:05,  1.82s/it]
  3%|▎         | 535/18627 [17:14<9:34:54,  1.91s/it]
  3%|▎         | 536/18627 [17:16<9:54:29,  1.97s/it]
  3%|▎         | 537/18627 [17:19<10:08:25,  2.02s/it]
  3%|▎         | 538/18627 [17:21<10:16:38,  2.05s/it]
  3%|▎         | 539/18627 [17:23<10:20:53,  2.06s/it]
  3%|▎         | 540/18627 [17:25<10:24:39,  2.07s/it]
                                                      
{'loss': 1.4168, 'grad_norm': 12.825884819030762, 'learning_rate': 4.83005366726297e-06, 'epoch': 0.03}
+
  3%|▎         | 540/18627 [17:25<10:24:39,  2.07s/it]
  3%|▎         | 541/18627 [17:26<9:04:09,  1.81s/it] 
  3%|▎         | 542/18627 [17:28<9:34:29,  1.91s/it]
  3%|▎         | 543/18627 [17:30<9:55:01,  1.97s/it]
  3%|▎         | 544/18627 [17:31<8:43:55,  1.74s/it]
  3%|▎         | 545/18627 [17:34<9:18:15,  1.85s/it]
  3%|▎         | 546/18627 [17:36<9:42:26,  1.93s/it]
  3%|▎         | 547/18627 [17:38<10:00:47,  1.99s/it]
  3%|▎         | 548/18627 [17:40<10:11:57,  2.03s/it]
  3%|▎         | 549/18627 [17:42<10:20:22,  2.06s/it]
  3%|▎         | 550/18627 [17:43<9:02:16,  1.80s/it] 
                                                     
{'loss': 1.6703, 'grad_norm': 17.444271087646484, 'learning_rate': 4.919499105545618e-06, 'epoch': 0.03}
+
  3%|▎         | 550/18627 [17:43<9:02:16,  1.80s/it]
  3%|▎         | 551/18627 [17:45<9:28:58,  1.89s/it]
  3%|▎         | 552/18627 [17:47<9:48:57,  1.96s/it]
  3%|▎         | 553/18627 [17:50<10:01:55,  2.00s/it]
  3%|▎         | 554/18627 [17:52<10:13:20,  2.04s/it]
  3%|▎         | 555/18627 [17:54<10:21:23,  2.06s/it]
  3%|▎         | 556/18627 [17:56<10:26:55,  2.08s/it]
  3%|▎         | 557/18627 [17:57<8:59:58,  1.79s/it] 
  3%|▎         | 558/18627 [17:59<9:32:00,  1.90s/it]
  3%|▎         | 559/18627 [18:01<9:51:25,  1.96s/it]
  3%|▎         | 560/18627 [18:03<10:06:36,  2.01s/it]
                                                      
{'loss': 1.3585, 'grad_norm': 13.558156967163086, 'learning_rate': 4.999999962208895e-06, 'epoch': 0.03}
+
  3%|▎         | 560/18627 [18:03<10:06:36,  2.01s/it]
  3%|▎         | 561/18627 [18:05<8:52:13,  1.77s/it] 
  3%|▎         | 562/18627 [18:06<7:53:10,  1.57s/it]
  3%|▎         | 563/18627 [18:08<8:42:28,  1.74s/it]
  3%|▎         | 564/18627 [18:10<9:16:21,  1.85s/it]
  3%|▎         | 565/18627 [18:12<9:40:08,  1.93s/it]
  3%|▎         | 566/18627 [18:13<8:33:02,  1.70s/it]
  3%|▎         | 567/18627 [18:15<9:09:10,  1.82s/it]
  3%|▎         | 568/18627 [18:18<9:35:16,  1.91s/it]
  3%|▎         | 569/18627 [18:20<9:53:47,  1.97s/it]
  3%|▎         | 570/18627 [18:22<10:08:29,  2.02s/it]
                                                      
{'loss': 1.663, 'grad_norm': 14.0825834274292, 'learning_rate': 4.999995427277649e-06, 'epoch': 0.03}
+
  3%|▎         | 570/18627 [18:22<10:08:29,  2.02s/it]
  3%|▎         | 571/18627 [18:24<10:18:23,  2.05s/it]
  3%|▎         | 572/18627 [18:26<10:21:37,  2.07s/it]
  3%|▎         | 573/18627 [18:28<10:25:33,  2.08s/it]
  3%|▎         | 574/18627 [18:30<10:31:03,  2.10s/it]
  3%|▎         | 575/18627 [18:32<10:32:18,  2.10s/it]
  3%|▎         | 576/18627 [18:34<10:34:55,  2.11s/it]
  3%|▎         | 577/18627 [18:37<10:34:19,  2.11s/it]
  3%|▎         | 578/18627 [18:39<10:36:04,  2.11s/it]
  3%|▎         | 579/18627 [18:41<10:35:47,  2.11s/it]
  3%|▎         | 580/18627 [18:43<10:35:55,  2.11s/it]
                                                      
{'loss': 1.2031, 'grad_norm': 13.44847297668457, 'learning_rate': 4.9999833341410635e-06, 'epoch': 0.03}
+
  3%|▎         | 580/18627 [18:43<10:35:55,  2.11s/it]
  3%|▎         | 581/18627 [18:44<9:13:15,  1.84s/it] 
  3%|▎         | 582/18627 [18:46<9:38:50,  1.92s/it]
  3%|▎         | 583/18627 [18:48<9:57:02,  1.99s/it]
  3%|▎         | 584/18627 [18:50<8:45:22,  1.75s/it]
  3%|▎         | 585/18627 [18:51<7:54:39,  1.58s/it]
  3%|▎         | 586/18627 [18:53<8:41:42,  1.74s/it]
  3%|▎         | 587/18627 [18:55<9:16:27,  1.85s/it]
  3%|▎         | 588/18627 [18:56<8:09:07,  1.63s/it]
  3%|▎         | 589/18627 [18:58<8:53:51,  1.78s/it]
  3%|▎         | 590/18627 [19:00<9:26:33,  1.88s/it]
                                                     
{'loss': 1.7972, 'grad_norm': 14.0843505859375, 'learning_rate': 4.9999636828357e-06, 'epoch': 0.03}
+
  3%|▎         | 590/18627 [19:00<9:26:33,  1.88s/it]
  3%|▎         | 591/18627 [19:02<9:50:21,  1.96s/it]
  3%|▎         | 592/18627 [19:05<10:04:50,  2.01s/it]
  3%|▎         | 593/18627 [19:07<10:14:35,  2.04s/it]
  3%|▎         | 594/18627 [19:09<10:24:49,  2.08s/it]
  3%|▎         | 595/18627 [19:11<10:26:48,  2.09s/it]
  3%|▎         | 596/18627 [19:13<10:28:52,  2.09s/it]
  3%|▎         | 597/18627 [19:14<9:07:12,  1.82s/it] 
  3%|▎         | 598/18627 [19:15<8:10:45,  1.63s/it]
  3%|▎         | 599/18627 [19:18<8:55:58,  1.78s/it]
  3%|▎         | 600/18627 [19:20<9:26:45,  1.89s/it]
                                                     
{'loss': 1.4777, 'grad_norm': 15.103700637817383, 'learning_rate': 4.999936473420971e-06, 'epoch': 0.03}
+
  3%|▎         | 600/18627 [19:20<9:26:45,  1.89s/it]
  3%|▎         | 601/18627 [19:21<8:16:16,  1.65s/it]
  3%|▎         | 602/18627 [19:23<8:59:09,  1.79s/it]
  3%|▎         | 603/18627 [19:24<7:57:03,  1.59s/it]
  3%|▎         | 604/18627 [19:26<8:43:43,  1.74s/it]
  3%|▎         | 605/18627 [19:27<7:54:26,  1.58s/it]
  3%|▎         | 606/18627 [19:30<8:47:00,  1.75s/it]
  3%|▎         | 607/18627 [19:32<9:23:03,  1.87s/it]
  3%|▎         | 608/18627 [19:34<9:46:53,  1.95s/it]
  3%|▎         | 609/18627 [19:36<10:04:02,  2.01s/it]
  3%|▎         | 610/18627 [19:38<10:14:37,  2.05s/it]
                                                      
{'loss': 1.7091, 'grad_norm': 16.787925720214844, 'learning_rate': 4.999901705979138e-06, 'epoch': 0.03}
+
  3%|▎         | 610/18627 [19:38<10:14:37,  2.05s/it]
  3%|▎         | 611/18627 [19:40<10:19:19,  2.06s/it]
  3%|▎         | 612/18627 [19:42<10:22:39,  2.07s/it]
  3%|▎         | 613/18627 [19:44<10:28:35,  2.09s/it]
  3%|▎         | 614/18627 [19:47<10:31:21,  2.10s/it]
  3%|▎         | 615/18627 [19:49<10:31:41,  2.10s/it]
  3%|▎         | 616/18627 [19:51<10:30:52,  2.10s/it]
  3%|▎         | 617/18627 [19:53<10:31:35,  2.10s/it]
  3%|▎         | 618/18627 [19:55<10:32:05,  2.11s/it]
  3%|▎         | 619/18627 [19:57<10:34:11,  2.11s/it]
  3%|▎         | 620/18627 [19:59<10:32:49,  2.11s/it]
                                                      
{'loss': 1.2518, 'grad_norm': 11.518420219421387, 'learning_rate': 4.9998593806153125e-06, 'epoch': 0.03}
+
  3%|▎         | 620/18627 [19:59<10:32:49,  2.11s/it]
  3%|▎         | 621/18627 [20:01<10:36:47,  2.12s/it]
  3%|▎         | 622/18627 [20:04<10:35:54,  2.12s/it]
  3%|▎         | 623/18627 [20:06<10:36:04,  2.12s/it]
  3%|▎         | 624/18627 [20:07<9:06:46,  1.82s/it] 
  3%|▎         | 625/18627 [20:08<8:10:03,  1.63s/it]
  3%|▎         | 626/18627 [20:10<8:53:21,  1.78s/it]
  3%|▎         | 627/18627 [20:12<9:23:28,  1.88s/it]
  3%|▎         | 628/18627 [20:14<9:43:30,  1.95s/it]
  3%|▎         | 629/18627 [20:16<9:58:47,  2.00s/it]
  3%|▎         | 630/18627 [20:19<10:09:48,  2.03s/it]
                                                      
{'loss': 1.5757, 'grad_norm': 11.527627944946289, 'learning_rate': 4.999809497457457e-06, 'epoch': 0.03}
+
  3%|▎         | 630/18627 [20:19<10:09:48,  2.03s/it]
  3%|▎         | 631/18627 [20:20<8:46:05,  1.75s/it] 
  3%|▎         | 632/18627 [20:22<9:19:30,  1.87s/it]
  3%|▎         | 633/18627 [20:23<8:18:47,  1.66s/it]
  3%|▎         | 634/18627 [20:25<9:00:54,  1.80s/it]
  3%|▎         | 635/18627 [20:27<9:29:41,  1.90s/it]
  3%|▎         | 636/18627 [20:29<9:51:26,  1.97s/it]
  3%|▎         | 637/18627 [20:31<10:05:49,  2.02s/it]
  3%|▎         | 638/18627 [20:34<10:12:19,  2.04s/it]
  3%|▎         | 639/18627 [20:36<10:21:50,  2.07s/it]
  3%|▎         | 640/18627 [20:38<10:26:01,  2.09s/it]
                                                      
{'loss': 1.5223, 'grad_norm': 10.920843124389648, 'learning_rate': 4.999752056656382e-06, 'epoch': 0.03}
+
  3%|▎         | 640/18627 [20:38<10:26:01,  2.09s/it]
  3%|▎         | 641/18627 [20:40<10:29:42,  2.10s/it]
  3%|▎         | 642/18627 [20:42<10:31:49,  2.11s/it]
  3%|▎         | 643/18627 [20:43<9:10:30,  1.84s/it] 
  3%|▎         | 644/18627 [20:45<9:38:39,  1.93s/it]
  3%|▎         | 645/18627 [20:48<9:54:09,  1.98s/it]
  3%|▎         | 646/18627 [20:49<8:45:00,  1.75s/it]
  3%|▎         | 647/18627 [20:51<9:17:01,  1.86s/it]
  3%|▎         | 648/18627 [20:53<9:40:16,  1.94s/it]
  3%|▎         | 649/18627 [20:55<9:58:01,  2.00s/it]
  3%|▎         | 650/18627 [20:56<8:44:53,  1.75s/it]
                                                     
{'loss': 1.7055, 'grad_norm': 19.04953956604004, 'learning_rate': 4.999687058385747e-06, 'epoch': 0.03}
+
  3%|▎         | 650/18627 [20:56<8:44:53,  1.75s/it]
  3%|▎         | 651/18627 [20:58<9:18:11,  1.86s/it]
  4%|▎         | 652/18627 [21:00<8:18:16,  1.66s/it]
  4%|▎         | 653/18627 [21:02<9:01:06,  1.81s/it]
  4%|▎         | 654/18627 [21:04<9:29:53,  1.90s/it]
  4%|▎         | 655/18627 [21:06<9:49:52,  1.97s/it]
  4%|▎         | 656/18627 [21:08<10:01:29,  2.01s/it]
  4%|▎         | 657/18627 [21:10<10:10:50,  2.04s/it]
  4%|▎         | 658/18627 [21:12<10:19:46,  2.07s/it]
  4%|▎         | 659/18627 [21:14<10:26:03,  2.09s/it]
  4%|▎         | 660/18627 [21:17<10:29:26,  2.10s/it]
                                                      
{'loss': 1.3072, 'grad_norm': 11.170331954956055, 'learning_rate': 4.9996145028420615e-06, 'epoch': 0.04}
+
  4%|▎         | 660/18627 [21:17<10:29:26,  2.10s/it]
  4%|▎         | 661/18627 [21:18<9:08:47,  1.83s/it] 
  4%|▎         | 662/18627 [21:19<8:10:59,  1.64s/it]
  4%|▎         | 663/18627 [21:21<8:55:01,  1.79s/it]
  4%|▎         | 664/18627 [21:23<9:27:04,  1.89s/it]
  4%|▎         | 665/18627 [21:25<9:49:32,  1.97s/it]
  4%|▎         | 666/18627 [21:28<10:01:57,  2.01s/it]
  4%|▎         | 667/18627 [21:30<10:12:38,  2.05s/it]
  4%|▎         | 668/18627 [21:32<10:19:21,  2.07s/it]
  4%|▎         | 669/18627 [21:34<10:23:17,  2.08s/it]
  4%|▎         | 670/18627 [21:35<9:02:38,  1.81s/it] 
                                                     
{'loss': 1.6721, 'grad_norm': 17.498394012451172, 'learning_rate': 4.999534390244681e-06, 'epoch': 0.04}
+
  4%|▎         | 670/18627 [21:35<9:02:38,  1.81s/it]
  4%|▎         | 671/18627 [21:36<7:57:01,  1.59s/it]
  4%|▎         | 672/18627 [21:38<8:42:40,  1.75s/it]
  4%|▎         | 673/18627 [21:40<9:15:50,  1.86s/it]
  4%|▎         | 674/18627 [21:43<9:40:07,  1.94s/it]
  4%|▎         | 675/18627 [21:45<9:56:38,  1.99s/it]
  4%|▎         | 676/18627 [21:47<10:09:46,  2.04s/it]
  4%|▎         | 677/18627 [21:49<10:16:23,  2.06s/it]
  4%|▎         | 678/18627 [21:51<10:21:43,  2.08s/it]
  4%|▎         | 679/18627 [21:53<10:25:15,  2.09s/it]
  4%|▎         | 680/18627 [21:54<8:21:11,  1.68s/it] 
                                                     
{'loss': 1.2925, 'grad_norm': 12.15459156036377, 'learning_rate': 4.9994467208358115e-06, 'epoch': 0.04}
+
  4%|▎         | 680/18627 [21:54<8:21:11,  1.68s/it]
  4%|▎         | 681/18627 [21:56<9:02:08,  1.81s/it]
  4%|▎         | 682/18627 [21:57<8:06:31,  1.63s/it]
  4%|▎         | 683/18627 [21:58<7:26:50,  1.49s/it]
  4%|▎         | 684/18627 [22:00<8:23:47,  1.68s/it]
  4%|▎         | 685/18627 [22:03<9:01:24,  1.81s/it]
  4%|▎         | 686/18627 [22:05<9:30:48,  1.91s/it]
  4%|▎         | 687/18627 [22:07<9:49:17,  1.97s/it]
  4%|▎         | 688/18627 [22:09<10:01:57,  2.01s/it]
  4%|▎         | 689/18627 [22:11<10:15:19,  2.06s/it]
  4%|▎         | 690/18627 [22:13<10:23:59,  2.09s/it]
                                                      
{'loss': 1.3774, 'grad_norm': 11.764025688171387, 'learning_rate': 4.999351494880499e-06, 'epoch': 0.04}
+
  4%|▎         | 690/18627 [22:13<10:23:59,  2.09s/it]
  4%|▎         | 691/18627 [22:15<10:27:42,  2.10s/it]
  4%|▎         | 692/18627 [22:18<10:28:55,  2.10s/it]
  4%|▎         | 693/18627 [22:20<10:32:35,  2.12s/it]
  4%|▎         | 694/18627 [22:22<10:32:24,  2.12s/it]
  4%|▎         | 695/18627 [22:23<9:00:58,  1.81s/it] 
  4%|▎         | 696/18627 [22:24<7:21:42,  1.48s/it]
  4%|▎         | 697/18627 [22:26<8:21:00,  1.68s/it]
  4%|▎         | 698/18627 [22:28<9:02:05,  1.81s/it]
  4%|▍         | 699/18627 [22:30<9:28:53,  1.90s/it]
  4%|▍         | 700/18627 [22:32<9:47:56,  1.97s/it]
                                                     
{'loss': 1.2881, 'grad_norm': 10.374739646911621, 'learning_rate': 4.99924871266664e-06, 'epoch': 0.04}
+
  4%|▍         | 700/18627 [22:32<9:47:56,  1.97s/it]
  4%|▍         | 701/18627 [22:34<10:00:33,  2.01s/it]
  4%|▍         | 702/18627 [22:36<10:11:55,  2.05s/it]
  4%|▍         | 703/18627 [22:38<10:19:00,  2.07s/it]
  4%|▍         | 704/18627 [22:41<10:22:19,  2.08s/it]
  4%|▍         | 705/18627 [22:43<10:23:18,  2.09s/it]
  4%|▍         | 706/18627 [22:43<8:18:06,  1.67s/it] 
  4%|▍         | 707/18627 [22:45<8:59:35,  1.81s/it]
  4%|▍         | 708/18627 [22:48<9:26:57,  1.90s/it]
  4%|▍         | 709/18627 [22:50<9:46:34,  1.96s/it]
  4%|▍         | 710/18627 [22:52<9:58:39,  2.00s/it]
                                                     
{'loss': 1.1331, 'grad_norm': 11.001517295837402, 'learning_rate': 4.999138374504977e-06, 'epoch': 0.04}
+
  4%|▍         | 710/18627 [22:52<9:58:39,  2.00s/it]
  4%|▍         | 711/18627 [22:54<10:07:51,  2.04s/it]
  4%|▍         | 712/18627 [22:56<10:16:14,  2.06s/it]
  4%|▍         | 713/18627 [22:57<8:58:17,  1.80s/it] 
  4%|▍         | 714/18627 [22:59<9:25:55,  1.90s/it]
  4%|▍         | 715/18627 [23:01<9:46:11,  1.96s/it]
  4%|▍         | 716/18627 [23:04<10:00:57,  2.01s/it]
  4%|▍         | 717/18627 [23:05<8:50:48,  1.78s/it] 
  4%|▍         | 718/18627 [23:07<9:21:28,  1.88s/it]
  4%|▍         | 719/18627 [23:09<9:43:45,  1.96s/it]
  4%|▍         | 720/18627 [23:11<9:58:15,  2.00s/it]
                                                     
{'loss': 1.5097, 'grad_norm': 13.292499542236328, 'learning_rate': 4.999020480729092e-06, 'epoch': 0.04}
+
  4%|▍         | 720/18627 [23:11<9:58:15,  2.00s/it]
  4%|▍         | 721/18627 [23:13<10:10:58,  2.05s/it]
  4%|▍         | 722/18627 [23:15<10:17:29,  2.07s/it]
  4%|▍         | 723/18627 [23:18<10:22:47,  2.09s/it]
  4%|▍         | 724/18627 [23:20<10:24:18,  2.09s/it]
  4%|▍         | 725/18627 [23:21<9:04:36,  1.83s/it] 
  4%|▍         | 726/18627 [23:23<9:30:53,  1.91s/it]
  4%|▍         | 727/18627 [23:25<9:49:06,  1.97s/it]
  4%|▍         | 728/18627 [23:27<10:03:54,  2.02s/it]
  4%|▍         | 729/18627 [23:28<8:42:01,  1.75s/it] 
  4%|▍         | 730/18627 [23:31<9:15:51,  1.86s/it]
                                                     
{'loss': 1.4131, 'grad_norm': 9.959339141845703, 'learning_rate': 4.998895031695412e-06, 'epoch': 0.04}
+
  4%|▍         | 730/18627 [23:31<9:15:51,  1.86s/it]
  4%|▍         | 731/18627 [23:33<9:39:32,  1.94s/it]
  4%|▍         | 732/18627 [23:34<8:24:12,  1.69s/it]
  4%|▍         | 733/18627 [23:36<9:02:58,  1.82s/it]
  4%|▍         | 734/18627 [23:38<9:27:17,  1.90s/it]
  4%|▍         | 735/18627 [23:39<8:23:08,  1.69s/it]
  4%|▍         | 736/18627 [23:41<9:02:26,  1.82s/it]
  4%|▍         | 737/18627 [23:43<9:27:12,  1.90s/it]
  4%|▍         | 738/18627 [23:44<8:15:23,  1.66s/it]
  4%|▍         | 739/18627 [23:47<8:56:57,  1.80s/it]
  4%|▍         | 740/18627 [23:48<8:04:21,  1.62s/it]
                                                     
{'loss': 1.9142, 'grad_norm': 17.609384536743164, 'learning_rate': 4.998762027783206e-06, 'epoch': 0.04}
+
  4%|▍         | 740/18627 [23:48<8:04:21,  1.62s/it]
  4%|▍         | 741/18627 [23:49<7:18:26,  1.47s/it]
  4%|▍         | 742/18627 [23:50<6:53:08,  1.39s/it]
  4%|▍         | 743/18627 [23:52<7:58:08,  1.60s/it]
  4%|▍         | 744/18627 [23:54<8:45:38,  1.76s/it]
  4%|▍         | 745/18627 [23:56<9:17:48,  1.87s/it]
  4%|▍         | 746/18627 [23:59<9:38:44,  1.94s/it]
  4%|▍         | 747/18627 [24:00<8:32:00,  1.72s/it]
  4%|▍         | 748/18627 [24:02<9:08:09,  1.84s/it]
  4%|▍         | 749/18627 [24:04<9:34:42,  1.93s/it]
  4%|▍         | 750/18627 [24:05<8:27:56,  1.70s/it]
                                                     
{'loss': 1.8993, 'grad_norm': 16.252046585083008, 'learning_rate': 4.998621469394582e-06, 'epoch': 0.04}
+
  4%|▍         | 750/18627 [24:05<8:27:56,  1.70s/it]
  4%|▍         | 751/18627 [24:06<7:44:33,  1.56s/it]
  4%|▍         | 752/18627 [24:08<7:11:37,  1.45s/it]
  4%|▍         | 753/18627 [24:10<8:10:30,  1.65s/it]
  4%|▍         | 754/18627 [24:12<8:52:31,  1.79s/it]
  4%|▍         | 755/18627 [24:14<9:21:12,  1.88s/it]
  4%|▍         | 756/18627 [24:16<9:41:04,  1.95s/it]
  4%|▍         | 757/18627 [24:18<9:57:14,  2.01s/it]
  4%|▍         | 758/18627 [24:20<10:06:00,  2.03s/it]
  4%|▍         | 759/18627 [24:22<10:13:40,  2.06s/it]
  4%|▍         | 760/18627 [24:25<10:18:24,  2.08s/it]
                                                      
{'loss': 1.4534, 'grad_norm': 9.33243465423584, 'learning_rate': 4.99847335695449e-06, 'epoch': 0.04}
+
  4%|▍         | 760/18627 [24:25<10:18:24,  2.08s/it]
  4%|▍         | 761/18627 [24:27<10:21:27,  2.09s/it]
  4%|▍         | 762/18627 [24:29<10:24:52,  2.10s/it]
  4%|▍         | 763/18627 [24:31<10:24:44,  2.10s/it]
  4%|▍         | 764/18627 [24:33<10:25:20,  2.10s/it]
  4%|▍         | 765/18627 [24:34<9:04:21,  1.83s/it] 
  4%|▍         | 766/18627 [24:35<8:07:49,  1.64s/it]
  4%|▍         | 767/18627 [24:38<8:52:49,  1.79s/it]
  4%|▍         | 768/18627 [24:39<7:59:26,  1.61s/it]
  4%|▍         | 769/18627 [24:41<8:46:31,  1.77s/it]
  4%|▍         | 770/18627 [24:43<9:19:09,  1.88s/it]
                                                     
{'loss': 1.4666, 'grad_norm': 11.797158241271973, 'learning_rate': 4.998317690910716e-06, 'epoch': 0.04}
+
  4%|▍         | 770/18627 [24:43<9:19:09,  1.88s/it]
  4%|▍         | 771/18627 [24:45<9:40:03,  1.95s/it]
  4%|▍         | 772/18627 [24:47<9:55:01,  2.00s/it]
  4%|▍         | 773/18627 [24:49<10:04:31,  2.03s/it]
  4%|▍         | 774/18627 [24:51<10:13:06,  2.06s/it]
  4%|▍         | 775/18627 [24:53<8:55:05,  1.80s/it] 
  4%|▍         | 776/18627 [24:55<9:24:15,  1.90s/it]
  4%|▍         | 777/18627 [24:57<9:46:28,  1.97s/it]
  4%|▍         | 778/18627 [24:59<10:00:25,  2.02s/it]
  4%|▍         | 779/18627 [25:01<10:09:36,  2.05s/it]
  4%|▍         | 780/18627 [25:03<10:15:50,  2.07s/it]
                                                      
{'loss': 1.1603, 'grad_norm': 13.059012413024902, 'learning_rate': 4.9981544717338824e-06, 'epoch': 0.04}
+
  4%|▍         | 780/18627 [25:03<10:15:50,  2.07s/it]
  4%|▍         | 781/18627 [25:05<10:19:16,  2.08s/it]
  4%|▍         | 782/18627 [25:08<10:25:06,  2.10s/it]
  4%|▍         | 783/18627 [25:10<10:28:38,  2.11s/it]
  4%|▍         | 784/18627 [25:12<10:27:18,  2.11s/it]
  4%|▍         | 785/18627 [25:14<10:29:15,  2.12s/it]
  4%|▍         | 786/18627 [25:15<9:06:33,  1.84s/it] 
  4%|▍         | 787/18627 [25:17<9:31:58,  1.92s/it]
  4%|▍         | 788/18627 [25:19<9:48:11,  1.98s/it]
  4%|▍         | 789/18627 [25:21<10:00:02,  2.02s/it]
  4%|▍         | 790/18627 [25:24<10:10:05,  2.05s/it]
                                                      
{'loss': 1.2212, 'grad_norm': 15.020527839660645, 'learning_rate': 4.997983699917448e-06, 'epoch': 0.04}
+
  4%|▍         | 790/18627 [25:24<10:10:05,  2.05s/it]
  4%|▍         | 791/18627 [25:26<10:17:54,  2.08s/it]
  4%|▍         | 792/18627 [25:28<10:20:40,  2.09s/it]
  4%|▍         | 793/18627 [25:30<10:24:36,  2.10s/it]
  4%|▍         | 794/18627 [25:32<10:24:25,  2.10s/it]
  4%|▍         | 795/18627 [25:34<10:28:20,  2.11s/it]
  4%|▍         | 796/18627 [25:36<10:28:31,  2.11s/it]
  4%|▍         | 797/18627 [25:38<9:06:15,  1.84s/it] 
  4%|▍         | 798/18627 [25:40<9:32:11,  1.93s/it]
  4%|▍         | 799/18627 [25:41<8:27:36,  1.71s/it]
  4%|▍         | 800/18627 [25:43<9:04:33,  1.83s/it]
                                                     
{'loss': 1.3623, 'grad_norm': 11.446976661682129, 'learning_rate': 4.997805375977707e-06, 'epoch': 0.04}
+
  4%|▍         | 800/18627 [25:43<9:04:33,  1.83s/it]
  4%|▍         | 801/18627 [25:45<9:29:37,  1.92s/it]
  4%|▍         | 802/18627 [25:46<8:24:35,  1.70s/it]
  4%|▍         | 803/18627 [25:48<9:00:32,  1.82s/it]
  4%|▍         | 804/18627 [25:50<9:27:01,  1.91s/it]
  4%|▍         | 805/18627 [25:53<9:46:30,  1.97s/it]
  4%|▍         | 806/18627 [25:55<9:57:56,  2.01s/it]
  4%|▍         | 807/18627 [25:57<10:07:36,  2.05s/it]
  4%|▍         | 808/18627 [25:59<10:12:23,  2.06s/it]
  4%|▍         | 809/18627 [26:01<10:18:33,  2.08s/it]
  4%|▍         | 810/18627 [26:03<10:23:17,  2.10s/it]
                                                      
{'loss': 1.2487, 'grad_norm': 9.434560775756836, 'learning_rate': 4.997619500453781e-06, 'epoch': 0.04}
+
  4%|▍         | 810/18627 [26:03<10:23:17,  2.10s/it]
  4%|▍         | 811/18627 [26:04<9:03:10,  1.83s/it] 
  4%|▍         | 812/18627 [26:06<8:07:12,  1.64s/it]
  4%|▍         | 813/18627 [26:08<8:49:47,  1.78s/it]
  4%|▍         | 814/18627 [26:10<9:24:49,  1.90s/it]
  4%|▍         | 815/18627 [26:12<9:45:06,  1.97s/it]
  4%|▍         | 816/18627 [26:14<9:58:50,  2.02s/it]
  4%|▍         | 817/18627 [26:16<10:07:14,  2.05s/it]
  4%|▍         | 818/18627 [26:17<8:52:35,  1.79s/it] 
  4%|▍         | 819/18627 [26:20<9:21:55,  1.89s/it]
  4%|▍         | 820/18627 [26:22<9:42:11,  1.96s/it]
                                                     
{'loss': 1.5364, 'grad_norm': 14.297173500061035, 'learning_rate': 4.997426073907626e-06, 'epoch': 0.04}
+
  4%|▍         | 820/18627 [26:22<9:42:11,  1.96s/it]
  4%|▍         | 821/18627 [26:22<7:49:16,  1.58s/it]
  4%|▍         | 822/18627 [26:23<6:30:10,  1.31s/it]
  4%|▍         | 823/18627 [26:25<7:40:37,  1.55s/it]
  4%|▍         | 824/18627 [26:27<8:31:25,  1.72s/it]
  4%|▍         | 825/18627 [26:29<9:08:07,  1.85s/it]
  4%|▍         | 826/18627 [26:32<9:32:35,  1.93s/it]
  4%|▍         | 827/18627 [26:33<8:26:20,  1.71s/it]
  4%|▍         | 828/18627 [26:35<9:03:33,  1.83s/it]
  4%|▍         | 829/18627 [26:36<7:56:19,  1.61s/it]
  4%|▍         | 830/18627 [26:38<8:43:08,  1.76s/it]
                                                     
{'loss': 1.3807, 'grad_norm': 12.35734748840332, 'learning_rate': 4.997225096924028e-06, 'epoch': 0.04}
+
  4%|▍         | 830/18627 [26:38<8:43:08,  1.76s/it]
  4%|▍         | 831/18627 [26:40<9:13:58,  1.87s/it]
  4%|▍         | 832/18627 [26:42<9:37:03,  1.95s/it]
  4%|▍         | 833/18627 [26:44<9:52:25,  2.00s/it]
  4%|▍         | 834/18627 [26:47<10:02:34,  2.03s/it]
  4%|▍         | 835/18627 [26:48<8:47:23,  1.78s/it] 
  4%|▍         | 836/18627 [26:49<7:39:24,  1.55s/it]
  4%|▍         | 837/18627 [26:50<7:07:43,  1.44s/it]
  4%|▍         | 838/18627 [26:52<8:08:28,  1.65s/it]
  5%|▍         | 839/18627 [26:54<8:49:37,  1.79s/it]
  5%|▍         | 840/18627 [26:56<9:18:08,  1.88s/it]
                                                     
{'loss': 1.5869, 'grad_norm': 10.338175773620605, 'learning_rate': 4.997016570110597e-06, 'epoch': 0.05}
+
  5%|▍         | 840/18627 [26:56<9:18:08,  1.88s/it]
  5%|▍         | 841/18627 [26:58<9:39:47,  1.96s/it]
  5%|▍         | 842/18627 [26:59<8:16:46,  1.68s/it]
  5%|▍         | 843/18627 [27:02<8:58:45,  1.82s/it]
  5%|▍         | 844/18627 [27:04<9:28:53,  1.92s/it]
  5%|▍         | 845/18627 [27:05<8:24:30,  1.70s/it]
  5%|▍         | 846/18627 [27:06<7:40:44,  1.55s/it]
  5%|▍         | 847/18627 [27:08<8:30:46,  1.72s/it]
  5%|▍         | 848/18627 [27:10<9:05:19,  1.84s/it]
  5%|▍         | 849/18627 [27:13<9:29:09,  1.92s/it]
  5%|▍         | 850/18627 [27:15<9:46:44,  1.98s/it]
                                                     
{'loss': 1.5674, 'grad_norm': 14.130302429199219, 'learning_rate': 4.996800494097768e-06, 'epoch': 0.05}
+
  5%|▍         | 850/18627 [27:15<9:46:44,  1.98s/it]
  5%|▍         | 851/18627 [27:16<8:36:42,  1.74s/it]
  5%|▍         | 852/18627 [27:18<9:09:04,  1.85s/it]
  5%|▍         | 853/18627 [27:19<7:25:57,  1.51s/it]
  5%|▍         | 854/18627 [27:20<6:57:43,  1.41s/it]
  5%|▍         | 855/18627 [27:22<8:00:50,  1.62s/it]
  5%|▍         | 856/18627 [27:24<8:44:12,  1.77s/it]
  5%|▍         | 857/18627 [27:25<7:08:36,  1.45s/it]
  5%|▍         | 858/18627 [27:27<8:09:40,  1.65s/it]
  5%|▍         | 859/18627 [27:28<6:45:13,  1.37s/it]
  5%|▍         | 860/18627 [27:30<7:52:10,  1.59s/it]
                                                     
{'loss': 1.3932, 'grad_norm': 11.551674842834473, 'learning_rate': 4.996576869538805e-06, 'epoch': 0.05}
+
  5%|▍         | 860/18627 [27:30<7:52:10,  1.59s/it]
  5%|▍         | 861/18627 [27:30<6:33:39,  1.33s/it]
  5%|▍         | 862/18627 [27:33<7:42:28,  1.56s/it]
  5%|▍         | 863/18627 [27:35<8:32:46,  1.73s/it]
  5%|▍         | 864/18627 [27:37<9:06:41,  1.85s/it]
  5%|▍         | 865/18627 [27:39<9:36:27,  1.95s/it]
  5%|▍         | 866/18627 [27:41<9:49:27,  1.99s/it]
  5%|▍         | 867/18627 [27:43<10:01:19,  2.03s/it]
  5%|▍         | 868/18627 [27:44<8:46:20,  1.78s/it] 
  5%|▍         | 869/18627 [27:45<7:39:10,  1.55s/it]
  5%|▍         | 870/18627 [27:48<8:31:15,  1.73s/it]
                                                     
{'loss': 1.4229, 'grad_norm': 16.6386775970459, 'learning_rate': 4.996345697109785e-06, 'epoch': 0.05}
+
  5%|▍         | 870/18627 [27:48<8:31:15,  1.73s/it]
  5%|▍         | 871/18627 [27:49<7:32:42,  1.53s/it]
  5%|▍         | 872/18627 [27:51<8:25:31,  1.71s/it]
  5%|▍         | 873/18627 [27:53<9:01:56,  1.83s/it]
  5%|▍         | 874/18627 [27:55<9:27:59,  1.92s/it]
  5%|▍         | 875/18627 [27:57<9:46:56,  1.98s/it]
  5%|▍         | 876/18627 [27:58<8:36:47,  1.75s/it]
  5%|▍         | 877/18627 [28:00<9:09:22,  1.86s/it]
  5%|▍         | 878/18627 [28:03<9:30:45,  1.93s/it]
  5%|▍         | 879/18627 [28:05<9:45:54,  1.98s/it]
  5%|▍         | 880/18627 [28:07<9:58:47,  2.02s/it]
                                                     
{'loss': 1.399, 'grad_norm': 10.258707046508789, 'learning_rate': 4.99610697750961e-06, 'epoch': 0.05}
+
  5%|▍         | 880/18627 [28:07<9:58:47,  2.02s/it]
  5%|▍         | 881/18627 [28:09<10:07:09,  2.05s/it]
  5%|▍         | 882/18627 [28:10<8:50:31,  1.79s/it] 
  5%|▍         | 883/18627 [28:11<7:57:08,  1.61s/it]
  5%|▍         | 884/18627 [28:13<8:41:55,  1.76s/it]
  5%|▍         | 885/18627 [28:15<9:12:29,  1.87s/it]
  5%|▍         | 886/18627 [28:18<9:35:21,  1.95s/it]
  5%|▍         | 887/18627 [28:20<9:50:12,  2.00s/it]
  5%|▍         | 888/18627 [28:20<7:55:28,  1.61s/it]
  5%|▍         | 889/18627 [28:23<8:41:26,  1.76s/it]
  5%|▍         | 890/18627 [28:25<9:10:52,  1.86s/it]
                                                     
{'loss': 1.3652, 'grad_norm': 12.27953052520752, 'learning_rate': 4.99586071146e-06, 'epoch': 0.05}
+
  5%|▍         | 890/18627 [28:25<9:10:52,  1.86s/it]
  5%|▍         | 891/18627 [28:28<11:41:58,  2.37s/it]
  5%|▍         | 892/18627 [28:29<9:47:47,  1.99s/it] 
  5%|▍         | 893/18627 [28:31<10:00:04,  2.03s/it]
  5%|▍         | 894/18627 [28:34<10:06:20,  2.05s/it]
  5%|▍         | 895/18627 [28:35<8:50:44,  1.80s/it] 
  5%|▍         | 896/18627 [28:36<7:56:21,  1.61s/it]
  5%|▍         | 897/18627 [28:38<8:43:08,  1.77s/it]
  5%|▍         | 898/18627 [28:40<9:11:46,  1.87s/it]
  5%|▍         | 899/18627 [28:42<9:33:42,  1.94s/it]
  5%|▍         | 900/18627 [28:43<8:19:08,  1.69s/it]
                                                     
{'loss': 1.7541, 'grad_norm': 18.203651428222656, 'learning_rate': 4.995606899705486e-06, 'epoch': 0.05}
+
  5%|▍         | 900/18627 [28:43<8:19:08,  1.69s/it]
  5%|▍         | 901/18627 [28:45<8:59:22,  1.83s/it]
  5%|▍         | 902/18627 [28:48<9:27:37,  1.92s/it]
  5%|▍         | 903/18627 [28:50<9:43:17,  1.97s/it]
  5%|▍         | 904/18627 [28:52<9:56:30,  2.02s/it]
  5%|▍         | 905/18627 [28:54<10:05:20,  2.05s/it]
  5%|▍         | 906/18627 [28:56<10:11:30,  2.07s/it]
  5%|▍         | 907/18627 [28:58<10:18:18,  2.09s/it]
  5%|▍         | 908/18627 [29:00<10:20:04,  2.10s/it]
  5%|▍         | 909/18627 [29:02<10:21:50,  2.11s/it]
  5%|▍         | 910/18627 [29:05<10:21:57,  2.11s/it]
                                                      
{'loss': 0.9897, 'grad_norm': 11.889490127563477, 'learning_rate': 4.9953455430134144e-06, 'epoch': 0.05}
+
  5%|▍         | 910/18627 [29:05<10:21:57,  2.11s/it]
  5%|▍         | 911/18627 [29:07<10:23:53,  2.11s/it]
  5%|▍         | 912/18627 [29:09<10:24:54,  2.12s/it]
  5%|▍         | 913/18627 [29:11<10:24:01,  2.11s/it]
  5%|▍         | 914/18627 [29:13<10:23:44,  2.11s/it]
  5%|▍         | 915/18627 [29:15<10:25:05,  2.12s/it]
  5%|▍         | 916/18627 [29:16<8:18:53,  1.69s/it] 
  5%|▍         | 917/18627 [29:18<8:55:32,  1.81s/it]
  5%|▍         | 918/18627 [29:20<9:21:52,  1.90s/it]
  5%|▍         | 919/18627 [29:22<9:41:15,  1.97s/it]
  5%|▍         | 920/18627 [29:24<9:56:42,  2.02s/it]
                                                     
{'loss': 0.9678, 'grad_norm': 9.412690162658691, 'learning_rate': 4.995076642173941e-06, 'epoch': 0.05}
+
  5%|▍         | 920/18627 [29:24<9:56:42,  2.02s/it]
  5%|▍         | 921/18627 [29:26<10:05:18,  2.05s/it]
  5%|▍         | 922/18627 [29:29<10:12:12,  2.07s/it]
  5%|▍         | 923/18627 [29:31<10:18:06,  2.09s/it]
  5%|▍         | 924/18627 [29:32<8:57:36,  1.82s/it] 
  5%|▍         | 925/18627 [29:34<9:25:04,  1.92s/it]
  5%|▍         | 926/18627 [29:35<8:21:21,  1.70s/it]
  5%|▍         | 927/18627 [29:36<7:36:29,  1.55s/it]
  5%|▍         | 928/18627 [29:38<7:04:42,  1.44s/it]
  5%|▍         | 929/18627 [29:40<8:05:01,  1.64s/it]
  5%|▍         | 930/18627 [29:42<8:46:02,  1.78s/it]
                                                     
{'loss': 1.6555, 'grad_norm': 16.13521957397461, 'learning_rate': 4.994800198000033e-06, 'epoch': 0.05}
+
  5%|▍         | 930/18627 [29:42<8:46:02,  1.78s/it]
  5%|▍         | 931/18627 [29:43<7:53:27,  1.61s/it]
  5%|▌         | 932/18627 [29:44<6:36:11,  1.34s/it]
  5%|▌         | 933/18627 [29:45<6:22:51,  1.30s/it]
  5%|▌         | 934/18627 [29:47<7:35:34,  1.54s/it]
  5%|▌         | 935/18627 [29:49<8:25:14,  1.71s/it]
  5%|▌         | 936/18627 [29:51<9:01:29,  1.84s/it]
  5%|▌         | 937/18627 [29:53<9:27:41,  1.93s/it]
  5%|▌         | 938/18627 [29:56<9:43:26,  1.98s/it]
  5%|▌         | 939/18627 [29:57<8:33:29,  1.74s/it]
  5%|▌         | 940/18627 [29:58<7:43:59,  1.57s/it]
                                                     
{'loss': 1.6006, 'grad_norm': 20.689594268798828, 'learning_rate': 4.9945162113274585e-06, 'epoch': 0.05}
+
  5%|▌         | 940/18627 [29:58<7:43:59,  1.57s/it]
  5%|▌         | 941/18627 [30:00<8:31:23,  1.73s/it]
  5%|▌         | 942/18627 [30:01<7:43:02,  1.57s/it]
  5%|▌         | 943/18627 [30:03<8:30:15,  1.73s/it]
  5%|▌         | 944/18627 [30:05<9:04:27,  1.85s/it]
  5%|▌         | 945/18627 [30:07<8:05:27,  1.65s/it]
  5%|▌         | 946/18627 [30:09<8:46:25,  1.79s/it]
  5%|▌         | 947/18627 [30:10<7:53:38,  1.61s/it]
  5%|▌         | 948/18627 [30:12<8:38:00,  1.76s/it]
  5%|▌         | 949/18627 [30:14<9:08:49,  1.86s/it]
  5%|▌         | 950/18627 [30:15<8:09:04,  1.66s/it]
                                                     
{'loss': 1.7834, 'grad_norm': 18.512807846069336, 'learning_rate': 4.994224683014791e-06, 'epoch': 0.05}
+
  5%|▌         | 950/18627 [30:15<8:09:04,  1.66s/it]
  5%|▌         | 951/18627 [30:17<7:28:13,  1.52s/it]
  5%|▌         | 952/18627 [30:19<8:23:51,  1.71s/it]
  5%|▌         | 953/18627 [30:21<9:01:14,  1.84s/it]
  5%|▌         | 954/18627 [30:22<8:05:09,  1.65s/it]
  5%|▌         | 955/18627 [30:24<8:45:39,  1.78s/it]
  5%|▌         | 956/18627 [30:26<9:15:17,  1.89s/it]
  5%|▌         | 957/18627 [30:28<9:37:10,  1.96s/it]
  5%|▌         | 958/18627 [30:30<8:29:16,  1.73s/it]
  5%|▌         | 959/18627 [30:32<9:01:19,  1.84s/it]
  5%|▌         | 960/18627 [30:34<9:25:46,  1.92s/it]
                                                     
{'loss': 1.6591, 'grad_norm': 10.637067794799805, 'learning_rate': 4.993925613943405e-06, 'epoch': 0.05}
+
  5%|▌         | 960/18627 [30:34<9:25:46,  1.92s/it]
  5%|▌         | 961/18627 [30:36<9:43:14,  1.98s/it]
  5%|▌         | 962/18627 [30:38<9:55:27,  2.02s/it]
  5%|▌         | 963/18627 [30:40<10:04:33,  2.05s/it]
  5%|▌         | 964/18627 [30:42<10:10:04,  2.07s/it]
  5%|▌         | 965/18627 [30:44<10:15:29,  2.09s/it]
  5%|▌         | 966/18627 [30:45<8:12:21,  1.67s/it] 
  5%|▌         | 967/18627 [30:47<8:52:07,  1.81s/it]
  5%|▌         | 968/18627 [30:49<9:21:37,  1.91s/it]
  5%|▌         | 969/18627 [30:51<9:38:31,  1.97s/it]
  5%|▌         | 970/18627 [30:54<9:53:14,  2.02s/it]
                                                     
{'loss': 1.0251, 'grad_norm': 10.417563438415527, 'learning_rate': 4.993619005017472e-06, 'epoch': 0.05}
+
  5%|▌         | 970/18627 [30:54<9:53:14,  2.02s/it]
  5%|▌         | 971/18627 [30:56<10:03:00,  2.05s/it]
  5%|▌         | 972/18627 [30:57<8:32:24,  1.74s/it] 
  5%|▌         | 973/18627 [30:59<9:05:43,  1.85s/it]
  5%|▌         | 974/18627 [31:01<9:31:03,  1.94s/it]
  5%|▌         | 975/18627 [31:03<9:47:21,  2.00s/it]
  5%|▌         | 976/18627 [31:05<9:58:45,  2.04s/it]
  5%|▌         | 977/18627 [31:06<8:45:50,  1.79s/it]
  5%|▌         | 978/18627 [31:08<7:52:55,  1.61s/it]
  5%|▌         | 979/18627 [31:10<8:38:35,  1.76s/it]
  5%|▌         | 980/18627 [31:12<9:10:19,  1.87s/it]
                                                     
{'loss': 1.5916, 'grad_norm': 12.903435707092285, 'learning_rate': 4.993304857163959e-06, 'epoch': 0.05}
+
  5%|▌         | 980/18627 [31:12<9:10:19,  1.87s/it]
  5%|▌         | 981/18627 [31:13<8:10:20,  1.67s/it]
  5%|▌         | 982/18627 [31:15<8:51:24,  1.81s/it]
  5%|▌         | 983/18627 [31:17<9:16:50,  1.89s/it]
  5%|▌         | 984/18627 [31:19<9:35:17,  1.96s/it]
  5%|▌         | 985/18627 [31:22<9:51:27,  2.01s/it]
  5%|▌         | 986/18627 [31:24<10:00:22,  2.04s/it]
  5%|▌         | 987/18627 [31:26<10:06:51,  2.06s/it]
  5%|▌         | 988/18627 [31:28<10:11:12,  2.08s/it]
  5%|▌         | 989/18627 [31:29<8:37:40,  1.76s/it] 
  5%|▌         | 990/18627 [31:31<9:09:31,  1.87s/it]
                                                     
{'loss': 1.3659, 'grad_norm': 11.987259864807129, 'learning_rate': 4.992983171332625e-06, 'epoch': 0.05}
+
  5%|▌         | 990/18627 [31:31<9:09:31,  1.87s/it]
  5%|▌         | 991/18627 [31:33<9:31:58,  1.95s/it]
  5%|▌         | 992/18627 [31:34<7:41:41,  1.57s/it]
  5%|▌         | 993/18627 [31:36<8:30:49,  1.74s/it]
  5%|▌         | 994/18627 [31:38<9:04:27,  1.85s/it]
  5%|▌         | 995/18627 [31:40<9:26:26,  1.93s/it]
  5%|▌         | 996/18627 [31:41<8:21:08,  1.71s/it]
  5%|▌         | 997/18627 [31:44<8:58:27,  1.83s/it]
  5%|▌         | 998/18627 [31:46<9:24:23,  1.92s/it]
  5%|▌         | 999/18627 [31:48<9:43:11,  1.99s/it]
  5%|▌         | 1000/18627 [31:50<9:53:14,  2.02s/it]
                                                      
{'loss': 1.1913, 'grad_norm': 14.259053230285645, 'learning_rate': 4.99265394849602e-06, 'epoch': 0.05}
+
  5%|▌         | 1000/18627 [31:50<9:53:14,  2.02s/it]/home/zhengduo/workspace/LLaMA-Factory/src/qwen_vl/model/modeling_qwen2_5_vl.py:1979: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
+  with torch.cuda.amp.autocast(dtype=dtype):
+/home/zhengduo/miniconda3/envs/vgllm/lib/python3.10/site-packages/torch/utils/checkpoint.py:87: UserWarning: None of the inputs have requires_grad=True. Gradients will be None
+  warnings.warn(
+
  5%|▌         | 1001/18627 [32:29<64:12:45, 13.12s/it]
  5%|▌         | 1002/18627 [32:31<48:03:30,  9.82s/it]
  5%|▌         | 1003/18627 [32:33<36:48:59,  7.52s/it]
  5%|▌         | 1004/18627 [32:35<28:57:17,  5.91s/it]
  5%|▌         | 1005/18627 [32:38<23:23:32,  4.78s/it]
  5%|▌         | 1006/18627 [32:40<19:28:35,  3.98s/it]
  5%|▌         | 1007/18627 [32:42<16:42:52,  3.41s/it]
  5%|▌         | 1008/18627 [32:42<12:43:20,  2.60s/it]
  5%|▌         | 1009/18627 [32:45<11:59:54,  2.45s/it]
  5%|▌         | 1010/18627 [32:47<11:31:52,  2.36s/it]
                                                       
{'loss': 0.9519, 'grad_norm': 10.52511215209961, 'learning_rate': 4.992317189649478e-06, 'epoch': 0.05}
+
  5%|▌         | 1010/18627 [32:47<11:31:52,  2.36s/it]
  5%|▌         | 1011/18627 [32:48<9:41:51,  1.98s/it] 
  5%|▌         | 1012/18627 [32:50<9:55:05,  2.03s/it]
  5%|▌         | 1013/18627 [32:52<10:02:06,  2.05s/it]
  5%|▌         | 1014/18627 [32:54<10:08:38,  2.07s/it]
  5%|▌         | 1015/18627 [32:56<10:13:08,  2.09s/it]
  5%|▌         | 1016/18627 [32:58<10:18:37,  2.11s/it]
  5%|▌         | 1017/18627 [33:01<10:20:43,  2.11s/it]
  5%|▌         | 1018/18627 [33:03<10:20:59,  2.12s/it]
  5%|▌         | 1019/18627 [33:04<8:59:36,  1.84s/it] 
  5%|▌         | 1020/18627 [33:06<9:24:46,  1.92s/it]
                                                      
{'loss': 1.3438, 'grad_norm': 11.156561851501465, 'learning_rate': 4.991972895811119e-06, 'epoch': 0.05}
+
  5%|▌         | 1020/18627 [33:06<9:24:46,  1.92s/it]
  5%|▌         | 1021/18627 [33:08<9:42:13,  1.98s/it]
  5%|▌         | 1022/18627 [33:10<9:53:16,  2.02s/it]
  5%|▌         | 1023/18627 [33:11<8:39:59,  1.77s/it]
  5%|▌         | 1024/18627 [33:14<9:10:01,  1.87s/it]
  6%|▌         | 1025/18627 [33:14<7:34:36,  1.55s/it]
  6%|▌         | 1026/18627 [33:16<8:24:16,  1.72s/it]
  6%|▌         | 1027/18627 [33:18<7:38:54,  1.56s/it]
  6%|▌         | 1028/18627 [33:19<7:06:57,  1.46s/it]
  6%|▌         | 1029/18627 [33:21<8:05:24,  1.65s/it]
  6%|▌         | 1030/18627 [33:22<7:24:30,  1.52s/it]
                                                      
{'loss': 1.6984, 'grad_norm': 16.05250358581543, 'learning_rate': 4.991621068021842e-06, 'epoch': 0.06}
+
  6%|▌         | 1030/18627 [33:22<7:24:30,  1.52s/it]
  6%|▌         | 1031/18627 [33:24<8:14:46,  1.69s/it]
  6%|▌         | 1032/18627 [33:25<7:16:02,  1.49s/it]
  6%|▌         | 1033/18627 [33:27<8:10:45,  1.67s/it]
  6%|▌         | 1034/18627 [33:28<7:18:47,  1.50s/it]
  6%|▌         | 1035/18627 [33:31<8:12:47,  1.68s/it]
  6%|▌         | 1036/18627 [33:33<8:50:38,  1.81s/it]
  6%|▌         | 1037/18627 [33:34<7:56:37,  1.63s/it]
  6%|▌         | 1038/18627 [33:36<8:42:55,  1.78s/it]
  6%|▌         | 1039/18627 [33:38<9:14:25,  1.89s/it]
  6%|▌         | 1040/18627 [33:40<9:36:39,  1.97s/it]
                                                      
{'loss': 1.5235, 'grad_norm': 15.92250919342041, 'learning_rate': 4.991261707345323e-06, 'epoch': 0.06}
+
  6%|▌         | 1040/18627 [33:40<9:36:39,  1.97s/it]
  6%|▌         | 1041/18627 [33:42<9:51:06,  2.02s/it]
  6%|▌         | 1042/18627 [33:45<9:59:45,  2.05s/it]
  6%|▌         | 1043/18627 [33:47<10:09:02,  2.08s/it]
  6%|▌         | 1044/18627 [33:49<10:13:01,  2.09s/it]
  6%|▌         | 1045/18627 [33:51<10:15:10,  2.10s/it]
  6%|▌         | 1046/18627 [33:53<10:17:51,  2.11s/it]
  6%|▌         | 1047/18627 [33:55<10:19:37,  2.11s/it]
  6%|▌         | 1048/18627 [33:57<10:21:15,  2.12s/it]
  6%|▌         | 1049/18627 [33:58<8:48:01,  1.80s/it] 
  6%|▌         | 1050/18627 [34:01<9:18:21,  1.91s/it]
                                                      
{'loss': 1.1451, 'grad_norm': 13.844340324401855, 'learning_rate': 4.990894814868012e-06, 'epoch': 0.06}
+
  6%|▌         | 1050/18627 [34:01<9:18:21,  1.91s/it]
  6%|▌         | 1051/18627 [34:03<9:36:59,  1.97s/it]
  6%|▌         | 1052/18627 [34:05<9:51:52,  2.02s/it]
  6%|▌         | 1053/18627 [34:07<10:01:27,  2.05s/it]
  6%|▌         | 1054/18627 [34:09<10:07:37,  2.07s/it]
  6%|▌         | 1055/18627 [34:11<10:15:03,  2.10s/it]
  6%|▌         | 1056/18627 [34:13<10:19:16,  2.11s/it]
  6%|▌         | 1057/18627 [34:16<10:24:09,  2.13s/it]
  6%|▌         | 1058/18627 [34:17<9:04:28,  1.86s/it] 
  6%|▌         | 1059/18627 [34:19<9:27:47,  1.94s/it]
  6%|▌         | 1060/18627 [34:21<9:44:40,  2.00s/it]
                                                      
{'loss': 1.2051, 'grad_norm': 13.630294799804688, 'learning_rate': 4.990520391699133e-06, 'epoch': 0.06}
+
  6%|▌         | 1060/18627 [34:21<9:44:40,  2.00s/it]
  6%|▌         | 1061/18627 [34:23<9:53:50,  2.03s/it]
  6%|▌         | 1062/18627 [34:24<8:24:17,  1.72s/it]
  6%|▌         | 1063/18627 [34:26<8:59:47,  1.84s/it]
  6%|▌         | 1064/18627 [34:28<9:24:16,  1.93s/it]
  6%|▌         | 1065/18627 [34:30<9:39:50,  1.98s/it]
  6%|▌         | 1066/18627 [34:33<9:50:49,  2.02s/it]
  6%|▌         | 1067/18627 [34:35<10:00:52,  2.05s/it]
  6%|▌         | 1068/18627 [34:37<10:06:16,  2.07s/it]
  6%|▌         | 1069/18627 [34:39<10:09:55,  2.08s/it]
  6%|▌         | 1070/18627 [34:40<8:51:16,  1.82s/it] 
                                                      
{'loss': 1.2836, 'grad_norm': 15.19543170928955, 'learning_rate': 4.990138438970673e-06, 'epoch': 0.06}
+
  6%|▌         | 1070/18627 [34:40<8:51:16,  1.82s/it]
  6%|▌         | 1071/18627 [34:41<7:57:35,  1.63s/it]
  6%|▌         | 1072/18627 [34:43<7:19:08,  1.50s/it]
  6%|▌         | 1073/18627 [34:45<8:12:50,  1.68s/it]
  6%|▌         | 1074/18627 [34:47<8:50:11,  1.81s/it]
  6%|▌         | 1075/18627 [34:49<9:16:36,  1.90s/it]
  6%|▌         | 1076/18627 [34:50<8:04:27,  1.66s/it]
  6%|▌         | 1077/18627 [34:51<7:23:32,  1.52s/it]
  6%|▌         | 1078/18627 [34:53<8:14:22,  1.69s/it]
  6%|▌         | 1079/18627 [34:55<8:51:30,  1.82s/it]
  6%|▌         | 1080/18627 [34:57<9:19:39,  1.91s/it]
                                                      
{'loss': 1.6094, 'grad_norm': 14.003798484802246, 'learning_rate': 4.9897489578373855e-06, 'epoch': 0.06}
+
  6%|▌         | 1080/18627 [34:57<9:19:39,  1.91s/it]
  6%|▌         | 1081/18627 [35:00<9:38:29,  1.98s/it]
  6%|▌         | 1082/18627 [35:01<8:28:58,  1.74s/it]
  6%|▌         | 1083/18627 [35:03<9:02:45,  1.86s/it]
  6%|▌         | 1084/18627 [35:05<9:24:54,  1.93s/it]
  6%|▌         | 1085/18627 [35:07<9:40:38,  1.99s/it]
  6%|▌         | 1086/18627 [35:09<9:51:51,  2.02s/it]
  6%|▌         | 1087/18627 [35:11<9:59:08,  2.05s/it]
  6%|▌         | 1088/18627 [35:14<10:07:40,  2.08s/it]
  6%|▌         | 1089/18627 [35:16<10:11:41,  2.09s/it]
  6%|▌         | 1090/18627 [35:18<10:14:26,  2.10s/it]
                                                       
{'loss': 1.1312, 'grad_norm': 10.518413543701172, 'learning_rate': 4.989351949476785e-06, 'epoch': 0.06}
+
  6%|▌         | 1090/18627 [35:18<10:14:26,  2.10s/it]
  6%|▌         | 1091/18627 [35:19<8:54:10,  1.83s/it] 
  6%|▌         | 1092/18627 [35:20<7:48:58,  1.60s/it]
  6%|▌         | 1093/18627 [35:22<8:33:46,  1.76s/it]
  6%|▌         | 1094/18627 [35:23<7:43:20,  1.59s/it]
  6%|▌         | 1095/18627 [35:25<8:32:08,  1.75s/it]
  6%|▌         | 1096/18627 [35:28<9:02:26,  1.86s/it]
  6%|▌         | 1097/18627 [35:29<8:03:35,  1.66s/it]
  6%|▌         | 1098/18627 [35:31<8:43:13,  1.79s/it]
  6%|▌         | 1099/18627 [35:33<9:12:27,  1.89s/it]
  6%|▌         | 1100/18627 [35:35<9:31:56,  1.96s/it]
                                                      
{'loss': 1.5919, 'grad_norm': 11.736445426940918, 'learning_rate': 4.9889474150891396e-06, 'epoch': 0.06}
+
  6%|▌         | 1100/18627 [35:35<9:31:56,  1.96s/it]
  6%|▌         | 1101/18627 [35:37<9:45:56,  2.01s/it]
  6%|▌         | 1102/18627 [35:39<9:53:22,  2.03s/it]
  6%|▌         | 1103/18627 [35:41<8:40:25,  1.78s/it]
  6%|▌         | 1104/18627 [35:43<9:11:00,  1.89s/it]
  6%|▌         | 1105/18627 [35:45<9:30:22,  1.95s/it]
  6%|▌         | 1106/18627 [35:47<9:45:31,  2.01s/it]
  6%|▌         | 1107/18627 [35:48<8:34:11,  1.76s/it]
  6%|▌         | 1108/18627 [35:50<9:04:46,  1.87s/it]
  6%|▌         | 1109/18627 [35:52<9:27:00,  1.94s/it]
  6%|▌         | 1110/18627 [35:54<9:41:22,  1.99s/it]
                                                      
{'loss': 1.3014, 'grad_norm': 10.473881721496582, 'learning_rate': 4.988535355897475e-06, 'epoch': 0.06}
+
  6%|▌         | 1110/18627 [35:54<9:41:22,  1.99s/it]
  6%|▌         | 1111/18627 [35:56<8:30:53,  1.75s/it]
  6%|▌         | 1112/18627 [35:58<9:04:09,  1.86s/it]
  6%|▌         | 1113/18627 [36:00<9:26:52,  1.94s/it]
  6%|▌         | 1114/18627 [36:01<8:20:21,  1.71s/it]
  6%|▌         | 1115/18627 [36:03<8:55:37,  1.84s/it]
  6%|▌         | 1116/18627 [36:05<9:20:28,  1.92s/it]
  6%|▌         | 1117/18627 [36:07<9:37:54,  1.98s/it]
  6%|▌         | 1118/18627 [36:09<9:48:17,  2.02s/it]
  6%|▌         | 1119/18627 [36:12<9:57:56,  2.05s/it]
  6%|▌         | 1120/18627 [36:14<10:02:29,  2.06s/it]
                                                       
{'loss': 1.2997, 'grad_norm': 10.007888793945312, 'learning_rate': 4.988115773147563e-06, 'epoch': 0.06}
+
  6%|▌         | 1120/18627 [36:14<10:02:29,  2.06s/it]
  6%|▌         | 1121/18627 [36:16<10:07:19,  2.08s/it]
  6%|▌         | 1122/18627 [36:18<10:08:22,  2.09s/it]
  6%|▌         | 1123/18627 [36:20<10:12:10,  2.10s/it]
  6%|▌         | 1124/18627 [36:21<8:52:37,  1.83s/it] 
  6%|▌         | 1125/18627 [36:22<7:42:17,  1.58s/it]
  6%|▌         | 1126/18627 [36:23<7:09:28,  1.47s/it]
  6%|▌         | 1127/18627 [36:26<8:05:53,  1.67s/it]
  6%|▌         | 1128/18627 [36:28<8:46:57,  1.81s/it]
  6%|▌         | 1129/18627 [36:30<9:15:06,  1.90s/it]
  6%|▌         | 1130/18627 [36:31<8:11:35,  1.69s/it]
                                                      
{'loss': 1.7064, 'grad_norm': 17.150348663330078, 'learning_rate': 4.987688668107926e-06, 'epoch': 0.06}
+
  6%|▌         | 1130/18627 [36:31<8:11:35,  1.69s/it]
  6%|▌         | 1131/18627 [36:33<8:50:30,  1.82s/it]
  6%|▌         | 1132/18627 [36:35<9:15:34,  1.91s/it]
  6%|▌         | 1133/18627 [36:37<9:31:48,  1.96s/it]
  6%|▌         | 1134/18627 [36:39<9:45:40,  2.01s/it]
  6%|▌         | 1135/18627 [36:42<9:52:58,  2.03s/it]
  6%|▌         | 1136/18627 [36:44<10:01:08,  2.06s/it]
  6%|▌         | 1137/18627 [36:45<8:44:38,  1.80s/it] 
  6%|▌         | 1138/18627 [36:47<9:13:00,  1.90s/it]
  6%|▌         | 1139/18627 [36:49<9:30:28,  1.96s/it]
  6%|▌         | 1140/18627 [36:51<9:45:14,  2.01s/it]
                                                      
{'loss': 0.9953, 'grad_norm': 11.549002647399902, 'learning_rate': 4.987254042069823e-06, 'epoch': 0.06}
+
  6%|▌         | 1140/18627 [36:51<9:45:14,  2.01s/it]
  6%|▌         | 1141/18627 [36:53<9:54:37,  2.04s/it]
  6%|▌         | 1142/18627 [36:55<9:59:31,  2.06s/it]
  6%|▌         | 1143/18627 [36:57<8:42:37,  1.79s/it]
  6%|▌         | 1144/18627 [36:59<9:11:35,  1.89s/it]
  6%|▌         | 1145/18627 [37:01<9:31:51,  1.96s/it]
  6%|▌         | 1146/18627 [37:03<9:45:10,  2.01s/it]
  6%|▌         | 1147/18627 [37:05<9:55:11,  2.04s/it]
  6%|▌         | 1148/18627 [37:07<10:01:04,  2.06s/it]
  6%|▌         | 1149/18627 [37:08<8:46:41,  1.81s/it] 
  6%|▌         | 1150/18627 [37:11<9:15:39,  1.91s/it]
                                                      
{'loss': 1.2676, 'grad_norm': 12.428756713867188, 'learning_rate': 4.986811896347254e-06, 'epoch': 0.06}
+
  6%|▌         | 1150/18627 [37:11<9:15:39,  1.91s/it]
  6%|▌         | 1151/18627 [37:12<8:04:15,  1.66s/it]
  6%|▌         | 1152/18627 [37:14<8:44:11,  1.80s/it]
  6%|▌         | 1153/18627 [37:16<9:13:20,  1.90s/it]
  6%|▌         | 1154/18627 [37:18<9:31:31,  1.96s/it]
  6%|▌         | 1155/18627 [37:20<9:45:27,  2.01s/it]
  6%|▌         | 1156/18627 [37:22<9:53:36,  2.04s/it]
  6%|▌         | 1157/18627 [37:24<9:59:06,  2.06s/it]
  6%|▌         | 1158/18627 [37:26<10:03:04,  2.07s/it]
  6%|▌         | 1159/18627 [37:28<8:45:05,  1.80s/it] 
  6%|▌         | 1160/18627 [37:30<9:12:23,  1.90s/it]
                                                      
{'loss': 1.4153, 'grad_norm': 8.988959312438965, 'learning_rate': 4.986362232276953e-06, 'epoch': 0.06}
+
  6%|▌         | 1160/18627 [37:30<9:12:23,  1.90s/it]
  6%|▌         | 1161/18627 [37:32<9:31:57,  1.96s/it]
  6%|▌         | 1162/18627 [37:33<8:24:22,  1.73s/it]
  6%|▌         | 1163/18627 [37:34<7:36:11,  1.57s/it]
  6%|▌         | 1164/18627 [37:36<8:24:31,  1.73s/it]
  6%|▋         | 1165/18627 [37:39<8:58:12,  1.85s/it]
  6%|▋         | 1166/18627 [37:40<8:00:56,  1.65s/it]
  6%|▋         | 1167/18627 [37:40<6:37:47,  1.37s/it]
  6%|▋         | 1168/18627 [37:43<7:42:51,  1.59s/it]
  6%|▋         | 1169/18627 [37:45<8:32:09,  1.76s/it]
  6%|▋         | 1170/18627 [37:45<7:05:17,  1.46s/it]
                                                      
{'loss': 1.3826, 'grad_norm': 11.30033016204834, 'learning_rate': 4.9859050512183835e-06, 'epoch': 0.06}
+
  6%|▋         | 1170/18627 [37:45<7:05:17,  1.46s/it]
  6%|▋         | 1171/18627 [37:48<8:05:08,  1.67s/it]
  6%|▋         | 1172/18627 [37:49<7:23:40,  1.53s/it]
  6%|▋         | 1173/18627 [37:50<6:54:06,  1.42s/it]
  6%|▋         | 1174/18627 [37:52<7:58:08,  1.64s/it]
  6%|▋         | 1175/18627 [37:53<7:17:23,  1.50s/it]
  6%|▋         | 1176/18627 [37:55<8:14:18,  1.70s/it]
  6%|▋         | 1177/18627 [37:58<8:54:36,  1.84s/it]
  6%|▋         | 1178/18627 [38:00<9:22:31,  1.93s/it]
  6%|▋         | 1179/18627 [38:02<9:43:12,  2.01s/it]
  6%|▋         | 1180/18627 [38:04<9:55:56,  2.05s/it]
                                                      
{'loss': 1.5355, 'grad_norm': 9.830764770507812, 'learning_rate': 4.985440354553735e-06, 'epoch': 0.06}
+
  6%|▋         | 1180/18627 [38:04<9:55:56,  2.05s/it]
  6%|▋         | 1181/18627 [38:06<10:05:20,  2.08s/it]
  6%|▋         | 1182/18627 [38:08<10:11:26,  2.10s/it]
  6%|▋         | 1183/18627 [38:11<10:15:18,  2.12s/it]
  6%|▋         | 1184/18627 [38:13<10:17:14,  2.12s/it]
  6%|▋         | 1185/18627 [38:15<10:15:58,  2.12s/it]
  6%|▋         | 1186/18627 [38:17<10:16:35,  2.12s/it]
  6%|▋         | 1187/18627 [38:19<10:16:21,  2.12s/it]
  6%|▋         | 1188/18627 [38:21<10:16:53,  2.12s/it]
  6%|▋         | 1189/18627 [38:22<8:12:36,  1.69s/it] 
  6%|▋         | 1190/18627 [38:23<7:28:41,  1.54s/it]
                                                      
{'loss': 1.169, 'grad_norm': 19.010128021240234, 'learning_rate': 4.98496814368792e-06, 'epoch': 0.06}
+
  6%|▋         | 1190/18627 [38:23<7:28:41,  1.54s/it]
  6%|▋         | 1191/18627 [38:25<8:18:45,  1.72s/it]
  6%|▋         | 1192/18627 [38:26<6:49:33,  1.41s/it]
  6%|▋         | 1193/18627 [38:28<7:49:45,  1.62s/it]
  6%|▋         | 1194/18627 [38:30<8:32:58,  1.77s/it]
  6%|▋         | 1195/18627 [38:32<9:03:48,  1.87s/it]
  6%|▋         | 1196/18627 [38:34<9:25:59,  1.95s/it]
  6%|▋         | 1197/18627 [38:36<9:42:58,  2.01s/it]
  6%|▋         | 1198/18627 [38:38<8:32:45,  1.77s/it]
  6%|▋         | 1199/18627 [38:40<9:04:26,  1.87s/it]
  6%|▋         | 1200/18627 [38:42<9:27:19,  1.95s/it]
                                                      
{'loss': 1.093, 'grad_norm': 12.570661544799805, 'learning_rate': 4.984488420048567e-06, 'epoch': 0.06}
+
  6%|▋         | 1200/18627 [38:42<9:27:19,  1.95s/it]
  6%|▋         | 1201/18627 [38:43<8:21:21,  1.73s/it]
  6%|▋         | 1202/18627 [38:45<8:56:19,  1.85s/it]
  6%|▋         | 1203/18627 [38:46<7:58:51,  1.65s/it]
  6%|▋         | 1204/18627 [38:49<8:41:29,  1.80s/it]
  6%|▋         | 1205/18627 [38:51<9:09:47,  1.89s/it]
  6%|▋         | 1206/18627 [38:53<9:30:51,  1.97s/it]
  6%|▋         | 1207/18627 [38:55<9:42:49,  2.01s/it]
  6%|▋         | 1208/18627 [38:57<9:52:43,  2.04s/it]
  6%|▋         | 1209/18627 [38:59<9:58:15,  2.06s/it]
  6%|▋         | 1210/18627 [39:00<8:42:50,  1.80s/it]
                                                      
{'loss': 1.3871, 'grad_norm': 19.42620277404785, 'learning_rate': 4.98400118508602e-06, 'epoch': 0.06}
+
  6%|▋         | 1210/18627 [39:00<8:42:50,  1.80s/it]
  7%|▋         | 1211/18627 [39:03<9:11:55,  1.90s/it]
  7%|▋         | 1212/18627 [39:05<9:30:48,  1.97s/it]
  7%|▋         | 1213/18627 [39:06<8:23:56,  1.74s/it]
  7%|▋         | 1214/18627 [39:08<8:56:59,  1.85s/it]
  7%|▋         | 1215/18627 [39:10<9:20:13,  1.93s/it]
  7%|▋         | 1216/18627 [39:12<9:37:35,  1.99s/it]
  7%|▋         | 1217/18627 [39:13<8:28:37,  1.75s/it]
  7%|▋         | 1218/18627 [39:16<8:59:28,  1.86s/it]
  7%|▋         | 1219/18627 [39:18<9:21:24,  1.94s/it]
  7%|▋         | 1220/18627 [39:19<8:18:46,  1.72s/it]
                                                      
{'loss': 1.5047, 'grad_norm': 14.955121994018555, 'learning_rate': 4.9835064402733285e-06, 'epoch': 0.07}
+
  7%|▋         | 1220/18627 [39:19<8:18:46,  1.72s/it]
  7%|▋         | 1221/18627 [39:21<8:52:57,  1.84s/it]
  7%|▋         | 1222/18627 [39:22<7:55:50,  1.64s/it]
  7%|▋         | 1223/18627 [39:24<8:37:17,  1.78s/it]
  7%|▋         | 1224/18627 [39:25<7:45:52,  1.61s/it]
  7%|▋         | 1225/18627 [39:28<8:30:47,  1.76s/it]
  7%|▋         | 1226/18627 [39:30<9:01:49,  1.87s/it]
  7%|▋         | 1227/18627 [39:32<9:22:32,  1.94s/it]
  7%|▋         | 1228/18627 [39:34<9:39:10,  2.00s/it]
  7%|▋         | 1229/18627 [39:36<9:50:46,  2.04s/it]
  7%|▋         | 1230/18627 [39:37<8:29:43,  1.76s/it]
                                                      
{'loss': 1.4022, 'grad_norm': 16.1697998046875, 'learning_rate': 4.98300418710625e-06, 'epoch': 0.07}
+
  7%|▋         | 1230/18627 [39:37<8:29:43,  1.76s/it]
  7%|▋         | 1231/18627 [39:39<8:59:25,  1.86s/it]
  7%|▋         | 1232/18627 [39:41<9:22:08,  1.94s/it]
  7%|▋         | 1233/18627 [39:44<9:40:19,  2.00s/it]
  7%|▋         | 1234/18627 [39:46<9:49:54,  2.04s/it]
  7%|▋         | 1235/18627 [39:47<8:36:00,  1.78s/it]
  7%|▋         | 1236/18627 [39:48<7:02:30,  1.46s/it]
  7%|▋         | 1237/18627 [39:49<6:39:23,  1.38s/it]
  7%|▋         | 1238/18627 [39:51<7:44:43,  1.60s/it]
  7%|▋         | 1239/18627 [39:53<8:28:57,  1.76s/it]
  7%|▋         | 1240/18627 [39:55<8:58:54,  1.86s/it]
                                                      
{'loss': 1.2772, 'grad_norm': 11.658538818359375, 'learning_rate': 4.982494427103239e-06, 'epoch': 0.07}
+
  7%|▋         | 1240/18627 [39:55<8:58:54,  1.86s/it]
  7%|▋         | 1241/18627 [39:57<9:22:44,  1.94s/it]
  7%|▋         | 1242/18627 [39:59<9:38:05,  2.00s/it]
  7%|▋         | 1243/18627 [40:01<9:46:19,  2.02s/it]
  7%|▋         | 1244/18627 [40:03<8:34:00,  1.77s/it]
  7%|▋         | 1245/18627 [40:05<9:06:13,  1.89s/it]
  7%|▋         | 1246/18627 [40:07<9:27:23,  1.96s/it]
  7%|▋         | 1247/18627 [40:09<9:41:06,  2.01s/it]
  7%|▋         | 1248/18627 [40:11<9:50:59,  2.04s/it]
  7%|▋         | 1249/18627 [40:13<9:57:12,  2.06s/it]
  7%|▋         | 1250/18627 [40:14<8:34:41,  1.78s/it]
                                                      
{'loss': 1.2066, 'grad_norm': 16.234020233154297, 'learning_rate': 4.981977161805448e-06, 'epoch': 0.07}
+
  7%|▋         | 1250/18627 [40:14<8:34:41,  1.78s/it]
  7%|▋         | 1251/18627 [40:16<9:04:16,  1.88s/it]
  7%|▋         | 1252/18627 [40:19<9:24:29,  1.95s/it]
  7%|▋         | 1253/18627 [40:20<8:18:26,  1.72s/it]
  7%|▋         | 1254/18627 [40:22<8:51:49,  1.84s/it]
  7%|▋         | 1255/18627 [40:24<9:16:49,  1.92s/it]
  7%|▋         | 1256/18627 [40:26<9:34:10,  1.98s/it]
  7%|▋         | 1257/18627 [40:28<9:45:10,  2.02s/it]
  7%|▋         | 1258/18627 [40:30<9:55:17,  2.06s/it]
  7%|▋         | 1259/18627 [40:32<10:01:21,  2.08s/it]
  7%|▋         | 1260/18627 [40:35<10:05:21,  2.09s/it]
                                                       
{'loss': 1.1038, 'grad_norm': 11.486591339111328, 'learning_rate': 4.981452392776718e-06, 'epoch': 0.07}
+
  7%|▋         | 1260/18627 [40:35<10:05:21,  2.09s/it]
  7%|▋         | 1261/18627 [40:36<8:46:48,  1.82s/it] 
  7%|▋         | 1262/18627 [40:37<7:52:43,  1.63s/it]
  7%|▋         | 1263/18627 [40:39<8:35:09,  1.78s/it]
  7%|▋         | 1264/18627 [40:41<9:05:05,  1.88s/it]
  7%|▋         | 1265/18627 [40:43<9:24:37,  1.95s/it]
  7%|▋         | 1266/18627 [40:44<8:05:42,  1.68s/it]
  7%|▋         | 1267/18627 [40:46<8:43:03,  1.81s/it]
  7%|▋         | 1268/18627 [40:48<7:49:05,  1.62s/it]
  7%|▋         | 1269/18627 [40:50<8:32:20,  1.77s/it]
  7%|▋         | 1270/18627 [40:51<6:59:19,  1.45s/it]
                                                      
{'loss': 1.5938, 'grad_norm': 8.788490295410156, 'learning_rate': 4.980920121603577e-06, 'epoch': 0.07}
+
  7%|▋         | 1270/18627 [40:51<6:59:19,  1.45s/it]
  7%|▋         | 1271/18627 [40:53<7:56:55,  1.65s/it]
  7%|▋         | 1272/18627 [40:55<8:38:07,  1.79s/it]
  7%|▋         | 1273/18627 [40:57<9:08:48,  1.90s/it]
  7%|▋         | 1274/18627 [40:59<9:27:22,  1.96s/it]
  7%|▋         | 1275/18627 [41:01<9:40:57,  2.01s/it]
  7%|▋         | 1276/18627 [41:03<9:52:51,  2.05s/it]
  7%|▋         | 1277/18627 [41:05<10:05:14,  2.09s/it]
  7%|▋         | 1278/18627 [41:08<10:09:28,  2.11s/it]
  7%|▋         | 1279/18627 [41:10<10:08:53,  2.11s/it]
  7%|▋         | 1280/18627 [41:12<10:09:16,  2.11s/it]
                                                       
{'loss': 0.8626, 'grad_norm': 10.579232215881348, 'learning_rate': 4.980380349895232e-06, 'epoch': 0.07}
+
  7%|▋         | 1280/18627 [41:12<10:09:16,  2.11s/it]
  7%|▋         | 1281/18627 [41:14<10:09:46,  2.11s/it]
  7%|▋         | 1282/18627 [41:16<10:12:16,  2.12s/it]
  7%|▋         | 1283/18627 [41:18<10:12:36,  2.12s/it]
  7%|▋         | 1284/18627 [41:20<10:11:58,  2.12s/it]
  7%|▋         | 1285/18627 [41:22<10:12:33,  2.12s/it]
  7%|▋         | 1286/18627 [41:23<8:10:15,  1.70s/it] 
  7%|▋         | 1287/18627 [41:25<8:47:37,  1.83s/it]
  7%|▋         | 1288/18627 [41:27<9:14:26,  1.92s/it]
  7%|▋         | 1289/18627 [41:29<8:11:29,  1.70s/it]
  7%|▋         | 1290/18627 [41:30<7:20:34,  1.52s/it]
                                                      
{'loss': 1.2095, 'grad_norm': 15.67060661315918, 'learning_rate': 4.979833079283572e-06, 'epoch': 0.07}
+
  7%|▋         | 1290/18627 [41:30<7:20:34,  1.52s/it]
  7%|▋         | 1291/18627 [41:32<8:12:01,  1.70s/it]
  7%|▋         | 1292/18627 [41:34<8:46:43,  1.82s/it]
  7%|▋         | 1293/18627 [41:36<9:11:34,  1.91s/it]
  7%|▋         | 1294/18627 [41:37<8:10:22,  1.70s/it]
  7%|▋         | 1295/18627 [41:38<6:43:21,  1.40s/it]
  7%|▋         | 1296/18627 [41:40<7:49:50,  1.63s/it]
  7%|▋         | 1297/18627 [41:41<7:11:04,  1.49s/it]
  7%|▋         | 1298/18627 [41:43<8:07:11,  1.69s/it]
  7%|▋         | 1299/18627 [41:45<7:17:01,  1.51s/it]
  7%|▋         | 1300/18627 [41:47<8:11:04,  1.70s/it]
                                                      
{'loss': 1.4065, 'grad_norm': 12.815376281738281, 'learning_rate': 4.97927831142315e-06, 'epoch': 0.07}
+
  7%|▋         | 1300/18627 [41:47<8:11:04,  1.70s/it]
  7%|▋         | 1301/18627 [41:49<8:46:33,  1.82s/it]
  7%|▋         | 1302/18627 [41:50<7:51:51,  1.63s/it]
  7%|▋         | 1303/18627 [41:52<8:34:36,  1.78s/it]
  7%|▋         | 1304/18627 [41:54<9:04:07,  1.88s/it]
  7%|▋         | 1305/18627 [41:55<8:03:30,  1.67s/it]
  7%|▋         | 1306/18627 [41:57<7:21:30,  1.53s/it]
  7%|▋         | 1307/18627 [41:59<8:13:47,  1.71s/it]
  7%|▋         | 1308/18627 [42:01<8:49:02,  1.83s/it]
  7%|▋         | 1309/18627 [42:02<7:54:01,  1.64s/it]
  7%|▋         | 1310/18627 [42:04<8:36:26,  1.79s/it]
                                                      
{'loss': 1.5561, 'grad_norm': 15.765726089477539, 'learning_rate': 4.978716047991191e-06, 'epoch': 0.07}
+
  7%|▋         | 1310/18627 [42:04<8:36:26,  1.79s/it]
  7%|▋         | 1311/18627 [42:06<9:04:56,  1.89s/it]
  7%|▋         | 1312/18627 [42:08<9:22:59,  1.95s/it]
  7%|▋         | 1313/18627 [42:11<9:37:58,  2.00s/it]
  7%|▋         | 1314/18627 [42:12<8:28:11,  1.76s/it]
  7%|▋         | 1315/18627 [42:14<8:59:20,  1.87s/it]
  7%|▋         | 1316/18627 [42:16<9:22:44,  1.95s/it]
  7%|▋         | 1317/18627 [42:18<9:37:17,  2.00s/it]
  7%|▋         | 1318/18627 [42:20<9:49:21,  2.04s/it]
  7%|▋         | 1319/18627 [42:22<10:00:33,  2.08s/it]
  7%|▋         | 1320/18627 [42:25<10:03:59,  2.09s/it]
                                                       
{'loss': 1.1205, 'grad_norm': 12.804259300231934, 'learning_rate': 4.978146290687578e-06, 'epoch': 0.07}
+
  7%|▋         | 1320/18627 [42:25<10:03:59,  2.09s/it]
  7%|▋         | 1321/18627 [42:27<10:07:10,  2.11s/it]
  7%|▋         | 1322/18627 [42:28<8:47:47,  1.83s/it] 
  7%|▋         | 1323/18627 [42:30<9:13:49,  1.92s/it]
  7%|▋         | 1324/18627 [42:32<9:31:13,  1.98s/it]
  7%|▋         | 1325/18627 [42:33<8:22:43,  1.74s/it]
  7%|▋         | 1326/18627 [42:35<8:54:03,  1.85s/it]
  7%|▋         | 1327/18627 [42:37<9:15:46,  1.93s/it]
  7%|▋         | 1328/18627 [42:40<9:33:06,  1.99s/it]
  7%|▋         | 1329/18627 [42:41<8:23:25,  1.75s/it]
  7%|▋         | 1330/18627 [42:43<8:58:15,  1.87s/it]
                                                      
{'loss': 1.4068, 'grad_norm': 11.372343063354492, 'learning_rate': 4.977569041234851e-06, 'epoch': 0.07}
+
  7%|▋         | 1330/18627 [42:43<8:58:15,  1.87s/it]
  7%|▋         | 1331/18627 [42:44<8:00:06,  1.67s/it]
  7%|▋         | 1332/18627 [42:46<8:40:59,  1.81s/it]
  7%|▋         | 1333/18627 [42:48<9:08:58,  1.90s/it]
  7%|▋         | 1334/18627 [42:50<8:07:31,  1.69s/it]
  7%|▋         | 1335/18627 [42:52<8:44:20,  1.82s/it]
  7%|▋         | 1336/18627 [42:53<7:40:32,  1.60s/it]
  7%|▋         | 1337/18627 [42:55<8:26:05,  1.76s/it]
  7%|▋         | 1338/18627 [42:57<8:58:47,  1.87s/it]
  7%|▋         | 1339/18627 [42:58<7:59:53,  1.67s/it]
  7%|▋         | 1340/18627 [43:00<8:39:33,  1.80s/it]
                                                      
{'loss': 1.5787, 'grad_norm': 10.030970573425293, 'learning_rate': 4.9769843013782016e-06, 'epoch': 0.07}
+
  7%|▋         | 1340/18627 [43:00<8:39:33,  1.80s/it]
  7%|▋         | 1341/18627 [43:02<7:45:48,  1.62s/it]
  7%|▋         | 1342/18627 [43:04<8:29:22,  1.77s/it]
  7%|▋         | 1343/18627 [43:06<8:58:34,  1.87s/it]
  7%|▋         | 1344/18627 [43:08<9:21:10,  1.95s/it]
  7%|▋         | 1345/18627 [43:10<9:35:22,  2.00s/it]
  7%|▋         | 1346/18627 [43:11<8:25:41,  1.76s/it]
  7%|▋         | 1347/18627 [43:13<8:58:07,  1.87s/it]
  7%|▋         | 1348/18627 [43:16<9:21:21,  1.95s/it]
  7%|▋         | 1349/18627 [43:17<8:16:45,  1.73s/it]
  7%|▋         | 1350/18627 [43:19<8:52:50,  1.85s/it]
                                                      
{'loss': 1.4609, 'grad_norm': 10.832488059997559, 'learning_rate': 4.9763920728854674e-06, 'epoch': 0.07}
+
  7%|▋         | 1350/18627 [43:19<8:52:50,  1.85s/it]
  7%|▋         | 1351/18627 [43:21<9:18:58,  1.94s/it]
  7%|▋         | 1352/18627 [43:23<9:38:08,  2.01s/it]
  7%|▋         | 1353/18627 [43:25<9:51:45,  2.06s/it]
  7%|▋         | 1354/18627 [43:27<9:58:01,  2.08s/it]
  7%|▋         | 1355/18627 [43:30<10:02:29,  2.09s/it]
  7%|▋         | 1356/18627 [43:32<10:02:22,  2.09s/it]
  7%|▋         | 1357/18627 [43:34<10:05:32,  2.10s/it]
  7%|▋         | 1358/18627 [43:36<10:06:24,  2.11s/it]
  7%|▋         | 1359/18627 [43:38<10:07:55,  2.11s/it]
  7%|▋         | 1360/18627 [43:40<10:08:33,  2.11s/it]
                                                       
{'loss': 0.8255, 'grad_norm': 8.727134704589844, 'learning_rate': 4.975792357547124e-06, 'epoch': 0.07}
+
  7%|▋         | 1360/18627 [43:40<10:08:33,  2.11s/it]
  7%|▋         | 1361/18627 [43:42<10:09:28,  2.12s/it]
  7%|▋         | 1362/18627 [43:44<10:09:37,  2.12s/it]
  7%|▋         | 1363/18627 [43:47<10:09:00,  2.12s/it]
  7%|▋         | 1364/18627 [43:49<10:07:44,  2.11s/it]
  7%|▋         | 1365/18627 [43:51<10:07:43,  2.11s/it]
  7%|▋         | 1366/18627 [43:53<10:08:59,  2.12s/it]
  7%|▋         | 1367/18627 [43:55<10:11:44,  2.13s/it]
  7%|▋         | 1368/18627 [43:57<10:12:28,  2.13s/it]
  7%|▋         | 1369/18627 [43:59<10:14:19,  2.14s/it]
  7%|▋         | 1370/18627 [44:01<10:11:42,  2.13s/it]
                                                       
{'loss': 0.8334, 'grad_norm': 11.516378402709961, 'learning_rate': 4.975185157176284e-06, 'epoch': 0.07}
+
  7%|▋         | 1370/18627 [44:01<10:11:42,  2.13s/it]
  7%|▋         | 1371/18627 [44:04<10:10:49,  2.12s/it]
  7%|▋         | 1372/18627 [44:06<10:11:31,  2.13s/it]
  7%|▋         | 1373/18627 [44:08<10:10:13,  2.12s/it]
  7%|▋         | 1374/18627 [44:10<10:13:08,  2.13s/it]
  7%|▋         | 1375/18627 [44:12<10:15:12,  2.14s/it]
  7%|▋         | 1376/18627 [44:14<10:14:53,  2.14s/it]
  7%|▋         | 1377/18627 [44:16<10:12:59,  2.13s/it]
  7%|▋         | 1378/18627 [44:18<10:11:13,  2.13s/it]
  7%|▋         | 1379/18627 [44:21<10:12:11,  2.13s/it]
  7%|▋         | 1380/18627 [44:23<10:12:54,  2.13s/it]
                                                       
{'loss': 0.8389, 'grad_norm': 10.055022239685059, 'learning_rate': 4.9745704736086894e-06, 'epoch': 0.07}
+
  7%|▋         | 1380/18627 [44:23<10:12:54,  2.13s/it]
  7%|▋         | 1381/18627 [44:25<10:13:05,  2.13s/it]
  7%|▋         | 1382/18627 [44:27<10:13:25,  2.13s/it]
  7%|▋         | 1383/18627 [44:29<10:13:51,  2.14s/it]
  7%|▋         | 1384/18627 [44:31<10:13:48,  2.14s/it]
  7%|▋         | 1385/18627 [44:33<10:13:15,  2.13s/it]
  7%|▋         | 1386/18627 [44:36<10:12:57,  2.13s/it]
  7%|▋         | 1387/18627 [44:38<10:14:16,  2.14s/it]
  7%|▋         | 1388/18627 [44:40<10:10:12,  2.12s/it]
  7%|▋         | 1389/18627 [44:42<10:07:35,  2.11s/it]
  7%|▋         | 1390/18627 [44:44<10:08:55,  2.12s/it]
                                                       
{'loss': 0.8668, 'grad_norm': 10.08651065826416, 'learning_rate': 4.9739483087027056e-06, 'epoch': 0.07}
+
  7%|▋         | 1390/18627 [44:44<10:08:55,  2.12s/it]
  7%|▋         | 1391/18627 [44:46<10:12:10,  2.13s/it]
  7%|▋         | 1392/18627 [44:48<10:14:43,  2.14s/it]
  7%|▋         | 1393/18627 [44:50<8:53:24,  1.86s/it] 
  7%|▋         | 1394/18627 [44:51<7:48:20,  1.63s/it]
  7%|▋         | 1395/18627 [44:53<8:31:04,  1.78s/it]
  7%|▋         | 1396/18627 [44:55<9:00:42,  1.88s/it]
  7%|▋         | 1397/18627 [44:57<9:21:01,  1.95s/it]
  8%|▊         | 1398/18627 [44:59<9:35:05,  2.00s/it]
  8%|▊         | 1399/18627 [45:01<9:45:12,  2.04s/it]
  8%|▊         | 1400/18627 [45:03<9:53:11,  2.07s/it]
                                                      
{'loss': 1.1964, 'grad_norm': 9.150856018066406, 'learning_rate': 4.973318664339314e-06, 'epoch': 0.08}
+
  8%|▊         | 1400/18627 [45:03<9:53:11,  2.07s/it]
  8%|▊         | 1401/18627 [45:06<10:00:38,  2.09s/it]
  8%|▊         | 1402/18627 [45:07<8:42:04,  1.82s/it] 
  8%|▊         | 1403/18627 [45:09<9:05:56,  1.90s/it]
  8%|▊         | 1404/18627 [45:10<8:05:00,  1.69s/it]
  8%|▊         | 1405/18627 [45:12<8:44:04,  1.83s/it]
  8%|▊         | 1406/18627 [45:14<9:10:49,  1.92s/it]
  8%|▊         | 1407/18627 [45:16<9:27:16,  1.98s/it]
  8%|▊         | 1408/18627 [45:18<9:39:04,  2.02s/it]
  8%|▊         | 1409/18627 [45:21<9:51:07,  2.06s/it]
  8%|▊         | 1410/18627 [45:23<9:56:38,  2.08s/it]
                                                      
{'loss': 1.1585, 'grad_norm': 8.913542747497559, 'learning_rate': 4.972681542422113e-06, 'epoch': 0.08}
+
  8%|▊         | 1410/18627 [45:23<9:56:38,  2.08s/it]
  8%|▊         | 1411/18627 [45:25<10:00:31,  2.09s/it]
  8%|▊         | 1412/18627 [45:27<10:02:59,  2.10s/it]
  8%|▊         | 1413/18627 [45:28<8:44:05,  1.83s/it] 
  8%|▊         | 1414/18627 [45:30<9:09:05,  1.91s/it]
  8%|▊         | 1415/18627 [45:32<9:24:33,  1.97s/it]
  8%|▊         | 1416/18627 [45:35<9:38:28,  2.02s/it]
  8%|▊         | 1417/18627 [45:37<9:47:26,  2.05s/it]
  8%|▊         | 1418/18627 [45:39<9:53:15,  2.07s/it]
  8%|▊         | 1419/18627 [45:41<9:56:03,  2.08s/it]
  8%|▊         | 1420/18627 [45:43<9:59:43,  2.09s/it]
                                                      
{'loss': 1.0935, 'grad_norm': 7.615073204040527, 'learning_rate': 4.972036944877304e-06, 'epoch': 0.08}
+
  8%|▊         | 1420/18627 [45:43<9:59:43,  2.09s/it]
  8%|▊         | 1421/18627 [45:45<10:01:58,  2.10s/it]
  8%|▊         | 1422/18627 [45:47<10:03:55,  2.11s/it]
  8%|▊         | 1423/18627 [45:49<10:06:39,  2.12s/it]
  8%|▊         | 1424/18627 [45:52<10:09:03,  2.12s/it]
  8%|▊         | 1425/18627 [45:54<10:08:21,  2.12s/it]
  8%|▊         | 1426/18627 [45:56<10:07:18,  2.12s/it]
  8%|▊         | 1427/18627 [45:58<10:08:05,  2.12s/it]
  8%|▊         | 1428/18627 [46:00<10:11:48,  2.13s/it]
  8%|▊         | 1429/18627 [46:02<10:13:05,  2.14s/it]
  8%|▊         | 1430/18627 [46:03<8:36:48,  1.80s/it] 
                                                      
{'loss': 1.0599, 'grad_norm': 15.864347457885742, 'learning_rate': 4.971384873653693e-06, 'epoch': 0.08}
+
  8%|▊         | 1430/18627 [46:03<8:36:48,  1.80s/it]
  8%|▊         | 1431/18627 [46:05<9:03:06,  1.90s/it]
  8%|▊         | 1432/18627 [46:08<9:28:16,  1.98s/it]
  8%|▊         | 1433/18627 [46:10<9:44:32,  2.04s/it]
  8%|▊         | 1434/18627 [46:12<9:54:49,  2.08s/it]
  8%|▊         | 1435/18627 [46:14<10:02:44,  2.10s/it]
  8%|▊         | 1436/18627 [46:15<8:46:11,  1.84s/it] 
  8%|▊         | 1437/18627 [46:17<9:09:58,  1.92s/it]
  8%|▊         | 1438/18627 [46:19<9:26:07,  1.98s/it]
  8%|▊         | 1439/18627 [46:22<9:37:25,  2.02s/it]
  8%|▊         | 1440/18627 [46:24<9:46:23,  2.05s/it]
                                                      
{'loss': 0.9812, 'grad_norm': 13.237309455871582, 'learning_rate': 4.970725330722676e-06, 'epoch': 0.08}
+
  8%|▊         | 1440/18627 [46:24<9:46:23,  2.05s/it]
  8%|▊         | 1441/18627 [46:26<9:53:29,  2.07s/it]
  8%|▊         | 1442/18627 [46:28<9:56:37,  2.08s/it]
  8%|▊         | 1443/18627 [46:30<9:58:56,  2.09s/it]
  8%|▊         | 1444/18627 [46:32<10:00:49,  2.10s/it]
  8%|▊         | 1445/18627 [46:33<8:44:54,  1.83s/it] 
  8%|▊         | 1446/18627 [46:35<7:49:18,  1.64s/it]
  8%|▊         | 1447/18627 [46:35<6:27:59,  1.36s/it]
  8%|▊         | 1448/18627 [46:36<6:14:23,  1.31s/it]
  8%|▊         | 1449/18627 [46:39<7:22:02,  1.54s/it]
  8%|▊         | 1450/18627 [46:41<8:09:34,  1.71s/it]
                                                      
{'loss': 1.4464, 'grad_norm': 8.938522338867188, 'learning_rate': 4.970058318078241e-06, 'epoch': 0.08}
+
  8%|▊         | 1450/18627 [46:41<8:09:34,  1.71s/it]
  8%|▊         | 1451/18627 [46:42<7:25:45,  1.56s/it]
  8%|▊         | 1452/18627 [46:44<8:13:16,  1.72s/it]
  8%|▊         | 1453/18627 [46:46<8:47:12,  1.84s/it]
  8%|▊         | 1454/18627 [46:48<9:10:55,  1.92s/it]
  8%|▊         | 1455/18627 [46:49<7:58:21,  1.67s/it]
  8%|▊         | 1456/18627 [46:51<8:38:12,  1.81s/it]
  8%|▊         | 1457/18627 [46:53<7:44:19,  1.62s/it]
  8%|▊         | 1458/18627 [46:55<8:26:38,  1.77s/it]
  8%|▊         | 1459/18627 [46:57<8:56:43,  1.88s/it]
  8%|▊         | 1460/18627 [46:59<9:20:21,  1.96s/it]
                                                      
{'loss': 1.4037, 'grad_norm': 14.562503814697266, 'learning_rate': 4.969383837736962e-06, 'epoch': 0.08}
+
  8%|▊         | 1460/18627 [46:59<9:20:21,  1.96s/it]
  8%|▊         | 1461/18627 [47:01<9:34:13,  2.01s/it]
  8%|▊         | 1462/18627 [47:03<9:44:53,  2.04s/it]
  8%|▊         | 1463/18627 [47:04<8:31:15,  1.79s/it]
  8%|▊         | 1464/18627 [47:07<9:00:53,  1.89s/it]
  8%|▊         | 1465/18627 [47:09<9:21:20,  1.96s/it]
  8%|▊         | 1466/18627 [47:10<8:02:15,  1.69s/it]
  8%|▊         | 1467/18627 [47:12<8:39:28,  1.82s/it]
  8%|▊         | 1468/18627 [47:14<9:04:17,  1.90s/it]
  8%|▊         | 1469/18627 [47:16<9:23:50,  1.97s/it]
  8%|▊         | 1470/18627 [47:18<9:39:01,  2.02s/it]
                                                      
{'loss': 1.1917, 'grad_norm': 13.009746551513672, 'learning_rate': 4.968701891737984e-06, 'epoch': 0.08}
+
  8%|▊         | 1470/18627 [47:18<9:39:01,  2.02s/it]
  8%|▊         | 1471/18627 [47:20<9:47:20,  2.05s/it]
  8%|▊         | 1472/18627 [47:22<9:53:26,  2.08s/it]
  8%|▊         | 1473/18627 [47:25<9:59:36,  2.10s/it]
  8%|▊         | 1474/18627 [47:27<10:01:14,  2.10s/it]
  8%|▊         | 1475/18627 [47:29<10:02:56,  2.11s/it]
  8%|▊         | 1476/18627 [47:31<10:04:34,  2.11s/it]
  8%|▊         | 1477/18627 [47:33<10:06:18,  2.12s/it]
  8%|▊         | 1478/18627 [47:35<10:04:05,  2.11s/it]
  8%|▊         | 1479/18627 [47:37<10:04:44,  2.12s/it]
  8%|▊         | 1480/18627 [47:39<10:05:56,  2.12s/it]
                                                       
{'loss': 0.7777, 'grad_norm': 8.079020500183105, 'learning_rate': 4.968012482143029e-06, 'epoch': 0.08}
+
  8%|▊         | 1480/18627 [47:39<10:05:56,  2.12s/it]
  8%|▊         | 1481/18627 [47:42<10:05:35,  2.12s/it]
  8%|▊         | 1482/18627 [47:44<10:03:31,  2.11s/it]
  8%|▊         | 1483/18627 [47:46<10:02:46,  2.11s/it]
  8%|▊         | 1484/18627 [47:48<10:03:01,  2.11s/it]
  8%|▊         | 1485/18627 [47:50<10:03:49,  2.11s/it]
  8%|▊         | 1486/18627 [47:52<10:03:50,  2.11s/it]
  8%|▊         | 1487/18627 [47:53<8:45:26,  1.84s/it] 
  8%|▊         | 1488/18627 [47:55<9:09:34,  1.92s/it]
  8%|▊         | 1489/18627 [47:58<9:24:54,  1.98s/it]
  8%|▊         | 1490/18627 [48:00<9:38:09,  2.02s/it]
                                                      
{'loss': 1.0018, 'grad_norm': 7.997498512268066, 'learning_rate': 4.967315611036377e-06, 'epoch': 0.08}
+
  8%|▊         | 1490/18627 [48:00<9:38:09,  2.02s/it]
  8%|▊         | 1491/18627 [48:02<9:46:36,  2.05s/it]
  8%|▊         | 1492/18627 [48:04<9:53:52,  2.08s/it]
  8%|▊         | 1493/18627 [48:05<8:37:10,  1.81s/it]
  8%|▊         | 1494/18627 [48:07<9:08:26,  1.92s/it]
  8%|▊         | 1495/18627 [48:09<9:25:25,  1.98s/it]
  8%|▊         | 1496/18627 [48:12<9:37:52,  2.02s/it]
  8%|▊         | 1497/18627 [48:13<8:26:17,  1.77s/it]
  8%|▊         | 1498/18627 [48:15<8:55:48,  1.88s/it]
  8%|▊         | 1499/18627 [48:17<9:18:59,  1.96s/it]
  8%|▊         | 1500/18627 [48:19<9:31:34,  2.00s/it]
                                                      
{'loss': 1.176, 'grad_norm': 13.039105415344238, 'learning_rate': 4.966611280524873e-06, 'epoch': 0.08}
+
  8%|▊         | 1500/18627 [48:19<9:31:34,  2.00s/it]
  8%|▊         | 1501/18627 [48:21<9:42:32,  2.04s/it]
  8%|▊         | 1502/18627 [48:23<9:49:52,  2.07s/it]
  8%|▊         | 1503/18627 [48:25<8:34:37,  1.80s/it]
  8%|▊         | 1504/18627 [48:27<9:03:09,  1.90s/it]
  8%|▊         | 1505/18627 [48:28<8:02:08,  1.69s/it]
  8%|▊         | 1506/18627 [48:30<8:40:26,  1.82s/it]
  8%|▊         | 1507/18627 [48:32<9:05:30,  1.91s/it]
  8%|▊         | 1508/18627 [48:34<9:23:47,  1.98s/it]
  8%|▊         | 1509/18627 [48:36<9:36:29,  2.02s/it]
  8%|▊         | 1510/18627 [48:38<9:44:25,  2.05s/it]
                                                      
{'loss': 1.3196, 'grad_norm': 12.74972152709961, 'learning_rate': 4.965899492737909e-06, 'epoch': 0.08}
+
  8%|▊         | 1510/18627 [48:38<9:44:25,  2.05s/it]
  8%|▊         | 1511/18627 [48:41<9:51:00,  2.07s/it]
  8%|▊         | 1512/18627 [48:43<9:54:54,  2.09s/it]
  8%|▊         | 1513/18627 [48:45<9:58:27,  2.10s/it]
  8%|▊         | 1514/18627 [48:46<8:40:14,  1.82s/it]
  8%|▊         | 1515/18627 [48:48<9:04:28,  1.91s/it]
  8%|▊         | 1516/18627 [48:50<9:25:16,  1.98s/it]
  8%|▊         | 1517/18627 [48:52<9:37:18,  2.02s/it]
  8%|▊         | 1518/18627 [48:54<8:25:55,  1.77s/it]
  8%|▊         | 1519/18627 [48:56<8:58:45,  1.89s/it]
  8%|▊         | 1520/18627 [48:58<9:22:15,  1.97s/it]
                                                      
{'loss': 1.1877, 'grad_norm': 11.960359573364258, 'learning_rate': 4.965180249827425e-06, 'epoch': 0.08}
+
  8%|▊         | 1520/18627 [48:58<9:22:15,  1.97s/it]
  8%|▊         | 1521/18627 [49:00<9:39:34,  2.03s/it]
  8%|▊         | 1522/18627 [49:02<9:47:25,  2.06s/it]
  8%|▊         | 1523/18627 [49:04<9:51:49,  2.08s/it]
  8%|▊         | 1524/18627 [49:06<8:35:55,  1.81s/it]
  8%|▊         | 1525/18627 [49:07<7:42:36,  1.62s/it]
  8%|▊         | 1526/18627 [49:09<8:24:26,  1.77s/it]
  8%|▊         | 1527/18627 [49:11<8:54:07,  1.87s/it]
  8%|▊         | 1528/18627 [49:13<9:15:43,  1.95s/it]
  8%|▊         | 1529/18627 [49:15<9:30:58,  2.00s/it]
  8%|▊         | 1530/18627 [49:16<8:22:28,  1.76s/it]
                                                      
{'loss': 1.3422, 'grad_norm': 16.979598999023438, 'learning_rate': 4.964453553967899e-06, 'epoch': 0.08}
+
  8%|▊         | 1530/18627 [49:16<8:22:28,  1.76s/it]
  8%|▊         | 1531/18627 [49:18<7:33:26,  1.59s/it]
  8%|▊         | 1532/18627 [49:20<8:18:43,  1.75s/it]
  8%|▊         | 1533/18627 [49:22<8:51:01,  1.86s/it]
  8%|▊         | 1534/18627 [49:24<9:12:58,  1.94s/it]
  8%|▊         | 1535/18627 [49:26<9:29:21,  2.00s/it]
  8%|▊         | 1536/18627 [49:28<9:39:01,  2.03s/it]
  8%|▊         | 1537/18627 [49:29<8:26:31,  1.78s/it]
  8%|▊         | 1538/18627 [49:32<8:54:26,  1.88s/it]
  8%|▊         | 1539/18627 [49:34<9:16:49,  1.96s/it]
  8%|▊         | 1540/18627 [49:36<9:29:43,  2.00s/it]
                                                      
{'loss': 1.1504, 'grad_norm': 9.746969223022461, 'learning_rate': 4.96371940735634e-06, 'epoch': 0.08}
+
  8%|▊         | 1540/18627 [49:36<9:29:43,  2.00s/it]
  8%|▊         | 1541/18627 [49:37<8:20:18,  1.76s/it]
  8%|▊         | 1542/18627 [49:38<7:30:50,  1.58s/it]
  8%|▊         | 1543/18627 [49:39<6:56:52,  1.46s/it]
  8%|▊         | 1544/18627 [49:41<7:52:31,  1.66s/it]
  8%|▊         | 1545/18627 [49:44<8:32:02,  1.80s/it]
  8%|▊         | 1546/18627 [49:46<9:01:38,  1.90s/it]
  8%|▊         | 1547/18627 [49:47<7:51:21,  1.66s/it]
  8%|▊         | 1548/18627 [49:49<8:32:36,  1.80s/it]
  8%|▊         | 1549/18627 [49:51<9:01:00,  1.90s/it]
  8%|▊         | 1550/18627 [49:52<8:00:55,  1.69s/it]
                                                      
{'loss': 1.6627, 'grad_norm': 16.20099449157715, 'learning_rate': 4.962977812212287e-06, 'epoch': 0.08}
+
  8%|▊         | 1550/18627 [49:52<8:00:55,  1.69s/it]
  8%|▊         | 1551/18627 [49:54<8:38:12,  1.82s/it]
  8%|▊         | 1552/18627 [49:56<9:02:03,  1.90s/it]
  8%|▊         | 1553/18627 [49:59<9:19:30,  1.97s/it]
  8%|▊         | 1554/18627 [50:01<9:32:40,  2.01s/it]
  8%|▊         | 1555/18627 [50:03<9:41:54,  2.05s/it]
  8%|▊         | 1556/18627 [50:05<9:47:21,  2.06s/it]
  8%|▊         | 1557/18627 [50:07<9:51:57,  2.08s/it]
  8%|▊         | 1558/18627 [50:09<9:56:30,  2.10s/it]
  8%|▊         | 1559/18627 [50:11<9:56:57,  2.10s/it]
  8%|▊         | 1560/18627 [50:12<7:57:26,  1.68s/it]
                                                      
{'loss': 0.794, 'grad_norm': 10.558563232421875, 'learning_rate': 4.962228770777795e-06, 'epoch': 0.08}
+
  8%|▊         | 1560/18627 [50:12<7:57:26,  1.68s/it]
  8%|▊         | 1561/18627 [50:13<6:35:35,  1.39s/it]
  8%|▊         | 1562/18627 [50:15<7:36:37,  1.61s/it]
  8%|▊         | 1563/18627 [50:17<8:22:28,  1.77s/it]
  8%|▊         | 1564/18627 [50:18<7:33:18,  1.59s/it]
  8%|▊         | 1565/18627 [50:20<8:17:16,  1.75s/it]
  8%|▊         | 1566/18627 [50:22<8:48:37,  1.86s/it]
  8%|▊         | 1567/18627 [50:24<7:52:45,  1.66s/it]
  8%|▊         | 1568/18627 [50:26<8:32:01,  1.80s/it]
  8%|▊         | 1569/18627 [50:28<9:00:28,  1.90s/it]
  8%|▊         | 1570/18627 [50:30<9:20:25,  1.97s/it]
                                                      
{'loss': 1.1804, 'grad_norm': 9.463888168334961, 'learning_rate': 4.96147228531743e-06, 'epoch': 0.08}
+
  8%|▊         | 1570/18627 [50:30<9:20:25,  1.97s/it]
  8%|▊         | 1571/18627 [50:32<9:32:18,  2.01s/it]
  8%|▊         | 1572/18627 [50:33<8:21:52,  1.77s/it]
  8%|▊         | 1573/18627 [50:35<8:51:02,  1.87s/it]
  8%|▊         | 1574/18627 [50:37<9:11:11,  1.94s/it]
  8%|▊         | 1575/18627 [50:39<8:06:48,  1.71s/it]
  8%|▊         | 1576/18627 [50:41<8:41:57,  1.84s/it]
  8%|▊         | 1577/18627 [50:42<7:46:33,  1.64s/it]
  8%|▊         | 1578/18627 [50:44<8:27:35,  1.79s/it]
  8%|▊         | 1579/18627 [50:46<8:56:18,  1.89s/it]
  8%|▊         | 1580/18627 [50:48<9:15:20,  1.95s/it]
                                                      
{'loss': 1.5006, 'grad_norm': 19.106138229370117, 'learning_rate': 4.960708358118267e-06, 'epoch': 0.08}
+
  8%|▊         | 1580/18627 [50:48<9:15:20,  1.95s/it]
  8%|▊         | 1581/18627 [50:50<9:28:19,  2.00s/it]
  8%|▊         | 1582/18627 [50:53<9:37:43,  2.03s/it]
  8%|▊         | 1583/18627 [50:53<7:43:44,  1.63s/it]
  9%|▊         | 1584/18627 [50:55<8:25:05,  1.78s/it]
  9%|▊         | 1585/18627 [50:57<8:52:43,  1.88s/it]
  9%|▊         | 1586/18627 [50:59<7:49:58,  1.65s/it]
  9%|▊         | 1587/18627 [51:01<8:29:48,  1.80s/it]
  9%|▊         | 1588/18627 [51:03<8:56:28,  1.89s/it]
  9%|▊         | 1589/18627 [51:05<9:17:34,  1.96s/it]
  9%|▊         | 1590/18627 [51:07<9:30:38,  2.01s/it]
                                                      
{'loss': 1.079, 'grad_norm': 9.386693000793457, 'learning_rate': 4.959936991489876e-06, 'epoch': 0.09}
+
  9%|▊         | 1590/18627 [51:07<9:30:38,  2.01s/it]
  9%|▊         | 1591/18627 [51:09<9:40:55,  2.05s/it]
  9%|▊         | 1592/18627 [51:11<9:48:24,  2.07s/it]
  9%|▊         | 1593/18627 [51:13<9:50:28,  2.08s/it]
  9%|▊         | 1594/18627 [51:16<9:53:04,  2.09s/it]
  9%|▊         | 1595/18627 [51:18<9:53:33,  2.09s/it]
  9%|▊         | 1596/18627 [51:20<9:55:34,  2.10s/it]
  9%|▊         | 1597/18627 [51:22<9:58:02,  2.11s/it]
  9%|▊         | 1598/18627 [51:23<8:41:03,  1.84s/it]
  9%|▊         | 1599/18627 [51:24<7:04:49,  1.50s/it]
  9%|▊         | 1600/18627 [51:25<6:31:47,  1.38s/it]
                                                      
{'loss': 1.2437, 'grad_norm': 15.190378189086914, 'learning_rate': 4.959158187764323e-06, 'epoch': 0.09}
+
  9%|▊         | 1600/18627 [51:25<6:31:47,  1.38s/it]
  9%|▊         | 1601/18627 [51:26<6:15:09,  1.32s/it]
  9%|▊         | 1602/18627 [51:28<7:22:26,  1.56s/it]
  9%|▊         | 1603/18627 [51:30<8:09:57,  1.73s/it]
  9%|▊         | 1604/18627 [51:32<7:24:06,  1.57s/it]
  9%|▊         | 1605/18627 [51:34<8:09:48,  1.73s/it]
  9%|▊         | 1606/18627 [51:35<7:23:49,  1.56s/it]
  9%|▊         | 1607/18627 [51:37<8:09:33,  1.73s/it]
  9%|▊         | 1608/18627 [51:39<8:44:27,  1.85s/it]
  9%|▊         | 1609/18627 [51:41<9:28:11,  2.00s/it]
  9%|▊         | 1610/18627 [51:44<9:38:01,  2.04s/it]
                                                      
{'loss': 1.346, 'grad_norm': 9.169631004333496, 'learning_rate': 4.958371949296153e-06, 'epoch': 0.09}
+
  9%|▊         | 1610/18627 [51:44<9:38:01,  2.04s/it]
  9%|▊         | 1611/18627 [51:45<8:25:47,  1.78s/it]
  9%|▊         | 1612/18627 [51:46<7:35:16,  1.61s/it]
  9%|▊         | 1613/18627 [51:47<6:45:55,  1.43s/it]
  9%|▊         | 1614/18627 [51:48<6:25:06,  1.36s/it]
  9%|▊         | 1615/18627 [51:50<7:28:32,  1.58s/it]
  9%|▊         | 1616/18627 [51:51<6:56:11,  1.47s/it]
  9%|▊         | 1617/18627 [51:54<7:56:34,  1.68s/it]
  9%|▊         | 1618/18627 [51:55<7:08:32,  1.51s/it]
  9%|▊         | 1619/18627 [51:56<6:25:40,  1.36s/it]
  9%|▊         | 1620/18627 [51:58<7:29:34,  1.59s/it]
                                                      
{'loss': 2.0485, 'grad_norm': 9.272505760192871, 'learning_rate': 4.95757827846239e-06, 'epoch': 0.09}
+
  9%|▊         | 1620/18627 [51:58<7:29:34,  1.59s/it]
  9%|▊         | 1621/18627 [52:00<8:14:47,  1.75s/it]
  9%|▊         | 1622/18627 [52:02<8:48:34,  1.87s/it]
  9%|▊         | 1623/18627 [52:04<9:13:32,  1.95s/it]
  9%|▊         | 1624/18627 [52:06<9:31:30,  2.02s/it]
  9%|▊         | 1625/18627 [52:08<8:20:54,  1.77s/it]
  9%|▊         | 1626/18627 [52:10<8:50:16,  1.87s/it]
  9%|▊         | 1627/18627 [52:12<9:11:10,  1.95s/it]
  9%|▊         | 1628/18627 [52:14<9:26:12,  2.00s/it]
  9%|▊         | 1629/18627 [52:15<8:17:05,  1.75s/it]
  9%|▉         | 1630/18627 [52:16<7:29:44,  1.59s/it]
                                                      
{'loss': 1.4599, 'grad_norm': 19.823707580566406, 'learning_rate': 4.956777177662533e-06, 'epoch': 0.09}
+
  9%|▉         | 1630/18627 [52:16<7:29:44,  1.59s/it]
  9%|▉         | 1631/18627 [52:17<6:41:01,  1.42s/it]
  9%|▉         | 1632/18627 [52:20<7:41:23,  1.63s/it]
  9%|▉         | 1633/18627 [52:22<8:24:10,  1.78s/it]
  9%|▉         | 1634/18627 [52:23<7:33:41,  1.60s/it]
  9%|▉         | 1635/18627 [52:25<8:18:26,  1.76s/it]
  9%|▉         | 1636/18627 [52:26<7:20:44,  1.56s/it]
  9%|▉         | 1637/18627 [52:27<6:49:12,  1.45s/it]
  9%|▉         | 1638/18627 [52:29<7:44:45,  1.64s/it]
  9%|▉         | 1639/18627 [52:31<7:05:49,  1.50s/it]
  9%|▉         | 1640/18627 [52:32<6:28:29,  1.37s/it]
                                                      
{'loss': 1.8431, 'grad_norm': 17.77994728088379, 'learning_rate': 4.955968649318539e-06, 'epoch': 0.09}
+
  9%|▉         | 1640/18627 [52:32<6:28:29,  1.37s/it]
  9%|▉         | 1641/18627 [52:33<6:05:31,  1.29s/it]
  9%|▉         | 1642/18627 [52:35<7:15:22,  1.54s/it]
  9%|▉         | 1643/18627 [52:37<8:05:30,  1.72s/it]
  9%|▉         | 1644/18627 [52:39<8:40:41,  1.84s/it]
  9%|▉         | 1645/18627 [52:41<9:02:00,  1.91s/it]
  9%|▉         | 1646/18627 [52:43<9:18:39,  1.97s/it]
  9%|▉         | 1647/18627 [52:44<8:11:52,  1.74s/it]
  9%|▉         | 1648/18627 [52:47<8:44:06,  1.85s/it]
  9%|▉         | 1649/18627 [52:49<9:06:25,  1.93s/it]
  9%|▉         | 1650/18627 [52:50<8:03:54,  1.71s/it]
                                                      
{'loss': 1.3929, 'grad_norm': 19.136898040771484, 'learning_rate': 4.9551526958748195e-06, 'epoch': 0.09}
+
  9%|▉         | 1650/18627 [52:50<8:03:54,  1.71s/it]
  9%|▉         | 1651/18627 [52:51<7:19:21,  1.55s/it]
  9%|▉         | 1652/18627 [52:53<8:10:02,  1.73s/it]
  9%|▉         | 1653/18627 [52:54<7:24:51,  1.57s/it]
  9%|▉         | 1654/18627 [52:57<8:11:17,  1.74s/it]
  9%|▉         | 1655/18627 [52:58<7:24:20,  1.57s/it]
  9%|▉         | 1656/18627 [53:00<8:13:21,  1.74s/it]
  9%|▉         | 1657/18627 [53:02<8:43:33,  1.85s/it]
  9%|��         | 1658/18627 [53:04<9:07:20,  1.94s/it]
  9%|▉         | 1659/18627 [53:06<9:21:14,  1.98s/it]
  9%|▉         | 1660/18627 [53:07<8:13:25,  1.74s/it]
                                                      
{'loss': 1.4569, 'grad_norm': 15.444260597229004, 'learning_rate': 4.954329319798239e-06, 'epoch': 0.09}
+
  9%|▉         | 1660/18627 [53:07<8:13:25,  1.74s/it]
  9%|▉         | 1661/18627 [53:09<8:43:28,  1.85s/it]
  9%|▉         | 1662/18627 [53:12<9:04:48,  1.93s/it]
  9%|▉         | 1663/18627 [53:14<9:21:07,  1.98s/it]
  9%|▉         | 1664/18627 [53:16<9:30:00,  2.02s/it]
  9%|▉         | 1665/18627 [53:18<9:38:21,  2.05s/it]
  9%|▉         | 1666/18627 [53:19<8:26:41,  1.79s/it]
  9%|▉         | 1667/18627 [53:21<8:53:04,  1.89s/it]
  9%|▉         | 1668/18627 [53:22<7:54:13,  1.68s/it]
  9%|▉         | 1669/18627 [53:24<7:12:29,  1.53s/it]
  9%|▉         | 1670/18627 [53:25<6:43:38,  1.43s/it]
                                                      
{'loss': 1.5632, 'grad_norm': 16.38795280456543, 'learning_rate': 4.953498523578098e-06, 'epoch': 0.09}
+
  9%|▉         | 1670/18627 [53:25<6:43:38,  1.43s/it]
  9%|▉         | 1671/18627 [53:26<5:50:12,  1.24s/it]
  9%|▉         | 1672/18627 [53:28<7:07:23,  1.51s/it]
  9%|▉         | 1673/18627 [53:30<7:58:44,  1.69s/it]
  9%|▉         | 1674/18627 [53:32<8:33:47,  1.82s/it]
  9%|▉         | 1675/18627 [53:33<7:40:38,  1.63s/it]
  9%|▉         | 1676/18627 [53:35<8:20:46,  1.77s/it]
  9%|▉         | 1677/18627 [53:37<8:50:01,  1.88s/it]
  9%|▉         | 1678/18627 [53:38<7:11:20,  1.53s/it]
  9%|▉         | 1679/18627 [53:40<8:05:22,  1.72s/it]
  9%|▉         | 1680/18627 [53:42<8:42:41,  1.85s/it]
                                                      
{'loss': 1.1102, 'grad_norm': 8.564276695251465, 'learning_rate': 4.952660309726135e-06, 'epoch': 0.09}
+
  9%|▉         | 1680/18627 [53:42<8:42:41,  1.85s/it]
  9%|▉         | 1681/18627 [53:45<9:09:26,  1.95s/it]
  9%|▉         | 1682/18627 [53:47<9:28:05,  2.01s/it]
  9%|▉         | 1683/18627 [53:49<9:41:23,  2.06s/it]
  9%|▉         | 1684/18627 [53:51<9:50:49,  2.09s/it]
  9%|▉         | 1685/18627 [53:53<9:56:44,  2.11s/it]
  9%|▉         | 1686/18627 [53:55<9:56:49,  2.11s/it]
  9%|▉         | 1687/18627 [53:57<8:38:44,  1.84s/it]
  9%|▉         | 1688/18627 [53:59<9:02:14,  1.92s/it]
  9%|▉         | 1689/18627 [54:01<9:17:11,  1.97s/it]
  9%|▉         | 1690/18627 [54:03<9:28:37,  2.01s/it]
                                                      
{'loss': 0.9917, 'grad_norm': 8.032626152038574, 'learning_rate': 4.95181468077651e-06, 'epoch': 0.09}
+
  9%|▉         | 1690/18627 [54:03<9:28:37,  2.01s/it]
  9%|▉         | 1691/18627 [54:04<8:03:54,  1.71s/it]
  9%|▉         | 1692/18627 [54:06<8:39:29,  1.84s/it]
  9%|▉         | 1693/18627 [54:08<9:03:57,  1.93s/it]
  9%|▉         | 1694/18627 [54:10<9:19:03,  1.98s/it]
  9%|▉         | 1695/18627 [54:12<9:30:04,  2.02s/it]
  9%|▉         | 1696/18627 [54:14<8:19:34,  1.77s/it]
  9%|▉         | 1697/18627 [54:16<8:49:16,  1.88s/it]
  9%|▉         | 1698/18627 [54:18<9:08:56,  1.95s/it]
  9%|▉         | 1699/18627 [54:20<9:23:55,  2.00s/it]
  9%|▉         | 1700/18627 [54:22<9:34:11,  2.04s/it]
                                                      
{'loss': 1.1924, 'grad_norm': 13.140206336975098, 'learning_rate': 4.950961639285803e-06, 'epoch': 0.09}
+
  9%|▉         | 1700/18627 [54:22<9:34:11,  2.04s/it]
  9%|▉         | 1701/18627 [54:24<9:39:27,  2.05s/it]
  9%|▉         | 1702/18627 [54:26<9:43:52,  2.07s/it]
  9%|▉         | 1703/18627 [54:28<9:48:56,  2.09s/it]
  9%|▉         | 1704/18627 [54:30<9:50:55,  2.10s/it]
  9%|▉         | 1705/18627 [54:33<9:53:27,  2.10s/it]
  9%|▉         | 1706/18627 [54:35<9:53:27,  2.10s/it]
  9%|▉         | 1707/18627 [54:36<8:35:53,  1.83s/it]
  9%|▉         | 1708/18627 [54:37<7:42:12,  1.64s/it]
  9%|▉         | 1709/18627 [54:39<8:24:35,  1.79s/it]
  9%|▉         | 1710/18627 [54:41<8:52:15,  1.89s/it]
                                                      
{'loss': 1.1445, 'grad_norm': 10.744111061096191, 'learning_rate': 4.950101187833004e-06, 'epoch': 0.09}
+
  9%|▉         | 1710/18627 [54:41<8:52:15,  1.89s/it]
  9%|▉         | 1711/18627 [54:43<7:54:01,  1.68s/it]
  9%|▉         | 1712/18627 [54:44<7:13:28,  1.54s/it]
  9%|▉         | 1713/18627 [54:46<8:02:27,  1.71s/it]
  9%|▉         | 1714/18627 [54:48<8:39:57,  1.84s/it]
  9%|▉         | 1715/18627 [54:49<7:45:11,  1.65s/it]
  9%|▉         | 1716/18627 [54:51<8:24:39,  1.79s/it]
  9%|▉         | 1717/18627 [54:53<8:51:12,  1.88s/it]
  9%|▉         | 1718/18627 [54:56<9:09:33,  1.95s/it]
  9%|▉         | 1719/18627 [54:58<9:24:40,  2.00s/it]
  9%|▉         | 1720/18627 [55:00<9:35:56,  2.04s/it]
                                                      
{'loss': 1.3699, 'grad_norm': 9.438612937927246, 'learning_rate': 4.949233329019505e-06, 'epoch': 0.09}
+
  9%|▉         | 1720/18627 [55:00<9:35:56,  2.04s/it]
  9%|▉         | 1721/18627 [55:01<8:25:11,  1.79s/it]
  9%|▉         | 1722/18627 [55:03<8:51:54,  1.89s/it]
  9%|▉         | 1723/18627 [55:04<7:52:54,  1.68s/it]
  9%|▉         | 1724/18627 [55:06<8:28:16,  1.80s/it]
  9%|▉         | 1725/18627 [55:08<7:36:42,  1.62s/it]
  9%|▉         | 1726/18627 [55:10<8:17:52,  1.77s/it]
  9%|▉         | 1727/18627 [55:11<7:28:57,  1.59s/it]
  9%|▉         | 1728/18627 [55:13<8:12:20,  1.75s/it]
  9%|▉         | 1729/18627 [55:15<8:46:29,  1.87s/it]
  9%|▉         | 1730/18627 [55:17<9:08:12,  1.95s/it]
                                                      
{'loss': 1.4553, 'grad_norm': 9.144245147705078, 'learning_rate': 4.9483580654690925e-06, 'epoch': 0.09}
+
  9%|▉         | 1730/18627 [55:17<9:08:12,  1.95s/it]
  9%|▉         | 1731/18627 [55:18<8:03:38,  1.72s/it]
  9%|▉         | 1732/18627 [55:21<8:37:17,  1.84s/it]
  9%|▉         | 1733/18627 [55:22<7:43:16,  1.65s/it]
  9%|▉         | 1734/18627 [55:24<8:22:05,  1.78s/it]
  9%|▉         | 1735/18627 [55:26<8:49:07,  1.88s/it]
  9%|▉         | 1736/18627 [55:28<9:07:52,  1.95s/it]
  9%|▉         | 1737/18627 [55:30<9:22:52,  2.00s/it]
  9%|▉         | 1738/18627 [55:31<7:44:52,  1.65s/it]
  9%|▉         | 1739/18627 [55:33<8:24:35,  1.79s/it]
  9%|▉         | 1740/18627 [55:35<8:54:54,  1.90s/it]
                                                      
{'loss': 1.1819, 'grad_norm': 10.721570014953613, 'learning_rate': 4.947475399827941e-06, 'epoch': 0.09}
+
  9%|▉         | 1740/18627 [55:35<8:54:54,  1.90s/it]
  9%|▉         | 1741/18627 [55:36<7:15:21,  1.55s/it]
  9%|▉         | 1742/18627 [55:37<6:37:05,  1.41s/it]
  9%|▉         | 1743/18627 [55:38<6:19:50,  1.35s/it]
  9%|▉         | 1744/18627 [55:39<5:26:03,  1.16s/it]
  9%|▉         | 1745/18627 [55:40<5:28:29,  1.17s/it]
  9%|▉         | 1746/18627 [55:41<5:30:17,  1.17s/it]
  9%|▉         | 1747/18627 [55:44<6:50:09,  1.46s/it]
  9%|▉         | 1748/18627 [55:46<7:44:32,  1.65s/it]
  9%|▉         | 1749/18627 [55:48<8:24:12,  1.79s/it]
  9%|▉         | 1750/18627 [55:50<8:55:02,  1.90s/it]
                                                      
{'loss': 1.5456, 'grad_norm': 8.538431167602539, 'learning_rate': 4.946585334764601e-06, 'epoch': 0.09}
+
  9%|▉         | 1750/18627 [55:50<8:55:02,  1.90s/it]
  9%|▉         | 1751/18627 [55:51<7:55:17,  1.69s/it]
  9%|▉         | 1752/18627 [55:52<7:04:08,  1.51s/it]
  9%|▉         | 1753/18627 [55:53<6:37:00,  1.41s/it]
  9%|▉         | 1754/18627 [55:55<6:17:58,  1.34s/it]
  9%|▉         | 1755/18627 [55:56<6:05:56,  1.30s/it]
  9%|▉         | 1756/18627 [55:57<5:56:05,  1.27s/it]
  9%|▉         | 1757/18627 [55:59<7:08:18,  1.52s/it]
  9%|▉         | 1758/18627 [56:01<7:59:58,  1.71s/it]
  9%|▉         | 1759/18627 [56:03<8:35:45,  1.83s/it]
  9%|▉         | 1760/18627 [56:05<8:58:12,  1.91s/it]
                                                      
{'loss': 1.8654, 'grad_norm': 8.384821891784668, 'learning_rate': 4.945687872969997e-06, 'epoch': 0.09}
+
  9%|▉         | 1760/18627 [56:05<8:58:12,  1.91s/it]
  9%|▉         | 1761/18627 [56:08<9:16:48,  1.98s/it]
  9%|▉         | 1762/18627 [56:10<9:27:04,  2.02s/it]
  9%|▉         | 1763/18627 [56:12<9:36:07,  2.05s/it]
  9%|▉         | 1764/18627 [56:14<9:43:24,  2.08s/it]
  9%|▉         | 1765/18627 [56:16<9:46:34,  2.09s/it]
  9%|▉         | 1766/18627 [56:17<7:49:56,  1.67s/it]
  9%|▉         | 1767/18627 [56:19<8:27:49,  1.81s/it]
  9%|▉         | 1768/18627 [56:21<8:53:16,  1.90s/it]
  9%|▉         | 1769/18627 [56:23<9:14:44,  1.97s/it]
 10%|▉         | 1770/18627 [56:25<9:26:24,  2.02s/it]
                                                      
{'loss': 0.7645, 'grad_norm': 10.373551368713379, 'learning_rate': 4.944783017157413e-06, 'epoch': 0.1}
+
 10%|▉         | 1770/18627 [56:25<9:26:24,  2.02s/it]
 10%|▉         | 1771/18627 [56:27<9:34:11,  2.04s/it]
 10%|▉         | 1772/18627 [56:30<9:40:27,  2.07s/it]
 10%|▉         | 1773/18627 [56:31<8:27:17,  1.81s/it]
 10%|▉         | 1774/18627 [56:33<8:54:01,  1.90s/it]
 10%|▉         | 1775/18627 [56:35<9:12:06,  1.97s/it]
 10%|▉         | 1776/18627 [56:37<9:27:57,  2.02s/it]
 10%|▉         | 1777/18627 [56:39<9:35:22,  2.05s/it]
 10%|▉         | 1778/18627 [56:40<8:23:18,  1.79s/it]
 10%|▉         | 1779/18627 [56:43<8:50:17,  1.89s/it]
 10%|▉         | 1780/18627 [56:45<9:10:00,  1.96s/it]
                                                      
{'loss': 1.2206, 'grad_norm': 10.283239364624023, 'learning_rate': 4.943870770062489e-06, 'epoch': 0.1}
+
 10%|▉         | 1780/18627 [56:45<9:10:00,  1.96s/it]
 10%|▉         | 1781/18627 [56:47<9:22:39,  2.00s/it]
 10%|▉         | 1782/18627 [56:49<9:31:03,  2.03s/it]
 10%|▉         | 1783/18627 [56:50<8:19:51,  1.78s/it]
 10%|▉         | 1784/18627 [56:52<8:48:57,  1.88s/it]
 10%|▉         | 1785/18627 [56:53<7:50:14,  1.68s/it]
 10%|▉         | 1786/18627 [56:56<8:28:35,  1.81s/it]
 10%|▉         | 1787/18627 [56:58<8:54:33,  1.90s/it]
 10%|▉         | 1788/18627 [57:00<9:13:30,  1.97s/it]
 10%|▉         | 1789/18627 [57:02<9:24:49,  2.01s/it]
 10%|▉         | 1790/18627 [57:04<9:33:18,  2.04s/it]
                                                      
{'loss': 1.1307, 'grad_norm': 10.200223922729492, 'learning_rate': 4.94295113444321e-06, 'epoch': 0.1}
+
 10%|▉         | 1790/18627 [57:04<9:33:18,  2.04s/it]
 10%|▉         | 1791/18627 [57:05<8:20:58,  1.79s/it]
 10%|▉         | 1792/18627 [57:06<7:31:07,  1.61s/it]
 10%|▉         | 1793/18627 [57:08<8:15:12,  1.77s/it]
 10%|▉         | 1794/18627 [57:11<8:46:47,  1.88s/it]
 10%|▉         | 1795/18627 [57:13<9:07:04,  1.95s/it]
 10%|▉         | 1796/18627 [57:14<8:03:57,  1.73s/it]
 10%|▉         | 1797/18627 [57:15<7:19:08,  1.57s/it]
 10%|▉         | 1798/18627 [57:17<8:06:26,  1.73s/it]
 10%|▉         | 1799/18627 [57:18<7:21:16,  1.57s/it]
 10%|▉         | 1800/18627 [57:21<8:04:47,  1.73s/it]
                                                      
{'loss': 1.8022, 'grad_norm': 10.227203369140625, 'learning_rate': 4.9420241130799e-06, 'epoch': 0.1}
+
 10%|▉         | 1800/18627 [57:21<8:04:47,  1.73s/it]
 10%|▉         | 1801/18627 [57:23<8:37:46,  1.85s/it]
 10%|▉         | 1802/18627 [57:25<9:00:42,  1.93s/it]
 10%|▉         | 1803/18627 [57:26<7:59:03,  1.71s/it]
 10%|▉         | 1804/18627 [57:27<7:16:42,  1.56s/it]
 10%|▉         | 1805/18627 [57:29<8:02:50,  1.72s/it]
 10%|▉         | 1806/18627 [57:31<8:33:58,  1.83s/it]
 10%|▉         | 1807/18627 [57:34<8:58:41,  1.92s/it]
 10%|▉         | 1808/18627 [57:36<9:14:10,  1.98s/it]
 10%|▉         | 1809/18627 [57:38<9:26:39,  2.02s/it]
 10%|▉         | 1810/18627 [57:38<7:35:10,  1.62s/it]
                                                      
{'loss': 1.2727, 'grad_norm': 10.653651237487793, 'learning_rate': 4.94108970877521e-06, 'epoch': 0.1}
+
 10%|▉         | 1810/18627 [57:38<7:35:10,  1.62s/it]
 10%|▉         | 1811/18627 [57:41<8:17:39,  1.78s/it]
 10%|▉         | 1812/18627 [57:43<8:47:09,  1.88s/it]
 10%|▉         | 1813/18627 [57:45<9:06:40,  1.95s/it]
 10%|▉         | 1814/18627 [57:47<9:21:28,  2.00s/it]
 10%|▉         | 1815/18627 [57:49<9:29:23,  2.03s/it]
 10%|▉         | 1816/18627 [57:51<9:36:31,  2.06s/it]
 10%|▉         | 1817/18627 [57:53<9:42:00,  2.08s/it]
 10%|▉         | 1818/18627 [57:55<9:44:32,  2.09s/it]
 10%|▉         | 1819/18627 [57:58<9:47:29,  2.10s/it]
 10%|▉         | 1820/18627 [58:00<9:49:40,  2.11s/it]
                                                      
{'loss': 0.7975, 'grad_norm': 10.678781509399414, 'learning_rate': 4.940147924354112e-06, 'epoch': 0.1}
+
 10%|▉         | 1820/18627 [58:00<9:49:40,  2.11s/it]
 10%|▉         | 1821/18627 [58:02<9:54:08,  2.12s/it]
 10%|▉         | 1822/18627 [58:03<8:37:55,  1.85s/it]
 10%|▉         | 1823/18627 [58:05<9:04:11,  1.94s/it]
 10%|▉         | 1824/18627 [58:07<9:20:30,  2.00s/it]
 10%|▉         | 1825/18627 [58:09<9:31:47,  2.04s/it]
 10%|▉         | 1826/18627 [58:12<9:38:08,  2.06s/it]
 10%|▉         | 1827/18627 [58:14<9:41:52,  2.08s/it]
 10%|▉         | 1828/18627 [58:16<9:45:19,  2.09s/it]
 10%|▉         | 1829/18627 [58:18<9:46:39,  2.10s/it]
 10%|▉         | 1830/18627 [58:20<9:48:29,  2.10s/it]
                                                      
{'loss': 1.0851, 'grad_norm': 9.054078102111816, 'learning_rate': 4.939198762663895e-06, 'epoch': 0.1}
+
 10%|▉         | 1830/18627 [58:20<9:48:29,  2.10s/it]
 10%|▉         | 1831/18627 [58:22<9:51:25,  2.11s/it]
 10%|▉         | 1832/18627 [58:23<7:53:22,  1.69s/it]
 10%|▉         | 1833/18627 [58:25<8:33:41,  1.84s/it]
 10%|▉         | 1834/18627 [58:27<8:55:54,  1.91s/it]
 10%|▉         | 1835/18627 [58:29<9:11:22,  1.97s/it]
 10%|▉         | 1836/18627 [58:30<8:05:56,  1.74s/it]
 10%|▉         | 1837/18627 [58:33<8:36:07,  1.84s/it]
 10%|▉         | 1838/18627 [58:35<9:00:06,  1.93s/it]
 10%|▉         | 1839/18627 [58:37<9:15:44,  1.99s/it]
 10%|▉         | 1840/18627 [58:39<9:26:26,  2.02s/it]
                                                      
{'loss': 1.0273, 'grad_norm': 8.074854850769043, 'learning_rate': 4.938242226574144e-06, 'epoch': 0.1}
+
 10%|▉         | 1840/18627 [58:39<9:26:26,  2.02s/it]
 10%|▉         | 1841/18627 [58:40<8:16:26,  1.77s/it]
 10%|▉         | 1842/18627 [58:42<8:43:48,  1.87s/it]
 10%|▉         | 1843/18627 [58:44<9:05:11,  1.95s/it]
 10%|▉         | 1844/18627 [58:46<8:02:14,  1.72s/it]
 10%|▉         | 1845/18627 [58:48<8:41:12,  1.86s/it]
 10%|▉         | 1846/18627 [58:50<9:04:35,  1.95s/it]
 10%|▉         | 1847/18627 [58:52<9:18:40,  2.00s/it]
 10%|▉         | 1848/18627 [58:54<9:29:45,  2.04s/it]
 10%|▉         | 1849/18627 [58:56<9:35:08,  2.06s/it]
 10%|▉         | 1850/18627 [58:58<9:40:43,  2.08s/it]
                                                      
{'loss': 1.1771, 'grad_norm': 9.461112976074219, 'learning_rate': 4.9372783189767455e-06, 'epoch': 0.1}
+
 10%|▉         | 1850/18627 [58:58<9:40:43,  2.08s/it]
 10%|▉         | 1851/18627 [59:00<9:44:43,  2.09s/it]
 10%|▉         | 1852/18627 [59:03<9:47:57,  2.10s/it]
 10%|▉         | 1853/18627 [59:05<9:51:42,  2.12s/it]
 10%|▉         | 1854/18627 [59:07<9:50:44,  2.11s/it]
 10%|▉         | 1855/18627 [59:08<8:28:21,  1.82s/it]
 10%|▉         | 1856/18627 [59:10<8:53:45,  1.91s/it]
 10%|▉         | 1857/18627 [59:12<9:10:31,  1.97s/it]
 10%|▉         | 1858/18627 [59:14<9:21:11,  2.01s/it]
 10%|▉         | 1859/18627 [59:16<9:31:40,  2.05s/it]
 10%|▉         | 1860/18627 [59:19<9:38:46,  2.07s/it]
                                                      
{'loss': 0.9638, 'grad_norm': 10.421544075012207, 'learning_rate': 4.936307042785867e-06, 'epoch': 0.1}
+
 10%|▉         | 1860/18627 [59:19<9:38:46,  2.07s/it]
 10%|▉         | 1861/18627 [59:19<7:43:45,  1.66s/it]
 10%|▉         | 1862/18627 [59:21<8:24:31,  1.81s/it]
 10%|█         | 1863/18627 [59:23<8:49:55,  1.90s/it]
 10%|█         | 1864/18627 [59:25<7:50:40,  1.68s/it]
 10%|█         | 1865/18627 [59:27<8:26:27,  1.81s/it]
 10%|█         | 1866/18627 [59:29<8:51:18,  1.90s/it]
 10%|█         | 1867/18627 [59:31<9:09:35,  1.97s/it]
 10%|█         | 1868/18627 [59:33<9:23:10,  2.02s/it]
 10%|█         | 1869/18627 [59:35<9:31:08,  2.04s/it]
 10%|█         | 1870/18627 [59:37<9:37:51,  2.07s/it]
                                                      
{'loss': 1.0195, 'grad_norm': 8.310418128967285, 'learning_rate': 4.935328400937959e-06, 'epoch': 0.1}
+
 10%|█         | 1870/18627 [59:37<9:37:51,  2.07s/it]
 10%|█         | 1871/18627 [59:40<9:44:22,  2.09s/it]
 10%|█         | 1872/18627 [59:41<8:29:49,  1.83s/it]
 10%|█         | 1873/18627 [59:43<8:56:08,  1.92s/it]
 10%|█         | 1874/18627 [59:45<9:14:47,  1.99s/it]
 10%|█         | 1875/18627 [59:46<7:26:01,  1.60s/it]
 10%|█         | 1876/18627 [59:48<8:08:13,  1.75s/it]
 10%|█         | 1877/18627 [59:50<8:38:24,  1.86s/it]
 10%|█         | 1878/18627 [59:52<8:58:32,  1.93s/it]
 10%|█         | 1879/18627 [59:53<7:17:12,  1.57s/it]
 10%|█         | 1880/18627 [59:55<8:01:41,  1.73s/it]
                                                      
{'loss': 0.9511, 'grad_norm': 9.356950759887695, 'learning_rate': 4.934342396391734e-06, 'epoch': 0.1}
+
 10%|█         | 1880/18627 [59:55<8:01:41,  1.73s/it]
 10%|█         | 1881/18627 [59:57<8:35:23,  1.85s/it]
 10%|█         | 1882/18627 [59:59<8:58:39,  1.93s/it]
 10%|█         | 1883/18627 [1:00:01<9:14:54,  1.99s/it]
 10%|█         | 1884/18627 [1:00:03<9:26:12,  2.03s/it]
 10%|█         | 1885/18627 [1:00:04<8:06:45,  1.74s/it]
 10%|█         | 1886/18627 [1:00:07<8:38:57,  1.86s/it]
 10%|█         | 1887/18627 [1:00:09<9:01:40,  1.94s/it]
 10%|█         | 1888/18627 [1:00:11<9:14:33,  1.99s/it]
 10%|█         | 1889/18627 [1:00:13<9:26:00,  2.03s/it]
 10%|█         | 1890/18627 [1:00:15<9:34:40,  2.06s/it]
                                                        
{'loss': 1.0369, 'grad_norm': 7.01657772064209, 'learning_rate': 4.9333490321281695e-06, 'epoch': 0.1}
+
 10%|█         | 1890/18627 [1:00:15<9:34:40,  2.06s/it]
 10%|█         | 1891/18627 [1:00:17<9:39:32,  2.08s/it]
 10%|█         | 1892/18627 [1:00:19<9:41:58,  2.09s/it]
 10%|█         | 1893/18627 [1:00:20<8:26:58,  1.82s/it]
 10%|█         | 1894/18627 [1:00:23<8:53:43,  1.91s/it]
 10%|█         | 1895/18627 [1:00:25<9:10:19,  1.97s/it]
 10%|█         | 1896/18627 [1:00:26<8:05:04,  1.74s/it]
 10%|█         | 1897/18627 [1:00:28<8:36:59,  1.85s/it]
 10%|█         | 1898/18627 [1:00:29<7:41:38,  1.66s/it]
 10%|█         | 1899/18627 [1:00:30<7:02:12,  1.51s/it]
 10%|█         | 1900/18627 [1:00:33<7:53:14,  1.70s/it]
                                                        
{'loss': 1.6856, 'grad_norm': 10.344090461730957, 'learning_rate': 4.93234831115049e-06, 'epoch': 0.1}
+
 10%|█         | 1900/18627 [1:00:33<7:53:14,  1.70s/it]
 10%|█         | 1901/18627 [1:00:35<8:30:02,  1.83s/it]
 10%|█         | 1902/18627 [1:00:36<7:29:16,  1.61s/it]
 10%|█         | 1903/18627 [1:00:38<8:13:48,  1.77s/it]
 10%|█         | 1904/18627 [1:00:40<8:44:33,  1.88s/it]
 10%|█         | 1905/18627 [1:00:42<9:05:04,  1.96s/it]
 10%|█         | 1906/18627 [1:00:44<9:18:57,  2.01s/it]
 10%|█         | 1907/18627 [1:00:46<9:28:49,  2.04s/it]
 10%|█         | 1908/18627 [1:00:49<9:35:51,  2.07s/it]
 10%|█         | 1909/18627 [1:00:49<7:41:48,  1.66s/it]
 10%|█         | 1910/18627 [1:00:51<8:20:39,  1.80s/it]
                                                        
{'loss': 1.0313, 'grad_norm': 8.103877067565918, 'learning_rate': 4.9313402364841636e-06, 'epoch': 0.1}
+
 10%|█         | 1910/18627 [1:00:51<8:20:39,  1.80s/it]
 10%|█         | 1911/18627 [1:00:53<8:47:29,  1.89s/it]
 10%|█         | 1912/18627 [1:00:56<9:07:51,  1.97s/it]
 10%|█         | 1913/18627 [1:00:58<9:21:18,  2.01s/it]
 10%|█         | 1914/18627 [1:01:00<9:31:18,  2.05s/it]
 10%|█         | 1915/18627 [1:01:01<8:20:05,  1.80s/it]
 10%|█         | 1916/18627 [1:01:02<7:29:53,  1.62s/it]
 10%|█         | 1917/18627 [1:01:03<6:14:07,  1.34s/it]
 10%|█         | 1918/18627 [1:01:04<6:01:16,  1.30s/it]
 10%|█         | 1919/18627 [1:01:06<7:09:00,  1.54s/it]
 10%|█         | 1920/18627 [1:01:07<6:39:53,  1.44s/it]
                                                        
{'loss': 1.5853, 'grad_norm': 18.170270919799805, 'learning_rate': 4.9303248111768905e-06, 'epoch': 0.1}
+
 10%|█         | 1920/18627 [1:01:07<6:39:53,  1.44s/it]
 10%|█         | 1921/18627 [1:01:10<7:35:44,  1.64s/it]
 10%|█         | 1922/18627 [1:01:12<8:17:00,  1.79s/it]
 10%|█         | 1923/18627 [1:01:14<8:43:59,  1.88s/it]
 10%|█         | 1924/18627 [1:01:16<9:03:18,  1.95s/it]
 10%|█         | 1925/18627 [1:01:17<7:52:17,  1.70s/it]
 10%|█         | 1926/18627 [1:01:19<8:28:35,  1.83s/it]
 10%|█         | 1927/18627 [1:01:21<8:53:23,  1.92s/it]
 10%|█         | 1928/18627 [1:01:23<9:10:55,  1.98s/it]
 10%|█         | 1929/18627 [1:01:26<9:25:55,  2.03s/it]
 10%|█         | 1930/18627 [1:01:28<9:33:27,  2.06s/it]
                                                        
{'loss': 0.9836, 'grad_norm': 8.632328987121582, 'learning_rate': 4.929302038298593e-06, 'epoch': 0.1}
+
 10%|█         | 1930/18627 [1:01:28<9:33:27,  2.06s/it]
 10%|█         | 1931/18627 [1:01:30<9:40:10,  2.08s/it]
 10%|█         | 1932/18627 [1:01:32<9:44:35,  2.10s/it]
 10%|█         | 1933/18627 [1:01:34<9:45:51,  2.11s/it]
 10%|█         | 1934/18627 [1:01:36<9:47:24,  2.11s/it]
 10%|█         | 1935/18627 [1:01:38<9:46:44,  2.11s/it]
 10%|█         | 1936/18627 [1:01:39<8:23:18,  1.81s/it]
 10%|█         | 1937/18627 [1:01:42<8:48:04,  1.90s/it]
 10%|█         | 1938/18627 [1:01:44<9:07:48,  1.97s/it]
 10%|█         | 1939/18627 [1:01:45<8:03:31,  1.74s/it]
 10%|█         | 1940/18627 [1:01:47<8:34:47,  1.85s/it]
                                                        
{'loss': 1.2209, 'grad_norm': 10.443403244018555, 'learning_rate': 4.928271920941407e-06, 'epoch': 0.1}
+
 10%|█         | 1940/18627 [1:01:47<8:34:47,  1.85s/it]
 10%|█         | 1941/18627 [1:01:49<8:57:29,  1.93s/it]
 10%|█         | 1942/18627 [1:01:51<9:12:07,  1.99s/it]
 10%|█         | 1943/18627 [1:01:53<9:23:11,  2.03s/it]
 10%|█         | 1944/18627 [1:01:55<9:31:19,  2.05s/it]
 10%|█         | 1945/18627 [1:01:56<7:41:26,  1.66s/it]
 10%|█         | 1946/18627 [1:01:58<8:24:10,  1.81s/it]
 10%|█         | 1947/18627 [1:02:01<8:53:29,  1.92s/it]
 10%|█         | 1948/18627 [1:02:03<9:12:55,  1.99s/it]
 10%|█         | 1949/18627 [1:02:05<9:27:29,  2.04s/it]
 10%|█         | 1950/18627 [1:02:07<9:31:32,  2.06s/it]
                                                        
{'loss': 0.749, 'grad_norm': 10.140669822692871, 'learning_rate': 4.927234462219675e-06, 'epoch': 0.1}
+
 10%|█         | 1950/18627 [1:02:07<9:31:32,  2.06s/it]
 10%|█         | 1951/18627 [1:02:09<9:37:04,  2.08s/it]
 10%|█         | 1952/18627 [1:02:10<7:41:44,  1.66s/it]
 10%|█         | 1953/18627 [1:02:12<8:19:22,  1.80s/it]
 10%|█         | 1954/18627 [1:02:14<8:46:58,  1.90s/it]
 10%|█         | 1955/18627 [1:02:15<7:47:14,  1.68s/it]
 11%|█         | 1956/18627 [1:02:17<8:24:52,  1.82s/it]
 11%|█         | 1957/18627 [1:02:19<7:32:21,  1.63s/it]
 11%|█         | 1958/18627 [1:02:20<6:56:46,  1.50s/it]
 11%|█         | 1959/18627 [1:02:22<7:52:32,  1.70s/it]
 11%|█         | 1960/18627 [1:02:24<8:25:54,  1.82s/it]
                                                        
{'loss': 1.335, 'grad_norm': 7.7643141746521, 'learning_rate': 4.926189665269933e-06, 'epoch': 0.11}
+
 11%|█         | 1960/18627 [1:02:24<8:25:54,  1.82s/it]
 11%|█         | 1961/18627 [1:02:26<8:51:05,  1.91s/it]
 11%|█         | 1962/18627 [1:02:28<9:08:49,  1.98s/it]
 11%|█         | 1963/18627 [1:02:30<9:20:20,  2.02s/it]
 11%|█         | 1964/18627 [1:02:32<9:28:26,  2.05s/it]
 11%|█         | 1965/18627 [1:02:35<9:33:44,  2.07s/it]
 11%|█         | 1966/18627 [1:02:35<7:40:05,  1.66s/it]
 11%|█         | 1967/18627 [1:02:37<8:16:27,  1.79s/it]
 11%|█         | 1968/18627 [1:02:39<8:42:11,  1.88s/it]
 11%|█         | 1969/18627 [1:02:42<8:59:55,  1.94s/it]
 11%|█         | 1970/18627 [1:02:44<9:13:22,  1.99s/it]
                                                        
{'loss': 0.8093, 'grad_norm': 11.005542755126953, 'learning_rate': 4.9251375332509015e-06, 'epoch': 0.11}
+
 11%|█         | 1970/18627 [1:02:44<9:13:22,  1.99s/it]
 11%|█         | 1971/18627 [1:02:46<9:26:55,  2.04s/it]
 11%|█         | 1972/18627 [1:02:48<9:32:40,  2.06s/it]
 11%|█         | 1973/18627 [1:02:50<9:36:23,  2.08s/it]
 11%|█         | 1974/18627 [1:02:51<8:21:55,  1.81s/it]
 11%|█         | 1975/18627 [1:02:52<7:15:58,  1.57s/it]
 11%|█         | 1976/18627 [1:02:54<8:05:43,  1.75s/it]
 11%|█         | 1977/18627 [1:02:56<7:19:20,  1.58s/it]
 11%|█         | 1978/18627 [1:02:58<8:04:36,  1.75s/it]
 11%|█         | 1979/18627 [1:03:00<8:34:50,  1.86s/it]
 11%|█         | 1980/18627 [1:03:02<8:56:09,  1.93s/it]
                                                        
{'loss': 1.1904, 'grad_norm': 7.971055030822754, 'learning_rate': 4.9240780693434794e-06, 'epoch': 0.11}
+
 11%|█         | 1980/18627 [1:03:02<8:56:09,  1.93s/it]
 11%|█         | 1981/18627 [1:03:04<9:13:08,  1.99s/it]
 11%|█         | 1982/18627 [1:03:06<9:25:30,  2.04s/it]
 11%|█         | 1983/18627 [1:03:08<9:29:35,  2.05s/it]
 11%|█         | 1984/18627 [1:03:10<9:36:49,  2.08s/it]
 11%|█         | 1985/18627 [1:03:13<9:40:22,  2.09s/it]
 11%|█         | 1986/18627 [1:03:14<8:24:26,  1.82s/it]
 11%|█         | 1987/18627 [1:03:16<8:49:30,  1.91s/it]
 11%|█         | 1988/18627 [1:03:17<7:08:34,  1.55s/it]
 11%|█         | 1989/18627 [1:03:19<7:58:13,  1.72s/it]
 11%|█         | 1990/18627 [1:03:21<8:29:08,  1.84s/it]
                                                        
{'loss': 1.0182, 'grad_norm': 9.07944107055664, 'learning_rate': 4.9230112767507295e-06, 'epoch': 0.11}
+
 11%|█         | 1990/18627 [1:03:21<8:29:08,  1.84s/it]
 11%|█         | 1991/18627 [1:03:23<8:53:25,  1.92s/it]
 11%|█         | 1992/18627 [1:03:25<9:10:27,  1.99s/it]
 11%|█         | 1993/18627 [1:03:26<8:03:56,  1.75s/it]
 11%|█         | 1994/18627 [1:03:28<8:34:10,  1.85s/it]
 11%|█         | 1995/18627 [1:03:29<6:58:36,  1.51s/it]
 11%|█         | 1996/18627 [1:03:31<7:49:34,  1.69s/it]
 11%|█         | 1997/18627 [1:03:33<8:25:29,  1.82s/it]
 11%|█         | 1998/18627 [1:03:35<7:32:34,  1.63s/it]
 11%|█         | 1999/18627 [1:03:36<6:55:37,  1.50s/it]
 11%|█         | 2000/18627 [1:03:38<7:46:44,  1.68s/it]
                                                        
{'loss': 1.3015, 'grad_norm': 10.084610939025879, 'learning_rate': 4.921937158697874e-06, 'epoch': 0.11}
+
 11%|█         | 2000/18627 [1:03:38<7:46:44,  1.68s/it]/home/zhengduo/workspace/LLaMA-Factory/src/qwen_vl/model/modeling_qwen2_5_vl.py:1979: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
+  with torch.cuda.amp.autocast(dtype=dtype):
+/home/zhengduo/miniconda3/envs/vgllm/lib/python3.10/site-packages/torch/utils/checkpoint.py:87: UserWarning: None of the inputs have requires_grad=True. Gradients will be None
+  warnings.warn(
+
 11%|█         | 2001/18627 [1:04:20<63:12:52, 13.69s/it]
 11%|█         | 2002/18627 [1:04:21<45:54:07,  9.94s/it]
 11%|█         | 2003/18627 [1:04:23<35:04:05,  7.59s/it]
 11%|█         | 2004/18627 [1:04:24<26:11:43,  5.67s/it]
 11%|█         | 2005/18627 [1:04:26<21:14:46,  4.60s/it]
 11%|█         | 2006/18627 [1:04:28<17:47:46,  3.85s/it]
 11%|█         | 2007/18627 [1:04:30<15:24:01,  3.34s/it]
 11%|█         | 2008/18627 [1:04:32<13:43:21,  2.97s/it]
 11%|█         | 2009/18627 [1:04:35<12:31:30,  2.71s/it]
 11%|█         | 2010/18627 [1:04:37<11:40:52,  2.53s/it]
                                                         
{'loss': 1.1102, 'grad_norm': 8.114959716796875, 'learning_rate': 4.920855718432282e-06, 'epoch': 0.11}
+
 11%|█         | 2010/18627 [1:04:37<11:40:52,  2.53s/it]
 11%|█         | 2011/18627 [1:04:38<9:50:26,  2.13s/it] 
 11%|█         | 2012/18627 [1:04:40<9:53:42,  2.14s/it]
 11%|█         | 2013/18627 [1:04:42<9:53:54,  2.14s/it]
 11%|█         | 2014/18627 [1:04:44<9:54:17,  2.15s/it]
 11%|█         | 2015/18627 [1:04:46<8:34:49,  1.86s/it]
 11%|█         | 2016/18627 [1:04:47<7:41:31,  1.67s/it]
 11%|█         | 2017/18627 [1:04:49<8:19:31,  1.80s/it]
 11%|█         | 2018/18627 [1:04:51<8:46:54,  1.90s/it]
 11%|█         | 2019/18627 [1:04:53<9:04:39,  1.97s/it]
 11%|█         | 2020/18627 [1:04:55<9:19:14,  2.02s/it]
                                                        
{'loss': 1.3611, 'grad_norm': 8.14728832244873, 'learning_rate': 4.919766959223456e-06, 'epoch': 0.11}
+
 11%|█         | 2020/18627 [1:04:55<9:19:14,  2.02s/it]
 11%|█         | 2021/18627 [1:04:57<9:28:34,  2.05s/it]
 11%|█         | 2022/18627 [1:05:00<9:35:43,  2.08s/it]
 11%|█         | 2023/18627 [1:05:02<9:36:52,  2.08s/it]
 11%|█         | 2024/18627 [1:05:04<9:37:36,  2.09s/it]
 11%|█         | 2025/18627 [1:05:06<9:42:39,  2.11s/it]
 11%|█         | 2026/18627 [1:05:08<9:42:22,  2.10s/it]
 11%|█         | 2027/18627 [1:05:10<9:45:28,  2.12s/it]
 11%|█         | 2028/18627 [1:05:12<9:44:13,  2.11s/it]
 11%|█         | 2029/18627 [1:05:13<8:27:18,  1.83s/it]
 11%|█         | 2030/18627 [1:05:16<8:51:07,  1.92s/it]
                                                        
{'loss': 0.9366, 'grad_norm': 10.183863639831543, 'learning_rate': 4.918670884363028e-06, 'epoch': 0.11}
+
 11%|█         | 2030/18627 [1:05:16<8:51:07,  1.92s/it]
 11%|█         | 2031/18627 [1:05:17<7:50:00,  1.70s/it]
 11%|█         | 2032/18627 [1:05:19<8:24:45,  1.82s/it]
 11%|█         | 2033/18627 [1:05:20<7:31:40,  1.63s/it]
 11%|█         | 2034/18627 [1:05:22<8:12:06,  1.78s/it]
 11%|█         | 2035/18627 [1:05:24<8:38:33,  1.88s/it]
 11%|█         | 2036/18627 [1:05:26<8:59:03,  1.95s/it]
 11%|█         | 2037/18627 [1:05:28<7:56:12,  1.72s/it]
 11%|█         | 2038/18627 [1:05:30<8:27:09,  1.83s/it]
 11%|█         | 2039/18627 [1:05:31<7:34:31,  1.64s/it]
 11%|█         | 2040/18627 [1:05:33<8:15:32,  1.79s/it]
                                                        
{'loss': 1.5853, 'grad_norm': 11.224695205688477, 'learning_rate': 4.91756749716475e-06, 'epoch': 0.11}
+
 11%|█         | 2040/18627 [1:05:33<8:15:32,  1.79s/it]
 11%|█         | 2041/18627 [1:05:34<7:25:47,  1.61s/it]
 11%|█         | 2042/18627 [1:05:36<8:09:13,  1.77s/it]
 11%|█         | 2043/18627 [1:05:38<7:20:15,  1.59s/it]
 11%|█         | 2044/18627 [1:05:39<6:46:26,  1.47s/it]
 11%|█         | 2045/18627 [1:05:41<7:40:47,  1.67s/it]
 11%|█         | 2046/18627 [1:05:42<7:00:43,  1.52s/it]
 11%|█         | 2047/18627 [1:05:44<7:49:55,  1.70s/it]
 11%|█         | 2048/18627 [1:05:46<8:23:07,  1.82s/it]
 11%|█         | 2049/18627 [1:05:48<8:46:55,  1.91s/it]
 11%|█         | 2050/18627 [1:05:51<9:06:06,  1.98s/it]
                                                        
{'loss': 1.5224, 'grad_norm': 9.279414176940918, 'learning_rate': 4.916456800964478e-06, 'epoch': 0.11}
+
 11%|█         | 2050/18627 [1:05:51<9:06:06,  1.98s/it]
 11%|█         | 2051/18627 [1:05:53<9:16:05,  2.01s/it]
 11%|█         | 2052/18627 [1:05:55<9:24:55,  2.04s/it]
 11%|█         | 2053/18627 [1:05:57<9:31:21,  2.07s/it]
 11%|█         | 2054/18627 [1:05:59<9:34:19,  2.08s/it]
 11%|█         | 2055/18627 [1:06:01<9:37:24,  2.09s/it]
 11%|█         | 2056/18627 [1:06:03<9:41:08,  2.10s/it]
 11%|█         | 2057/18627 [1:06:05<9:43:26,  2.11s/it]
 11%|█         | 2058/18627 [1:06:07<9:44:58,  2.12s/it]
 11%|█         | 2059/18627 [1:06:08<7:53:22,  1.71s/it]
 11%|█         | 2060/18627 [1:06:10<8:25:37,  1.83s/it]
                                                        
{'loss': 0.816, 'grad_norm': 7.420502185821533, 'learning_rate': 4.915338799120165e-06, 'epoch': 0.11}
+
 11%|█         | 2060/18627 [1:06:10<8:25:37,  1.83s/it]
 11%|█         | 2061/18627 [1:06:12<8:49:56,  1.92s/it]
 11%|█         | 2062/18627 [1:06:15<9:06:22,  1.98s/it]
 11%|█         | 2063/18627 [1:06:17<9:15:52,  2.01s/it]
 11%|█         | 2064/18627 [1:06:19<9:22:43,  2.04s/it]
 11%|█         | 2065/18627 [1:06:21<9:28:42,  2.06s/it]
 11%|█         | 2066/18627 [1:06:23<9:33:08,  2.08s/it]
 11%|█         | 2067/18627 [1:06:25<9:37:07,  2.09s/it]
 11%|█         | 2068/18627 [1:06:27<9:40:07,  2.10s/it]
 11%|█         | 2069/18627 [1:06:28<8:23:49,  1.83s/it]
 11%|█         | 2070/18627 [1:06:30<7:33:51,  1.64s/it]
                                                        
{'loss': 1.1427, 'grad_norm': 17.925199508666992, 'learning_rate': 4.914213495011854e-06, 'epoch': 0.11}
+
 11%|█         | 2070/18627 [1:06:30<7:33:51,  1.64s/it]
 11%|█         | 2071/18627 [1:06:32<8:13:06,  1.79s/it]
 11%|█         | 2072/18627 [1:06:33<7:23:58,  1.61s/it]
 11%|█         | 2073/18627 [1:06:35<8:06:28,  1.76s/it]
 11%|█         | 2074/18627 [1:06:37<8:37:25,  1.88s/it]
 11%|█         | 2075/18627 [1:06:39<8:57:42,  1.95s/it]
 11%|█         | 2076/18627 [1:06:41<7:55:24,  1.72s/it]
 11%|█         | 2077/18627 [1:06:43<8:28:01,  1.84s/it]
 11%|█         | 2078/18627 [1:06:45<8:52:00,  1.93s/it]
 11%|█         | 2079/18627 [1:06:47<9:10:48,  2.00s/it]
 11%|█         | 2080/18627 [1:06:49<9:21:37,  2.04s/it]
                                                        
{'loss': 1.0403, 'grad_norm': 7.620808124542236, 'learning_rate': 4.913080892041661e-06, 'epoch': 0.11}
+
 11%|█         | 2080/18627 [1:06:49<9:21:37,  2.04s/it]
 11%|█         | 2081/18627 [1:06:50<8:02:26,  1.75s/it]
 11%|█         | 2082/18627 [1:06:51<7:07:05,  1.55s/it]
 11%|█         | 2083/18627 [1:06:53<7:54:24,  1.72s/it]
 11%|█         | 2084/18627 [1:06:55<8:27:11,  1.84s/it]
 11%|█         | 2085/18627 [1:06:58<8:48:40,  1.92s/it]
 11%|█         | 2086/18627 [1:06:59<7:49:42,  1.70s/it]
 11%|█         | 2087/18627 [1:07:01<8:24:36,  1.83s/it]
 11%|█         | 2088/18627 [1:07:03<8:48:52,  1.92s/it]
 11%|█         | 2089/18627 [1:07:05<9:05:36,  1.98s/it]
 11%|█         | 2090/18627 [1:07:07<9:16:55,  2.02s/it]
                                                        
{'loss': 1.3583, 'grad_norm': 12.792862892150879, 'learning_rate': 4.911940993633772e-06, 'epoch': 0.11}
+
 11%|█         | 2090/18627 [1:07:07<9:16:55,  2.02s/it]
 11%|█         | 2091/18627 [1:07:08<8:08:09,  1.77s/it]
 11%|█         | 2092/18627 [1:07:11<8:36:55,  1.88s/it]
 11%|█         | 2093/18627 [1:07:13<8:55:11,  1.94s/it]
 11%|█         | 2094/18627 [1:07:14<7:52:44,  1.72s/it]
 11%|█         | 2095/18627 [1:07:16<8:26:01,  1.84s/it]
 11%|█▏        | 2096/18627 [1:07:18<8:52:47,  1.93s/it]
 11%|█▏        | 2097/18627 [1:07:19<7:50:22,  1.71s/it]
 11%|█▏        | 2098/18627 [1:07:21<8:23:56,  1.83s/it]
 11%|█▏        | 2099/18627 [1:07:23<7:31:49,  1.64s/it]
 11%|█▏        | 2100/18627 [1:07:24<6:54:28,  1.50s/it]
                                                        
{'loss': 1.6445, 'grad_norm': 17.190523147583008, 'learning_rate': 4.9107938032344275e-06, 'epoch': 0.11}
+
 11%|█▏        | 2100/18627 [1:07:24<6:54:28,  1.50s/it]
 11%|█▏        | 2101/18627 [1:07:26<7:44:32,  1.69s/it]
 11%|█▏        | 2102/18627 [1:07:28<8:19:36,  1.81s/it]
 11%|█▏        | 2103/18627 [1:07:29<7:28:24,  1.63s/it]
 11%|█▏        | 2104/18627 [1:07:31<8:09:52,  1.78s/it]
 11%|█▏        | 2105/18627 [1:07:33<8:38:26,  1.88s/it]
 11%|█▏        | 2106/18627 [1:07:36<8:57:43,  1.95s/it]
 11%|█▏        | 2107/18627 [1:07:37<7:54:14,  1.72s/it]
 11%|█▏        | 2108/18627 [1:07:39<8:24:48,  1.83s/it]
 11%|█▏        | 2109/18627 [1:07:41<8:45:52,  1.91s/it]
 11%|█▏        | 2110/18627 [1:07:43<9:02:35,  1.97s/it]
                                                        
{'loss': 1.2134, 'grad_norm': 7.116855144500732, 'learning_rate': 4.909639324311914e-06, 'epoch': 0.11}
+
 11%|█▏        | 2110/18627 [1:07:43<9:02:35,  1.97s/it]
 11%|█▏        | 2111/18627 [1:07:45<9:14:31,  2.01s/it]
 11%|█▏        | 2112/18627 [1:07:47<9:22:12,  2.04s/it]
 11%|█▏        | 2113/18627 [1:07:49<9:27:31,  2.06s/it]
 11%|█▏        | 2114/18627 [1:07:52<9:31:18,  2.08s/it]
 11%|█▏        | 2115/18627 [1:07:53<8:17:41,  1.81s/it]
 11%|█▏        | 2116/18627 [1:07:55<8:41:39,  1.90s/it]
 11%|█▏        | 2117/18627 [1:07:56<7:37:40,  1.66s/it]
 11%|█▏        | 2118/18627 [1:07:58<8:15:25,  1.80s/it]
 11%|█▏        | 2119/18627 [1:08:00<8:42:54,  1.90s/it]
 11%|█▏        | 2120/18627 [1:08:02<9:02:28,  1.97s/it]
                                                        
{'loss': 1.2422, 'grad_norm': 8.309609413146973, 'learning_rate': 4.908477560356554e-06, 'epoch': 0.11}
+
 11%|█▏        | 2120/18627 [1:08:02<9:02:28,  1.97s/it]
 11%|█▏        | 2121/18627 [1:08:04<7:57:16,  1.73s/it]
 11%|█▏        | 2122/18627 [1:08:06<8:27:59,  1.85s/it]
 11%|█▏        | 2123/18627 [1:08:08<8:50:25,  1.93s/it]
 11%|█▏        | 2124/18627 [1:08:10<9:06:53,  1.99s/it]
 11%|█▏        | 2125/18627 [1:08:12<9:16:51,  2.02s/it]
 11%|█▏        | 2126/18627 [1:08:14<9:22:56,  2.05s/it]
 11%|█▏        | 2127/18627 [1:08:15<8:13:04,  1.79s/it]
 11%|█▏        | 2128/18627 [1:08:16<7:23:55,  1.61s/it]
 11%|█▏        | 2129/18627 [1:08:19<8:05:10,  1.76s/it]
 11%|█▏        | 2130/18627 [1:08:21<8:35:21,  1.87s/it]
                                                        
{'loss': 1.411, 'grad_norm': 11.83594036102295, 'learning_rate': 4.907308514880693e-06, 'epoch': 0.11}
+
 11%|█▏        | 2130/18627 [1:08:21<8:35:21,  1.87s/it]
 11%|█▏        | 2131/18627 [1:08:22<7:38:53,  1.67s/it]
 11%|█▏        | 2132/18627 [1:08:24<8:15:47,  1.80s/it]
 11%|█▏        | 2133/18627 [1:08:25<7:25:06,  1.62s/it]
 11%|█▏        | 2134/18627 [1:08:27<8:05:23,  1.77s/it]
 11%|█▏        | 2135/18627 [1:08:29<8:35:09,  1.87s/it]
 11%|█▏        | 2136/18627 [1:08:32<8:55:41,  1.95s/it]
 11%|█▏        | 2137/18627 [1:08:33<7:54:05,  1.73s/it]
 11%|█▏        | 2138/18627 [1:08:35<8:25:11,  1.84s/it]
 11%|█▏        | 2139/18627 [1:08:36<7:24:50,  1.62s/it]
 11%|█▏        | 2140/18627 [1:08:38<8:07:08,  1.77s/it]
                                                        
{'loss': 1.6207, 'grad_norm': 10.184101104736328, 'learning_rate': 4.9061321914186925e-06, 'epoch': 0.11}
+
 11%|█▏        | 2140/18627 [1:08:38<8:07:08,  1.77s/it]
 11%|█▏        | 2141/18627 [1:08:40<8:35:18,  1.88s/it]
 11%|█▏        | 2142/18627 [1:08:42<8:56:24,  1.95s/it]
 12%|█▏        | 2143/18627 [1:08:44<7:52:42,  1.72s/it]
 12%|█▏        | 2144/18627 [1:08:46<8:25:46,  1.84s/it]
 12%|█▏        | 2145/18627 [1:08:48<8:48:10,  1.92s/it]
 12%|█▏        | 2146/18627 [1:08:50<9:03:02,  1.98s/it]
 12%|█▏        | 2147/18627 [1:08:52<9:15:38,  2.02s/it]
 12%|█▏        | 2148/18627 [1:08:53<7:29:49,  1.64s/it]
 12%|█▏        | 2149/18627 [1:08:55<8:11:21,  1.79s/it]
 12%|█▏        | 2150/18627 [1:08:57<8:37:25,  1.88s/it]
                                                        
{'loss': 0.9211, 'grad_norm': 11.125953674316406, 'learning_rate': 4.904948593526916e-06, 'epoch': 0.12}
+
 12%|█▏        | 2150/18627 [1:08:57<8:37:25,  1.88s/it]
 12%|█▏        | 2151/18627 [1:08:58<7:40:56,  1.68s/it]
 12%|█▏        | 2152/18627 [1:08:59<6:21:43,  1.39s/it]
 12%|█▏        | 2153/18627 [1:09:01<7:23:00,  1.61s/it]
 12%|█▏        | 2154/18627 [1:09:03<8:05:08,  1.77s/it]
 12%|█▏        | 2155/18627 [1:09:05<8:32:22,  1.87s/it]
 12%|█▏        | 2156/18627 [1:09:07<8:52:42,  1.94s/it]
 12%|█▏        | 2157/18627 [1:09:10<9:08:09,  2.00s/it]
 12%|█▏        | 2158/18627 [1:09:12<9:19:05,  2.04s/it]
 12%|█▏        | 2159/18627 [1:09:14<9:25:19,  2.06s/it]
 12%|█▏        | 2160/18627 [1:09:16<9:31:59,  2.08s/it]
                                                        
{'loss': 0.9311, 'grad_norm': 9.776846885681152, 'learning_rate': 4.9037577247837205e-06, 'epoch': 0.12}
+
 12%|█▏        | 2160/18627 [1:09:16<9:31:59,  2.08s/it]
 12%|█▏        | 2161/18627 [1:09:18<9:35:19,  2.10s/it]
 12%|█▏        | 2162/18627 [1:09:20<9:38:54,  2.11s/it]
 12%|█▏        | 2163/18627 [1:09:22<9:40:00,  2.11s/it]
 12%|█▏        | 2164/18627 [1:09:24<9:40:42,  2.12s/it]
 12%|█▏        | 2165/18627 [1:09:27<9:41:57,  2.12s/it]
 12%|█▏        | 2166/18627 [1:09:29<9:43:07,  2.13s/it]
 12%|█▏        | 2167/18627 [1:09:31<9:43:13,  2.13s/it]
 12%|█▏        | 2168/18627 [1:09:33<9:41:19,  2.12s/it]
 12%|█▏        | 2169/18627 [1:09:35<9:42:09,  2.12s/it]
 12%|█▏        | 2170/18627 [1:09:37<9:39:39,  2.11s/it]
                                                        
{'loss': 0.7538, 'grad_norm': 11.215271949768066, 'learning_rate': 4.902559588789446e-06, 'epoch': 0.12}
+
 12%|█▏        | 2170/18627 [1:09:37<9:39:39,  2.11s/it]
 12%|█▏        | 2171/18627 [1:09:39<9:39:54,  2.11s/it]
 12%|█▏        | 2172/18627 [1:09:41<9:38:42,  2.11s/it]
 12%|█▏        | 2173/18627 [1:09:43<9:40:53,  2.12s/it]
 12%|█▏        | 2174/18627 [1:09:45<8:23:59,  1.84s/it]
 12%|█▏        | 2175/18627 [1:09:47<8:48:29,  1.93s/it]
 12%|█▏        | 2176/18627 [1:09:48<7:48:20,  1.71s/it]
 12%|█▏        | 2177/18627 [1:09:49<7:05:57,  1.55s/it]
 12%|█▏        | 2178/18627 [1:09:51<7:51:31,  1.72s/it]
 12%|█▏        | 2179/18627 [1:09:53<8:23:58,  1.84s/it]
 12%|█▏        | 2180/18627 [1:09:56<8:46:15,  1.92s/it]
                                                        
{'loss': 1.3579, 'grad_norm': 10.415063858032227, 'learning_rate': 4.901354189166402e-06, 'epoch': 0.12}
+
 12%|█▏        | 2180/18627 [1:09:56<8:46:15,  1.92s/it]
 12%|█▏        | 2181/18627 [1:09:56<7:09:38,  1.57s/it]
 12%|█▏        | 2182/18627 [1:09:57<6:28:47,  1.42s/it]
 12%|█▏        | 2183/18627 [1:09:59<7:27:00,  1.63s/it]
 12%|█▏        | 2184/18627 [1:10:02<8:07:00,  1.78s/it]
 12%|█▏        | 2185/18627 [1:10:04<8:33:57,  1.88s/it]
 12%|█▏        | 2186/18627 [1:10:06<8:54:02,  1.95s/it]
 12%|█▏        | 2187/18627 [1:10:08<9:09:07,  2.00s/it]
 12%|█▏        | 2188/18627 [1:10:10<9:17:20,  2.03s/it]
 12%|█▏        | 2189/18627 [1:10:11<8:07:36,  1.78s/it]
 12%|█▏        | 2190/18627 [1:10:13<8:36:39,  1.89s/it]
                                                        
{'loss': 1.1946, 'grad_norm': 11.879575729370117, 'learning_rate': 4.900141529558857e-06, 'epoch': 0.12}
+
 12%|█▏        | 2190/18627 [1:10:13<8:36:39,  1.89s/it]
 12%|█▏        | 2191/18627 [1:10:16<8:57:00,  1.96s/it]
 12%|█▏        | 2192/18627 [1:10:18<9:10:51,  2.01s/it]
 12%|█▏        | 2193/18627 [1:10:20<9:21:50,  2.05s/it]
 12%|█▏        | 2194/18627 [1:10:22<9:25:52,  2.07s/it]
 12%|█▏        | 2195/18627 [1:10:24<9:30:56,  2.08s/it]
 12%|█▏        | 2196/18627 [1:10:26<9:33:30,  2.09s/it]
 12%|█▏        | 2197/18627 [1:10:27<8:19:18,  1.82s/it]
 12%|█▏        | 2198/18627 [1:10:29<8:41:46,  1.91s/it]
 12%|█▏        | 2199/18627 [1:10:32<8:57:44,  1.96s/it]
 12%|█▏        | 2200/18627 [1:10:34<9:10:30,  2.01s/it]
                                                        
{'loss': 0.9325, 'grad_norm': 8.953886032104492, 'learning_rate': 4.898921613633031e-06, 'epoch': 0.12}
+
 12%|█▏        | 2200/18627 [1:10:34<9:10:30,  2.01s/it]
 12%|█▏        | 2201/18627 [1:10:36<9:19:22,  2.04s/it]
 12%|█▏        | 2202/18627 [1:10:37<7:54:51,  1.73s/it]
 12%|█▏        | 2203/18627 [1:10:39<8:26:09,  1.85s/it]
 12%|█▏        | 2204/18627 [1:10:41<8:49:09,  1.93s/it]
 12%|█▏        | 2205/18627 [1:10:43<9:04:47,  1.99s/it]
 12%|█▏        | 2206/18627 [1:10:44<7:59:54,  1.75s/it]
 12%|█▏        | 2207/18627 [1:10:46<7:13:20,  1.58s/it]
 12%|█▏        | 2208/18627 [1:10:48<7:56:01,  1.74s/it]
 12%|█▏        | 2209/18627 [1:10:50<8:27:52,  1.86s/it]
 12%|█▏        | 2210/18627 [1:10:52<8:48:51,  1.93s/it]
                                                        
{'loss': 1.3203, 'grad_norm': 8.201791763305664, 'learning_rate': 4.897694445077081e-06, 'epoch': 0.12}
+
 12%|█▏        | 2210/18627 [1:10:52<8:48:51,  1.93s/it]
 12%|█▏        | 2211/18627 [1:10:54<9:05:08,  1.99s/it]
 12%|█▏        | 2212/18627 [1:10:56<9:14:16,  2.03s/it]
 12%|█▏        | 2213/18627 [1:10:58<9:21:41,  2.05s/it]
 12%|█▏        | 2214/18627 [1:11:00<9:27:45,  2.08s/it]
 12%|█▏        | 2215/18627 [1:11:02<9:30:34,  2.09s/it]
 12%|█▏        | 2216/18627 [1:11:05<9:33:39,  2.10s/it]
 12%|█▏        | 2217/18627 [1:11:07<9:36:03,  2.11s/it]
 12%|█▏        | 2218/18627 [1:11:08<8:20:52,  1.83s/it]
 12%|█▏        | 2219/18627 [1:11:10<8:44:20,  1.92s/it]
 12%|█▏        | 2220/18627 [1:11:12<9:00:40,  1.98s/it]
                                                        
{'loss': 0.9742, 'grad_norm': 7.558645248413086, 'learning_rate': 4.896460027601089e-06, 'epoch': 0.12}
+
 12%|█▏        | 2220/18627 [1:11:12<9:00:40,  1.98s/it]
 12%|█▏        | 2221/18627 [1:11:14<9:13:17,  2.02s/it]
 12%|█▏        | 2222/18627 [1:11:16<9:22:27,  2.06s/it]
 12%|█▏        | 2223/18627 [1:11:19<9:27:27,  2.08s/it]
 12%|█▏        | 2224/18627 [1:11:21<9:29:10,  2.08s/it]
 12%|█▏        | 2225/18627 [1:11:23<9:31:52,  2.09s/it]
 12%|█▏        | 2226/18627 [1:11:25<9:34:43,  2.10s/it]
 12%|█▏        | 2227/18627 [1:11:27<9:36:23,  2.11s/it]
 12%|█▏        | 2228/18627 [1:11:29<9:38:44,  2.12s/it]
 12%|█▏        | 2229/18627 [1:11:30<8:22:23,  1.84s/it]
 12%|█▏        | 2230/18627 [1:11:32<8:45:08,  1.92s/it]
                                                        
{'loss': 0.93, 'grad_norm': 7.915459632873535, 'learning_rate': 4.895218364937056e-06, 'epoch': 0.12}
+
 12%|█▏        | 2230/18627 [1:11:32<8:45:08,  1.92s/it]
 12%|█▏        | 2231/18627 [1:11:35<9:01:33,  1.98s/it]
 12%|█▏        | 2232/18627 [1:11:36<7:56:13,  1.74s/it]
 12%|█▏        | 2233/18627 [1:11:38<8:26:35,  1.85s/it]
 12%|█▏        | 2234/18627 [1:11:40<8:50:15,  1.94s/it]
 12%|█▏        | 2235/18627 [1:11:42<9:06:02,  2.00s/it]
 12%|█▏        | 2236/18627 [1:11:44<9:17:28,  2.04s/it]
 12%|█▏        | 2237/18627 [1:11:45<8:07:30,  1.78s/it]
 12%|█▏        | 2238/18627 [1:11:48<8:35:52,  1.89s/it]
 12%|█▏        | 2239/18627 [1:11:50<8:54:19,  1.96s/it]
 12%|█▏        | 2240/18627 [1:11:52<9:07:19,  2.00s/it]
                                                        
{'loss': 1.2277, 'grad_norm': 9.397565841674805, 'learning_rate': 4.8939694608388835e-06, 'epoch': 0.12}
+
 12%|█▏        | 2240/18627 [1:11:52<9:07:19,  2.00s/it]
 12%|█▏        | 2241/18627 [1:11:53<8:00:46,  1.76s/it]
 12%|█▏        | 2242/18627 [1:11:55<8:30:10,  1.87s/it]
 12%|█▏        | 2243/18627 [1:11:57<8:49:32,  1.94s/it]
 12%|█▏        | 2244/18627 [1:11:59<9:04:08,  1.99s/it]
 12%|█▏        | 2245/18627 [1:12:01<7:58:49,  1.75s/it]
 12%|█▏        | 2246/18627 [1:12:03<8:29:01,  1.86s/it]
 12%|█▏        | 2247/18627 [1:12:05<8:51:03,  1.95s/it]
 12%|█▏        | 2248/18627 [1:12:07<9:05:41,  2.00s/it]
 12%|█▏        | 2249/18627 [1:12:09<9:14:38,  2.03s/it]
 12%|█▏        | 2250/18627 [1:12:10<8:06:14,  1.78s/it]
                                                        
{'loss': 1.3145, 'grad_norm': 15.724913597106934, 'learning_rate': 4.892713319082369e-06, 'epoch': 0.12}
+
 12%|█▏        | 2250/18627 [1:12:10<8:06:14,  1.78s/it]
 12%|█▏        | 2251/18627 [1:12:12<8:34:04,  1.88s/it]
 12%|█▏        | 2252/18627 [1:12:14<8:51:26,  1.95s/it]
 12%|█▏        | 2253/18627 [1:12:17<9:04:43,  2.00s/it]
 12%|█▏        | 2254/18627 [1:12:18<7:58:36,  1.75s/it]
 12%|█▏        | 2255/18627 [1:12:19<7:12:25,  1.58s/it]
 12%|█▏        | 2256/18627 [1:12:21<7:56:01,  1.74s/it]
 12%|█▏        | 2257/18627 [1:12:23<8:24:43,  1.85s/it]
 12%|█▏        | 2258/18627 [1:12:25<8:44:46,  1.92s/it]
 12%|█▏        | 2259/18627 [1:12:27<9:01:21,  1.98s/it]
 12%|█▏        | 2260/18627 [1:12:29<9:11:54,  2.02s/it]
                                                        
{'loss': 1.2269, 'grad_norm': 7.835499286651611, 'learning_rate': 4.89144994346519e-06, 'epoch': 0.12}
+
 12%|█▏        | 2260/18627 [1:12:29<9:11:54,  2.02s/it]
 12%|█▏        | 2261/18627 [1:12:31<8:03:17,  1.77s/it]
 12%|█▏        | 2262/18627 [1:12:33<8:30:57,  1.87s/it]
 12%|█▏        | 2263/18627 [1:12:35<8:52:02,  1.95s/it]
 12%|█▏        | 2264/18627 [1:12:37<9:06:07,  2.00s/it]
 12%|█▏        | 2265/18627 [1:12:39<9:16:13,  2.04s/it]
 12%|█▏        | 2266/18627 [1:12:41<9:21:39,  2.06s/it]
 12%|█▏        | 2267/18627 [1:12:43<9:28:02,  2.08s/it]
 12%|█▏        | 2268/18627 [1:12:46<9:32:10,  2.10s/it]
 12%|█▏        | 2269/18627 [1:12:48<9:37:36,  2.12s/it]
 12%|█▏        | 2270/18627 [1:12:50<9:38:21,  2.12s/it]
                                                        
{'loss': 0.9395, 'grad_norm': 18.65610694885254, 'learning_rate': 4.890179337806896e-06, 'epoch': 0.12}
+
 12%|█▏        | 2270/18627 [1:12:50<9:38:21,  2.12s/it]
 12%|█▏        | 2271/18627 [1:12:52<9:37:22,  2.12s/it]
 12%|█▏        | 2272/18627 [1:12:54<9:37:55,  2.12s/it]
 12%|█▏        | 2273/18627 [1:12:56<9:38:30,  2.12s/it]
 12%|█▏        | 2274/18627 [1:12:58<9:37:56,  2.12s/it]
 12%|█▏        | 2275/18627 [1:12:59<8:21:11,  1.84s/it]
 12%|█▏        | 2276/18627 [1:13:01<7:27:50,  1.64s/it]
 12%|█▏        | 2277/18627 [1:13:02<6:50:23,  1.51s/it]
 12%|█▏        | 2278/18627 [1:13:03<6:12:18,  1.37s/it]
 12%|█▏        | 2279/18627 [1:13:05<7:14:30,  1.59s/it]
 12%|█▏        | 2280/18627 [1:13:06<6:01:32,  1.33s/it]
                                                        
{'loss': 1.6445, 'grad_norm': 7.627039909362793, 'learning_rate': 4.888901505948892e-06, 'epoch': 0.12}
+
 12%|█▏        | 2280/18627 [1:13:06<6:01:32,  1.33s/it]
 12%|█▏        | 2281/18627 [1:13:08<7:06:44,  1.57s/it]
 12%|█▏        | 2282/18627 [1:13:10<7:52:35,  1.73s/it]
 12%|█��        | 2283/18627 [1:13:12<8:23:58,  1.85s/it]
 12%|█▏        | 2284/18627 [1:13:14<8:44:21,  1.93s/it]
 12%|█▏        | 2285/18627 [1:13:16<9:00:40,  1.99s/it]
 12%|█▏        | 2286/18627 [1:13:18<7:55:11,  1.74s/it]
 12%|█▏        | 2287/18627 [1:13:20<8:23:23,  1.85s/it]
 12%|█▏        | 2288/18627 [1:13:22<8:45:48,  1.93s/it]
 12%|█▏        | 2289/18627 [1:13:24<9:00:22,  1.98s/it]
 12%|█▏        | 2290/18627 [1:13:26<9:10:53,  2.02s/it]
                                                        
{'loss': 0.9958, 'grad_norm': 10.299637794494629, 'learning_rate': 4.88761645175443e-06, 'epoch': 0.12}
+
 12%|█▏        | 2290/18627 [1:13:26<9:10:53,  2.02s/it]
 12%|█▏        | 2291/18627 [1:13:28<9:19:21,  2.05s/it]
 12%|█▏        | 2292/18627 [1:13:29<7:29:19,  1.65s/it]
 12%|█▏        | 2293/18627 [1:13:31<8:08:53,  1.80s/it]
 12%|█▏        | 2294/18627 [1:13:33<8:35:11,  1.89s/it]
 12%|█▏        | 2295/18627 [1:13:34<7:37:46,  1.68s/it]
 12%|█▏        | 2296/18627 [1:13:36<8:16:50,  1.83s/it]
 12%|█▏        | 2297/18627 [1:13:38<7:25:33,  1.64s/it]
 12%|█▏        | 2298/18627 [1:13:39<6:48:45,  1.50s/it]
 12%|█▏        | 2299/18627 [1:13:41<7:41:16,  1.70s/it]
 12%|█▏        | 2300/18627 [1:13:43<8:16:01,  1.82s/it]
                                                        
{'loss': 1.3692, 'grad_norm': 11.802054405212402, 'learning_rate': 4.886324179108601e-06, 'epoch': 0.12}
+
 12%|█▏        | 2300/18627 [1:13:43<8:16:01,  1.82s/it]
 12%|█▏        | 2301/18627 [1:13:45<8:41:50,  1.92s/it]
 12%|█▏        | 2302/18627 [1:13:47<8:58:00,  1.98s/it]
 12%|█▏        | 2303/18627 [1:13:49<9:08:32,  2.02s/it]
 12%|█▏        | 2304/18627 [1:13:52<9:17:05,  2.05s/it]
 12%|█▏        | 2305/18627 [1:13:54<9:23:06,  2.07s/it]
 12%|█▏        | 2306/18627 [1:13:56<9:26:38,  2.08s/it]
 12%|█▏        | 2307/18627 [1:13:58<9:29:51,  2.10s/it]
 12%|█▏        | 2308/18627 [1:14:00<9:32:23,  2.10s/it]
 12%|█▏        | 2309/18627 [1:14:02<9:34:31,  2.11s/it]
 12%|█▏        | 2310/18627 [1:14:04<9:34:08,  2.11s/it]
                                                        
{'loss': 0.7659, 'grad_norm': 7.465868949890137, 'learning_rate': 4.8850246919183156e-06, 'epoch': 0.12}
+
 12%|█▏        | 2310/18627 [1:14:04<9:34:08,  2.11s/it]
 12%|█▏        | 2311/18627 [1:14:06<9:37:04,  2.12s/it]
 12%|█▏        | 2312/18627 [1:14:07<7:40:56,  1.70s/it]
 12%|█▏        | 2313/18627 [1:14:09<8:16:22,  1.83s/it]
 12%|█▏        | 2314/18627 [1:14:10<7:23:35,  1.63s/it]
 12%|█▏        | 2315/18627 [1:14:13<8:04:22,  1.78s/it]
 12%|█▏        | 2316/18627 [1:14:15<8:30:55,  1.88s/it]
 12%|█▏        | 2317/18627 [1:14:16<7:34:22,  1.67s/it]
 12%|█▏        | 2318/18627 [1:14:18<8:09:35,  1.80s/it]
 12%|█▏        | 2319/18627 [1:14:19<7:19:05,  1.62s/it]
 12%|█▏        | 2320/18627 [1:14:21<8:01:03,  1.77s/it]
                                                        
{'loss': 1.3204, 'grad_norm': 8.01265811920166, 'learning_rate': 4.883717994112297e-06, 'epoch': 0.12}
+
 12%|█▏        | 2320/18627 [1:14:21<8:01:03,  1.77s/it]
 12%|█▏        | 2321/18627 [1:14:23<8:29:39,  1.88s/it]
 12%|█▏        | 2322/18627 [1:14:25<8:49:50,  1.95s/it]
 12%|█▏        | 2323/18627 [1:14:28<9:04:41,  2.00s/it]
 12%|█▏        | 2324/18627 [1:14:29<7:43:47,  1.71s/it]
 12%|█▏        | 2325/18627 [1:14:31<8:16:53,  1.83s/it]
 12%|█▏        | 2326/18627 [1:14:33<8:42:49,  1.92s/it]
 12%|█▏        | 2327/18627 [1:14:35<8:58:52,  1.98s/it]
 12%|█▏        | 2328/18627 [1:14:37<9:10:08,  2.03s/it]
 13%|█▎        | 2329/18627 [1:14:39<9:17:39,  2.05s/it]
 13%|█▎        | 2330/18627 [1:14:41<9:21:09,  2.07s/it]
                                                        
{'loss': 0.9778, 'grad_norm': 18.70553970336914, 'learning_rate': 4.882404089641069e-06, 'epoch': 0.13}
+
 13%|█▎        | 2330/18627 [1:14:41<9:21:09,  2.07s/it]
 13%|█▎        | 2331/18627 [1:14:43<9:25:58,  2.08s/it]
 13%|█▎        | 2332/18627 [1:14:46<9:29:50,  2.10s/it]
 13%|█▎        | 2333/18627 [1:14:48<9:31:51,  2.11s/it]
 13%|█▎        | 2334/18627 [1:14:50<9:32:31,  2.11s/it]
 13%|█▎        | 2335/18627 [1:14:51<7:38:12,  1.69s/it]
 13%|█▎        | 2336/18627 [1:14:53<8:15:23,  1.82s/it]
 13%|█▎        | 2337/18627 [1:14:55<8:39:14,  1.91s/it]
 13%|█▎        | 2338/18627 [1:14:57<8:54:28,  1.97s/it]
 13%|█▎        | 2339/18627 [1:14:59<9:07:27,  2.02s/it]
 13%|█▎        | 2340/18627 [1:15:01<9:16:51,  2.05s/it]
                                                        
{'loss': 0.777, 'grad_norm': 8.817856788635254, 'learning_rate': 4.881082982476945e-06, 'epoch': 0.13}
+
 13%|█▎        | 2340/18627 [1:15:01<9:16:51,  2.05s/it]
 13%|█▎        | 2341/18627 [1:15:03<9:23:14,  2.08s/it]
 13%|█▎        | 2342/18627 [1:15:05<9:25:26,  2.08s/it]
 13%|█▎        | 2343/18627 [1:15:08<9:28:56,  2.10s/it]
 13%|█▎        | 2344/18627 [1:15:10<9:29:39,  2.10s/it]
 13%|█▎        | 2345/18627 [1:15:12<9:32:03,  2.11s/it]
 13%|█▎        | 2346/18627 [1:15:13<8:17:11,  1.83s/it]
 13%|█▎        | 2347/18627 [1:15:15<8:40:44,  1.92s/it]
 13%|█▎        | 2348/18627 [1:15:17<8:56:40,  1.98s/it]
 13%|█▎        | 2349/18627 [1:15:19<9:09:27,  2.03s/it]
 13%|█▎        | 2350/18627 [1:15:21<9:16:13,  2.05s/it]
                                                        
{'loss': 0.9001, 'grad_norm': 8.14919376373291, 'learning_rate': 4.879754676614008e-06, 'epoch': 0.13}
+
 13%|█▎        | 2350/18627 [1:15:21<9:16:13,  2.05s/it]
 13%|█▎        | 2351/18627 [1:15:23<8:07:08,  1.80s/it]
 13%|█▎        | 2352/18627 [1:15:25<8:34:18,  1.90s/it]
 13%|█▎        | 2353/18627 [1:15:27<8:51:19,  1.96s/it]
 13%|█▎        | 2354/18627 [1:15:29<9:04:21,  2.01s/it]
 13%|█▎        | 2355/18627 [1:15:31<9:12:52,  2.04s/it]
 13%|█▎        | 2356/18627 [1:15:33<9:19:27,  2.06s/it]
 13%|█▎        | 2357/18627 [1:15:35<9:24:20,  2.08s/it]
 13%|█▎        | 2358/18627 [1:15:37<8:11:46,  1.81s/it]
 13%|█▎        | 2359/18627 [1:15:39<8:34:51,  1.90s/it]
 13%|█▎        | 2360/18627 [1:15:41<8:52:22,  1.96s/it]
                                                        
{'loss': 1.1594, 'grad_norm': 6.689359188079834, 'learning_rate': 4.878419176068111e-06, 'epoch': 0.13}
+
 13%|█▎        | 2360/18627 [1:15:41<8:52:22,  1.96s/it]
 13%|█▎        | 2361/18627 [1:15:43<9:04:58,  2.01s/it]
 13%|█▎        | 2362/18627 [1:15:44<7:18:36,  1.62s/it]
 13%|█▎        | 2363/18627 [1:15:46<7:59:36,  1.77s/it]
 13%|█▎        | 2364/18627 [1:15:47<7:12:37,  1.60s/it]
 13%|█▎        | 2365/18627 [1:15:49<7:55:01,  1.75s/it]
 13%|█▎        | 2366/18627 [1:15:51<8:25:27,  1.87s/it]
 13%|█▎        | 2367/18627 [1:15:52<7:30:40,  1.66s/it]
 13%|█▎        | 2368/18627 [1:15:54<8:08:00,  1.80s/it]
 13%|█▎        | 2369/18627 [1:15:56<7:18:13,  1.62s/it]
 13%|█▎        | 2370/18627 [1:15:56<6:03:44,  1.34s/it]
                                                        
{'loss': 1.3507, 'grad_norm': 7.795644283294678, 'learning_rate': 4.877076484876856e-06, 'epoch': 0.13}
+
 13%|█▎        | 2370/18627 [1:15:56<6:03:44,  1.34s/it]
 13%|█▎        | 2371/18627 [1:15:58<5:51:15,  1.30s/it]
 13%|█▎        | 2372/18627 [1:16:00<6:57:25,  1.54s/it]
 13%|█▎        | 2373/18627 [1:16:02<7:44:50,  1.72s/it]
 13%|█▎        | 2374/18627 [1:16:03<7:02:03,  1.56s/it]
 13%|█▎        | 2375/18627 [1:16:05<7:46:54,  1.72s/it]
 13%|█▎        | 2376/18627 [1:16:07<8:19:10,  1.84s/it]
 13%|█▎        | 2377/18627 [1:16:09<8:41:42,  1.93s/it]
 13%|█▎        | 2378/18627 [1:16:11<8:58:26,  1.99s/it]
 13%|█▎        | 2379/18627 [1:16:14<9:09:57,  2.03s/it]
 13%|█▎        | 2380/18627 [1:16:15<8:01:45,  1.78s/it]
                                                        
{'loss': 1.3009, 'grad_norm': 18.106992721557617, 'learning_rate': 4.875726607099586e-06, 'epoch': 0.13}
+
 13%|█▎        | 2380/18627 [1:16:15<8:01:45,  1.78s/it]
 13%|█▎        | 2381/18627 [1:16:17<8:29:51,  1.88s/it]
 13%|█▎        | 2382/18627 [1:16:18<7:35:04,  1.68s/it]
 13%|█▎        | 2383/18627 [1:16:20<8:11:02,  1.81s/it]
 13%|█▎        | 2384/18627 [1:16:22<8:36:35,  1.91s/it]
 13%|█▎        | 2385/18627 [1:16:24<8:55:46,  1.98s/it]
 13%|█▎        | 2386/18627 [1:16:27<9:08:20,  2.03s/it]
 13%|█▎        | 2387/18627 [1:16:29<9:14:08,  2.05s/it]
 13%|█▎        | 2388/18627 [1:16:30<8:04:09,  1.79s/it]
 13%|█▎        | 2389/18627 [1:16:31<7:15:11,  1.61s/it]
 13%|█▎        | 2390/18627 [1:16:33<7:58:38,  1.77s/it]
                                                        
{'loss': 1.2801, 'grad_norm': 7.038511753082275, 'learning_rate': 4.874369546817367e-06, 'epoch': 0.13}
+
 13%|█▎        | 2390/18627 [1:16:33<7:58:38,  1.77s/it]
 13%|█▎        | 2391/18627 [1:16:35<8:26:59,  1.87s/it]
 13%|█▎        | 2392/18627 [1:16:37<8:47:04,  1.95s/it]
 13%|█▎        | 2393/18627 [1:16:40<9:00:07,  2.00s/it]
 13%|█▎        | 2394/18627 [1:16:41<7:54:42,  1.75s/it]
 13%|█▎        | 2395/18627 [1:16:43<8:25:37,  1.87s/it]
 13%|█▎        | 2396/18627 [1:16:45<8:46:51,  1.95s/it]
 13%|█▎        | 2397/18627 [1:16:47<9:00:04,  2.00s/it]
 13%|█▎        | 2398/18627 [1:16:49<9:12:02,  2.04s/it]
 13%|█▎        | 2399/18627 [1:16:51<9:19:27,  2.07s/it]
 13%|█▎        | 2400/18627 [1:16:54<9:22:09,  2.08s/it]
                                                        
{'loss': 0.9816, 'grad_norm': 8.319135665893555, 'learning_rate': 4.873005308132985e-06, 'epoch': 0.13}
+
 13%|█▎        | 2400/18627 [1:16:54<9:22:09,  2.08s/it]
 13%|█▎        | 2401/18627 [1:16:56<9:25:06,  2.09s/it]
 13%|█▎        | 2402/18627 [1:16:58<9:29:07,  2.10s/it]
 13%|█▎        | 2403/18627 [1:17:00<9:31:11,  2.11s/it]
 13%|█▎        | 2404/18627 [1:17:02<9:32:29,  2.12s/it]
 13%|█▎        | 2405/18627 [1:17:04<9:30:25,  2.11s/it]
 13%|█▎        | 2406/18627 [1:17:06<9:30:22,  2.11s/it]
 13%|█▎        | 2407/18627 [1:17:08<9:31:14,  2.11s/it]
 13%|█▎        | 2408/18627 [1:17:11<9:32:59,  2.12s/it]
 13%|█▎        | 2409/18627 [1:17:13<9:32:36,  2.12s/it]
 13%|█▎        | 2410/18627 [1:17:15<9:32:05,  2.12s/it]
                                                        
{'loss': 0.7602, 'grad_norm': 7.838551998138428, 'learning_rate': 4.871633895170924e-06, 'epoch': 0.13}
+
 13%|█▎        | 2410/18627 [1:17:15<9:32:05,  2.12s/it]
 13%|█▎        | 2411/18627 [1:17:16<8:10:33,  1.82s/it]
 13%|█▎        | 2412/18627 [1:17:18<8:37:15,  1.91s/it]
 13%|█▎        | 2413/18627 [1:17:20<8:55:00,  1.98s/it]
 13%|█▎        | 2414/18627 [1:17:22<9:05:49,  2.02s/it]
 13%|█▎        | 2415/18627 [1:17:23<7:58:15,  1.77s/it]
 13%|█▎        | 2416/18627 [1:17:26<8:26:22,  1.87s/it]
 13%|█▎        | 2417/18627 [1:17:27<7:31:26,  1.67s/it]
 13%|█▎        | 2418/18627 [1:17:29<8:05:45,  1.80s/it]
 13%|█▎        | 2419/18627 [1:17:31<8:32:10,  1.90s/it]
 13%|█▎        | 2420/18627 [1:17:33<8:49:43,  1.96s/it]
                                                        
{'loss': 1.3234, 'grad_norm': 8.532792091369629, 'learning_rate': 4.870255312077362e-06, 'epoch': 0.13}
+
 13%|█▎        | 2420/18627 [1:17:33<8:49:43,  1.96s/it]
 13%|█▎        | 2421/18627 [1:17:35<9:00:50,  2.00s/it]
 13%|█▎        | 2422/18627 [1:17:36<7:54:50,  1.76s/it]
 13%|█▎        | 2423/18627 [1:17:38<7:08:39,  1.59s/it]
 13%|█▎        | 2424/18627 [1:17:40<7:52:37,  1.75s/it]
 13%|█▎        | 2425/18627 [1:17:42<8:23:05,  1.86s/it]
 13%|█▎        | 2426/18627 [1:17:44<8:45:26,  1.95s/it]
 13%|█▎        | 2427/18627 [1:17:45<7:45:05,  1.72s/it]
 13%|█▎        | 2428/18627 [1:17:47<8:16:55,  1.84s/it]
 13%|█▎        | 2429/18627 [1:17:49<8:39:55,  1.93s/it]
 13%|█▎        | 2430/18627 [1:17:52<8:55:07,  1.98s/it]
                                                        
{'loss': 1.3425, 'grad_norm': 7.035489082336426, 'learning_rate': 4.868869563020152e-06, 'epoch': 0.13}
+
 13%|█▎        | 2430/18627 [1:17:52<8:55:07,  1.98s/it]
 13%|█▎        | 2431/18627 [1:17:54<9:06:20,  2.02s/it]
 13%|█▎        | 2432/18627 [1:17:56<9:11:57,  2.04s/it]
 13%|█▎        | 2433/18627 [1:17:58<9:20:51,  2.08s/it]
 13%|█▎        | 2434/18627 [1:18:00<9:25:18,  2.09s/it]
 13%|█▎        | 2435/18627 [1:18:01<8:11:59,  1.82s/it]
 13%|█▎        | 2436/18627 [1:18:02<7:06:08,  1.58s/it]
 13%|█▎        | 2437/18627 [1:18:04<7:48:43,  1.74s/it]
 13%|█▎        | 2438/18627 [1:18:06<8:20:28,  1.85s/it]
 13%|█▎        | 2439/18627 [1:18:09<8:43:14,  1.94s/it]
 13%|█▎        | 2440/18627 [1:18:11<8:58:24,  2.00s/it]
                                                        
{'loss': 1.1401, 'grad_norm': 10.039881706237793, 'learning_rate': 4.86747665218881e-06, 'epoch': 0.13}
+
 13%|█▎        | 2440/18627 [1:18:11<8:58:24,  2.00s/it]
 13%|█▎        | 2441/18627 [1:18:13<9:08:27,  2.03s/it]
 13%|█▎        | 2442/18627 [1:18:15<9:14:49,  2.06s/it]
 13%|█▎        | 2443/18627 [1:18:16<7:58:17,  1.77s/it]
 13%|█▎        | 2444/18627 [1:18:17<7:11:12,  1.60s/it]
 13%|█▎        | 2445/18627 [1:18:19<7:51:55,  1.75s/it]
 13%|█▎        | 2446/18627 [1:18:20<6:27:59,  1.44s/it]
 13%|█▎        | 2447/18627 [1:18:21<6:08:18,  1.37s/it]
 13%|█▎        | 2448/18627 [1:18:23<7:09:36,  1.59s/it]
 13%|█▎        | 2449/18627 [1:18:26<7:53:10,  1.75s/it]
 13%|█▎        | 2450/18627 [1:18:27<7:07:50,  1.59s/it]
                                                        
{'loss': 1.527, 'grad_norm': 15.929950714111328, 'learning_rate': 4.866076583794508e-06, 'epoch': 0.13}
+
 13%|█▎        | 2450/18627 [1:18:27<7:07:50,  1.59s/it]
 13%|█▎        | 2451/18627 [1:18:29<7:49:31,  1.74s/it]
 13%|█▎        | 2452/18627 [1:18:30<7:05:32,  1.58s/it]
 13%|█▎        | 2453/18627 [1:18:32<7:48:05,  1.74s/it]
 13%|█▎        | 2454/18627 [1:18:34<8:19:39,  1.85s/it]
 13%|█▎        | 2455/18627 [1:18:36<8:39:56,  1.93s/it]
 13%|█▎        | 2456/18627 [1:18:38<7:40:08,  1.71s/it]
 13%|█▎        | 2457/18627 [1:18:40<8:13:42,  1.83s/it]
 13%|█▎        | 2458/18627 [1:18:42<8:37:34,  1.92s/it]
 13%|█▎        | 2459/18627 [1:18:43<7:40:01,  1.71s/it]
 13%|█▎        | 2460/18627 [1:18:44<6:19:11,  1.41s/it]
                                                        
{'loss': 1.3244, 'grad_norm': 8.255677223205566, 'learning_rate': 4.864669362070055e-06, 'epoch': 0.13}
+
 13%|█▎        | 2460/18627 [1:18:44<6:19:11,  1.41s/it]
 13%|█▎        | 2461/18627 [1:18:45<5:58:23,  1.33s/it]
 13%|█▎        | 2462/18627 [1:18:47<7:01:56,  1.57s/it]
 13%|█▎        | 2463/18627 [1:18:48<6:31:15,  1.45s/it]
 13%|█▎        | 2464/18627 [1:18:50<7:25:04,  1.65s/it]
 13%|█▎        | 2465/18627 [1:18:52<8:04:18,  1.80s/it]
 13%|█▎        | 2466/18627 [1:18:55<8:28:55,  1.89s/it]
 13%|█▎        | 2467/18627 [1:18:55<6:52:05,  1.53s/it]
 13%|█▎        | 2468/18627 [1:18:57<7:39:53,  1.71s/it]
 13%|█▎        | 2469/18627 [1:18:59<8:13:02,  1.83s/it]
 13%|█▎        | 2470/18627 [1:19:02<8:35:51,  1.92s/it]
                                                        
{'loss': 1.1138, 'grad_norm': 7.576117038726807, 'learning_rate': 4.863254991269888e-06, 'epoch': 0.13}
+
 13%|█▎        | 2470/18627 [1:19:02<8:35:51,  1.92s/it]
 13%|█▎        | 2471/18627 [1:19:02<6:58:27,  1.55s/it]
 13%|█▎        | 2472/18627 [1:19:04<7:43:16,  1.72s/it]
 13%|█▎        | 2473/18627 [1:19:07<8:16:36,  1.84s/it]
 13%|█▎        | 2474/18627 [1:19:09<8:41:03,  1.94s/it]
 13%|█▎        | 2475/18627 [1:19:11<8:57:54,  2.00s/it]
 13%|█▎        | 2476/18627 [1:19:13<9:08:10,  2.04s/it]
 13%|█▎        | 2477/18627 [1:19:14<7:54:11,  1.76s/it]
 13%|█▎        | 2478/18627 [1:19:16<8:24:15,  1.87s/it]
 13%|█▎        | 2479/18627 [1:19:18<8:42:27,  1.94s/it]
 13%|█▎        | 2480/18627 [1:19:20<8:56:09,  1.99s/it]
                                                        
{'loss': 0.9075, 'grad_norm': 8.603239059448242, 'learning_rate': 4.861833475670055e-06, 'epoch': 0.13}
+
 13%|█▎        | 2480/18627 [1:19:20<8:56:09,  1.99s/it]
 13%|█▎        | 2481/18627 [1:19:22<7:51:52,  1.75s/it]
 13%|█▎        | 2482/18627 [1:19:24<8:21:31,  1.86s/it]
 13%|█▎        | 2483/18627 [1:19:26<8:41:57,  1.94s/it]
 13%|█▎        | 2484/18627 [1:19:27<7:41:16,  1.71s/it]
 13%|█▎        | 2485/18627 [1:19:31<10:05:23,  2.25s/it]
 13%|█▎        | 2486/18627 [1:19:33<9:54:39,  2.21s/it] 
 13%|█▎        | 2487/18627 [1:19:35<9:46:24,  2.18s/it]
 13%|█▎        | 2488/18627 [1:19:37<9:45:45,  2.18s/it]
 13%|█▎        | 2489/18627 [1:19:39<9:41:41,  2.16s/it]
 13%|█▎        | 2490/18627 [1:19:41<9:37:14,  2.15s/it]
                                                        
{'loss': 1.1651, 'grad_norm': 7.866832733154297, 'learning_rate': 4.8604048195682065e-06, 'epoch': 0.13}
+
 13%|█▎        | 2490/18627 [1:19:41<9:37:14,  2.15s/it]
 13%|█▎        | 2491/18627 [1:19:43<9:34:18,  2.14s/it]
 13%|█▎        | 2492/18627 [1:19:45<9:32:15,  2.13s/it]
 13%|█▎        | 2493/18627 [1:19:47<9:30:54,  2.12s/it]
 13%|█▎        | 2494/18627 [1:19:48<7:36:14,  1.70s/it]
 13%|█▎        | 2495/18627 [1:19:50<8:08:31,  1.82s/it]
 13%|█▎        | 2496/18627 [1:19:51<7:18:40,  1.63s/it]
 13%|█▎        | 2497/18627 [1:19:54<7:57:35,  1.78s/it]
 13%|█▎        | 2498/18627 [1:19:55<7:10:16,  1.60s/it]
 13%|█▎        | 2499/18627 [1:19:57<7:51:20,  1.75s/it]
 13%|█▎        | 2500/18627 [1:19:58<7:05:25,  1.58s/it]
                                                        
{'loss': 1.3481, 'grad_norm': 15.445428848266602, 'learning_rate': 4.858969027283583e-06, 'epoch': 0.13}
+
 13%|█▎        | 2500/18627 [1:19:58<7:05:25,  1.58s/it]
 13%|█▎        | 2501/18627 [1:20:00<7:47:33,  1.74s/it]
 13%|█▎        | 2502/18627 [1:20:02<8:18:13,  1.85s/it]
 13%|█▎        | 2503/18627 [1:20:04<8:37:06,  1.92s/it]
 13%|█▎        | 2504/18627 [1:20:07<8:52:24,  1.98s/it]
 13%|█▎        | 2505/18627 [1:20:09<9:02:46,  2.02s/it]
 13%|█▎        | 2506/18627 [1:20:10<7:56:09,  1.77s/it]
 13%|█▎        | 2507/18627 [1:20:12<8:24:44,  1.88s/it]
 13%|█▎        | 2508/18627 [1:20:13<7:30:01,  1.68s/it]
 13%|█▎        | 2509/18627 [1:20:15<8:05:37,  1.81s/it]
 13%|█▎        | 2510/18627 [1:20:17<8:29:36,  1.90s/it]
                                                        
{'loss': 1.2102, 'grad_norm': 8.422149658203125, 'learning_rate': 4.857526103156997e-06, 'epoch': 0.13}
+
 13%|█▎        | 2510/18627 [1:20:17<8:29:36,  1.90s/it]
 13%|█▎        | 2511/18627 [1:20:19<7:32:27,  1.68s/it]
 13%|█▎        | 2512/18627 [1:20:20<6:52:13,  1.53s/it]
 13%|█▎        | 2513/18627 [1:20:22<7:39:12,  1.71s/it]
 13%|█▎        | 2514/18627 [1:20:23<6:56:39,  1.55s/it]
 14%|█▎        | 2515/18627 [1:20:24<6:27:24,  1.44s/it]
 14%|█▎        | 2516/18627 [1:20:26<7:24:01,  1.65s/it]
 14%|█▎        | 2517/18627 [1:20:28<7:59:54,  1.79s/it]
 14%|█▎        | 2518/18627 [1:20:31<8:25:26,  1.88s/it]
 14%|█▎        | 2519/18627 [1:20:33<8:46:06,  1.96s/it]
 14%|█▎        | 2520/18627 [1:20:35<9:00:51,  2.01s/it]
                                                        
{'loss': 1.4626, 'grad_norm': 8.69704532623291, 'learning_rate': 4.856076051550821e-06, 'epoch': 0.14}
+
 14%|█▎        | 2520/18627 [1:20:35<9:00:51,  2.01s/it]
 14%|█▎        | 2521/18627 [1:20:37<9:09:36,  2.05s/it]
 14%|█▎        | 2522/18627 [1:20:39<9:14:03,  2.06s/it]
 14%|█▎        | 2523/18627 [1:20:40<8:03:36,  1.80s/it]
 14%|█▎        | 2524/18627 [1:20:42<8:30:17,  1.90s/it]
 14%|█▎        | 2525/18627 [1:20:45<8:48:56,  1.97s/it]
 14%|█▎        | 2526/18627 [1:20:46<7:46:19,  1.74s/it]
 14%|█▎        | 2527/18627 [1:20:48<8:16:15,  1.85s/it]
 14%|█▎        | 2528/18627 [1:20:50<8:36:44,  1.93s/it]
 14%|█▎        | 2529/18627 [1:20:52<8:51:55,  1.98s/it]
 14%|█▎        | 2530/18627 [1:20:54<9:02:09,  2.02s/it]
                                                        
{'loss': 1.2638, 'grad_norm': 13.394875526428223, 'learning_rate': 4.854618876848981e-06, 'epoch': 0.14}
+
 14%|█▎        | 2530/18627 [1:20:54<9:02:09,  2.02s/it]
 14%|█▎        | 2531/18627 [1:20:56<9:11:09,  2.05s/it]
 14%|█▎        | 2532/18627 [1:20:58<9:16:57,  2.08s/it]
 14%|█▎        | 2533/18627 [1:21:01<9:19:25,  2.09s/it]
 14%|█▎        | 2534/18627 [1:21:03<9:25:20,  2.11s/it]
 14%|█▎        | 2535/18627 [1:21:04<8:11:15,  1.83s/it]
 14%|█▎        | 2536/18627 [1:21:06<8:33:30,  1.91s/it]
 14%|█▎        | 2537/18627 [1:21:08<8:49:46,  1.98s/it]
 14%|█▎        | 2538/18627 [1:21:10<9:02:04,  2.02s/it]
 14%|█▎        | 2539/18627 [1:21:12<9:10:40,  2.05s/it]
 14%|█▎        | 2540/18627 [1:21:14<9:14:39,  2.07s/it]
                                                        
{'loss': 0.851, 'grad_norm': 10.491499900817871, 'learning_rate': 4.853154583456933e-06, 'epoch': 0.14}
+
 14%|█▎        | 2540/18627 [1:21:14<9:14:39,  2.07s/it]
 14%|█▎        | 2541/18627 [1:21:17<9:19:24,  2.09s/it]
 14%|█▎        | 2542/18627 [1:21:19<9:23:13,  2.10s/it]
 14%|█▎        | 2543/18627 [1:21:21<9:23:33,  2.10s/it]
 14%|█▎        | 2544/18627 [1:21:22<8:10:22,  1.83s/it]
 14%|█▎        | 2545/18627 [1:21:24<8:34:06,  1.92s/it]
 14%|█▎        | 2546/18627 [1:21:26<8:51:40,  1.98s/it]
 14%|█▎        | 2547/18627 [1:21:28<9:01:13,  2.02s/it]
 14%|█▎        | 2548/18627 [1:21:30<7:54:37,  1.77s/it]
 14%|█▎        | 2549/18627 [1:21:32<8:24:23,  1.88s/it]
 14%|█▎        | 2550/18627 [1:21:34<8:46:32,  1.97s/it]
                                                        
{'loss': 1.174, 'grad_norm': 10.67133903503418, 'learning_rate': 4.851683175801659e-06, 'epoch': 0.14}
+
 14%|█▎        | 2550/18627 [1:21:34<8:46:32,  1.97s/it]
 14%|█▎        | 2551/18627 [1:21:36<8:59:34,  2.01s/it]
 14%|█▎        | 2552/18627 [1:21:38<9:09:10,  2.05s/it]
 14%|█▎        | 2553/18627 [1:21:40<9:15:16,  2.07s/it]
 14%|█▎        | 2554/18627 [1:21:42<9:19:38,  2.09s/it]
 14%|█▎        | 2555/18627 [1:21:44<8:07:24,  1.82s/it]
 14%|█▎        | 2556/18627 [1:21:44<6:39:03,  1.49s/it]
 14%|█▎        | 2557/18627 [1:21:46<7:27:38,  1.67s/it]
 14%|█▎        | 2558/18627 [1:21:49<8:03:18,  1.80s/it]
 14%|█▎        | 2559/18627 [1:21:51<8:28:06,  1.90s/it]
 14%|█▎        | 2560/18627 [1:21:52<7:31:38,  1.69s/it]
                                                        
{'loss': 1.1255, 'grad_norm': 15.081131935119629, 'learning_rate': 4.850204658331647e-06, 'epoch': 0.14}
+
 14%|█▎        | 2560/18627 [1:21:52<7:31:38,  1.69s/it]
 14%|█▎        | 2561/18627 [1:21:54<8:08:14,  1.82s/it]
 14%|█▍        | 2562/18627 [1:21:56<8:31:36,  1.91s/it]
 14%|█▍        | 2563/18627 [1:21:58<8:48:46,  1.98s/it]
 14%|█▍        | 2564/18627 [1:22:00<9:01:25,  2.02s/it]
 14%|█▍        | 2565/18627 [1:22:02<7:55:12,  1.78s/it]
 14%|█▍        | 2566/18627 [1:22:04<8:23:33,  1.88s/it]
 14%|█▍        | 2567/18627 [1:22:06<8:43:47,  1.96s/it]
 14%|█▍        | 2568/18627 [1:22:08<8:57:21,  2.01s/it]
 14%|█▍        | 2569/18627 [1:22:10<9:06:24,  2.04s/it]
 14%|█▍        | 2570/18627 [1:22:12<9:13:21,  2.07s/it]
                                                        
{'loss': 1.0273, 'grad_norm': 9.563088417053223, 'learning_rate': 4.84871903551688e-06, 'epoch': 0.14}
+
 14%|█▍        | 2570/18627 [1:22:12<9:13:21,  2.07s/it]
 14%|█▍        | 2571/18627 [1:22:14<9:18:11,  2.09s/it]
 14%|█▍        | 2572/18627 [1:22:16<9:20:58,  2.10s/it]
 14%|█▍        | 2573/18627 [1:22:19<9:22:39,  2.10s/it]
 14%|█▍        | 2574/18627 [1:22:21<9:25:27,  2.11s/it]
 14%|█▍        | 2575/18627 [1:22:23<9:27:02,  2.12s/it]
 14%|█▍        | 2576/18627 [1:22:25<9:26:34,  2.12s/it]
 14%|█▍        | 2577/18627 [1:22:27<9:26:28,  2.12s/it]
 14%|█▍        | 2578/18627 [1:22:29<9:27:16,  2.12s/it]
 14%|█▍        | 2579/18627 [1:22:31<9:28:43,  2.13s/it]
 14%|█▍        | 2580/18627 [1:22:33<9:29:07,  2.13s/it]
                                                        
{'loss': 0.731, 'grad_norm': 8.483844757080078, 'learning_rate': 4.847226311848824e-06, 'epoch': 0.14}
+
 14%|█▍        | 2580/18627 [1:22:33<9:29:07,  2.13s/it]
 14%|█▍        | 2581/18627 [1:22:34<7:34:51,  1.70s/it]
 14%|█▍        | 2582/18627 [1:22:36<8:07:46,  1.82s/it]
 14%|█▍        | 2583/18627 [1:22:38<8:31:42,  1.91s/it]
 14%|█▍        | 2584/18627 [1:22:41<8:47:02,  1.97s/it]
 14%|█▍        | 2585/18627 [1:22:43<8:59:57,  2.02s/it]
 14%|█▍        | 2586/18627 [1:22:44<7:52:56,  1.77s/it]
 14%|█▍        | 2587/18627 [1:22:45<6:27:26,  1.45s/it]
 14%|█▍        | 2588/18627 [1:22:47<7:20:28,  1.65s/it]
 14%|█▍        | 2589/18627 [1:22:48<6:43:19,  1.51s/it]
 14%|█▍        | 2590/18627 [1:22:49<6:18:09,  1.41s/it]
                                                        
{'loss': 1.4118, 'grad_norm': 17.15674591064453, 'learning_rate': 4.845726491840411e-06, 'epoch': 0.14}
+
 14%|█▍        | 2590/18627 [1:22:49<6:18:09,  1.41s/it]
 14%|█▍        | 2591/18627 [1:22:51<7:13:16,  1.62s/it]
 14%|█▍        | 2592/18627 [1:22:53<7:54:06,  1.77s/it]
 14%|█▍        | 2593/18627 [1:22:55<8:20:44,  1.87s/it]
 14%|█▍        | 2594/18627 [1:22:57<7:25:44,  1.67s/it]
 14%|█▍        | 2595/18627 [1:22:59<8:03:02,  1.81s/it]
 14%|█▍        | 2596/18627 [1:23:01<8:30:03,  1.91s/it]
 14%|█▍        | 2597/18627 [1:23:03<8:47:17,  1.97s/it]
 14%|█▍        | 2598/18627 [1:23:04<7:44:32,  1.74s/it]
 14%|█▍        | 2599/18627 [1:23:06<8:15:47,  1.86s/it]
 14%|█▍        | 2600/18627 [1:23:07<7:22:26,  1.66s/it]
                                                        
{'loss': 1.3442, 'grad_norm': 16.473581314086914, 'learning_rate': 4.84421958002603e-06, 'epoch': 0.14}
+
 14%|█▍        | 2600/18627 [1:23:07<7:22:26,  1.66s/it]
 14%|█▍        | 2601/18627 [1:23:10<8:00:59,  1.80s/it]
 14%|█▍        | 2602/18627 [1:23:12<8:27:31,  1.90s/it]
 14%|█▍        | 2603/18627 [1:23:14<8:43:59,  1.96s/it]
 14%|█▍        | 2604/18627 [1:23:16<8:56:17,  2.01s/it]
 14%|█▍        | 2605/18627 [1:23:18<9:04:49,  2.04s/it]
 14%|█▍        | 2606/18627 [1:23:20<9:12:10,  2.07s/it]
 14%|█▍        | 2607/18627 [1:23:22<9:15:23,  2.08s/it]
 14%|█▍        | 2608/18627 [1:23:24<9:19:09,  2.09s/it]
 14%|█▍        | 2609/18627 [1:23:27<9:22:03,  2.11s/it]
 14%|█▍        | 2610/18627 [1:23:29<9:23:50,  2.11s/it]
                                                        
{'loss': 0.7458, 'grad_norm': 9.595731735229492, 'learning_rate': 4.842705580961507e-06, 'epoch': 0.14}
+
 14%|█▍        | 2610/18627 [1:23:29<9:23:50,  2.11s/it]
 14%|█▍        | 2611/18627 [1:23:31<9:26:43,  2.12s/it]
 14%|█▍        | 2612/18627 [1:23:33<9:27:28,  2.13s/it]
 14%|█▍        | 2613/18627 [1:23:35<9:25:35,  2.12s/it]
 14%|█▍        | 2614/18627 [1:23:36<8:10:58,  1.84s/it]
 14%|█▍        | 2615/18627 [1:23:37<7:10:33,  1.61s/it]
 14%|█▍        | 2616/18627 [1:23:39<7:51:47,  1.77s/it]
 14%|█▍        | 2617/18627 [1:23:42<8:19:42,  1.87s/it]
 14%|█▍        | 2618/18627 [1:23:44<8:38:56,  1.94s/it]
 14%|█▍        | 2619/18627 [1:23:46<8:53:25,  2.00s/it]
 14%|█▍        | 2620/18627 [1:23:48<9:02:24,  2.03s/it]
                                                        
{'loss': 1.0499, 'grad_norm': 10.634049415588379, 'learning_rate': 4.841184499224098e-06, 'epoch': 0.14}
+
 14%|█▍        | 2620/18627 [1:23:48<9:02:24,  2.03s/it]
 14%|█▍        | 2621/18627 [1:23:49<7:55:54,  1.78s/it]
 14%|█▍        | 2622/18627 [1:23:51<8:23:32,  1.89s/it]
 14%|█▍        | 2623/18627 [1:23:52<7:27:08,  1.68s/it]
 14%|█▍        | 2624/18627 [1:23:55<8:03:19,  1.81s/it]
 14%|█▍        | 2625/18627 [1:23:57<8:28:56,  1.91s/it]
 14%|█▍        | 2626/18627 [1:23:59<8:46:15,  1.97s/it]
 14%|█▍        | 2627/18627 [1:24:01<8:58:54,  2.02s/it]
 14%|█▍        | 2628/18627 [1:24:03<9:04:58,  2.04s/it]
 14%|█▍        | 2629/18627 [1:24:04<7:56:30,  1.79s/it]
 14%|█▍        | 2630/18627 [1:24:05<7:08:23,  1.61s/it]
                                                        
{'loss': 1.4751, 'grad_norm': 14.892288208007812, 'learning_rate': 4.83965633941247e-06, 'epoch': 0.14}
+
 14%|█▍        | 2630/18627 [1:24:05<7:08:23,  1.61s/it]
 14%|█▍        | 2631/18627 [1:24:07<6:34:35,  1.48s/it]
 14%|█▍        | 2632/18627 [1:24:09<7:24:44,  1.67s/it]
 14%|█▍        | 2633/18627 [1:24:11<7:59:38,  1.80s/it]
 14%|█▍        | 2634/18627 [1:24:13<8:24:59,  1.89s/it]
 14%|█▍        | 2635/18627 [1:24:14<7:28:11,  1.68s/it]
 14%|█▍        | 2636/18627 [1:24:16<8:01:46,  1.81s/it]
 14%|█▍        | 2637/18627 [1:24:17<6:32:50,  1.47s/it]
 14%|█▍        | 2638/18627 [1:24:19<7:25:09,  1.67s/it]
 14%|█▍        | 2639/18627 [1:24:21<8:00:50,  1.80s/it]
 14%|█▍        | 2640/18627 [1:24:22<7:11:36,  1.62s/it]
                                                        
{'loss': 1.2603, 'grad_norm': 14.603962898254395, 'learning_rate': 4.8381211061466904e-06, 'epoch': 0.14}
+
 14%|█▍        | 2640/18627 [1:24:22<7:11:36,  1.62s/it]
 14%|█▍        | 2641/18627 [1:24:25<7:50:45,  1.77s/it]
 14%|█▍        | 2642/18627 [1:24:27<8:18:57,  1.87s/it]
 14%|█▍        | 2643/18627 [1:24:28<7:25:50,  1.67s/it]
 14%|█▍        | 2644/18627 [1:24:30<8:03:21,  1.81s/it]
 14%|█▍        | 2645/18627 [1:24:32<8:28:34,  1.91s/it]
 14%|█▍        | 2646/18627 [1:24:34<8:46:39,  1.98s/it]
 14%|█▍        | 2647/18627 [1:24:35<7:44:00,  1.74s/it]
 14%|█▍        | 2648/18627 [1:24:38<8:13:06,  1.85s/it]
 14%|█▍        | 2649/18627 [1:24:40<8:32:48,  1.93s/it]
 14%|█▍        | 2650/18627 [1:24:42<8:49:06,  1.99s/it]
                                                        
{'loss': 1.1345, 'grad_norm': 12.587336540222168, 'learning_rate': 4.83657880406821e-06, 'epoch': 0.14}
+
 14%|█▍        | 2650/18627 [1:24:42<8:49:06,  1.99s/it]
 14%|█▍        | 2651/18627 [1:24:44<8:59:24,  2.03s/it]
 14%|█▍        | 2652/18627 [1:24:46<9:08:13,  2.06s/it]
 14%|█▍        | 2653/18627 [1:24:48<9:11:55,  2.07s/it]
 14%|█▍        | 2654/18627 [1:24:50<9:16:06,  2.09s/it]
 14%|█▍        | 2655/18627 [1:24:52<9:17:28,  2.09s/it]
 14%|█▍        | 2656/18627 [1:24:55<9:23:39,  2.12s/it]
 14%|█▍        | 2657/18627 [1:24:57<9:23:09,  2.12s/it]
 14%|█▍        | 2658/18627 [1:24:58<8:09:49,  1.84s/it]
 14%|█▍        | 2659/18627 [1:25:00<8:33:55,  1.93s/it]
 14%|█▍        | 2660/18627 [1:25:02<8:47:46,  1.98s/it]
                                                        
{'loss': 0.9015, 'grad_norm': 12.996410369873047, 'learning_rate': 4.835029437839852e-06, 'epoch': 0.14}
+
 14%|█▍        | 2660/18627 [1:25:02<8:47:46,  1.98s/it]
 14%|█▍        | 2661/18627 [1:25:04<9:00:01,  2.03s/it]
 14%|█▍        | 2662/18627 [1:25:06<9:06:12,  2.05s/it]
 14%|█▍        | 2663/18627 [1:25:08<9:12:33,  2.08s/it]
 14%|█▍        | 2664/18627 [1:25:12<10:55:06,  2.46s/it]
 14%|█▍        | 2665/18627 [1:25:14<10:26:38,  2.36s/it]
 14%|█▍        | 2666/18627 [1:25:16<10:05:48,  2.28s/it]
 14%|█▍        | 2667/18627 [1:25:17<8:39:02,  1.95s/it] 
 14%|█▍        | 2668/18627 [1:25:19<8:51:37,  2.00s/it]
 14%|█▍        | 2669/18627 [1:25:21<9:01:29,  2.04s/it]
 14%|█▍        | 2670/18627 [1:25:24<9:06:29,  2.05s/it]
                                                        
{'loss': 0.9886, 'grad_norm': 8.754450798034668, 'learning_rate': 4.8334730121457976e-06, 'epoch': 0.14}
+
 14%|█▍        | 2670/18627 [1:25:24<9:06:29,  2.05s/it]
 14%|█▍        | 2671/18627 [1:25:26<9:11:48,  2.08s/it]
 14%|█▍        | 2672/18627 [1:25:28<9:15:44,  2.09s/it]
 14%|█▍        | 2673/18627 [1:25:30<9:18:07,  2.10s/it]
 14%|█▍        | 2674/18627 [1:25:31<8:05:37,  1.83s/it]
 14%|█▍        | 2675/18627 [1:25:33<8:26:22,  1.90s/it]
 14%|█▍        | 2676/18627 [1:25:35<8:42:55,  1.97s/it]
 14%|█▍        | 2677/18627 [1:25:37<8:54:40,  2.01s/it]
 14%|█▍        | 2678/18627 [1:25:40<9:01:42,  2.04s/it]
 14%|█▍        | 2679/18627 [1:25:42<9:09:14,  2.07s/it]
 14%|█▍        | 2680/18627 [1:25:44<9:12:03,  2.08s/it]
                                                        
{'loss': 0.9807, 'grad_norm': 8.651375770568848, 'learning_rate': 4.831909531691566e-06, 'epoch': 0.14}
+
 14%|█▍        | 2680/18627 [1:25:44<9:12:03,  2.08s/it]
 14%|█▍        | 2681/18627 [1:25:46<9:12:54,  2.08s/it]
 14%|█▍        | 2682/18627 [1:25:48<9:17:11,  2.10s/it]
 14%|█▍        | 2683/18627 [1:25:49<8:04:46,  1.82s/it]
 14%|█▍        | 2684/18627 [1:25:51<8:30:28,  1.92s/it]
 14%|█▍        | 2685/18627 [1:25:53<8:47:10,  1.98s/it]
 14%|█▍        | 2686/18627 [1:25:55<7:44:12,  1.75s/it]
 14%|█▍        | 2687/18627 [1:25:57<8:14:57,  1.86s/it]
 14%|█▍        | 2688/18627 [1:25:58<7:21:49,  1.66s/it]
 14%|█▍        | 2689/18627 [1:26:00<7:59:03,  1.80s/it]
 14%|█▍        | 2690/18627 [1:26:02<8:24:56,  1.90s/it]
                                                        
{'loss': 1.4295, 'grad_norm': 7.417424201965332, 'learning_rate': 4.830339001204013e-06, 'epoch': 0.14}
+
 14%|█▍        | 2690/18627 [1:26:02<8:24:56,  1.90s/it]
 14%|█▍        | 2691/18627 [1:26:05<9:17:29,  2.10s/it]
 14%|█▍        | 2692/18627 [1:26:07<9:19:23,  2.11s/it]
 14%|█▍        | 2693/18627 [1:26:09<9:21:38,  2.11s/it]
 14%|█▍        | 2694/18627 [1:26:11<9:22:14,  2.12s/it]
 14%|█▍        | 2695/18627 [1:26:13<9:22:19,  2.12s/it]
 14%|█▍        | 2696/18627 [1:26:15<9:21:32,  2.11s/it]
 14%|█▍        | 2697/18627 [1:26:18<9:21:07,  2.11s/it]
 14%|█▍        | 2698/18627 [1:26:20<9:21:39,  2.12s/it]
 14%|█▍        | 2699/18627 [1:26:22<9:24:24,  2.13s/it]
 14%|█▍        | 2700/18627 [1:26:23<8:09:09,  1.84s/it]
                                                        
{'loss': 1.0203, 'grad_norm': 14.969225883483887, 'learning_rate': 4.8287614254313e-06, 'epoch': 0.14}
+
 14%|█▍        | 2700/18627 [1:26:23<8:09:09,  1.84s/it]
 15%|█▍        | 2701/18627 [1:26:25<8:30:34,  1.92s/it]
 15%|█▍        | 2702/18627 [1:26:28<10:19:07,  2.33s/it]
 15%|█▍        | 2703/18627 [1:26:30<10:02:34,  2.27s/it]
 15%|█▍        | 2704/18627 [1:26:33<9:50:17,  2.22s/it] 
 15%|█▍        | 2705/18627 [1:26:35<9:41:10,  2.19s/it]
 15%|█▍        | 2706/18627 [1:26:37<9:36:10,  2.17s/it]
 15%|█▍        | 2707/18627 [1:26:39<9:31:42,  2.15s/it]
 15%|█▍        | 2708/18627 [1:26:41<9:29:02,  2.14s/it]
 15%|█▍        | 2709/18627 [1:26:42<8:13:47,  1.86s/it]
 15%|█▍        | 2710/18627 [1:26:44<8:33:22,  1.94s/it]
                                                        
{'loss': 1.0072, 'grad_norm': 9.608999252319336, 'learning_rate': 4.827176809142895e-06, 'epoch': 0.15}
+
 15%|█▍        | 2710/18627 [1:26:44<8:33:22,  1.94s/it]
 15%|█▍        | 2711/18627 [1:26:47<8:48:51,  1.99s/it]
 15%|█▍        | 2712/18627 [1:26:49<8:59:41,  2.03s/it]
 15%|█▍        | 2713/18627 [1:26:51<9:07:15,  2.06s/it]
 15%|█▍        | 2714/18627 [1:26:52<7:57:12,  1.80s/it]
 15%|█▍        | 2715/18627 [1:26:54<8:22:04,  1.89s/it]
 15%|█▍        | 2716/18627 [1:26:56<8:39:55,  1.96s/it]
 15%|█▍        | 2717/18627 [1:26:58<8:52:55,  2.01s/it]
 15%|█▍        | 2718/18627 [1:27:00<9:01:24,  2.04s/it]
 15%|█▍        | 2719/18627 [1:27:03<9:07:04,  2.06s/it]
 15%|█▍        | 2720/18627 [1:27:04<7:56:57,  1.80s/it]
                                                        
{'loss': 1.0366, 'grad_norm': 15.650749206542969, 'learning_rate': 4.825585157129547e-06, 'epoch': 0.15}
+
 15%|█▍        | 2720/18627 [1:27:04<7:56:57,  1.80s/it]
 15%|█▍        | 2721/18627 [1:27:05<7:08:40,  1.62s/it]
 15%|█▍        | 2722/18627 [1:27:07<7:47:07,  1.76s/it]
 15%|█▍        | 2723/18627 [1:27:08<7:01:28,  1.59s/it]
 15%|█▍        | 2724/18627 [1:27:10<7:43:12,  1.75s/it]
 15%|█▍        | 2725/18627 [1:27:12<8:13:26,  1.86s/it]
 15%|█▍        | 2726/18627 [1:27:15<8:34:43,  1.94s/it]
 15%|█▍        | 2727/18627 [1:27:17<8:50:09,  2.00s/it]
 15%|█▍        | 2728/18627 [1:27:19<8:58:58,  2.03s/it]
 15%|█▍        | 2729/18627 [1:27:20<7:50:58,  1.78s/it]
 15%|█▍        | 2730/18627 [1:27:24<10:08:26,  2.30s/it]
                                                         
{'loss': 1.2627, 'grad_norm': 7.252901554107666, 'learning_rate': 4.823986474203279e-06, 'epoch': 0.15}
+
 15%|█▍        | 2730/18627 [1:27:24<10:08:26,  2.30s/it]
 15%|█▍        | 2731/18627 [1:27:26<9:54:23,  2.24s/it] 
 15%|█▍        | 2732/18627 [1:27:28<9:45:30,  2.21s/it]
 15%|█▍        | 2733/18627 [1:27:30<9:36:35,  2.18s/it]
 15%|█▍        | 2734/18627 [1:27:32<9:30:57,  2.16s/it]
 15%|█▍        | 2735/18627 [1:27:34<9:28:39,  2.15s/it]
 15%|█▍        | 2736/18627 [1:27:36<9:28:02,  2.14s/it]
 15%|█▍        | 2737/18627 [1:27:38<9:26:53,  2.14s/it]
 15%|█▍        | 2738/18627 [1:27:40<9:22:43,  2.12s/it]
 15%|█▍        | 2739/18627 [1:27:43<9:23:06,  2.13s/it]
 15%|█▍        | 2740/18627 [1:27:43<7:29:48,  1.70s/it]
                                                        
{'loss': 0.7549, 'grad_norm': 7.711190223693848, 'learning_rate': 4.82238076519737e-06, 'epoch': 0.15}
+
 15%|█▍        | 2740/18627 [1:27:43<7:29:48,  1.70s/it]
 15%|█▍        | 2741/18627 [1:27:45<8:04:10,  1.83s/it]
 15%|█▍        | 2742/18627 [1:27:48<8:27:56,  1.92s/it]
 15%|█▍        | 2743/18627 [1:27:50<8:42:49,  1.97s/it]
 15%|█▍        | 2744/18627 [1:27:51<7:39:57,  1.74s/it]
 15%|█▍        | 2745/18627 [1:27:53<8:09:05,  1.85s/it]
 15%|█▍        | 2746/18627 [1:27:55<8:30:11,  1.93s/it]
 15%|█▍        | 2747/18627 [1:27:57<8:46:46,  1.99s/it]
 15%|█▍        | 2748/18627 [1:27:59<8:56:46,  2.03s/it]
 15%|█▍        | 2749/18627 [1:28:00<7:12:27,  1.63s/it]
 15%|█▍        | 2750/18627 [1:28:02<7:51:26,  1.78s/it]
                                                        
{'loss': 1.02, 'grad_norm': 7.226834297180176, 'learning_rate': 4.820768034966339e-06, 'epoch': 0.15}
+
 15%|█▍        | 2750/18627 [1:28:02<7:51:26,  1.78s/it]
 15%|█▍        | 2751/18627 [1:28:04<8:19:01,  1.89s/it]
 15%|█▍        | 2752/18627 [1:28:06<8:38:46,  1.96s/it]
 15%|█▍        | 2753/18627 [1:28:09<8:51:13,  2.01s/it]
 15%|█▍        | 2754/18627 [1:28:11<8:59:46,  2.04s/it]
 15%|█▍        | 2755/18627 [1:28:13<9:06:26,  2.07s/it]
 15%|█▍        | 2756/18627 [1:28:15<9:21:50,  2.12s/it]
 15%|█▍        | 2757/18627 [1:28:17<9:22:04,  2.13s/it]
 15%|█▍        | 2758/18627 [1:28:19<9:22:32,  2.13s/it]
 15%|█▍        | 2759/18627 [1:28:21<9:22:31,  2.13s/it]
 15%|█▍        | 2760/18627 [1:28:24<9:21:40,  2.12s/it]
                                                        
{'loss': 0.7146, 'grad_norm': 9.336762428283691, 'learning_rate': 4.819148288385934e-06, 'epoch': 0.15}
+
 15%|█▍        | 2760/18627 [1:28:24<9:21:40,  2.12s/it]
 15%|█▍        | 2761/18627 [1:28:26<9:20:12,  2.12s/it]
 15%|█▍        | 2762/18627 [1:28:28<9:19:26,  2.12s/it]
 15%|█▍        | 2763/18627 [1:28:30<9:18:22,  2.11s/it]
 15%|█▍        | 2764/18627 [1:28:32<9:18:37,  2.11s/it]
 15%|█▍        | 2765/18627 [1:28:33<8:05:11,  1.84s/it]
 15%|█▍        | 2766/18627 [1:28:34<7:14:35,  1.64s/it]
 15%|█▍        | 2767/18627 [1:28:36<7:51:46,  1.78s/it]
 15%|█▍        | 2768/18627 [1:28:38<6:58:55,  1.58s/it]
 15%|█▍        | 2769/18627 [1:28:41<9:19:13,  2.12s/it]
 15%|█▍        | 2770/18627 [1:28:43<9:21:21,  2.12s/it]
                                                        
{'loss': 1.4538, 'grad_norm': 8.765752792358398, 'learning_rate': 4.817521530353114e-06, 'epoch': 0.15}
+
 15%|█▍        | 2770/18627 [1:28:43<9:21:21,  2.12s/it]
 15%|█▍        | 2771/18627 [1:28:45<9:22:27,  2.13s/it]
 15%|█▍        | 2772/18627 [1:28:46<8:07:31,  1.84s/it]
 15%|█▍        | 2773/18627 [1:28:49<8:31:53,  1.94s/it]
 15%|█▍        | 2774/18627 [1:28:51<8:47:35,  2.00s/it]
 15%|█▍        | 2775/18627 [1:28:51<7:05:22,  1.61s/it]
 15%|█▍        | 2776/18627 [1:28:54<7:45:55,  1.76s/it]
 15%|█▍        | 2777/18627 [1:28:56<8:12:25,  1.86s/it]
 15%|█▍        | 2778/18627 [1:28:58<8:31:17,  1.94s/it]
 15%|█▍        | 2779/18627 [1:28:59<7:23:18,  1.68s/it]
 15%|█▍        | 2780/18627 [1:29:01<7:58:31,  1.81s/it]
                                                        
{'loss': 1.1009, 'grad_norm': 6.761751651763916, 'learning_rate': 4.815887765786037e-06, 'epoch': 0.15}
+
 15%|█▍        | 2780/18627 [1:29:01<7:58:31,  1.81s/it]
 15%|█▍        | 2781/18627 [1:29:03<8:21:46,  1.90s/it]
 15%|█▍        | 2782/18627 [1:29:05<8:39:06,  1.97s/it]
 15%|█▍        | 2783/18627 [1:29:07<8:50:54,  2.01s/it]
 15%|█▍        | 2784/18627 [1:29:09<8:59:16,  2.04s/it]
 15%|█▍        | 2785/18627 [1:29:12<9:05:15,  2.07s/it]
 15%|█▍        | 2786/18627 [1:29:14<9:09:06,  2.08s/it]
 15%|█▍        | 2787/18627 [1:29:15<7:59:00,  1.81s/it]
 15%|█▍        | 2788/18627 [1:29:18<9:53:43,  2.25s/it]
 15%|█▍        | 2789/18627 [1:29:20<9:41:53,  2.20s/it]
 15%|█▍        | 2790/18627 [1:29:22<9:34:53,  2.18s/it]
                                                        
{'loss': 0.9509, 'grad_norm': 11.577098846435547, 'learning_rate': 4.814246999624043e-06, 'epoch': 0.15}
+
 15%|█▍        | 2790/18627 [1:29:22<9:34:53,  2.18s/it]
 15%|█▍        | 2791/18627 [1:29:24<9:30:27,  2.16s/it]
 15%|█▍        | 2792/18627 [1:29:27<9:27:51,  2.15s/it]
 15%|█▍        | 2793/18627 [1:29:29<9:25:28,  2.14s/it]
 15%|█▍        | 2794/18627 [1:29:30<8:10:13,  1.86s/it]
 15%|█▌        | 2795/18627 [1:29:31<7:17:02,  1.66s/it]
 15%|█▌        | 2796/18627 [1:29:33<7:55:34,  1.80s/it]
 15%|█▌        | 2797/18627 [1:29:34<7:07:01,  1.62s/it]
 15%|█▌        | 2798/18627 [1:29:36<7:46:04,  1.77s/it]
 15%|█▌        | 2799/18627 [1:29:39<8:14:55,  1.88s/it]
 15%|█▌        | 2800/18627 [1:29:41<8:35:00,  1.95s/it]
                                                        
{'loss': 1.3031, 'grad_norm': 7.294278621673584, 'learning_rate': 4.812599236827642e-06, 'epoch': 0.15}
+
 15%|█▌        | 2800/18627 [1:29:41<8:35:00,  1.95s/it]
 15%|█▌        | 2801/18627 [1:29:43<8:48:46,  2.00s/it]
 15%|█▌        | 2802/18627 [1:29:45<8:58:26,  2.04s/it]
 15%|█▌        | 2803/18627 [1:29:46<7:50:26,  1.78s/it]
 15%|█▌        | 2804/18627 [1:29:48<8:17:12,  1.89s/it]
 15%|█▌        | 2805/18627 [1:29:50<8:34:50,  1.95s/it]
 15%|█▌        | 2806/18627 [1:29:53<8:48:20,  2.00s/it]
 15%|█▌        | 2807/18627 [1:29:55<8:57:43,  2.04s/it]
 15%|█▌        | 2808/18627 [1:29:57<9:04:30,  2.07s/it]
 15%|█▌        | 2809/18627 [1:29:58<7:41:41,  1.75s/it]
 15%|█▌        | 2810/18627 [1:29:59<6:57:56,  1.59s/it]
                                                        
{'loss': 1.3366, 'grad_norm': 16.320560455322266, 'learning_rate': 4.810944482378493e-06, 'epoch': 0.15}
+
 15%|█▌        | 2810/18627 [1:29:59<6:57:56,  1.59s/it]
 15%|█▌        | 2811/18627 [1:30:01<7:40:55,  1.75s/it]
 15%|█▌        | 2812/18627 [1:30:03<8:08:42,  1.85s/it]
 15%|█▌        | 2813/18627 [1:30:05<8:29:10,  1.93s/it]
 15%|█▌        | 2814/18627 [1:30:07<7:30:09,  1.71s/it]
 15%|█▌        | 2815/18627 [1:30:09<8:03:29,  1.83s/it]
 15%|█▌        | 2816/18627 [1:30:11<8:26:35,  1.92s/it]
 15%|█▌        | 2817/18627 [1:30:13<8:39:57,  1.97s/it]
 15%|█▌        | 2818/18627 [1:30:15<8:50:13,  2.01s/it]
 15%|█▌        | 2819/18627 [1:30:18<10:47:43,  2.46s/it]
 15%|█▌        | 2820/18627 [1:30:20<9:07:11,  2.08s/it] 
                                                        
{'loss': 1.1496, 'grad_norm': 15.34796142578125, 'learning_rate': 4.809282741279395e-06, 'epoch': 0.15}
+
 15%|█▌        | 2820/18627 [1:30:20<9:07:11,  2.08s/it]
 15%|█▌        | 2821/18627 [1:30:21<7:57:21,  1.81s/it]
 15%|█▌        | 2822/18627 [1:30:22<6:29:58,  1.48s/it]
 15%|█▌        | 2823/18627 [1:30:24<7:21:08,  1.67s/it]
 15%|█▌        | 2824/18627 [1:30:26<7:55:56,  1.81s/it]
 15%|█▌        | 2825/18627 [1:30:27<7:07:01,  1.62s/it]
 15%|█▌        | 2826/18627 [1:30:29<7:47:03,  1.77s/it]
 15%|█▌        | 2827/18627 [1:30:31<8:13:34,  1.87s/it]
 15%|█▌        | 2828/18627 [1:30:33<8:32:33,  1.95s/it]
 15%|█▌        | 2829/18627 [1:30:35<8:46:11,  2.00s/it]
 15%|█▌        | 2830/18627 [1:30:37<7:41:50,  1.75s/it]
                                                        
{'loss': 1.3646, 'grad_norm': 16.42742347717285, 'learning_rate': 4.80761401855427e-06, 'epoch': 0.15}
+
 15%|█▌        | 2830/18627 [1:30:37<7:41:50,  1.75s/it]
 15%|█▌        | 2831/18627 [1:30:39<8:11:13,  1.87s/it]
 15%|█▌        | 2832/18627 [1:30:41<8:31:43,  1.94s/it]
 15%|█▌        | 2833/18627 [1:30:42<7:31:52,  1.72s/it]
 15%|█▌        | 2834/18627 [1:30:44<8:01:26,  1.83s/it]
 15%|█▌        | 2835/18627 [1:30:46<8:23:58,  1.91s/it]
 15%|█▌        | 2836/18627 [1:30:48<8:40:01,  1.98s/it]
 15%|█▌        | 2837/18627 [1:30:51<8:53:06,  2.03s/it]
 15%|█▌        | 2838/18627 [1:30:53<9:01:16,  2.06s/it]
 15%|█▌        | 2839/18627 [1:30:55<9:07:28,  2.08s/it]
 15%|█▌        | 2840/18627 [1:30:57<9:09:32,  2.09s/it]
                                                        
{'loss': 0.9573, 'grad_norm': 7.500369071960449, 'learning_rate': 4.805938319248145e-06, 'epoch': 0.15}
+
 15%|█▌        | 2840/18627 [1:30:57<9:09:32,  2.09s/it]
 15%|█▌        | 2841/18627 [1:30:59<9:13:24,  2.10s/it]
 15%|█▌        | 2842/18627 [1:31:01<9:14:28,  2.11s/it]
 15%|█▌        | 2843/18627 [1:31:03<9:17:29,  2.12s/it]
 15%|█▌        | 2844/18627 [1:31:05<9:17:41,  2.12s/it]
 15%|█▌        | 2845/18627 [1:31:08<9:18:47,  2.12s/it]
 15%|█▌        | 2846/18627 [1:31:10<9:17:40,  2.12s/it]
 15%|█▌        | 2847/18627 [1:31:12<9:16:26,  2.12s/it]
 15%|█▌        | 2848/18627 [1:31:14<9:15:38,  2.11s/it]
 15%|█▌        | 2849/18627 [1:31:16<9:15:27,  2.11s/it]
 15%|█▌        | 2850/18627 [1:31:18<9:15:24,  2.11s/it]
                                                        
{'loss': 0.7209, 'grad_norm': 8.783166885375977, 'learning_rate': 4.804255648427143e-06, 'epoch': 0.15}
+
 15%|█▌        | 2850/18627 [1:31:18<9:15:24,  2.11s/it]
 15%|█▌        | 2851/18627 [1:31:20<9:15:19,  2.11s/it]
 15%|█▌        | 2852/18627 [1:31:22<9:14:41,  2.11s/it]
 15%|█▌        | 2853/18627 [1:31:24<8:02:12,  1.83s/it]
 15%|█▌        | 2854/18627 [1:31:26<8:22:51,  1.91s/it]
 15%|█▌        | 2855/18627 [1:31:27<7:19:09,  1.67s/it]
 15%|█▌        | 2856/18627 [1:31:28<6:41:06,  1.53s/it]
 15%|█▌        | 2857/18627 [1:31:30<7:27:49,  1.70s/it]
 15%|█▌        | 2858/18627 [1:31:32<7:59:26,  1.82s/it]
 15%|█▌        | 2859/18627 [1:31:34<8:22:01,  1.91s/it]
 15%|█▌        | 2860/18627 [1:31:36<8:38:55,  1.97s/it]
                                                        
{'loss': 1.2434, 'grad_norm': 6.690563678741455, 'learning_rate': 4.802566011178462e-06, 'epoch': 0.15}
+
 15%|█▌        | 2860/18627 [1:31:36<8:38:55,  1.97s/it]
 15%|█▌        | 2861/18627 [1:31:39<8:49:32,  2.02s/it]
 15%|█▌        | 2862/18627 [1:31:41<8:56:19,  2.04s/it]
 15%|█▌        | 2863/18627 [1:31:42<7:49:01,  1.79s/it]
 15%|█▌        | 2864/18627 [1:31:44<8:14:57,  1.88s/it]
 15%|█▌        | 2865/18627 [1:31:45<7:20:25,  1.68s/it]
 15%|█▌        | 2866/18627 [1:31:47<7:55:58,  1.81s/it]
 15%|█▌        | 2867/18627 [1:31:49<8:20:47,  1.91s/it]
 15%|█▌        | 2868/18627 [1:31:51<7:24:12,  1.69s/it]
 15%|█▌        | 2869/18627 [1:31:53<7:57:15,  1.82s/it]
 15%|█▌        | 2870/18627 [1:31:55<8:21:39,  1.91s/it]
                                                        
{'loss': 1.1704, 'grad_norm': 10.404061317443848, 'learning_rate': 4.8008694126103606e-06, 'epoch': 0.15}
+
 15%|█▌        | 2870/18627 [1:31:55<8:21:39,  1.91s/it]
 15%|█▌        | 2871/18627 [1:31:57<8:37:55,  1.97s/it]
 15%|█▌        | 2872/18627 [1:31:59<8:48:02,  2.01s/it]
 15%|█▌        | 2873/18627 [1:32:01<8:56:27,  2.04s/it]
 15%|█▌        | 2874/18627 [1:32:03<9:01:04,  2.06s/it]
 15%|█▌        | 2875/18627 [1:32:04<7:52:11,  1.80s/it]
 15%|█▌        | 2876/18627 [1:32:06<7:03:54,  1.61s/it]
 15%|█▌        | 2877/18627 [1:32:08<7:41:55,  1.76s/it]
 15%|█▌        | 2878/18627 [1:32:10<8:09:56,  1.87s/it]
 15%|█▌        | 2879/18627 [1:32:12<8:29:18,  1.94s/it]
 15%|█▌        | 2880/18627 [1:32:14<8:44:05,  2.00s/it]
                                                        
{'loss': 1.0608, 'grad_norm': 12.526078224182129, 'learning_rate': 4.799165857852144e-06, 'epoch': 0.15}
+
 15%|█▌        | 2880/18627 [1:32:14<8:44:05,  2.00s/it]
 15%|█▌        | 2881/18627 [1:32:16<8:52:33,  2.03s/it]
 15%|█▌        | 2882/18627 [1:32:18<8:59:51,  2.06s/it]
 15%|█▌        | 2883/18627 [1:32:20<9:04:13,  2.07s/it]
 15%|█▌        | 2884/18627 [1:32:22<7:54:42,  1.81s/it]
 15%|█▌        | 2885/18627 [1:32:24<8:19:11,  1.90s/it]
 15%|█▌        | 2886/18627 [1:32:26<8:35:46,  1.97s/it]
 15%|█▌        | 2887/18627 [1:32:28<8:48:21,  2.01s/it]
 16%|█▌        | 2888/18627 [1:32:29<7:43:42,  1.77s/it]
 16%|█▌        | 2889/18627 [1:32:31<8:12:07,  1.88s/it]
 16%|█▌        | 2890/18627 [1:32:33<8:32:34,  1.95s/it]
                                                        
{'loss': 1.1065, 'grad_norm': 10.033717155456543, 'learning_rate': 4.797455352054149e-06, 'epoch': 0.16}
+
 16%|█▌        | 2890/18627 [1:32:33<8:32:34,  1.95s/it]
 16%|█▌        | 2891/18627 [1:32:36<8:44:45,  2.00s/it]
 16%|█▌        | 2892/18627 [1:32:38<8:54:17,  2.04s/it]
 16%|█▌        | 2893/18627 [1:32:40<9:03:51,  2.07s/it]
 16%|█▌        | 2894/18627 [1:32:42<9:07:57,  2.09s/it]
 16%|█▌        | 2895/18627 [1:32:44<9:09:58,  2.10s/it]
 16%|█▌        | 2896/18627 [1:32:46<9:11:53,  2.10s/it]
 16%|█▌        | 2897/18627 [1:32:48<9:13:24,  2.11s/it]
 16%|█���        | 2898/18627 [1:32:50<9:14:46,  2.12s/it]
 16%|█▌        | 2899/18627 [1:32:52<8:05:04,  1.85s/it]
 16%|█▌        | 2900/18627 [1:32:54<8:31:31,  1.95s/it]
                                                        
{'loss': 1.0353, 'grad_norm': 10.32773494720459, 'learning_rate': 4.795737900387727e-06, 'epoch': 0.16}
+
 16%|█▌        | 2900/18627 [1:32:54<8:31:31,  1.95s/it]
 16%|█▌        | 2901/18627 [1:32:55<7:32:11,  1.73s/it]
 16%|█▌        | 2902/18627 [1:32:57<8:05:00,  1.85s/it]
 16%|█▌        | 2903/18627 [1:32:59<8:27:57,  1.94s/it]
 16%|█▌        | 2904/18627 [1:33:01<8:45:09,  2.00s/it]
 16%|█▌        | 2905/18627 [1:33:04<8:57:12,  2.05s/it]
 16%|█▌        | 2906/18627 [1:33:06<9:01:51,  2.07s/it]
 16%|█▌        | 2907/18627 [1:33:08<9:06:30,  2.09s/it]
 16%|█▌        | 2908/18627 [1:33:10<9:09:16,  2.10s/it]
 16%|█▌        | 2909/18627 [1:33:12<9:08:54,  2.10s/it]
 16%|█▌        | 2910/18627 [1:33:14<9:11:25,  2.11s/it]
                                                        
{'loss': 0.9874, 'grad_norm': 16.10841178894043, 'learning_rate': 4.794013508045228e-06, 'epoch': 0.16}
+
 16%|█▌        | 2910/18627 [1:33:14<9:11:25,  2.11s/it]
 16%|█▌        | 2911/18627 [1:33:15<8:00:15,  1.83s/it]
 16%|█▌        | 2912/18627 [1:33:18<8:21:46,  1.92s/it]
 16%|█▌        | 2913/18627 [1:33:19<7:25:44,  1.70s/it]
 16%|█▌        | 2914/18627 [1:33:21<7:56:29,  1.82s/it]
 16%|█▌        | 2915/18627 [1:33:23<8:19:45,  1.91s/it]
 16%|█▌        | 2916/18627 [1:33:25<8:37:30,  1.98s/it]
 16%|█▌        | 2917/18627 [1:33:26<7:35:46,  1.74s/it]
 16%|█▌        | 2918/18627 [1:33:28<8:08:25,  1.87s/it]
 16%|█▌        | 2919/18627 [1:33:31<8:29:39,  1.95s/it]
 16%|█▌        | 2920/18627 [1:33:33<8:43:44,  2.00s/it]
                                                        
{'loss': 1.3791, 'grad_norm': 10.619322776794434, 'learning_rate': 4.792282180239985e-06, 'epoch': 0.16}
+
 16%|█▌        | 2920/18627 [1:33:33<8:43:44,  2.00s/it]
 16%|█▌        | 2921/18627 [1:33:34<7:39:38,  1.76s/it]
 16%|█▌        | 2922/18627 [1:33:36<8:05:54,  1.86s/it]
 16%|█▌        | 2923/18627 [1:33:38<8:27:33,  1.94s/it]
 16%|█▌        | 2924/18627 [1:33:39<7:29:13,  1.72s/it]
 16%|█▌        | 2925/18627 [1:33:41<8:01:34,  1.84s/it]
 16%|█▌        | 2926/18627 [1:33:44<8:21:11,  1.92s/it]
 16%|█▌        | 2927/18627 [1:33:46<8:37:09,  1.98s/it]
 16%|█▌        | 2928/18627 [1:33:48<8:48:26,  2.02s/it]
 16%|█▌        | 2929/18627 [1:33:50<8:55:57,  2.05s/it]
 16%|█▌        | 2930/18627 [1:33:52<9:01:08,  2.07s/it]
                                                        
{'loss': 1.215, 'grad_norm': 9.037995338439941, 'learning_rate': 4.790543922206302e-06, 'epoch': 0.16}
+
 16%|█▌        | 2930/18627 [1:33:52<9:01:08,  2.07s/it]
 16%|█▌        | 2931/18627 [1:33:54<9:05:44,  2.09s/it]
 16%|█▌        | 2932/18627 [1:33:56<9:08:28,  2.10s/it]
 16%|█▌        | 2933/18627 [1:33:58<9:10:11,  2.10s/it]
 16%|█▌        | 2934/18627 [1:34:00<7:58:44,  1.83s/it]
 16%|█▌        | 2935/18627 [1:34:01<7:07:45,  1.64s/it]
 16%|█▌        | 2936/18627 [1:34:03<7:46:41,  1.78s/it]
 16%|█▌        | 2937/18627 [1:34:05<8:15:22,  1.89s/it]
 16%|█▌        | 2938/18627 [1:34:07<8:33:48,  1.96s/it]
 16%|█▌        | 2939/18627 [1:34:08<6:56:39,  1.59s/it]
 16%|█▌        | 2940/18627 [1:34:10<7:38:47,  1.75s/it]
                                                        
{'loss': 1.0512, 'grad_norm': 11.363986015319824, 'learning_rate': 4.788798739199431e-06, 'epoch': 0.16}
+
 16%|█▌        | 2940/18627 [1:34:10<7:38:47,  1.75s/it]
 16%|█▌        | 2941/18627 [1:34:12<8:07:52,  1.87s/it]
 16%|█▌        | 2942/18627 [1:34:14<8:29:48,  1.95s/it]
 16%|█▌        | 2943/18627 [1:34:16<8:43:25,  2.00s/it]
 16%|█▌        | 2944/18627 [1:34:19<8:52:03,  2.04s/it]
 16%|█▌        | 2945/18627 [1:34:21<8:59:07,  2.06s/it]
 16%|█▌        | 2946/18627 [1:34:21<7:12:39,  1.66s/it]
 16%|█▌        | 2947/18627 [1:34:23<7:49:57,  1.80s/it]
 16%|█▌        | 2948/18627 [1:34:25<7:02:04,  1.62s/it]
 16%|█▌        | 2949/18627 [1:34:27<7:40:08,  1.76s/it]
 16%|█▌        | 2950/18627 [1:34:28<6:41:02,  1.53s/it]
                                                        
{'loss': 1.0164, 'grad_norm': 14.480243682861328, 'learning_rate': 4.7870466364955645e-06, 'epoch': 0.16}
+
 16%|█▌        | 2950/18627 [1:34:28<6:41:02,  1.53s/it]
 16%|█▌        | 2951/18627 [1:34:29<6:06:50,  1.40s/it]
 16%|█▌        | 2952/18627 [1:34:31<7:03:53,  1.62s/it]
 16%|█▌        | 2953/18627 [1:34:32<6:30:33,  1.50s/it]
 16%|█▌        | 2954/18627 [1:34:34<7:18:45,  1.68s/it]
 16%|█▌        | 2955/18627 [1:34:35<6:40:47,  1.53s/it]
 16%|█▌        | 2956/18627 [1:34:38<7:26:03,  1.71s/it]
 16%|█▌        | 2957/18627 [1:34:39<6:45:33,  1.55s/it]
 16%|█▌        | 2958/18627 [1:34:41<7:29:57,  1.72s/it]
 16%|█▌        | 2959/18627 [1:34:43<8:02:29,  1.85s/it]
 16%|█▌        | 2960/18627 [1:34:45<8:23:40,  1.93s/it]
                                                        
{'loss': 1.5059, 'grad_norm': 7.173501014709473, 'learning_rate': 4.785287619391811e-06, 'epoch': 0.16}
+
 16%|█▌        | 2960/18627 [1:34:45<8:23:40,  1.93s/it]
 16%|█▌        | 2961/18627 [1:34:46<7:25:54,  1.71s/it]
 16%|█▌        | 2962/18627 [1:34:48<7:58:00,  1.83s/it]
 16%|█▌        | 2963/18627 [1:34:51<8:20:49,  1.92s/it]
 16%|█▌        | 2964/18627 [1:34:52<7:24:38,  1.70s/it]
 16%|█▌        | 2965/18627 [1:34:54<7:57:13,  1.83s/it]
 16%|█▌        | 2966/18627 [1:34:56<8:19:23,  1.91s/it]
 16%|█▌        | 2967/18627 [1:34:58<8:35:46,  1.98s/it]
 16%|█▌        | 2968/18627 [1:35:00<8:44:54,  2.01s/it]
 16%|█▌        | 2969/18627 [1:35:02<8:54:07,  2.05s/it]
 16%|█▌        | 2970/18627 [1:35:05<8:59:38,  2.07s/it]
                                                        
{'loss': 1.1783, 'grad_norm': 13.592036247253418, 'learning_rate': 4.783521693206187e-06, 'epoch': 0.16}
+
 16%|█▌        | 2970/18627 [1:35:05<8:59:38,  2.07s/it]
 16%|█▌        | 2971/18627 [1:35:07<9:04:07,  2.09s/it]
 16%|█▌        | 2972/18627 [1:35:09<9:07:12,  2.10s/it]
 16%|█▌        | 2973/18627 [1:35:10<7:56:56,  1.83s/it]
 16%|█▌        | 2974/18627 [1:35:11<7:06:47,  1.64s/it]
 16%|█▌        | 2975/18627 [1:35:13<7:44:21,  1.78s/it]
 16%|█▌        | 2976/18627 [1:35:15<8:12:32,  1.89s/it]
 16%|█▌        | 2977/18627 [1:35:18<8:31:39,  1.96s/it]
 16%|█▌        | 2978/18627 [1:35:20<8:44:34,  2.01s/it]
 16%|█▌        | 2979/18627 [1:35:22<8:55:14,  2.05s/it]
 16%|█▌        | 2980/18627 [1:35:24<9:01:52,  2.08s/it]
                                                        
{'loss': 1.1444, 'grad_norm': 7.588423728942871, 'learning_rate': 4.781748863277593e-06, 'epoch': 0.16}
+
 16%|█▌        | 2980/18627 [1:35:24<9:01:52,  2.08s/it]
 16%|█▌        | 2981/18627 [1:35:26<9:08:52,  2.10s/it]
 16%|█▌        | 2982/18627 [1:35:28<9:09:59,  2.11s/it]
 16%|█▌        | 2983/18627 [1:35:30<9:11:45,  2.12s/it]
 16%|█▌        | 2984/18627 [1:35:32<9:11:48,  2.12s/it]
 16%|█▌        | 2985/18627 [1:35:34<7:59:26,  1.84s/it]
 16%|█▌        | 2986/18627 [1:35:36<8:22:42,  1.93s/it]
 16%|█▌        | 2987/18627 [1:35:37<6:46:26,  1.56s/it]
 16%|█▌        | 2988/18627 [1:35:38<6:17:02,  1.45s/it]
 16%|█▌        | 2989/18627 [1:35:40<7:10:24,  1.65s/it]
 16%|█▌        | 2990/18627 [1:35:42<7:46:10,  1.79s/it]
                                                        
{'loss': 1.1792, 'grad_norm': 10.718658447265625, 'learning_rate': 4.7799691349658065e-06, 'epoch': 0.16}
+
 16%|█▌        | 2990/18627 [1:35:42<7:46:10,  1.79s/it]
 16%|█▌        | 2991/18627 [1:35:44<8:12:39,  1.89s/it]
 16%|█▌        | 2992/18627 [1:35:46<8:30:01,  1.96s/it]
 16%|█▌        | 2993/18627 [1:35:47<7:29:47,  1.73s/it]
 16%|█▌        | 2994/18627 [1:35:49<7:58:41,  1.84s/it]
 16%|█▌        | 2995/18627 [1:35:52<8:19:38,  1.92s/it]
 16%|█▌        | 2996/18627 [1:35:54<8:35:06,  1.98s/it]
 16%|█▌        | 2997/18627 [1:35:56<8:45:41,  2.02s/it]
 16%|█▌        | 2998/18627 [1:35:58<8:53:19,  2.05s/it]
 16%|█▌        | 2999/18627 [1:36:00<8:56:54,  2.06s/it]
 16%|█▌        | 3000/18627 [1:36:01<7:48:00,  1.80s/it]
                                                        
{'loss': 1.0944, 'grad_norm': 16.28377914428711, 'learning_rate': 4.778182513651456e-06, 'epoch': 0.16}
+
 16%|█▌        | 3000/18627 [1:36:01<7:48:00,  1.80s/it]/home/zhengduo/workspace/LLaMA-Factory/src/qwen_vl/model/modeling_qwen2_5_vl.py:1979: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
+  with torch.cuda.amp.autocast(dtype=dtype):
+/home/zhengduo/miniconda3/envs/vgllm/lib/python3.10/site-packages/torch/utils/checkpoint.py:87: UserWarning: None of the inputs have requires_grad=True. Gradients will be None
+  warnings.warn(
+
 16%|█▌        | 3001/18627 [1:36:42<58:08:21, 13.39s/it]
 16%|█▌        | 3002/18627 [1:36:44<43:28:41, 10.02s/it]
 16%|█▌        | 3003/18627 [1:36:46<33:11:49,  7.65s/it]
 16%|█▌        | 3004/18627 [1:36:48<26:00:51,  5.99s/it]
 16%|█▌        | 3005/18627 [1:36:50<20:58:28,  4.83s/it]
 16%|█▌        | 3006/18627 [1:36:52<17:28:26,  4.03s/it]
 16%|█▌        | 3007/18627 [1:36:54<14:59:30,  3.46s/it]
 16%|█▌        | 3008/18627 [1:36:57<13:16:14,  3.06s/it]
 16%|█▌        | 3009/18627 [1:36:59<12:03:09,  2.78s/it]
 16%|█▌        | 3010/18627 [1:37:00<9:59:00,  2.30s/it] 
                                                        
{'loss': 0.9336, 'grad_norm': 14.51433277130127, 'learning_rate': 4.776389004736014e-06, 'epoch': 0.16}
+
 16%|█▌        | 3010/18627 [1:37:00<9:59:00,  2.30s/it]
 16%|█▌        | 3011/18627 [1:37:02<9:45:51,  2.25s/it]
 16%|█▌        | 3012/18627 [1:37:04<9:36:21,  2.21s/it]
 16%|█▌        | 3013/18627 [1:37:06<9:28:40,  2.19s/it]
 16%|█▌        | 3014/18627 [1:37:08<9:22:58,  2.16s/it]
 16%|█▌        | 3015/18627 [1:37:10<9:18:53,  2.15s/it]
 16%|█▌        | 3016/18627 [1:37:13<9:19:02,  2.15s/it]
 16%|█▌        | 3017/18627 [1:37:15<9:17:28,  2.14s/it]
 16%|█▌        | 3018/18627 [1:37:17<9:15:55,  2.14s/it]
 16%|█▌        | 3019/18627 [1:37:18<8:01:36,  1.85s/it]
 16%|█▌        | 3020/18627 [1:37:19<7:09:42,  1.65s/it]
                                                        
{'loss': 1.1706, 'grad_norm': 16.199079513549805, 'learning_rate': 4.7745886136417715e-06, 'epoch': 0.16}
+
 16%|█▌        | 3020/18627 [1:37:19<7:09:42,  1.65s/it]
 16%|█▌        | 3021/18627 [1:37:21<7:46:08,  1.79s/it]
 16%|█▌        | 3022/18627 [1:37:24<8:13:44,  1.90s/it]
 16%|█▌        | 3023/18627 [1:37:24<6:43:02,  1.55s/it]
 16%|█▌        | 3024/18627 [1:37:26<7:26:41,  1.72s/it]
 16%|█▌        | 3025/18627 [1:37:28<7:59:30,  1.84s/it]
 16%|█▌        | 3026/18627 [1:37:31<8:21:16,  1.93s/it]
 16%|█▋        | 3027/18627 [1:37:33<8:36:00,  1.98s/it]
 16%|█▋        | 3028/18627 [1:37:34<7:33:37,  1.74s/it]
 16%|█▋        | 3029/18627 [1:37:36<8:03:01,  1.86s/it]
 16%|█▋        | 3030/18627 [1:37:38<8:24:39,  1.94s/it]
                                                        
{'loss': 0.9536, 'grad_norm': 8.45963191986084, 'learning_rate': 4.77278134581183e-06, 'epoch': 0.16}
+
 16%|█▋        | 3030/18627 [1:37:38<8:24:39,  1.94s/it]
 16%|█▋        | 3031/18627 [1:37:40<8:38:14,  1.99s/it]
 16%|█▋        | 3032/18627 [1:37:41<7:36:07,  1.75s/it]
 16%|█▋        | 3033/18627 [1:37:44<8:02:07,  1.86s/it]
 16%|█▋        | 3034/18627 [1:37:46<8:22:14,  1.93s/it]
 16%|█▋        | 3035/18627 [1:37:48<8:37:21,  1.99s/it]
 16%|█▋        | 3036/18627 [1:37:49<7:35:10,  1.75s/it]
 16%|█▋        | 3037/18627 [1:37:51<8:01:47,  1.85s/it]
 16%|█▋        | 3038/18627 [1:37:53<8:21:31,  1.93s/it]
 16%|█▋        | 3039/18627 [1:37:54<7:23:46,  1.71s/it]
 16%|█▋        | 3040/18627 [1:37:57<7:54:17,  1.83s/it]
                                                        
{'loss': 1.3605, 'grad_norm': 6.116096496582031, 'learning_rate': 4.770967206710079e-06, 'epoch': 0.16}
+
 16%|█▋        | 3040/18627 [1:37:57<7:54:17,  1.83s/it]
 16%|█▋        | 3041/18627 [1:37:59<8:18:59,  1.92s/it]
 16%|█▋        | 3042/18627 [1:38:01<8:35:27,  1.98s/it]
 16%|█▋        | 3043/18627 [1:38:03<8:46:50,  2.03s/it]
 16%|█▋        | 3044/18627 [1:38:05<8:52:37,  2.05s/it]
 16%|█▋        | 3045/18627 [1:38:06<7:45:11,  1.79s/it]
 16%|█▋        | 3046/18627 [1:38:08<8:08:47,  1.88s/it]
 16%|█▋        | 3047/18627 [1:38:09<7:02:13,  1.63s/it]
 16%|█▋        | 3048/18627 [1:38:11<6:30:55,  1.51s/it]
 16%|█▋        | 3049/18627 [1:38:12<6:06:39,  1.41s/it]
 16%|█▋        | 3050/18627 [1:38:14<7:02:41,  1.63s/it]
                                                        
{'loss': 1.5195, 'grad_norm': 7.275299549102783, 'learning_rate': 4.769146201821184e-06, 'epoch': 0.16}
+
 16%|█▋        | 3050/18627 [1:38:14<7:02:41,  1.63s/it]
 16%|█▋        | 3051/18627 [1:38:15<6:29:00,  1.50s/it]
 16%|█▋        | 3052/18627 [1:38:16<6:05:02,  1.41s/it]
 16%|█▋        | 3053/18627 [1:38:17<5:47:59,  1.34s/it]
 16%|█▋        | 3054/18627 [1:38:20<6:47:49,  1.57s/it]
 16%|█▋        | 3055/18627 [1:38:22<7:29:32,  1.73s/it]
 16%|█▋        | 3056/18627 [1:38:22<6:10:23,  1.43s/it]
 16%|█▋        | 3057/18627 [1:38:24<7:03:46,  1.63s/it]
 16%|█▋        | 3058/18627 [1:38:27<7:41:32,  1.78s/it]
 16%|█▋        | 3059/18627 [1:38:29<8:07:56,  1.88s/it]
 16%|█▋        | 3060/18627 [1:38:31<8:27:09,  1.95s/it]
                                                        
{'loss': 1.3784, 'grad_norm': 7.1150102615356445, 'learning_rate': 4.767318336650567e-06, 'epoch': 0.16}
+
 16%|█▋        | 3060/18627 [1:38:31<8:27:09,  1.95s/it]
 16%|█▋        | 3061/18627 [1:38:33<8:40:21,  2.01s/it]
 16%|█▋        | 3062/18627 [1:38:34<6:59:28,  1.62s/it]
 16%|█▋        | 3063/18627 [1:38:36<7:39:37,  1.77s/it]
 16%|█▋        | 3064/18627 [1:38:38<8:06:42,  1.88s/it]
 16%|█▋        | 3065/18627 [1:38:40<8:26:50,  1.95s/it]
 16%|█▋        | 3066/18627 [1:38:42<8:41:59,  2.01s/it]
 16%|█▋        | 3067/18627 [1:38:44<8:51:51,  2.05s/it]
 16%|█▋        | 3068/18627 [1:38:46<8:56:30,  2.07s/it]
 16%|█▋        | 3069/18627 [1:38:49<9:00:13,  2.08s/it]
 16%|█▋        | 3070/18627 [1:38:51<9:02:09,  2.09s/it]
                                                        
{'loss': 0.7422, 'grad_norm': 15.433606147766113, 'learning_rate': 4.765483616724389e-06, 'epoch': 0.16}
+
 16%|█▋        | 3070/18627 [1:38:51<9:02:09,  2.09s/it]
 16%|█▋        | 3071/18627 [1:38:53<9:04:28,  2.10s/it]
 16%|█▋        | 3072/18627 [1:38:55<9:06:22,  2.11s/it]
 16%|█▋        | 3073/18627 [1:38:57<9:09:46,  2.12s/it]
 17%|█▋        | 3074/18627 [1:38:59<9:09:38,  2.12s/it]
 17%|█▋        | 3075/18627 [1:39:00<7:56:51,  1.84s/it]
 17%|█��        | 3076/18627 [1:39:02<7:06:17,  1.64s/it]
 17%|█▋        | 3077/18627 [1:39:04<7:43:01,  1.79s/it]
 17%|█▋        | 3078/18627 [1:39:06<8:08:53,  1.89s/it]
 17%|█▋        | 3079/18627 [1:39:07<7:14:33,  1.68s/it]
 17%|█▋        | 3080/18627 [1:39:09<7:47:17,  1.80s/it]
                                                        
{'loss': 1.3474, 'grad_norm': 9.253497123718262, 'learning_rate': 4.763642047589536e-06, 'epoch': 0.17}
+
 17%|█▋        | 3080/18627 [1:39:09<7:47:17,  1.80s/it]
 17%|█▋        | 3081/18627 [1:39:11<8:12:51,  1.90s/it]
 17%|█▋        | 3082/18627 [1:39:13<8:31:26,  1.97s/it]
 17%|█▋        | 3083/18627 [1:39:16<8:44:06,  2.02s/it]
 17%|█▋        | 3084/18627 [1:39:18<8:52:34,  2.06s/it]
 17%|█▋        | 3085/18627 [1:39:20<8:55:28,  2.07s/it]
 17%|█▋        | 3086/18627 [1:39:22<8:59:24,  2.08s/it]
 17%|█▋        | 3087/18627 [1:39:24<9:02:28,  2.09s/it]
 17%|█▋        | 3088/18627 [1:39:26<9:03:39,  2.10s/it]
 17%|█▋        | 3089/18627 [1:39:27<7:52:45,  1.83s/it]
 17%|█▋        | 3090/18627 [1:39:29<8:15:55,  1.92s/it]
                                                        
{'loss': 1.0052, 'grad_norm': 12.682035446166992, 'learning_rate': 4.761793634813602e-06, 'epoch': 0.17}
+
 17%|█▋        | 3090/18627 [1:39:29<8:15:55,  1.92s/it]
 17%|█▋        | 3091/18627 [1:39:32<8:31:30,  1.98s/it]
 17%|█▋        | 3092/18627 [1:39:34<8:42:27,  2.02s/it]
 17%|█▋        | 3093/18627 [1:39:35<7:38:39,  1.77s/it]
 17%|█▋        | 3094/18627 [1:39:37<8:05:04,  1.87s/it]
 17%|█▋        | 3095/18627 [1:39:38<7:11:34,  1.67s/it]
 17%|█▋        | 3096/18627 [1:39:40<7:46:46,  1.80s/it]
 17%|█▋        | 3097/18627 [1:39:42<8:11:09,  1.90s/it]
 17%|█▋        | 3098/18627 [1:39:45<8:28:39,  1.97s/it]
 17%|█▋        | 3099/18627 [1:39:46<7:28:02,  1.73s/it]
 17%|█▋        | 3100/18627 [1:39:48<7:58:25,  1.85s/it]
                                                        
{'loss': 1.2812, 'grad_norm': 6.347700595855713, 'learning_rate': 4.7599383839848706e-06, 'epoch': 0.17}
+
 17%|█▋        | 3100/18627 [1:39:48<7:58:25,  1.85s/it]
 17%|█▋        | 3101/18627 [1:39:49<7:06:45,  1.65s/it]
 17%|█▋        | 3102/18627 [1:39:51<7:44:08,  1.79s/it]
 17%|█▋        | 3103/18627 [1:39:53<8:09:04,  1.89s/it]
 17%|█▋        | 3104/18627 [1:39:54<6:37:49,  1.54s/it]
 17%|█▋        | 3105/18627 [1:39:56<7:23:26,  1.71s/it]
 17%|█▋        | 3106/18627 [1:39:57<6:42:31,  1.56s/it]
 17%|█▋        | 3107/18627 [1:39:58<6:06:41,  1.42s/it]
 17%|█▋        | 3108/18627 [1:40:00<5:49:24,  1.35s/it]
 17%|█▋        | 3109/18627 [1:40:02<6:49:27,  1.58s/it]
 17%|█▋        | 3110/18627 [1:40:03<6:18:52,  1.47s/it]
                                                        
{'loss': 1.646, 'grad_norm': 14.941232681274414, 'learning_rate': 4.758076300712299e-06, 'epoch': 0.17}
+
 17%|█▋        | 3110/18627 [1:40:03<6:18:52,  1.47s/it]
 17%|█▋        | 3111/18627 [1:40:04<5:51:26,  1.36s/it]
 17%|█▋        | 3112/18627 [1:40:06<6:50:45,  1.59s/it]
 17%|█▋        | 3113/18627 [1:40:08<7:32:43,  1.75s/it]
 17%|█▋        | 3114/18627 [1:40:10<8:00:25,  1.86s/it]
 17%|█▋        | 3115/18627 [1:40:12<7:08:24,  1.66s/it]
 17%|█▋        | 3116/18627 [1:40:13<6:31:58,  1.52s/it]
 17%|█▋        | 3117/18627 [1:40:15<7:19:34,  1.70s/it]
 17%|█▋        | 3118/18627 [1:40:17<7:52:13,  1.83s/it]
 17%|█▋        | 3119/18627 [1:40:18<7:02:36,  1.64s/it]
 17%|█▋        | 3120/18627 [1:40:19<6:27:32,  1.50s/it]
                                                        
{'loss': 1.7149, 'grad_norm': 15.418540954589844, 'learning_rate': 4.756207390625499e-06, 'epoch': 0.17}
+
 17%|█▋        | 3120/18627 [1:40:19<6:27:32,  1.50s/it]
 17%|█▋        | 3121/18627 [1:40:21<7:14:35,  1.68s/it]
 17%|█▋        | 3122/18627 [1:40:24<7:46:58,  1.81s/it]
 17%|█▋        | 3123/18627 [1:40:26<8:11:02,  1.90s/it]
 17%|█▋        | 3124/18627 [1:40:27<7:15:21,  1.68s/it]
 17%|█▋        | 3125/18627 [1:40:29<7:50:08,  1.82s/it]
 17%|█▋        | 3126/18627 [1:40:30<7:00:34,  1.63s/it]
 17%|█▋        | 3127/18627 [1:40:31<6:27:05,  1.50s/it]
 17%|█▋        | 3128/18627 [1:40:32<5:25:04,  1.26s/it]
 17%|█▋        | 3129/18627 [1:40:33<4:44:11,  1.10s/it]
 17%|█▋        | 3130/18627 [1:40:35<6:04:45,  1.41s/it]
                                                        
{'loss': 1.3781, 'grad_norm': 6.847586154937744, 'learning_rate': 4.754331659374725e-06, 'epoch': 0.17}
+
 17%|█▋        | 3130/18627 [1:40:35<6:04:45,  1.41s/it]
 17%|█▋        | 3131/18627 [1:40:37<6:59:45,  1.63s/it]
 17%|█▋        | 3132/18627 [1:40:38<6:25:35,  1.49s/it]
 17%|█▋        | 3133/18627 [1:40:39<6:02:52,  1.41s/it]
 17%|█▋        | 3134/18627 [1:40:42<6:59:08,  1.62s/it]
 17%|█▋        | 3135/18627 [1:40:44<7:36:46,  1.77s/it]
 17%|█▋        | 3136/18627 [1:40:46<8:04:11,  1.88s/it]
 17%|█▋        | 3137/18627 [1:40:47<6:32:46,  1.52s/it]
 17%|█▋        | 3138/18627 [1:40:49<7:22:00,  1.71s/it]
 17%|█▋        | 3139/18627 [1:40:50<6:41:40,  1.56s/it]
 17%|█▋        | 3140/18627 [1:40:51<6:15:48,  1.46s/it]
                                                        
{'loss': 1.4679, 'grad_norm': 17.684858322143555, 'learning_rate': 4.752449112630851e-06, 'epoch': 0.17}
+
 17%|█▋        | 3140/18627 [1:40:51<6:15:48,  1.46s/it]
 17%|█▋        | 3141/18627 [1:40:53<7:07:52,  1.66s/it]
 17%|█▋        | 3142/18627 [1:40:55<7:43:20,  1.80s/it]
 17%|█▋        | 3143/18627 [1:40:57<8:09:20,  1.90s/it]
 17%|█▋        | 3144/18627 [1:41:00<8:28:35,  1.97s/it]
 17%|█▋        | 3145/18627 [1:41:02<8:43:11,  2.03s/it]
 17%|█▋        | 3146/18627 [1:41:04<8:49:24,  2.05s/it]
 17%|█▋        | 3147/18627 [1:41:06<8:54:35,  2.07s/it]
 17%|█▋        | 3148/18627 [1:41:08<9:00:02,  2.09s/it]
 17%|█▋        | 3149/18627 [1:41:10<9:02:15,  2.10s/it]
 17%|█▋        | 3150/18627 [1:41:11<7:51:33,  1.83s/it]
                                                        
{'loss': 0.927, 'grad_norm': 17.267284393310547, 'learning_rate': 4.750559756085359e-06, 'epoch': 0.17}
+
 17%|█▋        | 3150/18627 [1:41:11<7:51:33,  1.83s/it]
 17%|█▋        | 3151/18627 [1:41:14<8:14:05,  1.92s/it]
 17%|█▋        | 3152/18627 [1:41:16<8:29:36,  1.98s/it]
 17%|█▋        | 3153/18627 [1:41:18<8:41:00,  2.02s/it]
 17%|█▋        | 3154/18627 [1:41:20<8:47:07,  2.04s/it]
 17%|█▋        | 3155/18627 [1:41:22<8:52:44,  2.07s/it]
 17%|█▋        | 3156/18627 [1:41:24<8:58:28,  2.09s/it]
 17%|█▋        | 3157/18627 [1:41:26<9:02:31,  2.10s/it]
 17%|█▋        | 3158/18627 [1:41:28<9:04:16,  2.11s/it]
 17%|█▋        | 3159/18627 [1:41:29<7:14:24,  1.69s/it]
 17%|█▋        | 3160/18627 [1:41:30<6:36:12,  1.54s/it]
                                                        
{'loss': 0.889, 'grad_norm': 16.058073043823242, 'learning_rate': 4.748663595450316e-06, 'epoch': 0.17}
+
 17%|█▋        | 3160/18627 [1:41:30<6:36:12,  1.54s/it]
 17%|█▋        | 3161/18627 [1:41:32<7:22:34,  1.72s/it]
 17%|█▋        | 3162/18627 [1:41:34<6:41:29,  1.56s/it]
 17%|█▋        | 3163/18627 [1:41:36<7:25:27,  1.73s/it]
 17%|█▋        | 3164/18627 [1:41:38<7:55:10,  1.84s/it]
 17%|█▋        | 3165/18627 [1:41:40<8:17:57,  1.93s/it]
 17%|█▋        | 3166/18627 [1:41:42<8:32:28,  1.99s/it]
 17%|█▋        | 3167/18627 [1:41:44<8:42:38,  2.03s/it]
 17%|█▋        | 3168/18627 [1:41:46<8:50:12,  2.06s/it]
 17%|█▋        | 3169/18627 [1:41:49<8:55:55,  2.08s/it]
 17%|█▋        | 3170/18627 [1:41:51<8:58:43,  2.09s/it]
                                                        
{'loss': 0.9169, 'grad_norm': 5.927127838134766, 'learning_rate': 4.746760636458362e-06, 'epoch': 0.17}
+
 17%|█▋        | 3170/18627 [1:41:51<8:58:43,  2.09s/it]
 17%|█▋        | 3171/18627 [1:41:53<9:01:35,  2.10s/it]
 17%|█▋        | 3172/18627 [1:41:54<7:51:15,  1.83s/it]
 17%|█▋        | 3173/18627 [1:41:56<8:14:48,  1.92s/it]
 17%|█▋        | 3174/18627 [1:41:58<8:31:16,  1.99s/it]
 17%|█▋        | 3175/18627 [1:42:00<8:42:29,  2.03s/it]
 17%|█▋        | 3176/18627 [1:42:03<8:51:13,  2.06s/it]
 17%|█▋        | 3177/18627 [1:42:05<8:54:34,  2.08s/it]
 17%|█▋        | 3178/18627 [1:42:07<8:57:25,  2.09s/it]
 17%|█▋        | 3179/18627 [1:42:09<8:59:09,  2.09s/it]
 17%|█▋        | 3180/18627 [1:42:11<8:59:45,  2.10s/it]
                                                        
{'loss': 0.8619, 'grad_norm': 9.675300598144531, 'learning_rate': 4.744850884862688e-06, 'epoch': 0.17}
+
 17%|█▋        | 3180/18627 [1:42:11<8:59:45,  2.10s/it]
 17%|█▋        | 3181/18627 [1:42:13<9:01:36,  2.10s/it]
 17%|█▋        | 3182/18627 [1:42:15<9:02:49,  2.11s/it]
 17%|█▋        | 3183/18627 [1:42:16<7:51:28,  1.83s/it]
 17%|█▋        | 3184/18627 [1:42:18<7:02:13,  1.64s/it]
 17%|█▋        | 3185/18627 [1:42:20<7:38:52,  1.78s/it]
 17%|█▋        | 3186/18627 [1:42:22<8:05:09,  1.89s/it]
 17%|█▋        | 3187/18627 [1:42:24<8:24:55,  1.96s/it]
 17%|█▋        | 3188/18627 [1:42:26<8:36:07,  2.01s/it]
 17%|█▋        | 3189/18627 [1:42:27<7:33:26,  1.76s/it]
 17%|█▋        | 3190/18627 [1:42:29<7:58:51,  1.86s/it]
                                                        
{'loss': 1.2914, 'grad_norm': 11.003336906433105, 'learning_rate': 4.742934346437024e-06, 'epoch': 0.17}
+
 17%|█▋        | 3190/18627 [1:42:29<7:58:51,  1.86s/it]
 17%|█▋        | 3191/18627 [1:42:31<8:16:55,  1.93s/it]
 17%|█▋        | 3192/18627 [1:42:34<8:30:05,  1.98s/it]
 17%|█▋        | 3193/18627 [1:42:35<7:29:11,  1.75s/it]
 17%|█▋        | 3194/18627 [1:42:37<7:58:39,  1.86s/it]
 17%|█▋        | 3195/18627 [1:42:39<8:17:56,  1.94s/it]
 17%|█▋        | 3196/18627 [1:42:40<7:20:17,  1.71s/it]
 17%|█▋        | 3197/18627 [1:42:42<7:50:47,  1.83s/it]
 17%|█▋        | 3198/18627 [1:42:44<8:12:31,  1.92s/it]
 17%|█▋        | 3199/18627 [1:42:47<8:29:28,  1.98s/it]
 17%|█▋        | 3200/18627 [1:42:49<8:38:59,  2.02s/it]
                                                        
{'loss': 1.1146, 'grad_norm': 7.526270389556885, 'learning_rate': 4.741011026975615e-06, 'epoch': 0.17}
+
 17%|█▋        | 3200/18627 [1:42:49<8:38:59,  2.02s/it]
 17%|█▋        | 3201/18627 [1:42:50<7:34:59,  1.77s/it]
 17%|█▋        | 3202/18627 [1:42:52<8:00:58,  1.87s/it]
 17%|█▋        | 3203/18627 [1:42:54<8:19:54,  1.94s/it]
 17%|█▋        | 3204/18627 [1:42:56<8:33:16,  2.00s/it]
 17%|█▋        | 3205/18627 [1:42:57<7:31:06,  1.76s/it]
 17%|█▋        | 3206/18627 [1:42:59<6:47:19,  1.58s/it]
 17%|█▋        | 3207/18627 [1:43:01<7:29:05,  1.75s/it]
 17%|█▋        | 3208/18627 [1:43:03<7:58:49,  1.86s/it]
 17%|█▋        | 3209/18627 [1:43:05<8:19:13,  1.94s/it]
 17%|█▋        | 3210/18627 [1:43:07<8:32:40,  2.00s/it]
                                                        
{'loss': 1.324, 'grad_norm': 7.820011615753174, 'learning_rate': 4.739080932293211e-06, 'epoch': 0.17}
+
 17%|█▋        | 3210/18627 [1:43:07<8:32:40,  2.00s/it]
 17%|█▋        | 3211/18627 [1:43:09<8:42:16,  2.03s/it]
 17%|█▋        | 3212/18627 [1:43:11<8:50:16,  2.06s/it]
 17%|█▋        | 3213/18627 [1:43:13<8:55:23,  2.08s/it]
 17%|█▋        | 3214/18627 [1:43:16<8:55:51,  2.09s/it]
 17%|█▋        | 3215/18627 [1:43:18<8:59:13,  2.10s/it]
 17%|█▋        | 3216/18627 [1:43:20<9:01:15,  2.11s/it]
 17%|█▋        | 3217/18627 [1:43:22<9:01:17,  2.11s/it]
 17%|█▋        | 3218/18627 [1:43:24<9:03:11,  2.12s/it]
 17%|█▋        | 3219/18627 [1:43:26<9:04:26,  2.12s/it]
 17%|█▋        | 3220/18627 [1:43:28<9:03:48,  2.12s/it]
                                                        
{'loss': 0.7544, 'grad_norm': 9.366170883178711, 'learning_rate': 4.737144068225043e-06, 'epoch': 0.17}
+
 17%|█▋        | 3220/18627 [1:43:28<9:03:48,  2.12s/it]
 17%|█▋        | 3221/18627 [1:43:30<9:04:48,  2.12s/it]
 17%|█▋        | 3222/18627 [1:43:32<7:54:55,  1.85s/it]
 17%|█▋        | 3223/18627 [1:43:34<8:15:30,  1.93s/it]
 17%|█▋        | 3224/18627 [1:43:35<7:18:21,  1.71s/it]
 17%|█▋        | 3225/18627 [1:43:36<6:30:27,  1.52s/it]
 17%|█▋        | 3226/18627 [1:43:38<7:16:54,  1.70s/it]
 17%|█▋        | 3227/18627 [1:43:40<7:47:42,  1.82s/it]
 17%|█▋        | 3228/18627 [1:43:42<8:10:24,  1.91s/it]
 17%|█▋        | 3229/18627 [1:43:44<8:26:53,  1.98s/it]
 17%|█▋        | 3230/18627 [1:43:46<7:26:34,  1.74s/it]
                                                        
{'loss': 1.5546, 'grad_norm': 16.416717529296875, 'learning_rate': 4.735200440626808e-06, 'epoch': 0.17}
+
 17%|█▋        | 3230/18627 [1:43:46<7:26:34,  1.74s/it]
 17%|█▋        | 3231/18627 [1:43:48<7:55:40,  1.85s/it]
 17%|█▋        | 3232/18627 [1:43:50<8:16:14,  1.93s/it]
 17%|█▋        | 3233/18627 [1:43:51<7:18:43,  1.71s/it]
 17%|█▋        | 3234/18627 [1:43:52<6:38:26,  1.55s/it]
 17%|█▋        | 3235/18627 [1:43:54<7:23:02,  1.73s/it]
 17%|█▋        | 3236/18627 [1:43:57<7:54:10,  1.85s/it]
 17%|█▋        | 3237/18627 [1:43:59<8:14:43,  1.93s/it]
 17%|█▋        | 3238/18627 [1:44:01<8:30:13,  1.99s/it]
 17%|█▋        | 3239/18627 [1:44:02<7:29:01,  1.75s/it]
 17%|█▋        | 3240/18627 [1:44:03<6:45:39,  1.58s/it]
                                                        
{'loss': 1.4925, 'grad_norm': 15.016637802124023, 'learning_rate': 4.733250055374651e-06, 'epoch': 0.17}
+
 17%|█▋        | 3240/18627 [1:44:03<6:45:39,  1.58s/it]
 17%|█▋        | 3241/18627 [1:44:04<6:15:09,  1.46s/it]
 17%|█▋        | 3242/18627 [1:44:06<5:53:46,  1.38s/it]
 17%|█▋        | 3243/18627 [1:44:08<6:51:48,  1.61s/it]
 17%|█▋        | 3244/18627 [1:44:10<7:31:00,  1.76s/it]
 17%|█▋        | 3245/18627 [1:44:12<7:57:38,  1.86s/it]
 17%|█▋        | 3246/18627 [1:44:14<8:16:43,  1.94s/it]
 17%|█▋        | 3247/18627 [1:44:15<7:18:36,  1.71s/it]
 17%|█▋        | 3248/18627 [1:44:17<7:49:40,  1.83s/it]
 17%|█▋        | 3249/18627 [1:44:18<7:00:22,  1.64s/it]
 17%|█▋        | 3250/18627 [1:44:21<7:36:50,  1.78s/it]
                                                        
{'loss': 1.5428, 'grad_norm': 22.81689453125, 'learning_rate': 4.731292918365148e-06, 'epoch': 0.17}
+
 17%|█▋        | 3250/18627 [1:44:21<7:36:50,  1.78s/it]
 17%|█▋        | 3251/18627 [1:44:23<8:01:51,  1.88s/it]
 17%|█▋        | 3252/18627 [1:44:24<7:09:25,  1.68s/it]
 17%|█▋        | 3253/18627 [1:44:25<6:31:39,  1.53s/it]
 17%|█▋        | 3254/18627 [1:44:27<7:16:13,  1.70s/it]
 17%|█▋        | 3255/18627 [1:44:29<7:49:13,  1.83s/it]
 17%|█▋        | 3256/18627 [1:44:31<6:59:47,  1.64s/it]
 17%|█▋        | 3257/18627 [1:44:33<7:35:06,  1.78s/it]
 17%|█▋        | 3258/18627 [1:44:35<8:02:18,  1.88s/it]
 17%|█▋        | 3259/18627 [1:44:37<8:20:05,  1.95s/it]
 18%|█▊        | 3260/18627 [1:44:38<7:21:32,  1.72s/it]
                                                        
{'loss': 1.6626, 'grad_norm': 16.02472496032715, 'learning_rate': 4.729329035515287e-06, 'epoch': 0.18}
+
 18%|█▊        | 3260/18627 [1:44:38<7:21:32,  1.72s/it]
 18%|█▊        | 3261/18627 [1:44:39<6:41:11,  1.57s/it]
 18%|█▊        | 3262/18627 [1:44:41<7:23:16,  1.73s/it]
 18%|█▊        | 3263/18627 [1:44:44<7:54:28,  1.85s/it]
 18%|█▊        | 3264/18627 [1:44:46<8:13:59,  1.93s/it]
 18%|█▊        | 3265/18627 [1:44:48<8:28:51,  1.99s/it]
 18%|█▊        | 3266/18627 [1:44:50<8:39:40,  2.03s/it]
 18%|█▊        | 3267/18627 [1:44:51<7:27:51,  1.75s/it]
 18%|█▊        | 3268/18627 [1:44:53<7:57:26,  1.87s/it]
 18%|█▊        | 3269/18627 [1:44:54<7:05:04,  1.66s/it]
 18%|█▊        | 3270/18627 [1:44:56<7:41:04,  1.80s/it]
                                                        
{'loss': 1.442, 'grad_norm': 7.230112075805664, 'learning_rate': 4.727358412762452e-06, 'epoch': 0.18}
+
 18%|█▊        | 3270/18627 [1:44:56<7:41:04,  1.80s/it]
 18%|█▊        | 3271/18627 [1:44:58<6:53:53,  1.62s/it]
 18%|█▊        | 3272/18627 [1:45:00<7:34:02,  1.77s/it]
 18%|█▊        | 3273/18627 [1:45:02<8:01:34,  1.88s/it]
 18%|█▊        | 3274/18627 [1:45:04<8:20:14,  1.95s/it]
 18%|█▊        | 3275/18627 [1:45:06<8:31:41,  2.00s/it]
 18%|█▊        | 3276/18627 [1:45:08<8:42:27,  2.04s/it]
 18%|█▊        | 3277/18627 [1:45:10<8:49:01,  2.07s/it]
 18%|█▊        | 3278/18627 [1:45:12<7:41:43,  1.80s/it]
 18%|█▊        | 3279/18627 [1:45:14<8:05:50,  1.90s/it]
 18%|█▊        | 3280/18627 [1:45:15<7:11:11,  1.69s/it]
                                                        
{'loss': 1.3902, 'grad_norm': 15.36326789855957, 'learning_rate': 4.7253810560643995e-06, 'epoch': 0.18}
+
 18%|█▊        | 3280/18627 [1:45:15<7:11:11,  1.69s/it]
 18%|█▊        | 3281/18627 [1:45:17<7:43:45,  1.81s/it]
 18%|█▊        | 3282/18627 [1:45:18<6:55:51,  1.63s/it]
 18%|█▊        | 3283/18627 [1:45:20<7:34:11,  1.78s/it]
 18%|█▊        | 3284/18627 [1:45:22<8:00:42,  1.88s/it]
 18%|█▊        | 3285/18627 [1:45:25<8:17:43,  1.95s/it]
 18%|█▊        | 3286/18627 [1:45:27<8:30:12,  2.00s/it]
 18%|█▊        | 3287/18627 [1:45:28<7:28:52,  1.76s/it]
 18%|█▊        | 3288/18627 [1:45:30<8:09:37,  1.92s/it]
 18%|█▊        | 3289/18627 [1:45:31<7:13:51,  1.70s/it]
 18%|█▊        | 3290/18627 [1:45:33<7:47:33,  1.83s/it]
                                                        
{'loss': 1.2639, 'grad_norm': 6.477517604827881, 'learning_rate': 4.723396971399251e-06, 'epoch': 0.18}
+
 18%|█▊        | 3290/18627 [1:45:33<7:47:33,  1.83s/it]
 18%|█▊        | 3291/18627 [1:45:36<8:10:09,  1.92s/it]
 18%|█▊        | 3292/18627 [1:45:37<7:14:33,  1.70s/it]
 18%|█▊        | 3293/18627 [1:45:39<7:46:30,  1.83s/it]
 18%|█▊        | 3294/18627 [1:45:40<6:57:49,  1.64s/it]
 18%|█▊        | 3295/18627 [1:45:42<7:35:08,  1.78s/it]
 18%|█▊        | 3296/18627 [1:45:44<7:59:21,  1.88s/it]
 18%|█▊        | 3297/18627 [1:45:46<8:16:43,  1.94s/it]
 18%|█▊        | 3298/18627 [1:45:48<8:29:51,  2.00s/it]
 18%|█▊        | 3299/18627 [1:45:51<8:40:29,  2.04s/it]
 18%|█▊        | 3300/18627 [1:45:53<8:46:23,  2.06s/it]
                                                        
{'loss': 1.0775, 'grad_norm': 10.079001426696777, 'learning_rate': 4.721406164765464e-06, 'epoch': 0.18}
+
 18%|█▊        | 3300/18627 [1:45:53<8:46:23,  2.06s/it]
 18%|█▊        | 3301/18627 [1:45:55<8:50:36,  2.08s/it]
 18%|█▊        | 3302/18627 [1:45:57<8:54:04,  2.09s/it]
 18%|█▊        | 3303/18627 [1:45:59<8:57:55,  2.11s/it]
 18%|█▊        | 3304/18627 [1:46:01<8:59:16,  2.11s/it]
 18%|█▊        | 3305/18627 [1:46:03<8:58:01,  2.11s/it]
 18%|█▊        | 3306/18627 [1:46:05<8:59:08,  2.11s/it]
 18%|█▊        | 3307/18627 [1:46:08<8:58:28,  2.11s/it]
 18%|█▊        | 3308/18627 [1:46:10<8:59:53,  2.11s/it]
 18%|█▊        | 3309/18627 [1:46:12<8:59:41,  2.11s/it]
 18%|█▊        | 3310/18627 [1:46:14<8:59:34,  2.11s/it]
                                                        
{'loss': 0.7374, 'grad_norm': 6.46017599105835, 'learning_rate': 4.719408642181819e-06, 'epoch': 0.18}
+
 18%|█▊        | 3310/18627 [1:46:14<8:59:34,  2.11s/it]
 18%|█▊        | 3311/18627 [1:46:16<9:00:16,  2.12s/it]
 18%|█▊        | 3312/18627 [1:46:18<8:58:45,  2.11s/it]
 18%|█▊        | 3313/18627 [1:46:20<8:58:09,  2.11s/it]
 18%|█▊        | 3314/18627 [1:46:22<8:57:10,  2.10s/it]
 18%|█▊        | 3315/18627 [1:46:24<7:47:04,  1.83s/it]
 18%|█▊        | 3316/18627 [1:46:25<6:57:48,  1.64s/it]
 18%|█▊        | 3317/18627 [1:46:27<7:33:21,  1.78s/it]
 18%|█▊        | 3318/18627 [1:46:29<7:59:12,  1.88s/it]
 18%|█▊        | 3319/18627 [1:46:31<8:16:27,  1.95s/it]
 18%|█▊        | 3320/18627 [1:46:33<8:29:05,  2.00s/it]
                                                        
{'loss': 1.0972, 'grad_norm': 7.657912731170654, 'learning_rate': 4.717404409687401e-06, 'epoch': 0.18}
+
 18%|█▊        | 3320/18627 [1:46:33<8:29:05,  2.00s/it]
 18%|█▊        | 3321/18627 [1:46:35<8:38:23,  2.03s/it]
 18%|█▊        | 3322/18627 [1:46:36<7:33:48,  1.78s/it]
 18%|█▊        | 3323/18627 [1:46:39<7:59:08,  1.88s/it]
 18%|█▊        | 3324/18627 [1:46:41<8:16:40,  1.95s/it]
 18%|█▊        | 3325/18627 [1:46:43<8:30:21,  2.00s/it]
 18%|█▊        | 3326/18627 [1:46:44<6:51:36,  1.61s/it]
 18%|█▊        | 3327/18627 [1:46:46<7:29:49,  1.76s/it]
 18%|█▊        | 3328/18627 [1:46:48<7:55:17,  1.86s/it]
 18%|█▊        | 3329/18627 [1:46:50<8:14:39,  1.94s/it]
 18%|█▊        | 3330/18627 [1:46:52<8:30:46,  2.00s/it]
                                                        
{'loss': 0.9075, 'grad_norm': 6.374989986419678, 'learning_rate': 4.715393473341583e-06, 'epoch': 0.18}
+
 18%|█▊        | 3330/18627 [1:46:52<8:30:46,  2.00s/it]
 18%|█▊        | 3331/18627 [1:46:54<8:40:16,  2.04s/it]
 18%|█▊        | 3332/18627 [1:46:56<8:47:12,  2.07s/it]
 18%|█▊        | 3333/18627 [1:46:58<8:51:51,  2.09s/it]
 18%|█▊        | 3334/18627 [1:47:00<8:53:26,  2.09s/it]
 18%|█▊        | 3335/18627 [1:47:03<8:55:44,  2.10s/it]
 18%|█▊        | 3336/18627 [1:47:04<7:45:51,  1.83s/it]
 18%|█▊        | 3337/18627 [1:47:06<8:07:44,  1.91s/it]
 18%|█▊        | 3338/18627 [1:47:08<8:23:12,  1.97s/it]
 18%|█▊        | 3339/18627 [1:47:10<8:34:48,  2.02s/it]
 18%|█▊        | 3340/18627 [1:47:11<7:31:13,  1.77s/it]
                                                        
{'loss': 1.1557, 'grad_norm': 16.391956329345703, 'learning_rate': 4.713375839224003e-06, 'epoch': 0.18}
+
 18%|█▊        | 3340/18627 [1:47:11<7:31:13,  1.77s/it]
 18%|█▊        | 3341/18627 [1:47:13<7:57:26,  1.87s/it]
 18%|█▊        | 3342/18627 [1:47:16<8:15:31,  1.95s/it]
 18%|█▊        | 3343/18627 [1:47:18<8:28:32,  2.00s/it]
 18%|█▊        | 3344/18627 [1:47:19<7:26:42,  1.75s/it]
 18%|█▊        | 3345/18627 [1:47:20<6:43:49,  1.59s/it]
 18%|█▊        | 3346/18627 [1:47:21<6:01:06,  1.42s/it]
 18%|█▊        | 3347/18627 [1:47:23<6:54:09,  1.63s/it]
 18%|█▊        | 3348/18627 [1:47:25<7:31:24,  1.77s/it]
 18%|█▊        | 3349/18627 [1:47:27<7:57:37,  1.88s/it]
 18%|█▊        | 3350/18627 [1:47:30<8:14:59,  1.94s/it]
                                                        
{'loss': 1.3464, 'grad_norm': 10.927242279052734, 'learning_rate': 4.711351513434549e-06, 'epoch': 0.18}
+
 18%|█▊        | 3350/18627 [1:47:30<8:14:59,  1.94s/it]
 18%|█▊        | 3351/18627 [1:47:31<7:16:55,  1.72s/it]
 18%|█▊        | 3352/18627 [1:47:32<6:36:56,  1.56s/it]
 18%|█▊        | 3353/18627 [1:47:33<6:02:11,  1.42s/it]
 18%|█▊        | 3354/18627 [1:47:35<6:54:50,  1.63s/it]
 18%|█▊        | 3355/18627 [1:47:37<7:31:12,  1.77s/it]
 18%|█▊        | 3356/18627 [1:47:39<7:55:31,  1.87s/it]
 18%|█▊        | 3357/18627 [1:47:41<8:15:06,  1.95s/it]
 18%|█▊        | 3358/18627 [1:47:44<8:29:02,  2.00s/it]
 18%|█▊        | 3359/18627 [1:47:46<8:37:54,  2.04s/it]
 18%|█▊        | 3360/18627 [1:47:48<8:43:16,  2.06s/it]
                                                        
{'loss': 1.2944, 'grad_norm': 22.445711135864258, 'learning_rate': 4.7093205020933405e-06, 'epoch': 0.18}
+
 18%|█▊        | 3360/18627 [1:47:48<8:43:16,  2.06s/it]
 18%|█▊        | 3361/18627 [1:47:49<6:59:08,  1.65s/it]
 18%|█▊        | 3362/18627 [1:47:50<6:24:15,  1.51s/it]
 18%|█▊        | 3363/18627 [1:47:50<5:22:46,  1.27s/it]
 18%|█▊        | 3364/18627 [1:47:53<6:27:37,  1.52s/it]
 18%|█▊        | 3365/18627 [1:47:55<7:12:57,  1.70s/it]
 18%|█▊        | 3366/18627 [1:47:57<7:43:30,  1.82s/it]
 18%|█▊        | 3367/18627 [1:47:59<8:05:58,  1.91s/it]
 18%|█▊        | 3368/18627 [1:48:01<8:21:35,  1.97s/it]
 18%|█▊        | 3369/18627 [1:48:03<8:32:48,  2.02s/it]
 18%|█▊        | 3370/18627 [1:48:05<8:40:30,  2.05s/it]
                                                        
{'loss': 0.9778, 'grad_norm': 7.17965030670166, 'learning_rate': 4.707282811340711e-06, 'epoch': 0.18}
+
 18%|█▊        | 3370/18627 [1:48:05<8:40:30,  2.05s/it]
 18%|█▊        | 3371/18627 [1:48:06<7:36:26,  1.80s/it]
 18%|█▊        | 3372/18627 [1:48:09<8:01:24,  1.89s/it]
 18%|█▊        | 3373/18627 [1:48:11<8:17:43,  1.96s/it]
 18%|█▊        | 3374/18627 [1:48:13<8:28:51,  2.00s/it]
 18%|█▊        | 3375/18627 [1:48:15<8:36:06,  2.03s/it]
 18%|█▊        | 3376/18627 [1:48:17<8:43:25,  2.06s/it]
 18%|█▊        | 3377/18627 [1:48:19<8:48:00,  2.08s/it]
 18%|█▊        | 3378/18627 [1:48:21<8:52:18,  2.09s/it]
 18%|█▊        | 3379/18627 [1:48:22<7:43:13,  1.82s/it]
 18%|█▊        | 3380/18627 [1:48:25<8:07:07,  1.92s/it]
                                                        
{'loss': 1.164, 'grad_norm': 5.98141622543335, 'learning_rate': 4.705238447337182e-06, 'epoch': 0.18}
+
 18%|█▊        | 3380/18627 [1:48:25<8:07:07,  1.92s/it]
 18%|█▊        | 3381/18627 [1:48:27<8:22:44,  1.98s/it]
 18%|█▊        | 3382/18627 [1:48:29<8:33:45,  2.02s/it]
 18%|█▊        | 3383/18627 [1:48:31<8:38:51,  2.04s/it]
 18%|█▊        | 3384/18627 [1:48:33<8:44:40,  2.07s/it]
 18%|█▊        | 3385/18627 [1:48:35<8:49:24,  2.08s/it]
 18%|█▊        | 3386/18627 [1:48:36<7:41:34,  1.82s/it]
 18%|█▊        | 3387/18627 [1:48:38<8:04:58,  1.91s/it]
 18%|█▊        | 3388/18627 [1:48:40<7:10:35,  1.70s/it]
 18%|█▊        | 3389/18627 [1:48:41<6:31:27,  1.54s/it]
 18%|█▊        | 3390/18627 [1:48:43<7:15:18,  1.71s/it]
                                                        
{'loss': 1.3916, 'grad_norm': 8.436300277709961, 'learning_rate': 4.703187416263458e-06, 'epoch': 0.18}
+
 18%|█▊        | 3390/18627 [1:48:43<7:15:18,  1.71s/it]
 18%|█▊        | 3391/18627 [1:48:44<6:36:19,  1.56s/it]
 18%|█▊        | 3392/18627 [1:48:46<7:17:41,  1.72s/it]
 18%|█▊        | 3393/18627 [1:48:48<7:47:42,  1.84s/it]
 18%|█▊        | 3394/18627 [1:48:50<6:58:01,  1.65s/it]
 18%|█▊        | 3395/18627 [1:48:52<7:35:15,  1.79s/it]
 18%|█▊        | 3396/18627 [1:48:53<6:35:39,  1.56s/it]
 18%|█▊        | 3397/18627 [1:48:54<6:08:08,  1.45s/it]
 18%|█▊        | 3398/18627 [1:48:56<6:58:02,  1.65s/it]
 18%|█▊        | 3399/18627 [1:48:57<6:23:04,  1.51s/it]
 18%|█▊        | 3400/18627 [1:48:58<5:21:27,  1.27s/it]
                                                        
{'loss': 1.7676, 'grad_norm': 7.561093330383301, 'learning_rate': 4.701129724320393e-06, 'epoch': 0.18}
+
 18%|█▊        | 3400/18627 [1:48:58<5:21:27,  1.27s/it]
 18%|█▊        | 3401/18627 [1:48:59<5:08:56,  1.22s/it]
 18%|█▊        | 3402/18627 [1:49:01<6:17:07,  1.49s/it]
 18%|█▊        | 3403/18627 [1:49:03<7:04:35,  1.67s/it]
 18%|█▊        | 3404/18627 [1:49:05<7:37:51,  1.80s/it]
 18%|█▊        | 3405/18627 [1:49:07<8:00:54,  1.90s/it]
 18%|█▊        | 3406/18627 [1:49:10<8:17:32,  1.96s/it]
 18%|█▊        | 3407/18627 [1:49:12<8:29:54,  2.01s/it]
 18%|█▊        | 3408/18627 [1:49:13<7:27:13,  1.76s/it]
 18%|█▊        | 3409/18627 [1:49:15<7:52:04,  1.86s/it]
 18%|█▊        | 3410/18627 [1:49:17<8:10:58,  1.94s/it]
                                                        
{'loss': 1.1369, 'grad_norm': 8.606691360473633, 'learning_rate': 4.699065377728983e-06, 'epoch': 0.18}
+
 18%|█▊        | 3410/18627 [1:49:17<8:10:58,  1.94s/it]
 18%|█▊        | 3411/18627 [1:49:19<8:24:47,  1.99s/it]
 18%|█▊        | 3412/18627 [1:49:21<8:34:01,  2.03s/it]
 18%|█▊        | 3413/18627 [1:49:23<8:39:22,  2.05s/it]
 18%|█▊        | 3414/18627 [1:49:26<8:42:54,  2.06s/it]
 18%|█▊        | 3415/18627 [1:49:27<7:36:04,  1.80s/it]
 18%|█▊        | 3416/18627 [1:49:29<7:59:27,  1.89s/it]
 18%|█▊        | 3417/18627 [1:49:31<8:17:46,  1.96s/it]
 18%|█▊        | 3418/18627 [1:49:32<7:19:38,  1.73s/it]
 18%|█▊        | 3419/18627 [1:49:33<6:00:47,  1.42s/it]
 18%|█▊        | 3420/18627 [1:49:35<6:53:11,  1.63s/it]
                                                        
{'loss': 1.1455, 'grad_norm': 8.222113609313965, 'learning_rate': 4.696994382730341e-06, 'epoch': 0.18}
+
 18%|█▊        | 3420/18627 [1:49:35<6:53:11,  1.63s/it]
 18%|█▊        | 3421/18627 [1:49:37<7:30:48,  1.78s/it]
 18%|█▊        | 3422/18627 [1:49:39<8:08:14,  1.93s/it]
 18%|█▊        | 3423/18627 [1:49:41<8:25:20,  1.99s/it]
 18%|█▊        | 3424/18627 [1:49:44<8:35:33,  2.03s/it]
 18%|█▊        | 3425/18627 [1:49:46<8:41:43,  2.06s/it]
 18%|█▊        | 3426/18627 [1:49:48<8:47:24,  2.08s/it]
 18%|█▊        | 3427/18627 [1:49:50<8:50:06,  2.09s/it]
 18%|█▊        | 3428/18627 [1:49:52<8:51:22,  2.10s/it]
 18%|█▊        | 3429/18627 [1:49:54<8:51:48,  2.10s/it]
 18%|█▊        | 3430/18627 [1:49:55<7:42:35,  1.83s/it]
                                                        
{'loss': 0.9619, 'grad_norm': 17.85607147216797, 'learning_rate': 4.694916745585681e-06, 'epoch': 0.18}
+
 18%|█▊        | 3430/18627 [1:49:55<7:42:35,  1.83s/it]
 18%|█▊        | 3431/18627 [1:49:57<6:53:59,  1.63s/it]
 18%|█▊        | 3432/18627 [1:54:31<352:22:12, 83.48s/it]
 18%|█▊        | 3433/18627 [1:54:33<249:20:19, 59.08s/it]
 18%|█▊        | 3434/18627 [1:54:34<176:02:16, 41.71s/it]
 18%|█▊        | 3435/18627 [1:54:36<125:53:41, 29.83s/it]
 18%|█▊        | 3436/18627 [1:54:38<89:38:21, 21.24s/it] 
 18%|█▊        | 3437/18627 [1:54:40<65:26:16, 15.51s/it]
 18%|█▊        | 3438/18627 [1:54:42<48:27:31, 11.49s/it]
 18%|█▊        | 3439/18627 [1:54:44<36:33:21,  8.66s/it]
 18%|█▊        | 3440/18627 [1:54:46<28:16:19,  6.70s/it]
                                                         
{'loss': 1.3925, 'grad_norm': 9.32919692993164, 'learning_rate': 4.692832472576298e-06, 'epoch': 0.18}
+
 18%|█▊        | 3440/18627 [1:54:46<28:16:19,  6.70s/it]
 18%|█▊        | 3441/18627 [1:54:48<22:27:54,  5.33s/it]
 18%|█▊        | 3442/18627 [1:54:49<17:13:36,  4.08s/it]
 18%|█▊        | 3443/18627 [1:54:52<14:46:04,  3.50s/it]
 18%|█▊        | 3444/18627 [1:54:54<13:00:23,  3.08s/it]
 18%|█▊        | 3445/18627 [1:54:56<11:47:34,  2.80s/it]
 19%|█▊        | 3446/18627 [1:54:57<9:45:51,  2.32s/it] 
 19%|█▊        | 3447/18627 [1:54:59<9:30:00,  2.25s/it]
 19%|█▊        | 3448/18627 [1:55:01<9:20:30,  2.22s/it]
 19%|█▊        | 3449/18627 [1:55:02<8:02:19,  1.91s/it]
 19%|█▊        | 3450/18627 [1:55:04<7:07:25,  1.69s/it]
                                                        
{'loss': 1.5476, 'grad_norm': 14.702792167663574, 'learning_rate': 4.690741570003548e-06, 'epoch': 0.19}
+
 19%|█▊        | 3450/18627 [1:55:04<7:07:25,  1.69s/it]
 19%|█▊        | 3451/18627 [1:55:05<6:30:00,  1.54s/it]
 19%|█▊        | 3452/18627 [1:55:05<5:26:07,  1.29s/it]
 19%|█▊        | 3453/18627 [1:55:08<6:27:17,  1.53s/it]
 19%|█▊        | 3454/18627 [1:55:10<7:13:50,  1.72s/it]
 19%|█▊        | 3455/18627 [1:55:11<6:32:41,  1.55s/it]
 19%|█▊        | 3456/18627 [1:55:13<7:16:03,  1.72s/it]
 19%|█▊        | 3457/18627 [1:55:15<7:46:39,  1.85s/it]
 19%|█▊        | 3458/18627 [1:55:16<6:56:36,  1.65s/it]
 19%|█▊        | 3459/18627 [1:55:18<7:32:50,  1.79s/it]
 19%|█▊        | 3460/18627 [1:55:19<6:09:49,  1.46s/it]
                                                        
{'loss': 1.3116, 'grad_norm': 7.033271312713623, 'learning_rate': 4.68864404418883e-06, 'epoch': 0.19}
+
 19%|█▊        | 3460/18627 [1:55:19<6:09:49,  1.46s/it]
 19%|█▊        | 3461/18627 [1:55:21<6:59:43,  1.66s/it]
 19%|█▊        | 3462/18627 [1:55:23<7:32:59,  1.79s/it]
 19%|█▊        | 3463/18627 [1:55:26<7:57:37,  1.89s/it]
 19%|█▊        | 3464/18627 [1:55:28<8:14:44,  1.96s/it]
 19%|█▊        | 3465/18627 [1:55:30<8:27:36,  2.01s/it]
 19%|█▊        | 3466/18627 [1:55:32<8:36:01,  2.04s/it]
 19%|█▊        | 3467/18627 [1:55:33<7:32:07,  1.79s/it]
 19%|█▊        | 3468/18627 [1:55:35<7:59:11,  1.90s/it]
 19%|█▊        | 3469/18627 [1:55:37<8:15:14,  1.96s/it]
 19%|█▊        | 3470/18627 [1:55:38<7:08:35,  1.70s/it]
                                                        
{'loss': 1.1645, 'grad_norm': 14.276156425476074, 'learning_rate': 4.686539901473572e-06, 'epoch': 0.19}
+
 19%|█▊        | 3470/18627 [1:55:38<7:08:35,  1.70s/it]
 19%|█▊        | 3471/18627 [1:55:41<7:40:03,  1.82s/it]
 19%|█▊        | 3472/18627 [1:55:43<8:02:46,  1.91s/it]
 19%|█▊        | 3473/18627 [1:55:45<8:18:07,  1.97s/it]
 19%|█▊        | 3474/18627 [1:55:47<8:27:53,  2.01s/it]
 19%|█▊        | 3475/18627 [1:55:49<8:36:22,  2.04s/it]
 19%|█▊        | 3476/18627 [1:55:51<8:41:17,  2.06s/it]
 19%|█▊        | 3477/18627 [1:55:53<8:44:11,  2.08s/it]
 19%|█▊        | 3478/18627 [1:55:55<8:45:51,  2.08s/it]
 19%|█▊        | 3479/18627 [1:55:57<8:48:36,  2.09s/it]
 19%|█▊        | 3480/18627 [1:56:00<8:50:50,  2.10s/it]
                                                        
{'loss': 0.7534, 'grad_norm': 9.640213012695312, 'learning_rate': 4.684429148219199e-06, 'epoch': 0.19}
+
 19%|█▊        | 3480/18627 [1:56:00<8:50:50,  2.10s/it]
 19%|█▊        | 3481/18627 [1:56:02<8:51:55,  2.11s/it]
 19%|█▊        | 3482/18627 [1:56:03<7:42:09,  1.83s/it]
 19%|█▊        | 3483/18627 [1:56:05<8:03:45,  1.92s/it]
 19%|█▊        | 3484/18627 [1:56:07<8:20:32,  1.98s/it]
 19%|█▊        | 3485/18627 [1:56:08<7:20:11,  1.74s/it]
 19%|█▊        | 3486/18627 [1:56:10<7:49:54,  1.86s/it]
 19%|█▊        | 3487/18627 [1:56:13<8:09:55,  1.94s/it]
 19%|█▊        | 3488/18627 [1:56:15<8:22:49,  1.99s/it]
 19%|█▊        | 3489/18627 [1:56:17<8:32:55,  2.03s/it]
 19%|█▊        | 3490/18627 [1:56:18<7:20:45,  1.75s/it]
                                                        
{'loss': 1.4295, 'grad_norm': 15.213176727294922, 'learning_rate': 4.6823117908071265e-06, 'epoch': 0.19}
+
 19%|█▊        | 3490/18627 [1:56:18<7:20:45,  1.75s/it]
 19%|█▊        | 3491/18627 [1:56:20<7:48:20,  1.86s/it]
 19%|█▊        | 3492/18627 [1:56:22<8:09:05,  1.94s/it]
 19%|█▉        | 3493/18627 [1:56:23<7:12:29,  1.71s/it]
 19%|█▉        | 3494/18627 [1:56:25<7:43:41,  1.84s/it]
 19%|█▉        | 3495/18627 [1:56:28<8:04:18,  1.92s/it]
 19%|█▉        | 3496/18627 [1:56:30<8:18:17,  1.98s/it]
 19%|█▉        | 3497/18627 [1:56:32<8:28:36,  2.02s/it]
 19%|█▉        | 3498/18627 [1:56:33<7:27:09,  1.77s/it]
 19%|█▉        | 3499/18627 [1:56:35<7:53:14,  1.88s/it]
 19%|█▉        | 3500/18627 [1:56:37<8:12:00,  1.95s/it]
                                                        
{'loss': 1.1231, 'grad_norm': 8.654376983642578, 'learning_rate': 4.6801878356387345e-06, 'epoch': 0.19}
+
 19%|█▉        | 3500/18627 [1:56:37<8:12:00,  1.95s/it]
 19%|█▉        | 3501/18627 [1:56:39<8:23:06,  2.00s/it]
 19%|█▉        | 3502/18627 [1:56:41<8:32:34,  2.03s/it]
 19%|█▉        | 3503/18627 [1:56:44<8:40:04,  2.06s/it]
 19%|█▉        | 3504/18627 [1:56:46<8:44:45,  2.08s/it]
 19%|█▉        | 3505/18627 [1:56:47<7:36:52,  1.81s/it]
 19%|█▉        | 3506/18627 [1:56:48<6:12:16,  1.48s/it]
 19%|█▉        | 3507/18627 [1:56:49<5:50:07,  1.39s/it]
 19%|█▉        | 3508/18627 [1:56:51<6:45:27,  1.61s/it]
 19%|█▉        | 3509/18627 [1:56:53<7:24:19,  1.76s/it]
 19%|█▉        | 3510/18627 [1:56:55<7:51:50,  1.87s/it]
                                                        
{'loss': 1.0757, 'grad_norm': 10.635987281799316, 'learning_rate': 4.678057289135351e-06, 'epoch': 0.19}
+
 19%|█▉        | 3510/18627 [1:56:55<7:51:50,  1.87s/it]
 19%|█▉        | 3511/18627 [1:56:56<7:00:34,  1.67s/it]
 19%|█▉        | 3512/18627 [1:56:58<7:32:28,  1.80s/it]
 19%|█▉        | 3513/18627 [1:57:01<7:56:49,  1.89s/it]
 19%|█▉        | 3514/18627 [1:57:03<8:15:01,  1.97s/it]
 19%|█▉        | 3515/18627 [1:57:05<8:25:34,  2.01s/it]
 19%|█▉        | 3516/18627 [1:57:07<8:36:20,  2.05s/it]
 19%|█▉        | 3517/18627 [1:57:08<7:31:30,  1.79s/it]
 19%|█▉        | 3518/18627 [1:57:10<7:55:41,  1.89s/it]
 19%|█▉        | 3519/18627 [1:57:12<8:13:32,  1.96s/it]
 19%|█▉        | 3520/18627 [1:57:13<6:41:51,  1.60s/it]
                                                        
{'loss': 1.0959, 'grad_norm': 9.857612609863281, 'learning_rate': 4.675920157738232e-06, 'epoch': 0.19}
+
 19%|█▉        | 3520/18627 [1:57:13<6:41:51,  1.60s/it]
 19%|█▉        | 3521/18627 [1:57:15<7:24:49,  1.77s/it]
 19%|█▉        | 3522/18627 [1:57:17<7:54:52,  1.89s/it]
 19%|█▉        | 3523/18627 [1:57:20<8:14:53,  1.97s/it]
 19%|█▉        | 3524/18627 [1:57:21<7:17:18,  1.74s/it]
 19%|█▉        | 3525/18627 [1:57:23<7:45:27,  1.85s/it]
 19%|█▉        | 3526/18627 [1:57:24<6:56:09,  1.65s/it]
 19%|█▉        | 3527/18627 [1:57:25<6:21:41,  1.52s/it]
 19%|█▉        | 3528/18627 [1:57:27<7:05:58,  1.69s/it]
 19%|█▉        | 3529/18627 [1:57:30<7:39:03,  1.82s/it]
 19%|█▉        | 3530/18627 [1:57:32<8:01:01,  1.91s/it]
                                                        
{'loss': 1.468, 'grad_norm': 8.886661529541016, 'learning_rate': 4.673776447908538e-06, 'epoch': 0.19}
+
 19%|█▉        | 3530/18627 [1:57:32<8:01:01,  1.91s/it]
 19%|█▉        | 3531/18627 [1:57:34<8:16:56,  1.98s/it]
 19%|█▉        | 3532/18627 [1:57:36<8:26:24,  2.01s/it]
 19%|█▉        | 3533/18627 [1:57:38<8:33:40,  2.04s/it]
 19%|█▉        | 3534/18627 [1:57:40<8:39:17,  2.06s/it]
 19%|█▉        | 3535/18627 [1:57:42<8:43:02,  2.08s/it]
 19%|█▉        | 3536/18627 [1:57:44<8:47:59,  2.10s/it]
 19%|█▉        | 3537/18627 [1:57:46<8:49:01,  2.10s/it]
 19%|█▉        | 3538/18627 [1:57:49<8:51:49,  2.11s/it]
 19%|█▉        | 3539/18627 [1:57:51<8:50:32,  2.11s/it]
 19%|█▉        | 3540/18627 [1:57:52<7:41:43,  1.84s/it]
                                                        
{'loss': 0.9711, 'grad_norm': 16.391977310180664, 'learning_rate': 4.671626166127323e-06, 'epoch': 0.19}
+
 19%|█▉        | 3540/18627 [1:57:52<7:41:43,  1.84s/it]
 19%|█▉        | 3541/18627 [1:57:54<8:03:44,  1.92s/it]
 19%|█▉        | 3542/18627 [1:57:56<8:18:29,  1.98s/it]
 19%|█▉        | 3543/18627 [1:57:58<8:28:49,  2.02s/it]
 19%|█▉        | 3544/18627 [1:58:00<8:36:34,  2.05s/it]
 19%|█▉        | 3545/18627 [1:58:03<8:41:14,  2.07s/it]
 19%|█▉        | 3546/18627 [1:58:04<7:34:22,  1.81s/it]
 19%|█▉        | 3547/18627 [1:58:05<6:47:11,  1.62s/it]
 19%|█▉        | 3548/18627 [1:58:07<7:23:48,  1.77s/it]
 19%|█▉        | 3549/18627 [1:58:09<7:49:23,  1.87s/it]
 19%|█▉        | 3550/18627 [1:58:11<8:08:22,  1.94s/it]
                                                        
{'loss': 1.0924, 'grad_norm': 7.238748550415039, 'learning_rate': 4.669469318895505e-06, 'epoch': 0.19}
+
 19%|█▉        | 3550/18627 [1:58:11<8:08:22,  1.94s/it]
 19%|█▉        | 3551/18627 [1:58:12<7:11:50,  1.72s/it]
 19%|█▉        | 3552/18627 [1:58:14<6:32:29,  1.56s/it]
 19%|█▉        | 3553/18627 [1:58:16<7:16:16,  1.74s/it]
 19%|█▉        | 3554/18627 [1:58:18<7:48:00,  1.86s/it]
 19%|█▉        | 3555/18627 [1:58:19<6:57:22,  1.66s/it]
 19%|█▉        | 3556/18627 [1:58:20<6:22:04,  1.52s/it]
 19%|█▉        | 3557/18627 [1:58:22<7:05:33,  1.69s/it]
 19%|█▉        | 3558/18627 [1:58:24<7:37:23,  1.82s/it]
 19%|█▉        | 3559/18627 [1:58:27<7:58:42,  1.91s/it]
 19%|█▉        | 3560/18627 [1:58:28<7:01:17,  1.68s/it]
                                                        
{'loss': 1.6906, 'grad_norm': 15.763091087341309, 'learning_rate': 4.667305912733856e-06, 'epoch': 0.19}
+
 19%|█▉        | 3560/18627 [1:58:28<7:01:17,  1.68s/it]
 19%|█▉        | 3561/18627 [1:58:30<7:33:56,  1.81s/it]
 19%|█▉        | 3562/18627 [1:58:32<7:57:35,  1.90s/it]
 19%|█▉        | 3563/18627 [1:58:33<7:03:52,  1.69s/it]
 19%|█▉        | 3564/18627 [1:58:35<7:33:59,  1.81s/it]
 19%|█▉        | 3565/18627 [1:58:37<7:55:47,  1.90s/it]
 19%|█▉        | 3566/18627 [1:58:38<6:58:30,  1.67s/it]
 19%|█▉        | 3567/18627 [1:58:39<5:45:28,  1.38s/it]
 19%|█▉        | 3568/18627 [1:58:41<6:40:54,  1.60s/it]
 19%|█▉        | 3569/18627 [1:58:43<7:19:48,  1.75s/it]
 19%|█▉        | 3570/18627 [1:58:45<6:37:47,  1.59s/it]
                                                        
{'loss': 1.3764, 'grad_norm': 14.378523826599121, 'learning_rate': 4.665135954182974e-06, 'epoch': 0.19}
+
 19%|█▉        | 3570/18627 [1:58:45<6:37:47,  1.59s/it]
 19%|█▉        | 3571/18627 [1:58:46<6:07:56,  1.47s/it]
 19%|█▉        | 3572/18627 [1:58:47<5:38:57,  1.35s/it]
 19%|█▉        | 3573/18627 [1:58:48<5:27:31,  1.31s/it]
 19%|█▉        | 3574/18627 [1:58:50<6:28:22,  1.55s/it]
 19%|█▉        | 3575/18627 [1:58:52<7:13:15,  1.73s/it]
 19%|█▉        | 3576/18627 [1:58:54<6:32:46,  1.57s/it]
 19%|█▉        | 3577/18627 [1:58:56<7:14:39,  1.73s/it]
 19%|█▉        | 3578/18627 [1:58:58<7:43:28,  1.85s/it]
 19%|█▉        | 3579/18627 [1:59:00<8:06:36,  1.94s/it]
 19%|█▉        | 3580/18627 [1:59:02<8:21:15,  2.00s/it]
                                                        
{'loss': 1.5632, 'grad_norm': 8.820269584655762, 'learning_rate': 4.6629594498032674e-06, 'epoch': 0.19}
+
 19%|█▉        | 3580/18627 [1:59:02<8:21:15,  2.00s/it]
 19%|█▉        | 3581/18627 [1:59:04<8:28:39,  2.03s/it]
 19%|█▉        | 3582/18627 [1:59:06<8:35:31,  2.06s/it]
 19%|█▉        | 3583/18627 [1:59:07<7:30:36,  1.80s/it]
 19%|█▉        | 3584/18627 [1:59:10<7:55:49,  1.90s/it]
 19%|█▉        | 3585/18627 [1:59:12<8:12:18,  1.96s/it]
 19%|█▉        | 3586/18627 [1:59:14<8:24:30,  2.01s/it]
 19%|█▉        | 3587/18627 [1:59:16<8:32:32,  2.04s/it]
 19%|█▉        | 3588/18627 [1:59:18<8:39:09,  2.07s/it]
 19%|█▉        | 3589/18627 [1:59:20<8:42:57,  2.09s/it]
 19%|█▉        | 3590/18627 [1:59:21<7:35:27,  1.82s/it]
                                                        
{'loss': 1.0799, 'grad_norm': 16.15110206604004, 'learning_rate': 4.660776406174936e-06, 'epoch': 0.19}
+
 19%|█▉        | 3590/18627 [1:59:21<7:35:27,  1.82s/it]
 19%|█▉        | 3591/18627 [1:59:24<7:58:07,  1.91s/it]
 19%|█▉        | 3592/18627 [1:59:26<8:14:32,  1.97s/it]
 19%|█▉        | 3593/18627 [1:59:28<8:25:19,  2.02s/it]
 19%|█▉        | 3594/18627 [1:59:30<8:30:45,  2.04s/it]
 19%|█▉        | 3595/18627 [1:59:32<8:37:21,  2.07s/it]
 19%|█▉        | 3596/18627 [1:59:33<7:32:10,  1.80s/it]
 19%|█▉        | 3597/18627 [1:59:35<7:59:02,  1.91s/it]
 19%|█▉        | 3598/18627 [1:59:37<8:14:44,  1.98s/it]
 19%|█▉        | 3599/18627 [1:59:39<7:16:58,  1.74s/it]
 19%|█▉        | 3600/18627 [1:59:40<6:35:54,  1.58s/it]
                                                        
{'loss': 1.3501, 'grad_norm': 17.78586196899414, 'learning_rate': 4.658586829897947e-06, 'epoch': 0.19}
+
 19%|█▉        | 3600/18627 [1:59:40<6:35:54,  1.58s/it]
 19%|█▉        | 3601/18627 [1:59:42<7:15:29,  1.74s/it]
 19%|█▉        | 3602/18627 [1:59:43<6:34:04,  1.57s/it]
 19%|█▉        | 3603/18627 [1:59:44<6:05:53,  1.46s/it]
 19%|█▉        | 3604/18627 [1:59:46<6:54:00,  1.65s/it]
 19%|█▉        | 3605/18627 [1:59:49<7:28:08,  1.79s/it]
 19%|█▉        | 3606/18627 [1:59:51<7:52:59,  1.89s/it]
 19%|█▉        | 3607/18627 [1:59:52<6:52:27,  1.65s/it]
 19%|█▉        | 3608/18627 [1:59:54<7:28:24,  1.79s/it]
 19%|█▉        | 3609/18627 [1:59:55<6:42:26,  1.61s/it]
 19%|█▉        | 3610/18627 [1:59:57<7:21:24,  1.76s/it]
                                                        
{'loss': 1.591, 'grad_norm': 6.3675079345703125, 'learning_rate': 4.656390727592023e-06, 'epoch': 0.19}
+
 19%|█▉        | 3610/18627 [1:59:57<7:21:24,  1.76s/it]
 19%|█▉        | 3611/18627 [1:59:59<7:48:20,  1.87s/it]
 19%|█▉        | 3612/18627 [2:00:00<6:21:30,  1.52s/it]
 19%|█▉        | 3613/18627 [2:00:02<7:07:09,  1.71s/it]
 19%|█▉        | 3614/18627 [2:00:03<6:27:31,  1.55s/it]
 19%|█▉        | 3615/18627 [2:00:06<7:12:28,  1.73s/it]
 19%|█▉        | 3616/18627 [2:00:07<6:32:16,  1.57s/it]
 19%|█▉        | 3617/18627 [2:00:09<7:13:32,  1.73s/it]
 19%|█▉        | 3618/18627 [2:00:11<7:42:45,  1.85s/it]
 19%|█▉        | 3619/18627 [2:00:13<8:02:59,  1.93s/it]
 19%|█▉        | 3620/18627 [2:00:15<8:17:30,  1.99s/it]
                                                        
{'loss': 1.1597, 'grad_norm': 8.369041442871094, 'learning_rate': 4.654188105896609e-06, 'epoch': 0.19}
+
 19%|█▉        | 3620/18627 [2:00:15<8:17:30,  1.99s/it]
 19%|█▉        | 3621/18627 [2:00:16<7:17:07,  1.75s/it]
 19%|█▉        | 3622/18627 [2:00:18<6:34:58,  1.58s/it]
 19%|█▉        | 3623/18627 [2:00:19<6:05:59,  1.46s/it]
 19%|█▉        | 3624/18627 [2:00:21<6:53:40,  1.65s/it]
 19%|█▉        | 3625/18627 [2:00:23<7:29:02,  1.80s/it]
 19%|█▉        | 3626/18627 [2:00:25<7:53:07,  1.89s/it]
 19%|█▉        | 3627/18627 [2:00:27<8:11:04,  1.96s/it]
 19%|█▉        | 3628/18627 [2:00:29<8:23:22,  2.01s/it]
 19%|█▉        | 3629/18627 [2:00:31<8:30:47,  2.04s/it]
 19%|█▉        | 3630/18627 [2:00:34<8:37:05,  2.07s/it]
                                                        
{'loss': 1.2871, 'grad_norm': 8.07697868347168, 'learning_rate': 4.651978971470865e-06, 'epoch': 0.19}
+
 19%|█▉        | 3630/18627 [2:00:34<8:37:05,  2.07s/it]
 19%|█▉        | 3631/18627 [2:00:36<8:40:15,  2.08s/it]
 19%|█▉        | 3632/18627 [2:00:38<8:44:03,  2.10s/it]
 20%|█▉        | 3633/18627 [2:00:40<8:44:26,  2.10s/it]
 20%|█▉        | 3634/18627 [2:00:42<8:46:31,  2.11s/it]
 20%|█▉        | 3635/18627 [2:00:44<8:47:49,  2.11s/it]
 20%|█▉        | 3636/18627 [2:00:46<8:48:10,  2.11s/it]
 20%|█▉        | 3637/18627 [2:00:48<8:47:51,  2.11s/it]
 20%|█▉        | 3638/18627 [2:00:51<8:47:56,  2.11s/it]
 20%|█▉        | 3639/18627 [2:00:53<8:50:00,  2.12s/it]
 20%|█▉        | 3640/18627 [2:00:55<8:49:18,  2.12s/it]
                                                        
{'loss': 0.715, 'grad_norm': 6.705743789672852, 'learning_rate': 4.649763330993642e-06, 'epoch': 0.2}
+
 20%|█▉        | 3640/18627 [2:00:55<8:49:18,  2.12s/it]
 20%|█▉        | 3641/18627 [2:00:57<8:51:01,  2.13s/it]
 20%|█▉        | 3642/18627 [2:00:58<7:40:45,  1.84s/it]
 20%|█▉        | 3643/18627 [2:00:59<6:14:55,  1.50s/it]
 20%|█▉        | 3644/18627 [2:01:01<6:59:58,  1.68s/it]
 20%|█▉        | 3645/18627 [2:01:03<7:32:09,  1.81s/it]
 20%|█▉        | 3646/18627 [2:01:05<7:54:58,  1.90s/it]
 20%|█▉        | 3647/18627 [2:01:06<7:01:27,  1.69s/it]
 20%|█▉        | 3648/18627 [2:01:08<7:33:56,  1.82s/it]
 20%|█▉        | 3649/18627 [2:01:11<7:56:00,  1.91s/it]
 20%|█▉        | 3650/18627 [2:01:13<8:11:55,  1.97s/it]
                                                        
{'loss': 1.141, 'grad_norm': 9.2643404006958, 'learning_rate': 4.647541191163457e-06, 'epoch': 0.2}
+
 20%|█▉        | 3650/18627 [2:01:13<8:11:55,  1.97s/it]
 20%|█▉        | 3651/18627 [2:01:15<8:22:54,  2.01s/it]
 20%|█▉        | 3652/18627 [2:01:16<7:20:23,  1.76s/it]
 20%|█▉        | 3653/18627 [2:01:18<7:46:17,  1.87s/it]
 20%|█▉        | 3654/18627 [2:01:20<8:05:38,  1.95s/it]
 20%|█▉        | 3655/18627 [2:01:22<8:18:15,  2.00s/it]
 20%|█▉        | 3656/18627 [2:01:24<8:27:22,  2.03s/it]
 20%|█▉        | 3657/18627 [2:01:25<6:47:03,  1.63s/it]
 20%|█▉        | 3658/18627 [2:01:27<7:22:43,  1.77s/it]
 20%|█▉        | 3659/18627 [2:01:29<7:47:57,  1.88s/it]
 20%|█▉        | 3660/18627 [2:01:32<8:05:08,  1.94s/it]
                                                        
{'loss': 0.9469, 'grad_norm': 7.585899829864502, 'learning_rate': 4.645312558698477e-06, 'epoch': 0.2}
+
 20%|█▉        | 3660/18627 [2:01:32<8:05:08,  1.94s/it]
 20%|█▉        | 3661/18627 [2:01:34<8:16:35,  1.99s/it]
 20%|█▉        | 3662/18627 [2:01:36<8:25:40,  2.03s/it]
 20%|█▉        | 3663/18627 [2:01:38<8:33:27,  2.06s/it]
 20%|█▉        | 3664/18627 [2:01:40<8:35:41,  2.07s/it]
 20%|█▉        | 3665/18627 [2:01:42<8:39:59,  2.09s/it]
 20%|█▉        | 3666/18627 [2:01:44<8:41:43,  2.09s/it]
 20%|█▉        | 3667/18627 [2:01:46<8:44:57,  2.11s/it]
 20%|█▉        | 3668/18627 [2:01:48<8:46:44,  2.11s/it]
 20%|█▉        | 3669/18627 [2:01:51<8:48:07,  2.12s/it]
 20%|█▉        | 3670/18627 [2:01:53<8:49:43,  2.13s/it]
                                                        
{'loss': 0.6867, 'grad_norm': 8.355679512023926, 'learning_rate': 4.643077440336501e-06, 'epoch': 0.2}
+
 20%|█▉        | 3670/18627 [2:01:53<8:49:43,  2.13s/it]
 20%|█▉        | 3671/18627 [2:01:54<7:39:17,  1.84s/it]
 20%|█▉        | 3672/18627 [2:01:56<8:01:56,  1.93s/it]
 20%|█▉        | 3673/18627 [2:01:58<8:15:07,  1.99s/it]
 20%|█▉        | 3674/18627 [2:02:00<8:24:46,  2.03s/it]
 20%|█▉        | 3675/18627 [2:02:02<8:32:17,  2.06s/it]
 20%|█▉        | 3676/18627 [2:02:05<8:37:47,  2.08s/it]
 20%|█▉        | 3677/18627 [2:02:07<8:42:22,  2.10s/it]
 20%|█▉        | 3678/18627 [2:02:09<8:44:18,  2.10s/it]
 20%|█▉        | 3679/18627 [2:02:11<8:45:21,  2.11s/it]
 20%|█▉        | 3680/18627 [2:02:13<8:46:01,  2.11s/it]
                                                        
{'loss': 0.8878, 'grad_norm': 7.4701738357543945, 'learning_rate': 4.640835842834933e-06, 'epoch': 0.2}
+
 20%|█▉        | 3680/18627 [2:02:13<8:46:01,  2.11s/it]
 20%|█▉        | 3681/18627 [2:02:15<8:46:50,  2.11s/it]
 20%|█▉        | 3682/18627 [2:02:17<8:47:08,  2.12s/it]
 20%|█▉        | 3683/18627 [2:02:19<8:46:55,  2.12s/it]
 20%|█▉        | 3684/18627 [2:02:21<8:47:15,  2.12s/it]
 20%|█▉        | 3685/18627 [2:02:24<8:47:46,  2.12s/it]
 20%|█▉        | 3686/18627 [2:02:26<8:48:46,  2.12s/it]
 20%|█▉        | 3687/18627 [2:02:28<8:48:43,  2.12s/it]
 20%|█▉        | 3688/18627 [2:02:29<7:39:24,  1.85s/it]
 20%|█▉        | 3689/18627 [2:02:31<7:59:46,  1.93s/it]
 20%|█▉        | 3690/18627 [2:02:33<8:12:50,  1.98s/it]
                                                        
{'loss': 0.87, 'grad_norm': 7.792985439300537, 'learning_rate': 4.638587772970768e-06, 'epoch': 0.2}
+
 20%|█▉        | 3690/18627 [2:02:33<8:12:50,  1.98s/it]
 20%|█▉        | 3691/18627 [2:02:35<8:22:54,  2.02s/it]
 20%|█▉        | 3692/18627 [2:02:38<8:29:20,  2.05s/it]
 20%|█▉        | 3693/18627 [2:02:40<8:36:40,  2.08s/it]
 20%|█▉        | 3694/18627 [2:02:42<8:41:29,  2.10s/it]
 20%|█▉        | 3695/18627 [2:02:43<7:33:35,  1.82s/it]
 20%|█▉        | 3696/18627 [2:02:45<7:56:12,  1.91s/it]
 20%|█▉        | 3697/18627 [2:02:47<8:12:34,  1.98s/it]
 20%|█▉        | 3698/18627 [2:02:49<8:23:44,  2.02s/it]
 20%|█▉        | 3699/18627 [2:02:51<8:30:20,  2.05s/it]
 20%|█▉        | 3700/18627 [2:02:54<8:35:48,  2.07s/it]
                                                        
{'loss': 0.8852, 'grad_norm': 8.34260368347168, 'learning_rate': 4.636333237540568e-06, 'epoch': 0.2}
+
 20%|█▉        | 3700/18627 [2:02:54<8:35:48,  2.07s/it]
 20%|█▉        | 3701/18627 [2:02:56<8:39:43,  2.09s/it]
 20%|█▉        | 3702/18627 [2:02:58<8:42:23,  2.10s/it]
 20%|█▉        | 3703/18627 [2:03:00<8:43:04,  2.10s/it]
 20%|█▉        | 3704/18627 [2:03:02<8:43:22,  2.10s/it]
 20%|█▉        | 3705/18627 [2:03:04<8:44:41,  2.11s/it]
 20%|█▉        | 3706/18627 [2:03:06<8:44:45,  2.11s/it]
 20%|█▉        | 3707/18627 [2:03:08<8:44:25,  2.11s/it]
 20%|█▉        | 3708/18627 [2:03:10<7:35:50,  1.83s/it]
 20%|█▉        | 3709/18627 [2:03:12<7:56:31,  1.92s/it]
 20%|█▉        | 3710/18627 [2:03:14<8:09:52,  1.97s/it]
                                                        
{'loss': 0.9154, 'grad_norm': 10.951787948608398, 'learning_rate': 4.634072243360442e-06, 'epoch': 0.2}
+
 20%|█▉        | 3710/18627 [2:03:14<8:09:52,  1.97s/it]
 20%|█▉        | 3711/18627 [2:03:16<8:20:59,  2.02s/it]
 20%|█▉        | 3712/18627 [2:03:18<8:27:15,  2.04s/it]
 20%|█▉        | 3713/18627 [2:03:20<8:32:51,  2.06s/it]
 20%|█▉        | 3714/18627 [2:03:21<7:27:38,  1.80s/it]
 20%|█▉        | 3715/18627 [2:03:23<6:41:47,  1.62s/it]
 20%|█▉        | 3716/18627 [2:03:25<7:20:32,  1.77s/it]
 20%|█▉        | 3717/18627 [2:03:27<7:44:52,  1.87s/it]
 20%|█▉        | 3718/18627 [2:03:29<8:02:27,  1.94s/it]
 20%|█▉        | 3719/18627 [2:03:31<8:16:05,  2.00s/it]
 20%|█▉        | 3720/18627 [2:03:32<7:16:44,  1.76s/it]
                                                        
{'loss': 1.4312, 'grad_norm': 16.14868927001953, 'learning_rate': 4.631804797266025e-06, 'epoch': 0.2}
+
 20%|█▉        | 3720/18627 [2:03:32<7:16:44,  1.76s/it]
 20%|█▉        | 3721/18627 [2:03:34<7:44:26,  1.87s/it]
 20%|█▉        | 3722/18627 [2:03:36<8:04:16,  1.95s/it]
 20%|█▉        | 3723/18627 [2:03:39<8:16:46,  2.00s/it]
 20%|█▉        | 3724/18627 [2:03:41<8:25:39,  2.04s/it]
 20%|█▉        | 3725/18627 [2:03:42<7:23:01,  1.78s/it]
 20%|██        | 3726/18627 [2:03:43<6:32:08,  1.58s/it]
 20%|██        | 3727/18627 [2:03:45<7:13:53,  1.75s/it]
 20%|██        | 3728/18627 [2:03:47<7:40:23,  1.85s/it]
 20%|██        | 3729/18627 [2:03:48<6:50:59,  1.66s/it]
 20%|██        | 3730/18627 [2:03:50<6:16:31,  1.52s/it]
                                                        
{'loss': 1.4565, 'grad_norm': 14.808999061584473, 'learning_rate': 4.62953090611246e-06, 'epoch': 0.2}
+
 20%|██        | 3730/18627 [2:03:50<6:16:31,  1.52s/it]
 20%|██        | 3731/18627 [2:03:52<7:01:57,  1.70s/it]
 20%|██        | 3732/18627 [2:03:54<7:32:06,  1.82s/it]
 20%|██        | 3733/18627 [2:03:56<7:54:49,  1.91s/it]
 20%|██        | 3734/18627 [2:03:57<7:01:11,  1.70s/it]
 20%|██        | 3735/18627 [2:03:59<7:33:05,  1.83s/it]
 20%|██        | 3736/18627 [2:04:01<7:54:43,  1.91s/it]
 20%|██        | 3737/18627 [2:04:04<8:10:07,  1.98s/it]
 20%|██        | 3738/18627 [2:04:05<7:11:17,  1.74s/it]
 20%|██        | 3739/18627 [2:04:07<7:38:19,  1.85s/it]
 20%|██        | 3740/18627 [2:04:08<6:13:08,  1.50s/it]
                                                        
{'loss': 1.1886, 'grad_norm': 6.217281341552734, 'learning_rate': 4.6272505767743745e-06, 'epoch': 0.2}
+
 20%|██        | 3740/18627 [2:04:08<6:13:08,  1.50s/it]
 20%|██        | 3741/18627 [2:04:09<5:49:43,  1.41s/it]
 20%|██        | 3742/18627 [2:04:11<6:43:10,  1.63s/it]
 20%|██        | 3743/18627 [2:04:13<7:18:09,  1.77s/it]
 20%|██        | 3744/18627 [2:04:15<7:43:47,  1.87s/it]
 20%|██        | 3745/18627 [2:04:16<6:55:17,  1.67s/it]
 20%|██        | 3746/18627 [2:04:18<7:28:26,  1.81s/it]
 20%|██        | 3747/18627 [2:04:21<7:50:46,  1.90s/it]
 20%|██        | 3748/18627 [2:04:23<8:07:11,  1.96s/it]
 20%|██        | 3749/18627 [2:04:25<8:19:31,  2.01s/it]
 20%|██        | 3750/18627 [2:04:27<8:27:31,  2.05s/it]
                                                        
{'loss': 1.0689, 'grad_norm': 6.214573860168457, 'learning_rate': 4.624963816145858e-06, 'epoch': 0.2}
+
 20%|██        | 3750/18627 [2:04:27<8:27:31,  2.05s/it]
 20%|██        | 3751/18627 [2:04:29<8:34:02,  2.07s/it]
 20%|██        | 3752/18627 [2:04:31<8:37:20,  2.09s/it]
 20%|██        | 3753/18627 [2:04:32<7:32:11,  1.82s/it]
 20%|██        | 3754/18627 [2:04:34<7:53:21,  1.91s/it]
 20%|██        | 3755/18627 [2:04:37<8:09:51,  1.98s/it]
 20%|██        | 3756/18627 [2:04:39<8:20:19,  2.02s/it]
 20%|██        | 3757/18627 [2:04:41<8:26:42,  2.04s/it]
 20%|██        | 3758/18627 [2:04:43<8:33:07,  2.07s/it]
 20%|██        | 3759/18627 [2:04:44<6:51:02,  1.66s/it]
 20%|██        | 3760/18627 [2:04:46<7:26:07,  1.80s/it]
                                                        
{'loss': 0.927, 'grad_norm': 13.440322875976562, 'learning_rate': 4.622670631140447e-06, 'epoch': 0.2}
+
 20%|██        | 3760/18627 [2:04:46<7:26:07,  1.80s/it]
 20%|██        | 3761/18627 [2:04:48<7:49:33,  1.90s/it]
 20%|██        | 3762/18627 [2:04:50<8:06:43,  1.96s/it]
 20%|██        | 3763/18627 [2:04:51<6:32:55,  1.59s/it]
 20%|██        | 3764/18627 [2:04:53<7:13:07,  1.75s/it]
 20%|██        | 3765/18627 [2:04:55<7:39:45,  1.86s/it]
 20%|██        | 3766/18627 [2:04:57<7:58:15,  1.93s/it]
 20%|██        | 3767/18627 [2:04:58<7:03:30,  1.71s/it]
 20%|██        | 3768/18627 [2:04:59<6:25:33,  1.56s/it]
 20%|██        | 3769/18627 [2:05:01<5:58:05,  1.45s/it]
 20%|██        | 3770/18627 [2:05:03<6:45:40,  1.64s/it]
                                                        
{'loss': 1.2594, 'grad_norm': 6.943330764770508, 'learning_rate': 4.6203710286910995e-06, 'epoch': 0.2}
+
 20%|██        | 3770/18627 [2:05:03<6:45:40,  1.64s/it]
 20%|██        | 3771/18627 [2:05:05<7:21:13,  1.78s/it]
 20%|██        | 3772/18627 [2:05:07<7:44:33,  1.88s/it]
 20%|██        | 3773/18627 [2:05:09<8:02:21,  1.95s/it]
 20%|██        | 3774/18627 [2:05:11<8:15:32,  2.00s/it]
 20%|██        | 3775/18627 [2:05:12<7:15:03,  1.76s/it]
 20%|██        | 3776/18627 [2:05:14<7:42:03,  1.87s/it]
 20%|██        | 3777/18627 [2:05:17<8:02:09,  1.95s/it]
 20%|██        | 3778/18627 [2:05:19<8:14:31,  2.00s/it]
 20%|██        | 3779/18627 [2:05:21<8:24:22,  2.04s/it]
 20%|██        | 3780/18627 [2:05:23<8:30:10,  2.06s/it]
                                                        
{'loss': 0.8903, 'grad_norm': 6.708223342895508, 'learning_rate': 4.618065015750175e-06, 'epoch': 0.2}
+
 20%|██        | 3780/18627 [2:05:23<8:30:10,  2.06s/it]
 20%|██        | 3781/18627 [2:05:25<8:34:16,  2.08s/it]
 20%|██        | 3782/18627 [2:05:26<6:52:03,  1.67s/it]
 20%|██        | 3783/18627 [2:05:28<7:26:17,  1.80s/it]
 20%|██        | 3784/18627 [2:05:30<7:48:41,  1.89s/it]
 20%|██        | 3785/18627 [2:05:31<6:57:10,  1.69s/it]
 20%|██        | 3786/18627 [2:05:33<7:30:25,  1.82s/it]
 20%|██        | 3787/18627 [2:05:35<7:51:56,  1.91s/it]
 20%|██        | 3788/18627 [2:05:38<8:06:01,  1.97s/it]
 20%|██        | 3789/18627 [2:05:40<8:15:30,  2.00s/it]
 20%|██        | 3790/18627 [2:05:42<8:23:43,  2.04s/it]
                                                        
{'loss': 0.9476, 'grad_norm': 11.089399337768555, 'learning_rate': 4.615752599289415e-06, 'epoch': 0.2}
+
 20%|██        | 3790/18627 [2:05:42<8:23:43,  2.04s/it]
 20%|██        | 3791/18627 [2:05:44<8:30:21,  2.06s/it]
 20%|██        | 3792/18627 [2:05:45<7:25:20,  1.80s/it]
 20%|██        | 3793/18627 [2:05:47<7:46:55,  1.89s/it]
 20%|██        | 3794/18627 [2:05:49<8:03:21,  1.96s/it]
 20%|██        | 3795/18627 [2:05:51<8:16:27,  2.01s/it]
 20%|██        | 3796/18627 [2:05:54<8:23:17,  2.04s/it]
 20%|██        | 3797/18627 [2:05:55<7:20:28,  1.78s/it]
 20%|██        | 3798/18627 [2:05:57<7:44:43,  1.88s/it]
 20%|██        | 3799/18627 [2:05:59<8:01:41,  1.95s/it]
 20%|██        | 3800/18627 [2:06:01<8:14:12,  2.00s/it]
                                                        
{'loss': 1.1134, 'grad_norm': 8.430839538574219, 'learning_rate': 4.6134337862999175e-06, 'epoch': 0.2}
+
 20%|██        | 3800/18627 [2:06:01<8:14:12,  2.00s/it]
 20%|██        | 3801/18627 [2:06:03<8:22:34,  2.03s/it]
 20%|██        | 3802/18627 [2:06:05<8:28:37,  2.06s/it]
 20%|██        | 3803/18627 [2:06:07<8:31:21,  2.07s/it]
 20%|██        | 3804/18627 [2:06:10<8:37:02,  2.09s/it]
 20%|██        | 3805/18627 [2:06:11<7:31:07,  1.83s/it]
 20%|██        | 3806/18627 [2:06:12<6:43:49,  1.63s/it]
 20%|██        | 3807/18627 [2:06:14<7:19:02,  1.78s/it]
 20%|██        | 3808/18627 [2:06:16<7:46:03,  1.89s/it]
 20%|██        | 3809/18627 [2:06:18<8:03:08,  1.96s/it]
 20%|██        | 3810/18627 [2:06:19<7:06:30,  1.73s/it]
                                                        
{'loss': 1.2967, 'grad_norm': 16.15500831604004, 'learning_rate': 4.6111085837921224e-06, 'epoch': 0.2}
+
 20%|██        | 3810/18627 [2:06:19<7:06:30,  1.73s/it]
 20%|██        | 3811/18627 [2:06:21<6:20:29,  1.54s/it]
 20%|██        | 3812/18627 [2:06:23<7:01:50,  1.71s/it]
 20%|██        | 3813/18627 [2:06:25<7:32:26,  1.83s/it]
 20%|██        | 3814/18627 [2:06:27<7:53:15,  1.92s/it]
 20%|██        | 3815/18627 [2:06:28<6:59:22,  1.70s/it]
 20%|██        | 3816/18627 [2:06:30<7:29:39,  1.82s/it]
 20%|██        | 3817/18627 [2:06:32<7:51:18,  1.91s/it]
 20%|██        | 3818/18627 [2:06:34<8:07:11,  1.97s/it]
 21%|██        | 3819/18627 [2:06:37<8:17:51,  2.02s/it]
 21%|██        | 3820/18627 [2:06:39<8:26:49,  2.05s/it]
                                                        
{'loss': 1.0297, 'grad_norm': 7.458437442779541, 'learning_rate': 4.608776998795786e-06, 'epoch': 0.21}
+
 21%|██        | 3820/18627 [2:06:39<8:26:49,  2.05s/it]
 21%|██        | 3821/18627 [2:06:41<8:29:42,  2.07s/it]
 21%|██        | 3822/18627 [2:06:43<8:34:11,  2.08s/it]
 21%|██        | 3823/18627 [2:06:44<7:27:52,  1.82s/it]
 21%|██        | 3824/18627 [2:06:45<6:41:13,  1.63s/it]
 21%|██        | 3825/18627 [2:06:47<7:18:45,  1.78s/it]
 21%|██        | 3826/18627 [2:06:49<6:34:28,  1.60s/it]
 21%|██        | 3827/18627 [2:06:51<7:13:41,  1.76s/it]
 21%|██        | 3828/18627 [2:06:53<7:40:54,  1.87s/it]
 21%|██        | 3829/18627 [2:06:55<7:59:15,  1.94s/it]
 21%|██        | 3830/18627 [2:06:57<8:12:23,  2.00s/it]
                                                        
{'loss': 1.2447, 'grad_norm': 7.544371604919434, 'learning_rate': 4.60643903835996e-06, 'epoch': 0.21}
+
 21%|██        | 3830/18627 [2:06:57<8:12:23,  2.00s/it]
 21%|██        | 3831/18627 [2:06:58<7:12:25,  1.75s/it]
 21%|██        | 3832/18627 [2:07:00<7:39:01,  1.86s/it]
 21%|██        | 3833/18627 [2:07:03<7:57:49,  1.94s/it]
 21%|██        | 3834/18627 [2:07:05<8:12:45,  2.00s/it]
 21%|██        | 3835/18627 [2:07:07<8:23:26,  2.04s/it]
 21%|██        | 3836/18627 [2:07:09<8:30:12,  2.07s/it]
 21%|██        | 3837/18627 [2:07:10<6:49:07,  1.66s/it]
 21%|██        | 3838/18627 [2:07:12<7:25:22,  1.81s/it]
 21%|██        | 3839/18627 [2:07:14<7:48:49,  1.90s/it]
 21%|██        | 3840/18627 [2:07:16<8:03:52,  1.96s/it]
                                                        
{'loss': 0.9469, 'grad_norm': 6.732742786407471, 'learning_rate': 4.604094709552972e-06, 'epoch': 0.21}
+
 21%|██        | 3840/18627 [2:07:16<8:03:52,  1.96s/it]
 21%|██        | 3841/18627 [2:07:18<8:14:09,  2.01s/it]
 21%|██        | 3842/18627 [2:07:20<8:23:07,  2.04s/it]
 21%|██        | 3843/18627 [2:07:21<7:20:05,  1.79s/it]
 21%|██        | 3844/18627 [2:07:24<7:46:00,  1.89s/it]
 21%|██        | 3845/18627 [2:07:26<8:02:04,  1.96s/it]
 21%|██        | 3846/18627 [2:07:27<7:04:40,  1.72s/it]
 21%|██        | 3847/18627 [2:07:29<7:33:11,  1.84s/it]
 21%|██        | 3848/18627 [2:07:31<7:54:27,  1.93s/it]
 21%|██        | 3849/18627 [2:07:33<8:09:18,  1.99s/it]
 21%|██        | 3850/18627 [2:07:35<8:19:44,  2.03s/it]
                                                        
{'loss': 1.0445, 'grad_norm': 8.69398021697998, 'learning_rate': 4.601744019462402e-06, 'epoch': 0.21}
+
 21%|██        | 3850/18627 [2:07:35<8:19:44,  2.03s/it]
 21%|██        | 3851/18627 [2:07:37<8:25:28,  2.05s/it]
 21%|██        | 3852/18627 [2:07:40<8:29:42,  2.07s/it]
 21%|██        | 3853/18627 [2:07:42<8:31:53,  2.08s/it]
 21%|██        | 3854/18627 [2:07:44<8:35:26,  2.09s/it]
 21%|██        | 3855/18627 [2:07:46<8:37:08,  2.10s/it]
 21%|██        | 3856/18627 [2:07:48<8:38:28,  2.11s/it]
 21%|██        | 3857/18627 [2:07:50<8:39:39,  2.11s/it]
 21%|██        | 3858/18627 [2:07:52<8:41:31,  2.12s/it]
 21%|██        | 3859/18627 [2:07:54<8:41:17,  2.12s/it]
 21%|██        | 3860/18627 [2:07:55<7:19:49,  1.79s/it]
                                                        
{'loss': 0.8682, 'grad_norm': 15.88547420501709, 'learning_rate': 4.599386975195062e-06, 'epoch': 0.21}
+
 21%|██        | 3860/18627 [2:07:55<7:19:49,  1.79s/it]
 21%|██        | 3861/18627 [2:07:58<7:44:34,  1.89s/it]
 21%|██        | 3862/18627 [2:08:00<8:02:29,  1.96s/it]
 21%|██        | 3863/18627 [2:08:02<8:13:41,  2.01s/it]
 21%|██        | 3864/18627 [2:08:03<7:13:08,  1.76s/it]
 21%|██        | 3865/18627 [2:08:05<7:41:10,  1.87s/it]
 21%|██        | 3866/18627 [2:08:07<7:58:40,  1.95s/it]
 21%|██        | 3867/18627 [2:08:09<8:11:19,  2.00s/it]
 21%|██        | 3868/18627 [2:08:11<7:11:43,  1.76s/it]
 21%|██        | 3869/18627 [2:08:13<7:36:45,  1.86s/it]
 21%|██        | 3870/18627 [2:08:13<6:11:13,  1.51s/it]
                                                        
{'loss': 1.1128, 'grad_norm': 6.793948173522949, 'learning_rate': 4.597023583876975e-06, 'epoch': 0.21}
+
 21%|██        | 3870/18627 [2:08:13<6:11:13,  1.51s/it]
 21%|██        | 3871/18627 [2:08:15<6:56:20,  1.69s/it]
 21%|██        | 3872/18627 [2:08:18<7:28:32,  1.82s/it]
 21%|██        | 3873/18627 [2:08:20<7:50:57,  1.92s/it]
 21%|██        | 3874/18627 [2:08:22<8:05:14,  1.97s/it]
 21%|██        | 3875/18627 [2:08:24<8:14:14,  2.01s/it]
 21%|██        | 3876/18627 [2:08:26<8:21:50,  2.04s/it]
 21%|██        | 3877/18627 [2:08:28<8:26:27,  2.06s/it]
 21%|██        | 3878/18627 [2:08:30<8:31:14,  2.08s/it]
 21%|██        | 3879/18627 [2:08:31<7:25:22,  1.81s/it]
 21%|██        | 3880/18627 [2:08:34<7:46:28,  1.90s/it]
                                                        
{'loss': 0.8921, 'grad_norm': 10.572693824768066, 'learning_rate': 4.594653852653354e-06, 'epoch': 0.21}
+
 21%|██        | 3880/18627 [2:08:34<7:46:28,  1.90s/it]
 21%|██        | 3881/18627 [2:08:36<8:02:14,  1.96s/it]
 21%|██        | 3882/18627 [2:08:38<8:14:19,  2.01s/it]
 21%|██        | 3883/18627 [2:08:39<7:13:50,  1.77s/it]
 21%|██        | 3884/18627 [2:08:40<6:25:32,  1.57s/it]
 21%|██        | 3885/18627 [2:08:42<7:05:15,  1.73s/it]
 21%|██        | 3886/18627 [2:08:43<5:50:09,  1.43s/it]
 21%|██        | 3887/18627 [2:08:45<6:39:18,  1.63s/it]
 21%|██        | 3888/18627 [2:08:46<6:06:48,  1.49s/it]
 21%|██        | 3889/18627 [2:08:48<6:52:46,  1.68s/it]
 21%|██        | 3890/18627 [2:08:50<7:25:11,  1.81s/it]
                                                        
{'loss': 1.3462, 'grad_norm': 9.42236614227295, 'learning_rate': 4.592277788688575e-06, 'epoch': 0.21}
+
 21%|██        | 3890/18627 [2:08:50<7:25:11,  1.81s/it]
 21%|██        | 3891/18627 [2:08:53<7:48:16,  1.91s/it]
 21%|██        | 3892/18627 [2:08:55<8:04:26,  1.97s/it]
 21%|██        | 3893/18627 [2:08:57<8:13:59,  2.01s/it]
 21%|██        | 3894/18627 [2:08:59<8:23:13,  2.05s/it]
 21%|██        | 3895/18627 [2:09:01<8:28:33,  2.07s/it]
 21%|██        | 3896/18627 [2:09:03<8:32:46,  2.09s/it]
 21%|██        | 3897/18627 [2:09:05<8:35:32,  2.10s/it]
 21%|██        | 3898/18627 [2:09:07<7:28:38,  1.83s/it]
 21%|██        | 3899/18627 [2:09:09<7:48:36,  1.91s/it]
 21%|██        | 3900/18627 [2:09:09<6:19:30,  1.55s/it]
                                                        
{'loss': 0.8884, 'grad_norm': 7.901852607727051, 'learning_rate': 4.5898953991661665e-06, 'epoch': 0.21}
+
 21%|██        | 3900/18627 [2:09:09<6:19:30,  1.55s/it]
 21%|██        | 3901/18627 [2:09:11<7:01:26,  1.72s/it]
 21%|██        | 3902/18627 [2:09:14<7:31:36,  1.84s/it]
 21%|██        | 3903/18627 [2:09:16<7:51:21,  1.92s/it]
 21%|██        | 3904/18627 [2:09:17<6:56:57,  1.70s/it]
 21%|██        | 3905/18627 [2:09:18<6:19:52,  1.55s/it]
 21%|██        | 3906/18627 [2:09:20<7:00:55,  1.72s/it]
 21%|██        | 3907/18627 [2:09:22<7:31:04,  1.84s/it]
 21%|██        | 3908/18627 [2:09:24<7:52:34,  1.93s/it]
 21%|██        | 3909/18627 [2:09:26<6:58:34,  1.71s/it]
 21%|██        | 3910/18627 [2:09:28<7:27:24,  1.82s/it]
                                                        
{'loss': 1.3021, 'grad_norm': 10.362621307373047, 'learning_rate': 4.587506691288776e-06, 'epoch': 0.21}
+
 21%|██        | 3910/18627 [2:09:28<7:27:24,  1.82s/it]
 21%|██        | 3911/18627 [2:09:30<7:49:32,  1.91s/it]
 21%|██        | 3912/18627 [2:09:32<8:04:03,  1.97s/it]
 21%|██        | 3913/18627 [2:09:34<8:13:11,  2.01s/it]
 21%|██        | 3914/18627 [2:09:35<7:12:56,  1.77s/it]
 21%|██        | 3915/18627 [2:09:36<6:30:50,  1.59s/it]
 21%|██        | 3916/18627 [2:09:39<7:08:22,  1.75s/it]
 21%|██        | 3917/18627 [2:09:41<7:35:02,  1.86s/it]
 21%|██        | 3918/18627 [2:09:43<7:54:43,  1.94s/it]
 21%|██        | 3919/18627 [2:09:45<8:07:14,  1.99s/it]
 21%|██        | 3920/18627 [2:09:47<8:14:48,  2.02s/it]
                                                        
{'loss': 1.123, 'grad_norm': 9.954042434692383, 'learning_rate': 4.585111672278154e-06, 'epoch': 0.21}
+
 21%|██        | 3920/18627 [2:09:47<8:14:48,  2.02s/it]
 21%|██        | 3921/18627 [2:09:48<7:14:29,  1.77s/it]
 21%|██        | 3922/18627 [2:09:50<7:38:32,  1.87s/it]
 21%|██        | 3923/18627 [2:09:52<7:54:32,  1.94s/it]
 21%|██        | 3924/18627 [2:09:54<8:08:16,  1.99s/it]
 21%|██        | 3925/18627 [2:09:56<7:10:00,  1.75s/it]
 21%|██        | 3926/18627 [2:09:58<7:37:21,  1.87s/it]
 21%|██        | 3927/18627 [2:09:59<6:39:15,  1.63s/it]
 21%|██        | 3928/18627 [2:10:00<6:07:24,  1.50s/it]
 21%|██        | 3929/18627 [2:10:02<6:52:49,  1.69s/it]
 21%|██        | 3930/18627 [2:10:04<7:24:40,  1.82s/it]
                                                        
{'loss': 1.5889, 'grad_norm': 10.41720199584961, 'learning_rate': 4.582710349375133e-06, 'epoch': 0.21}
+
 21%|██        | 3930/18627 [2:10:04<7:24:40,  1.82s/it]
 21%|██        | 3931/18627 [2:10:06<7:46:55,  1.91s/it]
 21%|██        | 3932/18627 [2:10:09<8:01:30,  1.97s/it]
 21%|██        | 3933/18627 [2:10:11<8:12:33,  2.01s/it]
 21%|██        | 3934/18627 [2:10:13<8:21:38,  2.05s/it]
 21%|██        | 3935/18627 [2:10:15<8:25:21,  2.06s/it]
 21%|██        | 3936/18627 [2:10:16<7:21:55,  1.80s/it]
 21%|██        | 3937/18627 [2:10:17<6:36:26,  1.62s/it]
 21%|██        | 3938/18627 [2:10:19<7:12:28,  1.77s/it]
 21%|██        | 3939/18627 [2:10:22<7:38:41,  1.87s/it]
 21%|██        | 3940/18627 [2:10:23<6:48:33,  1.67s/it]
                                                        
{'loss': 1.2988, 'grad_norm': 14.810001373291016, 'learning_rate': 4.580302729839601e-06, 'epoch': 0.21}
+
 21%|██        | 3940/18627 [2:10:23<6:48:33,  1.67s/it]
 21%|██        | 3941/18627 [2:10:25<7:20:26,  1.80s/it]
 21%|██        | 3942/18627 [2:10:27<7:44:18,  1.90s/it]
 21%|██        | 3943/18627 [2:10:29<7:59:33,  1.96s/it]
 21%|██        | 3944/18627 [2:10:31<8:11:03,  2.01s/it]
 21%|██        | 3945/18627 [2:10:33<8:20:00,  2.04s/it]
 21%|██        | 3946/18627 [2:10:35<8:26:12,  2.07s/it]
 21%|██        | 3947/18627 [2:10:38<8:30:29,  2.09s/it]
 21%|██        | 3948/18627 [2:10:38<6:49:11,  1.67s/it]
 21%|██        | 3949/18627 [2:10:40<7:22:23,  1.81s/it]
 21%|██        | 3950/18627 [2:10:42<7:44:38,  1.90s/it]
                                                        
{'loss': 0.7598, 'grad_norm': 7.708392143249512, 'learning_rate': 4.5778888209504865e-06, 'epoch': 0.21}
+
 21%|██        | 3950/18627 [2:10:42<7:44:38,  1.90s/it]
 21%|██        | 3951/18627 [2:10:45<7:59:17,  1.96s/it]
 21%|██        | 3952/18627 [2:10:47<8:09:59,  2.00s/it]
 21%|██        | 3953/18627 [2:10:49<8:18:05,  2.04s/it]
 21%|██        | 3954/18627 [2:10:51<8:24:45,  2.06s/it]
 21%|██        | 3955/18627 [2:10:53<8:27:35,  2.08s/it]
 21%|██        | 3956/18627 [2:10:55<8:30:14,  2.09s/it]
 21%|██        | 3957/18627 [2:10:57<8:32:40,  2.10s/it]
 21%|██        | 3958/18627 [2:10:58<6:52:35,  1.69s/it]
 21%|██▏       | 3959/18627 [2:11:00<7:23:55,  1.82s/it]
 21%|██▏       | 3960/18627 [2:11:02<7:46:22,  1.91s/it]
                                                        
{'loss': 0.7634, 'grad_norm': 6.894002914428711, 'learning_rate': 4.57546863000573e-06, 'epoch': 0.21}
+
 21%|██▏       | 3960/18627 [2:11:02<7:46:22,  1.91s/it]
 21%|██▏       | 3961/18627 [2:11:04<8:02:39,  1.97s/it]
 21%|██▏       | 3962/18627 [2:11:06<8:13:18,  2.02s/it]
 21%|██▏       | 3963/18627 [2:11:09<8:21:40,  2.05s/it]
 21%|██▏       | 3964/18627 [2:11:11<8:27:58,  2.08s/it]
 21%|██▏       | 3965/18627 [2:11:13<8:31:37,  2.09s/it]
 21%|██▏       | 3966/18627 [2:11:14<7:25:50,  1.82s/it]
 21%|██▏       | 3967/18627 [2:11:15<6:08:57,  1.51s/it]
 21%|██▏       | 3968/18627 [2:11:17<6:54:35,  1.70s/it]
 21%|██▏       | 3969/18627 [2:11:19<7:24:58,  1.82s/it]
 21%|██▏       | 3970/18627 [2:11:21<7:47:13,  1.91s/it]
                                                        
{'loss': 0.8674, 'grad_norm': 10.723433494567871, 'learning_rate': 4.573042164322264e-06, 'epoch': 0.21}
+
 21%|██▏       | 3970/18627 [2:11:21<7:47:13,  1.91s/it]
 21%|██▏       | 3971/18627 [2:11:23<8:01:53,  1.97s/it]
 21%|██▏       | 3972/18627 [2:11:25<8:12:13,  2.02s/it]
 21%|██▏       | 3973/18627 [2:11:28<8:19:46,  2.05s/it]
 21%|██▏       | 3974/18627 [2:11:29<7:16:32,  1.79s/it]
 21%|██▏       | 3975/18627 [2:11:31<7:43:52,  1.90s/it]
 21%|██▏       | 3976/18627 [2:11:33<8:00:56,  1.97s/it]
 21%|██▏       | 3977/18627 [2:11:35<8:10:50,  2.01s/it]
 21%|██▏       | 3978/18627 [2:11:37<8:19:24,  2.05s/it]
 21%|██▏       | 3979/18627 [2:11:39<8:25:12,  2.07s/it]
 21%|██▏       | 3980/18627 [2:11:42<8:28:35,  2.08s/it]
                                                        
{'loss': 0.851, 'grad_norm': 7.566904544830322, 'learning_rate': 4.570609431235993e-06, 'epoch': 0.21}
+
 21%|██▏       | 3980/18627 [2:11:42<8:28:35,  2.08s/it]
 21%|██▏       | 3981/18627 [2:11:44<8:32:00,  2.10s/it]
 21%|██▏       | 3982/18627 [2:11:46<8:32:29,  2.10s/it]
 21%|██▏       | 3983/18627 [2:11:48<8:32:26,  2.10s/it]
 21%|██▏       | 3984/18627 [2:11:50<8:33:04,  2.10s/it]
 21%|██▏       | 3985/18627 [2:11:52<8:33:24,  2.10s/it]
 21%|██▏       | 3986/18627 [2:11:54<8:36:32,  2.12s/it]
 21%|██▏       | 3987/18627 [2:11:55<6:59:57,  1.72s/it]
 21%|██▏       | 3988/18627 [2:11:57<7:31:35,  1.85s/it]
 21%|██▏       | 3989/18627 [2:11:59<7:54:30,  1.94s/it]
 21%|██▏       | 3990/18627 [2:12:01<6:59:40,  1.72s/it]
                                                        
{'loss': 0.9303, 'grad_norm': 16.29374122619629, 'learning_rate': 4.568170438101769e-06, 'epoch': 0.21}
+
 21%|██▏       | 3990/18627 [2:12:01<6:59:40,  1.72s/it]
 21%|██▏       | 3991/18627 [2:12:03<7:30:47,  1.85s/it]
 21%|██▏       | 3992/18627 [2:12:05<7:50:35,  1.93s/it]
 21%|██▏       | 3993/18627 [2:12:06<6:56:01,  1.71s/it]
 21%|██▏       | 3994/18627 [2:12:08<7:27:00,  1.83s/it]
 21%|██▏       | 3995/18627 [2:12:10<7:47:41,  1.92s/it]
 21%|██▏       | 3996/18627 [2:12:12<8:02:05,  1.98s/it]
 21%|██▏       | 3997/18627 [2:12:14<8:11:57,  2.02s/it]
 21%|██▏       | 3998/18627 [2:12:17<8:18:03,  2.04s/it]
 21%|██▏       | 3999/18627 [2:12:19<8:22:25,  2.06s/it]
 21%|██▏       | 4000/18627 [2:12:21<8:27:43,  2.08s/it]
                                                        
{'loss': 0.9644, 'grad_norm': 7.390998840332031, 'learning_rate': 4.56572519229337e-06, 'epoch': 0.21}
+
 21%|██▏       | 4000/18627 [2:12:21<8:27:43,  2.08s/it]/home/zhengduo/workspace/LLaMA-Factory/src/qwen_vl/model/modeling_qwen2_5_vl.py:1979: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
+  with torch.cuda.amp.autocast(dtype=dtype):
+/home/zhengduo/miniconda3/envs/vgllm/lib/python3.10/site-packages/torch/utils/checkpoint.py:87: UserWarning: None of the inputs have requires_grad=True. Gradients will be None
+  warnings.warn(
+
 21%|██▏       | 4001/18627 [2:13:02<55:53:01, 13.76s/it]
 21%|██▏       | 4002/18627 [2:13:04<41:42:00, 10.26s/it]
 21%|██▏       | 4003/18627 [2:13:06<31:47:01,  7.82s/it]
 21%|██▏       | 4004/18627 [2:13:07<23:41:57,  5.83s/it]
 22%|██▏       | 4005/18627 [2:13:09<19:11:09,  4.72s/it]
 22%|██▏       | 4006/18627 [2:13:11<16:00:46,  3.94s/it]
 22%|██▏       | 4007/18627 [2:13:14<13:47:49,  3.40s/it]
 22%|██▏       | 4008/18627 [2:13:16<12:15:00,  3.02s/it]
 22%|██▏       | 4009/18627 [2:13:18<11:07:20,  2.74s/it]
 22%|██▏       | 4010/18627 [2:13:19<9:14:22,  2.28s/it] 
                                                        
{'loss': 1.0735, 'grad_norm': 13.77817440032959, 'learning_rate': 4.563273701203477e-06, 'epoch': 0.22}
+
 22%|██▏       | 4010/18627 [2:13:19<9:14:22,  2.28s/it]
 22%|██▏       | 4011/18627 [2:13:20<7:55:04,  1.95s/it]
 22%|██▏       | 4012/18627 [2:13:22<8:05:49,  1.99s/it]
 22%|██▏       | 4013/18627 [2:13:24<8:15:11,  2.03s/it]
 22%|██▏       | 4014/18627 [2:13:26<7:13:31,  1.78s/it]
 22%|██▏       | 4015/18627 [2:13:27<6:30:04,  1.60s/it]
 22%|██▏       | 4016/18627 [2:13:29<7:09:25,  1.76s/it]
 22%|██▏       | 4017/18627 [2:13:30<6:28:04,  1.59s/it]
 22%|██▏       | 4018/18627 [2:13:32<7:05:47,  1.75s/it]
 22%|██▏       | 4019/18627 [2:13:33<6:24:50,  1.58s/it]
 22%|██▏       | 4020/18627 [2:13:35<5:56:32,  1.46s/it]
                                                        
{'loss': 1.8325, 'grad_norm': 14.56369686126709, 'learning_rate': 4.560815972243653e-06, 'epoch': 0.22}
+
 22%|██▏       | 4020/18627 [2:13:35<5:56:32,  1.46s/it]
 22%|██▏       | 4021/18627 [2:13:37<6:45:44,  1.67s/it]
 22%|██▏       | 4022/18627 [2:13:39<7:18:52,  1.80s/it]
 22%|██▏       | 4023/18627 [2:13:41<7:43:40,  1.91s/it]
 22%|██▏       | 4024/18627 [2:13:43<7:59:20,  1.97s/it]
 22%|██▏       | 4025/18627 [2:13:45<8:09:19,  2.01s/it]
 22%|██▏       | 4026/18627 [2:13:47<8:15:39,  2.04s/it]
 22%|██▏       | 4027/18627 [2:13:49<8:22:31,  2.07s/it]
 22%|██▏       | 4028/18627 [2:13:52<8:24:27,  2.07s/it]
 22%|██▏       | 4029/18627 [2:13:54<8:27:24,  2.09s/it]
 22%|██▏       | 4030/18627 [2:13:56<8:28:47,  2.09s/it]
                                                        
{'loss': 0.6602, 'grad_norm': 11.950592994689941, 'learning_rate': 4.5583520128443185e-06, 'epoch': 0.22}
+
 22%|██▏       | 4030/18627 [2:13:56<8:28:47,  2.09s/it]
 22%|██▏       | 4031/18627 [2:13:58<8:29:41,  2.10s/it]
 22%|██▏       | 4032/18627 [2:13:59<7:24:11,  1.83s/it]
 22%|██▏       | 4033/18627 [2:14:00<6:37:44,  1.64s/it]
 22%|██▏       | 4034/18627 [2:14:02<7:13:11,  1.78s/it]
 22%|██▏       | 4035/18627 [2:14:04<6:30:16,  1.60s/it]
 22%|██▏       | 4036/18627 [2:14:06<7:07:42,  1.76s/it]
 22%|██▏       | 4037/18627 [2:14:08<7:35:09,  1.87s/it]
 22%|██▏       | 4038/18627 [2:14:10<7:52:46,  1.94s/it]
 22%|██▏       | 4039/18627 [2:14:12<8:05:19,  2.00s/it]
 22%|██▏       | 4040/18627 [2:14:13<7:06:22,  1.75s/it]
                                                        
{'loss': 1.4431, 'grad_norm': 13.711971282958984, 'learning_rate': 4.555881830454733e-06, 'epoch': 0.22}
+
 22%|██▏       | 4040/18627 [2:14:13<7:06:22,  1.75s/it]
 22%|██▏       | 4041/18627 [2:14:15<7:31:49,  1.86s/it]
 22%|██▏       | 4042/18627 [2:14:18<7:51:05,  1.94s/it]
 22%|██▏       | 4043/18627 [2:14:19<6:56:17,  1.71s/it]
 22%|██▏       | 4044/18627 [2:14:20<6:18:08,  1.56s/it]
 22%|██▏       | 4045/18627 [2:14:22<6:58:08,  1.72s/it]
 22%|██▏       | 4046/18627 [2:14:23<6:18:42,  1.56s/it]
 22%|██▏       | 4047/18627 [2:14:25<6:59:33,  1.73s/it]
 22%|██▏       | 4048/18627 [2:14:27<7:26:36,  1.84s/it]
 22%|██▏       | 4049/18627 [2:14:30<7:46:33,  1.92s/it]
 22%|██▏       | 4050/18627 [2:14:32<8:02:32,  1.99s/it]
                                                        
{'loss': 1.2922, 'grad_norm': 6.953356742858887, 'learning_rate': 4.55340543254297e-06, 'epoch': 0.22}
+
 22%|██▏       | 4050/18627 [2:14:32<8:02:32,  1.99s/it]
 22%|██▏       | 4051/18627 [2:14:33<7:04:00,  1.75s/it]
 22%|██▏       | 4052/18627 [2:14:35<7:30:43,  1.86s/it]
 22%|██▏       | 4053/18627 [2:14:36<6:42:39,  1.66s/it]
 22%|██▏       | 4054/18627 [2:14:37<6:08:57,  1.52s/it]
 22%|██▏       | 4055/18627 [2:14:39<6:50:36,  1.69s/it]
 22%|██▏       | 4056/18627 [2:14:42<7:20:42,  1.81s/it]
 22%|██▏       | 4057/18627 [2:14:44<7:41:04,  1.90s/it]
 22%|██▏       | 4058/18627 [2:14:46<7:57:47,  1.97s/it]
 22%|██▏       | 4059/18627 [2:14:48<8:08:33,  2.01s/it]
 22%|██▏       | 4060/18627 [2:14:50<8:15:13,  2.04s/it]
                                                        
{'loss': 1.2742, 'grad_norm': 6.404519557952881, 'learning_rate': 4.55092282659589e-06, 'epoch': 0.22}
+
 22%|██▏       | 4060/18627 [2:14:50<8:15:13,  2.04s/it]
 22%|██▏       | 4061/18627 [2:14:52<8:21:46,  2.07s/it]
 22%|██▏       | 4062/18627 [2:14:54<8:24:04,  2.08s/it]
 22%|██▏       | 4063/18627 [2:14:56<8:27:43,  2.09s/it]
 22%|██▏       | 4064/18627 [2:14:58<8:30:57,  2.11s/it]
 22%|██▏       | 4065/18627 [2:15:01<8:33:26,  2.12s/it]
 22%|██▏       | 4066/18627 [2:15:03<8:34:35,  2.12s/it]
 22%|██▏       | 4067/18627 [2:15:05<8:37:40,  2.13s/it]
 22%|██▏       | 4068/18627 [2:15:07<8:36:20,  2.13s/it]
 22%|██▏       | 4069/18627 [2:15:09<8:34:47,  2.12s/it]
 22%|██▏       | 4070/18627 [2:15:11<8:34:28,  2.12s/it]
                                                        
{'loss': 0.6859, 'grad_norm': 6.642707824707031, 'learning_rate': 4.548434020119125e-06, 'epoch': 0.22}
+
 22%|██▏       | 4070/18627 [2:15:11<8:34:28,  2.12s/it]
 22%|██▏       | 4071/18627 [2:15:12<7:26:47,  1.84s/it]
 22%|██▏       | 4072/18627 [2:15:15<7:46:23,  1.92s/it]
 22%|██▏       | 4073/18627 [2:15:17<7:59:55,  1.98s/it]
 22%|██▏       | 4074/18627 [2:15:19<8:10:30,  2.02s/it]
 22%|██▏       | 4075/18627 [2:15:20<6:56:41,  1.72s/it]
 22%|██▏       | 4076/18627 [2:15:22<7:25:58,  1.84s/it]
 22%|██▏       | 4077/18627 [2:15:23<6:38:32,  1.64s/it]
 22%|██▏       | 4078/18627 [2:15:25<7:11:47,  1.78s/it]
 22%|██▏       | 4079/18627 [2:15:27<7:34:32,  1.87s/it]
 22%|██▏       | 4080/18627 [2:15:29<7:51:49,  1.95s/it]
                                                        
{'loss': 1.2755, 'grad_norm': 8.551284790039062, 'learning_rate': 4.545939020637053e-06, 'epoch': 0.22}
+
 22%|██▏       | 4080/18627 [2:15:29<7:51:49,  1.95s/it]
 22%|██▏       | 4081/18627 [2:15:32<8:04:49,  2.00s/it]
 22%|██▏       | 4082/18627 [2:15:34<8:14:48,  2.04s/it]
 22%|██▏       | 4083/18627 [2:15:36<8:20:07,  2.06s/it]
 22%|██▏       | 4084/18627 [2:15:37<7:17:32,  1.81s/it]
 22%|██▏       | 4085/18627 [2:15:39<7:41:44,  1.91s/it]
 22%|██▏       | 4086/18627 [2:15:41<7:55:36,  1.96s/it]
 22%|██▏       | 4087/18627 [2:15:43<8:06:57,  2.01s/it]
 22%|██▏       | 4088/18627 [2:15:45<8:13:44,  2.04s/it]
 22%|██▏       | 4089/18627 [2:15:48<8:20:23,  2.07s/it]
 22%|██▏       | 4090/18627 [2:15:49<7:16:26,  1.80s/it]
                                                        
{'loss': 1.1543, 'grad_norm': 14.737210273742676, 'learning_rate': 4.543437835692778e-06, 'epoch': 0.22}
+
 22%|██▏       | 4090/18627 [2:15:49<7:16:26,  1.80s/it]
 22%|██▏       | 4091/18627 [2:15:50<6:31:41,  1.62s/it]
 22%|██▏       | 4092/18627 [2:15:52<7:07:52,  1.77s/it]
 22%|██▏       | 4093/18627 [2:15:53<5:50:05,  1.45s/it]
 22%|██▏       | 4094/18627 [2:15:55<6:38:59,  1.65s/it]
 22%|██▏       | 4095/18627 [2:15:57<7:14:15,  1.79s/it]
 22%|██▏       | 4096/18627 [2:15:59<7:39:27,  1.90s/it]
 22%|██▏       | 4097/18627 [2:16:01<7:57:13,  1.97s/it]
 22%|██▏       | 4098/18627 [2:16:03<8:08:12,  2.02s/it]
 22%|██▏       | 4099/18627 [2:16:06<8:15:23,  2.05s/it]
 22%|██▏       | 4100/18627 [2:16:08<8:21:29,  2.07s/it]
                                                        
{'loss': 0.9638, 'grad_norm': 7.733486652374268, 'learning_rate': 4.540930472848098e-06, 'epoch': 0.22}
+
 22%|██▏       | 4100/18627 [2:16:08<8:21:29,  2.07s/it]
 22%|██▏       | 4101/18627 [2:16:10<8:25:35,  2.09s/it]
 22%|██▏       | 4102/18627 [2:16:12<8:28:35,  2.10s/it]
 22%|██▏       | 4103/18627 [2:16:13<7:21:39,  1.82s/it]
 22%|██▏       | 4104/18627 [2:16:14<6:34:50,  1.63s/it]
 22%|██▏       | 4105/18627 [2:16:16<7:10:49,  1.78s/it]
 22%|██▏       | 4106/18627 [2:16:19<7:36:05,  1.88s/it]
 22%|██▏       | 4107/18627 [2:16:21<7:52:59,  1.95s/it]
 22%|██▏       | 4108/18627 [2:16:21<6:23:47,  1.59s/it]
 22%|██▏       | 4109/18627 [2:16:23<5:54:34,  1.47s/it]
 22%|██▏       | 4110/18627 [2:16:25<6:42:56,  1.67s/it]
                                                        
{'loss': 1.2959, 'grad_norm': 7.7226152420043945, 'learning_rate': 4.538416939683494e-06, 'epoch': 0.22}
+
 22%|██▏       | 4110/18627 [2:16:25<6:42:56,  1.67s/it]
 22%|██▏       | 4111/18627 [2:16:26<6:08:35,  1.52s/it]
 22%|██▏       | 4112/18627 [2:16:28<6:52:30,  1.71s/it]
 22%|██▏       | 4113/18627 [2:16:29<6:09:21,  1.53s/it]
 22%|██▏       | 4114/18627 [2:16:31<6:53:35,  1.71s/it]
 22%|██▏       | 4115/18627 [2:16:33<7:22:30,  1.83s/it]
 22%|██▏       | 4116/18627 [2:16:35<7:43:29,  1.92s/it]
 22%|██▏       | 4117/18627 [2:16:38<7:57:26,  1.97s/it]
 22%|██▏       | 4118/18627 [2:16:40<8:08:26,  2.02s/it]
 22%|██▏       | 4119/18627 [2:16:42<8:14:43,  2.05s/it]
 22%|██▏       | 4120/18627 [2:16:43<7:12:29,  1.79s/it]
                                                        
{'loss': 1.3088, 'grad_norm': 14.974197387695312, 'learning_rate': 4.535897243798099e-06, 'epoch': 0.22}
+
 22%|██▏       | 4120/18627 [2:16:43<7:12:29,  1.79s/it]
 22%|██▏       | 4121/18627 [2:16:45<7:36:00,  1.89s/it]
 22%|██▏       | 4122/18627 [2:16:47<7:53:00,  1.96s/it]
 22%|██▏       | 4123/18627 [2:16:49<8:03:37,  2.00s/it]
 22%|██▏       | 4124/18627 [2:16:51<8:11:57,  2.04s/it]
 22%|██▏       | 4125/18627 [2:16:53<7:10:27,  1.78s/it]
 22%|██▏       | 4126/18627 [2:16:55<7:37:39,  1.89s/it]
 22%|██▏       | 4127/18627 [2:16:56<6:46:07,  1.68s/it]
 22%|██▏       | 4128/18627 [2:16:58<7:17:06,  1.81s/it]
 22%|██▏       | 4129/18627 [2:17:00<7:39:17,  1.90s/it]
 22%|██▏       | 4130/18627 [2:17:01<6:47:17,  1.69s/it]
                                                        
{'loss': 1.2555, 'grad_norm': 15.22043228149414, 'learning_rate': 4.5333713928096806e-06, 'epoch': 0.22}
+
 22%|██▏       | 4130/18627 [2:17:01<6:47:17,  1.69s/it]
 22%|██▏       | 4131/18627 [2:17:04<7:19:01,  1.82s/it]
 22%|██▏       | 4132/18627 [2:17:06<7:41:13,  1.91s/it]
 22%|██▏       | 4133/18627 [2:17:07<6:49:53,  1.70s/it]
 22%|██▏       | 4134/18627 [2:17:09<7:21:11,  1.83s/it]
 22%|██▏       | 4135/18627 [2:17:11<7:42:45,  1.92s/it]
 22%|██▏       | 4136/18627 [2:17:12<6:50:09,  1.70s/it]
 22%|██▏       | 4137/18627 [2:17:14<7:20:38,  1.82s/it]
 22%|██▏       | 4138/18627 [2:17:17<7:42:44,  1.92s/it]
 22%|██▏       | 4139/18627 [2:17:19<7:56:58,  1.98s/it]
 22%|██▏       | 4140/18627 [2:17:21<8:07:44,  2.02s/it]
                                                        
{'loss': 1.1724, 'grad_norm': 8.928786277770996, 'learning_rate': 4.530839394354611e-06, 'epoch': 0.22}
+
 22%|██▏       | 4140/18627 [2:17:21<8:07:44,  2.02s/it]
 22%|██▏       | 4141/18627 [2:17:23<8:15:06,  2.05s/it]
 22%|██▏       | 4142/18627 [2:17:25<8:18:51,  2.07s/it]
 22%|██▏       | 4143/18627 [2:17:27<8:22:30,  2.08s/it]
 22%|██▏       | 4144/18627 [2:17:29<8:25:30,  2.09s/it]
 22%|██▏       | 4145/18627 [2:17:31<8:27:38,  2.10s/it]
 22%|██▏       | 4146/18627 [2:17:34<8:29:30,  2.11s/it]
 22%|██▏       | 4147/18627 [2:17:36<8:29:43,  2.11s/it]
 22%|██▏       | 4148/18627 [2:17:38<8:29:17,  2.11s/it]
 22%|██▏       | 4149/18627 [2:17:40<8:27:55,  2.10s/it]
 22%|██▏       | 4150/18627 [2:17:42<8:29:26,  2.11s/it]
                                                        
{'loss': 0.7379, 'grad_norm': 7.477701187133789, 'learning_rate': 4.528301256087849e-06, 'epoch': 0.22}
+
 22%|██▏       | 4150/18627 [2:17:42<8:29:26,  2.11s/it]
 22%|██▏       | 4151/18627 [2:17:44<8:29:13,  2.11s/it]
 22%|██▏       | 4152/18627 [2:17:46<8:30:02,  2.11s/it]
 22%|██▏       | 4153/18627 [2:17:48<8:30:35,  2.12s/it]
 22%|██▏       | 4154/18627 [2:17:50<8:31:41,  2.12s/it]
 22%|██▏       | 4155/18627 [2:17:53<8:32:34,  2.13s/it]
 22%|██▏       | 4156/18627 [2:17:55<8:32:13,  2.12s/it]
 22%|██▏       | 4157/18627 [2:17:57<8:32:30,  2.13s/it]
 22%|██▏       | 4158/18627 [2:17:59<8:30:34,  2.12s/it]
 22%|██▏       | 4159/18627 [2:18:02<10:13:37,  2.54s/it]
 22%|██▏       | 4160/18627 [2:18:05<9:42:09,  2.41s/it] 
                                                        
{'loss': 0.6724, 'grad_norm': 5.9267473220825195, 'learning_rate': 4.525756985682918e-06, 'epoch': 0.22}
+
 22%|██▏       | 4160/18627 [2:18:05<9:42:09,  2.41s/it]
 22%|██▏       | 4161/18627 [2:18:07<9:20:16,  2.32s/it]
 22%|██▏       | 4162/18627 [2:18:09<9:05:37,  2.26s/it]
 22%|██▏       | 4163/18627 [2:18:11<8:53:22,  2.21s/it]
 22%|██▏       | 4164/18627 [2:18:13<8:48:05,  2.19s/it]
 22%|██▏       | 4165/18627 [2:18:14<7:36:21,  1.89s/it]
 22%|██▏       | 4166/18627 [2:18:16<7:54:04,  1.97s/it]
 22%|██▏       | 4167/18627 [2:18:18<8:05:41,  2.02s/it]
 22%|██▏       | 4168/18627 [2:18:21<8:13:28,  2.05s/it]
 22%|██▏       | 4169/18627 [2:18:23<8:18:24,  2.07s/it]
 22%|██▏       | 4170/18627 [2:18:25<8:22:11,  2.08s/it]
                                                        
{'loss': 0.902, 'grad_norm': 8.869007110595703, 'learning_rate': 4.523206590831879e-06, 'epoch': 0.22}
+
 22%|██▏       | 4170/18627 [2:18:25<8:22:11,  2.08s/it]
 22%|██▏       | 4171/18627 [2:18:27<8:23:45,  2.09s/it]
 22%|██▏       | 4172/18627 [2:18:29<8:24:50,  2.10s/it]
 22%|██▏       | 4173/18627 [2:18:31<8:26:15,  2.10s/it]
 22%|██▏       | 4174/18627 [2:18:32<7:19:52,  1.83s/it]
 22%|██▏       | 4175/18627 [2:18:34<7:39:12,  1.91s/it]
 22%|██▏       | 4176/18627 [2:18:37<7:52:55,  1.96s/it]
 22%|██▏       | 4177/18627 [2:18:39<8:02:51,  2.00s/it]
 22%|██▏       | 4178/18627 [2:18:41<8:11:24,  2.04s/it]
 22%|██▏       | 4179/18627 [2:18:42<7:09:36,  1.78s/it]
 22%|██▏       | 4180/18627 [2:18:43<6:26:30,  1.61s/it]
                                                        
{'loss': 1.2025, 'grad_norm': 15.672279357910156, 'learning_rate': 4.52065007924531e-06, 'epoch': 0.22}
+
 22%|██▏       | 4180/18627 [2:18:43<6:26:30,  1.61s/it]
 22%|██▏       | 4181/18627 [2:18:44<5:56:29,  1.48s/it]
 22%|██▏       | 4182/18627 [2:18:46<6:41:11,  1.67s/it]
 22%|██▏       | 4183/18627 [2:18:47<5:53:35,  1.47s/it]
 22%|██▏       | 4184/18627 [2:18:49<5:33:22,  1.38s/it]
 22%|██▏       | 4185/18627 [2:18:51<6:26:17,  1.60s/it]
 22%|██▏       | 4186/18627 [2:18:53<7:03:27,  1.76s/it]
 22%|██▏       | 4187/18627 [2:18:55<7:28:26,  1.86s/it]
 22%|██▏       | 4188/18627 [2:18:57<7:48:59,  1.95s/it]
 22%|██▏       | 4189/18627 [2:18:59<8:02:22,  2.00s/it]
 22%|██▏       | 4190/18627 [2:19:01<8:10:06,  2.04s/it]
                                                        
{'loss': 1.3093, 'grad_norm': 10.466862678527832, 'learning_rate': 4.5180874586522815e-06, 'epoch': 0.22}
+
 22%|██▏       | 4190/18627 [2:19:01<8:10:06,  2.04s/it]
 22%|██▏       | 4191/18627 [2:19:04<8:16:02,  2.06s/it]
 23%|██▎       | 4192/18627 [2:19:04<6:38:05,  1.65s/it]
 23%|██▎       | 4193/18627 [2:19:06<7:11:16,  1.79s/it]
 23%|██▎       | 4194/18627 [2:19:08<7:34:01,  1.89s/it]
 23%|██▎       | 4195/18627 [2:19:11<7:51:00,  1.96s/it]
 23%|██▎       | 4196/18627 [2:19:13<8:02:55,  2.01s/it]
 23%|██▎       | 4197/18627 [2:19:15<8:09:28,  2.04s/it]
 23%|██▎       | 4198/18627 [2:19:16<7:00:41,  1.75s/it]
 23%|██▎       | 4199/18627 [2:19:18<7:27:42,  1.86s/it]
 23%|██▎       | 4200/18627 [2:19:19<6:38:42,  1.66s/it]
                                                        
{'loss': 1.0487, 'grad_norm': 16.26676368713379, 'learning_rate': 4.51551873680033e-06, 'epoch': 0.23}
+
 23%|██▎       | 4200/18627 [2:19:19<6:38:42,  1.66s/it]
 23%|██▎       | 4201/18627 [2:19:21<7:10:04,  1.79s/it]
 23%|██▎       | 4202/18627 [2:19:22<6:26:17,  1.61s/it]
 23%|██▎       | 4203/18627 [2:19:24<5:55:55,  1.48s/it]
 23%|██▎       | 4204/18627 [2:19:26<6:40:24,  1.67s/it]
 23%|██▎       | 4205/18627 [2:19:28<7:12:46,  1.80s/it]
 23%|██▎       | 4206/18627 [2:19:29<5:53:09,  1.47s/it]
 23%|██▎       | 4207/18627 [2:19:31<6:40:07,  1.66s/it]
 23%|██▎       | 4208/18627 [2:19:33<7:11:44,  1.80s/it]
 23%|██▎       | 4209/18627 [2:19:35<7:33:19,  1.89s/it]
 23%|██▎       | 4210/18627 [2:19:37<7:50:20,  1.96s/it]
                                                        
{'loss': 1.1198, 'grad_norm': 8.973531723022461, 'learning_rate': 4.512943921455443e-06, 'epoch': 0.23}
+
 23%|██▎       | 4210/18627 [2:19:37<7:50:20,  1.96s/it]
 23%|██▎       | 4211/18627 [2:19:39<8:00:58,  2.00s/it]
 23%|██▎       | 4212/18627 [2:19:41<8:09:47,  2.04s/it]
 23%|██▎       | 4213/18627 [2:19:43<8:14:57,  2.06s/it]
 23%|██▎       | 4214/18627 [2:19:45<7:12:03,  1.80s/it]
 23%|██▎       | 4215/18627 [2:19:47<7:34:37,  1.89s/it]
 23%|██▎       | 4216/18627 [2:19:48<6:32:29,  1.63s/it]
 23%|██▎       | 4217/18627 [2:19:50<7:08:06,  1.78s/it]
 23%|██▎       | 4218/18627 [2:19:51<6:26:06,  1.61s/it]
 23%|██▎       | 4219/18627 [2:19:52<5:55:47,  1.48s/it]
 23%|██▎       | 4220/18627 [2:19:54<6:41:24,  1.67s/it]
                                                        
{'loss': 1.5771, 'grad_norm': 7.409702777862549, 'learning_rate': 4.510363020402027e-06, 'epoch': 0.23}
+
 23%|██▎       | 4220/18627 [2:19:54<6:41:24,  1.67s/it]
 23%|██▎       | 4221/18627 [2:19:55<6:06:41,  1.53s/it]
 23%|██▎       | 4222/18627 [2:19:58<6:48:09,  1.70s/it]
 23%|██▎       | 4223/18627 [2:19:59<6:11:17,  1.55s/it]
 23%|██▎       | 4224/18627 [2:20:01<6:52:36,  1.72s/it]
 23%|██▎       | 4225/18627 [2:20:03<7:31:44,  1.88s/it]
 23%|██▎       | 4226/18627 [2:20:05<7:49:23,  1.96s/it]
 23%|██▎       | 4227/18627 [2:20:07<8:00:21,  2.00s/it]
 23%|██▎       | 4228/18627 [2:20:10<8:09:55,  2.04s/it]
 23%|██▎       | 4229/18627 [2:20:12<8:16:17,  2.07s/it]
 23%|██▎       | 4230/18627 [2:20:14<8:20:56,  2.09s/it]
                                                        
{'loss': 1.0369, 'grad_norm': 7.401615619659424, 'learning_rate': 4.50777604144289e-06, 'epoch': 0.23}
+
 23%|██▎       | 4230/18627 [2:20:14<8:20:56,  2.09s/it]
 23%|██▎       | 4231/18627 [2:20:16<8:24:15,  2.10s/it]
 23%|██▎       | 4232/18627 [2:20:17<7:18:10,  1.83s/it]
 23%|██▎       | 4233/18627 [2:20:19<7:40:20,  1.92s/it]
 23%|██▎       | 4234/18627 [2:20:21<7:54:13,  1.98s/it]
 23%|██▎       | 4235/18627 [2:20:23<8:03:58,  2.02s/it]
 23%|██▎       | 4236/18627 [2:20:26<8:11:48,  2.05s/it]
 23%|██▎       | 4237/18627 [2:20:28<8:15:32,  2.07s/it]
 23%|██▎       | 4238/18627 [2:20:30<8:18:16,  2.08s/it]
 23%|██▎       | 4239/18627 [2:20:32<8:20:49,  2.09s/it]
 23%|██▎       | 4240/18627 [2:20:33<7:16:29,  1.82s/it]
                                                        
{'loss': 1.1304, 'grad_norm': 13.788765907287598, 'learning_rate': 4.505182992399211e-06, 'epoch': 0.23}
+
 23%|██▎       | 4240/18627 [2:20:33<7:16:29,  1.82s/it]
 23%|██▎       | 4241/18627 [2:20:35<7:37:37,  1.91s/it]
 23%|██▎       | 4242/18627 [2:20:37<7:52:43,  1.97s/it]
 23%|██▎       | 4243/18627 [2:20:39<8:02:03,  2.01s/it]
 23%|██▎       | 4244/18627 [2:20:42<8:09:52,  2.04s/it]
 23%|██▎       | 4245/18627 [2:20:44<8:15:10,  2.07s/it]
 23%|██▎       | 4246/18627 [2:20:46<8:19:33,  2.08s/it]
 23%|██▎       | 4247/18627 [2:20:47<7:15:28,  1.82s/it]
 23%|██▎       | 4248/18627 [2:20:49<7:36:15,  1.90s/it]
 23%|██▎       | 4249/18627 [2:20:51<7:50:32,  1.96s/it]
 23%|██▎       | 4250/18627 [2:20:53<8:01:12,  2.01s/it]
                                                        
{'loss': 0.9463, 'grad_norm': 5.465855121612549, 'learning_rate': 4.502583881110524e-06, 'epoch': 0.23}
+
 23%|██▎       | 4250/18627 [2:20:53<8:01:12,  2.01s/it]
 23%|██▎       | 4251/18627 [2:20:55<8:09:19,  2.04s/it]
 23%|██▎       | 4252/18627 [2:20:58<8:15:26,  2.07s/it]
 23%|██▎       | 4253/18627 [2:21:00<8:19:31,  2.09s/it]
 23%|██▎       | 4254/18627 [2:21:01<7:15:31,  1.82s/it]
 23%|██▎       | 4255/18627 [2:21:02<6:29:51,  1.63s/it]
 23%|██▎       | 4256/18627 [2:21:04<7:05:55,  1.78s/it]
 23%|██▎       | 4257/18627 [2:21:06<7:29:42,  1.88s/it]
 23%|██▎       | 4258/18627 [2:21:08<6:40:05,  1.67s/it]
 23%|██▎       | 4259/18627 [2:21:10<7:13:06,  1.81s/it]
 23%|██▎       | 4260/18627 [2:21:12<7:34:06,  1.90s/it]
                                                        
{'loss': 1.4107, 'grad_norm': 6.6857428550720215, 'learning_rate': 4.499978715434691e-06, 'epoch': 0.23}
+
 23%|██▎       | 4260/18627 [2:21:12<7:34:06,  1.90s/it]
 23%|██▎       | 4261/18627 [2:21:14<7:50:08,  1.96s/it]
 23%|██▎       | 4262/18627 [2:21:16<8:03:13,  2.02s/it]
 23%|██▎       | 4263/18627 [2:21:17<7:03:10,  1.77s/it]
 23%|██▎       | 4264/18627 [2:21:19<7:28:32,  1.87s/it]
 23%|██▎       | 4265/18627 [2:21:21<7:44:31,  1.94s/it]
 23%|██▎       | 4266/18627 [2:21:24<7:57:07,  1.99s/it]
 23%|██▎       | 4267/18627 [2:21:26<8:04:40,  2.03s/it]
 23%|██▎       | 4268/18627 [2:21:28<8:10:39,  2.05s/it]
 23%|██▎       | 4269/18627 [2:21:29<7:08:11,  1.79s/it]
 23%|██▎       | 4270/18627 [2:21:30<6:25:10,  1.61s/it]
                                                        
{'loss': 1.2539, 'grad_norm': 16.070804595947266, 'learning_rate': 4.4973675032478764e-06, 'epoch': 0.23}
+
 23%|██▎       | 4270/18627 [2:21:30<6:25:10,  1.61s/it]
 23%|██▎       | 4271/18627 [2:21:32<7:00:57,  1.76s/it]
 23%|██▎       | 4272/18627 [2:21:34<7:26:46,  1.87s/it]
 23%|██▎       | 4273/18627 [2:21:36<7:43:37,  1.94s/it]
 23%|██▎       | 4274/18627 [2:21:39<7:57:11,  1.99s/it]
 23%|██▎       | 4275/18627 [2:21:40<6:50:35,  1.72s/it]
 23%|██▎       | 4276/18627 [2:21:42<7:20:18,  1.84s/it]
 23%|██▎       | 4277/18627 [2:21:44<7:41:00,  1.93s/it]
 23%|██▎       | 4278/18627 [2:21:46<7:54:33,  1.98s/it]
 23%|██▎       | 4279/18627 [2:21:48<8:03:53,  2.02s/it]
 23%|██▎       | 4280/18627 [2:21:50<8:09:43,  2.05s/it]
                                                        
{'loss': 0.9522, 'grad_norm': 8.641080856323242, 'learning_rate': 4.494750252444526e-06, 'epoch': 0.23}
+
 23%|██▎       | 4280/18627 [2:21:50<8:09:43,  2.05s/it]
 23%|██▎       | 4281/18627 [2:21:52<8:12:58,  2.06s/it]
 23%|██▎       | 4282/18627 [2:21:54<8:16:28,  2.08s/it]
 23%|██▎       | 4283/18627 [2:21:57<8:19:00,  2.09s/it]
 23%|██▎       | 4284/18627 [2:21:59<8:21:26,  2.10s/it]
 23%|██▎       | 4285/18627 [2:22:01<8:21:19,  2.10s/it]
 23%|██▎       | 4286/18627 [2:22:03<8:21:39,  2.10s/it]
 23%|██▎       | 4287/18627 [2:22:05<8:22:28,  2.10s/it]
 23%|██▎       | 4288/18627 [2:22:07<8:22:27,  2.10s/it]
 23%|██▎       | 4289/18627 [2:22:08<7:17:01,  1.83s/it]
 23%|██▎       | 4290/18627 [2:22:10<7:37:45,  1.92s/it]
                                                        
{'loss': 0.9396, 'grad_norm': 5.626453399658203, 'learning_rate': 4.492126970937343e-06, 'epoch': 0.23}
+
 23%|██▎       | 4290/18627 [2:22:10<7:37:45,  1.92s/it]
 23%|██▎       | 4291/18627 [2:22:12<7:51:40,  1.97s/it]
 23%|██▎       | 4292/18627 [2:22:14<6:55:00,  1.74s/it]
 23%|██▎       | 4293/18627 [2:22:16<7:22:42,  1.85s/it]
 23%|██▎       | 4294/18627 [2:22:18<7:40:55,  1.93s/it]
 23%|██▎       | 4295/18627 [2:22:19<6:48:07,  1.71s/it]
 23%|██▎       | 4296/18627 [2:22:21<7:18:20,  1.84s/it]
 23%|██▎       | 4297/18627 [2:22:23<7:37:44,  1.92s/it]
 23%|██▎       | 4298/18627 [2:22:25<7:52:13,  1.98s/it]
 23%|██▎       | 4299/18627 [2:22:27<6:55:55,  1.74s/it]
 23%|██▎       | 4300/18627 [2:22:29<7:23:32,  1.86s/it]
                                                        
{'loss': 1.2279, 'grad_norm': 7.115457534790039, 'learning_rate': 4.48949766665726e-06, 'epoch': 0.23}
+
 23%|██▎       | 4300/18627 [2:22:29<7:23:32,  1.86s/it]
 23%|██▎       | 4301/18627 [2:22:31<7:43:31,  1.94s/it]
 23%|██▎       | 4302/18627 [2:22:33<7:55:58,  1.99s/it]
 23%|██▎       | 4303/18627 [2:22:35<8:05:07,  2.03s/it]
 23%|██▎       | 4304/18627 [2:22:37<8:12:40,  2.06s/it]
 23%|██▎       | 4305/18627 [2:22:38<6:57:54,  1.75s/it]
 23%|██▎       | 4306/18627 [2:22:40<7:24:44,  1.86s/it]
 23%|██▎       | 4307/18627 [2:22:43<7:43:25,  1.94s/it]
 23%|██▎       | 4308/18627 [2:22:45<7:57:28,  2.00s/it]
 23%|██▎       | 4309/18627 [2:22:47<8:04:57,  2.03s/it]
 23%|██▎       | 4310/18627 [2:22:49<8:11:48,  2.06s/it]
                                                        
{'loss': 0.9312, 'grad_norm': 7.935233116149902, 'learning_rate': 4.486862347553421e-06, 'epoch': 0.23}
+
 23%|██▎       | 4310/18627 [2:22:49<8:11:48,  2.06s/it]
 23%|██▎       | 4311/18627 [2:22:51<8:15:48,  2.08s/it]
 23%|██▎       | 4312/18627 [2:22:53<8:19:06,  2.09s/it]
 23%|██▎       | 4313/18627 [2:22:55<8:20:00,  2.10s/it]
 23%|██▎       | 4314/18627 [2:22:57<8:21:07,  2.10s/it]
 23%|██▎       | 4315/18627 [2:23:00<8:23:10,  2.11s/it]
 23%|██▎       | 4316/18627 [2:23:02<8:24:13,  2.11s/it]
 23%|██▎       | 4317/18627 [2:23:04<8:23:20,  2.11s/it]
 23%|██▎       | 4318/18627 [2:23:06<8:22:35,  2.11s/it]
 23%|██▎       | 4319/18627 [2:23:08<8:23:06,  2.11s/it]
 23%|██▎       | 4320/18627 [2:23:10<8:23:05,  2.11s/it]
                                                        
{'loss': 0.81, 'grad_norm': 9.546976089477539, 'learning_rate': 4.484221021593154e-06, 'epoch': 0.23}
+
 23%|██▎       | 4320/18627 [2:23:10<8:23:05,  2.11s/it]
 23%|██▎       | 4321/18627 [2:23:12<8:23:45,  2.11s/it]
 23%|██▎       | 4322/18627 [2:23:14<8:23:08,  2.11s/it]
 23%|██▎       | 4323/18627 [2:23:16<8:22:36,  2.11s/it]
 23%|██▎       | 4324/18627 [2:23:19<8:23:47,  2.11s/it]
 23%|██▎       | 4325/18627 [2:23:20<7:17:43,  1.84s/it]
 23%|██▎       | 4326/18627 [2:23:22<7:36:27,  1.92s/it]
 23%|██▎       | 4327/18627 [2:23:24<7:49:25,  1.97s/it]
 23%|██▎       | 4328/18627 [2:23:25<6:53:47,  1.74s/it]
 23%|██▎       | 4329/18627 [2:23:27<7:21:16,  1.85s/it]
 23%|██▎       | 4330/18627 [2:23:28<6:33:52,  1.65s/it]
                                                        
{'loss': 1.2958, 'grad_norm': 16.181175231933594, 'learning_rate': 4.4815736967619475e-06, 'epoch': 0.23}
+
 23%|██▎       | 4330/18627 [2:23:28<6:33:52,  1.65s/it]
 23%|██▎       | 4331/18627 [2:23:31<7:06:36,  1.79s/it]
 23%|██▎       | 4332/18627 [2:23:33<7:28:57,  1.88s/it]
 23%|██▎       | 4333/18627 [2:23:35<7:44:45,  1.95s/it]
 23%|██▎       | 4334/18627 [2:23:37<7:56:38,  2.00s/it]
 23%|██▎       | 4335/18627 [2:23:39<8:05:28,  2.04s/it]
 23%|██▎       | 4336/18627 [2:23:41<8:10:25,  2.06s/it]
 23%|██▎       | 4337/18627 [2:23:43<8:15:04,  2.08s/it]
 23%|██▎       | 4338/18627 [2:23:44<7:11:10,  1.81s/it]
 23%|██▎       | 4339/18627 [2:23:47<7:34:06,  1.91s/it]
 23%|██▎       | 4340/18627 [2:23:49<7:50:04,  1.97s/it]
                                                        
{'loss': 0.8708, 'grad_norm': 8.141857147216797, 'learning_rate': 4.478920381063426e-06, 'epoch': 0.23}
+
 23%|██▎       | 4340/18627 [2:23:49<7:50:04,  1.97s/it]
 23%|██▎       | 4341/18627 [2:23:51<7:59:00,  2.01s/it]
 23%|██▎       | 4342/18627 [2:23:53<8:05:42,  2.04s/it]
 23%|██▎       | 4343/18627 [2:23:55<8:12:39,  2.07s/it]
 23%|██▎       | 4344/18627 [2:23:57<8:15:36,  2.08s/it]
 23%|██▎       | 4345/18627 [2:23:59<8:17:26,  2.09s/it]
 23%|██▎       | 4346/18627 [2:24:01<8:19:55,  2.10s/it]
 23%|██▎       | 4347/18627 [2:24:03<7:15:34,  1.83s/it]
 23%|██▎       | 4348/18627 [2:24:05<7:36:55,  1.92s/it]
 23%|██▎       | 4349/18627 [2:24:06<6:44:59,  1.70s/it]
 23%|██▎       | 4350/18627 [2:24:07<6:00:47,  1.52s/it]
                                                        
{'loss': 1.3204, 'grad_norm': 15.22232723236084, 'learning_rate': 4.476261082519325e-06, 'epoch': 0.23}
+
 23%|██▎       | 4350/18627 [2:24:07<6:00:47,  1.52s/it]
 23%|██▎       | 4351/18627 [2:24:09<6:44:28,  1.70s/it]
 23%|██▎       | 4352/18627 [2:24:11<7:15:19,  1.83s/it]
 23%|██▎       | 4353/18627 [2:24:13<7:35:50,  1.92s/it]
 23%|██▎       | 4354/18627 [2:24:15<7:50:08,  1.98s/it]
 23%|██▎       | 4355/18627 [2:24:18<8:00:14,  2.02s/it]
 23%|██▎       | 4356/18627 [2:24:20<8:07:45,  2.05s/it]
 23%|██▎       | 4357/18627 [2:24:22<8:12:26,  2.07s/it]
 23%|██▎       | 4358/18627 [2:24:24<8:15:38,  2.08s/it]
 23%|██▎       | 4359/18627 [2:24:25<7:11:31,  1.81s/it]
 23%|██▎       | 4360/18627 [2:24:27<7:33:43,  1.91s/it]
                                                        
{'loss': 0.957, 'grad_norm': 5.935036659240723, 'learning_rate': 4.47359580916947e-06, 'epoch': 0.23}
+
 23%|██▎       | 4360/18627 [2:24:27<7:33:43,  1.91s/it]
 23%|██▎       | 4361/18627 [2:24:29<7:49:47,  1.98s/it]
 23%|██▎       | 4362/18627 [2:24:31<7:58:36,  2.01s/it]
 23%|██▎       | 4363/18627 [2:24:34<8:06:29,  2.05s/it]
 23%|██▎       | 4364/18627 [2:24:35<7:05:35,  1.79s/it]
 23%|██▎       | 4365/18627 [2:24:36<6:17:07,  1.59s/it]
 23%|██▎       | 4366/18627 [2:24:38<6:57:18,  1.76s/it]
 23%|██▎       | 4367/18627 [2:24:39<6:16:05,  1.58s/it]
 23%|██▎       | 4368/18627 [2:24:41<6:54:25,  1.74s/it]
 23%|██▎       | 4369/18627 [2:24:43<6:15:04,  1.58s/it]
 23%|██▎       | 4370/18627 [2:24:45<6:52:58,  1.74s/it]
                                                        
{'loss': 1.4029, 'grad_norm': 5.433386325836182, 'learning_rate': 4.47092456907175e-06, 'epoch': 0.23}
+
 23%|██▎       | 4370/18627 [2:24:45<6:52:58,  1.74s/it]
 23%|██▎       | 4371/18627 [2:24:47<7:20:07,  1.85s/it]
 23%|██▎       | 4372/18627 [2:24:49<7:37:15,  1.92s/it]
 23%|██▎       | 4373/18627 [2:24:50<6:44:29,  1.70s/it]
 23%|██▎       | 4374/18627 [2:24:52<7:13:02,  1.82s/it]
 23%|██▎       | 4375/18627 [2:24:54<7:34:43,  1.91s/it]
 23%|██▎       | 4376/18627 [2:24:55<6:43:01,  1.70s/it]
 23%|██▎       | 4377/18627 [2:24:57<6:06:47,  1.54s/it]
 24%|██▎       | 4378/18627 [2:24:59<6:47:09,  1.71s/it]
 24%|██▎       | 4379/18627 [2:25:01<7:16:21,  1.84s/it]
 24%|██▎       | 4380/18627 [2:25:02<6:30:23,  1.64s/it]
                                                        
{'loss': 1.4933, 'grad_norm': 15.733256340026855, 'learning_rate': 4.468247370302089e-06, 'epoch': 0.24}
+
 24%|██▎       | 4380/18627 [2:25:02<6:30:23,  1.64s/it]
 24%|██▎       | 4381/18627 [2:25:04<7:04:12,  1.79s/it]
 24%|██▎       | 4382/18627 [2:25:05<6:21:53,  1.61s/it]
 24%|██▎       | 4383/18627 [2:25:08<6:58:10,  1.76s/it]
 24%|██▎       | 4384/18627 [2:25:09<6:18:43,  1.60s/it]
 24%|██▎       | 4385/18627 [2:25:10<5:49:49,  1.47s/it]
 24%|██▎       | 4386/18627 [2:25:11<5:29:40,  1.39s/it]
 24%|██▎       | 4387/18627 [2:25:13<6:20:42,  1.60s/it]
 24%|██▎       | 4388/18627 [2:25:14<5:44:58,  1.45s/it]
 24%|██▎       | 4389/18627 [2:25:16<6:31:45,  1.65s/it]
 24%|██▎       | 4390/18627 [2:25:19<7:04:09,  1.79s/it]
                                                        
{'loss': 1.684, 'grad_norm': 13.008451461791992, 'learning_rate': 4.465564220954433e-06, 'epoch': 0.24}
+
 24%|██▎       | 4390/18627 [2:25:19<7:04:09,  1.79s/it]
 24%|██▎       | 4391/18627 [2:25:21<7:27:12,  1.88s/it]
 24%|██▎       | 4392/18627 [2:25:23<7:43:43,  1.95s/it]
 24%|██▎       | 4393/18627 [2:25:24<6:41:39,  1.69s/it]
 24%|██▎       | 4394/18627 [2:25:25<6:05:37,  1.54s/it]
 24%|██▎       | 4395/18627 [2:25:27<6:46:56,  1.72s/it]
 24%|██▎       | 4396/18627 [2:25:28<6:09:29,  1.56s/it]
 24%|██▎       | 4397/18627 [2:25:30<6:49:39,  1.73s/it]
 24%|██▎       | 4398/18627 [2:25:33<7:18:10,  1.85s/it]
 24%|██▎       | 4399/18627 [2:25:35<7:38:27,  1.93s/it]
 24%|██▎       | 4400/18627 [2:25:37<7:53:28,  2.00s/it]
                                                        
{'loss': 1.2625, 'grad_norm': 6.19653844833374, 'learning_rate': 4.462875129140711e-06, 'epoch': 0.24}
+
 24%|██▎       | 4400/18627 [2:25:37<7:53:28,  2.00s/it]
 24%|██▎       | 4401/18627 [2:25:39<8:02:47,  2.04s/it]
 24%|██▎       | 4402/18627 [2:25:41<8:10:20,  2.07s/it]
 24%|██▎       | 4403/18627 [2:25:43<8:12:27,  2.08s/it]
 24%|██▎       | 4404/18627 [2:25:45<8:15:37,  2.09s/it]
 24%|██▎       | 4405/18627 [2:25:47<8:17:31,  2.10s/it]
 24%|██▎       | 4406/18627 [2:25:50<8:18:44,  2.10s/it]
 24%|██▎       | 4407/18627 [2:25:52<8:18:42,  2.10s/it]
 24%|██▎       | 4408/18627 [2:25:54<8:20:25,  2.11s/it]
 24%|██▎       | 4409/18627 [2:25:56<8:20:44,  2.11s/it]
 24%|██▎       | 4410/18627 [2:25:58<8:20:38,  2.11s/it]
                                                        
{'loss': 0.6682, 'grad_norm': 7.171701908111572, 'learning_rate': 4.460180102990823e-06, 'epoch': 0.24}
+
 24%|██▎       | 4410/18627 [2:25:58<8:20:38,  2.11s/it]
 24%|██▎       | 4411/18627 [2:26:00<8:20:12,  2.11s/it]
 24%|██▎       | 4412/18627 [2:26:02<8:20:20,  2.11s/it]
 24%|██▎       | 4413/18627 [2:26:04<8:20:10,  2.11s/it]
 24%|██▎       | 4414/18627 [2:26:07<8:21:38,  2.12s/it]
 24%|██▎       | 4415/18627 [2:26:09<8:23:24,  2.13s/it]
 24%|██▎       | 4416/18627 [2:26:11<8:21:34,  2.12s/it]
 24%|██▎       | 4417/18627 [2:26:13<8:21:10,  2.12s/it]
 24%|██▎       | 4418/18627 [2:26:15<8:20:30,  2.11s/it]
 24%|██▎       | 4419/18627 [2:26:17<8:21:14,  2.12s/it]
 24%|██▎       | 4420/18627 [2:26:19<8:22:22,  2.12s/it]
                                                        
{'loss': 0.71, 'grad_norm': 5.662790775299072, 'learning_rate': 4.457479150652607e-06, 'epoch': 0.24}
+
 24%|██▎       | 4420/18627 [2:26:19<8:22:22,  2.12s/it]
 24%|██▎       | 4421/18627 [2:26:21<8:22:47,  2.12s/it]
 24%|██▎       | 4422/18627 [2:26:23<8:22:46,  2.12s/it]
 24%|██▎       | 4423/18627 [2:26:26<8:21:35,  2.12s/it]
 24%|██▍       | 4424/18627 [2:26:27<7:15:03,  1.84s/it]
 24%|██▍       | 4425/18627 [2:26:29<7:36:32,  1.93s/it]
 24%|██▍       | 4426/18627 [2:26:31<7:51:43,  1.99s/it]
 24%|██▍       | 4427/18627 [2:26:33<8:00:46,  2.03s/it]
 24%|██▍       | 4428/18627 [2:26:34<7:01:08,  1.78s/it]
 24%|██▍       | 4429/18627 [2:26:37<7:25:54,  1.88s/it]
 24%|██▍       | 4430/18627 [2:26:39<7:41:50,  1.95s/it]
                                                        
{'loss': 1.028, 'grad_norm': 7.740350246429443, 'learning_rate': 4.454772280291821e-06, 'epoch': 0.24}
+
 24%|██▍       | 4430/18627 [2:26:39<7:41:50,  1.95s/it]
 24%|██▍       | 4431/18627 [2:26:41<7:54:10,  2.00s/it]
 24%|██▍       | 4432/18627 [2:26:43<8:03:06,  2.04s/it]
 24%|██▍       | 4433/18627 [2:26:45<8:09:24,  2.07s/it]
 24%|██▍       | 4434/18627 [2:26:47<8:12:41,  2.08s/it]
 24%|██▍       | 4435/18627 [2:26:50<8:37:43,  2.19s/it]
 24%|██▍       | 4436/18627 [2:26:52<8:31:51,  2.16s/it]
 24%|██▍       | 4437/18627 [2:26:54<8:29:06,  2.15s/it]
 24%|██▍       | 4438/18627 [2:26:56<8:27:18,  2.15s/it]
 24%|██▍       | 4439/18627 [2:26:58<8:25:59,  2.14s/it]
 24%|██▍       | 4440/18627 [2:27:00<8:22:56,  2.13s/it]
                                                        
{'loss': 0.6745, 'grad_norm': 7.4511399269104, 'learning_rate': 4.452059500092111e-06, 'epoch': 0.24}
+
 24%|██▍       | 4440/18627 [2:27:00<8:22:56,  2.13s/it]
 24%|██▍       | 4441/18627 [2:27:02<8:21:55,  2.12s/it]
 24%|██▍       | 4442/18627 [2:27:04<8:21:59,  2.12s/it]
 24%|██▍       | 4443/18627 [2:27:07<8:21:41,  2.12s/it]
 24%|██▍       | 4444/18627 [2:27:09<8:20:54,  2.12s/it]
 24%|██▍       | 4445/18627 [2:27:11<8:21:02,  2.12s/it]
 24%|██▍       | 4446/18627 [2:27:13<8:21:38,  2.12s/it]
 24%|██▍       | 4447/18627 [2:27:15<8:20:36,  2.12s/it]
 24%|██▍       | 4448/18627 [2:27:17<8:20:32,  2.12s/it]
 24%|██▍       | 4449/18627 [2:27:19<8:20:46,  2.12s/it]
 24%|██▍       | 4450/18627 [2:27:20<7:14:59,  1.84s/it]
                                                        
{'loss': 0.9609, 'grad_norm': 14.263236045837402, 'learning_rate': 4.449340818254992e-06, 'epoch': 0.24}
+
 24%|██▍       | 4450/18627 [2:27:20<7:14:59,  1.84s/it]
 24%|██▍       | 4451/18627 [2:27:21<5:54:47,  1.50s/it]
 24%|██▍       | 4452/18627 [2:27:22<4:58:16,  1.26s/it]
 24%|██▍       | 4453/18627 [2:27:24<6:00:44,  1.53s/it]
 24%|██▍       | 4454/18627 [2:27:26<6:42:50,  1.71s/it]
 24%|██▍       | 4455/18627 [2:27:28<7:11:02,  1.82s/it]
 24%|██▍       | 4456/18627 [2:27:29<6:13:59,  1.58s/it]
 24%|██▍       | 4457/18627 [2:27:31<6:52:42,  1.75s/it]
 24%|██▍       | 4458/18627 [2:27:33<7:19:49,  1.86s/it]
 24%|██▍       | 4459/18627 [2:27:36<7:38:04,  1.94s/it]
 24%|██▍       | 4460/18627 [2:27:37<6:41:16,  1.70s/it]
                                                        
{'loss': 1.153, 'grad_norm': 13.44764232635498, 'learning_rate': 4.446616242999822e-06, 'epoch': 0.24}
+
 24%|██▍       | 4460/18627 [2:27:37<6:41:16,  1.70s/it]
 24%|██▍       | 4461/18627 [2:27:39<7:09:49,  1.82s/it]
 24%|██▍       | 4462/18627 [2:27:41<7:30:55,  1.91s/it]
 24%|██▍       | 4463/18627 [2:27:43<7:45:29,  1.97s/it]
 24%|██▍       | 4464/18627 [2:27:45<7:56:27,  2.02s/it]
 24%|██▍       | 4465/18627 [2:27:47<8:04:18,  2.05s/it]
 24%|██▍       | 4466/18627 [2:27:49<8:07:26,  2.07s/it]
 24%|██▍       | 4467/18627 [2:27:52<8:10:02,  2.08s/it]
 24%|██▍       | 4468/18627 [2:27:54<8:13:04,  2.09s/it]
 24%|██▍       | 4469/18627 [2:27:55<6:57:11,  1.77s/it]
 24%|██▍       | 4470/18627 [2:27:57<7:22:54,  1.88s/it]
                                                        
{'loss': 0.9713, 'grad_norm': 9.663025856018066, 'learning_rate': 4.443885782563775e-06, 'epoch': 0.24}
+
 24%|██▍       | 4470/18627 [2:27:57<7:22:54,  1.88s/it]
 24%|██▍       | 4471/18627 [2:27:58<6:33:58,  1.67s/it]
 24%|██▍       | 4472/18627 [2:28:00<7:05:47,  1.80s/it]
 24%|██▍       | 4473/18627 [2:28:01<6:22:55,  1.62s/it]
 24%|██▍       | 4474/18627 [2:28:03<6:57:50,  1.77s/it]
 24%|██▍       | 4475/18627 [2:28:06<7:21:44,  1.87s/it]
 24%|██▍       | 4476/18627 [2:28:08<7:39:50,  1.95s/it]
 24%|██▍       | 4477/18627 [2:28:09<6:46:23,  1.72s/it]
 24%|██▍       | 4478/18627 [2:28:11<7:15:05,  1.85s/it]
 24%|██▍       | 4479/18627 [2:28:13<7:35:09,  1.93s/it]
 24%|██▍       | 4480/18627 [2:28:14<6:42:14,  1.71s/it]
                                                        
{'loss': 1.4246, 'grad_norm': 14.856078147888184, 'learning_rate': 4.4411494452018185e-06, 'epoch': 0.24}
+
 24%|██▍       | 4480/18627 [2:28:14<6:42:14,  1.71s/it]
 24%|██▍       | 4481/18627 [2:28:16<7:10:25,  1.83s/it]
 24%|██▍       | 4482/18627 [2:28:19<7:30:52,  1.91s/it]
 24%|██▍       | 4483/18627 [2:28:21<7:44:21,  1.97s/it]
 24%|██▍       | 4484/18627 [2:28:23<7:56:09,  2.02s/it]
 24%|██▍       | 4485/18627 [2:28:25<8:04:06,  2.05s/it]
 24%|██▍       | 4486/18627 [2:28:26<6:28:51,  1.65s/it]
 24%|██▍       | 4487/18627 [2:28:28<7:02:14,  1.79s/it]
 24%|██▍       | 4488/18627 [2:28:30<7:25:34,  1.89s/it]
 24%|██▍       | 4489/18627 [2:28:32<7:41:28,  1.96s/it]
 24%|██▍       | 4490/18627 [2:28:34<7:53:08,  2.01s/it]
                                                        
{'loss': 0.7398, 'grad_norm': 7.624509334564209, 'learning_rate': 4.438407239186689e-06, 'epoch': 0.24}
+
 24%|██▍       | 4490/18627 [2:28:34<7:53:08,  2.01s/it]
 24%|██▍       | 4491/18627 [2:28:36<8:00:26,  2.04s/it]
 24%|██▍       | 4492/18627 [2:28:38<8:05:43,  2.06s/it]
 24%|██▍       | 4493/18627 [2:28:40<8:08:43,  2.07s/it]
 24%|██▍       | 4494/18627 [2:28:43<8:11:14,  2.09s/it]
 24%|██▍       | 4495/18627 [2:28:45<8:13:27,  2.10s/it]
 24%|██▍       | 4496/18627 [2:28:47<8:14:30,  2.10s/it]
 24%|██▍       | 4497/18627 [2:28:49<8:14:32,  2.10s/it]
 24%|██▍       | 4498/18627 [2:28:51<8:17:39,  2.11s/it]
 24%|██▍       | 4499/18627 [2:28:53<8:17:00,  2.11s/it]
 24%|██▍       | 4500/18627 [2:28:55<8:16:24,  2.11s/it]
                                                        
{'loss': 0.7128, 'grad_norm': 6.766681671142578, 'learning_rate': 4.435659172808862e-06, 'epoch': 0.24}
+
 24%|██▍       | 4500/18627 [2:28:55<8:16:24,  2.11s/it]
 24%|██▍       | 4501/18627 [2:28:57<8:17:31,  2.11s/it]
 24%|██▍       | 4502/18627 [2:28:59<8:17:17,  2.11s/it]
 24%|██▍       | 4503/18627 [2:29:02<8:17:41,  2.11s/it]
 24%|██▍       | 4504/18627 [2:29:03<7:12:47,  1.84s/it]
 24%|██▍       | 4505/18627 [2:29:05<7:31:24,  1.92s/it]
 24%|██▍       | 4506/18627 [2:29:07<7:46:12,  1.98s/it]
 24%|██▍       | 4507/18627 [2:29:09<7:55:17,  2.02s/it]
 24%|██▍       | 4508/18627 [2:29:11<8:00:12,  2.04s/it]
 24%|██▍       | 4509/18627 [2:29:13<8:04:48,  2.06s/it]
 24%|██▍       | 4510/18627 [2:29:15<8:09:53,  2.08s/it]
                                                        
{'loss': 0.8224, 'grad_norm': 7.576805591583252, 'learning_rate': 4.432905254376534e-06, 'epoch': 0.24}
+
 24%|██▍       | 4510/18627 [2:29:15<8:09:53,  2.08s/it]
 24%|██▍       | 4511/18627 [2:29:18<8:15:08,  2.10s/it]
 24%|██▍       | 4512/18627 [2:29:20<8:16:14,  2.11s/it]
 24%|██▍       | 4513/18627 [2:29:20<6:37:29,  1.69s/it]
 24%|██▍       | 4514/18627 [2:29:21<5:55:19,  1.51s/it]
 24%|██▍       | 4515/18627 [2:29:24<6:38:00,  1.69s/it]
 24%|██▍       | 4516/18627 [2:29:25<6:02:49,  1.54s/it]
 24%|██▍       | 4517/18627 [2:29:26<5:37:50,  1.44s/it]
 24%|██▍       | 4518/18627 [2:29:28<6:26:34,  1.64s/it]
 24%|██▍       | 4519/18627 [2:29:30<6:59:03,  1.78s/it]
 24%|██▍       | 4520/18627 [2:29:32<7:22:54,  1.88s/it]
                                                        
{'loss': 1.2548, 'grad_norm': 6.9703192710876465, 'learning_rate': 4.4301454922155915e-06, 'epoch': 0.24}
+
 24%|██▍       | 4520/18627 [2:29:32<7:22:54,  1.88s/it]
 24%|██▍       | 4521/18627 [2:29:34<7:38:49,  1.95s/it]
 24%|██▍       | 4522/18627 [2:29:37<7:50:46,  2.00s/it]
 24%|██▍       | 4523/18627 [2:29:39<7:57:48,  2.03s/it]
 24%|██▍       | 4524/18627 [2:29:41<8:04:04,  2.06s/it]
 24%|██▍       | 4525/18627 [2:29:43<8:08:39,  2.08s/it]
 24%|██▍       | 4526/18627 [2:29:45<8:12:06,  2.09s/it]
 24%|██▍       | 4527/18627 [2:29:47<8:14:20,  2.10s/it]
 24%|██▍       | 4528/18627 [2:29:49<8:16:14,  2.11s/it]
 24%|██▍       | 4529/18627 [2:29:51<7:11:23,  1.84s/it]
 24%|██▍       | 4530/18627 [2:29:52<6:25:38,  1.64s/it]
                                                        
{'loss': 1.0456, 'grad_norm': 15.487590789794922, 'learning_rate': 4.427379894669591e-06, 'epoch': 0.24}
+
 24%|██▍       | 4530/18627 [2:29:52<6:25:38,  1.64s/it]
 24%|██▍       | 4531/18627 [2:29:53<5:54:20,  1.51s/it]
 24%|██▍       | 4532/18627 [2:29:54<5:31:50,  1.41s/it]
 24%|██▍       | 4533/18627 [2:29:56<6:22:19,  1.63s/it]
 24%|██▍       | 4534/18627 [2:29:58<6:54:44,  1.77s/it]
 24%|██▍       | 4535/18627 [2:30:00<7:19:47,  1.87s/it]
 24%|██▍       | 4536/18627 [2:30:03<7:35:56,  1.94s/it]
 24%|██▍       | 4537/18627 [2:30:05<7:48:45,  2.00s/it]
 24%|██▍       | 4538/18627 [2:30:07<7:57:26,  2.03s/it]
 24%|██▍       | 4539/18627 [2:30:08<6:57:29,  1.78s/it]
 24%|██▍       | 4540/18627 [2:30:09<6:10:48,  1.58s/it]
                                                        
{'loss': 1.4801, 'grad_norm': 15.843807220458984, 'learning_rate': 4.424608470099728e-06, 'epoch': 0.24}
+
 24%|██▍       | 4540/18627 [2:30:09<6:10:48,  1.58s/it]
 24%|██▍       | 4541/18627 [2:30:11<6:48:33,  1.74s/it]
 24%|██▍       | 4542/18627 [2:30:13<7:14:21,  1.85s/it]
 24%|██▍       | 4543/18627 [2:30:15<7:35:06,  1.94s/it]
 24%|██▍       | 4544/18627 [2:30:18<7:48:36,  2.00s/it]
 24%|██▍       | 4545/18627 [2:30:20<7:58:53,  2.04s/it]
 24%|██▍       | 4546/18627 [2:30:22<8:05:07,  2.07s/it]
 24%|██▍       | 4547/18627 [2:30:24<8:07:46,  2.08s/it]
 24%|██▍       | 4548/18627 [2:30:26<8:11:14,  2.09s/it]
 24%|██▍       | 4549/18627 [2:30:27<7:07:04,  1.82s/it]
 24%|██▍       | 4550/18627 [2:30:29<7:28:20,  1.91s/it]
                                                        
{'loss': 0.892, 'grad_norm': 7.35399055480957, 'learning_rate': 4.421831226884817e-06, 'epoch': 0.24}
+
 24%|██▍       | 4550/18627 [2:30:29<7:28:20,  1.91s/it]
 24%|██▍       | 4551/18627 [2:30:31<7:43:10,  1.97s/it]
 24%|██▍       | 4552/18627 [2:30:33<6:47:42,  1.74s/it]
 24%|██▍       | 4553/18627 [2:30:34<6:08:47,  1.57s/it]
 24%|██▍       | 4554/18627 [2:30:36<6:46:04,  1.73s/it]
 24%|██▍       | 4555/18627 [2:30:38<7:12:45,  1.85s/it]
 24%|██▍       | 4556/18627 [2:30:39<6:26:10,  1.65s/it]
 24%|██▍       | 4557/18627 [2:30:41<6:59:22,  1.79s/it]
 24%|██▍       | 4558/18627 [2:30:44<7:22:26,  1.89s/it]
 24%|██▍       | 4559/18627 [2:30:46<7:39:26,  1.96s/it]
 24%|██▍       | 4560/18627 [2:30:48<7:51:40,  2.01s/it]
                                                        
{'loss': 1.2843, 'grad_norm': 4.954522609710693, 'learning_rate': 4.419048173421262e-06, 'epoch': 0.24}
+
 24%|██▍       | 4560/18627 [2:30:48<7:51:40,  2.01s/it]
 24%|██▍       | 4561/18627 [2:30:49<6:53:27,  1.76s/it]
 24%|██▍       | 4562/18627 [2:30:50<6:14:03,  1.60s/it]
 24%|██▍       | 4563/18627 [2:30:51<5:47:52,  1.48s/it]
 25%|██▍       | 4564/18627 [2:30:54<6:33:11,  1.68s/it]
 25%|██▍       | 4565/18627 [2:30:56<7:04:10,  1.81s/it]
 25%|██▍       | 4566/18627 [2:30:58<7:24:50,  1.90s/it]
 25%|██▍       | 4567/18627 [2:31:00<7:39:50,  1.96s/it]
 25%|██▍       | 4568/18627 [2:31:01<6:45:26,  1.73s/it]
 25%|██▍       | 4569/18627 [2:31:03<7:14:11,  1.85s/it]
 25%|██▍       | 4570/18627 [2:31:05<7:32:20,  1.93s/it]
                                                        
{'loss': 1.6394, 'grad_norm': 8.045175552368164, 'learning_rate': 4.416259318123036e-06, 'epoch': 0.25}
+
 25%|██▍       | 4570/18627 [2:31:05<7:32:20,  1.93s/it]
 25%|██▍       | 4571/18627 [2:31:07<7:46:38,  1.99s/it]
 25%|██▍       | 4572/18627 [2:31:09<6:49:55,  1.75s/it]
 25%|██▍       | 4573/18627 [2:31:11<7:14:59,  1.86s/it]
 25%|██▍       | 4574/18627 [2:31:13<7:32:30,  1.93s/it]
 25%|██▍       | 4575/18627 [2:31:15<7:45:55,  1.99s/it]
 25%|██▍       | 4576/18627 [2:31:16<6:50:10,  1.75s/it]
 25%|██▍       | 4577/18627 [2:31:17<5:38:46,  1.45s/it]
 25%|██▍       | 4578/18627 [2:31:19<6:24:42,  1.64s/it]
 25%|██▍       | 4579/18627 [2:31:21<6:58:42,  1.79s/it]
 25%|██▍       | 4580/18627 [2:31:22<6:16:38,  1.61s/it]
                                                        
{'loss': 1.2693, 'grad_norm': 15.304893493652344, 'learning_rate': 4.41346466942165e-06, 'epoch': 0.25}
+
 25%|██▍       | 4580/18627 [2:31:22<6:16:38,  1.61s/it]
 25%|██▍       | 4581/18627 [2:31:24<6:52:15,  1.76s/it]
 25%|██▍       | 4582/18627 [2:31:27<7:16:22,  1.86s/it]
 25%|██▍       | 4583/18627 [2:31:30<9:04:01,  2.32s/it]
 25%|██▍       | 4584/18627 [2:31:31<7:44:17,  1.98s/it]
 25%|██▍       | 4585/18627 [2:31:33<7:53:47,  2.02s/it]
 25%|██▍       | 4586/18627 [2:31:34<6:54:53,  1.77s/it]
 25%|██▍       | 4587/18627 [2:31:36<6:07:39,  1.57s/it]
 25%|██▍       | 4588/18627 [2:31:38<6:45:10,  1.73s/it]
 25%|██▍       | 4589/18627 [2:31:40<7:12:59,  1.85s/it]
 25%|██▍       | 4590/18627 [2:31:42<7:32:08,  1.93s/it]
                                                        
{'loss': 1.3058, 'grad_norm': 8.72211742401123, 'learning_rate': 4.410664235766131e-06, 'epoch': 0.25}
+
 25%|██▍       | 4590/18627 [2:31:42<7:32:08,  1.93s/it]
 25%|██▍       | 4591/18627 [2:31:44<7:45:14,  1.99s/it]
 25%|██▍       | 4592/18627 [2:31:45<6:50:26,  1.75s/it]
 25%|██▍       | 4593/18627 [2:31:46<5:36:36,  1.44s/it]
 25%|██▍       | 4594/18627 [2:31:48<6:24:27,  1.64s/it]
 25%|██▍       | 4595/18627 [2:31:50<6:58:06,  1.79s/it]
 25%|██▍       | 4596/18627 [2:31:52<7:21:39,  1.89s/it]
 25%|██▍       | 4597/18627 [2:31:54<7:38:14,  1.96s/it]
 25%|██▍       | 4598/18627 [2:31:56<7:48:17,  2.00s/it]
 25%|██▍       | 4599/18627 [2:31:58<6:50:17,  1.75s/it]
 25%|██▍       | 4600/18627 [2:31:58<5:36:30,  1.44s/it]
                                                        
{'loss': 0.9553, 'grad_norm': 11.725997924804688, 'learning_rate': 4.4078580256229956e-06, 'epoch': 0.25}
+
 25%|██▍       | 4600/18627 [2:31:58<5:36:30,  1.44s/it]
 25%|██▍       | 4601/18627 [2:32:00<6:22:58,  1.64s/it]
 25%|██▍       | 4602/18627 [2:32:02<5:44:12,  1.47s/it]
 25%|██▍       | 4603/18627 [2:32:02<4:50:11,  1.24s/it]
 25%|██▍       | 4604/18627 [2:32:04<5:51:33,  1.50s/it]
 25%|██▍       | 4605/18627 [2:32:06<6:34:19,  1.69s/it]
 25%|██▍       | 4606/18627 [2:32:09<7:03:30,  1.81s/it]
 25%|██▍       | 4607/18627 [2:32:10<6:19:52,  1.63s/it]
 25%|██▍       | 4608/18627 [2:32:12<6:53:50,  1.77s/it]
 25%|██▍       | 4609/18627 [2:32:14<7:18:36,  1.88s/it]
 25%|██▍       | 4610/18627 [2:32:15<6:20:27,  1.63s/it]
                                                        
{'loss': 1.298, 'grad_norm': 15.206296920776367, 'learning_rate': 4.405046047476224e-06, 'epoch': 0.25}
+
 25%|██▍       | 4610/18627 [2:32:15<6:20:27,  1.63s/it]
 25%|██▍       | 4611/18627 [2:32:17<6:56:02,  1.78s/it]
 25%|██▍       | 4612/18627 [2:32:18<6:15:10,  1.61s/it]
 25%|██▍       | 4613/18627 [2:32:20<5:46:20,  1.48s/it]
 25%|██▍       | 4614/18627 [2:32:22<6:30:35,  1.67s/it]
 25%|██▍       | 4615/18627 [2:32:24<7:01:18,  1.80s/it]
 25%|██▍       | 4616/18627 [2:32:26<7:22:38,  1.90s/it]
 25%|██▍       | 4617/18627 [2:32:28<7:37:09,  1.96s/it]
 25%|██▍       | 4618/18627 [2:32:30<7:48:16,  2.01s/it]
 25%|██▍       | 4619/18627 [2:32:32<7:54:41,  2.03s/it]
 25%|██▍       | 4620/18627 [2:32:34<8:00:53,  2.06s/it]
                                                        
{'loss': 1.1366, 'grad_norm': 8.865577697753906, 'learning_rate': 4.402228309827234e-06, 'epoch': 0.25}
+
 25%|██▍       | 4620/18627 [2:32:34<8:00:53,  2.06s/it]
 25%|██▍       | 4621/18627 [2:32:35<6:25:56,  1.65s/it]
 25%|██▍       | 4622/18627 [2:32:37<6:59:17,  1.80s/it]
 25%|██▍       | 4623/18627 [2:32:38<5:41:52,  1.46s/it]
 25%|██▍       | 4624/18627 [2:32:39<5:23:13,  1.38s/it]
 25%|██▍       | 4625/18627 [2:32:41<6:14:45,  1.61s/it]
 25%|██▍       | 4626/18627 [2:32:42<5:46:02,  1.48s/it]
 25%|██▍       | 4627/18627 [2:32:45<6:31:36,  1.68s/it]
 25%|██▍       | 4628/18627 [2:32:47<7:01:41,  1.81s/it]
 25%|██▍       | 4629/18627 [2:32:49<7:25:07,  1.91s/it]
 25%|██▍       | 4630/18627 [2:32:50<6:34:37,  1.69s/it]
                                                        
{'loss': 1.2946, 'grad_norm': 16.285247802734375, 'learning_rate': 4.399404821194859e-06, 'epoch': 0.25}
+
 25%|██▍       | 4630/18627 [2:32:50<6:34:37,  1.69s/it]
 25%|██▍       | 4631/18627 [2:32:52<7:04:41,  1.82s/it]
 25%|██▍       | 4632/18627 [2:32:54<7:24:35,  1.91s/it]
 25%|██▍       | 4633/18627 [2:32:55<6:34:32,  1.69s/it]
 25%|██▍       | 4634/18627 [2:32:58<7:03:28,  1.82s/it]
 25%|██▍       | 4635/18627 [2:33:00<7:23:31,  1.90s/it]
 25%|██▍       | 4636/18627 [2:33:02<7:39:25,  1.97s/it]
 25%|██▍       | 4637/18627 [2:33:04<7:49:13,  2.01s/it]
 25%|██▍       | 4638/18627 [2:33:06<7:56:59,  2.05s/it]
 25%|██▍       | 4639/18627 [2:33:08<8:02:09,  2.07s/it]
 25%|██▍       | 4640/18627 [2:33:10<8:05:27,  2.08s/it]
                                                        
{'loss': 0.9195, 'grad_norm': 9.764721870422363, 'learning_rate': 4.396575590115317e-06, 'epoch': 0.25}
+
 25%|██▍       | 4640/18627 [2:33:10<8:05:27,  2.08s/it]
 25%|██▍       | 4641/18627 [2:33:12<8:08:22,  2.10s/it]
 25%|██▍       | 4642/18627 [2:33:14<8:10:16,  2.10s/it]
 25%|██▍       | 4643/18627 [2:33:17<8:11:36,  2.11s/it]
 25%|██▍       | 4644/18627 [2:33:19<8:12:49,  2.11s/it]
 25%|██▍       | 4645/18627 [2:33:20<7:08:11,  1.84s/it]
 25%|██▍       | 4646/18627 [2:33:22<7:27:17,  1.92s/it]
 25%|██▍       | 4647/18627 [2:33:24<7:40:43,  1.98s/it]
 25%|██▍       | 4648/18627 [2:33:26<7:50:51,  2.02s/it]
 25%|█���▍       | 4649/18627 [2:33:28<7:59:22,  2.06s/it]
 25%|██▍       | 4650/18627 [2:33:31<8:04:31,  2.08s/it]
                                                        
{'loss': 0.9134, 'grad_norm': 6.334207057952881, 'learning_rate': 4.393740625142187e-06, 'epoch': 0.25}
+
 25%|██▍       | 4650/18627 [2:33:31<8:04:31,  2.08s/it]
 25%|██▍       | 4651/18627 [2:33:33<8:08:47,  2.10s/it]
 25%|██▍       | 4652/18627 [2:33:35<8:08:51,  2.10s/it]
 25%|██▍       | 4653/18627 [2:33:36<7:06:15,  1.83s/it]
 25%|██▍       | 4654/18627 [2:33:38<7:27:25,  1.92s/it]
 25%|██▍       | 4655/18627 [2:33:40<7:41:05,  1.98s/it]
 25%|██▍       | 4656/18627 [2:33:42<7:52:41,  2.03s/it]
 25%|██▌       | 4657/18627 [2:33:44<7:58:45,  2.06s/it]
 25%|██▌       | 4658/18627 [2:33:46<6:59:24,  1.80s/it]
 25%|██▌       | 4659/18627 [2:33:48<7:22:32,  1.90s/it]
 25%|██▌       | 4660/18627 [2:33:49<6:32:54,  1.69s/it]
                                                        
{'loss': 1.2896, 'grad_norm': 14.907050132751465, 'learning_rate': 4.390899934846383e-06, 'epoch': 0.25}
+
 25%|██▌       | 4660/18627 [2:33:49<6:32:54,  1.69s/it]
 25%|██▌       | 4661/18627 [2:33:51<7:02:09,  1.81s/it]
 25%|██▌       | 4662/18627 [2:33:53<7:23:46,  1.91s/it]
 25%|██▌       | 4663/18627 [2:33:55<7:39:10,  1.97s/it]
 25%|██▌       | 4664/18627 [2:33:57<7:47:57,  2.01s/it]
 25%|██▌       | 4665/18627 [2:34:00<7:53:57,  2.04s/it]
 25%|██▌       | 4666/18627 [2:34:02<7:59:11,  2.06s/it]
 25%|██▌       | 4667/18627 [2:34:03<6:57:56,  1.80s/it]
 25%|██▌       | 4668/18627 [2:34:05<7:21:56,  1.90s/it]
 25%|██▌       | 4669/18627 [2:34:07<7:36:44,  1.96s/it]
 25%|██▌       | 4670/18627 [2:34:09<7:47:04,  2.01s/it]
                                                        
{'loss': 0.9204, 'grad_norm': 5.170718669891357, 'learning_rate': 4.388053527816131e-06, 'epoch': 0.25}
+
 25%|██▌       | 4670/18627 [2:34:09<7:47:04,  2.01s/it]
 25%|██▌       | 4671/18627 [2:34:10<6:18:12,  1.63s/it]
 25%|██▌       | 4672/18627 [2:34:11<5:14:07,  1.35s/it]
 25%|██▌       | 4673/18627 [2:34:13<6:07:43,  1.58s/it]
 25%|██▌       | 4674/18627 [2:34:15<6:45:06,  1.74s/it]
 25%|██▌       | 4675/18627 [2:34:17<7:11:39,  1.86s/it]
 25%|██▌       | 4676/18627 [2:34:19<7:28:47,  1.93s/it]
 25%|██▌       | 4677/18627 [2:34:21<7:41:12,  1.98s/it]
 25%|██▌       | 4678/18627 [2:34:23<7:49:32,  2.02s/it]
 25%|██▌       | 4679/18627 [2:34:25<7:56:03,  2.05s/it]
 25%|██▌       | 4680/18627 [2:34:27<6:56:09,  1.79s/it]
                                                        
{'loss': 0.8742, 'grad_norm': 16.60308265686035, 'learning_rate': 4.3852014126569355e-06, 'epoch': 0.25}
+
 25%|██▌       | 4680/18627 [2:34:27<6:56:09,  1.79s/it]
 25%|██▌       | 4681/18627 [2:34:27<5:39:50,  1.46s/it]
 25%|██▌       | 4682/18627 [2:34:29<6:25:57,  1.66s/it]
 25%|██▌       | 4683/18627 [2:34:32<6:58:27,  1.80s/it]
 25%|██▌       | 4684/18627 [2:34:33<6:15:11,  1.61s/it]
 25%|██▌       | 4685/18627 [2:34:35<6:50:13,  1.77s/it]
 25%|██▌       | 4686/18627 [2:34:37<7:15:00,  1.87s/it]
 25%|██▌       | 4687/18627 [2:34:38<6:27:49,  1.67s/it]
 25%|██▌       | 4688/18627 [2:34:40<7:00:03,  1.81s/it]
 25%|██▌       | 4689/18627 [2:34:42<7:21:52,  1.90s/it]
 25%|██▌       | 4690/18627 [2:34:45<7:37:17,  1.97s/it]
                                                        
{'loss': 1.0942, 'grad_norm': 7.8123016357421875, 'learning_rate': 4.382343597991563e-06, 'epoch': 0.25}
+
 25%|██▌       | 4690/18627 [2:34:45<7:37:17,  1.97s/it]
 25%|██▌       | 4691/18627 [2:34:47<7:48:10,  2.02s/it]
 25%|██▌       | 4692/18627 [2:34:49<7:55:45,  2.05s/it]
 25%|██▌       | 4693/18627 [2:34:51<8:00:54,  2.07s/it]
 25%|██▌       | 4694/18627 [2:34:53<8:03:12,  2.08s/it]
 25%|██▌       | 4695/18627 [2:34:55<8:06:35,  2.10s/it]
 25%|██▌       | 4696/18627 [2:34:57<8:08:14,  2.10s/it]
 25%|██▌       | 4697/18627 [2:34:58<6:56:15,  1.79s/it]
 25%|██▌       | 4698/18627 [2:35:01<7:19:36,  1.89s/it]
 25%|██▌       | 4699/18627 [2:35:03<7:34:29,  1.96s/it]
 25%|██▌       | 4700/18627 [2:35:05<7:45:41,  2.01s/it]
                                                        
{'loss': 0.9558, 'grad_norm': 9.841978073120117, 'learning_rate': 4.379480092460009e-06, 'epoch': 0.25}
+
 25%|██▌       | 4700/18627 [2:35:05<7:45:41,  2.01s/it]
 25%|██▌       | 4701/18627 [2:35:06<6:50:47,  1.77s/it]
 25%|██▌       | 4702/18627 [2:35:08<7:15:54,  1.88s/it]
 25%|██▌       | 4703/18627 [2:35:09<6:28:00,  1.67s/it]
 25%|██▌       | 4704/18627 [2:35:11<7:00:02,  1.81s/it]
 25%|██▌       | 4705/18627 [2:35:14<7:21:34,  1.90s/it]
 25%|██▌       | 4706/18627 [2:35:15<6:32:00,  1.69s/it]
 25%|██▌       | 4707/18627 [2:35:16<5:57:43,  1.54s/it]
 25%|██▌       | 4708/18627 [2:35:17<5:33:06,  1.44s/it]
 25%|██▌       | 4709/18627 [2:35:19<6:19:46,  1.64s/it]
 25%|██▌       | 4710/18627 [2:35:21<6:54:00,  1.78s/it]
                                                        
{'loss': 1.6471, 'grad_norm': 11.607048988342285, 'learning_rate': 4.3766109047194735e-06, 'epoch': 0.25}
+
 25%|██▌       | 4710/18627 [2:35:21<6:54:00,  1.78s/it]
 25%|██▌       | 4711/18627 [2:35:23<7:16:52,  1.88s/it]
 25%|██▌       | 4712/18627 [2:35:26<7:34:21,  1.96s/it]
 25%|██▌       | 4713/18627 [2:35:28<7:43:28,  2.00s/it]
 25%|██▌       | 4714/18627 [2:35:29<6:39:51,  1.72s/it]
 25%|██▌       | 4715/18627 [2:35:31<7:07:20,  1.84s/it]
 25%|██▌       | 4716/18627 [2:35:33<7:28:15,  1.93s/it]
 25%|██▌       | 4717/18627 [2:35:35<7:43:01,  2.00s/it]
 25%|██▌       | 4718/18627 [2:35:36<6:46:42,  1.75s/it]
 25%|██▌       | 4719/18627 [2:35:39<7:14:17,  1.87s/it]
 25%|██▌       | 4720/18627 [2:35:41<7:31:28,  1.95s/it]
                                                        
{'loss': 1.075, 'grad_norm': 7.525402545928955, 'learning_rate': 4.373736043444338e-06, 'epoch': 0.25}
+
 25%|██▌       | 4720/18627 [2:35:41<7:31:28,  1.95s/it]
 25%|██▌       | 4721/18627 [2:35:43<7:43:45,  2.00s/it]
 25%|██▌       | 4722/18627 [2:35:45<7:53:04,  2.04s/it]
 25%|██▌       | 4723/18627 [2:35:46<6:53:46,  1.79s/it]
 25%|██▌       | 4724/18627 [2:35:48<7:17:18,  1.89s/it]
 25%|██▌       | 4725/18627 [2:35:50<7:41:33,  1.99s/it]
 25%|██▌       | 4726/18627 [2:35:53<7:50:40,  2.03s/it]
 25%|██▌       | 4727/18627 [2:35:55<7:56:14,  2.06s/it]
 25%|██▌       | 4728/18627 [2:35:57<7:59:36,  2.07s/it]
 25%|██▌       | 4729/18627 [2:35:59<8:02:03,  2.08s/it]
 25%|██▌       | 4730/18627 [2:36:01<8:03:44,  2.09s/it]
                                                        
{'loss': 1.066, 'grad_norm': 8.131234169006348, 'learning_rate': 4.370855517326133e-06, 'epoch': 0.25}
+
 25%|██▌       | 4730/18627 [2:36:01<8:03:44,  2.09s/it]
 25%|██▌       | 4731/18627 [2:36:03<8:06:16,  2.10s/it]
 25%|██▌       | 4732/18627 [2:36:04<7:03:18,  1.83s/it]
 25%|██▌       | 4733/18627 [2:36:06<7:24:59,  1.92s/it]
 25%|██▌       | 4734/18627 [2:36:07<6:00:48,  1.56s/it]
 25%|██▌       | 4735/18627 [2:36:09<6:38:59,  1.72s/it]
 25%|██▌       | 4736/18627 [2:36:10<5:55:12,  1.53s/it]
 25%|██▌       | 4737/18627 [2:36:13<6:37:22,  1.72s/it]
 25%|██▌       | 4738/18627 [2:36:15<7:07:29,  1.85s/it]
 25%|██▌       | 4739/18627 [2:36:16<6:22:14,  1.65s/it]
 25%|██▌       | 4740/18627 [2:36:17<5:49:59,  1.51s/it]
                                                        
{'loss': 1.4238, 'grad_norm': 16.357973098754883, 'learning_rate': 4.36796933507352e-06, 'epoch': 0.25}
+
 25%|██▌       | 4740/18627 [2:36:17<5:49:59,  1.51s/it]
 25%|██▌       | 4741/18627 [2:36:19<6:33:00,  1.70s/it]
 25%|██▌       | 4742/18627 [2:36:21<7:02:52,  1.83s/it]
 25%|██▌       | 4743/18627 [2:36:23<7:22:21,  1.91s/it]
 25%|██▌       | 4744/18627 [2:36:25<6:32:18,  1.70s/it]
 25%|██▌       | 4745/18627 [2:36:26<5:57:29,  1.55s/it]
 25%|██▌       | 4746/18627 [2:36:28<6:37:39,  1.72s/it]
 25%|██▌       | 4747/18627 [2:36:29<6:00:25,  1.56s/it]
 25%|██▌       | 4748/18627 [2:36:31<6:39:18,  1.73s/it]
 25%|██▌       | 4749/18627 [2:36:33<7:07:08,  1.85s/it]
 26%|██▌       | 4750/18627 [2:36:35<7:24:30,  1.92s/it]
                                                        
{'loss': 1.3864, 'grad_norm': 7.250372886657715, 'learning_rate': 4.365077505412256e-06, 'epoch': 0.26}
+
 26%|██▌       | 4750/18627 [2:36:35<7:24:30,  1.92s/it]
 26%|██▌       | 4751/18627 [2:36:38<7:38:48,  1.98s/it]
 26%|██▌       | 4752/18627 [2:36:40<7:47:44,  2.02s/it]
 26%|██▌       | 4753/18627 [2:36:42<7:54:14,  2.05s/it]
 26%|██▌       | 4754/18627 [2:36:44<7:59:26,  2.07s/it]
 26%|██▌       | 4755/18627 [2:36:46<8:02:32,  2.09s/it]
 26%|██▌       | 4756/18627 [2:36:48<8:05:51,  2.10s/it]
 26%|██▌       | 4757/18627 [2:36:49<7:03:14,  1.83s/it]
 26%|██▌       | 4758/18627 [2:36:51<7:21:40,  1.91s/it]
 26%|██▌       | 4759/18627 [2:36:54<7:36:55,  1.98s/it]
 26%|██▌       | 4760/18627 [2:36:55<6:42:13,  1.74s/it]
                                                        
{'loss': 1.1255, 'grad_norm': 14.458534240722656, 'learning_rate': 4.362180037085177e-06, 'epoch': 0.26}
+
 26%|██▌       | 4760/18627 [2:36:55<6:42:13,  1.74s/it]
 26%|██▌       | 4761/18627 [2:36:57<7:08:08,  1.85s/it]
 26%|██▌       | 4762/18627 [2:36:59<7:28:01,  1.94s/it]
 26%|██▌       | 4763/18627 [2:37:00<6:35:45,  1.71s/it]
 26%|██▌       | 4764/18627 [2:37:02<7:05:02,  1.84s/it]
 26%|██▌       | 4765/18627 [2:37:04<6:20:07,  1.65s/it]
 26%|██▌       | 4766/18627 [2:37:06<6:51:16,  1.78s/it]
 26%|██▌       | 4767/18627 [2:37:06<5:35:58,  1.45s/it]
 26%|██▌       | 4768/18627 [2:37:08<5:17:18,  1.37s/it]
 26%|██▌       | 4769/18627 [2:37:09<5:04:26,  1.32s/it]
 26%|██▌       | 4770/18627 [2:37:10<4:56:26,  1.28s/it]
                                                        
{'loss': 1.739, 'grad_norm': 16.62920570373535, 'learning_rate': 4.359276938852159e-06, 'epoch': 0.26}
+
 26%|██▌       | 4770/18627 [2:37:10<4:56:26,  1.28s/it]
 26%|██▌       | 4771/18627 [2:37:11<4:50:17,  1.26s/it]
 26%|██▌       | 4772/18627 [2:37:13<5:48:49,  1.51s/it]
 26%|██▌       | 4773/18627 [2:37:15<6:29:33,  1.69s/it]
 26%|██▌       | 4774/18627 [2:37:17<5:55:23,  1.54s/it]
 26%|██▌       | 4775/18627 [2:37:19<6:34:13,  1.71s/it]
 26%|██▌       | 4776/18627 [2:37:20<5:47:41,  1.51s/it]
 26%|██▌       | 4777/18627 [2:37:22<6:30:13,  1.69s/it]
 26%|██▌       | 4778/18627 [2:37:24<6:59:43,  1.82s/it]
 26%|██▌       | 4779/18627 [2:37:26<7:20:49,  1.91s/it]
 26%|██▌       | 4780/18627 [2:37:27<6:30:41,  1.69s/it]
                                                        
{'loss': 1.3442, 'grad_norm': 13.904657363891602, 'learning_rate': 4.356368219490107e-06, 'epoch': 0.26}
+
 26%|██▌       | 4780/18627 [2:37:27<6:30:41,  1.69s/it]
 26%|██▌       | 4781/18627 [2:37:28<5:56:27,  1.54s/it]
 26%|██▌       | 4782/18627 [2:37:30<5:32:25,  1.44s/it]
 26%|██▌       | 4783/18627 [2:37:31<5:14:36,  1.36s/it]
 26%|██▌       | 4784/18627 [2:37:32<5:02:52,  1.31s/it]
 26%|██▌       | 4785/18627 [2:37:34<5:59:27,  1.56s/it]
 26%|██▌       | 4786/18627 [2:37:36<6:38:37,  1.73s/it]
 26%|██▌       | 4787/18627 [2:37:38<7:05:35,  1.85s/it]
 26%|██▌       | 4788/18627 [2:37:40<7:24:17,  1.93s/it]
 26%|██▌       | 4789/18627 [2:37:43<7:37:47,  1.98s/it]
 26%|██▌       | 4790/18627 [2:37:45<7:47:30,  2.03s/it]
                                                        
{'loss': 1.5581, 'grad_norm': 6.338520050048828, 'learning_rate': 4.3534538877929135e-06, 'epoch': 0.26}
+
 26%|██▌       | 4790/18627 [2:37:45<7:47:30,  2.03s/it]
 26%|██▌       | 4791/18627 [2:37:46<6:49:57,  1.78s/it]
 26%|██▌       | 4792/18627 [2:37:48<7:14:11,  1.88s/it]
 26%|██▌       | 4793/18627 [2:37:50<7:30:36,  1.95s/it]
 26%|██▌       | 4794/18627 [2:37:52<7:41:36,  2.00s/it]
 26%|██▌       | 4795/18627 [2:37:54<7:49:20,  2.04s/it]
 26%|██▌       | 4796/18627 [2:37:57<7:56:24,  2.07s/it]
 26%|██▌       | 4797/18627 [2:37:58<6:55:24,  1.80s/it]
 26%|██▌       | 4798/18627 [2:38:00<7:16:48,  1.90s/it]
 26%|██▌       | 4799/18627 [2:38:02<7:32:03,  1.96s/it]
 26%|██▌       | 4800/18627 [2:38:04<7:41:18,  2.00s/it]
                                                        
{'loss': 1.1037, 'grad_norm': 5.327105522155762, 'learning_rate': 4.350533952571444e-06, 'epoch': 0.26}
+
 26%|██▌       | 4800/18627 [2:38:04<7:41:18,  2.00s/it]
 26%|██▌       | 4801/18627 [2:38:06<7:50:21,  2.04s/it]
 26%|██▌       | 4802/18627 [2:38:08<7:55:08,  2.06s/it]
 26%|██▌       | 4803/18627 [2:38:09<6:49:17,  1.78s/it]
 26%|██▌       | 4804/18627 [2:38:12<7:16:23,  1.89s/it]
 26%|██▌       | 4805/18627 [2:38:14<7:33:40,  1.97s/it]
 26%|██▌       | 4806/18627 [2:38:15<6:39:43,  1.74s/it]
 26%|██▌       | 4807/18627 [2:38:17<7:06:07,  1.85s/it]
 26%|██▌       | 4808/18627 [2:38:19<7:26:15,  1.94s/it]
 26%|██▌       | 4809/18627 [2:38:21<7:38:13,  1.99s/it]
 26%|██▌       | 4810/18627 [2:38:23<7:47:40,  2.03s/it]
                                                        
{'loss': 1.1269, 'grad_norm': 5.9833292961120605, 'learning_rate': 4.3476084226535e-06, 'epoch': 0.26}
+
 26%|██▌       | 4810/18627 [2:38:23<7:47:40,  2.03s/it]
 26%|██▌       | 4811/18627 [2:38:25<6:50:01,  1.78s/it]
 26%|██▌       | 4812/18627 [2:38:27<7:15:21,  1.89s/it]
 26%|██▌       | 4813/18627 [2:38:29<7:30:32,  1.96s/it]
 26%|██▌       | 4814/18627 [2:38:31<7:41:39,  2.01s/it]
 26%|██▌       | 4815/18627 [2:38:33<7:50:48,  2.05s/it]
 26%|██▌       | 4816/18627 [2:38:35<7:56:27,  2.07s/it]
 26%|██▌       | 4817/18627 [2:38:37<7:59:57,  2.09s/it]
 26%|██▌       | 4818/18627 [2:38:40<8:02:54,  2.10s/it]
 26%|██▌       | 4819/18627 [2:38:42<8:04:58,  2.11s/it]
 26%|██▌       | 4820/18627 [2:38:44<8:04:17,  2.10s/it]
                                                        
{'loss': 0.9375, 'grad_norm': 7.314708709716797, 'learning_rate': 4.344677306883802e-06, 'epoch': 0.26}
+
 26%|██▌       | 4820/18627 [2:38:44<8:04:17,  2.10s/it]
 26%|██▌       | 4821/18627 [2:38:46<8:05:54,  2.11s/it]
 26%|██▌       | 4822/18627 [2:38:48<8:05:47,  2.11s/it]
 26%|██▌       | 4823/18627 [2:38:50<8:06:36,  2.12s/it]
 26%|██▌       | 4824/18627 [2:38:51<7:02:31,  1.84s/it]
 26%|██▌       | 4825/18627 [2:38:53<7:21:19,  1.92s/it]
 26%|██▌       | 4826/18627 [2:38:55<7:33:38,  1.97s/it]
 26%|██▌       | 4827/18627 [2:38:57<6:40:22,  1.74s/it]
 26%|██▌       | 4828/18627 [2:38:59<7:05:51,  1.85s/it]
 26%|██▌       | 4829/18627 [2:39:01<7:26:28,  1.94s/it]
 26%|██▌       | 4830/18627 [2:39:03<7:39:02,  2.00s/it]
                                                        
{'loss': 1.1505, 'grad_norm': 7.332765102386475, 'learning_rate': 4.341740614123956e-06, 'epoch': 0.26}
+
 26%|██▌       | 4830/18627 [2:39:03<7:39:02,  2.00s/it]
 26%|██▌       | 4831/18627 [2:39:05<7:45:54,  2.03s/it]
 26%|██▌       | 4832/18627 [2:39:06<6:40:23,  1.74s/it]
 26%|██▌       | 4833/18627 [2:39:08<7:06:35,  1.86s/it]
 26%|██▌       | 4834/18627 [2:39:10<7:24:43,  1.93s/it]
 26%|██▌       | 4835/18627 [2:39:13<7:35:52,  1.98s/it]
 26%|██▌       | 4836/18627 [2:39:15<7:43:29,  2.02s/it]
 26%|██▌       | 4837/18627 [2:39:17<7:48:20,  2.04s/it]
 26%|██▌       | 4838/18627 [2:39:19<7:53:40,  2.06s/it]
 26%|██▌       | 4839/18627 [2:39:21<7:55:38,  2.07s/it]
 26%|██▌       | 4840/18627 [2:39:23<7:59:47,  2.09s/it]
                                                        
{'loss': 0.8946, 'grad_norm': 6.323610782623291, 'learning_rate': 4.338798353252429e-06, 'epoch': 0.26}
+
 26%|██▌       | 4840/18627 [2:39:23<7:59:47,  2.09s/it]
 26%|██▌       | 4841/18627 [2:39:24<6:57:45,  1.82s/it]
 26%|██▌       | 4842/18627 [2:39:26<7:18:33,  1.91s/it]
 26%|██▌       | 4843/18627 [2:39:29<7:33:15,  1.97s/it]
 26%|██▌       | 4844/18627 [2:39:30<6:40:09,  1.74s/it]
 26%|██▌       | 4845/18627 [2:39:32<7:06:21,  1.86s/it]
 26%|██▌       | 4846/18627 [2:39:34<7:24:34,  1.94s/it]
 26%|██▌       | 4847/18627 [2:39:35<6:32:56,  1.71s/it]
 26%|██▌       | 4848/18627 [2:39:36<5:56:57,  1.55s/it]
 26%|██▌       | 4849/18627 [2:39:39<6:37:38,  1.73s/it]
 26%|██▌       | 4850/18627 [2:39:41<7:04:25,  1.85s/it]
                                                        
{'loss': 1.5263, 'grad_norm': 7.983599662780762, 'learning_rate': 4.33585053316452e-06, 'epoch': 0.26}
+
 26%|██▌       | 4850/18627 [2:39:41<7:04:25,  1.85s/it]
 26%|██▌       | 4851/18627 [2:39:43<7:25:03,  1.94s/it]
 26%|██▌       | 4852/18627 [2:39:45<7:37:32,  1.99s/it]
 26%|██▌       | 4853/18627 [2:39:47<7:47:49,  2.04s/it]
 26%|██▌       | 4854/18627 [2:39:48<6:49:16,  1.78s/it]
 26%|██▌       | 4855/18627 [2:39:49<5:34:33,  1.46s/it]
 26%|██▌       | 4856/18627 [2:39:51<6:20:39,  1.66s/it]
 26%|██▌       | 4857/18627 [2:39:53<6:52:07,  1.80s/it]
 26%|██▌       | 4858/18627 [2:39:55<7:15:22,  1.90s/it]
 26%|██▌       | 4859/18627 [2:39:57<7:31:00,  1.97s/it]
 26%|██▌       | 4860/18627 [2:40:00<7:41:39,  2.01s/it]
                                                        
{'loss': 0.9208, 'grad_norm': 9.077059745788574, 'learning_rate': 4.33289716277234e-06, 'epoch': 0.26}
+
 26%|██▌       | 4860/18627 [2:40:00<7:41:39,  2.01s/it]
 26%|██▌       | 4861/18627 [2:40:02<7:48:45,  2.04s/it]
 26%|██▌       | 4862/18627 [2:40:03<6:49:39,  1.79s/it]
 26%|██▌       | 4863/18627 [2:40:05<7:11:57,  1.88s/it]
 26%|██▌       | 4864/18627 [2:40:07<7:28:39,  1.96s/it]
 26%|██▌       | 4865/18627 [2:40:09<7:40:00,  2.01s/it]
 26%|██▌       | 4866/18627 [2:40:11<7:46:15,  2.03s/it]
 26%|██▌       | 4867/18627 [2:40:12<6:47:50,  1.78s/it]
 26%|██▌       | 4868/18627 [2:40:15<7:12:24,  1.89s/it]
 26%|██▌       | 4869/18627 [2:40:17<7:27:09,  1.95s/it]
 26%|██▌       | 4870/18627 [2:40:19<7:39:36,  2.00s/it]
                                                        
{'loss': 1.0719, 'grad_norm': 6.8345513343811035, 'learning_rate': 4.329938251004776e-06, 'epoch': 0.26}
+
 26%|██▌       | 4870/18627 [2:40:19<7:39:36,  2.00s/it]
 26%|██▌       | 4871/18627 [2:40:21<7:49:25,  2.05s/it]
 26%|██▌       | 4872/18627 [2:40:23<7:55:34,  2.07s/it]
 26%|██▌       | 4873/18627 [2:40:25<7:59:08,  2.09s/it]
 26%|██▌       | 4874/18627 [2:40:27<8:02:30,  2.11s/it]
 26%|██▌       | 4875/18627 [2:40:30<8:04:25,  2.11s/it]
 26%|██▌       | 4876/18627 [2:40:32<8:03:47,  2.11s/it]
 26%|██▌       | 4877/18627 [2:40:33<7:00:09,  1.83s/it]
 26%|██▌       | 4878/18627 [2:40:35<7:19:31,  1.92s/it]
 26%|██▌       | 4879/18627 [2:40:36<6:21:41,  1.67s/it]
 26%|██▌       | 4880/18627 [2:40:38<6:51:51,  1.80s/it]
                                                        
{'loss': 1.0648, 'grad_norm': 5.419963359832764, 'learning_rate': 4.326973806807468e-06, 'epoch': 0.26}
+
 26%|██▌       | 4880/18627 [2:40:38<6:51:51,  1.80s/it]
 26%|██▌       | 4881/18627 [2:40:40<7:12:59,  1.89s/it]
 26%|██▌       | 4882/18627 [2:40:42<7:29:01,  1.96s/it]
 26%|██▌       | 4883/18627 [2:40:43<6:01:52,  1.58s/it]
 26%|██▌       | 4884/18627 [2:40:45<6:39:07,  1.74s/it]
 26%|██▌       | 4885/18627 [2:40:47<7:05:08,  1.86s/it]
 26%|██▌       | 4886/18627 [2:40:49<7:22:40,  1.93s/it]
 26%|██▌       | 4887/18627 [2:40:52<7:36:07,  1.99s/it]
 26%|██▌       | 4888/18627 [2:40:53<6:40:50,  1.75s/it]
 26%|██▌       | 4889/18627 [2:40:55<7:07:18,  1.87s/it]
 26%|██▋       | 4890/18627 [2:40:57<7:24:35,  1.94s/it]
                                                        
{'loss': 0.9139, 'grad_norm': 6.073435306549072, 'learning_rate': 4.3240038391427865e-06, 'epoch': 0.26}
+
 26%|██▋       | 4890/18627 [2:40:57<7:24:35,  1.94s/it]
 26%|██▋       | 4891/18627 [2:40:59<7:35:49,  1.99s/it]
 26%|██▋       | 4892/18627 [2:41:00<6:41:05,  1.75s/it]
 26%|██▋       | 4893/18627 [2:41:01<6:02:45,  1.58s/it]
 26%|██▋       | 4894/18627 [2:41:04<6:40:56,  1.75s/it]
 26%|██▋       | 4895/18627 [2:41:06<7:06:41,  1.86s/it]
 26%|██▋       | 4896/18627 [2:41:07<6:21:19,  1.67s/it]
 26%|██▋       | 4897/18627 [2:41:09<6:52:30,  1.80s/it]
 26%|██▋       | 4898/18627 [2:41:11<7:16:51,  1.91s/it]
 26%|██▋       | 4899/18627 [2:41:13<7:30:21,  1.97s/it]
 26%|██▋       | 4900/18627 [2:41:15<6:36:54,  1.73s/it]
                                                        
{'loss': 1.4703, 'grad_norm': 17.399202346801758, 'learning_rate': 4.321028356989797e-06, 'epoch': 0.26}
+
 26%|██▋       | 4900/18627 [2:41:15<6:36:54,  1.73s/it]
 26%|██▋       | 4901/18627 [2:41:16<5:59:39,  1.57s/it]
 26%|██▋       | 4902/18627 [2:41:17<5:33:21,  1.46s/it]
 26%|██▋       | 4903/18627 [2:41:19<6:17:48,  1.65s/it]
 26%|██▋       | 4904/18627 [2:41:20<5:46:02,  1.51s/it]
 26%|██▋       | 4905/18627 [2:41:22<6:26:15,  1.69s/it]
 26%|██▋       | 4906/18627 [2:41:24<6:54:46,  1.81s/it]
 26%|██▋       | 4907/18627 [2:41:26<7:14:32,  1.90s/it]
 26%|██▋       | 4908/18627 [2:41:28<6:26:37,  1.69s/it]
 26%|██▋       | 4909/18627 [2:41:30<6:55:19,  1.82s/it]
 26%|██▋       | 4910/18627 [2:41:31<6:12:01,  1.63s/it]
                                                        
{'loss': 1.6238, 'grad_norm': 14.989962577819824, 'learning_rate': 4.318047369344236e-06, 'epoch': 0.26}
+
 26%|██▋       | 4910/18627 [2:41:31<6:12:01,  1.63s/it]
 26%|██▋       | 4911/18627 [2:41:33<6:46:06,  1.78s/it]
 26%|██▋       | 4912/18627 [2:41:34<6:05:53,  1.60s/it]
 26%|██▋       | 4913/18627 [2:41:36<6:41:30,  1.76s/it]
 26%|██▋       | 4914/18627 [2:41:39<7:05:09,  1.86s/it]
 26%|██▋       | 4915/18627 [2:41:41<7:21:12,  1.93s/it]
 26%|██▋       | 4916/18627 [2:41:42<6:30:33,  1.71s/it]
 26%|██▋       | 4917/18627 [2:41:43<5:54:56,  1.55s/it]
 26%|██▋       | 4918/18627 [2:41:45<6:33:17,  1.72s/it]
 26%|██▋       | 4919/18627 [2:41:47<6:58:47,  1.83s/it]
 26%|██▋       | 4920/18627 [2:41:49<7:18:50,  1.92s/it]
                                                        
{'loss': 1.3254, 'grad_norm': 10.254854202270508, 'learning_rate': 4.3150608852184895e-06, 'epoch': 0.26}
+
 26%|██▋       | 4920/18627 [2:41:49<7:18:50,  1.92s/it]
 26%|██▋       | 4921/18627 [2:41:50<6:21:30,  1.67s/it]
 26%|██▋       | 4922/18627 [2:41:51<5:37:23,  1.48s/it]
 26%|██▋       | 4923/18627 [2:41:53<5:17:03,  1.39s/it]
 26%|██▋       | 4924/18627 [2:41:55<6:07:30,  1.61s/it]
 26%|██▋       | 4925/18627 [2:41:57<6:43:31,  1.77s/it]
 26%|██▋       | 4926/18627 [2:41:58<6:04:32,  1.60s/it]
 26%|██▋       | 4927/18627 [2:42:00<6:39:25,  1.75s/it]
 26%|██▋       | 4928/18627 [2:42:01<6:01:02,  1.58s/it]
 26%|██▋       | 4929/18627 [2:42:04<6:37:37,  1.74s/it]
 26%|██▋       | 4930/18627 [2:42:06<7:04:14,  1.86s/it]
                                                        
{'loss': 1.6861, 'grad_norm': 10.899751663208008, 'learning_rate': 4.312068913641556e-06, 'epoch': 0.26}
+
 26%|██▋       | 4930/18627 [2:42:06<7:04:14,  1.86s/it]
 26%|██▋       | 4931/18627 [2:42:08<7:22:19,  1.94s/it]
 26%|██▋       | 4932/18627 [2:42:10<7:34:53,  1.99s/it]
 26%|██▋       | 4933/18627 [2:42:12<7:42:34,  2.03s/it]
 26%|██▋       | 4934/18627 [2:42:14<7:49:24,  2.06s/it]
 26%|██▋       | 4935/18627 [2:42:16<7:53:24,  2.07s/it]
 26%|██▋       | 4936/18627 [2:42:18<7:57:16,  2.09s/it]
 27%|██▋       | 4937/18627 [2:42:20<6:54:49,  1.82s/it]
 27%|██▋       | 4938/18627 [2:42:22<7:14:41,  1.91s/it]
 27%|██▋       | 4939/18627 [2:42:24<7:28:26,  1.97s/it]
 27%|██▋       | 4940/18627 [2:42:27<9:00:28,  2.37s/it]
                                                        
{'loss': 0.8937, 'grad_norm': 8.73538875579834, 'learning_rate': 4.309071463659028e-06, 'epoch': 0.27}
+
 27%|██▋       | 4940/18627 [2:42:27<9:00:28,  2.37s/it]
 27%|██▋       | 4941/18627 [2:42:29<8:43:08,  2.29s/it]
 27%|██▋       | 4942/18627 [2:42:30<7:28:47,  1.97s/it]
 27%|██▋       | 4943/18627 [2:42:33<7:39:23,  2.01s/it]
 27%|██▋       | 4944/18627 [2:42:35<7:47:42,  2.05s/it]
 27%|██▋       | 4945/18627 [2:42:37<7:53:38,  2.08s/it]
 27%|██▋       | 4946/18627 [2:42:39<7:56:36,  2.09s/it]
 27%|██▋       | 4947/18627 [2:42:40<6:54:38,  1.82s/it]
 27%|██▋       | 4948/18627 [2:42:41<6:01:07,  1.58s/it]
 27%|██▋       | 4949/18627 [2:42:42<5:33:51,  1.46s/it]
 27%|██▋       | 4950/18627 [2:42:44<6:18:16,  1.66s/it]
                                                        
{'loss': 1.3974, 'grad_norm': 5.894237518310547, 'learning_rate': 4.306068544333057e-06, 'epoch': 0.27}
+
 27%|██▋       | 4950/18627 [2:42:44<6:18:16,  1.66s/it]
 27%|██▋       | 4951/18627 [2:42:47<6:49:31,  1.80s/it]
 27%|██▋       | 4952/18627 [2:42:49<7:11:28,  1.89s/it]
 27%|██▋       | 4953/18627 [2:42:51<7:27:27,  1.96s/it]
 27%|██▋       | 4954/18627 [2:42:53<7:39:00,  2.01s/it]
 27%|██▋       | 4955/18627 [2:42:55<7:45:45,  2.04s/it]
 27%|██▋       | 4956/18627 [2:42:57<7:52:13,  2.07s/it]
 27%|██▋       | 4957/18627 [2:42:58<6:52:01,  1.81s/it]
 27%|██▋       | 4958/18627 [2:43:00<7:13:13,  1.90s/it]
 27%|██▋       | 4959/18627 [2:43:03<7:29:43,  1.97s/it]
 27%|██▋       | 4960/18627 [2:43:05<7:38:47,  2.01s/it]
                                                        
{'loss': 0.9099, 'grad_norm': 8.414528846740723, 'learning_rate': 4.303060164742334e-06, 'epoch': 0.27}
+
 27%|██▋       | 4960/18627 [2:43:05<7:38:47,  2.01s/it]
 27%|██▋       | 4961/18627 [2:43:07<7:46:37,  2.05s/it]
 27%|██▋       | 4962/18627 [2:43:08<6:37:46,  1.75s/it]
 27%|██▋       | 4963/18627 [2:43:10<7:03:49,  1.86s/it]
 27%|██▋       | 4964/18627 [2:43:12<7:21:18,  1.94s/it]
 27%|██▋       | 4965/18627 [2:43:14<7:35:14,  2.00s/it]
 27%|██▋       | 4966/18627 [2:43:16<7:43:33,  2.04s/it]
 27%|██▋       | 4967/18627 [2:43:19<7:50:23,  2.07s/it]
 27%|██▋       | 4968/18627 [2:43:20<6:50:27,  1.80s/it]
 27%|██▋       | 4969/18627 [2:43:21<6:08:39,  1.62s/it]
 27%|██▋       | 4970/18627 [2:43:23<6:43:29,  1.77s/it]
                                                        
{'loss': 1.3039, 'grad_norm': 7.305602073669434, 'learning_rate': 4.300046333982056e-06, 'epoch': 0.27}
+
 27%|██▋       | 4970/18627 [2:43:23<6:43:29,  1.77s/it]
 27%|██▋       | 4971/18627 [2:43:25<7:08:49,  1.88s/it]
 27%|██▋       | 4972/18627 [2:43:27<7:26:23,  1.96s/it]
 27%|██▋       | 4973/18627 [2:43:29<7:37:35,  2.01s/it]
 27%|██▋       | 4974/18627 [2:43:32<7:44:27,  2.04s/it]
 27%|██▋       | 4975/18627 [2:43:34<7:49:55,  2.07s/it]
 27%|██▋       | 4976/18627 [2:43:36<8:13:41,  2.17s/it]
 27%|██▋       | 4977/18627 [2:43:38<8:10:07,  2.15s/it]
 27%|██▋       | 4978/18627 [2:43:40<8:07:13,  2.14s/it]
 27%|██▋       | 4979/18627 [2:43:42<8:05:44,  2.14s/it]
 27%|██▋       | 4980/18627 [2:43:44<6:54:04,  1.82s/it]
                                                        
{'loss': 1.0622, 'grad_norm': 14.907671928405762, 'learning_rate': 4.297027061163898e-06, 'epoch': 0.27}
+
 27%|██▋       | 4980/18627 [2:43:44<6:54:04,  1.82s/it]
 27%|██▋       | 4981/18627 [2:43:44<5:37:11,  1.48s/it]
 27%|██▋       | 4982/18627 [2:43:46<6:20:01,  1.67s/it]
 27%|██▋       | 4983/18627 [2:43:48<6:50:49,  1.81s/it]
 27%|██▋       | 4984/18627 [2:43:51<7:11:10,  1.90s/it]
 27%|██▋       | 4985/18627 [2:43:52<6:22:30,  1.68s/it]
 27%|██▋       | 4986/18627 [2:43:53<5:49:01,  1.54s/it]
 27%|██▋       | 4987/18627 [2:43:55<6:30:00,  1.72s/it]
 27%|██▋       | 4988/18627 [2:43:57<6:58:23,  1.84s/it]
 27%|██▋       | 4989/18627 [2:43:59<7:17:33,  1.93s/it]
 27%|██▋       | 4990/18627 [2:44:02<7:32:18,  1.99s/it]
                                                        
{'loss': 1.0926, 'grad_norm': 6.324799060821533, 'learning_rate': 4.294002355415992e-06, 'epoch': 0.27}
+
 27%|██▋       | 4990/18627 [2:44:02<7:32:18,  1.99s/it]
 27%|██▋       | 4991/18627 [2:44:04<7:43:02,  2.04s/it]
 27%|██▋       | 4992/18627 [2:44:05<6:45:20,  1.78s/it]
 27%|██▋       | 4993/18627 [2:44:06<6:04:46,  1.61s/it]
 27%|██▋       | 4994/18627 [2:44:08<6:39:50,  1.76s/it]
 27%|██▋       | 4995/18627 [2:44:10<7:04:36,  1.87s/it]
 27%|██▋       | 4996/18627 [2:44:11<6:18:19,  1.67s/it]
 27%|██▋       | 4997/18627 [2:44:14<6:50:05,  1.81s/it]
 27%|██▋       | 4998/18627 [2:44:15<6:07:46,  1.62s/it]
 27%|██▋       | 4999/18627 [2:44:17<6:41:49,  1.77s/it]
 27%|██▋       | 5000/18627 [2:44:19<7:05:38,  1.87s/it]
                                                        
{'loss': 1.4569, 'grad_norm': 8.188894271850586, 'learning_rate': 4.290972225882894e-06, 'epoch': 0.27}
+
 27%|██▋       | 5000/18627 [2:44:19<7:05:38,  1.87s/it]/home/zhengduo/workspace/LLaMA-Factory/src/qwen_vl/model/modeling_qwen2_5_vl.py:1979: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
+  with torch.cuda.amp.autocast(dtype=dtype):
+/home/zhengduo/miniconda3/envs/vgllm/lib/python3.10/site-packages/torch/utils/checkpoint.py:87: UserWarning: None of the inputs have requires_grad=True. Gradients will be None
+  warnings.warn(
+
 27%|██▋       | 5001/18627 [2:45:00<51:21:07, 13.57s/it]
 27%|██▋       | 5002/18627 [2:45:02<38:22:14, 10.14s/it]
 27%|██▋       | 5003/18627 [2:45:04<29:14:27,  7.73s/it]
 27%|██▋       | 5004/18627 [2:45:06<22:51:38,  6.04s/it]
 27%|██▋       | 5005/18627 [2:45:07<17:20:46,  4.58s/it]
 27%|██▋       | 5006/18627 [2:45:08<13:19:39,  3.52s/it]
 27%|██▋       | 5007/18627 [2:45:11<11:44:00,  3.10s/it]
 27%|██▋       | 5008/18627 [2:45:13<10:36:57,  2.81s/it]
 27%|██▋       | 5009/18627 [2:45:15<9:50:51,  2.60s/it] 
 27%|██▋       | 5010/18627 [2:45:17<9:17:56,  2.46s/it]
                                                        
{'loss': 1.1138, 'grad_norm': 7.2968573570251465, 'learning_rate': 4.287936681725556e-06, 'epoch': 0.27}
+
 27%|██▋       | 5010/18627 [2:45:17<9:17:56,  2.46s/it]
 27%|██▋       | 5011/18627 [2:45:19<8:55:21,  2.36s/it]
 27%|██▋       | 5012/18627 [2:45:21<8:39:40,  2.29s/it]
 27%|██▋       | 5013/18627 [2:45:23<8:27:02,  2.23s/it]
 27%|██▋       | 5014/18627 [2:45:25<8:19:03,  2.20s/it]
 27%|██▋       | 5015/18627 [2:45:28<8:14:20,  2.18s/it]
 27%|██▋       | 5016/18627 [2:45:31<9:59:02,  2.64s/it]
 27%|██▋       | 5017/18627 [2:45:33<9:25:26,  2.49s/it]
 27%|██▋       | 5018/18627 [2:45:36<9:00:34,  2.38s/it]
 27%|██▋       | 5019/18627 [2:45:37<7:39:16,  2.03s/it]
 27%|██▋       | 5020/18627 [2:45:40<9:06:21,  2.41s/it]
                                                        
{'loss': 0.8433, 'grad_norm': 7.2989726066589355, 'learning_rate': 4.284895732121302e-06, 'epoch': 0.27}
+
 27%|██▋       | 5020/18627 [2:45:40<9:06:21,  2.41s/it]
 27%|██▋       | 5021/18627 [2:45:41<7:43:44,  2.05s/it]
 27%|██▋       | 5022/18627 [2:45:43<7:48:42,  2.07s/it]
 27%|██▋       | 5023/18627 [2:45:46<7:54:45,  2.09s/it]
 27%|██▋       | 5024/18627 [2:45:48<7:58:45,  2.11s/it]
 27%|██▋       | 5025/18627 [2:45:49<6:55:50,  1.83s/it]
 27%|██▋       | 5026/18627 [2:45:51<7:14:16,  1.92s/it]
 27%|██▋       | 5027/18627 [2:45:53<7:28:30,  1.98s/it]
 27%|██▋       | 5028/18627 [2:45:55<7:37:48,  2.02s/it]
 27%|██▋       | 5029/18627 [2:45:57<7:44:49,  2.05s/it]
 27%|██▋       | 5030/18627 [2:45:59<7:48:55,  2.07s/it]
                                                        
{'loss': 1.0306, 'grad_norm': 7.334841728210449, 'learning_rate': 4.281849386263797e-06, 'epoch': 0.27}
+
 27%|██▋       | 5030/18627 [2:45:59<7:48:55,  2.07s/it]
 27%|██▋       | 5031/18627 [2:46:02<7:54:35,  2.09s/it]
 27%|██▋       | 5032/18627 [2:46:04<7:56:28,  2.10s/it]
 27%|██▋       | 5033/18627 [2:46:06<7:57:41,  2.11s/it]
 27%|██▋       | 5034/18627 [2:46:07<6:55:11,  1.83s/it]
 27%|██▋       | 5035/18627 [2:46:09<7:15:01,  1.92s/it]
 27%|██▋       | 5036/18627 [2:46:11<7:27:24,  1.98s/it]
 27%|██▋       | 5037/18627 [2:46:13<7:39:05,  2.03s/it]
 27%|██▋       | 5038/18627 [2:46:16<7:45:13,  2.05s/it]
 27%|██▋       | 5039/18627 [2:46:17<6:46:18,  1.79s/it]
 27%|██▋       | 5040/18627 [2:46:20<8:29:24,  2.25s/it]
                                                        
{'loss': 1.1629, 'grad_norm': 6.495748996734619, 'learning_rate': 4.278797653363021e-06, 'epoch': 0.27}
+
 27%|██▋       | 5040/18627 [2:46:20<8:29:24,  2.25s/it]
 27%|██▋       | 5041/18627 [2:46:21<7:17:35,  1.93s/it]
 27%|██▋       | 5042/18627 [2:46:23<7:31:36,  1.99s/it]
 27%|██▋       | 5043/18627 [2:46:25<7:40:23,  2.03s/it]
 27%|██▋       | 5044/18627 [2:46:28<7:44:35,  2.05s/it]
 27%|██▋       | 5045/18627 [2:46:29<6:45:24,  1.79s/it]
 27%|██▋       | 5046/18627 [2:46:31<7:08:32,  1.89s/it]
 27%|██▋       | 5047/18627 [2:46:33<7:24:19,  1.96s/it]
 27%|██▋       | 5048/18627 [2:46:35<7:33:26,  2.00s/it]
 27%|██▋       | 5049/18627 [2:46:36<6:33:59,  1.74s/it]
 27%|██▋       | 5050/18627 [2:46:38<6:58:56,  1.85s/it]
                                                        
{'loss': 1.2897, 'grad_norm': 8.87549114227295, 'learning_rate': 4.27574054264524e-06, 'epoch': 0.27}
+
 27%|██▋       | 5050/18627 [2:46:38<6:58:56,  1.85s/it]
 27%|██▋       | 5051/18627 [2:46:40<7:17:46,  1.93s/it]
 27%|██▋       | 5052/18627 [2:46:43<7:28:40,  1.98s/it]
 27%|██▋       | 5053/18627 [2:46:45<7:38:12,  2.03s/it]
 27%|██▋       | 5054/18627 [2:46:47<7:43:40,  2.05s/it]
 27%|██▋       | 5055/18627 [2:46:48<6:45:02,  1.79s/it]
 27%|██▋       | 5056/18627 [2:46:50<7:06:41,  1.89s/it]
 27%|██▋       | 5057/18627 [2:46:52<7:21:41,  1.95s/it]
 27%|██▋       | 5058/18627 [2:46:53<6:29:58,  1.72s/it]
 27%|██▋       | 5059/18627 [2:46:55<6:55:57,  1.84s/it]
 27%|██▋       | 5060/18627 [2:46:57<6:11:38,  1.64s/it]
                                                        
{'loss': 1.332, 'grad_norm': 16.03841781616211, 'learning_rate': 4.272678063352981e-06, 'epoch': 0.27}
+
 27%|██▋       | 5060/18627 [2:46:57<6:11:38,  1.64s/it]
 27%|██▋       | 5061/18627 [2:46:59<6:45:17,  1.79s/it]
 27%|██▋       | 5062/18627 [2:47:01<7:07:36,  1.89s/it]
 27%|██▋       | 5063/18627 [2:47:03<7:24:22,  1.97s/it]
 27%|██▋       | 5064/18627 [2:47:05<7:34:47,  2.01s/it]
 27%|██▋       | 5065/18627 [2:47:07<7:41:29,  2.04s/it]
 27%|██▋       | 5066/18627 [2:47:09<7:46:05,  2.06s/it]
 27%|██▋       | 5067/18627 [2:47:12<7:51:11,  2.08s/it]
 27%|██▋       | 5068/18627 [2:47:14<7:54:26,  2.10s/it]
 27%|██▋       | 5069/18627 [2:47:15<6:52:18,  1.82s/it]
 27%|██▋       | 5070/18627 [2:47:17<7:11:39,  1.91s/it]
                                                        
{'loss': 0.8517, 'grad_norm': 5.857301235198975, 'learning_rate': 4.269610224744996e-06, 'epoch': 0.27}
+
 27%|██▋       | 5070/18627 [2:47:17<7:11:39,  1.91s/it]
 27%|██▋       | 5071/18627 [2:47:19<7:25:35,  1.97s/it]
 27%|██▋       | 5072/18627 [2:47:21<7:36:07,  2.02s/it]
 27%|██▋       | 5073/18627 [2:47:23<7:41:26,  2.04s/it]
 27%|██▋       | 5074/18627 [2:47:25<7:46:12,  2.06s/it]
 27%|██▋       | 5075/18627 [2:47:28<7:50:53,  2.08s/it]
 27%|██▋       | 5076/18627 [2:47:30<7:51:35,  2.09s/it]
 27%|██▋       | 5077/18627 [2:47:30<6:17:58,  1.67s/it]
 27%|██▋       | 5078/18627 [2:47:33<6:49:07,  1.81s/it]
 27%|██▋       | 5079/18627 [2:47:34<6:08:01,  1.63s/it]
 27%|██▋       | 5080/18627 [2:47:36<6:39:39,  1.77s/it]
                                                        
{'loss': 0.9129, 'grad_norm': 7.983285427093506, 'learning_rate': 4.266537036096249e-06, 'epoch': 0.27}
+
 27%|██▋       | 5080/18627 [2:47:36<6:39:39,  1.77s/it]
 27%|██▋       | 5081/18627 [2:47:37<6:00:12,  1.60s/it]
 27%|██▋       | 5082/18627 [2:47:39<6:36:15,  1.76s/it]
 27%|██▋       | 5083/18627 [2:47:41<7:00:41,  1.86s/it]
 27%|██▋       | 5084/18627 [2:47:43<7:18:35,  1.94s/it]
 27%|██▋       | 5085/18627 [2:47:46<7:30:49,  2.00s/it]
 27%|██▋       | 5086/18627 [2:47:48<7:38:45,  2.03s/it]
 27%|██▋       | 5087/18627 [2:47:50<7:45:02,  2.06s/it]
 27%|██▋       | 5088/18627 [2:47:52<7:49:51,  2.08s/it]
 27%|██▋       | 5089/18627 [2:47:53<6:49:16,  1.81s/it]
 27%|██▋       | 5090/18627 [2:47:55<7:10:31,  1.91s/it]
                                                        
{'loss': 1.0733, 'grad_norm': 6.68778133392334, 'learning_rate': 4.263458506697869e-06, 'epoch': 0.27}
+
 27%|██▋       | 5090/18627 [2:47:55<7:10:31,  1.91s/it]
 27%|██▋       | 5091/18627 [2:47:57<7:24:49,  1.97s/it]
 27%|██▋       | 5092/18627 [2:47:59<7:32:58,  2.01s/it]
 27%|██▋       | 5093/18627 [2:48:01<6:37:28,  1.76s/it]
 27%|██▋       | 5094/18627 [2:48:03<7:03:42,  1.88s/it]
 27%|██▋       | 5095/18627 [2:48:05<7:20:47,  1.95s/it]
 27%|██▋       | 5096/18627 [2:48:07<7:30:32,  2.00s/it]
 27%|██▋       | 5097/18627 [2:48:08<6:36:43,  1.76s/it]
 27%|██▋       | 5098/18627 [2:48:10<7:02:03,  1.87s/it]
 27%|██▋       | 5099/18627 [2:48:12<7:20:52,  1.96s/it]
 27%|██▋       | 5100/18627 [2:48:15<7:32:02,  2.01s/it]
                                                        
{'loss': 1.0463, 'grad_norm': 7.71735143661499, 'learning_rate': 4.260374645857137e-06, 'epoch': 0.27}
+
 27%|██▋       | 5100/18627 [2:48:15<7:32:02,  2.01s/it]
 27%|██▋       | 5101/18627 [2:48:17<7:39:04,  2.04s/it]
 27%|██▋       | 5102/18627 [2:48:19<7:45:19,  2.06s/it]
 27%|██▋       | 5103/18627 [2:48:21<7:48:17,  2.08s/it]
 27%|██▋       | 5104/18627 [2:48:23<7:50:40,  2.09s/it]
 27%|██▋       | 5105/18627 [2:48:25<7:51:59,  2.09s/it]
 27%|██▋       | 5106/18627 [2:48:27<7:52:15,  2.10s/it]
 27%|██▋       | 5107/18627 [2:48:29<7:52:51,  2.10s/it]
 27%|██▋       | 5108/18627 [2:48:31<6:51:10,  1.82s/it]
 27%|██▋       | 5109/18627 [2:48:33<7:11:11,  1.91s/it]
 27%|██▋       | 5110/18627 [2:48:35<7:24:43,  1.97s/it]
                                                        
{'loss': 0.9092, 'grad_norm': 8.701825141906738, 'learning_rate': 4.2572854628974525e-06, 'epoch': 0.27}
+
 27%|██▋       | 5110/18627 [2:48:35<7:24:43,  1.97s/it]
 27%|██▋       | 5111/18627 [2:48:36<6:32:09,  1.74s/it]
 27%|██▋       | 5112/18627 [2:48:38<6:56:51,  1.85s/it]
 27%|██▋       | 5113/18627 [2:48:40<7:16:13,  1.94s/it]
 27%|██▋       | 5114/18627 [2:48:42<7:28:53,  1.99s/it]
 27%|██▋       | 5115/18627 [2:48:44<7:37:47,  2.03s/it]
 27%|██▋       | 5116/18627 [2:48:47<7:42:49,  2.06s/it]
 27%|██▋       | 5117/18627 [2:48:49<7:46:17,  2.07s/it]
 27%|██▋       | 5118/18627 [2:48:51<7:49:42,  2.09s/it]
 27%|██▋       | 5119/18627 [2:48:53<7:51:51,  2.10s/it]
 27%|██▋       | 5120/18627 [2:48:55<7:54:10,  2.11s/it]
                                                        
{'loss': 0.8516, 'grad_norm': 6.744924068450928, 'learning_rate': 4.2541909671583035e-06, 'epoch': 0.27}
+
 27%|██▋       | 5120/18627 [2:48:55<7:54:10,  2.11s/it]
 27%|██▋       | 5121/18627 [2:48:57<7:54:42,  2.11s/it]
 27%|██▋       | 5122/18627 [2:48:59<7:55:51,  2.11s/it]
 28%|██▊       | 5123/18627 [2:49:01<7:55:27,  2.11s/it]
 28%|██▊       | 5124/18627 [2:49:04<7:54:36,  2.11s/it]
 28%|██▊       | 5125/18627 [2:49:06<7:56:07,  2.12s/it]
 28%|██▊       | 5126/18627 [2:49:08<7:56:50,  2.12s/it]
 28%|██▊       | 5127/18627 [2:49:10<7:56:34,  2.12s/it]
 28%|██▊       | 5128/18627 [2:49:12<7:55:09,  2.11s/it]
 28%|██▊       | 5129/18627 [2:49:14<7:54:34,  2.11s/it]
 28%|██▊       | 5130/18627 [2:49:16<7:56:13,  2.12s/it]
                                                        
{'loss': 0.6841, 'grad_norm': 7.448139667510986, 'learning_rate': 4.2510911679952405e-06, 'epoch': 0.28}
+
 28%|██▊       | 5130/18627 [2:49:16<7:56:13,  2.12s/it]
 28%|██▊       | 5131/18627 [2:49:18<7:55:45,  2.12s/it]
 28%|██▊       | 5132/18627 [2:49:20<7:56:38,  2.12s/it]
 28%|██▊       | 5133/18627 [2:49:23<7:56:44,  2.12s/it]
 28%|██▊       | 5134/18627 [2:49:26<9:12:28,  2.46s/it]
 28%|██▊       | 5135/18627 [2:49:28<8:49:06,  2.35s/it]
 28%|██▊       | 5136/18627 [2:49:29<7:30:34,  2.00s/it]
 28%|██▊       | 5137/18627 [2:49:30<6:36:18,  1.76s/it]
 28%|██▊       | 5138/18627 [2:49:32<6:59:46,  1.87s/it]
 28%|██▊       | 5139/18627 [2:49:34<6:13:43,  1.66s/it]
 28%|██▊       | 5140/18627 [2:49:36<6:42:42,  1.79s/it]
                                                        
{'loss': 1.2909, 'grad_norm': 5.38520622253418, 'learning_rate': 4.247986074779849e-06, 'epoch': 0.28}
+
 28%|██▊       | 5140/18627 [2:49:36<6:42:42,  1.79s/it]
 28%|██▊       | 5141/18627 [2:49:37<6:01:55,  1.61s/it]
 28%|██▊       | 5142/18627 [2:49:39<6:35:31,  1.76s/it]
 28%|██▊       | 5143/18627 [2:49:41<6:59:06,  1.86s/it]
 28%|██▊       | 5144/18627 [2:49:43<7:15:44,  1.94s/it]
 28%|██▊       | 5145/18627 [2:49:44<6:25:10,  1.71s/it]
 28%|██▊       | 5146/18627 [2:49:45<5:37:36,  1.50s/it]
 28%|██▊       | 5147/18627 [2:49:47<5:16:52,  1.41s/it]
 28%|██▊       | 5148/18627 [2:49:47<4:28:52,  1.20s/it]
 28%|██▊       | 5149/18627 [2:49:49<5:29:09,  1.47s/it]
 28%|██▊       | 5150/18627 [2:49:52<6:11:56,  1.66s/it]
                                                        
{'loss': 1.5932, 'grad_norm': 7.976722240447998, 'learning_rate': 4.244875696899718e-06, 'epoch': 0.28}
+
 28%|██▊       | 5150/18627 [2:49:52<6:11:56,  1.66s/it]
 28%|██▊       | 5151/18627 [2:49:53<5:40:07,  1.51s/it]
 28%|██▊       | 5152/18627 [2:49:54<5:18:05,  1.42s/it]
 28%|██▊       | 5153/18627 [2:49:56<6:05:33,  1.63s/it]
 28%|██▊       | 5154/18627 [2:49:57<5:03:35,  1.35s/it]
 28%|██▊       | 5155/18627 [2:49:59<5:55:32,  1.58s/it]
 28%|██▊       | 5156/18627 [2:50:00<4:56:55,  1.32s/it]
 28%|██▊       | 5157/18627 [2:50:02<5:50:22,  1.56s/it]
 28%|██▊       | 5158/18627 [2:50:04<6:30:24,  1.74s/it]
 28%|██▊       | 5159/18627 [2:50:06<6:56:18,  1.85s/it]
 28%|██▊       | 5160/18627 [2:50:07<6:12:08,  1.66s/it]
                                                        
{'loss': 1.2443, 'grad_norm': 14.545150756835938, 'learning_rate': 4.241760043758415e-06, 'epoch': 0.28}
+
 28%|██▊       | 5160/18627 [2:50:07<6:12:08,  1.66s/it]
 28%|██▊       | 5161/18627 [2:50:09<6:43:30,  1.80s/it]
 28%|██▊       | 5162/18627 [2:50:10<6:02:53,  1.62s/it]
 28%|██▊       | 5163/18627 [2:50:13<6:36:27,  1.77s/it]
 28%|██▊       | 5164/18627 [2:50:15<7:00:03,  1.87s/it]
 28%|██▊       | 5165/18627 [2:50:17<7:16:19,  1.94s/it]
 28%|██▊       | 5166/18627 [2:50:18<6:25:10,  1.72s/it]
 28%|██▊       | 5167/18627 [2:50:20<6:52:03,  1.84s/it]
 28%|██▊       | 5168/18627 [2:50:22<7:10:36,  1.92s/it]
 28%|██▊       | 5169/18627 [2:50:24<7:23:20,  1.98s/it]
 28%|██▊       | 5170/18627 [2:50:26<7:33:21,  2.02s/it]
                                                        
{'loss': 0.967, 'grad_norm': 6.2145676612854, 'learning_rate': 4.238639124775456e-06, 'epoch': 0.28}
+
 28%|██▊       | 5170/18627 [2:50:26<7:33:21,  2.02s/it]
 28%|██▊       | 5171/18627 [2:50:29<7:40:47,  2.05s/it]
 28%|██▊       | 5172/18627 [2:50:29<6:10:01,  1.65s/it]
 28%|██▊       | 5173/18627 [2:50:31<6:42:09,  1.79s/it]
 28%|██▊       | 5174/18627 [2:50:34<7:04:50,  1.89s/it]
 28%|██▊       | 5175/18627 [2:50:36<7:17:53,  1.95s/it]
 28%|██▊       | 5176/18627 [2:50:37<6:26:41,  1.72s/it]
 28%|██▊       | 5177/18627 [2:50:39<6:52:40,  1.84s/it]
 28%|██▊       | 5178/18627 [2:50:41<7:09:40,  1.92s/it]
 28%|██▊       | 5179/18627 [2:50:42<6:20:26,  1.70s/it]
 28%|██▊       | 5180/18627 [2:50:43<5:46:48,  1.55s/it]
                                                        
{'loss': 1.2965, 'grad_norm': 16.64395523071289, 'learning_rate': 4.2355129493862765e-06, 'epoch': 0.28}
+
 28%|██▊       | 5180/18627 [2:50:43<5:46:48,  1.55s/it]
 28%|██▊       | 5181/18627 [2:50:46<6:25:58,  1.72s/it]
 28%|██▊       | 5182/18627 [2:50:46<5:18:12,  1.42s/it]
 28%|██▊       | 5183/18627 [2:50:48<6:04:51,  1.63s/it]
 28%|██▊       | 5184/18627 [2:50:51<6:39:13,  1.78s/it]
 28%|██▊       | 5185/18627 [2:50:52<5:58:53,  1.60s/it]
 28%|██▊       | 5186/18627 [2:50:54<6:34:18,  1.76s/it]
 28%|██▊       | 5187/18627 [2:50:56<6:58:29,  1.87s/it]
 28%|██▊       | 5188/18627 [2:50:58<7:14:04,  1.94s/it]
 28%|██▊       | 5189/18627 [2:51:00<7:24:36,  1.99s/it]
 28%|██▊       | 5190/18627 [2:51:02<7:33:41,  2.03s/it]
                                                        
{'loss': 0.8345, 'grad_norm': 5.606618404388428, 'learning_rate': 4.232381527042203e-06, 'epoch': 0.28}
+
 28%|██▊       | 5190/18627 [2:51:02<7:33:41,  2.03s/it]
 28%|██▊       | 5191/18627 [2:51:04<7:39:39,  2.05s/it]
 28%|██▊       | 5192/18627 [2:51:07<7:43:31,  2.07s/it]
 28%|██▊       | 5193/18627 [2:51:09<7:46:21,  2.08s/it]
 28%|██▊       | 5194/18627 [2:51:11<7:47:49,  2.09s/it]
 28%|██▊       | 5195/18627 [2:51:13<7:49:20,  2.10s/it]
 28%|██▊       | 5196/18627 [2:51:14<6:48:06,  1.82s/it]
 28%|██▊       | 5197/18627 [2:51:15<6:05:42,  1.63s/it]
 28%|██▊       | 5198/18627 [2:51:16<5:35:53,  1.50s/it]
 28%|██▊       | 5199/18627 [2:51:19<6:18:54,  1.69s/it]
 28%|██▊       | 5200/18627 [2:51:21<6:46:10,  1.82s/it]
                                                        
{'loss': 1.3318, 'grad_norm': 9.887587547302246, 'learning_rate': 4.2292448672104296e-06, 'epoch': 0.28}
+
 28%|██▊       | 5200/18627 [2:51:21<6:46:10,  1.82s/it]
 28%|██▊       | 5201/18627 [2:51:23<7:05:23,  1.90s/it]
 28%|██▊       | 5202/18627 [2:51:24<6:17:15,  1.69s/it]
 28%|██▊       | 5203/18627 [2:51:25<5:44:09,  1.54s/it]
 28%|██▊       | 5204/18627 [2:51:27<6:23:50,  1.72s/it]
 28%|██▊       | 5205/18627 [2:51:29<6:50:35,  1.84s/it]
 28%|██▊       | 5206/18627 [2:51:31<7:08:41,  1.92s/it]
 28%|██▊       | 5207/18627 [2:51:34<7:22:16,  1.98s/it]
 28%|██▊       | 5208/18627 [2:51:36<7:31:03,  2.02s/it]
 28%|██▊       | 5209/18627 [2:51:38<7:38:32,  2.05s/it]
 28%|██▊       | 5210/18627 [2:51:40<7:42:31,  2.07s/it]
                                                        
{'loss': 1.1236, 'grad_norm': 7.770495891571045, 'learning_rate': 4.226102979373977e-06, 'epoch': 0.28}
+
 28%|██▊       | 5210/18627 [2:51:40<7:42:31,  2.07s/it]
 28%|██▊       | 5211/18627 [2:51:42<7:46:35,  2.09s/it]
 28%|██▊       | 5212/18627 [2:51:44<7:49:01,  2.10s/it]
 28%|██▊       | 5213/18627 [2:51:45<6:47:38,  1.82s/it]
 28%|██▊       | 5214/18627 [2:51:47<7:06:31,  1.91s/it]
 28%|██▊       | 5215/18627 [2:51:49<6:17:58,  1.69s/it]
 28%|██▊       | 5216/18627 [2:51:51<6:47:01,  1.82s/it]
 28%|██▊       | 5217/18627 [2:51:53<7:06:02,  1.91s/it]
 28%|██▊       | 5218/18627 [2:51:54<6:17:46,  1.69s/it]
 28%|██▊       | 5219/18627 [2:51:56<6:47:35,  1.82s/it]
 28%|██▊       | 5220/18627 [2:51:57<6:04:39,  1.63s/it]
                                                        
{'loss': 1.4303, 'grad_norm': 14.979072570800781, 'learning_rate': 4.222955873031678e-06, 'epoch': 0.28}
+
 28%|██▊       | 5220/18627 [2:51:57<6:04:39,  1.63s/it]
 28%|██▊       | 5221/18627 [2:51:59<5:35:44,  1.50s/it]
 28%|██▊       | 5222/18627 [2:52:00<5:15:02,  1.41s/it]
 28%|██▊       | 5223/18627 [2:52:01<4:54:59,  1.32s/it]
 28%|██▊       | 5224/18627 [2:52:03<5:49:19,  1.56s/it]
 28%|██▊       | 5225/18627 [2:52:05<6:26:10,  1.73s/it]
 28%|██▊       | 5226/18627 [2:52:07<6:52:26,  1.85s/it]
 28%|██▊       | 5227/18627 [2:52:09<7:11:40,  1.93s/it]
 28%|██▊       | 5228/18627 [2:52:12<7:23:28,  1.99s/it]
 28%|██▊       | 5229/18627 [2:52:14<7:31:14,  2.02s/it]
 28%|██▊       | 5230/18627 [2:52:16<7:37:25,  2.05s/it]
                                                        
{'loss': 1.2761, 'grad_norm': 5.548940181732178, 'learning_rate': 4.21980355769814e-06, 'epoch': 0.28}
+
 28%|██▊       | 5230/18627 [2:52:16<7:37:25,  2.05s/it]
 28%|██▊       | 5231/18627 [2:52:18<7:40:39,  2.06s/it]
 28%|██▊       | 5232/18627 [2:52:19<6:41:58,  1.80s/it]
 28%|██▊       | 5233/18627 [2:52:21<7:02:55,  1.89s/it]
 28%|██▊       | 5234/18627 [2:52:22<6:15:35,  1.68s/it]
 28%|██▊       | 5235/18627 [2:52:23<5:37:32,  1.51s/it]
 28%|██▊       | 5236/18627 [2:52:26<6:17:33,  1.69s/it]
 28%|██▊       | 5237/18627 [2:52:28<6:45:35,  1.82s/it]
 28%|██▊       | 5238/18627 [2:52:30<7:07:34,  1.92s/it]
 28%|██▊       | 5239/18627 [2:52:32<7:21:39,  1.98s/it]
 28%|██▊       | 5240/18627 [2:52:34<7:31:17,  2.02s/it]
                                                        
{'loss': 1.4008, 'grad_norm': 9.678160667419434, 'learning_rate': 4.21664604290372e-06, 'epoch': 0.28}
+
 28%|██▊       | 5240/18627 [2:52:34<7:31:17,  2.02s/it]
 28%|██▊       | 5241/18627 [2:52:36<7:37:15,  2.05s/it]
 28%|██▊       | 5242/18627 [2:52:37<6:07:30,  1.65s/it]
 28%|██▊       | 5243/18627 [2:52:39<6:39:43,  1.79s/it]
 28%|██▊       | 5244/18627 [2:52:41<7:02:08,  1.89s/it]
 28%|██▊       | 5245/18627 [2:52:43<7:17:27,  1.96s/it]
 28%|██▊       | 5246/18627 [2:52:44<5:52:20,  1.58s/it]
 28%|██▊       | 5247/18627 [2:52:46<6:29:36,  1.75s/it]
 28%|██▊       | 5248/18627 [2:52:48<6:55:38,  1.86s/it]
 28%|██▊       | 5249/18627 [2:52:50<7:12:30,  1.94s/it]
 28%|██▊       | 5250/18627 [2:52:52<6:22:10,  1.71s/it]
                                                        
{'loss': 0.8581, 'grad_norm': 13.987789154052734, 'learning_rate': 4.213483338194492e-06, 'epoch': 0.28}
+
 28%|██▊       | 5250/18627 [2:52:52<6:22:10,  1.71s/it]
 28%|██▊       | 5251/18627 [2:52:54<6:49:59,  1.84s/it]
 28%|██▊       | 5252/18627 [2:52:56<7:07:07,  1.92s/it]
 28%|██▊       | 5253/18627 [2:52:58<7:21:28,  1.98s/it]
 28%|██▊       | 5254/18627 [2:53:00<7:29:57,  2.02s/it]
 28%|██▊       | 5255/18627 [2:53:02<7:37:47,  2.05s/it]
 28%|██▊       | 5256/18627 [2:53:04<7:43:09,  2.08s/it]
 28%|██▊       | 5257/18627 [2:53:06<7:45:11,  2.09s/it]
 28%|██▊       | 5258/18627 [2:53:08<6:44:58,  1.82s/it]
 28%|██▊       | 5259/18627 [2:53:09<5:56:01,  1.60s/it]
 28%|██▊       | 5260/18627 [2:53:10<5:29:12,  1.48s/it]
                                                        
{'loss': 1.2654, 'grad_norm': 14.557610511779785, 'learning_rate': 4.210315453132224e-06, 'epoch': 0.28}
+
 28%|██▊       | 5260/18627 [2:53:10<5:29:12,  1.48s/it]
 28%|██▊       | 5261/18627 [2:53:12<6:12:20,  1.67s/it]
 28%|██▊       | 5262/18627 [2:53:14<6:41:47,  1.80s/it]
 28%|██▊       | 5263/18627 [2:53:15<6:00:42,  1.62s/it]
 28%|██▊       | 5264/18627 [2:53:17<6:34:26,  1.77s/it]
 28%|██▊       | 5265/18627 [2:53:20<6:57:03,  1.87s/it]
 28%|██▊       | 5266/18627 [2:53:22<7:13:57,  1.95s/it]
 28%|██▊       | 5267/18627 [2:53:23<6:23:05,  1.72s/it]
 28%|██▊       | 5268/18627 [2:53:24<5:47:17,  1.56s/it]
 28%|██▊       | 5269/18627 [2:53:26<6:25:00,  1.73s/it]
 28%|██▊       | 5270/18627 [2:53:27<5:48:41,  1.57s/it]
                                                        
{'loss': 1.4547, 'grad_norm': 14.873409271240234, 'learning_rate': 4.207142397294342e-06, 'epoch': 0.28}
+
 28%|██▊       | 5270/18627 [2:53:27<5:48:41,  1.57s/it]
 28%|██▊       | 5271/18627 [2:53:29<5:25:55,  1.46s/it]
 28%|██▊       | 5272/18627 [2:53:30<5:08:48,  1.39s/it]
 28%|██▊       | 5273/18627 [2:53:32<5:56:25,  1.60s/it]
 28%|██▊       | 5274/18627 [2:53:34<6:31:55,  1.76s/it]
 28%|██▊       | 5275/18627 [2:53:36<6:57:08,  1.87s/it]
 28%|██▊       | 5276/18627 [2:53:38<7:13:25,  1.95s/it]
 28%|██▊       | 5277/18627 [2:53:40<7:25:13,  2.00s/it]
 28%|██▊       | 5278/18627 [2:53:43<7:35:14,  2.05s/it]
 28%|██▊       | 5279/18627 [2:53:44<6:38:29,  1.79s/it]
 28%|██▊       | 5280/18627 [2:53:46<7:00:33,  1.89s/it]
                                                        
{'loss': 1.2765, 'grad_norm': 8.590595245361328, 'learning_rate': 4.2039641802739076e-06, 'epoch': 0.28}
+
 28%|██▊       | 5280/18627 [2:53:46<7:00:33,  1.89s/it]
 28%|██▊       | 5281/18627 [2:53:47<6:08:06,  1.65s/it]
 28%|██▊       | 5282/18627 [2:53:49<6:38:47,  1.79s/it]
 28%|██▊       | 5283/18627 [2:53:51<7:00:09,  1.89s/it]
 28%|██▊       | 5284/18627 [2:53:53<7:15:47,  1.96s/it]
 28%|██▊       | 5285/18627 [2:53:54<6:23:33,  1.72s/it]
 28%|██▊       | 5286/18627 [2:53:56<5:40:28,  1.53s/it]
 28%|██▊       | 5287/18627 [2:53:58<6:17:23,  1.70s/it]
 28%|██▊       | 5288/18627 [2:54:00<6:45:28,  1.82s/it]
 28%|██▊       | 5289/18627 [2:54:02<7:03:47,  1.91s/it]
 28%|██▊       | 5290/18627 [2:54:03<5:45:14,  1.55s/it]
                                                        
{'loss': 1.3259, 'grad_norm': 6.578243732452393, 'learning_rate': 4.200780811679584e-06, 'epoch': 0.28}
+
 28%|██▊       | 5290/18627 [2:54:03<5:45:14,  1.55s/it]
 28%|██▊       | 5291/18627 [2:54:04<5:21:41,  1.45s/it]
 28%|██▊       | 5292/18627 [2:54:05<5:04:26,  1.37s/it]
 28%|██▊       | 5293/18627 [2:54:07<5:52:31,  1.59s/it]
 28%|██▊       | 5294/18627 [2:54:09<6:29:05,  1.75s/it]
 28%|██▊       | 5295/18627 [2:54:11<6:54:45,  1.87s/it]
 28%|██▊       | 5296/18627 [2:54:13<6:09:31,  1.66s/it]
 28%|██▊       | 5297/18627 [2:54:15<6:38:40,  1.79s/it]
 28%|██▊       | 5298/18627 [2:54:16<5:57:54,  1.61s/it]
 28%|██▊       | 5299/18627 [2:54:18<6:31:12,  1.76s/it]
 28%|██▊       | 5300/18627 [2:54:20<6:53:45,  1.86s/it]
                                                        
{'loss': 1.4357, 'grad_norm': 6.469790935516357, 'learning_rate': 4.197592301135611e-06, 'epoch': 0.28}
+
 28%|██▊       | 5300/18627 [2:54:20<6:53:45,  1.86s/it]
 28%|██▊       | 5301/18627 [2:54:22<7:10:54,  1.94s/it]
 28%|██▊       | 5302/18627 [2:54:24<7:22:44,  1.99s/it]
 28%|██▊       | 5303/18627 [2:54:26<7:29:57,  2.03s/it]
 28%|██▊       | 5304/18627 [2:54:28<6:34:24,  1.78s/it]
 28%|██▊       | 5305/18627 [2:54:30<6:58:33,  1.89s/it]
 28%|██▊       | 5306/18627 [2:54:31<6:12:18,  1.68s/it]
 28%|██▊       | 5307/18627 [2:54:32<5:40:56,  1.54s/it]
 28%|██▊       | 5308/18627 [2:54:34<6:19:47,  1.71s/it]
 29%|██▊       | 5309/18627 [2:54:36<6:46:19,  1.83s/it]
 29%|██▊       | 5310/18627 [2:54:38<7:06:35,  1.92s/it]
                                                        
{'loss': 1.3226, 'grad_norm': 6.369358539581299, 'learning_rate': 4.194398658281774e-06, 'epoch': 0.29}
+
 29%|██▊       | 5310/18627 [2:54:38<7:06:35,  1.92s/it]
 29%|██▊       | 5311/18627 [2:54:40<6:18:42,  1.71s/it]
 29%|██▊       | 5312/18627 [2:54:42<6:46:59,  1.83s/it]
 29%|██▊       | 5313/18627 [2:54:43<6:04:28,  1.64s/it]
 29%|██▊       | 5314/18627 [2:54:45<6:35:10,  1.78s/it]
 29%|██▊       | 5315/18627 [2:54:47<6:57:49,  1.88s/it]
 29%|██▊       | 5316/18627 [2:54:49<7:14:01,  1.96s/it]
 29%|██▊       | 5317/18627 [2:54:50<6:17:16,  1.70s/it]
 29%|██▊       | 5318/18627 [2:54:53<6:45:44,  1.83s/it]
 29%|██▊       | 5319/18627 [2:54:54<6:02:54,  1.64s/it]
 29%|██▊       | 5320/18627 [2:54:54<5:00:36,  1.36s/it]
                                                        
{'loss': 1.5481, 'grad_norm': 6.664346218109131, 'learning_rate': 4.191199892773373e-06, 'epoch': 0.29}
+
 29%|██▊       | 5320/18627 [2:54:54<5:00:36,  1.36s/it]
 29%|██▊       | 5321/18627 [2:54:57<5:50:15,  1.58s/it]
 29%|██▊       | 5322/18627 [2:54:59<6:25:06,  1.74s/it]
 29%|██▊       | 5323/18627 [2:55:01<6:53:35,  1.87s/it]
 29%|██▊       | 5324/18627 [2:55:03<7:11:27,  1.95s/it]
 29%|██▊       | 5325/18627 [2:55:04<6:21:25,  1.72s/it]
 29%|██▊       | 5326/18627 [2:55:06<6:48:20,  1.84s/it]
 29%|██▊       | 5327/18627 [2:55:07<6:04:51,  1.65s/it]
 29%|██▊       | 5328/18627 [2:55:10<6:37:11,  1.79s/it]
 29%|██▊       | 5329/18627 [2:55:12<6:58:52,  1.89s/it]
 29%|██▊       | 5330/18627 [2:55:14<7:14:32,  1.96s/it]
                                                        
{'loss': 1.1174, 'grad_norm': 11.716681480407715, 'learning_rate': 4.187996014281197e-06, 'epoch': 0.29}
+
 29%|██▊       | 5330/18627 [2:55:14<7:14:32,  1.96s/it]
 29%|██▊       | 5331/18627 [2:55:16<7:25:41,  2.01s/it]
 29%|██▊       | 5332/18627 [2:55:18<7:33:38,  2.05s/it]
 29%|██▊       | 5333/18627 [2:55:20<7:38:06,  2.07s/it]
 29%|██▊       | 5334/18627 [2:55:22<7:41:50,  2.08s/it]
 29%|██▊       | 5335/18627 [2:55:24<7:43:06,  2.09s/it]
 29%|██▊       | 5336/18627 [2:55:27<7:44:14,  2.10s/it]
 29%|██▊       | 5337/18627 [2:55:29<7:45:22,  2.10s/it]
 29%|██▊       | 5338/18627 [2:55:30<6:44:47,  1.83s/it]
 29%|██▊       | 5339/18627 [2:55:32<7:04:30,  1.92s/it]
 29%|██▊       | 5340/18627 [2:55:34<7:17:25,  1.98s/it]
                                                        
{'loss': 0.8648, 'grad_norm': 9.262947082519531, 'learning_rate': 4.184787032491491e-06, 'epoch': 0.29}
+
 29%|██▊       | 5340/18627 [2:55:34<7:17:25,  1.98s/it]
 29%|██▊       | 5341/18627 [2:55:35<6:25:33,  1.74s/it]
 29%|██▊       | 5342/18627 [2:55:37<6:50:05,  1.85s/it]
 29%|██▊       | 5343/18627 [2:55:40<7:07:05,  1.93s/it]
 29%|██▊       | 5344/18627 [2:55:42<7:20:55,  1.99s/it]
 29%|██▊       | 5345/18627 [2:55:44<7:30:21,  2.03s/it]
 29%|██▊       | 5346/18627 [2:55:46<7:36:05,  2.06s/it]
 29%|██▊       | 5347/18627 [2:55:48<7:39:07,  2.07s/it]
 29%|██▊       | 5348/18627 [2:55:50<7:42:32,  2.09s/it]
 29%|██▊       | 5349/18627 [2:55:51<6:09:56,  1.67s/it]
 29%|██▊       | 5350/18627 [2:55:53<6:39:56,  1.81s/it]
                                                        
{'loss': 0.9386, 'grad_norm': 6.628335952758789, 'learning_rate': 4.181572957105932e-06, 'epoch': 0.29}
+
 29%|██▊       | 5350/18627 [2:55:53<6:39:56,  1.81s/it]
 29%|██▊       | 5351/18627 [2:55:55<6:59:00,  1.89s/it]
 29%|██▊       | 5352/18627 [2:55:57<7:12:40,  1.96s/it]
 29%|██▊       | 5353/18627 [2:55:59<7:22:07,  2.00s/it]
 29%|██▊       | 5354/18627 [2:56:00<6:29:32,  1.76s/it]
 29%|██▊       | 5355/18627 [2:56:03<6:52:42,  1.87s/it]
 29%|██▉       | 5356/18627 [2:56:04<5:56:10,  1.61s/it]
 29%|██▉       | 5357/18627 [2:56:06<6:29:10,  1.76s/it]
 29%|██▉       | 5358/18627 [2:56:08<6:52:31,  1.87s/it]
 29%|██▉       | 5359/18627 [2:56:10<7:09:45,  1.94s/it]
 29%|██▉       | 5360/18627 [2:56:11<6:20:50,  1.72s/it]
                                                        
{'loss': 1.3468, 'grad_norm': 16.815969467163086, 'learning_rate': 4.178353797841592e-06, 'epoch': 0.29}
+
 29%|██▉       | 5360/18627 [2:56:11<6:20:50,  1.72s/it]
 29%|██▉       | 5361/18627 [2:56:12<5:46:03,  1.57s/it]
 29%|██▉       | 5362/18627 [2:56:14<6:23:35,  1.74s/it]
 29%|██▉       | 5363/18627 [2:56:17<6:48:40,  1.85s/it]
 29%|██▉       | 5364/18627 [2:56:18<6:05:03,  1.65s/it]
 29%|██▉       | 5365/18627 [2:56:20<6:35:46,  1.79s/it]
 29%|██▉       | 5366/18627 [2:56:22<6:57:07,  1.89s/it]
 29%|██▉       | 5367/18627 [2:56:24<7:15:12,  1.97s/it]
 29%|██▉       | 5368/18627 [2:56:26<7:25:14,  2.01s/it]
 29%|██▉       | 5369/18627 [2:56:28<7:33:02,  2.05s/it]
 29%|██▉       | 5370/18627 [2:56:29<6:03:44,  1.65s/it]
                                                        
{'loss': 1.183, 'grad_norm': 9.601729393005371, 'learning_rate': 4.175129564430919e-06, 'epoch': 0.29}
+
 29%|██▉       | 5370/18627 [2:56:29<6:03:44,  1.65s/it]
 29%|██▉       | 5371/18627 [2:56:31<6:34:42,  1.79s/it]
 29%|██▉       | 5372/18627 [2:56:33<6:57:18,  1.89s/it]
 29%|██▉       | 5373/18627 [2:56:35<7:12:39,  1.96s/it]
 29%|██▉       | 5374/18627 [2:56:38<7:25:14,  2.02s/it]
 29%|██▉       | 5375/18627 [2:56:40<7:32:53,  2.05s/it]
 29%|██▉       | 5376/18627 [2:56:42<7:36:46,  2.07s/it]
 29%|██▉       | 5377/18627 [2:56:44<7:42:23,  2.09s/it]
 29%|██▉       | 5378/18627 [2:56:46<7:44:21,  2.10s/it]
 29%|██▉       | 5379/18627 [2:56:47<6:11:19,  1.68s/it]
 29%|██▉       | 5380/18627 [2:56:48<5:38:47,  1.53s/it]
                                                        
{'loss': 0.8782, 'grad_norm': 16.4572696685791, 'learning_rate': 4.171900266621695e-06, 'epoch': 0.29}
+
 29%|██▉       | 5380/18627 [2:56:48<5:38:47,  1.53s/it]
 29%|██▉       | 5381/18627 [2:56:50<6:16:15,  1.70s/it]
 29%|██▉       | 5382/18627 [2:56:52<6:44:28,  1.83s/it]
 29%|██▉       | 5383/18627 [2:56:54<7:03:27,  1.92s/it]
 29%|██▉       | 5384/18627 [2:56:57<7:17:02,  1.98s/it]
 29%|██▉       | 5385/18627 [2:56:59<7:25:59,  2.02s/it]
 29%|██▉       | 5386/18627 [2:57:00<6:31:15,  1.77s/it]
 29%|██▉       | 5387/18627 [2:57:02<6:54:38,  1.88s/it]
 29%|██▉       | 5388/18627 [2:57:03<6:09:04,  1.67s/it]
 29%|██▉       | 5389/18627 [2:57:05<6:37:48,  1.80s/it]
 29%|██▉       | 5390/18627 [2:57:07<6:59:39,  1.90s/it]
                                                        
{'loss': 0.9901, 'grad_norm': 7.192978858947754, 'learning_rate': 4.1686659141770166e-06, 'epoch': 0.29}
+
 29%|██▉       | 5390/18627 [2:57:07<6:59:39,  1.90s/it]
 29%|██▉       | 5391/18627 [2:57:09<7:13:51,  1.97s/it]
 29%|██▉       | 5392/18627 [2:57:11<6:22:41,  1.73s/it]
 29%|██▉       | 5393/18627 [2:57:14<8:21:32,  2.27s/it]
 29%|██▉       | 5394/18627 [2:57:16<8:11:28,  2.23s/it]
 29%|██▉       | 5395/18627 [2:57:18<8:02:33,  2.19s/it]
 29%|██▉       | 5396/18627 [2:57:21<7:57:01,  2.16s/it]
 29%|██▉       | 5397/18627 [2:57:23<7:53:52,  2.15s/it]
 29%|██▉       | 5398/18627 [2:57:25<7:52:09,  2.14s/it]
 29%|██▉       | 5399/18627 [2:57:25<6:17:32,  1.71s/it]
 29%|██▉       | 5400/18627 [2:57:26<5:11:23,  1.41s/it]
                                                        
{'loss': 0.8602, 'grad_norm': 7.557520866394043, 'learning_rate': 4.165426516875263e-06, 'epoch': 0.29}
+
 29%|██▉       | 5400/18627 [2:57:26<5:11:23,  1.41s/it]
 29%|██▉       | 5401/18627 [2:57:27<4:57:20,  1.35s/it]
 29%|██▉       | 5402/18627 [2:57:30<5:46:45,  1.57s/it]
 29%|██▉       | 5403/18627 [2:57:32<6:21:16,  1.73s/it]
 29%|██▉       | 5404/18627 [2:57:34<6:47:34,  1.85s/it]
 29%|██▉       | 5405/18627 [2:57:36<7:05:32,  1.93s/it]
 29%|██▉       | 5406/18627 [2:57:38<7:18:41,  1.99s/it]
 29%|██▉       | 5407/18627 [2:57:40<7:27:04,  2.03s/it]
 29%|██▉       | 5408/18627 [2:57:42<7:32:38,  2.05s/it]
 29%|██▉       | 5409/18627 [2:57:44<7:37:16,  2.08s/it]
 29%|██▉       | 5410/18627 [2:57:46<7:39:05,  2.08s/it]
                                                        
{'loss': 0.8847, 'grad_norm': 5.464683532714844, 'learning_rate': 4.1621820845100624e-06, 'epoch': 0.29}
+
 29%|██▉       | 5410/18627 [2:57:46<7:39:05,  2.08s/it]
 29%|██▉       | 5411/18627 [2:57:49<7:42:39,  2.10s/it]
 29%|██▉       | 5412/18627 [2:57:50<6:42:37,  1.83s/it]
 29%|██▉       | 5413/18627 [2:57:52<7:01:23,  1.91s/it]
 29%|██▉       | 5414/18627 [2:57:53<5:40:58,  1.55s/it]
 29%|██▉       | 5415/18627 [2:57:55<6:18:58,  1.72s/it]
 29%|██▉       | 5416/18627 [2:57:57<6:45:56,  1.84s/it]
 29%|██▉       | 5417/18627 [2:57:59<7:04:10,  1.93s/it]
 29%|██▉       | 5418/18627 [2:58:00<6:15:44,  1.71s/it]
 29%|██▉       | 5419/18627 [2:58:02<6:42:42,  1.83s/it]
 29%|██▉       | 5420/18627 [2:58:04<7:02:43,  1.92s/it]
                                                        
{'loss': 1.1192, 'grad_norm': 12.930455207824707, 'learning_rate': 4.158932626890269e-06, 'epoch': 0.29}
+
 29%|██▉       | 5420/18627 [2:58:04<7:02:43,  1.92s/it]
 29%|██▉       | 5421/18627 [2:58:07<7:15:32,  1.98s/it]
 29%|██▉       | 5422/18627 [2:58:09<7:23:29,  2.02s/it]
 29%|██▉       | 5423/18627 [2:58:10<6:28:31,  1.77s/it]
 29%|██▉       | 5424/18627 [2:58:12<6:51:38,  1.87s/it]
 29%|██▉       | 5425/18627 [2:58:14<7:06:55,  1.94s/it]
 29%|██▉       | 5426/18627 [2:58:15<6:17:05,  1.71s/it]
 29%|██▉       | 5427/18627 [2:58:17<6:43:02,  1.83s/it]
 29%|██▉       | 5428/18627 [2:58:19<7:02:40,  1.92s/it]
 29%|██▉       | 5429/18627 [2:58:22<7:15:21,  1.98s/it]
 29%|██▉       | 5430/18627 [2:58:24<7:24:55,  2.02s/it]
                                                        
{'loss': 1.0656, 'grad_norm': 6.370907306671143, 'learning_rate': 4.155678153839927e-06, 'epoch': 0.29}
+
 29%|██▉       | 5430/18627 [2:58:24<7:24:55,  2.02s/it]
 29%|██▉       | 5431/18627 [2:58:26<7:30:17,  2.05s/it]
 29%|██▉       | 5432/18627 [2:58:27<6:33:55,  1.79s/it]
 29%|██▉       | 5433/18627 [2:58:29<6:56:04,  1.89s/it]
 29%|██▉       | 5434/18627 [2:58:31<7:12:14,  1.97s/it]
 29%|██▉       | 5435/18627 [2:58:33<7:24:02,  2.02s/it]
 29%|██▉       | 5436/18627 [2:58:36<7:30:52,  2.05s/it]
 29%|██▉       | 5437/18627 [2:58:38<7:34:47,  2.07s/it]
 29%|██▉       | 5438/18627 [2:58:40<7:40:16,  2.09s/it]
 29%|██▉       | 5439/18627 [2:58:42<7:42:21,  2.10s/it]
 29%|██▉       | 5440/18627 [2:58:44<7:44:18,  2.11s/it]
                                                        
{'loss': 0.8562, 'grad_norm': 6.343400001525879, 'learning_rate': 4.152418675198245e-06, 'epoch': 0.29}
+
 29%|██▉       | 5440/18627 [2:58:44<7:44:18,  2.11s/it]
 29%|██▉       | 5441/18627 [2:58:46<7:45:50,  2.12s/it]
 29%|██▉       | 5442/18627 [2:58:48<7:45:12,  2.12s/it]
 29%|██▉       | 5443/18627 [2:58:50<7:44:46,  2.12s/it]
 29%|██▉       | 5444/18627 [2:58:52<6:44:02,  1.84s/it]
 29%|██▉       | 5445/18627 [2:58:54<7:01:09,  1.92s/it]
 29%|██▉       | 5446/18627 [2:58:56<7:14:56,  1.98s/it]
 29%|██▉       | 5447/18627 [2:58:57<6:22:41,  1.74s/it]
 29%|██▉       | 5448/18627 [2:58:59<6:45:42,  1.85s/it]
 29%|██▉       | 5449/18627 [2:59:01<7:03:43,  1.93s/it]
 29%|██▉       | 5450/18627 [2:59:02<6:14:59,  1.71s/it]
                                                        
{'loss': 1.2986, 'grad_norm': 15.066439628601074, 'learning_rate': 4.149154200819564e-06, 'epoch': 0.29}
+
 29%|██▉       | 5450/18627 [2:59:02<6:14:59,  1.71s/it]
 29%|██▉       | 5451/18627 [2:59:05<6:40:52,  1.83s/it]
 29%|██▉       | 5452/18627 [2:59:07<7:00:27,  1.91s/it]
 29%|██▉       | 5453/18627 [2:59:09<7:13:18,  1.97s/it]
 29%|██▉       | 5454/18627 [2:59:11<7:23:47,  2.02s/it]
 29%|██▉       | 5455/18627 [2:59:13<7:32:18,  2.06s/it]
 29%|██▉       | 5456/18627 [2:59:15<7:36:26,  2.08s/it]
 29%|██▉       | 5457/18627 [2:59:17<7:38:43,  2.09s/it]
 29%|██▉       | 5458/18627 [2:59:19<7:40:24,  2.10s/it]
 29%|██▉       | 5459/18627 [2:59:21<7:41:45,  2.10s/it]
 29%|██▉       | 5460/18627 [2:59:24<7:42:09,  2.11s/it]
                                                        
{'loss': 0.646, 'grad_norm': 7.98778772354126, 'learning_rate': 4.145884740573329e-06, 'epoch': 0.29}
+
 29%|██▉       | 5460/18627 [2:59:24<7:42:09,  2.11s/it]
 29%|██▉       | 5461/18627 [2:59:26<7:44:14,  2.12s/it]
 29%|██▉       | 5462/18627 [2:59:28<7:44:20,  2.12s/it]
 29%|██▉       | 5463/18627 [2:59:29<6:42:53,  1.84s/it]
 29%|██▉       | 5464/18627 [2:59:31<7:03:16,  1.93s/it]
 29%|██▉       | 5465/18627 [2:59:33<7:16:25,  1.99s/it]
 29%|██▉       | 5466/18627 [2:59:35<6:23:24,  1.75s/it]
 29%|██▉       | 5467/18627 [2:59:37<6:48:24,  1.86s/it]
 29%|██▉       | 5468/18627 [2:59:39<7:04:58,  1.94s/it]
 29%|██▉       | 5469/18627 [2:59:41<7:16:55,  1.99s/it]
 29%|██▉       | 5470/18627 [2:59:43<7:26:38,  2.04s/it]
                                                        
{'loss': 1.0345, 'grad_norm': 6.593790531158447, 'learning_rate': 4.142610304344061e-06, 'epoch': 0.29}
+
 29%|██▉       | 5470/18627 [2:59:43<7:26:38,  2.04s/it]
 29%|██▉       | 5471/18627 [2:59:45<7:33:18,  2.07s/it]
 29%|██▉       | 5472/18627 [2:59:47<7:35:34,  2.08s/it]
 29%|██▉       | 5473/18627 [2:59:49<7:37:54,  2.09s/it]
 29%|██▉       | 5474/18627 [2:59:50<6:06:02,  1.67s/it]
 29%|██▉       | 5475/18627 [2:59:52<6:35:56,  1.81s/it]
 29%|██▉       | 5476/18627 [2:59:54<6:55:48,  1.90s/it]
 29%|██▉       | 5477/18627 [2:59:56<7:10:01,  1.96s/it]
 29%|██▉       | 5478/18627 [2:59:59<7:19:10,  2.00s/it]
 29%|██▉       | 5479/18627 [3:00:01<7:25:02,  2.03s/it]
 29%|██▉       | 5480/18627 [3:00:03<7:31:23,  2.06s/it]
                                                        
{'loss': 0.6564, 'grad_norm': 8.7335205078125, 'learning_rate': 4.139330902031319e-06, 'epoch': 0.29}
+
 29%|██▉       | 5480/18627 [3:00:03<7:31:23,  2.06s/it]
 29%|██▉       | 5481/18627 [3:00:05<7:33:56,  2.07s/it]
 29%|██▉       | 5482/18627 [3:00:07<7:35:51,  2.08s/it]
 29%|██▉       | 5483/18627 [3:00:09<7:38:48,  2.09s/it]
 29%|██▉       | 5484/18627 [3:00:10<6:39:36,  1.82s/it]
 29%|██▉       | 5485/18627 [3:00:12<7:00:10,  1.92s/it]
 29%|██▉       | 5486/18627 [3:00:15<7:14:22,  1.98s/it]
 29%|██▉       | 5487/18627 [3:00:16<6:22:55,  1.75s/it]
 29%|██▉       | 5488/18627 [3:00:18<6:47:45,  1.86s/it]
 29%|██▉       | 5489/18627 [3:00:19<5:31:26,  1.51s/it]
 29%|██▉       | 5490/18627 [3:00:20<5:09:54,  1.42s/it]
                                                        
{'loss': 1.321, 'grad_norm': 14.440370559692383, 'learning_rate': 4.136046543549683e-06, 'epoch': 0.29}
+
 29%|██▉       | 5490/18627 [3:00:20<5:09:54,  1.42s/it]
 29%|██▉       | 5491/18627 [3:00:22<5:56:10,  1.63s/it]
 29%|██▉       | 5492/18627 [3:00:23<5:27:09,  1.49s/it]
 29%|██▉       | 5493/18627 [3:00:25<6:08:58,  1.69s/it]
 29%|██▉       | 5494/18627 [3:00:26<5:36:01,  1.54s/it]
 30%|██▉       | 5495/18627 [3:00:27<4:41:52,  1.29s/it]
 30%|██▉       | 5496/18627 [3:00:29<5:35:34,  1.53s/it]
 30%|██▉       | 5497/18627 [3:00:31<6:14:51,  1.71s/it]
 30%|██▉       | 5498/18627 [3:00:33<6:40:01,  1.83s/it]
 30%|██▉       | 5499/18627 [3:00:36<6:59:49,  1.92s/it]
 30%|██▉       | 5500/18627 [3:00:37<6:11:54,  1.70s/it]
                                                        
{'loss': 1.3228, 'grad_norm': 14.811610221862793, 'learning_rate': 4.1327572388287105e-06, 'epoch': 0.3}
+
 30%|██▉       | 5500/18627 [3:00:37<6:11:54,  1.70s/it]
 30%|██▉       | 5501/18627 [3:00:39<6:41:21,  1.83s/it]
 30%|██▉       | 5502/18627 [3:00:40<5:26:27,  1.49s/it]
 30%|██▉       | 5503/18627 [3:00:41<5:01:03,  1.38s/it]
 30%|██▉       | 5504/18627 [3:00:43<5:50:40,  1.60s/it]
 30%|██▉       | 5505/18627 [3:00:45<6:23:40,  1.75s/it]
 30%|██▉       | 5506/18627 [3:00:47<6:47:37,  1.86s/it]
 30%|██▉       | 5507/18627 [3:00:49<7:04:44,  1.94s/it]
 30%|██▉       | 5508/18627 [3:00:51<7:16:46,  2.00s/it]
 30%|██▉       | 5509/18627 [3:00:53<7:24:34,  2.03s/it]
 30%|██▉       | 5510/18627 [3:00:56<7:29:59,  2.06s/it]
                                                        
{'loss': 0.9261, 'grad_norm': 7.305030822753906, 'learning_rate': 4.129462997812918e-06, 'epoch': 0.3}
+
 30%|██▉       | 5510/18627 [3:00:56<7:29:59,  2.06s/it]
 30%|██▉       | 5511/18627 [3:00:57<6:32:53,  1.80s/it]
 30%|██▉       | 5512/18627 [3:00:58<5:53:24,  1.62s/it]
 30%|██▉       | 5513/18627 [3:01:00<6:27:16,  1.77s/it]
 30%|██▉       | 5514/18627 [3:01:01<5:49:38,  1.60s/it]
 30%|██▉       | 5515/18627 [3:01:03<6:22:03,  1.75s/it]
 30%|██▉       | 5516/18627 [3:01:05<6:47:26,  1.86s/it]
 30%|██▉       | 5517/18627 [3:01:08<7:03:51,  1.94s/it]
 30%|██▉       | 5518/18627 [3:01:10<7:15:55,  2.00s/it]
 30%|██▉       | 5519/18627 [3:01:12<7:23:56,  2.03s/it]
 30%|██▉       | 5520/18627 [3:01:13<6:28:30,  1.78s/it]
                                                        
{'loss': 1.3928, 'grad_norm': 14.89034366607666, 'learning_rate': 4.126163830461744e-06, 'epoch': 0.3}
+
 30%|██▉       | 5520/18627 [3:01:13<6:28:30,  1.78s/it]
 30%|██▉       | 5521/18627 [3:01:15<6:50:45,  1.88s/it]
 30%|██▉       | 5522/18627 [3:01:17<7:05:39,  1.95s/it]
 30%|██▉       | 5523/18627 [3:01:19<7:15:47,  2.00s/it]
 30%|██▉       | 5524/18627 [3:01:21<7:24:47,  2.04s/it]
 30%|██▉       | 5525/18627 [3:01:24<7:30:31,  2.06s/it]
 30%|██▉       | 5526/18627 [3:01:25<6:33:03,  1.80s/it]
 30%|██▉       | 5527/18627 [3:01:27<6:53:45,  1.90s/it]
 30%|██▉       | 5528/18627 [3:01:29<7:09:12,  1.97s/it]
 30%|██▉       | 5529/18627 [3:01:31<7:20:58,  2.02s/it]
 30%|██▉       | 5530/18627 [3:01:33<7:27:00,  2.05s/it]
                                                        
{'loss': 0.9012, 'grad_norm': 9.645459175109863, 'learning_rate': 4.1228597467495185e-06, 'epoch': 0.3}
+
 30%|██▉       | 5530/18627 [3:01:33<7:27:00,  2.05s/it]
 30%|██▉       | 5531/18627 [3:01:35<7:30:52,  2.07s/it]
 30%|██▉       | 5532/18627 [3:01:38<7:35:03,  2.08s/it]
 30%|██▉       | 5533/18627 [3:01:40<7:37:42,  2.10s/it]
 30%|██▉       | 5534/18627 [3:01:42<7:39:35,  2.11s/it]
 30%|██▉       | 5535/18627 [3:01:44<7:39:45,  2.11s/it]
 30%|██▉       | 5536/18627 [3:01:46<7:39:19,  2.11s/it]
 30%|██▉       | 5537/18627 [3:01:48<7:39:37,  2.11s/it]
 30%|██▉       | 5538/18627 [3:01:50<7:40:36,  2.11s/it]
 30%|██▉       | 5539/18627 [3:01:52<7:40:40,  2.11s/it]
 30%|██▉       | 5540/18627 [3:01:54<7:39:47,  2.11s/it]
                                                        
{'loss': 0.6854, 'grad_norm': 12.189104080200195, 'learning_rate': 4.119550756665438e-06, 'epoch': 0.3}
+
 30%|██▉       | 5540/18627 [3:01:54<7:39:47,  2.11s/it]
 30%|██▉       | 5541/18627 [3:01:57<7:41:03,  2.11s/it]
 30%|██▉       | 5542/18627 [3:01:58<6:40:22,  1.84s/it]
 30%|██▉       | 5543/18627 [3:02:00<6:58:42,  1.92s/it]
 30%|██▉       | 5544/18627 [3:02:02<7:10:44,  1.98s/it]
 30%|██▉       | 5545/18627 [3:02:03<6:19:13,  1.74s/it]
 30%|██▉       | 5546/18627 [3:02:05<6:44:35,  1.86s/it]
 30%|██▉       | 5547/18627 [3:02:07<7:00:22,  1.93s/it]
 30%|██▉       | 5548/18627 [3:02:08<6:04:20,  1.67s/it]
 30%|██▉       | 5549/18627 [3:02:11<6:33:43,  1.81s/it]
 30%|██▉       | 5550/18627 [3:02:13<6:54:49,  1.90s/it]
                                                        
{'loss': 1.3102, 'grad_norm': 4.6539411544799805, 'learning_rate': 4.1162368702135326e-06, 'epoch': 0.3}
+
 30%|██▉       | 5550/18627 [3:02:13<6:54:49,  1.90s/it]
 30%|██▉       | 5551/18627 [3:02:15<7:09:35,  1.97s/it]
 30%|██▉       | 5552/18627 [3:02:17<7:18:37,  2.01s/it]
 30%|██▉       | 5553/18627 [3:02:19<7:24:58,  2.04s/it]
 30%|██▉       | 5554/18627 [3:02:20<5:56:55,  1.64s/it]
 30%|██▉       | 5555/18627 [3:02:22<6:28:44,  1.78s/it]
 30%|██▉       | 5556/18627 [3:02:24<6:50:58,  1.89s/it]
 30%|██▉       | 5557/18627 [3:02:25<6:04:57,  1.68s/it]
 30%|██▉       | 5558/18627 [3:02:27<6:33:12,  1.81s/it]
 30%|██▉       | 5559/18627 [3:02:29<6:53:12,  1.90s/it]
 30%|██▉       | 5560/18627 [3:02:31<6:06:24,  1.68s/it]
                                                        
{'loss': 1.1095, 'grad_norm': 15.340166091918945, 'learning_rate': 4.112918097412633e-06, 'epoch': 0.3}
+
 30%|██▉       | 5560/18627 [3:02:31<6:06:24,  1.68s/it]
 30%|██▉       | 5561/18627 [3:02:32<5:33:45,  1.53s/it]
 30%|██▉       | 5562/18627 [3:02:34<6:11:07,  1.70s/it]
 30%|██▉       | 5563/18627 [3:02:36<6:36:43,  1.82s/it]
 30%|██▉       | 5564/18627 [3:02:38<6:56:37,  1.91s/it]
 30%|██▉       | 5565/18627 [3:02:40<7:08:07,  1.97s/it]
 30%|██▉       | 5566/18627 [3:02:42<7:18:58,  2.02s/it]
 30%|██▉       | 5567/18627 [3:02:44<7:24:25,  2.04s/it]
 30%|██▉       | 5568/18627 [3:02:47<7:28:47,  2.06s/it]
 30%|██▉       | 5569/18627 [3:02:49<7:31:48,  2.08s/it]
 30%|██▉       | 5570/18627 [3:02:50<6:34:02,  1.81s/it]
                                                        
{'loss': 1.0254, 'grad_norm': 15.2852144241333, 'learning_rate': 4.1095944482963436e-06, 'epoch': 0.3}
+
 30%|██▉       | 5570/18627 [3:02:50<6:34:02,  1.81s/it]
 30%|██▉       | 5571/18627 [3:02:52<6:53:21,  1.90s/it]
 30%|██▉       | 5572/18627 [3:02:54<7:08:53,  1.97s/it]
 30%|██▉       | 5573/18627 [3:02:55<5:45:34,  1.59s/it]
 30%|██▉       | 5574/18627 [3:02:56<5:19:29,  1.47s/it]
 30%|██▉       | 5575/18627 [3:02:57<5:01:48,  1.39s/it]
 30%|██▉       | 5576/18627 [3:02:59<5:48:08,  1.60s/it]
 30%|██▉       | 5577/18627 [3:03:01<6:22:59,  1.76s/it]
 30%|██▉       | 5578/18627 [3:03:04<6:46:48,  1.87s/it]
 30%|██▉       | 5579/18627 [3:03:05<6:02:53,  1.67s/it]
 30%|██▉       | 5580/18627 [3:03:07<6:33:49,  1.81s/it]
                                                        
{'loss': 1.2259, 'grad_norm': 9.176116943359375, 'learning_rate': 4.106265932913013e-06, 'epoch': 0.3}
+
 30%|██▉       | 5580/18627 [3:03:07<6:33:49,  1.81s/it]
 30%|██▉       | 5581/18627 [3:03:09<6:53:46,  1.90s/it]
 30%|██▉       | 5582/18627 [3:03:11<7:09:23,  1.97s/it]
 30%|██▉       | 5583/18627 [3:03:13<7:18:22,  2.02s/it]
 30%|██▉       | 5584/18627 [3:03:15<7:23:56,  2.04s/it]
 30%|██▉       | 5585/18627 [3:03:17<6:27:39,  1.78s/it]
 30%|██▉       | 5586/18627 [3:03:18<5:48:49,  1.60s/it]
 30%|██▉       | 5587/18627 [3:03:20<6:22:13,  1.76s/it]
 30%|██▉       | 5588/18627 [3:03:22<6:46:04,  1.87s/it]
 30%|███       | 5589/18627 [3:03:24<7:02:47,  1.95s/it]
 30%|███       | 5590/18627 [3:03:25<5:41:27,  1.57s/it]
                                                        
{'loss': 1.1049, 'grad_norm': 7.110942363739014, 'learning_rate': 4.102932561325699e-06, 'epoch': 0.3}
+
 30%|███       | 5590/18627 [3:03:25<5:41:27,  1.57s/it]
 30%|███       | 5591/18627 [3:03:27<6:17:51,  1.74s/it]
 30%|███       | 5592/18627 [3:03:29<6:43:00,  1.86s/it]
 30%|███       | 5593/18627 [3:03:31<6:59:14,  1.93s/it]
 30%|███       | 5594/18627 [3:03:33<7:11:49,  1.99s/it]
 30%|███       | 5595/18627 [3:03:34<6:20:04,  1.75s/it]
 30%|███       | 5596/18627 [3:03:36<5:43:16,  1.58s/it]
 30%|███       | 5597/18627 [3:03:38<6:17:36,  1.74s/it]
 30%|███       | 5598/18627 [3:03:40<6:42:27,  1.85s/it]
 30%|███       | 5599/18627 [3:03:41<5:52:19,  1.62s/it]
 30%|███       | 5600/18627 [3:03:42<5:23:59,  1.49s/it]
                                                        
{'loss': 1.4992, 'grad_norm': 14.922239303588867, 'learning_rate': 4.099594343612146e-06, 'epoch': 0.3}
+
 30%|███       | 5600/18627 [3:03:42<5:23:59,  1.49s/it]
 30%|███       | 5601/18627 [3:03:43<4:49:12,  1.33s/it]
 30%|███       | 5602/18627 [3:03:44<4:13:56,  1.17s/it]
 30%|███       | 5603/18627 [3:03:45<4:16:17,  1.18s/it]
 30%|███       | 5604/18627 [3:03:47<5:17:53,  1.46s/it]
 30%|███       | 5605/18627 [3:03:48<4:59:47,  1.38s/it]
 30%|███       | 5606/18627 [3:03:51<5:46:48,  1.60s/it]
 30%|███       | 5607/18627 [3:03:53<6:20:28,  1.75s/it]
 30%|███       | 5608/18627 [3:03:55<6:44:27,  1.86s/it]
 30%|███       | 5609/18627 [3:03:57<7:03:13,  1.95s/it]
 30%|███       | 5610/18627 [3:03:58<6:13:35,  1.72s/it]
                                                        
{'loss': 1.4807, 'grad_norm': 16.830324172973633, 'learning_rate': 4.096251289864743e-06, 'epoch': 0.3}
+
 30%|███       | 5610/18627 [3:03:58<6:13:35,  1.72s/it]
 30%|███       | 5611/18627 [3:04:00<6:40:01,  1.84s/it]
 30%|███       | 5612/18627 [3:04:01<5:57:49,  1.65s/it]
 30%|███       | 5613/18627 [3:04:04<6:29:53,  1.80s/it]
 30%|███       | 5614/18627 [3:04:05<5:49:34,  1.61s/it]
 30%|███       | 5615/18627 [3:04:07<6:22:21,  1.76s/it]
 30%|███       | 5616/18627 [3:04:09<6:46:05,  1.87s/it]
 30%|███       | 5617/18627 [3:04:10<6:01:37,  1.67s/it]
 30%|███       | 5618/18627 [3:04:12<6:30:39,  1.80s/it]
 30%|███       | 5619/18627 [3:04:13<5:50:36,  1.62s/it]
 30%|███       | 5620/18627 [3:04:16<6:22:42,  1.77s/it]
                                                        
{'loss': 1.4759, 'grad_norm': 5.010047912597656, 'learning_rate': 4.092903410190505e-06, 'epoch': 0.3}
+
 30%|███       | 5620/18627 [3:04:16<6:22:42,  1.77s/it]
 30%|███       | 5621/18627 [3:04:18<6:46:00,  1.87s/it]
 30%|███       | 5622/18627 [3:04:20<7:01:39,  1.95s/it]
 30%|███       | 5623/18627 [3:04:22<7:13:22,  2.00s/it]
 30%|███       | 5624/18627 [3:04:23<6:20:33,  1.76s/it]
 30%|███       | 5625/18627 [3:04:25<6:43:44,  1.86s/it]
 30%|███       | 5626/18627 [3:04:27<6:59:30,  1.94s/it]
 30%|███       | 5627/18627 [3:04:29<7:10:07,  1.99s/it]
 30%|███       | 5628/18627 [3:04:32<7:18:11,  2.02s/it]
 30%|███       | 5629/18627 [3:04:32<5:52:30,  1.63s/it]
 30%|███       | 5630/18627 [3:04:34<6:24:30,  1.78s/it]
                                                        
{'loss': 0.807, 'grad_norm': 8.636159896850586, 'learning_rate': 4.0895507147110345e-06, 'epoch': 0.3}
+
 30%|███       | 5630/18627 [3:04:34<6:24:30,  1.78s/it]
 30%|███       | 5631/18627 [3:04:36<6:45:46,  1.87s/it]
 30%|███       | 5632/18627 [3:04:39<7:01:50,  1.95s/it]
 30%|███       | 5633/18627 [3:04:40<6:12:15,  1.72s/it]
 30%|███       | 5634/18627 [3:04:42<6:39:08,  1.84s/it]
 30%|███       | 5635/18627 [3:04:44<6:57:14,  1.93s/it]
 30%|███       | 5636/18627 [3:04:46<7:11:20,  1.99s/it]
 30%|███       | 5637/18627 [3:04:48<7:20:35,  2.04s/it]
 30%|███       | 5638/18627 [3:04:50<7:26:29,  2.06s/it]
 30%|███       | 5639/18627 [3:04:53<7:29:36,  2.08s/it]
 30%|███       | 5640/18627 [3:04:55<7:32:49,  2.09s/it]
                                                        
{'loss': 0.8583, 'grad_norm': 10.074819564819336, 'learning_rate': 4.086193213562495e-06, 'epoch': 0.3}
+
 30%|███       | 5640/18627 [3:04:55<7:32:49,  2.09s/it]
 30%|███       | 5641/18627 [3:04:57<7:36:04,  2.11s/it]
 30%|███       | 5642/18627 [3:04:59<7:36:15,  2.11s/it]
 30%|███       | 5643/18627 [3:05:01<7:36:14,  2.11s/it]
 30%|███       | 5644/18627 [3:05:03<7:36:57,  2.11s/it]
 30%|███       | 5645/18627 [3:05:05<7:37:15,  2.11s/it]
 30%|███       | 5646/18627 [3:05:06<6:37:07,  1.84s/it]
 30%|███       | 5647/18627 [3:05:09<6:53:46,  1.91s/it]
 30%|███       | 5648/18627 [3:05:11<7:07:38,  1.98s/it]
 30%|███       | 5649/18627 [3:05:13<7:17:20,  2.02s/it]
 30%|███       | 5650/18627 [3:05:15<7:23:18,  2.05s/it]
                                                        
{'loss': 0.8729, 'grad_norm': 7.2127366065979, 'learning_rate': 4.082830916895578e-06, 'epoch': 0.3}
+
 30%|███       | 5650/18627 [3:05:15<7:23:18,  2.05s/it]
 30%|███       | 5651/18627 [3:05:16<6:28:04,  1.79s/it]
 30%|███       | 5652/18627 [3:05:17<5:49:27,  1.62s/it]
 30%|███       | 5653/18627 [3:05:19<5:21:53,  1.49s/it]
 30%|███       | 5654/18627 [3:05:21<6:03:14,  1.68s/it]
 30%|███       | 5655/18627 [3:05:23<6:31:48,  1.81s/it]
 30%|███       | 5656/18627 [3:05:25<6:51:41,  1.90s/it]
 30%|███       | 5657/18627 [3:05:27<7:06:04,  1.97s/it]
 30%|███       | 5658/18627 [3:05:29<7:16:27,  2.02s/it]
 30%|███       | 5659/18627 [3:05:31<7:21:17,  2.04s/it]
 30%|███       | 5660/18627 [3:05:33<7:25:42,  2.06s/it]
                                                        
{'loss': 1.3094, 'grad_norm': 7.143892288208008, 'learning_rate': 4.079463834875474e-06, 'epoch': 0.3}
+
 30%|███       | 5660/18627 [3:05:33<7:25:42,  2.06s/it]
 30%|███       | 5661/18627 [3:05:35<7:29:01,  2.08s/it]
 30%|███       | 5662/18627 [3:05:38<7:31:57,  2.09s/it]
 30%|███       | 5663/18627 [3:05:40<7:33:27,  2.10s/it]
 30%|███       | 5664/18627 [3:05:42<7:34:24,  2.10s/it]
 30%|███       | 5665/18627 [3:05:44<7:35:40,  2.11s/it]
 30%|███       | 5666/18627 [3:05:46<7:36:01,  2.11s/it]
 30%|███       | 5667/18627 [3:05:48<7:36:31,  2.11s/it]
 30%|███       | 5668/18627 [3:05:49<6:37:23,  1.84s/it]
 30%|███       | 5669/18627 [3:05:50<5:46:03,  1.60s/it]
 30%|███       | 5670/18627 [3:05:53<6:19:44,  1.76s/it]
                                                        
{'loss': 1.1032, 'grad_norm': 6.193157196044922, 'learning_rate': 4.07609197768184e-06, 'epoch': 0.3}
+
 30%|███       | 5670/18627 [3:05:53<6:19:44,  1.76s/it]
 30%|███       | 5671/18627 [3:05:55<6:43:28,  1.87s/it]
 30%|███       | 5672/18627 [3:05:56<5:59:39,  1.67s/it]
 30%|███       | 5673/18627 [3:05:57<5:28:34,  1.52s/it]
 30%|███       | 5674/18627 [3:05:59<6:05:36,  1.69s/it]
 30%|███       | 5675/18627 [3:06:01<6:32:47,  1.82s/it]
 30%|███       | 5676/18627 [3:06:03<6:52:13,  1.91s/it]
 30%|███       | 5677/18627 [3:06:05<7:04:16,  1.97s/it]
 30%|███       | 5678/18627 [3:06:07<6:14:28,  1.74s/it]
 30%|███       | 5679/18627 [3:06:09<6:38:30,  1.85s/it]
 30%|███       | 5680/18627 [3:06:11<6:55:31,  1.93s/it]
                                                        
{'loss': 1.352, 'grad_norm': 7.1880083084106445, 'learning_rate': 4.07271535550877e-06, 'epoch': 0.3}
+
 30%|███       | 5680/18627 [3:06:11<6:55:31,  1.93s/it]
 30%|███       | 5681/18627 [3:06:13<7:07:12,  1.98s/it]
 31%|███       | 5682/18627 [3:06:14<6:15:36,  1.74s/it]
 31%|███       | 5683/18627 [3:06:16<6:39:29,  1.85s/it]
 31%|███       | 5684/18627 [3:06:17<5:52:39,  1.63s/it]
 31%|███       | 5685/18627 [3:06:20<6:22:15,  1.77s/it]
 31%|███       | 5686/18627 [3:06:22<6:43:36,  1.87s/it]
 31%|███       | 5687/18627 [3:06:24<6:57:59,  1.94s/it]
 31%|███       | 5688/18627 [3:06:25<6:10:00,  1.72s/it]
 31%|███       | 5689/18627 [3:06:27<6:37:00,  1.84s/it]
 31%|███       | 5690/18627 [3:06:28<5:23:00,  1.50s/it]
                                                        
{'loss': 1.2552, 'grad_norm': 8.085671424865723, 'learning_rate': 4.0693339785647655e-06, 'epoch': 0.31}
+
 31%|███       | 5690/18627 [3:06:28<5:23:00,  1.50s/it]
 31%|███       | 5691/18627 [3:06:30<6:02:06,  1.68s/it]
 31%|███       | 5692/18627 [3:06:32<6:29:20,  1.81s/it]
 31%|███       | 5693/18627 [3:06:33<5:49:32,  1.62s/it]
 31%|███       | 5694/18627 [3:06:35<6:21:01,  1.77s/it]
 31%|███       | 5695/18627 [3:06:37<6:44:36,  1.88s/it]
 31%|███       | 5696/18627 [3:06:39<7:00:19,  1.95s/it]
 31%|███       | 5697/18627 [3:06:41<6:04:14,  1.69s/it]
 31%|███       | 5698/18627 [3:06:43<6:32:51,  1.82s/it]
 31%|███       | 5699/18627 [3:06:45<6:51:59,  1.91s/it]
 31%|███       | 5700/18627 [3:06:47<7:06:28,  1.98s/it]
                                                        
{'loss': 1.0968, 'grad_norm': 5.260685443878174, 'learning_rate': 4.065947857072701e-06, 'epoch': 0.31}
+
 31%|███       | 5700/18627 [3:06:47<7:06:28,  1.98s/it]
 31%|███       | 5701/18627 [3:06:48<6:16:15,  1.75s/it]
 31%|███       | 5702/18627 [3:06:50<6:39:44,  1.86s/it]
 31%|███       | 5703/18627 [3:06:51<5:57:07,  1.66s/it]
 31%|███       | 5704/18627 [3:06:53<5:26:27,  1.52s/it]
 31%|███       | 5705/18627 [3:06:55<6:03:48,  1.69s/it]
 31%|███       | 5706/18627 [3:06:57<6:32:29,  1.82s/it]
 31%|███       | 5707/18627 [3:06:58<5:20:18,  1.49s/it]
 31%|███       | 5708/18627 [3:07:00<6:01:29,  1.68s/it]
 31%|███       | 5709/18627 [3:07:01<5:29:44,  1.53s/it]
 31%|███       | 5710/18627 [3:07:03<6:07:10,  1.71s/it]
                                                        
{'loss': 1.4507, 'grad_norm': 7.686944484710693, 'learning_rate': 4.0625570012697944e-06, 'epoch': 0.31}
+
 31%|███       | 5710/18627 [3:07:03<6:07:10,  1.71s/it]
 31%|███       | 5711/18627 [3:07:05<6:33:11,  1.83s/it]
 31%|███       | 5712/18627 [3:07:07<6:52:00,  1.91s/it]
 31%|███       | 5713/18627 [3:07:09<7:05:53,  1.98s/it]
 31%|███       | 5714/18627 [3:07:12<7:15:50,  2.03s/it]
 31%|███       | 5715/18627 [3:07:13<6:21:17,  1.77s/it]
 31%|███       | 5716/18627 [3:07:15<6:42:59,  1.87s/it]
 31%|███       | 5717/18627 [3:07:17<6:58:28,  1.94s/it]
 31%|███       | 5718/18627 [3:07:19<7:10:05,  2.00s/it]
 31%|███       | 5719/18627 [3:07:21<7:17:35,  2.03s/it]
 31%|███       | 5720/18627 [3:07:22<6:23:34,  1.78s/it]
                                                        
{'loss': 1.0355, 'grad_norm': 13.07571029663086, 'learning_rate': 4.05916142140758e-06, 'epoch': 0.31}
+
 31%|███       | 5720/18627 [3:07:22<6:23:34,  1.78s/it]
 31%|███       | 5721/18627 [3:07:25<6:47:03,  1.89s/it]
 31%|███       | 5722/18627 [3:07:27<7:01:44,  1.96s/it]
 31%|███       | 5723/18627 [3:07:29<7:11:16,  2.01s/it]
 31%|███       | 5724/18627 [3:07:31<7:18:03,  2.04s/it]
 31%|███       | 5725/18627 [3:07:33<7:24:24,  2.07s/it]
 31%|███       | 5726/18627 [3:07:35<7:28:22,  2.09s/it]
 31%|███       | 5727/18627 [3:07:37<7:28:45,  2.09s/it]
 31%|███       | 5728/18627 [3:07:39<7:31:40,  2.10s/it]
 31%|███       | 5729/18627 [3:07:41<7:32:40,  2.11s/it]
 31%|███       | 5730/18627 [3:07:44<7:33:25,  2.11s/it]
                                                        
{'loss': 0.6393, 'grad_norm': 5.8516058921813965, 'learning_rate': 4.0557611277518725e-06, 'epoch': 0.31}
+
 31%|███       | 5730/18627 [3:07:44<7:33:25,  2.11s/it]
 31%|███       | 5731/18627 [3:07:46<7:33:34,  2.11s/it]
 31%|███       | 5732/18627 [3:07:48<7:35:14,  2.12s/it]
 31%|███       | 5733/18627 [3:07:49<6:35:07,  1.84s/it]
 31%|███       | 5734/18627 [3:07:50<5:53:35,  1.65s/it]
 31%|███       | 5735/18627 [3:07:51<5:24:16,  1.51s/it]
 31%|███       | 5736/18627 [3:07:53<6:02:22,  1.69s/it]
 31%|███       | 5737/18627 [3:07:56<6:28:55,  1.81s/it]
 31%|███       | 5738/18627 [3:07:58<6:48:40,  1.90s/it]
 31%|███       | 5739/18627 [3:08:00<7:02:19,  1.97s/it]
 31%|███       | 5740/18627 [3:08:01<6:12:27,  1.73s/it]
                                                        
{'loss': 1.442, 'grad_norm': 13.10562801361084, 'learning_rate': 4.052356130582738e-06, 'epoch': 0.31}
+
 31%|███       | 5740/18627 [3:08:01<6:12:27,  1.73s/it]
 31%|███       | 5741/18627 [3:08:03<6:36:51,  1.85s/it]
 31%|███       | 5742/18627 [3:08:05<6:55:21,  1.93s/it]
 31%|███       | 5743/18627 [3:08:07<7:07:47,  1.99s/it]
 31%|███       | 5744/18627 [3:08:10<7:17:02,  2.04s/it]
 31%|███       | 5745/18627 [3:08:11<6:22:24,  1.78s/it]
 31%|███       | 5746/18627 [3:08:13<6:43:22,  1.88s/it]
 31%|███       | 5747/18627 [3:08:15<6:57:39,  1.95s/it]
 31%|███       | 5748/18627 [3:08:16<6:08:45,  1.72s/it]
 31%|███       | 5749/18627 [3:08:18<6:34:52,  1.84s/it]
 31%|███       | 5750/18627 [3:08:20<6:52:25,  1.92s/it]
                                                        
{'loss': 1.1442, 'grad_norm': 6.067203044891357, 'learning_rate': 4.04894644019446e-06, 'epoch': 0.31}
+
 31%|███       | 5750/18627 [3:08:20<6:52:25,  1.92s/it]
 31%|███       | 5751/18627 [3:08:22<7:03:30,  1.97s/it]
 31%|███       | 5752/18627 [3:08:24<6:13:28,  1.74s/it]
 31%|███       | 5753/18627 [3:08:25<5:37:42,  1.57s/it]
 31%|███       | 5754/18627 [3:08:27<6:13:08,  1.74s/it]
 31%|███       | 5755/18627 [3:08:29<6:38:25,  1.86s/it]
 31%|███       | 5756/18627 [3:08:30<5:55:32,  1.66s/it]
 31%|███       | 5757/18627 [3:08:32<6:24:28,  1.79s/it]
 31%|███       | 5758/18627 [3:08:35<6:46:35,  1.90s/it]
 31%|███       | 5759/18627 [3:08:37<7:01:44,  1.97s/it]
 31%|███       | 5760/18627 [3:08:39<7:13:04,  2.02s/it]
                                                        
{'loss': 1.2771, 'grad_norm': 5.709597587585449, 'learning_rate': 4.045532066895516e-06, 'epoch': 0.31}
+
 31%|███       | 5760/18627 [3:08:39<7:13:04,  2.02s/it]
 31%|███       | 5761/18627 [3:08:41<7:18:54,  2.05s/it]
 31%|███       | 5762/18627 [3:08:42<6:23:52,  1.79s/it]
 31%|███       | 5763/18627 [3:08:44<6:44:03,  1.88s/it]
 31%|███       | 5764/18627 [3:08:46<7:01:02,  1.96s/it]
 31%|███       | 5765/18627 [3:08:48<7:10:34,  2.01s/it]
 31%|███       | 5766/18627 [3:08:51<7:18:00,  2.04s/it]
 31%|███       | 5767/18627 [3:08:53<7:23:21,  2.07s/it]
 31%|███       | 5768/18627 [3:08:55<7:25:17,  2.08s/it]
 31%|███       | 5769/18627 [3:08:57<7:27:22,  2.09s/it]
 31%|███       | 5770/18627 [3:08:59<7:29:36,  2.10s/it]
                                                        
{'loss': 0.9141, 'grad_norm': 7.8425774574279785, 'learning_rate': 4.042113021008538e-06, 'epoch': 0.31}
+
 31%|███       | 5770/18627 [3:08:59<7:29:36,  2.10s/it]
 31%|███       | 5771/18627 [3:09:01<7:31:11,  2.11s/it]
 31%|███       | 5772/18627 [3:09:03<7:32:49,  2.11s/it]
 31%|███       | 5773/18627 [3:09:05<7:31:38,  2.11s/it]
 31%|███       | 5774/18627 [3:09:08<7:32:38,  2.11s/it]
 31%|███       | 5775/18627 [3:09:10<7:32:30,  2.11s/it]
 31%|███       | 5776/18627 [3:09:12<7:32:16,  2.11s/it]
 31%|███       | 5777/18627 [3:09:14<7:32:33,  2.11s/it]
 31%|███       | 5778/18627 [3:09:16<7:32:54,  2.11s/it]
 31%|███       | 5779/18627 [3:09:18<7:33:14,  2.12s/it]
 31%|███       | 5780/18627 [3:09:20<7:32:18,  2.11s/it]
                                                        
{'loss': 0.6666, 'grad_norm': 7.025063991546631, 'learning_rate': 4.038689312870284e-06, 'epoch': 0.31}
+
 31%|███       | 5780/18627 [3:09:20<7:32:18,  2.11s/it]
 31%|███       | 5781/18627 [3:09:22<7:33:31,  2.12s/it]
 31%|███       | 5782/18627 [3:09:24<7:33:45,  2.12s/it]
 31%|███       | 5783/18627 [3:09:27<7:33:26,  2.12s/it]
 31%|███       | 5784/18627 [3:09:29<7:33:14,  2.12s/it]
 31%|███       | 5785/18627 [3:09:30<6:33:15,  1.84s/it]
 31%|███       | 5786/18627 [3:09:32<6:52:16,  1.93s/it]
 31%|███       | 5787/18627 [3:09:34<7:05:57,  1.99s/it]
 31%|███       | 5788/18627 [3:09:35<6:15:48,  1.76s/it]
 31%|███       | 5789/18627 [3:09:37<6:39:25,  1.87s/it]
 31%|███       | 5790/18627 [3:09:40<6:55:07,  1.94s/it]
                                                        
{'loss': 1.0256, 'grad_norm': 8.906791687011719, 'learning_rate': 4.03526095283161e-06, 'epoch': 0.31}
+
 31%|███       | 5790/18627 [3:09:40<6:55:07,  1.94s/it]
 31%|███       | 5791/18627 [3:09:42<7:06:59,  2.00s/it]
 31%|███       | 5792/18627 [3:09:44<7:13:24,  2.03s/it]
 31%|███       | 5793/18627 [3:09:46<7:20:02,  2.06s/it]
 31%|███       | 5794/18627 [3:09:48<7:23:36,  2.07s/it]
 31%|███       | 5795/18627 [3:09:50<7:26:10,  2.09s/it]
 31%|███       | 5796/18627 [3:09:52<7:27:03,  2.09s/it]
 31%|███       | 5797/18627 [3:09:54<7:28:10,  2.10s/it]
 31%|███       | 5798/18627 [3:09:56<7:28:25,  2.10s/it]
 31%|███       | 5799/18627 [3:09:59<7:29:07,  2.10s/it]
 31%|███       | 5800/18627 [3:10:01<7:30:34,  2.11s/it]
                                                        
{'loss': 0.6632, 'grad_norm': 4.622925758361816, 'learning_rate': 4.031827951257435e-06, 'epoch': 0.31}
+
 31%|███       | 5800/18627 [3:10:01<7:30:34,  2.11s/it]
 31%|███       | 5801/18627 [3:10:03<7:32:02,  2.11s/it]
 31%|███       | 5802/18627 [3:10:05<7:32:41,  2.12s/it]
 31%|███       | 5803/18627 [3:10:06<6:27:07,  1.81s/it]
 31%|███       | 5804/18627 [3:10:08<6:46:52,  1.90s/it]
 31%|███       | 5805/18627 [3:10:10<7:01:08,  1.97s/it]
 31%|███       | 5806/18627 [3:10:12<6:11:24,  1.74s/it]
 31%|███       | 5807/18627 [3:10:14<6:36:23,  1.86s/it]
 31%|███       | 5808/18627 [3:10:16<6:52:46,  1.93s/it]
 31%|███       | 5809/18627 [3:10:17<6:05:06,  1.71s/it]
 31%|███       | 5810/18627 [3:10:19<6:30:50,  1.83s/it]
                                                        
{'loss': 1.2302, 'grad_norm': 7.205989360809326, 'learning_rate': 4.028390318526709e-06, 'epoch': 0.31}
+
 31%|███       | 5810/18627 [3:10:19<6:30:50,  1.83s/it]
 31%|███       | 5811/18627 [3:10:21<6:49:17,  1.92s/it]
 31%|███       | 5812/18627 [3:10:23<7:02:54,  1.98s/it]
 31%|███       | 5813/18627 [3:10:24<6:12:12,  1.74s/it]
 31%|███       | 5814/18627 [3:10:27<6:35:51,  1.85s/it]
 31%|███       | 5815/18627 [3:10:28<5:53:53,  1.66s/it]
 31%|███       | 5816/18627 [3:10:30<6:23:08,  1.79s/it]
 31%|███       | 5817/18627 [3:10:32<6:46:06,  1.90s/it]
 31%|███       | 5818/18627 [3:10:33<6:00:26,  1.69s/it]
 31%|███       | 5819/18627 [3:10:35<6:28:19,  1.82s/it]
 31%|███       | 5820/18627 [3:10:37<6:47:47,  1.91s/it]
                                                        
{'loss': 1.2903, 'grad_norm': 11.43480110168457, 'learning_rate': 4.024948065032385e-06, 'epoch': 0.31}
+
 31%|███       | 5820/18627 [3:10:37<6:47:47,  1.91s/it]
 31%|███▏      | 5821/18627 [3:10:40<7:01:30,  1.97s/it]
 31%|███▏      | 5822/18627 [3:10:42<7:11:04,  2.02s/it]
 31%|███▏      | 5823/18627 [3:10:44<7:16:53,  2.05s/it]
 31%|███▏      | 5824/18627 [3:10:46<7:21:52,  2.07s/it]
 31%|███▏      | 5825/18627 [3:10:48<7:24:42,  2.08s/it]
 31%|███▏      | 5826/18627 [3:10:50<7:25:38,  2.09s/it]
 31%|███▏      | 5827/18627 [3:10:52<7:26:58,  2.10s/it]
 31%|███▏      | 5828/18627 [3:10:54<6:29:37,  1.83s/it]
 31%|███▏      | 5829/18627 [3:10:56<6:48:52,  1.92s/it]
 31%|███▏      | 5830/18627 [3:10:58<7:02:41,  1.98s/it]
                                                        
{'loss': 0.8516, 'grad_norm': 12.686202049255371, 'learning_rate': 4.021501201181386e-06, 'epoch': 0.31}
+
 31%|███▏      | 5830/18627 [3:10:58<7:02:41,  1.98s/it]
 31%|███▏      | 5831/18627 [3:11:00<7:12:24,  2.03s/it]
 31%|███▏      | 5832/18627 [3:11:01<6:19:20,  1.78s/it]
 31%|███▏      | 5833/18627 [3:11:03<6:42:06,  1.89s/it]
 31%|███▏      | 5834/18627 [3:11:05<6:57:23,  1.96s/it]
 31%|███▏      | 5835/18627 [3:11:06<6:02:46,  1.70s/it]
 31%|███▏      | 5836/18627 [3:11:09<6:28:22,  1.82s/it]
 31%|███▏      | 5837/18627 [3:11:11<6:48:07,  1.91s/it]
 31%|███▏      | 5838/18627 [3:11:13<7:01:05,  1.98s/it]
 31%|███▏      | 5839/18627 [3:11:15<7:10:22,  2.02s/it]
 31%|███▏      | 5840/18627 [3:11:16<5:45:47,  1.62s/it]
                                                        
{'loss': 1.0664, 'grad_norm': 5.660851955413818, 'learning_rate': 4.018049737394572e-06, 'epoch': 0.31}
+
 31%|███▏      | 5840/18627 [3:11:16<5:45:47,  1.62s/it]
 31%|███▏      | 5841/18627 [3:11:16<4:46:55,  1.35s/it]
 31%|███▏      | 5842/18627 [3:11:18<5:36:12,  1.58s/it]
 31%|███▏      | 5843/18627 [3:11:21<6:09:40,  1.74s/it]
 31%|███▏      | 5844/18627 [3:11:22<5:34:40,  1.57s/it]
 31%|███▏      | 5845/18627 [3:11:23<5:02:23,  1.42s/it]
 31%|███▏      | 5846/18627 [3:11:25<5:45:11,  1.62s/it]
 31%|███▏      | 5847/18627 [3:11:27<6:15:06,  1.76s/it]
 31%|███▏      | 5848/18627 [3:11:29<6:38:43,  1.87s/it]
 31%|███▏      | 5849/18627 [3:11:30<5:55:08,  1.67s/it]
 31%|███▏      | 5850/18627 [3:11:31<4:54:31,  1.38s/it]
                                                        
{'loss': 1.2384, 'grad_norm': 7.292784214019775, 'learning_rate': 4.014593684106713e-06, 'epoch': 0.31}
+
 31%|███▏      | 5850/18627 [3:11:31<4:54:31,  1.38s/it]
 31%|███▏      | 5851/18627 [3:11:33<5:40:16,  1.60s/it]
 31%|███▏      | 5852/18627 [3:11:35<6:15:05,  1.76s/it]
 31%|███▏      | 5853/18627 [3:11:37<6:37:55,  1.87s/it]
 31%|███▏      | 5854/18627 [3:11:40<6:53:47,  1.94s/it]
 31%|███▏      | 5855/18627 [3:11:42<7:05:01,  2.00s/it]
 31%|███▏      | 5856/18627 [3:11:44<7:12:57,  2.03s/it]
 31%|███▏      | 5857/18627 [3:11:46<7:18:44,  2.06s/it]
 31%|███▏      | 5858/18627 [3:11:48<7:22:51,  2.08s/it]
 31%|███▏      | 5859/18627 [3:11:50<7:26:36,  2.10s/it]
 31%|███▏      | 5860/18627 [3:11:52<7:27:22,  2.10s/it]
                                                        
{'loss': 0.7139, 'grad_norm': 5.596077919006348, 'learning_rate': 4.011133051766451e-06, 'epoch': 0.31}
+
 31%|███▏      | 5860/18627 [3:11:52<7:27:22,  2.10s/it]
 31%|███▏      | 5861/18627 [3:11:54<7:27:47,  2.10s/it]
 31%|███▏      | 5862/18627 [3:11:56<6:29:06,  1.83s/it]
 31%|███▏      | 5863/18627 [3:11:58<6:46:05,  1.91s/it]
 31%|███▏      | 5864/18627 [3:12:00<6:59:36,  1.97s/it]
 31%|███▏      | 5865/18627 [3:12:02<7:07:23,  2.01s/it]
 31%|███▏      | 5866/18627 [3:12:03<6:15:33,  1.77s/it]
 31%|███▏      | 5867/18627 [3:12:05<6:38:38,  1.87s/it]
 32%|███▏      | 5868/18627 [3:12:07<6:54:26,  1.95s/it]
 32%|███▏      | 5869/18627 [3:12:09<7:05:56,  2.00s/it]
 32%|███▏      | 5870/18627 [3:12:11<6:14:01,  1.76s/it]
                                                        
{'loss': 1.1554, 'grad_norm': 14.660816192626953, 'learning_rate': 4.007667850836271e-06, 'epoch': 0.32}
+
 32%|███▏      | 5870/18627 [3:12:11<6:14:01,  1.76s/it]
 32%|███▏      | 5871/18627 [3:12:12<5:37:48,  1.59s/it]
 32%|███▏      | 5872/18627 [3:12:14<6:11:38,  1.75s/it]
 32%|███▏      | 5873/18627 [3:12:16<6:34:55,  1.86s/it]
 32%|███▏      | 5874/18627 [3:12:18<6:51:39,  1.94s/it]
 32%|███▏      | 5875/18627 [3:12:19<6:03:47,  1.71s/it]
 32%|███▏      | 5876/18627 [3:12:20<4:59:58,  1.41s/it]
 32%|███▏      | 5877/18627 [3:12:22<5:43:32,  1.62s/it]
 32%|███▏      | 5878/18627 [3:12:24<6:15:53,  1.77s/it]
 32%|███▏      | 5879/18627 [3:12:26<6:38:12,  1.87s/it]
 32%|███▏      | 5880/18627 [3:12:28<5:55:33,  1.67s/it]
                                                        
{'loss': 1.3381, 'grad_norm': 15.471632957458496, 'learning_rate': 4.004198091792475e-06, 'epoch': 0.32}
+
 32%|███▏      | 5880/18627 [3:12:28<5:55:33,  1.67s/it]
 32%|███▏      | 5881/18627 [3:12:30<6:24:59,  1.81s/it]
 32%|███▏      | 5882/18627 [3:12:32<6:44:19,  1.90s/it]
 32%|███▏      | 5883/18627 [3:12:34<6:58:10,  1.97s/it]
 32%|███▏      | 5884/18627 [3:12:36<7:06:48,  2.01s/it]
 32%|███▏      | 5885/18627 [3:12:38<7:13:25,  2.04s/it]
 32%|███▏      | 5886/18627 [3:12:40<7:18:05,  2.06s/it]
 32%|███▏      | 5887/18627 [3:12:42<7:21:49,  2.08s/it]
 32%|███▏      | 5888/18627 [3:12:45<7:23:32,  2.09s/it]
 32%|███▏      | 5889/18627 [3:12:46<6:25:36,  1.82s/it]
 32%|███▏      | 5890/18627 [3:12:48<6:44:33,  1.91s/it]
                                                        
{'loss': 0.8823, 'grad_norm': 6.381232261657715, 'learning_rate': 4.000723785125142e-06, 'epoch': 0.32}
+
 32%|███▏      | 5890/18627 [3:12:48<6:44:33,  1.91s/it]
 32%|███▏      | 5891/18627 [3:12:49<5:59:20,  1.69s/it]
 32%|███▏      | 5892/18627 [3:12:51<6:28:08,  1.83s/it]
 32%|███▏      | 5893/18627 [3:12:53<6:45:39,  1.91s/it]
 32%|███▏      | 5894/18627 [3:12:54<5:55:48,  1.68s/it]
 32%|███▏      | 5895/18627 [3:12:57<6:23:09,  1.81s/it]
 32%|███▏      | 5896/18627 [3:12:59<6:42:51,  1.90s/it]
 32%|███▏      | 5897/18627 [3:13:01<6:57:32,  1.97s/it]
 32%|███▏      | 5898/18627 [3:13:02<6:07:32,  1.73s/it]
 32%|███▏      | 5899/18627 [3:13:04<6:31:37,  1.85s/it]
 32%|███▏      | 5900/18627 [3:13:06<6:49:00,  1.93s/it]
                                                        
{'loss': 1.3409, 'grad_norm': 5.229235649108887, 'learning_rate': 3.997244941338101e-06, 'epoch': 0.32}
+
 32%|███▏      | 5900/18627 [3:13:06<6:49:00,  1.93s/it]
 32%|███▏      | 5901/18627 [3:13:08<7:01:10,  1.99s/it]
 32%|███▏      | 5902/18627 [3:13:10<7:09:41,  2.03s/it]
 32%|███▏      | 5903/18627 [3:13:13<7:14:49,  2.05s/it]
 32%|███▏      | 5904/18627 [3:13:15<7:19:47,  2.07s/it]
 32%|███▏      | 5905/18627 [3:13:17<7:22:35,  2.09s/it]
 32%|███▏      | 5906/18627 [3:13:18<6:25:27,  1.82s/it]
 32%|███▏      | 5907/18627 [3:13:19<5:45:36,  1.63s/it]
 32%|███▏      | 5908/18627 [3:13:21<6:16:48,  1.78s/it]
 32%|███▏      | 5909/18627 [3:13:22<5:35:25,  1.58s/it]
 32%|███▏      | 5910/18627 [3:13:25<6:10:25,  1.75s/it]
                                                        
{'loss': 1.3034, 'grad_norm': 4.810164928436279, 'learning_rate': 3.993761570948896e-06, 'epoch': 0.32}
+
 32%|███▏      | 5910/18627 [3:13:25<6:10:25,  1.75s/it]
 32%|███▏      | 5911/18627 [3:13:27<6:33:00,  1.85s/it]
 32%|███▏      | 5912/18627 [3:13:29<6:49:31,  1.93s/it]
 32%|███▏      | 5913/18627 [3:13:31<7:02:16,  1.99s/it]
 32%|███▏      | 5914/18627 [3:13:32<6:11:05,  1.75s/it]
 32%|███▏      | 5915/18627 [3:13:34<6:35:07,  1.86s/it]
 32%|███▏      | 5916/18627 [3:13:36<6:51:03,  1.94s/it]
 32%|███▏      | 5917/18627 [3:13:38<7:01:49,  1.99s/it]
 32%|███▏      | 5918/18627 [3:13:41<7:10:15,  2.03s/it]
 32%|███▏      | 5919/18627 [3:13:43<7:16:21,  2.06s/it]
 32%|███▏      | 5920/18627 [3:13:45<7:19:39,  2.08s/it]
                                                        
{'loss': 0.9301, 'grad_norm': 5.827229022979736, 'learning_rate': 3.990273684488759e-06, 'epoch': 0.32}
+
 32%|███▏      | 5920/18627 [3:13:45<7:19:39,  2.08s/it]
 32%|███▏      | 5921/18627 [3:13:47<7:21:31,  2.08s/it]
 32%|███▏      | 5922/18627 [3:13:48<6:24:31,  1.82s/it]
 32%|███▏      | 5923/18627 [3:13:50<6:43:28,  1.91s/it]
 32%|███▏      | 5924/18627 [3:13:52<6:57:35,  1.97s/it]
 32%|███▏      | 5925/18627 [3:13:54<7:06:03,  2.01s/it]
 32%|███▏      | 5926/18627 [3:13:56<6:13:56,  1.77s/it]
 32%|███▏      | 5927/18627 [3:13:58<6:35:31,  1.87s/it]
 32%|███▏      | 5928/18627 [3:14:00<6:51:18,  1.94s/it]
 32%|███▏      | 5929/18627 [3:14:02<7:04:08,  2.00s/it]
 32%|███▏      | 5930/18627 [3:14:03<6:12:35,  1.76s/it]
                                                        
{'loss': 1.2238, 'grad_norm': 13.622220993041992, 'learning_rate': 3.9867812925025734e-06, 'epoch': 0.32}
+
 32%|███▏      | 5930/18627 [3:14:03<6:12:35,  1.76s/it]
 32%|███▏      | 5931/18627 [3:14:05<6:36:02,  1.87s/it]
 32%|███▏      | 5932/18627 [3:14:06<5:22:17,  1.52s/it]
 32%|███▏      | 5933/18627 [3:14:08<5:59:59,  1.70s/it]
 32%|███▏      | 5934/18627 [3:14:10<6:26:14,  1.83s/it]
 32%|███▏      | 5935/18627 [3:14:12<6:43:35,  1.91s/it]
 32%|███▏      | 5936/18627 [3:14:14<6:55:54,  1.97s/it]
 32%|███▏      | 5937/18627 [3:14:17<7:05:45,  2.01s/it]
 32%|███▏      | 5938/18627 [3:14:19<7:13:05,  2.05s/it]
 32%|███▏      | 5939/18627 [3:14:20<6:18:39,  1.79s/it]
 32%|███▏      | 5940/18627 [3:14:22<6:40:21,  1.89s/it]
                                                        
{'loss': 0.9263, 'grad_norm': 10.48565673828125, 'learning_rate': 3.983284405548846e-06, 'epoch': 0.32}
+
 32%|███▏      | 5940/18627 [3:14:22<6:40:21,  1.89s/it]
 32%|███▏      | 5941/18627 [3:14:24<6:54:22,  1.96s/it]
 32%|███▏      | 5942/18627 [3:14:25<6:05:29,  1.73s/it]
 32%|███▏      | 5943/18627 [3:14:27<6:28:33,  1.84s/it]
 32%|███▏      | 5944/18627 [3:14:30<6:46:20,  1.92s/it]
 32%|███▏      | 5945/18627 [3:14:32<6:59:21,  1.98s/it]
 32%|███▏      | 5946/18627 [3:14:33<6:08:52,  1.75s/it]
 32%|███▏      | 5947/18627 [3:14:35<6:33:01,  1.86s/it]
 32%|███▏      | 5948/18627 [3:14:37<6:49:24,  1.94s/it]
 32%|███▏      | 5949/18627 [3:14:39<6:59:52,  1.99s/it]
 32%|███▏      | 5950/18627 [3:14:41<7:07:07,  2.02s/it]
                                                        
{'loss': 1.1506, 'grad_norm': 8.33576488494873, 'learning_rate': 3.97978303419967e-06, 'epoch': 0.32}
+
 32%|███▏      | 5950/18627 [3:14:41<7:07:07,  2.02s/it]
 32%|███▏      | 5951/18627 [3:14:43<7:11:42,  2.04s/it]
 32%|███▏      | 5952/18627 [3:14:46<7:16:09,  2.06s/it]
 32%|███▏      | 5953/18627 [3:14:48<7:21:09,  2.09s/it]
 32%|███▏      | 5954/18627 [3:14:50<7:21:47,  2.09s/it]
 32%|█���█▏      | 5955/18627 [3:14:52<7:23:40,  2.10s/it]
 32%|███▏      | 5956/18627 [3:14:54<7:25:12,  2.11s/it]
 32%|███▏      | 5957/18627 [3:14:56<7:23:50,  2.10s/it]
 32%|███▏      | 5958/18627 [3:14:58<7:25:05,  2.11s/it]
 32%|███▏      | 5959/18627 [3:15:00<7:26:30,  2.11s/it]
 32%|███▏      | 5960/18627 [3:15:03<7:26:40,  2.12s/it]
                                                        
{'loss': 0.6912, 'grad_norm': 4.851088523864746, 'learning_rate': 3.9762771890407e-06, 'epoch': 0.32}
+
 32%|███▏      | 5960/18627 [3:15:03<7:26:40,  2.12s/it]
 32%|███▏      | 5961/18627 [3:15:04<6:28:19,  1.84s/it]
 32%|███▏      | 5962/18627 [3:15:06<6:45:56,  1.92s/it]
 32%|███▏      | 5963/18627 [3:15:08<6:58:37,  1.98s/it]
 32%|███▏      | 5964/18627 [3:15:10<7:07:20,  2.02s/it]
 32%|███▏      | 5965/18627 [3:15:12<7:13:35,  2.05s/it]
 32%|███▏      | 5966/18627 [3:15:13<6:18:21,  1.79s/it]
 32%|███▏      | 5967/18627 [3:15:15<6:39:13,  1.89s/it]
 32%|███▏      | 5968/18627 [3:15:18<6:54:44,  1.97s/it]
 32%|███▏      | 5969/18627 [3:15:20<7:04:55,  2.01s/it]
 32%|███▏      | 5970/18627 [3:15:22<7:11:44,  2.05s/it]
                                                        
{'loss': 1.1464, 'grad_norm': 13.780621528625488, 'learning_rate': 3.972766880671113e-06, 'epoch': 0.32}
+
 32%|███▏      | 5970/18627 [3:15:22<7:11:44,  2.05s/it]
 32%|███▏      | 5971/18627 [3:15:24<7:16:37,  2.07s/it]
 32%|███▏      | 5972/18627 [3:15:26<7:19:26,  2.08s/it]
 32%|███▏      | 5973/18627 [3:15:28<7:21:54,  2.10s/it]
 32%|███▏      | 5974/18627 [3:15:30<7:23:47,  2.10s/it]
 32%|███▏      | 5975/18627 [3:15:32<6:26:54,  1.83s/it]
 32%|███▏      | 5976/18627 [3:15:33<5:45:56,  1.64s/it]
 32%|███▏      | 5977/18627 [3:15:35<6:16:06,  1.78s/it]
 32%|███▏      | 5978/18627 [3:15:37<6:36:15,  1.88s/it]
 32%|███▏      | 5979/18627 [3:15:39<6:49:55,  1.94s/it]
 32%|███▏      | 5980/18627 [3:15:41<7:01:30,  2.00s/it]
                                                        
{'loss': 1.0302, 'grad_norm': 7.191285610198975, 'learning_rate': 3.969252119703583e-06, 'epoch': 0.32}
+
 32%|███▏      | 5980/18627 [3:15:41<7:01:30,  2.00s/it]
 32%|███▏      | 5981/18627 [3:15:43<7:08:27,  2.03s/it]
 32%|███▏      | 5982/18627 [3:15:45<6:15:35,  1.78s/it]
 32%|███▏      | 5983/18627 [3:15:47<6:37:36,  1.89s/it]
 32%|███▏      | 5984/18627 [3:15:49<6:55:56,  1.97s/it]
 32%|███▏      | 5985/18627 [3:15:51<7:04:54,  2.02s/it]
 32%|███▏      | 5986/18627 [3:15:53<7:11:08,  2.05s/it]
 32%|███▏      | 5987/18627 [3:15:55<7:16:19,  2.07s/it]
 32%|███▏      | 5988/18627 [3:15:57<7:19:11,  2.08s/it]
 32%|███▏      | 5989/18627 [3:15:59<6:23:13,  1.82s/it]
 32%|███▏      | 5990/18627 [3:16:00<5:43:34,  1.63s/it]
                                                        
{'loss': 1.3769, 'grad_norm': 13.856903076171875, 'learning_rate': 3.965732916764241e-06, 'epoch': 0.32}
+
 32%|███▏      | 5990/18627 [3:16:00<5:43:34,  1.63s/it]
 32%|███▏      | 5991/18627 [3:16:02<6:14:28,  1.78s/it]
 32%|███▏      | 5992/18627 [3:16:03<5:37:17,  1.60s/it]
 32%|███▏      | 5993/18627 [3:16:04<5:05:59,  1.45s/it]
 32%|███▏      | 5994/18627 [3:16:06<5:48:55,  1.66s/it]
 32%|███▏      | 5995/18627 [3:16:07<5:18:58,  1.52s/it]
 32%|███▏      | 5996/18627 [3:16:10<5:56:59,  1.70s/it]
 32%|███▏      | 5997/18627 [3:16:12<6:23:58,  1.82s/it]
 32%|███▏      | 5998/18627 [3:16:14<6:42:27,  1.91s/it]
 32%|███▏      | 5999/18627 [3:16:16<6:56:09,  1.98s/it]
 32%|███▏      | 6000/18627 [3:16:17<6:05:54,  1.74s/it]
                                                        
{'loss': 1.5762, 'grad_norm': 14.530896186828613, 'learning_rate': 3.962209282492652e-06, 'epoch': 0.32}
+
 32%|███▏      | 6000/18627 [3:16:17<6:05:54,  1.74s/it]/home/zhengduo/workspace/LLaMA-Factory/src/qwen_vl/model/modeling_qwen2_5_vl.py:1979: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
+  with torch.cuda.amp.autocast(dtype=dtype):
+/home/zhengduo/miniconda3/envs/vgllm/lib/python3.10/site-packages/torch/utils/checkpoint.py:87: UserWarning: None of the inputs have requires_grad=True. Gradients will be None
+  warnings.warn(
+
 32%|███▏      | 6001/18627 [3:16:58<47:21:55, 13.51s/it]
 32%|███▏      | 6002/18627 [3:16:59<34:24:11,  9.81s/it]
 32%|███▏      | 6003/18627 [3:17:01<26:18:00,  7.50s/it]
 32%|███▏      | 6004/18627 [3:17:04<20:41:12,  5.90s/it]
 32%|███▏      | 6005/18627 [3:17:06<16:43:17,  4.77s/it]
 32%|███▏      | 6006/18627 [3:17:08<13:56:42,  3.98s/it]
 32%|███▏      | 6007/18627 [3:17:10<12:00:08,  3.42s/it]
 32%|███▏      | 6008/18627 [3:17:12<10:38:29,  3.04s/it]
 32%|███▏      | 6009/18627 [3:17:14<9:40:37,  2.76s/it] 
 32%|███▏      | 6010/18627 [3:17:15<8:01:31,  2.29s/it]
                                                        
{'loss': 1.0321, 'grad_norm': 14.211827278137207, 'learning_rate': 3.958681227541775e-06, 'epoch': 0.32}
+
 32%|███▏      | 6010/18627 [3:17:15<8:01:31,  2.29s/it]
 32%|███▏      | 6011/18627 [3:17:17<7:50:32,  2.24s/it]
 32%|███▏      | 6012/18627 [3:17:19<6:44:01,  1.92s/it]
 32%|███▏      | 6013/18627 [3:17:20<5:57:39,  1.70s/it]
 32%|███▏      | 6014/18627 [3:17:22<6:24:22,  1.83s/it]
 32%|███▏      | 6015/18627 [3:17:24<6:42:41,  1.92s/it]
 32%|███▏      | 6016/18627 [3:17:26<6:54:09,  1.97s/it]
 32%|███▏      | 6017/18627 [3:17:27<5:57:18,  1.70s/it]
 32%|███▏      | 6018/18627 [3:17:29<6:24:03,  1.83s/it]
 32%|███▏      | 6019/18627 [3:17:31<6:41:02,  1.91s/it]
 32%|███▏      | 6020/18627 [3:17:33<5:55:56,  1.69s/it]
                                                        
{'loss': 1.4733, 'grad_norm': 15.23973560333252, 'learning_rate': 3.955148762577934e-06, 'epoch': 0.32}
+
 32%|███▏      | 6020/18627 [3:17:33<5:55:56,  1.69s/it]
 32%|███▏      | 6021/18627 [3:17:35<6:23:39,  1.83s/it]
 32%|███▏      | 6022/18627 [3:17:37<6:41:56,  1.91s/it]
 32%|███▏      | 6023/18627 [3:17:39<6:55:30,  1.98s/it]
 32%|███▏      | 6024/18627 [3:17:41<7:05:18,  2.02s/it]
 32%|███▏      | 6025/18627 [3:17:43<7:11:44,  2.06s/it]
 32%|███▏      | 6026/18627 [3:17:45<7:15:30,  2.07s/it]
 32%|███▏      | 6027/18627 [3:17:48<7:17:53,  2.09s/it]
 32%|███▏      | 6028/18627 [3:17:50<7:20:57,  2.10s/it]
 32%|███▏      | 6029/18627 [3:17:52<7:21:24,  2.10s/it]
 32%|███▏      | 6030/18627 [3:17:53<6:24:13,  1.83s/it]
                                                        
{'loss': 0.8615, 'grad_norm': 14.812249183654785, 'learning_rate': 3.951611898280789e-06, 'epoch': 0.32}
+
 32%|███▏      | 6030/18627 [3:17:53<6:24:13,  1.83s/it]
 32%|███▏      | 6031/18627 [3:17:55<6:42:50,  1.92s/it]
 32%|███▏      | 6032/18627 [3:17:57<6:55:32,  1.98s/it]
 32%|███▏      | 6033/18627 [3:17:59<7:03:45,  2.02s/it]
 32%|███▏      | 6034/18627 [3:18:01<7:09:20,  2.05s/it]
 32%|███▏      | 6035/18627 [3:18:04<7:13:39,  2.07s/it]
 32%|███▏      | 6036/18627 [3:18:05<6:18:11,  1.80s/it]
 32%|███▏      | 6037/18627 [3:18:07<6:37:12,  1.89s/it]
 32%|███▏      | 6038/18627 [3:18:08<5:52:54,  1.68s/it]
 32%|███▏      | 6039/18627 [3:18:10<6:19:56,  1.81s/it]
 32%|███▏      | 6040/18627 [3:18:12<6:38:34,  1.90s/it]
                                                        
{'loss': 1.1625, 'grad_norm': 5.459301948547363, 'learning_rate': 3.948070645343297e-06, 'epoch': 0.32}
+
 32%|███▏      | 6040/18627 [3:18:12<6:38:34,  1.90s/it]
 32%|███▏      | 6041/18627 [3:18:14<6:51:31,  1.96s/it]
 32%|███▏      | 6042/18627 [3:18:16<7:02:11,  2.01s/it]
 32%|███▏      | 6043/18627 [3:18:18<6:10:09,  1.76s/it]
 32%|███▏      | 6044/18627 [3:18:20<6:33:06,  1.87s/it]
 32%|███▏      | 6045/18627 [3:18:22<6:48:30,  1.95s/it]
 32%|███▏      | 6046/18627 [3:18:23<6:00:43,  1.72s/it]
 32%|███▏      | 6047/18627 [3:18:25<6:26:16,  1.84s/it]
 32%|███▏      | 6048/18627 [3:18:26<5:45:01,  1.65s/it]
 32%|███▏      | 6049/18627 [3:18:29<6:16:16,  1.79s/it]
 32%|███▏      | 6050/18627 [3:18:31<6:36:50,  1.89s/it]
                                                        
{'loss': 1.2098, 'grad_norm': 6.634744644165039, 'learning_rate': 3.944525014471684e-06, 'epoch': 0.32}
+
 32%|███▏      | 6050/18627 [3:18:31<6:36:50,  1.89s/it]
 32%|███▏      | 6051/18627 [3:18:33<6:50:04,  1.96s/it]
 32%|███▏      | 6052/18627 [3:18:35<6:58:57,  2.00s/it]
 32%|███▏      | 6053/18627 [3:18:37<7:07:10,  2.04s/it]
 33%|███▎      | 6054/18627 [3:18:39<7:11:09,  2.06s/it]
 33%|███▎      | 6055/18627 [3:18:41<7:13:30,  2.07s/it]
 33%|███▎      | 6056/18627 [3:18:42<6:07:04,  1.75s/it]
 33%|███▎      | 6057/18627 [3:18:44<6:30:18,  1.86s/it]
 33%|███▎      | 6058/18627 [3:18:46<5:48:03,  1.66s/it]
 33%|███▎      | 6059/18627 [3:18:47<5:17:59,  1.52s/it]
 33%|███▎      | 6060/18627 [3:18:49<5:56:34,  1.70s/it]
                                                        
{'loss': 1.2521, 'grad_norm': 9.980705261230469, 'learning_rate': 3.940975016385414e-06, 'epoch': 0.33}
+
 33%|███▎      | 6060/18627 [3:18:49<5:56:34,  1.70s/it]
 33%|███▎      | 6061/18627 [3:18:51<6:22:30,  1.83s/it]
 33%|███▎      | 6062/18627 [3:18:53<6:41:00,  1.91s/it]
 33%|███▎      | 6063/18627 [3:18:55<6:54:19,  1.98s/it]
 33%|███▎      | 6064/18627 [3:18:57<7:03:40,  2.02s/it]
 33%|███▎      | 6065/18627 [3:19:00<7:10:42,  2.06s/it]
 33%|███▎      | 6066/18627 [3:19:02<7:14:58,  2.08s/it]
 33%|███▎      | 6067/18627 [3:19:03<6:19:04,  1.81s/it]
 33%|███▎      | 6068/18627 [3:19:05<6:37:57,  1.90s/it]
 33%|███▎      | 6069/18627 [3:19:07<6:51:19,  1.97s/it]
 33%|███▎      | 6070/18627 [3:19:09<7:02:34,  2.02s/it]
                                                        
{'loss': 0.804, 'grad_norm': 5.953916072845459, 'learning_rate': 3.937420661817149e-06, 'epoch': 0.33}
+
 33%|███▎      | 6070/18627 [3:19:09<7:02:34,  2.02s/it]
 33%|███▎      | 6071/18627 [3:19:11<7:08:01,  2.05s/it]
 33%|███▎      | 6072/18627 [3:19:13<7:13:25,  2.07s/it]
 33%|███▎      | 6073/18627 [3:19:16<7:14:51,  2.08s/it]
 33%|███▎      | 6074/18627 [3:19:18<7:16:37,  2.09s/it]
 33%|███▎      | 6075/18627 [3:19:20<7:18:03,  2.09s/it]
 33%|███▎      | 6076/18627 [3:19:22<7:20:09,  2.10s/it]
 33%|███▎      | 6077/18627 [3:19:24<7:20:06,  2.10s/it]
 33%|███▎      | 6078/18627 [3:19:25<6:22:40,  1.83s/it]
 33%|███▎      | 6079/18627 [3:19:26<5:42:48,  1.64s/it]
 33%|███▎      | 6080/18627 [3:19:28<6:14:36,  1.79s/it]
                                                        
{'loss': 1.0544, 'grad_norm': 6.309021949768066, 'learning_rate': 3.933861961512728e-06, 'epoch': 0.33}
+
 33%|███▎      | 6080/18627 [3:19:29<6:14:36,  1.79s/it]
 33%|███▎      | 6081/18627 [3:19:31<6:37:51,  1.90s/it]
 33%|███▎      | 6082/18627 [3:19:33<6:50:03,  1.96s/it]
 33%|███▎      | 6083/18627 [3:19:34<6:01:04,  1.73s/it]
 33%|███▎      | 6084/18627 [3:19:36<6:24:53,  1.84s/it]
 33%|███▎      | 6085/18627 [3:19:38<6:41:21,  1.92s/it]
 33%|███▎      | 6086/18627 [3:19:40<6:54:43,  1.98s/it]
 33%|███▎      | 6087/18627 [3:19:42<7:02:42,  2.02s/it]
 33%|███▎      | 6088/18627 [3:19:44<6:11:17,  1.78s/it]
 33%|███▎      | 6089/18627 [3:19:46<6:31:09,  1.87s/it]
 33%|███▎      | 6090/18627 [3:19:48<6:45:49,  1.94s/it]
                                                        
{'loss': 1.1879, 'grad_norm': 7.107146263122559, 'learning_rate': 3.930298926231125e-06, 'epoch': 0.33}
+
 33%|███▎      | 6090/18627 [3:19:48<6:45:49,  1.94s/it]
 33%|███▎      | 6091/18627 [3:19:49<5:28:06,  1.57s/it]
 33%|███▎      | 6092/18627 [3:19:51<6:04:27,  1.74s/it]
 33%|███▎      | 6093/18627 [3:19:52<5:29:55,  1.58s/it]
 33%|███▎      | 6094/18627 [3:19:54<6:04:41,  1.75s/it]
 33%|███▎      | 6095/18627 [3:19:56<6:28:46,  1.86s/it]
 33%|███▎      | 6096/18627 [3:19:58<6:44:23,  1.94s/it]
 33%|███▎      | 6097/18627 [3:20:00<6:55:19,  1.99s/it]
 33%|███▎      | 6098/18627 [3:20:02<7:03:18,  2.03s/it]
 33%|███▎      | 6099/18627 [3:20:05<7:07:59,  2.05s/it]
 33%|███▎      | 6100/18627 [3:20:05<5:44:23,  1.65s/it]
                                                        
{'loss': 0.7911, 'grad_norm': 7.152374744415283, 'learning_rate': 3.92673156674442e-06, 'epoch': 0.33}
+
 33%|███▎      | 6100/18627 [3:20:05<5:44:23,  1.65s/it]
 33%|███▎      | 6101/18627 [3:20:07<6:14:25,  1.79s/it]
 33%|███▎      | 6102/18627 [3:20:10<6:35:26,  1.89s/it]
 33%|███▎      | 6103/18627 [3:20:11<5:51:35,  1.68s/it]
 33%|███▎      | 6104/18627 [3:20:12<5:21:00,  1.54s/it]
 33%|███▎      | 6105/18627 [3:20:14<5:56:08,  1.71s/it]
 33%|███▎      | 6106/18627 [3:20:16<6:23:51,  1.84s/it]
 33%|███▎      | 6107/18627 [3:20:18<6:41:20,  1.92s/it]
 33%|███▎      | 6108/18627 [3:20:19<5:55:46,  1.71s/it]
 33%|███▎      | 6109/18627 [3:20:22<6:22:25,  1.83s/it]
 33%|███▎      | 6110/18627 [3:20:24<6:40:11,  1.92s/it]
                                                        
{'loss': 1.2758, 'grad_norm': 7.954428672790527, 'learning_rate': 3.923159893837768e-06, 'epoch': 0.33}
+
 33%|███▎      | 6110/18627 [3:20:24<6:40:11,  1.92s/it]
 33%|███▎      | 6111/18627 [3:20:26<6:52:32,  1.98s/it]
 33%|███▎      | 6112/18627 [3:20:28<7:00:02,  2.01s/it]
 33%|███▎      | 6113/18627 [3:20:30<7:07:29,  2.05s/it]
 33%|███▎      | 6114/18627 [3:20:32<7:11:45,  2.07s/it]
 33%|███▎      | 6115/18627 [3:20:34<7:15:17,  2.09s/it]
 33%|███▎      | 6116/18627 [3:20:36<7:16:09,  2.09s/it]
 33%|███▎      | 6117/18627 [3:20:39<7:18:32,  2.10s/it]
 33%|███▎      | 6118/18627 [3:20:41<7:19:03,  2.11s/it]
 33%|███▎      | 6119/18627 [3:20:43<7:20:32,  2.11s/it]
 33%|███▎      | 6120/18627 [3:20:44<6:23:25,  1.84s/it]
                                                        
{'loss': 0.8328, 'grad_norm': 14.012812614440918, 'learning_rate': 3.919583918309363e-06, 'epoch': 0.33}
+
 33%|███▎      | 6120/18627 [3:20:44<6:23:25,  1.84s/it]
 33%|███▎      | 6121/18627 [3:20:46<6:41:18,  1.93s/it]
 33%|███▎      | 6122/18627 [3:20:48<6:53:11,  1.98s/it]
 33%|███▎      | 6123/18627 [3:20:50<7:00:40,  2.02s/it]
 33%|███▎      | 6124/18627 [3:20:52<7:06:19,  2.05s/it]
 33%|███▎      | 6125/18627 [3:20:54<6:12:55,  1.79s/it]
 33%|███▎      | 6126/18627 [3:20:56<6:33:42,  1.89s/it]
 33%|███▎      | 6127/18627 [3:20:57<5:49:42,  1.68s/it]
 33%|███▎      | 6128/18627 [3:20:59<6:17:21,  1.81s/it]
 33%|███▎      | 6129/18627 [3:21:00<5:07:13,  1.47s/it]
 33%|███▎      | 6130/18627 [3:21:02<5:48:54,  1.68s/it]
                                                        
{'loss': 1.0507, 'grad_norm': 8.462418556213379, 'learning_rate': 3.916003650970408e-06, 'epoch': 0.33}
+
 33%|███▎      | 6130/18627 [3:21:02<5:48:54,  1.68s/it]
 33%|███▎      | 6131/18627 [3:21:04<6:16:44,  1.81s/it]
 33%|███▎      | 6132/18627 [3:21:05<5:38:28,  1.63s/it]
 33%|███▎      | 6133/18627 [3:21:06<5:11:22,  1.50s/it]
 33%|███▎      | 6134/18627 [3:21:08<4:52:11,  1.40s/it]
 33%|███▎      | 6135/18627 [3:21:10<5:36:50,  1.62s/it]
 33%|███▎      | 6136/18627 [3:21:12<6:07:23,  1.76s/it]
 33%|███▎      | 6137/18627 [3:21:14<6:28:59,  1.87s/it]
 33%|███▎      | 6138/18627 [3:21:16<6:45:29,  1.95s/it]
 33%|███▎      | 6139/18627 [3:21:18<6:56:32,  2.00s/it]
 33%|███▎      | 6140/18627 [3:21:19<5:59:27,  1.73s/it]
                                                        
{'loss': 1.5476, 'grad_norm': 16.7008113861084, 'learning_rate': 3.912419102645082e-06, 'epoch': 0.33}
+
 33%|███▎      | 6140/18627 [3:21:19<5:59:27,  1.73s/it]
 33%|███▎      | 6141/18627 [3:21:20<5:25:55,  1.57s/it]
 33%|███▎      | 6142/18627 [3:21:23<6:00:14,  1.73s/it]
 33%|███▎      | 6143/18627 [3:21:24<5:26:20,  1.57s/it]
 33%|███▎      | 6144/18627 [3:21:26<6:01:35,  1.74s/it]
 33%|███▎      | 6145/18627 [3:21:28<6:26:15,  1.86s/it]
 33%|███▎      | 6146/18627 [3:21:30<6:42:16,  1.93s/it]
 33%|███▎      | 6147/18627 [3:21:32<6:53:08,  1.99s/it]
 33%|███▎      | 6148/18627 [3:21:34<7:01:46,  2.03s/it]
 33%|███▎      | 6149/18627 [3:21:37<7:07:00,  2.05s/it]
 33%|███▎      | 6150/18627 [3:21:39<7:11:11,  2.07s/it]
                                                        
{'loss': 1.0944, 'grad_norm': 9.216864585876465, 'learning_rate': 3.9088302841705026e-06, 'epoch': 0.33}
+
 33%|███▎      | 6150/18627 [3:21:39<7:11:11,  2.07s/it]
 33%|███▎      | 6151/18627 [3:21:39<5:45:57,  1.66s/it]
 33%|███▎      | 6152/18627 [3:21:40<4:45:59,  1.38s/it]
 33%|███▎      | 6153/18627 [3:21:41<4:34:30,  1.32s/it]
 33%|███▎      | 6154/18627 [3:21:43<5:24:47,  1.56s/it]
 33%|███▎      | 6155/18627 [3:21:45<5:59:44,  1.73s/it]
 33%|███▎      | 6156/18627 [3:21:48<6:23:13,  1.84s/it]
 33%|███▎      | 6157/18627 [3:21:49<5:42:17,  1.65s/it]
 33%|███▎      | 6158/18627 [3:21:51<6:11:47,  1.79s/it]
 33%|███▎      | 6159/18627 [3:21:53<6:32:20,  1.89s/it]
 33%|███▎      | 6160/18627 [3:21:55<6:46:23,  1.96s/it]
                                                        
{'loss': 1.0736, 'grad_norm': 5.740019798278809, 'learning_rate': 3.9052372063967045e-06, 'epoch': 0.33}
+
 33%|███▎      | 6160/18627 [3:21:55<6:46:23,  1.96s/it]
 33%|███▎      | 6161/18627 [3:21:56<5:58:16,  1.72s/it]
 33%|███▎      | 6162/18627 [3:21:58<6:21:15,  1.84s/it]
 33%|███▎      | 6163/18627 [3:22:00<5:41:15,  1.64s/it]
 33%|███▎      | 6164/18627 [3:22:02<6:10:32,  1.78s/it]
 33%|███▎      | 6165/18627 [3:22:04<6:31:17,  1.88s/it]
 33%|███▎      | 6166/18627 [3:22:06<6:46:27,  1.96s/it]
 33%|███▎      | 6167/18627 [3:22:07<5:58:34,  1.73s/it]
 33%|███▎      | 6168/18627 [3:22:09<6:22:13,  1.84s/it]
 33%|███▎      | 6169/18627 [3:22:11<6:38:59,  1.92s/it]
 33%|███▎      | 6170/18627 [3:22:14<6:52:29,  1.99s/it]
                                                        
{'loss': 1.3061, 'grad_norm': 7.434545993804932, 'learning_rate': 3.901639880186592e-06, 'epoch': 0.33}
+
 33%|███▎      | 6170/18627 [3:22:14<6:52:29,  1.99s/it]
 33%|███▎      | 6171/18627 [3:22:16<7:00:55,  2.03s/it]
 33%|███▎      | 6172/18627 [3:22:18<7:06:29,  2.05s/it]
 33%|███▎      | 6173/18627 [3:22:20<7:11:24,  2.08s/it]
 33%|███▎      | 6174/18627 [3:22:22<7:13:37,  2.09s/it]
 33%|███▎      | 6175/18627 [3:22:24<7:15:35,  2.10s/it]
 33%|███▎      | 6176/18627 [3:22:26<7:16:45,  2.10s/it]
 33%|███▎      | 6177/18627 [3:22:28<7:17:38,  2.11s/it]
 33%|███▎      | 6178/18627 [3:22:31<7:20:20,  2.12s/it]
 33%|███▎      | 6179/18627 [3:22:33<7:20:28,  2.12s/it]
 33%|███▎      | 6180/18627 [3:22:35<7:20:10,  2.12s/it]
                                                        
{'loss': 0.6575, 'grad_norm': 8.90616226196289, 'learning_rate': 3.89803831641592e-06, 'epoch': 0.33}
+
 33%|███▎      | 6180/18627 [3:22:35<7:20:10,  2.12s/it]
 33%|███▎      | 6181/18627 [3:22:37<7:19:52,  2.12s/it]
 33%|███▎      | 6182/18627 [3:22:39<7:18:24,  2.11s/it]
 33%|███▎      | 6183/18627 [3:22:41<7:18:28,  2.11s/it]
 33%|███▎      | 6184/18627 [3:22:42<6:20:29,  1.83s/it]
 33%|███▎      | 6185/18627 [3:22:44<6:37:17,  1.92s/it]
 33%|███▎      | 6186/18627 [3:22:47<6:50:43,  1.98s/it]
 33%|███▎      | 6187/18627 [3:22:49<6:59:29,  2.02s/it]
 33%|███▎      | 6188/18627 [3:22:51<7:05:03,  2.05s/it]
 33%|███▎      | 6189/18627 [3:22:53<7:08:53,  2.07s/it]
 33%|███▎      | 6190/18627 [3:22:55<7:10:55,  2.08s/it]
                                                        
{'loss': 0.9277, 'grad_norm': 5.130674362182617, 'learning_rate': 3.894432525973248e-06, 'epoch': 0.33}
+
 33%|███▎      | 6190/18627 [3:22:55<7:10:55,  2.08s/it]
 33%|███▎      | 6191/18627 [3:22:57<7:13:52,  2.09s/it]
 33%|███▎      | 6192/18627 [3:22:58<6:11:00,  1.79s/it]
 33%|███▎      | 6193/18627 [3:22:59<5:34:48,  1.62s/it]
 33%|███▎      | 6194/18627 [3:23:01<5:07:51,  1.49s/it]
 33%|███▎      | 6195/18627 [3:23:02<4:44:22,  1.37s/it]
 33%|███▎      | 6196/18627 [3:23:03<4:35:34,  1.33s/it]
 33%|███▎      | 6197/18627 [3:23:04<4:28:24,  1.30s/it]
 33%|███▎      | 6198/18627 [3:23:06<5:22:48,  1.56s/it]
 33%|███▎      | 6199/18627 [3:23:08<5:59:03,  1.73s/it]
 33%|███▎      | 6200/18627 [3:23:11<6:23:56,  1.85s/it]
                                                        
{'loss': 1.8148, 'grad_norm': 8.033321380615234, 'learning_rate': 3.8908225197599225e-06, 'epoch': 0.33}
+
 33%|███▎      | 6200/18627 [3:23:11<6:23:56,  1.85s/it]
 33%|███▎      | 6201/18627 [3:23:13<6:40:50,  1.94s/it]
 33%|███▎      | 6202/18627 [3:23:15<6:53:19,  2.00s/it]
 33%|███▎      | 6203/18627 [3:23:17<7:02:08,  2.04s/it]
 33%|███▎      | 6204/18627 [3:23:19<7:10:10,  2.08s/it]
 33%|███▎      | 6205/18627 [3:23:21<7:13:12,  2.09s/it]
 33%|███▎      | 6206/18627 [3:23:22<6:17:24,  1.82s/it]
 33%|███▎      | 6207/18627 [3:23:25<6:35:05,  1.91s/it]
 33%|███▎      | 6208/18627 [3:23:25<5:20:34,  1.55s/it]
 33%|███▎      | 6209/18627 [3:23:27<5:55:22,  1.72s/it]
 33%|███▎      | 6210/18627 [3:23:29<5:23:01,  1.56s/it]
                                                        
{'loss': 1.1482, 'grad_norm': 14.978224754333496, 'learning_rate': 3.887208308690027e-06, 'epoch': 0.33}
+
 33%|███▎      | 6210/18627 [3:23:29<5:23:01,  1.56s/it]
 33%|███▎      | 6211/18627 [3:23:31<5:58:11,  1.73s/it]
 33%|███▎      | 6212/18627 [3:23:33<6:21:48,  1.85s/it]
 33%|███▎      | 6213/18627 [3:23:34<5:40:25,  1.65s/it]
 33%|███▎      | 6214/18627 [3:23:36<6:09:08,  1.78s/it]
 33%|███▎      | 6215/18627 [3:23:38<6:29:43,  1.88s/it]
 33%|███▎      | 6216/18627 [3:23:40<6:43:38,  1.95s/it]
 33%|███▎      | 6217/18627 [3:23:42<6:52:58,  2.00s/it]
 33%|███▎      | 6218/18627 [3:23:45<7:00:28,  2.03s/it]
 33%|███▎      | 6219/18627 [3:23:47<7:04:52,  2.05s/it]
 33%|███▎      | 6220/18627 [3:23:48<6:08:29,  1.78s/it]
                                                        
{'loss': 1.0421, 'grad_norm': 14.250873565673828, 'learning_rate': 3.883589903690363e-06, 'epoch': 0.33}
+
 33%|███▎      | 6220/18627 [3:23:48<6:08:29,  1.78s/it]
 33%|███▎      | 6221/18627 [3:23:50<6:29:02,  1.88s/it]
 33%|███▎      | 6222/18627 [3:23:52<6:42:10,  1.95s/it]
 33%|███▎      | 6223/18627 [3:23:53<5:49:06,  1.69s/it]
 33%|███▎      | 6224/18627 [3:23:55<6:16:01,  1.82s/it]
 33%|███▎      | 6225/18627 [3:23:56<5:36:33,  1.63s/it]
 33%|███▎      | 6226/18627 [3:23:59<6:07:15,  1.78s/it]
 33%|███▎      | 6227/18627 [3:24:00<5:31:01,  1.60s/it]
 33%|███▎      | 6228/18627 [3:24:02<6:03:40,  1.76s/it]
 33%|███▎      | 6229/18627 [3:24:04<6:25:45,  1.87s/it]
 33%|███▎      | 6230/18627 [3:24:06<6:41:53,  1.95s/it]
                                                        
{'loss': 1.2419, 'grad_norm': 6.839808940887451, 'learning_rate': 3.879967315700409e-06, 'epoch': 0.33}
+
 33%|███▎      | 6230/18627 [3:24:06<6:41:53,  1.95s/it]
 33%|███▎      | 6231/18627 [3:24:08<6:53:10,  2.00s/it]
 33%|███▎      | 6232/18627 [3:24:10<7:01:08,  2.04s/it]
 33%|███▎      | 6233/18627 [3:24:12<7:07:06,  2.07s/it]
 33%|███▎      | 6234/18627 [3:24:15<7:10:46,  2.09s/it]
 33%|███▎      | 6235/18627 [3:24:17<7:12:23,  2.09s/it]
 33%|███▎      | 6236/18627 [3:24:19<7:13:32,  2.10s/it]
 33%|███▎      | 6237/18627 [3:24:21<7:14:59,  2.11s/it]
 33%|███▎      | 6238/18627 [3:24:23<7:16:01,  2.11s/it]
 33%|███▎      | 6239/18627 [3:24:24<6:14:06,  1.81s/it]
 33%|███▎      | 6240/18627 [3:24:26<6:35:32,  1.92s/it]
                                                        
{'loss': 0.8643, 'grad_norm': 5.03639554977417, 'learning_rate': 3.876340555672291e-06, 'epoch': 0.33}
+
 33%|█���█▎      | 6240/18627 [3:24:26<6:35:32,  1.92s/it]
 34%|███▎      | 6241/18627 [3:24:29<6:51:38,  1.99s/it]
 34%|███▎      | 6242/18627 [3:24:31<7:00:36,  2.04s/it]
 34%|███▎      | 6243/18627 [3:24:32<6:08:28,  1.79s/it]
 34%|███▎      | 6244/18627 [3:24:34<6:29:21,  1.89s/it]
 34%|███▎      | 6245/18627 [3:24:35<5:45:57,  1.68s/it]
 34%|███▎      | 6246/18627 [3:24:37<6:13:03,  1.81s/it]
 34%|███▎      | 6247/18627 [3:24:38<5:34:46,  1.62s/it]
 34%|███▎      | 6248/18627 [3:24:41<6:05:19,  1.77s/it]
 34%|███▎      | 6249/18627 [3:24:43<6:26:39,  1.87s/it]
 34%|███▎      | 6250/18627 [3:24:45<6:42:00,  1.95s/it]
                                                        
{'loss': 1.3, 'grad_norm': 5.803928375244141, 'learning_rate': 3.872709634570748e-06, 'epoch': 0.34}
+
 34%|███▎      | 6250/18627 [3:24:45<6:42:00,  1.95s/it]
 34%|███▎      | 6251/18627 [3:24:47<6:51:30,  2.00s/it]
 34%|███▎      | 6252/18627 [3:24:49<6:59:12,  2.03s/it]
 34%|███▎      | 6253/18627 [3:24:50<5:56:08,  1.73s/it]
 34%|███▎      | 6254/18627 [3:24:52<6:20:44,  1.85s/it]
 34%|███▎      | 6255/18627 [3:24:54<6:36:54,  1.92s/it]
 34%|███▎      | 6256/18627 [3:24:56<6:49:09,  1.98s/it]
 34%|███▎      | 6257/18627 [3:24:59<6:58:07,  2.03s/it]
 34%|███▎      | 6258/18627 [3:25:01<7:02:48,  2.05s/it]
 34%|███▎      | 6259/18627 [3:25:03<7:05:53,  2.07s/it]
 34%|███▎      | 6260/18627 [3:25:05<7:09:18,  2.08s/it]
                                                        
{'loss': 0.8798, 'grad_norm': 8.988733291625977, 'learning_rate': 3.869074563373099e-06, 'epoch': 0.34}
+
 34%|███▎      | 6260/18627 [3:25:05<7:09:18,  2.08s/it]
 34%|███▎      | 6261/18627 [3:25:06<6:13:39,  1.81s/it]
 34%|███▎      | 6262/18627 [3:25:08<6:33:01,  1.91s/it]
 34%|███▎      | 6263/18627 [3:25:10<6:45:56,  1.97s/it]
 34%|███▎      | 6264/18627 [3:25:12<5:57:43,  1.74s/it]
 34%|███▎      | 6265/18627 [3:25:14<6:21:03,  1.85s/it]
 34%|███▎      | 6266/18627 [3:25:14<5:10:19,  1.51s/it]
 34%|███▎      | 6267/18627 [3:25:15<4:40:45,  1.36s/it]
 34%|███▎      | 6268/18627 [3:25:17<4:30:16,  1.31s/it]
 34%|███▎      | 6269/18627 [3:25:19<5:19:53,  1.55s/it]
 34%|███▎      | 6270/18627 [3:25:21<5:54:44,  1.72s/it]
                                                        
{'loss': 1.434, 'grad_norm': 7.779637336730957, 'learning_rate': 3.865435353069209e-06, 'epoch': 0.34}
+
 34%|███▎      | 6270/18627 [3:25:21<5:54:44,  1.72s/it]
 34%|███▎      | 6271/18627 [3:25:23<6:20:02,  1.85s/it]
 34%|███▎      | 6272/18627 [3:25:25<6:36:59,  1.93s/it]
 34%|███▎      | 6273/18627 [3:25:27<6:48:31,  1.98s/it]
 34%|███▎      | 6274/18627 [3:25:29<6:58:26,  2.03s/it]
 34%|███▎      | 6275/18627 [3:25:30<6:01:07,  1.75s/it]
 34%|███▎      | 6276/18627 [3:25:33<6:24:17,  1.87s/it]
 34%|███▎      | 6277/18627 [3:25:35<6:38:34,  1.94s/it]
 34%|███▎      | 6278/18627 [3:25:37<6:50:06,  1.99s/it]
 34%|███▎      | 6279/18627 [3:25:39<6:57:48,  2.03s/it]
 34%|███▎      | 6280/18627 [3:25:41<7:04:45,  2.06s/it]
                                                        
{'loss': 0.8852, 'grad_norm': 8.458354949951172, 'learning_rate': 3.861792014661458e-06, 'epoch': 0.34}
+
 34%|███▎      | 6280/18627 [3:25:41<7:04:45,  2.06s/it]
 34%|███▎      | 6281/18627 [3:25:42<6:10:35,  1.80s/it]
 34%|███▎      | 6282/18627 [3:25:43<5:32:36,  1.62s/it]
 34%|███▎      | 6283/18627 [3:25:46<6:03:59,  1.77s/it]
 34%|███▎      | 6284/18627 [3:25:48<6:25:12,  1.87s/it]
 34%|███▎      | 6285/18627 [3:25:50<6:40:12,  1.95s/it]
 34%|███▎      | 6286/18627 [3:25:51<5:53:11,  1.72s/it]
 34%|███▍      | 6287/18627 [3:25:52<4:51:14,  1.42s/it]
 34%|███▍      | 6288/18627 [3:25:54<5:33:17,  1.62s/it]
 34%|███▍      | 6289/18627 [3:25:56<6:02:57,  1.77s/it]
 34%|███▍      | 6290/18627 [3:25:57<5:27:52,  1.59s/it]
                                                        
{'loss': 1.4662, 'grad_norm': 15.749419212341309, 'learning_rate': 3.858144559164707e-06, 'epoch': 0.34}
+
 34%|███▍      | 6290/18627 [3:25:57<5:27:52,  1.59s/it]
 34%|███▍      | 6291/18627 [3:25:59<6:01:08,  1.76s/it]
 34%|███▍      | 6292/18627 [3:26:01<6:24:09,  1.87s/it]
 34%|███▍      | 6293/18627 [3:26:03<6:39:07,  1.94s/it]
 34%|███▍      | 6294/18627 [3:26:06<6:50:04,  2.00s/it]
 34%|███▍      | 6295/18627 [3:26:08<6:57:57,  2.03s/it]
 34%|███▍      | 6296/18627 [3:26:10<7:02:20,  2.06s/it]
 34%|███▍      | 6297/18627 [3:26:12<7:05:56,  2.07s/it]
 34%|███▍      | 6298/18627 [3:26:13<5:46:57,  1.69s/it]
 34%|███▍      | 6299/18627 [3:26:15<6:16:26,  1.83s/it]
 34%|███▍      | 6300/18627 [3:26:17<6:36:07,  1.93s/it]
                                                        
{'loss': 0.6994, 'grad_norm': 7.231342792510986, 'learning_rate': 3.854492997606262e-06, 'epoch': 0.34}
+
 34%|███▍      | 6300/18627 [3:26:17<6:36:07,  1.93s/it]
 34%|███▍      | 6301/18627 [3:26:19<6:47:23,  1.98s/it]
 34%|███▍      | 6302/18627 [3:26:21<6:56:31,  2.03s/it]
 34%|███▍      | 6303/18627 [3:26:22<6:04:25,  1.77s/it]
 34%|███▍      | 6304/18627 [3:26:25<6:27:07,  1.88s/it]
 34%|███▍      | 6305/18627 [3:26:26<5:44:03,  1.68s/it]
 34%|███▍      | 6306/18627 [3:26:27<5:07:34,  1.50s/it]
 34%|███▍      | 6307/18627 [3:26:29<5:44:45,  1.68s/it]
 34%|███▍      | 6308/18627 [3:26:30<5:15:03,  1.53s/it]
 34%|███▍      | 6309/18627 [3:26:32<5:51:56,  1.71s/it]
 34%|███▍      | 6310/18627 [3:26:34<6:17:47,  1.84s/it]
                                                        
{'loss': 1.5055, 'grad_norm': 7.197060585021973, 'learning_rate': 3.850837341025846e-06, 'epoch': 0.34}
+
 34%|███▍      | 6310/18627 [3:26:34<6:17:47,  1.84s/it]
 34%|███▍      | 6311/18627 [3:26:37<6:34:13,  1.92s/it]
 34%|███▍      | 6312/18627 [3:26:39<6:47:21,  1.98s/it]
 34%|███▍      | 6313/18627 [3:26:41<6:56:38,  2.03s/it]
 34%|███▍      | 6314/18627 [3:26:43<7:00:33,  2.05s/it]
 34%|███▍      | 6315/18627 [3:26:44<6:07:54,  1.79s/it]
 34%|███▍      | 6316/18627 [3:26:45<5:31:18,  1.61s/it]
 34%|███▍      | 6317/18627 [3:26:47<6:02:37,  1.77s/it]
 34%|███▍      | 6318/18627 [3:26:48<4:57:11,  1.45s/it]
 34%|███▍      | 6319/18627 [3:26:49<4:11:31,  1.23s/it]
 34%|███▍      | 6320/18627 [3:26:51<5:06:51,  1.50s/it]
                                                        
{'loss': 1.1282, 'grad_norm': 6.228940010070801, 'learning_rate': 3.847177600475558e-06, 'epoch': 0.34}
+
 34%|███▍      | 6320/18627 [3:26:51<5:06:51,  1.50s/it]
 34%|███▍      | 6321/18627 [3:26:52<4:47:34,  1.40s/it]
 34%|███▍      | 6322/18627 [3:26:54<5:30:59,  1.61s/it]
 34%|███▍      | 6323/18627 [3:26:56<6:02:00,  1.77s/it]
 34%|███▍      | 6324/18627 [3:26:58<6:23:28,  1.87s/it]
 34%|███▍      | 6325/18627 [3:27:01<6:39:30,  1.95s/it]
 34%|███▍      | 6326/18627 [3:27:03<6:49:27,  2.00s/it]
 34%|███▍      | 6327/18627 [3:27:05<6:57:19,  2.04s/it]
 34%|███▍      | 6328/18627 [3:27:07<7:00:57,  2.05s/it]
 34%|███▍      | 6329/18627 [3:27:09<7:05:16,  2.07s/it]
 34%|███▍      | 6330/18627 [3:27:11<7:08:44,  2.09s/it]
                                                        
{'loss': 0.928, 'grad_norm': 6.368260860443115, 'learning_rate': 3.843513787019847e-06, 'epoch': 0.34}
+
 34%|███▍      | 6330/18627 [3:27:11<7:08:44,  2.09s/it]
 34%|███▍      | 6331/18627 [3:27:13<7:10:43,  2.10s/it]
 34%|███▍      | 6332/18627 [3:27:15<7:11:16,  2.10s/it]
 34%|███▍      | 6333/18627 [3:27:18<7:11:19,  2.11s/it]
 34%|███▍      | 6334/18627 [3:27:20<7:12:28,  2.11s/it]
 34%|███▍      | 6335/18627 [3:27:22<7:13:32,  2.12s/it]
 34%|███▍      | 6336/18627 [3:27:24<7:13:20,  2.12s/it]
 34%|███▍      | 6337/18627 [3:27:25<6:16:17,  1.84s/it]
 34%|███▍      | 6338/18627 [3:27:27<6:33:20,  1.92s/it]
 34%|███▍      | 6339/18627 [3:27:29<6:46:27,  1.98s/it]
 34%|███▍      | 6340/18627 [3:27:31<6:55:48,  2.03s/it]
                                                        
{'loss': 0.7534, 'grad_norm': 6.207856178283691, 'learning_rate': 3.839845911735476e-06, 'epoch': 0.34}
+
 34%|███▍      | 6340/18627 [3:27:31<6:55:48,  2.03s/it]
 34%|███▍      | 6341/18627 [3:27:34<7:00:09,  2.05s/it]
 34%|███▍      | 6342/18627 [3:27:36<7:04:57,  2.08s/it]
 34%|███▍      | 6343/18627 [3:27:38<7:07:10,  2.09s/it]
 34%|███▍      | 6344/18627 [3:27:40<7:09:13,  2.10s/it]
 34%|███▍      | 6345/18627 [3:27:42<7:09:44,  2.10s/it]
 34%|███▍      | 6346/18627 [3:27:43<6:13:50,  1.83s/it]
 34%|███▍      | 6347/18627 [3:27:45<6:31:29,  1.91s/it]
 34%|███▍      | 6348/18627 [3:27:47<6:44:27,  1.98s/it]
 34%|███▍      | 6349/18627 [3:27:50<6:52:23,  2.02s/it]
 34%|███▍      | 6350/18627 [3:27:52<6:58:11,  2.04s/it]
                                                        
{'loss': 0.8441, 'grad_norm': 7.9572553634643555, 'learning_rate': 3.836173985711486e-06, 'epoch': 0.34}
+
 34%|███▍      | 6350/18627 [3:27:52<6:58:11,  2.04s/it]
 34%|███▍      | 6351/18627 [3:27:54<7:02:42,  2.07s/it]
 34%|███▍      | 6352/18627 [3:27:55<6:08:59,  1.80s/it]
 34%|███▍      | 6353/18627 [3:27:57<6:27:20,  1.89s/it]
 34%|███▍      | 6354/18627 [3:27:58<5:44:40,  1.69s/it]
 34%|███▍      | 6355/18627 [3:28:00<6:09:58,  1.81s/it]
 34%|███▍      | 6356/18627 [3:28:03<6:29:46,  1.91s/it]
 34%|███▍      | 6357/18627 [3:28:05<6:42:45,  1.97s/it]
 34%|███▍      | 6358/18627 [3:28:07<6:52:00,  2.01s/it]
 34%|█���█▍      | 6359/18627 [3:28:09<6:58:43,  2.05s/it]
 34%|███▍      | 6360/18627 [3:28:11<7:02:18,  2.07s/it]
                                                        
{'loss': 1.0152, 'grad_norm': 5.6978912353515625, 'learning_rate': 3.832498020049167e-06, 'epoch': 0.34}
+
 34%|███▍      | 6360/18627 [3:28:11<7:02:18,  2.07s/it]
 34%|███▍      | 6361/18627 [3:28:13<7:05:27,  2.08s/it]
 34%|███▍      | 6362/18627 [3:28:15<7:06:25,  2.09s/it]
 34%|███▍      | 6363/18627 [3:28:17<7:08:27,  2.10s/it]
 34%|███▍      | 6364/18627 [3:28:19<7:11:27,  2.11s/it]
 34%|███▍      | 6365/18627 [3:28:22<7:11:34,  2.11s/it]
 34%|███▍      | 6366/18627 [3:28:24<7:11:11,  2.11s/it]
 34%|███▍      | 6367/18627 [3:28:26<7:11:49,  2.11s/it]
 34%|███▍      | 6368/18627 [3:28:27<6:05:35,  1.79s/it]
 34%|███▍      | 6369/18627 [3:28:29<6:28:49,  1.90s/it]
 34%|███▍      | 6370/18627 [3:28:30<5:15:35,  1.54s/it]
                                                        
{'loss': 0.9793, 'grad_norm': 7.680033206939697, 'learning_rate': 3.82881802586202e-06, 'epoch': 0.34}
+
 34%|███▍      | 6370/18627 [3:28:30<5:15:35,  1.54s/it]
 34%|███▍      | 6371/18627 [3:28:32<5:49:57,  1.71s/it]
 34%|███▍      | 6372/18627 [3:28:34<6:16:21,  1.84s/it]
 34%|███▍      | 6373/18627 [3:28:36<6:33:36,  1.93s/it]
 34%|███▍      | 6374/18627 [3:28:38<6:46:14,  1.99s/it]
 34%|███▍      | 6375/18627 [3:28:39<5:57:13,  1.75s/it]
 34%|███▍      | 6376/18627 [3:28:41<5:23:24,  1.58s/it]
 34%|███▍      | 6377/18627 [3:28:43<5:55:40,  1.74s/it]
 34%|███▍      | 6378/18627 [3:28:45<6:18:03,  1.85s/it]
 34%|███▍      | 6379/18627 [3:28:47<6:34:43,  1.93s/it]
 34%|███▍      | 6380/18627 [3:28:49<6:45:28,  1.99s/it]
                                                        
{'loss': 1.0765, 'grad_norm': 5.439462184906006, 'learning_rate': 3.825134014275725e-06, 'epoch': 0.34}
+
 34%|███▍      | 6380/18627 [3:28:49<6:45:28,  1.99s/it]
 34%|███▍      | 6381/18627 [3:28:51<6:53:14,  2.02s/it]
 34%|███▍      | 6382/18627 [3:28:52<5:57:00,  1.75s/it]
 34%|███▍      | 6383/18627 [3:28:54<6:19:13,  1.86s/it]
 34%|███▍      | 6384/18627 [3:28:57<6:35:29,  1.94s/it]
 34%|███▍      | 6385/18627 [3:28:59<6:46:21,  1.99s/it]
 34%|███▍      | 6386/18627 [3:29:00<5:58:47,  1.76s/it]
 34%|███▍      | 6387/18627 [3:29:02<6:20:31,  1.87s/it]
 34%|███▍      | 6388/18627 [3:29:04<6:36:49,  1.95s/it]
 34%|███▍      | 6389/18627 [3:29:06<6:46:38,  1.99s/it]
 34%|███▍      | 6390/18627 [3:29:08<6:53:36,  2.03s/it]
                                                        
{'loss': 1.1338, 'grad_norm': 10.672872543334961, 'learning_rate': 3.821445996428111e-06, 'epoch': 0.34}
+
 34%|███▍      | 6390/18627 [3:29:08<6:53:36,  2.03s/it]
 34%|███▍      | 6391/18627 [3:29:10<6:59:10,  2.06s/it]
 34%|███▍      | 6392/18627 [3:29:12<6:06:03,  1.80s/it]
 34%|███▍      | 6393/18627 [3:29:14<6:25:46,  1.89s/it]
 34%|███▍      | 6394/18627 [3:29:15<5:43:09,  1.68s/it]
 34%|███▍      | 6395/18627 [3:29:16<5:12:46,  1.53s/it]
 34%|███▍      | 6396/18627 [3:29:18<5:48:23,  1.71s/it]
 34%|███▍      | 6397/18627 [3:29:20<6:14:10,  1.84s/it]
 34%|███▍      | 6398/18627 [3:29:22<6:32:34,  1.93s/it]
 34%|███▍      | 6399/18627 [3:29:25<6:44:55,  1.99s/it]
 34%|███▍      | 6400/18627 [3:29:26<5:56:22,  1.75s/it]
                                                        
{'loss': 1.3784, 'grad_norm': 12.346087455749512, 'learning_rate': 3.8177539834691145e-06, 'epoch': 0.34}
+
 34%|███▍      | 6400/18627 [3:29:26<5:56:22,  1.75s/it]
 34%|███▍      | 6401/18627 [3:29:28<6:17:57,  1.85s/it]
 34%|███▍      | 6402/18627 [3:29:30<6:34:18,  1.94s/it]
 34%|███▍      | 6403/18627 [3:29:32<6:46:08,  1.99s/it]
 34%|███▍      | 6404/18627 [3:29:34<6:55:01,  2.04s/it]
 34%|███▍      | 6405/18627 [3:29:36<6:59:55,  2.06s/it]
 34%|███▍      | 6406/18627 [3:29:38<6:07:12,  1.80s/it]
 34%|███▍      | 6407/18627 [3:29:40<6:26:09,  1.90s/it]
 34%|███▍      | 6408/18627 [3:29:42<6:39:05,  1.96s/it]
 34%|███▍      | 6409/18627 [3:29:44<6:48:59,  2.01s/it]
 34%|███▍      | 6410/18627 [3:29:46<6:56:10,  2.04s/it]
                                                        
{'loss': 0.8561, 'grad_norm': 5.221214771270752, 'learning_rate': 3.8140579865607548e-06, 'epoch': 0.34}
+
 34%|███▍      | 6410/18627 [3:29:46<6:56:10,  2.04s/it]
 34%|███▍      | 6411/18627 [3:29:48<7:00:43,  2.07s/it]
 34%|███▍      | 6412/18627 [3:29:50<7:03:21,  2.08s/it]
 34%|███▍      | 6413/18627 [3:29:51<6:02:24,  1.78s/it]
 34%|███▍      | 6414/18627 [3:29:53<5:26:10,  1.60s/it]
 34%|███▍      | 6415/18627 [3:29:55<5:56:25,  1.75s/it]
 34%|███▍      | 6416/18627 [3:29:57<6:18:52,  1.86s/it]
 34%|███▍      | 6417/18627 [3:29:59<6:33:09,  1.93s/it]
 34%|███▍      | 6418/18627 [3:30:01<6:44:51,  1.99s/it]
 34%|███▍      | 6419/18627 [3:30:03<6:52:29,  2.03s/it]
 34%|███▍      | 6420/18627 [3:30:05<6:58:23,  2.06s/it]
                                                        
{'loss': 1.0338, 'grad_norm': 8.814742088317871, 'learning_rate': 3.810358016877092e-06, 'epoch': 0.34}
+
 34%|███▍      | 6420/18627 [3:30:05<6:58:23,  2.06s/it]
 34%|███▍      | 6421/18627 [3:30:07<7:03:15,  2.08s/it]
 34%|███▍      | 6422/18627 [3:30:10<7:04:53,  2.09s/it]
 34%|███▍      | 6423/18627 [3:30:11<6:10:10,  1.82s/it]
 34%|███▍      | 6424/18627 [3:30:13<6:29:47,  1.92s/it]
 34%|███▍      | 6425/18627 [3:30:15<6:40:58,  1.97s/it]
 34%|███▍      | 6426/18627 [3:30:17<6:50:08,  2.02s/it]
 35%|███▍      | 6427/18627 [3:30:19<6:55:30,  2.04s/it]
 35%|███▍      | 6428/18627 [3:30:21<7:00:47,  2.07s/it]
 35%|███▍      | 6429/18627 [3:30:23<7:04:05,  2.09s/it]
 35%|███▍      | 6430/18627 [3:30:26<7:04:56,  2.09s/it]
                                                        
{'loss': 0.8517, 'grad_norm': 7.251826286315918, 'learning_rate': 3.8066540856042e-06, 'epoch': 0.35}
+
 35%|███▍      | 6430/18627 [3:30:26<7:04:56,  2.09s/it]
 35%|███▍      | 6431/18627 [3:30:28<7:07:38,  2.10s/it]
 35%|███▍      | 6432/18627 [3:30:30<7:08:43,  2.11s/it]
 35%|███▍      | 6433/18627 [3:30:32<7:08:30,  2.11s/it]
 35%|███▍      | 6434/18627 [3:30:34<7:07:36,  2.10s/it]
 35%|███▍      | 6435/18627 [3:30:36<7:09:11,  2.11s/it]
 35%|███▍      | 6436/18627 [3:30:38<7:09:36,  2.11s/it]
 35%|███▍      | 6437/18627 [3:30:40<7:08:59,  2.11s/it]
 35%|███▍      | 6438/18627 [3:30:42<6:12:59,  1.84s/it]
 35%|███▍      | 6439/18627 [3:30:44<6:29:33,  1.92s/it]
 35%|███▍      | 6440/18627 [3:30:46<6:42:57,  1.98s/it]
                                                        
{'loss': 0.913, 'grad_norm': 8.360296249389648, 'learning_rate': 3.8029462039401283e-06, 'epoch': 0.35}
+
 35%|███▍      | 6440/18627 [3:30:46<6:42:57,  1.98s/it]
 35%|███▍      | 6441/18627 [3:30:48<6:52:00,  2.03s/it]
 35%|███▍      | 6442/18627 [3:30:50<7:00:38,  2.07s/it]
 35%|███▍      | 6443/18627 [3:30:51<6:06:33,  1.81s/it]
 35%|███▍      | 6444/18627 [3:30:53<6:25:57,  1.90s/it]
 35%|███▍      | 6445/18627 [3:30:56<6:40:04,  1.97s/it]
 35%|███▍      | 6446/18627 [3:30:57<5:52:13,  1.73s/it]
 35%|███▍      | 6447/18627 [3:30:59<6:15:08,  1.85s/it]
 35%|███▍      | 6448/18627 [3:31:01<6:32:15,  1.93s/it]
 35%|███▍      | 6449/18627 [3:31:03<6:43:33,  1.99s/it]
 35%|███▍      | 6450/18627 [3:31:05<6:53:58,  2.04s/it]
                                                        
{'loss': 1.1002, 'grad_norm': 6.797801971435547, 'learning_rate': 3.7992343830948697e-06, 'epoch': 0.35}
+
 35%|███▍      | 6450/18627 [3:31:05<6:53:58,  2.04s/it]
 35%|███▍      | 6451/18627 [3:31:07<6:59:25,  2.07s/it]
 35%|███▍      | 6452/18627 [3:31:10<7:03:51,  2.09s/it]
 35%|███▍      | 6453/18627 [3:31:12<7:04:32,  2.09s/it]
 35%|███▍      | 6454/18627 [3:31:14<7:05:40,  2.10s/it]
 35%|███▍      | 6455/18627 [3:31:16<7:06:51,  2.10s/it]
 35%|███▍      | 6456/18627 [3:31:18<7:06:26,  2.10s/it]
 35%|███▍      | 6457/18627 [3:31:20<7:07:12,  2.11s/it]
 35%|███▍      | 6458/18627 [3:31:21<6:05:29,  1.80s/it]
 35%|███▍      | 6459/18627 [3:31:23<6:23:52,  1.89s/it]
 35%|███▍      | 6460/18627 [3:31:25<6:36:22,  1.95s/it]
                                                        
{'loss': 0.8652, 'grad_norm': 5.847208023071289, 'learning_rate': 3.7955186342903256e-06, 'epoch': 0.35}
+
 35%|███▍      | 6460/18627 [3:31:25<6:36:22,  1.95s/it]
 35%|███▍      | 6461/18627 [3:31:27<6:47:46,  2.01s/it]
 35%|███▍      | 6462/18627 [3:31:29<5:59:02,  1.77s/it]
 35%|███▍      | 6463/18627 [3:31:31<6:21:07,  1.88s/it]
 35%|███▍      | 6464/18627 [3:31:33<6:35:36,  1.95s/it]
 35%|███▍      | 6465/18627 [3:31:35<6:45:45,  2.00s/it]
 35%|███▍      | 6466/18627 [3:31:37<6:52:50,  2.04s/it]
 35%|███▍      | 6467/18627 [3:31:39<6:57:43,  2.06s/it]
 35%|███▍      | 6468/18627 [3:31:41<6:04:44,  1.80s/it]
 35%|███▍      | 6469/18627 [3:31:42<5:27:36,  1.62s/it]
 35%|███▍      | 6470/18627 [3:31:44<5:56:40,  1.76s/it]
                                                        
{'loss': 1.219, 'grad_norm': 7.225777626037598, 'learning_rate': 3.7917989687602747e-06, 'epoch': 0.35}
+
 35%|███▍      | 6470/18627 [3:31:44<5:56:40,  1.76s/it]
 35%|███▍      | 6471/18627 [3:31:45<4:52:41,  1.44s/it]
 35%|███▍      | 6472/18627 [3:31:47<5:32:50,  1.64s/it]
 35%|███▍      | 6473/18627 [3:31:49<6:00:18,  1.78s/it]
 35%|███▍      | 6474/18627 [3:31:51<6:21:37,  1.88s/it]
 35%|███▍      | 6475/18627 [3:31:53<6:35:38,  1.95s/it]
 35%|███▍      | 6476/18627 [3:31:55<6:46:06,  2.01s/it]
 35%|███▍      | 6477/18627 [3:31:57<6:53:31,  2.04s/it]
 35%|███▍      | 6478/18627 [3:31:58<6:02:09,  1.79s/it]
 35%|███▍      | 6479/18627 [3:32:01<6:21:27,  1.88s/it]
 35%|███▍      | 6480/18627 [3:32:02<5:39:34,  1.68s/it]
                                                        
{'loss': 1.0296, 'grad_norm': 15.496307373046875, 'learning_rate': 3.788075397750336e-06, 'epoch': 0.35}
+
 35%|███▍      | 6480/18627 [3:32:02<5:39:34,  1.68s/it]
 35%|███▍      | 6481/18627 [3:32:04<6:05:55,  1.81s/it]
 35%|███▍      | 6482/18627 [3:32:05<5:28:38,  1.62s/it]
 35%|███▍      | 6483/18627 [3:32:06<5:02:59,  1.50s/it]
 35%|███▍      | 6484/18627 [3:32:08<5:40:32,  1.68s/it]
 35%|███▍      | 6485/18627 [3:32:10<6:06:30,  1.81s/it]
 35%|███▍      | 6486/18627 [3:32:13<6:24:38,  1.90s/it]
 35%|███▍      | 6487/18627 [3:32:15<6:38:03,  1.97s/it]
 35%|███▍      | 6488/18627 [3:32:17<6:47:34,  2.01s/it]
 35%|███▍      | 6489/18627 [3:32:19<6:54:25,  2.05s/it]
 35%|███▍      | 6490/18627 [3:32:20<6:01:50,  1.79s/it]
                                                        
{'loss': 1.27, 'grad_norm': 14.892407417297363, 'learning_rate': 3.7843479325179345e-06, 'epoch': 0.35}
+
 35%|███▍      | 6490/18627 [3:32:20<6:01:50,  1.79s/it]
 35%|███▍      | 6491/18627 [3:32:22<6:20:32,  1.88s/it]
 35%|███▍      | 6492/18627 [3:32:24<6:35:39,  1.96s/it]
 35%|███▍      | 6493/18627 [3:32:26<6:46:08,  2.01s/it]
 35%|███▍      | 6494/18627 [3:32:29<6:53:07,  2.04s/it]
 35%|███▍      | 6495/18627 [3:32:30<6:01:30,  1.79s/it]
 35%|███▍      | 6496/18627 [3:32:32<6:21:35,  1.89s/it]
 35%|███▍      | 6497/18627 [3:32:34<6:36:04,  1.96s/it]
 35%|███▍      | 6498/18627 [3:32:36<6:44:45,  2.00s/it]
 35%|███▍      | 6499/18627 [3:32:38<6:51:52,  2.04s/it]
 35%|███▍      | 6500/18627 [3:32:40<6:56:18,  2.06s/it]
                                                        
{'loss': 0.8956, 'grad_norm': 6.993169784545898, 'learning_rate': 3.78061658433227e-06, 'epoch': 0.35}
+
 35%|███▍      | 6500/18627 [3:32:40<6:56:18,  2.06s/it]
 35%|███▍      | 6501/18627 [3:32:42<7:00:08,  2.08s/it]
 35%|███▍      | 6502/18627 [3:32:45<7:02:16,  2.09s/it]
 35%|███▍      | 6503/18627 [3:32:47<7:04:07,  2.10s/it]
 35%|███▍      | 6504/18627 [3:32:48<6:09:25,  1.83s/it]
 35%|███▍      | 6505/18627 [3:32:50<6:26:13,  1.91s/it]
 35%|███▍      | 6506/18627 [3:32:51<5:42:38,  1.70s/it]
 35%|███▍      | 6507/18627 [3:32:52<5:11:49,  1.54s/it]
 35%|███▍      | 6508/18627 [3:32:54<4:50:25,  1.44s/it]
 35%|███▍      | 6509/18627 [3:32:56<5:32:11,  1.64s/it]
 35%|███▍      | 6510/18627 [3:32:58<6:02:11,  1.79s/it]
                                                        
{'loss': 1.4104, 'grad_norm': 7.215968608856201, 'learning_rate': 3.776881364474283e-06, 'epoch': 0.35}
+
 35%|███▍      | 6510/18627 [3:32:58<6:02:11,  1.79s/it]
 35%|███▍      | 6511/18627 [3:33:00<6:21:10,  1.89s/it]
 35%|███▍      | 6512/18627 [3:33:02<6:34:22,  1.95s/it]
 35%|███▍      | 6513/18627 [3:33:04<6:44:42,  2.00s/it]
 35%|███▍      | 6514/18627 [3:33:05<5:52:21,  1.75s/it]
 35%|███▍      | 6515/18627 [3:33:07<6:15:18,  1.86s/it]
 35%|███▍      | 6516/18627 [3:33:10<6:30:02,  1.93s/it]
 35%|███▍      | 6517/18627 [3:33:12<6:41:07,  1.99s/it]
 35%|███▍      | 6518/18627 [3:33:14<6:48:35,  2.02s/it]
 35%|███▍      | 6519/18627 [3:33:16<6:53:34,  2.05s/it]
 35%|███▌      | 6520/18627 [3:33:17<6:01:20,  1.79s/it]
                                                        
{'loss': 1.0045, 'grad_norm': 13.780594825744629, 'learning_rate': 3.7731422842366165e-06, 'epoch': 0.35}
+
 35%|███▌      | 6520/18627 [3:33:17<6:01:20,  1.79s/it]
 35%|███▌      | 6521/18627 [3:33:19<6:21:03,  1.89s/it]
 35%|███▌      | 6522/18627 [3:33:21<6:34:10,  1.95s/it]
 35%|███▌      | 6523/18627 [3:33:23<6:43:49,  2.00s/it]
 35%|███▌      | 6524/18627 [3:33:26<6:50:52,  2.04s/it]
 35%|███▌      | 6525/18627 [3:33:27<5:59:19,  1.78s/it]
 35%|███▌      | 6526/18627 [3:33:28<5:23:38,  1.60s/it]
 35%|███▌      | 6527/18627 [3:33:30<5:54:47,  1.76s/it]
 35%|███▌      | 6528/18627 [3:33:32<6:17:27,  1.87s/it]
 35%|███▌      | 6529/18627 [3:33:34<6:31:56,  1.94s/it]
 35%|███▌      | 6530/18627 [3:33:36<6:42:20,  2.00s/it]
                                                        
{'loss': 1.0188, 'grad_norm': 6.66459321975708, 'learning_rate': 3.7693993549235864e-06, 'epoch': 0.35}
+
 35%|███▌      | 6530/18627 [3:33:36<6:42:20,  2.00s/it]
 35%|███▌      | 6531/18627 [3:33:39<6:51:07,  2.04s/it]
 35%|███▌      | 6532/18627 [3:33:41<6:56:58,  2.07s/it]
 35%|███▌      | 6533/18627 [3:33:43<7:00:21,  2.09s/it]
 35%|███▌      | 6534/18627 [3:33:45<7:02:33,  2.10s/it]
 35%|███▌      | 6535/18627 [3:33:47<7:03:28,  2.10s/it]
 35%|███▌      | 6536/18627 [3:33:49<7:04:29,  2.11s/it]
 35%|███▌      | 6537/18627 [3:33:51<7:04:21,  2.11s/it]
 35%|███▌      | 6538/18627 [3:33:53<7:04:05,  2.10s/it]
 35%|███▌      | 6539/18627 [3:33:55<7:04:58,  2.11s/it]
 35%|███▌      | 6540/18627 [3:33:58<7:05:54,  2.11s/it]
                                                        
{'loss': 0.6912, 'grad_norm': 6.512606143951416, 'learning_rate': 3.765652587851143e-06, 'epoch': 0.35}
+
 35%|███▌      | 6540/18627 [3:33:58<7:05:54,  2.11s/it]
 35%|███▌      | 6541/18627 [3:33:59<6:10:11,  1.84s/it]
 35%|███▌      | 6542/18627 [3:34:00<5:01:11,  1.50s/it]
 35%|███▌      | 6543/18627 [3:34:02<5:38:24,  1.68s/it]
 35%|███▌      | 6544/18627 [3:34:04<6:04:49,  1.81s/it]
 35%|███▌      | 6545/18627 [3:34:06<6:22:43,  1.90s/it]
 35%|███▌      | 6546/18627 [3:34:07<5:39:39,  1.69s/it]
 35%|███▌      | 6547/18627 [3:34:09<6:05:56,  1.82s/it]
 35%|███▌      | 6548/18627 [3:34:10<5:28:47,  1.63s/it]
 35%|███▌      | 6549/18627 [3:34:12<5:58:27,  1.78s/it]
 35%|███▌      | 6550/18627 [3:34:15<6:17:31,  1.88s/it]
                                                        
{'loss': 1.231, 'grad_norm': 5.75010347366333, 'learning_rate': 3.7619019943468432e-06, 'epoch': 0.35}
+
 35%|███▌      | 6550/18627 [3:34:15<6:17:31,  1.88s/it]
 35%|███▌      | 6551/18627 [3:34:17<6:34:11,  1.96s/it]
 35%|███▌      | 6552/18627 [3:34:18<5:48:27,  1.73s/it]
 35%|███▌      | 6553/18627 [3:34:20<6:14:53,  1.86s/it]
 35%|███▌      | 6554/18627 [3:34:22<6:33:28,  1.96s/it]
 35%|███▌      | 6555/18627 [3:34:24<6:46:00,  2.02s/it]
 35%|███▌      | 6556/18627 [3:34:27<6:54:05,  2.06s/it]
 35%|███▌      | 6557/18627 [3:34:29<6:59:33,  2.09s/it]
 35%|███▌      | 6558/18627 [3:34:30<6:06:32,  1.82s/it]
 35%|███▌      | 6559/18627 [3:34:32<6:24:09,  1.91s/it]
 35%|███▌      | 6560/18627 [3:34:34<6:37:18,  1.98s/it]
                                                        
{'loss': 1.0525, 'grad_norm': 6.6261372566223145, 'learning_rate': 3.758147585749809e-06, 'epoch': 0.35}
+
 35%|███▌      | 6560/18627 [3:34:34<6:37:18,  1.98s/it]
 35%|███▌      | 6561/18627 [3:34:36<6:45:10,  2.01s/it]
 35%|███▌      | 6562/18627 [3:34:38<6:51:18,  2.05s/it]
 35%|███▌      | 6563/18627 [3:34:41<6:55:17,  2.07s/it]
 35%|███▌      | 6564/18627 [3:34:43<6:58:25,  2.08s/it]
 35%|███▌      | 6565/18627 [3:34:45<7:01:32,  2.10s/it]
 35%|███▌      | 6566/18627 [3:34:46<6:03:43,  1.81s/it]
 35%|███▌      | 6567/18627 [3:34:48<6:20:48,  1.89s/it]
 35%|███▌      | 6568/18627 [3:34:50<6:35:03,  1.97s/it]
 35%|███▌      | 6569/18627 [3:34:52<6:43:33,  2.01s/it]
 35%|███▌      | 6570/18627 [3:34:54<6:50:00,  2.04s/it]
                                                        
{'loss': 1.0022, 'grad_norm': 7.9663825035095215, 'learning_rate': 3.7543893734106974e-06, 'epoch': 0.35}
+
 35%|███▌      | 6570/18627 [3:34:54<6:50:00,  2.04s/it]
 35%|███▌      | 6571/18627 [3:34:55<5:30:18,  1.64s/it]
 35%|███▌      | 6572/18627 [3:34:56<5:03:03,  1.51s/it]
 35%|███▌      | 6573/18627 [3:34:58<5:40:24,  1.69s/it]
 35%|███▌      | 6574/18627 [3:35:00<5:10:06,  1.54s/it]
 35%|███▌      | 6575/18627 [3:35:02<5:44:09,  1.71s/it]
 35%|███▌      | 6576/18627 [3:35:04<6:09:40,  1.84s/it]
 35%|███▌      | 6577/18627 [3:35:05<5:30:03,  1.64s/it]
 35%|███▌      | 6578/18627 [3:35:07<5:58:53,  1.79s/it]
 35%|███▌      | 6579/18627 [3:35:09<6:19:42,  1.89s/it]
 35%|███▌      | 6580/18627 [3:35:11<6:33:05,  1.96s/it]
                                                        
{'loss': 1.3049, 'grad_norm': 7.838817119598389, 'learning_rate': 3.7506273686916654e-06, 'epoch': 0.35}
+
 35%|███▌      | 6580/18627 [3:35:11<6:33:05,  1.96s/it]
 35%|███▌      | 6581/18627 [3:35:14<6:42:23,  2.00s/it]
 35%|███▌      | 6582/18627 [3:35:16<6:49:16,  2.04s/it]
 35%|███▌      | 6583/18627 [3:35:18<6:54:44,  2.07s/it]
 35%|███▌      | 6584/18627 [3:35:20<6:57:05,  2.08s/it]
 35%|███▌      | 6585/18627 [3:35:22<6:59:49,  2.09s/it]
 35%|███▌      | 6586/18627 [3:35:24<7:01:47,  2.10s/it]
 35%|███▌      | 6587/18627 [3:35:26<7:02:29,  2.11s/it]
 35%|███▌      | 6588/18627 [3:35:28<7:03:38,  2.11s/it]
 35%|███▌      | 6589/18627 [3:35:30<7:04:28,  2.12s/it]
 35%|███▌      | 6590/18627 [3:35:32<6:08:43,  1.84s/it]
                                                        
{'loss': 0.8755, 'grad_norm': 13.217529296875, 'learning_rate': 3.7468615829663357e-06, 'epoch': 0.35}
+
 35%|███▌      | 6590/18627 [3:35:32<6:08:43,  1.84s/it]
 35%|███▌      | 6591/18627 [3:35:34<6:26:12,  1.93s/it]
 35%|███▌      | 6592/18627 [3:35:36<6:37:54,  1.98s/it]
 35%|███▌      | 6593/18627 [3:35:38<6:45:41,  2.02s/it]
 35%|███▌      | 6594/18627 [3:35:39<5:55:17,  1.77s/it]
 35%|███▌      | 6595/18627 [3:35:40<5:13:12,  1.56s/it]
 35%|███▌      | 6596/18627 [3:35:41<4:51:11,  1.45s/it]
 35%|███▌      | 6597/18627 [3:35:44<5:32:03,  1.66s/it]
 35%|███▌      | 6598/18627 [3:35:46<6:00:22,  1.80s/it]
 35%|███▌      | 6599/18627 [3:35:48<6:19:35,  1.89s/it]
 35%|███▌      | 6600/18627 [3:35:50<6:33:32,  1.96s/it]
                                                        
{'loss': 1.2115, 'grad_norm': 6.292603015899658, 'learning_rate': 3.7430920276197623e-06, 'epoch': 0.35}
+
 35%|███▌      | 6600/18627 [3:35:50<6:33:32,  1.96s/it]
 35%|███▌      | 6601/18627 [3:35:52<6:42:06,  2.01s/it]
 35%|███▌      | 6602/18627 [3:35:54<6:49:43,  2.04s/it]
 35%|███▌      | 6603/18627 [3:35:55<5:58:36,  1.79s/it]
 35%|███▌      | 6604/18627 [3:35:57<5:23:06,  1.61s/it]
 35%|███▌      | 6605/18627 [3:35:59<5:53:51,  1.77s/it]
 35%|███▌      | 6606/18627 [3:36:01<6:15:21,  1.87s/it]
 35%|███▌      | 6607/18627 [3:36:03<6:31:07,  1.95s/it]
 35%|███▌      | 6608/18627 [3:36:05<6:40:42,  2.00s/it]
 35%|███▌      | 6609/18627 [3:36:07<6:47:47,  2.04s/it]
 35%|███▌      | 6610/18627 [3:36:09<6:51:54,  2.06s/it]
                                                        
{'loss': 1.094, 'grad_norm': 7.872997760772705, 'learning_rate': 3.739318714048395e-06, 'epoch': 0.35}
+
 35%|███▌      | 6610/18627 [3:36:09<6:51:54,  2.06s/it]
 35%|███▌      | 6611/18627 [3:36:11<6:56:34,  2.08s/it]
 35%|███▌      | 6612/18627 [3:36:14<6:59:31,  2.10s/it]
 36%|███▌      | 6613/18627 [3:36:16<6:59:40,  2.10s/it]
 36%|███▌      | 6614/18627 [3:36:17<6:05:12,  1.82s/it]
 36%|███▌      | 6615/18627 [3:36:19<6:22:33,  1.91s/it]
 36%|███▌      | 6616/18627 [3:36:21<6:35:41,  1.98s/it]
 36%|███▌      | 6617/18627 [3:36:23<6:45:16,  2.02s/it]
 36%|███▌      | 6618/18627 [3:36:25<6:50:40,  2.05s/it]
 36%|███▌      | 6619/18627 [3:36:27<5:59:34,  1.80s/it]
 36%|███▌      | 6620/18627 [3:36:29<6:18:43,  1.89s/it]
                                                        
{'loss': 1.0761, 'grad_norm': 5.330392360687256, 'learning_rate': 3.735541653660046e-06, 'epoch': 0.36}
+
 36%|███▌      | 6620/18627 [3:36:29<6:18:43,  1.89s/it]
 36%|███▌      | 6621/18627 [3:36:30<5:29:40,  1.65s/it]
 36%|███▌      | 6622/18627 [3:36:32<5:58:43,  1.79s/it]
 36%|███▌      | 6623/18627 [3:36:34<6:17:04,  1.88s/it]
 36%|███▌      | 6624/18627 [3:36:36<6:31:03,  1.95s/it]
 36%|███▌      | 6625/18627 [3:36:37<5:44:59,  1.72s/it]
 36%|███▌      | 6626/18627 [3:36:39<5:12:44,  1.56s/it]
 36%|███▌      | 6627/18627 [3:36:41<5:45:33,  1.73s/it]
 36%|███▌      | 6628/18627 [3:36:43<6:07:29,  1.84s/it]
 36%|███▌      | 6629/18627 [3:36:45<6:24:26,  1.92s/it]
 36%|███▌      | 6630/18627 [3:36:47<6:36:17,  1.98s/it]
                                                        
{'loss': 1.2746, 'grad_norm': 5.569629192352295, 'learning_rate': 3.7317608578738553e-06, 'epoch': 0.36}
+
 36%|███▌      | 6630/18627 [3:36:47<6:36:17,  1.98s/it]
 36%|███▌      | 6631/18627 [3:36:49<6:44:50,  2.02s/it]
 36%|███▌      | 6632/18627 [3:36:51<6:51:15,  2.06s/it]
 36%|███▌      | 6633/18627 [3:36:53<6:55:07,  2.08s/it]
 36%|███▌      | 6634/18627 [3:36:55<6:58:19,  2.09s/it]
 36%|███▌      | 6635/18627 [3:36:58<7:00:16,  2.10s/it]
 36%|███▌      | 6636/18627 [3:37:00<7:01:14,  2.11s/it]
 36%|███▌      | 6637/18627 [3:37:02<7:01:17,  2.11s/it]
 36%|███▌      | 6638/18627 [3:37:04<7:02:18,  2.11s/it]
 36%|███▌      | 6639/18627 [3:37:06<7:02:09,  2.11s/it]
 36%|███▌      | 6640/18627 [3:37:07<6:06:57,  1.84s/it]
                                                        
{'loss': 0.9218, 'grad_norm': 15.43908405303955, 'learning_rate': 3.727976338120256e-06, 'epoch': 0.36}
+
 36%|███▌      | 6640/18627 [3:37:07<6:06:57,  1.84s/it]
 36%|███▌      | 6641/18627 [3:37:09<6:24:31,  1.92s/it]
 36%|███▌      | 6642/18627 [3:37:11<6:35:55,  1.98s/it]
 36%|███▌      | 6643/18627 [3:37:14<6:42:34,  2.02s/it]
 36%|███▌      | 6644/18627 [3:37:16<6:49:42,  2.05s/it]
 36%|███▌      | 6645/18627 [3:37:17<5:57:54,  1.79s/it]
 36%|███▌      | 6646/18627 [3:37:19<6:18:46,  1.90s/it]
 36%|███▌      | 6647/18627 [3:37:21<6:32:13,  1.96s/it]
 36%|███▌      | 6648/18627 [3:37:23<6:41:16,  2.01s/it]
 36%|███▌      | 6649/18627 [3:37:24<5:23:47,  1.62s/it]
 36%|███▌      | 6650/18627 [3:37:26<5:53:31,  1.77s/it]
                                                        
{'loss': 0.8843, 'grad_norm': 7.962451934814453, 'learning_rate': 3.7241881058409413e-06, 'epoch': 0.36}
+
 36%|███▌      | 6650/18627 [3:37:26<5:53:31,  1.77s/it]
 36%|███▌      | 6651/18627 [3:37:28<6:15:15,  1.88s/it]
 36%|███▌      | 6652/18627 [3:37:30<6:31:03,  1.96s/it]
 36%|███▌      | 6653/18627 [3:37:33<6:40:35,  2.01s/it]
 36%|███▌      | 6654/18627 [3:37:35<6:47:30,  2.04s/it]
 36%|███▌      | 6655/18627 [3:37:37<6:51:02,  2.06s/it]
 36%|███▌      | 6656/18627 [3:37:39<6:54:46,  2.08s/it]
 36%|███▌      | 6657/18627 [3:37:41<6:57:47,  2.09s/it]
 36%|███▌      | 6658/18627 [3:37:43<6:58:05,  2.10s/it]
 36%|███▌      | 6659/18627 [3:37:45<7:00:05,  2.11s/it]
 36%|███▌      | 6660/18627 [3:37:47<7:00:44,  2.11s/it]
                                                        
{'loss': 0.6699, 'grad_norm': 7.071935653686523, 'learning_rate': 3.720396172488825e-06, 'epoch': 0.36}
+
 36%|███▌      | 6660/18627 [3:37:47<7:00:44,  2.11s/it]
 36%|███▌      | 6661/18627 [3:37:49<7:01:36,  2.11s/it]
 36%|███▌      | 6662/18627 [3:37:52<7:02:19,  2.12s/it]
 36%|███▌      | 6663/18627 [3:37:54<7:01:36,  2.11s/it]
 36%|███▌      | 6664/18627 [3:37:56<7:01:45,  2.12s/it]
 36%|███▌      | 6665/18627 [3:37:57<6:06:42,  1.84s/it]
 36%|███▌      | 6666/18627 [3:37:59<6:24:23,  1.93s/it]
 36%|███▌      | 6667/18627 [3:38:01<6:34:54,  1.98s/it]
 36%|███▌      | 6668/18627 [3:38:03<6:43:21,  2.02s/it]
 36%|███▌      | 6669/18627 [3:38:06<6:49:21,  2.05s/it]
 36%|███▌      | 6670/18627 [3:38:08<6:53:46,  2.08s/it]
                                                        
{'loss': 0.8456, 'grad_norm': 4.685774803161621, 'learning_rate': 3.7166005495280144e-06, 'epoch': 0.36}
+
 36%|███▌      | 6670/18627 [3:38:08<6:53:46,  2.08s/it]
 36%|███▌      | 6671/18627 [3:38:10<6:56:11,  2.09s/it]
 36%|███▌      | 6672/18627 [3:38:12<6:58:08,  2.10s/it]
 36%|███▌      | 6673/18627 [3:38:14<6:59:51,  2.11s/it]
 36%|███▌      | 6674/18627 [3:38:16<7:01:17,  2.11s/it]
 36%|███▌      | 6675/18627 [3:38:18<7:02:05,  2.12s/it]
 36%|███▌      | 6676/18627 [3:38:19<6:07:24,  1.84s/it]
 36%|███▌      | 6677/18627 [3:38:22<6:25:21,  1.93s/it]
 36%|███▌      | 6678/18627 [3:38:24<6:37:33,  2.00s/it]
 36%|███▌      | 6679/18627 [3:38:26<6:45:17,  2.04s/it]
 36%|███▌      | 6680/18627 [3:38:28<6:49:19,  2.06s/it]
                                                        
{'loss': 0.8254, 'grad_norm': 8.111054420471191, 'learning_rate': 3.7128012484337687e-06, 'epoch': 0.36}
+
 36%|███▌      | 6680/18627 [3:38:28<6:49:19,  2.06s/it]
 36%|███▌      | 6681/18627 [3:38:29<5:57:36,  1.80s/it]
 36%|███▌      | 6682/18627 [3:38:31<6:16:45,  1.89s/it]
 36%|███▌      | 6683/18627 [3:38:33<6:30:25,  1.96s/it]
 36%|███▌      | 6684/18627 [3:38:36<6:39:36,  2.01s/it]
 36%|███▌      | 6685/18627 [3:38:37<5:50:25,  1.76s/it]
 36%|███▌      | 6686/18627 [3:38:39<6:13:10,  1.88s/it]
 36%|███▌      | 6687/18627 [3:38:40<5:32:07,  1.67s/it]
 36%|███▌      | 6688/18627 [3:38:41<4:36:39,  1.39s/it]
 36%|███▌      | 6689/18627 [3:38:43<5:19:49,  1.61s/it]
 36%|███▌      | 6690/18627 [3:38:45<5:50:14,  1.76s/it]
                                                        
{'loss': 1.295, 'grad_norm': 5.9869585037231445, 'learning_rate': 3.7089982806924677e-06, 'epoch': 0.36}
+
 36%|███▌      | 6690/18627 [3:38:45<5:50:14,  1.76s/it]
 36%|███▌      | 6691/18627 [3:38:47<6:12:30,  1.87s/it]
 36%|███▌      | 6692/18627 [3:38:48<5:32:13,  1.67s/it]
 36%|███▌      | 6693/18627 [3:38:50<5:59:15,  1.81s/it]
 36%|███▌      | 6694/18627 [3:38:52<5:22:57,  1.62s/it]
 36%|███▌      | 6695/18627 [3:38:54<6:20:36,  1.91s/it]
 36%|███▌      | 6696/18627 [3:38:56<6:31:51,  1.97s/it]
 36%|███▌      | 6697/18627 [3:38:58<6:40:03,  2.01s/it]
 36%|███▌      | 6698/18627 [3:39:00<5:51:26,  1.77s/it]
 36%|███▌      | 6699/18627 [3:39:02<6:11:36,  1.87s/it]
 36%|███▌      | 6700/18627 [3:39:04<6:26:46,  1.95s/it]
                                                        
{'loss': 1.4206, 'grad_norm': 6.055211067199707, 'learning_rate': 3.705191657801578e-06, 'epoch': 0.36}
+
 36%|███▌      | 6700/18627 [3:39:04<6:26:46,  1.95s/it]
 36%|███▌      | 6701/18627 [3:39:06<6:35:52,  1.99s/it]
 36%|███▌      | 6702/18627 [3:39:08<6:43:47,  2.03s/it]
 36%|███▌      | 6703/18627 [3:39:10<6:50:17,  2.06s/it]
 36%|███▌      | 6704/18627 [3:39:12<6:53:56,  2.08s/it]
 36%|███▌      | 6705/18627 [3:39:15<6:57:01,  2.10s/it]
 36%|███▌      | 6706/18627 [3:39:17<6:58:59,  2.11s/it]
 36%|███▌      | 6707/18627 [3:39:19<6:59:44,  2.11s/it]
 36%|███▌      | 6708/18627 [3:39:21<7:00:24,  2.12s/it]
 36%|███▌      | 6709/18627 [3:39:23<7:00:04,  2.11s/it]
 36%|███▌      | 6710/18627 [3:39:24<6:05:17,  1.84s/it]
                                                        
{'loss': 0.9116, 'grad_norm': 15.921135902404785, 'learning_rate': 3.7013813912696137e-06, 'epoch': 0.36}
+
 36%|███▌      | 6710/18627 [3:39:24<6:05:17,  1.84s/it]
 36%|███▌      | 6711/18627 [3:39:26<6:22:39,  1.93s/it]
 36%|███▌      | 6712/18627 [3:39:28<6:33:58,  1.98s/it]
 36%|███▌      | 6713/18627 [3:39:31<6:42:37,  2.03s/it]
 36%|███▌      | 6714/18627 [3:39:33<6:46:25,  2.05s/it]
 36%|███▌      | 6715/18627 [3:39:35<6:51:10,  2.07s/it]
 36%|███▌      | 6716/18627 [3:39:37<6:53:02,  2.08s/it]
 36%|███▌      | 6717/18627 [3:39:38<6:00:02,  1.81s/it]
 36%|███▌      | 6718/18627 [3:39:40<6:16:52,  1.90s/it]
 36%|███▌      | 6719/18627 [3:39:42<6:28:56,  1.96s/it]
 36%|███▌      | 6720/18627 [3:39:44<6:39:02,  2.01s/it]
                                                        
{'loss': 0.8416, 'grad_norm': 6.481049537658691, 'learning_rate': 3.697567492616109e-06, 'epoch': 0.36}
+
 36%|███▌      | 6720/18627 [3:39:44<6:39:02,  2.01s/it]
 36%|███▌      | 6721/18627 [3:39:45<5:21:35,  1.62s/it]
 36%|███▌      | 6722/18627 [3:39:47<5:51:41,  1.77s/it]
 36%|███▌      | 6723/18627 [3:39:48<4:48:34,  1.45s/it]
 36%|███▌      | 6724/18627 [3:39:50<5:27:01,  1.65s/it]
 36%|███▌      | 6725/18627 [3:39:52<5:54:49,  1.79s/it]
 36%|███▌      | 6726/18627 [3:39:54<6:15:32,  1.89s/it]
 36%|███▌      | 6727/18627 [3:39:56<6:28:55,  1.96s/it]
 36%|███▌      | 6728/18627 [3:39:59<6:36:48,  2.00s/it]
 36%|███▌      | 6729/18627 [3:40:00<5:48:18,  1.76s/it]
 36%|███▌      | 6730/18627 [3:40:02<6:08:09,  1.86s/it]
                                                        
{'loss': 0.8294, 'grad_norm': 5.300196647644043, 'learning_rate': 3.6937499733715754e-06, 'epoch': 0.36}
+
 36%|███▌      | 6730/18627 [3:40:02<6:08:09,  1.86s/it]
 36%|███▌      | 6731/18627 [3:40:04<6:23:55,  1.94s/it]
 36%|███▌      | 6732/18627 [3:40:06<6:34:41,  1.99s/it]
 36%|███▌      | 6733/18627 [3:40:08<6:41:52,  2.03s/it]
 36%|███▌      | 6734/18627 [3:40:10<6:47:29,  2.06s/it]
 36%|███▌      | 6735/18627 [3:40:11<5:55:39,  1.79s/it]
 36%|███▌      | 6736/18627 [3:40:14<6:15:39,  1.90s/it]
 36%|███▌      | 6737/18627 [3:40:15<5:34:21,  1.69s/it]
 36%|███▌      | 6738/18627 [3:40:17<6:00:07,  1.82s/it]
 36%|███▌      | 6739/18627 [3:40:18<5:23:03,  1.63s/it]
 36%|███▌      | 6740/18627 [3:40:20<5:50:55,  1.77s/it]
                                                        
{'loss': 1.213, 'grad_norm': 7.917925834655762, 'learning_rate': 3.6899288450774713e-06, 'epoch': 0.36}
+
 36%|███▌      | 6740/18627 [3:40:20<5:50:55,  1.77s/it]
 36%|███▌      | 6741/18627 [3:40:22<6:13:22,  1.88s/it]
 36%|███▌      | 6742/18627 [3:40:24<5:32:38,  1.68s/it]
 36%|███▌      | 6743/18627 [3:40:25<5:03:15,  1.53s/it]
 36%|███▌      | 6744/18627 [3:40:27<5:38:20,  1.71s/it]
 36%|███▌      | 6745/18627 [3:40:29<6:01:35,  1.83s/it]
 36%|███▌      | 6746/18627 [3:40:31<6:18:23,  1.91s/it]
 36%|███▌      | 6747/18627 [3:40:33<6:32:32,  1.98s/it]
 36%|███▌      | 6748/18627 [3:40:34<5:45:51,  1.75s/it]
 36%|███▌      | 6749/18627 [3:40:37<6:07:51,  1.86s/it]
 36%|███▌      | 6750/18627 [3:40:38<5:28:38,  1.66s/it]
                                                        
{'loss': 1.3886, 'grad_norm': 13.216570854187012, 'learning_rate': 3.6861041192861667e-06, 'epoch': 0.36}
+
 36%|███▌      | 6750/18627 [3:40:38<5:28:38,  1.66s/it]
 36%|███▌      | 6751/18627 [3:40:40<5:56:12,  1.80s/it]
 36%|███▌      | 6752/18627 [3:40:42<6:16:30,  1.90s/it]
 36%|███▋      | 6753/18627 [3:40:44<6:29:29,  1.97s/it]
 36%|███▋      | 6754/18627 [3:40:45<5:34:29,  1.69s/it]
 36%|███▋      | 6755/18627 [3:40:47<5:59:20,  1.82s/it]
 36%|███▋      | 6756/18627 [3:40:49<6:18:02,  1.91s/it]
 36%|███▋      | 6757/18627 [3:40:52<6:30:56,  1.98s/it]
 36%|███▋      | 6758/18627 [3:40:54<6:39:36,  2.02s/it]
 36%|███▋      | 6759/18627 [3:40:55<5:50:35,  1.77s/it]
 36%|███▋      | 6760/18627 [3:40:57<6:12:14,  1.88s/it]
                                                        
{'loss': 1.0481, 'grad_norm': 6.413755893707275, 'learning_rate': 3.6822758075609093e-06, 'epoch': 0.36}
+
 36%|███▋      | 6760/18627 [3:40:57<6:12:14,  1.88s/it]
 36%|███▋      | 6761/18627 [3:40:58<5:30:57,  1.67s/it]
 36%|███▋      | 6762/18627 [3:41:00<5:56:55,  1.80s/it]
 36%|███▋      | 6763/18627 [3:41:02<6:14:20,  1.89s/it]
 36%|███▋      | 6764/18627 [3:41:05<6:28:22,  1.96s/it]
 36%|███▋      | 6765/18627 [3:41:07<6:38:38,  2.02s/it]
 36%|███▋      | 6766/18627 [3:41:08<5:50:00,  1.77s/it]
 36%|███▋      | 6767/18627 [3:41:09<5:15:58,  1.60s/it]
 36%|███▋      | 6768/18627 [3:41:10<4:51:40,  1.48s/it]
 36%|███▋      | 6769/18627 [3:41:12<5:30:46,  1.67s/it]
 36%|███▋      | 6770/18627 [3:41:15<5:56:48,  1.81s/it]
                                                        
{'loss': 1.2706, 'grad_norm': 4.810872554779053, 'learning_rate': 3.6784439214757862e-06, 'epoch': 0.36}
+
 36%|███▋      | 6770/18627 [3:41:15<5:56:48,  1.81s/it]
 36%|███▋      | 6771/18627 [3:41:17<6:14:34,  1.90s/it]
 36%|███▋      | 6772/18627 [3:41:19<6:28:25,  1.97s/it]
 36%|███▋      | 6773/18627 [3:41:20<5:42:11,  1.73s/it]
 36%|███▋      | 6774/18627 [3:41:22<6:05:03,  1.85s/it]
 36%|███▋      | 6775/18627 [3:41:23<5:25:54,  1.65s/it]
 36%|███▋      | 6776/18627 [3:41:24<4:58:53,  1.51s/it]
 36%|███▋      | 6777/18627 [3:41:27<5:33:52,  1.69s/it]
 36%|███▋      | 6778/18627 [3:41:29<5:58:16,  1.81s/it]
 36%|███▋      | 6779/18627 [3:41:30<5:20:48,  1.62s/it]
 36%|███▋      | 6780/18627 [3:41:32<5:51:07,  1.78s/it]
                                                        
{'loss': 1.3631, 'grad_norm': 5.1552276611328125, 'learning_rate': 3.6746084726156906e-06, 'epoch': 0.36}
+
 36%|███▋      | 6780/18627 [3:41:32<5:51:07,  1.78s/it]
 36%|███▋      | 6781/18627 [3:41:34<6:12:23,  1.89s/it]
 36%|███▋      | 6782/18627 [3:41:35<5:26:34,  1.65s/it]
 36%|███▋      | 6783/18627 [3:41:37<5:53:47,  1.79s/it]
 36%|███▋      | 6784/18627 [3:41:39<6:13:36,  1.89s/it]
 36%|███▋      | 6785/18627 [3:41:41<5:32:00,  1.68s/it]
 36%|███▋      | 6786/18627 [3:41:43<5:56:57,  1.81s/it]
 36%|███▋      | 6787/18627 [3:41:45<6:13:35,  1.89s/it]
 36%|███▋      | 6788/18627 [3:41:47<6:26:22,  1.96s/it]
 36%|███▋      | 6789/18627 [3:41:49<6:35:01,  2.00s/it]
 36%|███▋      | 6790/18627 [3:41:51<6:41:42,  2.04s/it]
                                                        
{'loss': 1.0553, 'grad_norm': 6.352284908294678, 'learning_rate': 3.670769472576288e-06, 'epoch': 0.36}
+
 36%|███▋      | 6790/18627 [3:41:51<6:41:42,  2.04s/it]
 36%|███▋      | 6791/18627 [3:41:53<6:46:55,  2.06s/it]
 36%|███▋      | 6792/18627 [3:41:54<5:26:19,  1.65s/it]
 36%|███▋      | 6793/18627 [3:41:56<5:52:58,  1.79s/it]
 36%|███▋      | 6794/18627 [3:41:58<6:11:37,  1.88s/it]
 36%|███▋      | 6795/18627 [3:42:00<6:26:02,  1.96s/it]
 36%|███▋      | 6796/18627 [3:42:02<6:36:14,  2.01s/it]
 36%|███▋      | 6797/18627 [3:42:05<6:42:34,  2.04s/it]
 36%|███▋      | 6798/18627 [3:42:07<6:47:21,  2.07s/it]
 37%|███▋      | 6799/18627 [3:42:09<6:50:15,  2.08s/it]
 37%|███▋      | 6800/18627 [3:42:11<6:51:47,  2.09s/it]
                                                        
{'loss': 0.7704, 'grad_norm': 15.445347785949707, 'learning_rate': 3.66692693296398e-06, 'epoch': 0.37}
+
 37%|███▋      | 6800/18627 [3:42:11<6:51:47,  2.09s/it]
 37%|███▋      | 6801/18627 [3:42:13<6:53:44,  2.10s/it]
 37%|███▋      | 6802/18627 [3:42:15<6:55:52,  2.11s/it]
 37%|███▋      | 6803/18627 [3:42:17<6:56:59,  2.12s/it]
 37%|███▋      | 6804/18627 [3:42:19<6:57:52,  2.12s/it]
 37%|███▋      | 6805/18627 [3:42:21<6:02:34,  1.84s/it]
 37%|███▋      | 6806/18627 [3:42:23<6:18:06,  1.92s/it]
 37%|███▋      | 6807/18627 [3:42:25<6:29:41,  1.98s/it]
 37%|███▋      | 6808/18627 [3:42:27<6:39:12,  2.03s/it]
 37%|███▋      | 6809/18627 [3:42:29<6:45:09,  2.06s/it]
 37%|███▋      | 6810/18627 [3:42:31<6:57:05,  2.12s/it]
                                                        
{'loss': 0.8296, 'grad_norm': 11.609169006347656, 'learning_rate': 3.6630808653958694e-06, 'epoch': 0.37}
+
 37%|███▋      | 6810/18627 [3:42:31<6:57:05,  2.12s/it]
 37%|███▋      | 6811/18627 [3:42:33<6:01:57,  1.84s/it]
 37%|███▋      | 6812/18627 [3:42:35<6:18:19,  1.92s/it]
 37%|███▋      | 6813/18627 [3:42:36<5:35:26,  1.70s/it]
 37%|███▋      | 6814/18627 [3:42:38<6:00:21,  1.83s/it]
 37%|███▋      | 6815/18627 [3:42:40<6:17:19,  1.92s/it]
 37%|███▋      | 6816/18627 [3:42:42<6:29:53,  1.98s/it]
 37%|███▋      | 6817/18627 [3:42:44<6:37:27,  2.02s/it]
 37%|███▋      | 6818/18627 [3:42:46<6:43:34,  2.05s/it]
 37%|███▋      | 6819/18627 [3:42:49<6:47:18,  2.07s/it]
 37%|███▋      | 6820/18627 [3:42:50<5:55:38,  1.81s/it]
                                                        
{'loss': 1.2943, 'grad_norm': 13.41612434387207, 'learning_rate': 3.6592312814997244e-06, 'epoch': 0.37}
+
 37%|███▋      | 6820/18627 [3:42:50<5:55:38,  1.81s/it]
 37%|███▋      | 6821/18627 [3:42:52<6:14:26,  1.90s/it]
 37%|███▋      | 6822/18627 [3:42:53<5:32:24,  1.69s/it]
 37%|███▋      | 6823/18627 [3:42:55<5:57:24,  1.82s/it]
 37%|███▋      | 6824/18627 [3:42:57<6:16:54,  1.92s/it]
 37%|███▋      | 6825/18627 [3:43:00<6:29:43,  1.98s/it]
 37%|███▋      | 6826/18627 [3:43:02<6:39:49,  2.03s/it]
 37%|███▋      | 6827/18627 [3:43:04<6:45:22,  2.06s/it]
 37%|███▋      | 6828/18627 [3:43:06<6:49:43,  2.08s/it]
 37%|███▋      | 6829/18627 [3:43:08<6:51:27,  2.09s/it]
 37%|███▋      | 6830/18627 [3:43:10<6:53:22,  2.10s/it]
                                                        
{'loss': 0.9025, 'grad_norm': 8.015268325805664, 'learning_rate': 3.6553781929139454e-06, 'epoch': 0.37}
+
 37%|███▋      | 6830/18627 [3:43:10<6:53:22,  2.10s/it]
 37%|███▋      | 6831/18627 [3:43:11<5:59:37,  1.83s/it]
 37%|███▋      | 6832/18627 [3:43:13<5:21:35,  1.64s/it]
 37%|███▋      | 6833/18627 [3:43:15<5:50:28,  1.78s/it]
 37%|███▋      | 6834/18627 [3:43:17<6:11:03,  1.89s/it]
 37%|███▋      | 6835/18627 [3:43:19<6:24:33,  1.96s/it]
 37%|███▋      | 6836/18627 [3:43:21<6:34:34,  2.01s/it]
 37%|███▋      | 6837/18627 [3:43:22<5:46:49,  1.77s/it]
 37%|███▋      | 6838/18627 [3:43:24<6:08:19,  1.87s/it]
 37%|███▋      | 6839/18627 [3:43:26<5:28:27,  1.67s/it]
 37%|███▋      | 6840/18627 [3:43:28<5:54:21,  1.80s/it]
                                                        
{'loss': 1.5019, 'grad_norm': 7.323563575744629, 'learning_rate': 3.651521611287527e-06, 'epoch': 0.37}
+
 37%|███▋      | 6840/18627 [3:43:28<5:54:21,  1.80s/it]
 37%|███▋      | 6841/18627 [3:43:30<6:13:05,  1.90s/it]
 37%|███▋      | 6842/18627 [3:43:32<6:26:02,  1.97s/it]
 37%|███▋      | 6843/18627 [3:43:33<5:39:50,  1.73s/it]
 37%|███▋      | 6844/18627 [3:43:34<5:08:32,  1.57s/it]
 37%|███▋      | 6845/18627 [3:43:36<5:40:45,  1.74s/it]
 37%|███▋      | 6846/18627 [3:43:38<5:08:36,  1.57s/it]
 37%|███▋      | 6847/18627 [3:43:39<4:46:04,  1.46s/it]
 37%|███▋      | 6848/18627 [3:43:41<5:28:07,  1.67s/it]
 37%|███▋      | 6849/18627 [3:43:43<5:56:40,  1.82s/it]
 37%|███▋      | 6850/18627 [3:43:45<6:15:18,  1.91s/it]
                                                        
{'loss': 1.4556, 'grad_norm': 5.2865166664123535, 'learning_rate': 3.647661548280024e-06, 'epoch': 0.37}
+
 37%|███▋      | 6850/18627 [3:43:45<6:15:18,  1.91s/it]
 37%|███▋      | 6851/18627 [3:43:47<6:27:18,  1.97s/it]
 37%|███▋      | 6852/18627 [3:43:49<6:34:46,  2.01s/it]
 37%|███▋      | 6853/18627 [3:43:52<6:39:52,  2.04s/it]
 37%|███▋      | 6854/18627 [3:43:54<6:44:31,  2.06s/it]
 37%|███▋      | 6855/18627 [3:43:55<5:53:21,  1.80s/it]
 37%|███▋      | 6856/18627 [3:43:57<6:11:41,  1.89s/it]
 37%|███▋      | 6857/18627 [3:43:59<6:23:38,  1.96s/it]
 37%|███▋      | 6858/18627 [3:44:00<5:38:12,  1.72s/it]
 37%|███▋      | 6859/18627 [3:44:02<6:01:34,  1.84s/it]
 37%|███▋      | 6860/18627 [3:44:05<6:16:38,  1.92s/it]
                                                        
{'loss': 1.1787, 'grad_norm': 6.444762706756592, 'learning_rate': 3.6437980155615187e-06, 'epoch': 0.37}
+
 37%|███▋      | 6860/18627 [3:44:05<6:16:38,  1.92s/it]
 37%|███▋      | 6861/18627 [3:44:07<6:28:41,  1.98s/it]
 37%|███▋      | 6862/18627 [3:44:07<5:13:45,  1.60s/it]
 37%|███▋      | 6863/18627 [3:44:09<4:49:40,  1.48s/it]
 37%|███▋      | 6864/18627 [3:44:11<5:28:29,  1.68s/it]
 37%|███▋      | 6865/18627 [3:44:12<4:59:50,  1.53s/it]
 37%|███▋      | 6866/18627 [3:44:14<5:33:20,  1.70s/it]
 37%|███▋      | 6867/18627 [3:44:16<5:57:59,  1.83s/it]
 37%|███▋      | 6868/18627 [3:44:17<5:15:30,  1.61s/it]
 37%|███▋      | 6869/18627 [3:44:19<5:46:29,  1.77s/it]
 37%|███▋      | 6870/18627 [3:44:21<6:08:05,  1.88s/it]
                                                        
{'loss': 1.3888, 'grad_norm': 4.8634724617004395, 'learning_rate': 3.6399310248125807e-06, 'epoch': 0.37}
+
 37%|███▋      | 6870/18627 [3:44:21<6:08:05,  1.88s/it]
 37%|███▋      | 6871/18627 [3:44:23<5:27:15,  1.67s/it]
 37%|███▋      | 6872/18627 [3:44:25<5:54:45,  1.81s/it]
 37%|███▋      | 6873/18627 [3:44:27<6:14:09,  1.91s/it]
 37%|███▋      | 6874/18627 [3:44:29<6:29:34,  1.99s/it]
 37%|███▋      | 6875/18627 [3:44:31<6:41:33,  2.05s/it]
 37%|███▋      | 6876/18627 [3:44:33<5:53:30,  1.80s/it]
 37%|███▋      | 6877/18627 [3:44:35<6:12:45,  1.90s/it]
 37%|███▋      | 6878/18627 [3:44:37<6:24:12,  1.96s/it]
 37%|███▋      | 6879/18627 [3:44:39<6:33:07,  2.01s/it]
 37%|███▋      | 6880/18627 [3:44:41<6:39:24,  2.04s/it]
                                                        
{'loss': 1.099, 'grad_norm': 5.923860549926758, 'learning_rate': 3.6360605877242374e-06, 'epoch': 0.37}
+
 37%|███▋      | 6880/18627 [3:44:41<6:39:24,  2.04s/it]
 37%|███▋      | 6881/18627 [3:44:43<6:44:01,  2.06s/it]
 37%|███▋      | 6882/18627 [3:44:45<6:47:32,  2.08s/it]
 37%|███▋      | 6883/18627 [3:44:46<5:55:57,  1.82s/it]
 37%|███▋      | 6884/18627 [3:44:49<6:13:53,  1.91s/it]
 37%|███▋      | 6885/18627 [3:44:51<6:25:39,  1.97s/it]
 37%|███▋      | 6886/18627 [3:44:53<6:34:36,  2.02s/it]
 37%|███▋      | 6887/18627 [3:44:55<6:40:20,  2.05s/it]
 37%|███▋      | 6888/18627 [3:44:57<6:45:46,  2.07s/it]
 37%|███▋      | 6889/18627 [3:44:58<5:54:13,  1.81s/it]
 37%|███▋      | 6890/18627 [3:45:00<6:11:39,  1.90s/it]
                                                        
{'loss': 1.0395, 'grad_norm': 15.466293334960938, 'learning_rate': 3.6321867159979305e-06, 'epoch': 0.37}
+
 37%|███▋      | 6890/18627 [3:45:00<6:11:39,  1.90s/it]
 37%|███▋      | 6891/18627 [3:45:02<6:24:31,  1.97s/it]
 37%|███▋      | 6892/18627 [3:45:03<5:10:10,  1.59s/it]
 37%|███▋      | 6893/18627 [3:45:05<5:41:52,  1.75s/it]
 37%|███▋      | 6894/18627 [3:45:07<6:04:54,  1.87s/it]
 37%|███▋      | 6895/18627 [3:45:08<5:11:51,  1.59s/it]
 37%|███▋      | 6896/18627 [3:45:10<4:47:54,  1.47s/it]
 37%|███▋      | 6897/18627 [3:45:12<5:30:20,  1.69s/it]
 37%|███▋      | 6898/18627 [3:45:13<5:01:44,  1.54s/it]
 37%|███▋      | 6899/18627 [3:45:15<5:36:25,  1.72s/it]
 37%|███▋      | 6900/18627 [3:45:16<5:05:05,  1.56s/it]
                                                        
{'loss': 1.4445, 'grad_norm': 13.171893119812012, 'learning_rate': 3.628309421345492e-06, 'epoch': 0.37}
+
 37%|███▋      | 6900/18627 [3:45:16<5:05:05,  1.56s/it]
 37%|███▋      | 6901/18627 [3:45:18<5:38:34,  1.73s/it]
 37%|███▋      | 6902/18627 [3:45:21<6:00:59,  1.85s/it]
 37%|███▋      | 6903/18627 [3:45:21<4:53:48,  1.50s/it]
 37%|███▋      | 6904/18627 [3:45:23<5:29:33,  1.69s/it]
 37%|███▋      | 6905/18627 [3:45:25<5:53:41,  1.81s/it]
 37%|███▋      | 6906/18627 [3:45:28<6:11:32,  1.90s/it]
 37%|███▋      | 6907/18627 [3:45:28<5:01:03,  1.54s/it]
 37%|███▋      | 6908/18627 [3:45:30<5:34:18,  1.71s/it]
 37%|███▋      | 6909/18627 [3:45:33<5:58:16,  1.83s/it]
 37%|███▋      | 6910/18627 [3:45:34<5:21:16,  1.65s/it]
                                                        
{'loss': 0.9264, 'grad_norm': 15.021842956542969, 'learning_rate': 3.6244287154890955e-06, 'epoch': 0.37}
+
 37%|███▋      | 6910/18627 [3:45:34<5:21:16,  1.65s/it]
 37%|███▋      | 6911/18627 [3:45:35<4:54:21,  1.51s/it]
 37%|███▋      | 6912/18627 [3:45:37<5:30:16,  1.69s/it]
 37%|███▋      | 6913/18627 [3:45:39<5:55:26,  1.82s/it]
 37%|███▋      | 6914/18627 [3:45:41<6:12:36,  1.91s/it]
 37%|███▋      | 6915/18627 [3:45:43<6:25:33,  1.98s/it]
 37%|███▋      | 6916/18627 [3:45:45<6:33:07,  2.01s/it]
 37%|███▋      | 6917/18627 [3:45:47<5:45:00,  1.77s/it]
 37%|███▋      | 6918/18627 [3:45:48<5:11:13,  1.59s/it]
 37%|███▋      | 6919/18627 [3:45:50<5:41:47,  1.75s/it]
 37%|███▋      | 6920/18627 [3:45:52<6:04:20,  1.87s/it]
                                                        
{'loss': 1.1987, 'grad_norm': 6.725245952606201, 'learning_rate': 3.6205446101612346e-06, 'epoch': 0.37}
+
 37%|███▋      | 6920/18627 [3:45:52<6:04:20,  1.87s/it]
 37%|███▋      | 6921/18627 [3:45:53<5:25:30,  1.67s/it]
 37%|███▋      | 6922/18627 [3:45:55<5:52:42,  1.81s/it]
 37%|███▋      | 6923/18627 [3:45:58<6:10:38,  1.90s/it]
 37%|███▋      | 6924/18627 [3:46:00<6:23:29,  1.97s/it]
 37%|███▋      | 6925/18627 [3:46:02<6:32:12,  2.01s/it]
 37%|███▋      | 6926/18627 [3:46:04<6:38:15,  2.04s/it]
 37%|███▋      | 6927/18627 [3:46:06<6:42:55,  2.07s/it]
 37%|███▋      | 6928/18627 [3:46:08<6:48:44,  2.10s/it]
 37%|███▋      | 6929/18627 [3:46:11<7:03:15,  2.17s/it]
 37%|███▋      | 6930/18627 [3:46:13<7:15:40,  2.23s/it]
                                                        
{'loss': 0.8448, 'grad_norm': 7.165319442749023, 'learning_rate': 3.6166571171046744e-06, 'epoch': 0.37}
+
 37%|███▋      | 6930/18627 [3:46:13<7:15:40,  2.23s/it]
 37%|███▋      | 6931/18627 [3:46:14<6:30:20,  2.00s/it]
 37%|███▋      | 6932/18627 [3:46:16<5:56:20,  1.83s/it]
 37%|███▋      | 6933/18627 [3:46:18<6:23:54,  1.97s/it]
 37%|███▋      | 6934/18627 [3:46:20<6:44:20,  2.07s/it]
 37%|███▋      | 6935/18627 [3:46:23<7:00:13,  2.16s/it]
 37%|███▋      | 6936/18627 [3:46:24<6:10:37,  1.90s/it]
 37%|███▋      | 6937/18627 [3:46:26<6:35:46,  2.03s/it]
 37%|███▋      | 6938/18627 [3:46:29<6:53:44,  2.12s/it]
 37%|███▋      | 6939/18627 [3:46:31<7:06:33,  2.19s/it]
 37%|███▋      | 6940/18627 [3:46:34<7:18:28,  2.25s/it]
                                                        
{'loss': 1.134, 'grad_norm': 6.477161407470703, 'learning_rate': 3.612766248072426e-06, 'epoch': 0.37}
+
 37%|███▋      | 6940/18627 [3:46:34<7:18:28,  2.25s/it]
 37%|███▋      | 6941/18627 [3:46:35<6:26:53,  1.99s/it]
 37%|███▋      | 6942/18627 [3:46:36<5:49:50,  1.80s/it]
 37%|███▋      | 6943/18627 [3:46:39<6:21:48,  1.96s/it]
 37%|███▋      | 6944/18627 [3:46:41<6:45:23,  2.08s/it]
 37%|███▋      | 6945/18627 [3:46:43<7:03:47,  2.18s/it]
 37%|███▋      | 6946/18627 [3:46:46<7:14:11,  2.23s/it]
 37%|███▋      | 6947/18627 [3:46:47<6:24:51,  1.98s/it]
 37%|███▋      | 6948/18627 [3:46:49<6:45:34,  2.08s/it]
 37%|███▋      | 6949/18627 [3:46:52<7:00:25,  2.16s/it]
 37%|███▋      | 6950/18627 [3:46:54<7:10:58,  2.21s/it]
                                                        
{'loss': 1.2794, 'grad_norm': 9.55146598815918, 'learning_rate': 3.6088720148277047e-06, 'epoch': 0.37}
+
 37%|███▋      | 6950/18627 [3:46:54<7:10:58,  2.21s/it]
 37%|███▋      | 6951/18627 [3:46:55<6:12:18,  1.91s/it]
 37%|███▋      | 6952/18627 [3:46:58<6:37:09,  2.04s/it]
 37%|███▋      | 6953/18627 [3:46:59<5:56:50,  1.83s/it]
 37%|███▋      | 6954/18627 [3:47:00<5:32:08,  1.71s/it]
 37%|███▋      | 6955/18627 [3:47:02<5:11:57,  1.60s/it]
 37%|███▋      | 6956/18627 [3:47:04<5:55:28,  1.83s/it]
 37%|███▋      | 6957/18627 [3:47:06<6:24:22,  1.98s/it]
 37%|███▋      | 6958/18627 [3:47:09<6:45:19,  2.08s/it]
 37%|███▋      | 6959/18627 [3:47:11<6:59:51,  2.16s/it]
 37%|███▋      | 6960/18627 [3:47:13<6:16:19,  1.94s/it]
                                                        
{'loss': 1.4887, 'grad_norm': 15.76423168182373, 'learning_rate': 3.604974429143899e-06, 'epoch': 0.37}
+
 37%|███▋      | 6960/18627 [3:47:13<6:16:19,  1.94s/it]
 37%|███▋      | 6961/18627 [3:47:15<6:38:35,  2.05s/it]
 37%|███▋      | 6962/18627 [3:47:17<6:55:27,  2.14s/it]
 37%|███▋      | 6963/18627 [3:47:19<6:11:28,  1.91s/it]
 37%|███▋      | 6964/18627 [3:47:21<6:38:41,  2.05s/it]
 37%|███▋      | 6965/18627 [3:47:23<6:57:13,  2.15s/it]
 37%|███▋      | 6966/18627 [3:47:26<7:09:50,  2.21s/it]
 37%|███▋      | 6967/18627 [3:47:28<7:16:45,  2.25s/it]
 37%|███▋      | 6968/18627 [3:47:30<7:22:35,  2.28s/it]
 37%|███▋      | 6969/18627 [3:47:32<6:33:32,  2.03s/it]
 37%|███▋      | 6970/18627 [3:47:34<6:52:10,  2.12s/it]
                                                        
{'loss': 1.0018, 'grad_norm': 8.252311706542969, 'learning_rate': 3.6010735028045313e-06, 'epoch': 0.37}
+
 37%|███▋      | 6970/18627 [3:47:34<6:52:10,  2.12s/it]
 37%|███▋      | 6971/18627 [3:47:37<7:08:05,  2.20s/it]
 37%|███▋      | 6972/18627 [3:47:39<7:17:42,  2.25s/it]
 37%|███▋      | 6973/18627 [3:47:41<7:19:57,  2.27s/it]
 37%|███▋      | 6974/18627 [3:47:43<6:26:27,  1.99s/it]
 37%|███▋      | 6975/18627 [3:47:44<5:51:32,  1.81s/it]
 37%|███▋      | 6976/18627 [3:47:46<6:23:32,  1.98s/it]
 37%|███▋      | 6977/18627 [3:47:51<8:33:24,  2.64s/it]
 37%|███▋      | 6978/18627 [3:47:52<7:22:44,  2.28s/it]
 37%|███▋      | 6979/18627 [3:47:54<7:26:17,  2.30s/it]
 37%|███▋      | 6980/18627 [3:47:57<7:29:17,  2.31s/it]
                                                        
{'loss': 1.2952, 'grad_norm': 8.215311050415039, 'learning_rate': 3.597169247603224e-06, 'epoch': 0.37}
+
 37%|███▋      | 6980/18627 [3:47:57<7:29:17,  2.31s/it]
 37%|███▋      | 6981/18627 [3:47:59<7:32:00,  2.33s/it]
 37%|███▋      | 6982/18627 [3:48:00<6:35:51,  2.04s/it]
 37%|███▋      | 6983/18627 [3:48:02<5:58:18,  1.85s/it]
 37%|███▋      | 6984/18627 [3:48:04<6:26:50,  1.99s/it]
 37%|███▋      | 6985/18627 [3:48:06<6:47:30,  2.10s/it]
 38%|███▊      | 6986/18627 [3:48:09<7:02:49,  2.18s/it]
 38%|███▊      | 6987/18627 [3:48:11<7:11:57,  2.23s/it]
 38%|███▊      | 6988/18627 [3:48:13<7:18:11,  2.26s/it]
 38%|███▊      | 6989/18627 [3:48:15<6:23:39,  1.98s/it]
 38%|███▊      | 6990/18627 [3:48:16<5:52:22,  1.82s/it]
                                                        
{'loss': 1.466, 'grad_norm': 15.390189170837402, 'learning_rate': 3.5932616753436633e-06, 'epoch': 0.38}
+
 38%|███▊      | 6990/18627 [3:48:16<5:52:22,  1.82s/it]
 38%|███▊      | 6991/18627 [3:48:19<6:22:52,  1.97s/it]
 38%|███▊      | 6992/18627 [3:48:21<6:45:01,  2.09s/it]
 38%|███▊      | 6993/18627 [3:48:23<6:58:24,  2.16s/it]
 38%|███▊      | 6994/18627 [3:48:25<6:11:14,  1.91s/it]
 38%|███▊      | 6995/18627 [3:48:27<6:38:00,  2.05s/it]
 38%|███▊      | 6996/18627 [3:48:29<6:55:01,  2.14s/it]
 38%|███▊      | 6997/18627 [3:48:32<7:04:19,  2.19s/it]
 38%|███▊      | 6998/18627 [3:48:34<7:12:33,  2.23s/it]
 38%|███▊      | 6999/18627 [3:48:36<7:18:51,  2.26s/it]
 38%|███▊      | 7000/18627 [3:48:39<7:24:31,  2.29s/it]
                                                        
{'loss': 0.9247, 'grad_norm': 5.137002944946289, 'learning_rate': 3.589350797839565e-06, 'epoch': 0.38}
+
 38%|███▊      | 7000/18627 [3:48:39<7:24:31,  2.29s/it]/home/zhengduo/workspace/LLaMA-Factory/src/qwen_vl/model/modeling_qwen2_5_vl.py:1979: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
+  with torch.cuda.amp.autocast(dtype=dtype):
+/home/zhengduo/miniconda3/envs/vgllm/lib/python3.10/site-packages/torch/utils/checkpoint.py:87: UserWarning: None of the inputs have requires_grad=True. Gradients will be None
+  warnings.warn(
+
 38%|███▊      | 7001/18627 [3:49:37<61:51:39, 19.16s/it]
 38%|███▊      | 7002/18627 [3:49:39<45:33:43, 14.11s/it]
 38%|███▊      | 7003/18627 [3:49:42<34:09:38, 10.58s/it]
 38%|███▊      | 7004/18627 [3:49:44<26:10:22,  8.11s/it]
 38%|███▊      | 7005/18627 [3:49:47<20:36:06,  6.38s/it]
 38%|███▊      | 7006/18627 [3:49:49<16:41:07,  5.17s/it]
 38%|███▊      | 7007/18627 [3:49:51<13:56:47,  4.32s/it]
 38%|███▊      | 7008/18627 [3:49:54<12:02:25,  3.73s/it]
 38%|███▊      | 7009/18627 [3:49:56<10:42:26,  3.32s/it]
 38%|███▊      | 7010/18627 [3:49:58<9:43:25,  3.01s/it] 
                                                        
{'loss': 0.6605, 'grad_norm': 6.00876522064209, 'learning_rate': 3.5854366269146364e-06, 'epoch': 0.38}
+
 38%|███▊      | 7010/18627 [3:49:58<9:43:25,  3.01s/it]
 38%|███▊      | 7011/18627 [3:50:00<8:58:22,  2.78s/it]
 38%|███▊      | 7012/18627 [3:50:02<7:26:52,  2.31s/it]
 38%|███▊      | 7013/18627 [3:50:04<7:19:09,  2.27s/it]
 38%|███▊      | 7014/18627 [3:50:06<7:15:42,  2.25s/it]
 38%|███▊      | 7015/18627 [3:50:08<7:11:25,  2.23s/it]
 38%|███▊      | 7016/18627 [3:50:10<7:09:36,  2.22s/it]
 38%|███▊      | 7017/18627 [3:50:13<7:11:41,  2.23s/it]
 38%|███▊      | 7018/18627 [3:50:15<7:12:44,  2.24s/it]
 38%|███▊      | 7019/18627 [3:50:17<7:15:40,  2.25s/it]
 38%|███▊      | 7020/18627 [3:50:20<7:16:57,  2.26s/it]
                                                        
{'loss': 0.8728, 'grad_norm': 7.127292633056641, 'learning_rate': 3.581519174402541e-06, 'epoch': 0.38}
+
 38%|███▊      | 7020/18627 [3:50:20<7:16:57,  2.26s/it]
 38%|███▊      | 7021/18627 [3:50:21<6:23:36,  1.98s/it]
 38%|███▊      | 7022/18627 [3:50:22<5:41:01,  1.76s/it]
 38%|███▊      | 7023/18627 [3:50:23<5:15:39,  1.63s/it]
 38%|███▊      | 7024/18627 [3:50:25<4:58:07,  1.54s/it]
 38%|███▊      | 7025/18627 [3:50:27<5:41:39,  1.77s/it]
 38%|███▊      | 7026/18627 [3:50:29<6:10:55,  1.92s/it]
 38%|███▊      | 7027/18627 [3:50:32<6:28:10,  2.01s/it]
 38%|███▊      | 7028/18627 [3:50:34<6:39:05,  2.06s/it]
 38%|███▊      | 7029/18627 [3:50:36<6:47:45,  2.11s/it]
 38%|███▊      | 7030/18627 [3:50:37<5:58:42,  1.86s/it]
                                                        
{'loss': 1.6742, 'grad_norm': 15.037744522094727, 'learning_rate': 3.577598452146867e-06, 'epoch': 0.38}
+
 38%|███▊      | 7030/18627 [3:50:37<5:58:42,  1.86s/it]
 38%|███▊      | 7031/18627 [3:50:39<6:21:06,  1.97s/it]
 38%|███▊      | 7032/18627 [3:50:42<6:33:53,  2.04s/it]
 38%|███▊      | 7033/18627 [3:50:44<6:43:51,  2.09s/it]
 38%|███▊      | 7034/18627 [3:50:45<5:54:26,  1.83s/it]
 38%|███▊      | 7035/18627 [3:50:47<6:16:08,  1.95s/it]
 38%|███▊      | 7036/18627 [3:50:50<6:33:19,  2.04s/it]
 38%|███▊      | 7037/18627 [3:50:51<5:50:49,  1.82s/it]
 38%|███▊      | 7038/18627 [3:50:53<6:19:07,  1.96s/it]
 38%|███▊      | 7039/18627 [3:50:55<5:47:25,  1.80s/it]
 38%|███▊      | 7040/18627 [3:50:57<6:18:37,  1.96s/it]
                                                        
{'loss': 1.3105, 'grad_norm': 5.431376934051514, 'learning_rate': 3.573674472001084e-06, 'epoch': 0.38}
+
 38%|███▊      | 7040/18627 [3:50:57<6:18:37,  1.96s/it]
 38%|███▊      | 7041/18627 [3:50:58<5:47:46,  1.80s/it]
 38%|███▊      | 7042/18627 [3:51:01<6:21:29,  1.98s/it]
 38%|███▊      | 7043/18627 [3:51:03<6:40:17,  2.07s/it]
 38%|███▊      | 7044/18627 [3:51:05<6:55:11,  2.15s/it]
 38%|███▊      | 7045/18627 [3:51:08<7:02:56,  2.19s/it]
 38%|███▊      | 7046/18627 [3:51:10<7:09:05,  2.22s/it]
 38%|███▊      | 7047/18627 [3:51:12<7:13:00,  2.24s/it]
 38%|███▊      | 7048/18627 [3:51:13<6:14:33,  1.94s/it]
 38%|███▊      | 7049/18627 [3:51:16<6:34:52,  2.05s/it]
 38%|███▊      | 7050/18627 [3:51:18<6:47:57,  2.11s/it]
                                                        
{'loss': 1.0223, 'grad_norm': 6.964735507965088, 'learning_rate': 3.569747245828513e-06, 'epoch': 0.38}
+
 38%|███▊      | 7050/18627 [3:51:18<6:47:57,  2.11s/it]
 38%|█���█▊      | 7051/18627 [3:51:19<6:02:37,  1.88s/it]
 38%|███▊      | 7052/18627 [3:51:21<5:27:23,  1.70s/it]
 38%|███▊      | 7053/18627 [3:51:23<6:02:38,  1.88s/it]
 38%|███▊      | 7054/18627 [3:51:24<5:12:14,  1.62s/it]
 38%|███▊      | 7055/18627 [3:51:26<5:49:04,  1.81s/it]
 38%|███▊      | 7056/18627 [3:51:28<6:14:14,  1.94s/it]
 38%|███▊      | 7057/18627 [3:51:31<6:34:15,  2.04s/it]
 38%|███▊      | 7058/18627 [3:51:32<5:52:46,  1.83s/it]
 38%|███▊      | 7059/18627 [3:51:33<5:23:46,  1.68s/it]
 38%|███▊      | 7060/18627 [3:51:36<5:57:50,  1.86s/it]
                                                        
{'loss': 1.5261, 'grad_norm': 6.909992694854736, 'learning_rate': 3.5658167855022897e-06, 'epoch': 0.38}
+
 38%|███▊      | 7060/18627 [3:51:36<5:57:50,  1.86s/it]
 38%|███▊      | 7061/18627 [3:51:37<5:26:09,  1.69s/it]
 38%|███▊      | 7062/18627 [3:51:39<6:00:40,  1.87s/it]
 38%|███▊      | 7063/18627 [3:51:42<6:24:35,  2.00s/it]
 38%|███▊      | 7064/18627 [3:51:43<5:48:37,  1.81s/it]
 38%|███▊      | 7065/18627 [3:51:45<6:17:21,  1.96s/it]
 38%|███▊      | 7066/18627 [3:51:48<6:38:29,  2.07s/it]
 38%|███▊      | 7067/18627 [3:51:49<5:52:39,  1.83s/it]
 38%|███▊      | 7068/18627 [3:51:51<6:20:38,  1.98s/it]
 38%|███▊      | 7069/18627 [3:51:53<6:41:25,  2.08s/it]
 38%|███▊      | 7070/18627 [3:51:56<6:53:39,  2.15s/it]
                                                        
{'loss': 1.2489, 'grad_norm': 5.938471794128418, 'learning_rate': 3.5618831029053245e-06, 'epoch': 0.38}
+
 38%|███▊      | 7070/18627 [3:51:56<6:53:39,  2.15s/it]
 38%|███▊      | 7071/18627 [3:51:58<7:03:11,  2.20s/it]
 38%|███▊      | 7072/18627 [3:52:00<7:08:45,  2.23s/it]
 38%|███▊      | 7073/18627 [3:52:03<7:14:20,  2.26s/it]
 38%|███▊      | 7074/18627 [3:52:05<7:17:46,  2.27s/it]
 38%|███▊      | 7075/18627 [3:52:07<7:19:42,  2.28s/it]
 38%|███▊      | 7076/18627 [3:52:10<7:21:49,  2.29s/it]
 38%|███▊      | 7077/18627 [3:52:11<6:19:03,  1.97s/it]
 38%|███▊      | 7078/18627 [3:52:13<6:37:52,  2.07s/it]
 38%|███▊      | 7079/18627 [3:52:14<5:49:50,  1.82s/it]
 38%|███▊      | 7080/18627 [3:52:17<6:16:53,  1.96s/it]
                                                        
{'loss': 1.068, 'grad_norm': 8.535066604614258, 'learning_rate': 3.5579462099302726e-06, 'epoch': 0.38}
+
 38%|███▊      | 7080/18627 [3:52:17<6:16:53,  1.96s/it]
 38%|███▊      | 7081/18627 [3:52:18<5:41:14,  1.77s/it]
 38%|███▊      | 7082/18627 [3:52:19<4:57:49,  1.55s/it]
 38%|███▊      | 7083/18627 [3:52:21<5:40:51,  1.77s/it]
 38%|███▊      | 7084/18627 [3:52:24<6:11:26,  1.93s/it]
 38%|███▊      | 7085/18627 [3:52:26<6:32:24,  2.04s/it]
 38%|███▊      | 7086/18627 [3:52:28<6:45:35,  2.11s/it]
 38%|███▊      | 7087/18627 [3:52:30<6:55:31,  2.16s/it]
 38%|███▊      | 7088/18627 [3:52:33<7:04:44,  2.21s/it]
 38%|███▊      | 7089/18627 [3:52:35<7:09:37,  2.23s/it]
 38%|███▊      | 7090/18627 [3:52:37<7:12:40,  2.25s/it]
                                                        
{'loss': 0.8173, 'grad_norm': 5.7000508308410645, 'learning_rate': 3.5540061184794932e-06, 'epoch': 0.38}
+
 38%|███▊      | 7090/18627 [3:52:37<7:12:40,  2.25s/it]
 38%|███▊      | 7091/18627 [3:52:40<7:15:33,  2.27s/it]
 38%|███▊      | 7092/18627 [3:52:42<7:15:52,  2.27s/it]
 38%|███▊      | 7093/18627 [3:52:44<7:16:37,  2.27s/it]
 38%|███▊      | 7094/18627 [3:52:46<7:15:58,  2.27s/it]
 38%|███▊      | 7095/18627 [3:52:48<6:19:04,  1.97s/it]
 38%|███▊      | 7096/18627 [3:52:49<5:44:35,  1.79s/it]
 38%|███▊      | 7097/18627 [3:52:51<5:20:03,  1.67s/it]
 38%|███▊      | 7098/18627 [3:52:52<4:52:06,  1.52s/it]
 38%|███▊      | 7099/18627 [3:52:54<5:36:23,  1.75s/it]
 38%|███▊      | 7100/18627 [3:52:56<6:07:41,  1.91s/it]
                                                        
{'loss': 1.4385, 'grad_norm': 5.840850353240967, 'learning_rate': 3.550062840465017e-06, 'epoch': 0.38}
+
 38%|███▊      | 7100/18627 [3:52:56<6:07:41,  1.91s/it]
 38%|███▊      | 7101/18627 [3:52:59<6:31:24,  2.04s/it]
 38%|███▊      | 7102/18627 [3:53:01<6:48:23,  2.13s/it]
 38%|███▊      | 7103/18627 [3:53:03<6:59:11,  2.18s/it]
 38%|███▊      | 7104/18627 [3:53:05<6:11:44,  1.94s/it]
 38%|███▊      | 7105/18627 [3:53:07<6:31:50,  2.04s/it]
 38%|███▊      | 7106/18627 [3:53:09<6:45:41,  2.11s/it]
 38%|███▊      | 7107/18627 [3:53:11<6:53:57,  2.16s/it]
 38%|███▊      | 7108/18627 [3:53:14<7:02:50,  2.20s/it]
 38%|███▊      | 7109/18627 [3:53:16<7:08:16,  2.23s/it]
 38%|███▊      | 7110/18627 [3:53:18<7:11:05,  2.25s/it]
                                                        
{'loss': 0.8166, 'grad_norm': 6.917598247528076, 'learning_rate': 3.546116387808507e-06, 'epoch': 0.38}
+
 38%|███▊      | 7110/18627 [3:53:18<7:11:05,  2.25s/it]
 38%|███▊      | 7111/18627 [3:53:21<7:13:50,  2.26s/it]
 38%|███▊      | 7112/18627 [3:53:23<7:16:42,  2.28s/it]
 38%|███▊      | 7113/18627 [3:53:25<7:18:13,  2.28s/it]
 38%|███▊      | 7114/18627 [3:53:27<6:22:32,  1.99s/it]
 38%|███▊      | 7115/18627 [3:53:29<6:39:10,  2.08s/it]
 38%|███▊      | 7116/18627 [3:53:31<6:50:40,  2.14s/it]
 38%|███▊      | 7117/18627 [3:53:33<6:59:42,  2.19s/it]
 38%|███▊      | 7118/18627 [3:53:36<7:05:20,  2.22s/it]
 38%|███▊      | 7119/18627 [3:53:37<6:15:20,  1.96s/it]
 38%|███▊      | 7120/18627 [3:53:39<6:34:14,  2.06s/it]
                                                        
{'loss': 1.1095, 'grad_norm': 6.453629016876221, 'learning_rate': 3.5421667724412257e-06, 'epoch': 0.38}
+
 38%|███▊      | 7120/18627 [3:53:39<6:34:14,  2.06s/it]
 38%|███▊      | 7121/18627 [3:53:41<5:50:39,  1.83s/it]
 38%|███▊      | 7122/18627 [3:53:43<6:17:22,  1.97s/it]
 38%|███▊      | 7123/18627 [3:53:45<6:37:15,  2.07s/it]
 38%|███▊      | 7124/18627 [3:53:47<6:47:50,  2.13s/it]
 38%|███▊      | 7125/18627 [3:53:50<6:57:37,  2.18s/it]
 38%|███▊      | 7126/18627 [3:53:52<7:02:51,  2.21s/it]
 38%|███▊      | 7127/18627 [3:53:53<6:11:13,  1.94s/it]
 38%|███▊      | 7128/18627 [3:53:56<6:31:52,  2.04s/it]
 38%|███▊      | 7129/18627 [3:53:58<6:46:23,  2.12s/it]
 38%|███▊      | 7130/18627 [3:54:00<6:56:34,  2.17s/it]
                                                        
{'loss': 0.9876, 'grad_norm': 4.757483959197998, 'learning_rate': 3.538214006303996e-06, 'epoch': 0.38}
+
 38%|███▊      | 7130/18627 [3:54:00<6:56:34,  2.17s/it]
 38%|███▊      | 7131/18627 [3:54:02<6:07:38,  1.92s/it]
 38%|███▊      | 7132/18627 [3:54:04<6:26:59,  2.02s/it]
 38%|███▊      | 7133/18627 [3:54:06<6:42:35,  2.10s/it]
 38%|███▊      | 7134/18627 [3:54:07<5:58:45,  1.87s/it]
 38%|███▊      | 7135/18627 [3:54:10<6:23:02,  2.00s/it]
 38%|███▊      | 7136/18627 [3:54:12<6:39:27,  2.09s/it]
 38%|███▊      | 7137/18627 [3:54:14<6:50:10,  2.14s/it]
 38%|███▊      | 7138/18627 [3:54:17<6:57:25,  2.18s/it]
 38%|███▊      | 7139/18627 [3:54:19<7:04:37,  2.22s/it]
 38%|███▊      | 7140/18627 [3:54:21<7:09:49,  2.25s/it]
                                                        
{'loss': 1.0527, 'grad_norm': 5.638991355895996, 'learning_rate': 3.5342581013471668e-06, 'epoch': 0.38}
+
 38%|███▊      | 7140/18627 [3:54:21<7:09:49,  2.25s/it]
 38%|███▊      | 7141/18627 [3:54:24<7:13:52,  2.27s/it]
 38%|███▊      | 7142/18627 [3:54:26<7:15:36,  2.28s/it]
 38%|███▊      | 7143/18627 [3:54:28<7:15:43,  2.28s/it]
 38%|███▊      | 7144/18627 [3:54:30<7:14:43,  2.27s/it]
 38%|███▊      | 7145/18627 [3:54:33<7:16:00,  2.28s/it]
 38%|███▊      | 7146/18627 [3:54:35<7:18:05,  2.29s/it]
 38%|███▊      | 7147/18627 [3:54:36<6:23:16,  2.00s/it]
 38%|███▊      | 7148/18627 [3:54:39<6:39:00,  2.09s/it]
 38%|███▊      | 7149/18627 [3:54:40<5:35:11,  1.75s/it]
 38%|███▊      | 7150/18627 [3:54:42<6:05:41,  1.91s/it]
                                                        
{'loss': 0.9288, 'grad_norm': 6.814233779907227, 'learning_rate': 3.5302990695305774e-06, 'epoch': 0.38}
+
 38%|███▊      | 7150/18627 [3:54:42<6:05:41,  1.91s/it]
 38%|███▊      | 7151/18627 [3:54:44<6:27:43,  2.03s/it]
 38%|███▊      | 7152/18627 [3:54:46<6:43:14,  2.11s/it]
 38%|███▊      | 7153/18627 [3:54:49<6:54:18,  2.17s/it]
 38%|███▊      | 7154/18627 [3:54:51<6:59:53,  2.20s/it]
 38%|███▊      | 7155/18627 [3:54:53<7:06:09,  2.23s/it]
 38%|███▊      | 7156/18627 [3:54:55<6:08:37,  1.93s/it]
 38%|███▊      | 7157/18627 [3:54:57<6:29:33,  2.04s/it]
 38%|███▊      | 7158/18627 [3:54:58<5:48:41,  1.82s/it]
 38%|███▊      | 7159/18627 [3:55:00<6:15:52,  1.97s/it]
 38%|███▊      | 7160/18627 [3:55:02<5:35:11,  1.75s/it]
                                                        
{'loss': 1.2538, 'grad_norm': 12.772412300109863, 'learning_rate': 3.526336922823519e-06, 'epoch': 0.38}
+
 38%|███▊      | 7160/18627 [3:55:02<5:35:11,  1.75s/it]
 38%|███▊      | 7161/18627 [3:55:03<5:12:11,  1.63s/it]
 38%|███▊      | 7162/18627 [3:55:04<4:48:13,  1.51s/it]
 38%|███▊      | 7163/18627 [3:55:05<4:18:10,  1.35s/it]
 38%|███▊      | 7164/18627 [3:55:08<5:11:45,  1.63s/it]
 38%|███▊      | 7165/18627 [3:55:10<5:47:39,  1.82s/it]
 38%|███▊      | 7166/18627 [3:55:12<6:13:38,  1.96s/it]
 38%|███▊      | 7167/18627 [3:55:14<6:33:52,  2.06s/it]
 38%|███▊      | 7168/18627 [3:55:16<5:48:52,  1.83s/it]
 38%|███▊      | 7169/18627 [3:55:18<6:15:23,  1.97s/it]
 38%|███▊      | 7170/18627 [3:55:20<6:32:58,  2.06s/it]
                                                        
{'loss': 1.2826, 'grad_norm': 6.531960964202881, 'learning_rate': 3.5223716732047016e-06, 'epoch': 0.38}
+
 38%|███▊      | 7170/18627 [3:55:20<6:32:58,  2.06s/it]
 38%|███▊      | 7171/18627 [3:55:23<6:45:50,  2.13s/it]
 39%|███▊      | 7172/18627 [3:55:24<5:58:45,  1.88s/it]
 39%|███▊      | 7173/18627 [3:55:26<6:20:04,  1.99s/it]
 39%|███▊      | 7174/18627 [3:55:28<6:39:18,  2.09s/it]
 39%|███▊      | 7175/18627 [3:55:31<6:50:26,  2.15s/it]
 39%|███▊      | 7176/18627 [3:55:33<6:58:32,  2.19s/it]
 39%|███▊      | 7177/18627 [3:55:35<7:04:49,  2.23s/it]
 39%|███▊      | 7178/18627 [3:55:38<7:08:42,  2.25s/it]
 39%|███▊      | 7179/18627 [3:55:40<7:11:18,  2.26s/it]
 39%|███▊      | 7180/18627 [3:55:42<7:12:38,  2.27s/it]
                                                        
{'loss': 0.9213, 'grad_norm': 6.4597578048706055, 'learning_rate': 3.518403332662215e-06, 'epoch': 0.39}
+
 39%|███▊      | 7180/18627 [3:55:42<7:12:38,  2.27s/it]
 39%|███▊      | 7181/18627 [3:55:44<7:15:52,  2.28s/it]
 39%|███▊      | 7182/18627 [3:55:47<7:17:22,  2.29s/it]
 39%|███▊      | 7183/18627 [3:55:49<7:16:06,  2.29s/it]
 39%|███▊      | 7184/18627 [3:55:51<7:17:26,  2.29s/it]
 39%|███▊      | 7185/18627 [3:55:52<6:04:49,  1.91s/it]
 39%|███▊      | 7186/18627 [3:55:55<6:27:41,  2.03s/it]
 39%|███▊      | 7187/18627 [3:55:57<6:41:48,  2.11s/it]
 39%|███▊      | 7188/18627 [3:56:00<7:19:41,  2.31s/it]
 39%|███▊      | 7189/18627 [3:56:02<7:19:08,  2.30s/it]
 39%|███▊      | 7190/18627 [3:56:04<7:18:17,  2.30s/it]
                                                        
{'loss': 0.9748, 'grad_norm': 8.034599304199219, 'learning_rate': 3.5144319131934947e-06, 'epoch': 0.39}
+
 39%|███▊      | 7190/18627 [3:56:04<7:18:17,  2.30s/it]
 39%|███▊      | 7191/18627 [3:56:07<7:18:29,  2.30s/it]
 39%|███▊      | 7192/18627 [3:56:09<7:16:00,  2.29s/it]
 39%|███▊      | 7193/18627 [3:56:10<6:20:26,  2.00s/it]
 39%|███▊      | 7194/18627 [3:56:12<6:35:32,  2.08s/it]
 39%|███▊      | 7195/18627 [3:56:15<6:47:36,  2.14s/it]
 39%|███▊      | 7196/18627 [3:56:16<6:02:17,  1.90s/it]
 39%|███▊      | 7197/18627 [3:56:17<5:29:28,  1.73s/it]
 39%|███▊      | 7198/18627 [3:56:18<4:48:01,  1.51s/it]
 39%|███▊      | 7199/18627 [3:56:20<4:40:07,  1.47s/it]
 39%|███▊      | 7200/18627 [3:56:22<5:27:54,  1.72s/it]
                                                        
{'loss': 1.4504, 'grad_norm': 7.181233882904053, 'learning_rate': 3.5104574268052836e-06, 'epoch': 0.39}
+
 39%|███▊      | 7200/18627 [3:56:22<5:27:54,  1.72s/it]
 39%|███▊      | 7201/18627 [3:56:24<6:00:00,  1.89s/it]
 39%|███▊      | 7202/18627 [3:56:27<6:22:26,  2.01s/it]
 39%|███▊      | 7203/18627 [3:56:29<6:40:18,  2.10s/it]
 39%|███▊      | 7204/18627 [3:56:31<6:51:29,  2.16s/it]
 39%|███▊      | 7205/18627 [3:56:34<6:58:39,  2.20s/it]
 39%|███▊      | 7206/18627 [3:56:35<6:10:33,  1.95s/it]
 39%|███▊      | 7207/18627 [3:56:37<6:31:11,  2.06s/it]
 39%|███▊      | 7208/18627 [3:56:40<6:45:53,  2.13s/it]
 39%|███▊      | 7209/18627 [3:56:41<5:56:08,  1.87s/it]
 39%|███▊      | 7210/18627 [3:56:42<5:24:20,  1.70s/it]
                                                        
{'loss': 1.4265, 'grad_norm': 16.946565628051758, 'learning_rate': 3.5064798855135984e-06, 'epoch': 0.39}
+
 39%|███▊      | 7210/18627 [3:56:42<5:24:20,  1.70s/it]
 39%|███▊      | 7211/18627 [3:56:44<5:04:57,  1.60s/it]
 39%|███▊      | 7212/18627 [3:56:46<5:44:36,  1.81s/it]
 39%|███▊      | 7213/18627 [3:56:48<6:09:11,  1.94s/it]
 39%|███▊      | 7214/18627 [3:56:50<6:28:42,  2.04s/it]
 39%|███▊      | 7215/18627 [3:56:52<5:40:20,  1.79s/it]
 39%|███▊      | 7216/18627 [3:56:53<5:09:17,  1.63s/it]
 39%|███▊      | 7217/18627 [3:56:55<5:48:33,  1.83s/it]
 39%|███▉      | 7218/18627 [3:56:57<6:15:32,  1.97s/it]
 39%|███▉      | 7219/18627 [3:57:00<6:34:23,  2.07s/it]
 39%|███▉      | 7220/18627 [3:57:02<6:47:10,  2.14s/it]
                                                        
{'loss': 1.2822, 'grad_norm': 10.453386306762695, 'learning_rate': 3.5024993013436894e-06, 'epoch': 0.39}
+
 39%|███▉      | 7220/18627 [3:57:02<6:47:10,  2.14s/it]
 39%|███▉      | 7221/18627 [3:57:04<6:52:19,  2.17s/it]
 39%|███▉      | 7222/18627 [3:57:07<6:57:31,  2.20s/it]
 39%|███▉      | 7223/18627 [3:57:09<6:59:07,  2.21s/it]
 39%|███▉      | 7224/18627 [3:57:10<6:02:47,  1.91s/it]
 39%|███▉      | 7225/18627 [3:57:12<6:25:39,  2.03s/it]
 39%|███▉      | 7226/18627 [3:57:15<6:37:35,  2.09s/it]
 39%|███▉      | 7227/18627 [3:57:17<6:49:30,  2.16s/it]
 39%|███▉      | 7228/18627 [3:57:21<8:27:03,  2.67s/it]
 39%|███▉      | 7229/18627 [3:57:23<8:04:08,  2.55s/it]
 39%|███▉      | 7230/18627 [3:57:25<7:48:09,  2.46s/it]
                                                        
{'loss': 0.8473, 'grad_norm': 6.382622241973877, 'learning_rate': 3.498515686330008e-06, 'epoch': 0.39}
+
 39%|███▉      | 7230/18627 [3:57:25<7:48:09,  2.46s/it]
 39%|███▉      | 7231/18627 [3:57:27<7:36:04,  2.40s/it]
 39%|███▉      | 7232/18627 [3:57:30<7:27:29,  2.36s/it]
 39%|███▉      | 7233/18627 [3:57:32<7:19:46,  2.32s/it]
 39%|███▉      | 7234/18627 [3:57:34<7:21:03,  2.32s/it]
 39%|███▉      | 7235/18627 [3:57:36<6:20:03,  2.00s/it]
 39%|███▉      | 7236/18627 [3:57:38<6:35:13,  2.08s/it]
 39%|███▉      | 7237/18627 [3:57:40<6:48:20,  2.15s/it]
 39%|███▉      | 7238/18627 [3:57:41<5:41:09,  1.80s/it]
 39%|███▉      | 7239/18627 [3:57:43<6:07:46,  1.94s/it]
 39%|███▉      | 7240/18627 [3:57:46<6:25:46,  2.03s/it]
                                                        
{'loss': 0.8154, 'grad_norm': 6.078360557556152, 'learning_rate': 3.494529052516169e-06, 'epoch': 0.39}
+
 39%|███▉      | 7240/18627 [3:57:46<6:25:46,  2.03s/it]
 39%|███▉      | 7241/18627 [3:57:48<6:40:15,  2.11s/it]
 39%|███▉      | 7242/18627 [3:57:50<6:50:21,  2.16s/it]
 39%|███▉      | 7243/18627 [3:57:52<6:58:07,  2.20s/it]
 39%|███▉      | 7244/18627 [3:57:55<7:01:40,  2.22s/it]
 39%|███▉      | 7245/18627 [3:57:57<7:05:16,  2.24s/it]
 39%|███▉      | 7246/18627 [3:57:59<7:07:09,  2.25s/it]
 39%|███▉      | 7247/18627 [3:58:02<7:08:09,  2.26s/it]
 39%|███▉      | 7248/18627 [3:58:03<6:15:23,  1.98s/it]
 39%|███▉      | 7249/18627 [3:58:05<6:33:49,  2.08s/it]
 39%|███▉      | 7250/18627 [3:58:08<6:48:41,  2.16s/it]
                                                        
{'loss': 0.8272, 'grad_norm': 7.111160755157471, 'learning_rate': 3.4905394119549125e-06, 'epoch': 0.39}
+
 39%|███▉      | 7250/18627 [3:58:08<6:48:41,  2.16s/it]
 39%|███▉      | 7251/18627 [3:58:09<6:02:24,  1.91s/it]
 39%|███▉      | 7252/18627 [3:58:11<6:24:03,  2.03s/it]
 39%|███▉      | 7253/18627 [3:58:12<5:38:37,  1.79s/it]
 39%|███▉      | 7254/18627 [3:58:15<6:05:24,  1.93s/it]
 39%|███▉      | 7255/18627 [3:58:17<6:23:54,  2.03s/it]
 39%|███▉      | 7256/18627 [3:58:19<6:39:08,  2.11s/it]
 39%|███▉      | 7257/18627 [3:58:22<6:48:48,  2.16s/it]
 39%|███▉      | 7258/18627 [3:58:24<6:54:40,  2.19s/it]
 39%|███▉      | 7259/18627 [3:58:26<7:01:06,  2.22s/it]
 39%|███▉      | 7260/18627 [3:58:28<7:05:31,  2.25s/it]
                                                        
{'loss': 1.018, 'grad_norm': 6.3529953956604, 'learning_rate': 3.48654677670807e-06, 'epoch': 0.39}
+
 39%|███▉      | 7260/18627 [3:58:28<7:05:31,  2.25s/it]
 39%|███▉      | 7261/18627 [3:58:29<5:55:50,  1.88s/it]
 39%|███▉      | 7262/18627 [3:58:31<5:22:42,  1.70s/it]
 39%|███▉      | 7263/18627 [3:58:33<5:56:13,  1.88s/it]
 39%|███▉      | 7264/18627 [3:58:35<6:19:16,  2.00s/it]
 39%|███▉      | 7265/18627 [3:58:38<6:33:02,  2.08s/it]
 39%|███▉      | 7266/18627 [3:58:39<5:50:11,  1.85s/it]
 39%|███▉      | 7267/18627 [3:58:41<6:13:14,  1.97s/it]
 39%|███▉      | 7268/18627 [3:58:43<6:31:11,  2.07s/it]
 39%|███▉      | 7269/18627 [3:58:46<6:43:55,  2.13s/it]
 39%|███▉      | 7270/18627 [3:58:48<6:52:16,  2.18s/it]
                                                        
{'loss': 1.0163, 'grad_norm': 9.240813255310059, 'learning_rate': 3.4825511588465256e-06, 'epoch': 0.39}
+
 39%|███▉      | 7270/18627 [3:58:48<6:52:16,  2.18s/it]
 39%|███▉      | 7271/18627 [3:58:50<7:00:04,  2.22s/it]
 39%|███▉      | 7272/18627 [3:58:53<7:02:33,  2.23s/it]
 39%|███▉      | 7273/18627 [3:58:55<7:04:27,  2.24s/it]
 39%|███▉      | 7274/18627 [3:58:57<7:05:44,  2.25s/it]
 39%|███▉      | 7275/18627 [3:58:59<7:07:29,  2.26s/it]
 39%|███▉      | 7276/18627 [3:59:04<9:00:41,  2.86s/it]
 39%|███▉      | 7277/18627 [3:59:05<7:31:30,  2.39s/it]
 39%|███▉      | 7278/18627 [3:59:06<6:27:20,  2.05s/it]
 39%|███▉      | 7279/18627 [3:59:08<6:42:57,  2.13s/it]
 39%|███▉      | 7280/18627 [3:59:11<6:51:15,  2.17s/it]
                                                        
{'loss': 1.1041, 'grad_norm': 6.422667980194092, 'learning_rate': 3.4785525704501825e-06, 'epoch': 0.39}
+
 39%|███▉      | 7280/18627 [3:59:11<6:51:15,  2.17s/it]
 39%|███▉      | 7281/18627 [3:59:13<6:56:06,  2.20s/it]
 39%|███▉      | 7282/18627 [3:59:15<6:59:38,  2.22s/it]
 39%|███▉      | 7283/18627 [3:59:18<7:02:58,  2.24s/it]
 39%|███▉      | 7284/18627 [3:59:20<7:05:07,  2.25s/it]
 39%|███▉      | 7285/18627 [3:59:22<7:07:27,  2.26s/it]
 39%|███▉      | 7286/18627 [3:59:24<7:08:43,  2.27s/it]
 39%|███▉      | 7287/18627 [3:59:27<7:09:26,  2.27s/it]
 39%|███▉      | 7288/18627 [3:59:28<6:16:25,  1.99s/it]
 39%|███▉      | 7289/18627 [3:59:29<5:36:58,  1.78s/it]
 39%|███▉      | 7290/18627 [3:59:32<6:04:06,  1.93s/it]
                                                        
{'loss': 1.0534, 'grad_norm': 7.837161064147949, 'learning_rate': 3.4745510236079232e-06, 'epoch': 0.39}
+
 39%|███▉      | 7290/18627 [3:59:32<6:04:06,  1.93s/it]
 39%|███▉      | 7291/18627 [3:59:34<6:24:28,  2.04s/it]
 39%|███▉      | 7292/18627 [3:59:36<6:36:02,  2.10s/it]
 39%|███▉      | 7293/18627 [3:59:38<6:44:23,  2.14s/it]
 39%|███▉      | 7294/18627 [3:59:41<6:49:05,  2.17s/it]
 39%|███▉      | 7295/18627 [3:59:42<5:58:56,  1.90s/it]
 39%|███▉      | 7296/18627 [3:59:44<6:19:30,  2.01s/it]
 39%|███▉      | 7297/18627 [3:59:46<6:34:40,  2.09s/it]
 39%|███▉      | 7298/18627 [3:59:48<5:49:58,  1.85s/it]
 39%|███▉      | 7299/18627 [3:59:49<5:16:39,  1.68s/it]
 39%|███▉      | 7300/18627 [3:59:51<5:50:41,  1.86s/it]
                                                        
{'loss': 1.2526, 'grad_norm': 6.506810188293457, 'learning_rate': 3.4705465304175746e-06, 'epoch': 0.39}
+
 39%|███▉      | 7300/18627 [3:59:51<5:50:41,  1.86s/it]
 39%|███▉      | 7301/18627 [3:59:54<6:15:27,  1.99s/it]
 39%|███▉      | 7302/18627 [3:59:55<5:37:18,  1.79s/it]
 39%|███▉      | 7303/18627 [3:59:57<6:04:17,  1.93s/it]
 39%|███▉      | 7304/18627 [3:59:59<6:24:42,  2.04s/it]
 39%|███▉      | 7305/18627 [4:00:01<5:41:34,  1.81s/it]
 39%|███▉      | 7306/18627 [4:00:03<6:07:50,  1.95s/it]
 39%|███▉      | 7307/18627 [4:00:05<6:25:55,  2.05s/it]
 39%|███▉      | 7308/18627 [4:00:06<5:41:33,  1.81s/it]
 39%|███▉      | 7309/18627 [4:00:10<7:35:55,  2.42s/it]
 39%|███▉      | 7310/18627 [4:00:12<6:32:51,  2.08s/it]
                                                        
{'loss': 1.3739, 'grad_norm': 14.010235786437988, 'learning_rate': 3.466539102985873e-06, 'epoch': 0.39}
+
 39%|███▉      | 7310/18627 [4:00:12<6:32:51,  2.08s/it]
 39%|███▉      | 7311/18627 [4:00:14<6:42:36,  2.13s/it]
 39%|███▉      | 7312/18627 [4:00:16<6:51:52,  2.18s/it]
 39%|███▉      | 7313/18627 [4:00:18<6:02:32,  1.92s/it]
 39%|███▉      | 7314/18627 [4:00:20<6:23:19,  2.03s/it]
 39%|███▉      | 7315/18627 [4:00:21<5:42:39,  1.82s/it]
 39%|███▉      | 7316/18627 [4:00:23<6:09:47,  1.96s/it]
 39%|███▉      | 7317/18627 [4:00:26<6:27:23,  2.06s/it]
 39%|███▉      | 7318/18627 [4:00:30<8:12:51,  2.61s/it]
 39%|███▉      | 7319/18627 [4:00:32<7:52:22,  2.51s/it]
 39%|███▉      | 7320/18627 [4:00:34<7:37:23,  2.43s/it]
                                                        
{'loss': 1.1527, 'grad_norm': 4.4636077880859375, 'learning_rate': 3.4625287534284224e-06, 'epoch': 0.39}
+
 39%|███▉      | 7320/18627 [4:00:34<7:37:23,  2.43s/it]
 39%|███▉      | 7321/18627 [4:00:36<7:28:26,  2.38s/it]
 39%|███▉      | 7322/18627 [4:00:39<7:19:56,  2.33s/it]
 39%|███▉      | 7323/18627 [4:00:41<7:14:53,  2.31s/it]
 39%|███▉      | 7324/18627 [4:00:43<7:12:32,  2.30s/it]
 39%|███▉      | 7325/18627 [4:00:45<7:10:31,  2.29s/it]
 39%|███▉      | 7326/18627 [4:00:48<7:08:28,  2.27s/it]
 39%|███▉      | 7327/18627 [4:00:50<7:05:41,  2.26s/it]
 39%|███▉      | 7328/18627 [4:00:51<6:10:50,  1.97s/it]
 39%|███▉      | 7329/18627 [4:00:53<6:27:59,  2.06s/it]
 39%|███▉      | 7330/18627 [4:00:55<5:45:12,  1.83s/it]
                                                        
{'loss': 1.0723, 'grad_norm': 13.947052955627441, 'learning_rate': 3.458515493869664e-06, 'epoch': 0.39}
+
 39%|███▉      | 7330/18627 [4:00:55<5:45:12,  1.83s/it]
 39%|███▉      | 7331/18627 [4:00:57<6:09:18,  1.96s/it]
 39%|███▉      | 7332/18627 [4:00:59<6:29:14,  2.07s/it]
 39%|███▉      | 7333/18627 [4:01:01<5:43:37,  1.83s/it]
 39%|███▉      | 7334/18627 [4:01:02<5:14:54,  1.67s/it]
 39%|███▉      | 7335/18627 [4:01:04<5:48:22,  1.85s/it]
 39%|███▉      | 7336/18627 [4:01:06<6:13:21,  1.98s/it]
 39%|███▉      | 7337/18627 [4:01:09<6:30:44,  2.08s/it]
 39%|███▉      | 7338/18627 [4:01:10<5:48:49,  1.85s/it]
 39%|███▉      | 7339/18627 [4:01:12<6:14:36,  1.99s/it]
 39%|███▉      | 7340/18627 [4:01:15<6:30:44,  2.08s/it]
                                                        
{'loss': 1.3235, 'grad_norm': 7.284400939941406, 'learning_rate': 3.4544993364428363e-06, 'epoch': 0.39}
+
 39%|███▉      | 7340/18627 [4:01:15<6:30:44,  2.08s/it]
 39%|███▉      | 7341/18627 [4:01:16<5:45:04,  1.83s/it]
 39%|███▉      | 7342/18627 [4:01:18<6:10:04,  1.97s/it]
 39%|███▉      | 7343/18627 [4:01:20<5:33:38,  1.77s/it]
 39%|███▉      | 7344/18627 [4:01:22<6:01:45,  1.92s/it]
 39%|███▉      | 7345/18627 [4:01:24<6:23:44,  2.04s/it]
 39%|███▉      | 7346/18627 [4:01:25<5:44:11,  1.83s/it]
 39%|███▉      | 7347/18627 [4:01:28<6:09:37,  1.97s/it]
 39%|███▉      | 7348/18627 [4:01:30<6:26:18,  2.05s/it]
 39%|███▉      | 7349/18627 [4:01:32<6:38:44,  2.12s/it]
 39%|███▉      | 7350/18627 [4:01:35<6:48:20,  2.17s/it]
                                                        
{'loss': 1.2866, 'grad_norm': 6.541877746582031, 'learning_rate': 3.4504802932899383e-06, 'epoch': 0.39}
+
 39%|███▉      | 7350/18627 [4:01:35<6:48:20,  2.17s/it]
 39%|███▉      | 7351/18627 [4:01:37<6:55:23,  2.21s/it]
 39%|███▉      | 7352/18627 [4:01:39<6:58:24,  2.23s/it]
 39%|███▉      | 7353/18627 [4:01:41<7:01:10,  2.24s/it]
 39%|███▉      | 7354/18627 [4:01:44<7:01:35,  2.24s/it]
 39%|███▉      | 7355/18627 [4:01:46<7:03:00,  2.25s/it]
 39%|███▉      | 7356/18627 [4:01:48<7:03:40,  2.26s/it]
 39%|███▉      | 7357/18627 [4:01:50<7:05:28,  2.27s/it]
 40%|███▉      | 7358/18627 [4:01:52<6:09:06,  1.97s/it]
 40%|███▉      | 7359/18627 [4:01:54<6:27:01,  2.06s/it]
 40%|███▉      | 7360/18627 [4:01:56<6:38:51,  2.12s/it]
                                                        
{'loss': 0.8158, 'grad_norm': 5.859488487243652, 'learning_rate': 3.446458376561693e-06, 'epoch': 0.4}
+
 40%|███▉      | 7360/18627 [4:01:56<6:38:51,  2.12s/it]
 40%|███▉      | 7361/18627 [4:01:59<6:47:01,  2.17s/it]
 40%|███▉      | 7362/18627 [4:02:01<6:54:05,  2.21s/it]
 40%|███▉      | 7363/18627 [4:02:03<6:58:41,  2.23s/it]
 40%|███▉      | 7364/18627 [4:02:05<7:00:27,  2.24s/it]
 40%|███▉      | 7365/18627 [4:02:08<7:04:37,  2.26s/it]
 40%|███▉      | 7366/18627 [4:02:10<7:06:21,  2.27s/it]
 40%|███▉      | 7367/18627 [4:02:11<5:53:41,  1.88s/it]
 40%|███▉      | 7368/18627 [4:02:13<6:16:34,  2.01s/it]
 40%|███▉      | 7369/18627 [4:02:16<6:33:50,  2.10s/it]
 40%|███▉      | 7370/18627 [4:02:18<6:43:57,  2.15s/it]
                                                        
{'loss': 0.6145, 'grad_norm': 5.667947769165039, 'learning_rate': 3.4424335984175127e-06, 'epoch': 0.4}
+
 40%|███▉      | 7370/18627 [4:02:18<6:43:57,  2.15s/it]
 40%|███▉      | 7371/18627 [4:02:20<6:51:48,  2.20s/it]
 40%|███▉      | 7372/18627 [4:02:22<6:55:00,  2.21s/it]
 40%|███▉      | 7373/18627 [4:02:24<5:58:54,  1.91s/it]
 40%|███▉      | 7374/18627 [4:02:26<6:20:16,  2.03s/it]
 40%|███▉      | 7375/18627 [4:02:28<6:32:49,  2.09s/it]
 40%|███▉      | 7376/18627 [4:02:30<6:45:22,  2.16s/it]
 40%|███▉      | 7377/18627 [4:02:32<5:59:02,  1.91s/it]
 40%|███▉      | 7378/18627 [4:02:34<6:20:02,  2.03s/it]
 40%|███▉      | 7379/18627 [4:02:36<6:32:22,  2.09s/it]
 40%|███▉      | 7380/18627 [4:02:39<6:42:15,  2.15s/it]
                                                        
{'loss': 1.0586, 'grad_norm': 4.055055618286133, 'learning_rate': 3.4384059710254586e-06, 'epoch': 0.4}
+
 40%|███▉      | 7380/18627 [4:02:39<6:42:15,  2.15s/it]
 40%|███▉      | 7381/18627 [4:02:41<6:49:32,  2.18s/it]
 40%|███▉      | 7382/18627 [4:02:43<6:52:19,  2.20s/it]
 40%|███▉      | 7383/18627 [4:02:45<6:54:48,  2.21s/it]
 40%|███▉      | 7384/18627 [4:02:48<6:55:12,  2.22s/it]
 40%|███▉      | 7385/18627 [4:02:50<6:58:19,  2.23s/it]
 40%|███▉      | 7386/18627 [4:02:52<7:00:17,  2.24s/it]
 40%|███▉      | 7387/18627 [4:02:54<7:02:50,  2.26s/it]
 40%|███▉      | 7388/18627 [4:02:56<6:12:27,  1.99s/it]
 40%|███▉      | 7389/18627 [4:02:58<6:28:21,  2.07s/it]
 40%|███▉      | 7390/18627 [4:03:00<6:42:15,  2.15s/it]
                                                        
{'loss': 0.7898, 'grad_norm': 9.238423347473145, 'learning_rate': 3.4343755065622075e-06, 'epoch': 0.4}
+
 40%|███▉      | 7390/18627 [4:03:00<6:42:15,  2.15s/it]
 40%|███▉      | 7391/18627 [4:03:03<6:50:38,  2.19s/it]
 40%|███▉      | 7392/18627 [4:03:05<6:55:36,  2.22s/it]
 40%|███▉      | 7393/18627 [4:03:07<7:00:25,  2.25s/it]
 40%|███▉      | 7394/18627 [4:03:10<7:01:04,  2.25s/it]
 40%|███▉      | 7395/18627 [4:03:12<7:01:15,  2.25s/it]
 40%|███▉      | 7396/18627 [4:03:13<5:46:22,  1.85s/it]
 40%|███▉      | 7397/18627 [4:03:14<5:08:29,  1.65s/it]
 40%|███▉      | 7398/18627 [4:03:16<5:41:19,  1.82s/it]
 40%|███▉      | 7399/18627 [4:03:17<5:08:25,  1.65s/it]
 40%|███▉      | 7400/18627 [4:03:20<5:37:23,  1.80s/it]
                                                        
{'loss': 1.1088, 'grad_norm': 4.55826997756958, 'learning_rate': 3.4303422172130124e-06, 'epoch': 0.4}
+
 40%|███▉      | 7400/18627 [4:03:20<5:37:23,  1.80s/it]
 40%|███▉      | 7401/18627 [4:03:22<5:57:20,  1.91s/it]
 40%|███▉      | 7402/18627 [4:03:24<6:10:01,  1.98s/it]
 40%|███▉      | 7403/18627 [4:03:25<5:25:40,  1.74s/it]
 40%|███▉      | 7404/18627 [4:03:26<4:57:34,  1.59s/it]
 40%|███▉      | 7405/18627 [4:03:27<4:35:14,  1.47s/it]
 40%|███▉      | 7406/18627 [4:03:28<3:52:10,  1.24s/it]
 40%|███▉      | 7407/18627 [4:03:30<4:42:13,  1.51s/it]
 40%|███▉      | 7408/18627 [4:03:32<5:16:09,  1.69s/it]
 40%|███▉      | 7409/18627 [4:03:35<5:40:54,  1.82s/it]
 40%|███▉      | 7410/18627 [4:03:37<5:56:36,  1.91s/it]
                                                        
{'loss': 1.2367, 'grad_norm': 12.696539878845215, 'learning_rate': 3.426306115171667e-06, 'epoch': 0.4}
+
 40%|███▉      | 7410/18627 [4:03:37<5:56:36,  1.91s/it]
 40%|███▉      | 7411/18627 [4:03:38<5:16:11,  1.69s/it]
 40%|███▉      | 7412/18627 [4:03:40<5:39:39,  1.82s/it]
 40%|███▉      | 7413/18627 [4:03:42<5:58:08,  1.92s/it]
 40%|███▉      | 7414/18627 [4:03:44<6:09:59,  1.98s/it]
 40%|███▉      | 7415/18627 [4:03:45<5:25:33,  1.74s/it]
 40%|███▉      | 7416/18627 [4:03:48<5:47:32,  1.86s/it]
 40%|███▉      | 7417/18627 [4:03:50<6:02:17,  1.94s/it]
 40%|███▉      | 7418/18627 [4:03:52<6:14:36,  2.01s/it]
 40%|███▉      | 7419/18627 [4:03:54<6:24:05,  2.06s/it]
 40%|███▉      | 7420/18627 [4:03:56<6:29:03,  2.08s/it]
                                                        
{'loss': 1.0136, 'grad_norm': 7.366796970367432, 'learning_rate': 3.42226721264047e-06, 'epoch': 0.4}
+
 40%|███▉      | 7420/18627 [4:03:56<6:29:03,  2.08s/it]
 40%|███▉      | 7421/18627 [4:03:58<6:31:36,  2.10s/it]
 40%|███▉      | 7422/18627 [4:04:00<6:34:05,  2.11s/it]
 40%|███▉      | 7423/18627 [4:04:03<6:37:30,  2.13s/it]
 40%|███▉      | 7424/18627 [4:04:05<6:41:06,  2.15s/it]
 40%|███▉      | 7425/18627 [4:04:07<6:44:24,  2.17s/it]
 40%|███▉      | 7426/18627 [4:04:09<6:59:33,  2.25s/it]
 40%|███▉      | 7427/18627 [4:04:11<6:01:13,  1.94s/it]
 40%|███▉      | 7428/18627 [4:04:12<5:19:50,  1.71s/it]
 40%|███▉      | 7429/18627 [4:04:14<5:44:45,  1.85s/it]
 40%|███▉      | 7430/18627 [4:04:16<6:02:30,  1.94s/it]
                                                        
{'loss': 1.0199, 'grad_norm': 6.594090938568115, 'learning_rate': 3.4182255218301853e-06, 'epoch': 0.4}
+
 40%|███▉      | 7430/18627 [4:04:16<6:02:30,  1.94s/it]
 40%|███▉      | 7431/18627 [4:04:18<6:14:19,  2.01s/it]
 40%|███▉      | 7432/18627 [4:04:20<6:22:49,  2.05s/it]
 40%|███▉      | 7433/18627 [4:04:23<6:28:36,  2.08s/it]
 40%|███▉      | 7434/18627 [4:04:25<6:31:02,  2.10s/it]
 40%|███▉      | 7435/18627 [4:04:27<6:33:34,  2.11s/it]
 40%|███▉      | 7436/18627 [4:04:28<5:36:17,  1.80s/it]
 40%|███▉      | 7437/18627 [4:04:29<5:01:58,  1.62s/it]
 40%|███▉      | 7438/18627 [4:04:31<5:30:19,  1.77s/it]
 40%|███▉      | 7439/18627 [4:04:33<5:49:52,  1.88s/it]
 40%|███▉      | 7440/18627 [4:04:36<6:04:23,  1.95s/it]
                                                        
{'loss': 1.0349, 'grad_norm': 4.732074737548828, 'learning_rate': 3.414181054960005e-06, 'epoch': 0.4}
+
 40%|███▉      | 7440/18627 [4:04:36<6:04:23,  1.95s/it]
 40%|███▉      | 7441/18627 [4:04:38<6:13:09,  2.00s/it]
 40%|███▉      | 7442/18627 [4:04:40<6:20:37,  2.04s/it]
 40%|███▉      | 7443/18627 [4:04:42<6:25:36,  2.07s/it]
 40%|███▉      | 7444/18627 [4:04:44<6:29:20,  2.09s/it]
 40%|███▉      | 7445/18627 [4:04:46<6:31:04,  2.10s/it]
 40%|███▉      | 7446/18627 [4:04:48<6:33:19,  2.11s/it]
 40%|███▉      | 7447/18627 [4:04:50<6:35:33,  2.12s/it]
 40%|███▉      | 7448/18627 [4:04:53<6:36:07,  2.13s/it]
 40%|███▉      | 7449/18627 [4:04:53<5:23:07,  1.73s/it]
 40%|███▉      | 7450/18627 [4:04:55<4:52:44,  1.57s/it]
                                                        
{'loss': 0.8068, 'grad_norm': 13.215561866760254, 'learning_rate': 3.4101338242575184e-06, 'epoch': 0.4}
+
 40%|███▉      | 7450/18627 [4:04:55<4:52:44,  1.57s/it]
 40%|████      | 7451/18627 [4:04:57<5:25:12,  1.75s/it]
 40%|████      | 7452/18627 [4:04:59<5:46:20,  1.86s/it]
 40%|████      | 7453/18627 [4:05:01<6:02:33,  1.95s/it]
 40%|████      | 7454/18627 [4:05:02<5:20:25,  1.72s/it]
 40%|████      | 7455/18627 [4:05:04<5:45:51,  1.86s/it]
 40%|████      | 7456/18627 [4:05:07<6:02:24,  1.95s/it]
 40%|████      | 7457/18627 [4:05:09<6:12:49,  2.00s/it]
 40%|████      | 7458/18627 [4:05:11<6:21:03,  2.05s/it]
 40%|████      | 7459/18627 [4:05:13<6:25:58,  2.07s/it]
 40%|████      | 7460/18627 [4:05:15<6:29:02,  2.09s/it]
                                                        
{'loss': 0.9249, 'grad_norm': 4.427876949310303, 'learning_rate': 3.406083841958667e-06, 'epoch': 0.4}
+
 40%|████      | 7460/18627 [4:05:15<6:29:02,  2.09s/it]
 40%|████      | 7461/18627 [4:05:17<6:31:31,  2.10s/it]
 40%|████      | 7462/18627 [4:05:19<5:44:23,  1.85s/it]
 40%|████      | 7463/18627 [4:05:21<6:01:53,  1.94s/it]
 40%|████      | 7464/18627 [4:05:23<6:11:50,  2.00s/it]
 40%|████      | 7465/18627 [4:05:24<5:26:38,  1.76s/it]
 40%|████      | 7466/18627 [4:05:26<5:47:56,  1.87s/it]
 40%|████      | 7467/18627 [4:05:28<6:00:35,  1.94s/it]
 40%|████      | 7468/18627 [4:05:30<6:11:31,  2.00s/it]
 40%|████      | 7469/18627 [4:05:31<5:17:14,  1.71s/it]
 40%|████      | 7470/18627 [4:05:34<5:41:20,  1.84s/it]
                                                        
{'loss': 1.3636, 'grad_norm': 6.425782203674316, 'learning_rate': 3.4020311203077123e-06, 'epoch': 0.4}
+
 40%|████      | 7470/18627 [4:05:34<5:41:20,  1.84s/it]
 40%|████      | 7471/18627 [4:05:36<5:57:37,  1.92s/it]
 40%|████      | 7472/18627 [4:05:37<5:16:33,  1.70s/it]
 40%|████      | 7473/18627 [4:05:39<5:38:54,  1.82s/it]
 40%|████      | 7474/18627 [4:05:41<6:11:36,  2.00s/it]
 40%|████      | 7475/18627 [4:05:43<6:19:08,  2.04s/it]
 40%|████      | 7476/18627 [4:05:46<6:24:21,  2.07s/it]
 40%|████      | 7477/18627 [4:05:47<5:35:33,  1.81s/it]
 40%|████      | 7478/18627 [4:05:49<5:52:58,  1.90s/it]
 40%|████      | 7479/18627 [4:05:51<6:05:56,  1.97s/it]
 40%|████      | 7480/18627 [4:05:53<6:15:07,  2.02s/it]
                                                        
{'loss': 1.1058, 'grad_norm': 5.951717853546143, 'learning_rate': 3.3979756715571976e-06, 'epoch': 0.4}
+
 40%|████      | 7480/18627 [4:05:53<6:15:07,  2.02s/it]
 40%|████      | 7481/18627 [4:05:55<6:21:44,  2.05s/it]
 40%|████      | 7482/18627 [4:05:57<5:33:47,  1.80s/it]
 40%|████      | 7483/18627 [4:05:57<4:34:32,  1.48s/it]
 40%|████      | 7484/18627 [4:05:59<5:12:16,  1.68s/it]
 40%|████      | 7485/18627 [4:06:02<5:37:54,  1.82s/it]
 40%|████      | 7486/18627 [4:06:04<5:56:07,  1.92s/it]
 40%|████      | 7487/18627 [4:06:06<6:07:42,  1.98s/it]
 40%|████      | 7488/18627 [4:06:07<5:23:33,  1.74s/it]
 40%|████      | 7489/18627 [4:06:08<4:29:04,  1.45s/it]
 40%|████      | 7490/18627 [4:06:10<5:07:01,  1.65s/it]
                                                        
{'loss': 1.1174, 'grad_norm': 6.354161262512207, 'learning_rate': 3.3939175079679116e-06, 'epoch': 0.4}
+
 40%|████      | 7490/18627 [4:06:10<5:07:01,  1.65s/it]
 40%|████      | 7491/18627 [4:06:12<5:33:35,  1.80s/it]
 40%|████      | 7492/18627 [4:06:14<5:53:20,  1.90s/it]
 40%|████      | 7493/18627 [4:06:15<5:13:30,  1.69s/it]
 40%|████      | 7494/18627 [4:06:18<5:38:26,  1.82s/it]
 40%|████      | 7495/18627 [4:06:20<5:54:33,  1.91s/it]
 40%|████      | 7496/18627 [4:06:22<6:07:11,  1.98s/it]
 40%|████      | 7497/18627 [4:06:24<6:16:40,  2.03s/it]
 40%|████      | 7498/18627 [4:06:26<6:22:52,  2.06s/it]
 40%|████      | 7499/18627 [4:06:28<6:26:22,  2.08s/it]
 40%|████      | 7500/18627 [4:06:29<5:09:55,  1.67s/it]
                                                        
{'loss': 0.8335, 'grad_norm': 5.274303913116455, 'learning_rate': 3.3898566418088496e-06, 'epoch': 0.4}
+
 40%|████      | 7500/18627 [4:06:29<5:09:55,  1.67s/it]
 40%|████      | 7501/18627 [4:06:31<5:35:26,  1.81s/it]
 40%|████      | 7502/18627 [4:06:32<5:00:59,  1.62s/it]
 40%|████      | 7503/18627 [4:06:33<4:36:59,  1.49s/it]
 40%|████      | 7504/18627 [4:06:36<5:12:19,  1.68s/it]
 40%|████      | 7505/18627 [4:06:38<5:36:43,  1.82s/it]
 40%|████      | 7506/18627 [4:06:40<5:52:26,  1.90s/it]
 40%|████      | 7507/18627 [4:06:42<6:03:17,  1.96s/it]
 40%|████      | 7508/18627 [4:06:44<6:12:12,  2.01s/it]
 40%|████      | 7509/18627 [4:06:46<6:18:58,  2.05s/it]
 40%|████      | 7510/18627 [4:06:48<6:22:52,  2.07s/it]
                                                        
{'loss': 1.1274, 'grad_norm': 7.011325836181641, 'learning_rate': 3.385793085357179e-06, 'epoch': 0.4}
+
 40%|████      | 7510/18627 [4:06:48<6:22:52,  2.07s/it]
 40%|████      | 7511/18627 [4:06:50<6:24:33,  2.08s/it]
 40%|████      | 7512/18627 [4:06:52<6:28:04,  2.09s/it]
 40%|████      | 7513/18627 [4:06:54<5:38:07,  1.83s/it]
 40%|████      | 7514/18627 [4:06:56<5:54:51,  1.92s/it]
 40%|████      | 7515/18627 [4:06:58<6:05:32,  1.97s/it]
 40%|████      | 7516/18627 [4:07:00<6:14:18,  2.02s/it]
 40%|████      | 7517/18627 [4:07:02<6:20:32,  2.06s/it]
 40%|████      | 7518/18627 [4:07:04<6:23:13,  2.07s/it]
 40%|████      | 7519/18627 [4:07:05<5:34:12,  1.81s/it]
 40%|████      | 7520/18627 [4:07:08<5:50:05,  1.89s/it]
                                                        
{'loss': 1.1187, 'grad_norm': 7.219629764556885, 'learning_rate': 3.381726850898199e-06, 'epoch': 0.4}
+
 40%|████      | 7520/18627 [4:07:08<5:50:05,  1.89s/it]
 40%|████      | 7521/18627 [4:07:10<6:03:01,  1.96s/it]
 40%|████      | 7522/18627 [4:07:12<6:11:21,  2.01s/it]
 40%|████      | 7523/18627 [4:07:14<6:17:41,  2.04s/it]
 40%|████      | 7524/18627 [4:07:16<6:22:04,  2.06s/it]
 40%|████      | 7525/18627 [4:07:19<6:51:39,  2.22s/it]
 40%|████      | 7526/18627 [4:07:21<6:46:48,  2.20s/it]
 40%|████      | 7527/18627 [4:07:23<6:44:06,  2.18s/it]
 40%|████      | 7528/18627 [4:07:24<5:49:11,  1.89s/it]
 40%|████      | 7529/18627 [4:07:26<6:02:09,  1.96s/it]
 40%|████      | 7530/18627 [4:07:28<6:11:13,  2.01s/it]
                                                        
{'loss': 0.8275, 'grad_norm': 4.974544525146484, 'learning_rate': 3.377657950725307e-06, 'epoch': 0.4}
+
 40%|████      | 7530/18627 [4:07:28<6:11:13,  2.01s/it]
 40%|████      | 7531/18627 [4:07:31<6:17:46,  2.04s/it]
 40%|████      | 7532/18627 [4:07:33<6:22:39,  2.07s/it]
 40%|████      | 7533/18627 [4:07:35<6:25:03,  2.08s/it]
 40%|████      | 7534/18627 [4:07:37<6:26:33,  2.09s/it]
 40%|████      | 7535/18627 [4:07:39<6:28:05,  2.10s/it]
 40%|████      | 7536/18627 [4:07:41<6:29:33,  2.11s/it]
 40%|████      | 7537/18627 [4:07:43<6:29:06,  2.11s/it]
 40%|████      | 7538/18627 [4:07:45<6:28:49,  2.10s/it]
 40%|████      | 7539/18627 [4:07:47<6:30:40,  2.11s/it]
 40%|████      | 7540/18627 [4:07:49<5:39:28,  1.84s/it]
                                                        
{'loss': 0.8031, 'grad_norm': 12.928973197937012, 'learning_rate': 3.3735863971399572e-06, 'epoch': 0.4}
+
 40%|████      | 7540/18627 [4:07:49<5:39:28,  1.84s/it]
 40%|████      | 7541/18627 [4:07:51<5:55:45,  1.93s/it]
 40%|████      | 7542/18627 [4:07:53<6:06:28,  1.98s/it]
 40%|████      | 7543/18627 [4:07:54<5:22:17,  1.74s/it]
 41%|████      | 7544/18627 [4:07:56<5:44:54,  1.87s/it]
 41%|████      | 7545/18627 [4:07:58<6:00:19,  1.95s/it]
 41%|████      | 7546/18627 [4:08:01<6:24:09,  2.08s/it]
 41%|████      | 7547/18627 [4:08:02<5:53:47,  1.92s/it]
 41%|████      | 7548/18627 [4:08:04<5:32:09,  1.80s/it]
 41%|████      | 7549/18627 [4:08:06<6:06:35,  1.99s/it]
 41%|████      | 7550/18627 [4:08:08<5:39:11,  1.84s/it]
                                                        
{'loss': 1.3894, 'grad_norm': 13.713397026062012, 'learning_rate': 3.369512202451629e-06, 'epoch': 0.41}
+
 41%|████      | 7550/18627 [4:08:08<5:39:11,  1.84s/it]
 41%|████      | 7551/18627 [4:08:10<6:10:20,  2.01s/it]
 41%|████      | 7552/18627 [4:08:13<6:36:42,  2.15s/it]
 41%|████      | 7553/18627 [4:08:15<6:54:07,  2.24s/it]
 41%|████      | 7554/18627 [4:08:17<6:13:53,  2.03s/it]
 41%|████      | 7555/18627 [4:08:18<5:43:33,  1.86s/it]
 41%|████      | 7556/18627 [4:08:20<6:15:02,  2.03s/it]
 41%|████      | 7557/18627 [4:08:23<6:35:44,  2.14s/it]
 41%|████      | 7558/18627 [4:08:25<6:50:46,  2.23s/it]
 41%|████      | 7559/18627 [4:08:28<7:00:43,  2.28s/it]
 41%|████      | 7560/18627 [4:08:30<7:05:41,  2.31s/it]
                                                        
{'loss': 1.0684, 'grad_norm': 9.99059772491455, 'learning_rate': 3.365435378977785e-06, 'epoch': 0.41}
+
 41%|████      | 7560/18627 [4:08:30<7:05:41,  2.31s/it]
 41%|████      | 7561/18627 [4:08:32<6:22:05,  2.07s/it]
 41%|████      | 7562/18627 [4:08:33<5:34:11,  1.81s/it]
 41%|████      | 7563/18627 [4:08:35<6:09:09,  2.00s/it]
 41%|████      | 7564/18627 [4:08:38<6:31:26,  2.12s/it]
 41%|████      | 7565/18627 [4:08:40<6:46:43,  2.21s/it]
 41%|████      | 7566/18627 [4:08:43<7:00:02,  2.28s/it]
 41%|████      | 7567/18627 [4:08:45<7:07:24,  2.32s/it]
 41%|████      | 7568/18627 [4:08:47<7:12:13,  2.35s/it]
 41%|████      | 7569/18627 [4:08:50<7:16:46,  2.37s/it]
 41%|████      | 7570/18627 [4:08:51<6:23:36,  2.08s/it]
                                                        
{'loss': 1.1266, 'grad_norm': 13.649321556091309, 'learning_rate': 3.361355939043833e-06, 'epoch': 0.41}
+
 41%|████      | 7570/18627 [4:08:51<6:23:36,  2.08s/it]
 41%|████      | 7571/18627 [4:08:54<6:42:38,  2.19s/it]
 41%|████      | 7572/18627 [4:08:56<6:56:08,  2.26s/it]
 41%|████      | 7573/18627 [4:08:58<6:16:43,  2.04s/it]
 41%|████      | 7574/18627 [4:09:00<6:36:59,  2.16s/it]
 41%|████      | 7575/18627 [4:09:02<6:53:25,  2.24s/it]
 41%|████      | 7576/18627 [4:09:05<7:03:43,  2.30s/it]
 41%|████      | 7577/18627 [4:09:07<7:11:17,  2.34s/it]
 41%|████      | 7578/18627 [4:09:09<6:22:14,  2.08s/it]
 41%|████      | 7579/18627 [4:09:11<6:42:25,  2.19s/it]
 41%|████      | 7580/18627 [4:09:14<6:56:38,  2.26s/it]
                                                        
{'loss': 1.0098, 'grad_norm': 5.922238349914551, 'learning_rate': 3.3572738949830965e-06, 'epoch': 0.41}
+
 41%|████      | 7580/18627 [4:09:14<6:56:38,  2.26s/it]
 41%|████      | 7581/18627 [4:09:16<7:06:06,  2.31s/it]
 41%|████      | 7582/18627 [4:09:19<7:12:29,  2.35s/it]
 41%|████      | 7583/18627 [4:09:21<7:16:54,  2.37s/it]
 41%|████      | 7584/18627 [4:09:23<7:20:56,  2.40s/it]
 41%|████      | 7585/18627 [4:09:26<7:22:15,  2.40s/it]
 41%|████      | 7586/18627 [4:09:28<7:25:45,  2.42s/it]
 41%|████      | 7587/18627 [4:09:31<7:24:21,  2.41s/it]
 41%|████      | 7588/18627 [4:09:33<7:24:20,  2.42s/it]
 41%|████      | 7589/18627 [4:09:36<7:24:26,  2.42s/it]
 41%|████      | 7590/18627 [4:09:37<6:36:26,  2.16s/it]
                                                        
{'loss': 0.8531, 'grad_norm': 11.885506629943848, 'learning_rate': 3.353189259136767e-06, 'epoch': 0.41}
+
 41%|████      | 7590/18627 [4:09:37<6:36:26,  2.16s/it]
 41%|████      | 7591/18627 [4:09:39<6:50:45,  2.23s/it]
 41%|████      | 7592/18627 [4:09:42<6:58:35,  2.28s/it]
 41%|████      | 7593/18627 [4:09:44<7:04:42,  2.31s/it]
 41%|████      | 7594/18627 [4:09:47<7:07:25,  2.32s/it]
 41%|████      | 7595/18627 [4:09:49<7:11:18,  2.35s/it]
 41%|████      | 7596/18627 [4:09:51<7:16:13,  2.37s/it]
 41%|████      | 7597/18627 [4:09:54<7:15:02,  2.37s/it]
 41%|████      | 7598/18627 [4:09:56<7:16:03,  2.37s/it]
 41%|████      | 7599/18627 [4:09:59<7:16:51,  2.38s/it]
 41%|████      | 7600/18627 [4:10:01<7:18:42,  2.39s/it]
                                                        
{'loss': 0.6209, 'grad_norm': 4.846322536468506, 'learning_rate': 3.349102043853874e-06, 'epoch': 0.41}
+
 41%|████      | 7600/18627 [4:10:01<7:18:42,  2.39s/it]
 41%|████      | 7601/18627 [4:10:03<7:20:50,  2.40s/it]
 41%|████      | 7602/18627 [4:10:05<6:34:22,  2.15s/it]
 41%|████      | 7603/18627 [4:10:06<5:57:56,  1.95s/it]
 41%|████      | 7604/18627 [4:10:09<6:23:06,  2.09s/it]
 41%|████      | 7605/18627 [4:10:11<6:42:53,  2.19s/it]
 41%|████      | 7606/18627 [4:10:14<6:56:32,  2.27s/it]
 41%|████      | 7607/18627 [4:10:16<7:05:15,  2.32s/it]
 41%|████      | 7608/18627 [4:10:19<7:09:09,  2.34s/it]
 41%|████      | 7609/18627 [4:10:21<7:14:14,  2.36s/it]
 41%|████      | 7610/18627 [4:10:23<7:19:21,  2.39s/it]
                                                        
{'loss': 0.9809, 'grad_norm': 6.769039154052734, 'learning_rate': 3.345012261491245e-06, 'epoch': 0.41}
+
 41%|████      | 7610/18627 [4:10:23<7:19:21,  2.39s/it]
 41%|████      | 7611/18627 [4:10:25<6:38:24,  2.17s/it]
 41%|████      | 7612/18627 [4:10:27<6:51:39,  2.24s/it]
 41%|████      | 7613/18627 [4:10:30<7:00:32,  2.29s/it]
 41%|████      | 7614/18627 [4:10:32<7:07:52,  2.33s/it]
 41%|████      | 7615/18627 [4:10:35<7:12:44,  2.36s/it]
 41%|████      | 7616/18627 [4:10:37<7:16:21,  2.38s/it]
 41%|████      | 7617/18627 [4:10:40<7:18:33,  2.39s/it]
 41%|████      | 7618/18627 [4:10:42<7:21:19,  2.41s/it]
 41%|████      | 7619/18627 [4:10:44<7:22:32,  2.41s/it]
 41%|████      | 7620/18627 [4:10:47<7:25:37,  2.43s/it]
                                                        
{'loss': 0.8612, 'grad_norm': 5.1128740310668945, 'learning_rate': 3.340919924413469e-06, 'epoch': 0.41}
+
 41%|████      | 7620/18627 [4:10:47<7:25:37,  2.43s/it]
 41%|████      | 7621/18627 [4:10:49<7:24:54,  2.43s/it]
 41%|████      | 7622/18627 [4:10:52<7:23:25,  2.42s/it]
 41%|████      | 7623/18627 [4:10:54<7:12:42,  2.36s/it]
 41%|████      | 7624/18627 [4:10:56<7:03:48,  2.31s/it]
 41%|████      | 7625/18627 [4:10:58<6:57:45,  2.28s/it]
 41%|████      | 7626/18627 [4:11:01<6:53:06,  2.25s/it]
 41%|████      | 7627/18627 [4:11:03<6:49:49,  2.24s/it]
 41%|████      | 7628/18627 [4:11:05<6:47:15,  2.22s/it]
 41%|████      | 7629/18627 [4:11:07<6:46:59,  2.22s/it]
 41%|████      | 7630/18627 [4:11:09<6:48:07,  2.23s/it]
                                                        
{'loss': 0.6683, 'grad_norm': 6.599756717681885, 'learning_rate': 3.3368250449928583e-06, 'epoch': 0.41}
+
 41%|████      | 7630/18627 [4:11:09<6:48:07,  2.23s/it]
 41%|████      | 7631/18627 [4:11:11<5:55:11,  1.94s/it]
 41%|████      | 7632/18627 [4:11:13<6:09:52,  2.02s/it]
 41%|████      | 7633/18627 [4:11:15<6:21:45,  2.08s/it]
 41%|████      | 7634/18627 [4:11:17<6:28:04,  2.12s/it]
 41%|████      | 7635/18627 [4:11:20<6:37:16,  2.17s/it]
 41%|████      | 7636/18627 [4:11:22<6:41:01,  2.19s/it]
 41%|████      | 7637/18627 [4:11:23<5:33:46,  1.82s/it]
 41%|████      | 7638/18627 [4:11:25<5:57:19,  1.95s/it]
 41%|████      | 7639/18627 [4:11:27<6:16:19,  2.05s/it]
 41%|████      | 7640/18627 [4:11:30<6:29:04,  2.12s/it]
                                                        
{'loss': 0.8461, 'grad_norm': 9.075181007385254, 'learning_rate': 3.332727635609411e-06, 'epoch': 0.41}
+
 41%|████      | 7640/18627 [4:11:30<6:29:04,  2.12s/it]
 41%|████      | 7641/18627 [4:11:32<6:40:14,  2.19s/it]
 41%|████      | 7642/18627 [4:11:34<6:46:18,  2.22s/it]
 41%|████      | 7643/18627 [4:11:35<5:39:36,  1.86s/it]
 41%|████      | 7644/18627 [4:11:38<6:01:34,  1.98s/it]
 41%|████      | 7645/18627 [4:11:40<6:18:24,  2.07s/it]
 41%|████      | 7646/18627 [4:11:42<6:30:39,  2.13s/it]
 41%|████      | 7647/18627 [4:11:44<6:38:44,  2.18s/it]
 41%|████      | 7648/18627 [4:11:47<6:45:22,  2.22s/it]
 41%|████      | 7649/18627 [4:11:49<6:49:50,  2.24s/it]
 41%|████      | 7650/18627 [4:11:51<6:53:22,  2.26s/it]
                                                        
{'loss': 0.6353, 'grad_norm': 7.2439422607421875, 'learning_rate': 3.328627708650774e-06, 'epoch': 0.41}
+
 41%|████      | 7650/18627 [4:11:51<6:53:22,  2.26s/it]
 41%|████      | 7651/18627 [4:11:54<6:55:35,  2.27s/it]
 41%|████      | 7652/18627 [4:11:56<6:55:12,  2.27s/it]
 41%|████      | 7653/18627 [4:11:57<6:02:43,  1.98s/it]
 41%|████      | 7654/18627 [4:11:59<6:20:09,  2.08s/it]
 41%|████      | 7655/18627 [4:12:02<6:32:01,  2.14s/it]
 41%|████      | 7656/18627 [4:12:04<6:38:50,  2.18s/it]
 41%|████      | 7657/18627 [4:12:06<6:44:38,  2.21s/it]
 41%|████      | 7658/18627 [4:12:09<6:47:28,  2.23s/it]
 41%|████      | 7659/18627 [4:12:10<5:55:48,  1.95s/it]
 41%|████      | 7660/18627 [4:12:12<6:13:10,  2.04s/it]
                                                        
{'loss': 1.1172, 'grad_norm': 7.6226887702941895, 'learning_rate': 3.3245252765122062e-06, 'epoch': 0.41}
+
 41%|████      | 7660/18627 [4:12:12<6:13:10,  2.04s/it]
 41%|████      | 7661/18627 [4:12:13<5:32:52,  1.82s/it]
 41%|████      | 7662/18627 [4:12:15<5:04:59,  1.67s/it]
 41%|████      | 7663/18627 [4:12:16<4:44:25,  1.56s/it]
 41%|████      | 7664/18627 [4:12:18<5:22:39,  1.77s/it]
 41%|████      | 7665/18627 [4:12:20<4:56:31,  1.62s/it]
 41%|████      | 7666/18627 [4:12:22<5:32:42,  1.82s/it]
 41%|████      | 7667/18627 [4:12:24<5:57:34,  1.96s/it]
 41%|████      | 7668/18627 [4:12:26<6:14:18,  2.05s/it]
 41%|████      | 7669/18627 [4:12:27<5:16:22,  1.73s/it]
 41%|████      | 7670/18627 [4:12:29<4:51:29,  1.60s/it]
                                                        
{'loss': 1.7491, 'grad_norm': 14.243639945983887, 'learning_rate': 3.3204203515965406e-06, 'epoch': 0.41}
+
 41%|████      | 7670/18627 [4:12:29<4:51:29,  1.60s/it]
 41%|████      | 7671/18627 [4:12:31<5:30:24,  1.81s/it]
 41%|████      | 7672/18627 [4:12:33<5:56:47,  1.95s/it]
 41%|████      | 7673/18627 [4:12:36<6:14:51,  2.05s/it]
 41%|████      | 7674/18627 [4:12:38<6:28:04,  2.13s/it]
 41%|████      | 7675/18627 [4:12:40<6:36:50,  2.17s/it]
 41%|████      | 7676/18627 [4:12:42<6:43:20,  2.21s/it]
 41%|████      | 7677/18627 [4:12:45<6:47:17,  2.23s/it]
 41%|████      | 7678/18627 [4:12:47<6:50:57,  2.25s/it]
 41%|████      | 7679/18627 [4:12:49<6:52:59,  2.26s/it]
 41%|████      | 7680/18627 [4:12:52<6:53:17,  2.27s/it]
                                                        
{'loss': 0.6509, 'grad_norm': 5.846193790435791, 'learning_rate': 3.316312946314145e-06, 'epoch': 0.41}
+
 41%|████      | 7680/18627 [4:12:52<6:53:17,  2.27s/it]
 41%|████      | 7681/18627 [4:12:54<6:52:58,  2.26s/it]
 41%|████      | 7682/18627 [4:12:56<6:52:55,  2.26s/it]
 41%|████      | 7683/18627 [4:12:58<6:54:12,  2.27s/it]
 41%|████▏     | 7684/18627 [4:13:01<6:58:42,  2.30s/it]
 41%|████▏     | 7685/18627 [4:13:03<6:56:37,  2.28s/it]
 41%|████▏     | 7686/18627 [4:13:05<6:55:36,  2.28s/it]
 41%|████▏     | 7687/18627 [4:13:08<6:55:51,  2.28s/it]
 41%|████▏     | 7688/18627 [4:13:10<6:58:22,  2.29s/it]
 41%|████▏     | 7689/18627 [4:13:12<6:58:22,  2.29s/it]
 41%|████▏     | 7690/18627 [4:13:14<6:57:40,  2.29s/it]
                                                        
{'loss': 0.611, 'grad_norm': 8.373210906982422, 'learning_rate': 3.3122030730828884e-06, 'epoch': 0.41}
+
 41%|████▏     | 7690/18627 [4:13:14<6:57:40,  2.29s/it]
 41%|████▏     | 7691/18627 [4:13:17<6:56:19,  2.28s/it]
 41%|████▏     | 7692/18627 [4:13:19<6:56:34,  2.29s/it]
 41%|████▏     | 7693/18627 [4:13:21<6:57:54,  2.29s/it]
 41%|████▏     | 7694/18627 [4:13:22<5:46:04,  1.90s/it]
 41%|████▏     | 7695/18627 [4:13:25<6:08:09,  2.02s/it]
 41%|████▏     | 7696/18627 [4:13:27<6:21:26,  2.09s/it]
 41%|████▏     | 7697/18627 [4:13:28<5:37:32,  1.85s/it]
 41%|████▏     | 7698/18627 [4:13:30<5:58:45,  1.97s/it]
 41%|████▏     | 7699/18627 [4:13:33<6:14:17,  2.06s/it]
 41%|████▏     | 7700/18627 [4:13:35<6:26:20,  2.12s/it]
                                                        
{'loss': 0.789, 'grad_norm': 4.798507213592529, 'learning_rate': 3.308090744328099e-06, 'epoch': 0.41}
+
 41%|████▏     | 7700/18627 [4:13:35<6:26:20,  2.12s/it]
 41%|████▏     | 7701/18627 [4:13:36<5:38:33,  1.86s/it]
 41%|████▏     | 7702/18627 [4:13:38<5:10:20,  1.70s/it]
 41%|████▏     | 7703/18627 [4:13:40<5:39:51,  1.87s/it]
 41%|████▏     | 7704/18627 [4:13:41<5:08:10,  1.69s/it]
 41%|████▏     | 7705/18627 [4:13:42<4:46:30,  1.57s/it]
 41%|████▏     | 7706/18627 [4:13:45<5:25:33,  1.79s/it]
 41%|████▏     | 7707/18627 [4:13:46<4:57:27,  1.63s/it]
 41%|████▏     | 7708/18627 [4:13:47<4:38:34,  1.53s/it]
 41%|████▏     | 7709/18627 [4:13:49<4:27:26,  1.47s/it]
 41%|████▏     | 7710/18627 [4:13:51<5:14:18,  1.73s/it]
                                                        
{'loss': 2.0175, 'grad_norm': 6.225909233093262, 'learning_rate': 3.3039759724825302e-06, 'epoch': 0.41}
+
 41%|████▏     | 7710/18627 [4:13:51<5:14:18,  1.73s/it]
 41%|████▏     | 7711/18627 [4:13:53<5:45:23,  1.90s/it]
 41%|████▏     | 7712/18627 [4:13:55<6:06:21,  2.01s/it]
 41%|████▏     | 7713/18627 [4:13:58<6:21:44,  2.10s/it]
 41%|████▏     | 7714/18627 [4:14:00<6:33:17,  2.16s/it]
 41%|████▏     | 7715/18627 [4:14:02<6:41:16,  2.21s/it]
 41%|████▏     | 7716/18627 [4:14:05<6:45:51,  2.23s/it]
 41%|████▏     | 7717/18627 [4:14:07<6:48:26,  2.25s/it]
 41%|████▏     | 7718/18627 [4:14:09<6:49:37,  2.25s/it]
 41%|████▏     | 7719/18627 [4:14:11<6:50:59,  2.26s/it]
 41%|████▏     | 7720/18627 [4:14:14<6:51:13,  2.26s/it]
                                                        
{'loss': 0.6919, 'grad_norm': 6.156948566436768, 'learning_rate': 3.2998587699863205e-06, 'epoch': 0.41}
+
 41%|████▏     | 7720/18627 [4:14:14<6:51:13,  2.26s/it]
 41%|████▏     | 7721/18627 [4:14:16<6:52:31,  2.27s/it]
 41%|████▏     | 7722/18627 [4:14:18<6:54:24,  2.28s/it]
 41%|████▏     | 7723/18627 [4:14:21<6:53:23,  2.27s/it]
 41%|████▏     | 7724/18627 [4:14:22<6:00:32,  1.98s/it]
 41%|████▏     | 7725/18627 [4:14:23<5:24:15,  1.78s/it]
 41%|████▏     | 7726/18627 [4:14:25<4:57:49,  1.64s/it]
 41%|████▏     | 7727/18627 [4:14:27<5:32:26,  1.83s/it]
 41%|████▏     | 7728/18627 [4:14:29<5:56:29,  1.96s/it]
 41%|████▏     | 7729/18627 [4:14:31<6:14:52,  2.06s/it]
 41%|████▏     | 7730/18627 [4:14:34<6:29:02,  2.14s/it]
                                                        
{'loss': 1.3024, 'grad_norm': 6.317250728607178, 'learning_rate': 3.2957391492869566e-06, 'epoch': 0.41}
+
 41%|████▏     | 7730/18627 [4:14:34<6:29:02,  2.14s/it]
 42%|████▏     | 7731/18627 [4:14:36<6:36:45,  2.18s/it]
 42%|████▏     | 7732/18627 [4:14:37<5:46:12,  1.91s/it]
 42%|████▏     | 7733/18627 [4:14:40<6:06:40,  2.02s/it]
 42%|████▏     | 7734/18627 [4:14:42<6:20:43,  2.10s/it]
 42%|████▏     | 7735/18627 [4:14:44<6:30:19,  2.15s/it]
 42%|████▏     | 7736/18627 [4:14:45<5:39:17,  1.87s/it]
 42%|████▏     | 7737/18627 [4:14:48<6:01:03,  1.99s/it]
 42%|████▏     | 7738/18627 [4:14:50<6:15:56,  2.07s/it]
 42%|████▏     | 7739/18627 [4:14:52<6:26:09,  2.13s/it]
 42%|████▏     | 7740/18627 [4:14:54<6:33:00,  2.17s/it]
                                                        
{'loss': 1.029, 'grad_norm': 6.011458396911621, 'learning_rate': 3.2916171228392386e-06, 'epoch': 0.42}
+
 42%|████▏     | 7740/18627 [4:14:54<6:33:00,  2.17s/it]
 42%|████▏     | 7741/18627 [4:14:57<6:37:51,  2.19s/it]
 42%|████▏     | 7742/18627 [4:14:58<5:46:03,  1.91s/it]
 42%|████▏     | 7743/18627 [4:15:00<6:06:07,  2.02s/it]
 42%|████▏     | 7744/18627 [4:15:02<6:18:54,  2.09s/it]
 42%|████▏     | 7745/18627 [4:15:03<5:14:57,  1.74s/it]
 42%|████▏     | 7746/18627 [4:15:06<5:43:38,  1.89s/it]
 42%|████▏     | 7747/18627 [4:15:08<6:05:47,  2.02s/it]
 42%|████▏     | 7748/18627 [4:15:10<6:21:36,  2.10s/it]
 42%|████▏     | 7749/18627 [4:15:11<5:34:50,  1.85s/it]
 42%|████▏     | 7750/18627 [4:15:14<5:57:28,  1.97s/it]
                                                        
{'loss': 0.9962, 'grad_norm': 6.314315319061279, 'learning_rate': 3.2874927031052377e-06, 'epoch': 0.42}
+
 42%|████▏     | 7750/18627 [4:15:14<5:57:28,  1.97s/it]
 42%|████▏     | 7751/18627 [4:15:15<5:19:15,  1.76s/it]
 42%|████▏     | 7752/18627 [4:15:17<5:48:13,  1.92s/it]
 42%|████▏     | 7753/18627 [4:15:19<6:05:52,  2.02s/it]
 42%|████▏     | 7754/18627 [4:15:22<6:20:11,  2.10s/it]
 42%|████▏     | 7755/18627 [4:15:23<5:34:34,  1.85s/it]
 42%|████▏     | 7756/18627 [4:15:25<5:59:01,  1.98s/it]
 42%|████▏     | 7757/18627 [4:15:28<6:15:22,  2.07s/it]
 42%|████▏     | 7758/18627 [4:15:30<6:26:42,  2.13s/it]
 42%|████▏     | 7759/18627 [4:15:32<6:35:11,  2.18s/it]
 42%|████▏     | 7760/18627 [4:15:34<6:40:08,  2.21s/it]
                                                        
{'loss': 1.1461, 'grad_norm': 5.141458511352539, 'learning_rate': 3.28336590255426e-06, 'epoch': 0.42}
+
 42%|████▏     | 7760/18627 [4:15:34<6:40:08,  2.21s/it]
 42%|████▏     | 7761/18627 [4:15:37<6:45:08,  2.24s/it]
 42%|████▏     | 7762/18627 [4:15:38<5:55:26,  1.96s/it]
 42%|████▏     | 7763/18627 [4:15:39<5:11:24,  1.72s/it]
 42%|████▏     | 7764/18627 [4:15:40<4:44:33,  1.57s/it]
 42%|████▏     | 7765/18627 [4:15:43<5:21:46,  1.78s/it]
 42%|████▏     | 7766/18627 [4:15:45<5:48:12,  1.92s/it]
 42%|████▏     | 7767/18627 [4:15:46<5:12:10,  1.72s/it]
 42%|████▏     | 7768/18627 [4:15:49<5:42:05,  1.89s/it]
 42%|████▏     | 7769/18627 [4:15:50<5:07:57,  1.70s/it]
 42%|████▏     | 7770/18627 [4:15:51<4:30:24,  1.49s/it]
                                                        
{'loss': 1.7377, 'grad_norm': 7.1783294677734375, 'learning_rate': 3.2792367336628126e-06, 'epoch': 0.42}
+
 42%|████▏     | 7770/18627 [4:15:51<4:30:24,  1.49s/it]
 42%|████▏     | 7771/18627 [4:15:52<4:20:21,  1.44s/it]
 42%|████▏     | 7772/18627 [4:15:53<4:13:02,  1.40s/it]
 42%|████▏     | 7773/18627 [4:15:55<4:09:18,  1.38s/it]
 42%|████▏     | 7774/18627 [4:15:58<6:08:45,  2.04s/it]
 42%|████▏     | 7775/18627 [4:16:01<6:23:03,  2.12s/it]
 42%|████▏     | 7776/18627 [4:16:03<6:30:32,  2.16s/it]
 42%|████▏     | 7777/18627 [4:16:05<6:35:53,  2.19s/it]
 42%|████▏     | 7778/18627 [4:16:06<5:46:45,  1.92s/it]
 42%|████▏     | 7779/18627 [4:16:09<6:05:46,  2.02s/it]
 42%|████▏     | 7780/18627 [4:16:10<5:27:25,  1.81s/it]
                                                        
{'loss': 1.6056, 'grad_norm': 13.391210556030273, 'learning_rate': 3.2751052089145597e-06, 'epoch': 0.42}
+
 42%|████▏     | 7780/18627 [4:16:10<5:27:25,  1.81s/it]
 42%|████▏     | 7781/18627 [4:16:12<5:52:43,  1.95s/it]
 42%|████▏     | 7782/18627 [4:16:15<6:10:06,  2.05s/it]
 42%|████▏     | 7783/18627 [4:16:17<6:24:00,  2.12s/it]
 42%|████▏     | 7784/18627 [4:16:19<6:32:57,  2.17s/it]
 42%|████▏     | 7785/18627 [4:16:21<6:34:59,  2.19s/it]
 42%|████▏     | 7786/18627 [4:16:23<5:44:10,  1.90s/it]
 42%|████▏     | 7787/18627 [4:16:25<6:04:00,  2.01s/it]
 42%|████▏     | 7788/18627 [4:16:27<6:17:58,  2.09s/it]
 42%|████▏     | 7789/18627 [4:16:29<6:29:40,  2.16s/it]
 42%|████▏     | 7790/18627 [4:16:32<6:37:01,  2.20s/it]
                                                        
{'loss': 0.864, 'grad_norm': 6.9083051681518555, 'learning_rate': 3.270971340800289e-06, 'epoch': 0.42}
+
 42%|████▏     | 7790/18627 [4:16:32<6:37:01,  2.20s/it]
 42%|████▏     | 7791/18627 [4:16:33<5:47:24,  1.92s/it]
 42%|████▏     | 7792/18627 [4:16:35<6:07:17,  2.03s/it]
 42%|████▏     | 7793/18627 [4:16:38<6:20:12,  2.11s/it]
 42%|████▏     | 7794/18627 [4:16:40<6:30:59,  2.17s/it]
 42%|████▏     | 7795/18627 [4:16:42<6:38:31,  2.21s/it]
 42%|████▏     | 7796/18627 [4:16:44<6:42:10,  2.23s/it]
 42%|████▏     | 7797/18627 [4:16:47<6:42:27,  2.23s/it]
 42%|████▏     | 7798/18627 [4:16:49<6:43:13,  2.23s/it]
 42%|████▏     | 7799/18627 [4:16:51<6:43:33,  2.24s/it]
 42%|████▏     | 7800/18627 [4:16:52<5:51:29,  1.95s/it]
                                                        
{'loss': 1.0406, 'grad_norm': 15.448162078857422, 'learning_rate': 3.2668351418178735e-06, 'epoch': 0.42}
+
 42%|████▏     | 7800/18627 [4:16:52<5:51:29,  1.95s/it]
 42%|████▏     | 7801/18627 [4:16:55<6:07:34,  2.04s/it]
 42%|████▏     | 7802/18627 [4:16:57<6:21:30,  2.11s/it]
 42%|████▏     | 7803/18627 [4:16:58<5:34:47,  1.86s/it]
 42%|████▏     | 7804/18627 [4:17:01<5:56:16,  1.98s/it]
 42%|████▏     | 7805/18627 [4:17:03<6:12:07,  2.06s/it]
 42%|████▏     | 7806/18627 [4:17:04<5:23:37,  1.79s/it]
 42%|████▏     | 7807/18627 [4:17:06<5:50:17,  1.94s/it]
 42%|████▏     | 7808/18627 [4:17:09<6:07:36,  2.04s/it]
 42%|████▏     | 7809/18627 [4:17:09<5:10:00,  1.72s/it]
 42%|████▏     | 7810/18627 [4:17:12<5:39:00,  1.88s/it]
                                                        
{'loss': 1.1413, 'grad_norm': 8.051915168762207, 'learning_rate': 3.2626966244722325e-06, 'epoch': 0.42}
+
 42%|████▏     | 7810/18627 [4:17:12<5:39:00,  1.88s/it]
 42%|████▏     | 7811/18627 [4:17:14<6:00:34,  2.00s/it]
 42%|████▏     | 7812/18627 [4:17:16<6:12:54,  2.07s/it]
 42%|████▏     | 7813/18627 [4:17:18<6:21:51,  2.12s/it]
 42%|████▏     | 7814/18627 [4:17:21<6:27:41,  2.15s/it]
 42%|████▏     | 7815/18627 [4:17:22<5:42:55,  1.90s/it]
 42%|████▏     | 7816/18627 [4:17:24<6:03:06,  2.02s/it]
 42%|████▏     | 7817/18627 [4:17:27<6:13:08,  2.07s/it]
 42%|████▏     | 7818/18627 [4:17:28<5:31:40,  1.84s/it]
 42%|████▏     | 7819/18627 [4:17:30<5:54:44,  1.97s/it]
 42%|████▏     | 7820/18627 [4:17:32<6:10:20,  2.06s/it]
                                                        
{'loss': 1.1617, 'grad_norm': 9.55269718170166, 'learning_rate': 3.2585558012752943e-06, 'epoch': 0.42}
+
 42%|████▏     | 7820/18627 [4:17:32<6:10:20,  2.06s/it]
 42%|████▏     | 7821/18627 [4:17:34<5:25:16,  1.81s/it]
 42%|████▏     | 7822/18627 [4:17:36<5:49:31,  1.94s/it]
 42%|████▏     | 7823/18627 [4:17:38<6:08:42,  2.05s/it]
 42%|████▏     | 7824/18627 [4:17:40<6:20:22,  2.11s/it]
 42%|████▏     | 7825/18627 [4:17:43<6:29:23,  2.16s/it]
 42%|████▏     | 7826/18627 [4:17:44<5:39:11,  1.88s/it]
 42%|████▏     | 7827/18627 [4:17:46<6:00:17,  2.00s/it]
 42%|████▏     | 7828/18627 [4:17:48<6:15:30,  2.09s/it]
 42%|████▏     | 7829/18627 [4:17:51<6:26:44,  2.15s/it]
 42%|████▏     | 7830/18627 [4:17:52<5:38:00,  1.88s/it]
                                                        
{'loss': 1.2802, 'grad_norm': 14.073532104492188, 'learning_rate': 3.2544126847459586e-06, 'epoch': 0.42}
+
 42%|████▏     | 7830/18627 [4:17:52<5:38:00,  1.88s/it]
 42%|████▏     | 7831/18627 [4:17:54<5:59:01,  2.00s/it]
 42%|████▏     | 7832/18627 [4:17:56<5:21:12,  1.79s/it]
 42%|████▏     | 7833/18627 [4:17:58<5:47:05,  1.93s/it]
 42%|████▏     | 7834/18627 [4:18:00<6:04:43,  2.03s/it]
 42%|████▏     | 7835/18627 [4:18:02<6:17:26,  2.10s/it]
 42%|████▏     | 7836/18627 [4:18:05<6:26:47,  2.15s/it]
 42%|████▏     | 7837/18627 [4:18:07<6:31:34,  2.18s/it]
 42%|████▏     | 7838/18627 [4:18:09<6:36:07,  2.20s/it]
 42%|████▏     | 7839/18627 [4:18:11<6:37:38,  2.21s/it]
 42%|████▏     | 7840/18627 [4:18:14<6:40:08,  2.23s/it]
                                                        
{'loss': 0.8846, 'grad_norm': 4.811277389526367, 'learning_rate': 3.2502672874100584e-06, 'epoch': 0.42}
+
 42%|████▏     | 7840/18627 [4:18:14<6:40:08,  2.23s/it]
 42%|████▏     | 7841/18627 [4:18:16<6:41:22,  2.23s/it]
 42%|████▏     | 7842/18627 [4:18:18<6:44:29,  2.25s/it]
 42%|████▏     | 7843/18627 [4:18:20<6:42:57,  2.24s/it]
 42%|████▏     | 7844/18627 [4:18:23<6:45:35,  2.26s/it]
 42%|████▏     | 7845/18627 [4:18:25<6:45:20,  2.26s/it]
 42%|████▏     | 7846/18627 [4:18:27<6:45:41,  2.26s/it]
 42%|████▏     | 7847/18627 [4:18:29<6:45:03,  2.25s/it]
 42%|████▏     | 7848/18627 [4:18:31<5:47:01,  1.93s/it]
 42%|████▏     | 7849/18627 [4:18:33<6:04:04,  2.03s/it]
 42%|████▏     | 7850/18627 [4:18:35<6:17:52,  2.10s/it]
                                                        
{'loss': 0.7893, 'grad_norm': 6.619278430938721, 'learning_rate': 3.2461196218003225e-06, 'epoch': 0.42}
+
 42%|████▏     | 7850/18627 [4:18:35<6:17:52,  2.10s/it]
 42%|████▏     | 7851/18627 [4:18:37<6:24:57,  2.14s/it]
 42%|████▏     | 7852/18627 [4:18:40<6:33:21,  2.19s/it]
 42%|████▏     | 7853/18627 [4:18:42<6:38:56,  2.22s/it]
 42%|████▏     | 7854/18627 [4:18:44<6:43:25,  2.25s/it]
 42%|████▏     | 7855/18627 [4:18:47<6:44:15,  2.25s/it]
 42%|████▏     | 7856/18627 [4:18:49<6:43:44,  2.25s/it]
 42%|████▏     | 7857/18627 [4:18:50<5:52:52,  1.97s/it]
 42%|████▏     | 7858/18627 [4:18:52<6:10:44,  2.07s/it]
 42%|████▏     | 7859/18627 [4:18:54<5:27:02,  1.82s/it]
 42%|████▏     | 7860/18627 [4:18:56<5:50:09,  1.95s/it]
                                                        
{'loss': 1.0149, 'grad_norm': 5.148118019104004, 'learning_rate': 3.241969700456337e-06, 'epoch': 0.42}
+
 42%|████▏     | 7860/18627 [4:18:56<5:50:09,  1.95s/it]
 42%|████▏     | 7861/18627 [4:18:57<5:14:25,  1.75s/it]
 42%|████▏     | 7862/18627 [4:18:59<5:38:54,  1.89s/it]
 42%|████▏     | 7863/18627 [4:19:02<5:58:10,  2.00s/it]
 42%|████▏     | 7864/18627 [4:19:04<6:11:22,  2.07s/it]
 42%|████▏     | 7865/18627 [4:19:06<6:19:36,  2.12s/it]
 42%|████▏     | 7866/18627 [4:19:08<6:27:43,  2.16s/it]
 42%|████▏     | 7867/18627 [4:19:10<5:43:20,  1.91s/it]
 42%|████▏     | 7868/18627 [4:19:11<5:03:53,  1.69s/it]
 42%|████▏     | 7869/18627 [4:19:13<5:34:34,  1.87s/it]
 42%|████▏     | 7870/18627 [4:19:14<5:03:16,  1.69s/it]
                                                        
{'loss': 1.4499, 'grad_norm': 13.387690544128418, 'learning_rate': 3.237817535924509e-06, 'epoch': 0.42}
+
 42%|████▏     | 7870/18627 [4:19:14<5:03:16,  1.69s/it]
 42%|████▏     | 7871/18627 [4:19:15<4:25:36,  1.48s/it]
 42%|████▏     | 7872/18627 [4:19:18<5:07:26,  1.72s/it]
 42%|████▏     | 7873/18627 [4:19:20<5:36:34,  1.88s/it]
 42%|████▏     | 7874/18627 [4:19:21<5:03:49,  1.70s/it]
 42%|████▏     | 7875/18627 [4:19:23<5:34:45,  1.87s/it]
 42%|████▏     | 7876/18627 [4:19:26<5:56:13,  1.99s/it]
 42%|████▏     | 7877/18627 [4:19:28<6:12:11,  2.08s/it]
 42%|████▏     | 7878/18627 [4:19:30<6:19:27,  2.12s/it]
 42%|████▏     | 7879/18627 [4:19:32<5:32:32,  1.86s/it]
 42%|████▏     | 7880/18627 [4:19:34<5:55:59,  1.99s/it]
                                                        
{'loss': 1.0466, 'grad_norm': 5.4978108406066895, 'learning_rate': 3.2336631407580248e-06, 'epoch': 0.42}
+
 42%|████▏     | 7880/18627 [4:19:34<5:55:59,  1.99s/it]
 42%|████▏     | 7881/18627 [4:19:36<6:10:28,  2.07s/it]
 42%|████▏     | 7882/18627 [4:19:37<5:26:26,  1.82s/it]
 42%|████▏     | 7883/18627 [4:19:40<5:49:57,  1.95s/it]
 42%|████▏     | 7884/18627 [4:19:41<5:15:28,  1.76s/it]
 42%|████▏     | 7885/18627 [4:19:43<5:42:40,  1.91s/it]
 42%|████▏     | 7886/18627 [4:19:44<5:08:46,  1.72s/it]
 42%|████▏     | 7887/18627 [4:19:47<5:37:55,  1.89s/it]
 42%|████▏     | 7888/18627 [4:19:49<5:57:50,  2.00s/it]
 42%|████▏     | 7889/18627 [4:19:51<6:11:29,  2.08s/it]
 42%|████▏     | 7890/18627 [4:19:53<6:21:34,  2.13s/it]
                                                        
{'loss': 1.2951, 'grad_norm': 7.373843193054199, 'learning_rate': 3.2295065275168165e-06, 'epoch': 0.42}
+
 42%|████▏     | 7890/18627 [4:19:53<6:21:34,  2.13s/it]
 42%|████▏     | 7891/18627 [4:19:56<6:27:24,  2.17s/it]
 42%|████▏     | 7892/18627 [4:19:57<5:37:16,  1.89s/it]
 42%|████▏     | 7893/18627 [4:19:59<5:57:31,  2.00s/it]
 42%|████▏     | 7894/18627 [4:20:01<6:11:06,  2.07s/it]
 42%|████▏     | 7895/18627 [4:20:04<6:19:05,  2.12s/it]
 42%|████▏     | 7896/18627 [4:20:06<6:25:40,  2.16s/it]
 42%|████▏     | 7897/18627 [4:20:07<5:37:01,  1.88s/it]
 42%|████▏     | 7898/18627 [4:20:09<5:58:26,  2.00s/it]
 42%|████▏     | 7899/18627 [4:20:11<5:18:11,  1.78s/it]
 42%|████▏     | 7900/18627 [4:20:13<5:46:31,  1.94s/it]
                                                        
{'loss': 1.1763, 'grad_norm': 8.450349807739258, 'learning_rate': 3.2253477087675217e-06, 'epoch': 0.42}
+
 42%|████▏     | 7900/18627 [4:20:13<5:46:31,  1.94s/it]
 42%|████▏     | 7901/18627 [4:20:15<6:06:32,  2.05s/it]
 42%|████▏     | 7902/18627 [4:20:18<6:17:19,  2.11s/it]
 42%|████▏     | 7903/18627 [4:20:20<6:24:36,  2.15s/it]
 42%|████▏     | 7904/18627 [4:20:22<6:28:43,  2.18s/it]
 42%|████▏     | 7905/18627 [4:20:24<6:33:01,  2.20s/it]
 42%|████▏     | 7906/18627 [4:20:27<6:37:20,  2.22s/it]
 42%|████▏     | 7907/18627 [4:20:29<6:39:56,  2.24s/it]
 42%|████▏     | 7908/18627 [4:20:31<6:42:13,  2.25s/it]
 42%|████▏     | 7909/18627 [4:20:33<6:42:29,  2.25s/it]
 42%|████▏     | 7910/18627 [4:20:35<5:50:42,  1.96s/it]
                                                        
{'loss': 0.8166, 'grad_norm': 15.04806137084961, 'learning_rate': 3.2211866970834464e-06, 'epoch': 0.42}
+
 42%|████▏     | 7910/18627 [4:20:35<5:50:42,  1.96s/it]
 42%|████▏     | 7911/18627 [4:20:37<6:05:57,  2.05s/it]
 42%|████▏     | 7912/18627 [4:20:38<5:21:38,  1.80s/it]
 42%|████▏     | 7913/18627 [4:20:40<5:48:06,  1.95s/it]
 42%|████▏     | 7914/18627 [4:20:43<6:05:18,  2.05s/it]
 42%|████▏     | 7915/18627 [4:20:44<5:22:49,  1.81s/it]
 42%|████▏     | 7916/18627 [4:20:46<5:49:06,  1.96s/it]
 43%|████▎     | 7917/18627 [4:20:49<6:07:17,  2.06s/it]
 43%|████▎     | 7918/18627 [4:20:51<6:18:25,  2.12s/it]
 43%|████▎     | 7919/18627 [4:20:53<6:24:33,  2.15s/it]
 43%|████▎     | 7920/18627 [4:20:55<6:30:07,  2.19s/it]
                                                        
{'loss': 1.0539, 'grad_norm': 9.653497695922852, 'learning_rate': 3.2170235050445255e-06, 'epoch': 0.43}
+
 43%|████▎     | 7920/18627 [4:20:55<6:30:07,  2.19s/it]
 43%|████▎     | 7921/18627 [4:20:58<6:34:27,  2.21s/it]
 43%|████▎     | 7922/18627 [4:20:59<5:40:32,  1.91s/it]
 43%|████▎     | 7923/18627 [4:21:00<5:08:00,  1.73s/it]
 43%|████▎     | 7924/18627 [4:21:02<5:37:20,  1.89s/it]
 43%|████▎     | 7925/18627 [4:21:05<5:55:56,  2.00s/it]
 43%|████▎     | 7926/18627 [4:21:07<6:09:21,  2.07s/it]
 43%|████▎     | 7927/18627 [4:21:09<6:19:08,  2.13s/it]
 43%|████▎     | 7928/18627 [4:21:11<6:26:55,  2.17s/it]
 43%|████▎     | 7929/18627 [4:21:14<6:30:47,  2.19s/it]
 43%|████▎     | 7930/18627 [4:21:16<6:34:36,  2.21s/it]
                                                        
{'loss': 1.033, 'grad_norm': 9.561826705932617, 'learning_rate': 3.212858145237285e-06, 'epoch': 0.43}
+
 43%|████▎     | 7930/18627 [4:21:16<6:34:36,  2.21s/it]
 43%|████▎     | 7931/18627 [4:21:18<6:40:06,  2.24s/it]
 43%|████▎     | 7932/18627 [4:21:21<6:41:43,  2.25s/it]
 43%|████▎     | 7933/18627 [4:21:23<6:43:19,  2.26s/it]
 43%|████▎     | 7934/18627 [4:21:24<5:51:03,  1.97s/it]
 43%|████▎     | 7935/18627 [4:21:26<6:09:21,  2.07s/it]
 43%|████▎     | 7936/18627 [4:21:29<6:21:57,  2.14s/it]
 43%|████▎     | 7937/18627 [4:21:31<6:29:18,  2.19s/it]
 43%|████▎     | 7938/18627 [4:21:33<6:36:06,  2.22s/it]
 43%|████▎     | 7939/18627 [4:21:36<6:37:38,  2.23s/it]
 43%|████▎     | 7940/18627 [4:21:38<6:40:10,  2.25s/it]
                                                        
{'loss': 0.8647, 'grad_norm': 6.748555660247803, 'learning_rate': 3.208690630254808e-06, 'epoch': 0.43}
+
 43%|████▎     | 7940/18627 [4:21:38<6:40:10,  2.25s/it]
 43%|████▎     | 7941/18627 [4:21:40<6:43:17,  2.26s/it]
 43%|████▎     | 7942/18627 [4:21:41<5:50:21,  1.97s/it]
 43%|████▎     | 7943/18627 [4:21:44<6:05:54,  2.05s/it]
 43%|████▎     | 7944/18627 [4:21:45<5:07:35,  1.73s/it]
 43%|████▎     | 7945/18627 [4:21:47<5:36:19,  1.89s/it]
 43%|████▎     | 7946/18627 [4:21:49<5:55:56,  2.00s/it]
 43%|████▎     | 7947/18627 [4:21:51<6:11:10,  2.09s/it]
 43%|████▎     | 7948/18627 [4:21:54<6:19:57,  2.13s/it]
 43%|████▎     | 7949/18627 [4:21:55<5:35:55,  1.89s/it]
 43%|████▎     | 7950/18627 [4:21:57<5:56:18,  2.00s/it]
                                                        
{'loss': 1.0937, 'grad_norm': 4.782110691070557, 'learning_rate': 3.2045209726966895e-06, 'epoch': 0.43}
+
 43%|████▎     | 7950/18627 [4:21:57<5:56:18,  2.00s/it]
 43%|████▎     | 7951/18627 [4:22:00<6:09:46,  2.08s/it]
 43%|████▎     | 7952/18627 [4:22:01<5:28:00,  1.84s/it]
 43%|████▎     | 7953/18627 [4:22:03<5:49:41,  1.97s/it]
 43%|████▎     | 7954/18627 [4:22:05<6:07:03,  2.06s/it]
 43%|████▎     | 7955/18627 [4:22:08<6:17:48,  2.12s/it]
 43%|████▎     | 7956/18627 [4:22:10<6:25:29,  2.17s/it]
 43%|████▎     | 7957/18627 [4:22:12<6:29:54,  2.19s/it]
 43%|████▎     | 7958/18627 [4:22:14<6:34:49,  2.22s/it]
 43%|████▎     | 7959/18627 [4:22:17<6:37:13,  2.23s/it]
 43%|████▎     | 7960/18627 [4:22:19<6:40:23,  2.25s/it]
                                                        
{'loss': 0.9565, 'grad_norm': 5.785258769989014, 'learning_rate': 3.200349185169004e-06, 'epoch': 0.43}
+
 43%|████▎     | 7960/18627 [4:22:19<6:40:23,  2.25s/it]
 43%|████▎     | 7961/18627 [4:22:21<6:40:52,  2.26s/it]
 43%|████▎     | 7962/18627 [4:22:23<5:49:36,  1.97s/it]
 43%|████▎     | 7963/18627 [4:22:25<6:06:31,  2.06s/it]
 43%|████▎     | 7964/18627 [4:22:27<6:16:47,  2.12s/it]
 43%|████▎     | 7965/18627 [4:22:28<5:27:43,  1.84s/it]
 43%|████▎     | 7966/18627 [4:22:31<5:50:11,  1.97s/it]
 43%|████▎     | 7967/18627 [4:22:33<6:05:33,  2.06s/it]
 43%|████▎     | 7968/18627 [4:22:35<6:16:57,  2.12s/it]
 43%|████▎     | 7969/18627 [4:22:36<5:33:27,  1.88s/it]
 43%|████▎     | 7970/18627 [4:22:39<5:55:52,  2.00s/it]
                                                        
{'loss': 1.3113, 'grad_norm': 4.441384792327881, 'learning_rate': 3.196175280284266e-06, 'epoch': 0.43}
+
 43%|████▎     | 7970/18627 [4:22:39<5:55:52,  2.00s/it]
 43%|████▎     | 7971/18627 [4:22:41<6:10:35,  2.09s/it]
 43%|████▎     | 7972/18627 [4:22:43<6:23:16,  2.16s/it]
 43%|████▎     | 7973/18627 [4:22:46<6:30:39,  2.20s/it]
 43%|████▎     | 7974/18627 [4:22:48<6:35:21,  2.23s/it]
 43%|████▎     | 7975/18627 [4:22:50<6:36:57,  2.24s/it]
 43%|████▎     | 7976/18627 [4:22:52<6:37:45,  2.24s/it]
 43%|████▎     | 7977/18627 [4:22:55<6:39:28,  2.25s/it]
 43%|���███▎     | 7978/18627 [4:22:57<6:41:07,  2.26s/it]
 43%|████▎     | 7979/18627 [4:22:59<6:41:43,  2.26s/it]
 43%|████▎     | 7980/18627 [4:23:02<6:42:38,  2.27s/it]
                                                        
{'loss': 0.6548, 'grad_norm': 7.647723197937012, 'learning_rate': 3.1919992706613902e-06, 'epoch': 0.43}
+
 43%|████▎     | 7980/18627 [4:23:02<6:42:38,  2.27s/it]
 43%|████▎     | 7981/18627 [4:23:03<5:50:11,  1.97s/it]
 43%|████▎     | 7982/18627 [4:23:05<6:05:12,  2.06s/it]
 43%|████▎     | 7983/18627 [4:23:07<6:16:22,  2.12s/it]
 43%|████▎     | 7984/18627 [4:23:10<6:25:21,  2.17s/it]
 43%|████▎     | 7985/18627 [4:23:12<6:30:15,  2.20s/it]
 43%|████▎     | 7986/18627 [4:23:14<6:35:09,  2.23s/it]
 43%|████▎     | 7987/18627 [4:23:16<6:36:51,  2.24s/it]
 43%|████▎     | 7988/18627 [4:23:19<6:38:02,  2.24s/it]
 43%|████▎     | 7989/18627 [4:23:21<6:38:18,  2.25s/it]
 43%|████▎     | 7990/18627 [4:23:22<5:44:46,  1.94s/it]
                                                        
{'loss': 1.0902, 'grad_norm': 13.294577598571777, 'learning_rate': 3.187821168925655e-06, 'epoch': 0.43}
+
 43%|████▎     | 7990/18627 [4:23:22<5:44:46,  1.94s/it]
 43%|████▎     | 7991/18627 [4:23:24<6:01:51,  2.04s/it]
 43%|████▎     | 7992/18627 [4:23:27<6:13:52,  2.11s/it]
 43%|████▎     | 7993/18627 [4:23:29<6:24:11,  2.17s/it]
 43%|████▎     | 7994/18627 [4:23:30<5:38:58,  1.91s/it]
 43%|████▎     | 7995/18627 [4:23:32<5:07:13,  1.73s/it]
 43%|████▎     | 7996/18627 [4:23:34<5:36:24,  1.90s/it]
 43%|████▎     | 7997/18627 [4:23:36<5:56:27,  2.01s/it]
 43%|████▎     | 7998/18627 [4:23:38<5:18:35,  1.80s/it]
 43%|████▎     | 7999/18627 [4:23:40<5:42:29,  1.93s/it]
 43%|████▎     | 8000/18627 [4:23:42<5:59:30,  2.03s/it]
                                                        
{'loss': 1.3198, 'grad_norm': 5.372689723968506, 'learning_rate': 3.1836409877086638e-06, 'epoch': 0.43}
+
 43%|████▎     | 8000/18627 [4:23:42<5:59:30,  2.03s/it]/home/zhengduo/workspace/LLaMA-Factory/src/qwen_vl/model/modeling_qwen2_5_vl.py:1979: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
+  with torch.cuda.amp.autocast(dtype=dtype):
+/home/zhengduo/miniconda3/envs/vgllm/lib/python3.10/site-packages/torch/utils/checkpoint.py:87: UserWarning: None of the inputs have requires_grad=True. Gradients will be None
+  warnings.warn(
+
 43%|████▎     | 8001/18627 [4:24:27<44:23:40, 15.04s/it]
 43%|████▎     | 8002/18627 [4:24:30<33:04:41, 11.21s/it]
 43%|████▎     | 8003/18627 [4:24:31<24:17:25,  8.23s/it]
 43%|████▎     | 8004/18627 [4:24:32<18:05:03,  6.13s/it]
 43%|████▎     | 8005/18627 [4:24:34<14:36:58,  4.95s/it]
 43%|████▎     | 8006/18627 [4:24:37<12:11:50,  4.13s/it]
 43%|████▎     | 8007/18627 [4:24:39<10:28:46,  3.55s/it]
 43%|████▎     | 8008/18627 [4:24:41<9:18:02,  3.15s/it] 
 43%|████▎     | 8009/18627 [4:24:43<8:27:26,  2.87s/it]
 43%|████▎     | 8010/18627 [4:24:44<7:00:49,  2.38s/it]
                                                        
{'loss': 1.4518, 'grad_norm': 13.624720573425293, 'learning_rate': 3.179458739648307e-06, 'epoch': 0.43}
+
 43%|████▎     | 8010/18627 [4:24:44<7:00:49,  2.38s/it]
 43%|████▎     | 8011/18627 [4:24:47<6:51:57,  2.33s/it]
 43%|████▎     | 8012/18627 [4:24:49<6:46:05,  2.30s/it]
 43%|████▎     | 8013/18627 [4:24:51<6:41:06,  2.27s/it]
 43%|████▎     | 8014/18627 [4:24:53<6:37:03,  2.24s/it]
 43%|████▎     | 8015/18627 [4:24:55<6:33:55,  2.23s/it]
 43%|████▎     | 8016/18627 [4:24:57<5:43:23,  1.94s/it]
 43%|████▎     | 8017/18627 [4:24:58<5:04:38,  1.72s/it]
 43%|████▎     | 8018/18627 [4:25:00<5:33:52,  1.89s/it]
 43%|████▎     | 8019/18627 [4:25:02<5:51:42,  1.99s/it]
 43%|████▎     | 8020/18627 [4:25:05<6:03:34,  2.06s/it]
                                                        
{'loss': 1.0651, 'grad_norm': 5.792115211486816, 'learning_rate': 3.1752744373887227e-06, 'epoch': 0.43}
+
 43%|████▎     | 8020/18627 [4:25:05<6:03:34,  2.06s/it]
 43%|████▎     | 8021/18627 [4:25:07<6:11:24,  2.10s/it]
 43%|████▎     | 8022/18627 [4:25:09<6:15:53,  2.13s/it]
 43%|████▎     | 8023/18627 [4:25:11<6:22:41,  2.17s/it]
 43%|████▎     | 8024/18627 [4:25:14<6:26:08,  2.19s/it]
 43%|████▎     | 8025/18627 [4:25:16<6:28:42,  2.20s/it]
 43%|████▎     | 8026/18627 [4:25:17<5:37:35,  1.91s/it]
 43%|████▎     | 8027/18627 [4:25:19<5:53:43,  2.00s/it]
 43%|████▎     | 8028/18627 [4:25:21<6:04:43,  2.06s/it]
 43%|████▎     | 8029/18627 [4:25:23<5:20:54,  1.82s/it]
 43%|████▎     | 8030/18627 [4:25:25<5:38:34,  1.92s/it]
                                                        
{'loss': 1.0533, 'grad_norm': 6.3531084060668945, 'learning_rate': 3.1710880935802624e-06, 'epoch': 0.43}
+
 43%|████▎     | 8030/18627 [4:25:25<5:38:34,  1.92s/it]
 43%|████▎     | 8031/18627 [4:25:27<5:51:15,  1.99s/it]
 43%|████▎     | 8032/18627 [4:25:29<6:00:03,  2.04s/it]
 43%|████▎     | 8033/18627 [4:25:30<5:16:48,  1.79s/it]
 43%|████▎     | 8034/18627 [4:25:33<5:36:22,  1.91s/it]
 43%|████▎     | 8035/18627 [4:25:35<5:50:51,  1.99s/it]
 43%|████▎     | 8036/18627 [4:25:36<4:51:20,  1.65s/it]
 43%|████▎     | 8037/18627 [4:25:37<4:30:18,  1.53s/it]
 43%|████▎     | 8038/18627 [4:25:39<5:07:15,  1.74s/it]
 43%|████▎     | 8039/18627 [4:25:40<4:22:40,  1.49s/it]
 43%|████▎     | 8040/18627 [4:25:41<4:11:44,  1.43s/it]
                                                        
{'loss': 1.2189, 'grad_norm': 13.421025276184082, 'learning_rate': 3.1668997208794455e-06, 'epoch': 0.43}
+
 43%|████▎     | 8040/18627 [4:25:41<4:11:44,  1.43s/it]
 43%|████▎     | 8041/18627 [4:25:44<4:58:14,  1.69s/it]
 43%|████▎     | 8042/18627 [4:25:46<5:32:16,  1.88s/it]
 43%|████▎     | 8043/18627 [4:25:48<6:00:47,  2.05s/it]
 43%|████▎     | 8044/18627 [4:25:51<6:25:35,  2.19s/it]
 43%|████▎     | 8045/18627 [4:25:53<6:39:12,  2.26s/it]
 43%|████▎     | 8046/18627 [4:25:55<6:06:00,  2.08s/it]
 43%|████▎     | 8047/18627 [4:25:57<6:23:25,  2.17s/it]
 43%|████▎     | 8048/18627 [4:25:59<5:54:01,  2.01s/it]
 43%|████▎     | 8049/18627 [4:26:01<6:18:52,  2.15s/it]
 43%|████▎     | 8050/18627 [4:26:04<6:36:13,  2.25s/it]
                                                        
{'loss': 1.0535, 'grad_norm': 7.6082305908203125, 'learning_rate': 3.1627093319489288e-06, 'epoch': 0.43}
+
 43%|████▎     | 8050/18627 [4:26:04<6:36:13,  2.25s/it]
 43%|████▎     | 8051/18627 [4:26:06<6:46:18,  2.31s/it]
 43%|████▎     | 8052/18627 [4:26:08<6:07:33,  2.09s/it]
 43%|████▎     | 8053/18627 [4:26:09<5:37:05,  1.91s/it]
 43%|████▎     | 8054/18627 [4:26:12<5:52:50,  2.00s/it]
 43%|████▎     | 8055/18627 [4:26:14<6:04:28,  2.07s/it]
 43%|████▎     | 8056/18627 [4:26:16<6:12:19,  2.11s/it]
 43%|████▎     | 8057/18627 [4:26:17<5:25:54,  1.85s/it]
 43%|████▎     | 8058/18627 [4:26:20<5:44:27,  1.96s/it]
 43%|████▎     | 8059/18627 [4:26:20<4:47:48,  1.63s/it]
 43%|████▎     | 8060/18627 [4:26:23<5:19:44,  1.82s/it]
                                                        
{'loss': 1.2032, 'grad_norm': 6.5823187828063965, 'learning_rate': 3.1585169394574626e-06, 'epoch': 0.43}
+
 43%|████▎     | 8060/18627 [4:26:23<5:19:44,  1.82s/it]
 43%|████▎     | 8061/18627 [4:26:25<5:41:01,  1.94s/it]
 43%|████▎     | 8062/18627 [4:26:27<5:54:53,  2.02s/it]
 43%|████▎     | 8063/18627 [4:26:28<5:17:10,  1.80s/it]
 43%|████▎     | 8064/18627 [4:26:31<5:39:24,  1.93s/it]
 43%|████▎     | 8065/18627 [4:26:32<5:05:08,  1.73s/it]
 43%|████▎     | 8066/18627 [4:26:34<5:32:37,  1.89s/it]
 43%|████▎     | 8067/18627 [4:26:36<5:50:41,  1.99s/it]
 43%|████▎     | 8068/18627 [4:26:39<6:04:10,  2.07s/it]
 43%|████▎     | 8069/18627 [4:26:41<6:13:33,  2.12s/it]
 43%|████▎     | 8070/18627 [4:26:42<5:26:20,  1.85s/it]
                                                        
{'loss': 1.3118, 'grad_norm': 13.519118309020996, 'learning_rate': 3.1543225560798575e-06, 'epoch': 0.43}
+
 43%|████▎     | 8070/18627 [4:26:42<5:26:20,  1.85s/it]
 43%|████▎     | 8071/18627 [4:26:44<5:46:16,  1.97s/it]
 43%|████▎     | 8072/18627 [4:26:46<5:08:59,  1.76s/it]
 43%|████▎     | 8073/18627 [4:26:48<5:33:08,  1.89s/it]
 43%|████▎     | 8074/18627 [4:26:49<4:58:02,  1.69s/it]
 43%|████▎     | 8075/18627 [4:26:51<5:26:21,  1.86s/it]
 43%|████▎     | 8076/18627 [4:26:53<5:45:46,  1.97s/it]
 43%|████▎     | 8077/18627 [4:26:55<5:07:15,  1.75s/it]
 43%|████▎     | 8078/18627 [4:26:57<5:34:33,  1.90s/it]
 43%|████▎     | 8079/18627 [4:26:59<5:53:06,  2.01s/it]
 43%|████▎     | 8080/18627 [4:27:02<6:07:06,  2.09s/it]
                                                        
{'loss': 1.2894, 'grad_norm': 5.419600963592529, 'learning_rate': 3.150126194496938e-06, 'epoch': 0.43}
+
 43%|████▎     | 8080/18627 [4:27:02<6:07:06,  2.09s/it]
 43%|████▎     | 8081/18627 [4:27:03<5:24:15,  1.84s/it]
 43%|████▎     | 8082/18627 [4:27:05<5:45:59,  1.97s/it]
 43%|████▎     | 8083/18627 [4:27:07<6:00:41,  2.05s/it]
 43%|████▎     | 8084/18627 [4:27:10<6:11:02,  2.11s/it]
 43%|████▎     | 8085/18627 [4:27:12<6:20:29,  2.17s/it]
 43%|���███▎     | 8086/18627 [4:27:14<6:24:33,  2.19s/it]
 43%|████▎     | 8087/18627 [4:27:16<6:27:59,  2.21s/it]
 43%|████▎     | 8088/18627 [4:27:19<6:29:46,  2.22s/it]
 43%|████▎     | 8089/18627 [4:27:21<6:33:09,  2.24s/it]
 43%|████▎     | 8090/18627 [4:27:23<6:34:42,  2.25s/it]
                                                        
{'loss': 0.7806, 'grad_norm': 7.544774055480957, 'learning_rate': 3.145927867395514e-06, 'epoch': 0.43}
+
 43%|████▎     | 8090/18627 [4:27:23<6:34:42,  2.25s/it]
 43%|████▎     | 8091/18627 [4:27:25<6:37:32,  2.26s/it]
 43%|████▎     | 8092/18627 [4:27:28<6:37:09,  2.26s/it]
 43%|████▎     | 8093/18627 [4:27:29<5:45:35,  1.97s/it]
 43%|████▎     | 8094/18627 [4:27:31<5:58:39,  2.04s/it]
 43%|████▎     | 8095/18627 [4:27:33<6:08:34,  2.10s/it]
 43%|████▎     | 8096/18627 [4:27:36<6:17:17,  2.15s/it]
 43%|████▎     | 8097/18627 [4:27:38<6:20:17,  2.17s/it]
 43%|████▎     | 8098/18627 [4:27:40<6:24:07,  2.19s/it]
 43%|████▎     | 8099/18627 [4:27:41<5:34:21,  1.91s/it]
 43%|████▎     | 8100/18627 [4:27:44<5:52:40,  2.01s/it]
                                                        
{'loss': 1.0386, 'grad_norm': 15.159143447875977, 'learning_rate': 3.1417275874683357e-06, 'epoch': 0.43}
+
 43%|████▎     | 8100/18627 [4:27:44<5:52:40,  2.01s/it]
 43%|████▎     | 8101/18627 [4:27:46<6:05:58,  2.09s/it]
 43%|████▎     | 8102/18627 [4:27:48<6:14:33,  2.14s/it]
 44%|████▎     | 8103/18627 [4:27:49<5:28:15,  1.87s/it]
 44%|████▎     | 8104/18627 [4:27:52<5:46:27,  1.98s/it]
 44%|████▎     | 8105/18627 [4:27:53<5:07:21,  1.75s/it]
 44%|████▎     | 8106/18627 [4:27:54<4:40:27,  1.60s/it]
 44%|████▎     | 8107/18627 [4:27:56<5:14:47,  1.80s/it]
 44%|████▎     | 8108/18627 [4:27:59<5:40:12,  1.94s/it]
 44%|████▎     | 8109/18627 [4:28:01<5:56:51,  2.04s/it]
 44%|████▎     | 8110/18627 [4:28:03<6:07:36,  2.10s/it]
                                                        
{'loss': 1.3027, 'grad_norm': 5.28191614151001, 'learning_rate': 3.1375253674140583e-06, 'epoch': 0.44}
+
 44%|████▎     | 8110/18627 [4:28:03<6:07:36,  2.10s/it]
 44%|████▎     | 8111/18627 [4:28:05<6:16:43,  2.15s/it]
 44%|████▎     | 8112/18627 [4:28:07<5:28:51,  1.88s/it]
 44%|████▎     | 8113/18627 [4:28:08<4:51:42,  1.66s/it]
 44%|████▎     | 8114/18627 [4:28:10<5:23:44,  1.85s/it]
 44%|████▎     | 8115/18627 [4:28:12<5:45:03,  1.97s/it]
 44%|████▎     | 8116/18627 [4:28:15<5:57:59,  2.04s/it]
 44%|████▎     | 8117/18627 [4:28:17<6:08:42,  2.10s/it]
 44%|████▎     | 8118/18627 [4:28:19<6:15:22,  2.14s/it]
 44%|████▎     | 8119/18627 [4:28:21<6:20:21,  2.17s/it]
 44%|████▎     | 8120/18627 [4:28:23<6:21:47,  2.18s/it]
                                                        
{'loss': 1.0102, 'grad_norm': 6.061928749084473, 'learning_rate': 3.1333212199372006e-06, 'epoch': 0.44}
+
 44%|████▎     | 8120/18627 [4:28:23<6:21:47,  2.18s/it]
 44%|████▎     | 8121/18627 [4:28:26<6:24:30,  2.20s/it]
 44%|████▎     | 8122/18627 [4:28:28<6:27:52,  2.22s/it]
 44%|████▎     | 8123/18627 [4:28:29<5:37:56,  1.93s/it]
 44%|████▎     | 8124/18627 [4:28:31<5:54:35,  2.03s/it]
 44%|████▎     | 8125/18627 [4:28:34<6:05:46,  2.09s/it]
 44%|████▎     | 8126/18627 [4:28:35<5:24:07,  1.85s/it]
 44%|████▎     | 8127/18627 [4:28:37<5:46:19,  1.98s/it]
 44%|████▎     | 8128/18627 [4:28:40<6:00:33,  2.06s/it]
 44%|████▎     | 8129/18627 [4:28:41<5:21:16,  1.84s/it]
 44%|████▎     | 8130/18627 [4:28:43<5:45:19,  1.97s/it]
                                                        
{'loss': 1.3013, 'grad_norm': 9.4578857421875, 'learning_rate': 3.1291151577481106e-06, 'epoch': 0.44}
+
 44%|████▎     | 8130/18627 [4:28:43<5:45:19,  1.97s/it]
 44%|████▎     | 8131/18627 [4:28:45<5:59:39,  2.06s/it]
 44%|████▎     | 8132/18627 [4:28:46<5:00:48,  1.72s/it]
 44%|████▎     | 8133/18627 [4:28:48<4:35:06,  1.57s/it]
 44%|████▎     | 8134/18627 [4:28:50<5:10:47,  1.78s/it]
 44%|████▎     | 8135/18627 [4:28:52<5:39:40,  1.94s/it]
 44%|████▎     | 8136/18627 [4:28:54<5:58:02,  2.05s/it]
 44%|████▎     | 8137/18627 [4:28:57<6:10:12,  2.12s/it]
 44%|████▎     | 8138/18627 [4:28:58<5:23:43,  1.85s/it]
 44%|████▎     | 8139/18627 [4:29:00<5:44:03,  1.97s/it]
 44%|████▎     | 8140/18627 [4:29:02<5:58:03,  2.05s/it]
                                                        
{'loss': 1.1411, 'grad_norm': 6.47383451461792, 'learning_rate': 3.124907193562925e-06, 'epoch': 0.44}
+
 44%|████▎     | 8140/18627 [4:29:02<5:58:03,  2.05s/it]
 44%|████▎     | 8141/18627 [4:29:04<5:17:22,  1.82s/it]
 44%|████▎     | 8142/18627 [4:29:05<4:43:09,  1.62s/it]
 44%|████▎     | 8143/18627 [4:29:07<5:16:46,  1.81s/it]
 44%|████▎     | 8144/18627 [4:29:09<5:38:03,  1.93s/it]
 44%|████▎     | 8145/18627 [4:29:12<5:53:38,  2.02s/it]
 44%|████▎     | 8146/18627 [4:29:13<5:12:56,  1.79s/it]
 44%|████▎     | 8147/18627 [4:29:15<5:35:21,  1.92s/it]
 44%|████▎     | 8148/18627 [4:29:17<5:53:11,  2.02s/it]
 44%|████▎     | 8149/18627 [4:29:20<6:06:29,  2.10s/it]
 44%|████▍     | 8150/18627 [4:29:22<6:13:46,  2.14s/it]
                                                        
{'loss': 1.1994, 'grad_norm': 8.019689559936523, 'learning_rate': 3.120697340103528e-06, 'epoch': 0.44}
+
 44%|████▍     | 8150/18627 [4:29:22<6:13:46,  2.14s/it]
 44%|████▍     | 8151/18627 [4:29:24<6:19:32,  2.17s/it]
 44%|████▍     | 8152/18627 [4:29:26<6:25:07,  2.21s/it]
 44%|████▍     | 8153/18627 [4:29:29<6:28:41,  2.23s/it]
 44%|████▍     | 8154/18627 [4:29:31<6:29:06,  2.23s/it]
 44%|████▍     | 8155/18627 [4:29:32<5:37:07,  1.93s/it]
 44%|████▍     | 8156/18627 [4:29:33<5:00:11,  1.72s/it]
 44%|████▍     | 8157/18627 [4:29:36<5:28:50,  1.88s/it]
 44%|████▍     | 8158/18627 [4:29:38<5:46:25,  1.99s/it]
 44%|████▍     | 8159/18627 [4:29:40<6:00:47,  2.07s/it]
 44%|████▍     | 8160/18627 [4:29:42<6:08:28,  2.11s/it]
                                                        
{'loss': 1.0763, 'grad_norm': 7.016213893890381, 'learning_rate': 3.1164856100975196e-06, 'epoch': 0.44}
+
 44%|████▍     | 8160/18627 [4:29:42<6:08:28,  2.11s/it]
 44%|████▍     | 8161/18627 [4:29:43<5:07:18,  1.76s/it]
 44%|████▍     | 8162/18627 [4:29:46<5:37:12,  1.93s/it]
 44%|████▍     | 8163/18627 [4:29:47<5:03:22,  1.74s/it]
 44%|████▍     | 8164/18627 [4:29:49<5:29:36,  1.89s/it]
 44%|████▍     | 8165/18627 [4:29:51<5:49:37,  2.01s/it]
 44%|████▍     | 8166/18627 [4:29:54<6:02:58,  2.08s/it]
 44%|████▍     | 8167/18627 [4:29:56<6:11:21,  2.13s/it]
 44%|████▍     | 8168/18627 [4:29:58<6:18:47,  2.17s/it]
 44%|████▍     | 8169/18627 [4:30:00<6:24:32,  2.21s/it]
 44%|████▍     | 8170/18627 [4:30:03<6:27:12,  2.22s/it]
                                                        
{'loss': 0.9407, 'grad_norm': 7.428996562957764, 'learning_rate': 3.1122720162781694e-06, 'epoch': 0.44}
+
 44%|████▍     | 8170/18627 [4:30:03<6:27:12,  2.22s/it]
 44%|████▍     | 8171/18627 [4:30:05<6:31:28,  2.25s/it]
 44%|████▍     | 8172/18627 [4:30:07<6:33:18,  2.26s/it]
 44%|████▍     | 8173/18627 [4:30:10<6:33:57,  2.26s/it]
 44%|████▍     | 8174/18627 [4:30:12<6:33:49,  2.26s/it]
 44%|████▍     | 8175/18627 [4:30:14<6:36:30,  2.28s/it]
 44%|████▍     | 8176/18627 [4:30:15<5:42:53,  1.97s/it]
 44%|████▍     | 8177/18627 [4:30:16<4:50:23,  1.67s/it]
 44%|████▍     | 8178/18627 [4:30:19<5:24:47,  1.87s/it]
 44%|████▍     | 8179/18627 [4:30:21<5:44:25,  1.98s/it]
 44%|████▍     | 8180/18627 [4:30:22<5:08:03,  1.77s/it]
                                                        
{'loss': 0.9885, 'grad_norm': 14.086064338684082, 'learning_rate': 3.1080565713843857e-06, 'epoch': 0.44}
+
 44%|████▍     | 8180/18627 [4:30:22<5:08:03,  1.77s/it]
 44%|████▍     | 8181/18627 [4:30:24<5:32:40,  1.91s/it]
 44%|████▍     | 8182/18627 [4:30:26<4:59:23,  1.72s/it]
 44%|████▍     | 8183/18627 [4:30:28<5:26:57,  1.88s/it]
 44%|████▍     | 8184/18627 [4:30:30<5:47:14,  2.00s/it]
 44%|████▍     | 8185/18627 [4:30:32<6:01:14,  2.08s/it]
 44%|████▍     | 8186/18627 [4:30:35<6:11:41,  2.14s/it]
 44%|████▍     | 8187/18627 [4:30:37<6:18:24,  2.17s/it]
 44%|████▍     | 8188/18627 [4:30:39<6:23:34,  2.20s/it]
 44%|████▍     | 8189/18627 [4:30:42<6:24:56,  2.21s/it]
 44%|████▍     | 8190/18627 [4:30:43<5:35:41,  1.93s/it]
                                                        
{'loss': 1.0636, 'grad_norm': 13.369497299194336, 'learning_rate': 3.1038392881606695e-06, 'epoch': 0.44}
+
 44%|████▍     | 8190/18627 [4:30:43<5:35:41,  1.93s/it]
 44%|████▍     | 8191/18627 [4:30:45<5:52:05,  2.02s/it]
 44%|████▍     | 8192/18627 [4:30:47<6:04:58,  2.10s/it]
 44%|████▍     | 8193/18627 [4:30:50<6:10:52,  2.13s/it]
 44%|████▍     | 8194/18627 [4:30:52<6:17:37,  2.17s/it]
 44%|████▍     | 8195/18627 [4:30:54<6:21:49,  2.20s/it]
 44%|████▍     | 8196/18627 [4:30:56<6:23:34,  2.21s/it]
 44%|████▍     | 8197/18627 [4:30:59<6:27:27,  2.23s/it]
 44%|████▍     | 8198/18627 [4:31:00<5:31:57,  1.91s/it]
 44%|████▍     | 8199/18627 [4:31:01<4:42:15,  1.62s/it]
 44%|████▍     | 8200/18627 [4:31:03<5:15:22,  1.81s/it]
                                                        
{'loss': 0.8163, 'grad_norm': 7.1421895027160645, 'learning_rate': 3.0996201793570807e-06, 'epoch': 0.44}
+
 44%|████▍     | 8200/18627 [4:31:03<5:15:22,  1.81s/it]
 44%|████▍     | 8201/18627 [4:31:04<4:48:07,  1.66s/it]
 44%|████▍     | 8202/18627 [4:31:06<5:18:59,  1.84s/it]
 44%|████▍     | 8203/18627 [4:31:09<5:41:42,  1.97s/it]
 44%|████▍     | 8204/18627 [4:31:11<5:55:40,  2.05s/it]
 44%|████▍     | 8205/18627 [4:31:12<5:16:23,  1.82s/it]
 44%|████▍     | 8206/18627 [4:31:15<5:39:58,  1.96s/it]
 44%|████▍     | 8207/18627 [4:31:17<5:56:58,  2.06s/it]
 44%|████▍     | 8208/18627 [4:31:19<6:09:22,  2.13s/it]
 44%|████▍     | 8209/18627 [4:31:21<6:16:06,  2.17s/it]
 44%|████▍     | 8210/18627 [4:31:24<6:21:03,  2.19s/it]
                                                        
{'loss': 1.0462, 'grad_norm': 6.2819695472717285, 'learning_rate': 3.0953992577292003e-06, 'epoch': 0.44}
+
 44%|████▍     | 8210/18627 [4:31:24<6:21:03,  2.19s/it]
 44%|████▍     | 8211/18627 [4:31:25<5:16:13,  1.82s/it]
 44%|████▍     | 8212/18627 [4:31:26<4:49:55,  1.67s/it]
 44%|████▍     | 8213/18627 [4:31:28<5:21:35,  1.85s/it]
 44%|████▍     | 8214/18627 [4:31:30<5:42:23,  1.97s/it]
 44%|████▍     | 8215/18627 [4:31:33<5:58:14,  2.06s/it]
 44%|████▍     | 8216/18627 [4:31:35<6:09:29,  2.13s/it]
 44%|████▍     | 8217/18627 [4:31:37<6:18:22,  2.18s/it]
 44%|████▍     | 8218/18627 [4:31:38<5:25:46,  1.88s/it]
 44%|████▍     | 8219/18627 [4:31:41<5:46:24,  2.00s/it]
 44%|████▍     | 8220/18627 [4:31:43<5:59:32,  2.07s/it]
                                                        
{'loss': 1.0379, 'grad_norm': 4.765186786651611, 'learning_rate': 3.0911765360380887e-06, 'epoch': 0.44}
+
 44%|████▍     | 8220/18627 [4:31:43<5:59:32,  2.07s/it]
 44%|████▍     | 8221/18627 [4:31:45<6:09:32,  2.13s/it]
 44%|████▍     | 8222/18627 [4:31:48<6:17:14,  2.18s/it]
 44%|████▍     | 8223/18627 [4:31:50<6:22:21,  2.21s/it]
 44%|████▍     | 8224/18627 [4:31:52<6:24:49,  2.22s/it]
 44%|████▍     | 8225/18627 [4:31:53<5:35:15,  1.93s/it]
 44%|████▍     | 8226/18627 [4:31:56<5:51:45,  2.03s/it]
 44%|████▍     | 8227/18627 [4:31:58<6:02:07,  2.09s/it]
 44%|████▍     | 8228/18627 [4:32:00<6:10:23,  2.14s/it]
 44%|████▍     | 8229/18627 [4:32:01<5:27:35,  1.89s/it]
 44%|████▍     | 8230/18627 [4:32:04<5:47:44,  2.01s/it]
                                                        
{'loss': 1.0819, 'grad_norm': 9.777909278869629, 'learning_rate': 3.0869520270502473e-06, 'epoch': 0.44}
+
 44%|████▍     | 8230/18627 [4:32:04<5:47:44,  2.01s/it]
 44%|████▍     | 8231/18627 [4:32:06<6:01:47,  2.09s/it]
 44%|████▍     | 8232/18627 [4:32:08<6:12:35,  2.15s/it]
 44%|████▍     | 8233/18627 [4:32:11<6:20:04,  2.19s/it]
 44%|████▍     | 8234/18627 [4:32:13<6:24:18,  2.22s/it]
 44%|████▍     | 8235/18627 [4:32:14<5:35:30,  1.94s/it]
 44%|████▍     | 8236/18627 [4:32:16<5:52:54,  2.04s/it]
 44%|████▍     | 8237/18627 [4:32:19<6:04:28,  2.10s/it]
 44%|████▍     | 8238/18627 [4:32:21<6:11:48,  2.15s/it]
 44%|████▍     | 8239/18627 [4:32:23<6:19:20,  2.19s/it]
 44%|████▍     | 8240/18627 [4:32:25<6:23:03,  2.21s/it]
                                                        
{'loss': 0.8826, 'grad_norm': 4.489352226257324, 'learning_rate': 3.0827257435375818e-06, 'epoch': 0.44}
+
 44%|████▍     | 8240/18627 [4:32:25<6:23:03,  2.21s/it]
 44%|████▍     | 8241/18627 [4:32:28<6:26:16,  2.23s/it]
 44%|████▍     | 8242/18627 [4:32:29<5:36:04,  1.94s/it]
 44%|████▍     | 8243/18627 [4:32:30<4:59:35,  1.73s/it]
 44%|████▍     | 8244/18627 [4:32:31<4:35:52,  1.59s/it]
 44%|████▍     | 8245/18627 [4:32:33<4:19:11,  1.50s/it]
 44%|████▍     | 8246/18627 [4:32:35<4:59:55,  1.73s/it]
 44%|████▍     | 8247/18627 [4:32:37<5:29:44,  1.91s/it]
 44%|████▍     | 8248/18627 [4:32:40<5:48:44,  2.02s/it]
 44%|████▍     | 8249/18627 [4:32:42<6:03:23,  2.10s/it]
 44%|████▍     | 8250/18627 [4:32:44<6:10:02,  2.14s/it]
                                                        
{'loss': 1.4524, 'grad_norm': 9.617182731628418, 'learning_rate': 3.078497698277364e-06, 'epoch': 0.44}
+
 44%|████▍     | 8250/18627 [4:32:44<6:10:02,  2.14s/it]
 44%|████▍     | 8251/18627 [4:32:45<5:24:56,  1.88s/it]
 44%|████▍     | 8252/18627 [4:32:48<5:45:12,  2.00s/it]
 44%|████▍     | 8253/18627 [4:32:49<5:08:37,  1.78s/it]
 44%|████▍     | 8254/18627 [4:32:53<6:42:22,  2.33s/it]
 44%|████▍     | 8255/18627 [4:32:55<6:40:45,  2.32s/it]
 44%|████▍     | 8256/18627 [4:32:56<5:28:50,  1.90s/it]
 44%|████▍     | 8257/18627 [4:32:57<4:57:16,  1.72s/it]
 44%|████▍     | 8258/18627 [4:32:59<5:26:00,  1.89s/it]
 44%|████▍     | 8259/18627 [4:33:02<5:45:08,  2.00s/it]
 44%|████▍     | 8260/18627 [4:33:04<5:59:53,  2.08s/it]
                                                        
{'loss': 1.2826, 'grad_norm': 7.790722370147705, 'learning_rate': 3.0742679040521908e-06, 'epoch': 0.44}
+
 44%|████▍     | 8260/18627 [4:33:04<5:59:53,  2.08s/it]
 44%|████▍     | 8261/18627 [4:33:06<6:10:53,  2.15s/it]
 44%|████▍     | 8262/18627 [4:33:09<6:19:08,  2.19s/it]
 44%|████▍     | 8263/18627 [4:33:11<6:22:56,  2.22s/it]
 44%|████▍     | 8264/18627 [4:33:13<6:24:23,  2.23s/it]
 44%|████▍     | 8265/18627 [4:33:14<5:34:02,  1.93s/it]
 44%|████▍     | 8266/18627 [4:33:17<5:50:34,  2.03s/it]
 44%|████▍     | 8267/18627 [4:33:19<6:03:29,  2.11s/it]
 44%|████▍     | 8268/18627 [4:33:20<5:20:53,  1.86s/it]
 44%|████▍     | 8269/18627 [4:33:22<5:41:55,  1.98s/it]
 44%|████▍     | 8270/18627 [4:33:24<5:06:21,  1.77s/it]
                                                        
{'loss': 1.2471, 'grad_norm': 13.48586654663086, 'learning_rate': 3.0700363736499465e-06, 'epoch': 0.44}
+
 44%|████▍     | 8270/18627 [4:33:24<5:06:21,  1.77s/it]
 44%|████▍     | 8271/18627 [4:33:26<5:31:54,  1.92s/it]
 44%|████▍     | 8272/18627 [4:33:28<5:49:16,  2.02s/it]
 44%|████▍     | 8273/18627 [4:33:30<5:13:44,  1.82s/it]
 44%|████▍     | 8274/18627 [4:33:32<5:37:15,  1.95s/it]
 44%|████▍     | 8275/18627 [4:33:34<5:53:07,  2.05s/it]
 44%|████▍     | 8276/18627 [4:33:36<6:03:33,  2.11s/it]
 44%|████▍     | 8277/18627 [4:33:39<6:13:24,  2.16s/it]
 44%|████▍     | 8278/18627 [4:33:41<6:17:36,  2.19s/it]
 44%|████▍     | 8279/18627 [4:33:43<6:21:33,  2.21s/it]
 44%|████▍     | 8280/18627 [4:33:44<5:32:10,  1.93s/it]
                                                        
{'loss': 1.1114, 'grad_norm': 15.685275077819824, 'learning_rate': 3.065803119863765e-06, 'epoch': 0.44}
+
 44%|████▍     | 8280/18627 [4:33:44<5:32:10,  1.93s/it]
 44%|████▍     | 8281/18627 [4:33:47<5:50:21,  2.03s/it]
 44%|████▍     | 8282/18627 [4:33:49<6:02:16,  2.10s/it]
 44%|████▍     | 8283/18627 [4:33:51<6:11:58,  2.16s/it]
 44%|████▍     | 8284/18627 [4:33:53<6:18:06,  2.19s/it]
 44%|████▍     | 8285/18627 [4:33:56<6:22:55,  2.22s/it]
 44%|████▍     | 8286/18627 [4:33:58<6:24:28,  2.23s/it]
 44%|████▍     | 8287/18627 [4:34:00<6:28:08,  2.25s/it]
 44%|████▍     | 8288/18627 [4:34:03<6:31:19,  2.27s/it]
 44%|████▍     | 8289/18627 [4:34:05<6:29:50,  2.26s/it]
 45%|████▍     | 8290/18627 [4:34:07<6:28:12,  2.25s/it]
                                                        
{'loss': 0.6111, 'grad_norm': 4.37431526184082, 'learning_rate': 3.06156815549199e-06, 'epoch': 0.45}
+
 45%|████▍     | 8290/18627 [4:34:07<6:28:12,  2.25s/it]
 45%|████▍     | 8291/18627 [4:34:09<6:29:18,  2.26s/it]
 45%|████▍     | 8292/18627 [4:34:12<6:29:21,  2.26s/it]
 45%|████▍     | 8293/18627 [4:34:13<5:39:15,  1.97s/it]
 45%|████▍     | 8294/18627 [4:34:15<5:55:01,  2.06s/it]
 45%|████▍     | 8295/18627 [4:34:17<6:03:50,  2.11s/it]
 45%|████▍     | 8296/18627 [4:34:20<6:12:15,  2.16s/it]
 45%|████▍     | 8297/18627 [4:34:22<6:15:29,  2.18s/it]
 45%|████▍     | 8298/18627 [4:34:24<6:19:20,  2.20s/it]
 45%|████▍     | 8299/18627 [4:34:26<6:21:29,  2.22s/it]
 45%|████▍     | 8300/18627 [4:34:29<6:23:13,  2.23s/it]
                                                        
{'loss': 0.8288, 'grad_norm': 4.879363536834717, 'learning_rate': 3.057331493338138e-06, 'epoch': 0.45}
+
 45%|████▍     | 8300/18627 [4:34:29<6:23:13,  2.23s/it]
 45%|████▍     | 8301/18627 [4:34:31<6:25:37,  2.24s/it]
 45%|████▍     | 8302/18627 [4:34:33<6:25:41,  2.24s/it]
 45%|████▍     | 8303/18627 [4:34:35<5:38:05,  1.96s/it]
 45%|████▍     | 8304/18627 [4:34:36<5:05:51,  1.78s/it]
 45%|████▍     | 8305/18627 [4:34:38<5:29:49,  1.92s/it]
 45%|████▍     | 8306/18627 [4:34:40<5:48:15,  2.02s/it]
 45%|████▍     | 8307/18627 [4:34:43<6:03:19,  2.11s/it]
 45%|████▍     | 8308/18627 [4:34:45<6:12:00,  2.16s/it]
 45%|████▍     | 8309/18627 [4:34:47<6:17:08,  2.19s/it]
 45%|████▍     | 8310/18627 [4:34:50<6:21:41,  2.22s/it]
                                                        
{'loss': 1.117, 'grad_norm': 6.886136054992676, 'learning_rate': 3.053093146210856e-06, 'epoch': 0.45}
+
 45%|████▍     | 8310/18627 [4:34:50<6:21:41,  2.22s/it]
 45%|████▍     | 8311/18627 [4:34:52<6:24:54,  2.24s/it]
 45%|████▍     | 8312/18627 [4:34:54<6:25:50,  2.24s/it]
 45%|████▍     | 8313/18627 [4:34:56<6:27:00,  2.25s/it]
 45%|████▍     | 8314/18627 [4:34:59<6:28:10,  2.26s/it]
 45%|████▍     | 8315/18627 [4:35:01<6:27:15,  2.25s/it]
 45%|████▍     | 8316/18627 [4:35:03<6:30:23,  2.27s/it]
 45%|████▍     | 8317/18627 [4:35:06<6:34:53,  2.30s/it]
 45%|████▍     | 8318/18627 [4:35:07<5:45:49,  2.01s/it]
 45%|████▍     | 8319/18627 [4:35:09<6:01:24,  2.10s/it]
 45%|████▍     | 8320/18627 [4:35:11<5:20:14,  1.86s/it]
                                                        
{'loss': 0.9849, 'grad_norm': 13.912681579589844, 'learning_rate': 3.0488531269238874e-06, 'epoch': 0.45}
+
 45%|████▍     | 8320/18627 [4:35:11<5:20:14,  1.86s/it]
 45%|████▍     | 8321/18627 [4:35:13<5:43:32,  2.00s/it]
 45%|████▍     | 8322/18627 [4:35:15<5:57:47,  2.08s/it]
 45%|████▍     | 8323/18627 [4:35:17<6:07:17,  2.14s/it]
 45%|████▍     | 8324/18627 [4:35:19<5:24:42,  1.89s/it]
 45%|████▍     | 8325/18627 [4:35:21<5:44:52,  2.01s/it]
 45%|████▍     | 8326/18627 [4:35:22<5:08:16,  1.80s/it]
 45%|████▍     | 8327/18627 [4:35:25<5:31:49,  1.93s/it]
 45%|████▍     | 8328/18627 [4:35:27<5:49:46,  2.04s/it]
 45%|████▍     | 8329/18627 [4:35:29<6:02:38,  2.11s/it]
 45%|████▍     | 8330/18627 [4:35:31<6:11:45,  2.17s/it]
                                                        
{'loss': 1.0391, 'grad_norm': 5.316504955291748, 'learning_rate': 3.04461144829603e-06, 'epoch': 0.45}
+
 45%|████▍     | 8330/18627 [4:35:31<6:11:45,  2.17s/it]
 45%|████▍     | 8331/18627 [4:35:33<5:27:06,  1.91s/it]
 45%|████▍     | 8332/18627 [4:35:35<5:46:32,  2.02s/it]
 45%|████▍     | 8333/18627 [4:35:36<5:09:29,  1.80s/it]
 45%|████▍     | 8334/18627 [4:35:39<5:34:21,  1.95s/it]
 45%|████▍     | 8335/18627 [4:35:40<5:03:06,  1.77s/it]
 45%|████▍     | 8336/18627 [4:35:42<5:28:19,  1.91s/it]
 45%|████▍     | 8337/18627 [4:35:43<4:58:17,  1.74s/it]
 45%|████▍     | 8338/18627 [4:35:46<5:26:52,  1.91s/it]
 45%|████▍     | 8339/18627 [4:35:47<4:56:56,  1.73s/it]
 45%|████▍     | 8340/18627 [4:35:49<5:25:37,  1.90s/it]
                                                        
{'loss': 1.62, 'grad_norm': 5.916789531707764, 'learning_rate': 3.040368123151098e-06, 'epoch': 0.45}
+
 45%|████▍     | 8340/18627 [4:35:49<5:25:37,  1.90s/it]
 45%|████▍     | 8341/18627 [4:35:52<5:44:47,  2.01s/it]
 45%|████▍     | 8342/18627 [4:35:54<5:58:29,  2.09s/it]
 45%|████▍     | 8343/18627 [4:35:56<6:08:19,  2.15s/it]
 45%|████▍     | 8344/18627 [4:35:58<5:24:05,  1.89s/it]
 45%|████▍     | 8345/18627 [4:36:00<5:42:20,  2.00s/it]
 45%|████▍     | 8346/18627 [4:36:02<5:56:32,  2.08s/it]
 45%|████▍     | 8347/18627 [4:36:04<6:06:02,  2.14s/it]
 45%|████▍     | 8348/18627 [4:36:07<6:11:29,  2.17s/it]
 45%|████▍     | 8349/18627 [4:36:09<6:15:31,  2.19s/it]
 45%|████▍     | 8350/18627 [4:36:11<6:18:58,  2.21s/it]
                                                        
{'loss': 0.8633, 'grad_norm': 5.085758209228516, 'learning_rate': 3.036123164317886e-06, 'epoch': 0.45}
+
 45%|████▍     | 8350/18627 [4:36:11<6:18:58,  2.21s/it]
 45%|████▍     | 8351/18627 [4:36:13<6:20:55,  2.22s/it]
 45%|████▍     | 8352/18627 [4:36:16<6:25:24,  2.25s/it]
 45%|████▍     | 8353/18627 [4:36:18<6:27:52,  2.27s/it]
 45%|████▍     | 8354/18627 [4:36:19<5:36:32,  1.97s/it]
 45%|████▍     | 8355/18627 [4:36:21<5:51:45,  2.05s/it]
 45%|████▍     | 8356/18627 [4:36:24<6:02:54,  2.12s/it]
 45%|████▍     | 8357/18627 [4:36:26<6:12:16,  2.17s/it]
 45%|████▍     | 8358/18627 [4:36:28<6:17:30,  2.21s/it]
 45%|████▍     | 8359/18627 [4:36:31<6:21:56,  2.23s/it]
 45%|████▍     | 8360/18627 [4:36:33<6:23:31,  2.24s/it]
                                                        
{'loss': 0.764, 'grad_norm': 5.440527439117432, 'learning_rate': 3.0318765846301234e-06, 'epoch': 0.45}
+
 45%|████▍     | 8360/18627 [4:36:33<6:23:31,  2.24s/it]
 45%|████▍     | 8361/18627 [4:36:35<6:26:06,  2.26s/it]
 45%|████▍     | 8362/18627 [4:36:37<6:26:59,  2.26s/it]
 45%|████▍     | 8363/18627 [4:36:40<6:27:50,  2.27s/it]
 45%|████▍     | 8364/18627 [4:36:42<6:28:12,  2.27s/it]
 45%|████▍     | 8365/18627 [4:36:44<6:27:16,  2.26s/it]
 45%|████▍     | 8366/18627 [4:36:46<6:25:21,  2.25s/it]
 45%|████▍     | 8367/18627 [4:36:49<6:26:39,  2.26s/it]
 45%|████▍     | 8368/18627 [4:36:51<6:26:08,  2.26s/it]
 45%|████▍     | 8369/18627 [4:36:53<6:26:21,  2.26s/it]
 45%|████▍     | 8370/18627 [4:36:55<5:36:12,  1.97s/it]
                                                        
{'loss': 0.9262, 'grad_norm': 16.20299530029297, 'learning_rate': 3.0276283969264424e-06, 'epoch': 0.45}
+
 45%|████▍     | 8370/18627 [4:36:55<5:36:12,  1.97s/it]
 45%|████▍     | 8371/18627 [4:36:57<5:51:54,  2.06s/it]
 45%|████▍     | 8372/18627 [4:36:59<6:04:37,  2.13s/it]
 45%|████▍     | 8373/18627 [4:37:01<6:12:13,  2.18s/it]
 45%|████▍     | 8374/18627 [4:37:04<6:16:12,  2.20s/it]
 45%|████▍     | 8375/18627 [4:37:06<6:18:49,  2.22s/it]
 45%|████▍     | 8376/18627 [4:37:08<6:22:06,  2.24s/it]
 45%|████▍     | 8377/18627 [4:37:11<6:25:54,  2.26s/it]
 45%|████▍     | 8378/18627 [4:37:13<6:24:58,  2.25s/it]
 45%|████▍     | 8379/18627 [4:37:15<6:25:26,  2.26s/it]
 45%|████▍     | 8380/18627 [4:37:17<6:27:07,  2.27s/it]
                                                        
{'loss': 0.6383, 'grad_norm': 8.615994453430176, 'learning_rate': 3.0233786140503386e-06, 'epoch': 0.45}
+
 45%|████▍     | 8380/18627 [4:37:17<6:27:07,  2.27s/it]
 45%|████▍     | 8381/18627 [4:37:20<6:25:07,  2.26s/it]
 45%|████▍     | 8382/18627 [4:37:22<6:25:23,  2.26s/it]
 45%|████▌     | 8383/18627 [4:37:24<6:24:36,  2.25s/it]
 45%|████▌     | 8384/18627 [4:37:26<6:27:48,  2.27s/it]
 45%|████▌     | 8385/18627 [4:37:29<6:26:06,  2.26s/it]
 45%|████▌     | 8386/18627 [4:37:31<6:26:59,  2.27s/it]
 45%|████▌     | 8387/18627 [4:37:33<6:26:00,  2.26s/it]
 45%|████▌     | 8388/18627 [4:37:35<6:31:26,  2.29s/it]
 45%|████▌     | 8389/18627 [4:37:37<5:35:26,  1.97s/it]
 45%|████▌     | 8390/18627 [4:37:39<5:52:10,  2.06s/it]
                                                        
{'loss': 0.9084, 'grad_norm': 5.990074634552002, 'learning_rate': 3.0191272488501257e-06, 'epoch': 0.45}
+
 45%|████▌     | 8390/18627 [4:37:39<5:52:10,  2.06s/it]
 45%|████▌     | 8391/18627 [4:37:40<5:09:48,  1.82s/it]
 45%|████▌     | 8392/18627 [4:37:42<5:31:18,  1.94s/it]
 45%|████▌     | 8393/18627 [4:37:45<5:46:34,  2.03s/it]
 45%|████▌     | 8394/18627 [4:37:46<4:53:32,  1.72s/it]
 45%|████▌     | 8395/18627 [4:37:48<5:22:49,  1.89s/it]
 45%|████▌     | 8396/18627 [4:37:50<5:43:15,  2.01s/it]
 45%|████▌     | 8397/18627 [4:37:53<5:56:11,  2.09s/it]
 45%|████▌     | 8398/18627 [4:37:55<6:05:08,  2.14s/it]
 45%|████▌     | 8399/18627 [4:37:57<6:14:30,  2.20s/it]
 45%|████▌     | 8400/18627 [4:37:59<6:19:38,  2.23s/it]
                                                        
{'loss': 0.9203, 'grad_norm': 8.876919746398926, 'learning_rate': 3.0148743141789043e-06, 'epoch': 0.45}
+
 45%|████▌     | 8400/18627 [4:37:59<6:19:38,  2.23s/it]
 45%|████▌     | 8401/18627 [4:38:02<6:21:40,  2.24s/it]
 45%|████▌     | 8402/18627 [4:38:03<5:33:44,  1.96s/it]
 45%|████▌     | 8403/18627 [4:38:05<5:48:22,  2.04s/it]
 45%|████▌     | 8404/18627 [4:38:07<5:57:03,  2.10s/it]
 45%|████▌     | 8405/18627 [4:38:08<4:58:18,  1.75s/it]
 45%|████▌     | 8406/18627 [4:38:11<5:23:07,  1.90s/it]
 45%|████▌     | 8407/18627 [4:38:13<5:41:09,  2.00s/it]
 45%|████▌     | 8408/18627 [4:38:15<5:53:25,  2.08s/it]
 45%|████▌     | 8409/18627 [4:38:16<4:54:14,  1.73s/it]
 45%|████▌     | 8410/18627 [4:38:17<4:32:12,  1.60s/it]
                                                        
{'loss': 1.0704, 'grad_norm': 14.23562240600586, 'learning_rate': 3.01061982289452e-06, 'epoch': 0.45}
+
 45%|████▌     | 8410/18627 [4:38:17<4:32:12,  1.60s/it]
 45%|████▌     | 8411/18627 [4:38:18<4:01:52,  1.42s/it]
 45%|████▌     | 8412/18627 [4:38:21<4:46:06,  1.68s/it]
 45%|████▌     | 8413/18627 [4:38:23<5:17:36,  1.87s/it]
 45%|████▌     | 8414/18627 [4:38:25<5:39:11,  1.99s/it]
 45%|████▌     | 8415/18627 [4:38:28<5:54:44,  2.08s/it]
 45%|████▌     | 8416/18627 [4:38:30<6:03:41,  2.14s/it]
 45%|████▌     | 8417/18627 [4:38:32<6:09:29,  2.17s/it]
 45%|████▌     | 8418/18627 [4:38:33<5:06:34,  1.80s/it]
 45%|████▌     | 8419/18627 [4:38:35<5:30:48,  1.94s/it]
 45%|████▌     | 8420/18627 [4:38:38<5:46:41,  2.04s/it]
                                                        
{'loss': 0.6396, 'grad_norm': 4.706974983215332, 'learning_rate': 3.0063637878595235e-06, 'epoch': 0.45}
+
 45%|████▌     | 8420/18627 [4:38:38<5:46:41,  2.04s/it]
 45%|████▌     | 8421/18627 [4:38:38<4:51:12,  1.71s/it]
 45%|████▌     | 8422/18627 [4:38:40<4:31:33,  1.60s/it]
 45%|████▌     | 8423/18627 [4:38:42<5:07:56,  1.81s/it]
 45%|████▌     | 8424/18627 [4:38:44<5:30:37,  1.94s/it]
 45%|████▌     | 8425/18627 [4:38:47<5:46:40,  2.04s/it]
 45%|████▌     | 8426/18627 [4:38:49<5:59:54,  2.12s/it]
 45%|████▌     | 8427/18627 [4:38:51<6:09:13,  2.17s/it]
 45%|████▌     | 8428/18627 [4:38:53<6:13:55,  2.20s/it]
 45%|████▌     | 8429/18627 [4:38:56<6:17:38,  2.22s/it]
 45%|████▌     | 8430/18627 [4:38:58<6:21:30,  2.24s/it]
                                                        
{'loss': 0.8722, 'grad_norm': 7.573390483856201, 'learning_rate': 3.0021062219411333e-06, 'epoch': 0.45}
+
 45%|████▌     | 8430/18627 [4:38:58<6:21:30,  2.24s/it]
 45%|████▌     | 8431/18627 [4:39:00<6:24:43,  2.26s/it]
 45%|████▌     | 8432/18627 [4:39:03<6:25:59,  2.27s/it]
 45%|████▌     | 8433/18627 [4:39:05<6:27:27,  2.28s/it]
 45%|████▌     | 8434/18627 [4:39:06<5:38:59,  2.00s/it]
 45%|████▌     | 8435/18627 [4:39:07<4:46:45,  1.69s/it]
 45%|████▌     | 8436/18627 [4:39:08<4:20:48,  1.54s/it]
 45%|████▌     | 8437/18627 [4:39:10<4:07:48,  1.46s/it]
 45%|████▌     | 8438/18627 [4:39:12<4:49:38,  1.71s/it]
 45%|████▌     | 8439/18627 [4:39:14<5:20:22,  1.89s/it]
 45%|████▌     | 8440/18627 [4:39:17<5:40:18,  2.00s/it]
                                                        
{'loss': 1.2815, 'grad_norm': 6.700966835021973, 'learning_rate': 2.9978471380111952e-06, 'epoch': 0.45}
+
 45%|████▌     | 8440/18627 [4:39:17<5:40:18,  2.00s/it]
 45%|████▌     | 8441/18627 [4:39:19<5:54:29,  2.09s/it]
 45%|████▌     | 8442/18627 [4:39:20<4:58:37,  1.76s/it]
 45%|████▌     | 8443/18627 [4:39:22<5:24:42,  1.91s/it]
 45%|████▌     | 8444/18627 [4:39:24<5:45:03,  2.03s/it]
 45%|████▌     | 8445/18627 [4:39:26<5:04:57,  1.80s/it]
 45%|████▌     | 8446/18627 [4:39:28<5:28:37,  1.94s/it]
 45%|████▌     | 8447/18627 [4:39:29<4:56:58,  1.75s/it]
 45%|████▌     | 8448/18627 [4:39:32<5:25:32,  1.92s/it]
 45%|████▌     | 8449/18627 [4:39:34<5:41:15,  2.01s/it]
 45%|████▌     | 8450/18627 [4:39:36<5:54:53,  2.09s/it]
                                                        
{'loss': 1.112, 'grad_norm': 6.776036739349365, 'learning_rate': 2.993586548946146e-06, 'epoch': 0.45}
+
 45%|████▌     | 8450/18627 [4:39:36<5:54:53,  2.09s/it]
 45%|████▌     | 8451/18627 [4:39:38<6:04:38,  2.15s/it]
 45%|████▌     | 8452/18627 [4:39:41<6:11:15,  2.19s/it]
 45%|████▌     | 8453/18627 [4:39:43<6:15:50,  2.22s/it]
 45%|████▌     | 8454/18627 [4:39:45<6:19:02,  2.24s/it]
 45%|████▌     | 8455/18627 [4:39:47<5:30:47,  1.95s/it]
 45%|████▌     | 8456/18627 [4:39:49<5:48:07,  2.05s/it]
 45%|████▌     | 8457/18627 [4:39:51<6:00:07,  2.12s/it]
 45%|████▌     | 8458/18627 [4:39:53<6:08:56,  2.18s/it]
 45%|████▌     | 8459/18627 [4:39:56<6:14:45,  2.21s/it]
 45%|████▌     | 8460/18627 [4:39:58<6:19:40,  2.24s/it]
                                                        
{'loss': 0.8624, 'grad_norm': 8.76427173614502, 'learning_rate': 2.989324467626971e-06, 'epoch': 0.45}
+
 45%|████▌     | 8460/18627 [4:39:58<6:19:40,  2.24s/it]
 45%|████▌     | 8461/18627 [4:40:00<6:22:17,  2.26s/it]
 45%|████▌     | 8462/18627 [4:40:03<6:23:15,  2.26s/it]
 45%|████▌     | 8463/18627 [4:40:05<6:24:35,  2.27s/it]
 45%|████▌     | 8464/18627 [4:40:07<6:24:54,  2.27s/it]
 45%|████▌     | 8465/18627 [4:40:09<6:24:11,  2.27s/it]
 45%|████▌     | 8466/18627 [4:40:12<6:46:47,  2.40s/it]
 45%|████▌     | 8467/18627 [4:40:14<6:40:15,  2.36s/it]
 45%|████▌     | 8468/18627 [4:40:17<6:37:15,  2.35s/it]
 45%|████▌     | 8469/18627 [4:40:18<5:44:21,  2.03s/it]
 45%|████▌     | 8470/18627 [4:40:20<5:58:44,  2.12s/it]
                                                        
{'loss': 0.7923, 'grad_norm': 7.917606353759766, 'learning_rate': 2.985060906939169e-06, 'epoch': 0.45}
+
 45%|████▌     | 8470/18627 [4:40:20<5:58:44,  2.12s/it]
 45%|████▌     | 8471/18627 [4:40:23<6:07:00,  2.17s/it]
 45%|████▌     | 8472/18627 [4:40:25<6:13:59,  2.21s/it]
 45%|████▌     | 8473/18627 [4:40:27<6:16:15,  2.22s/it]
 45%|████▌     | 8474/18627 [4:40:29<6:19:35,  2.24s/it]
 45%|████▌     | 8475/18627 [4:40:31<5:27:21,  1.93s/it]
 46%|████▌     | 8476/18627 [4:40:33<5:45:52,  2.04s/it]
 46%|████▌     | 8477/18627 [4:40:35<5:58:26,  2.12s/it]
 46%|████▌     | 8478/18627 [4:40:38<6:07:11,  2.17s/it]
 46%|████▌     | 8479/18627 [4:40:40<6:12:38,  2.20s/it]
 46%|████▌     | 8480/18627 [4:40:42<6:16:30,  2.23s/it]
                                                        
{'loss': 0.8665, 'grad_norm': 5.39255428314209, 'learning_rate': 2.98079587977271e-06, 'epoch': 0.46}
+
 46%|████▌     | 8480/18627 [4:40:42<6:16:30,  2.23s/it]
 46%|████▌     | 8481/18627 [4:40:44<6:19:37,  2.24s/it]
 46%|████▌     | 8482/18627 [4:40:47<6:21:06,  2.25s/it]
 46%|████▌     | 8483/18627 [4:40:49<6:22:31,  2.26s/it]
 46%|████▌     | 8484/18627 [4:40:51<6:22:00,  2.26s/it]
 46%|████▌     | 8485/18627 [4:40:53<6:23:57,  2.27s/it]
 46%|████▌     | 8486/18627 [4:40:55<5:36:46,  1.99s/it]
 46%|████▌     | 8487/18627 [4:40:57<5:50:16,  2.07s/it]
 46%|████▌     | 8488/18627 [4:40:59<5:58:58,  2.12s/it]
 46%|████▌     | 8489/18627 [4:41:01<5:14:59,  1.86s/it]
 46%|████▌     | 8490/18627 [4:41:03<5:35:13,  1.98s/it]
                                                        
{'loss': 1.0379, 'grad_norm': 5.295314788818359, 'learning_rate': 2.9765293990219977e-06, 'epoch': 0.46}
+
 46%|████▌     | 8490/18627 [4:41:03<5:35:13,  1.98s/it]
 46%|████▌     | 8491/18627 [4:41:05<5:48:11,  2.06s/it]
 46%|████▌     | 8492/18627 [4:41:07<5:56:43,  2.11s/it]
 46%|████▌     | 8493/18627 [4:41:09<5:12:29,  1.85s/it]
 46%|████▌     | 8494/18627 [4:41:11<5:32:25,  1.97s/it]
 46%|████▌     | 8495/18627 [4:41:13<5:46:03,  2.05s/it]
 46%|████▌     | 8496/18627 [4:41:15<5:58:13,  2.12s/it]
 46%|████▌     | 8497/18627 [4:41:18<6:07:07,  2.17s/it]
 46%|████▌     | 8498/18627 [4:41:19<5:20:56,  1.90s/it]
 46%|████▌     | 8499/18627 [4:41:21<5:39:28,  2.01s/it]
 46%|████▌     | 8500/18627 [4:41:23<5:54:13,  2.10s/it]
                                                        
{'loss': 1.0946, 'grad_norm': 6.090220928192139, 'learning_rate': 2.9722614775858328e-06, 'epoch': 0.46}
+
 46%|████▌     | 8500/18627 [4:41:23<5:54:13,  2.10s/it]
 46%|████▌     | 8501/18627 [4:41:26<6:03:09,  2.15s/it]
 46%|████▌     | 8502/18627 [4:41:28<6:09:15,  2.19s/it]
 46%|████▌     | 8503/18627 [4:41:29<5:22:27,  1.91s/it]
 46%|████▌     | 8504/18627 [4:41:32<5:40:00,  2.02s/it]
 46%|████▌     | 8505/18627 [4:41:34<5:51:50,  2.09s/it]
 46%|████▌     | 8506/18627 [4:41:36<6:00:16,  2.14s/it]
 46%|████▌     | 8507/18627 [4:41:37<5:18:07,  1.89s/it]
 46%|████▌     | 8508/18627 [4:41:40<5:36:58,  2.00s/it]
 46%|████▌     | 8509/18627 [4:41:41<5:00:24,  1.78s/it]
 46%|████▌     | 8510/18627 [4:41:42<4:36:41,  1.64s/it]
                                                        
{'loss': 1.4609, 'grad_norm': 13.752340316772461, 'learning_rate': 2.9679921283673663e-06, 'epoch': 0.46}
+
 46%|████▌     | 8510/18627 [4:41:42<4:36:41,  1.64s/it]
 46%|████▌     | 8511/18627 [4:41:44<5:08:07,  1.83s/it]
 46%|████▌     | 8512/18627 [4:41:47<5:30:16,  1.96s/it]
 46%|████▌     | 8513/18627 [4:41:48<4:56:48,  1.76s/it]
 46%|████▌     | 8514/18627 [4:41:50<5:21:34,  1.91s/it]
 46%|████▌     | 8515/18627 [4:41:52<4:51:25,  1.73s/it]
 46%|████▌     | 8516/18627 [4:41:53<4:29:47,  1.60s/it]
 46%|████▌     | 8517/18627 [4:41:55<5:03:35,  1.80s/it]
 46%|████▌     | 8518/18627 [4:41:57<5:26:12,  1.94s/it]
 46%|████▌     | 8519/18627 [4:42:00<5:43:55,  2.04s/it]
 46%|████▌     | 8520/18627 [4:42:02<5:55:57,  2.11s/it]
                                                        
{'loss': 1.2311, 'grad_norm': 5.422159194946289, 'learning_rate': 2.9637213642740707e-06, 'epoch': 0.46}
+
 46%|████▌     | 8520/18627 [4:42:02<5:55:57,  2.11s/it]
 46%|████▌     | 8521/18627 [4:42:03<5:14:26,  1.87s/it]
 46%|████▌     | 8522/18627 [4:42:05<4:45:27,  1.69s/it]
 46%|████▌     | 8523/18627 [4:42:07<5:13:39,  1.86s/it]
 46%|████▌     | 8524/18627 [4:42:09<5:36:28,  2.00s/it]
 46%|████▌     | 8525/18627 [4:42:11<5:48:26,  2.07s/it]
 46%|████▌     | 8526/18627 [4:42:13<5:08:32,  1.83s/it]
 46%|████▌     | 8527/18627 [4:42:14<4:42:34,  1.68s/it]
 46%|████▌     | 8528/18627 [4:42:16<5:11:42,  1.85s/it]
 46%|████▌     | 8529/18627 [4:42:19<5:34:44,  1.99s/it]
 46%|████▌     | 8530/18627 [4:42:21<5:47:25,  2.06s/it]
                                                        
{'loss': 1.4937, 'grad_norm': 5.118160247802734, 'learning_rate': 2.959449198217695e-06, 'epoch': 0.46}
+
 46%|████▌     | 8530/18627 [4:42:21<5:47:25,  2.06s/it]
 46%|████▌     | 8531/18627 [4:42:23<5:58:16,  2.13s/it]
 46%|████▌     | 8532/18627 [4:42:25<6:04:15,  2.16s/it]
 46%|████▌     | 8533/18627 [4:42:27<5:19:52,  1.90s/it]
 46%|████▌     | 8534/18627 [4:42:28<4:49:32,  1.72s/it]
 46%|████▌     | 8535/18627 [4:42:30<5:18:17,  1.89s/it]
 46%|████▌     | 8536/18627 [4:42:32<5:35:54,  2.00s/it]
 46%|████▌     | 8537/18627 [4:42:34<4:59:04,  1.78s/it]
 46%|████▌     | 8538/18627 [4:42:35<4:33:56,  1.63s/it]
 46%|████▌     | 8539/18627 [4:42:37<5:05:51,  1.82s/it]
 46%|████▌     | 8540/18627 [4:42:40<5:28:23,  1.95s/it]
                                                        
{'loss': 1.455, 'grad_norm': 7.013083457946777, 'learning_rate': 2.9551756431142253e-06, 'epoch': 0.46}
+
 46%|████▌     | 8540/18627 [4:42:40<5:28:23,  1.95s/it]
 46%|████▌     | 8541/18627 [4:42:42<5:46:03,  2.06s/it]
 46%|████▌     | 8542/18627 [4:42:44<5:54:49,  2.11s/it]
 46%|████▌     | 8543/18627 [4:42:46<6:01:33,  2.15s/it]
 46%|████▌     | 8544/18627 [4:42:49<6:06:25,  2.18s/it]
 46%|████▌     | 8545/18627 [4:42:51<6:10:41,  2.21s/it]
 46%|████▌     | 8546/18627 [4:42:53<6:13:55,  2.23s/it]
 46%|████▌     | 8547/18627 [4:42:54<5:29:18,  1.96s/it]
 46%|████▌     | 8548/18627 [4:42:57<5:44:02,  2.05s/it]
 46%|████▌     | 8549/18627 [4:42:58<4:47:27,  1.71s/it]
 46%|████▌     | 8550/18627 [4:43:00<5:14:50,  1.87s/it]
                                                        
{'loss': 0.7972, 'grad_norm': 4.98638916015625, 'learning_rate': 2.9509007118838485e-06, 'epoch': 0.46}
+
 46%|████▌     | 8550/18627 [4:43:00<5:14:50,  1.87s/it]
 46%|████▌     | 8551/18627 [4:43:01<4:43:24,  1.69s/it]
 46%|████▌     | 8552/18627 [4:43:03<5:15:17,  1.88s/it]
 46%|████▌     | 8553/18627 [4:43:05<4:45:48,  1.70s/it]
 46%|████▌     | 8554/18627 [4:43:07<5:15:05,  1.88s/it]
 46%|████▌     | 8555/18627 [4:43:09<5:34:25,  1.99s/it]
 46%|████▌     | 8556/18627 [4:43:12<5:47:47,  2.07s/it]
 46%|████▌     | 8557/18627 [4:43:14<5:59:40,  2.14s/it]
 46%|████▌     | 8558/18627 [4:43:16<6:08:24,  2.20s/it]
 46%|████▌     | 8559/18627 [4:43:18<6:09:43,  2.20s/it]
 46%|████▌     | 8560/18627 [4:43:21<6:13:59,  2.23s/it]
                                                        
{'loss': 1.0719, 'grad_norm': 4.9389543533325195, 'learning_rate': 2.9466244174509106e-06, 'epoch': 0.46}
+
 46%|████▌     | 8560/18627 [4:43:21<6:13:59,  2.23s/it]
 46%|████▌     | 8561/18627 [4:43:23<6:16:14,  2.24s/it]
 46%|████▌     | 8562/18627 [4:43:25<6:18:06,  2.25s/it]
 46%|████▌     | 8563/18627 [4:43:28<6:20:01,  2.27s/it]
 46%|████▌     | 8564/18627 [4:43:30<6:17:30,  2.25s/it]
 46%|████▌     | 8565/18627 [4:43:31<5:28:19,  1.96s/it]
 46%|████▌     | 8566/18627 [4:43:33<5:44:46,  2.06s/it]
 46%|████▌     | 8567/18627 [4:43:36<5:56:26,  2.13s/it]
 46%|████▌     | 8568/18627 [4:43:38<6:03:23,  2.17s/it]
 46%|████▌     | 8569/18627 [4:43:40<6:05:15,  2.18s/it]
 46%|████▌     | 8570/18627 [4:43:41<5:13:59,  1.87s/it]
                                                        
{'loss': 1.0612, 'grad_norm': 14.276679039001465, 'learning_rate': 2.9423467727438805e-06, 'epoch': 0.46}
+
 46%|████▌     | 8570/18627 [4:43:41<5:13:59,  1.87s/it]
 46%|████▌     | 8571/18627 [4:43:43<5:34:04,  1.99s/it]
 46%|████▌     | 8572/18627 [4:43:45<4:58:18,  1.78s/it]
 46%|████▌     | 8573/18627 [4:43:47<5:20:56,  1.92s/it]
 46%|████▌     | 8574/18627 [4:43:48<4:51:07,  1.74s/it]
 46%|████▌     | 8575/18627 [4:43:51<5:18:42,  1.90s/it]
 46%|████▌     | 8576/18627 [4:43:53<5:36:35,  2.01s/it]
 46%|████▌     | 8577/18627 [4:43:55<5:47:40,  2.08s/it]
 46%|████▌     | 8578/18627 [4:43:57<5:55:33,  2.12s/it]
 46%|████▌     | 8579/18627 [4:44:00<6:02:15,  2.16s/it]
 46%|████▌     | 8580/18627 [4:44:02<6:06:29,  2.19s/it]
                                                        
{'loss': 1.0483, 'grad_norm': 10.387727737426758, 'learning_rate': 2.9380677906953087e-06, 'epoch': 0.46}
+
 46%|████▌     | 8580/18627 [4:44:02<6:06:29,  2.19s/it]
 46%|████▌     | 8581/18627 [4:44:03<5:20:45,  1.92s/it]
 46%|████▌     | 8582/18627 [4:44:05<5:37:13,  2.01s/it]
 46%|████▌     | 8583/18627 [4:44:07<5:00:13,  1.79s/it]
 46%|████▌     | 8584/18627 [4:44:09<5:23:17,  1.93s/it]
 46%|████▌     | 8585/18627 [4:44:10<4:51:07,  1.74s/it]
 46%|████▌     | 8586/18627 [4:44:12<5:19:18,  1.91s/it]
 46%|████▌     | 8587/18627 [4:44:14<4:51:34,  1.74s/it]
 46%|████▌     | 8588/18627 [4:44:16<5:18:40,  1.90s/it]
 46%|████▌     | 8589/18627 [4:44:17<4:46:43,  1.71s/it]
 46%|████▌     | 8590/18627 [4:44:20<5:14:30,  1.88s/it]
                                                        
{'loss': 1.7042, 'grad_norm': 8.601621627807617, 'learning_rate': 2.933787484241789e-06, 'epoch': 0.46}
+
 46%|████▌     | 8590/18627 [4:44:20<5:14:30,  1.88s/it]
 46%|████▌     | 8591/18627 [4:44:21<4:45:48,  1.71s/it]
 46%|████▌     | 8592/18627 [4:44:22<4:11:03,  1.50s/it]
 46%|████▌     | 8593/18627 [4:44:23<4:01:42,  1.45s/it]
 46%|████▌     | 8594/18627 [4:44:26<4:44:37,  1.70s/it]
 46%|████▌     | 8595/18627 [4:44:28<5:11:41,  1.86s/it]
 46%|████▌     | 8596/18627 [4:44:29<4:43:47,  1.70s/it]
 46%|████▌     | 8597/18627 [4:44:31<5:12:16,  1.87s/it]
 46%|████▌     | 8598/18627 [4:44:34<5:30:05,  1.97s/it]
 46%|████▌     | 8599/18627 [4:44:36<5:44:32,  2.06s/it]
 46%|████▌     | 8600/18627 [4:44:38<5:53:50,  2.12s/it]
                                                        
{'loss': 1.2368, 'grad_norm': 8.846522331237793, 'learning_rate': 2.9295058663239185e-06, 'epoch': 0.46}
+
 46%|████▌     | 8600/18627 [4:44:38<5:53:50,  2.12s/it]
 46%|████▌     | 8601/18627 [4:44:40<6:02:26,  2.17s/it]
 46%|████▌     | 8602/18627 [4:44:42<5:16:53,  1.90s/it]
 46%|████▌     | 8603/18627 [4:44:43<4:45:08,  1.71s/it]
 46%|████▌     | 8604/18627 [4:44:45<5:12:59,  1.87s/it]
 46%|████▌     | 8605/18627 [4:44:47<5:31:57,  1.99s/it]
 46%|████▌     | 8606/18627 [4:44:50<5:43:58,  2.06s/it]
 46%|████▌     | 8607/18627 [4:44:52<5:56:39,  2.14s/it]
 46%|████▌     | 8608/18627 [4:44:54<6:03:31,  2.18s/it]
 46%|████▌     | 8609/18627 [4:44:57<6:07:12,  2.20s/it]
 46%|████▌     | 8610/18627 [4:44:59<6:07:57,  2.20s/it]
                                                        
{'loss': 1.017, 'grad_norm': 7.508945465087891, 'learning_rate': 2.92522294988626e-06, 'epoch': 0.46}
+
 46%|████▌     | 8610/18627 [4:44:59<6:07:57,  2.20s/it]
 46%|████▌     | 8611/18627 [4:45:00<5:23:15,  1.94s/it]
 46%|████▌     | 8612/18627 [4:45:01<4:49:13,  1.73s/it]
 46%|████▌     | 8613/18627 [4:45:04<5:15:33,  1.89s/it]
 46%|████▌     | 8614/18627 [4:45:05<4:46:10,  1.71s/it]
 46%|████▋     | 8615/18627 [4:45:07<5:13:08,  1.88s/it]
 46%|████▋     | 8616/18627 [4:45:09<5:31:36,  1.99s/it]
 46%|████▋     | 8617/18627 [4:45:12<5:45:39,  2.07s/it]
 46%|████▋     | 8618/18627 [4:45:14<5:53:28,  2.12s/it]
 46%|████▋     | 8619/18627 [4:45:15<5:10:24,  1.86s/it]
 46%|████▋     | 8620/18627 [4:45:16<4:40:10,  1.68s/it]
                                                        
{'loss': 1.7619, 'grad_norm': 14.352953910827637, 'learning_rate': 2.920938747877302e-06, 'epoch': 0.46}
+
 46%|████▋     | 8620/18627 [4:45:16<4:40:10,  1.68s/it]
 46%|████▋     | 8621/18627 [4:45:19<5:10:23,  1.86s/it]
 46%|████▋     | 8622/18627 [4:45:20<4:41:57,  1.69s/it]
 46%|████▋     | 8623/18627 [4:45:22<5:10:24,  1.86s/it]
 46%|████▋     | 8624/18627 [4:45:24<5:30:12,  1.98s/it]
 46%|████▋     | 8625/18627 [4:45:26<4:55:10,  1.77s/it]
 46%|████▋     | 8626/18627 [4:45:28<5:19:38,  1.92s/it]
 46%|████▋     | 8627/18627 [4:45:30<5:37:21,  2.02s/it]
 46%|████▋     | 8628/18627 [4:45:33<5:49:43,  2.10s/it]
 46%|████▋     | 8629/18627 [4:45:35<5:56:38,  2.14s/it]
 46%|████▋     | 8630/18627 [4:45:36<5:13:38,  1.88s/it]
                                                        
{'loss': 1.4116, 'grad_norm': 15.417244911193848, 'learning_rate': 2.91665327324942e-06, 'epoch': 0.46}
+
 46%|████▋     | 8630/18627 [4:45:36<5:13:38,  1.88s/it]
 46%|████▋     | 8631/18627 [4:45:38<5:31:54,  1.99s/it]
 46%|████▋     | 8632/18627 [4:45:41<5:44:51,  2.07s/it]
 46%|████▋     | 8633/18627 [4:45:42<5:04:45,  1.83s/it]
 46%|████▋     | 8634/18627 [4:45:44<5:25:58,  1.96s/it]
 46%|████▋     | 8635/18627 [4:45:45<4:49:24,  1.74s/it]
 46%|████▋     | 8636/18627 [4:45:48<5:16:43,  1.90s/it]
 46%|████▋     | 8637/18627 [4:45:50<5:36:01,  2.02s/it]
 46%|████▋     | 8638/18627 [4:45:51<4:58:15,  1.79s/it]
 46%|████▋     | 8639/18627 [4:45:53<5:20:44,  1.93s/it]
 46%|████▋     | 8640/18627 [4:45:56<5:39:09,  2.04s/it]
                                                        
{'loss': 1.2536, 'grad_norm': 5.905208587646484, 'learning_rate': 2.9123665389588364e-06, 'epoch': 0.46}
+
 46%|████▋     | 8640/18627 [4:45:56<5:39:09,  2.04s/it]
 46%|████▋     | 8641/18627 [4:45:58<5:52:45,  2.12s/it]
 46%|████▋     | 8642/18627 [4:46:00<6:02:07,  2.18s/it]
 46%|████▋     | 8643/18627 [4:46:02<5:18:07,  1.91s/it]
 46%|████▋     | 8644/18627 [4:46:04<5:37:05,  2.03s/it]
 46%|████▋     | 8645/18627 [4:46:06<5:49:42,  2.10s/it]
 46%|████▋     | 8646/18627 [4:46:08<5:55:41,  2.14s/it]
 46%|████▋     | 8647/18627 [4:46:11<6:01:01,  2.17s/it]
 46%|████▋     | 8648/18627 [4:46:13<6:06:55,  2.21s/it]
 46%|████▋     | 8649/18627 [4:46:15<6:12:46,  2.24s/it]
 46%|████▋     | 8650/18627 [4:46:18<6:14:28,  2.25s/it]
                                                        
{'loss': 0.9175, 'grad_norm': 5.763683795928955, 'learning_rate': 2.9080785579655813e-06, 'epoch': 0.46}
+
 46%|████▋     | 8650/18627 [4:46:18<6:14:28,  2.25s/it]
 46%|████▋     | 8651/18627 [4:46:19<5:25:29,  1.96s/it]
 46%|████▋     | 8652/18627 [4:46:21<5:40:45,  2.05s/it]
 46%|████▋     | 8653/18627 [4:46:23<5:51:32,  2.11s/it]
 46%|██���█▋     | 8654/18627 [4:46:26<5:59:16,  2.16s/it]
 46%|████▋     | 8655/18627 [4:46:28<6:05:48,  2.20s/it]
 46%|████▋     | 8656/18627 [4:46:30<6:09:45,  2.22s/it]
 46%|████▋     | 8657/18627 [4:46:33<6:14:57,  2.26s/it]
 46%|████▋     | 8658/18627 [4:46:35<6:16:57,  2.27s/it]
 46%|████▋     | 8659/18627 [4:46:37<6:18:36,  2.28s/it]
 46%|████▋     | 8660/18627 [4:46:39<6:19:40,  2.29s/it]
                                                        
{'loss': 0.8137, 'grad_norm': 6.967824935913086, 'learning_rate': 2.9037893432334563e-06, 'epoch': 0.46}
+
 46%|████▋     | 8660/18627 [4:46:39<6:19:40,  2.29s/it]
 46%|████▋     | 8661/18627 [4:46:42<6:19:54,  2.29s/it]
 47%|████▋     | 8662/18627 [4:46:44<6:20:00,  2.29s/it]
 47%|████▋     | 8663/18627 [4:46:46<6:20:33,  2.29s/it]
 47%|████▋     | 8664/18627 [4:46:49<6:18:59,  2.28s/it]
 47%|████▋     | 8665/18627 [4:46:50<5:27:18,  1.97s/it]
 47%|████▋     | 8666/18627 [4:46:51<4:35:13,  1.66s/it]
 47%|████▋     | 8667/18627 [4:46:52<4:13:37,  1.53s/it]
 47%|████▋     | 8668/18627 [4:46:54<4:50:10,  1.75s/it]
 47%|████▋     | 8669/18627 [4:46:56<4:28:34,  1.62s/it]
 47%|████▋     | 8670/18627 [4:46:58<5:00:24,  1.81s/it]
                                                        
{'loss': 1.3179, 'grad_norm': 6.0751118659973145, 'learning_rate': 2.8994989077299902e-06, 'epoch': 0.47}
+
 47%|████▋     | 8670/18627 [4:46:58<5:00:24,  1.81s/it]
 47%|████▋     | 8671/18627 [4:47:00<5:22:39,  1.94s/it]
 47%|████▋     | 8672/18627 [4:47:01<4:51:17,  1.76s/it]
 47%|████▋     | 8673/18627 [4:47:02<4:13:34,  1.53s/it]
 47%|████▋     | 8674/18627 [4:47:04<3:56:56,  1.43s/it]
 47%|████▋     | 8675/18627 [4:47:05<3:50:50,  1.39s/it]
 47%|████▋     | 8676/18627 [4:47:06<3:45:45,  1.36s/it]
 47%|████▋     | 8677/18627 [4:47:07<3:42:27,  1.34s/it]
 47%|████▋     | 8678/18627 [4:47:09<3:40:05,  1.33s/it]
 47%|████▋     | 8679/18627 [4:47:10<3:41:06,  1.33s/it]
 47%|████▋     | 8680/18627 [4:47:11<3:35:03,  1.30s/it]
                                                        
{'loss': 2.2477, 'grad_norm': 13.38095474243164, 'learning_rate': 2.8952072644264035e-06, 'epoch': 0.47}
+
 47%|████▋     | 8680/18627 [4:47:11<3:35:03,  1.30s/it]
 47%|████▋     | 8681/18627 [4:47:13<3:33:29,  1.29s/it]
 47%|████▋     | 8682/18627 [4:47:15<4:22:02,  1.58s/it]
 47%|████▋     | 8683/18627 [4:47:17<4:57:51,  1.80s/it]
 47%|████▋     | 8684/18627 [4:47:19<5:21:11,  1.94s/it]
 47%|████▋     | 8685/18627 [4:47:22<5:38:44,  2.04s/it]
 47%|████▋     | 8686/18627 [4:47:23<5:02:21,  1.82s/it]
 47%|████▋     | 8687/18627 [4:47:24<4:34:35,  1.66s/it]
 47%|████▋     | 8688/18627 [4:47:27<5:05:35,  1.84s/it]
 47%|████▋     | 8689/18627 [4:47:29<5:27:40,  1.98s/it]
 47%|████▋     | 8690/18627 [4:47:30<4:53:42,  1.77s/it]
                                                        
{'loss': 1.5232, 'grad_norm': 13.883898735046387, 'learning_rate': 2.8909144262975696e-06, 'epoch': 0.47}
+
 47%|████▋     | 8690/18627 [4:47:30<4:53:42,  1.77s/it]
 47%|████▋     | 8691/18627 [4:47:32<5:18:50,  1.93s/it]
 47%|████▋     | 8692/18627 [4:47:35<5:35:47,  2.03s/it]
 47%|████▋     | 8693/18627 [4:47:37<5:47:06,  2.10s/it]
 47%|████▋     | 8694/18627 [4:47:39<5:56:11,  2.15s/it]
 47%|████▋     | 8695/18627 [4:47:42<6:01:33,  2.18s/it]
 47%|████▋     | 8696/18627 [4:47:44<6:06:28,  2.21s/it]
 47%|████▋     | 8697/18627 [4:47:46<6:11:08,  2.24s/it]
 47%|████▋     | 8698/18627 [4:47:48<6:13:37,  2.26s/it]
 47%|████▋     | 8699/18627 [4:47:51<6:15:51,  2.27s/it]
 47%|████▋     | 8700/18627 [4:47:52<5:23:05,  1.95s/it]
                                                        
{'loss': 0.8771, 'grad_norm': 13.699193954467773, 'learning_rate': 2.8866204063219726e-06, 'epoch': 0.47}
+
 47%|████▋     | 8700/18627 [4:47:52<5:23:05,  1.95s/it]
 47%|████▋     | 8701/18627 [4:47:54<5:38:11,  2.04s/it]
 47%|████▋     | 8702/18627 [4:47:55<5:01:24,  1.82s/it]
 47%|████▋     | 8703/18627 [4:47:58<5:22:19,  1.95s/it]
 47%|████▋     | 8704/18627 [4:48:00<5:39:42,  2.05s/it]
 47%|████▋     | 8705/18627 [4:48:02<5:47:56,  2.10s/it]
 47%|████▋     | 8706/18627 [4:48:04<5:54:32,  2.14s/it]
 47%|████▋     | 8707/18627 [4:48:06<5:12:24,  1.89s/it]
 47%|████▋     | 8708/18627 [4:48:07<4:42:32,  1.71s/it]
 47%|████▋     | 8709/18627 [4:48:08<4:21:12,  1.58s/it]
 47%|████▋     | 8710/18627 [4:48:11<4:52:48,  1.77s/it]
                                                        
{'loss': 1.4851, 'grad_norm': 6.6185526847839355, 'learning_rate': 2.8823252174816697e-06, 'epoch': 0.47}
+
 47%|████▋     | 8710/18627 [4:48:11<4:52:48,  1.77s/it]
 47%|████▋     | 8711/18627 [4:48:13<5:16:17,  1.91s/it]
 47%|████▋     | 8712/18627 [4:48:15<5:31:35,  2.01s/it]
 47%|████▋     | 8713/18627 [4:48:17<5:45:09,  2.09s/it]
 47%|████▋     | 8714/18627 [4:48:19<5:01:30,  1.82s/it]
 47%|████▋     | 8715/18627 [4:48:20<4:34:12,  1.66s/it]
 47%|████▋     | 8716/18627 [4:48:22<5:04:54,  1.85s/it]
 47%|████▋     | 8717/18627 [4:48:23<4:36:25,  1.67s/it]
 47%|████▋     | 8718/18627 [4:48:26<5:05:38,  1.85s/it]
 47%|████▋     | 8719/18627 [4:48:28<5:25:48,  1.97s/it]
 47%|████▋     | 8720/18627 [4:48:30<5:38:19,  2.05s/it]
                                                        
{'loss': 1.2309, 'grad_norm': 5.803572654724121, 'learning_rate': 2.8780288727622525e-06, 'epoch': 0.47}
+
 47%|████▋     | 8720/18627 [4:48:30<5:38:19,  2.05s/it]
 47%|████▋     | 8721/18627 [4:48:31<4:44:41,  1.72s/it]
 47%|████▋     | 8722/18627 [4:48:33<5:12:19,  1.89s/it]
 47%|████▋     | 8723/18627 [4:48:36<5:30:26,  2.00s/it]
 47%|████▋     | 8724/18627 [4:48:38<5:42:13,  2.07s/it]
 47%|████▋     | 8725/18627 [4:48:40<5:49:44,  2.12s/it]
 47%|████▋     | 8726/18627 [4:48:42<5:55:40,  2.16s/it]
 47%|████▋     | 8727/18627 [4:48:45<5:59:59,  2.18s/it]
 47%|████▋     | 8728/18627 [4:48:47<6:03:09,  2.20s/it]
 47%|████▋     | 8729/18627 [4:48:49<6:06:17,  2.22s/it]
 47%|████▋     | 8730/18627 [4:48:50<5:20:18,  1.94s/it]
                                                        
{'loss': 0.8749, 'grad_norm': 15.1759033203125, 'learning_rate': 2.8737313851528072e-06, 'epoch': 0.47}
+
 47%|████▋     | 8730/18627 [4:48:50<5:20:18,  1.94s/it]
 47%|████▋     | 8731/18627 [4:48:52<4:50:53,  1.76s/it]
 47%|████▋     | 8732/18627 [4:48:53<4:29:28,  1.63s/it]
 47%|████▋     | 8733/18627 [4:48:55<4:58:34,  1.81s/it]
 47%|████▋     | 8734/18627 [4:48:58<5:20:11,  1.94s/it]
 47%|████▋     | 8735/18627 [4:48:59<4:46:02,  1.74s/it]
 47%|████▋     | 8736/18627 [4:49:01<5:11:06,  1.89s/it]
 47%|████▋     | 8737/18627 [4:49:03<5:29:31,  2.00s/it]
 47%|████▋     | 8738/18627 [4:49:05<5:41:16,  2.07s/it]
 47%|████▋     | 8739/18627 [4:49:07<5:01:07,  1.83s/it]
 47%|████▋     | 8740/18627 [4:49:09<5:20:57,  1.95s/it]
                                                        
{'loss': 1.6008, 'grad_norm': 5.809321880340576, 'learning_rate': 2.8694327676458757e-06, 'epoch': 0.47}
+
 47%|████▋     | 8740/18627 [4:49:09<5:20:57,  1.95s/it]
 47%|████▋     | 8741/18627 [4:49:11<5:36:25,  2.04s/it]
 47%|████▋     | 8742/18627 [4:49:14<5:47:54,  2.11s/it]
 47%|████▋     | 8743/18627 [4:49:16<5:52:40,  2.14s/it]
 47%|████▋     | 8744/18627 [4:49:18<5:58:00,  2.17s/it]
 47%|████▋     | 8745/18627 [4:49:20<6:00:56,  2.19s/it]
 47%|████▋     | 8746/18627 [4:49:21<5:14:12,  1.91s/it]
 47%|████▋     | 8747/18627 [4:49:24<5:32:45,  2.02s/it]
 47%|████▋     | 8748/18627 [4:49:26<5:44:14,  2.09s/it]
 47%|████▋     | 8749/18627 [4:49:28<5:50:07,  2.13s/it]
 47%|████▋     | 8750/18627 [4:49:30<5:55:47,  2.16s/it]
                                                        
{'loss': 0.8292, 'grad_norm': 5.8425421714782715, 'learning_rate': 2.8651330332374138e-06, 'epoch': 0.47}
+
 47%|████▋     | 8750/18627 [4:49:30<5:55:47,  2.16s/it]
 47%|████▋     | 8751/18627 [4:49:33<5:59:15,  2.18s/it]
 47%|████▋     | 8752/18627 [4:49:34<5:07:35,  1.87s/it]
 47%|████▋     | 8753/18627 [4:49:35<4:37:51,  1.69s/it]
 47%|████▋     | 8754/18627 [4:49:36<4:18:09,  1.57s/it]
 47%|████▋     | 8755/18627 [4:49:39<4:50:47,  1.77s/it]
 47%|████▋     | 8756/18627 [4:49:41<5:12:21,  1.90s/it]
 47%|████▋     | 8757/18627 [4:49:43<5:27:58,  1.99s/it]
 47%|████▋     | 8758/18627 [4:49:45<5:37:18,  2.05s/it]
 47%|████▋     | 8759/18627 [4:49:47<5:43:23,  2.09s/it]
 47%|████▋     | 8760/18627 [4:49:50<5:47:52,  2.12s/it]
                                                        
{'loss': 1.2469, 'grad_norm': 6.753209114074707, 'learning_rate': 2.8608321949267564e-06, 'epoch': 0.47}
+
 47%|████▋     | 8760/18627 [4:49:50<5:47:52,  2.12s/it]
 47%|████▋     | 8761/18627 [4:49:52<5:54:21,  2.16s/it]
 47%|████▋     | 8762/18627 [4:49:54<5:58:36,  2.18s/it]
 47%|████▋     | 8763/18627 [4:49:55<5:13:20,  1.91s/it]
 47%|████▋     | 8764/18627 [4:49:58<5:29:23,  2.00s/it]
 47%|████▋     | 8765/18627 [4:49:58<4:35:19,  1.68s/it]
 47%|████▋     | 8766/18627 [4:50:01<5:00:55,  1.83s/it]
 47%|████▋     | 8767/18627 [4:50:01<4:10:18,  1.52s/it]
 47%|████▋     | 8768/18627 [4:50:04<4:44:22,  1.73s/it]
 47%|████▋     | 8769/18627 [4:50:05<4:22:11,  1.60s/it]
 47%|████▋     | 8770/18627 [4:50:06<4:08:05,  1.51s/it]
                                                        
{'loss': 1.2593, 'grad_norm': 16.480758666992188, 'learning_rate': 2.856530265716575e-06, 'epoch': 0.47}
+
 47%|████▋     | 8770/18627 [4:50:06<4:08:05,  1.51s/it]
 47%|████▋     | 8771/18627 [4:50:08<4:43:49,  1.73s/it]
 47%|████▋     | 8772/18627 [4:50:11<5:09:10,  1.88s/it]
 47%|████▋     | 8773/18627 [4:50:13<5:27:31,  1.99s/it]
 47%|████▋     | 8774/18627 [4:50:15<5:38:32,  2.06s/it]
 47%|████▋     | 8775/18627 [4:50:17<5:46:05,  2.11s/it]
 47%|████▋     | 8776/18627 [4:50:20<5:51:51,  2.14s/it]
 47%|████▋     | 8777/18627 [4:50:21<5:08:02,  1.88s/it]
 47%|████▋     | 8778/18627 [4:50:23<5:27:08,  1.99s/it]
 47%|████▋     | 8779/18627 [4:50:25<5:40:48,  2.08s/it]
 47%|████▋     | 8780/18627 [4:50:28<5:47:53,  2.12s/it]
                                                        
{'loss': 0.8623, 'grad_norm': 5.503910064697266, 'learning_rate': 2.8522272586128397e-06, 'epoch': 0.47}
+
 47%|████▋     | 8780/18627 [4:50:28<5:47:53,  2.12s/it]
 47%|████▋     | 8781/18627 [4:50:30<5:55:46,  2.17s/it]
 47%|████▋     | 8782/18627 [4:50:32<5:59:03,  2.19s/it]
 47%|████▋     | 8783/18627 [4:50:34<6:01:19,  2.20s/it]
 47%|████▋     | 8784/18627 [4:50:37<6:01:33,  2.20s/it]
 47%|████▋     | 8785/18627 [4:50:39<6:02:07,  2.21s/it]
 47%|████▋     | 8786/18627 [4:50:40<5:14:21,  1.92s/it]
 47%|████▋     | 8787/18627 [4:50:42<5:29:34,  2.01s/it]
 47%|████▋     | 8788/18627 [4:50:45<5:38:46,  2.07s/it]
 47%|████▋     | 8789/18627 [4:50:47<6:12:23,  2.27s/it]
 47%|████▋     | 8790/18627 [4:50:49<6:10:39,  2.26s/it]
                                                        
{'loss': 0.8724, 'grad_norm': 6.338234901428223, 'learning_rate': 2.847923186624777e-06, 'epoch': 0.47}
+
 47%|████▋     | 8790/18627 [4:50:49<6:10:39,  2.26s/it]
 47%|████▋     | 8791/18627 [4:50:51<5:18:46,  1.94s/it]
 47%|████▋     | 8792/18627 [4:50:53<5:32:31,  2.03s/it]
 47%|████▋     | 8793/18627 [4:50:54<4:54:10,  1.79s/it]
 47%|████▋     | 8794/18627 [4:50:56<5:16:06,  1.93s/it]
 47%|████▋     | 8795/18627 [4:50:58<4:45:25,  1.74s/it]
 47%|████▋     | 8796/18627 [4:51:00<5:10:52,  1.90s/it]
 47%|████▋     | 8797/18627 [4:51:02<5:27:31,  2.00s/it]
 47%|████▋     | 8798/18627 [4:51:04<5:37:28,  2.06s/it]
 47%|████▋     | 8799/18627 [4:51:07<5:43:41,  2.10s/it]
 47%|████▋     | 8800/18627 [4:51:09<5:48:00,  2.12s/it]
                                                        
{'loss': 1.2689, 'grad_norm': 7.611451148986816, 'learning_rate': 2.843618062764836e-06, 'epoch': 0.47}
+
 47%|████▋     | 8800/18627 [4:51:09<5:48:00,  2.12s/it]
 47%|████▋     | 8801/18627 [4:51:13<7:29:05,  2.74s/it]
 47%|████▋     | 8802/18627 [4:51:15<7:03:57,  2.59s/it]
 47%|████▋     | 8803/18627 [4:51:16<5:56:37,  2.18s/it]
 47%|████▋     | 8804/18627 [4:51:19<5:56:15,  2.18s/it]
 47%|████▋     | 8805/18627 [4:51:20<5:11:11,  1.90s/it]
 47%|████▋     | 8806/18627 [4:51:22<5:28:43,  2.01s/it]
 47%|████▋     | 8807/18627 [4:51:24<5:39:34,  2.07s/it]
 47%|████▋     | 8808/18627 [4:51:25<4:41:04,  1.72s/it]
 47%|████▋     | 8809/18627 [4:51:26<4:18:55,  1.58s/it]
 47%|████▋     | 8810/18627 [4:51:29<4:49:33,  1.77s/it]
                                                        
{'loss': 1.174, 'grad_norm': 6.86128044128418, 'learning_rate': 2.8393119000486443e-06, 'epoch': 0.47}
+
 47%|████▋     | 8810/18627 [4:51:29<4:49:33,  1.77s/it]
 47%|████▋     | 8811/18627 [4:51:31<5:10:32,  1.90s/it]
 47%|████▋     | 8812/18627 [4:51:33<5:25:21,  1.99s/it]
 47%|████▋     | 8813/18627 [4:51:35<5:33:51,  2.04s/it]
 47%|████▋     | 8814/18627 [4:51:36<4:47:09,  1.76s/it]
 47%|████▋     | 8815/18627 [4:51:39<5:07:26,  1.88s/it]
 47%|████▋     | 8816/18627 [4:51:40<4:33:56,  1.68s/it]
 47%|████▋     | 8817/18627 [4:51:42<4:59:51,  1.83s/it]
 47%|████▋     | 8818/18627 [4:51:44<5:18:37,  1.95s/it]
 47%|████▋     | 8819/18627 [4:51:46<5:31:56,  2.03s/it]
 47%|████▋     | 8820/18627 [4:51:47<4:32:02,  1.66s/it]
                                                        
{'loss': 1.0469, 'grad_norm': 5.686656951904297, 'learning_rate': 2.8350047114949703e-06, 'epoch': 0.47}
+
 47%|████▋     | 8820/18627 [4:51:47<4:32:02,  1.66s/it]
 47%|████▋     | 8821/18627 [4:51:49<4:57:20,  1.82s/it]
 47%|████▋     | 8822/18627 [4:51:52<5:15:26,  1.93s/it]
 47%|████▋     | 8823/18627 [4:51:54<5:28:45,  2.01s/it]
 47%|████▋     | 8824/18627 [4:51:55<4:43:17,  1.73s/it]
 47%|████▋     | 8825/18627 [4:51:56<4:18:27,  1.58s/it]
 47%|████▋     | 8826/18627 [4:51:58<4:51:56,  1.79s/it]
 47%|████▋     | 8827/18627 [4:52:00<4:25:06,  1.62s/it]
 47%|████▋     | 8828/18627 [4:52:02<4:55:17,  1.81s/it]
 47%|████▋     | 8829/18627 [4:52:04<5:14:39,  1.93s/it]
 47%|████▋     | 8830/18627 [4:52:06<5:28:24,  2.01s/it]
                                                        
{'loss': 1.1025, 'grad_norm': 4.990889072418213, 'learning_rate': 2.8306965101256846e-06, 'epoch': 0.47}
+
 47%|████▋     | 8830/18627 [4:52:06<5:28:24,  2.01s/it]
 47%|████▋     | 8831/18627 [4:52:07<4:51:42,  1.79s/it]
 47%|████▋     | 8832/18627 [4:52:10<5:13:38,  1.92s/it]
 47%|████▋     | 8833/18627 [4:52:12<5:27:46,  2.01s/it]
 47%|████▋     | 8834/18627 [4:52:14<5:38:20,  2.07s/it]
 47%|████▋     | 8835/18627 [4:52:15<4:55:28,  1.81s/it]
 47%|████▋     | 8836/18627 [4:52:18<5:14:05,  1.92s/it]
 47%|████▋     | 8837/18627 [4:52:20<5:28:51,  2.02s/it]
 47%|████▋     | 8838/18627 [4:52:22<5:38:42,  2.08s/it]
 47%|████▋     | 8839/18627 [4:52:24<5:45:33,  2.12s/it]
 47%|████▋     | 8840/18627 [4:52:26<5:51:19,  2.15s/it]
                                                        
{'loss': 1.012, 'grad_norm': 6.497244834899902, 'learning_rate': 2.8263873089657186e-06, 'epoch': 0.47}
+
 47%|████▋     | 8840/18627 [4:52:26<5:51:19,  2.15s/it]
 47%|████▋     | 8841/18627 [4:52:29<5:54:23,  2.17s/it]
 47%|████▋     | 8842/18627 [4:52:31<5:57:16,  2.19s/it]
 47%|████▋     | 8843/18627 [4:52:33<5:58:10,  2.20s/it]
 47%|████▋     | 8844/18627 [4:52:35<5:59:52,  2.21s/it]
 47%|████▋     | 8845/18627 [4:52:38<6:02:08,  2.22s/it]
 47%|████▋     | 8846/18627 [4:52:39<5:15:04,  1.93s/it]
 47%|████▋     | 8847/18627 [4:52:41<5:30:05,  2.03s/it]
 48%|████▊     | 8848/18627 [4:52:42<4:51:33,  1.79s/it]
 48%|████▊     | 8849/18627 [4:52:45<5:11:01,  1.91s/it]
 48%|████▊     | 8850/18627 [4:52:47<5:23:35,  1.99s/it]
                                                        
{'loss': 1.1054, 'grad_norm': 6.101356029510498, 'learning_rate': 2.822077121043027e-06, 'epoch': 0.48}
+
 48%|████▊     | 8850/18627 [4:52:47<5:23:35,  1.99s/it]
 48%|████▊     | 8851/18627 [4:52:49<5:34:09,  2.05s/it]
 48%|████▊     | 8852/18627 [4:52:51<5:41:34,  2.10s/it]
 48%|████▊     | 8853/18627 [4:52:52<4:43:00,  1.74s/it]
 48%|████▊     | 8854/18627 [4:52:53<4:16:54,  1.58s/it]
 48%|████▊     | 8855/18627 [4:52:54<3:54:55,  1.44s/it]
 48%|████▊     | 8856/18627 [4:52:57<4:33:24,  1.68s/it]
 48%|████▊     | 8857/18627 [4:52:59<4:59:35,  1.84s/it]
 48%|████▊     | 8858/18627 [4:53:00<4:28:24,  1.65s/it]
 48%|████▊     | 8859/18627 [4:53:01<4:11:36,  1.55s/it]
 48%|████▊     | 8860/18627 [4:53:03<4:44:13,  1.75s/it]
                                                        
{'loss': 1.433, 'grad_norm': 7.093038558959961, 'learning_rate': 2.8177659593885486e-06, 'epoch': 0.48}
+
 48%|████▊     | 8860/18627 [4:53:03<4:44:13,  1.75s/it]
 48%|████▊     | 8861/18627 [4:53:06<5:06:30,  1.88s/it]
 48%|████▊     | 8862/18627 [4:53:07<4:33:52,  1.68s/it]
 48%|████▊     | 8863/18627 [4:53:08<4:10:33,  1.54s/it]
 48%|████▊     | 8864/18627 [4:53:10<4:43:22,  1.74s/it]
 48%|████▊     | 8865/18627 [4:53:12<5:03:41,  1.87s/it]
 48%|████▊     | 8866/18627 [4:53:15<5:17:13,  1.95s/it]
 48%|████▊     | 8867/18627 [4:53:17<5:28:15,  2.02s/it]
 48%|████▊     | 8868/18627 [4:53:18<4:48:34,  1.77s/it]
 48%|████▊     | 8869/18627 [4:53:20<5:08:53,  1.90s/it]
 48%|████▊     | 8870/18627 [4:53:21<4:18:49,  1.59s/it]
                                                        
{'loss': 1.3347, 'grad_norm': 4.789013862609863, 'learning_rate': 2.813453837036164e-06, 'epoch': 0.48}
+
 48%|████▊     | 8870/18627 [4:53:21<4:18:49,  1.59s/it]
 48%|████▊     | 8871/18627 [4:53:22<4:00:49,  1.48s/it]
 48%|████▊     | 8872/18627 [4:53:24<3:50:31,  1.42s/it]
 48%|████▊     | 8873/18627 [4:53:26<4:29:49,  1.66s/it]
 48%|████▊     | 8874/18627 [4:53:28<4:56:33,  1.82s/it]
 48%|████▊     | 8875/18627 [4:53:30<5:14:37,  1.94s/it]
 48%|████▊     | 8876/18627 [4:53:32<5:28:10,  2.02s/it]
 48%|████▊     | 8877/18627 [4:53:35<5:37:23,  2.08s/it]
 48%|████▊     | 8878/18627 [4:53:37<5:45:03,  2.12s/it]
 48%|████▊     | 8879/18627 [4:53:38<4:45:25,  1.76s/it]
 48%|████▊     | 8880/18627 [4:53:40<5:08:15,  1.90s/it]
                                                        
{'loss': 1.0866, 'grad_norm': 8.099120140075684, 'learning_rate': 2.809140767022661e-06, 'epoch': 0.48}
+
 48%|████▊     | 8880/18627 [4:53:40<5:08:15,  1.90s/it]
 48%|████▊     | 8881/18627 [4:53:42<5:24:46,  2.00s/it]
 48%|████▊     | 8882/18627 [4:53:44<5:35:58,  2.07s/it]
 48%|████▊     | 8883/18627 [4:53:46<4:56:18,  1.82s/it]
 48%|████▊     | 8884/18627 [4:53:48<5:13:51,  1.93s/it]
 48%|████▊     | 8885/18627 [4:53:50<5:24:40,  2.00s/it]
 48%|████▊     | 8886/18627 [4:53:51<4:46:07,  1.76s/it]
 48%|████▊     | 8887/18627 [4:53:52<4:19:00,  1.60s/it]
 48%|████▊     | 8888/18627 [4:53:54<3:58:14,  1.47s/it]
 48%|████▊     | 8889/18627 [4:53:55<3:47:10,  1.40s/it]
 48%|████▊     | 8890/18627 [4:53:56<3:38:37,  1.35s/it]
                                                        
{'loss': 1.8879, 'grad_norm': 13.19403076171875, 'learning_rate': 2.8048267623876896e-06, 'epoch': 0.48}
+
 48%|████▊     | 8890/18627 [4:53:56<3:38:37,  1.35s/it]
 48%|████▊     | 8891/18627 [4:53:57<3:16:23,  1.21s/it]
 48%|████▊     | 8892/18627 [4:53:58<3:16:35,  1.21s/it]
 48%|████▊     | 8893/18627 [4:54:00<4:02:26,  1.49s/it]
 48%|████▊     | 8894/18627 [4:54:02<3:47:53,  1.40s/it]
 48%|████▊     | 8895/18627 [4:54:04<4:26:39,  1.64s/it]
 48%|████▊     | 8896/18627 [4:54:06<4:52:46,  1.81s/it]
 48%|████▊     | 8897/18627 [4:54:08<5:08:34,  1.90s/it]
 48%|████▊     | 8898/18627 [4:54:10<5:22:47,  1.99s/it]
 48%|████▊     | 8899/18627 [4:54:12<5:33:07,  2.05s/it]
 48%|████▊     | 8900/18627 [4:54:15<5:36:36,  2.08s/it]
                                                        
{'loss': 1.0088, 'grad_norm': 5.094458103179932, 'learning_rate': 2.8005118361737276e-06, 'epoch': 0.48}
+
 48%|████▊     | 8900/18627 [4:54:15<5:36:36,  2.08s/it]
 48%|████▊     | 8901/18627 [4:54:17<5:39:23,  2.09s/it]
 48%|████▊     | 8902/18627 [4:54:19<5:41:44,  2.11s/it]
 48%|████▊     | 8903/18627 [4:54:21<5:43:11,  2.12s/it]
 48%|████▊     | 8904/18627 [4:54:23<5:44:04,  2.12s/it]
 48%|████▊     | 8905/18627 [4:54:25<5:44:10,  2.12s/it]
 48%|████▊     | 8906/18627 [4:54:27<5:44:41,  2.13s/it]
 48%|████▊     | 8907/18627 [4:54:30<5:44:16,  2.13s/it]
 48%|████▊     | 8908/18627 [4:54:31<5:01:16,  1.86s/it]
 48%|████▊     | 8909/18627 [4:54:33<5:18:06,  1.96s/it]
 48%|████▊     | 8910/18627 [4:54:34<4:42:30,  1.74s/it]
                                                        
{'loss': 1.0877, 'grad_norm': 14.920623779296875, 'learning_rate': 2.796196001426038e-06, 'epoch': 0.48}
+
 48%|████▊     | 8910/18627 [4:54:34<4:42:30,  1.74s/it]
 48%|████▊     | 8911/18627 [4:54:35<4:17:13,  1.59s/it]
 48%|████▊     | 8912/18627 [4:54:38<4:43:17,  1.75s/it]
 48%|████▊     | 8913/18627 [4:54:40<5:01:41,  1.86s/it]
 48%|████▊     | 8914/18627 [4:54:42<5:15:24,  1.95s/it]
 48%|████▊     | 8915/18627 [4:54:44<5:24:01,  2.00s/it]
 48%|████▊     | 8916/18627 [4:54:46<5:30:36,  2.04s/it]
 48%|████▊     | 8917/18627 [4:54:48<5:35:41,  2.07s/it]
 48%|████▊     | 8918/18627 [4:54:50<5:38:18,  2.09s/it]
 48%|████▊     | 8919/18627 [4:54:53<5:41:46,  2.11s/it]
 48%|████▊     | 8920/18627 [4:54:54<4:57:38,  1.84s/it]
                                                        
{'loss': 1.0661, 'grad_norm': 14.863776206970215, 'learning_rate': 2.7918792711926307e-06, 'epoch': 0.48}
+
 48%|████▊     | 8920/18627 [4:54:54<4:57:38,  1.84s/it]
 48%|████▊     | 8921/18627 [4:54:56<5:13:08,  1.94s/it]
 48%|████▊     | 8922/18627 [4:54:57<4:36:54,  1.71s/it]
 48%|████▊     | 8923/18627 [4:54:59<4:57:03,  1.84s/it]
 48%|████▊     | 8924/18627 [4:55:01<5:13:26,  1.94s/it]
 48%|████▊     | 8925/18627 [4:55:03<4:37:26,  1.72s/it]
 48%|████▊     | 8926/18627 [4:55:04<4:12:23,  1.56s/it]
 48%|████▊     | 8927/18627 [4:55:06<4:40:38,  1.74s/it]
 48%|████▊     | 8928/18627 [4:55:08<4:59:59,  1.86s/it]
 48%|████▊     | 8929/18627 [4:55:10<5:13:44,  1.94s/it]
 48%|████▊     | 8930/18627 [4:55:12<5:23:23,  2.00s/it]
                                                        
{'loss': 1.2764, 'grad_norm': 5.470398902893066, 'learning_rate': 2.787561658524223e-06, 'epoch': 0.48}
+
 48%|████▊     | 8930/18627 [4:55:12<5:23:23,  2.00s/it]
 48%|████▊     | 8931/18627 [4:55:14<5:29:38,  2.04s/it]
 48%|████▊     | 8932/18627 [4:55:17<5:33:33,  2.06s/it]
 48%|████▊     | 8933/18627 [4:55:19<5:37:43,  2.09s/it]
 48%|████▊     | 8934/18627 [4:55:21<5:39:13,  2.10s/it]
 48%|████▊     | 8935/18627 [4:55:22<4:35:46,  1.71s/it]
 48%|████▊     | 8936/18627 [4:55:24<4:56:00,  1.83s/it]
 48%|████▊     | 8937/18627 [4:55:25<4:24:49,  1.64s/it]
 48%|████▊     | 8938/18627 [4:55:27<4:51:51,  1.81s/it]
 48%|████▊     | 8939/18627 [4:55:29<5:10:48,  1.92s/it]
 48%|████▊     | 8940/18627 [4:55:31<5:20:44,  1.99s/it]
                                                        
{'loss': 0.8507, 'grad_norm': 5.766788959503174, 'learning_rate': 2.7832431764742015e-06, 'epoch': 0.48}
+
 48%|████▊     | 8940/18627 [4:55:32<5:20:44,  1.99s/it]
 48%|████▊     | 8941/18627 [4:55:34<5:28:09,  2.03s/it]
 48%|████▊     | 8942/18627 [4:55:35<4:47:27,  1.78s/it]
 48%|████▊     | 8943/18627 [4:55:37<5:05:11,  1.89s/it]
 48%|████▊     | 8944/18627 [4:55:39<5:17:19,  1.97s/it]
 48%|████▊     | 8945/18627 [4:55:41<5:27:29,  2.03s/it]
 48%|████▊     | 8946/18627 [4:55:43<4:49:51,  1.80s/it]
 48%|████▊     | 8947/18627 [4:55:45<5:07:18,  1.90s/it]
 48%|████▊     | 8948/18627 [4:55:47<5:18:34,  1.97s/it]
 48%|████▊     | 8949/18627 [4:55:49<5:25:53,  2.02s/it]
 48%|████▊     | 8950/18627 [4:55:50<4:45:56,  1.77s/it]
                                                        
{'loss': 1.1618, 'grad_norm': 25.32003402709961, 'learning_rate': 2.7789238380985765e-06, 'epoch': 0.48}
+
 48%|████▊     | 8950/18627 [4:55:50<4:45:56,  1.77s/it]
 48%|████▊     | 8951/18627 [4:55:51<4:18:45,  1.60s/it]
 48%|████▊     | 8952/18627 [4:55:54<4:44:58,  1.77s/it]
 48%|████▊     | 8953/18627 [4:55:55<4:14:11,  1.58s/it]
 48%|████▊     | 8954/18627 [4:55:56<3:55:29,  1.46s/it]
 48%|████▊     | 8955/18627 [4:55:58<4:29:31,  1.67s/it]
 48%|████▊     | 8956/18627 [4:55:59<4:07:11,  1.53s/it]
 48%|████▊     | 8957/18627 [4:56:01<4:37:27,  1.72s/it]
 48%|████▊     | 8958/18627 [4:56:04<4:59:05,  1.86s/it]
 48%|████▊     | 8959/18627 [4:56:06<5:13:48,  1.95s/it]
 48%|████▊     | 8960/18627 [4:56:08<5:24:56,  2.02s/it]
                                                        
{'loss': 1.5303, 'grad_norm': 6.201385974884033, 'learning_rate': 2.7746036564559525e-06, 'epoch': 0.48}
+
 48%|████▊     | 8960/18627 [4:56:08<5:24:56,  2.02s/it]
 48%|████▊     | 8961/18627 [4:56:09<4:46:09,  1.78s/it]
 48%|████▊     | 8962/18627 [4:56:10<4:18:39,  1.61s/it]
 48%|████▊     | 8963/18627 [4:56:12<4:45:55,  1.78s/it]
 48%|████▊     | 8964/18627 [4:56:15<5:04:55,  1.89s/it]
 48%|████▊     | 8965/18627 [4:56:17<5:17:58,  1.97s/it]
 48%|████▊     | 8966/18627 [4:56:19<5:27:20,  2.03s/it]
 48%|████▊     | 8967/18627 [4:56:20<4:43:50,  1.76s/it]
 48%|████▊     | 8968/18627 [4:56:22<5:03:16,  1.88s/it]
 48%|████▊     | 8969/18627 [4:56:24<5:17:37,  1.97s/it]
 48%|████▊     | 8970/18627 [4:56:27<5:27:27,  2.03s/it]
                                                        
{'loss': 1.1878, 'grad_norm': 7.039514064788818, 'learning_rate': 2.7702826446074816e-06, 'epoch': 0.48}
+
 48%|████▊     | 8970/18627 [4:56:27<5:27:27,  2.03s/it]
 48%|████▊     | 8971/18627 [4:56:29<5:34:12,  2.08s/it]
 48%|████▊     | 8972/18627 [4:56:31<5:38:08,  2.10s/it]
 48%|████▊     | 8973/18627 [4:56:33<5:40:42,  2.12s/it]
 48%|████▊     | 8974/18627 [4:56:35<5:43:41,  2.14s/it]
 48%|████▊     | 8975/18627 [4:56:37<4:59:46,  1.86s/it]
 48%|████▊     | 8976/18627 [4:56:39<5:11:53,  1.94s/it]
 48%|████▊     | 8977/18627 [4:56:41<5:20:17,  1.99s/it]
 48%|████▊     | 8978/18627 [4:56:43<5:25:35,  2.02s/it]
 48%|████▊     | 8979/18627 [4:56:45<5:30:36,  2.06s/it]
 48%|████▊     | 8980/18627 [4:56:47<5:33:19,  2.07s/it]
                                                        
{'loss': 0.8973, 'grad_norm': 5.601466655731201, 'learning_rate': 2.7659608156168248e-06, 'epoch': 0.48}
+
 48%|████▊     | 8980/18627 [4:56:47<5:33:19,  2.07s/it]
 48%|████▊     | 8981/18627 [4:56:49<5:36:13,  2.09s/it]
 48%|████▊     | 8982/18627 [4:56:51<5:37:19,  2.10s/it]
 48%|████▊     | 8983/18627 [4:56:53<5:37:23,  2.10s/it]
 48%|████▊     | 8984/18627 [4:56:56<5:37:52,  2.10s/it]
 48%|████▊     | 8985/18627 [4:56:57<4:54:11,  1.83s/it]
 48%|████▊     | 8986/18627 [4:56:59<5:08:38,  1.92s/it]
 48%|████▊     | 8987/18627 [4:57:01<5:17:34,  1.98s/it]
 48%|████▊     | 8988/18627 [4:57:02<4:40:06,  1.74s/it]
 48%|████▊     | 8989/18627 [4:57:03<4:15:46,  1.59s/it]
 48%|████▊     | 8990/18627 [4:57:06<4:41:26,  1.75s/it]
                                                        
{'loss': 1.3379, 'grad_norm': 8.506794929504395, 'learning_rate': 2.761638182550115e-06, 'epoch': 0.48}
+
 48%|████▊     | 8990/18627 [4:57:06<4:41:26,  1.75s/it]
 48%|████▊     | 8991/18627 [4:57:08<4:58:44,  1.86s/it]
 48%|████▊     | 8992/18627 [4:57:09<4:26:45,  1.66s/it]
 48%|████▊     | 8993/18627 [4:57:11<4:49:51,  1.81s/it]
 48%|████▊     | 8994/18627 [4:57:13<5:04:56,  1.90s/it]
 48%|████▊     | 8995/18627 [4:57:14<4:24:46,  1.65s/it]
 48%|████▊     | 8996/18627 [4:57:15<4:02:21,  1.51s/it]
 48%|████▊     | 8997/18627 [4:57:17<3:46:45,  1.41s/it]
 48%|████▊     | 8998/18627 [4:57:19<4:20:54,  1.63s/it]
 48%|████▊     | 8999/18627 [4:57:21<4:45:03,  1.78s/it]
 48%|████▊     | 9000/18627 [4:57:22<3:52:32,  1.45s/it]
                                                        
{'loss': 1.4305, 'grad_norm': 4.990551948547363, 'learning_rate': 2.7573147584759146e-06, 'epoch': 0.48}
+
 48%|████▊     | 9000/18627 [4:57:22<3:52:32,  1.45s/it]/home/zhengduo/workspace/LLaMA-Factory/src/qwen_vl/model/modeling_qwen2_5_vl.py:1979: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
+  with torch.cuda.amp.autocast(dtype=dtype):
+/home/zhengduo/miniconda3/envs/vgllm/lib/python3.10/site-packages/torch/utils/checkpoint.py:87: UserWarning: None of the inputs have requires_grad=True. Gradients will be None
+  warnings.warn(
+
 48%|████▊     | 9001/18627 [4:58:01<34:36:17, 12.94s/it]
 48%|████▊     | 9002/18627 [4:58:03<25:57:44,  9.71s/it]
 48%|████▊     | 9003/18627 [4:58:05<19:07:41,  7.16s/it]
 48%|████▊     | 9004/18627 [4:58:07<15:05:14,  5.64s/it]
 48%|████▊     | 9005/18627 [4:58:09<12:15:11,  4.58s/it]
 48%|████▊     | 9006/18627 [4:58:11<10:16:52,  3.85s/it]
 48%|████▊     | 9007/18627 [4:58:13<8:53:28,  3.33s/it] 
 48%|████▊     | 9008/18627 [4:58:15<7:54:39,  2.96s/it]
 48%|████▊     | 9009/18627 [4:58:17<7:15:08,  2.71s/it]
 48%|████▊     | 9010/18627 [4:58:19<6:01:33,  2.26s/it]
                                                        
{'loss': 1.0015, 'grad_norm': 16.103673934936523, 'learning_rate': 2.7529905564651793e-06, 'epoch': 0.48}
+
 48%|████▊     | 9010/18627 [4:58:19<6:01:33,  2.26s/it]
 48%|████▊     | 9011/18627 [4:58:20<5:10:23,  1.94s/it]
 48%|████▊     | 9012/18627 [4:58:21<4:34:09,  1.71s/it]
 48%|████▊     | 9013/18627 [4:58:23<4:53:44,  1.83s/it]
 48%|████▊     | 9014/18627 [4:58:25<5:07:58,  1.92s/it]
 48%|████▊     | 9015/18627 [4:58:27<5:16:53,  1.98s/it]
 48%|████▊     | 9016/18627 [4:58:29<5:24:00,  2.02s/it]
 48%|████▊     | 9017/18627 [4:58:32<5:29:08,  2.06s/it]
 48%|████▊     | 9018/18627 [4:58:32<4:24:21,  1.65s/it]
 48%|████▊     | 9019/18627 [4:58:33<4:02:33,  1.51s/it]
 48%|████▊     | 9020/18627 [4:58:36<4:31:51,  1.70s/it]
                                                        
{'loss': 1.336, 'grad_norm': 5.858074188232422, 'learning_rate': 2.748665589591215e-06, 'epoch': 0.48}
+
 48%|████▊     | 9020/18627 [4:58:36<4:31:51,  1.70s/it]
 48%|████▊     | 9021/18627 [4:58:38<4:52:47,  1.83s/it]
 48%|████▊     | 9022/18627 [4:58:40<5:07:43,  1.92s/it]
 48%|████▊     | 9023/18627 [4:58:42<5:17:15,  1.98s/it]
 48%|████▊     | 9024/18627 [4:58:44<5:23:39,  2.02s/it]
 48%|████▊     | 9025/18627 [4:58:46<5:28:03,  2.05s/it]
 48%|████▊     | 9026/18627 [4:58:47<4:46:44,  1.79s/it]
 48%|████▊     | 9027/18627 [4:58:49<5:01:42,  1.89s/it]
 48%|████▊     | 9028/18627 [4:58:52<5:12:22,  1.95s/it]
 48%|████▊     | 9029/18627 [4:58:54<5:19:06,  1.99s/it]
 48%|████▊     | 9030/18627 [4:58:56<5:24:09,  2.03s/it]
                                                        
{'loss': 0.8517, 'grad_norm': 6.731173992156982, 'learning_rate': 2.744339870929641e-06, 'epoch': 0.48}
+
 48%|████▊     | 9030/18627 [4:58:56<5:24:09,  2.03s/it]
 48%|████▊     | 9031/18627 [4:58:58<5:28:11,  2.05s/it]
 48%|████▊     | 9032/18627 [4:58:59<4:46:41,  1.79s/it]
 48%|████▊     | 9033/18627 [4:59:01<5:02:52,  1.89s/it]
 48%|████▊     | 9034/18627 [4:59:02<4:29:31,  1.69s/it]
 49%|████▊     | 9035/18627 [4:59:03<3:44:55,  1.41s/it]
 49%|████▊     | 9036/18627 [4:59:04<3:35:15,  1.35s/it]
 49%|████▊     | 9037/18627 [4:59:07<4:12:40,  1.58s/it]
 49%|████▊     | 9038/18627 [4:59:09<4:38:26,  1.74s/it]
 49%|████▊     | 9039/18627 [4:59:11<4:56:49,  1.86s/it]
 49%|████▊     | 9040/18627 [4:59:13<5:08:52,  1.93s/it]
                                                        
{'loss': 1.2465, 'grad_norm': 7.600436210632324, 'learning_rate': 2.7400134135583487e-06, 'epoch': 0.49}
+
 49%|████▊     | 9040/18627 [4:59:13<5:08:52,  1.93s/it]
 49%|████▊     | 9041/18627 [4:59:15<5:16:54,  1.98s/it]
 49%|████▊     | 9042/18627 [4:59:17<5:23:14,  2.02s/it]
 49%|████▊     | 9043/18627 [4:59:18<4:42:49,  1.77s/it]
 49%|████▊     | 9044/18627 [4:59:20<4:59:02,  1.87s/it]
 49%|████▊     | 9045/18627 [4:59:22<5:10:32,  1.94s/it]
 49%|████▊     | 9046/18627 [4:59:25<5:18:06,  1.99s/it]
 49%|████▊     | 9047/18627 [4:59:27<5:23:40,  2.03s/it]
 49%|████▊     | 9048/18627 [4:59:28<4:25:23,  1.66s/it]
 49%|████▊     | 9049/18627 [4:59:29<4:03:00,  1.52s/it]
 49%|████▊     | 9050/18627 [4:59:31<4:32:15,  1.71s/it]
                                                        
{'loss': 0.9554, 'grad_norm': 6.899526119232178, 'learning_rate': 2.735686230557464e-06, 'epoch': 0.49}
+
 49%|████▊     | 9050/18627 [4:59:31<4:32:15,  1.71s/it]
 49%|████▊     | 9051/18627 [4:59:33<4:52:20,  1.83s/it]
 49%|████▊     | 9052/18627 [4:59:35<5:05:25,  1.91s/it]
 49%|████▊     | 9053/18627 [4:59:36<4:31:33,  1.70s/it]
 49%|████▊     | 9054/18627 [4:59:38<4:51:42,  1.83s/it]
 49%|████▊     | 9055/18627 [4:59:41<5:06:02,  1.92s/it]
 49%|████▊     | 9056/18627 [4:59:43<5:15:20,  1.98s/it]
 49%|████▊     | 9057/18627 [4:59:45<5:22:39,  2.02s/it]
 49%|████▊     | 9058/18627 [4:59:47<5:26:31,  2.05s/it]
 49%|████▊     | 9059/18627 [4:59:49<5:30:31,  2.07s/it]
 49%|████▊     | 9060/18627 [4:59:51<5:33:20,  2.09s/it]
                                                        
{'loss': 0.8911, 'grad_norm': 8.104578971862793, 'learning_rate': 2.731358335009306e-06, 'epoch': 0.49}
+
 49%|████▊     | 9060/18627 [4:59:51<5:33:20,  2.09s/it]
 49%|████▊     | 9061/18627 [4:59:52<4:50:27,  1.82s/it]
 49%|████▊     | 9062/18627 [4:59:54<5:04:18,  1.91s/it]
 49%|████▊     | 9063/18627 [4:59:56<4:29:45,  1.69s/it]
 49%|████▊     | 9064/18627 [4:59:57<4:05:21,  1.54s/it]
 49%|████▊     | 9065/18627 [4:59:59<4:32:37,  1.71s/it]
 49%|████▊     | 9066/18627 [5:00:01<4:52:38,  1.84s/it]
 49%|████▊     | 9067/18627 [5:00:03<5:06:25,  1.92s/it]
 49%|████▊     | 9068/18627 [5:00:05<5:16:05,  1.98s/it]
 49%|████▊     | 9069/18627 [5:00:07<5:22:39,  2.03s/it]
 49%|████▊     | 9070/18627 [5:00:10<5:26:54,  2.05s/it]
                                                        
{'loss': 1.2492, 'grad_norm': 3.9029831886291504, 'learning_rate': 2.727029739998347e-06, 'epoch': 0.49}
+
 49%|████▊     | 9070/18627 [5:00:10<5:26:54,  2.05s/it]
 49%|████▊     | 9071/18627 [5:00:12<5:30:19,  2.07s/it]
 49%|████▊     | 9072/18627 [5:00:14<5:31:09,  2.08s/it]
 49%|████▊     | 9073/18627 [5:00:16<5:31:52,  2.08s/it]
 49%|████▊     | 9074/18627 [5:00:17<4:49:05,  1.82s/it]
 49%|████▊     | 9075/18627 [5:00:18<4:18:43,  1.63s/it]
 49%|████▊     | 9076/18627 [5:00:20<4:41:35,  1.77s/it]
 49%|████▊     | 9077/18627 [5:00:24<5:59:30,  2.26s/it]
 49%|████▊     | 9078/18627 [5:00:26<5:52:57,  2.22s/it]
 49%|████▊     | 9079/18627 [5:00:28<5:48:09,  2.19s/it]
 49%|████▊     | 9080/18627 [5:00:30<5:43:53,  2.16s/it]
                                                        
{'loss': 1.0941, 'grad_norm': 7.332837104797363, 'learning_rate': 2.722700458611175e-06, 'epoch': 0.49}
+
 49%|████▊     | 9080/18627 [5:00:30<5:43:53,  2.16s/it]
 49%|████▉     | 9081/18627 [5:00:32<5:43:03,  2.16s/it]
 49%|████▉     | 9082/18627 [5:00:34<5:41:24,  2.15s/it]
 49%|████▉     | 9083/18627 [5:00:36<4:55:54,  1.86s/it]
 49%|████▉     | 9084/18627 [5:00:37<4:24:08,  1.66s/it]
 49%|████▉     | 9085/18627 [5:00:39<4:46:05,  1.80s/it]
 49%|████▉     | 9086/18627 [5:00:40<4:16:48,  1.62s/it]
 49%|████▉     | 9087/18627 [5:00:42<4:39:57,  1.76s/it]
 49%|████▉     | 9088/18627 [5:00:43<4:12:39,  1.59s/it]
 49%|████▉     | 9089/18627 [5:00:45<4:37:42,  1.75s/it]
 49%|████▉     | 9090/18627 [5:00:48<4:54:50,  1.85s/it]
                                                        
{'loss': 1.4759, 'grad_norm': 5.136799335479736, 'learning_rate': 2.7183705039364537e-06, 'epoch': 0.49}
+
 49%|████▉     | 9090/18627 [5:00:48<4:54:50,  1.85s/it]
 49%|████▉     | 9091/18627 [5:00:50<5:06:45,  1.93s/it]
 49%|████▉     | 9092/18627 [5:00:52<5:15:41,  1.99s/it]
 49%|████▉     | 9093/18627 [5:00:54<5:21:06,  2.02s/it]
 49%|████▉     | 9094/18627 [5:00:56<5:25:47,  2.05s/it]
 49%|████▉     | 9095/18627 [5:00:58<5:29:47,  2.08s/it]
 49%|████▉     | 9096/18627 [5:01:00<5:32:10,  2.09s/it]
 49%|████▉     | 9097/18627 [5:01:02<5:33:07,  2.10s/it]
 49%|████▉     | 9098/18627 [5:01:04<5:33:22,  2.10s/it]
 49%|████▉     | 9099/18627 [5:01:07<5:34:40,  2.11s/it]
 49%|████▉     | 9100/18627 [5:01:09<5:35:26,  2.11s/it]
                                                        
{'loss': 0.6203, 'grad_norm': 5.5781331062316895, 'learning_rate': 2.714039889064882e-06, 'epoch': 0.49}
+
 49%|████▉     | 9100/18627 [5:01:09<5:35:26,  2.11s/it]
 49%|████▉     | 9101/18627 [5:01:11<5:37:04,  2.12s/it]
 49%|████▉     | 9102/18627 [5:01:13<5:37:33,  2.13s/it]
 49%|████▉     | 9103/18627 [5:01:14<4:53:00,  1.85s/it]
 49%|████▉     | 9104/18627 [5:01:16<5:06:26,  1.93s/it]
 49%|████▉     | 9105/18627 [5:01:18<5:15:39,  1.99s/it]
 49%|████▉     | 9106/18627 [5:01:21<5:21:55,  2.03s/it]
 49%|████▉     | 9107/18627 [5:01:23<5:26:21,  2.06s/it]
 49%|████▉     | 9108/18627 [5:01:25<5:29:51,  2.08s/it]
 49%|████▉     | 9109/18627 [5:01:27<5:31:30,  2.09s/it]
 49%|████▉     | 9110/18627 [5:01:28<4:45:04,  1.80s/it]
                                                        
{'loss': 0.9831, 'grad_norm': 13.513917922973633, 'learning_rate': 2.7097086270891533e-06, 'epoch': 0.49}
+
 49%|████▉     | 9110/18627 [5:01:28<4:45:04,  1.80s/it]
 49%|████▉     | 9111/18627 [5:01:30<5:00:23,  1.89s/it]
 49%|████▉     | 9112/18627 [5:01:32<5:10:00,  1.95s/it]
 49%|████▉     | 9113/18627 [5:01:34<5:17:18,  2.00s/it]
 49%|████▉     | 9114/18627 [5:01:36<4:38:47,  1.76s/it]
 49%|████▉     | 9115/18627 [5:01:38<4:55:38,  1.86s/it]
 49%|████▉     | 9116/18627 [5:01:40<5:07:30,  1.94s/it]
 49%|████▉     | 9117/18627 [5:01:42<5:15:04,  1.99s/it]
 49%|████▉     | 9118/18627 [5:01:44<5:20:59,  2.03s/it]
 49%|████▉     | 9119/18627 [5:01:45<4:42:41,  1.78s/it]
 49%|████▉     | 9120/18627 [5:01:47<4:57:29,  1.88s/it]
                                                        
{'loss': 1.1232, 'grad_norm': 6.96962308883667, 'learning_rate': 2.7053767311039186e-06, 'epoch': 0.49}
+
 49%|████▉     | 9120/18627 [5:01:47<4:57:29,  1.88s/it]
 49%|████▉     | 9121/18627 [5:01:49<5:11:13,  1.96s/it]
 49%|████▉     | 9122/18627 [5:01:52<5:28:49,  2.08s/it]
 49%|████▉     | 9123/18627 [5:01:54<5:42:37,  2.16s/it]
 49%|████▉     | 9124/18627 [5:01:57<5:54:05,  2.24s/it]
 49%|████▉     | 9125/18627 [5:01:59<5:59:20,  2.27s/it]
 49%|████▉     | 9126/18627 [5:02:00<5:19:57,  2.02s/it]
 49%|████▉     | 9127/18627 [5:02:03<5:34:56,  2.12s/it]
 49%|████▉     | 9128/18627 [5:02:05<5:45:57,  2.19s/it]
 49%|████▉     | 9129/18627 [5:02:07<5:10:58,  1.96s/it]
 49%|████▉     | 9130/18627 [5:02:09<5:27:36,  2.07s/it]
                                                        
{'loss': 1.0134, 'grad_norm': 5.68367338180542, 'learning_rate': 2.7010442142057466e-06, 'epoch': 0.49}
+
 49%|████▉     | 9130/18627 [5:02:09<5:27:36,  2.07s/it]
 49%|████▉     | 9131/18627 [5:02:11<5:42:04,  2.16s/it]
 49%|████▉     | 9132/18627 [5:02:13<5:07:45,  1.94s/it]
 49%|████▉     | 9133/18627 [5:02:15<5:28:44,  2.08s/it]
 49%|████▉     | 9134/18627 [5:02:16<4:57:11,  1.88s/it]
 49%|████▉     | 9135/18627 [5:02:19<5:19:58,  2.02s/it]
 49%|████▉     | 9136/18627 [5:02:21<5:34:02,  2.11s/it]
 49%|████▉     | 9137/18627 [5:02:23<5:45:13,  2.18s/it]
 49%|████▉     | 9138/18627 [5:02:26<5:55:47,  2.25s/it]
 49%|████▉     | 9139/18627 [5:02:28<5:59:47,  2.28s/it]
 49%|████▉     | 9140/18627 [5:02:31<6:04:49,  2.31s/it]
                                                        
{'loss': 1.0194, 'grad_norm': 4.899198532104492, 'learning_rate': 2.6967110894930814e-06, 'epoch': 0.49}
+
 49%|████▉     | 9140/18627 [5:02:31<6:04:49,  2.31s/it]
 49%|████▉     | 9141/18627 [5:02:32<5:07:00,  1.94s/it]
 49%|████▉     | 9142/18627 [5:02:34<5:25:54,  2.06s/it]
 49%|████▉     | 9143/18627 [5:02:36<5:39:25,  2.15s/it]
 49%|████▉     | 9144/18627 [5:02:38<5:02:33,  1.91s/it]
 49%|████▉     | 9145/18627 [5:02:40<5:25:40,  2.06s/it]
 49%|████▉     | 9146/18627 [5:02:42<5:37:42,  2.14s/it]
 49%|████▉     | 9147/18627 [5:02:45<5:47:40,  2.20s/it]
 49%|████▉     | 9148/18627 [5:02:46<5:07:02,  1.94s/it]
 49%|████▉     | 9149/18627 [5:02:49<5:26:45,  2.07s/it]
 49%|████▉     | 9150/18627 [5:02:50<4:59:35,  1.90s/it]
                                                        
{'loss': 1.2725, 'grad_norm': 14.799491882324219, 'learning_rate': 2.692377370066206e-06, 'epoch': 0.49}
+
 49%|████▉     | 9150/18627 [5:02:50<4:59:35,  1.90s/it]
 49%|████▉     | 9151/18627 [5:02:52<5:22:24,  2.04s/it]
 49%|████▉     | 9152/18627 [5:02:55<5:35:42,  2.13s/it]
 49%|████▉     | 9153/18627 [5:02:57<5:44:58,  2.18s/it]
 49%|████▉     | 9154/18627 [5:02:59<5:52:04,  2.23s/it]
 49%|████▉     | 9155/18627 [5:03:02<5:58:05,  2.27s/it]
 49%|████▉     | 9156/18627 [5:03:04<6:02:55,  2.30s/it]
 49%|████▉     | 9157/18627 [5:03:06<6:05:51,  2.32s/it]
 49%|████▉     | 9158/18627 [5:03:09<6:07:51,  2.33s/it]
 49%|████▉     | 9159/18627 [5:03:11<6:10:22,  2.35s/it]
 49%|████▉     | 9160/18627 [5:03:14<6:10:33,  2.35s/it]
                                                        
{'loss': 0.6603, 'grad_norm': 5.474850654602051, 'learning_rate': 2.6880430690271997e-06, 'epoch': 0.49}
+
 49%|████▉     | 9160/18627 [5:03:14<6:10:33,  2.35s/it]
 49%|████▉     | 9161/18627 [5:03:16<6:10:47,  2.35s/it]
 49%|████▉     | 9162/18627 [5:03:18<6:13:07,  2.37s/it]
 49%|████▉     | 9163/18627 [5:03:21<6:12:24,  2.36s/it]
 49%|████▉     | 9164/18627 [5:03:23<6:09:55,  2.35s/it]
 49%|████▉     | 9165/18627 [5:03:25<6:08:15,  2.34s/it]
 49%|████▉     | 9166/18627 [5:03:28<6:10:59,  2.35s/it]
 49%|████▉     | 9167/18627 [5:03:30<6:13:39,  2.37s/it]
 49%|████▉     | 9168/18627 [5:03:32<6:13:38,  2.37s/it]
 49%|████▉     | 9169/18627 [5:03:35<6:11:53,  2.36s/it]
 49%|████▉     | 9170/18627 [5:03:36<5:30:34,  2.10s/it]
                                                        
{'loss': 0.7897, 'grad_norm': 14.193864822387695, 'learning_rate': 2.683708199479903e-06, 'epoch': 0.49}
+
 49%|████▉     | 9170/18627 [5:03:36<5:30:34,  2.10s/it]
 49%|████▉     | 9171/18627 [5:03:39<5:44:44,  2.19s/it]
 49%|████▉     | 9172/18627 [5:03:41<5:51:17,  2.23s/it]
 49%|████▉     | 9173/18627 [5:03:42<5:12:42,  1.98s/it]
 49%|████▉     | 9174/18627 [5:03:45<5:32:11,  2.11s/it]
 49%|████▉     | 9175/18627 [5:03:47<5:43:10,  2.18s/it]
 49%|████▉     | 9176/18627 [5:03:49<5:48:51,  2.21s/it]
 49%|████▉     | 9177/18627 [5:03:52<5:54:31,  2.25s/it]
 49%|████▉     | 9178/18627 [5:03:53<5:16:16,  2.01s/it]
 49%|████▉     | 9179/18627 [5:03:55<4:48:06,  1.83s/it]
 49%|████▉     | 9180/18627 [5:03:57<5:12:52,  1.99s/it]
                                                        
{'loss': 1.1785, 'grad_norm': 5.760058403015137, 'learning_rate': 2.6793727745298727e-06, 'epoch': 0.49}
+
 49%|████▉     | 9180/18627 [5:03:57<5:12:52,  1.99s/it]
 49%|████▉     | 9181/18627 [5:03:59<5:28:20,  2.09s/it]
 49%|████▉     | 9182/18627 [5:04:02<5:40:35,  2.16s/it]
 49%|████▉     | 9183/18627 [5:04:04<5:47:53,  2.21s/it]
 49%|████▉     | 9184/18627 [5:04:05<5:11:16,  1.98s/it]
 49%|████▉     | 9185/18627 [5:04:08<5:32:28,  2.11s/it]
 49%|████▉     | 9186/18627 [5:04:09<4:44:46,  1.81s/it]
 49%|████▉     | 9187/18627 [5:04:11<5:09:56,  1.97s/it]
 49%|████▉     | 9188/18627 [5:04:14<5:29:51,  2.10s/it]
 49%|████▉     | 9189/18627 [5:04:16<5:41:09,  2.17s/it]
 49%|████▉     | 9190/18627 [5:04:18<5:49:00,  2.22s/it]
                                                        
{'loss': 0.8139, 'grad_norm': 6.650750160217285, 'learning_rate': 2.675036807284345e-06, 'epoch': 0.49}
+
 49%|████▉     | 9190/18627 [5:04:18<5:49:00,  2.22s/it]
 49%|████▉     | 9191/18627 [5:04:20<5:13:00,  1.99s/it]
 49%|████▉     | 9192/18627 [5:04:22<5:28:34,  2.09s/it]
 49%|████▉     | 9193/18627 [5:04:25<5:41:26,  2.17s/it]
 49%|████▉     | 9194/18627 [5:04:27<5:49:32,  2.22s/it]
 49%|████▉     | 9195/18627 [5:04:29<5:55:06,  2.26s/it]
 49%|████▉     | 9196/18627 [5:04:31<5:12:04,  1.99s/it]
 49%|████▉     | 9197/18627 [5:04:33<5:28:53,  2.09s/it]
 49%|████▉     | 9198/18627 [5:04:35<5:40:13,  2.16s/it]
 49%|████▉     | 9199/18627 [5:04:38<5:48:02,  2.21s/it]
 49%|████▉     | 9200/18627 [5:04:40<5:53:40,  2.25s/it]
                                                        
{'loss': 1.002, 'grad_norm': 4.7902607917785645, 'learning_rate': 2.670700310852198e-06, 'epoch': 0.49}
+
 49%|████▉     | 9200/18627 [5:04:40<5:53:40,  2.25s/it]
 49%|████▉     | 9201/18627 [5:04:41<5:12:31,  1.99s/it]
 49%|████▉     | 9202/18627 [5:04:44<5:28:42,  2.09s/it]
 49%|████▉     | 9203/18627 [5:04:45<4:55:27,  1.88s/it]
 49%|████▉     | 9204/18627 [5:04:47<5:20:12,  2.04s/it]
 49%|████▉     | 9205/18627 [5:04:49<4:50:03,  1.85s/it]
 49%|████▉     | 9206/18627 [5:04:51<5:14:51,  2.01s/it]
 49%|████▉     | 9207/18627 [5:04:54<5:32:52,  2.12s/it]
 49%|████▉     | 9208/18627 [5:04:56<5:43:06,  2.19s/it]
 49%|████▉     | 9209/18627 [5:04:58<5:51:59,  2.24s/it]
 49%|████▉     | 9210/18627 [5:05:01<5:54:22,  2.26s/it]
                                                        
{'loss': 1.3122, 'grad_norm': 6.029917240142822, 'learning_rate': 2.6663632983439064e-06, 'epoch': 0.49}
+
 49%|████▉     | 9210/18627 [5:05:01<5:54:22,  2.26s/it]
 49%|████▉     | 9211/18627 [5:05:02<5:12:37,  1.99s/it]
 49%|████▉     | 9212/18627 [5:05:04<5:28:05,  2.09s/it]
 49%|████▉     | 9213/18627 [5:05:07<5:40:38,  2.17s/it]
 49%|████▉     | 9214/18627 [5:05:09<5:47:12,  2.21s/it]
 49%|████▉     | 9215/18627 [5:05:11<5:52:27,  2.25s/it]
 49%|████▉     | 9216/18627 [5:05:14<5:55:26,  2.27s/it]
 49%|████▉     | 9217/18627 [5:05:15<5:15:16,  2.01s/it]
 49%|████▉     | 9218/18627 [5:05:16<4:46:17,  1.83s/it]
 49%|████▉     | 9219/18627 [5:05:18<4:24:52,  1.69s/it]
 49%|████▉     | 9220/18627 [5:05:20<4:54:28,  1.88s/it]
                                                        
{'loss': 1.3819, 'grad_norm': 7.9476637840271, 'learning_rate': 2.662025782871506e-06, 'epoch': 0.49}
+
 49%|████▉     | 9220/18627 [5:05:20<4:54:28,  1.88s/it]
 50%|████▉     | 9221/18627 [5:05:22<5:16:21,  2.02s/it]
 50%|████▉     | 9222/18627 [5:05:24<4:44:26,  1.81s/it]
 50%|████▉     | 9223/18627 [5:05:25<4:26:08,  1.70s/it]
 50%|████▉     | 9224/18627 [5:05:28<4:57:15,  1.90s/it]
 50%|████▉     | 9225/18627 [5:05:30<5:16:37,  2.02s/it]
 50%|████▉     | 9226/18627 [5:05:32<5:33:31,  2.13s/it]
 50%|████▉     | 9227/18627 [5:05:35<5:43:25,  2.19s/it]
 50%|████▉     | 9228/18627 [5:05:37<5:51:20,  2.24s/it]
 50%|████▉     | 9229/18627 [5:05:38<5:14:13,  2.01s/it]
 50%|████▉     | 9230/18627 [5:05:41<5:29:38,  2.10s/it]
                                                        
{'loss': 1.1977, 'grad_norm': 7.198119640350342, 'learning_rate': 2.657687777548556e-06, 'epoch': 0.5}
+
 50%|████▉     | 9230/18627 [5:05:41<5:29:38,  2.10s/it]
 50%|████▉     | 9231/18627 [5:05:43<5:40:53,  2.18s/it]
 50%|████▉     | 9232/18627 [5:05:45<5:47:56,  2.22s/it]
 50%|████▉     | 9233/18627 [5:05:48<5:54:12,  2.26s/it]
 50%|████▉     | 9234/18627 [5:05:50<5:57:40,  2.28s/it]
 50%|████▉     | 9235/18627 [5:05:52<6:02:42,  2.32s/it]
 50%|████▉     | 9236/18627 [5:05:55<6:04:07,  2.33s/it]
 50%|████▉     | 9237/18627 [5:05:56<5:20:27,  2.05s/it]
 50%|████▉     | 9238/18627 [5:05:59<5:34:58,  2.14s/it]
 50%|████▉     | 9239/18627 [5:06:01<5:43:40,  2.20s/it]
 50%|████▉     | 9240/18627 [5:06:03<5:50:31,  2.24s/it]
                                                        
{'loss': 0.8688, 'grad_norm': 7.108974933624268, 'learning_rate': 2.6533492954900923e-06, 'epoch': 0.5}
+
 50%|████▉     | 9240/18627 [5:06:03<5:50:31,  2.24s/it]
 50%|████▉     | 9241/18627 [5:06:06<5:57:09,  2.28s/it]
 50%|████▉     | 9242/18627 [5:06:07<5:12:19,  2.00s/it]
 50%|████▉     | 9243/18627 [5:06:09<5:28:11,  2.10s/it]
 50%|████▉     | 9244/18627 [5:06:11<4:52:56,  1.87s/it]
 50%|████▉     | 9245/18627 [5:06:13<5:15:56,  2.02s/it]
 50%|████▉     | 9246/18627 [5:06:15<5:30:53,  2.12s/it]
 50%|████▉     | 9247/18627 [5:06:18<5:40:11,  2.18s/it]
 50%|████▉     | 9248/18627 [5:06:20<5:47:58,  2.23s/it]
 50%|████▉     | 9249/18627 [5:06:22<5:54:33,  2.27s/it]
 50%|████▉     | 9250/18627 [5:06:25<5:58:36,  2.29s/it]
                                                        
{'loss': 0.9967, 'grad_norm': 5.569089889526367, 'learning_rate': 2.6490103498125953e-06, 'epoch': 0.5}
+
 50%|████▉     | 9250/18627 [5:06:25<5:58:36,  2.29s/it]
 50%|████▉     | 9251/18627 [5:06:27<6:02:18,  2.32s/it]
 50%|████▉     | 9252/18627 [5:06:28<5:09:47,  1.98s/it]
 50%|████▉     | 9253/18627 [5:06:31<5:26:27,  2.09s/it]
 50%|████▉     | 9254/18627 [5:06:32<4:53:19,  1.88s/it]
 50%|████▉     | 9255/18627 [5:06:34<5:14:36,  2.01s/it]
 50%|████▉     | 9256/18627 [5:06:36<4:43:07,  1.81s/it]
 50%|████▉     | 9257/18627 [5:06:38<5:02:16,  1.94s/it]
 50%|████▉     | 9258/18627 [5:06:40<5:16:27,  2.03s/it]
 50%|████▉     | 9259/18627 [5:06:41<4:41:48,  1.80s/it]
 50%|████▉     | 9260/18627 [5:06:44<5:00:56,  1.93s/it]
                                                        
{'loss': 1.2604, 'grad_norm': 5.307986736297607, 'learning_rate': 2.6446709536339464e-06, 'epoch': 0.5}
+
 50%|████▉     | 9260/18627 [5:06:44<5:00:56,  1.93s/it]
 50%|████▉     | 9261/18627 [5:06:46<5:14:02,  2.01s/it]
 50%|████▉     | 9262/18627 [5:06:48<5:31:19,  2.12s/it]
 50%|████▉     | 9263/18627 [5:06:51<5:37:41,  2.16s/it]
 50%|████▉     | 9264/18627 [5:06:52<4:55:05,  1.89s/it]
 50%|████▉     | 9265/18627 [5:06:53<4:24:40,  1.70s/it]
 50%|████▉     | 9266/18627 [5:06:55<4:49:36,  1.86s/it]
 50%|████▉     | 9267/18627 [5:06:57<5:07:09,  1.97s/it]
 50%|████▉     | 9268/18627 [5:06:59<4:32:49,  1.75s/it]
 50%|████▉     | 9269/18627 [5:07:00<4:11:01,  1.61s/it]
 50%|████▉     | 9270/18627 [5:07:02<4:41:48,  1.81s/it]
                                                        
{'loss': 1.4797, 'grad_norm': 5.181243419647217, 'learning_rate': 2.640331120073387e-06, 'epoch': 0.5}
+
 50%|████▉     | 9270/18627 [5:07:02<4:41:48,  1.81s/it]
 50%|████▉     | 9271/18627 [5:07:05<5:03:57,  1.95s/it]
 50%|████▉     | 9272/18627 [5:07:07<5:18:30,  2.04s/it]
 50%|████▉     | 9273/18627 [5:07:09<5:28:28,  2.11s/it]
 50%|████▉     | 9274/18627 [5:07:11<5:36:40,  2.16s/it]
 50%|████▉     | 9275/18627 [5:07:14<5:41:38,  2.19s/it]
 50%|████▉     | 9276/18627 [5:07:16<5:45:02,  2.21s/it]
 50%|████▉     | 9277/18627 [5:07:18<5:48:08,  2.23s/it]
 50%|████▉     | 9278/18627 [5:07:20<5:50:12,  2.25s/it]
 50%|████▉     | 9279/18627 [5:07:23<5:52:07,  2.26s/it]
 50%|████▉     | 9280/18627 [5:07:25<5:52:53,  2.27s/it]
                                                        
{'loss': 0.6561, 'grad_norm': 6.483076572418213, 'learning_rate': 2.6359908622514836e-06, 'epoch': 0.5}
+
 50%|████▉     | 9280/18627 [5:07:25<5:52:53,  2.27s/it]
 50%|████▉     | 9281/18627 [5:07:27<5:52:28,  2.26s/it]
 50%|████▉     | 9282/18627 [5:07:30<5:52:35,  2.26s/it]
 50%|████▉     | 9283/18627 [5:07:32<5:53:40,  2.27s/it]
 50%|████▉     | 9284/18627 [5:07:33<5:09:37,  1.99s/it]
 50%|████▉     | 9285/18627 [5:07:35<5:24:27,  2.08s/it]
 50%|████▉     | 9286/18627 [5:07:38<5:34:59,  2.15s/it]
 50%|████▉     | 9287/18627 [5:07:40<5:41:35,  2.19s/it]
 50%|████▉     | 9288/18627 [5:07:42<5:45:27,  2.22s/it]
 50%|████▉     | 9289/18627 [5:07:45<5:47:26,  2.23s/it]
 50%|████▉     | 9290/18627 [5:07:47<5:50:18,  2.25s/it]
                                                        
{'loss': 0.7234, 'grad_norm': 5.926863193511963, 'learning_rate': 2.6316501932900836e-06, 'epoch': 0.5}
+
 50%|████▉     | 9290/18627 [5:07:47<5:50:18,  2.25s/it]
 50%|████▉     | 9291/18627 [5:07:48<5:07:05,  1.97s/it]
 50%|████▉     | 9292/18627 [5:07:51<5:24:08,  2.08s/it]
 50%|████▉     | 9293/18627 [5:07:52<4:59:25,  1.92s/it]
 50%|████▉     | 9294/18627 [5:07:54<5:17:20,  2.04s/it]
 50%|████▉     | 9295/18627 [5:07:57<5:27:28,  2.11s/it]
 50%|████▉     | 9296/18627 [5:07:59<5:37:49,  2.17s/it]
 50%|████▉     | 9297/18627 [5:08:00<4:46:01,  1.84s/it]
 50%|████▉     | 9298/18627 [5:08:01<4:17:54,  1.66s/it]
 50%|████▉     | 9299/18627 [5:08:04<4:47:10,  1.85s/it]
 50%|████▉     | 9300/18627 [5:08:06<5:08:35,  1.99s/it]
                                                        
{'loss': 1.1774, 'grad_norm': 4.85757303237915, 'learning_rate': 2.6273091263122787e-06, 'epoch': 0.5}
+
 50%|████▉     | 9300/18627 [5:08:06<5:08:35,  1.99s/it]
 50%|████▉     | 9301/18627 [5:08:08<5:23:50,  2.08s/it]
 50%|████▉     | 9302/18627 [5:08:10<4:48:49,  1.86s/it]
 50%|████▉     | 9303/18627 [5:08:11<4:24:14,  1.70s/it]
 50%|████▉     | 9304/18627 [5:08:13<4:52:18,  1.88s/it]
 50%|████▉     | 9305/18627 [5:08:15<5:12:20,  2.01s/it]
 50%|████▉     | 9306/18627 [5:08:17<4:41:10,  1.81s/it]
 50%|████▉     | 9307/18627 [5:08:19<5:03:21,  1.95s/it]
 50%|████▉     | 9308/18627 [5:08:21<5:21:01,  2.07s/it]
 50%|████▉     | 9309/18627 [5:08:24<5:30:29,  2.13s/it]
 50%|████▉     | 9310/18627 [5:08:25<4:53:13,  1.89s/it]
                                                        
{'loss': 1.5183, 'grad_norm': 14.247601509094238, 'learning_rate': 2.6229676744423617e-06, 'epoch': 0.5}
+
 50%|████▉     | 9310/18627 [5:08:25<4:53:13,  1.89s/it]
 50%|████▉     | 9311/18627 [5:08:27<5:15:04,  2.03s/it]
 50%|████▉     | 9312/18627 [5:08:30<5:27:46,  2.11s/it]
 50%|████▉     | 9313/18627 [5:08:31<4:53:25,  1.89s/it]
 50%|█████     | 9314/18627 [5:08:33<5:12:57,  2.02s/it]
 50%|█████     | 9315/18627 [5:08:36<5:25:46,  2.10s/it]
 50%|█████     | 9316/18627 [5:08:38<5:39:55,  2.19s/it]
 50%|█████     | 9317/18627 [5:08:39<5:00:05,  1.93s/it]
 50%|█████     | 9318/18627 [5:08:42<5:16:58,  2.04s/it]
 50%|█████     | 9319/18627 [5:08:44<5:28:53,  2.12s/it]
 50%|█████     | 9320/18627 [5:08:46<5:34:53,  2.16s/it]
                                                        
{'loss': 1.0759, 'grad_norm': 8.299243927001953, 'learning_rate': 2.6186258508057915e-06, 'epoch': 0.5}
+
 50%|█████     | 9320/18627 [5:08:46<5:34:53,  2.16s/it]
 50%|█████     | 9321/18627 [5:08:48<5:05:24,  1.97s/it]
 50%|█████     | 9322/18627 [5:08:49<4:20:39,  1.68s/it]
 50%|█████     | 9323/18627 [5:08:51<4:49:21,  1.87s/it]
 50%|█████     | 9324/18627 [5:08:53<5:08:16,  1.99s/it]
 50%|█████     | 9325/18627 [5:08:56<5:20:47,  2.07s/it]
 50%|█████     | 9326/18627 [5:08:57<4:46:15,  1.85s/it]
 50%|█████     | 9327/18627 [5:08:59<5:06:11,  1.98s/it]
 50%|█████     | 9328/18627 [5:09:02<5:22:29,  2.08s/it]
 50%|█████     | 9329/18627 [5:09:04<5:33:51,  2.15s/it]
 50%|█████     | 9330/18627 [5:09:05<4:54:08,  1.90s/it]
                                                        
{'loss': 1.3081, 'grad_norm': 13.628233909606934, 'learning_rate': 2.61428366852915e-06, 'epoch': 0.5}
+
 50%|█████     | 9330/18627 [5:09:05<4:54:08,  1.90s/it]
 50%|█████     | 9331/18627 [5:09:08<5:13:58,  2.03s/it]
 50%|█████     | 9332/18627 [5:09:10<5:26:23,  2.11s/it]
 50%|█████     | 9333/18627 [5:09:12<5:34:50,  2.16s/it]
 50%|█████     | 9334/18627 [5:09:13<4:53:32,  1.90s/it]
 50%|█████     | 9335/18627 [5:09:16<5:13:30,  2.02s/it]
 50%|█████     | 9336/18627 [5:09:17<4:42:15,  1.82s/it]
 50%|█████     | 9337/18627 [5:09:19<5:05:01,  1.97s/it]
 50%|█████     | 9338/18627 [5:09:22<5:21:14,  2.07s/it]
 50%|█████     | 9339/18627 [5:09:24<5:31:06,  2.14s/it]
 50%|█████     | 9340/18627 [5:09:26<5:38:46,  2.19s/it]
                                                        
{'loss': 1.0562, 'grad_norm': 6.781693458557129, 'learning_rate': 2.609941140740103e-06, 'epoch': 0.5}
+
 50%|█████     | 9340/18627 [5:09:26<5:38:46,  2.19s/it]
 50%|█████     | 9341/18627 [5:09:29<5:43:54,  2.22s/it]
 50%|█████     | 9342/18627 [5:09:31<5:47:07,  2.24s/it]
 50%|█████     | 9343/18627 [5:09:32<4:51:14,  1.88s/it]
 50%|█████     | 9344/18627 [5:09:34<5:12:22,  2.02s/it]
 50%|█████     | 9345/18627 [5:09:37<5:25:50,  2.11s/it]
 50%|█████     | 9346/18627 [5:09:38<4:49:56,  1.87s/it]
 50%|█████     | 9347/18627 [5:09:40<5:09:58,  2.00s/it]
 50%|█████     | 9348/18627 [5:09:43<5:24:33,  2.10s/it]
 50%|█████     | 9349/18627 [5:09:45<5:46:13,  2.24s/it]
 50%|█████     | 9350/18627 [5:09:47<5:48:20,  2.25s/it]
                                                        
{'loss': 0.8365, 'grad_norm': 6.7921552658081055, 'learning_rate': 2.6055982805673615e-06, 'epoch': 0.5}
+
 50%|█████     | 9350/18627 [5:09:47<5:48:20,  2.25s/it]
 50%|█████     | 9351/18627 [5:09:49<5:06:26,  1.98s/it]
 50%|█████     | 9352/18627 [5:09:51<5:20:45,  2.07s/it]
 50%|█████     | 9353/18627 [5:09:53<5:31:43,  2.15s/it]
 50%|█████     | 9354/18627 [5:09:56<5:42:49,  2.22s/it]
 50%|█████     | 9355/18627 [5:09:58<6:03:58,  2.36s/it]
 50%|█████     | 9356/18627 [5:10:01<6:00:24,  2.33s/it]
 50%|█████     | 9357/18627 [5:10:03<5:57:40,  2.32s/it]
 50%|█████     | 9358/18627 [5:10:05<5:56:57,  2.31s/it]
 50%|█████     | 9359/18627 [5:10:08<5:57:18,  2.31s/it]
 50%|█████     | 9360/18627 [5:10:10<5:56:56,  2.31s/it]
                                                        
{'loss': 0.8947, 'grad_norm': 5.792592525482178, 'learning_rate': 2.6012551011406406e-06, 'epoch': 0.5}
+
 50%|█████     | 9360/18627 [5:10:10<5:56:56,  2.31s/it]
 50%|█████     | 9361/18627 [5:10:12<5:54:54,  2.30s/it]
 50%|█████     | 9362/18627 [5:10:14<5:55:48,  2.30s/it]
 50%|█████     | 9363/18627 [5:10:17<5:55:33,  2.30s/it]
 50%|█████     | 9364/18627 [5:10:19<5:55:44,  2.30s/it]
 50%|█████     | 9365/18627 [5:10:21<5:55:10,  2.30s/it]
 50%|█████     | 9366/18627 [5:10:23<5:11:23,  2.02s/it]
 50%|█████     | 9367/18627 [5:10:25<5:22:43,  2.09s/it]
 50%|█████     | 9368/18627 [5:10:27<5:30:40,  2.14s/it]
 50%|█████     | 9369/18627 [5:10:29<5:36:37,  2.18s/it]
 50%|█████     | 9370/18627 [5:10:32<5:42:43,  2.22s/it]
                                                        
{'loss': 0.8611, 'grad_norm': 4.700439453125, 'learning_rate': 2.5969116155906216e-06, 'epoch': 0.5}
+
 50%|█████     | 9370/18627 [5:10:32<5:42:43,  2.22s/it]
 50%|█████     | 9371/18627 [5:10:33<5:03:26,  1.97s/it]
 50%|█████     | 9372/18627 [5:10:35<5:18:24,  2.06s/it]
 50%|█████     | 9373/18627 [5:10:38<5:29:06,  2.13s/it]
 50%|█████     | 9374/18627 [5:10:40<5:36:54,  2.18s/it]
 50%|█████     | 9375/18627 [5:10:42<5:42:07,  2.22s/it]
 50%|█████     | 9376/18627 [5:10:45<5:45:53,  2.24s/it]
 50%|█████     | 9377/18627 [5:10:47<5:47:45,  2.26s/it]
 50%|█████     | 9378/18627 [5:10:49<5:51:14,  2.28s/it]
 50%|█████     | 9379/18627 [5:10:52<5:52:03,  2.28s/it]
 50%|█████     | 9380/18627 [5:10:54<5:55:36,  2.31s/it]
                                                        
{'loss': 0.8875, 'grad_norm': 10.391050338745117, 'learning_rate': 2.5925678370489103e-06, 'epoch': 0.5}
+
 50%|█████     | 9380/18627 [5:10:54<5:55:36,  2.31s/it]
 50%|█████     | 9381/18627 [5:10:56<5:55:24,  2.31s/it]
 50%|█████     | 9382/18627 [5:10:59<5:55:32,  2.31s/it]
 50%|█████     | 9383/18627 [5:11:01<5:55:02,  2.30s/it]
 50%|█████     | 9384/18627 [5:11:03<5:54:21,  2.30s/it]
 50%|█████     | 9385/18627 [5:11:04<5:08:17,  2.00s/it]
 50%|█████     | 9386/18627 [5:11:07<5:23:02,  2.10s/it]
 50%|█████     | 9387/18627 [5:11:09<5:32:26,  2.16s/it]
 50%|█████     | 9388/18627 [5:11:11<5:38:47,  2.20s/it]
 50%|█████     | 9389/18627 [5:11:14<5:43:53,  2.23s/it]
 50%|█████     | 9390/18627 [5:11:16<5:47:57,  2.26s/it]
                                                        
{'loss': 0.8481, 'grad_norm': 6.474730968475342, 'learning_rate': 2.588223778647999e-06, 'epoch': 0.5}
+
 50%|█████     | 9390/18627 [5:11:16<5:47:57,  2.26s/it]
 50%|█████     | 9391/18627 [5:11:18<5:53:33,  2.30s/it]
 50%|█████     | 9392/18627 [5:11:21<5:53:01,  2.29s/it]
 50%|█████     | 9393/18627 [5:11:22<5:07:25,  2.00s/it]
 50%|█████     | 9394/18627 [5:11:25<5:34:42,  2.18s/it]
 50%|█████     | 9395/18627 [5:11:27<5:38:46,  2.20s/it]
 50%|█████     | 9396/18627 [5:11:28<4:59:25,  1.95s/it]
 50%|█████     | 9397/18627 [5:11:30<5:14:32,  2.04s/it]
 50%|█████     | 9398/18627 [5:11:33<5:26:03,  2.12s/it]
 50%|█████     | 9399/18627 [5:11:35<5:33:55,  2.17s/it]
 50%|█████     | 9400/18627 [5:11:37<5:39:26,  2.21s/it]
                                                        
{'loss': 1.0298, 'grad_norm': 11.23221492767334, 'learning_rate': 2.5838794535212262e-06, 'epoch': 0.5}
+
 50%|█████     | 9400/18627 [5:11:37<5:39:26,  2.21s/it]
 50%|█████     | 9401/18627 [5:11:40<5:43:15,  2.23s/it]
 50%|█████     | 9402/18627 [5:11:41<5:00:33,  1.95s/it]
 50%|█████     | 9403/18627 [5:11:43<5:17:24,  2.06s/it]
 50%|█████     | 9404/18627 [5:11:46<5:29:25,  2.14s/it]
 50%|█████     | 9405/18627 [5:11:47<4:39:19,  1.82s/it]
 50%|█████     | 9406/18627 [5:11:49<5:00:42,  1.96s/it]
 51%|█████     | 9407/18627 [5:11:50<4:17:00,  1.67s/it]
 51%|█████     | 9408/18627 [5:11:52<4:47:06,  1.87s/it]
 51%|█████     | 9409/18627 [5:11:55<5:07:32,  2.00s/it]
 51%|█████     | 9410/18627 [5:11:56<4:34:19,  1.79s/it]
                                                        
{'loss': 1.0182, 'grad_norm': 13.635296821594238, 'learning_rate': 2.579534874802736e-06, 'epoch': 0.51}
+
 51%|█████     | 9410/18627 [5:11:56<4:34:19,  1.79s/it]
 51%|█████     | 9411/18627 [5:11:57<4:12:45,  1.65s/it]
 51%|█████     | 9412/18627 [5:11:59<4:41:39,  1.83s/it]
 51%|█████     | 9413/18627 [5:12:02<5:02:00,  1.97s/it]
 51%|█████     | 9414/18627 [5:12:04<5:15:00,  2.05s/it]
 51%|█████     | 9415/18627 [5:12:06<5:26:21,  2.13s/it]
 51%|█████     | 9416/18627 [5:12:09<5:35:25,  2.18s/it]
 51%|█████     | 9417/18627 [5:12:11<5:39:40,  2.21s/it]
 51%|█████     | 9418/18627 [5:12:13<5:44:23,  2.24s/it]
 51%|█████     | 9419/18627 [5:12:15<5:46:58,  2.26s/it]
 51%|█████     | 9420/18627 [5:12:18<5:48:26,  2.27s/it]
                                                        
{'loss': 0.8952, 'grad_norm': 5.06599235534668, 'learning_rate': 2.57519005562744e-06, 'epoch': 0.51}
+
 51%|█████     | 9420/18627 [5:12:18<5:48:26,  2.27s/it]
 51%|█████     | 9421/18627 [5:12:20<5:47:40,  2.27s/it]
 51%|█████     | 9422/18627 [5:12:21<5:04:18,  1.98s/it]
 51%|█████     | 9423/18627 [5:12:24<5:16:47,  2.07s/it]
 51%|█████     | 9424/18627 [5:12:26<5:26:24,  2.13s/it]
 51%|█████     | 9425/18627 [5:12:28<5:32:59,  2.17s/it]
 51%|█████     | 9426/18627 [5:12:30<5:39:38,  2.21s/it]
 51%|█████     | 9427/18627 [5:12:33<5:42:13,  2.23s/it]
 51%|█████     | 9428/18627 [5:12:35<5:44:49,  2.25s/it]
 51%|█████     | 9429/18627 [5:12:36<5:01:10,  1.96s/it]
 51%|█████     | 9430/18627 [5:12:39<5:15:47,  2.06s/it]
                                                        
{'loss': 0.9294, 'grad_norm': 5.236781597137451, 'learning_rate': 2.570845009130976e-06, 'epoch': 0.51}
+
 51%|█████     | 9430/18627 [5:12:39<5:15:47,  2.06s/it]
 51%|█████     | 9431/18627 [5:12:41<5:26:45,  2.13s/it]
 51%|█████     | 9432/18627 [5:12:43<5:34:09,  2.18s/it]
 51%|█████     | 9433/18627 [5:12:45<4:56:01,  1.93s/it]
 51%|█████     | 9434/18627 [5:12:46<4:24:27,  1.73s/it]
 51%|█████     | 9435/18627 [5:12:48<4:51:57,  1.91s/it]
 51%|█████     | 9436/18627 [5:12:50<5:09:15,  2.02s/it]
 51%|█████     | 9437/18627 [5:12:53<5:21:41,  2.10s/it]
 51%|█████     | 9438/18627 [5:12:55<5:30:39,  2.16s/it]
 51%|█████     | 9439/18627 [5:12:57<5:37:43,  2.21s/it]
 51%|█████     | 9440/18627 [5:13:00<5:42:31,  2.24s/it]
                                                        
{'loss': 1.0637, 'grad_norm': 10.683716773986816, 'learning_rate': 2.5664997484496697e-06, 'epoch': 0.51}
+
 51%|█████     | 9440/18627 [5:13:00<5:42:31,  2.24s/it]
 51%|█████     | 9441/18627 [5:13:02<5:45:49,  2.26s/it]
 51%|█████     | 9442/18627 [5:13:04<5:52:48,  2.30s/it]
 51%|█████     | 9443/18627 [5:13:07<5:52:25,  2.30s/it]
 51%|█████     | 9444/18627 [5:13:09<5:54:47,  2.32s/it]
 51%|█████     | 9445/18627 [5:13:11<5:55:22,  2.32s/it]
 51%|█████     | 9446/18627 [5:13:14<5:53:29,  2.31s/it]
 51%|█████     | 9447/18627 [5:13:16<5:55:06,  2.32s/it]
 51%|█████     | 9448/18627 [5:13:18<5:55:04,  2.32s/it]
 51%|█████     | 9449/18627 [5:13:21<5:54:35,  2.32s/it]
 51%|█████     | 9450/18627 [5:13:23<5:53:13,  2.31s/it]
                                                        
{'loss': 0.6422, 'grad_norm': 4.415614604949951, 'learning_rate': 2.5621542867204945e-06, 'epoch': 0.51}
+
 51%|█████     | 9450/18627 [5:13:23<5:53:13,  2.31s/it]
 51%|█████     | 9451/18627 [5:13:25<5:52:32,  2.31s/it]
 51%|█████     | 9452/18627 [5:13:27<5:51:39,  2.30s/it]
 51%|█████     | 9453/18627 [5:13:30<5:50:53,  2.29s/it]
 51%|█████     | 9454/18627 [5:13:31<4:53:12,  1.92s/it]
 51%|█████     | 9455/18627 [5:13:33<5:10:23,  2.03s/it]
 51%|█████     | 9456/18627 [5:13:35<5:26:40,  2.14s/it]
 51%|█████     | 9457/18627 [5:13:38<5:33:26,  2.18s/it]
 51%|█████     | 9458/18627 [5:13:39<4:52:07,  1.91s/it]
 51%|█████     | 9459/18627 [5:13:41<5:08:51,  2.02s/it]
 51%|█████     | 9460/18627 [5:13:44<5:20:54,  2.10s/it]
                                                        
{'loss': 0.8282, 'grad_norm': 5.092986583709717, 'learning_rate': 2.5578086370810308e-06, 'epoch': 0.51}
+
 51%|█████     | 9460/18627 [5:13:44<5:20:54,  2.10s/it]
 51%|█████     | 9461/18627 [5:13:45<4:46:06,  1.87s/it]
 51%|█████     | 9462/18627 [5:13:46<4:21:28,  1.71s/it]
 51%|█████     | 9463/18627 [5:13:49<4:48:22,  1.89s/it]
 51%|█████     | 9464/18627 [5:13:50<4:20:43,  1.71s/it]
 51%|█████     | 9465/18627 [5:13:52<4:47:51,  1.89s/it]
 51%|█████     | 9466/18627 [5:13:54<5:06:14,  2.01s/it]
 51%|█████     | 9467/18627 [5:13:57<5:21:51,  2.11s/it]
 51%|█████     | 9468/18627 [5:13:59<5:30:11,  2.16s/it]
 51%|█████     | 9469/18627 [5:14:03<6:58:16,  2.74s/it]
 51%|█████     | 9470/18627 [5:14:05<6:38:12,  2.61s/it]
                                                        
{'loss': 1.2372, 'grad_norm': 5.710644721984863, 'learning_rate': 2.553462812669428e-06, 'epoch': 0.51}
+
 51%|█████     | 9470/18627 [5:14:05<6:38:12,  2.61s/it]
 51%|█████     | 9471/18627 [5:14:08<6:23:55,  2.52s/it]
 51%|█████     | 9472/18627 [5:14:10<6:14:02,  2.45s/it]
 51%|█████     | 9473/18627 [5:14:12<6:06:03,  2.40s/it]
 51%|█████     | 9474/18627 [5:14:15<6:08:11,  2.41s/it]
 51%|█████     | 9475/18627 [5:14:17<6:01:48,  2.37s/it]
 51%|█████     | 9476/18627 [5:14:18<5:11:20,  2.04s/it]
 51%|█████     | 9477/18627 [5:14:20<4:36:34,  1.81s/it]
 51%|█████     | 9478/18627 [5:14:22<4:57:46,  1.95s/it]
 51%|█████     | 9479/18627 [5:14:24<5:13:44,  2.06s/it]
 51%|█████     | 9480/18627 [5:14:27<5:24:58,  2.13s/it]
                                                        
{'loss': 1.0339, 'grad_norm': 6.715170383453369, 'learning_rate': 2.549116826624362e-06, 'epoch': 0.51}
+
 51%|█████     | 9480/18627 [5:14:27<5:24:58,  2.13s/it]
 51%|█████     | 9481/18627 [5:14:29<5:32:00,  2.18s/it]
 51%|█████     | 9482/18627 [5:14:31<5:36:45,  2.21s/it]
 51%|█████     | 9483/18627 [5:14:33<5:45:11,  2.26s/it]
 51%|█████     | 9484/18627 [5:14:36<5:48:32,  2.29s/it]
 51%|█████     | 9485/18627 [5:14:38<5:49:43,  2.30s/it]
 51%|█████     | 9486/18627 [5:14:39<5:06:28,  2.01s/it]
 51%|█████     | 9487/18627 [5:14:42<5:19:42,  2.10s/it]
 51%|█████     | 9488/18627 [5:14:44<5:27:40,  2.15s/it]
 51%|█████     | 9489/18627 [5:14:48<6:52:03,  2.71s/it]
 51%|█████     | 9490/18627 [5:14:50<6:34:06,  2.59s/it]
                                                        
{'loss': 0.8563, 'grad_norm': 9.063458442687988, 'learning_rate': 2.544770692085001e-06, 'epoch': 0.51}
+
 51%|█████     | 9490/18627 [5:14:50<6:34:06,  2.59s/it]
 51%|█████     | 9491/18627 [5:14:53<6:20:39,  2.50s/it]
 51%|█████     | 9492/18627 [5:14:54<5:25:37,  2.14s/it]
 51%|█████     | 9493/18627 [5:14:56<5:33:10,  2.19s/it]
 51%|█████     | 9494/18627 [5:14:59<5:38:09,  2.22s/it]
 51%|█████     | 9495/18627 [5:15:01<5:42:58,  2.25s/it]
 51%|█████     | 9496/18627 [5:15:03<5:44:12,  2.26s/it]
 51%|█████     | 9497/18627 [5:15:05<5:43:59,  2.26s/it]
 51%|█████     | 9498/18627 [5:15:07<5:01:24,  1.98s/it]
 51%|█████     | 9499/18627 [5:15:09<5:16:04,  2.08s/it]
 51%|█████     | 9500/18627 [5:15:11<5:25:49,  2.14s/it]
                                                        
{'loss': 1.0873, 'grad_norm': 6.3464860916137695, 'learning_rate': 2.5404244221909586e-06, 'epoch': 0.51}
+
 51%|█████     | 9500/18627 [5:15:11<5:25:49,  2.14s/it]
 51%|███��█     | 9501/18627 [5:15:13<4:48:01,  1.89s/it]
 51%|█████     | 9502/18627 [5:15:15<5:06:40,  2.02s/it]
 51%|█████     | 9503/18627 [5:15:17<5:19:05,  2.10s/it]
 51%|█████     | 9504/18627 [5:15:20<5:29:18,  2.17s/it]
 51%|█████     | 9505/18627 [5:15:22<5:35:25,  2.21s/it]
 51%|█████     | 9506/18627 [5:15:24<5:38:16,  2.23s/it]
 51%|█████     | 9507/18627 [5:15:26<5:42:33,  2.25s/it]
 51%|█████     | 9508/18627 [5:15:29<5:43:20,  2.26s/it]
 51%|█████     | 9509/18627 [5:15:31<5:44:24,  2.27s/it]
 51%|█████     | 9510/18627 [5:15:34<5:56:47,  2.35s/it]
                                                        
{'loss': 0.8599, 'grad_norm': 5.757299423217773, 'learning_rate': 2.536078030082259e-06, 'epoch': 0.51}
+
 51%|█████     | 9510/18627 [5:15:34<5:56:47,  2.35s/it]
 51%|█████     | 9511/18627 [5:15:36<5:55:06,  2.34s/it]
 51%|█████     | 9512/18627 [5:15:38<5:52:42,  2.32s/it]
 51%|█████     | 9513/18627 [5:15:40<5:50:00,  2.30s/it]
 51%|█████     | 9514/18627 [5:15:43<5:48:55,  2.30s/it]
 51%|█████     | 9515/18627 [5:15:45<5:47:13,  2.29s/it]
 51%|█████     | 9516/18627 [5:15:46<5:01:30,  1.99s/it]
 51%|█████     | 9517/18627 [5:15:49<5:17:17,  2.09s/it]
 51%|█████     | 9518/18627 [5:15:51<5:38:42,  2.23s/it]
 51%|█████     | 9519/18627 [5:15:53<5:42:10,  2.25s/it]
 51%|█████     | 9520/18627 [5:15:56<5:44:12,  2.27s/it]
                                                        
{'loss': 0.7956, 'grad_norm': 6.454016208648682, 'learning_rate': 2.531731528899296e-06, 'epoch': 0.51}
+
 51%|█████     | 9520/18627 [5:15:56<5:44:12,  2.27s/it]
 51%|█████     | 9521/18627 [5:15:58<5:53:41,  2.33s/it]
 51%|█████     | 9522/18627 [5:16:01<5:50:47,  2.31s/it]
 51%|█████     | 9523/18627 [5:16:03<5:48:54,  2.30s/it]
 51%|█████     | 9524/18627 [5:16:05<5:47:56,  2.29s/it]
 51%|█████     | 9525/18627 [5:16:07<5:48:56,  2.30s/it]
 51%|█████     | 9526/18627 [5:16:09<5:03:12,  2.00s/it]
 51%|█████     | 9527/18627 [5:16:10<4:43:16,  1.87s/it]
 51%|█████     | 9528/18627 [5:16:13<5:02:17,  1.99s/it]
 51%|█████     | 9529/18627 [5:16:15<5:18:50,  2.10s/it]
 51%|█████     | 9530/18627 [5:16:17<5:27:58,  2.16s/it]
                                                        
{'loss': 1.093, 'grad_norm': 6.02449369430542, 'learning_rate': 2.5273849317827938e-06, 'epoch': 0.51}
+
 51%|█████     | 9530/18627 [5:16:17<5:27:58,  2.16s/it]
 51%|█████     | 9531/18627 [5:16:19<5:31:29,  2.19s/it]
 51%|█████     | 9532/18627 [5:16:24<7:12:03,  2.85s/it]
 51%|█████     | 9533/18627 [5:16:26<6:47:34,  2.69s/it]
 51%|█████     | 9534/18627 [5:16:27<5:43:02,  2.26s/it]
 51%|█████     | 9535/18627 [5:16:30<5:44:05,  2.27s/it]
 51%|█████     | 9536/18627 [5:16:32<5:45:57,  2.28s/it]
 51%|█████     | 9537/18627 [5:16:34<5:46:06,  2.28s/it]
 51%|█████     | 9538/18627 [5:16:37<5:46:21,  2.29s/it]
 51%|█████     | 9539/18627 [5:16:38<5:03:35,  2.00s/it]
 51%|█████     | 9540/18627 [5:16:39<4:31:44,  1.79s/it]
                                                        
{'loss': 1.2814, 'grad_norm': 15.334836959838867, 'learning_rate': 2.5230382518737646e-06, 'epoch': 0.51}
+
 51%|█████     | 9540/18627 [5:16:39<4:31:44,  1.79s/it]
 51%|█████     | 9541/18627 [5:16:41<4:09:38,  1.65s/it]
 51%|█████     | 9542/18627 [5:16:42<3:54:36,  1.55s/it]
 51%|█████     | 9543/18627 [5:16:43<3:35:17,  1.42s/it]
 51%|█████     | 9544/18627 [5:16:45<3:41:25,  1.46s/it]
 51%|█████     | 9545/18627 [5:16:47<4:20:28,  1.72s/it]
 51%|█████     | 9546/18627 [5:16:49<4:46:07,  1.89s/it]
 51%|█████▏    | 9547/18627 [5:16:51<5:03:20,  2.00s/it]
 51%|█████▏    | 9548/18627 [5:16:54<5:17:14,  2.10s/it]
 51%|█████▏    | 9549/18627 [5:16:56<5:26:48,  2.16s/it]
 51%|█████▏    | 9550/18627 [5:16:58<5:33:07,  2.20s/it]
                                                        
{'loss': 1.1824, 'grad_norm': 5.372366428375244, 'learning_rate': 2.518691502313472e-06, 'epoch': 0.51}
+
 51%|█████▏    | 9550/18627 [5:16:58<5:33:07,  2.20s/it]
 51%|█████▏    | 9551/18627 [5:17:01<5:37:16,  2.23s/it]
 51%|█████▏    | 9552/18627 [5:17:03<5:38:52,  2.24s/it]
 51%|█████▏    | 9553/18627 [5:17:04<4:55:23,  1.95s/it]
 51%|█████▏    | 9554/18627 [5:17:06<5:10:19,  2.05s/it]
 51%|█████▏    | 9555/18627 [5:17:08<4:37:57,  1.84s/it]
 51%|█████▏    | 9556/18627 [5:17:10<5:01:35,  1.99s/it]
 51%|█████▏    | 9557/18627 [5:17:12<5:14:35,  2.08s/it]
 51%|█████▏    | 9558/18627 [5:17:15<5:22:02,  2.13s/it]
 51%|█████▏    | 9559/18627 [5:17:17<5:29:29,  2.18s/it]
 51%|█████▏    | 9560/18627 [5:17:19<5:35:03,  2.22s/it]
                                                        
{'loss': 1.1357, 'grad_norm': 7.319975852966309, 'learning_rate': 2.514344696243392e-06, 'epoch': 0.51}
+
 51%|█████▏    | 9560/18627 [5:17:19<5:35:03,  2.22s/it]
 51%|█████▏    | 9561/18627 [5:17:22<5:36:52,  2.23s/it]
 51%|█████▏    | 9562/18627 [5:17:24<5:39:43,  2.25s/it]
 51%|█████▏    | 9563/18627 [5:17:25<4:55:06,  1.95s/it]
 51%|█████▏    | 9564/18627 [5:17:27<5:09:33,  2.05s/it]
 51%|█████▏    | 9565/18627 [5:17:30<5:20:06,  2.12s/it]
 51%|█████▏    | 9566/18627 [5:17:32<5:26:45,  2.16s/it]
 51%|█████▏    | 9567/18627 [5:17:33<4:47:39,  1.91s/it]
 51%|█████▏    | 9568/18627 [5:17:36<5:04:45,  2.02s/it]
 51%|█████▏    | 9569/18627 [5:17:37<4:33:51,  1.81s/it]
 51%|█████▏    | 9570/18627 [5:17:39<4:55:31,  1.96s/it]
                                                        
{'loss': 1.2812, 'grad_norm': 6.582172870635986, 'learning_rate': 2.509997846805169e-06, 'epoch': 0.51}
+
 51%|█████▏    | 9570/18627 [5:17:39<4:55:31,  1.96s/it]
 51%|█████▏    | 9571/18627 [5:17:42<5:13:32,  2.08s/it]
 51%|█████▏    | 9572/18627 [5:17:44<5:28:51,  2.18s/it]
 51%|█████▏    | 9573/18627 [5:17:46<5:32:48,  2.21s/it]
 51%|█████▏    | 9574/18627 [5:17:47<4:51:26,  1.93s/it]
 51%|█████▏    | 9575/18627 [5:17:50<5:07:47,  2.04s/it]
 51%|█████▏    | 9576/18627 [5:17:52<5:20:15,  2.12s/it]
 51%|█████▏    | 9577/18627 [5:17:53<4:44:10,  1.88s/it]
 51%|█████▏    | 9578/18627 [5:17:56<5:02:24,  2.01s/it]
 51%|█████▏    | 9579/18627 [5:17:58<5:14:15,  2.08s/it]
 51%|█████▏    | 9580/18627 [5:18:00<5:24:38,  2.15s/it]
                                                        
{'loss': 0.9976, 'grad_norm': 4.069430351257324, 'learning_rate': 2.5056509671405784e-06, 'epoch': 0.51}
+
 51%|█████▏    | 9580/18627 [5:18:00<5:24:38,  2.15s/it]
 51%|█████▏    | 9581/18627 [5:18:03<5:32:08,  2.20s/it]
 51%|█████▏    | 9582/18627 [5:18:05<5:37:42,  2.24s/it]
 51%|█████▏    | 9583/18627 [5:18:06<4:56:52,  1.97s/it]
 51%|█████▏    | 9584/18627 [5:18:08<4:28:43,  1.78s/it]
 51%|█████▏    | 9585/18627 [5:18:10<4:51:53,  1.94s/it]
 51%|█████▏    | 9586/18627 [5:18:12<5:07:18,  2.04s/it]
 51%|█████▏    | 9587/18627 [5:18:14<5:17:37,  2.11s/it]
 51%|█████▏    | 9588/18627 [5:18:17<5:26:26,  2.17s/it]
 51%|█████▏    | 9589/18627 [5:18:19<5:31:47,  2.20s/it]
 51%|█████▏    | 9590/18627 [5:18:21<5:35:40,  2.23s/it]
                                                        
{'loss': 1.0681, 'grad_norm': 6.521454334259033, 'learning_rate': 2.5013040703914883e-06, 'epoch': 0.51}
+
 51%|█████▏    | 9590/18627 [5:18:21<5:35:40,  2.23s/it]
 51%|█████▏    | 9591/18627 [5:18:24<5:39:09,  2.25s/it]
 51%|█████▏    | 9592/18627 [5:18:26<5:42:20,  2.27s/it]
 52%|█████▏    | 9593/18627 [5:18:28<5:43:31,  2.28s/it]
 52%|█████▏    | 9594/18627 [5:18:30<4:57:45,  1.98s/it]
 52%|█████▏    | 9595/18627 [5:18:31<4:16:40,  1.71s/it]
 52%|█████▏    | 9596/18627 [5:18:33<4:45:36,  1.90s/it]
 52%|█████▏    | 9597/18627 [5:18:35<5:03:04,  2.01s/it]
 52%|█████▏    | 9598/18627 [5:18:38<5:15:43,  2.10s/it]
 52%|█████▏    | 9599/18627 [5:18:40<5:29:45,  2.19s/it]
 52%|█████▏    | 9600/18627 [5:18:42<5:37:35,  2.24s/it]
                                                        
{'loss': 0.8507, 'grad_norm': 9.823486328125, 'learning_rate': 2.4969571696998186e-06, 'epoch': 0.52}
+
 52%|█████▏    | 9600/18627 [5:18:42<5:37:35,  2.24s/it]
 52%|█████▏    | 9601/18627 [5:18:44<4:53:51,  1.95s/it]
 52%|█████▏    | 9602/18627 [5:18:46<5:06:41,  2.04s/it]
 52%|█████▏    | 9603/18627 [5:18:47<4:35:04,  1.83s/it]
 52%|█████▏    | 9604/18627 [5:18:51<6:15:56,  2.50s/it]
 52%|█████▏    | 9605/18627 [5:18:53<5:20:41,  2.13s/it]
 52%|█████▏    | 9606/18627 [5:18:54<4:44:46,  1.89s/it]
 52%|█████▏    | 9607/18627 [5:18:56<5:02:55,  2.02s/it]
 52%|█████▏    | 9608/18627 [5:18:57<4:31:36,  1.81s/it]
 52%|█████▏    | 9609/18627 [5:18:59<4:10:04,  1.66s/it]
 52%|█████▏    | 9610/18627 [5:19:01<4:40:41,  1.87s/it]
                                                        
{'loss': 1.7461, 'grad_norm': 4.95862340927124, 'learning_rate': 2.4926102782075e-06, 'epoch': 0.52}
+
 52%|█████▏    | 9610/18627 [5:19:01<4:40:41,  1.87s/it]
 52%|█████▏    | 9611/18627 [5:19:03<5:00:16,  2.00s/it]
 52%|█████▏    | 9612/18627 [5:19:06<5:16:32,  2.11s/it]
 52%|█████▏    | 9613/18627 [5:19:08<5:26:53,  2.18s/it]
 52%|█���███▏    | 9614/18627 [5:19:09<4:49:43,  1.93s/it]
 52%|█████▏    | 9615/18627 [5:19:12<5:09:05,  2.06s/it]
 52%|█████▏    | 9616/18627 [5:19:16<6:34:55,  2.63s/it]
 52%|█████▏    | 9617/18627 [5:19:18<6:20:35,  2.53s/it]
 52%|█████▏    | 9618/18627 [5:19:20<6:10:20,  2.47s/it]
 52%|█████▏    | 9619/18627 [5:19:23<6:02:11,  2.41s/it]
 52%|█████▏    | 9620/18627 [5:19:24<5:10:29,  2.07s/it]
                                                        
{'loss': 1.0893, 'grad_norm': 14.465418815612793, 'learning_rate': 2.488263409056437e-06, 'epoch': 0.52}
+
 52%|█████▏    | 9620/18627 [5:19:24<5:10:29,  2.07s/it]
 52%|█████▏    | 9621/18627 [5:19:26<5:20:30,  2.14s/it]
 52%|█████▏    | 9622/18627 [5:19:29<5:26:55,  2.18s/it]
 52%|█████▏    | 9623/18627 [5:19:31<5:31:57,  2.21s/it]
 52%|█████▏    | 9624/18627 [5:19:33<5:34:19,  2.23s/it]
 52%|█████▏    | 9625/18627 [5:19:34<4:46:56,  1.91s/it]
 52%|█████▏    | 9626/18627 [5:19:37<5:04:00,  2.03s/it]
 52%|█████▏    | 9627/18627 [5:19:39<5:16:30,  2.11s/it]
 52%|█████▏    | 9628/18627 [5:19:41<5:25:14,  2.17s/it]
 52%|█████▏    | 9629/18627 [5:19:43<5:29:26,  2.20s/it]
 52%|█████▏    | 9630/18627 [5:19:45<4:50:03,  1.93s/it]
                                                        
{'loss': 1.1536, 'grad_norm': 13.26216983795166, 'learning_rate': 2.4839165753884634e-06, 'epoch': 0.52}
+
 52%|█████▏    | 9630/18627 [5:19:45<4:50:03,  1.93s/it]
 52%|█████▏    | 9631/18627 [5:19:46<4:23:36,  1.76s/it]
 52%|█████▏    | 9632/18627 [5:19:48<4:49:32,  1.93s/it]
 52%|█████▏    | 9633/18627 [5:19:51<5:07:11,  2.05s/it]
 52%|█████▏    | 9634/18627 [5:19:53<5:17:31,  2.12s/it]
 52%|█████▏    | 9635/18627 [5:19:55<5:24:40,  2.17s/it]
 52%|█████▏    | 9636/18627 [5:19:57<4:45:42,  1.91s/it]
 52%|█████▏    | 9637/18627 [5:19:59<5:03:12,  2.02s/it]
 52%|█████▏    | 9638/18627 [5:20:01<5:16:17,  2.11s/it]
 52%|█████▏    | 9639/18627 [5:20:04<5:23:33,  2.16s/it]
 52%|█████▏    | 9640/18627 [5:20:06<5:30:09,  2.20s/it]
                                                        
{'loss': 0.9448, 'grad_norm': 6.435298442840576, 'learning_rate': 2.47956979034531e-06, 'epoch': 0.52}
+
 52%|█████▏    | 9640/18627 [5:20:06<5:30:09,  2.20s/it]
 52%|█████▏    | 9641/18627 [5:20:08<5:33:04,  2.22s/it]
 52%|█████▏    | 9642/18627 [5:20:10<5:35:15,  2.24s/it]
 52%|█████▏    | 9643/18627 [5:20:13<5:37:01,  2.25s/it]
 52%|█████▏    | 9644/18627 [5:20:15<5:37:57,  2.26s/it]
 52%|█████▏    | 9645/18627 [5:20:17<5:40:36,  2.28s/it]
 52%|█████▏    | 9646/18627 [5:20:20<5:40:17,  2.27s/it]
 52%|█████▏    | 9647/18627 [5:20:22<5:41:42,  2.28s/it]
 52%|█████▏    | 9648/18627 [5:20:24<5:42:30,  2.29s/it]
 52%|█████▏    | 9649/18627 [5:20:26<5:42:52,  2.29s/it]
 52%|█████▏    | 9650/18627 [5:20:28<4:59:41,  2.00s/it]
                                                        
{'loss': 0.926, 'grad_norm': 14.428033828735352, 'learning_rate': 2.475223067068557e-06, 'epoch': 0.52}
+
 52%|█████▏    | 9650/18627 [5:20:28<4:59:41,  2.00s/it]
 52%|█████▏    | 9651/18627 [5:20:30<5:12:32,  2.09s/it]
 52%|█████▏    | 9652/18627 [5:20:32<5:21:23,  2.15s/it]
 52%|█████▏    | 9653/18627 [5:20:35<5:26:15,  2.18s/it]
 52%|█████▏    | 9654/18627 [5:20:36<4:45:20,  1.91s/it]
 52%|█████▏    | 9655/18627 [5:20:38<5:01:25,  2.02s/it]
 52%|█████▏    | 9656/18627 [5:20:40<5:12:22,  2.09s/it]
 52%|█████▏    | 9657/18627 [5:20:42<4:37:05,  1.85s/it]
 52%|█████▏    | 9658/18627 [5:20:43<4:12:05,  1.69s/it]
 52%|█████▏    | 9659/18627 [5:20:45<4:39:43,  1.87s/it]
 52%|█████▏    | 9660/18627 [5:20:47<4:14:43,  1.70s/it]
                                                        
{'loss': 1.389, 'grad_norm': 17.441368103027344, 'learning_rate': 2.4708764186996002e-06, 'epoch': 0.52}
+
 52%|█████▏    | 9660/18627 [5:20:47<4:14:43,  1.70s/it]
 52%|█████▏    | 9661/18627 [5:20:48<3:53:26,  1.56s/it]
 52%|█████▏    | 9662/18627 [5:20:50<4:25:43,  1.78s/it]
 52%|█████▏    | 9663/18627 [5:20:51<4:04:18,  1.64s/it]
 52%|█████▏    | 9664/18627 [5:20:52<3:35:18,  1.44s/it]
 52%|█████▏    | 9665/18627 [5:20:54<3:29:15,  1.40s/it]
 52%|█████▏    | 9666/18627 [5:20:56<4:10:17,  1.68s/it]
 52%|█████▏    | 9667/18627 [5:20:58<4:39:18,  1.87s/it]
 52%|█████▏    | 9668/18627 [5:21:00<4:13:40,  1.70s/it]
 52%|█████▏    | 9669/18627 [5:21:02<4:39:21,  1.87s/it]
 52%|█████▏    | 9670/18627 [5:21:04<5:00:59,  2.02s/it]
                                                        
{'loss': 1.4342, 'grad_norm': 4.637203693389893, 'learning_rate': 2.4665298583796064e-06, 'epoch': 0.52}
+
 52%|█████▏    | 9670/18627 [5:21:04<5:00:59,  2.02s/it]
 52%|█████▏    | 9671/18627 [5:21:07<5:13:03,  2.10s/it]
 52%|█████▏    | 9672/18627 [5:21:09<5:21:53,  2.16s/it]
 52%|█████▏    | 9673/18627 [5:21:11<5:27:45,  2.20s/it]
 52%|█████▏    | 9674/18627 [5:21:13<5:29:34,  2.21s/it]
 52%|█████▏    | 9675/18627 [5:21:16<5:31:29,  2.22s/it]
 52%|█████▏    | 9676/18627 [5:21:18<5:32:13,  2.23s/it]
 52%|█████▏    | 9677/18627 [5:21:20<5:33:58,  2.24s/it]
 52%|█████▏    | 9678/18627 [5:21:22<5:35:38,  2.25s/it]
 52%|█████▏    | 9679/18627 [5:21:25<5:34:52,  2.25s/it]
 52%|█████▏    | 9680/18627 [5:21:26<4:52:47,  1.96s/it]
                                                        
{'loss': 0.8645, 'grad_norm': 15.724663734436035, 'learning_rate': 2.4621833992494804e-06, 'epoch': 0.52}
+
 52%|█████▏    | 9680/18627 [5:21:26<4:52:47,  1.96s/it]
 52%|█████▏    | 9681/18627 [5:21:28<5:07:45,  2.06s/it]
 52%|█████▏    | 9682/18627 [5:21:30<5:15:32,  2.12s/it]
 52%|█████▏    | 9683/18627 [5:21:33<5:23:11,  2.17s/it]
 52%|█████▏    | 9684/18627 [5:21:35<5:27:28,  2.20s/it]
 52%|█████▏    | 9685/18627 [5:21:37<5:29:28,  2.21s/it]
 52%|█████▏    | 9686/18627 [5:21:40<5:31:06,  2.22s/it]
 52%|█████▏    | 9687/18627 [5:21:42<5:34:41,  2.25s/it]
 52%|█████▏    | 9688/18627 [5:21:44<5:34:24,  2.24s/it]
 52%|█████▏    | 9689/18627 [5:21:45<4:54:26,  1.98s/it]
 52%|█████▏    | 9690/18627 [5:21:48<5:08:10,  2.07s/it]
                                                        
{'loss': 0.864, 'grad_norm': 9.625112533569336, 'learning_rate': 2.4578370544498164e-06, 'epoch': 0.52}
+
 52%|█████▏    | 9690/18627 [5:21:48<5:08:10,  2.07s/it]
 52%|█████▏    | 9691/18627 [5:21:50<5:16:44,  2.13s/it]
 52%|█████▏    | 9692/18627 [5:21:53<5:35:09,  2.25s/it]
 52%|█████▏    | 9693/18627 [5:21:54<4:54:10,  1.98s/it]
 52%|█████▏    | 9694/18627 [5:21:55<4:25:11,  1.78s/it]
 52%|█████▏    | 9695/18627 [5:21:56<4:04:31,  1.64s/it]
 52%|█████▏    | 9696/18627 [5:21:59<4:33:50,  1.84s/it]
 52%|█████▏    | 9697/18627 [5:22:01<4:53:26,  1.97s/it]
 52%|█████▏    | 9698/18627 [5:22:03<5:06:31,  2.06s/it]
 52%|█████▏    | 9699/18627 [5:22:06<5:17:42,  2.14s/it]
 52%|█████▏    | 9700/18627 [5:22:08<5:23:58,  2.18s/it]
                                                        
{'loss': 1.2308, 'grad_norm': 6.728978633880615, 'learning_rate': 2.4534908371208657e-06, 'epoch': 0.52}
+
 52%|█████▏    | 9700/18627 [5:22:08<5:23:58,  2.18s/it]
 52%|█████▏    | 9701/18627 [5:22:10<5:29:13,  2.21s/it]
 52%|█████▏    | 9702/18627 [5:22:12<5:31:38,  2.23s/it]
 52%|█████▏    | 9703/18627 [5:22:15<5:34:20,  2.25s/it]
 52%|█████▏    | 9704/18627 [5:22:17<5:33:57,  2.25s/it]
 52%|█████▏    | 9705/18627 [5:22:19<5:35:35,  2.26s/it]
 52%|█████▏    | 9706/18627 [5:22:22<5:37:04,  2.27s/it]
 52%|█████▏    | 9707/18627 [5:22:24<5:39:53,  2.29s/it]
 52%|█████▏    | 9708/18627 [5:22:25<4:56:49,  2.00s/it]
 52%|█████▏    | 9709/18627 [5:22:28<5:10:09,  2.09s/it]
 52%|█████▏    | 9710/18627 [5:22:30<5:17:59,  2.14s/it]
                                                        
{'loss': 0.8617, 'grad_norm': 6.635209560394287, 'learning_rate': 2.4491447604024924e-06, 'epoch': 0.52}
+
 52%|█████▏    | 9710/18627 [5:22:30<5:17:59,  2.14s/it]
 52%|█████▏    | 9711/18627 [5:22:32<5:30:59,  2.23s/it]
 52%|█████▏    | 9712/18627 [5:22:35<5:33:18,  2.24s/it]
 52%|█████▏    | 9713/18627 [5:22:37<5:35:50,  2.26s/it]
 52%|█████▏    | 9714/18627 [5:22:38<4:52:53,  1.97s/it]
 52%|█████▏    | 9715/18627 [5:22:40<5:08:13,  2.08s/it]
 52%|█████▏    | 9716/18627 [5:22:42<4:33:59,  1.84s/it]
 52%|█████▏    | 9717/18627 [5:22:44<4:54:29,  1.98s/it]
 52%|█████▏    | 9718/18627 [5:22:45<4:12:52,  1.70s/it]
 52%|█████▏    | 9719/18627 [5:22:46<3:54:35,  1.58s/it]
 52%|█████▏    | 9720/18627 [5:22:49<4:25:57,  1.79s/it]
                                                        
{'loss': 1.1633, 'grad_norm': 5.731926918029785, 'learning_rate': 2.4447988374341393e-06, 'epoch': 0.52}
+
 52%|█████▏    | 9720/18627 [5:22:49<4:25:57,  1.79s/it]
 52%|█████▏    | 9721/18627 [5:22:51<4:51:07,  1.96s/it]
 52%|█████▏    | 9722/18627 [5:22:53<5:07:01,  2.07s/it]
 52%|█████▏    | 9723/18627 [5:22:55<4:32:59,  1.84s/it]
 52%|█████▏    | 9724/18627 [5:22:57<4:53:15,  1.98s/it]
 52%|█████▏    | 9725/18627 [5:22:59<5:07:16,  2.07s/it]
 52%|█████▏    | 9726/18627 [5:23:02<5:17:17,  2.14s/it]
 52%|█████▏    | 9727/18627 [5:23:04<5:28:22,  2.21s/it]
 52%|█████▏    | 9728/18627 [5:23:06<5:30:29,  2.23s/it]
 52%|█████▏    | 9729/18627 [5:23:08<5:33:26,  2.25s/it]
 52%|█████▏    | 9730/18627 [5:23:10<4:51:51,  1.97s/it]
                                                        
{'loss': 1.0918, 'grad_norm': 14.250811576843262, 'learning_rate': 2.440453081354779e-06, 'epoch': 0.52}
+
 52%|█████▏    | 9730/18627 [5:23:10<4:51:51,  1.97s/it]
 52%|█████▏    | 9731/18627 [5:23:12<5:06:07,  2.06s/it]
 52%|█████▏    | 9732/18627 [5:23:14<5:15:06,  2.13s/it]
 52%|█████▏    | 9733/18627 [5:23:17<5:21:10,  2.17s/it]
 52%|█████▏    | 9734/18627 [5:23:19<5:26:51,  2.21s/it]
 52%|█████▏    | 9735/18627 [5:23:21<5:31:03,  2.23s/it]
 52%|█████▏    | 9736/18627 [5:23:23<4:50:14,  1.96s/it]
 52%|█████▏    | 9737/18627 [5:23:24<4:19:44,  1.75s/it]
 52%|█████▏    | 9738/18627 [5:23:26<4:43:13,  1.91s/it]
 52%|█████▏    | 9739/18627 [5:23:29<5:10:18,  2.09s/it]
 52%|█████▏    | 9740/18627 [5:23:31<5:18:35,  2.15s/it]
                                                        
{'loss': 0.9989, 'grad_norm': 6.089971542358398, 'learning_rate': 2.4361075053028844e-06, 'epoch': 0.52}
+
 52%|█████▏    | 9740/18627 [5:23:31<5:18:35,  2.15s/it]
 52%|█████▏    | 9741/18627 [5:23:32<4:42:05,  1.90s/it]
 52%|█████▏    | 9742/18627 [5:23:35<5:02:30,  2.04s/it]
 52%|█████▏    | 9743/18627 [5:23:37<5:14:24,  2.12s/it]
 52%|█████▏    | 9744/18627 [5:23:39<5:21:50,  2.17s/it]
 52%|█████▏    | 9745/18627 [5:23:42<5:27:58,  2.22s/it]
 52%|█████▏    | 9746/18627 [5:23:44<5:32:40,  2.25s/it]
 52%|█████▏    | 9747/18627 [5:23:46<5:35:29,  2.27s/it]
 52%|█████▏    | 9748/18627 [5:23:49<5:53:42,  2.39s/it]
 52%|█████▏    | 9749/18627 [5:23:51<5:50:01,  2.37s/it]
 52%|█████▏    | 9750/18627 [5:23:53<5:46:23,  2.34s/it]
                                                        
{'loss': 0.81, 'grad_norm': 5.811034679412842, 'learning_rate': 2.431762122416379e-06, 'epoch': 0.52}
+
 52%|█████▏    | 9750/18627 [5:23:53<5:46:23,  2.34s/it]
 52%|█████▏    | 9751/18627 [5:23:55<5:01:23,  2.04s/it]
 52%|█████▏    | 9752/18627 [5:23:57<5:15:09,  2.13s/it]
 52%|█████▏    | 9753/18627 [5:23:58<4:35:35,  1.86s/it]
 52%|█████▏    | 9754/18627 [5:24:00<4:05:32,  1.66s/it]
 52%|█████▏    | 9755/18627 [5:24:01<3:51:51,  1.57s/it]
 52%|█████▏    | 9756/18627 [5:24:03<4:21:55,  1.77s/it]
 52%|█████▏    | 9757/18627 [5:24:05<4:43:20,  1.92s/it]
 52%|█████▏    | 9758/18627 [5:24:08<5:00:25,  2.03s/it]
 52%|█████▏    | 9759/18627 [5:24:10<5:12:32,  2.11s/it]
 52%|█████▏    | 9760/18627 [5:24:12<5:21:01,  2.17s/it]
                                                        
{'loss': 1.3868, 'grad_norm': 7.727179050445557, 'learning_rate': 2.427416945832609e-06, 'epoch': 0.52}
+
 52%|█████▏    | 9760/18627 [5:24:12<5:21:01,  2.17s/it]
 52%|█████▏    | 9761/18627 [5:24:15<5:27:08,  2.21s/it]
 52%|█████▏    | 9762/18627 [5:24:17<5:32:19,  2.25s/it]
 52%|█████▏    | 9763/18627 [5:24:19<5:35:13,  2.27s/it]
 52%|█████▏    | 9764/18627 [5:24:22<5:39:07,  2.30s/it]
 52%|█████▏    | 9765/18627 [5:24:24<5:44:50,  2.33s/it]
 52%|█████▏    | 9766/18627 [5:24:26<5:43:37,  2.33s/it]
 52%|█████▏    | 9767/18627 [5:24:29<5:42:37,  2.32s/it]
 52%|█████▏    | 9768/18627 [5:24:31<5:42:17,  2.32s/it]
 52%|█████▏    | 9769/18627 [5:24:33<5:40:42,  2.31s/it]
 52%|█████▏    | 9770/18627 [5:24:36<5:41:44,  2.32s/it]
                                                        
{'loss': 0.5712, 'grad_norm': 7.159693717956543, 'learning_rate': 2.4230719886882907e-06, 'epoch': 0.52}
+
 52%|█████▏    | 9770/18627 [5:24:36<5:41:44,  2.32s/it]
 52%|█████▏    | 9771/18627 [5:24:38<5:50:15,  2.37s/it]
 52%|█████▏    | 9772/18627 [5:24:40<5:47:09,  2.35s/it]
 52%|█████▏    | 9773/18627 [5:24:43<5:43:38,  2.33s/it]
 52%|█████▏    | 9774/18627 [5:24:44<4:58:01,  2.02s/it]
 52%|█████▏    | 9775/18627 [5:24:46<5:10:34,  2.11s/it]
 52%|█████▏    | 9776/18627 [5:24:49<5:19:15,  2.16s/it]
 52%|█████▏    | 9777/18627 [5:24:51<5:25:22,  2.21s/it]
 52%|█████▏    | 9778/18627 [5:24:53<5:29:38,  2.24s/it]
 52%|█████▏    | 9779/18627 [5:24:56<5:34:05,  2.27s/it]
 53%|█████▎    | 9780/18627 [5:24:58<5:36:36,  2.28s/it]
                                                        
{'loss': 0.9225, 'grad_norm': 11.708717346191406, 'learning_rate': 2.418727264119481e-06, 'epoch': 0.53}
+
 53%|█████▎    | 9780/18627 [5:24:58<5:36:36,  2.28s/it]
 53%|█████▎    | 9781/18627 [5:24:59<4:51:20,  1.98s/it]
 53%|█████▎    | 9782/18627 [5:25:01<5:05:51,  2.07s/it]
 53%|█████▎    | 9783/18627 [5:25:04<5:16:04,  2.14s/it]
 53%|█████▎    | 9784/18627 [5:25:06<5:23:09,  2.19s/it]
 53%|█████▎    | 9785/18627 [5:25:08<5:24:51,  2.20s/it]
 53%|█████▎    | 9786/18627 [5:25:09<4:31:23,  1.84s/it]
 53%|█████▎    | 9787/18627 [5:25:11<4:08:09,  1.68s/it]
 53%|█████▎    | 9788/18627 [5:25:13<4:35:01,  1.87s/it]
 53%|█████▎    | 9789/18627 [5:25:14<4:10:01,  1.70s/it]
 53%|█████▎    | 9790/18627 [5:25:16<4:36:48,  1.88s/it]
                                                        
{'loss': 1.2411, 'grad_norm': 5.6728596687316895, 'learning_rate': 2.4143827852615297e-06, 'epoch': 0.53}
+
 53%|█████▎    | 9790/18627 [5:25:16<4:36:48,  1.88s/it]
 53%|█████▎    | 9791/18627 [5:25:19<4:53:39,  1.99s/it]
 53%|█████▎    | 9792/18627 [5:25:21<5:06:36,  2.08s/it]
 53%|█████▎    | 9793/18627 [5:25:23<5:14:47,  2.14s/it]
 53%|█████▎    | 9794/18627 [5:25:26<5:20:54,  2.18s/it]
 53%|█████▎    | 9795/18627 [5:25:28<5:26:35,  2.22s/it]
 53%|█████▎    | 9796/18627 [5:25:29<4:45:05,  1.94s/it]
 53%|█████▎    | 9797/18627 [5:25:31<5:00:33,  2.04s/it]
 53%|█████▎    | 9798/18627 [5:25:34<5:11:20,  2.12s/it]
 53%|█████▎    | 9799/18627 [5:25:36<5:18:23,  2.16s/it]
 53%|█████▎    | 9800/18627 [5:25:38<5:23:08,  2.20s/it]
                                                        
{'loss': 0.8303, 'grad_norm': 7.466031074523926, 'learning_rate': 2.4100385652490487e-06, 'epoch': 0.53}
+
 53%|█████▎    | 9800/18627 [5:25:38<5:23:08,  2.20s/it]
 53%|█████▎    | 9801/18627 [5:25:40<4:45:31,  1.94s/it]
 53%|█████▎    | 9802/18627 [5:25:42<5:02:26,  2.06s/it]
 53%|█████▎    | 9803/18627 [5:25:44<5:13:16,  2.13s/it]
 53%|█████▎    | 9804/18627 [5:25:47<5:21:04,  2.18s/it]
 53%|█████▎    | 9805/18627 [5:25:49<5:29:39,  2.24s/it]
 53%|█████▎    | 9806/18627 [5:25:51<5:32:48,  2.26s/it]
 53%|█████▎    | 9807/18627 [5:25:54<5:47:37,  2.36s/it]
 53%|█████▎    | 9808/18627 [5:25:56<5:43:00,  2.33s/it]
 53%|█████▎    | 9809/18627 [5:25:58<5:45:16,  2.35s/it]
 53%|█████▎    | 9810/18627 [5:26:01<5:58:16,  2.44s/it]
                                                        
{'loss': 0.8249, 'grad_norm': 8.255541801452637, 'learning_rate': 2.4056946172158633e-06, 'epoch': 0.53}
+
 53%|█████▎    | 9810/18627 [5:26:01<5:58:16,  2.44s/it]
 53%|█████▎    | 9811/18627 [5:26:03<5:53:06,  2.40s/it]
 53%|█████▎    | 9812/18627 [5:26:06<5:47:30,  2.37s/it]
 53%|█████▎    | 9813/18627 [5:26:07<5:02:55,  2.06s/it]
 53%|█████▎    | 9814/18627 [5:26:09<5:15:59,  2.15s/it]
 53%|█████▎    | 9815/18627 [5:26:12<5:22:36,  2.20s/it]
 53%|█████▎    | 9816/18627 [5:26:14<5:28:59,  2.24s/it]
 53%|█████▎    | 9817/18627 [5:26:16<5:31:07,  2.26s/it]
 53%|█████▎    | 9818/18627 [5:26:18<4:59:43,  2.04s/it]
 53%|█████▎    | 9819/18627 [5:26:19<4:27:47,  1.82s/it]
 53%|█████▎    | 9820/18627 [5:26:21<4:05:54,  1.68s/it]
                                                        
{'loss': 1.4965, 'grad_norm': 14.389863967895508, 'learning_rate': 2.401350954294979e-06, 'epoch': 0.53}
+
 53%|█████▎    | 9820/18627 [5:26:21<4:05:54,  1.68s/it]
 53%|█████▎    | 9821/18627 [5:26:23<4:32:35,  1.86s/it]
 53%|█████▎    | 9822/18627 [5:26:25<4:51:48,  1.99s/it]
 53%|█████▎    | 9823/18627 [5:26:26<4:21:40,  1.78s/it]
 53%|█████▎    | 9824/18627 [5:26:29<4:43:15,  1.93s/it]
 53%|█████▎    | 9825/18627 [5:26:31<5:00:04,  2.05s/it]
 53%|█████▎    | 9826/18627 [5:26:33<5:11:17,  2.12s/it]
 53%|█████▎    | 9827/18627 [5:26:36<5:17:51,  2.17s/it]
 53%|█████▎    | 9828/18627 [5:26:38<5:23:58,  2.21s/it]
 53%|█████▎    | 9829/18627 [5:26:40<5:26:26,  2.23s/it]
 53%|█████▎    | 9830/18627 [5:26:42<5:29:33,  2.25s/it]
                                                        
{'loss': 0.9726, 'grad_norm': 6.184101581573486, 'learning_rate': 2.3970075896185365e-06, 'epoch': 0.53}
+
 53%|█████▎    | 9830/18627 [5:26:42<5:29:33,  2.25s/it]
 53%|█████▎    | 9831/18627 [5:26:44<4:35:56,  1.88s/it]
 53%|█████▎    | 9832/18627 [5:26:46<4:54:48,  2.01s/it]
 53%|█████▎    | 9833/18627 [5:26:48<5:06:48,  2.09s/it]
 53%|█████▎    | 9834/18627 [5:26:50<5:14:57,  2.15s/it]
 53%|█████▎    | 9835/18627 [5:26:53<5:22:23,  2.20s/it]
 53%|█████▎    | 9836/18627 [5:26:54<4:43:41,  1.94s/it]
 53%|█████▎    | 9837/18627 [5:26:56<4:58:54,  2.04s/it]
 53%|█████▎    | 9838/18627 [5:26:59<5:08:34,  2.11s/it]
 53%|█████▎    | 9839/18627 [5:27:01<5:17:45,  2.17s/it]
 53%|█████▎    | 9840/18627 [5:27:03<5:22:25,  2.20s/it]
                                                        
{'loss': 0.8279, 'grad_norm': 5.858983516693115, 'learning_rate': 2.392664536317779e-06, 'epoch': 0.53}
+
 53%|█████▎    | 9840/18627 [5:27:03<5:22:25,  2.20s/it]
 53%|█████▎    | 9841/18627 [5:27:05<5:26:55,  2.23s/it]
 53%|█████▎    | 9842/18627 [5:27:08<5:32:44,  2.27s/it]
 53%|█████▎    | 9843/18627 [5:27:10<5:34:21,  2.28s/it]
 53%|█████▎    | 9844/18627 [5:27:12<5:34:03,  2.28s/it]
 53%|█████▎    | 9845/18627 [5:27:15<5:34:24,  2.28s/it]
 53%|█████▎    | 9846/18627 [5:27:17<5:32:59,  2.28s/it]
 53%|█████▎    | 9847/18627 [5:27:18<4:35:11,  1.88s/it]
 53%|█████▎    | 9848/18627 [5:27:20<4:52:21,  2.00s/it]
 53%|█████▎    | 9849/18627 [5:27:22<5:03:47,  2.08s/it]
 53%|█████▎    | 9850/18627 [5:27:25<5:11:40,  2.13s/it]
                                                        
{'loss': 0.6051, 'grad_norm': 5.418534755706787, 'learning_rate': 2.388321807523005e-06, 'epoch': 0.53}
+
 53%|█████▎    | 9850/18627 [5:27:25<5:11:40,  2.13s/it]
 53%|█████▎    | 9851/18627 [5:27:27<5:18:11,  2.18s/it]
 53%|█████▎    | 9852/18627 [5:27:29<5:20:46,  2.19s/it]
 53%|█████▎    | 9853/18627 [5:27:31<4:42:06,  1.93s/it]
 53%|█████▎    | 9854/18627 [5:27:33<4:58:25,  2.04s/it]
 53%|█████▎    | 9855/18627 [5:27:35<5:08:54,  2.11s/it]
 53%|█████▎    | 9856/18627 [5:27:37<5:16:08,  2.16s/it]
 53%|█████▎    | 9857/18627 [5:27:40<5:21:45,  2.20s/it]
 53%|█████▎    | 9858/18627 [5:27:42<5:22:39,  2.21s/it]
 53%|█████▎    | 9859/18627 [5:27:44<5:26:00,  2.23s/it]
 53%|█████▎    | 9860/18627 [5:27:46<4:51:41,  2.00s/it]
                                                        
{'loss': 1.1741, 'grad_norm': 15.70361614227295, 'learning_rate': 2.383979416363532e-06, 'epoch': 0.53}
+
 53%|█████▎    | 9860/18627 [5:27:46<4:51:41,  2.00s/it]
 53%|█████▎    | 9861/18627 [5:27:47<4:21:00,  1.79s/it]
 53%|█████▎    | 9862/18627 [5:27:49<4:42:35,  1.93s/it]
 53%|█████▎    | 9863/18627 [5:27:50<4:10:39,  1.72s/it]
 53%|█████▎    | 9864/18627 [5:27:52<3:52:25,  1.59s/it]
 53%|█████▎    | 9865/18627 [5:27:53<3:38:22,  1.50s/it]
 53%|█████▎    | 9866/18627 [5:27:55<4:12:36,  1.73s/it]
 53%|█████▎    | 9867/18627 [5:27:58<4:36:58,  1.90s/it]
 53%|█████▎    | 9868/18627 [5:28:00<4:53:13,  2.01s/it]
 53%|█████▎    | 9869/18627 [5:28:01<4:23:13,  1.80s/it]
 53%|█████▎    | 9870/18627 [5:28:03<4:43:28,  1.94s/it]
                                                        
{'loss': 1.6329, 'grad_norm': 8.53310775756836, 'learning_rate': 2.379637375967659e-06, 'epoch': 0.53}
+
 53%|█████▎    | 9870/18627 [5:28:03<4:43:28,  1.94s/it]
 53%|█████▎    | 9871/18627 [5:28:05<4:14:18,  1.74s/it]
 53%|█████▎    | 9872/18627 [5:28:07<4:38:13,  1.91s/it]
 53%|█████▎    | 9873/18627 [5:28:09<4:53:39,  2.01s/it]
 53%|█████▎    | 9874/18627 [5:28:12<5:04:03,  2.08s/it]
 53%|█████▎    | 9875/18627 [5:28:14<5:13:36,  2.15s/it]
 53%|█████▎    | 9876/18627 [5:28:16<5:18:30,  2.18s/it]
 53%|█████▎    | 9877/18627 [5:28:18<5:22:28,  2.21s/it]
 53%|█████▎    | 9878/18627 [5:28:20<4:45:26,  1.96s/it]
 53%|█████▎    | 9879/18627 [5:28:21<4:19:06,  1.78s/it]
 53%|█████▎    | 9880/18627 [5:28:23<4:40:55,  1.93s/it]
                                                        
{'loss': 1.2267, 'grad_norm': 5.116423606872559, 'learning_rate': 2.3752956994626224e-06, 'epoch': 0.53}
+
 53%|█████▎    | 9880/18627 [5:28:23<4:40:55,  1.93s/it]
 53%|█████▎    | 9881/18627 [5:28:26<4:55:35,  2.03s/it]
 53%|█████▎    | 9882/18627 [5:28:28<5:05:17,  2.09s/it]
 53%|█████▎    | 9883/18627 [5:28:30<5:12:37,  2.15s/it]
 53%|█████▎    | 9884/18627 [5:28:32<5:18:53,  2.19s/it]
 53%|█████▎    | 9885/18627 [5:28:35<5:22:40,  2.21s/it]
 53%|█████▎    | 9886/18627 [5:28:39<6:37:26,  2.73s/it]
 53%|█████▎    | 9887/18627 [5:28:40<5:36:19,  2.31s/it]
 53%|█████▎    | 9888/18627 [5:28:42<5:35:50,  2.31s/it]
 53%|█████▎    | 9889/18627 [5:28:45<5:35:43,  2.31s/it]
 53%|█████▎    | 9890/18627 [5:28:47<5:33:23,  2.29s/it]
                                                        
{'loss': 0.8655, 'grad_norm': 5.640902996063232, 'learning_rate': 2.3709543999745602e-06, 'epoch': 0.53}
+
 53%|█████▎    | 9890/18627 [5:28:47<5:33:23,  2.29s/it]
 53%|█████▎    | 9891/18627 [5:28:49<5:37:17,  2.32s/it]
 53%|█████▎    | 9892/18627 [5:28:51<5:34:51,  2.30s/it]
 53%|█████▎    | 9893/18627 [5:28:54<5:33:53,  2.29s/it]
 53%|█████▎    | 9894/18627 [5:28:56<5:33:28,  2.29s/it]
 53%|█████▎    | 9895/18627 [5:28:58<5:34:29,  2.30s/it]
 53%|█████▎    | 9896/18627 [5:29:01<5:34:23,  2.30s/it]
 53%|█████▎    | 9897/18627 [5:29:02<5:02:32,  2.08s/it]
 53%|█████▎    | 9898/18627 [5:29:04<4:29:27,  1.85s/it]
 53%|█████▎    | 9899/18627 [5:29:06<5:00:47,  2.07s/it]
 53%|█████▎    | 9900/18627 [5:29:08<5:10:16,  2.13s/it]
                                                        
{'loss': 0.9526, 'grad_norm': 7.251457691192627, 'learning_rate': 2.3666134906284675e-06, 'epoch': 0.53}
+
 53%|█████▎    | 9900/18627 [5:29:08<5:10:16,  2.13s/it]
 53%|█████▎    | 9901/18627 [5:29:11<5:16:54,  2.18s/it]
 53%|█████▎    | 9902/18627 [5:29:13<5:21:06,  2.21s/it]
 53%|█████▎    | 9903/18627 [5:29:14<4:42:44,  1.94s/it]
 53%|█████▎    | 9904/18627 [5:29:16<4:55:30,  2.03s/it]
 53%|█████▎    | 9905/18627 [5:29:19<5:05:53,  2.10s/it]
 53%|█████▎    | 9906/18627 [5:29:21<5:11:45,  2.14s/it]
 53%|█████▎    | 9907/18627 [5:29:22<4:34:32,  1.89s/it]
 53%|█████▎    | 9908/18627 [5:29:25<4:51:15,  2.00s/it]
 53%|█████▎    | 9909/18627 [5:29:27<5:00:52,  2.07s/it]
 53%|█████▎    | 9910/18627 [5:29:29<5:12:35,  2.15s/it]
                                                        
{'loss': 1.048, 'grad_norm': 6.20330810546875, 'learning_rate': 2.3622729845481642e-06, 'epoch': 0.53}
+
 53%|█████▎    | 9910/18627 [5:29:29<5:12:35,  2.15s/it]
 53%|█████▎    | 9911/18627 [5:29:30<4:34:10,  1.89s/it]
 53%|█████▎    | 9912/18627 [5:29:33<4:50:10,  2.00s/it]
 53%|█████▎    | 9913/18627 [5:29:34<4:20:33,  1.79s/it]
 53%|█████▎    | 9914/18627 [5:29:36<4:39:42,  1.93s/it]
 53%|█████▎    | 9915/18627 [5:29:39<4:56:33,  2.04s/it]
 53%|█████▎    | 9916/18627 [5:29:41<5:07:48,  2.12s/it]
 53%|█████▎    | 9917/18627 [5:29:43<5:14:41,  2.17s/it]
 53%|█████▎    | 9918/18627 [5:29:45<5:18:05,  2.19s/it]
 53%|█████▎    | 9919/18627 [5:29:48<5:21:56,  2.22s/it]
 53%|█████▎    | 9920/18627 [5:29:50<5:24:37,  2.24s/it]
                                                        
{'loss': 1.0077, 'grad_norm': 7.451358318328857, 'learning_rate': 2.357932894856247e-06, 'epoch': 0.53}
+
 53%|█████▎    | 9920/18627 [5:29:50<5:24:37,  2.24s/it]
 53%|█████▎    | 9921/18627 [5:29:52<5:27:19,  2.26s/it]
 53%|█████▎    | 9922/18627 [5:29:54<5:27:04,  2.25s/it]
 53%|█████▎    | 9923/18627 [5:29:57<5:28:02,  2.26s/it]
 53%|█████▎    | 9924/18627 [5:29:59<5:29:12,  2.27s/it]
 53%|█████▎    | 9925/18627 [5:30:01<5:29:38,  2.27s/it]
 53%|█████▎    | 9926/18627 [5:30:04<5:30:08,  2.28s/it]
 53%|█████▎    | 9927/18627 [5:30:05<4:47:43,  1.98s/it]
 53%|█████▎    | 9928/18627 [5:30:07<5:01:25,  2.08s/it]
 53%|█████▎    | 9929/18627 [5:30:09<5:08:43,  2.13s/it]
 53%|█████▎    | 9930/18627 [5:30:12<5:15:08,  2.17s/it]
                                                        
{'loss': 0.9203, 'grad_norm': 5.705124378204346, 'learning_rate': 2.3535932346740583e-06, 'epoch': 0.53}
+
 53%|█████▎    | 9930/18627 [5:30:12<5:15:08,  2.17s/it]
 53%|█████▎    | 9931/18627 [5:30:14<5:19:36,  2.21s/it]
 53%|█████▎    | 9932/18627 [5:30:16<5:31:11,  2.29s/it]
 53%|█████▎    | 9933/18627 [5:30:19<5:30:56,  2.28s/it]
 53%|█████▎    | 9934/18627 [5:30:20<4:32:28,  1.88s/it]
 53%|█████▎    | 9935/18627 [5:30:22<5:04:51,  2.10s/it]
 53%|█████▎    | 9936/18627 [5:30:25<5:14:25,  2.17s/it]
 53%|█████▎    | 9937/18627 [5:30:26<4:38:17,  1.92s/it]
 53%|█████▎    | 9938/18627 [5:30:28<4:51:37,  2.01s/it]
 53%|█████▎    | 9939/18627 [5:30:31<5:17:45,  2.19s/it]
 53%|█████▎    | 9940/18627 [5:30:33<5:21:28,  2.22s/it]
                                                        
{'loss': 0.9224, 'grad_norm': 5.153573513031006, 'learning_rate': 2.3492540171216357e-06, 'epoch': 0.53}
+
 53%|█████▎    | 9940/18627 [5:30:33<5:21:28,  2.22s/it]
 53%|█████▎    | 9941/18627 [5:30:36<5:31:42,  2.29s/it]
 53%|█████▎    | 9942/18627 [5:30:38<5:29:48,  2.28s/it]
 53%|█████▎    | 9943/18627 [5:30:40<5:27:27,  2.26s/it]
 53%|█████▎    | 9944/18627 [5:30:42<5:27:21,  2.26s/it]
 53%|█████▎    | 9945/18627 [5:30:43<4:31:00,  1.87s/it]
 53%|█████▎    | 9946/18627 [5:30:45<4:05:05,  1.69s/it]
 53%|█████▎    | 9947/18627 [5:30:47<4:31:01,  1.87s/it]
 53%|█████▎    | 9948/18627 [5:30:48<4:06:13,  1.70s/it]
 53%|█████▎    | 9949/18627 [5:30:50<4:31:07,  1.87s/it]
 53%|█████▎    | 9950/18627 [5:30:53<4:48:34,  2.00s/it]
                                                        
{'loss': 0.9732, 'grad_norm': 4.571021556854248, 'learning_rate': 2.3449152553176845e-06, 'epoch': 0.53}
+
 53%|█████▎    | 9950/18627 [5:30:53<4:48:34,  2.00s/it]
 53%|█████▎    | 9951/18627 [5:30:55<4:59:38,  2.07s/it]
 53%|█████▎    | 9952/18627 [5:30:57<5:07:41,  2.13s/it]
 53%|█████▎    | 9953/18627 [5:30:59<5:13:42,  2.17s/it]
 53%|█████▎    | 9954/18627 [5:31:02<5:17:11,  2.19s/it]
 53%|█████▎    | 9955/18627 [5:31:04<5:20:12,  2.22s/it]
 53%|█████▎    | 9956/18627 [5:31:06<5:23:38,  2.24s/it]
 53%|█████▎    | 9957/18627 [5:31:09<5:25:58,  2.26s/it]
 53%|█████▎    | 9958/18627 [5:31:11<5:25:51,  2.26s/it]
 53%|█████▎    | 9959/18627 [5:31:13<5:24:54,  2.25s/it]
 53%|█████▎    | 9960/18627 [5:31:15<5:25:45,  2.26s/it]
                                                        
{'loss': 0.7306, 'grad_norm': 7.386058330535889, 'learning_rate': 2.3405769623795277e-06, 'epoch': 0.53}
+
 53%|█████▎    | 9960/18627 [5:31:15<5:25:45,  2.26s/it]
 53%|█████▎    | 9961/18627 [5:31:18<5:27:15,  2.27s/it]
 53%|█████▎    | 9962/18627 [5:31:20<5:26:44,  2.26s/it]
 53%|█████▎    | 9963/18627 [5:31:22<5:28:25,  2.27s/it]
 53%|█████▎    | 9964/18627 [5:31:24<5:28:52,  2.28s/it]
 53%|█████▎    | 9965/18627 [5:31:27<5:29:09,  2.28s/it]
 54%|█████▎    | 9966/18627 [5:31:29<5:30:54,  2.29s/it]
 54%|█████▎    | 9967/18627 [5:31:31<5:31:16,  2.30s/it]
 54%|█████▎    | 9968/18627 [5:31:33<4:47:56,  2.00s/it]
 54%|█████▎    | 9969/18627 [5:31:35<4:59:43,  2.08s/it]
 54%|█████▎    | 9970/18627 [5:31:37<5:07:30,  2.13s/it]
                                                        
{'loss': 0.8398, 'grad_norm': 5.412652015686035, 'learning_rate': 2.3362391514230756e-06, 'epoch': 0.54}
+
 54%|█████▎    | 9970/18627 [5:31:37<5:07:30,  2.13s/it]
 54%|█████▎    | 9971/18627 [5:31:39<5:13:21,  2.17s/it]
 54%|█████▎    | 9972/18627 [5:31:42<5:17:14,  2.20s/it]
 54%|█████▎    | 9973/18627 [5:31:43<4:37:18,  1.92s/it]
 54%|█████▎    | 9974/18627 [5:31:45<4:53:14,  2.03s/it]
 54%|█████▎    | 9975/18627 [5:31:48<5:04:21,  2.11s/it]
 54%|█████▎    | 9976/18627 [5:31:50<5:12:02,  2.16s/it]
 54%|█████▎    | 9977/18627 [5:31:52<5:18:34,  2.21s/it]
 54%|█████▎    | 9978/18627 [5:31:53<4:38:36,  1.93s/it]
 54%|█████▎    | 9979/18627 [5:31:56<4:52:24,  2.03s/it]
 54%|█████▎    | 9980/18627 [5:31:58<5:01:52,  2.09s/it]
                                                        
{'loss': 1.0431, 'grad_norm': 5.483310222625732, 'learning_rate': 2.3319018355627764e-06, 'epoch': 0.54}
+
 54%|█████▎    | 9980/18627 [5:31:58<5:01:52,  2.09s/it]
 54%|█████▎    | 9981/18627 [5:31:59<4:27:00,  1.85s/it]
 54%|█████▎    | 9982/18627 [5:32:02<4:48:15,  2.00s/it]
 54%|█████▎    | 9983/18627 [5:32:03<4:18:21,  1.79s/it]
 54%|█████▎    | 9984/18627 [5:32:04<3:57:19,  1.65s/it]
 54%|█████▎    | 9985/18627 [5:32:06<4:23:47,  1.83s/it]
 54%|█████▎    | 9986/18627 [5:32:09<4:42:56,  1.96s/it]
 54%|█████▎    | 9987/18627 [5:32:11<4:57:05,  2.06s/it]
 54%|█████▎    | 9988/18627 [5:32:12<4:21:53,  1.82s/it]
 54%|█████▎    | 9989/18627 [5:32:15<4:43:15,  1.97s/it]
 54%|█████▎    | 9990/18627 [5:32:17<4:55:48,  2.05s/it]
                                                        
{'loss': 1.4188, 'grad_norm': 6.073135852813721, 'learning_rate': 2.3275650279115847e-06, 'epoch': 0.54}
+
 54%|█████▎    | 9990/18627 [5:32:17<4:55:48,  2.05s/it]
 54%|█████▎    | 9991/18627 [5:32:19<5:03:47,  2.11s/it]
 54%|█████▎    | 9992/18627 [5:32:21<5:11:04,  2.16s/it]
 54%|█████▎    | 9993/18627 [5:32:22<4:19:03,  1.80s/it]
 54%|█████▎    | 9994/18627 [5:32:25<4:41:24,  1.96s/it]
 54%|█████▎    | 9995/18627 [5:32:27<4:55:56,  2.06s/it]
 54%|█████▎    | 9996/18627 [5:32:29<5:08:56,  2.15s/it]
 54%|█████▎    | 9997/18627 [5:32:32<5:12:13,  2.17s/it]
 54%|█████▎    | 9998/18627 [5:32:33<4:33:08,  1.90s/it]
 54%|█████▎    | 9999/18627 [5:32:35<4:49:59,  2.02s/it]
 54%|█████▎    | 10000/18627 [5:32:36<4:05:18,  1.71s/it]
                                                         
{'loss': 0.8893, 'grad_norm': 6.084858417510986, 'learning_rate': 2.3232287415809164e-06, 'epoch': 0.54}
+
 54%|█████▎    | 10000/18627 [5:32:36<4:05:18,  1.71s/it]/home/zhengduo/workspace/LLaMA-Factory/src/qwen_vl/model/modeling_qwen2_5_vl.py:1979: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
+  with torch.cuda.amp.autocast(dtype=dtype):
+/home/zhengduo/miniconda3/envs/vgllm/lib/python3.10/site-packages/torch/utils/checkpoint.py:87: UserWarning: None of the inputs have requires_grad=True. Gradients will be None
+  warnings.warn(
+
 54%|█████▎    | 10001/18627 [5:33:20<34:42:50, 14.49s/it]
 54%|█████▎    | 10002/18627 [5:33:23<25:55:48, 10.82s/it]
 54%|█████▎    | 10003/18627 [5:33:25<19:45:58,  8.25s/it]
 54%|█████▎    | 10004/18627 [5:33:27<15:30:08,  6.47s/it]
 54%|█████▎    | 10005/18627 [5:33:29<11:45:47,  4.91s/it]
 54%|█████▎    | 10006/18627 [5:33:30<9:07:25,  3.81s/it] 
 54%|█████▎    | 10007/18627 [5:33:32<7:59:22,  3.34s/it]
 54%|█████▎    | 10008/18627 [5:33:34<7:11:22,  3.00s/it]
 54%|█████▎    | 10009/18627 [5:33:36<6:38:00,  2.77s/it]
 54%|█████▎    | 10010/18627 [5:33:39<6:14:40,  2.61s/it]
                                                         
{'loss': 1.0515, 'grad_norm': 8.443501472473145, 'learning_rate': 2.318892989680614e-06, 'epoch': 0.54}
+
 54%|█████▎    | 10010/18627 [5:33:39<6:14:40,  2.61s/it]
 54%|█████▎    | 10011/18627 [5:33:41<5:57:09,  2.49s/it]
 54%|█████▎    | 10012/18627 [5:33:43<5:44:27,  2.40s/it]
 54%|█████▍    | 10013/18627 [5:33:45<5:35:45,  2.34s/it]
 54%|█████▍    | 10014/18627 [5:33:47<5:29:46,  2.30s/it]
 54%|█████▍    | 10015/18627 [5:33:49<4:46:41,  2.00s/it]
 54%|█████▍    | 10016/18627 [5:33:51<4:56:13,  2.06s/it]
 54%|█████▍    | 10017/18627 [5:33:52<4:20:32,  1.82s/it]
 54%|█████▍    | 10018/18627 [5:33:54<4:37:17,  1.93s/it]
 54%|█████▍    | 10019/18627 [5:33:55<3:50:05,  1.60s/it]
 54%|█████▍    | 10020/18627 [5:33:57<3:37:01,  1.51s/it]
                                                         
{'loss': 1.2015, 'grad_norm': 13.431053161621094, 'learning_rate': 2.3145577853189015e-06, 'epoch': 0.54}
+
 54%|█████▍    | 10020/18627 [5:33:57<3:37:01,  1.51s/it]
 54%|█████▍    | 10021/18627 [5:33:59<4:08:28,  1.73s/it]
 54%|█████▍    | 10022/18627 [5:34:00<3:48:34,  1.59s/it]
 54%|█████▍    | 10023/18627 [5:34:02<4:17:19,  1.79s/it]
 54%|█████▍    | 10024/18627 [5:34:04<3:55:34,  1.64s/it]
 54%|█████▍    | 10025/18627 [5:34:06<4:21:24,  1.82s/it]
 54%|█████▍    | 10026/18627 [5:34:08<4:39:44,  1.95s/it]
 54%|█████▍    | 10027/18627 [5:34:10<4:51:33,  2.03s/it]
 54%|█████▍    | 10028/18627 [5:34:13<5:00:16,  2.10s/it]
 54%|█████▍    | 10029/18627 [5:34:15<5:05:24,  2.13s/it]
 54%|█████▍    | 10030/18627 [5:34:17<5:10:07,  2.16s/it]
                                                         
{'loss': 1.0493, 'grad_norm': 4.795833110809326, 'learning_rate': 2.31022314160235e-06, 'epoch': 0.54}
+
 54%|█████▍    | 10030/18627 [5:34:17<5:10:07,  2.16s/it]
 54%|█████▍    | 10031/18627 [5:34:19<5:14:05,  2.19s/it]
 54%|█████▍    | 10032/18627 [5:34:22<5:15:35,  2.20s/it]
 54%|█████▍    | 10033/18627 [5:34:24<5:17:16,  2.22s/it]
 54%|█████▍    | 10034/18627 [5:34:26<5:17:39,  2.22s/it]
 54%|█████▍    | 10035/18627 [5:34:27<4:37:30,  1.94s/it]
 54%|█████▍    | 10036/18627 [5:34:30<4:51:01,  2.03s/it]
 54%|█████▍    | 10037/18627 [5:34:32<4:58:39,  2.09s/it]
 54%|█████▍    | 10038/18627 [5:34:34<5:05:50,  2.14s/it]
 54%|█████▍    | 10039/18627 [5:34:36<5:12:21,  2.18s/it]
 54%|█████▍    | 10040/18627 [5:34:39<5:16:30,  2.21s/it]
                                                         
{'loss': 0.792, 'grad_norm': 8.388006210327148, 'learning_rate': 2.305889071635833e-06, 'epoch': 0.54}
+
 54%|█████▍    | 10040/18627 [5:34:39<5:16:30,  2.21s/it]
 54%|█████▍    | 10041/18627 [5:34:41<5:18:18,  2.22s/it]
 54%|█████▍    | 10042/18627 [5:34:43<5:21:00,  2.24s/it]
 54%|█████▍    | 10043/18627 [5:34:45<5:21:06,  2.24s/it]
 54%|█████▍    | 10044/18627 [5:34:48<5:23:09,  2.26s/it]
 54%|█████▍    | 10045/18627 [5:34:50<5:26:36,  2.28s/it]
 54%|█████▍    | 10046/18627 [5:34:52<5:25:02,  2.27s/it]
 54%|█████▍    | 10047/18627 [5:34:54<4:45:39,  2.00s/it]
 54%|█████▍    | 10048/18627 [5:34:56<4:59:35,  2.10s/it]
 54%|█████▍    | 10049/18627 [5:34:57<4:27:15,  1.87s/it]
 54%|█████▍    | 10050/18627 [5:35:00<4:50:34,  2.03s/it]
                                                         
{'loss': 0.9103, 'grad_norm': 7.47572660446167, 'learning_rate': 2.301555588522492e-06, 'epoch': 0.54}
+
 54%|█████▍    | 10050/18627 [5:35:00<4:50:34,  2.03s/it]
 54%|█████▍    | 10051/18627 [5:35:01<4:06:42,  1.73s/it]
 54%|█████▍    | 10052/18627 [5:35:03<4:31:07,  1.90s/it]
 54%|█████▍    | 10053/18627 [5:35:05<4:50:07,  2.03s/it]
 54%|█████▍    | 10054/18627 [5:35:08<5:05:11,  2.14s/it]
 54%|█████▍    | 10055/18627 [5:35:10<5:16:25,  2.21s/it]
 54%|█████▍    | 10056/18627 [5:35:12<4:41:39,  1.97s/it]
 54%|█████▍    | 10057/18627 [5:35:14<4:58:57,  2.09s/it]
 54%|█████▍    | 10058/18627 [5:35:16<5:12:05,  2.19s/it]
 54%|█████▍    | 10059/18627 [5:35:19<5:18:52,  2.23s/it]
 54%|█████▍    | 10060/18627 [5:35:21<5:27:46,  2.30s/it]
                                                         
{'loss': 0.8275, 'grad_norm': 6.261465549468994, 'learning_rate': 2.297222705363692e-06, 'epoch': 0.54}
+
 54%|█████▍    | 10060/18627 [5:35:21<5:27:46,  2.30s/it]
 54%|█████▍    | 10061/18627 [5:35:23<4:53:29,  2.06s/it]
 54%|█████▍    | 10062/18627 [5:35:24<4:19:20,  1.82s/it]
 54%|█████▍    | 10063/18627 [5:35:26<4:44:36,  1.99s/it]
 54%|█████▍    | 10064/18627 [5:35:29<5:01:09,  2.11s/it]
 54%|█████▍    | 10065/18627 [5:35:31<5:13:06,  2.19s/it]
 54%|█████▍    | 10066/18627 [5:35:33<5:23:01,  2.26s/it]
 54%|█████▍    | 10067/18627 [5:35:36<5:26:12,  2.29s/it]
 54%|█████▍    | 10068/18627 [5:35:38<5:29:59,  2.31s/it]
 54%|█████▍    | 10069/18627 [5:35:41<5:31:56,  2.33s/it]
 54%|█████▍    | 10070/18627 [5:35:42<4:42:04,  1.98s/it]
                                                         
{'loss': 0.9062, 'grad_norm': 6.299708366394043, 'learning_rate': 2.292890435258986e-06, 'epoch': 0.54}
+
 54%|█████▍    | 10070/18627 [5:35:42<4:42:04,  1.98s/it]
 54%|█████▍    | 10071/18627 [5:35:44<4:58:54,  2.10s/it]
 54%|█████▍    | 10072/18627 [5:35:46<5:11:47,  2.19s/it]
 54%|█████▍    | 10073/18627 [5:35:48<4:38:39,  1.95s/it]
 54%|█████▍    | 10074/18627 [5:35:50<4:56:55,  2.08s/it]
 54%|█████▍    | 10075/18627 [5:35:52<4:28:55,  1.89s/it]
 54%|█████▍    | 10076/18627 [5:35:54<4:47:49,  2.02s/it]
 54%|█████▍    | 10077/18627 [5:35:55<4:21:40,  1.84s/it]
 54%|█████▍    | 10078/18627 [5:35:57<4:06:51,  1.73s/it]
 54%|█████▍    | 10079/18627 [5:35:58<3:54:24,  1.65s/it]
 54%|█████▍    | 10080/18627 [5:36:01<4:26:16,  1.87s/it]
                                                         
{'loss': 1.7082, 'grad_norm': 5.449352264404297, 'learning_rate': 2.2885587913060717e-06, 'epoch': 0.54}
+
 54%|█████▍    | 10080/18627 [5:36:01<4:26:16,  1.87s/it]
 54%|█████▍    | 10081/18627 [5:36:02<4:04:18,  1.72s/it]
 54%|█████▍    | 10082/18627 [5:36:04<4:28:55,  1.89s/it]
 54%|█████▍    | 10083/18627 [5:36:05<3:49:16,  1.61s/it]
 54%|█████▍    | 10084/18627 [5:36:07<3:36:01,  1.52s/it]
 54%|█████▍    | 10085/18627 [5:36:09<4:05:52,  1.73s/it]
 54%|█████▍    | 10086/18627 [5:36:11<4:26:59,  1.88s/it]
 54%|█████▍    | 10087/18627 [5:36:13<4:41:27,  1.98s/it]
 54%|█████▍    | 10088/18627 [5:36:16<4:53:20,  2.06s/it]
 54%|█████▍    | 10089/18627 [5:36:18<5:01:58,  2.12s/it]
 54%|█████▍    | 10090/18627 [5:36:19<4:27:11,  1.88s/it]
                                                         
{'loss': 1.2874, 'grad_norm': 15.12488842010498, 'learning_rate': 2.2842277866007563e-06, 'epoch': 0.54}
+
 54%|█████▍    | 10090/18627 [5:36:19<4:27:11,  1.88s/it]
 54%|█████▍    | 10091/18627 [5:36:21<4:42:33,  1.99s/it]
 54%|█████▍    | 10092/18627 [5:36:24<4:54:10,  2.07s/it]
 54%|█████▍    | 10093/18627 [5:36:26<5:01:33,  2.12s/it]
 54%|█████▍    | 10094/18627 [5:36:28<5:06:33,  2.16s/it]
 54%|█████▍    | 10095/18627 [5:36:30<5:11:10,  2.19s/it]
 54%|█████▍    | 10096/18627 [5:36:33<5:15:08,  2.22s/it]
 54%|█████▍    | 10097/18627 [5:36:35<5:17:21,  2.23s/it]
 54%|█████▍    | 10098/18627 [5:36:37<5:19:28,  2.25s/it]
 54%|█████▍    | 10099/18627 [5:36:39<4:39:27,  1.97s/it]
 54%|█████▍    | 10100/18627 [5:36:41<4:52:38,  2.06s/it]
                                                         
{'loss': 0.8503, 'grad_norm': 5.560448169708252, 'learning_rate': 2.279897434236912e-06, 'epoch': 0.54}
+
 54%|█████▍    | 10100/18627 [5:36:41<4:52:38,  2.06s/it]
 54%|█████▍    | 10101/18627 [5:36:42<4:18:48,  1.82s/it]
 54%|█████▍    | 10102/18627 [5:36:44<4:38:28,  1.96s/it]
 54%|█████▍    | 10103/18627 [5:36:47<4:53:24,  2.07s/it]
 54%|█████▍    | 10104/18627 [5:36:49<5:04:23,  2.14s/it]
 54%|█████▍    | 10105/18627 [5:36:51<5:11:13,  2.19s/it]
 54%|█████▍    | 10106/18627 [5:36:54<5:15:30,  2.22s/it]
 54%|█████▍    | 10107/18627 [5:36:56<5:21:01,  2.26s/it]
 54%|█████��    | 10108/18627 [5:36:58<5:22:50,  2.27s/it]
 54%|█████▍    | 10109/18627 [5:37:01<5:23:09,  2.28s/it]
 54%|█████▍    | 10110/18627 [5:37:03<5:23:37,  2.28s/it]
                                                         
{'loss': 0.892, 'grad_norm': 6.321796894073486, 'learning_rate': 2.2755677473064394e-06, 'epoch': 0.54}
+
 54%|█████▍    | 10110/18627 [5:37:03<5:23:37,  2.28s/it]
 54%|█████▍    | 10111/18627 [5:37:05<5:25:46,  2.30s/it]
 54%|█████▍    | 10112/18627 [5:37:07<5:25:26,  2.29s/it]
 54%|█████▍    | 10113/18627 [5:37:10<5:29:54,  2.32s/it]
 54%|█████▍    | 10114/18627 [5:37:12<5:27:55,  2.31s/it]
 54%|█████▍    | 10115/18627 [5:37:13<4:46:32,  2.02s/it]
 54%|█████▍    | 10116/18627 [5:37:16<4:57:10,  2.10s/it]
 54%|█████▍    | 10117/18627 [5:37:18<5:05:28,  2.15s/it]
 54%|█████▍    | 10118/18627 [5:37:20<5:09:52,  2.19s/it]
 54%|█████▍    | 10119/18627 [5:37:21<4:29:24,  1.90s/it]
 54%|█████▍    | 10120/18627 [5:37:23<4:04:43,  1.73s/it]
                                                         
{'loss': 1.1939, 'grad_norm': 12.117864608764648, 'learning_rate': 2.271238738899229e-06, 'epoch': 0.54}
+
 54%|█████▍    | 10120/18627 [5:37:23<4:04:43,  1.73s/it]
 54%|█████▍    | 10121/18627 [5:37:25<4:30:37,  1.91s/it]
 54%|█████▍    | 10122/18627 [5:37:27<4:45:01,  2.01s/it]
 54%|█████▍    | 10123/18627 [5:37:29<4:17:40,  1.82s/it]
 54%|█████▍    | 10124/18627 [5:37:31<4:38:27,  1.96s/it]
 54%|█████▍    | 10125/18627 [5:37:33<4:53:53,  2.07s/it]
 54%|█████▍    | 10126/18627 [5:37:35<4:23:07,  1.86s/it]
 54%|█████▍    | 10127/18627 [5:37:37<4:42:58,  2.00s/it]
 54%|█████▍    | 10128/18627 [5:37:39<4:55:30,  2.09s/it]
 54%|█████▍    | 10129/18627 [5:37:42<5:04:42,  2.15s/it]
 54%|█████▍    | 10130/18627 [5:37:43<4:24:59,  1.87s/it]
                                                         
{'loss': 1.0969, 'grad_norm': 13.869786262512207, 'learning_rate': 2.266910422103117e-06, 'epoch': 0.54}
+
 54%|█████▍    | 10130/18627 [5:37:43<4:24:59,  1.87s/it]
 54%|█████▍    | 10131/18627 [5:37:45<4:42:16,  1.99s/it]
 54%|█████▍    | 10132/18627 [5:37:47<4:56:05,  2.09s/it]
 54%|█████▍    | 10133/18627 [5:37:49<4:25:19,  1.87s/it]
 54%|█████▍    | 10134/18627 [5:37:50<4:03:20,  1.72s/it]
 54%|█████▍    | 10135/18627 [5:37:52<3:46:16,  1.60s/it]
 54%|█████▍    | 10136/18627 [5:37:53<3:35:37,  1.52s/it]
 54%|█████▍    | 10137/18627 [5:37:55<4:08:24,  1.76s/it]
 54%|█████▍    | 10138/18627 [5:37:57<4:31:49,  1.92s/it]
 54%|█████▍    | 10139/18627 [5:38:00<4:48:25,  2.04s/it]
 54%|█████▍    | 10140/18627 [5:38:01<4:17:54,  1.82s/it]
                                                         
{'loss': 1.5467, 'grad_norm': 14.814237594604492, 'learning_rate': 2.2625828100038513e-06, 'epoch': 0.54}
+
 54%|█████▍    | 10140/18627 [5:38:01<4:17:54,  1.82s/it]
 54%|█████▍    | 10141/18627 [5:38:02<3:43:47,  1.58s/it]
 54%|█████▍    | 10142/18627 [5:38:04<4:13:35,  1.79s/it]
 54%|█████▍    | 10143/18627 [5:38:06<3:53:43,  1.65s/it]
 54%|█████▍    | 10144/18627 [5:38:08<4:19:12,  1.83s/it]
 54%|█████▍    | 10145/18627 [5:38:10<4:38:42,  1.97s/it]
 54%|█████▍    | 10146/18627 [5:38:12<4:11:07,  1.78s/it]
 54%|█████▍    | 10147/18627 [5:38:14<4:31:30,  1.92s/it]
 54%|█████▍    | 10148/18627 [5:38:16<4:48:26,  2.04s/it]
 54%|█████▍    | 10149/18627 [5:38:18<4:58:12,  2.11s/it]
 54%|█████▍    | 10150/18627 [5:38:21<5:04:47,  2.16s/it]
                                                         
{'loss': 1.1127, 'grad_norm': 6.513592720031738, 'learning_rate': 2.2582559156850467e-06, 'epoch': 0.54}
+
 54%|█████▍    | 10150/18627 [5:38:21<5:04:47,  2.16s/it]
 54%|█████▍    | 10151/18627 [5:38:23<5:11:23,  2.20s/it]
 55%|█████▍    | 10152/18627 [5:38:25<5:14:29,  2.23s/it]
 55%|█████▍    | 10153/18627 [5:38:28<5:17:45,  2.25s/it]
 55%|█████▍    | 10154/18627 [5:38:29<4:38:45,  1.97s/it]
 55%|█████▍    | 10155/18627 [5:38:31<4:53:19,  2.08s/it]
 55%|█████▍    | 10156/18627 [5:38:34<5:03:22,  2.15s/it]
 55%|█████▍    | 10157/18627 [5:38:35<4:28:21,  1.90s/it]
 55%|█████▍    | 10158/18627 [5:38:37<4:44:12,  2.01s/it]
 55%|█████▍    | 10159/18627 [5:38:39<4:55:36,  2.09s/it]
 55%|█████▍    | 10160/18627 [5:38:42<5:03:30,  2.15s/it]
                                                         
{'loss': 1.0628, 'grad_norm': 7.026455879211426, 'learning_rate': 2.2539297522281514e-06, 'epoch': 0.55}
+
 55%|█████▍    | 10160/18627 [5:38:42<5:03:30,  2.15s/it]
 55%|█████��    | 10161/18627 [5:38:44<5:08:48,  2.19s/it]
 55%|█████▍    | 10162/18627 [5:38:46<5:11:25,  2.21s/it]
 55%|█████▍    | 10163/18627 [5:38:49<5:15:11,  2.23s/it]
 55%|█████▍    | 10164/18627 [5:38:51<5:17:12,  2.25s/it]
 55%|█████▍    | 10165/18627 [5:38:53<5:18:21,  2.26s/it]
 55%|█████▍    | 10166/18627 [5:38:55<5:19:18,  2.26s/it]
 55%|█████▍    | 10167/18627 [5:38:58<5:19:13,  2.26s/it]
 55%|█████▍    | 10168/18627 [5:38:59<4:41:15,  1.99s/it]
 55%|█████▍    | 10169/18627 [5:39:00<4:13:29,  1.80s/it]
 55%|█████▍    | 10170/18627 [5:39:02<3:54:15,  1.66s/it]
                                                         
{'loss': 1.2139, 'grad_norm': 13.22324275970459, 'learning_rate': 2.2496043327124005e-06, 'epoch': 0.55}
+
 55%|█████▍    | 10170/18627 [5:39:02<3:54:15,  1.66s/it]
 55%|█████▍    | 10171/18627 [5:39:04<4:20:17,  1.85s/it]
 55%|█████▍    | 10172/18627 [5:39:06<4:39:16,  1.98s/it]
 55%|█████▍    | 10173/18627 [5:39:09<4:51:40,  2.07s/it]
 55%|█████▍    | 10174/18627 [5:39:11<5:01:38,  2.14s/it]
 55%|█████▍    | 10175/18627 [5:39:13<5:08:47,  2.19s/it]
 55%|█████▍    | 10176/18627 [5:39:16<5:14:50,  2.24s/it]
 55%|█████▍    | 10177/18627 [5:39:18<5:16:25,  2.25s/it]
 55%|█████▍    | 10178/18627 [5:39:20<5:20:23,  2.28s/it]
 55%|█████▍    | 10179/18627 [5:39:22<5:19:46,  2.27s/it]
 55%|█████▍    | 10180/18627 [5:39:25<5:18:34,  2.26s/it]
                                                         
{'loss': 0.6608, 'grad_norm': 12.20676040649414, 'learning_rate': 2.2452796702147826e-06, 'epoch': 0.55}
+
 55%|█████▍    | 10180/18627 [5:39:25<5:18:34,  2.26s/it]
 55%|█████▍    | 10181/18627 [5:39:27<5:18:39,  2.26s/it]
 55%|█████▍    | 10182/18627 [5:39:29<5:18:58,  2.27s/it]
 55%|█████▍    | 10183/18627 [5:39:31<5:19:13,  2.27s/it]
 55%|█████▍    | 10184/18627 [5:39:33<4:39:49,  1.99s/it]
 55%|█████▍    | 10185/18627 [5:39:35<4:53:19,  2.08s/it]
 55%|█████▍    | 10186/18627 [5:39:37<5:02:14,  2.15s/it]
 55%|█████▍    | 10187/18627 [5:39:40<5:07:40,  2.19s/it]
 55%|█████▍    | 10188/18627 [5:39:41<4:32:14,  1.94s/it]
 55%|█████▍    | 10189/18627 [5:39:43<4:47:01,  2.04s/it]
 55%|█████▍    | 10190/18627 [5:39:45<4:15:43,  1.82s/it]
                                                         
{'loss': 1.2997, 'grad_norm': 15.429813385009766, 'learning_rate': 2.2409557778099945e-06, 'epoch': 0.55}
+
 55%|█████▍    | 10190/18627 [5:39:45<4:15:43,  1.82s/it]
 55%|█████▍    | 10191/18627 [5:39:47<4:35:10,  1.96s/it]
 55%|█████▍    | 10192/18627 [5:39:49<4:50:14,  2.06s/it]
 55%|█████▍    | 10193/18627 [5:39:52<4:59:42,  2.13s/it]
 55%|█████▍    | 10194/18627 [5:39:54<5:05:33,  2.17s/it]
 55%|█████▍    | 10195/18627 [5:39:56<5:09:21,  2.20s/it]
 55%|█████▍    | 10196/18627 [5:39:58<5:12:20,  2.22s/it]
 55%|█████▍    | 10197/18627 [5:40:01<5:14:56,  2.24s/it]
 55%|█████▍    | 10198/18627 [5:40:02<4:32:19,  1.94s/it]
 55%|█████▍    | 10199/18627 [5:40:04<4:47:40,  2.05s/it]
 55%|█████▍    | 10200/18627 [5:40:06<4:55:57,  2.11s/it]
                                                         
{'loss': 0.8456, 'grad_norm': 11.621594429016113, 'learning_rate': 2.236632668570409e-06, 'epoch': 0.55}
+
 55%|█████▍    | 10200/18627 [5:40:06<4:55:57,  2.11s/it]
 55%|█████▍    | 10201/18627 [5:40:09<5:03:17,  2.16s/it]
 55%|█████▍    | 10202/18627 [5:40:10<4:30:02,  1.92s/it]
 55%|█████▍    | 10203/18627 [5:40:12<4:44:50,  2.03s/it]
 55%|█████▍    | 10204/18627 [5:40:15<4:55:37,  2.11s/it]
 55%|█████▍    | 10205/18627 [5:40:17<5:03:13,  2.16s/it]
 55%|█████▍    | 10206/18627 [5:40:19<5:07:10,  2.19s/it]
 55%|█████▍    | 10207/18627 [5:40:20<4:30:19,  1.93s/it]
 55%|█████▍    | 10208/18627 [5:40:23<4:46:41,  2.04s/it]
 55%|█████▍    | 10209/18627 [5:40:25<4:56:57,  2.12s/it]
 55%|█████▍    | 10210/18627 [5:40:26<4:23:02,  1.88s/it]
                                                         
{'loss': 1.2392, 'grad_norm': 14.709559440612793, 'learning_rate': 2.232310355566028e-06, 'epoch': 0.55}
+
 55%|█████▍    | 10210/18627 [5:40:26<4:23:02,  1.88s/it]
 55%|█████▍    | 10211/18627 [5:40:29<4:39:09,  1.99s/it]
 55%|█████▍    | 10212/18627 [5:40:31<4:53:25,  2.09s/it]
 55%|█████▍    | 10213/18627 [5:40:32<4:25:16,  1.89s/it]
 55%|█████▍    | 10214/18627 [5:40:34<4:01:22,  1.72s/it]
 55%|█████▍    | 10215/18627 [5:40:35<3:40:24,  1.57s/it]
 55%|█████▍    | 10216/18627 [5:40:37<4:09:08,  1.78s/it]
 55%|█████▍    | 10217/18627 [5:40:39<4:30:46,  1.93s/it]
 55%|█████▍    | 10218/18627 [5:40:42<4:45:22,  2.04s/it]
 55%|█████▍    | 10219/18627 [5:40:43<4:15:00,  1.82s/it]
 55%|█████▍    | 10220/18627 [5:40:44<3:53:42,  1.67s/it]
                                                         
{'loss': 1.5849, 'grad_norm': 12.997626304626465, 'learning_rate': 2.2279888518644475e-06, 'epoch': 0.55}
+
 55%|█████▍    | 10220/18627 [5:40:44<3:53:42,  1.67s/it]
 55%|█████▍    | 10221/18627 [5:40:47<4:18:54,  1.85s/it]
 55%|█████▍    | 10222/18627 [5:40:48<3:54:45,  1.68s/it]
 55%|█████▍    | 10223/18627 [5:40:50<4:20:07,  1.86s/it]
 55%|█████▍    | 10224/18627 [5:40:52<4:37:05,  1.98s/it]
 55%|█████▍    | 10225/18627 [5:40:55<4:49:27,  2.07s/it]
 55%|█████▍    | 10226/18627 [5:40:56<4:15:24,  1.82s/it]
 55%|█████▍    | 10227/18627 [5:40:58<4:34:09,  1.96s/it]
 55%|█████▍    | 10228/18627 [5:41:01<4:46:32,  2.05s/it]
 55%|█████▍    | 10229/18627 [5:41:02<4:13:15,  1.81s/it]
 55%|█████▍    | 10230/18627 [5:41:03<3:50:48,  1.65s/it]
                                                         
{'loss': 1.318, 'grad_norm': 14.621047973632812, 'learning_rate': 2.223668170530815e-06, 'epoch': 0.55}
+
 55%|█████▍    | 10230/18627 [5:41:03<3:50:48,  1.65s/it]
 55%|█████▍    | 10231/18627 [5:41:05<4:16:39,  1.83s/it]
 55%|█████▍    | 10232/18627 [5:41:07<3:54:23,  1.68s/it]
 55%|█████▍    | 10233/18627 [5:41:09<4:19:51,  1.86s/it]
 55%|█████▍    | 10234/18627 [5:41:11<4:38:02,  1.99s/it]
 55%|█████▍    | 10235/18627 [5:41:13<4:49:58,  2.07s/it]
 55%|█████▍    | 10236/18627 [5:41:16<4:58:58,  2.14s/it]
 55%|█████▍    | 10237/18627 [5:41:17<4:23:17,  1.88s/it]
 55%|█████▍    | 10238/18627 [5:41:18<3:58:33,  1.71s/it]
 55%|█████▍    | 10239/18627 [5:41:20<3:42:15,  1.59s/it]
 55%|█████▍    | 10240/18627 [5:41:22<4:11:03,  1.80s/it]
                                                         
{'loss': 1.522, 'grad_norm': 6.242947578430176, 'learning_rate': 2.2193483246277954e-06, 'epoch': 0.55}
+
 55%|█████▍    | 10240/18627 [5:41:22<4:11:03,  1.80s/it]
 55%|█████▍    | 10241/18627 [5:41:24<4:32:34,  1.95s/it]
 55%|█████▍    | 10242/18627 [5:41:27<4:47:10,  2.05s/it]
 55%|█████▍    | 10243/18627 [5:41:29<4:57:00,  2.13s/it]
 55%|█████▍    | 10244/18627 [5:41:31<5:02:56,  2.17s/it]
 55%|█████▌    | 10245/18627 [5:41:32<4:26:16,  1.91s/it]
 55%|█████▌    | 10246/18627 [5:41:35<4:44:28,  2.04s/it]
 55%|█████▌    | 10247/18627 [5:41:37<4:54:38,  2.11s/it]
 55%|█████▌    | 10248/18627 [5:41:39<5:02:07,  2.16s/it]
 55%|█████▌    | 10249/18627 [5:41:42<5:07:34,  2.20s/it]
 55%|█████▌    | 10250/18627 [5:41:44<5:11:45,  2.23s/it]
                                                         
{'loss': 0.8574, 'grad_norm': 6.177704811096191, 'learning_rate': 2.215029327215523e-06, 'epoch': 0.55}
+
 55%|█████▌    | 10250/18627 [5:41:44<5:11:45,  2.23s/it]
 55%|█████▌    | 10251/18627 [5:41:45<4:33:10,  1.96s/it]
 55%|█████▌    | 10252/18627 [5:41:48<4:46:23,  2.05s/it]
 55%|█████▌    | 10253/18627 [5:41:49<4:14:47,  1.83s/it]
 55%|█████▌    | 10254/18627 [5:41:50<3:52:11,  1.66s/it]
 55%|█████▌    | 10255/18627 [5:41:52<4:18:02,  1.85s/it]
 55%|█████▌    | 10256/18627 [5:41:55<4:35:58,  1.98s/it]
 55%|█████▌    | 10257/18627 [5:41:57<4:51:02,  2.09s/it]
 55%|█████▌    | 10258/18627 [5:41:59<5:01:05,  2.16s/it]
 55%|█████▌    | 10259/18627 [5:42:02<5:07:44,  2.21s/it]
 55%|█████▌    | 10260/18627 [5:42:04<5:25:09,  2.33s/it]
                                                         
{'loss': 1.1515, 'grad_norm': 8.040987968444824, 'learning_rate': 2.2107111913515715e-06, 'epoch': 0.55}
+
 55%|█████▌    | 10260/18627 [5:42:04<5:25:09,  2.33s/it]
 55%|█████▌    | 10261/18627 [5:42:07<5:28:32,  2.36s/it]
 55%|█████▌    | 10262/18627 [5:42:09<5:27:31,  2.35s/it]
 55%|█████▌    | 10263/18627 [5:42:11<5:24:26,  2.33s/it]
 55%|█████▌    | 10264/18627 [5:42:14<5:22:32,  2.31s/it]
 55%|█████▌    | 10265/18627 [5:42:16<5:22:30,  2.31s/it]
 55%|█████▌    | 10266/18627 [5:42:18<5:21:42,  2.31s/it]
 55%|█████▌    | 10267/18627 [5:42:20<5:21:54,  2.31s/it]
 55%|█████▌    | 10268/18627 [5:42:22<4:39:54,  2.01s/it]
 55%|█████▌    | 10269/18627 [5:42:24<4:52:07,  2.10s/it]
 55%|█████▌    | 10270/18627 [5:42:26<5:00:39,  2.16s/it]
                                                         
{'loss': 0.8214, 'grad_norm': 6.494222164154053, 'learning_rate': 2.206393930090906e-06, 'epoch': 0.55}
+
 55%|█████▌    | 10270/18627 [5:42:26<5:00:39,  2.16s/it]
 55%|█████▌    | 10271/18627 [5:42:29<5:06:58,  2.20s/it]
 55%|█████▌    | 10272/18627 [5:42:31<5:10:48,  2.23s/it]
 55%|█████▌    | 10273/18627 [5:42:33<5:12:36,  2.25s/it]
 55%|█████▌    | 10274/18627 [5:42:36<5:15:13,  2.26s/it]
 55%|█████▌    | 10275/18627 [5:42:38<5:16:11,  2.27s/it]
 55%|█████▌    | 10276/18627 [5:42:40<5:15:16,  2.27s/it]
 55%|█████▌    | 10277/18627 [5:42:42<5:15:50,  2.27s/it]
 55%|█████▌    | 10278/18627 [5:42:45<5:15:55,  2.27s/it]
 55%|█████▌    | 10279/18627 [5:42:47<5:15:59,  2.27s/it]
 55%|█████▌    | 10280/18627 [5:42:49<5:17:31,  2.28s/it]
                                                         
{'loss': 0.5926, 'grad_norm': 5.411732196807861, 'learning_rate': 2.202077556485851e-06, 'epoch': 0.55}
+
 55%|█████▌    | 10280/18627 [5:42:49<5:17:31,  2.28s/it]
 55%|█████▌    | 10281/18627 [5:42:52<5:23:57,  2.33s/it]
 55%|█████▌    | 10282/18627 [5:42:54<5:24:02,  2.33s/it]
 55%|█████▌    | 10283/18627 [5:42:56<5:25:00,  2.34s/it]
 55%|█████▌    | 10284/18627 [5:42:59<5:22:41,  2.32s/it]
 55%|█████▌    | 10285/18627 [5:43:00<4:40:03,  2.01s/it]
 55%|█████▌    | 10286/18627 [5:43:02<4:52:21,  2.10s/it]
 55%|█████▌    | 10287/18627 [5:43:05<5:00:47,  2.16s/it]
 55%|█████▌    | 10288/18627 [5:43:07<5:05:47,  2.20s/it]
 55%|█████▌    | 10289/18627 [5:43:08<4:23:18,  1.89s/it]
 55%|█████▌    | 10290/18627 [5:43:10<4:38:50,  2.01s/it]
                                                         
{'loss': 1.0182, 'grad_norm': 6.2157721519470215, 'learning_rate': 2.197762083586044e-06, 'epoch': 0.55}
+
 55%|█████▌    | 10290/18627 [5:43:10<4:38:50,  2.01s/it]
 55%|█████▌    | 10291/18627 [5:43:12<4:12:19,  1.82s/it]
 55%|█████▌    | 10292/18627 [5:43:14<4:35:45,  1.99s/it]
 55%|█████▌    | 10293/18627 [5:43:15<4:05:19,  1.77s/it]
 55%|█████▌    | 10294/18627 [5:43:18<4:25:43,  1.91s/it]
 55%|█████▌    | 10295/18627 [5:43:20<4:52:27,  2.11s/it]
 55%|█████▌    | 10296/18627 [5:43:22<4:59:13,  2.16s/it]
 55%|█████▌    | 10297/18627 [5:43:25<5:04:21,  2.19s/it]
 55%|█████▌    | 10298/18627 [5:43:27<5:08:00,  2.22s/it]
 55%|█████▌    | 10299/18627 [5:43:29<5:11:39,  2.25s/it]
 55%|█████▌    | 10300/18627 [5:43:32<5:14:12,  2.26s/it]
                                                         
{'loss': 1.0658, 'grad_norm': 4.849903106689453, 'learning_rate': 2.1934475244384027e-06, 'epoch': 0.55}
+
 55%|█████▌    | 10300/18627 [5:43:32<5:14:12,  2.26s/it]
 55%|█████▌    | 10301/18627 [5:43:33<4:35:07,  1.98s/it]
 55%|█████▌    | 10302/18627 [5:43:35<4:47:39,  2.07s/it]
 55%|█████▌    | 10303/18627 [5:43:37<4:17:00,  1.85s/it]
 55%|█████▌    | 10304/18627 [5:43:38<3:54:07,  1.69s/it]
 55%|█████▌    | 10305/18627 [5:43:40<4:17:33,  1.86s/it]
 55%|█████▌    | 10306/18627 [5:43:42<4:35:38,  1.99s/it]
 55%|█████▌    | 10307/18627 [5:43:45<4:46:04,  2.06s/it]
 55%|█████▌    | 10308/18627 [5:43:47<4:54:50,  2.13s/it]
 55%|█████▌    | 10309/18627 [5:43:49<5:01:20,  2.17s/it]
 55%|█████▌    | 10310/18627 [5:43:51<5:05:54,  2.21s/it]
                                                         
{'loss': 1.2559, 'grad_norm': 8.549363136291504, 'learning_rate': 2.189133892087078e-06, 'epoch': 0.55}
+
 55%|█████▌    | 10310/18627 [5:43:51<5:05:54,  2.21s/it]
 55%|█████▌    | 10311/18627 [5:43:54<5:10:44,  2.24s/it]
 55%|█████▌    | 10312/18627 [5:43:56<5:14:14,  2.27s/it]
 55%|█████▌    | 10313/18627 [5:43:58<5:16:15,  2.28s/it]
 55%|█████▌    | 10314/18627 [5:44:01<5:15:31,  2.28s/it]
 55%|█████▌    | 10315/18627 [5:44:03<5:14:16,  2.27s/it]
 55%|█████▌    | 10316/18627 [5:44:05<5:15:05,  2.27s/it]
 55%|█████▌    | 10317/18627 [5:44:08<5:16:14,  2.28s/it]
 55%|█████▌    | 10318/18627 [5:44:10<5:17:52,  2.30s/it]
 55%|█████▌    | 10319/18627 [5:44:12<5:17:06,  2.29s/it]
 55%|█████▌    | 10320/18627 [5:44:14<5:16:30,  2.29s/it]
                                                         
{'loss': 0.6197, 'grad_norm': 5.534502983093262, 'learning_rate': 2.184821199573424e-06, 'epoch': 0.55}
+
 55%|█████▌    | 10320/18627 [5:44:14<5:16:30,  2.29s/it]
 55%|█████▌    | 10321/18627 [5:44:16<4:37:06,  2.00s/it]
 55%|█████▌    | 10322/18627 [5:44:18<4:47:45,  2.08s/it]
 55%|█████▌    | 10323/18627 [5:44:20<4:55:17,  2.13s/it]
 55%|█████▌    | 10324/18627 [5:44:23<5:01:52,  2.18s/it]
 55%|█████▌    | 10325/18627 [5:44:24<4:26:25,  1.93s/it]
 55%|█████▌    | 10326/18627 [5:44:26<4:41:21,  2.03s/it]
 55%|��████▌    | 10327/18627 [5:44:27<4:07:19,  1.79s/it]
 55%|█████▌    | 10328/18627 [5:44:30<4:27:42,  1.94s/it]
 55%|█████▌    | 10329/18627 [5:44:32<4:41:32,  2.04s/it]
 55%|█████▌    | 10330/18627 [5:44:34<4:50:04,  2.10s/it]
                                                         
{'loss': 1.2455, 'grad_norm': 7.615298271179199, 'learning_rate': 2.1805094599359496e-06, 'epoch': 0.55}
+
 55%|█████▌    | 10330/18627 [5:44:34<4:50:04,  2.10s/it]
 55%|█████▌    | 10331/18627 [5:44:36<4:57:42,  2.15s/it]
 55%|█████▌    | 10332/18627 [5:44:38<4:21:04,  1.89s/it]
 55%|█████▌    | 10333/18627 [5:44:40<4:38:58,  2.02s/it]
 55%|█████▌    | 10334/18627 [5:44:43<5:03:34,  2.20s/it]
 55%|█████▌    | 10335/18627 [5:44:45<5:05:00,  2.21s/it]
 55%|█████▌    | 10336/18627 [5:44:47<5:08:42,  2.23s/it]
 55%|█████▌    | 10337/18627 [5:44:49<5:10:41,  2.25s/it]
 56%|█████▌    | 10338/18627 [5:44:52<5:10:56,  2.25s/it]
 56%|█████▌    | 10339/18627 [5:44:54<5:12:26,  2.26s/it]
 56%|█████▌    | 10340/18627 [5:44:56<5:12:28,  2.26s/it]
                                                         
{'loss': 0.8249, 'grad_norm': 8.008660316467285, 'learning_rate': 2.1761986862102845e-06, 'epoch': 0.56}
+
 56%|█████▌    | 10340/18627 [5:44:56<5:12:28,  2.26s/it]
 56%|█████▌    | 10341/18627 [5:44:58<4:32:16,  1.97s/it]
 56%|█████▌    | 10342/18627 [5:45:00<4:44:30,  2.06s/it]
 56%|█████▌    | 10343/18627 [5:45:02<4:51:59,  2.11s/it]
 56%|█████▌    | 10344/18627 [5:45:04<4:57:28,  2.15s/it]
 56%|█████▌    | 10345/18627 [5:45:07<5:02:16,  2.19s/it]
 56%|█████▌    | 10346/18627 [5:45:09<5:05:46,  2.22s/it]
 56%|█████▌    | 10347/18627 [5:45:11<5:09:27,  2.24s/it]
 56%|█████▌    | 10348/18627 [5:45:13<4:30:45,  1.96s/it]
 56%|█████▌    | 10349/18627 [5:45:15<4:44:07,  2.06s/it]
 56%|█████▌    | 10350/18627 [5:45:18<5:11:30,  2.26s/it]
                                                         
{'loss': 1.1307, 'grad_norm': 5.708700656890869, 'learning_rate': 2.1718888914291365e-06, 'epoch': 0.56}
+
 56%|█████▌    | 10350/18627 [5:45:18<5:11:30,  2.26s/it]
 56%|█████▌    | 10351/18627 [5:45:20<5:14:17,  2.28s/it]
 56%|█████▌    | 10352/18627 [5:45:22<5:13:40,  2.27s/it]
 56%|█████▌    | 10353/18627 [5:45:24<5:13:13,  2.27s/it]
 56%|█████▌    | 10354/18627 [5:45:26<4:31:34,  1.97s/it]
 56%|█████▌    | 10355/18627 [5:45:28<4:46:10,  2.08s/it]
 56%|█████▌    | 10356/18627 [5:45:30<4:55:08,  2.14s/it]
 56%|█████▌    | 10357/18627 [5:45:33<5:01:04,  2.18s/it]
 56%|█████▌    | 10358/18627 [5:45:35<5:04:52,  2.21s/it]
 56%|█████▌    | 10359/18627 [5:45:37<5:07:56,  2.23s/it]
 56%|█████▌    | 10360/18627 [5:45:39<5:10:03,  2.25s/it]
                                                         
{'loss': 0.8307, 'grad_norm': 7.5365095138549805, 'learning_rate': 2.1675800886222566e-06, 'epoch': 0.56}
+
 56%|█████▌    | 10360/18627 [5:45:39<5:10:03,  2.25s/it]
 56%|█████▌    | 10361/18627 [5:45:42<5:10:36,  2.25s/it]
 56%|█████▌    | 10362/18627 [5:45:43<4:30:47,  1.97s/it]
 56%|█████▌    | 10363/18627 [5:45:45<4:44:31,  2.07s/it]
 56%|█████▌    | 10364/18627 [5:45:47<4:52:29,  2.12s/it]
 56%|█████▌    | 10365/18627 [5:45:50<5:00:37,  2.18s/it]
 56%|█████▌    | 10366/18627 [5:45:52<5:05:43,  2.22s/it]
 56%|█████▌    | 10367/18627 [5:45:54<5:09:11,  2.25s/it]
 56%|█████▌    | 10368/18627 [5:45:56<4:30:27,  1.96s/it]
 56%|█████▌    | 10369/18627 [5:45:58<4:44:13,  2.07s/it]
 56%|█████▌    | 10370/18627 [5:46:00<4:51:50,  2.12s/it]
                                                         
{'loss': 1.0264, 'grad_norm': 5.835506916046143, 'learning_rate': 2.163272290816394e-06, 'epoch': 0.56}
+
 56%|█████▌    | 10370/18627 [5:46:00<4:51:50,  2.12s/it]
 56%|█████▌    | 10371/18627 [5:46:03<4:58:13,  2.17s/it]
 56%|█████▌    | 10372/18627 [5:46:05<5:03:19,  2.20s/it]
 56%|█████▌    | 10373/18627 [5:46:07<5:06:25,  2.23s/it]
 56%|█████▌    | 10374/18627 [5:46:09<5:09:33,  2.25s/it]
 56%|█████▌    | 10375/18627 [5:46:12<5:11:05,  2.26s/it]
 56%|█████▌    | 10376/18627 [5:46:14<5:12:38,  2.27s/it]
 56%|█████▌    | 10377/18627 [5:46:15<4:32:46,  1.98s/it]
 56%|█████▌    | 10378/18627 [5:46:18<4:44:22,  2.07s/it]
 56%|█████▌    | 10379/18627 [5:46:20<4:54:42,  2.14s/it]
 56%|█████▌    | 10380/18627 [5:46:22<4:59:28,  2.18s/it]
                                                         
{'loss': 0.795, 'grad_norm': 5.597250938415527, 'learning_rate': 2.158965511035261e-06, 'epoch': 0.56}
+
 56%|█���███▌    | 10380/18627 [5:46:22<4:59:28,  2.18s/it]
 56%|█████▌    | 10381/18627 [5:46:24<5:04:02,  2.21s/it]
 56%|█████▌    | 10382/18627 [5:46:27<5:06:59,  2.23s/it]
 56%|█████▌    | 10383/18627 [5:46:29<5:20:43,  2.33s/it]
 56%|█████▌    | 10384/18627 [5:46:32<5:19:28,  2.33s/it]
 56%|█████▌    | 10385/18627 [5:46:34<5:17:51,  2.31s/it]
 56%|█████▌    | 10386/18627 [5:46:36<5:17:49,  2.31s/it]
 56%|█████▌    | 10387/18627 [5:46:39<5:18:16,  2.32s/it]
 56%|█████▌    | 10388/18627 [5:46:40<4:49:53,  2.11s/it]
 56%|█████▌    | 10389/18627 [5:46:42<4:56:52,  2.16s/it]
 56%|█████▌    | 10390/18627 [5:46:45<5:00:25,  2.19s/it]
                                                         
{'loss': 0.8265, 'grad_norm': 5.079643726348877, 'learning_rate': 2.1546597622994925e-06, 'epoch': 0.56}
+
 56%|█████▌    | 10390/18627 [5:46:45<5:00:25,  2.19s/it]
 56%|█████▌    | 10391/18627 [5:46:47<5:05:20,  2.22s/it]
 56%|█████▌    | 10392/18627 [5:46:48<4:26:11,  1.94s/it]
 56%|█████▌    | 10393/18627 [5:46:50<4:01:52,  1.76s/it]
 56%|█████▌    | 10394/18627 [5:46:52<4:23:12,  1.92s/it]
 56%|█████▌    | 10395/18627 [5:46:54<4:36:31,  2.02s/it]
 56%|█████▌    | 10396/18627 [5:46:56<4:47:37,  2.10s/it]
 56%|█████▌    | 10397/18627 [5:46:59<4:55:45,  2.16s/it]
 56%|█████▌    | 10398/18627 [5:47:01<5:01:10,  2.20s/it]
 56%|█████▌    | 10399/18627 [5:47:03<5:05:55,  2.23s/it]
 56%|█████▌    | 10400/18627 [5:47:06<5:08:05,  2.25s/it]
                                                         
{'loss': 1.0734, 'grad_norm': 5.55297327041626, 'learning_rate': 2.1503550576266046e-06, 'epoch': 0.56}
+
 56%|█████▌    | 10400/18627 [5:47:06<5:08:05,  2.25s/it]
 56%|█████▌    | 10401/18627 [5:47:08<5:09:27,  2.26s/it]
 56%|█████▌    | 10402/18627 [5:47:10<5:11:28,  2.27s/it]
 56%|█████▌    | 10403/18627 [5:47:13<5:12:06,  2.28s/it]
 56%|█████▌    | 10404/18627 [5:47:15<5:12:42,  2.28s/it]
 56%|█████▌    | 10405/18627 [5:47:17<5:13:23,  2.29s/it]
 56%|█████▌    | 10406/18627 [5:47:19<5:12:50,  2.28s/it]
 56%|█████▌    | 10407/18627 [5:47:22<5:14:18,  2.29s/it]
 56%|█████▌    | 10408/18627 [5:47:24<5:14:36,  2.30s/it]
 56%|█████▌    | 10409/18627 [5:47:26<5:13:53,  2.29s/it]
 56%|█████▌    | 10410/18627 [5:47:28<4:33:42,  2.00s/it]
                                                         
{'loss': 0.8254, 'grad_norm': 14.704153060913086, 'learning_rate': 2.146051410030959e-06, 'epoch': 0.56}
+
 56%|█████▌    | 10410/18627 [5:47:28<4:33:42,  2.00s/it]
 56%|█████▌    | 10411/18627 [5:47:29<4:05:37,  1.79s/it]
 56%|█████▌    | 10412/18627 [5:47:31<4:25:48,  1.94s/it]
 56%|█████▌    | 10413/18627 [5:47:33<4:39:35,  2.04s/it]
 56%|█████▌    | 10414/18627 [5:47:35<4:09:41,  1.82s/it]
 56%|█████▌    | 10415/18627 [5:47:37<4:28:31,  1.96s/it]
 56%|█████▌    | 10416/18627 [5:47:39<4:41:17,  2.06s/it]
 56%|█████▌    | 10417/18627 [5:47:41<4:06:02,  1.80s/it]
 56%|█████▌    | 10418/18627 [5:47:43<4:26:27,  1.95s/it]
 56%|█████▌    | 10419/18627 [5:47:45<4:39:58,  2.05s/it]
 56%|█████▌    | 10420/18627 [5:47:47<4:48:48,  2.11s/it]
                                                         
{'loss': 1.113, 'grad_norm': 5.373778343200684, 'learning_rate': 2.1417488325237182e-06, 'epoch': 0.56}
+
 56%|█████▌    | 10420/18627 [5:47:47<4:48:48,  2.11s/it]
 56%|█████▌    | 10421/18627 [5:47:50<4:56:12,  2.17s/it]
 56%|█████▌    | 10422/18627 [5:47:52<5:01:31,  2.20s/it]
 56%|█████▌    | 10423/18627 [5:47:54<5:05:01,  2.23s/it]
 56%|█████▌    | 10424/18627 [5:47:57<5:07:02,  2.25s/it]
 56%|█████▌    | 10425/18627 [5:47:59<5:07:49,  2.25s/it]
 56%|█████▌    | 10426/18627 [5:48:01<5:09:52,  2.27s/it]
 56%|█████▌    | 10427/18627 [5:48:03<5:09:27,  2.26s/it]
 56%|█████▌    | 10428/18627 [5:48:06<5:09:37,  2.27s/it]
 56%|█████▌    | 10429/18627 [5:48:07<4:28:35,  1.97s/it]
 56%|█████▌    | 10430/18627 [5:48:08<4:00:38,  1.76s/it]
                                                         
{'loss': 1.0807, 'grad_norm': 16.62543487548828, 'learning_rate': 2.1374473381128143e-06, 'epoch': 0.56}
+
 56%|█████▌    | 10430/18627 [5:48:08<4:00:38,  1.76s/it]
 56%|█████▌    | 10431/18627 [5:48:09<3:41:17,  1.62s/it]
 56%|█████▌    | 10432/18627 [5:48:11<3:28:32,  1.53s/it]
 56%|█████▌    | 10433/18627 [5:48:12<3:17:41,  1.45s/it]
 56%|█████▌    | 10434/18627 [5:48:14<3:51:01,  1.69s/it]
 56%|█████▌    | 10435/18627 [5:48:17<4:13:57,  1.86s/it]
 56%|█████▌    | 10436/18627 [5:48:19<4:30:14,  1.98s/it]
 56%|█████▌    | 10437/18627 [5:48:21<4:41:19,  2.06s/it]
 56%|█████▌    | 10438/18627 [5:48:23<4:50:55,  2.13s/it]
 56%|█████▌    | 10439/18627 [5:48:25<4:16:31,  1.88s/it]
 56%|█████▌    | 10440/18627 [5:48:27<4:32:24,  2.00s/it]
                                                         
{'loss': 1.4459, 'grad_norm': 5.273484706878662, 'learning_rate': 2.133146939802901e-06, 'epoch': 0.56}
+
 56%|█████▌    | 10440/18627 [5:48:27<4:32:24,  2.00s/it]
 56%|█████▌    | 10441/18627 [5:48:29<4:43:16,  2.08s/it]
 56%|█████▌    | 10442/18627 [5:48:31<4:52:02,  2.14s/it]
 56%|█████▌    | 10443/18627 [5:48:33<4:19:00,  1.90s/it]
 56%|█████▌    | 10444/18627 [5:48:35<4:35:14,  2.02s/it]
 56%|█████▌    | 10445/18627 [5:48:37<4:47:49,  2.11s/it]
 56%|█████▌    | 10446/18627 [5:48:40<4:57:07,  2.18s/it]
 56%|█████▌    | 10447/18627 [5:48:41<4:23:19,  1.93s/it]
 56%|█████▌    | 10448/18627 [5:48:43<4:38:55,  2.05s/it]
 56%|█████▌    | 10449/18627 [5:48:45<4:09:17,  1.83s/it]
 56%|█████▌    | 10450/18627 [5:48:47<4:29:25,  1.98s/it]
                                                         
{'loss': 1.2444, 'grad_norm': 5.3604230880737305, 'learning_rate': 2.1288476505953194e-06, 'epoch': 0.56}
+
 56%|█████▌    | 10450/18627 [5:48:47<4:29:25,  1.98s/it]
 56%|█████▌    | 10451/18627 [5:48:49<4:41:23,  2.06s/it]
 56%|█████▌    | 10452/18627 [5:48:51<4:10:26,  1.84s/it]
 56%|█████▌    | 10453/18627 [5:48:52<3:48:41,  1.68s/it]
 56%|█████▌    | 10454/18627 [5:48:53<3:34:38,  1.58s/it]
 56%|█████▌    | 10455/18627 [5:48:56<4:03:24,  1.79s/it]
 56%|█████▌    | 10456/18627 [5:48:58<4:23:12,  1.93s/it]
 56%|█████▌    | 10457/18627 [5:49:00<4:41:24,  2.07s/it]
 56%|█████▌    | 10458/18627 [5:49:02<4:09:56,  1.84s/it]
 56%|█████▌    | 10459/18627 [5:49:04<4:26:48,  1.96s/it]
 56%|█████▌    | 10460/18627 [5:49:05<4:00:39,  1.77s/it]
                                                         
{'loss': 1.7088, 'grad_norm': 17.38876724243164, 'learning_rate': 2.124549483488057e-06, 'epoch': 0.56}
+
 56%|█████▌    | 10460/18627 [5:49:05<4:00:39,  1.77s/it]
 56%|█████▌    | 10461/18627 [5:49:07<4:20:33,  1.91s/it]
 56%|█████▌    | 10462/18627 [5:49:10<4:35:22,  2.02s/it]
 56%|█████▌    | 10463/18627 [5:49:12<4:46:51,  2.11s/it]
 56%|█████▌    | 10464/18627 [5:49:14<4:52:31,  2.15s/it]
 56%|█████▌    | 10465/18627 [5:49:17<4:58:06,  2.19s/it]
 56%|█████▌    | 10466/18627 [5:49:19<5:01:46,  2.22s/it]
 56%|█████▌    | 10467/18627 [5:49:20<4:21:19,  1.92s/it]
 56%|█████▌    | 10468/18627 [5:49:22<4:35:14,  2.02s/it]
 56%|█████▌    | 10469/18627 [5:49:24<4:04:39,  1.80s/it]
 56%|█████▌    | 10470/18627 [5:49:26<4:23:01,  1.93s/it]
                                                         
{'loss': 0.9625, 'grad_norm': 6.306502342224121, 'learning_rate': 2.1202524514757113e-06, 'epoch': 0.56}
+
 56%|█████▌    | 10470/18627 [5:49:26<4:23:01,  1.93s/it]
 56%|█████▌    | 10471/18627 [5:49:28<4:37:42,  2.04s/it]
 56%|█████▌    | 10472/18627 [5:49:29<4:04:30,  1.80s/it]
 56%|█████▌    | 10473/18627 [5:49:31<3:45:38,  1.66s/it]
 56%|█████▌    | 10474/18627 [5:49:33<4:09:40,  1.84s/it]
 56%|█████▌    | 10475/18627 [5:49:35<4:28:10,  1.97s/it]
 56%|█████▌    | 10476/18627 [5:49:37<4:40:29,  2.06s/it]
 56%|█████▌    | 10477/18627 [5:49:40<4:54:03,  2.16s/it]
 56%|█████▋    | 10478/18627 [5:49:41<4:17:01,  1.89s/it]
 56%|█████▋    | 10479/18627 [5:49:42<3:39:37,  1.62s/it]
 56%|█████▋    | 10480/18627 [5:49:43<3:26:32,  1.52s/it]
                                                         
{'loss': 1.4251, 'grad_norm': 14.838828086853027, 'learning_rate': 2.1159565675494443e-06, 'epoch': 0.56}
+
 56%|█████▋    | 10480/18627 [5:49:43<3:26:32,  1.52s/it]
 56%|█████▋    | 10481/18627 [5:49:46<3:57:07,  1.75s/it]
 56%|█████▋    | 10482/18627 [5:49:48<4:18:33,  1.90s/it]
 56%|█████▋    | 10483/18627 [5:49:50<4:33:20,  2.01s/it]
 56%|█████▋    | 10484/18627 [5:49:53<4:44:15,  2.09s/it]
 56%|█████▋    | 10485/18627 [5:49:55<4:50:50,  2.14s/it]
 56%|█████▋    | 10486/18627 [5:49:57<4:55:44,  2.18s/it]
 56%|█████▋    | 10487/18627 [5:49:59<5:00:02,  2.21s/it]
 56%|█████▋    | 10488/18627 [5:50:02<5:02:58,  2.23s/it]
 56%|█████▋    | 10489/18627 [5:50:03<4:25:03,  1.95s/it]
 56%|█████▋    | 10490/18627 [5:50:05<4:39:42,  2.06s/it]
                                                         
{'loss': 0.8733, 'grad_norm': 6.216856479644775, 'learning_rate': 2.11166184469695e-06, 'epoch': 0.56}
+
 56%|█████▋    | 10490/18627 [5:50:05<4:39:42,  2.06s/it]
 56%|█████▋    | 10491/18627 [5:50:08<4:49:40,  2.14s/it]
 56%|█████▋    | 10492/18627 [5:50:10<4:55:48,  2.18s/it]
 56%|█████▋    | 10493/18627 [5:50:12<5:00:19,  2.22s/it]
 56%|█████▋    | 10494/18627 [5:50:14<5:03:58,  2.24s/it]
 56%|█████▋    | 10495/18627 [5:50:17<5:04:03,  2.24s/it]
 56%|█████▋    | 10496/18627 [5:50:19<5:05:19,  2.25s/it]
 56%|█████▋    | 10497/18627 [5:50:21<5:06:23,  2.26s/it]
 56%|█████▋    | 10498/18627 [5:50:24<5:07:29,  2.27s/it]
 56%|█████▋    | 10499/18627 [5:50:26<5:08:09,  2.27s/it]
 56%|█████▋    | 10500/18627 [5:50:27<4:24:30,  1.95s/it]
                                                         
{'loss': 0.8504, 'grad_norm': 13.25187873840332, 'learning_rate': 2.10736829590241e-06, 'epoch': 0.56}
+
 56%|█████▋    | 10500/18627 [5:50:27<4:24:30,  1.95s/it]
 56%|█████▋    | 10501/18627 [5:50:28<3:59:03,  1.77s/it]
 56%|█████▋    | 10502/18627 [5:50:31<4:34:48,  2.03s/it]
 56%|█████▋    | 10503/18627 [5:50:32<4:01:26,  1.78s/it]
 56%|█████▋    | 10504/18627 [5:50:33<3:29:12,  1.55s/it]
 56%|█████▋    | 10505/18627 [5:50:35<3:58:59,  1.77s/it]
 56%|█████▋    | 10506/18627 [5:50:37<3:39:59,  1.63s/it]
 56%|█████▋    | 10507/18627 [5:50:39<4:05:48,  1.82s/it]
 56%|█████▋    | 10508/18627 [5:50:41<4:24:26,  1.95s/it]
 56%|█████▋    | 10509/18627 [5:50:44<4:36:25,  2.04s/it]
 56%|█████▋    | 10510/18627 [5:50:46<4:44:57,  2.11s/it]
                                                         
{'loss': 1.1758, 'grad_norm': 6.080920219421387, 'learning_rate': 2.1030759341464583e-06, 'epoch': 0.56}
+
 56%|█████▋    | 10510/18627 [5:50:46<4:44:57,  2.11s/it]
 56%|█████▋    | 10511/18627 [5:50:48<4:52:04,  2.16s/it]
 56%|█████▋    | 10512/18627 [5:50:50<4:57:53,  2.20s/it]
 56%|█████▋    | 10513/18627 [5:50:53<4:59:46,  2.22s/it]
 56%|█████▋    | 10514/18627 [5:50:55<5:02:30,  2.24s/it]
 56%|█████▋    | 10515/18627 [5:50:57<5:04:56,  2.26s/it]
 56%|█████▋    | 10516/18627 [5:50:59<5:05:10,  2.26s/it]
 56%|█████▋    | 10517/18627 [5:51:01<4:25:48,  1.97s/it]
 56%|█████▋    | 10518/18627 [5:51:03<4:37:40,  2.05s/it]
 56%|█████▋    | 10519/18627 [5:51:05<4:45:32,  2.11s/it]
 56%|█████▋    | 10520/18627 [5:51:08<4:50:41,  2.15s/it]
                                                         
{'loss': 0.8829, 'grad_norm': 5.644862651824951, 'learning_rate': 2.098784772406139e-06, 'epoch': 0.56}
+
 56%|█████▋    | 10520/18627 [5:51:08<4:50:41,  2.15s/it]
 56%|█████▋    | 10521/18627 [5:51:10<4:51:59,  2.16s/it]
 56%|█████▋    | 10522/18627 [5:51:11<4:13:16,  1.87s/it]
 56%|█████▋    | 10523/18627 [5:51:13<4:24:44,  1.96s/it]
 56%|█████▋    | 10524/18627 [5:51:15<4:34:06,  2.03s/it]
 57%|█████▋    | 10525/18627 [5:51:16<4:01:02,  1.79s/it]
 57%|█████▋    | 10526/18627 [5:51:19<4:14:24,  1.88s/it]
 57%|█████▋    | 10527/18627 [5:51:21<4:28:15,  1.99s/it]
 57%|█████▋    | 10528/18627 [5:51:22<3:57:23,  1.76s/it]
 57%|█████▋    | 10529/18627 [5:51:24<4:12:32,  1.87s/it]
 57%|█████▋    | 10530/18627 [5:51:26<4:22:48,  1.95s/it]
                                                         
{'loss': 1.1413, 'grad_norm': 6.920463562011719, 'learning_rate': 2.094494823654869e-06, 'epoch': 0.57}
+
 57%|█████▋    | 10530/18627 [5:51:26<4:22:48,  1.95s/it]
 57%|█████▋    | 10531/18627 [5:51:28<4:29:23,  2.00s/it]
 57%|█████▋    | 10532/18627 [5:51:31<4:48:25,  2.14s/it]
 57%|█████▋    | 10533/18627 [5:51:33<4:49:29,  2.15s/it]
 57%|█████▋    | 10534/18627 [5:51:35<4:50:02,  2.15s/it]
 57%|█████▋    | 10535/18627 [5:51:37<4:50:51,  2.16s/it]
 57%|█████▋    | 10536/18627 [5:51:40<5:01:44,  2.24s/it]
 57%|█████▋    | 10537/18627 [5:51:42<5:00:48,  2.23s/it]
 57%|█████▋    | 10538/18627 [5:51:43<4:19:13,  1.92s/it]
 57%|█████▋    | 10539/18627 [5:51:46<4:58:00,  2.21s/it]
 57%|█████▋    | 10540/18627 [5:51:49<5:08:17,  2.29s/it]
                                                         
{'loss': 0.791, 'grad_norm': 7.158686637878418, 'learning_rate': 2.090206100862396e-06, 'epoch': 0.57}
+
 57%|█████▋    | 10540/18627 [5:51:49<5:08:17,  2.29s/it]
 57%|█████▋    | 10541/18627 [5:51:51<5:22:03,  2.39s/it]
 57%|█████▋    | 10542/18627 [5:51:54<5:33:42,  2.48s/it]
 57%|█████▋    | 10543/18627 [5:51:56<5:31:14,  2.46s/it]
 57%|█████▋    | 10544/18627 [5:51:58<4:41:20,  2.09s/it]
 57%|█████▋    | 10545/18627 [5:52:00<4:43:47,  2.11s/it]
 57%|█████▋    | 10546/18627 [5:52:02<4:45:46,  2.12s/it]
 57%|█████▋    | 10547/18627 [5:52:04<4:45:38,  2.12s/it]
 57%|█████▋    | 10548/18627 [5:52:05<4:08:18,  1.84s/it]
 57%|█████▋    | 10549/18627 [5:52:06<3:41:54,  1.65s/it]
 57%|█████▋    | 10550/18627 [5:52:08<3:44:20,  1.67s/it]
                                                         
{'loss': 1.2963, 'grad_norm': 13.96137809753418, 'learning_rate': 2.085918616994764e-06, 'epoch': 0.57}
+
 57%|█████▋    | 10550/18627 [5:52:08<3:44:20,  1.67s/it]
 57%|█████▋    | 10551/18627 [5:52:10<4:02:47,  1.80s/it]
 57%|█████▋    | 10552/18627 [5:52:12<3:50:42,  1.71s/it]
 57%|█████▋    | 10553/18627 [5:52:14<4:33:43,  2.03s/it]
 57%|█████▋    | 10554/18627 [5:52:17<4:38:00,  2.07s/it]
 57%|█████▋    | 10555/18627 [5:52:18<4:04:41,  1.82s/it]
 57%|█████▋    | 10556/18627 [5:52:21<4:49:43,  2.15s/it]
 57%|█████▋    | 10557/18627 [5:52:23<4:48:01,  2.14s/it]
 57%|█████▋    | 10558/18627 [5:52:25<4:47:20,  2.14s/it]
 57%|█████▋    | 10559/18627 [5:52:26<4:09:37,  1.86s/it]
 57%|█████▋    | 10560/18627 [5:52:27<3:43:53,  1.67s/it]
                                                         
{'loss': 1.4476, 'grad_norm': 14.418923377990723, 'learning_rate': 2.0816323850142693e-06, 'epoch': 0.57}
+
 57%|█████▋    | 10560/18627 [5:52:28<3:43:53,  1.67s/it]
 57%|█████▋    | 10561/18627 [5:52:30<4:14:10,  1.89s/it]
 57%|█████▋    | 10562/18627 [5:52:32<4:26:15,  1.98s/it]
 57%|█████▋    | 10563/18627 [5:52:34<4:33:23,  2.03s/it]
 57%|█████▋    | 10564/18627 [5:52:36<4:38:20,  2.07s/it]
 57%|█████▋    | 10565/18627 [5:52:38<4:41:40,  2.10s/it]
 57%|█████▋    | 10566/18627 [5:52:41<4:43:14,  2.11s/it]
 57%|█████▋    | 10567/18627 [5:52:43<4:44:55,  2.12s/it]
 57%|█████▋    | 10568/18627 [5:52:45<4:46:06,  2.13s/it]
 57%|█████▋    | 10569/18627 [5:52:47<4:51:20,  2.17s/it]
 57%|█████▋    | 10570/18627 [5:52:49<4:50:50,  2.17s/it]
                                                         
{'loss': 0.6471, 'grad_norm': 6.3950514793396, 'learning_rate': 2.0773474178794256e-06, 'epoch': 0.57}
+
 57%|█████▋    | 10570/18627 [5:52:49<4:50:50,  2.17s/it]
 57%|█████▋    | 10571/18627 [5:52:52<4:52:37,  2.18s/it]
 57%|█████▋    | 10572/18627 [5:52:53<4:12:41,  1.88s/it]
 57%|█████▋    | 10573/18627 [5:52:54<3:44:48,  1.67s/it]
 57%|█████▋    | 10574/18627 [5:52:55<3:25:05,  1.53s/it]
 57%|█████▋    | 10575/18627 [5:52:57<3:48:49,  1.71s/it]
 57%|█████▋    | 10576/18627 [5:52:59<4:06:09,  1.83s/it]
 57%|█████▋    | 10577/18627 [5:53:03<4:59:45,  2.23s/it]
 57%|█████▋    | 10578/18627 [5:53:04<4:17:41,  1.92s/it]
 57%|█████▋    | 10579/18627 [5:53:06<4:25:58,  1.98s/it]
 57%|█████▋    | 10580/18627 [5:53:08<4:31:21,  2.02s/it]
                                                         
{'loss': 1.4947, 'grad_norm': 7.1297688484191895, 'learning_rate': 2.0730637285449186e-06, 'epoch': 0.57}
+
 57%|█████▋    | 10580/18627 [5:53:08<4:31:21,  2.02s/it]
 57%|█████▋    | 10581/18627 [5:53:10<4:35:16,  2.05s/it]
 57%|█████▋    | 10582/18627 [5:53:12<4:38:18,  2.08s/it]
 57%|█████▋    | 10583/18627 [5:53:14<4:40:42,  2.09s/it]
 57%|█████▋    | 10584/18627 [5:53:17<5:01:40,  2.25s/it]
 57%|█████▋    | 10585/18627 [5:53:18<4:15:38,  1.91s/it]
 57%|█████▋    | 10586/18627 [5:53:20<4:24:34,  1.97s/it]
 57%|█████▋    | 10587/18627 [5:53:22<4:31:18,  2.02s/it]
 57%|█████▋    | 10588/18627 [5:53:25<4:35:41,  2.06s/it]
 57%|█████▋    | 10589/18627 [5:53:27<5:02:38,  2.26s/it]
 57%|█████▋    | 10590/18627 [5:53:29<4:57:50,  2.22s/it]
                                                         
{'loss': 0.8903, 'grad_norm': 5.737969875335693, 'learning_rate': 2.0687813299615763e-06, 'epoch': 0.57}
+
 57%|█████▋    | 10590/18627 [5:53:30<4:57:50,  2.22s/it]
 57%|█████▋    | 10591/18627 [5:53:32<5:19:10,  2.38s/it]
 57%|█████▋    | 10592/18627 [5:53:33<4:14:43,  1.90s/it]
 57%|█████▋    | 10593/18627 [5:53:35<4:25:32,  1.98s/it]
 57%|█████▋    | 10594/18627 [5:53:37<4:33:49,  2.05s/it]
 57%|█████▋    | 10595/18627 [5:53:39<4:39:59,  2.09s/it]
 57%|█████▋    | 10596/18627 [5:53:41<4:21:16,  1.95s/it]
 57%|█████▋    | 10597/18627 [5:53:42<3:51:44,  1.73s/it]
 57%|█████▋    | 10598/18627 [5:53:45<4:10:23,  1.87s/it]
 57%|█████▋    | 10599/18627 [5:53:46<3:58:43,  1.78s/it]
 57%|█████▋    | 10600/18627 [5:53:48<4:13:41,  1.90s/it]
                                                         
{'loss': 1.207, 'grad_norm': 5.430782794952393, 'learning_rate': 2.0645002350763194e-06, 'epoch': 0.57}
+
 57%|█████▋    | 10600/18627 [5:53:48<4:13:41,  1.90s/it]
 57%|█████▋    | 10601/18627 [5:53:50<4:25:10,  1.98s/it]
 57%|█████▋    | 10602/18627 [5:53:53<4:33:43,  2.05s/it]
 57%|█████▋    | 10603/18627 [5:53:55<4:38:04,  2.08s/it]
 57%|█████▋    | 10604/18627 [5:53:57<4:42:00,  2.11s/it]
 57%|█████▋    | 10605/18627 [5:53:59<4:42:02,  2.11s/it]
 57%|█████▋    | 10606/18627 [5:54:00<4:05:08,  1.83s/it]
 57%|█████▋    | 10607/18627 [5:54:02<4:16:21,  1.92s/it]
 57%|█████▋    | 10608/18627 [5:54:05<4:25:41,  1.99s/it]
 57%|█████▋    | 10609/18627 [5:54:07<4:55:18,  2.21s/it]
 57%|█████▋    | 10610/18627 [5:54:09<4:52:56,  2.19s/it]
                                                         
{'loss': 0.8719, 'grad_norm': 8.414691925048828, 'learning_rate': 2.0602204568321305e-06, 'epoch': 0.57}
+
 57%|█████▋    | 10610/18627 [5:54:09<4:52:56,  2.19s/it]
 57%|█████▋    | 10611/18627 [5:54:12<4:51:15,  2.18s/it]
 57%|█████▋    | 10612/18627 [5:54:14<4:49:37,  2.17s/it]
 57%|█████▋    | 10613/18627 [5:54:15<4:10:41,  1.88s/it]
 57%|█████▋    | 10614/18627 [5:54:17<4:31:03,  2.03s/it]
 57%|█████▋    | 10615/18627 [5:54:19<4:35:13,  2.06s/it]
 57%|█████▋    | 10616/18627 [5:54:22<4:37:54,  2.08s/it]
 57%|█████▋    | 10617/18627 [5:54:24<4:41:04,  2.11s/it]
 57%|█████▋    | 10618/18627 [5:54:26<4:41:30,  2.11s/it]
 57%|█████▋    | 10619/18627 [5:54:28<4:43:56,  2.13s/it]
 57%|█████▋    | 10620/18627 [5:54:30<4:42:32,  2.12s/it]
                                                         
{'loss': 0.8664, 'grad_norm': 6.2966532707214355, 'learning_rate': 2.0559420081680094e-06, 'epoch': 0.57}
+
 57%|█████▋    | 10620/18627 [5:54:30<4:42:32,  2.12s/it]
 57%|█████▋    | 10621/18627 [5:54:32<4:42:11,  2.11s/it]
 57%|█████▋    | 10622/18627 [5:54:34<4:43:35,  2.13s/it]
 57%|█████▋    | 10623/18627 [5:54:37<4:45:37,  2.14s/it]
 57%|█████▋    | 10624/18627 [5:54:39<4:44:17,  2.13s/it]
 57%|█████▋    | 10625/18627 [5:54:41<4:47:06,  2.15s/it]
 57%|█████▋    | 10626/18627 [5:54:43<4:51:02,  2.18s/it]
 57%|█████▋    | 10627/18627 [5:54:45<4:53:42,  2.20s/it]
 57%|█████▋    | 10628/18627 [5:54:48<4:53:37,  2.20s/it]
 57%|█████▋    | 10629/18627 [5:54:50<4:51:16,  2.19s/it]
 57%|█████▋    | 10630/18627 [5:54:51<4:12:25,  1.89s/it]
                                                         
{'loss': 0.8102, 'grad_norm': 13.461386680603027, 'learning_rate': 2.051664902018938e-06, 'epoch': 0.57}
+
 57%|█████▋    | 10630/18627 [5:54:51<4:12:25,  1.89s/it]
 57%|█████▋    | 10631/18627 [5:54:53<4:23:09,  1.97s/it]
 57%|█████▋    | 10632/18627 [5:54:55<4:31:34,  2.04s/it]
 57%|█████▋    | 10633/18627 [5:54:57<4:36:26,  2.07s/it]
 57%|█████▋    | 10634/18627 [5:55:00<4:38:53,  2.09s/it]
 57%|█████▋    | 10635/18627 [5:55:02<4:49:51,  2.18s/it]
 57%|█████▋    | 10636/18627 [5:55:03<4:10:13,  1.88s/it]
 57%|█████▋    | 10637/18627 [5:55:05<4:25:41,  2.00s/it]
 57%|█████▋    | 10638/18627 [5:55:08<4:32:59,  2.05s/it]
 57%|█████▋    | 10639/18627 [5:55:09<4:01:24,  1.81s/it]
 57%|█████▋    | 10640/18627 [5:55:11<4:26:32,  2.00s/it]
                                                         
{'loss': 1.1167, 'grad_norm': 11.95564079284668, 'learning_rate': 2.0473891513158376e-06, 'epoch': 0.57}
+
 57%|█████▋    | 10640/18627 [5:55:13<4:26:32,  2.00s/it]
 57%|█████▋    | 10641/18627 [5:55:15<5:46:00,  2.60s/it]
 57%|█████▋    | 10642/18627 [5:55:18<5:33:52,  2.51s/it]
 57%|█████▋    | 10643/18627 [5:55:19<4:41:42,  2.12s/it]
 57%|█████▋    | 10644/18627 [5:55:20<4:06:54,  1.86s/it]
 57%|█████▋    | 10645/18627 [5:55:22<4:19:22,  1.95s/it]
 57%|█████▋    | 10646/18627 [5:55:23<3:38:59,  1.65s/it]
 57%|█████▋    | 10647/18627 [5:55:25<3:58:31,  1.79s/it]
 57%|█████▋    | 10648/18627 [5:55:28<4:22:07,  1.97s/it]
 57%|█████▋    | 10649/18627 [5:55:30<4:30:23,  2.03s/it]
 57%|█████▋    | 10650/18627 [5:55:32<4:42:03,  2.12s/it]
                                                         
{'loss': 1.0362, 'grad_norm': 8.70516300201416, 'learning_rate': 2.043114768985534e-06, 'epoch': 0.57}
+
 57%|█████▋    | 10650/18627 [5:55:32<4:42:03,  2.12s/it]
 57%|█████▋    | 10651/18627 [5:55:34<4:50:51,  2.19s/it]
 57%|█████▋    | 10652/18627 [5:55:37<4:49:32,  2.18s/it]
 57%|█████▋    | 10653/18627 [5:55:39<4:49:47,  2.18s/it]
 57%|█████▋    | 10654/18627 [5:55:41<4:47:29,  2.16s/it]
 57%|█████▋    | 10655/18627 [5:55:43<4:45:36,  2.15s/it]
 57%|█████▋    | 10656/18627 [5:55:45<4:44:39,  2.14s/it]
 57%|█████▋    | 10657/18627 [5:55:47<4:44:12,  2.14s/it]
 57%|█████▋    | 10658/18627 [5:55:49<4:43:36,  2.14s/it]
 57%|█████▋    | 10659/18627 [5:55:52<4:44:23,  2.14s/it]
 57%|█████▋    | 10660/18627 [5:55:53<4:06:30,  1.86s/it]
                                                         
{'loss': 0.88, 'grad_norm': 15.42365837097168, 'learning_rate': 2.0388417679507143e-06, 'epoch': 0.57}
+
 57%|█████▋    | 10660/18627 [5:55:53<4:06:30,  1.86s/it]
 57%|█████▋    | 10661/18627 [5:55:55<4:17:44,  1.94s/it]
 57%|█████▋    | 10662/18627 [5:55:57<4:24:57,  2.00s/it]
 57%|█████▋    | 10663/18627 [5:55:59<4:31:22,  2.04s/it]
 57%|█████▋    | 10664/18627 [5:56:01<4:35:05,  2.07s/it]
 57%|█████▋    | 10665/18627 [5:56:03<3:59:55,  1.81s/it]
 57%|█████▋    | 10666/18627 [5:56:05<4:12:34,  1.90s/it]
 57%|█████▋    | 10667/18627 [5:56:06<3:44:28,  1.69s/it]
 57%|█████▋    | 10668/18627 [5:56:08<4:02:45,  1.83s/it]
 57%|█████▋    | 10669/18627 [5:56:10<4:15:22,  1.93s/it]
 57%|█████▋    | 10670/18627 [5:56:12<4:23:35,  1.99s/it]
                                                         
{'loss': 1.084, 'grad_norm': 8.55978775024414, 'learning_rate': 2.034570161129888e-06, 'epoch': 0.57}
+
 57%|█████▋    | 10670/18627 [5:56:12<4:23:35,  1.99s/it]
 57%|█████▋    | 10671/18627 [5:56:14<4:28:06,  2.02s/it]
 57%|█████▋    | 10672/18627 [5:56:17<4:31:48,  2.05s/it]
 57%|█████▋    | 10673/18627 [5:56:19<4:34:32,  2.07s/it]
 57%|█████▋    | 10674/18627 [5:56:21<4:36:25,  2.09s/it]
 57%|█████▋    | 10675/18627 [5:56:23<4:37:13,  2.09s/it]
 57%|█████▋    | 10676/18627 [5:56:25<4:38:23,  2.10s/it]
 57%|█████▋    | 10677/18627 [5:56:26<4:01:52,  1.83s/it]
 57%|█████▋    | 10678/18627 [5:56:28<4:13:02,  1.91s/it]
 57%|█████▋    | 10679/18627 [5:56:29<3:44:21,  1.69s/it]
 57%|█████▋    | 10680/18627 [5:56:31<3:24:35,  1.54s/it]
                                                         
{'loss': 1.1979, 'grad_norm': 14.646122932434082, 'learning_rate': 2.030299961437354e-06, 'epoch': 0.57}
+
 57%|█████▋    | 10680/18627 [5:56:31<3:24:35,  1.54s/it]
 57%|█████▋    | 10681/18627 [5:56:33<3:49:51,  1.74s/it]
 57%|█████▋    | 10682/18627 [5:56:35<4:06:40,  1.86s/it]
 57%|█████▋    | 10683/18627 [5:56:36<3:36:55,  1.64s/it]
 57%|█████▋    | 10684/18627 [5:56:38<3:56:08,  1.78s/it]
 57%|█████▋    | 10685/18627 [5:56:40<4:08:55,  1.88s/it]
 57%|█████▋    | 10686/18627 [5:56:42<4:18:11,  1.95s/it]
 57%|█████▋    | 10687/18627 [5:56:45<4:24:18,  2.00s/it]
 57%|█████▋    | 10688/18627 [5:56:47<4:28:11,  2.03s/it]
 57%|█████▋    | 10689/18627 [5:56:49<4:32:11,  2.06s/it]
 57%|█████▋    | 10690/18627 [5:56:51<4:34:16,  2.07s/it]
                                                         
{'loss': 0.7672, 'grad_norm': 6.80014181137085, 'learning_rate': 2.026031181783152e-06, 'epoch': 0.57}
+
 57%|█████▋    | 10690/18627 [5:56:51<4:34:16,  2.07s/it]
 57%|█████▋    | 10691/18627 [5:56:53<4:35:59,  2.09s/it]
 57%|█████▋    | 10692/18627 [5:56:55<4:36:06,  2.09s/it]
 57%|█████▋    | 10693/18627 [5:56:56<4:00:17,  1.82s/it]
 57%|█████▋    | 10694/18627 [5:56:58<4:12:26,  1.91s/it]
 57%|█████▋    | 10695/18627 [5:57:01<4:20:34,  1.97s/it]
 57%|█████▋    | 10696/18627 [5:57:03<4:26:57,  2.02s/it]
 57%|█████▋    | 10697/18627 [5:57:04<3:54:03,  1.77s/it]
 57%|█████▋    | 10698/18627 [5:57:06<4:08:11,  1.88s/it]
 57%|█████▋    | 10699/18627 [5:57:07<3:21:18,  1.52s/it]
 57%|█████▋    | 10700/18627 [5:57:09<3:45:19,  1.71s/it]
                                                         
{'loss': 1.0441, 'grad_norm': 5.60532808303833, 'learning_rate': 2.0217638350730325e-06, 'epoch': 0.57}
+
 57%|█████▋    | 10700/18627 [5:57:09<3:45:19,  1.71s/it]
 57%|█████▋    | 10701/18627 [5:57:11<4:01:42,  1.83s/it]
 57%|█████▋    | 10702/18627 [5:57:13<4:12:27,  1.91s/it]
 57%|█████▋    | 10703/18627 [5:57:15<4:20:11,  1.97s/it]
 57%|█████▋    | 10704/18627 [5:57:17<4:26:16,  2.02s/it]
 57%|█████▋    | 10705/18627 [5:57:19<4:29:22,  2.04s/it]
 57%|█████▋    | 10706/18627 [5:57:21<4:31:24,  2.06s/it]
 57%|█████▋    | 10707/18627 [5:57:24<4:34:06,  2.08s/it]
 57%|█████▋    | 10708/18627 [5:57:26<4:37:21,  2.10s/it]
 57%|█████▋    | 10709/18627 [5:57:28<4:38:13,  2.11s/it]
 57%|█████▋    | 10710/18627 [5:57:29<4:01:35,  1.83s/it]
                                                         
{'loss': 0.808, 'grad_norm': 14.101771354675293, 'learning_rate': 2.017497934208411e-06, 'epoch': 0.57}
+
 57%|█████▋    | 10710/18627 [5:57:29<4:01:35,  1.83s/it]
 58%|█████▊    | 10711/18627 [5:57:31<4:12:31,  1.91s/it]
 58%|█████▊    | 10712/18627 [5:57:33<4:21:19,  1.98s/it]
 58%|█████▊    | 10713/18627 [5:57:35<4:27:01,  2.02s/it]
 58%|█████▊    | 10714/18627 [5:57:38<4:30:25,  2.05s/it]
 58%|█████▊    | 10715/18627 [5:57:39<3:56:52,  1.80s/it]
 58%|█████▊    | 10716/18627 [5:57:40<3:28:56,  1.58s/it]
 58%|█████▊    | 10717/18627 [5:57:42<3:50:24,  1.75s/it]
 58%|█████▊    | 10718/18627 [5:57:44<4:04:06,  1.85s/it]
 58%|█████▊    | 10719/18627 [5:57:46<4:15:11,  1.94s/it]
 58%|█████▊    | 10720/18627 [5:57:48<4:22:47,  1.99s/it]
                                                         
{'loss': 1.0105, 'grad_norm': 5.3644280433654785, 'learning_rate': 2.013233492086334e-06, 'epoch': 0.58}
+
 58%|█████▊    | 10720/18627 [5:57:48<4:22:47,  1.99s/it]
 58%|█████▊    | 10721/18627 [5:57:49<3:51:02,  1.75s/it]
 58%|█████▊    | 10722/18627 [5:57:52<4:04:24,  1.86s/it]
 58%|█████▊    | 10723/18627 [5:57:54<4:14:28,  1.93s/it]
 58%|█████▊    | 10724/18627 [5:57:56<4:21:08,  1.98s/it]
 58%|█████▊    | 10725/18627 [5:57:58<4:26:09,  2.02s/it]
 58%|█████▊    | 10726/18627 [5:58:00<4:32:38,  2.07s/it]
 58%|█████▊    | 10727/18627 [5:58:01<3:58:23,  1.81s/it]
 58%|█████▊    | 10728/18627 [5:58:02<3:30:01,  1.60s/it]
 58%|█████▊    | 10729/18627 [5:58:04<3:14:15,  1.48s/it]
 58%|█████▊    | 10730/18627 [5:58:06<3:39:40,  1.67s/it]
                                                         
{'loss': 1.4253, 'grad_norm': 7.483492851257324, 'learning_rate': 2.0089705215994357e-06, 'epoch': 0.58}
+
 58%|█████▊    | 10730/18627 [5:58:06<3:39:40,  1.67s/it]
 58%|█████▊    | 10731/18627 [5:58:08<3:58:09,  1.81s/it]
 58%|█████▊    | 10732/18627 [5:58:10<4:10:44,  1.91s/it]
 58%|█████▊    | 10733/18627 [5:58:11<3:23:03,  1.54s/it]
 58%|█████▊    | 10734/18627 [5:58:13<3:46:49,  1.72s/it]
 58%|█████▊    | 10735/18627 [5:58:15<4:01:59,  1.84s/it]
 58%|█████▊    | 10736/18627 [5:58:17<4:14:11,  1.93s/it]
 58%|█████▊    | 10737/18627 [5:58:19<4:21:51,  1.99s/it]
 58%|█████▊    | 10738/18627 [5:58:21<4:27:08,  2.03s/it]
 58%|█████▊    | 10739/18627 [5:58:23<4:32:09,  2.07s/it]
 58%|█████▊    | 10740/18627 [5:58:26<4:35:29,  2.10s/it]
                                                         
{'loss': 0.6524, 'grad_norm': 5.044948577880859, 'learning_rate': 2.0047090356359035e-06, 'epoch': 0.58}
+
 58%|█████▊    | 10740/18627 [5:58:26<4:35:29,  2.10s/it]
 58%|█████▊    | 10741/18627 [5:58:28<4:37:53,  2.11s/it]
 58%|█████▊    | 10742/18627 [5:58:30<4:38:32,  2.12s/it]
 58%|█████▊    | 10743/18627 [5:58:32<4:39:16,  2.13s/it]
 58%|█████▊    | 10744/18627 [5:58:34<4:39:55,  2.13s/it]
 58%|█████▊    | 10745/18627 [5:58:35<4:02:34,  1.85s/it]
 58%|█████▊    | 10746/18627 [5:58:38<4:14:45,  1.94s/it]
 58%|█████▊    | 10747/18627 [5:58:40<4:24:30,  2.01s/it]
 58%|█████▊    | 10748/18627 [5:58:42<4:30:27,  2.06s/it]
 58%|█████▊    | 10749/18627 [5:58:44<4:32:28,  2.08s/it]
 58%|█████▊    | 10750/18627 [5:58:45<3:57:51,  1.81s/it]
                                                         
{'loss': 1.0946, 'grad_norm': 15.486733436584473, 'learning_rate': 2.0004490470794336e-06, 'epoch': 0.58}
+
 58%|█████▊    | 10750/18627 [5:58:45<3:57:51,  1.81s/it]
 58%|█████▊    | 10751/18627 [5:58:47<4:09:40,  1.90s/it]
 58%|█████▊    | 10752/18627 [5:58:49<4:17:29,  1.96s/it]
 58%|█████▊    | 10753/18627 [5:58:52<4:23:28,  2.01s/it]
 58%|█████▊    | 10754/18627 [5:58:54<4:26:59,  2.03s/it]
 58%|█████▊    | 10755/18627 [5:58:56<4:30:40,  2.06s/it]
 58%|█████▊    | 10756/18627 [5:58:57<3:55:52,  1.80s/it]
 58%|█████▊    | 10757/18627 [5:58:59<4:08:50,  1.90s/it]
 58%|█████▊    | 10758/18627 [5:59:01<4:17:02,  1.96s/it]
 58%|█████▊    | 10759/18627 [5:59:02<3:47:06,  1.73s/it]
 58%|█████▊    | 10760/18627 [5:59:05<4:04:06,  1.86s/it]
                                                         
{'loss': 1.0529, 'grad_norm': 6.127838134765625, 'learning_rate': 1.996190568809199e-06, 'epoch': 0.58}
+
 58%|█████▊    | 10760/18627 [5:59:05<4:04:06,  1.86s/it]
 58%|█████▊    | 10761/18627 [5:59:05<3:18:23,  1.51s/it]
 58%|█████▊    | 10762/18627 [5:59:07<3:42:18,  1.70s/it]
 58%|█████▊    | 10763/18627 [5:59:09<3:58:19,  1.82s/it]
 58%|█████▊    | 10764/18627 [5:59:12<4:10:18,  1.91s/it]
 58%|█████▊    | 10765/18627 [5:59:14<4:18:42,  1.97s/it]
 58%|█████▊    | 10766/18627 [5:59:14<3:29:09,  1.60s/it]
 58%|█████▊    | 10767/18627 [5:59:15<2:53:41,  1.33s/it]
 58%|█████▊    | 10768/18627 [5:59:17<3:25:57,  1.57s/it]
 58%|█████▊    | 10769/18627 [5:59:19<3:48:45,  1.75s/it]
 58%|█████▊    | 10770/18627 [5:59:22<4:03:44,  1.86s/it]
                                                         
{'loss': 0.5933, 'grad_norm': 5.216161727905273, 'learning_rate': 1.9919336136998027e-06, 'epoch': 0.58}
+
 58%|█████▊    | 10770/18627 [5:59:22<4:03:44,  1.86s/it]
 58%|█████▊    | 10771/18627 [5:59:24<4:13:15,  1.93s/it]
 58%|█████▊    | 10772/18627 [5:59:26<4:21:19,  2.00s/it]
 58%|█████▊    | 10773/18627 [5:59:28<4:25:58,  2.03s/it]
 58%|█████▊    | 10774/18627 [5:59:30<4:31:26,  2.07s/it]
 58%|█████▊    | 10775/18627 [5:59:32<4:32:56,  2.09s/it]
 58%|█████▊    | 10776/18627 [5:59:34<4:34:00,  2.09s/it]
 58%|█████▊    | 10777/18627 [5:59:36<3:58:57,  1.83s/it]
 58%|█████▊    | 10778/18627 [5:59:38<4:10:21,  1.91s/it]
 58%|█████▊    | 10779/18627 [5:59:40<4:18:30,  1.98s/it]
 58%|█████▊    | 10780/18627 [5:59:42<4:24:18,  2.02s/it]
                                                         
{'loss': 0.8161, 'grad_norm': 7.068978786468506, 'learning_rate': 1.987678194621246e-06, 'epoch': 0.58}
+
 58%|█████▊    | 10780/18627 [5:59:42<4:24:18,  2.02s/it]
 58%|█████▊    | 10781/18627 [5:59:43<3:45:10,  1.72s/it]
 58%|█████▊    | 10782/18627 [5:59:44<3:24:35,  1.56s/it]
 58%|█████▊    | 10783/18627 [5:59:46<3:45:39,  1.73s/it]
 58%|█████▊    | 10784/18627 [5:59:48<4:01:27,  1.85s/it]
 58%|█████▊    | 10785/18627 [5:59:50<3:35:32,  1.65s/it]
 58%|█████▊    | 10786/18627 [5:59:52<3:52:52,  1.78s/it]
 58%|█████▊    | 10787/18627 [5:59:54<4:06:20,  1.89s/it]
 58%|█████▊    | 10788/18627 [5:59:56<4:14:24,  1.95s/it]
 58%|█████▊    | 10789/18627 [5:59:58<4:21:21,  2.00s/it]
 58%|█████▊    | 10790/18627 [6:00:00<4:26:09,  2.04s/it]
                                                         
{'loss': 1.3624, 'grad_norm': 5.224578380584717, 'learning_rate': 1.983424324438883e-06, 'epoch': 0.58}
+
 58%|█████▊    | 10790/18627 [6:00:00<4:26:09,  2.04s/it]
 58%|█████▊    | 10791/18627 [6:00:02<4:29:09,  2.06s/it]
 58%|█████▊    | 10792/18627 [6:00:04<4:31:02,  2.08s/it]
 58%|█████▊    | 10793/18627 [6:00:06<4:32:34,  2.09s/it]
 58%|█████▊    | 10794/18627 [6:00:09<4:32:54,  2.09s/it]
 58%|█████▊    | 10795/18627 [6:00:11<4:34:29,  2.10s/it]
 58%|█████▊    | 10796/18627 [6:00:13<4:34:57,  2.11s/it]
 58%|█████▊    | 10797/18627 [6:00:15<4:36:10,  2.12s/it]
 58%|█████▊    | 10798/18627 [6:00:17<4:36:53,  2.12s/it]
 58%|█████▊    | 10799/18627 [6:00:18<3:40:55,  1.69s/it]
 58%|█████▊    | 10800/18627 [6:00:20<3:58:03,  1.82s/it]
                                                         
{'loss': 0.6227, 'grad_norm': 4.429221153259277, 'learning_rate': 1.979172016013389e-06, 'epoch': 0.58}
+
 58%|█████▊    | 10800/18627 [6:00:20<3:58:03,  1.82s/it]
 58%|█████▊    | 10801/18627 [6:00:21<3:33:40,  1.64s/it]
 58%|█████▊    | 10802/18627 [6:00:23<3:52:47,  1.79s/it]
 58%|█████▊    | 10803/18627 [6:00:25<4:06:35,  1.89s/it]
 58%|█████▊    | 10804/18627 [6:00:27<4:15:23,  1.96s/it]
 58%|█████▊    | 10805/18627 [6:00:30<4:20:48,  2.00s/it]
 58%|█████▊    | 10806/18627 [6:00:32<4:24:52,  2.03s/it]
 58%|█████▊    | 10807/18627 [6:00:34<4:27:52,  2.06s/it]
 58%|█████▊    | 10808/18627 [6:00:35<3:50:12,  1.77s/it]
 58%|█████▊    | 10809/18627 [6:00:37<4:04:13,  1.87s/it]
 58%|█████▊    | 10810/18627 [6:00:39<4:13:16,  1.94s/it]
                                                         
{'loss': 0.942, 'grad_norm': 7.947227478027344, 'learning_rate': 1.974921282200714e-06, 'epoch': 0.58}
+
 58%|█████▊    | 10810/18627 [6:00:39<4:13:16,  1.94s/it]
 58%|█████▊    | 10811/18627 [6:00:41<4:20:32,  2.00s/it]
 58%|█████▊    | 10812/18627 [6:00:43<4:25:12,  2.04s/it]
 58%|█████▊    | 10813/18627 [6:00:45<4:29:14,  2.07s/it]
 58%|█████▊    | 10814/18627 [6:00:48<4:31:15,  2.08s/it]
 58%|█████▊    | 10815/18627 [6:00:50<4:32:24,  2.09s/it]
 58%|█████▊    | 10816/18627 [6:00:52<4:34:07,  2.11s/it]
 58%|█████▊    | 10817/18627 [6:00:53<3:58:17,  1.83s/it]
 58%|█████▊    | 10818/18627 [6:00:55<4:09:10,  1.91s/it]
 58%|█████▊    | 10819/18627 [6:00:57<4:16:55,  1.97s/it]
 58%|█████▊    | 10820/18627 [6:00:59<4:22:19,  2.02s/it]
                                                         
{'loss': 0.8019, 'grad_norm': 5.985844135284424, 'learning_rate': 1.970672135852051e-06, 'epoch': 0.58}
+
 58%|█████▊    | 10820/18627 [6:00:59<4:22:19,  2.02s/it]
 58%|█████▊    | 10821/18627 [6:01:00<3:30:51,  1.62s/it]
 58%|█████▊    | 10822/18627 [6:01:02<3:50:24,  1.77s/it]
 58%|█████▊    | 10823/18627 [6:01:03<3:27:29,  1.60s/it]
 58%|█████▊    | 10824/18627 [6:01:05<3:47:11,  1.75s/it]
 58%|█████▊    | 10825/18627 [6:01:08<4:02:02,  1.86s/it]
 58%|█████▊    | 10826/18627 [6:01:09<3:35:45,  1.66s/it]
 58%|█████▊    | 10827/18627 [6:01:11<3:53:09,  1.79s/it]
 58%|█████▊    | 10828/18627 [6:01:13<4:04:58,  1.88s/it]
 58%|█████▊    | 10829/18627 [6:01:15<4:13:07,  1.95s/it]
 58%|█████▊    | 10830/18627 [6:01:17<4:19:04,  1.99s/it]
                                                         
{'loss': 1.098, 'grad_norm': 9.886977195739746, 'learning_rate': 1.966424589813789e-06, 'epoch': 0.58}
+
 58%|█████▊    | 10830/18627 [6:01:17<4:19:04,  1.99s/it]
 58%|█████▊    | 10831/18627 [6:01:19<4:23:45,  2.03s/it]
 58%|█████▊    | 10832/18627 [6:01:21<4:26:36,  2.05s/it]
 58%|█████▊    | 10833/18627 [6:01:24<4:29:40,  2.08s/it]
 58%|█████▊    | 10834/18627 [6:01:26<4:31:31,  2.09s/it]
 58%|█████▊    | 10835/18627 [6:01:26<3:37:29,  1.67s/it]
 58%|█████▊    | 10836/18627 [6:01:29<3:54:30,  1.81s/it]
 58%|█████▊    | 10837/18627 [6:01:31<4:07:08,  1.90s/it]
 58%|█████▊    | 10838/18627 [6:01:33<4:15:41,  1.97s/it]
 58%|█████▊    | 10839/18627 [6:01:34<3:45:08,  1.73s/it]
 58%|█████▊    | 10840/18627 [6:01:36<3:59:34,  1.85s/it]
                                                         
{'loss': 0.772, 'grad_norm': 6.056911468505859, 'learning_rate': 1.9621786569274847e-06, 'epoch': 0.58}
+
 58%|█████▊    | 10840/18627 [6:01:36<3:59:34,  1.85s/it]
 58%|█████▊    | 10841/18627 [6:01:37<3:34:08,  1.65s/it]
 58%|█████▊    | 10842/18627 [6:01:39<3:51:53,  1.79s/it]
 58%|█████▊    | 10843/18627 [6:01:41<3:28:48,  1.61s/it]
 58%|█████▊    | 10844/18627 [6:01:43<3:48:07,  1.76s/it]
 58%|█████▊    | 10845/18627 [6:01:45<4:01:46,  1.86s/it]
 58%|█████▊    | 10846/18627 [6:01:46<3:35:44,  1.66s/it]
 58%|█████▊    | 10847/18627 [6:01:48<3:53:32,  1.80s/it]
 58%|█████▊    | 10848/18627 [6:01:50<4:06:02,  1.90s/it]
 58%|█████▊    | 10849/18627 [6:01:52<4:14:23,  1.96s/it]
 58%|█████▊    | 10850/18627 [6:01:54<4:20:57,  2.01s/it]
                                                         
{'loss': 1.2413, 'grad_norm': 8.438995361328125, 'learning_rate': 1.9579343500298124e-06, 'epoch': 0.58}
+
 58%|█████▊    | 10850/18627 [6:01:54<4:20:57,  2.01s/it]
 58%|█████▊    | 10851/18627 [6:01:57<4:25:02,  2.05s/it]
 58%|█████▊    | 10852/18627 [6:01:58<3:51:43,  1.79s/it]
 58%|█████▊    | 10853/18627 [6:02:00<4:04:24,  1.89s/it]
 58%|█████▊    | 10854/18627 [6:02:02<4:13:15,  1.95s/it]
 58%|█████▊    | 10855/18627 [6:02:04<4:19:26,  2.00s/it]
 58%|█████▊    | 10856/18627 [6:02:05<3:44:33,  1.73s/it]
 58%|█████▊    | 10857/18627 [6:02:07<3:59:10,  1.85s/it]
 58%|█████▊    | 10858/18627 [6:02:09<4:09:35,  1.93s/it]
 58%|█████▊    | 10859/18627 [6:02:11<3:40:42,  1.70s/it]
 58%|█████▊    | 10860/18627 [6:02:13<3:56:02,  1.82s/it]
                                                         
{'loss': 1.3006, 'grad_norm': 6.900991439819336, 'learning_rate': 1.953691681952534e-06, 'epoch': 0.58}
+
 58%|█████▊    | 10860/18627 [6:02:13<3:56:02,  1.82s/it]
 58%|█████▊    | 10861/18627 [6:02:15<4:06:29,  1.90s/it]
 58%|█████▊    | 10862/18627 [6:02:17<4:14:49,  1.97s/it]
 58%|█████▊    | 10863/18627 [6:02:19<4:20:49,  2.02s/it]
 58%|█████▊    | 10864/18627 [6:02:21<4:24:44,  2.05s/it]
 58%|█████▊    | 10865/18627 [6:02:23<4:28:22,  2.07s/it]
 58%|█████▊    | 10866/18627 [6:02:25<4:30:19,  2.09s/it]
 58%|█████▊    | 10867/18627 [6:02:28<4:32:06,  2.10s/it]
 58%|█████▊    | 10868/18627 [6:02:30<4:33:06,  2.11s/it]
 58%|█████▊    | 10869/18627 [6:02:31<3:57:07,  1.83s/it]
 58%|█████▊    | 10870/18627 [6:02:33<4:09:04,  1.93s/it]
                                                         
{'loss': 0.7948, 'grad_norm': 5.983034133911133, 'learning_rate': 1.949450665522454e-06, 'epoch': 0.58}
+
 58%|█████▊    | 10870/18627 [6:02:33<4:09:04,  1.93s/it]
 58%|█████▊    | 10871/18627 [6:02:34<3:40:19,  1.70s/it]
 58%|█████▊    | 10872/18627 [6:02:36<3:56:20,  1.83s/it]
 58%|█████▊    | 10873/18627 [6:02:38<4:06:28,  1.91s/it]
 58%|█████▊    | 10874/18627 [6:02:41<4:14:12,  1.97s/it]
 58%|█████▊    | 10875/18627 [6:02:42<3:43:54,  1.73s/it]
 58%|█████▊    | 10876/18627 [6:02:44<3:58:32,  1.85s/it]
 58%|█████▊    | 10877/18627 [6:02:46<4:08:18,  1.92s/it]
 58%|█████▊    | 10878/18627 [6:02:48<4:15:39,  1.98s/it]
 58%|█████▊    | 10879/18627 [6:02:50<4:21:02,  2.02s/it]
 58%|█████▊    | 10880/18627 [6:02:52<4:27:19,  2.07s/it]
                                                         
{'loss': 1.0413, 'grad_norm': 5.9684929847717285, 'learning_rate': 1.9452113135613866e-06, 'epoch': 0.58}
+
 58%|█████▊    | 10880/18627 [6:02:52<4:27:19,  2.07s/it]
 58%|█████▊    | 10881/18627 [6:02:54<3:53:58,  1.81s/it]
 58%|█████▊    | 10882/18627 [6:02:56<4:06:23,  1.91s/it]
 58%|█████▊    | 10883/18627 [6:02:58<4:15:02,  1.98s/it]
 58%|█████▊    | 10884/18627 [6:03:00<4:20:50,  2.02s/it]
 58%|█████▊    | 10885/18627 [6:03:02<4:24:59,  2.05s/it]
 58%|█████▊    | 10886/18627 [6:03:03<3:51:10,  1.79s/it]
 58%|█████▊    | 10887/18627 [6:03:05<4:03:38,  1.89s/it]
 58%|█████▊    | 10888/18627 [6:03:07<4:11:42,  1.95s/it]
 58%|█████▊    | 10889/18627 [6:03:09<3:42:35,  1.73s/it]
 58%|█████▊    | 10890/18627 [6:03:11<3:57:12,  1.84s/it]
                                                         
{'loss': 1.129, 'grad_norm': 5.101035118103027, 'learning_rate': 1.9409736388861117e-06, 'epoch': 0.58}
+
 58%|█████▊    | 10890/18627 [6:03:11<3:57:12,  1.84s/it]
 58%|█████▊    | 10891/18627 [6:03:12<3:32:32,  1.65s/it]
 58%|█████▊    | 10892/18627 [6:03:14<3:50:26,  1.79s/it]
 58%|█████▊    | 10893/18627 [6:03:16<4:03:12,  1.89s/it]
 58%|█████▊    | 10894/18627 [6:03:18<4:11:35,  1.95s/it]
 58%|█████▊    | 10895/18627 [6:03:20<4:17:26,  2.00s/it]
 58%|█████▊    | 10896/18627 [6:03:23<4:22:22,  2.04s/it]
 59%|█████▊    | 10897/18627 [6:03:24<3:50:01,  1.79s/it]
 59%|█████▊    | 10898/18627 [6:03:26<4:02:58,  1.89s/it]
 59%|█████▊    | 10899/18627 [6:03:28<4:11:41,  1.95s/it]
 59%|█████▊    | 10900/18627 [6:03:30<4:17:50,  2.00s/it]
                                                         
{'loss': 1.0269, 'grad_norm': 7.34773063659668, 'learning_rate': 1.9367376543083395e-06, 'epoch': 0.59}
+
 59%|█████▊    | 10900/18627 [6:03:30<4:17:50,  2.00s/it]
 59%|█████▊    | 10901/18627 [6:03:32<4:22:33,  2.04s/it]
 59%|█████▊    | 10902/18627 [6:03:34<4:26:10,  2.07s/it]
 59%|█████▊    | 10903/18627 [6:03:36<4:28:15,  2.08s/it]
 59%|█████▊    | 10904/18627 [6:03:39<4:28:27,  2.09s/it]
 59%|█████▊    | 10905/18627 [6:03:41<4:29:27,  2.09s/it]
 59%|█████▊    | 10906/18627 [6:03:42<3:54:14,  1.82s/it]
 59%|█████▊    | 10907/18627 [6:03:44<4:05:16,  1.91s/it]
 59%|█████▊    | 10908/18627 [6:03:45<3:33:42,  1.66s/it]
 59%|█████▊    | 10909/18627 [6:03:47<3:51:17,  1.80s/it]
 59%|█████▊    | 10910/18627 [6:03:49<4:03:41,  1.89s/it]
                                                         
{'loss': 0.9482, 'grad_norm': 9.493403434753418, 'learning_rate': 1.9325033726346683e-06, 'epoch': 0.59}
+
 59%|█████▊    | 10910/18627 [6:03:49<4:03:41,  1.89s/it]
 59%|█████▊    | 10911/18627 [6:03:51<4:12:27,  1.96s/it]
 59%|█████▊    | 10912/18627 [6:03:53<3:42:08,  1.73s/it]
 59%|█████▊    | 10913/18627 [6:03:54<3:21:44,  1.57s/it]
 59%|█████▊    | 10914/18627 [6:03:55<3:07:39,  1.46s/it]
 59%|█████▊    | 10915/18627 [6:03:57<3:32:43,  1.66s/it]
 59%|█████▊    | 10916/18627 [6:03:58<3:14:49,  1.52s/it]
 59%|█████▊    | 10917/18627 [6:04:00<3:38:17,  1.70s/it]
 59%|█████▊    | 10918/18627 [6:04:03<3:54:47,  1.83s/it]
 59%|█████▊    | 10919/18627 [6:04:05<4:08:33,  1.93s/it]
 59%|█████▊    | 10920/18627 [6:04:07<4:15:44,  1.99s/it]
                                                         
{'loss': 1.3943, 'grad_norm': 5.407600402832031, 'learning_rate': 1.9282708066665514e-06, 'epoch': 0.59}
+
 59%|█████▊    | 10920/18627 [6:04:07<4:15:44,  1.99s/it]
 59%|█████▊    | 10921/18627 [6:04:09<4:20:24,  2.03s/it]
 59%|█████▊    | 10922/18627 [6:04:11<4:24:45,  2.06s/it]
 59%|█████▊    | 10923/18627 [6:04:13<4:27:37,  2.08s/it]
 59%|█████▊    | 10924/18627 [6:04:15<4:29:09,  2.10s/it]
 59%|█████▊    | 10925/18627 [6:04:18<4:33:03,  2.13s/it]
 59%|█████▊    | 10926/18627 [6:04:19<3:54:31,  1.83s/it]
 59%|█████▊    | 10927/18627 [6:04:21<4:08:10,  1.93s/it]
 59%|█████▊    | 10928/18627 [6:04:23<4:14:57,  1.99s/it]
 59%|█████▊    | 10929/18627 [6:04:25<4:20:00,  2.03s/it]
 59%|█████▊    | 10930/18627 [6:04:27<4:23:09,  2.05s/it]
                                                         
{'loss': 0.8727, 'grad_norm': 8.104090690612793, 'learning_rate': 1.9240399692002527e-06, 'epoch': 0.59}
+
 59%|█████▊    | 10930/18627 [6:04:27<4:23:09,  2.05s/it]
 59%|█████▊    | 10931/18627 [6:04:29<4:25:49,  2.07s/it]
 59%|█████▊    | 10932/18627 [6:04:31<4:27:09,  2.08s/it]
 59%|█████▊    | 10933/18627 [6:04:33<3:52:38,  1.81s/it]
 59%|█████▊    | 10934/18627 [6:04:35<4:04:20,  1.91s/it]
 59%|█████▊    | 10935/18627 [6:04:36<3:36:47,  1.69s/it]
 59%|█████▊    | 10936/18627 [6:04:37<3:17:29,  1.54s/it]
 59%|█████▊    | 10937/18627 [6:04:40<3:59:39,  1.87s/it]
 59%|█████▊    | 10938/18627 [6:04:42<4:08:16,  1.94s/it]
 59%|█████▊    | 10939/18627 [6:04:43<3:39:18,  1.71s/it]
 59%|█████▊    | 10940/18627 [6:04:45<3:55:28,  1.84s/it]
                                                         
{'loss': 1.7202, 'grad_norm': 6.991078853607178, 'learning_rate': 1.91981087302681e-06, 'epoch': 0.59}
+
 59%|█████▊    | 10940/18627 [6:04:45<3:55:28,  1.84s/it]
 59%|█████▊    | 10941/18627 [6:04:47<4:06:38,  1.93s/it]
 59%|█████▊    | 10942/18627 [6:04:49<4:14:39,  1.99s/it]
 59%|█████▊    | 10943/18627 [6:04:52<4:19:30,  2.03s/it]
 59%|█████▉    | 10944/18627 [6:04:54<4:22:37,  2.05s/it]
 59%|█████▉    | 10945/18627 [6:04:56<4:25:05,  2.07s/it]
 59%|█████▉    | 10946/18627 [6:04:58<4:26:35,  2.08s/it]
 59%|█████▉    | 10947/18627 [6:05:00<4:27:38,  2.09s/it]
 59%|█████▉    | 10948/18627 [6:05:01<3:53:44,  1.83s/it]
 59%|█████▉    | 10949/18627 [6:05:02<3:29:09,  1.63s/it]
 59%|█████▉    | 10950/18627 [6:05:04<3:11:51,  1.50s/it]
                                                         
{'loss': 1.3177, 'grad_norm': 13.489028930664062, 'learning_rate': 1.9155835309319993e-06, 'epoch': 0.59}
+
 59%|█████▉    | 10950/18627 [6:05:04<3:11:51,  1.50s/it]
 59%|█████▉    | 10951/18627 [6:05:06<3:35:44,  1.69s/it]
 59%|█████▉    | 10952/18627 [6:05:08<3:52:01,  1.81s/it]
 59%|█████▉    | 10953/18627 [6:05:09<3:27:56,  1.63s/it]
 59%|█████▉    | 10954/18627 [6:05:11<3:46:24,  1.77s/it]
 59%|█████▉    | 10955/18627 [6:05:12<3:24:01,  1.60s/it]
 59%|█████▉    | 10956/18627 [6:05:14<3:08:30,  1.47s/it]
 59%|█████▉    | 10957/18627 [6:05:16<3:32:47,  1.66s/it]
 59%|█████▉    | 10958/18627 [6:05:18<3:49:30,  1.80s/it]
 59%|█████▉    | 10959/18627 [6:05:20<4:02:06,  1.89s/it]
 59%|█████▉    | 10960/18627 [6:05:22<4:10:17,  1.96s/it]
                                                         
{'loss': 1.2789, 'grad_norm': 6.147861957550049, 'learning_rate': 1.911357955696291e-06, 'epoch': 0.59}
+
 59%|█████▉    | 10960/18627 [6:05:22<4:10:17,  1.96s/it]
 59%|█████▉    | 10961/18627 [6:05:24<4:15:50,  2.00s/it]
 59%|█████▉    | 10962/18627 [6:05:25<3:44:50,  1.76s/it]
 59%|█████▉    | 10963/18627 [6:05:27<3:58:50,  1.87s/it]
 59%|█████▉    | 10964/18627 [6:05:29<4:08:14,  1.94s/it]
 59%|█████▉    | 10965/18627 [6:05:32<4:13:48,  1.99s/it]
 59%|█████▉    | 10966/18627 [6:05:33<3:43:21,  1.75s/it]
 59%|█████▉    | 10967/18627 [6:05:33<3:03:49,  1.44s/it]
 59%|█████▉    | 10968/18627 [6:05:36<3:29:55,  1.64s/it]
 59%|█████▉    | 10969/18627 [6:05:38<3:48:00,  1.79s/it]
 59%|█████▉    | 10970/18627 [6:05:39<3:25:09,  1.61s/it]
                                                         
{'loss': 1.3204, 'grad_norm': 15.281700134277344, 'learning_rate': 1.9071341600948145e-06, 'epoch': 0.59}
+
 59%|█████▉    | 10970/18627 [6:05:39<3:25:09,  1.61s/it]
 59%|█████▉    | 10971/18627 [6:05:41<3:44:33,  1.76s/it]
 59%|█████▉    | 10972/18627 [6:05:43<3:58:07,  1.87s/it]
 59%|█████▉    | 10973/18627 [6:05:45<4:07:06,  1.94s/it]
 59%|█████▉    | 10974/18627 [6:05:47<4:13:06,  1.98s/it]
 59%|█████▉    | 10975/18627 [6:05:49<3:42:44,  1.75s/it]
 59%|█████▉    | 10976/18627 [6:05:51<3:56:31,  1.85s/it]
 59%|█████▉    | 10977/18627 [6:05:53<4:06:30,  1.93s/it]
 59%|█████▉    | 10978/18627 [6:05:55<4:13:54,  1.99s/it]
 59%|█████▉    | 10979/18627 [6:05:56<3:43:05,  1.75s/it]
 59%|█████▉    | 10980/18627 [6:05:58<3:56:35,  1.86s/it]
                                                         
{'loss': 1.0566, 'grad_norm': 5.820309162139893, 'learning_rate': 1.902912156897318e-06, 'epoch': 0.59}
+
 59%|█████▉    | 10980/18627 [6:05:58<3:56:35,  1.86s/it]
 59%|█████▉    | 10981/18627 [6:05:59<3:30:49,  1.65s/it]
 59%|█████▉    | 10982/18627 [6:06:01<3:12:57,  1.51s/it]
 59%|█████▉    | 10983/18627 [6:06:03<3:36:18,  1.70s/it]
 59%|█████▉    | 10984/18627 [6:06:04<3:16:39,  1.54s/it]
 59%|█████▉    | 10985/18627 [6:06:06<3:40:27,  1.73s/it]
 59%|█████▉    | 10986/18627 [6:06:08<3:55:11,  1.85s/it]
 59%|█████▉    | 10987/18627 [6:06:10<4:05:30,  1.93s/it]
 59%|█████▉    | 10988/18627 [6:06:12<4:12:59,  1.99s/it]
 59%|█████▉    | 10989/18627 [6:06:15<4:18:20,  2.03s/it]
 59%|█████▉    | 10990/18627 [6:06:17<4:21:47,  2.06s/it]
                                                         
{'loss': 1.1956, 'grad_norm': 4.329510688781738, 'learning_rate': 1.8986919588681338e-06, 'epoch': 0.59}
+
 59%|█████▉    | 10990/18627 [6:06:17<4:21:47,  2.06s/it]
 59%|█████▉    | 10991/18627 [6:06:19<4:23:53,  2.07s/it]
 59%|█████▉    | 10992/18627 [6:06:21<4:25:11,  2.08s/it]
 59%|█████▉    | 10993/18627 [6:06:22<3:51:20,  1.82s/it]
 59%|█████▉    | 10994/18627 [6:06:24<4:03:21,  1.91s/it]
 59%|█████▉    | 10995/18627 [6:06:25<3:16:39,  1.55s/it]
 59%|█████▉    | 10996/18627 [6:06:26<3:02:54,  1.44s/it]
 59%|█████▉    | 10997/18627 [6:06:28<3:28:30,  1.64s/it]
 59%|█████▉    | 10998/18627 [6:06:30<3:46:55,  1.78s/it]
 59%|█████▉    | 10999/18627 [6:06:32<3:59:59,  1.89s/it]
 59%|█████▉    | 11000/18627 [6:06:35<4:08:29,  1.95s/it]
                                                         
{'loss': 1.0226, 'grad_norm': 5.589778423309326, 'learning_rate': 1.8944735787661332e-06, 'epoch': 0.59}
+
 59%|█████▉    | 11000/18627 [6:06:35<4:08:29,  1.95s/it]/home/zhengduo/workspace/LLaMA-Factory/src/qwen_vl/model/modeling_qwen2_5_vl.py:1979: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
+  with torch.cuda.amp.autocast(dtype=dtype):
+/home/zhengduo/miniconda3/envs/vgllm/lib/python3.10/site-packages/torch/utils/checkpoint.py:87: UserWarning: None of the inputs have requires_grad=True. Gradients will be None
+  warnings.warn(
+
 59%|█████▉    | 11001/18627 [6:07:17<30:01:06, 14.17s/it]
 59%|█████▉    | 11002/18627 [6:07:19<22:21:06, 10.55s/it]
 59%|█████▉    | 11003/18627 [6:07:21<17:00:06,  8.03s/it]
 59%|█████▉    | 11004/18627 [6:07:24<13:14:59,  6.26s/it]
 59%|█████▉    | 11005/18627 [6:07:25<10:03:26,  4.75s/it]
 59%|█████▉    | 11006/18627 [6:07:27<8:24:46,  3.97s/it] 
 59%|█████▉    | 11007/18627 [6:07:29<7:14:57,  3.42s/it]
 59%|█████▉    | 11008/18627 [6:07:31<6:26:28,  3.04s/it]
 59%|█████▉    | 11009/18627 [6:07:33<5:51:43,  2.77s/it]
 59%|█████▉    | 11010/18627 [6:07:35<4:51:30,  2.30s/it]
                                                         
{'loss': 1.078, 'grad_norm': 14.254372596740723, 'learning_rate': 1.890257029344693e-06, 'epoch': 0.59}
+
 59%|█████▉    | 11010/18627 [6:07:35<4:51:30,  2.30s/it]
 59%|█████▉    | 11011/18627 [6:07:37<4:44:47,  2.24s/it]
 59%|█████▉    | 11012/18627 [6:07:39<4:40:52,  2.21s/it]
 59%|█████▉    | 11013/18627 [6:07:41<4:39:51,  2.21s/it]
 59%|█████▉    | 11014/18627 [6:07:43<4:36:52,  2.18s/it]
 59%|█████▉    | 11015/18627 [6:07:45<4:34:02,  2.16s/it]
 59%|█████▉    | 11016/18627 [6:07:46<3:38:12,  1.72s/it]
 59%|█████▉    | 11017/18627 [6:07:48<3:53:15,  1.84s/it]
 59%|█████▉    | 11018/18627 [6:07:50<4:04:17,  1.93s/it]
 59%|█████▉    | 11019/18627 [6:07:52<4:11:27,  1.98s/it]
 59%|█████▉    | 11020/18627 [6:07:54<3:41:31,  1.75s/it]
                                                         
{'loss': 0.8113, 'grad_norm': 13.991432189941406, 'learning_rate': 1.8860423233516541e-06, 'epoch': 0.59}
+
 59%|█████▉    | 11020/18627 [6:07:54<3:41:31,  1.75s/it]
 59%|█████▉    | 11021/18627 [6:07:55<3:17:30,  1.56s/it]
 59%|█████▉    | 11022/18627 [6:07:57<3:40:30,  1.74s/it]
 59%|█████▉    | 11023/18627 [6:07:59<3:56:39,  1.87s/it]
 59%|█████▉    | 11024/18627 [6:08:01<4:07:16,  1.95s/it]
 59%|█████▉    | 11025/18627 [6:08:03<4:15:09,  2.01s/it]
 59%|█████▉    | 11026/18627 [6:08:05<4:20:00,  2.05s/it]
 59%|█████▉    | 11027/18627 [6:08:08<4:22:20,  2.07s/it]
 59%|█████▉    | 11028/18627 [6:08:10<4:23:42,  2.08s/it]
 59%|█████▉    | 11029/18627 [6:08:12<4:25:43,  2.10s/it]
 59%|█████▉    | 11030/18627 [6:08:13<3:51:41,  1.83s/it]
                                                         
{'loss': 1.0958, 'grad_norm': 13.541278839111328, 'learning_rate': 1.8818294735292872e-06, 'epoch': 0.59}
+
 59%|█████▉    | 11030/18627 [6:08:13<3:51:41,  1.83s/it]
 59%|█████▉    | 11031/18627 [6:08:15<4:02:41,  1.92s/it]
 59%|█████▉    | 11032/18627 [6:08:16<3:35:02,  1.70s/it]
 59%|█████▉    | 11033/18627 [6:08:18<3:15:38,  1.55s/it]
 59%|█████▉    | 11034/18627 [6:08:20<3:37:42,  1.72s/it]
 59%|█████▉    | 11035/18627 [6:08:21<3:17:19,  1.56s/it]
 59%|█████▉    | 11036/18627 [6:08:23<3:37:59,  1.72s/it]
 59%|█████▉    | 11037/18627 [6:08:25<3:52:56,  1.84s/it]
 59%|█████▉    | 11038/18627 [6:08:26<3:28:13,  1.65s/it]
 59%|█████▉    | 11039/18627 [6:08:28<3:45:43,  1.78s/it]
 59%|█████▉    | 11040/18627 [6:08:30<3:57:46,  1.88s/it]
                                                         
{'loss': 1.4131, 'grad_norm': 11.014192581176758, 'learning_rate': 1.8776184926142473e-06, 'epoch': 0.59}
+
 59%|█████▉    | 11040/18627 [6:08:30<3:57:46,  1.88s/it]
 59%|█████▉    | 11041/18627 [6:08:32<3:31:48,  1.68s/it]
 59%|█████▉    | 11042/18627 [6:08:34<3:49:13,  1.81s/it]
 59%|█████▉    | 11043/18627 [6:08:36<4:02:22,  1.92s/it]
 59%|█████▉    | 11044/18627 [6:08:37<3:34:36,  1.70s/it]
 59%|█████▉    | 11045/18627 [6:08:39<3:52:02,  1.84s/it]
 59%|█████▉    | 11046/18627 [6:08:41<4:04:17,  1.93s/it]
 59%|█████▉    | 11047/18627 [6:08:44<4:12:31,  2.00s/it]
 59%|█████▉    | 11048/18627 [6:08:45<3:42:07,  1.76s/it]
 59%|█████▉    | 11049/18627 [6:08:47<3:57:52,  1.88s/it]
 59%|█████▉    | 11050/18627 [6:08:49<4:08:18,  1.97s/it]
                                                         
{'loss': 1.2723, 'grad_norm': 4.97433614730835, 'learning_rate': 1.8734093933375426e-06, 'epoch': 0.59}
+
 59%|█████▉    | 11050/18627 [6:08:49<4:08:18,  1.97s/it]
 59%|█████▉    | 11051/18627 [6:08:51<4:16:15,  2.03s/it]
 59%|█████▉    | 11052/18627 [6:08:53<4:20:44,  2.07s/it]
 59%|█████▉    | 11053/18627 [6:08:56<4:23:27,  2.09s/it]
 59%|█████▉    | 11054/18627 [6:08:57<3:50:30,  1.83s/it]
 59%|█████▉    | 11055/18627 [6:08:59<4:02:34,  1.92s/it]
 59%|█████▉    | 11056/18627 [6:09:01<4:11:18,  1.99s/it]
 59%|█████▉    | 11057/18627 [6:09:03<4:17:31,  2.04s/it]
 59%|█████▉    | 11058/18627 [6:09:04<3:45:34,  1.79s/it]
 59%|█████▉    | 11059/18627 [6:09:06<3:20:46,  1.59s/it]
 59%|█████▉    | 11060/18627 [6:09:08<3:41:34,  1.76s/it]
                                                         
{'loss': 1.2023, 'grad_norm': 6.137520790100098, 'learning_rate': 1.8692021884244892e-06, 'epoch': 0.59}
+
 59%|█████▉    | 11060/18627 [6:09:08<3:41:34,  1.76s/it]
 59%|█████▉    | 11061/18627 [6:09:10<3:56:09,  1.87s/it]
 59%|█████▉    | 11062/18627 [6:09:12<4:06:34,  1.96s/it]
 59%|█████▉    | 11063/18627 [6:09:14<4:14:08,  2.02s/it]
 59%|█████▉    | 11064/18627 [6:09:16<4:18:52,  2.05s/it]
 59%|█████▉    | 11065/18627 [6:09:18<4:22:07,  2.08s/it]
 59%|█████▉    | 11066/18627 [6:09:21<4:25:03,  2.10s/it]
 59%|█████▉    | 11067/18627 [6:09:23<4:28:04,  2.13s/it]
 59%|█████▉    | 11068/18627 [6:09:25<4:29:52,  2.14s/it]
 59%|█████▉    | 11069/18627 [6:09:26<3:38:21,  1.73s/it]
 59%|█████▉    | 11070/18627 [6:09:28<3:54:11,  1.86s/it]
                                                         
{'loss': 0.6247, 'grad_norm': 7.318007469177246, 'learning_rate': 1.8649968905946797e-06, 'epoch': 0.59}
+
 59%|█████▉    | 11070/18627 [6:09:28<3:54:11,  1.86s/it]
 59%|█████▉    | 11071/18627 [6:09:30<4:05:47,  1.95s/it]
 59%|█████▉    | 11072/18627 [6:09:32<4:13:05,  2.01s/it]
 59%|█████▉    | 11073/18627 [6:09:34<4:18:34,  2.05s/it]
 59%|█████▉    | 11074/18627 [6:09:37<4:22:38,  2.09s/it]
 59%|█████▉    | 11075/18627 [6:09:39<4:25:18,  2.11s/it]
 59%|█████▉    | 11076/18627 [6:09:41<4:27:35,  2.13s/it]
 59%|█████▉    | 11077/18627 [6:09:42<3:52:52,  1.85s/it]
 59%|█████▉    | 11078/18627 [6:09:44<4:04:19,  1.94s/it]
 59%|█████▉    | 11079/18627 [6:09:46<4:12:07,  2.00s/it]
 59%|█████▉    | 11080/18627 [6:09:49<4:17:27,  2.05s/it]
                                                         
{'loss': 0.8565, 'grad_norm': 4.679361820220947, 'learning_rate': 1.860793512561938e-06, 'epoch': 0.59}
+
 59%|█████▉    | 11080/18627 [6:09:49<4:17:27,  2.05s/it]
 59%|█████▉    | 11081/18627 [6:09:50<3:45:42,  1.79s/it]
 59%|█████▉    | 11082/18627 [6:09:52<3:59:31,  1.90s/it]
 59%|█████▉    | 11083/18627 [6:09:54<4:08:43,  1.98s/it]
 60%|█████▉    | 11084/18627 [6:09:56<4:15:38,  2.03s/it]
 60%|█████▉    | 11085/18627 [6:09:58<4:20:20,  2.07s/it]
 60%|█████▉    | 11086/18627 [6:10:01<4:23:21,  2.10s/it]
 60%|█████▉    | 11087/18627 [6:10:02<3:49:46,  1.83s/it]
 60%|█████▉    | 11088/18627 [6:10:04<4:01:15,  1.92s/it]
 60%|█████▉    | 11089/18627 [6:10:06<4:08:56,  1.98s/it]
 60%|█████▉    | 11090/18627 [6:10:08<4:15:33,  2.03s/it]
                                                         
{'loss': 1.0128, 'grad_norm': 11.175265312194824, 'learning_rate': 1.8565920670342858e-06, 'epoch': 0.6}
+
 60%|█████▉    | 11090/18627 [6:10:08<4:15:33,  2.03s/it]
 60%|█████▉    | 11091/18627 [6:10:09<3:43:48,  1.78s/it]
 60%|█████▉    | 11092/18627 [6:10:11<3:56:38,  1.88s/it]
 60%|█████▉    | 11093/18627 [6:10:14<4:05:03,  1.95s/it]
 60%|█████▉    | 11094/18627 [6:10:16<4:10:20,  1.99s/it]
 60%|█████▉    | 11095/18627 [6:10:18<4:15:05,  2.03s/it]
 60%|█████▉    | 11096/18627 [6:10:20<4:18:52,  2.06s/it]
 60%|█████▉    | 11097/18627 [6:10:22<4:20:25,  2.08s/it]
 60%|█████▉    | 11098/18627 [6:10:24<4:21:51,  2.09s/it]
 60%|█████▉    | 11099/18627 [6:10:26<4:22:03,  2.09s/it]
 60%|█████▉    | 11100/18627 [6:10:27<3:48:26,  1.82s/it]
                                                         
{'loss': 1.0265, 'grad_norm': 12.925514221191406, 'learning_rate': 1.8523925667138999e-06, 'epoch': 0.6}
+
 60%|█████▉    | 11100/18627 [6:10:27<3:48:26,  1.82s/it]
 60%|█████▉    | 11101/18627 [6:10:30<3:59:17,  1.91s/it]
 60%|█████▉    | 11102/18627 [6:10:32<4:07:43,  1.98s/it]
 60%|█████▉    | 11103/18627 [6:10:34<4:13:23,  2.02s/it]
 60%|█████▉    | 11104/18627 [6:10:36<4:16:38,  2.05s/it]
 60%|█████▉    | 11105/18627 [6:10:38<4:19:17,  2.07s/it]
 60%|█████▉    | 11106/18627 [6:10:39<3:46:07,  1.80s/it]
 60%|█████▉    | 11107/18627 [6:10:41<3:58:04,  1.90s/it]
 60%|█████▉    | 11108/18627 [6:10:43<3:31:54,  1.69s/it]
 60%|█████▉    | 11109/18627 [6:10:45<3:48:56,  1.83s/it]
 60%|█████▉    | 11110/18627 [6:10:45<3:06:44,  1.49s/it]
                                                         
{'loss': 0.9675, 'grad_norm': 5.809579372406006, 'learning_rate': 1.8481950242970804e-06, 'epoch': 0.6}
+
 60%|█████▉    | 11110/18627 [6:10:45<3:06:44,  1.49s/it]
 60%|█████▉    | 11111/18627 [6:10:48<3:30:41,  1.68s/it]
 60%|█████▉    | 11112/18627 [6:10:50<3:46:19,  1.81s/it]
 60%|█████▉    | 11113/18627 [6:10:52<3:57:24,  1.90s/it]
 60%|█████▉    | 11114/18627 [6:10:54<4:05:51,  1.96s/it]
 60%|█████▉    | 11115/18627 [6:10:56<4:11:49,  2.01s/it]
 60%|█████▉    | 11116/18627 [6:10:58<4:15:21,  2.04s/it]
 60%|█████▉    | 11117/18627 [6:10:59<3:43:34,  1.79s/it]
 60%|█████▉    | 11118/18627 [6:11:01<3:55:36,  1.88s/it]
 60%|█████▉    | 11119/18627 [6:11:02<3:11:41,  1.53s/it]
 60%|█████▉    | 11120/18627 [6:11:04<3:33:37,  1.71s/it]
                                                         
{'loss': 0.9069, 'grad_norm': 5.329052448272705, 'learning_rate': 1.8439994524742033e-06, 'epoch': 0.6}
+
 60%|█████▉    | 11120/18627 [6:11:04<3:33:37,  1.71s/it]
 60%|█████▉    | 11121/18627 [6:11:05<3:14:32,  1.56s/it]
 60%|█████▉    | 11122/18627 [6:11:08<3:35:34,  1.72s/it]
 60%|█████▉    | 11123/18627 [6:11:10<3:49:58,  1.84s/it]
 60%|█████▉    | 11124/18627 [6:11:12<4:00:19,  1.92s/it]
 60%|█████▉    | 11125/18627 [6:11:14<4:07:20,  1.98s/it]
 60%|█████▉    | 11126/18627 [6:11:16<4:13:21,  2.03s/it]
 60%|█████▉    | 11127/18627 [6:11:18<4:15:53,  2.05s/it]
 60%|█████▉    | 11128/18627 [6:11:20<4:18:18,  2.07s/it]
 60%|█████▉    | 11129/18627 [6:11:22<4:20:19,  2.08s/it]
 60%|█████▉    | 11130/18627 [6:11:24<4:20:55,  2.09s/it]
                                                         
{'loss': 0.8376, 'grad_norm': 7.286038875579834, 'learning_rate': 1.8398058639296906e-06, 'epoch': 0.6}
+
 60%|█████▉    | 11130/18627 [6:11:24<4:20:55,  2.09s/it]
 60%|█████▉    | 11131/18627 [6:11:27<4:21:51,  2.10s/it]
 60%|█████▉    | 11132/18627 [6:11:29<4:23:02,  2.11s/it]
 60%|█████▉    | 11133/18627 [6:11:30<3:49:08,  1.83s/it]
 60%|█████▉    | 11134/18627 [6:11:32<4:01:01,  1.93s/it]
 60%|█████▉    | 11135/18627 [6:11:33<3:33:11,  1.71s/it]
 60%|█████▉    | 11136/18627 [6:11:34<3:13:51,  1.55s/it]
 60%|█████▉    | 11137/18627 [6:11:37<3:36:41,  1.74s/it]
 60%|█████▉    | 11138/18627 [6:11:38<3:16:28,  1.57s/it]
 60%|█████▉    | 11139/18627 [6:11:40<3:37:08,  1.74s/it]
 60%|█████▉    | 11140/18627 [6:11:42<3:50:52,  1.85s/it]
                                                         
{'loss': 1.3797, 'grad_norm': 5.797366619110107, 'learning_rate': 1.8356142713419652e-06, 'epoch': 0.6}
+
 60%|█████▉    | 11140/18627 [6:11:42<3:50:52,  1.85s/it]
 60%|█████▉    | 11141/18627 [6:11:44<3:59:57,  1.92s/it]
 60%|█████▉    | 11142/18627 [6:11:46<4:07:00,  1.98s/it]
 60%|█████▉    | 11143/18627 [6:11:47<3:37:54,  1.75s/it]
 60%|█████▉    | 11144/18627 [6:11:50<3:51:03,  1.85s/it]
 60%|█████▉    | 11145/18627 [6:11:51<3:26:19,  1.65s/it]
 60%|█████▉    | 11146/18627 [6:11:52<3:08:47,  1.51s/it]
 60%|█████▉    | 11147/18627 [6:11:53<2:52:43,  1.39s/it]
 60%|█████▉    | 11148/18627 [6:11:55<3:20:40,  1.61s/it]
 60%|█████▉    | 11149/18627 [6:11:56<3:04:52,  1.48s/it]
 60%|█████▉    | 11150/18627 [6:11:58<3:28:42,  1.67s/it]
                                                         
{'loss': 1.5248, 'grad_norm': 5.068739891052246, 'learning_rate': 1.8314246873834192e-06, 'epoch': 0.6}
+
 60%|█████▉    | 11150/18627 [6:11:58<3:28:42,  1.67s/it]
 60%|█████▉    | 11151/18627 [6:12:00<3:10:20,  1.53s/it]
 60%|█████▉    | 11152/18627 [6:12:01<2:57:51,  1.43s/it]
 60%|█████▉    | 11153/18627 [6:12:03<3:24:09,  1.64s/it]
 60%|█████▉    | 11154/18627 [6:12:04<3:06:59,  1.50s/it]
 60%|█████▉    | 11155/18627 [6:12:06<3:30:47,  1.69s/it]
 60%|█████▉    | 11156/18627 [6:12:07<3:12:22,  1.55s/it]
 60%|█████▉    | 11157/18627 [6:12:10<3:34:17,  1.72s/it]
 60%|█████▉    | 11158/18627 [6:12:12<3:49:16,  1.84s/it]
 60%|█████▉    | 11159/18627 [6:12:13<3:25:01,  1.65s/it]
 60%|█████▉    | 11160/18627 [6:12:14<3:07:57,  1.51s/it]
                                                         
{'loss': 1.6425, 'grad_norm': 11.302424430847168, 'learning_rate': 1.8272371247203692e-06, 'epoch': 0.6}
+
 60%|█████▉    | 11160/18627 [6:12:14<3:07:57,  1.51s/it]
 60%|█████▉    | 11161/18627 [6:12:16<3:30:49,  1.69s/it]
 60%|█████▉    | 11162/18627 [6:12:18<3:45:53,  1.82s/it]
 60%|█████▉    | 11163/18627 [6:12:19<3:04:00,  1.48s/it]
 60%|█████▉    | 11164/18627 [6:12:21<3:27:46,  1.67s/it]
 60%|█████▉    | 11165/18627 [6:12:23<3:44:36,  1.81s/it]
 60%|█████▉    | 11166/18627 [6:12:24<3:03:11,  1.47s/it]
 60%|█████▉    | 11167/18627 [6:12:26<3:27:37,  1.67s/it]
 60%|█████▉    | 11168/18627 [6:12:27<3:09:28,  1.52s/it]
 60%|█████▉    | 11169/18627 [6:12:28<2:56:52,  1.42s/it]
 60%|█████▉    | 11170/18627 [6:12:31<3:22:52,  1.63s/it]
                                                         
{'loss': 0.9262, 'grad_norm': 7.196443557739258, 'learning_rate': 1.8230515960130205e-06, 'epoch': 0.6}
+
 60%|█████▉    | 11170/18627 [6:12:31<3:22:52,  1.63s/it]
 60%|█████▉    | 11171/18627 [6:12:33<3:41:38,  1.78s/it]
 60%|█████▉    | 11172/18627 [6:12:35<3:54:44,  1.89s/it]
 60%|█████▉    | 11173/18627 [6:12:37<4:02:51,  1.95s/it]
 60%|█████▉    | 11174/18627 [6:12:39<4:09:07,  2.01s/it]
 60%|█████▉    | 11175/18627 [6:12:41<4:13:22,  2.04s/it]
 60%|█████▉    | 11176/18627 [6:12:43<4:17:30,  2.07s/it]
 60%|██████    | 11177/18627 [6:12:46<4:21:26,  2.11s/it]
 60%|██████    | 11178/18627 [6:12:47<3:47:11,  1.83s/it]
 60%|██████    | 11179/18627 [6:12:48<3:23:42,  1.64s/it]
 60%|██████    | 11180/18627 [6:12:49<3:02:35,  1.47s/it]
                                                         
{'loss': 1.1582, 'grad_norm': 13.76465892791748, 'learning_rate': 1.8188681139154329e-06, 'epoch': 0.6}
+
 60%|██████    | 11180/18627 [6:12:49<3:02:35,  1.47s/it]
 60%|██████    | 11181/18627 [6:12:51<3:26:06,  1.66s/it]
 60%|██████    | 11182/18627 [6:12:53<3:42:22,  1.79s/it]
 60%|██████    | 11183/18627 [6:12:54<3:19:44,  1.61s/it]
 60%|██████    | 11184/18627 [6:12:56<3:38:26,  1.76s/it]
 60%|██████    | 11185/18627 [6:12:59<3:51:38,  1.87s/it]
 60%|██████    | 11186/18627 [6:13:01<4:00:44,  1.94s/it]
 60%|██████    | 11187/18627 [6:13:03<4:09:34,  2.01s/it]
 60%|██████    | 11188/18627 [6:13:05<4:12:38,  2.04s/it]
 60%|██████    | 11189/18627 [6:13:07<4:15:42,  2.06s/it]
 60%|██████    | 11190/18627 [6:13:09<4:18:20,  2.08s/it]
                                                         
{'loss': 0.8825, 'grad_norm': 6.795285701751709, 'learning_rate': 1.8146866910754745e-06, 'epoch': 0.6}
+
 60%|██████    | 11190/18627 [6:13:09<4:18:20,  2.08s/it]
 60%|██████    | 11191/18627 [6:13:10<3:45:06,  1.82s/it]
 60%|██████    | 11192/18627 [6:13:13<3:56:32,  1.91s/it]
 60%|██████    | 11193/18627 [6:13:14<3:30:03,  1.70s/it]
 60%|██████    | 11194/18627 [6:13:16<3:45:53,  1.82s/it]
 60%|██████    | 11195/18627 [6:13:18<3:56:32,  1.91s/it]
 60%|██████    | 11196/18627 [6:13:20<4:04:19,  1.97s/it]
 60%|██████    | 11197/18627 [6:13:22<4:09:24,  2.01s/it]
 60%|██████    | 11198/18627 [6:13:24<4:13:16,  2.05s/it]
 60%|██████    | 11199/18627 [6:13:25<3:23:30,  1.64s/it]
 60%|██████    | 11200/18627 [6:13:27<3:41:35,  1.79s/it]
                                                         
{'loss': 1.1124, 'grad_norm': 8.118711471557617, 'learning_rate': 1.8105073401347905e-06, 'epoch': 0.6}
+
 60%|██████    | 11200/18627 [6:13:27<3:41:35,  1.79s/it]
 60%|██████    | 11201/18627 [6:13:29<3:53:40,  1.89s/it]
 60%|██████    | 11202/18627 [6:13:31<4:02:50,  1.96s/it]
 60%|██████    | 11203/18627 [6:13:34<4:09:15,  2.01s/it]
 60%|██████    | 11204/18627 [6:13:36<4:13:33,  2.05s/it]
 60%|██████    | 11205/18627 [6:13:38<4:16:41,  2.08s/it]
 60%|██████    | 11206/18627 [6:13:40<4:18:59,  2.09s/it]
 60%|██████    | 11207/18627 [6:13:42<4:19:11,  2.10s/it]
 60%|██████    | 11208/18627 [6:13:44<4:20:45,  2.11s/it]
 60%|██████    | 11209/18627 [6:13:46<4:21:09,  2.11s/it]
 60%|██████    | 11210/18627 [6:13:48<4:21:49,  2.12s/it]
                                                         
{'loss': 0.6077, 'grad_norm': 9.030536651611328, 'learning_rate': 1.8063300737287593e-06, 'epoch': 0.6}
+
 60%|██████    | 11210/18627 [6:13:48<4:21:49,  2.12s/it]
 60%|██████    | 11211/18627 [6:13:51<4:21:50,  2.12s/it]
 60%|██████    | 11212/18627 [6:13:52<3:47:27,  1.84s/it]
 60%|██████    | 11213/18627 [6:13:54<3:57:24,  1.92s/it]
 60%|██████    | 11214/18627 [6:13:55<3:29:58,  1.70s/it]
 60%|██████    | 11215/18627 [6:13:57<3:45:01,  1.82s/it]
 60%|██████    | 11216/18627 [6:13:59<3:55:55,  1.91s/it]
 60%|██████    | 11217/18627 [6:14:01<4:03:42,  1.97s/it]
 60%|██████    | 11218/18627 [6:14:03<3:34:45,  1.74s/it]
 60%|██████    | 11219/18627 [6:14:04<3:14:01,  1.57s/it]
 60%|██████    | 11220/18627 [6:14:06<3:34:25,  1.74s/it]
                                                         
{'loss': 1.4065, 'grad_norm': 4.620079517364502, 'learning_rate': 1.802154904486461e-06, 'epoch': 0.6}
+
 60%|██████    | 11220/18627 [6:14:06<3:34:25,  1.74s/it]
 60%|██████    | 11221/18627 [6:14:08<3:48:45,  1.85s/it]
 60%|██████    | 11222/18627 [6:14:10<3:58:09,  1.93s/it]
 60%|██████    | 11223/18627 [6:14:12<4:05:15,  1.99s/it]
 60%|██████    | 11224/18627 [6:14:14<4:09:05,  2.02s/it]
 60%|██████    | 11225/18627 [6:14:16<4:12:57,  2.05s/it]
 60%|██████    | 11226/18627 [6:14:19<4:16:47,  2.08s/it]
 60%|██████    | 11227/18627 [6:14:21<4:18:23,  2.10s/it]
 60%|██████    | 11228/18627 [6:14:22<3:44:45,  1.82s/it]
 60%|██████    | 11229/18627 [6:14:24<3:55:20,  1.91s/it]
 60%|██████    | 11230/18627 [6:14:26<4:03:28,  1.97s/it]
                                                         
{'loss': 0.819, 'grad_norm': 5.752904415130615, 'learning_rate': 1.7979818450306315e-06, 'epoch': 0.6}
+
 60%|██████    | 11230/18627 [6:14:26<4:03:28,  1.97s/it]
 60%|██████    | 11231/18627 [6:14:28<4:07:50,  2.01s/it]
 60%|██████    | 11232/18627 [6:14:29<3:37:18,  1.76s/it]
 60%|██████    | 11233/18627 [6:14:31<3:16:23,  1.59s/it]
 60%|██████    | 11234/18627 [6:14:33<3:35:13,  1.75s/it]
 60%|██████    | 11235/18627 [6:14:35<3:49:01,  1.86s/it]
 60%|██████    | 11236/18627 [6:14:37<3:58:43,  1.94s/it]
 60%|██████    | 11237/18627 [6:14:38<3:31:11,  1.71s/it]
 60%|██████    | 11238/18627 [6:14:40<3:45:41,  1.83s/it]
 60%|██████    | 11239/18627 [6:14:42<3:22:39,  1.65s/it]
 60%|██████    | 11240/18627 [6:14:44<3:39:58,  1.79s/it]
                                                         
{'loss': 1.331, 'grad_norm': 6.991641044616699, 'learning_rate': 1.7938109079776316e-06, 'epoch': 0.6}
+
 60%|██████    | 11240/18627 [6:14:44<3:39:58,  1.79s/it]
 60%|██████    | 11241/18627 [6:14:46<3:52:01,  1.88s/it]
 60%|██████    | 11242/18627 [6:14:48<3:59:30,  1.95s/it]
 60%|██████    | 11243/18627 [6:14:50<4:06:28,  2.00s/it]
 60%|██████    | 11244/18627 [6:14:52<4:10:45,  2.04s/it]
 60%|██████    | 11245/18627 [6:14:53<3:33:08,  1.73s/it]
 60%|██████    | 11246/18627 [6:14:55<3:48:55,  1.86s/it]
 60%|██████    | 11247/18627 [6:14:57<3:57:56,  1.93s/it]
 60%|██████    | 11248/18627 [6:14:59<4:05:02,  1.99s/it]
 60%|██████    | 11249/18627 [6:15:01<3:35:08,  1.75s/it]
 60%|██████    | 11250/18627 [6:15:03<3:48:33,  1.86s/it]
                                                         
{'loss': 1.1412, 'grad_norm': 5.738000869750977, 'learning_rate': 1.789642105937402e-06, 'epoch': 0.6}
+
 60%|██████    | 11250/18627 [6:15:03<3:48:33,  1.86s/it]
 60%|██████    | 11251/18627 [6:15:05<3:58:36,  1.94s/it]
 60%|██████    | 11252/18627 [6:15:07<4:04:32,  1.99s/it]
 60%|██████    | 11253/18627 [6:15:09<4:09:03,  2.03s/it]
 60%|██████    | 11254/18627 [6:15:10<3:38:05,  1.77s/it]
 60%|██████    | 11255/18627 [6:15:12<3:50:47,  1.88s/it]
 60%|██████    | 11256/18627 [6:15:15<3:59:39,  1.95s/it]
 60%|██████    | 11257/18627 [6:15:17<4:06:08,  2.00s/it]
 60%|██████    | 11258/18627 [6:15:19<4:10:36,  2.04s/it]
 60%|██████    | 11259/18627 [6:15:21<4:12:59,  2.06s/it]
 60%|██████    | 11260/18627 [6:15:23<4:15:15,  2.08s/it]
                                                         
{'loss': 0.7904, 'grad_norm': 5.73778772354126, 'learning_rate': 1.7854754515134323e-06, 'epoch': 0.6}
+
 60%|██████    | 11260/18627 [6:15:23<4:15:15,  2.08s/it]
 60%|██████    | 11261/18627 [6:15:25<4:16:25,  2.09s/it]
 60%|██████    | 11262/18627 [6:15:27<4:17:25,  2.10s/it]
 60%|██████    | 11263/18627 [6:15:29<4:17:51,  2.10s/it]
 60%|██████    | 11264/18627 [6:15:31<3:44:06,  1.83s/it]
 60%|██████    | 11265/18627 [6:15:33<3:54:30,  1.91s/it]
 60%|██████    | 11266/18627 [6:15:34<3:27:33,  1.69s/it]
 60%|██████    | 11267/18627 [6:15:36<3:42:32,  1.81s/it]
 60%|██████    | 11268/18627 [6:15:38<3:54:02,  1.91s/it]
 60%|██████    | 11269/18627 [6:15:40<4:01:06,  1.97s/it]
 61%|██████    | 11270/18627 [6:15:42<4:07:36,  2.02s/it]
                                                         
{'loss': 0.9814, 'grad_norm': 6.561555862426758, 'learning_rate': 1.781310957302716e-06, 'epoch': 0.61}
+
 61%|██████    | 11270/18627 [6:15:42<4:07:36,  2.02s/it]
 61%|██████    | 11271/18627 [6:15:44<3:37:02,  1.77s/it]
 61%|██████    | 11272/18627 [6:15:45<3:15:40,  1.60s/it]
 61%|██████    | 11273/18627 [6:15:47<3:35:02,  1.75s/it]
 61%|██████    | 11274/18627 [6:15:49<3:48:12,  1.86s/it]
 61%|██████    | 11275/18627 [6:15:51<3:57:37,  1.94s/it]
 61%|██████    | 11276/18627 [6:15:53<4:04:54,  2.00s/it]
 61%|██████    | 11277/18627 [6:15:55<4:09:22,  2.04s/it]
 61%|██████    | 11278/18627 [6:15:57<4:11:56,  2.06s/it]
 61%|██████    | 11279/18627 [6:16:00<4:13:34,  2.07s/it]
 61%|██████    | 11280/18627 [6:16:02<4:15:18,  2.09s/it]
                                                         
{'loss': 1.1191, 'grad_norm': 5.684905052185059, 'learning_rate': 1.7771486358957185e-06, 'epoch': 0.61}
+
 61%|██████    | 11280/18627 [6:16:02<4:15:18,  2.09s/it]
 61%|██████    | 11281/18627 [6:16:04<4:15:36,  2.09s/it]
 61%|██████    | 11282/18627 [6:16:06<4:16:29,  2.10s/it]
 61%|██████    | 11283/18627 [6:16:07<3:43:08,  1.82s/it]
 61%|██████    | 11284/18627 [6:16:09<3:54:52,  1.92s/it]
 61%|██████    | 11285/18627 [6:16:11<4:03:57,  1.99s/it]
 61%|██████    | 11286/18627 [6:16:14<4:09:29,  2.04s/it]
 61%|██████    | 11287/18627 [6:16:15<3:38:28,  1.79s/it]
 61%|██████    | 11288/18627 [6:16:17<3:50:45,  1.89s/it]
 61%|██████    | 11289/18627 [6:16:19<3:59:23,  1.96s/it]
 61%|██████    | 11290/18627 [6:16:20<3:13:58,  1.59s/it]
                                                         
{'loss': 0.9746, 'grad_norm': 5.973367214202881, 'learning_rate': 1.772988499876333e-06, 'epoch': 0.61}
+
 61%|██████    | 11290/18627 [6:16:20<3:13:58,  1.59s/it]
 61%|██████    | 11291/18627 [6:16:22<3:32:49,  1.74s/it]
 61%|██████    | 11292/18627 [6:16:24<3:46:05,  1.85s/it]
 61%|██████    | 11293/18627 [6:16:26<3:55:50,  1.93s/it]
 61%|██████    | 11294/18627 [6:16:27<3:28:55,  1.71s/it]
 61%|██████    | 11295/18627 [6:16:28<3:05:57,  1.52s/it]
 61%|██████    | 11296/18627 [6:16:29<2:53:49,  1.42s/it]
 61%|██████    | 11297/18627 [6:16:32<3:19:00,  1.63s/it]
 61%|██████    | 11298/18627 [6:16:34<3:37:00,  1.78s/it]
 61%|██████    | 11299/18627 [6:16:36<3:49:30,  1.88s/it]
 61%|██████    | 11300/18627 [6:16:38<3:58:10,  1.95s/it]
                                                         
{'loss': 1.3901, 'grad_norm': 3.9681596755981445, 'learning_rate': 1.7688305618218493e-06, 'epoch': 0.61}
+
 61%|██████    | 11300/18627 [6:16:38<3:58:10,  1.95s/it]
 61%|██████    | 11301/18627 [6:16:40<4:03:59,  2.00s/it]
 61%|██████    | 11302/18627 [6:16:42<4:08:44,  2.04s/it]
 61%|██████    | 11303/18627 [6:16:44<4:11:51,  2.06s/it]
 61%|██████    | 11304/18627 [6:16:46<4:13:25,  2.08s/it]
 61%|██████    | 11305/18627 [6:16:49<4:14:55,  2.09s/it]
 61%|██████    | 11306/18627 [6:16:51<4:36:53,  2.27s/it]
 61%|██████    | 11307/18627 [6:16:53<4:31:47,  2.23s/it]
 61%|██████    | 11308/18627 [6:16:55<4:27:09,  2.19s/it]
 61%|██████    | 11309/18627 [6:16:57<3:50:30,  1.89s/it]
 61%|██████    | 11310/18627 [6:16:59<3:58:10,  1.95s/it]
                                                         
{'loss': 0.9453, 'grad_norm': 6.46362829208374, 'learning_rate': 1.764674834302908e-06, 'epoch': 0.61}
+
 61%|██████    | 11310/18627 [6:16:59<3:58:10,  1.95s/it]
 61%|██████    | 11311/18627 [6:17:01<4:03:17,  2.00s/it]
 61%|██████    | 11312/18627 [6:17:03<4:08:10,  2.04s/it]
 61%|██████    | 11313/18627 [6:17:05<4:11:42,  2.06s/it]
 61%|██████    | 11314/18627 [6:17:06<3:39:56,  1.80s/it]
 61%|██████    | 11315/18627 [6:17:08<3:50:17,  1.89s/it]
 61%|██████    | 11316/18627 [6:17:10<3:24:50,  1.68s/it]
 61%|██████    | 11317/18627 [6:17:12<3:41:06,  1.81s/it]
 61%|██████    | 11318/18627 [6:17:14<3:52:07,  1.91s/it]
 61%|██████    | 11319/18627 [6:17:16<4:00:37,  1.98s/it]
 61%|██████    | 11320/18627 [6:17:18<4:06:28,  2.02s/it]
                                                         
{'loss': 0.9932, 'grad_norm': 5.062347412109375, 'learning_rate': 1.760521329883472e-06, 'epoch': 0.61}
+
 61%|██████    | 11320/18627 [6:17:18<4:06:28,  2.02s/it]
 61%|██████    | 11321/18627 [6:17:20<4:09:43,  2.05s/it]
 61%|██████    | 11322/18627 [6:17:22<4:11:35,  2.07s/it]
 61%|██████    | 11323/18627 [6:17:24<4:13:32,  2.08s/it]
 61%|██████    | 11324/18627 [6:17:27<4:14:30,  2.09s/it]
 61%|██████    | 11325/18627 [6:17:29<4:15:35,  2.10s/it]
 61%|██████    | 11326/18627 [6:17:30<3:42:26,  1.83s/it]
 61%|██████    | 11327/18627 [6:17:32<3:52:47,  1.91s/it]
 61%|██████    | 11328/18627 [6:17:33<3:26:35,  1.70s/it]
 61%|██████    | 11329/18627 [6:17:35<3:41:58,  1.82s/it]
 61%|██████    | 11330/18627 [6:17:37<3:52:03,  1.91s/it]
                                                         
{'loss': 1.1005, 'grad_norm': 4.840379238128662, 'learning_rate': 1.7563700611207765e-06, 'epoch': 0.61}
+
 61%|██████    | 11330/18627 [6:17:37<3:52:03,  1.91s/it]
 61%|██████    | 11331/18627 [6:17:39<3:59:52,  1.97s/it]
 61%|██████    | 11332/18627 [6:17:42<4:05:05,  2.02s/it]
 61%|██████    | 11333/18627 [6:17:44<4:09:20,  2.05s/it]
 61%|██████    | 11334/18627 [6:17:46<4:11:15,  2.07s/it]
 61%|██████    | 11335/18627 [6:17:48<4:12:03,  2.07s/it]
 61%|██████    | 11336/18627 [6:17:50<4:14:20,  2.09s/it]
 61%|██████    | 11337/18627 [6:17:52<4:15:13,  2.10s/it]
 61%|██████    | 11338/18627 [6:17:54<4:15:04,  2.10s/it]
 61%|██████    | 11339/18627 [6:17:56<4:15:55,  2.11s/it]
 61%|██████    | 11340/18627 [6:17:58<3:42:19,  1.83s/it]
                                                         
{'loss': 0.7855, 'grad_norm': 13.419822692871094, 'learning_rate': 1.7522210405653039e-06, 'epoch': 0.61}
+
 61%|██████    | 11340/18627 [6:17:58<3:42:19,  1.83s/it]
 61%|██████    | 11341/18627 [6:17:59<3:18:54,  1.64s/it]
 61%|██████    | 11342/18627 [6:18:01<3:36:01,  1.78s/it]
 61%|██████    | 11343/18627 [6:18:03<3:48:26,  1.88s/it]
 61%|██████    | 11344/18627 [6:18:04<3:23:00,  1.67s/it]
 61%|██████    | 11345/18627 [6:18:06<3:39:38,  1.81s/it]
 61%|██████    | 11346/18627 [6:18:08<3:50:58,  1.90s/it]
 61%|██████    | 11347/18627 [6:18:11<3:58:31,  1.97s/it]
 61%|██████    | 11348/18627 [6:18:12<3:30:19,  1.73s/it]
 61%|██████    | 11349/18627 [6:18:14<3:44:01,  1.85s/it]
 61%|██████    | 11350/18627 [6:18:16<3:53:37,  1.93s/it]
                                                         
{'loss': 1.2715, 'grad_norm': 6.106851577758789, 'learning_rate': 1.7480742807607346e-06, 'epoch': 0.61}
+
 61%|██████    | 11350/18627 [6:18:16<3:53:37,  1.93s/it]
 61%|██████    | 11351/18627 [6:18:18<4:00:05,  1.98s/it]
 61%|██████    | 11352/18627 [6:18:19<3:31:50,  1.75s/it]
 61%|██████    | 11353/18627 [6:18:21<3:45:44,  1.86s/it]
 61%|██████    | 11354/18627 [6:18:24<3:55:41,  1.94s/it]
 61%|██████    | 11355/18627 [6:18:25<3:22:03,  1.67s/it]
 61%|██████    | 11356/18627 [6:18:26<3:05:01,  1.53s/it]
 61%|██████    | 11357/18627 [6:18:28<3:26:21,  1.70s/it]
 61%|██████    | 11358/18627 [6:18:30<3:41:02,  1.82s/it]
 61%|██████    | 11359/18627 [6:18:32<3:51:01,  1.91s/it]
 61%|██████    | 11360/18627 [6:18:34<3:58:50,  1.97s/it]
                                                         
{'loss': 1.1329, 'grad_norm': 4.46468448638916, 'learning_rate': 1.7439297942439193e-06, 'epoch': 0.61}
+
 61%|██████    | 11360/18627 [6:18:34<3:58:50,  1.97s/it]
 61%|██████    | 11361/18627 [6:18:36<4:04:31,  2.02s/it]
 61%|██████    | 11362/18627 [6:18:38<4:08:00,  2.05s/it]
 61%|██████    | 11363/18627 [6:18:41<4:10:29,  2.07s/it]
 61%|██████    | 11364/18627 [6:18:43<4:12:05,  2.08s/it]
 61%|██████    | 11365/18627 [6:18:45<4:13:44,  2.10s/it]
 61%|██████    | 11366/18627 [6:18:47<4:14:12,  2.10s/it]
 61%|██████    | 11367/18627 [6:18:49<4:14:44,  2.11s/it]
 61%|██████    | 11368/18627 [6:18:51<4:15:31,  2.11s/it]
 61%|██████    | 11369/18627 [6:18:53<4:15:43,  2.11s/it]
 61%|██████    | 11370/18627 [6:18:55<4:15:49,  2.12s/it]
                                                         
{'loss': 0.6192, 'grad_norm': 7.136985778808594, 'learning_rate': 1.7397875935448294e-06, 'epoch': 0.61}
+
 61%|██████    | 11370/18627 [6:18:55<4:15:49,  2.12s/it]
 61%|██████    | 11371/18627 [6:18:58<4:15:41,  2.11s/it]
 61%|██████    | 11372/18627 [6:18:59<3:42:01,  1.84s/it]
 61%|██████    | 11373/18627 [6:19:01<3:52:56,  1.93s/it]
 61%|██████    | 11374/18627 [6:19:03<3:59:55,  1.98s/it]
 61%|██████    | 11375/18627 [6:19:04<3:31:17,  1.75s/it]
 61%|██████    | 11376/18627 [6:19:06<3:43:52,  1.85s/it]
 61%|██████    | 11377/18627 [6:19:08<3:53:47,  1.93s/it]
 61%|██████    | 11378/18627 [6:19:10<3:26:33,  1.71s/it]
 61%|██████    | 11379/18627 [6:19:11<3:07:54,  1.56s/it]
 61%|██████    | 11380/18627 [6:19:13<3:27:56,  1.72s/it]
                                                         
{'loss': 1.4043, 'grad_norm': 6.187252998352051, 'learning_rate': 1.735647691186532e-06, 'epoch': 0.61}
+
 61%|██████    | 11380/18627 [6:19:13<3:27:56,  1.72s/it]
 61%|██████    | 11381/18627 [6:19:15<3:42:19,  1.84s/it]
 61%|██████    | 11382/18627 [6:19:17<3:52:03,  1.92s/it]
 61%|██████    | 11383/18627 [6:19:19<3:59:16,  1.98s/it]
 61%|██████    | 11384/18627 [6:19:21<4:04:25,  2.02s/it]
 61%|██████    | 11385/18627 [6:19:23<4:08:05,  2.06s/it]
 61%|██████    | 11386/18627 [6:19:26<4:10:49,  2.08s/it]
 61%|██████    | 11387/18627 [6:19:27<3:38:47,  1.81s/it]
 61%|██████    | 11388/18627 [6:19:28<3:15:58,  1.62s/it]
 61%|██████    | 11389/18627 [6:19:29<3:00:05,  1.49s/it]
 61%|██████    | 11390/18627 [6:19:31<3:22:49,  1.68s/it]
                                                         
{'loss': 1.1557, 'grad_norm': 5.264932155609131, 'learning_rate': 1.7315100996851408e-06, 'epoch': 0.61}
+
 61%|██████    | 11390/18627 [6:19:31<3:22:49,  1.68s/it]
 61%|██████    | 11391/18627 [6:19:33<3:38:31,  1.81s/it]
 61%|██████    | 11392/18627 [6:19:36<3:49:36,  1.90s/it]
 61%|██████    | 11393/18627 [6:19:38<3:57:05,  1.97s/it]
 61%|██████    | 11394/18627 [6:19:40<4:02:43,  2.01s/it]
 61%|██████    | 11395/18627 [6:19:42<4:05:51,  2.04s/it]
 61%|██████    | 11396/18627 [6:19:44<4:08:14,  2.06s/it]
 61%|██████    | 11397/18627 [6:19:46<4:09:53,  2.07s/it]
 61%|██████    | 11398/18627 [6:19:48<4:11:41,  2.09s/it]
 61%|██████    | 11399/18627 [6:19:50<4:13:03,  2.10s/it]
 61%|██████    | 11400/18627 [6:19:52<4:13:09,  2.10s/it]
                                                         
{'loss': 0.6418, 'grad_norm': 5.118681907653809, 'learning_rate': 1.7273748315497873e-06, 'epoch': 0.61}
+
 61%|██████    | 11400/18627 [6:19:52<4:13:09,  2.10s/it]
 61%|██████    | 11401/18627 [6:19:55<4:13:33,  2.11s/it]
 61%|██████    | 11402/18627 [6:19:56<3:40:28,  1.83s/it]
 61%|██████    | 11403/18627 [6:19:58<3:50:15,  1.91s/it]
 61%|██████    | 11404/18627 [6:20:00<3:58:10,  1.98s/it]
 61%|██████    | 11405/18627 [6:20:02<4:02:41,  2.02s/it]
 61%|██████    | 11406/18627 [6:20:04<4:06:06,  2.04s/it]
 61%|██████    | 11407/18627 [6:20:05<3:32:17,  1.76s/it]
 61%|██████    | 11408/18627 [6:20:07<3:11:26,  1.59s/it]
 61%|██████    | 11409/18627 [6:20:09<3:30:19,  1.75s/it]
 61%|██████▏   | 11410/18627 [6:20:10<3:10:05,  1.58s/it]
                                                         
{'loss': 1.4859, 'grad_norm': 15.431357383728027, 'learning_rate': 1.7232418992825748e-06, 'epoch': 0.61}
+
 61%|██████▏   | 11410/18627 [6:20:10<3:10:05,  1.58s/it]
 61%|██████▏   | 11411/18627 [6:20:11<2:55:59,  1.46s/it]
 61%|██████▏   | 11412/18627 [6:20:12<2:46:09,  1.38s/it]
 61%|██████▏   | 11413/18627 [6:20:14<3:12:15,  1.60s/it]
 61%|██████▏   | 11414/18627 [6:20:16<3:31:24,  1.76s/it]
 61%|██████▏   | 11415/18627 [6:20:19<3:44:13,  1.87s/it]
 61%|██████▏   | 11416/18627 [6:20:21<3:53:15,  1.94s/it]
 61%|██████▏   | 11417/18627 [6:20:23<3:59:11,  1.99s/it]
 61%|█���████▏   | 11418/18627 [6:20:24<3:30:38,  1.75s/it]
 61%|██████▏   | 11419/18627 [6:20:26<3:44:10,  1.87s/it]
 61%|██████▏   | 11420/18627 [6:20:28<3:53:11,  1.94s/it]
                                                         
{'loss': 1.1158, 'grad_norm': 5.591870307922363, 'learning_rate': 1.7191113153785478e-06, 'epoch': 0.61}
+
 61%|██████▏   | 11420/18627 [6:20:28<3:53:11,  1.94s/it]
 61%|██████▏   | 11421/18627 [6:20:29<3:26:49,  1.72s/it]
 61%|██████▏   | 11422/18627 [6:20:32<3:41:02,  1.84s/it]
 61%|██████▏   | 11423/18627 [6:20:34<3:51:50,  1.93s/it]
 61%|██████▏   | 11424/18627 [6:20:36<3:58:05,  1.98s/it]
 61%|██████▏   | 11425/18627 [6:20:37<3:29:19,  1.74s/it]
 61%|██████▏   | 11426/18627 [6:20:39<3:43:35,  1.86s/it]
 61%|██████▏   | 11427/18627 [6:20:41<3:52:34,  1.94s/it]
 61%|██████▏   | 11428/18627 [6:20:43<3:58:15,  1.99s/it]
 61%|██████▏   | 11429/18627 [6:20:45<4:03:45,  2.03s/it]
 61%|██████▏   | 11430/18627 [6:20:48<4:05:51,  2.05s/it]
                                                         
{'loss': 1.1269, 'grad_norm': 9.556784629821777, 'learning_rate': 1.7149830923256477e-06, 'epoch': 0.61}
+
 61%|██████▏   | 11430/18627 [6:20:48<4:05:51,  2.05s/it]
 61%|██████▏   | 11431/18627 [6:20:50<4:08:32,  2.07s/it]
 61%|██████▏   | 11432/18627 [6:20:52<4:10:03,  2.09s/it]
 61%|██████▏   | 11433/18627 [6:20:53<3:37:45,  1.82s/it]
 61%|██████▏   | 11434/18627 [6:20:55<3:49:08,  1.91s/it]
 61%|██████▏   | 11435/18627 [6:20:57<3:55:48,  1.97s/it]
 61%|██████▏   | 11436/18627 [6:20:58<3:27:40,  1.73s/it]
 61%|██████▏   | 11437/18627 [6:21:01<3:41:52,  1.85s/it]
 61%|██████▏   | 11438/18627 [6:21:02<3:11:36,  1.60s/it]
 61%|██████▏   | 11439/18627 [6:21:04<3:30:35,  1.76s/it]
 61%|██████▏   | 11440/18627 [6:21:06<3:43:03,  1.86s/it]
                                                         
{'loss': 1.2005, 'grad_norm': 10.16091537475586, 'learning_rate': 1.7108572426046827e-06, 'epoch': 0.61}
+
 61%|██████▏   | 11440/18627 [6:21:06<3:43:03,  1.86s/it]
 61%|██████▏   | 11441/18627 [6:21:08<3:51:15,  1.93s/it]
 61%|██████▏   | 11442/18627 [6:21:10<3:56:58,  1.98s/it]
 61%|██████▏   | 11443/18627 [6:21:11<3:28:51,  1.74s/it]
 61%|██████▏   | 11444/18627 [6:21:13<3:42:49,  1.86s/it]
 61%|██████▏   | 11445/18627 [6:21:15<3:51:13,  1.93s/it]
 61%|██████▏   | 11446/18627 [6:21:17<3:57:18,  1.98s/it]
 61%|██████▏   | 11447/18627 [6:21:19<3:28:30,  1.74s/it]
 61%|██████▏   | 11448/18627 [6:21:20<3:05:57,  1.55s/it]
 61%|██████▏   | 11449/18627 [6:21:22<3:26:10,  1.72s/it]
 61%|██████▏   | 11450/18627 [6:21:24<3:40:30,  1.84s/it]
                                                         
{'loss': 1.2756, 'grad_norm': 11.08980655670166, 'learning_rate': 1.7067337786892824e-06, 'epoch': 0.61}
+
 61%|██████▏   | 11450/18627 [6:21:24<3:40:30,  1.84s/it]
 61%|██████▏   | 11451/18627 [6:21:26<3:50:45,  1.93s/it]
 61%|██████▏   | 11452/18627 [6:21:27<3:20:31,  1.68s/it]
 61%|██████▏   | 11453/18627 [6:21:29<3:35:52,  1.81s/it]
 61%|██████▏   | 11454/18627 [6:21:31<3:47:27,  1.90s/it]
 61%|██████▏   | 11455/18627 [6:21:34<3:55:34,  1.97s/it]
 62%|██████▏   | 11456/18627 [6:21:35<3:27:22,  1.74s/it]
 62%|██████▏   | 11457/18627 [6:21:37<3:41:36,  1.85s/it]
 62%|██████▏   | 11458/18627 [6:21:39<3:50:46,  1.93s/it]
 62%|██████▏   | 11459/18627 [6:21:41<3:57:51,  1.99s/it]
 62%|██████▏   | 11460/18627 [6:21:43<4:02:31,  2.03s/it]
                                                         
{'loss': 1.0098, 'grad_norm': 5.239773273468018, 'learning_rate': 1.7026127130458633e-06, 'epoch': 0.62}
+
 62%|██████▏   | 11460/18627 [6:21:43<4:02:31,  2.03s/it]
 62%|██████▏   | 11461/18627 [6:21:44<3:32:16,  1.78s/it]
 62%|██████▏   | 11462/18627 [6:21:47<3:44:12,  1.88s/it]
 62%|██████▏   | 11463/18627 [6:21:49<3:52:33,  1.95s/it]
 62%|██████▏   | 11464/18627 [6:21:51<3:58:34,  2.00s/it]
 62%|██████▏   | 11465/18627 [6:21:52<3:29:37,  1.76s/it]
 62%|██████▏   | 11466/18627 [6:21:54<3:42:06,  1.86s/it]
 62%|██████▏   | 11467/18627 [6:21:56<3:51:33,  1.94s/it]
 62%|██████▏   | 11468/18627 [6:21:58<3:58:07,  2.00s/it]
 62%|██████▏   | 11469/18627 [6:22:00<4:02:22,  2.03s/it]
 62%|██████▏   | 11470/18627 [6:22:03<4:04:40,  2.05s/it]
                                                         
{'loss': 0.9719, 'grad_norm': 6.5852952003479, 'learning_rate': 1.6984940581335946e-06, 'epoch': 0.62}
+
 62%|██████▏   | 11470/18627 [6:22:03<4:04:40,  2.05s/it]
 62%|██████▏   | 11471/18627 [6:22:05<4:07:14,  2.07s/it]
 62%|██████▏   | 11472/18627 [6:22:07<4:08:07,  2.08s/it]
 62%|██████▏   | 11473/18627 [6:22:09<4:08:28,  2.08s/it]
 62%|██████▏   | 11474/18627 [6:22:10<3:18:50,  1.67s/it]
 62%|██████▏   | 11475/18627 [6:22:11<3:01:40,  1.52s/it]
 62%|██████▏   | 11476/18627 [6:22:12<2:49:42,  1.42s/it]
 62%|██████▏   | 11477/18627 [6:22:13<2:41:24,  1.35s/it]
 62%|██████▏   | 11478/18627 [6:22:15<3:08:06,  1.58s/it]
 62%|██████▏   | 11479/18627 [6:22:16<2:54:23,  1.46s/it]
 62%|██████▏   | 11480/18627 [6:22:19<3:17:23,  1.66s/it]
                                                         
{'loss': 1.4538, 'grad_norm': 6.731161117553711, 'learning_rate': 1.694377826404353e-06, 'epoch': 0.62}
+
 62%|██████▏   | 11480/18627 [6:22:19<3:17:23,  1.66s/it]
 62%|██████▏   | 11481/18627 [6:22:21<3:33:25,  1.79s/it]
 62%|██████▏   | 11482/18627 [6:22:23<3:45:16,  1.89s/it]
 62%|██████▏   | 11483/18627 [6:22:24<3:20:21,  1.68s/it]
 62%|██████▏   | 11484/18627 [6:22:26<3:35:48,  1.81s/it]
 62%|██████▏   | 11485/18627 [6:22:28<3:46:03,  1.90s/it]
 62%|██████▏   | 11486/18627 [6:22:29<3:21:23,  1.69s/it]
 62%|██████▏   | 11487/18627 [6:22:32<3:37:11,  1.83s/it]
 62%|██████▏   | 11488/18627 [6:22:34<3:47:50,  1.91s/it]
 62%|██████▏   | 11489/18627 [6:22:36<3:55:02,  1.98s/it]
 62%|██████▏   | 11490/18627 [6:22:38<3:59:36,  2.01s/it]
                                                         
{'loss': 0.9764, 'grad_norm': 4.720456600189209, 'learning_rate': 1.6902640303026924e-06, 'epoch': 0.62}
+
 62%|██████▏   | 11490/18627 [6:22:38<3:59:36,  2.01s/it]
 62%|██████▏   | 11491/18627 [6:22:40<4:03:35,  2.05s/it]
 62%|██████▏   | 11492/18627 [6:22:42<4:05:52,  2.07s/it]
 62%|██████▏   | 11493/18627 [6:22:44<4:08:02,  2.09s/it]
 62%|██████▏   | 11494/18627 [6:22:46<4:09:24,  2.10s/it]
 62%|██████▏   | 11495/18627 [6:22:48<4:09:44,  2.10s/it]
 62%|██████▏   | 11496/18627 [6:22:50<3:37:09,  1.83s/it]
 62%|██████▏   | 11497/18627 [6:22:51<3:14:39,  1.64s/it]
 62%|██████▏   | 11498/18627 [6:22:52<2:58:31,  1.50s/it]
 62%|██████▏   | 11499/18627 [6:22:54<3:20:50,  1.69s/it]
 62%|██████▏   | 11500/18627 [6:22:56<3:36:07,  1.82s/it]
                                                         
{'loss': 1.2718, 'grad_norm': 6.29373025894165, 'learning_rate': 1.6861526822657997e-06, 'epoch': 0.62}
+
 62%|██████▏   | 11500/18627 [6:22:56<3:36:07,  1.82s/it]
 62%|██████▏   | 11501/18627 [6:22:58<3:46:37,  1.91s/it]
 62%|██████▏   | 11502/18627 [6:23:01<3:54:35,  1.98s/it]
 62%|██████▏   | 11503/18627 [6:23:03<3:59:34,  2.02s/it]
 62%|██████▏   | 11504/18627 [6:23:05<4:03:05,  2.05s/it]
 62%|██████▏   | 11505/18627 [6:23:06<3:32:25,  1.79s/it]
 62%|██████▏   | 11506/18627 [6:23:08<3:44:52,  1.89s/it]
 62%|██████▏   | 11507/18627 [6:23:10<3:52:55,  1.96s/it]
 62%|██████▏   | 11508/18627 [6:23:12<3:58:40,  2.01s/it]
 62%|██████▏   | 11509/18627 [6:23:14<4:01:34,  2.04s/it]
 62%|██████▏   | 11510/18627 [6:23:16<3:31:55,  1.79s/it]
                                                         
{'loss': 1.0908, 'grad_norm': 12.66163158416748, 'learning_rate': 1.6820437947234654e-06, 'epoch': 0.62}
+
 62%|██████▏   | 11510/18627 [6:23:16<3:31:55,  1.79s/it]
 62%|██████▏   | 11511/18627 [6:23:18<3:43:39,  1.89s/it]
 62%|██████▏   | 11512/18627 [6:23:20<3:52:09,  1.96s/it]
 62%|██████▏   | 11513/18627 [6:23:22<3:57:41,  2.00s/it]
 62%|██████▏   | 11514/18627 [6:23:24<4:01:31,  2.04s/it]
 62%|██████▏   | 11515/18627 [6:23:26<4:03:28,  2.05s/it]
 62%|██████▏   | 11516/18627 [6:23:28<4:05:52,  2.07s/it]
 62%|██████▏   | 11517/18627 [6:23:30<4:07:38,  2.09s/it]
 62%|██████▏   | 11518/18627 [6:23:33<4:08:18,  2.10s/it]
 62%|██████▏   | 11519/18627 [6:23:35<4:09:31,  2.11s/it]
 62%|██████▏   | 11520/18627 [6:23:37<4:09:52,  2.11s/it]
                                                         
{'loss': 0.6365, 'grad_norm': 7.9806952476501465, 'learning_rate': 1.677937380098037e-06, 'epoch': 0.62}
+
 62%|██████▏   | 11520/18627 [6:23:37<4:09:52,  2.11s/it]
 62%|██████▏   | 11521/18627 [6:23:39<4:09:54,  2.11s/it]
 62%|██████▏   | 11522/18627 [6:23:41<4:10:27,  2.12s/it]
 62%|██████▏   | 11523/18627 [6:23:43<4:10:45,  2.12s/it]
 62%|██████▏   | 11524/18627 [6:23:45<4:10:14,  2.11s/it]
 62%|██████▏   | 11525/18627 [6:23:47<4:09:54,  2.11s/it]
 62%|██████▏   | 11526/18627 [6:23:50<4:10:24,  2.12s/it]
 62%|██████▏   | 11527/18627 [6:23:50<3:19:47,  1.69s/it]
 62%|██████▏   | 11528/18627 [6:23:52<3:35:18,  1.82s/it]
 62%|██████▏   | 11529/18627 [6:23:53<2:57:35,  1.50s/it]
 62%|██████▏   | 11530/18627 [6:23:55<3:19:08,  1.68s/it]
                                                         
{'loss': 0.6515, 'grad_norm': 5.7943525314331055, 'learning_rate': 1.673833450804388e-06, 'epoch': 0.62}
+
 62%|██████▏   | 11530/18627 [6:23:55<3:19:08,  1.68s/it]
 62%|██████▏   | 11531/18627 [6:23:57<3:34:42,  1.82s/it]
 62%|██████▏   | 11532/18627 [6:23:59<3:12:35,  1.63s/it]
 62%|██████▏   | 11533/18627 [6:24:01<3:29:42,  1.77s/it]
 62%|██████▏   | 11534/18627 [6:24:03<3:41:10,  1.87s/it]
 62%|██████▏   | 11535/18627 [6:24:05<3:49:48,  1.94s/it]
 62%|██████▏   | 11536/18627 [6:24:06<3:23:13,  1.72s/it]
 62%|██████▏   | 11537/18627 [6:24:08<3:37:22,  1.84s/it]
 62%|██████▏   | 11538/18627 [6:24:10<3:46:26,  1.92s/it]
 62%|██████▏   | 11539/18627 [6:24:11<3:20:58,  1.70s/it]
 62%|██████▏   | 11540/18627 [6:24:14<3:34:58,  1.82s/it]
                                                         
{'loss': 1.2059, 'grad_norm': 5.5296196937561035, 'learning_rate': 1.6697320192498752e-06, 'epoch': 0.62}
+
 62%|██████▏   | 11540/18627 [6:24:14<3:34:58,  1.82s/it]
 62%|██████▏   | 11541/18627 [6:24:16<3:46:08,  1.91s/it]
 62%|██████▏   | 11542/18627 [6:24:17<3:20:25,  1.70s/it]
 62%|██████▏   | 11543/18627 [6:24:19<3:34:45,  1.82s/it]
 62%|██████▏   | 11544/18627 [6:24:21<3:44:41,  1.90s/it]
 62%|██████▏   | 11545/18627 [6:24:23<3:53:06,  1.97s/it]
 62%|██████▏   | 11546/18627 [6:24:25<3:59:59,  2.03s/it]
 62%|██████▏   | 11547/18627 [6:24:27<3:30:29,  1.78s/it]
 62%|██████▏   | 11548/18627 [6:24:29<3:43:29,  1.89s/it]
 62%|██████▏   | 11549/18627 [6:24:31<3:52:24,  1.97s/it]
 62%|██████▏   | 11550/18627 [6:24:33<3:58:54,  2.03s/it]
                                                         
{'loss': 1.0391, 'grad_norm': 6.310262680053711, 'learning_rate': 1.665633097834309e-06, 'epoch': 0.62}
+
 62%|██████▏   | 11550/18627 [6:24:33<3:58:54,  2.03s/it]
 62%|██████▏   | 11551/18627 [6:24:35<4:02:51,  2.06s/it]
 62%|██████▏   | 11552/18627 [6:24:37<4:05:43,  2.08s/it]
 62%|██████▏   | 11553/18627 [6:24:39<4:06:34,  2.09s/it]
 62%|██████▏   | 11554/18627 [6:24:42<4:07:18,  2.10s/it]
 62%|██████▏   | 11555/18627 [6:24:44<4:08:16,  2.11s/it]
 62%|██████▏   | 11556/18627 [6:24:46<4:08:51,  2.11s/it]
 62%|██████▏   | 11557/18627 [6:24:48<4:08:32,  2.11s/it]
 62%|██████▏   | 11558/18627 [6:24:50<4:09:19,  2.12s/it]
 62%|██████▏   | 11559/18627 [6:24:51<3:36:33,  1.84s/it]
 62%|██████▏   | 11560/18627 [6:24:53<3:46:30,  1.92s/it]
                                                         
{'loss': 0.766, 'grad_norm': 5.145846843719482, 'learning_rate': 1.661536698949906e-06, 'epoch': 0.62}
+
 62%|██████▏   | 11560/18627 [6:24:53<3:46:30,  1.92s/it]
 62%|██████▏   | 11561/18627 [6:24:55<3:52:48,  1.98s/it]
 62%|██████▏   | 11562/18627 [6:24:57<3:24:54,  1.74s/it]
 62%|██████▏   | 11563/18627 [6:24:59<3:38:16,  1.85s/it]
 62%|██████▏   | 11564/18627 [6:25:01<3:47:53,  1.94s/it]
 62%|██████▏   | 11565/18627 [6:25:02<3:21:13,  1.71s/it]
 62%|██████▏   | 11566/18627 [6:25:04<3:35:29,  1.83s/it]
 62%|██████▏   | 11567/18627 [6:25:06<3:44:49,  1.91s/it]
 62%|██████▏   | 11568/18627 [6:25:07<3:19:35,  1.70s/it]
 62%|██████▏   | 11569/18627 [6:25:10<3:34:51,  1.83s/it]
 62%|██████▏   | 11570/18627 [6:25:11<3:12:06,  1.63s/it]
                                                         
{'loss': 1.393, 'grad_norm': 14.83370590209961, 'learning_rate': 1.657442834981259e-06, 'epoch': 0.62}
+
 62%|██████▏   | 11570/18627 [6:25:11<3:12:06,  1.63s/it]
 62%|██████▏   | 11571/18627 [6:25:13<3:28:36,  1.77s/it]
 62%|██████▏   | 11572/18627 [6:25:15<3:40:38,  1.88s/it]
 62%|██████▏   | 11573/18627 [6:25:17<3:49:18,  1.95s/it]
 62%|██████▏   | 11574/18627 [6:25:19<3:54:40,  2.00s/it]
 62%|██████▏   | 11575/18627 [6:25:21<3:58:37,  2.03s/it]
 62%|██████▏   | 11576/18627 [6:25:23<4:01:58,  2.06s/it]
 62%|██████▏   | 11577/18627 [6:25:26<4:04:04,  2.08s/it]
 62%|██████▏   | 11578/18627 [6:25:27<3:32:41,  1.81s/it]
 62%|██████▏   | 11579/18627 [6:25:29<3:43:23,  1.90s/it]
 62%|██████▏   | 11580/18627 [6:25:31<3:50:28,  1.96s/it]
                                                         
{'loss': 0.8196, 'grad_norm': 5.297738552093506, 'learning_rate': 1.6533515183052957e-06, 'epoch': 0.62}
+
 62%|██████▏   | 11580/18627 [6:25:31<3:50:28,  1.96s/it]
 62%|██████▏   | 11581/18627 [6:25:33<3:55:39,  2.01s/it]
 62%|██████▏   | 11582/18627 [6:25:35<3:59:57,  2.04s/it]
 62%|██████▏   | 11583/18627 [6:25:37<4:02:35,  2.07s/it]
 62%|██████▏   | 11584/18627 [6:25:39<4:04:33,  2.08s/it]
 62%|██████▏   | 11585/18627 [6:25:41<3:33:40,  1.82s/it]
 62%|██████▏   | 11586/18627 [6:25:43<3:43:56,  1.91s/it]
 62%|██████▏   | 11587/18627 [6:25:44<3:19:05,  1.70s/it]
 62%|██████▏   | 11588/18627 [6:25:45<2:58:20,  1.52s/it]
 62%|██████▏   | 11589/18627 [6:25:46<2:46:52,  1.42s/it]
 62%|██████▏   | 11590/18627 [6:25:48<2:39:08,  1.36s/it]
                                                         
{'loss': 1.7275, 'grad_norm': 15.334333419799805, 'learning_rate': 1.6492627612912448e-06, 'epoch': 0.62}
+
 62%|██████▏   | 11590/18627 [6:25:48<2:39:08,  1.36s/it]
 62%|██████▏   | 11591/18627 [6:25:50<3:06:03,  1.59s/it]
 62%|██████▏   | 11592/18627 [6:25:52<3:24:53,  1.75s/it]
 62%|██████▏   | 11593/18627 [6:25:54<3:37:21,  1.85s/it]
 62%|██████▏   | 11594/18627 [6:25:55<3:13:43,  1.65s/it]
 62%|██████▏   | 11595/18627 [6:25:57<3:29:01,  1.78s/it]
 62%|██████▏   | 11596/18627 [6:25:58<3:04:23,  1.57s/it]
 62%|██████▏   | 11597/18627 [6:25:59<2:50:57,  1.46s/it]
 62%|██████▏   | 11598/18627 [6:26:02<3:14:03,  1.66s/it]
 62%|██████▏   | 11599/18627 [6:26:03<2:58:10,  1.52s/it]
 62%|██████▏   | 11600/18627 [6:26:04<2:43:28,  1.40s/it]
                                                         
{'loss': 1.5967, 'grad_norm': 13.57276725769043, 'learning_rate': 1.6451765763005937e-06, 'epoch': 0.62}
+
 62%|██████▏   | 11600/18627 [6:26:04<2:43:28,  1.40s/it]
 62%|██████▏   | 11601/18627 [6:26:06<3:08:58,  1.61s/it]
 62%|██████▏   | 11602/18627 [6:26:08<3:26:59,  1.77s/it]
 62%|██████▏   | 11603/18627 [6:26:10<3:38:59,  1.87s/it]
 62%|██████▏   | 11604/18627 [6:26:12<3:47:47,  1.95s/it]
 62%|██████▏   | 11605/18627 [6:26:14<3:53:26,  1.99s/it]
 62%|██████▏   | 11606/18627 [6:26:17<3:58:09,  2.04s/it]
 62%|██████▏   | 11607/18627 [6:26:19<4:00:42,  2.06s/it]
 62%|██████▏   | 11608/18627 [6:26:21<4:03:07,  2.08s/it]
 62%|██████▏   | 11609/18627 [6:26:23<4:04:03,  2.09s/it]
 62%|██████▏   | 11610/18627 [6:26:25<4:05:36,  2.10s/it]
                                                         
{'loss': 0.6083, 'grad_norm': 12.05284595489502, 'learning_rate': 1.6410929756870558e-06, 'epoch': 0.62}
+
 62%|██████▏   | 11610/18627 [6:26:25<4:05:36,  2.10s/it]
 62%|██████▏   | 11611/18627 [6:26:27<4:05:52,  2.10s/it]
 62%|██████▏   | 11612/18627 [6:26:28<3:34:11,  1.83s/it]
 62%|██████▏   | 11613/18627 [6:26:30<3:44:19,  1.92s/it]
 62%|██████▏   | 11614/18627 [6:26:33<3:51:24,  1.98s/it]
 62%|██████▏   | 11615/18627 [6:26:35<3:56:08,  2.02s/it]
 62%|██████▏   | 11616/18627 [6:26:36<3:26:52,  1.77s/it]
 62%|██████▏   | 11617/18627 [6:26:38<3:39:29,  1.88s/it]
 62%|██████▏   | 11618/18627 [6:26:39<3:15:29,  1.67s/it]
 62%|██████▏   | 11619/18627 [6:26:41<3:33:58,  1.83s/it]
 62%|██████▏   | 11620/18627 [6:26:44<3:44:21,  1.92s/it]
                                                         
{'loss': 1.1741, 'grad_norm': 7.541871070861816, 'learning_rate': 1.6370119717965293e-06, 'epoch': 0.62}
+
 62%|██████▏   | 11620/18627 [6:26:44<3:44:21,  1.92s/it]
 62%|██████▏   | 11621/18627 [6:26:46<3:51:31,  1.98s/it]
 62%|██████▏   | 11622/18627 [6:26:47<3:17:37,  1.69s/it]
 62%|██████▏   | 11623/18627 [6:26:49<3:32:40,  1.82s/it]
 62%|██████▏   | 11624/18627 [6:26:51<3:42:14,  1.90s/it]
 62%|██████▏   | 11625/18627 [6:26:53<3:49:13,  1.96s/it]
 62%|██████▏   | 11626/18627 [6:26:55<3:54:16,  2.01s/it]
 62%|██████▏   | 11627/18627 [6:26:57<3:57:48,  2.04s/it]
 62%|██████▏   | 11628/18627 [6:26:59<3:59:45,  2.06s/it]
 62%|██████▏   | 11629/18627 [6:27:02<4:03:49,  2.09s/it]
 62%|██████▏   | 11630/18627 [6:27:04<4:06:44,  2.12s/it]
                                                         
{'loss': 0.7808, 'grad_norm': 6.844648361206055, 'learning_rate': 1.6329335769670648e-06, 'epoch': 0.62}
+
 62%|████���█▏   | 11630/18627 [6:27:04<4:06:44,  2.12s/it]
 62%|██████▏   | 11631/18627 [6:27:06<4:06:58,  2.12s/it]
 62%|██████▏   | 11632/18627 [6:27:08<4:06:01,  2.11s/it]
 62%|██████▏   | 11633/18627 [6:27:10<4:06:26,  2.11s/it]
 62%|██████▏   | 11634/18627 [6:27:11<3:34:14,  1.84s/it]
 62%|██████▏   | 11635/18627 [6:27:13<3:44:25,  1.93s/it]
 62%|██████▏   | 11636/18627 [6:27:15<3:51:23,  1.99s/it]
 62%|██████▏   | 11637/18627 [6:27:18<3:55:33,  2.02s/it]
 62%|██████▏   | 11638/18627 [6:27:20<3:57:58,  2.04s/it]
 62%|██████▏   | 11639/18627 [6:27:22<3:59:49,  2.06s/it]
 62%|██████▏   | 11640/18627 [6:27:24<4:02:14,  2.08s/it]
                                                         
{'loss': 0.8447, 'grad_norm': 5.326478004455566, 'learning_rate': 1.6288578035288216e-06, 'epoch': 0.62}
+
 62%|██████▏   | 11640/18627 [6:27:24<4:02:14,  2.08s/it]
 62%|██████▏   | 11641/18627 [6:27:26<4:03:24,  2.09s/it]
 63%|██████▎   | 11642/18627 [6:27:28<4:03:30,  2.09s/it]
 63%|██████▎   | 11643/18627 [6:27:30<4:04:51,  2.10s/it]
 63%|██████▎   | 11644/18627 [6:27:32<4:05:06,  2.11s/it]
 63%|██████▎   | 11645/18627 [6:27:34<4:05:31,  2.11s/it]
 63%|██████▎   | 11646/18627 [6:27:37<4:05:10,  2.11s/it]
 63%|██████▎   | 11647/18627 [6:27:39<4:05:32,  2.11s/it]
 63%|██████▎   | 11648/18627 [6:27:41<4:05:52,  2.11s/it]
 63%|██████▎   | 11649/18627 [6:27:43<4:06:51,  2.12s/it]
 63%|██████▎   | 11650/18627 [6:27:45<4:06:59,  2.12s/it]
                                                         
{'loss': 0.5825, 'grad_norm': 5.819628715515137, 'learning_rate': 1.6247846638040365e-06, 'epoch': 0.63}
+
 63%|██████▎   | 11650/18627 [6:27:45<4:06:59,  2.12s/it]
 63%|██████▎   | 11651/18627 [6:27:47<4:06:48,  2.12s/it]
 63%|██████▎   | 11652/18627 [6:27:49<4:06:28,  2.12s/it]
 63%|██████▎   | 11653/18627 [6:27:51<4:05:59,  2.12s/it]
 63%|██████▎   | 11654/18627 [6:27:54<4:06:13,  2.12s/it]
 63%|██████▎   | 11655/18627 [6:27:55<3:33:49,  1.84s/it]
 63%|██████▎   | 11656/18627 [6:27:57<3:43:08,  1.92s/it]
 63%|██████▎   | 11657/18627 [6:27:59<3:49:37,  1.98s/it]
 63%|██████▎   | 11658/18627 [6:28:01<3:53:55,  2.01s/it]
 63%|██████▎   | 11659/18627 [6:28:03<3:57:29,  2.04s/it]
 63%|██████▎   | 11660/18627 [6:28:05<3:59:35,  2.06s/it]
                                                         
{'loss': 0.8654, 'grad_norm': 12.920642852783203, 'learning_rate': 1.6207141701069804e-06, 'epoch': 0.63}
+
 63%|██████▎   | 11660/18627 [6:28:05<3:59:35,  2.06s/it]
 63%|██████▎   | 11661/18627 [6:28:06<3:29:04,  1.80s/it]
 63%|██████▎   | 11662/18627 [6:28:09<3:40:03,  1.90s/it]
 63%|██████▎   | 11663/18627 [6:28:11<3:47:30,  1.96s/it]
 63%|██████▎   | 11664/18627 [6:28:13<3:53:19,  2.01s/it]
 63%|██████▎   | 11665/18627 [6:28:14<3:25:00,  1.77s/it]
 63%|██████▎   | 11666/18627 [6:28:16<3:37:25,  1.87s/it]
 63%|██████▎   | 11667/18627 [6:28:18<3:46:02,  1.95s/it]
 63%|██████▎   | 11668/18627 [6:28:20<3:53:09,  2.01s/it]
 63%|██████▎   | 11669/18627 [6:28:22<3:24:23,  1.76s/it]
 63%|██████▎   | 11670/18627 [6:28:24<3:37:08,  1.87s/it]
                                                         
{'loss': 1.0986, 'grad_norm': 4.553651332855225, 'learning_rate': 1.6166463347439292e-06, 'epoch': 0.63}
+
 63%|██████▎   | 11670/18627 [6:28:24<3:37:08,  1.87s/it]
 63%|██████▎   | 11671/18627 [6:28:25<3:13:22,  1.67s/it]
 63%|██████▎   | 11672/18627 [6:28:27<3:28:42,  1.80s/it]
 63%|██████▎   | 11673/18627 [6:28:29<3:40:02,  1.90s/it]
 63%|██████▎   | 11674/18627 [6:28:30<3:15:20,  1.69s/it]
 63%|██████▎   | 11675/18627 [6:28:32<3:29:40,  1.81s/it]
 63%|██████▎   | 11676/18627 [6:28:33<2:51:17,  1.48s/it]
 63%|██████▎   | 11677/18627 [6:28:35<3:12:57,  1.67s/it]
 63%|██████▎   | 11678/18627 [6:28:37<3:28:51,  1.80s/it]
 63%|██████▎   | 11679/18627 [6:28:40<3:39:57,  1.90s/it]
 63%|██████▎   | 11680/18627 [6:28:41<3:13:18,  1.67s/it]
                                                         
{'loss': 1.1029, 'grad_norm': 13.6753568649292, 'learning_rate': 1.6125811700131177e-06, 'epoch': 0.63}
+
 63%|██████▎   | 11680/18627 [6:28:41<3:13:18,  1.67s/it]
 63%|██████▎   | 11681/18627 [6:28:43<3:28:23,  1.80s/it]
 63%|██████▎   | 11682/18627 [6:28:44<3:07:33,  1.62s/it]
 63%|██████▎   | 11683/18627 [6:28:46<3:24:23,  1.77s/it]
 63%|██████▎   | 11684/18627 [6:28:47<3:04:48,  1.60s/it]
 63%|██████▎   | 11685/18627 [6:28:52<4:42:01,  2.44s/it]
 63%|██████▎   | 11686/18627 [6:28:54<4:40:02,  2.42s/it]
 63%|██████▎   | 11687/18627 [6:28:56<4:37:43,  2.40s/it]
 63%|██████▎   | 11688/18627 [6:28:59<4:35:57,  2.39s/it]
 63%|██████▎   | 11689/18627 [6:29:01<4:35:06,  2.38s/it]
 63%|██████▎   | 11690/18627 [6:29:03<4:33:57,  2.37s/it]
                                                         
{'loss': 1.0844, 'grad_norm': 7.241902828216553, 'learning_rate': 1.6085186882047098e-06, 'epoch': 0.63}
+
 63%|██████▎   | 11690/18627 [6:29:03<4:33:57,  2.37s/it]
 63%|██████▎   | 11691/18627 [6:29:06<4:32:59,  2.36s/it]
 63%|██████▎   | 11692/18627 [6:29:07<3:58:37,  2.06s/it]
 63%|██████▎   | 11693/18627 [6:29:10<4:09:45,  2.16s/it]
 63%|██████▎   | 11694/18627 [6:29:12<4:16:48,  2.22s/it]
 63%|██████▎   | 11695/18627 [6:29:13<3:47:08,  1.97s/it]
 63%|██████▎   | 11696/18627 [6:29:16<4:01:11,  2.09s/it]
 63%|██████▎   | 11697/18627 [6:29:18<4:10:00,  2.16s/it]
 63%|██████▎   | 11698/18627 [6:29:20<4:17:22,  2.23s/it]
 63%|██████▎   | 11699/18627 [6:29:23<4:20:00,  2.25s/it]
 63%|██████▎   | 11700/18627 [6:29:25<4:23:14,  2.28s/it]
                                                         
{'loss': 1.061, 'grad_norm': 10.133666038513184, 'learning_rate': 1.6044589016007545e-06, 'epoch': 0.63}
+
 63%|██████▎   | 11700/18627 [6:29:25<4:23:14,  2.28s/it]
 63%|██████▎   | 11701/18627 [6:29:26<3:42:58,  1.93s/it]
 63%|██████▎   | 11702/18627 [6:29:29<3:58:22,  2.07s/it]
 63%|██████▎   | 11703/18627 [6:29:31<4:08:05,  2.15s/it]
 63%|██████▎   | 11704/18627 [6:29:33<4:13:48,  2.20s/it]
 63%|██████▎   | 11705/18627 [6:29:35<3:46:07,  1.96s/it]
 63%|██████▎   | 11706/18627 [6:29:37<3:59:47,  2.08s/it]
 63%|██████▎   | 11707/18627 [6:29:39<4:14:35,  2.21s/it]
 63%|██████▎   | 11708/18627 [6:29:42<4:19:47,  2.25s/it]
 63%|██████▎   | 11709/18627 [6:29:43<3:51:07,  2.00s/it]
 63%|██████▎   | 11710/18627 [6:29:46<4:03:59,  2.12s/it]
                                                         
{'loss': 0.9622, 'grad_norm': 4.733138084411621, 'learning_rate': 1.600401822475156e-06, 'epoch': 0.63}
+
 63%|██████▎   | 11710/18627 [6:29:46<4:03:59,  2.12s/it]
 63%|██████▎   | 11711/18627 [6:29:48<4:12:55,  2.19s/it]
 63%|██████▎   | 11712/18627 [6:29:50<4:18:50,  2.25s/it]
 63%|██████▎   | 11713/18627 [6:29:52<3:47:44,  1.98s/it]
 63%|██████▎   | 11714/18627 [6:29:54<4:01:12,  2.09s/it]
 63%|██████▎   | 11715/18627 [6:29:55<3:37:00,  1.88s/it]
 63%|██████▎   | 11716/18627 [6:29:57<3:19:13,  1.73s/it]
 63%|██████▎   | 11717/18627 [6:29:59<3:40:38,  1.92s/it]
 63%|██████▎   | 11718/18627 [6:30:01<3:54:24,  2.04s/it]
 63%|██████▎   | 11719/18627 [6:30:04<4:05:06,  2.13s/it]
 63%|██████▎   | 11720/18627 [6:30:05<3:39:08,  1.90s/it]
                                                         
{'loss': 1.3745, 'grad_norm': 14.35534954071045, 'learning_rate': 1.5963474630936323e-06, 'epoch': 0.63}
+
 63%|██████▎   | 11720/18627 [6:30:05<3:39:08,  1.90s/it]
 63%|██████▎   | 11721/18627 [6:30:08<3:54:41,  2.04s/it]
 63%|██████▎   | 11722/18627 [6:30:10<4:05:09,  2.13s/it]
 63%|██████▎   | 11723/18627 [6:30:12<4:11:37,  2.19s/it]
 63%|██████▎   | 11724/18627 [6:30:15<4:16:17,  2.23s/it]
 63%|██████▎   | 11725/18627 [6:30:17<4:20:37,  2.27s/it]
 63%|██████▎   | 11726/18627 [6:30:19<4:23:24,  2.29s/it]
 63%|██████▎   | 11727/18627 [6:30:22<4:25:32,  2.31s/it]
 63%|██████▎   | 11728/18627 [6:30:23<3:54:15,  2.04s/it]
 63%|██████▎   | 11729/18627 [6:30:25<4:04:55,  2.13s/it]
 63%|██████▎   | 11730/18627 [6:30:27<3:42:02,  1.93s/it]
                                                         
{'loss': 1.0341, 'grad_norm': 15.016681671142578, 'learning_rate': 1.5922958357136759e-06, 'epoch': 0.63}
+
 63%|██████▎   | 11730/18627 [6:30:27<3:42:02,  1.93s/it]
 63%|██████▎   | 11731/18627 [6:30:29<3:55:45,  2.05s/it]
 63%|██████▎   | 11732/18627 [6:30:31<3:32:51,  1.85s/it]
 63%|██████▎   | 11733/18627 [6:30:32<3:15:30,  1.70s/it]
 63%|██████▎   | 11734/18627 [6:30:34<3:36:48,  1.89s/it]
 63%|██████▎   | 11735/18627 [6:30:36<3:19:52,  1.74s/it]
 63%|██████▎   | 11736/18627 [6:30:38<3:41:15,  1.93s/it]
 63%|██████▎   | 11737/18627 [6:30:39<3:19:53,  1.74s/it]
 63%|██████▎   | 11738/18627 [6:30:42<3:40:52,  1.92s/it]
 63%|██████▎   | 11739/18627 [6:30:44<3:54:13,  2.04s/it]
 63%|██████▎   | 11740/18627 [6:30:46<4:05:41,  2.14s/it]
                                                         
{'loss': 1.4679, 'grad_norm': 9.442488670349121, 'learning_rate': 1.5882469525845245e-06, 'epoch': 0.63}
+
 63%|██████▎   | 11740/18627 [6:30:46<4:05:41,  2.14s/it]
 63%|██████▎   | 11741/18627 [6:30:49<4:12:26,  2.20s/it]
 63%|██████▎   | 11742/18627 [6:30:50<3:44:37,  1.96s/it]
 63%|██████▎   | 11743/18627 [6:30:52<3:56:52,  2.06s/it]
 63%|██████▎   | 11744/18627 [6:30:55<4:06:48,  2.15s/it]
 63%|██████▎   | 11745/18627 [6:30:57<4:12:34,  2.20s/it]
 63%|██████▎   | 11746/18627 [6:30:58<3:46:13,  1.97s/it]
 63%|██████▎   | 11747/18627 [6:31:01<4:00:09,  2.09s/it]
 63%|██████▎   | 11748/18627 [6:31:03<4:08:51,  2.17s/it]
 63%|██████▎   | 11749/18627 [6:31:06<4:14:18,  2.22s/it]
 63%|██████▎   | 11750/18627 [6:31:07<3:45:37,  1.97s/it]
                                                         
{'loss': 1.2613, 'grad_norm': 13.3638334274292, 'learning_rate': 1.5842008259471143e-06, 'epoch': 0.63}
+
 63%|██████▎   | 11750/18627 [6:31:07<3:45:37,  1.97s/it]
 63%|██████▎   | 11751/18627 [6:31:09<4:00:53,  2.10s/it]
 63%|██████▎   | 11752/18627 [6:31:12<4:10:28,  2.19s/it]
 63%|██████▎   | 11753/18627 [6:31:13<3:43:01,  1.95s/it]
 63%|██████▎   | 11754/18627 [6:31:15<3:56:10,  2.06s/it]
 63%|██████▎   | 11755/18627 [6:31:18<4:05:24,  2.14s/it]
 63%|██████▎   | 11756/18627 [6:31:19<3:39:22,  1.92s/it]
 63%|██████▎   | 11757/18627 [6:31:22<3:54:00,  2.04s/it]
 63%|██████▎   | 11758/18627 [6:31:24<4:04:17,  2.13s/it]
 63%|██████▎   | 11759/18627 [6:31:26<4:11:16,  2.20s/it]
 63%|██████▎   | 11760/18627 [6:31:28<3:44:08,  1.96s/it]
                                                         
{'loss': 1.1518, 'grad_norm': 12.684722900390625, 'learning_rate': 1.5801574680340514e-06, 'epoch': 0.63}
+
 63%|██████▎   | 11760/18627 [6:31:28<3:44:08,  1.96s/it]
 63%|██████▎   | 11761/18627 [6:31:30<3:58:36,  2.09s/it]
 63%|██████▎   | 11762/18627 [6:31:31<3:35:18,  1.88s/it]
 63%|██████▎   | 11763/18627 [6:31:34<3:51:15,  2.02s/it]
 63%|██████▎   | 11764/18627 [6:31:36<4:01:35,  2.11s/it]
 63%|██████▎   | 11765/18627 [6:31:38<4:08:19,  2.17s/it]
 63%|██████▎   | 11766/18627 [6:31:40<3:40:58,  1.93s/it]
 63%|██████▎   | 11767/18627 [6:31:42<3:54:58,  2.06s/it]
 63%|██████▎   | 11768/18627 [6:31:43<3:32:11,  1.86s/it]
 63%|██████▎   | 11769/18627 [6:31:45<3:16:30,  1.72s/it]
 63%|██████▎   | 11770/18627 [6:31:47<3:38:37,  1.91s/it]
                                                         
{'loss': 1.4765, 'grad_norm': 4.621258735656738, 'learning_rate': 1.5761168910695687e-06, 'epoch': 0.63}
+
 63%|██████▎   | 11770/18627 [6:31:47<3:38:37,  1.91s/it]
 63%|██████▎   | 11771/18627 [6:31:50<3:54:49,  2.06s/it]
 63%|██████▎   | 11772/18627 [6:31:51<3:33:03,  1.86s/it]
 63%|██████▎   | 11773/18627 [6:31:53<3:49:39,  2.01s/it]
 63%|██████▎   | 11774/18627 [6:31:56<4:01:57,  2.12s/it]
 63%|██████▎   | 11775/18627 [6:31:58<4:10:07,  2.19s/it]
 63%|██████▎   | 11776/18627 [6:31:59<3:41:05,  1.94s/it]
 63%|██████▎   | 11777/18627 [6:32:02<3:54:43,  2.06s/it]
 63%|██████▎   | 11778/18627 [6:32:03<3:31:57,  1.86s/it]
 63%|██████▎   | 11779/18627 [6:32:06<3:49:24,  2.01s/it]
 63%|██████▎   | 11780/18627 [6:32:08<4:00:27,  2.11s/it]
                                                         
{'loss': 1.1081, 'grad_norm': 5.238236427307129, 'learning_rate': 1.5720791072694944e-06, 'epoch': 0.63}
+
 63%|██████▎   | 11780/18627 [6:32:08<4:00:27,  2.11s/it]
 63%|██████▎   | 11781/18627 [6:32:10<4:08:59,  2.18s/it]
 63%|██████▎   | 11782/18627 [6:32:13<4:14:21,  2.23s/it]
 63%|██████▎   | 11783/18627 [6:32:15<4:17:43,  2.26s/it]
 63%|██████▎   | 11784/18627 [6:32:17<4:20:23,  2.28s/it]
 63%|██████▎   | 11785/18627 [6:32:20<4:22:22,  2.30s/it]
 63%|██████▎   | 11786/18627 [6:32:22<4:22:47,  2.30s/it]
 63%|██████▎   | 11787/18627 [6:32:24<4:23:09,  2.31s/it]
 63%|██████▎   | 11788/18627 [6:32:27<4:23:09,  2.31s/it]
 63%|██████▎   | 11789/18627 [6:32:29<4:24:40,  2.32s/it]
 63%|██████▎   | 11790/18627 [6:32:30<3:48:59,  2.01s/it]
                                                         
{'loss': 0.7553, 'grad_norm': 12.762290000915527, 'learning_rate': 1.568044128841209e-06, 'epoch': 0.63}
+
 63%|██████▎   | 11790/18627 [6:32:30<3:48:59,  2.01s/it]
 63%|██████▎   | 11791/18627 [6:32:32<3:27:07,  1.82s/it]
 63%|██████▎   | 11792/18627 [6:32:34<3:44:24,  1.97s/it]
 63%|██████▎   | 11793/18627 [6:32:36<3:57:21,  2.08s/it]
 63%|██████▎   | 11794/18627 [6:32:39<4:05:38,  2.16s/it]
 63%|██████▎   | 11795/18627 [6:32:41<4:12:16,  2.22s/it]
 63%|██████▎   | 11796/18627 [6:32:43<4:16:42,  2.25s/it]
 63%|██████▎   | 11797/18627 [6:32:45<3:49:26,  2.02s/it]
 63%|██████▎   | 11798/18627 [6:32:47<4:01:12,  2.12s/it]
 63%|██████▎   | 11799/18627 [6:32:49<4:09:25,  2.19s/it]
 63%|██████▎   | 11800/18627 [6:32:52<4:15:15,  2.24s/it]
                                                         
{'loss': 0.9661, 'grad_norm': 5.382036209106445, 'learning_rate': 1.564011967983614e-06, 'epoch': 0.63}
+
 63%|██████▎   | 11800/18627 [6:32:52<4:15:15,  2.24s/it]
 63%|██████▎   | 11801/18627 [6:32:57<5:50:29,  3.08s/it]
 63%|██████▎   | 11802/18627 [6:33:01<6:39:23,  3.51s/it]
 63%|██████▎   | 11803/18627 [6:33:04<6:05:29,  3.21s/it]
 63%|██████▎   | 11804/18627 [6:33:06<5:35:35,  2.95s/it]
 63%|██████▎   | 11805/18627 [6:33:09<5:14:22,  2.76s/it]
 63%|██████▎   | 11806/18627 [6:33:11<4:59:31,  2.63s/it]
 63%|██████▎   | 11807/18627 [6:33:12<4:16:22,  2.26s/it]
 63%|██████▎   | 11808/18627 [6:33:14<3:48:02,  2.01s/it]
 63%|██████▎   | 11809/18627 [6:33:16<3:59:14,  2.11s/it]
 63%|██████▎   | 11810/18627 [6:33:18<4:06:51,  2.17s/it]
                                                         
{'loss': 1.0668, 'grad_norm': 5.524689197540283, 'learning_rate': 1.55998263688709e-06, 'epoch': 0.63}
+
 63%|██████▎   | 11810/18627 [6:33:18<4:06:51,  2.17s/it]
 63%|██████▎   | 11811/18627 [6:33:21<4:12:35,  2.22s/it]
 63%|██████▎   | 11812/18627 [6:33:22<3:45:07,  1.98s/it]
 63%|██████▎   | 11813/18627 [6:33:24<3:57:31,  2.09s/it]
 63%|██████▎   | 11814/18627 [6:33:27<4:05:53,  2.17s/it]
 63%|██████▎   | 11815/18627 [6:33:29<4:19:13,  2.28s/it]
 63%|██████▎   | 11816/18627 [6:33:32<4:20:31,  2.29s/it]
 63%|██████▎   | 11817/18627 [6:33:34<4:22:34,  2.31s/it]
 63%|██████▎   | 11818/18627 [6:33:36<4:23:52,  2.33s/it]
 63%|██████▎   | 11819/18627 [6:33:38<3:52:44,  2.05s/it]
 63%|██████▎   | 11820/18627 [6:33:39<3:31:37,  1.87s/it]
                                                         
{'loss': 1.2007, 'grad_norm': 13.151959419250488, 'learning_rate': 1.5559561477334662e-06, 'epoch': 0.63}
+
 63%|██████▎   | 11820/18627 [6:33:39<3:31:37,  1.87s/it]
 63%|██████▎   | 11821/18627 [6:33:41<3:15:35,  1.72s/it]
 63%|██████▎   | 11822/18627 [6:33:43<3:38:01,  1.92s/it]
 63%|██████▎   | 11823/18627 [6:33:45<3:54:02,  2.06s/it]
 63%|██████▎   | 11824/18627 [6:33:47<3:30:26,  1.86s/it]
 63%|██████▎   | 11825/18627 [6:33:49<3:48:19,  2.01s/it]
 63%|██████▎   | 11826/18627 [6:33:50<3:26:01,  1.82s/it]
 63%|██████▎   | 11827/18627 [6:33:53<3:42:52,  1.97s/it]
 63%|██████▎   | 11828/18627 [6:33:54<3:22:59,  1.79s/it]
 64%|██████▎   | 11829/18627 [6:33:56<3:40:39,  1.95s/it]
 64%|██████▎   | 11830/18627 [6:33:58<3:21:32,  1.78s/it]
                                                         
{'loss': 1.6128, 'grad_norm': 13.853103637695312, 'learning_rate': 1.5519325126959755e-06, 'epoch': 0.64}
+
 64%|██████▎   | 11830/18627 [6:33:58<3:21:32,  1.78s/it]
 64%|██████▎   | 11831/18627 [6:33:59<3:07:31,  1.66s/it]
 64%|██████▎   | 11832/18627 [6:34:01<2:57:57,  1.57s/it]
 64%|██████▎   | 11833/18627 [6:34:02<2:51:07,  1.51s/it]
 64%|██████▎   | 11834/18627 [6:34:03<2:47:14,  1.48s/it]
 64%|██████▎   | 11835/18627 [6:34:06<3:16:51,  1.74s/it]
 64%|██████▎   | 11836/18627 [6:34:08<3:37:42,  1.92s/it]
 64%|██████▎   | 11837/18627 [6:34:10<3:52:26,  2.05s/it]
 64%|██████▎   | 11838/18627 [6:34:13<4:01:59,  2.14s/it]
 64%|██████▎   | 11839/18627 [6:34:15<4:09:09,  2.20s/it]
 64%|██████▎   | 11840/18627 [6:34:17<4:14:17,  2.25s/it]
                                                         
{'loss': 1.3734, 'grad_norm': 6.439970970153809, 'learning_rate': 1.547911743939226e-06, 'epoch': 0.64}
+
 64%|██████▎   | 11840/18627 [6:34:18<4:14:17,  2.25s/it]
 64%|██████▎   | 11841/18627 [6:34:20<4:17:37,  2.28s/it]
 64%|██████▎   | 11842/18627 [6:34:21<3:48:51,  2.02s/it]
 64%|██████▎   | 11843/18627 [6:34:24<4:00:19,  2.13s/it]
 64%|██████▎   | 11844/18627 [6:34:26<4:07:16,  2.19s/it]
 64%|██████▎   | 11845/18627 [6:34:28<4:12:32,  2.23s/it]
 64%|██████▎   | 11846/18627 [6:34:30<3:43:49,  1.98s/it]
 64%|██████▎   | 11847/18627 [6:34:31<3:25:16,  1.82s/it]
 64%|██████▎   | 11848/18627 [6:34:33<3:43:15,  1.98s/it]
 64%|██████▎   | 11849/18627 [6:34:35<3:23:08,  1.80s/it]
 64%|██████▎   | 11850/18627 [6:34:37<3:42:50,  1.97s/it]
                                                         
{'loss': 1.3217, 'grad_norm': 8.293355941772461, 'learning_rate': 1.5438938536191567e-06, 'epoch': 0.64}
+
 64%|██████▎   | 11850/18627 [6:34:37<3:42:50,  1.97s/it]
 64%|██████▎   | 11851/18627 [6:34:40<3:55:49,  2.09s/it]
 64%|██████▎   | 11852/18627 [6:34:41<3:32:56,  1.89s/it]
 64%|██████▎   | 11853/18627 [6:34:43<3:47:27,  2.01s/it]
 64%|██████▎   | 11854/18627 [6:34:46<4:00:04,  2.13s/it]
 64%|██████▎   | 11855/18627 [6:34:48<4:07:42,  2.19s/it]
 64%|██████▎   | 11856/18627 [6:34:50<4:12:51,  2.24s/it]
 64%|██████▎   | 11857/18627 [6:34:52<3:45:56,  2.00s/it]
 64%|██████▎   | 11858/18627 [6:34:53<3:26:35,  1.83s/it]
 64%|██████▎   | 11859/18627 [6:34:56<3:44:35,  1.99s/it]
 64%|██████▎   | 11860/18627 [6:34:58<3:56:47,  2.10s/it]
                                                         
{'loss': 1.2256, 'grad_norm': 6.184427261352539, 'learning_rate': 1.5398788538830068e-06, 'epoch': 0.64}
+
 64%|██████▎   | 11860/18627 [6:34:58<3:56:47,  2.10s/it]
 64%|██████▎   | 11861/18627 [6:35:00<4:05:21,  2.18s/it]
 64%|██████▎   | 11862/18627 [6:35:03<4:12:26,  2.24s/it]
 64%|██████▎   | 11863/18627 [6:35:05<4:15:23,  2.27s/it]
 64%|██████▎   | 11864/18627 [6:35:07<4:19:37,  2.30s/it]
 64%|██████▎   | 11865/18627 [6:35:09<3:40:49,  1.96s/it]
 64%|██████▎   | 11866/18627 [6:35:11<3:54:04,  2.08s/it]
 64%|██████▎   | 11867/18627 [6:35:13<4:02:16,  2.15s/it]
 64%|██████▎   | 11868/18627 [6:35:15<3:36:07,  1.92s/it]
 64%|██████▎   | 11869/18627 [6:35:17<3:50:44,  2.05s/it]
 64%|██████▎   | 11870/18627 [6:35:19<4:00:12,  2.13s/it]
                                                         
{'loss': 0.7971, 'grad_norm': 5.882351398468018, 'learning_rate': 1.535866756869275e-06, 'epoch': 0.64}
+
 64%|██████▎   | 11870/18627 [6:35:19<4:00:12,  2.13s/it]
 64%|██████▎   | 11871/18627 [6:35:22<4:06:47,  2.19s/it]
 64%|██████▎   | 11872/18627 [6:35:24<4:10:42,  2.23s/it]
 64%|██████▎   | 11873/18627 [6:35:26<4:14:53,  2.26s/it]
 64%|██████▎   | 11874/18627 [6:35:29<4:17:33,  2.29s/it]
 64%|██████▍   | 11875/18627 [6:35:31<4:19:57,  2.31s/it]
 64%|██████▍   | 11876/18627 [6:35:33<4:21:15,  2.32s/it]
 64%|██████▍   | 11877/18627 [6:35:35<3:49:21,  2.04s/it]
 64%|██████▍   | 11878/18627 [6:35:37<3:58:58,  2.12s/it]
 64%|██████▍   | 11879/18627 [6:35:39<4:06:24,  2.19s/it]
 64%|██████▍   | 11880/18627 [6:35:41<3:40:09,  1.96s/it]
                                                         
{'loss': 0.9899, 'grad_norm': 15.61098575592041, 'learning_rate': 1.5318575747076856e-06, 'epoch': 0.64}
+
 64%|██████▍   | 11880/18627 [6:35:41<3:40:09,  1.96s/it]
 64%|██████▍   | 11881/18627 [6:35:43<3:52:20,  2.07s/it]
 64%|██████▍   | 11882/18627 [6:35:46<4:02:29,  2.16s/it]
 64%|██████▍   | 11883/18627 [6:35:47<3:38:27,  1.94s/it]
 64%|██████▍   | 11884/18627 [6:35:49<3:50:42,  2.05s/it]
 64%|██████▍   | 11885/18627 [6:35:52<3:59:23,  2.13s/it]
 64%|██████▍   | 11886/18627 [6:35:54<4:05:47,  2.19s/it]
 64%|██████▍   | 11887/18627 [6:35:56<4:11:46,  2.24s/it]
 64%|██████▍   | 11888/18627 [6:35:59<4:14:45,  2.27s/it]
 64%|██████▍   | 11889/18627 [6:36:01<4:17:29,  2.29s/it]
 64%|██████▍   | 11890/18627 [6:36:03<4:19:50,  2.31s/it]
                                                         
{'loss': 0.8131, 'grad_norm': 7.176275730133057, 'learning_rate': 1.5278513195191475e-06, 'epoch': 0.64}
+
 64%|██████▍   | 11890/18627 [6:36:03<4:19:50,  2.31s/it]
 64%|██████▍   | 11891/18627 [6:36:06<4:19:44,  2.31s/it]
 64%|██████▍   | 11892/18627 [6:36:08<4:21:05,  2.33s/it]
 64%|██████▍   | 11893/18627 [6:36:10<4:23:18,  2.35s/it]
 64%|██████▍   | 11894/18627 [6:36:12<3:51:47,  2.07s/it]
 64%|██████▍   | 11895/18627 [6:36:13<3:29:59,  1.87s/it]
 64%|██████▍   | 11896/18627 [6:36:16<3:45:55,  2.01s/it]
 64%|██████▍   | 11897/18627 [6:36:18<3:56:25,  2.11s/it]
 64%|██████▍   | 11898/18627 [6:36:20<4:03:45,  2.17s/it]
 64%|██████▍   | 11899/18627 [6:36:23<4:10:44,  2.24s/it]
 64%|████��█▍   | 11900/18627 [6:36:25<4:13:29,  2.26s/it]
                                                         
{'loss': 1.0058, 'grad_norm': 7.056249141693115, 'learning_rate': 1.5238480034157251e-06, 'epoch': 0.64}
+
 64%|██████▍   | 11900/18627 [6:36:25<4:13:29,  2.26s/it]
 64%|██████▍   | 11901/18627 [6:36:26<3:43:39,  2.00s/it]
 64%|██████▍   | 11902/18627 [6:36:29<3:57:52,  2.12s/it]
 64%|██████▍   | 11903/18627 [6:36:31<4:05:10,  2.19s/it]
 64%|██████▍   | 11904/18627 [6:36:33<4:11:35,  2.25s/it]
 64%|██████▍   | 11905/18627 [6:36:35<3:43:44,  2.00s/it]
 64%|██████▍   | 11906/18627 [6:36:36<3:23:16,  1.81s/it]
 64%|██████▍   | 11907/18627 [6:36:38<3:10:43,  1.70s/it]
 64%|██████▍   | 11908/18627 [6:36:40<3:34:44,  1.92s/it]
 64%|██████▍   | 11909/18627 [6:36:43<3:50:03,  2.05s/it]
 64%|██████▍   | 11910/18627 [6:36:47<5:19:44,  2.86s/it]
                                                         
{'loss': 1.4441, 'grad_norm': 8.19430160522461, 'learning_rate': 1.5198476385005931e-06, 'epoch': 0.64}
+
 64%|██████▍   | 11910/18627 [6:36:47<5:19:44,  2.86s/it]
 64%|██████▍   | 11911/18627 [6:36:49<4:33:26,  2.44s/it]
 64%|██████▍   | 11912/18627 [6:36:51<4:29:08,  2.40s/it]
 64%|██████▍   | 11913/18627 [6:36:53<4:26:23,  2.38s/it]
 64%|██████▍   | 11914/18627 [6:36:56<4:25:07,  2.37s/it]
 64%|██████▍   | 11915/18627 [6:36:58<4:24:07,  2.36s/it]
 64%|██████▍   | 11916/18627 [6:37:00<4:23:31,  2.36s/it]
 64%|██████▍   | 11917/18627 [6:37:03<4:24:02,  2.36s/it]
 64%|██████▍   | 11918/18627 [6:37:05<4:23:24,  2.36s/it]
 64%|██████▍   | 11919/18627 [6:37:07<4:23:49,  2.36s/it]
 64%|██████▍   | 11920/18627 [6:37:10<4:24:51,  2.37s/it]
                                                         
{'loss': 0.735, 'grad_norm': 7.751392364501953, 'learning_rate': 1.5158502368680062e-06, 'epoch': 0.64}
+
 64%|██████▍   | 11920/18627 [6:37:10<4:24:51,  2.37s/it]
 64%|██████▍   | 11921/18627 [6:37:12<4:24:38,  2.37s/it]
 64%|██████▍   | 11922/18627 [6:37:15<4:24:42,  2.37s/it]
 64%|██████▍   | 11923/18627 [6:37:17<4:23:04,  2.35s/it]
 64%|██████▍   | 11924/18627 [6:37:19<4:23:25,  2.36s/it]
 64%|██████▍   | 11925/18627 [6:37:21<3:51:56,  2.08s/it]
 64%|██████▍   | 11926/18627 [6:37:22<3:28:17,  1.87s/it]
 64%|██████▍   | 11927/18627 [6:37:24<3:45:09,  2.02s/it]
 64%|██████▍   | 11928/18627 [6:37:26<3:25:01,  1.84s/it]
 64%|██████▍   | 11929/18627 [6:37:28<3:43:10,  2.00s/it]
 64%|██████▍   | 11930/18627 [6:37:31<3:55:10,  2.11s/it]
                                                         
{'loss': 1.1384, 'grad_norm': 5.088303089141846, 'learning_rate': 1.5118558106032578e-06, 'epoch': 0.64}
+
 64%|██████▍   | 11930/18627 [6:37:31<3:55:10,  2.11s/it]
 64%|██████▍   | 11931/18627 [6:37:33<4:03:24,  2.18s/it]
 64%|██████▍   | 11932/18627 [6:37:35<4:09:20,  2.23s/it]
 64%|██████▍   | 11933/18627 [6:37:38<4:12:35,  2.26s/it]
 64%|██████▍   | 11934/18627 [6:37:40<4:15:26,  2.29s/it]
 64%|██████▍   | 11935/18627 [6:37:42<4:18:02,  2.31s/it]
 64%|██████▍   | 11936/18627 [6:37:45<4:19:23,  2.33s/it]
 64%|██████▍   | 11937/18627 [6:37:47<4:19:56,  2.33s/it]
 64%|██████▍   | 11938/18627 [6:37:48<3:39:08,  1.97s/it]
 64%|██████▍   | 11939/18627 [6:37:51<3:53:07,  2.09s/it]
 64%|██████▍   | 11940/18627 [6:37:53<4:02:23,  2.17s/it]
                                                         
{'loss': 0.5929, 'grad_norm': 4.272446632385254, 'learning_rate': 1.50786437178265e-06, 'epoch': 0.64}
+
 64%|██████▍   | 11940/18627 [6:37:53<4:02:23,  2.17s/it]
 64%|██████▍   | 11941/18627 [6:37:57<4:50:33,  2.61s/it]
 64%|██████▍   | 11942/18627 [6:37:59<4:43:47,  2.55s/it]
 64%|██████▍   | 11943/18627 [6:38:01<4:37:03,  2.49s/it]
 64%|██████▍   | 11944/18627 [6:38:04<4:31:59,  2.44s/it]
 64%|██████▍   | 11945/18627 [6:38:06<4:27:57,  2.41s/it]
 64%|██████▍   | 11946/18627 [6:38:07<3:54:11,  2.10s/it]
 64%|██████▍   | 11947/18627 [6:38:11<4:46:18,  2.57s/it]
 64%|██████▍   | 11948/18627 [6:38:13<4:38:49,  2.50s/it]
 64%|██████▍   | 11949/18627 [6:38:16<4:33:18,  2.46s/it]
 64%|██████▍   | 11950/18627 [6:38:18<4:28:57,  2.42s/it]
                                                         
{'loss': 1.0265, 'grad_norm': 6.062412261962891, 'learning_rate': 1.5038759324734487e-06, 'epoch': 0.64}
+
 64%|██████▍   | 11950/18627 [6:38:18<4:28:57,  2.42s/it]
 64%|██████▍   | 11951/18627 [6:38:20<4:26:54,  2.40s/it]
 64%|██████▍   | 11952/18627 [6:38:23<4:25:39,  2.39s/it]
 64%|██████▍   | 11953/18627 [6:38:25<4:24:42,  2.38s/it]
 64%|██████▍   | 11954/18627 [6:38:27<4:22:51,  2.36s/it]
 64%|██████▍   | 11955/18627 [6:38:30<4:22:50,  2.36s/it]
 64%|██████▍   | 11956/18627 [6:38:31<3:50:12,  2.07s/it]
 64%|██████▍   | 11957/18627 [6:38:33<3:27:42,  1.87s/it]
 64%|██████▍   | 11958/18627 [6:38:35<3:43:15,  2.01s/it]
 64%|██████▍   | 11959/18627 [6:38:36<3:20:06,  1.80s/it]
 64%|██████▍   | 11960/18627 [6:38:38<3:07:36,  1.69s/it]
                                                         
{'loss': 1.3603, 'grad_norm': 14.457653045654297, 'learning_rate': 1.4998905047338546e-06, 'epoch': 0.64}
+
 64%|██████▍   | 11960/18627 [6:38:38<3:07:36,  1.69s/it]
 64%|██████▍   | 11961/18627 [6:38:40<3:30:13,  1.89s/it]
 64%|██████▍   | 11962/18627 [6:38:41<3:13:14,  1.74s/it]
 64%|██████▍   | 11963/18627 [6:38:44<3:31:54,  1.91s/it]
 64%|██████▍   | 11964/18627 [6:38:46<3:46:06,  2.04s/it]
 64%|██████▍   | 11965/18627 [6:38:48<3:56:08,  2.13s/it]
 64%|██████▍   | 11966/18627 [6:38:51<4:03:48,  2.20s/it]
 64%|██████▍   | 11967/18627 [6:38:52<3:36:42,  1.95s/it]
 64%|██████▍   | 11968/18627 [6:38:54<3:17:39,  1.78s/it]
 64%|██████▍   | 11969/18627 [6:38:56<3:36:54,  1.95s/it]
 64%|██████▍   | 11970/18627 [6:38:58<3:50:09,  2.07s/it]
                                                         
{'loss': 1.229, 'grad_norm': 9.13637638092041, 'learning_rate': 1.4959081006129604e-06, 'epoch': 0.64}
+
 64%|██████▍   | 11970/18627 [6:38:58<3:50:09,  2.07s/it]
 64%|██████▍   | 11971/18627 [6:39:01<3:59:33,  2.16s/it]
 64%|██████▍   | 11972/18627 [6:39:03<4:06:37,  2.22s/it]
 64%|██████▍   | 11973/18627 [6:39:05<4:10:42,  2.26s/it]
 64%|██████▍   | 11974/18627 [6:39:07<4:06:45,  2.23s/it]
 64%|██████▍   | 11975/18627 [6:39:10<4:04:29,  2.21s/it]
 64%|██████▍   | 11976/18627 [6:39:12<4:00:51,  2.17s/it]
 64%|██████▍   | 11977/18627 [6:39:14<3:59:27,  2.16s/it]
 64%|██████▍   | 11978/18627 [6:39:16<3:57:50,  2.15s/it]
 64%|██████▍   | 11979/18627 [6:39:18<3:56:58,  2.14s/it]
 64%|██████▍   | 11980/18627 [6:39:20<3:56:09,  2.13s/it]
                                                         
{'loss': 0.6675, 'grad_norm': 8.49454402923584, 'learning_rate': 1.4919287321507203e-06, 'epoch': 0.64}
+
 64%|██████▍   | 11980/18627 [6:39:20<3:56:09,  2.13s/it]
 64%|██████▍   | 11981/18627 [6:39:21<3:09:17,  1.71s/it]
 64%|██████▍   | 11982/18627 [6:39:23<3:23:34,  1.84s/it]
 64%|██████▍   | 11983/18627 [6:39:24<3:02:36,  1.65s/it]
 64%|██████▍   | 11984/18627 [6:39:26<3:18:10,  1.79s/it]
 64%|██████▍   | 11985/18627 [6:39:28<3:09:14,  1.71s/it]
 64%|██████▍   | 11986/18627 [6:39:30<3:22:30,  1.83s/it]
 64%|██████▍   | 11987/18627 [6:39:32<3:32:08,  1.92s/it]
 64%|██████▍   | 11988/18627 [6:39:34<3:39:07,  1.98s/it]
 64%|██████▍   | 11989/18627 [6:39:36<3:44:08,  2.03s/it]
 64%|██████▍   | 11990/18627 [6:39:39<4:15:26,  2.31s/it]
                                                         
{'loss': 1.0696, 'grad_norm': 6.687187194824219, 'learning_rate': 1.4879524113779114e-06, 'epoch': 0.64}
+
 64%|██████▍   | 11990/18627 [6:39:39<4:15:26,  2.31s/it]
 64%|██████▍   | 11991/18627 [6:39:41<3:38:14,  1.97s/it]
 64%|██████▍   | 11992/18627 [6:39:43<3:43:13,  2.02s/it]
 64%|██████▍   | 11993/18627 [6:39:44<3:15:47,  1.77s/it]
 64%|██████▍   | 11994/18627 [6:39:46<3:26:32,  1.87s/it]
 64%|██████▍   | 11995/18627 [6:39:47<2:58:06,  1.61s/it]
 64%|██████▍   | 11996/18627 [6:39:49<3:14:36,  1.76s/it]
 64%|██████▍   | 11997/18627 [6:39:51<3:25:52,  1.86s/it]
 64%|██████▍   | 11998/18627 [6:39:53<3:36:27,  1.96s/it]
 64%|██████▍   | 11999/18627 [6:39:55<3:41:49,  2.01s/it]
 64%|██████▍   | 12000/18627 [6:39:58<3:44:49,  2.04s/it]
                                                         
{'loss': 1.2446, 'grad_norm': 5.548574447631836, 'learning_rate': 1.4839791503160926e-06, 'epoch': 0.64}
+
 64%|██████▍   | 12000/18627 [6:39:58<3:44:49,  2.04s/it]/home/zhengduo/workspace/LLaMA-Factory/src/qwen_vl/model/modeling_qwen2_5_vl.py:1979: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
+  with torch.cuda.amp.autocast(dtype=dtype):
+/home/zhengduo/miniconda3/envs/vgllm/lib/python3.10/site-packages/torch/utils/checkpoint.py:87: UserWarning: None of the inputs have requires_grad=True. Gradients will be None
+  warnings.warn(
+
 64%|██████▍   | 12001/18627 [6:40:45<28:54:41, 15.71s/it]
 64%|██████▍   | 12002/18627 [6:40:47<21:25:03, 11.64s/it]
 64%|██████▍   | 12003/18627 [6:40:49<16:10:53,  8.79s/it]
 64%|██████▍   | 12004/18627 [6:40:52<12:31:54,  6.81s/it]
 64%|██████▍   | 12005/18627 [6:40:54<9:55:41,  5.40s/it] 
 64%|██████▍   | 12006/18627 [6:40:56<8:07:04,  4.41s/it]
 64%|██████▍   | 12007/18627 [6:40:58<6:50:49,  3.72s/it]
 64%|██████▍   | 12008/18627 [6:41:00<5:57:44,  3.24s/it]
 64%|██████▍   | 12009/18627 [6:41:02<5:22:30,  2.92s/it]
 64%|██████▍   | 12010/18627 [6:41:04<4:55:17,  2.68s/it]
                                                         
{'loss': 0.6884, 'grad_norm': 5.2215352058410645, 'learning_rate': 1.4800089609775778e-06, 'epoch': 0.64}
+
 64%|██████▍   | 12010/18627 [6:41:04<4:55:17,  2.68s/it]
 64%|██████▍   | 12011/18627 [6:41:07<4:36:50,  2.51s/it]
 64%|██████▍   | 12012/18627 [6:41:09<4:23:28,  2.39s/it]
 64%|██████▍   | 12013/18627 [6:41:11<4:14:41,  2.31s/it]
 64%|██████▍   | 12014/18627 [6:41:13<4:08:43,  2.26s/it]
 65%|██████▍   | 12015/18627 [6:41:15<4:03:26,  2.21s/it]
 65%|██████▍   | 12016/18627 [6:41:17<4:00:28,  2.18s/it]
 65%|██████▍   | 12017/18627 [6:41:19<3:58:09,  2.16s/it]
 65%|██████▍   | 12018/18627 [6:41:21<3:56:28,  2.15s/it]
 65%|██████▍   | 12019/18627 [6:41:23<3:24:40,  1.86s/it]
 65%|██████▍   | 12020/18627 [6:41:25<3:33:22,  1.94s/it]
                                                         
{'loss': 0.8507, 'grad_norm': 7.686403274536133, 'learning_rate': 1.4760418553653889e-06, 'epoch': 0.65}
+
 65%|██████▍   | 12020/18627 [6:41:25<3:33:22,  1.94s/it]
 65%|██████▍   | 12021/18627 [6:41:27<3:39:28,  1.99s/it]
 65%|██████▍   | 12022/18627 [6:41:29<3:43:37,  2.03s/it]
 65%|██████▍   | 12023/18627 [6:41:30<3:15:45,  1.78s/it]
 65%|██████▍   | 12024/18627 [6:41:32<3:26:43,  1.88s/it]
 65%|██████▍   | 12025/18627 [6:41:34<3:34:10,  1.95s/it]
 65%|██████▍   | 12026/18627 [6:41:36<3:38:59,  1.99s/it]
 65%|██████▍   | 12027/18627 [6:41:39<3:43:22,  2.03s/it]
 65%|██████▍   | 12028/18627 [6:41:41<3:46:15,  2.06s/it]
 65%|██████▍   | 12029/18627 [6:41:43<3:48:09,  2.07s/it]
 65%|██████▍   | 12030/18627 [6:41:44<3:19:21,  1.81s/it]
                                                         
{'loss': 0.976, 'grad_norm': 13.461560249328613, 'learning_rate': 1.4720778454732295e-06, 'epoch': 0.65}
+
 65%|██████▍   | 12030/18627 [6:41:44<3:19:21,  1.81s/it]
 65%|██████▍   | 12031/18627 [6:41:45<2:58:55,  1.63s/it]
 65%|██████▍   | 12032/18627 [6:41:47<3:15:08,  1.78s/it]
 65%|██████▍   | 12033/18627 [6:41:49<3:27:01,  1.88s/it]
 65%|██████▍   | 12034/18627 [6:41:52<3:34:40,  1.95s/it]
 65%|██████▍   | 12035/18627 [6:41:53<3:09:26,  1.72s/it]
 65%|██████▍   | 12036/18627 [6:41:55<3:22:46,  1.85s/it]
 65%|██████▍   | 12037/18627 [6:41:57<3:31:36,  1.93s/it]
 65%|██████▍   | 12038/18627 [6:41:59<3:38:11,  1.99s/it]
 65%|██████▍   | 12039/18627 [6:42:01<3:42:12,  2.02s/it]
 65%|██████▍   | 12040/18627 [6:42:03<3:45:31,  2.05s/it]
                                                         
{'loss': 1.0611, 'grad_norm': 4.765686988830566, 'learning_rate': 1.4681169432854396e-06, 'epoch': 0.65}
+
 65%|██████▍   | 12040/18627 [6:42:03<3:45:31,  2.05s/it]
 65%|██████▍   | 12041/18627 [6:42:05<3:49:20,  2.09s/it]
 65%|██████▍   | 12042/18627 [6:42:08<3:51:17,  2.11s/it]
 65%|██████▍   | 12043/18627 [6:42:09<3:21:01,  1.83s/it]
 65%|██████▍   | 12044/18627 [6:42:11<3:29:54,  1.91s/it]
 65%|██████▍   | 12045/18627 [6:42:13<3:38:03,  1.99s/it]
 65%|██████▍   | 12046/18627 [6:42:15<3:42:19,  2.03s/it]
 65%|██████▍   | 12047/18627 [6:42:17<3:45:12,  2.05s/it]
 65%|██████▍   | 12048/18627 [6:42:19<3:47:30,  2.07s/it]
 65%|██████▍   | 12049/18627 [6:42:22<3:48:49,  2.09s/it]
 65%|██████▍   | 12050/18627 [6:42:24<3:49:44,  2.10s/it]
                                                         
{'loss': 0.8207, 'grad_norm': 8.464279174804688, 'learning_rate': 1.464159160776968e-06, 'epoch': 0.65}
+
 65%|██████▍   | 12050/18627 [6:42:24<3:49:44,  2.10s/it]
 65%|██████▍   | 12051/18627 [6:42:26<3:50:33,  2.10s/it]
 65%|██████▍   | 12052/18627 [6:42:28<3:51:11,  2.11s/it]
 65%|██████▍   | 12053/18627 [6:42:30<3:51:24,  2.11s/it]
 65%|██████▍   | 12054/18627 [6:42:32<3:51:51,  2.12s/it]
 65%|██████▍   | 12055/18627 [6:42:33<3:21:04,  1.84s/it]
 65%|██████▍   | 12056/18627 [6:42:36<3:31:39,  1.93s/it]
 65%|██████▍   | 12057/18627 [6:42:38<3:37:01,  1.98s/it]
 65%|██████▍   | 12058/18627 [6:42:39<3:09:20,  1.73s/it]
 65%|██████▍   | 12059/18627 [6:42:41<3:21:54,  1.84s/it]
 65%|██████▍   | 12060/18627 [6:42:42<2:44:59,  1.51s/it]
                                                         
{'loss': 0.9916, 'grad_norm': 6.104118347167969, 'learning_rate': 1.460204509913328e-06, 'epoch': 0.65}
+
 65%|██████▍   | 12060/18627 [6:42:42<2:44:59,  1.51s/it]
 65%|██████▍   | 12061/18627 [6:42:44<3:05:50,  1.70s/it]
 65%|██████▍   | 12062/18627 [6:42:45<2:49:32,  1.55s/it]
 65%|██████▍   | 12063/18627 [6:42:47<3:08:29,  1.72s/it]
 65%|██████▍   | 12064/18627 [6:42:49<3:22:04,  1.85s/it]
 65%|██████▍   | 12065/18627 [6:42:51<3:31:32,  1.93s/it]
 65%|██████▍   | 12066/18627 [6:42:52<3:03:44,  1.68s/it]
 65%|██████▍   | 12067/18627 [6:42:55<3:18:27,  1.82s/it]
 65%|██████▍   | 12068/18627 [6:42:57<3:28:44,  1.91s/it]
 65%|██████▍   | 12069/18627 [6:42:58<3:05:20,  1.70s/it]
 65%|██████▍   | 12070/18627 [6:43:00<3:19:20,  1.82s/it]
                                                         
{'loss': 1.0822, 'grad_norm': 9.488781929016113, 'learning_rate': 1.4562530026505686e-06, 'epoch': 0.65}
+
 65%|██████▍   | 12070/18627 [6:43:00<3:19:20,  1.82s/it]
 65%|██████▍   | 12071/18627 [6:43:02<3:29:32,  1.92s/it]
 65%|██████▍   | 12072/18627 [6:43:04<3:36:33,  1.98s/it]
 65%|██████▍   | 12073/18627 [6:43:06<3:40:33,  2.02s/it]
 65%|██████▍   | 12074/18627 [6:43:08<3:43:46,  2.05s/it]
 65%|██████▍   | 12075/18627 [6:43:11<3:46:17,  2.07s/it]
 65%|██████▍   | 12076/18627 [6:43:12<3:17:30,  1.81s/it]
 65%|██████▍   | 12077/18627 [6:43:14<3:27:53,  1.90s/it]
 65%|██████▍   | 12078/18627 [6:43:16<3:35:25,  1.97s/it]
 65%|██████▍   | 12079/18627 [6:43:18<3:39:59,  2.02s/it]
 65%|██████▍   | 12080/18627 [6:43:20<3:44:25,  2.06s/it]
                                                         
{'loss': 0.8211, 'grad_norm': 5.439093112945557, 'learning_rate': 1.4523046509352317e-06, 'epoch': 0.65}
+
 65%|██████▍   | 12080/18627 [6:43:20<3:44:25,  2.06s/it]
 65%|██████▍   | 12081/18627 [6:43:22<3:16:01,  1.80s/it]
 65%|██████▍   | 12082/18627 [6:43:24<3:26:23,  1.89s/it]
 65%|██████▍   | 12083/18627 [6:43:26<3:34:27,  1.97s/it]
 65%|██████▍   | 12084/18627 [6:43:27<3:06:40,  1.71s/it]
 65%|██████▍   | 12085/18627 [6:43:29<3:20:55,  1.84s/it]
 65%|██████▍   | 12086/18627 [6:43:31<3:30:17,  1.93s/it]
 65%|██████▍   | 12087/18627 [6:43:33<3:36:40,  1.99s/it]
 65%|██████▍   | 12088/18627 [6:43:35<3:10:47,  1.75s/it]
 65%|██████▍   | 12089/18627 [6:43:37<3:22:31,  1.86s/it]
 65%|██████▍   | 12090/18627 [6:43:38<3:00:40,  1.66s/it]
                                                         
{'loss': 1.3982, 'grad_norm': 12.87173080444336, 'learning_rate': 1.4483594667043205e-06, 'epoch': 0.65}
+
 65%|██████▍   | 12090/18627 [6:43:38<3:00:40,  1.66s/it]
 65%|██████▍   | 12091/18627 [6:43:40<3:15:58,  1.80s/it]
 65%|██████▍   | 12092/18627 [6:43:42<3:26:08,  1.89s/it]
 65%|██████▍   | 12093/18627 [6:43:44<3:33:52,  1.96s/it]
 65%|██████▍   | 12094/18627 [6:43:45<3:03:11,  1.68s/it]
 65%|██████▍   | 12095/18627 [6:43:47<3:17:31,  1.81s/it]
 65%|██████▍   | 12096/18627 [6:43:49<3:27:00,  1.90s/it]
 65%|██████▍   | 12097/18627 [6:43:52<3:34:15,  1.97s/it]
 65%|██████▍   | 12098/18627 [6:43:54<3:39:25,  2.02s/it]
 65%|██████▍   | 12099/18627 [6:43:56<3:43:07,  2.05s/it]
 65%|██████▍   | 12100/18627 [6:43:58<3:45:03,  2.07s/it]
                                                         
{'loss': 0.7672, 'grad_norm': 7.139204025268555, 'learning_rate': 1.4444174618852619e-06, 'epoch': 0.65}
+
 65%|██████▍   | 12100/18627 [6:43:58<3:45:03,  2.07s/it]
 65%|██████▍   | 12101/18627 [6:44:00<3:46:58,  2.09s/it]
 65%|██████▍   | 12102/18627 [6:44:02<3:48:14,  2.10s/it]
 65%|██████▍   | 12103/18627 [6:44:04<3:49:33,  2.11s/it]
 65%|██████▍   | 12104/18627 [6:44:06<3:50:01,  2.12s/it]
 65%|██████▍   | 12105/18627 [6:44:08<3:19:52,  1.84s/it]
 65%|██████▍   | 12106/18627 [6:44:09<2:58:53,  1.65s/it]
 65%|██████▍   | 12107/18627 [6:44:10<2:41:17,  1.48s/it]
 65%|██████▌   | 12108/18627 [6:44:12<3:02:19,  1.68s/it]
 65%|██████��   | 12109/18627 [6:44:14<3:16:51,  1.81s/it]
 65%|██████▌   | 12110/18627 [6:44:16<3:27:12,  1.91s/it]
                                                         
{'loss': 1.1895, 'grad_norm': 6.558509349822998, 'learning_rate': 1.4404786483958715e-06, 'epoch': 0.65}
+
 65%|██████▌   | 12110/18627 [6:44:16<3:27:12,  1.91s/it]
 65%|██████▌   | 12111/18627 [6:44:18<3:33:46,  1.97s/it]
 65%|██████▌   | 12112/18627 [6:44:20<3:08:35,  1.74s/it]
 65%|██████▌   | 12113/18627 [6:44:22<3:20:35,  1.85s/it]
 65%|██████▌   | 12114/18627 [6:44:23<2:59:07,  1.65s/it]
 65%|██████▌   | 12115/18627 [6:44:25<3:13:57,  1.79s/it]
 65%|██████▌   | 12116/18627 [6:44:26<2:54:30,  1.61s/it]
 65%|██████▌   | 12117/18627 [6:44:28<3:11:36,  1.77s/it]
 65%|██████▌   | 12118/18627 [6:44:30<3:23:02,  1.87s/it]
 65%|██████▌   | 12119/18627 [6:44:33<3:30:25,  1.94s/it]
 65%|██████▌   | 12120/18627 [6:44:35<3:35:53,  1.99s/it]
                                                         
{'loss': 1.1933, 'grad_norm': 5.284725666046143, 'learning_rate': 1.4365430381443124e-06, 'epoch': 0.65}
+
 65%|██████▌   | 12120/18627 [6:44:35<3:35:53,  1.99s/it]
 65%|██████▌   | 12121/18627 [6:44:37<3:41:42,  2.04s/it]
 65%|██████▌   | 12122/18627 [6:44:39<3:44:21,  2.07s/it]
 65%|██████▌   | 12123/18627 [6:44:41<3:46:41,  2.09s/it]
 65%|██████▌   | 12124/18627 [6:44:43<3:47:33,  2.10s/it]
 65%|██████▌   | 12125/18627 [6:44:44<3:18:00,  1.83s/it]
 65%|██████▌   | 12126/18627 [6:44:47<3:27:57,  1.92s/it]
 65%|██████▌   | 12127/18627 [6:44:48<3:04:23,  1.70s/it]
 65%|██████▌   | 12128/18627 [6:44:50<3:18:29,  1.83s/it]
 65%|██████▌   | 12129/18627 [6:44:52<3:27:09,  1.91s/it]
 65%|██████▌   | 12130/18627 [6:44:54<3:33:59,  1.98s/it]
                                                         
{'loss': 0.9746, 'grad_norm': 7.213533878326416, 'learning_rate': 1.432610643029071e-06, 'epoch': 0.65}
+
 65%|██████▌   | 12130/18627 [6:44:54<3:33:59,  1.98s/it]
 65%|██████▌   | 12131/18627 [6:44:56<3:39:43,  2.03s/it]
 65%|██████▌   | 12132/18627 [6:44:58<3:43:12,  2.06s/it]
 65%|██████▌   | 12133/18627 [6:45:01<3:44:56,  2.08s/it]
 65%|██████▌   | 12134/18627 [6:45:03<3:46:11,  2.09s/it]
 65%|██████▌   | 12135/18627 [6:45:05<3:46:53,  2.10s/it]
 65%|██████▌   | 12136/18627 [6:45:07<3:48:18,  2.11s/it]
 65%|██████▌   | 12137/18627 [6:45:09<3:48:51,  2.12s/it]
 65%|██████▌   | 12138/18627 [6:45:10<3:19:04,  1.84s/it]
 65%|██████▌   | 12139/18627 [6:45:12<3:28:24,  1.93s/it]
 65%|██████▌   | 12140/18627 [6:45:14<3:34:58,  1.99s/it]
                                                         
{'loss': 0.8389, 'grad_norm': 5.383005619049072, 'learning_rate': 1.4286814749389054e-06, 'epoch': 0.65}
+
 65%|██████▌   | 12140/18627 [6:45:15<3:34:58,  1.99s/it]
 65%|██████▌   | 12141/18627 [6:45:17<3:39:34,  2.03s/it]
 65%|██████▌   | 12142/18627 [6:45:19<3:42:49,  2.06s/it]
 65%|██████▌   | 12143/18627 [6:45:21<3:45:58,  2.09s/it]
 65%|██████▌   | 12144/18627 [6:45:23<3:47:09,  2.10s/it]
 65%|██████▌   | 12145/18627 [6:45:24<3:17:24,  1.83s/it]
 65%|██████▌   | 12146/18627 [6:45:26<3:27:25,  1.92s/it]
 65%|██████▌   | 12147/18627 [6:45:28<3:03:55,  1.70s/it]
 65%|██████▌   | 12148/18627 [6:45:30<3:17:17,  1.83s/it]
 65%|██████▌   | 12149/18627 [6:45:32<3:26:12,  1.91s/it]
 65%|██████▌   | 12150/18627 [6:45:33<3:02:38,  1.69s/it]
                                                         
{'loss': 1.2716, 'grad_norm': 15.090075492858887, 'learning_rate': 1.4247555457528229e-06, 'epoch': 0.65}
+
 65%|██████▌   | 12150/18627 [6:45:33<3:02:38,  1.69s/it]
 65%|██████▌   | 12151/18627 [6:45:35<3:16:52,  1.82s/it]
 65%|██████▌   | 12152/18627 [6:45:36<2:51:24,  1.59s/it]
 65%|██████▌   | 12153/18627 [6:45:37<2:38:30,  1.47s/it]
 65%|██████▌   | 12154/18627 [6:45:39<2:29:36,  1.39s/it]
 65%|██████▌   | 12155/18627 [6:45:40<2:18:51,  1.29s/it]
 65%|██████▌   | 12156/18627 [6:45:42<2:50:17,  1.58s/it]
 65%|██████▌   | 12157/18627 [6:45:43<2:43:48,  1.52s/it]
 65%|██████▌   | 12158/18627 [6:45:45<2:39:33,  1.48s/it]
 65%|██████▌   | 12159/18627 [6:45:46<2:37:43,  1.46s/it]
 65%|██████▌   | 12160/18627 [6:45:48<3:05:58,  1.73s/it]
                                                         
{'loss': 1.9856, 'grad_norm': 7.002494812011719, 'learning_rate': 1.4208328673400354e-06, 'epoch': 0.65}
+
 65%|██████▌   | 12160/18627 [6:45:48<3:05:58,  1.73s/it]
 65%|██████▌   | 12161/18627 [6:45:51<3:25:16,  1.90s/it]
 65%|██████▌   | 12162/18627 [6:45:53<3:39:18,  2.04s/it]
 65%|██████▌   | 12163/18627 [6:45:54<3:20:42,  1.86s/it]
 65%|██████▌   | 12164/18627 [6:45:57<3:38:08,  2.03s/it]
 65%|██████▌   | 12165/18627 [6:45:58<3:18:05,  1.84s/it]
 65%|██████▌   | 12166/18627 [6:46:01<3:34:13,  1.99s/it]
 65%|██████▌   | 12167/18627 [6:46:03<3:45:56,  2.10s/it]
 65%|██████▌   | 12168/18627 [6:46:05<3:54:54,  2.18s/it]
 65%|██████▌   | 12169/18627 [6:46:08<4:00:10,  2.23s/it]
 65%|██████▌   | 12170/18627 [6:46:10<4:03:47,  2.27s/it]
                                                         
{'loss': 1.071, 'grad_norm': 6.400459289550781, 'learning_rate': 1.4169134515599291e-06, 'epoch': 0.65}
+
 65%|██████▌   | 12170/18627 [6:46:10<4:03:47,  2.27s/it]
 65%|██████▌   | 12171/18627 [6:46:12<4:06:13,  2.29s/it]
 65%|██████▌   | 12172/18627 [6:46:15<4:07:34,  2.30s/it]
 65%|██████▌   | 12173/18627 [6:46:17<4:08:51,  2.31s/it]
 65%|██████▌   | 12174/18627 [6:46:19<4:09:39,  2.32s/it]
 65%|██████▌   | 12175/18627 [6:46:22<4:09:30,  2.32s/it]
 65%|██████▌   | 12176/18627 [6:46:24<4:10:33,  2.33s/it]
 65%|██████▌   | 12177/18627 [6:46:26<4:10:31,  2.33s/it]
 65%|██████▌   | 12178/18627 [6:46:29<4:10:13,  2.33s/it]
 65%|██████▌   | 12179/18627 [6:46:31<4:09:59,  2.33s/it]
 65%|██████▌   | 12180/18627 [6:46:32<3:35:42,  2.01s/it]
                                                         
{'loss': 0.8464, 'grad_norm': 14.319091796875, 'learning_rate': 1.4129973102620255e-06, 'epoch': 0.65}
+
 65%|██████▌   | 12180/18627 [6:46:32<3:35:42,  2.01s/it]
 65%|██████▌   | 12181/18627 [6:46:35<3:45:51,  2.10s/it]
 65%|██████▌   | 12182/18627 [6:46:37<3:52:29,  2.16s/it]
 65%|██████▌   | 12183/18627 [6:46:39<3:56:50,  2.21s/it]
 65%|██████▌   | 12184/18627 [6:46:41<3:26:14,  1.92s/it]
 65%|██████▌   | 12185/18627 [6:46:43<3:39:58,  2.05s/it]
 65%|██████▌   | 12186/18627 [6:46:44<3:19:12,  1.86s/it]
 65%|██████▌   | 12187/18627 [6:46:47<3:35:41,  2.01s/it]
 65%|██████▌   | 12188/18627 [6:46:49<3:46:36,  2.11s/it]
 65%|██████▌   | 12189/18627 [6:46:51<3:54:27,  2.19s/it]
 65%|██████▌   | 12190/18627 [6:46:54<3:58:27,  2.22s/it]
                                                         
{'loss': 1.0202, 'grad_norm': 6.071603298187256, 'learning_rate': 1.4090844552859465e-06, 'epoch': 0.65}
+
 65%|██████▌   | 12190/18627 [6:46:54<3:58:27,  2.22s/it]
 65%|██████▌   | 12191/18627 [6:46:56<4:01:40,  2.25s/it]
 65%|██████▌   | 12192/18627 [6:46:58<4:03:10,  2.27s/it]
 65%|██████▌   | 12193/18627 [6:47:01<4:05:19,  2.29s/it]
 65%|██████▌   | 12194/18627 [6:47:02<3:36:14,  2.02s/it]
 65%|██████▌   | 12195/18627 [6:47:04<3:44:51,  2.10s/it]
 65%|██████▌   | 12196/18627 [6:47:06<3:21:37,  1.88s/it]
 65%|██████▌   | 12197/18627 [6:47:08<3:35:21,  2.01s/it]
 65%|██████▌   | 12198/18627 [6:47:10<3:45:12,  2.10s/it]
 65%|██████▌   | 12199/18627 [6:47:13<3:51:41,  2.16s/it]
 65%|██████▌   | 12200/18627 [6:47:14<3:25:01,  1.91s/it]
                                                         
{'loss': 1.158, 'grad_norm': 13.050665855407715, 'learning_rate': 1.4051748984613758e-06, 'epoch': 0.65}
+
 65%|██████▌   | 12200/18627 [6:47:14<3:25:01,  1.91s/it]
 66%|██████▌   | 12201/18627 [6:47:15<2:57:18,  1.66s/it]
 66%|██████▌   | 12202/18627 [6:47:17<3:19:26,  1.86s/it]
 66%|██████▌   | 12203/18627 [6:47:20<3:33:27,  1.99s/it]
 66%|██████▌   | 12204/18627 [6:47:22<3:44:26,  2.10s/it]
 66%|██████▌   | 12205/18627 [6:47:24<3:51:34,  2.16s/it]
 66%|██████▌   | 12206/18627 [6:47:27<3:57:57,  2.22s/it]
 66%|██████▌   | 12207/18627 [6:47:29<4:02:56,  2.27s/it]
 66%|██████▌   | 12208/18627 [6:47:31<4:04:46,  2.29s/it]
 66%|██████▌   | 12209/18627 [6:47:34<4:05:59,  2.30s/it]
 66%|██████▌   | 12210/18627 [6:47:36<4:07:43,  2.32s/it]
                                                         
{'loss': 0.6297, 'grad_norm': 5.212625026702881, 'learning_rate': 1.4012686516080326e-06, 'epoch': 0.66}
+
 66%|██████▌   | 12210/18627 [6:47:36<4:07:43,  2.32s/it]
 66%|██████▌   | 12211/18627 [6:47:38<4:09:41,  2.33s/it]
 66%|██████▌   | 12212/18627 [6:47:41<4:09:11,  2.33s/it]
 66%|██████▌   | 12213/18627 [6:47:43<4:08:26,  2.32s/it]
 66%|██████▌   | 12214/18627 [6:47:46<4:12:47,  2.37s/it]
 66%|██████▌   | 12215/18627 [6:47:48<4:12:44,  2.36s/it]
 66%|██████▌   | 12216/18627 [6:47:50<4:10:42,  2.35s/it]
 66%|██████▌   | 12217/18627 [6:47:52<4:09:05,  2.33s/it]
 66%|██████▌   | 12218/18627 [6:47:55<4:10:04,  2.34s/it]
 66%|██████▌   | 12219/18627 [6:47:57<4:10:19,  2.34s/it]
 66%|██████▌   | 12220/18627 [6:48:00<4:09:43,  2.34s/it]
                                                         
{'loss': 0.6037, 'grad_norm': 7.38482666015625, 'learning_rate': 1.397365726535621e-06, 'epoch': 0.66}
+
 66%|██████▌   | 12220/18627 [6:48:00<4:09:43,  2.34s/it]
 66%|██████▌   | 12221/18627 [6:48:01<3:38:05,  2.04s/it]
 66%|██████▌   | 12222/18627 [6:48:03<3:47:35,  2.13s/it]
 66%|██████▌   | 12223/18627 [6:48:06<3:54:37,  2.20s/it]
 66%|██████▌   | 12224/18627 [6:48:10<5:09:44,  2.90s/it]
 66%|██████▌   | 12225/18627 [6:48:12<4:51:00,  2.73s/it]
 66%|██████▌   | 12226/18627 [6:48:15<4:37:39,  2.60s/it]
 66%|██████▌   | 12227/18627 [6:48:17<4:27:34,  2.51s/it]
 66%|██████▌   | 12228/18627 [6:48:19<4:21:06,  2.45s/it]
 66%|██████▌   | 12229/18627 [6:48:22<4:18:14,  2.42s/it]
 66%|██████▌   | 12230/18627 [6:48:24<4:14:58,  2.39s/it]
                                                         
{'loss': 0.8631, 'grad_norm': 6.944662094116211, 'learning_rate': 1.3934661350438083e-06, 'epoch': 0.66}
+
 66%|██████▌   | 12230/18627 [6:48:24<4:14:58,  2.39s/it]
 66%|██████▌   | 12231/18627 [6:48:26<4:13:55,  2.38s/it]
 66%|██████▌   | 12232/18627 [6:48:29<4:12:07,  2.37s/it]
 66%|██████▌   | 12233/18627 [6:48:31<4:10:00,  2.35s/it]
 66%|██████▌   | 12234/18627 [6:48:33<4:08:46,  2.33s/it]
 66%|██████▌   | 12235/18627 [6:48:36<4:08:05,  2.33s/it]
 66%|██████▌   | 12236/18627 [6:48:37<3:37:44,  2.04s/it]
 66%|██████▌   | 12237/18627 [6:48:39<3:46:17,  2.12s/it]
 66%|██████▌   | 12238/18627 [6:48:42<3:51:49,  2.18s/it]
 66%|██████▌   | 12239/18627 [6:48:44<3:56:30,  2.22s/it]
 66%|██████▌   | 12240/18627 [6:48:46<3:59:33,  2.25s/it]
                                                         
{'loss': 0.8346, 'grad_norm': 5.195940971374512, 'learning_rate': 1.3895698889221803e-06, 'epoch': 0.66}
+
 66%|██████▌   | 12240/18627 [6:48:46<3:59:33,  2.25s/it]
 66%|██████▌   | 12241/18627 [6:48:49<4:01:12,  2.27s/it]
 66%|██████▌   | 12242/18627 [6:48:50<3:23:18,  1.91s/it]
 66%|██████▌   | 12243/18627 [6:48:52<3:35:37,  2.03s/it]
 66%|██████▌   | 12244/18627 [6:48:54<3:45:16,  2.12s/it]
 66%|██████▌   | 12245/18627 [6:48:56<3:20:43,  1.89s/it]
 66%|██████▌   | 12246/18627 [6:48:58<3:34:07,  2.01s/it]
 66%|██████▌   | 12247/18627 [6:49:00<3:43:16,  2.10s/it]
 66%|██████▌   | 12248/18627 [6:49:03<3:50:20,  2.17s/it]
 66%|██████▌   | 12249/18627 [6:49:05<3:54:56,  2.21s/it]
 66%|██████▌   | 12250/18627 [6:49:07<3:58:11,  2.24s/it]
                                                         
{'loss': 0.8541, 'grad_norm': 10.415426254272461, 'learning_rate': 1.3856769999502107e-06, 'epoch': 0.66}
+
 66%|██████▌   | 12250/18627 [6:49:07<3:58:11,  2.24s/it]
 66%|██████▌   | 12251/18627 [6:49:09<4:00:48,  2.27s/it]
 66%|██████▌   | 12252/18627 [6:49:11<3:31:27,  1.99s/it]
 66%|██████▌   | 12253/18627 [6:49:13<3:41:58,  2.09s/it]
 66%|██████▌   | 12254/18627 [6:49:15<3:49:00,  2.16s/it]
 66%|██████▌   | 12255/18627 [6:49:18<3:52:59,  2.19s/it]
 66%|██████▌   | 12256/18627 [6:49:20<3:56:46,  2.23s/it]
 66%|██████▌   | 12257/18627 [6:49:22<4:00:14,  2.26s/it]
 66%|██████▌   | 12258/18627 [6:49:25<4:03:04,  2.29s/it]
 66%|██████▌   | 12259/18627 [6:49:27<4:03:44,  2.30s/it]
 66%|██████▌   | 12260/18627 [6:49:29<4:04:33,  2.30s/it]
                                                         
{'loss': 0.8138, 'grad_norm': 7.565029621124268, 'learning_rate': 1.381787479897222e-06, 'epoch': 0.66}
+
 66%|██████▌   | 12260/18627 [6:49:29<4:04:33,  2.30s/it]
 66%|██████▌   | 12261/18627 [6:49:32<4:06:12,  2.32s/it]
 66%|██████▌   | 12262/18627 [6:49:34<4:07:06,  2.33s/it]
 66%|██████▌   | 12263/18627 [6:49:36<4:05:49,  2.32s/it]
 66%|██████▌   | 12264/18627 [6:49:38<3:32:44,  2.01s/it]
 66%|██████▌   | 12265/18627 [6:49:39<3:10:58,  1.80s/it]
 66%|██████▌   | 12266/18627 [6:49:41<3:28:36,  1.97s/it]
 66%|██████▌   | 12267/18627 [6:49:44<3:38:53,  2.06s/it]
 66%|██████▌   | 12268/18627 [6:49:45<3:16:15,  1.85s/it]
 66%|██████▌   | 12269/18627 [6:49:47<3:31:15,  1.99s/it]
 66%|���█████▌   | 12270/18627 [6:49:49<3:11:06,  1.80s/it]
                                                         
{'loss': 1.324, 'grad_norm': 14.056329727172852, 'learning_rate': 1.377901340522353e-06, 'epoch': 0.66}
+
 66%|██████▌   | 12270/18627 [6:49:49<3:11:06,  1.80s/it]
 66%|██████▌   | 12271/18627 [6:49:51<3:27:21,  1.96s/it]
 66%|██████▌   | 12272/18627 [6:49:53<3:37:59,  2.06s/it]
 66%|██████▌   | 12273/18627 [6:49:56<3:46:04,  2.13s/it]
 66%|██████▌   | 12274/18627 [6:49:58<3:50:50,  2.18s/it]
 66%|██████▌   | 12275/18627 [6:50:00<3:54:20,  2.21s/it]
 66%|██████▌   | 12276/18627 [6:50:03<3:57:31,  2.24s/it]
 66%|██████▌   | 12277/18627 [6:50:04<3:28:47,  1.97s/it]
 66%|██████▌   | 12278/18627 [6:50:05<3:08:55,  1.79s/it]
 66%|██████▌   | 12279/18627 [6:50:07<2:54:48,  1.65s/it]
 66%|██████▌   | 12280/18627 [6:50:08<2:47:03,  1.58s/it]
                                                         
{'loss': 1.4309, 'grad_norm': 14.161043167114258, 'learning_rate': 1.37401859357452e-06, 'epoch': 0.66}
+
 66%|██████▌   | 12280/18627 [6:50:08<2:47:03,  1.58s/it]
 66%|██████▌   | 12281/18627 [6:50:10<3:10:29,  1.80s/it]
 66%|██████▌   | 12282/18627 [6:50:13<3:26:52,  1.96s/it]
 66%|██████▌   | 12283/18627 [6:50:15<3:37:22,  2.06s/it]
 66%|██████▌   | 12284/18627 [6:50:17<3:45:02,  2.13s/it]
 66%|██████▌   | 12285/18627 [6:50:19<3:50:22,  2.18s/it]
 66%|██████▌   | 12286/18627 [6:50:22<3:56:07,  2.23s/it]
 66%|██████▌   | 12287/18627 [6:50:24<3:59:13,  2.26s/it]
 66%|██████▌   | 12288/18627 [6:50:27<4:01:32,  2.29s/it]
 66%|██████▌   | 12289/18627 [6:50:29<4:02:41,  2.30s/it]
 66%|██████▌   | 12290/18627 [6:50:31<4:03:58,  2.31s/it]
                                                         
{'loss': 0.6536, 'grad_norm': 5.29819917678833, 'learning_rate': 1.370139250792385e-06, 'epoch': 0.66}
+
 66%|██████▌   | 12290/18627 [6:50:31<4:03:58,  2.31s/it]
 66%|██████▌   | 12291/18627 [6:50:33<4:04:10,  2.31s/it]
 66%|██████▌   | 12292/18627 [6:50:36<4:05:49,  2.33s/it]
 66%|██████▌   | 12293/18627 [6:50:38<4:04:34,  2.32s/it]
 66%|██████▌   | 12294/18627 [6:50:40<4:03:33,  2.31s/it]
 66%|██████▌   | 12295/18627 [6:50:43<4:02:25,  2.30s/it]
 66%|██████▌   | 12296/18627 [6:50:44<3:31:05,  2.00s/it]
 66%|██████▌   | 12297/18627 [6:50:46<3:40:22,  2.09s/it]
 66%|██████▌   | 12298/18627 [6:50:49<3:47:03,  2.15s/it]
 66%|██████▌   | 12299/18627 [6:50:50<3:20:34,  1.90s/it]
 66%|██████▌   | 12300/18627 [6:50:51<3:00:59,  1.72s/it]
                                                         
{'loss': 1.1378, 'grad_norm': 14.71925163269043, 'learning_rate': 1.3662633239043171e-06, 'epoch': 0.66}
+
 66%|██████▌   | 12300/18627 [6:50:51<3:00:59,  1.72s/it]
 66%|██████▌   | 12301/18627 [6:50:53<3:18:54,  1.89s/it]
 66%|██████▌   | 12302/18627 [6:50:56<3:32:17,  2.01s/it]
 66%|██████▌   | 12303/18627 [6:50:58<3:41:37,  2.10s/it]
 66%|██████▌   | 12304/18627 [6:50:59<3:18:16,  1.88s/it]
 66%|██████▌   | 12305/18627 [6:51:02<3:31:41,  2.01s/it]
 66%|██████▌   | 12306/18627 [6:51:03<3:02:29,  1.73s/it]
 66%|██████▌   | 12307/18627 [6:51:05<3:20:59,  1.91s/it]
 66%|██████▌   | 12308/18627 [6:51:08<3:34:25,  2.04s/it]
 66%|██████▌   | 12309/18627 [6:51:10<3:43:25,  2.12s/it]
 66%|██████▌   | 12310/18627 [6:51:12<3:48:26,  2.17s/it]
                                                         
{'loss': 0.8647, 'grad_norm': 4.825445652008057, 'learning_rate': 1.362390824628356e-06, 'epoch': 0.66}
+
 66%|██████▌   | 12310/18627 [6:51:12<3:48:26,  2.17s/it]
 66%|██████▌   | 12311/18627 [6:51:14<3:52:23,  2.21s/it]
 66%|██████▌   | 12312/18627 [6:51:16<3:25:02,  1.95s/it]
 66%|██████▌   | 12313/18627 [6:51:18<3:36:13,  2.05s/it]
 66%|██████▌   | 12314/18627 [6:51:20<3:43:05,  2.12s/it]
 66%|██████▌   | 12315/18627 [6:51:22<3:18:16,  1.88s/it]
 66%|██████▌   | 12316/18627 [6:51:24<3:31:02,  2.01s/it]
 66%|██████▌   | 12317/18627 [6:51:26<3:40:57,  2.10s/it]
 66%|██████▌   | 12318/18627 [6:51:29<3:48:03,  2.17s/it]
 66%|██████▌   | 12319/18627 [6:51:30<3:23:12,  1.93s/it]
 66%|██████▌   | 12320/18627 [6:51:31<3:04:18,  1.75s/it]
                                                         
{'loss': 1.4306, 'grad_norm': 13.501662254333496, 'learning_rate': 1.3585217646721846e-06, 'epoch': 0.66}
+
 66%|██████▌   | 12320/18627 [6:51:31<3:04:18,  1.75s/it]
 66%|██████▌   | 12321/18627 [6:51:34<3:22:23,  1.93s/it]
 66%|██████▌   | 12322/18627 [6:51:36<3:34:57,  2.05s/it]
 66%|██████▌   | 12323/18627 [6:51:38<3:43:14,  2.12s/it]
 66%|██████▌   | 12324/18627 [6:51:40<3:19:11,  1.90s/it]
 66%|██████▌   | 12325/18627 [6:51:42<3:32:39,  2.02s/it]
 66%|██████▌   | 12326/18627 [6:51:44<3:41:21,  2.11s/it]
 66%|██████▌   | 12327/18627 [6:51:46<3:17:45,  1.88s/it]
 66%|██████▌   | 12328/18627 [6:51:48<3:33:12,  2.03s/it]
 66%|██████▌   | 12329/18627 [6:51:50<3:41:28,  2.11s/it]
 66%|██████▌   | 12330/18627 [6:51:53<3:47:34,  2.17s/it]
                                                         
{'loss': 1.0999, 'grad_norm': 5.907223701477051, 'learning_rate': 1.3546561557330814e-06, 'epoch': 0.66}
+
 66%|██████▌   | 12330/18627 [6:51:53<3:47:34,  2.17s/it]
 66%|██████▌   | 12331/18627 [6:51:55<3:52:10,  2.21s/it]
 66%|██████▌   | 12332/18627 [6:51:57<3:54:57,  2.24s/it]
 66%|██████▌   | 12333/18627 [6:52:00<3:57:16,  2.26s/it]
 66%|██████▌   | 12334/18627 [6:52:01<3:31:35,  2.02s/it]
 66%|██████▌   | 12335/18627 [6:52:03<3:40:20,  2.10s/it]
 66%|██████▌   | 12336/18627 [6:52:06<3:48:07,  2.18s/it]
 66%|██████▌   | 12337/18627 [6:52:08<3:52:59,  2.22s/it]
 66%|██████▌   | 12338/18627 [6:52:09<3:25:43,  1.96s/it]
 66%|██████▌   | 12339/18627 [6:52:12<3:37:22,  2.07s/it]
 66%|██████▌   | 12340/18627 [6:52:13<3:14:08,  1.85s/it]
                                                         
{'loss': 1.1726, 'grad_norm': 13.576423645019531, 'learning_rate': 1.3507940094978953e-06, 'epoch': 0.66}
+
 66%|██████▌   | 12340/18627 [6:52:13<3:14:08,  1.85s/it]
 66%|██████▋   | 12341/18627 [6:52:15<3:29:15,  2.00s/it]
 66%|██████▋   | 12342/18627 [6:52:18<3:39:18,  2.09s/it]
 66%|██████▋   | 12343/18627 [6:52:20<3:45:48,  2.16s/it]
 66%|██████▋   | 12344/18627 [6:52:22<3:50:25,  2.20s/it]
 66%|██████▋   | 12345/18627 [6:52:25<3:53:48,  2.23s/it]
 66%|██████▋   | 12346/18627 [6:52:27<3:56:41,  2.26s/it]
 66%|██████▋   | 12347/18627 [6:52:29<3:58:30,  2.28s/it]
 66%|██████▋   | 12348/18627 [6:52:32<3:58:51,  2.28s/it]
 66%|██████▋   | 12349/18627 [6:52:34<3:59:28,  2.29s/it]
 66%|██████▋   | 12350/18627 [6:52:36<4:00:51,  2.30s/it]
                                                         
{'loss': 0.5968, 'grad_norm': 5.912051677703857, 'learning_rate': 1.346935337643005e-06, 'epoch': 0.66}
+
 66%|██████▋   | 12350/18627 [6:52:36<4:00:51,  2.30s/it]
 66%|██████▋   | 12351/18627 [6:52:38<4:01:47,  2.31s/it]
 66%|██████▋   | 12352/18627 [6:52:41<4:02:13,  2.32s/it]
 66%|██████▋   | 12353/18627 [6:52:43<4:02:41,  2.32s/it]
 66%|██████▋   | 12354/18627 [6:52:45<4:02:33,  2.32s/it]
 66%|██████▋   | 12355/18627 [6:52:46<3:22:17,  1.94s/it]
 66%|██████▋   | 12356/18627 [6:52:49<3:34:48,  2.06s/it]
 66%|██████▋   | 12357/18627 [6:52:51<3:42:28,  2.13s/it]
 66%|██████▋   | 12358/18627 [6:52:53<3:48:06,  2.18s/it]
 66%|██████▋   | 12359/18627 [6:52:56<3:52:07,  2.22s/it]
 66%|██████▋   | 12360/18627 [6:52:58<3:55:06,  2.25s/it]
                                                         
{'loss': 0.6175, 'grad_norm': 5.733963966369629, 'learning_rate': 1.3430801518342857e-06, 'epoch': 0.66}
+
 66%|██████▋   | 12360/18627 [6:52:58<3:55:06,  2.25s/it]
 66%|██████▋   | 12361/18627 [6:53:00<3:57:21,  2.27s/it]
 66%|██████▋   | 12362/18627 [6:53:03<3:59:07,  2.29s/it]
 66%|██████▋   | 12363/18627 [6:53:05<3:59:48,  2.30s/it]
 66%|██████▋   | 12364/18627 [6:53:07<4:00:25,  2.30s/it]
 66%|██████▋   | 12365/18627 [6:53:10<3:59:58,  2.30s/it]
 66%|██████▋   | 12366/18627 [6:53:12<4:01:06,  2.31s/it]
 66%|██████▋   | 12367/18627 [6:53:14<4:01:48,  2.32s/it]
 66%|██████▋   | 12368/18627 [6:53:16<3:33:56,  2.05s/it]
 66%|██████▋   | 12369/18627 [6:53:18<3:41:36,  2.12s/it]
 66%|██████▋   | 12370/18627 [6:53:20<3:47:22,  2.18s/it]
                                                         
{'loss': 0.8163, 'grad_norm': 6.955583095550537, 'learning_rate': 1.3392284637270732e-06, 'epoch': 0.66}
+
 66%|██████▋   | 12370/18627 [6:53:20<3:47:22,  2.18s/it]
 66%|██████▋   | 12371/18627 [6:53:23<3:52:44,  2.23s/it]
 66%|██████▋   | 12372/18627 [6:53:25<3:55:27,  2.26s/it]
 66%|██████▋   | 12373/18627 [6:53:27<3:58:06,  2.28s/it]
 66%|██████▋   | 12374/18627 [6:53:30<4:01:06,  2.31s/it]
 66%|██████▋   | 12375/18627 [6:53:32<4:01:07,  2.31s/it]
 66%|██████▋   | 12376/18627 [6:53:34<4:02:35,  2.33s/it]
 66%|██████▋   | 12377/18627 [6:53:37<4:03:48,  2.34s/it]
 66%|██████▋   | 12378/18627 [6:53:39<4:02:41,  2.33s/it]
 66%|██████▋   | 12379/18627 [6:53:40<3:23:39,  1.96s/it]
 66%|██████▋   | 12380/18627 [6:53:42<3:34:46,  2.06s/it]
                                                         
{'loss': 0.6663, 'grad_norm': 5.295736789703369, 'learning_rate': 1.3353802849661296e-06, 'epoch': 0.66}
+
 66%|██████▋   | 12380/18627 [6:53:43<3:34:46,  2.06s/it]
 66%|██████▋   | 12381/18627 [6:53:45<3:43:06,  2.14s/it]
 66%|██████▋   | 12382/18627 [6:53:47<3:47:32,  2.19s/it]
 66%|██████▋   | 12383/18627 [6:53:49<3:51:34,  2.23s/it]
 66%|██████▋   | 12384/18627 [6:53:51<3:24:31,  1.97s/it]
 66%|██████▋   | 12385/18627 [6:53:53<3:35:16,  2.07s/it]
 66%|██████▋   | 12386/18627 [6:53:54<3:14:21,  1.87s/it]
 67%|██████▋   | 12387/18627 [6:53:56<2:58:26,  1.72s/it]
 67%|██████▋   | 12388/18627 [6:53:58<3:16:23,  1.89s/it]
 67%|██████▋   | 12389/18627 [6:54:00<3:30:00,  2.02s/it]
 67%|██████▋   | 12390/18627 [6:54:02<3:08:38,  1.81s/it]
                                                         
{'loss': 1.3306, 'grad_norm': 15.844099044799805, 'learning_rate': 1.3315356271856034e-06, 'epoch': 0.67}
+
 67%|██████▋   | 12390/18627 [6:54:02<3:08:38,  1.81s/it]
 67%|██████▋   | 12391/18627 [6:54:04<3:23:52,  1.96s/it]
 67%|██████▋   | 12392/18627 [6:54:06<3:34:06,  2.06s/it]
 67%|██████▋   | 12393/18627 [6:54:08<3:11:32,  1.84s/it]
 67%|██████▋   | 12394/18627 [6:54:10<3:25:36,  1.98s/it]
 67%|██████▋   | 12395/18627 [6:54:11<3:05:58,  1.79s/it]
 67%|██████▋   | 12396/18627 [6:54:14<3:21:41,  1.94s/it]
 67%|██████▋   | 12397/18627 [6:54:16<3:32:26,  2.05s/it]
 67%|██████▋   | 12398/18627 [6:54:18<3:40:20,  2.12s/it]
 67%|██████▋   | 12399/18627 [6:54:21<3:46:15,  2.18s/it]
 67%|██████▋   | 12400/18627 [6:54:23<3:50:32,  2.22s/it]
                                                         
{'loss': 1.0683, 'grad_norm': 5.621187210083008, 'learning_rate': 1.3276945020090048e-06, 'epoch': 0.67}
+
 67%|██████▋   | 12400/18627 [6:54:23<3:50:32,  2.22s/it]
 67%|██████▋   | 12401/18627 [6:54:25<3:53:42,  2.25s/it]
 67%|██████▋   | 12402/18627 [6:54:28<3:55:34,  2.27s/it]
 67%|██████▋   | 12403/18627 [6:54:29<3:28:07,  2.01s/it]
 67%|██████▋   | 12404/18627 [6:54:31<3:38:12,  2.10s/it]
 67%|██████▋   | 12405/18627 [6:54:33<3:14:29,  1.88s/it]
 67%|██████▋   | 12406/18627 [6:54:35<3:29:23,  2.02s/it]
 67%|██████▋   | 12407/18627 [6:54:37<3:37:52,  2.10s/it]
 67%|██████▋   | 12408/18627 [6:54:39<3:14:51,  1.88s/it]
 67%|██████▋   | 12409/18627 [6:54:41<3:28:56,  2.02s/it]
 67%|██████▋   | 12410/18627 [6:54:43<3:38:20,  2.11s/it]
                                                         
{'loss': 1.2167, 'grad_norm': 6.563199996948242, 'learning_rate': 1.3238569210491572e-06, 'epoch': 0.67}
+
 67%|██████▋   | 12410/18627 [6:54:43<3:38:20,  2.11s/it]
 67%|██████▋   | 12411/18627 [6:54:46<3:45:36,  2.18s/it]
 67%|██████▋   | 12412/18627 [6:54:48<3:48:47,  2.21s/it]
 67%|██████▋   | 12413/18627 [6:54:49<3:21:55,  1.95s/it]
 67%|██████▋   | 12414/18627 [6:54:52<3:33:55,  2.07s/it]
 67%|██████▋   | 12415/18627 [6:54:54<3:41:45,  2.14s/it]
 67%|██████▋   | 12416/18627 [6:54:55<3:06:59,  1.81s/it]
 67%|██████▋   | 12417/18627 [6:54:56<2:53:17,  1.67s/it]
 67%|██████▋   | 12418/18627 [6:54:59<3:12:46,  1.86s/it]
 67%|██████▋   | 12419/18627 [6:55:01<3:27:15,  2.00s/it]
 67%|██████▋   | 12420/18627 [6:55:03<3:36:50,  2.10s/it]
                                                         
{'loss': 1.0659, 'grad_norm': 6.345821380615234, 'learning_rate': 1.3200228959081728e-06, 'epoch': 0.67}
+
 67%|██████▋   | 12420/18627 [6:55:03<3:36:50,  2.10s/it]
 67%|██████▋   | 12421/18627 [6:55:06<3:42:35,  2.15s/it]
 67%|██████▋   | 12422/18627 [6:55:08<3:47:38,  2.20s/it]
 67%|██████▋   | 12423/18627 [6:55:10<3:50:34,  2.23s/it]
 67%|██████▋   | 12424/18627 [6:55:12<3:53:32,  2.26s/it]
 67%|██████▋   | 12425/18627 [6:55:15<3:55:09,  2.28s/it]
 67%|██████▋   | 12426/18627 [6:55:17<3:56:46,  2.29s/it]
 67%|██████▋   | 12427/18627 [6:55:19<3:57:33,  2.30s/it]
 67%|██████▋   | 12428/18627 [6:55:22<3:57:54,  2.30s/it]
 67%|██████▋   | 12429/18627 [6:55:24<3:57:49,  2.30s/it]
 67%|██████▋   | 12430/18627 [6:55:26<3:58:04,  2.30s/it]
                                                         
{'loss': 0.6355, 'grad_norm': 8.502699851989746, 'learning_rate': 1.3161924381774126e-06, 'epoch': 0.67}
+
 67%|██████▋   | 12430/18627 [6:55:26<3:58:04,  2.30s/it]
 67%|██████▋   | 12431/18627 [6:55:28<3:29:36,  2.03s/it]
 67%|██████▋   | 12432/18627 [6:55:30<3:37:52,  2.11s/it]
 67%|██████▋   | 12433/18627 [6:55:32<3:44:31,  2.17s/it]
 67%|██████▋   | 12434/18627 [6:55:35<3:48:31,  2.21s/it]
 67%|██████▋   | 12435/18627 [6:55:37<3:51:25,  2.24s/it]
 67%|██████▋   | 12436/18627 [6:55:39<3:54:12,  2.27s/it]
 67%|██████▋   | 12437/18627 [6:55:42<3:56:11,  2.29s/it]
 67%|██████▋   | 12438/18627 [6:55:44<3:56:32,  2.29s/it]
 67%|██████▋   | 12439/18627 [6:55:46<3:56:42,  2.30s/it]
 67%|██████▋   | 12440/18627 [6:55:49<3:57:22,  2.30s/it]
                                                         
{'loss': 0.8689, 'grad_norm': 5.2658162117004395, 'learning_rate': 1.3123655594374523e-06, 'epoch': 0.67}
+
 67%|██████▋   | 12440/18627 [6:55:49<3:57:22,  2.30s/it]
 67%|██████▋   | 12441/18627 [6:55:51<3:58:02,  2.31s/it]
 67%|██████▋   | 12442/18627 [6:55:52<3:28:51,  2.03s/it]
 67%|██████▋   | 12443/18627 [6:55:54<3:07:49,  1.82s/it]
 67%|██████▋   | 12444/18627 [6:55:56<3:22:32,  1.97s/it]
 67%|██████▋   | 12445/18627 [6:55:58<3:33:34,  2.07s/it]
 67%|██████▋   | 12446/18627 [6:56:01<3:41:17,  2.15s/it]
 67%|██████▋   | 12447/18627 [6:56:03<3:46:53,  2.20s/it]
 67%|██████▋   | 12448/18627 [6:56:05<3:50:44,  2.24s/it]
 67%|██████▋   | 12449/18627 [6:56:07<3:52:35,  2.26s/it]
 67%|██████▋   | 12450/18627 [6:56:10<3:53:48,  2.27s/it]
                                                         
{'loss': 1.0245, 'grad_norm': 5.402364253997803, 'learning_rate': 1.3085422712580473e-06, 'epoch': 0.67}
+
 67%|██████▋   | 12450/18627 [6:56:10<3:53:48,  2.27s/it]
 67%|██████▋   | 12451/18627 [6:56:12<3:55:16,  2.29s/it]
 67%|██████▋   | 12452/18627 [6:56:14<3:55:37,  2.29s/it]
 67%|██████▋   | 12453/18627 [6:56:17<3:55:37,  2.29s/it]
 67%|██████▋   | 12454/18627 [6:56:19<3:54:48,  2.28s/it]
 67%|██████▋   | 12455/18627 [6:56:21<3:55:09,  2.29s/it]
 67%|██████▋   | 12456/18627 [6:56:24<3:55:30,  2.29s/it]
 67%|██████▋   | 12457/18627 [6:56:26<3:56:08,  2.30s/it]
 67%|██████▋   | 12458/18627 [6:56:27<3:27:14,  2.02s/it]
 67%|██████▋   | 12459/18627 [6:56:30<3:36:16,  2.10s/it]
 67%|██████▋   | 12460/18627 [6:56:32<3:43:12,  2.17s/it]
                                                         
{'loss': 0.8626, 'grad_norm': 6.024814605712891, 'learning_rate': 1.304722585198098e-06, 'epoch': 0.67}
+
 67%|██████▋   | 12460/18627 [6:56:32<3:43:12,  2.17s/it]
 67%|██████▋   | 12461/18627 [6:56:33<3:18:04,  1.93s/it]
 67%|██████▋   | 12462/18627 [6:56:36<3:29:48,  2.04s/it]
 67%|██████▋   | 12463/18627 [6:56:38<3:38:04,  2.12s/it]
 67%|██████▋   | 12464/18627 [6:56:39<3:04:11,  1.79s/it]
 67%|██████▋   | 12465/18627 [6:56:41<3:21:56,  1.97s/it]
 67%|██████▋   | 12466/18627 [6:56:43<3:02:22,  1.78s/it]
 67%|██████▋   | 12467/18627 [6:56:45<3:19:26,  1.94s/it]
 67%|██████▋   | 12468/18627 [6:56:47<3:31:17,  2.06s/it]
 67%|██████▋   | 12469/18627 [6:56:50<3:38:13,  2.13s/it]
 67%|██████▋   | 12470/18627 [6:56:51<3:12:04,  1.87s/it]
                                                         
{'loss': 1.2364, 'grad_norm': 13.036808013916016, 'learning_rate': 1.3009065128056118e-06, 'epoch': 0.67}
+
 67%|██████▋   | 12470/18627 [6:56:51<3:12:04,  1.87s/it]
 67%|██████▋   | 12471/18627 [6:56:53<3:26:32,  2.01s/it]
 67%|██████▋   | 12472/18627 [6:56:55<3:36:12,  2.11s/it]
 67%|██████▋   | 12473/18627 [6:56:58<3:42:05,  2.17s/it]
 67%|██████▋   | 12474/18627 [6:57:00<3:46:18,  2.21s/it]
 67%|██████▋   | 12475/18627 [6:57:02<3:48:36,  2.23s/it]
 67%|██████▋   | 12476/18627 [6:57:05<3:50:25,  2.25s/it]
 67%|██████▋   | 12477/18627 [6:57:07<3:51:55,  2.26s/it]
 67%|██████▋   | 12478/18627 [6:57:09<3:53:32,  2.28s/it]
 67%|██████▋   | 12479/18627 [6:57:12<3:54:33,  2.29s/it]
 67%|██████▋   | 12480/18627 [6:57:14<3:54:22,  2.29s/it]
                                                         
{'loss': 0.6918, 'grad_norm': 5.381145000457764, 'learning_rate': 1.2970940656176768e-06, 'epoch': 0.67}
+
 67%|██████▋   | 12480/18627 [6:57:14<3:54:22,  2.29s/it]
 67%|██████▋   | 12481/18627 [6:57:16<3:55:06,  2.30s/it]
 67%|█��████▋   | 12482/18627 [6:57:18<3:55:39,  2.30s/it]
 67%|██████▋   | 12483/18627 [6:57:21<3:56:10,  2.31s/it]
 67%|██████▋   | 12484/18627 [6:57:23<3:56:31,  2.31s/it]
 67%|██████▋   | 12485/18627 [6:57:25<3:55:39,  2.30s/it]
 67%|██████▋   | 12486/18627 [6:57:28<3:55:04,  2.30s/it]
 67%|██████▋   | 12487/18627 [6:57:30<3:55:31,  2.30s/it]
 67%|██████▋   | 12488/18627 [6:57:32<3:55:40,  2.30s/it]
 67%|██████▋   | 12489/18627 [6:57:35<3:55:30,  2.30s/it]
 67%|██████▋   | 12490/18627 [6:57:37<3:56:22,  2.31s/it]
                                                         
{'loss': 0.6644, 'grad_norm': 5.74312162399292, 'learning_rate': 1.2932852551604146e-06, 'epoch': 0.67}
+
 67%|██████▋   | 12490/18627 [6:57:37<3:56:22,  2.31s/it]
 67%|██████▋   | 12491/18627 [6:57:38<3:27:35,  2.03s/it]
 67%|██████▋   | 12492/18627 [6:57:41<3:36:28,  2.12s/it]
 67%|██████▋   | 12493/18627 [6:57:43<3:42:00,  2.17s/it]
 67%|██████▋   | 12494/18627 [6:57:45<3:46:17,  2.21s/it]
 67%|██████▋   | 12495/18627 [6:57:48<3:50:32,  2.26s/it]
 67%|██████▋   | 12496/18627 [6:57:50<3:52:22,  2.27s/it]
 67%|██████▋   | 12497/18627 [6:57:52<3:54:16,  2.29s/it]
 67%|██████▋   | 12498/18627 [6:57:55<3:54:21,  2.29s/it]
 67%|██████▋   | 12499/18627 [6:57:57<3:54:56,  2.30s/it]
 67%|██████▋   | 12500/18627 [6:57:59<3:57:22,  2.32s/it]
                                                         
{'loss': 0.82, 'grad_norm': 5.068548679351807, 'learning_rate': 1.2894800929489563e-06, 'epoch': 0.67}
+
 67%|██████▋   | 12500/18627 [6:57:59<3:57:22,  2.32s/it]
 67%|██████▋   | 12501/18627 [6:58:01<3:28:09,  2.04s/it]
 67%|██████▋   | 12502/18627 [6:58:03<3:36:06,  2.12s/it]
 67%|██████▋   | 12503/18627 [6:58:05<3:41:44,  2.17s/it]
 67%|██████▋   | 12504/18627 [6:58:08<3:46:47,  2.22s/it]
 67%|██████▋   | 12505/18627 [6:58:10<3:49:41,  2.25s/it]
 67%|██████▋   | 12506/18627 [6:58:12<3:51:41,  2.27s/it]
 67%|██████▋   | 12507/18627 [6:58:15<3:53:36,  2.29s/it]
 67%|██████▋   | 12508/18627 [6:58:17<3:54:11,  2.30s/it]
 67%|██████▋   | 12509/18627 [6:58:19<3:54:43,  2.30s/it]
 67%|██████▋   | 12510/18627 [6:58:21<3:54:54,  2.30s/it]
                                                         
{'loss': 0.7554, 'grad_norm': 6.714485168457031, 'learning_rate': 1.2856785904874015e-06, 'epoch': 0.67}
+
 67%|██████▋   | 12510/18627 [6:58:21<3:54:54,  2.30s/it]
 67%|██████▋   | 12511/18627 [6:58:24<3:54:36,  2.30s/it]
 67%|██████▋   | 12512/18627 [6:58:26<3:54:51,  2.30s/it]
 67%|██████▋   | 12513/18627 [6:58:28<3:54:59,  2.31s/it]
 67%|██████▋   | 12514/18627 [6:58:31<3:54:21,  2.30s/it]
 67%|██████▋   | 12515/18627 [6:58:33<3:53:42,  2.29s/it]
 67%|██████▋   | 12516/18627 [6:58:35<3:55:02,  2.31s/it]
 67%|██████▋   | 12517/18627 [6:58:38<3:55:12,  2.31s/it]
 67%|██████▋   | 12518/18627 [6:58:40<3:55:24,  2.31s/it]
 67%|██████▋   | 12519/18627 [6:58:42<3:56:11,  2.32s/it]
 67%|██████▋   | 12520/18627 [6:58:44<3:26:15,  2.03s/it]
                                                         
{'loss': 0.7694, 'grad_norm': 13.879148483276367, 'learning_rate': 1.281880759268786e-06, 'epoch': 0.67}
+
 67%|██████▋   | 12520/18627 [6:58:44<3:26:15,  2.03s/it]
 67%|██████▋   | 12521/18627 [6:58:46<3:36:17,  2.13s/it]
 67%|██████▋   | 12522/18627 [6:58:47<3:13:36,  1.90s/it]
 67%|██████▋   | 12523/18627 [6:58:50<3:26:26,  2.03s/it]
 67%|██████▋   | 12524/18627 [6:58:52<3:35:07,  2.12s/it]
 67%|██████▋   | 12525/18627 [6:58:54<3:41:17,  2.18s/it]
 67%|██████▋   | 12526/18627 [6:58:57<3:44:32,  2.21s/it]
 67%|██████▋   | 12527/18627 [6:58:59<3:48:15,  2.25s/it]
 67%|██████▋   | 12528/18627 [6:59:01<3:50:49,  2.27s/it]
 67%|██████▋   | 12529/18627 [6:59:03<3:22:45,  1.99s/it]
 67%|██████▋   | 12530/18627 [6:59:04<3:01:09,  1.78s/it]
                                                         
{'loss': 1.2169, 'grad_norm': 13.900629997253418, 'learning_rate': 1.2780866107750455e-06, 'epoch': 0.67}
+
 67%|██████▋   | 12530/18627 [6:59:04<3:01:09,  1.78s/it]
 67%|██████▋   | 12531/18627 [6:59:06<3:16:53,  1.94s/it]
 67%|██████▋   | 12532/18627 [6:59:08<3:28:04,  2.05s/it]
 67%|██████▋   | 12533/18627 [6:59:11<3:35:21,  2.12s/it]
 67%|██████▋   | 12534/18627 [6:59:13<3:40:32,  2.17s/it]
 67%|██████▋   | 12535/18627 [6:59:15<3:45:16,  2.22s/it]
 67%|██████▋   | 12536/18627 [6:59:18<3:46:41,  2.23s/it]
 67%|██████▋   | 12537/18627 [6:59:20<3:49:24,  2.26s/it]
 67%|██████▋   | 12538/18627 [6:59:22<3:51:58,  2.29s/it]
 67%|██████▋   | 12539/18627 [6:59:24<3:20:36,  1.98s/it]
 67%|██████▋   | 12540/18627 [6:59:26<3:31:11,  2.08s/it]
                                                         
{'loss': 0.8179, 'grad_norm': 6.083281993865967, 'learning_rate': 1.2742961564769832e-06, 'epoch': 0.67}
+
 67%|██████▋   | 12540/18627 [6:59:26<3:31:11,  2.08s/it]
 67%|██████▋   | 12541/18627 [6:59:28<3:37:57,  2.15s/it]
 67%|██████▋   | 12542/18627 [6:59:31<3:42:34,  2.19s/it]
 67%|██████▋   | 12543/18627 [6:59:33<3:45:15,  2.22s/it]
 67%|██████▋   | 12544/18627 [6:59:35<3:48:15,  2.25s/it]
 67%|██████▋   | 12545/18627 [6:59:37<3:51:33,  2.28s/it]
 67%|██████▋   | 12546/18627 [6:59:40<3:53:13,  2.30s/it]
 67%|██████▋   | 12547/18627 [6:59:42<3:53:57,  2.31s/it]
 67%|██████▋   | 12548/18627 [6:59:45<3:55:19,  2.32s/it]
 67%|██████▋   | 12549/18627 [6:59:47<3:55:37,  2.33s/it]
 67%|██████▋   | 12550/18627 [6:59:49<3:55:02,  2.32s/it]
                                                         
{'loss': 0.5999, 'grad_norm': 6.199990749359131, 'learning_rate': 1.2705094078342322e-06, 'epoch': 0.67}
+
 67%|██████▋   | 12550/18627 [6:59:49<3:55:02,  2.32s/it]
 67%|██████▋   | 12551/18627 [6:59:52<3:56:11,  2.33s/it]
 67%|██████▋   | 12552/18627 [6:59:54<3:56:11,  2.33s/it]
 67%|██████▋   | 12553/18627 [6:59:55<3:26:50,  2.04s/it]
 67%|██████▋   | 12554/18627 [6:59:58<3:34:14,  2.12s/it]
 67%|██████▋   | 12555/18627 [6:59:59<3:10:50,  1.89s/it]
 67%|██████▋   | 12556/18627 [7:00:00<2:54:52,  1.73s/it]
 67%|██████▋   | 12557/18627 [7:00:03<3:13:01,  1.91s/it]
 67%|██████▋   | 12558/18627 [7:00:05<3:25:51,  2.04s/it]
 67%|██████▋   | 12559/18627 [7:00:07<3:34:14,  2.12s/it]
 67%|██████▋   | 12560/18627 [7:00:09<3:39:24,  2.17s/it]
                                                         
{'loss': 1.2233, 'grad_norm': 7.213817119598389, 'learning_rate': 1.266726376295223e-06, 'epoch': 0.67}
+
 67%|██████▋   | 12560/18627 [7:00:09<3:39:24,  2.17s/it]
 67%|██████▋   | 12561/18627 [7:00:12<3:44:25,  2.22s/it]
 67%|██████▋   | 12562/18627 [7:00:14<3:47:55,  2.25s/it]
 67%|██████▋   | 12563/18627 [7:00:16<3:49:59,  2.28s/it]
 67%|██████▋   | 12564/18627 [7:00:19<3:50:48,  2.28s/it]
 67%|██████▋   | 12565/18627 [7:00:21<3:51:32,  2.29s/it]
 67%|██████▋   | 12566/18627 [7:00:23<3:51:57,  2.30s/it]
 67%|██████▋   | 12567/18627 [7:00:26<3:52:46,  2.30s/it]
 67%|██████▋   | 12568/18627 [7:00:28<3:53:08,  2.31s/it]
 67%|██████▋   | 12569/18627 [7:00:30<3:53:27,  2.31s/it]
 67%|██████▋   | 12570/18627 [7:00:33<3:53:32,  2.31s/it]
                                                         
{'loss': 0.606, 'grad_norm': 6.948901653289795, 'learning_rate': 1.2629470732971497e-06, 'epoch': 0.67}
+
 67%|██████▋   | 12570/18627 [7:00:33<3:53:32,  2.31s/it]
 67%|██████▋   | 12571/18627 [7:00:35<3:55:25,  2.33s/it]
 67%|██████▋   | 12572/18627 [7:00:37<3:55:58,  2.34s/it]
 67%|██████▋   | 12573/18627 [7:00:40<3:57:33,  2.35s/it]
 68%|██████▊   | 12574/18627 [7:00:42<3:56:29,  2.34s/it]
 68%|██████▊   | 12575/18627 [7:00:44<3:55:56,  2.34s/it]
 68%|██████▊   | 12576/18627 [7:00:47<3:55:26,  2.33s/it]
 68%|██████▊   | 12577/18627 [7:00:49<3:55:18,  2.33s/it]
 68%|██████▊   | 12578/18627 [7:00:51<3:54:26,  2.33s/it]
 68%|██████▊   | 12579/18627 [7:00:54<3:55:04,  2.33s/it]
 68%|██████▊   | 12580/18627 [7:00:55<3:25:54,  2.04s/it]
                                                         
{'loss': 0.7826, 'grad_norm': 14.305567741394043, 'learning_rate': 1.259171510265929e-06, 'epoch': 0.68}
+
 68%|██████▊   | 12580/18627 [7:00:55<3:25:54,  2.04s/it]
 68%|██████▊   | 12581/18627 [7:00:57<3:33:41,  2.12s/it]
 68%|██████▊   | 12582/18627 [7:01:00<3:41:19,  2.20s/it]
 68%|██████▊   | 12583/18627 [7:01:02<3:44:42,  2.23s/it]
 68%|██████▊   | 12584/18627 [7:01:03<3:18:17,  1.97s/it]
 68%|██████▊   | 12585/18627 [7:01:05<2:59:35,  1.78s/it]
 68%|██████▊   | 12586/18627 [7:01:07<3:16:17,  1.95s/it]
 68%|██████▊   | 12587/18627 [7:01:08<2:57:55,  1.77s/it]
 68%|██████▊   | 12588/18627 [7:01:11<3:14:09,  1.93s/it]
 68%|██████▊   | 12589/18627 [7:01:13<3:25:53,  2.05s/it]
 68%|██████▊   | 12590/18627 [7:01:15<3:33:20,  2.12s/it]
                                                         
{'loss': 1.245, 'grad_norm': 6.2015700340271, 'learning_rate': 1.2553996986161776e-06, 'epoch': 0.68}
+
 68%|██████▊   | 12590/18627 [7:01:15<3:33:20,  2.12s/it]
 68%|██████▊   | 12591/18627 [7:01:17<3:10:29,  1.89s/it]
 68%|██████▊   | 12592/18627 [7:01:19<3:22:42,  2.02s/it]
 68%|██████▊   | 12593/18627 [7:01:20<3:02:25,  1.81s/it]
 68%|██████▊   | 12594/18627 [7:01:23<3:17:39,  1.97s/it]
 68%|██████▊   | 12595/18627 [7:01:25<3:28:12,  2.07s/it]
 68%|██████▊   | 12596/18627 [7:01:27<3:35:17,  2.14s/it]
 68%|██████▊   | 12597/18627 [7:01:30<3:39:58,  2.19s/it]
 68%|██████▊   | 12598/18627 [7:01:32<3:43:10,  2.22s/it]
 68%|██████▊   | 12599/18627 [7:01:34<3:45:43,  2.25s/it]
 68%|██████▊   | 12600/18627 [7:01:37<3:47:47,  2.27s/it]
                                                         
{'loss': 1.0458, 'grad_norm': 5.527857780456543, 'learning_rate': 1.2516316497511648e-06, 'epoch': 0.68}
+
 68%|██████▊   | 12600/18627 [7:01:37<3:47:47,  2.27s/it]
 68%|██████▊   | 12601/18627 [7:01:39<3:50:20,  2.29s/it]
 68%|██████▊   | 12602/18627 [7:01:41<3:51:43,  2.31s/it]
 68%|██████▊   | 12603/18627 [7:01:44<3:51:11,  2.30s/it]
 68%|██████▊   | 12604/18627 [7:01:45<3:21:50,  2.01s/it]
 68%|██████▊   | 12605/18627 [7:01:46<3:02:15,  1.82s/it]
 68%|██████▊   | 12606/18627 [7:01:49<3:17:10,  1.96s/it]
 68%|██████▊   | 12607/18627 [7:01:51<3:27:23,  2.07s/it]
 68%|██████▊   | 12608/18627 [7:01:53<3:34:55,  2.14s/it]
 68%|██████▊   | 12609/18627 [7:01:55<3:10:03,  1.89s/it]
 68%|██████▊   | 12610/18627 [7:01:56<2:54:09,  1.74s/it]
                                                         
{'loss': 1.4125, 'grad_norm': 12.794093132019043, 'learning_rate': 1.2478673750627872e-06, 'epoch': 0.68}
+
 68%|██████▊   | 12610/18627 [7:01:56<2:54:09,  1.74s/it]
 68%|██████▊   | 12611/18627 [7:01:57<2:39:50,  1.59s/it]
 68%|██████▊   | 12612/18627 [7:01:59<3:01:46,  1.81s/it]
 68%|██████▊   | 12613/18627 [7:02:02<3:17:02,  1.97s/it]
 68%|██████▊   | 12614/18627 [7:02:04<3:27:59,  2.08s/it]
 68%|██████▊   | 12615/18627 [7:02:06<3:34:44,  2.14s/it]
 68%|██████▊   | 12616/18627 [7:02:09<3:39:57,  2.20s/it]
 68%|██████▊   | 12617/18627 [7:02:11<3:43:19,  2.23s/it]
 68%|██████▊   | 12618/18627 [7:02:13<3:46:24,  2.26s/it]
 68%|██████▊   | 12619/18627 [7:02:16<3:47:48,  2.28s/it]
 68%|██████▊   | 12620/18627 [7:02:17<3:20:23,  2.00s/it]
                                                         
{'loss': 1.0473, 'grad_norm': 12.989511489868164, 'learning_rate': 1.2441068859315304e-06, 'epoch': 0.68}
+
 68%|██████▊   | 12620/18627 [7:02:17<3:20:23,  2.00s/it]
 68%|██████▊   | 12621/18627 [7:02:18<2:49:05,  1.69s/it]
 68%|██████▊   | 12622/18627 [7:02:20<3:08:34,  1.88s/it]
 68%|██████▊   | 12623/18627 [7:02:23<3:20:17,  2.00s/it]
 68%|██████▊   | 12624/18627 [7:02:24<3:00:21,  1.80s/it]
 68%|██████▊   | 12625/18627 [7:02:25<2:46:54,  1.67s/it]
 68%|██████▊   | 12626/18627 [7:02:28<3:05:21,  1.85s/it]
 68%|██████▊   | 12627/18627 [7:02:30<3:22:17,  2.02s/it]
 68%|██████▊   | 12628/18627 [7:02:32<3:30:12,  2.10s/it]
 68%|██████▊   | 12629/18627 [7:02:35<3:36:32,  2.17s/it]
 68%|██████▊   | 12630/18627 [7:02:36<3:10:46,  1.91s/it]
                                                         
{'loss': 1.155, 'grad_norm': 12.967605590820312, 'learning_rate': 1.2403501937264337e-06, 'epoch': 0.68}
+
 68%|██████▊   | 12630/18627 [7:02:36<3:10:46,  1.91s/it]
 68%|██████▊   | 12631/18627 [7:02:38<3:22:51,  2.03s/it]
 68%|██████▊   | 12632/18627 [7:02:40<3:02:33,  1.83s/it]
 68%|██████▊   | 12633/18627 [7:02:42<3:17:35,  1.98s/it]
 68%|██████▊   | 12634/18627 [7:02:44<3:28:32,  2.09s/it]
 68%|██████▊   | 12635/18627 [7:02:47<3:35:36,  2.16s/it]
 68%|██████▊   | 12636/18627 [7:02:49<3:41:47,  2.22s/it]
 68%|██████▊   | 12637/18627 [7:02:50<3:16:54,  1.97s/it]
 68%|██████▊   | 12638/18627 [7:02:53<3:27:26,  2.08s/it]
 68%|██████▊   | 12639/18627 [7:02:55<3:34:52,  2.15s/it]
 68%|██████▊   | 12640/18627 [7:02:57<3:39:46,  2.20s/it]
                                                         
{'loss': 0.959, 'grad_norm': 8.510704040527344, 'learning_rate': 1.2365973098050593e-06, 'epoch': 0.68}
+
 68%|██████▊   | 12640/18627 [7:02:57<3:39:46,  2.20s/it]
 68%|██████▊   | 12641/18627 [7:03:00<3:43:52,  2.24s/it]
 68%|██████▊   | 12642/18627 [7:03:01<3:18:02,  1.99s/it]
 68%|██████▊   | 12643/18627 [7:03:02<2:57:23,  1.78s/it]
 68%|██████▊   | 12644/18627 [7:03:04<2:43:07,  1.64s/it]
 68%|██████▊   | 12645/18627 [7:03:06<3:03:53,  1.84s/it]
 68%|██████▊   | 12646/18627 [7:03:08<3:19:10,  2.00s/it]
 68%|██████▊   | 12647/18627 [7:03:09<2:50:00,  1.71s/it]
 68%|██████▊   | 12648/18627 [7:03:12<3:07:53,  1.89s/it]
 68%|██████▊   | 12649/18627 [7:03:14<3:20:06,  2.01s/it]
 68%|██████▊   | 12650/18627 [7:03:16<3:29:03,  2.10s/it]
                                                         
{'loss': 1.2159, 'grad_norm': 5.085075378417969, 'learning_rate': 1.2328482455134551e-06, 'epoch': 0.68}
+
 68%|██████▊   | 12650/18627 [7:03:16<3:29:03,  2.10s/it]
 68%|██████▊   | 12651/18627 [7:03:17<2:56:50,  1.78s/it]
 68%|██████▊   | 12652/18627 [7:03:20<3:12:48,  1.94s/it]
 68%|██████▊   | 12653/18627 [7:03:22<3:24:08,  2.05s/it]
 68%|██████▊   | 12654/18627 [7:03:24<3:31:55,  2.13s/it]
 68%|██████▊   | 12655/18627 [7:03:27<3:37:26,  2.18s/it]
 68%|██████▊   | 12656/18627 [7:03:29<3:40:41,  2.22s/it]
 68%|██████▊   | 12657/18627 [7:03:30<3:14:38,  1.96s/it]
 68%|██████▊   | 12658/18627 [7:03:33<3:26:01,  2.07s/it]
 68%|██████▊   | 12659/18627 [7:03:34<3:01:20,  1.82s/it]
 68%|██████▊   | 12660/18627 [7:03:36<3:15:30,  1.97s/it]
                                                         
{'loss': 1.088, 'grad_norm': 9.209644317626953, 'learning_rate': 1.229103012186119e-06, 'epoch': 0.68}
+
 68%|██████▊   | 12660/18627 [7:03:36<3:15:30,  1.97s/it]
 68%|██████▊   | 12661/18627 [7:03:38<3:26:44,  2.08s/it]
 68%|██████▊   | 12662/18627 [7:03:40<3:04:04,  1.85s/it]
 68%|██████▊   | 12663/18627 [7:03:42<3:18:13,  1.99s/it]
 68%|██████▊   | 12664/18627 [7:03:44<3:28:08,  2.09s/it]
 68%|██████▊   | 12665/18627 [7:03:47<3:35:15,  2.17s/it]
 68%|██████▊   | 12666/18627 [7:03:49<3:42:02,  2.24s/it]
 68%|██████▊   | 12667/18627 [7:03:51<3:44:13,  2.26s/it]
 68%|██████▊   | 12668/18627 [7:03:53<3:16:05,  1.97s/it]
 68%|██████▊   | 12669/18627 [7:03:55<3:25:47,  2.07s/it]
 68%|██████▊   | 12670/18627 [7:03:57<3:34:40,  2.16s/it]
                                                         
{'loss': 1.0747, 'grad_norm': 5.466617584228516, 'learning_rate': 1.2253616211459715e-06, 'epoch': 0.68}
+
 68%|██████▊   | 12670/18627 [7:03:58<3:34:40,  2.16s/it]
 68%|██████▊   | 12671/18627 [7:04:00<3:44:44,  2.26s/it]
 68%|██████▊   | 12672/18627 [7:04:02<3:46:18,  2.28s/it]
 68%|██████▊   | 12673/18627 [7:04:05<3:47:40,  2.29s/it]
 68%|██████▊   | 12674/18627 [7:04:07<3:48:57,  2.31s/it]
 68%|██████▊   | 12675/18627 [7:04:09<3:49:14,  2.31s/it]
 68%|██████▊   | 12676/18627 [7:04:11<3:21:01,  2.03s/it]
 68%|██████▊   | 12677/18627 [7:04:13<3:29:44,  2.11s/it]
 68%|██████▊   | 12678/18627 [7:04:15<3:37:16,  2.19s/it]
 68%|██████▊   | 12679/18627 [7:04:18<3:40:37,  2.23s/it]
 68%|██████▊   | 12680/18627 [7:04:20<3:43:41,  2.26s/it]
                                                         
{'loss': 0.862, 'grad_norm': 7.370673656463623, 'learning_rate': 1.2216240837043116e-06, 'epoch': 0.68}
+
 68%|██████▊   | 12680/18627 [7:04:20<3:43:41,  2.26s/it]
 68%|██████▊   | 12681/18627 [7:04:22<3:44:50,  2.27s/it]
 68%|██████▊   | 12682/18627 [7:04:25<3:46:12,  2.28s/it]
 68%|██████▊   | 12683/18627 [7:04:27<3:47:23,  2.30s/it]
 68%|██████▊   | 12684/18627 [7:04:29<3:47:11,  2.29s/it]
 68%|██████▊   | 12685/18627 [7:04:31<3:48:17,  2.31s/it]
 68%|██████▊   | 12686/18627 [7:04:33<3:20:05,  2.02s/it]
 68%|██████▊   | 12687/18627 [7:04:35<3:28:16,  2.10s/it]
 68%|██████▊   | 12688/18627 [7:04:37<3:35:07,  2.17s/it]
 68%|██████▊   | 12689/18627 [7:04:40<3:40:22,  2.23s/it]
 68%|██████▊   | 12690/18627 [7:04:42<3:42:38,  2.25s/it]
                                                         
{'loss': 0.7911, 'grad_norm': 5.093696117401123, 'learning_rate': 1.217890411160791e-06, 'epoch': 0.68}
+
 68%|██████▊   | 12690/18627 [7:04:42<3:42:38,  2.25s/it]
 68%|██████▊   | 12691/18627 [7:04:44<3:44:28,  2.27s/it]
 68%|██████▊   | 12692/18627 [7:04:46<3:18:21,  2.01s/it]
 68%|██████▊   | 12693/18627 [7:04:48<3:27:42,  2.10s/it]
 68%|██████▊   | 12694/18627 [7:04:50<3:34:51,  2.17s/it]
 68%|██████▊   | 12695/18627 [7:04:53<3:39:29,  2.22s/it]
 68%|██████▊   | 12696/18627 [7:04:55<3:42:04,  2.25s/it]
 68%|██████▊   | 12697/18627 [7:04:57<3:44:24,  2.27s/it]
 68%|██████▊   | 12698/18627 [7:05:00<3:48:33,  2.31s/it]
 68%|██████▊   | 12699/18627 [7:05:02<3:48:43,  2.32s/it]
 68%|██████▊   | 12700/18627 [7:05:04<3:20:48,  2.03s/it]
                                                         
{'loss': 0.9875, 'grad_norm': 13.383155822753906, 'learning_rate': 1.2141606148033763e-06, 'epoch': 0.68}
+
 68%|██████▊   | 12700/18627 [7:05:04<3:20:48,  2.03s/it]
 68%|██████▊   | 12701/18627 [7:05:06<3:29:41,  2.12s/it]
 68%|██████▊   | 12702/18627 [7:05:07<3:06:56,  1.89s/it]
 68%|██████▊   | 12703/18627 [7:05:10<3:19:45,  2.02s/it]
 68%|██████▊   | 12704/18627 [7:05:12<3:28:11,  2.11s/it]
 68%|██████▊   | 12705/18627 [7:05:14<3:34:10,  2.17s/it]
 68%|██████▊   | 12706/18627 [7:05:16<3:10:06,  1.93s/it]
 68%|██████▊   | 12707/18627 [7:05:18<3:20:00,  2.03s/it]
 68%|██████▊   | 12708/18627 [7:05:20<3:28:52,  2.12s/it]
 68%|██████▊   | 12709/18627 [7:05:22<3:10:12,  1.93s/it]
 68%|██████▊   | 12710/18627 [7:05:24<3:21:37,  2.04s/it]
                                                         
{'loss': 1.1963, 'grad_norm': 7.08881139755249, 'learning_rate': 1.2104347059083144e-06, 'epoch': 0.68}
+
 68%|██████▊   | 12710/18627 [7:05:24<3:21:37,  2.04s/it]
 68%|██████▊   | 12711/18627 [7:05:26<3:31:06,  2.14s/it]
 68%|██████▊   | 12712/18627 [7:05:27<2:58:26,  1.81s/it]
 68%|██████▊   | 12713/18627 [7:05:30<3:13:38,  1.96s/it]
 68%|██████▊   | 12714/18627 [7:05:32<3:24:12,  2.07s/it]
 68%|██████▊   | 12715/18627 [7:05:34<3:31:34,  2.15s/it]
 68%|██████▊   | 12716/18627 [7:05:37<3:36:01,  2.19s/it]
 68%|██████▊   | 12717/18627 [7:05:39<3:40:02,  2.23s/it]
 68%|██████▊   | 12718/18627 [7:05:41<3:42:10,  2.26s/it]
 68%|██████▊   | 12719/18627 [7:05:42<3:07:49,  1.91s/it]
 68%|██████▊   | 12720/18627 [7:05:44<2:48:20,  1.71s/it]
                                                         
{'loss': 0.8238, 'grad_norm': 13.814618110656738, 'learning_rate': 1.2067126957400995e-06, 'epoch': 0.68}
+
 68%|██████▊   | 12720/18627 [7:05:44<2:48:20,  1.71s/it]
 68%|██████▊   | 12721/18627 [7:05:45<2:38:50,  1.61s/it]
 68%|██████▊   | 12722/18627 [7:05:47<3:00:07,  1.83s/it]
 68%|██████▊   | 12723/18627 [7:05:50<3:14:57,  1.98s/it]
 68%|██████▊   | 12724/18627 [7:05:52<3:25:26,  2.09s/it]
 68%|██████▊   | 12725/18627 [7:05:54<3:32:53,  2.16s/it]
 68%|██████▊   | 12726/18627 [7:05:57<3:37:12,  2.21s/it]
 68%|██████▊   | 12727/18627 [7:05:58<3:11:46,  1.95s/it]
 68%|██████▊   | 12728/18627 [7:06:00<3:23:23,  2.07s/it]
 68%|██████▊   | 12729/18627 [7:06:02<3:03:07,  1.86s/it]
 68%|██████▊   | 12730/18627 [7:06:04<3:15:36,  1.99s/it]
                                                         
{'loss': 1.1849, 'grad_norm': 5.581386566162109, 'learning_rate': 1.2029945955514404e-06, 'epoch': 0.68}
+
 68%|██████▊   | 12730/18627 [7:06:04<3:15:36,  1.99s/it]
 68%|██████▊   | 12731/18627 [7:06:06<3:25:17,  2.09s/it]
 68%|██████▊   | 12732/18627 [7:06:09<3:31:19,  2.15s/it]
 68%|██████▊   | 12733/18627 [7:06:11<3:36:24,  2.20s/it]
 68%|██████▊   | 12734/18627 [7:06:13<3:39:12,  2.23s/it]
 68%|██████▊   | 12735/18627 [7:06:16<3:42:22,  2.26s/it]
 68%|██████▊   | 12736/18627 [7:06:18<3:43:59,  2.28s/it]
 68%|██████▊   | 12737/18627 [7:06:20<3:45:25,  2.30s/it]
 68%|██████▊   | 12738/18627 [7:06:23<3:46:29,  2.31s/it]
 68%|██████▊   | 12739/18627 [7:06:24<3:10:59,  1.95s/it]
 68%|██████▊   | 12740/18627 [7:06:26<3:21:38,  2.06s/it]
                                                         
{'loss': 0.5964, 'grad_norm': 6.0861124992370605, 'learning_rate': 1.1992804165832205e-06, 'epoch': 0.68}
+
 68%|██████▊   | 12740/18627 [7:06:26<3:21:38,  2.06s/it]
 68%|██████▊   | 12741/18627 [7:06:27<3:01:35,  1.85s/it]
 68%|██████▊   | 12742/18627 [7:06:29<2:44:42,  1.68s/it]
 68%|██████▊   | 12743/18627 [7:06:30<2:35:09,  1.58s/it]
 68%|██████▊   | 12744/18627 [7:06:32<2:56:35,  1.80s/it]
 68%|██████▊   | 12745/18627 [7:06:33<2:35:42,  1.59s/it]
 68%|██████▊   | 12746/18627 [7:06:36<2:57:08,  1.81s/it]
 68%|██████▊   | 12747/18627 [7:06:38<3:11:57,  1.96s/it]
 68%|██████▊   | 12748/18627 [7:06:39<2:44:11,  1.68s/it]
 68%|██████▊   | 12749/18627 [7:06:40<2:34:41,  1.58s/it]
 68%|██████▊   | 12750/18627 [7:06:43<2:56:30,  1.80s/it]
                                                         
{'loss': 1.4122, 'grad_norm': 6.717843532562256, 'learning_rate': 1.1955701700644753e-06, 'epoch': 0.68}
+
 68%|██████▊   | 12750/18627 [7:06:43<2:56:30,  1.80s/it]
 68%|██████▊   | 12751/18627 [7:06:45<3:12:27,  1.97s/it]
 68%|██████▊   | 12752/18627 [7:06:47<3:22:16,  2.07s/it]
 68%|██████▊   | 12753/18627 [7:06:50<3:32:56,  2.18s/it]
 68%|██████▊   | 12754/18627 [7:06:52<3:37:06,  2.22s/it]
 68%|██████▊   | 12755/18627 [7:06:53<3:11:35,  1.96s/it]
 68%|██████▊   | 12756/18627 [7:06:56<3:21:57,  2.06s/it]
 68%|██████▊   | 12757/18627 [7:06:58<3:29:03,  2.14s/it]
 68%|██████▊   | 12758/18627 [7:07:00<3:34:52,  2.20s/it]
 68%|██████▊   | 12759/18627 [7:07:02<3:08:05,  1.92s/it]
 69%|██████▊   | 12760/18627 [7:07:03<2:50:51,  1.75s/it]
                                                         
{'loss': 1.2364, 'grad_norm': 15.057908058166504, 'learning_rate': 1.1918638672123432e-06, 'epoch': 0.69}
+
 69%|██████▊   | 12760/18627 [7:07:03<2:50:51,  1.75s/it]
 69%|██████▊   | 12761/18627 [7:07:05<3:07:35,  1.92s/it]
 69%|██████▊   | 12762/18627 [7:07:07<2:52:22,  1.76s/it]
 69%|██████▊   | 12763/18627 [7:07:09<3:08:39,  1.93s/it]
 69%|██████▊   | 12764/18627 [7:07:10<2:49:46,  1.74s/it]
 69%|██████▊   | 12765/18627 [7:07:13<3:05:44,  1.90s/it]
 69%|██████▊   | 12766/18627 [7:07:15<3:18:42,  2.03s/it]
 69%|██████▊   | 12767/18627 [7:07:17<3:29:00,  2.14s/it]
 69%|██████▊   | 12768/18627 [7:07:20<3:41:08,  2.26s/it]
 69%|██████▊   | 12769/18627 [7:07:21<3:05:48,  1.90s/it]
 69%|██████▊   | 12770/18627 [7:07:22<2:49:57,  1.74s/it]
                                                         
{'loss': 1.2568, 'grad_norm': 14.847865104675293, 'learning_rate': 1.1881615192320478e-06, 'epoch': 0.69}
+
 69%|██████▊   | 12770/18627 [7:07:23<2:49:57,  1.74s/it]
 69%|██████▊   | 12771/18627 [7:07:25<3:10:42,  1.95s/it]
 69%|██████▊   | 12772/18627 [7:07:27<3:22:11,  2.07s/it]
 69%|██████▊   | 12773/18627 [7:07:29<3:01:35,  1.86s/it]
 69%|██████▊   | 12774/18627 [7:07:31<3:15:44,  2.01s/it]
 69%|██████▊   | 12775/18627 [7:07:32<2:57:16,  1.82s/it]
 69%|██████▊   | 12776/18627 [7:07:34<2:44:23,  1.69s/it]
 69%|██████▊   | 12777/18627 [7:07:36<3:02:38,  1.87s/it]
 69%|██████▊   | 12778/18627 [7:07:38<3:16:00,  2.01s/it]
 69%|██████▊   | 12779/18627 [7:07:40<2:56:16,  1.81s/it]
 69%|██████▊   | 12780/18627 [7:07:42<3:11:03,  1.96s/it]
                                                         
{'loss': 1.3847, 'grad_norm': 7.931560039520264, 'learning_rate': 1.1844631373168492e-06, 'epoch': 0.69}
+
 69%|██████▊   | 12780/18627 [7:07:42<3:11:03,  1.96s/it]
 69%|██████▊   | 12781/18627 [7:07:43<2:53:03,  1.78s/it]
 69%|██████▊   | 12782/18627 [7:07:46<3:09:08,  1.94s/it]
 69%|██████▊   | 12783/18627 [7:07:48<3:17:55,  2.03s/it]
 69%|██████▊   | 12784/18627 [7:07:50<3:25:34,  2.11s/it]
 69%|██████▊   | 12785/18627 [7:07:52<3:31:56,  2.18s/it]
 69%|██████▊   | 12786/18627 [7:07:55<3:35:49,  2.22s/it]
 69%|██████▊   | 12787/18627 [7:07:57<3:39:35,  2.26s/it]
 69%|██████▊   | 12788/18627 [7:07:59<3:41:27,  2.28s/it]
 69%|██████▊   | 12789/18627 [7:08:02<3:45:01,  2.31s/it]
 69%|██████▊   | 12790/18627 [7:08:04<3:45:34,  2.32s/it]
                                                         
{'loss': 0.7664, 'grad_norm': 6.195589542388916, 'learning_rate': 1.1807687326480199e-06, 'epoch': 0.69}
+
 69%|██████▊   | 12790/18627 [7:08:04<3:45:34,  2.32s/it]
 69%|██████▊   | 12791/18627 [7:08:06<3:45:33,  2.32s/it]
 69%|██████▊   | 12792/18627 [7:08:09<3:45:57,  2.32s/it]
 69%|██████▊   | 12793/18627 [7:08:11<3:46:01,  2.32s/it]
 69%|██████▊   | 12794/18627 [7:08:13<3:45:12,  2.32s/it]
 69%|██████▊   | 12795/18627 [7:08:16<3:45:32,  2.32s/it]
 69%|██████▊   | 12796/18627 [7:08:18<3:44:35,  2.31s/it]
 69%|██████▊   | 12797/18627 [7:08:20<3:44:36,  2.31s/it]
 69%|██████▊   | 12798/18627 [7:08:22<3:16:10,  2.02s/it]
 69%|██████▊   | 12799/18627 [7:08:24<3:24:38,  2.11s/it]
 69%|██████▊   | 12800/18627 [7:08:26<3:30:59,  2.17s/it]
                                                         
{'loss': 0.7734, 'grad_norm': 9.557731628417969, 'learning_rate': 1.177078316394811e-06, 'epoch': 0.69}
+
 69%|██████▊   | 12800/18627 [7:08:26<3:30:59,  2.17s/it]
 69%|██████▊   | 12801/18627 [7:08:28<3:07:20,  1.93s/it]
 69%|██████▊   | 12802/18627 [7:08:29<2:50:52,  1.76s/it]
 69%|██████▊   | 12803/18627 [7:08:31<3:06:35,  1.92s/it]
 69%|██████▊   | 12804/18627 [7:08:34<3:17:40,  2.04s/it]
 69%|██████▊   | 12805/18627 [7:08:35<2:57:10,  1.83s/it]
 69%|██████▊   | 12806/18627 [7:08:37<3:13:54,  2.00s/it]
 69%|██████▉   | 12807/18627 [7:08:40<3:24:30,  2.11s/it]
 69%|██████▉   | 12808/18627 [7:08:42<3:29:14,  2.16s/it]
 69%|██████▉   | 12809/18627 [7:08:43<3:06:53,  1.93s/it]
 69%|██████▉   | 12810/18627 [7:08:46<3:17:43,  2.04s/it]
                                                         
{'loss': 1.5835, 'grad_norm': 5.904951095581055, 'learning_rate': 1.1733918997144092e-06, 'epoch': 0.69}
+
 69%|██████▉   | 12810/18627 [7:08:46<3:17:43,  2.04s/it]
 69%|██████▉   | 12811/18627 [7:08:48<3:25:21,  2.12s/it]
 69%|██████▉   | 12812/18627 [7:08:49<2:53:57,  1.79s/it]
 69%|██████▉   | 12813/18627 [7:08:51<3:08:54,  1.95s/it]
 69%|██████▉   | 12814/18627 [7:08:53<2:51:56,  1.77s/it]
 69%|██████▉   | 12815/18627 [7:08:55<3:07:12,  1.93s/it]
 69%|██████▉   | 12816/18627 [7:08:57<3:18:20,  2.05s/it]
 69%|██████▉   | 12817/18627 [7:09:00<3:30:35,  2.17s/it]
 69%|██████▉   | 12818/18627 [7:09:01<3:07:22,  1.94s/it]
 69%|██████▉   | 12819/18627 [7:09:04<3:18:41,  2.05s/it]
 69%|██████▉   | 12820/18627 [7:09:05<2:58:37,  1.85s/it]
                                                         
{'loss': 1.1999, 'grad_norm': 16.28277015686035, 'learning_rate': 1.1697094937519137e-06, 'epoch': 0.69}
+
 69%|██████▉   | 12820/18627 [7:09:05<2:58:37,  1.85s/it]
 69%|██████▉   | 12821/18627 [7:09:06<2:35:59,  1.61s/it]
 69%|██████▉   | 12822/18627 [7:09:08<2:56:17,  1.82s/it]
 69%|██████▉   | 12823/18627 [7:09:11<3:09:42,  1.96s/it]
 69%|██████▉   | 12824/18627 [7:09:13<3:20:54,  2.08s/it]
 69%|██████▉   | 12825/18627 [7:09:15<3:28:21,  2.15s/it]
 69%|██████▉   | 12826/18627 [7:09:18<3:32:47,  2.20s/it]
 69%|██████▉   | 12827/18627 [7:09:20<3:35:31,  2.23s/it]
 69%|██████▉   | 12828/18627 [7:09:21<3:10:50,  1.97s/it]
 69%|██████▉   | 12829/18627 [7:09:24<3:20:45,  2.08s/it]
 69%|██████▉   | 12830/18627 [7:09:26<3:27:31,  2.15s/it]
                                                         
{'loss': 0.8195, 'grad_norm': 7.681834697723389, 'learning_rate': 1.166031109640297e-06, 'epoch': 0.69}
+
 69%|██████▉   | 12830/18627 [7:09:26<3:27:31,  2.15s/it]
 69%|██████▉   | 12831/18627 [7:09:28<3:32:09,  2.20s/it]
 69%|██████▉   | 12832/18627 [7:09:30<3:34:36,  2.22s/it]
 69%|██████▉   | 12833/18627 [7:09:33<3:35:33,  2.23s/it]
 69%|██████▉   | 12834/18627 [7:09:35<3:37:01,  2.25s/it]
 69%|██████▉   | 12835/18627 [7:09:37<3:39:11,  2.27s/it]
 69%|██████▉   | 12836/18627 [7:09:39<3:11:45,  1.99s/it]
 69%|██████▉   | 12837/18627 [7:09:41<3:20:53,  2.08s/it]
 69%|██████▉   | 12838/18627 [7:09:43<3:27:56,  2.16s/it]
 69%|██████▉   | 12839/18627 [7:09:46<3:32:59,  2.21s/it]
 69%|██████▉   | 12840/18627 [7:09:48<3:34:57,  2.23s/it]
                                                         
{'loss': 0.8131, 'grad_norm': 5.995288372039795, 'learning_rate': 1.1623567585003726e-06, 'epoch': 0.69}
+
 69%|██████▉   | 12840/18627 [7:09:48<3:34:57,  2.23s/it]
 69%|██████▉   | 12841/18627 [7:09:50<3:37:07,  2.25s/it]
 69%|██████▉   | 12842/18627 [7:09:51<3:01:00,  1.88s/it]
 69%|██████▉   | 12843/18627 [7:09:54<3:14:04,  2.01s/it]
 69%|██████▉   | 12844/18627 [7:09:56<3:22:23,  2.10s/it]
 69%|██████▉   | 12845/18627 [7:09:58<3:28:27,  2.16s/it]
 69%|██████▉   | 12846/18627 [7:10:00<3:32:26,  2.20s/it]
 69%|██████▉   | 12847/18627 [7:10:03<3:35:17,  2.23s/it]
 69%|██████▉   | 12848/18627 [7:10:05<3:37:11,  2.25s/it]
 69%|██████▉   | 12849/18627 [7:10:07<3:39:23,  2.28s/it]
 69%|██████▉   | 12850/18627 [7:10:10<3:46:31,  2.35s/it]
                                                         
{'loss': 0.6382, 'grad_norm': 5.842278003692627, 'learning_rate': 1.158686451440761e-06, 'epoch': 0.69}
+
 69%|██████▉   | 12850/18627 [7:10:10<3:46:31,  2.35s/it]
 69%|██████▉   | 12851/18627 [7:10:11<3:19:03,  2.07s/it]
 69%|██████▉   | 12852/18627 [7:10:12<2:49:09,  1.76s/it]
 69%|██████▉   | 12853/18627 [7:10:14<2:41:54,  1.68s/it]
 69%|██████▉   | 12854/18627 [7:10:16<3:01:49,  1.89s/it]
 69%|██████▉   | 12855/18627 [7:10:19<3:13:24,  2.01s/it]
 69%|██████▉   | 12856/18627 [7:10:20<2:51:10,  1.78s/it]
 69%|██████▉   | 12857/18627 [7:10:22<3:06:31,  1.94s/it]
 69%|████���█▉   | 12858/18627 [7:10:24<3:17:55,  2.06s/it]
 69%|██████▉   | 12859/18627 [7:10:27<3:25:31,  2.14s/it]
 69%|██████▉   | 12860/18627 [7:10:28<3:02:41,  1.90s/it]
                                                         
{'loss': 1.4128, 'grad_norm': 16.892210006713867, 'learning_rate': 1.1550201995578566e-06, 'epoch': 0.69}
+
 69%|██████▉   | 12860/18627 [7:10:28<3:02:41,  1.90s/it]
 69%|██████▉   | 12861/18627 [7:10:30<3:14:15,  2.02s/it]
 69%|██████▉   | 12862/18627 [7:10:33<3:22:26,  2.11s/it]
 69%|██████▉   | 12863/18627 [7:10:35<3:28:16,  2.17s/it]
 69%|██████▉   | 12864/18627 [7:10:37<3:32:32,  2.21s/it]
 69%|██████▉   | 12865/18627 [7:10:40<3:35:51,  2.25s/it]
 69%|██████▉   | 12866/18627 [7:10:42<3:36:46,  2.26s/it]
 69%|██████▉   | 12867/18627 [7:10:43<3:10:20,  1.98s/it]
 69%|██████▉   | 12868/18627 [7:10:46<3:20:26,  2.09s/it]
 69%|██████▉   | 12869/18627 [7:10:48<3:26:59,  2.16s/it]
 69%|██████▉   | 12870/18627 [7:10:50<3:31:59,  2.21s/it]
                                                         
{'loss': 0.9018, 'grad_norm': 6.019161701202393, 'learning_rate': 1.1513580139357914e-06, 'epoch': 0.69}
+
 69%|██████▉   | 12870/18627 [7:10:50<3:31:59,  2.21s/it]
 69%|██████▉   | 12871/18627 [7:10:53<3:34:17,  2.23s/it]
 69%|██████▉   | 12872/18627 [7:10:55<3:36:23,  2.26s/it]
 69%|██████▉   | 12873/18627 [7:10:56<3:00:22,  1.88s/it]
 69%|██████▉   | 12874/18627 [7:10:57<2:44:34,  1.72s/it]
 69%|██████▉   | 12875/18627 [7:11:00<3:01:42,  1.90s/it]
 69%|██████▉   | 12876/18627 [7:11:01<2:45:03,  1.72s/it]
 69%|██████▉   | 12877/18627 [7:11:02<2:34:14,  1.61s/it]
 69%|██████▉   | 12878/18627 [7:11:04<2:53:44,  1.81s/it]
 69%|██████▉   | 12879/18627 [7:11:07<3:08:05,  1.96s/it]
 69%|██████▉   | 12880/18627 [7:11:09<3:17:35,  2.06s/it]
                                                         
{'loss': 1.2916, 'grad_norm': 5.544093608856201, 'learning_rate': 1.1476999056464094e-06, 'epoch': 0.69}
+
 69%|██████▉   | 12880/18627 [7:11:09<3:17:35,  2.06s/it]
 69%|██████▉   | 12881/18627 [7:11:10<2:48:56,  1.76s/it]
 69%|██████▉   | 12882/18627 [7:11:12<3:04:51,  1.93s/it]
 69%|██████▉   | 12883/18627 [7:11:14<2:49:03,  1.77s/it]
 69%|██████▉   | 12884/18627 [7:11:16<3:06:21,  1.95s/it]
 69%|██████▉   | 12885/18627 [7:11:18<2:49:44,  1.77s/it]
 69%|██████▉   | 12886/18627 [7:11:20<3:06:42,  1.95s/it]
 69%|██████▉   | 12887/18627 [7:11:22<3:16:39,  2.06s/it]
 69%|██████▉   | 12888/18627 [7:11:25<3:23:20,  2.13s/it]
 69%|██████▉   | 12889/18627 [7:11:26<3:00:51,  1.89s/it]
 69%|██████▉   | 12890/18627 [7:11:28<3:14:18,  2.03s/it]
                                                         
{'loss': 1.2564, 'grad_norm': 5.346205711364746, 'learning_rate': 1.1440458857492217e-06, 'epoch': 0.69}
+
 69%|██████▉   | 12890/18627 [7:11:28<3:14:18,  2.03s/it]
 69%|██████▉   | 12891/18627 [7:11:31<3:22:34,  2.12s/it]
 69%|██████▉   | 12892/18627 [7:11:32<2:59:30,  1.88s/it]
 69%|██████▉   | 12893/18627 [7:11:34<3:11:48,  2.01s/it]
 69%|██████▉   | 12894/18627 [7:11:37<3:20:21,  2.10s/it]
 69%|██████▉   | 12895/18627 [7:11:39<3:26:55,  2.17s/it]
 69%|██████▉   | 12896/18627 [7:11:41<3:30:25,  2.20s/it]
 69%|██████▉   | 12897/18627 [7:11:43<3:32:43,  2.23s/it]
 69%|██████▉   | 12898/18627 [7:11:45<3:07:30,  1.96s/it]
 69%|██████▉   | 12899/18627 [7:11:47<3:17:38,  2.07s/it]
 69%|██████▉   | 12900/18627 [7:11:49<3:25:47,  2.16s/it]
                                                         
{'loss': 0.9416, 'grad_norm': 6.091616630554199, 'learning_rate': 1.140395965291382e-06, 'epoch': 0.69}
+
 69%|██████▉   | 12900/18627 [7:11:49<3:25:47,  2.16s/it]
 69%|██████▉   | 12901/18627 [7:11:52<3:30:30,  2.21s/it]
 69%|██████▉   | 12902/18627 [7:11:54<3:34:00,  2.24s/it]
 69%|██████▉   | 12903/18627 [7:11:55<3:08:33,  1.98s/it]
 69%|██████▉   | 12904/18627 [7:11:58<3:18:25,  2.08s/it]
 69%|██████▉   | 12905/18627 [7:12:00<3:25:01,  2.15s/it]
 69%|██████▉   | 12906/18627 [7:12:01<3:02:55,  1.92s/it]
 69%|██████▉   | 12907/18627 [7:12:04<3:13:49,  2.03s/it]
 69%|██████▉   | 12908/18627 [7:12:06<3:21:08,  2.11s/it]
 69%|██████▉   | 12909/18627 [7:12:07<2:59:27,  1.88s/it]
 69%|██████▉   | 12910/18627 [7:12:10<3:14:27,  2.04s/it]
                                                         
{'loss': 1.2828, 'grad_norm': 11.857626914978027, 'learning_rate': 1.1367501553076506e-06, 'epoch': 0.69}
+
 69%|██████▉   | 12910/18627 [7:12:10<3:14:27,  2.04s/it]
 69%|██████▉   | 12911/18627 [7:12:11<2:55:25,  1.84s/it]
 69%|██████▉   | 12912/18627 [7:12:13<3:08:55,  1.98s/it]
 69%|██████▉   | 12913/18627 [7:12:15<2:50:52,  1.79s/it]
 69%|██████▉   | 12914/18627 [7:12:17<3:06:20,  1.96s/it]
 69%|██████▉   | 12915/18627 [7:12:20<3:17:58,  2.08s/it]
 69%|██████▉   | 12916/18627 [7:12:22<3:24:07,  2.14s/it]
 69%|██████▉   | 12917/18627 [7:12:23<3:01:20,  1.91s/it]
 69%|██████▉   | 12918/18627 [7:12:26<3:13:00,  2.03s/it]
 69%|██████▉   | 12919/18627 [7:12:27<2:44:11,  1.73s/it]
 69%|██████▉   | 12920/18627 [7:12:29<3:01:35,  1.91s/it]
                                                         
{'loss': 1.1319, 'grad_norm': 6.510515213012695, 'learning_rate': 1.1331084668203593e-06, 'epoch': 0.69}
+
 69%|██████▉   | 12920/18627 [7:12:29<3:01:35,  1.91s/it]
 69%|██████▉   | 12921/18627 [7:12:30<2:46:28,  1.75s/it]
 69%|██████▉   | 12922/18627 [7:12:33<3:01:59,  1.91s/it]
 69%|██████▉   | 12923/18627 [7:12:35<3:14:36,  2.05s/it]
 69%|██████▉   | 12924/18627 [7:12:37<3:22:29,  2.13s/it]
 69%|██████▉   | 12925/18627 [7:12:40<3:27:12,  2.18s/it]
 69%|██████▉   | 12926/18627 [7:12:42<3:31:00,  2.22s/it]
 69%|██████▉   | 12927/18627 [7:12:44<3:32:51,  2.24s/it]
 69%|██████▉   | 12928/18627 [7:12:45<3:07:25,  1.97s/it]
 69%|██████▉   | 12929/18627 [7:12:48<3:18:02,  2.09s/it]
 69%|██████▉   | 12930/18627 [7:12:50<3:24:35,  2.15s/it]
                                                         
{'loss': 1.0008, 'grad_norm': 10.920360565185547, 'learning_rate': 1.12947091083938e-06, 'epoch': 0.69}
+
 69%|██████▉   | 12930/18627 [7:12:50<3:24:35,  2.15s/it]
 69%|██████▉   | 12931/18627 [7:12:52<3:30:16,  2.21s/it]
 69%|██████▉   | 12932/18627 [7:12:54<3:05:50,  1.96s/it]
 69%|██████▉   | 12933/18627 [7:12:56<3:15:58,  2.07s/it]
 69%|██████▉   | 12934/18627 [7:12:58<2:55:35,  1.85s/it]
 69%|██████▉   | 12935/18627 [7:12:59<2:42:54,  1.72s/it]
 69%|██████▉   | 12936/18627 [7:13:01<3:00:25,  1.90s/it]
 69%|██████▉   | 12937/18627 [7:13:04<3:11:47,  2.02s/it]
 69%|██████▉   | 12938/18627 [7:13:06<3:20:27,  2.11s/it]
 69%|██████▉   | 12939/18627 [7:13:08<3:26:09,  2.17s/it]
 69%|██████▉   | 12940/18627 [7:13:11<3:29:54,  2.21s/it]
                                                         
{'loss': 1.2193, 'grad_norm': 7.122239589691162, 'learning_rate': 1.1258374983620917e-06, 'epoch': 0.69}
+
 69%|██████▉   | 12940/18627 [7:13:11<3:29:54,  2.21s/it]
 69%|██████▉   | 12941/18627 [7:13:13<3:32:32,  2.24s/it]
 69%|██████▉   | 12942/18627 [7:13:15<3:34:39,  2.27s/it]
 69%|██████▉   | 12943/18627 [7:13:17<3:36:30,  2.29s/it]
 69%|██████▉   | 12944/18627 [7:13:20<3:37:22,  2.30s/it]
 69%|██████▉   | 12945/18627 [7:13:22<3:38:12,  2.30s/it]
 70%|██████▉   | 12946/18627 [7:13:24<3:38:02,  2.30s/it]
 70%|██████▉   | 12947/18627 [7:13:26<3:10:46,  2.02s/it]
 70%|██████▉   | 12948/18627 [7:13:28<3:19:56,  2.11s/it]
 70%|██████▉   | 12949/18627 [7:13:29<2:58:04,  1.88s/it]
 70%|██████▉   | 12950/18627 [7:13:32<3:10:04,  2.01s/it]
                                                         
{'loss': 0.9559, 'grad_norm': 5.291947841644287, 'learning_rate': 1.1222082403733434e-06, 'epoch': 0.7}
+
 70%|██████▉   | 12950/18627 [7:13:32<3:10:04,  2.01s/it]
 70%|██████▉   | 12951/18627 [7:13:34<3:18:00,  2.09s/it]
 70%|██████▉   | 12952/18627 [7:13:36<3:24:50,  2.17s/it]
 70%|██████▉   | 12953/18627 [7:13:39<3:28:47,  2.21s/it]
 70%|██████▉   | 12954/18627 [7:13:41<3:31:40,  2.24s/it]
 70%|██████▉   | 12955/18627 [7:13:43<3:33:31,  2.26s/it]
 70%|██████▉   | 12956/18627 [7:13:45<3:08:12,  1.99s/it]
 70%|██████▉   | 12957/18627 [7:13:47<3:17:56,  2.09s/it]
 70%|██████▉   | 12958/18627 [7:13:49<3:23:33,  2.15s/it]
 70%|██████▉   | 12959/18627 [7:13:52<3:28:07,  2.20s/it]
 70%|██████▉   | 12960/18627 [7:13:54<3:30:56,  2.23s/it]
                                                         
{'loss': 0.8432, 'grad_norm': 7.448797702789307, 'learning_rate': 1.1185831478454288e-06, 'epoch': 0.7}
+
 70%|██████▉   | 12960/18627 [7:13:54<3:30:56,  2.23s/it]
 70%|██████▉   | 12961/18627 [7:13:56<3:33:08,  2.26s/it]
 70%|██████▉   | 12962/18627 [7:13:59<3:34:23,  2.27s/it]
 70%|██████▉   | 12963/18627 [7:14:01<3:35:46,  2.29s/it]
 70%|█████���▉   | 12964/18627 [7:14:03<3:36:29,  2.29s/it]
 70%|██████▉   | 12965/18627 [7:14:04<3:09:23,  2.01s/it]
 70%|██████▉   | 12966/18627 [7:14:06<2:51:53,  1.82s/it]
 70%|██████▉   | 12967/18627 [7:14:08<3:03:21,  1.94s/it]
 70%|██████▉   | 12968/18627 [7:14:09<2:45:38,  1.76s/it]
 70%|██████▉   | 12969/18627 [7:14:12<3:01:01,  1.92s/it]
 70%|██████▉   | 12970/18627 [7:14:14<3:11:50,  2.03s/it]
                                                         
{'loss': 1.2043, 'grad_norm': 6.0150017738342285, 'learning_rate': 1.1149622317380431e-06, 'epoch': 0.7}
+
 70%|██████▉   | 12970/18627 [7:14:14<3:11:50,  2.03s/it]
 70%|██████▉   | 12971/18627 [7:14:16<3:20:10,  2.12s/it]
 70%|██████▉   | 12972/18627 [7:14:19<3:26:01,  2.19s/it]
 70%|██████▉   | 12973/18627 [7:14:21<3:29:45,  2.23s/it]
 70%|██████▉   | 12974/18627 [7:14:22<3:05:14,  1.97s/it]
 70%|██████▉   | 12975/18627 [7:14:24<2:48:39,  1.79s/it]
 70%|██████▉   | 12976/18627 [7:14:26<3:04:04,  1.95s/it]
 70%|██████▉   | 12977/18627 [7:14:28<3:13:43,  2.06s/it]
 70%|██████▉   | 12978/18627 [7:14:31<3:21:42,  2.14s/it]
 70%|██████▉   | 12979/18627 [7:14:33<3:26:35,  2.19s/it]
 70%|██████▉   | 12980/18627 [7:14:35<3:29:30,  2.23s/it]
                                                         
{'loss': 1.0074, 'grad_norm': 6.135735988616943, 'learning_rate': 1.1113455029982583e-06, 'epoch': 0.7}
+
 70%|██████▉   | 12980/18627 [7:14:35<3:29:30,  2.23s/it]
 70%|██████▉   | 12981/18627 [7:14:38<3:31:20,  2.25s/it]
 70%|██████▉   | 12982/18627 [7:14:40<3:33:46,  2.27s/it]
 70%|██████▉   | 12983/18627 [7:14:41<3:08:15,  2.00s/it]
 70%|██████▉   | 12984/18627 [7:14:44<3:17:21,  2.10s/it]
 70%|██████▉   | 12985/18627 [7:14:45<2:56:32,  1.88s/it]
 70%|██████▉   | 12986/18627 [7:14:46<2:41:36,  1.72s/it]
 70%|██████▉   | 12987/18627 [7:14:49<3:00:11,  1.92s/it]
 70%|██████▉   | 12988/18627 [7:14:51<3:11:22,  2.04s/it]
 70%|██████▉   | 12989/18627 [7:14:53<3:19:07,  2.12s/it]
 70%|██████▉   | 12990/18627 [7:14:56<3:24:20,  2.18s/it]
                                                         
{'loss': 1.2463, 'grad_norm': 5.808218002319336, 'learning_rate': 1.1077329725604857e-06, 'epoch': 0.7}
+
 70%|██████▉   | 12990/18627 [7:14:56<3:24:20,  2.18s/it]
 70%|██████▉   | 12991/18627 [7:14:58<3:27:39,  2.21s/it]
 70%|██████▉   | 12992/18627 [7:15:00<3:30:55,  2.25s/it]
 70%|██████▉   | 12993/18627 [7:15:03<3:32:56,  2.27s/it]
 70%|██████▉   | 12994/18627 [7:15:05<3:34:55,  2.29s/it]
 70%|██████▉   | 12995/18627 [7:15:06<3:07:49,  2.00s/it]
 70%|██████▉   | 12996/18627 [7:15:09<3:16:31,  2.09s/it]
 70%|██████▉   | 12997/18627 [7:15:10<2:57:49,  1.90s/it]
 70%|██████▉   | 12998/18627 [7:15:12<3:09:41,  2.02s/it]
 70%|██████▉   | 12999/18627 [7:15:15<3:18:25,  2.12s/it]
 70%|██████▉   | 13000/18627 [7:15:17<3:24:49,  2.18s/it]
                                                         
{'loss': 1.0582, 'grad_norm': 10.02350902557373, 'learning_rate': 1.104124651346444e-06, 'epoch': 0.7}
+
 70%|██████▉   | 13000/18627 [7:15:17<3:24:49,  2.18s/it]/home/zhengduo/workspace/LLaMA-Factory/src/qwen_vl/model/modeling_qwen2_5_vl.py:1979: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
+  with torch.cuda.amp.autocast(dtype=dtype):
+/home/zhengduo/miniconda3/envs/vgllm/lib/python3.10/site-packages/torch/utils/checkpoint.py:87: UserWarning: None of the inputs have requires_grad=True. Gradients will be None
+  warnings.warn(
+
 70%|██████▉   | 13001/18627 [7:16:26<34:34:37, 22.13s/it]
 70%|██████▉   | 13002/18627 [7:16:28<25:15:25, 16.16s/it]
 70%|██████▉   | 13003/18627 [7:16:30<18:45:04, 12.00s/it]
 70%|██████▉   | 13004/18627 [7:16:32<13:45:43,  8.81s/it]
 70%|██████▉   | 13005/18627 [7:16:34<10:43:34,  6.87s/it]
 70%|██████▉   | 13006/18627 [7:16:36<8:35:12,  5.50s/it] 
 70%|██████▉   | 13007/18627 [7:16:39<7:05:35,  4.54s/it]
 70%|██████▉   | 13008/18627 [7:16:41<6:02:56,  3.88s/it]
 70%|██████▉   | 13009/18627 [7:16:42<4:49:31,  3.09s/it]
 70%|██████▉   | 13010/18627 [7:16:43<3:59:11,  2.56s/it]
                                                         
{'loss': 1.1559, 'grad_norm': 13.884827613830566, 'learning_rate': 1.100520550265126e-06, 'epoch': 0.7}
+
 70%|██████▉   | 13010/18627 [7:16:43<3:59:11,  2.56s/it]
 70%|██████▉   | 13011/18627 [7:16:45<3:26:29,  2.21s/it]
 70%|██████▉   | 13012/18627 [7:16:46<3:01:49,  1.94s/it]
 70%|██████▉   | 13013/18627 [7:16:48<3:11:18,  2.04s/it]
 70%|██████▉   | 13014/18627 [7:16:51<3:18:00,  2.12s/it]
 70%|██████▉   | 13015/18627 [7:16:53<3:23:55,  2.18s/it]
 70%|██████▉   | 13016/18627 [7:16:54<2:51:57,  1.84s/it]
 70%|██████▉   | 13017/18627 [7:16:56<3:05:32,  1.98s/it]
 70%|██████▉   | 13018/18627 [7:16:59<3:15:05,  2.09s/it]
 70%|██████▉   | 13019/18627 [7:17:00<2:46:38,  1.78s/it]
 70%|██████▉   | 13020/18627 [7:17:02<3:01:59,  1.95s/it]
                                                         
{'loss': 1.0167, 'grad_norm': 5.233762741088867, 'learning_rate': 1.0969206802127678e-06, 'epoch': 0.7}
+
 70%|██████▉   | 13020/18627 [7:17:02<3:01:59,  1.95s/it]
 70%|██████▉   | 13021/18627 [7:17:04<3:12:12,  2.06s/it]
 70%|██████▉   | 13022/18627 [7:17:07<3:19:32,  2.14s/it]
 70%|██████▉   | 13023/18627 [7:17:09<3:24:13,  2.19s/it]
 70%|██████▉   | 13024/18627 [7:17:11<3:27:41,  2.22s/it]
 70%|██████▉   | 13025/18627 [7:17:14<3:30:02,  2.25s/it]
 70%|██████▉   | 13026/18627 [7:17:16<3:31:43,  2.27s/it]
 70%|██████▉   | 13027/18627 [7:17:17<3:01:58,  1.95s/it]
 70%|██████▉   | 13028/18627 [7:17:20<3:12:08,  2.06s/it]
 70%|██████▉   | 13029/18627 [7:17:21<2:50:33,  1.83s/it]
 70%|██████▉   | 13030/18627 [7:17:23<3:04:35,  1.98s/it]
                                                         
{'loss': 0.9959, 'grad_norm': 6.094729423522949, 'learning_rate': 1.093325052072808e-06, 'epoch': 0.7}
+
 70%|██████▉   | 13030/18627 [7:17:23<3:04:35,  1.98s/it]
 70%|██████▉   | 13031/18627 [7:17:24<2:38:20,  1.70s/it]
 70%|██████▉   | 13032/18627 [7:17:27<2:55:37,  1.88s/it]
 70%|██████▉   | 13033/18627 [7:17:28<2:40:50,  1.73s/it]
 70%|██████▉   | 13034/18627 [7:17:29<2:30:34,  1.62s/it]
 70%|██████▉   | 13035/18627 [7:17:32<2:50:22,  1.83s/it]
 70%|██████▉   | 13036/18627 [7:17:34<3:05:24,  1.99s/it]
 70%|██████▉   | 13037/18627 [7:17:35<2:48:10,  1.81s/it]
 70%|██████▉   | 13038/18627 [7:17:38<3:02:52,  1.96s/it]
 70%|███████   | 13039/18627 [7:17:40<3:13:57,  2.08s/it]
 70%|███████   | 13040/18627 [7:17:41<2:54:28,  1.87s/it]
                                                         
{'loss': 1.4729, 'grad_norm': 14.690372467041016, 'learning_rate': 1.0897336767158698e-06, 'epoch': 0.7}
+
 70%|███████   | 13040/18627 [7:17:41<2:54:28,  1.87s/it]
 70%|███████   | 13041/18627 [7:17:44<3:07:37,  2.02s/it]
 70%|███████   | 13042/18627 [7:17:46<3:16:13,  2.11s/it]
 70%|███████   | 13043/18627 [7:17:47<2:55:27,  1.89s/it]
 70%|███████   | 13044/18627 [7:17:50<3:07:34,  2.02s/it]
 70%|███████   | 13045/18627 [7:17:51<2:49:45,  1.82s/it]
 70%|███████   | 13046/18627 [7:17:53<3:02:41,  1.96s/it]
 70%|███████   | 13047/18627 [7:17:56<3:12:35,  2.07s/it]
 70%|███████   | 13048/18627 [7:17:58<3:20:04,  2.15s/it]
 70%|███████   | 13049/18627 [7:17:59<2:59:25,  1.93s/it]
 70%|███████   | 13050/18627 [7:18:02<3:10:44,  2.05s/it]
                                                         
{'loss': 1.2979, 'grad_norm': 5.273060321807861, 'learning_rate': 1.0861465649997104e-06, 'epoch': 0.7}
+
 70%|███████   | 13050/18627 [7:18:02<3:10:44,  2.05s/it]
 70%|███████   | 13051/18627 [7:18:03<2:52:33,  1.86s/it]
 70%|███████   | 13052/18627 [7:18:06<3:07:57,  2.02s/it]
 70%|███████   | 13053/18627 [7:18:08<3:16:03,  2.11s/it]
 70%|███████   | 13054/18627 [7:18:10<3:22:40,  2.18s/it]
 70%|███████   | 13055/18627 [7:18:13<3:26:59,  2.23s/it]
 70%|███████   | 13056/18627 [7:18:15<3:35:50,  2.32s/it]
 70%|███████   | 13057/18627 [7:18:17<3:35:00,  2.32s/it]
 70%|███████   | 13058/18627 [7:18:20<3:35:35,  2.32s/it]
 70%|███████   | 13059/18627 [7:18:22<3:35:03,  2.32s/it]
 70%|███████   | 13060/18627 [7:18:24<3:35:11,  2.32s/it]
                                                         
{'loss': 0.7733, 'grad_norm': 4.609137535095215, 'learning_rate': 1.0825637277692008e-06, 'epoch': 0.7}
+
 70%|███████   | 13060/18627 [7:18:24<3:35:11,  2.32s/it]
 70%|███████   | 13061/18627 [7:18:27<3:35:31,  2.32s/it]
 70%|███████   | 13062/18627 [7:18:29<3:36:07,  2.33s/it]
 70%|███████   | 13063/18627 [7:18:31<3:35:59,  2.33s/it]
 70%|███████   | 13064/18627 [7:18:34<3:35:18,  2.32s/it]
 70%|███████   | 13065/18627 [7:18:36<3:35:12,  2.32s/it]
 70%|███████   | 13066/18627 [7:18:38<3:35:23,  2.32s/it]
 70%|███████   | 13067/18627 [7:18:39<3:00:20,  1.95s/it]
 70%|███████   | 13068/18627 [7:18:41<2:43:16,  1.76s/it]
 70%|███████   | 13069/18627 [7:18:43<2:58:51,  1.93s/it]
 70%|███████   | 13070/18627 [7:18:45<3:09:40,  2.05s/it]
                                                         
{'loss': 0.7755, 'grad_norm': 10.123322486877441, 'learning_rate': 1.0789851758562913e-06, 'epoch': 0.7}
+
 70%|███████   | 13070/18627 [7:18:45<3:09:40,  2.05s/it]
 70%|███████   | 13071/18627 [7:18:48<3:17:05,  2.13s/it]
 70%|███████   | 13072/18627 [7:18:50<3:22:01,  2.18s/it]
 70%|███████   | 13073/18627 [7:18:52<3:26:00,  2.23s/it]
 70%|███████   | 13074/18627 [7:18:54<3:02:19,  1.97s/it]
 70%|███████   | 13075/18627 [7:18:56<3:12:34,  2.08s/it]
 70%|███████   | 13076/18627 [7:18:57<2:52:31,  1.86s/it]
 70%|███████   | 13077/18627 [7:19:00<3:04:48,  2.00s/it]
 70%|███████   | 13078/18627 [7:19:02<3:13:59,  2.10s/it]
 70%|███████   | 13079/18627 [7:19:04<3:21:18,  2.18s/it]
 70%|███████   | 13080/18627 [7:19:07<3:25:47,  2.23s/it]
                                                         
{'loss': 1.058, 'grad_norm': 9.748332023620605, 'learning_rate': 1.0754109200799708e-06, 'epoch': 0.7}
+
 70%|███████   | 13080/18627 [7:19:07<3:25:47,  2.23s/it]
 70%|███████   | 13081/18627 [7:19:08<3:02:41,  1.98s/it]
 70%|███████   | 13082/18627 [7:19:11<3:12:29,  2.08s/it]
 70%|███████   | 13083/18627 [7:19:13<3:19:55,  2.16s/it]
 70%|███████   | 13084/18627 [7:19:14<2:57:19,  1.92s/it]
 70%|███████   | 13085/18627 [7:19:16<2:42:51,  1.76s/it]
 70%|███████   | 13086/18627 [7:19:18<2:58:45,  1.94s/it]
 70%|███████   | 13087/18627 [7:19:20<3:09:28,  2.05s/it]
 70%|███████   | 13088/18627 [7:19:23<3:16:53,  2.13s/it]
 70%|███████   | 13089/18627 [7:19:25<3:22:22,  2.19s/it]
 70%|███████   | 13090/18627 [7:19:27<3:25:38,  2.23s/it]
                                                         
{'loss': 1.204, 'grad_norm': 6.044355869293213, 'learning_rate': 1.0718409712462438e-06, 'epoch': 0.7}
+
 70%|███████   | 13090/18627 [7:19:27<3:25:38,  2.23s/it]
 70%|███████   | 13091/18627 [7:19:30<3:28:21,  2.26s/it]
 70%|███████   | 13092/18627 [7:19:32<3:30:35,  2.28s/it]
 70%|███████   | 13093/18627 [7:19:34<3:31:49,  2.30s/it]
 70%|███████   | 13094/18627 [7:19:37<3:32:20,  2.30s/it]
 70%|███████   | 13095/18627 [7:19:39<3:32:40,  2.31s/it]
 70%|███████   | 13096/18627 [7:19:41<3:33:15,  2.31s/it]
 70%|███████   | 13097/18627 [7:19:44<3:33:42,  2.32s/it]
 70%|███████   | 13098/18627 [7:19:45<3:08:22,  2.04s/it]
 70%|███████   | 13099/18627 [7:19:47<3:16:49,  2.14s/it]
 70%|███████   | 13100/18627 [7:19:50<3:22:34,  2.20s/it]
                                                         
{'loss': 0.7768, 'grad_norm': 7.637211322784424, 'learning_rate': 1.0682753401480928e-06, 'epoch': 0.7}
+
 70%|███████   | 13100/18627 [7:19:50<3:22:34,  2.20s/it]
 70%|███████   | 13101/18627 [7:19:52<3:26:10,  2.24s/it]
 70%|███████   | 13102/18627 [7:19:54<3:29:45,  2.28s/it]
 70%|███████   | 13103/18627 [7:19:57<3:31:17,  2.29s/it]
 70%|███████   | 13104/18627 [7:19:59<3:33:34,  2.32s/it]
 70%|███████   | 13105/18627 [7:20:00<3:07:12,  2.03s/it]
 70%|███████   | 13106/18627 [7:20:03<3:15:42,  2.13s/it]
 70%|███████   | 13107/18627 [7:20:05<3:24:24,  2.22s/it]
 70%|███████   | 13108/18627 [7:20:08<3:26:58,  2.25s/it]
 70%|███████   | 13109/18627 [7:20:10<3:29:07,  2.27s/it]
 70%|███████   | 13110/18627 [7:20:12<3:30:18,  2.29s/it]
                                                         
{'loss': 0.7998, 'grad_norm': 4.9901041984558105, 'learning_rate': 1.064714037565446e-06, 'epoch': 0.7}
+
 70%|███████   | 13110/18627 [7:20:12<3:30:18,  2.29s/it]
 70%|███████   | 13111/18627 [7:20:14<3:05:09,  2.01s/it]
 70%|███████   | 13112/18627 [7:20:16<3:13:37,  2.11s/it]
 70%|███████   | 13113/18627 [7:20:18<3:19:51,  2.17s/it]
 70%|███████   | 13114/18627 [7:20:20<2:57:12,  1.93s/it]
 70%|███████   | 13115/18627 [7:20:22<3:08:23,  2.05s/it]
 70%|███████   | 13116/18627 [7:20:24<3:16:04,  2.13s/it]
 70%|███████   | 13117/18627 [7:20:27<3:20:56,  2.19s/it]
 70%|███████   | 13118/18627 [7:20:29<3:25:45,  2.24s/it]
 70%|███████   | 13119/18627 [7:20:31<3:27:55,  2.27s/it]
 70%|███████   | 13120/18627 [7:20:33<3:04:28,  2.01s/it]
                                                         
{'loss': 1.2348, 'grad_norm': 12.343181610107422, 'learning_rate': 1.0611570742651455e-06, 'epoch': 0.7}
+
 70%|███████   | 13120/18627 [7:20:33<3:04:28,  2.01s/it]
 70%|███████   | 13121/18627 [7:20:35<3:12:24,  2.10s/it]
 70%|███████   | 13122/18627 [7:20:36<2:53:32,  1.89s/it]
 70%|███████   | 13123/18627 [7:20:39<3:04:54,  2.02s/it]
 70%|███████   | 13124/18627 [7:20:41<3:13:53,  2.11s/it]
 70%|███████   | 13125/18627 [7:20:42<2:55:21,  1.91s/it]
 70%|███████   | 13126/18627 [7:20:45<3:07:06,  2.04s/it]
 70%|███████   | 13127/18627 [7:20:47<3:14:41,  2.12s/it]
 70%|███████   | 13128/18627 [7:20:49<3:19:44,  2.18s/it]
 70%|███████   | 13129/18627 [7:20:52<3:24:28,  2.23s/it]
 70%|███████   | 13130/18627 [7:20:54<3:26:08,  2.25s/it]
                                                         
{'loss': 1.0921, 'grad_norm': 6.451207637786865, 'learning_rate': 1.0576044610009157e-06, 'epoch': 0.7}
+
 70%|███████   | 13130/18627 [7:20:54<3:26:08,  2.25s/it]
 70%|███████   | 13131/18627 [7:20:56<3:27:57,  2.27s/it]
 70%|███████   | 13132/18627 [7:20:58<3:03:52,  2.01s/it]
 71%|███████   | 13133/18627 [7:20:59<2:51:46,  1.88s/it]
 71%|███████   | 13134/18627 [7:21:02<3:03:44,  2.01s/it]
 71%|███████   | 13135/18627 [7:21:04<3:12:20,  2.10s/it]
 71%|███████   | 13136/18627 [7:21:06<3:17:49,  2.16s/it]
 71%|███████   | 13137/18627 [7:21:09<3:21:47,  2.21s/it]
 71%|███████   | 13138/18627 [7:21:10<2:58:07,  1.95s/it]
 71%|███████   | 13139/18627 [7:21:12<3:08:29,  2.06s/it]
 71%|███████   | 13140/18627 [7:21:15<3:15:09,  2.13s/it]
                                                         
{'loss': 1.2067, 'grad_norm': 8.132711410522461, 'learning_rate': 1.0540562085133256e-06, 'epoch': 0.71}
+
 71%|███████   | 13140/18627 [7:21:15<3:15:09,  2.13s/it]
 71%|███████   | 13141/18627 [7:21:17<3:19:55,  2.19s/it]
 71%|███████   | 13142/18627 [7:21:19<3:23:37,  2.23s/it]
 71%|███████   | 13143/18627 [7:21:21<2:59:35,  1.96s/it]
 71%|███████   | 13144/18627 [7:21:23<3:09:51,  2.08s/it]
 71%|███████   | 13145/18627 [7:21:25<3:16:52,  2.15s/it]
 71%|███████   | 13146/18627 [7:21:28<3:20:55,  2.20s/it]
 71%|███████   | 13147/18627 [7:21:30<3:24:25,  2.24s/it]
 71%|███████   | 13148/18627 [7:21:31<3:02:38,  2.00s/it]
 71%|███████   | 13149/18627 [7:21:34<3:11:29,  2.10s/it]
 71%|███████   | 13150/18627 [7:21:36<3:17:12,  2.16s/it]
                                                         
{'loss': 1.0526, 'grad_norm': 6.082878112792969, 'learning_rate': 1.050512327529767e-06, 'epoch': 0.71}
+
 71%|███████   | 13150/18627 [7:21:36<3:17:12,  2.16s/it]
 71%|███████   | 13151/18627 [7:21:38<3:19:43,  2.19s/it]
 71%|███████   | 13152/18627 [7:21:40<3:22:29,  2.22s/it]
 71%|███████   | 13153/18627 [7:21:43<3:25:16,  2.25s/it]
 71%|███████   | 13154/18627 [7:21:44<3:06:57,  2.05s/it]
 71%|███████   | 13155/18627 [7:21:47<3:15:17,  2.14s/it]
 71%|███████   | 13156/18627 [7:21:48<2:52:41,  1.89s/it]
 71%|███████   | 13157/18627 [7:21:49<2:39:30,  1.75s/it]
 71%|███████   | 13158/18627 [7:21:51<2:26:24,  1.61s/it]
 71%|███████   | 13159/18627 [7:21:53<2:45:20,  1.81s/it]
 71%|███████   | 13160/18627 [7:21:55<3:00:07,  1.98s/it]
                                                         
{'loss': 1.4003, 'grad_norm': 8.207721710205078, 'learning_rate': 1.0469728287644087e-06, 'epoch': 0.71}
+
 71%|███████   | 13160/18627 [7:21:55<3:00:07,  1.98s/it]
 71%|███████   | 13161/18627 [7:21:58<3:09:30,  2.08s/it]
 71%|███████   | 13162/18627 [7:22:00<3:16:37,  2.16s/it]
 71%|███████   | 13163/18627 [7:22:02<3:20:50,  2.21s/it]
 71%|███████   | 13164/18627 [7:22:05<3:23:52,  2.24s/it]
 71%|███████   | 13165/18627 [7:22:06<2:59:33,  1.97s/it]
 71%|███████   | 13166/18627 [7:22:08<3:06:20,  2.05s/it]
 71%|███████   | 13167/18627 [7:22:11<3:13:45,  2.13s/it]
 71%|███████   | 13168/18627 [7:22:12<2:54:56,  1.92s/it]
 71%|███████   | 13169/18627 [7:22:14<3:05:51,  2.04s/it]
 71%|███████   | 13170/18627 [7:22:16<2:44:43,  1.81s/it]
                                                         
{'loss': 1.18, 'grad_norm': 14.246277809143066, 'learning_rate': 1.0434377229181745e-06, 'epoch': 0.71}
+
 71%|███████   | 13170/18627 [7:22:16<2:44:43,  1.81s/it]
 71%|███████   | 13171/18627 [7:22:18<2:58:49,  1.97s/it]
 71%|███████   | 13172/18627 [7:22:20<3:07:34,  2.06s/it]
 71%|███████   | 13173/18627 [7:22:23<3:14:47,  2.14s/it]
 71%|███████   | 13174/18627 [7:22:25<3:18:46,  2.19s/it]
 71%|███████   | 13175/18627 [7:22:27<3:21:45,  2.22s/it]
 71%|███████   | 13176/18627 [7:22:29<3:24:34,  2.25s/it]
 71%|███████   | 13177/18627 [7:22:32<3:26:43,  2.28s/it]
 71%|███████   | 13178/18627 [7:22:33<2:59:03,  1.97s/it]
 71%|███████   | 13179/18627 [7:22:35<3:08:36,  2.08s/it]
 71%|███████   | 13180/18627 [7:22:37<2:49:34,  1.87s/it]
                                                         
{'loss': 1.0161, 'grad_norm': 14.038466453552246, 'learning_rate': 1.0399070206787066e-06, 'epoch': 0.71}
+
 71%|███████   | 13180/18627 [7:22:37<2:49:34,  1.87s/it]
 71%|███████   | 13181/18627 [7:22:39<3:01:54,  2.00s/it]
 71%|███████   | 13182/18627 [7:22:41<3:09:57,  2.09s/it]
 71%|███████   | 13183/18627 [7:22:44<3:16:07,  2.16s/it]
 71%|███████   | 13184/18627 [7:22:46<3:20:35,  2.21s/it]
 71%|███████   | 13185/18627 [7:22:48<3:22:54,  2.24s/it]
 71%|███████   | 13186/18627 [7:22:51<3:25:31,  2.27s/it]
 71%|███████   | 13187/18627 [7:22:52<2:59:56,  1.98s/it]
 71%|███████   | 13188/18627 [7:22:54<3:09:41,  2.09s/it]
 71%|███████   | 13189/18627 [7:22:57<3:15:53,  2.16s/it]
 71%|███████   | 13190/18627 [7:22:59<3:20:00,  2.21s/it]
                                                         
{'loss': 0.7694, 'grad_norm': 7.873215198516846, 'learning_rate': 1.0363807327203328e-06, 'epoch': 0.71}
+
 71%|███████   | 13190/18627 [7:22:59<3:20:00,  2.21s/it]
 71%|███████   | 13191/18627 [7:23:00<2:58:21,  1.97s/it]
 71%|███████   | 13192/18627 [7:23:03<3:08:36,  2.08s/it]
 71%|███████   | 13193/18627 [7:23:05<3:15:35,  2.16s/it]
 71%|███████   | 13194/18627 [7:23:06<2:53:41,  1.92s/it]
 71%|███████   | 13195/18627 [7:23:09<3:04:48,  2.04s/it]
 71%|███████   | 13196/18627 [7:23:11<3:11:57,  2.12s/it]
 71%|███████   | 13197/18627 [7:23:13<3:16:42,  2.17s/it]
 71%|███████   | 13198/18627 [7:23:16<3:21:19,  2.23s/it]
 71%|███████   | 13199/18627 [7:23:17<2:58:04,  1.97s/it]
 71%|███████   | 13200/18627 [7:23:19<3:07:02,  2.07s/it]
                                                         
{'loss': 1.1258, 'grad_norm': 4.880125999450684, 'learning_rate': 1.0328588697040371e-06, 'epoch': 0.71}
+
 71%|███████   | 13200/18627 [7:23:19<3:07:02,  2.07s/it]
 71%|███████   | 13201/18627 [7:23:22<3:13:18,  2.14s/it]
 71%|███████   | 13202/18627 [7:23:24<3:17:34,  2.19s/it]
 71%|███████   | 13203/18627 [7:23:26<3:20:59,  2.22s/it]
 71%|███████   | 13204/18627 [7:23:28<2:57:15,  1.96s/it]
 71%|███████   | 13205/18627 [7:23:30<3:06:19,  2.06s/it]
 71%|███████   | 13206/18627 [7:23:32<3:12:39,  2.13s/it]
 71%|███████   | 13207/18627 [7:23:35<3:17:10,  2.18s/it]
 71%|███████   | 13208/18627 [7:23:37<3:21:00,  2.23s/it]
 71%|███████   | 13209/18627 [7:23:39<3:23:49,  2.26s/it]
 71%|███████   | 13210/18627 [7:23:41<3:24:42,  2.27s/it]
                                                         
{'loss': 0.8678, 'grad_norm': 4.894803524017334, 'learning_rate': 1.0293414422774245e-06, 'epoch': 0.71}
+
 71%|███████   | 13210/18627 [7:23:41<3:24:42,  2.27s/it]
 71%|███████   | 13211/18627 [7:23:44<3:25:52,  2.28s/it]
 71%|███████   | 13212/18627 [7:23:45<3:01:26,  2.01s/it]
 71%|███████   | 13213/18627 [7:23:48<3:10:21,  2.11s/it]
 71%|███████   | 13214/18627 [7:23:50<3:16:17,  2.18s/it]
 71%|███████   | 13215/18627 [7:23:52<3:20:23,  2.22s/it]
 71%|███████   | 13216/18627 [7:23:55<3:23:13,  2.25s/it]
 71%|███████   | 13217/18627 [7:23:57<3:24:37,  2.27s/it]
 71%|███████   | 13218/18627 [7:23:59<3:26:38,  2.29s/it]
 71%|███████   | 13219/18627 [7:24:01<3:27:06,  2.30s/it]
 71%|███████   | 13220/18627 [7:24:04<3:27:20,  2.30s/it]
                                                         
{'loss': 0.8609, 'grad_norm': 5.978727340698242, 'learning_rate': 1.0258284610746878e-06, 'epoch': 0.71}
+
 71%|███████   | 13220/18627 [7:24:04<3:27:20,  2.30s/it]
 71%|███████   | 13221/18627 [7:24:06<3:28:14,  2.31s/it]
 71%|███████   | 13222/18627 [7:24:08<3:28:25,  2.31s/it]
 71%|███████   | 13223/18627 [7:24:11<3:28:52,  2.32s/it]
 71%|███████   | 13224/18627 [7:24:13<3:28:07,  2.31s/it]
 71%|███████   | 13225/18627 [7:24:15<3:28:06,  2.31s/it]
 71%|███████   | 13226/18627 [7:24:18<3:27:07,  2.30s/it]
 71%|███████   | 13227/18627 [7:24:20<3:27:25,  2.30s/it]
 71%|██���████   | 13228/18627 [7:24:22<3:27:15,  2.30s/it]
 71%|███████   | 13229/18627 [7:24:25<3:26:46,  2.30s/it]
 71%|███████   | 13230/18627 [7:24:27<3:27:13,  2.30s/it]
                                                         
{'loss': 0.6022, 'grad_norm': 6.270735263824463, 'learning_rate': 1.022319936716583e-06, 'epoch': 0.71}
+
 71%|███████   | 13230/18627 [7:24:27<3:27:13,  2.30s/it]
 71%|███████   | 13231/18627 [7:24:29<3:27:21,  2.31s/it]
 71%|███████   | 13232/18627 [7:24:31<3:26:18,  2.29s/it]
 71%|███████   | 13233/18627 [7:24:33<3:02:07,  2.03s/it]
 71%|███████   | 13234/18627 [7:24:34<2:41:02,  1.79s/it]
 71%|███████   | 13235/18627 [7:24:36<2:55:23,  1.95s/it]
 71%|███████   | 13236/18627 [7:24:39<3:04:39,  2.06s/it]
 71%|███████   | 13237/18627 [7:24:41<3:12:16,  2.14s/it]
 71%|███████   | 13238/18627 [7:24:42<2:42:34,  1.81s/it]
 71%|███████   | 13239/18627 [7:24:44<2:57:05,  1.97s/it]
 71%|███████   | 13240/18627 [7:24:47<3:06:53,  2.08s/it]
                                                         
{'loss': 0.83, 'grad_norm': 6.012319087982178, 'learning_rate': 1.0188158798103856e-06, 'epoch': 0.71}
+
 71%|███████   | 13240/18627 [7:24:47<3:06:53,  2.08s/it]
 71%|███████   | 13241/18627 [7:24:49<3:14:13,  2.16s/it]
 71%|███████   | 13242/18627 [7:24:51<3:18:20,  2.21s/it]
 71%|███████   | 13243/18627 [7:24:54<3:20:59,  2.24s/it]
 71%|███████   | 13244/18627 [7:24:56<3:23:17,  2.27s/it]
 71%|███████   | 13245/18627 [7:24:57<2:59:10,  2.00s/it]
 71%|███████   | 13246/18627 [7:25:00<3:07:40,  2.09s/it]
 71%|███████   | 13247/18627 [7:25:02<3:12:54,  2.15s/it]
 71%|███████   | 13248/18627 [7:25:04<3:16:46,  2.19s/it]
 71%|███████   | 13249/18627 [7:25:07<3:19:14,  2.22s/it]
 71%|███████   | 13250/18627 [7:25:08<2:54:08,  1.94s/it]
                                                         
{'loss': 1.005, 'grad_norm': 12.503332138061523, 'learning_rate': 1.0153163009498692e-06, 'epoch': 0.71}
+
 71%|███████   | 13250/18627 [7:25:08<2:54:08,  1.94s/it]
 71%|███████   | 13251/18627 [7:25:10<3:08:30,  2.10s/it]
 71%|███████   | 13252/18627 [7:25:13<3:14:29,  2.17s/it]
 71%|███████   | 13253/18627 [7:25:15<3:18:32,  2.22s/it]
 71%|███████   | 13254/18627 [7:25:17<3:20:54,  2.24s/it]
 71%|███████   | 13255/18627 [7:25:20<3:23:44,  2.28s/it]
 71%|███████   | 13256/18627 [7:25:22<3:24:44,  2.29s/it]
 71%|███████   | 13257/18627 [7:25:24<3:25:27,  2.30s/it]
 71%|███████   | 13258/18627 [7:25:27<3:26:17,  2.31s/it]
 71%|███████   | 13259/18627 [7:25:29<3:27:03,  2.31s/it]
 71%|███████   | 13260/18627 [7:25:31<3:27:41,  2.32s/it]
                                                         
{'loss': 0.6914, 'grad_norm': 5.174591064453125, 'learning_rate': 1.0118212107152669e-06, 'epoch': 0.71}
+
 71%|███████   | 13260/18627 [7:25:31<3:27:41,  2.32s/it]
 71%|███████   | 13261/18627 [7:25:34<3:27:27,  2.32s/it]
 71%|███████   | 13262/18627 [7:25:36<3:26:47,  2.31s/it]
 71%|███████   | 13263/18627 [7:25:37<3:00:54,  2.02s/it]
 71%|███████   | 13264/18627 [7:25:40<3:09:15,  2.12s/it]
 71%|███████   | 13265/18627 [7:25:42<3:15:04,  2.18s/it]
 71%|███████   | 13266/18627 [7:25:44<3:18:36,  2.22s/it]
 71%|███████   | 13267/18627 [7:25:47<3:22:07,  2.26s/it]
 71%|███████   | 13268/18627 [7:25:49<3:24:00,  2.28s/it]
 71%|███████   | 13269/18627 [7:25:51<3:25:21,  2.30s/it]
 71%|███████   | 13270/18627 [7:25:54<3:26:28,  2.31s/it]
                                                         
{'loss': 0.8076, 'grad_norm': 5.366012096405029, 'learning_rate': 1.0083306196732423e-06, 'epoch': 0.71}
+
 71%|███████   | 13270/18627 [7:25:54<3:26:28,  2.31s/it]
 71%|███████   | 13271/18627 [7:25:56<3:27:09,  2.32s/it]
 71%|███████▏  | 13272/18627 [7:25:58<3:26:05,  2.31s/it]
 71%|███████▏  | 13273/18627 [7:26:01<3:25:49,  2.31s/it]
 71%|███████▏  | 13274/18627 [7:26:02<2:59:29,  2.01s/it]
 71%|███████▏  | 13275/18627 [7:26:04<3:08:46,  2.12s/it]
 71%|███████▏  | 13276/18627 [7:26:06<2:48:34,  1.89s/it]
 71%|███████▏  | 13277/18627 [7:26:08<2:59:47,  2.02s/it]
 71%|███████▏  | 13278/18627 [7:26:10<3:08:43,  2.12s/it]
 71%|███████▏  | 13279/18627 [7:26:13<3:13:47,  2.17s/it]
 71%|███████▏  | 13280/18627 [7:26:15<3:18:15,  2.22s/it]
                                                         
{'loss': 1.0181, 'grad_norm': 7.188870906829834, 'learning_rate': 1.0048445383768556e-06, 'epoch': 0.71}
+
 71%|███████▏  | 13280/18627 [7:26:15<3:18:15,  2.22s/it]
 71%|███████▏  | 13281/18627 [7:26:17<3:22:01,  2.27s/it]
 71%|███████▏  | 13282/18627 [7:26:20<3:23:35,  2.29s/it]
 71%|███████▏  | 13283/18627 [7:26:22<3:25:02,  2.30s/it]
 71%|███████▏  | 13284/18627 [7:26:24<3:26:26,  2.32s/it]
 71%|███████▏  | 13285/18627 [7:26:27<3:26:39,  2.32s/it]
 71%|███████▏  | 13286/18627 [7:26:29<3:27:23,  2.33s/it]
 71%|███████▏  | 13287/18627 [7:26:30<2:52:41,  1.94s/it]
 71%|███████▏  | 13288/18627 [7:26:32<2:39:50,  1.80s/it]
 71%|███████▏  | 13289/18627 [7:26:34<2:54:08,  1.96s/it]
 71%|███████▏  | 13290/18627 [7:26:35<2:38:23,  1.78s/it]
                                                         
{'loss': 0.9993, 'grad_norm': 12.918221473693848, 'learning_rate': 1.001362977365535e-06, 'epoch': 0.71}
+
 71%|███████▏  | 13290/18627 [7:26:35<2:38:23,  1.78s/it]
 71%|███████▏  | 13291/18627 [7:26:38<2:53:34,  1.95s/it]
 71%|███████▏  | 13292/18627 [7:26:40<3:03:18,  2.06s/it]
 71%|███████▏  | 13293/18627 [7:26:42<3:09:41,  2.13s/it]
 71%|███████▏  | 13294/18627 [7:26:44<3:13:34,  2.18s/it]
 71%|███████▏  | 13295/18627 [7:26:47<3:17:07,  2.22s/it]
 71%|███████▏  | 13296/18627 [7:26:49<3:19:55,  2.25s/it]
 71%|███████▏  | 13297/18627 [7:26:51<3:21:31,  2.27s/it]
 71%|███████▏  | 13298/18627 [7:26:53<2:56:39,  1.99s/it]
 71%|███████▏  | 13299/18627 [7:26:55<3:05:06,  2.08s/it]
 71%|███████▏  | 13300/18627 [7:26:57<3:11:57,  2.16s/it]
                                                         
{'loss': 0.8346, 'grad_norm': 6.333808422088623, 'learning_rate': 9.97885947165038e-07, 'epoch': 0.71}
+
 71%|███████▏  | 13300/18627 [7:26:57<3:11:57,  2.16s/it]
 71%|███████▏  | 13301/18627 [7:27:00<3:16:22,  2.21s/it]
 71%|███████▏  | 13302/18627 [7:27:01<2:53:09,  1.95s/it]
 71%|███████▏  | 13303/18627 [7:27:03<3:02:03,  2.05s/it]
 71%|███████▏  | 13304/18627 [7:27:06<3:06:35,  2.10s/it]
 71%|███████▏  | 13305/18627 [7:27:08<3:07:06,  2.11s/it]
 71%|███████▏  | 13306/18627 [7:27:10<3:07:40,  2.12s/it]
 71%|███████▏  | 13307/18627 [7:27:12<3:07:51,  2.12s/it]
 71%|███████▏  | 13308/18627 [7:27:14<3:07:50,  2.12s/it]
 71%|███████▏  | 13309/18627 [7:27:15<2:42:59,  1.84s/it]
 71%|███████▏  | 13310/18627 [7:27:17<2:50:16,  1.92s/it]
                                                         
{'loss': 1.0502, 'grad_norm': 6.002601146697998, 'learning_rate': 9.944134582874295e-07, 'epoch': 0.71}
+
 71%|███████▏  | 13310/18627 [7:27:17<2:50:16,  1.92s/it]
 71%|███████▏  | 13311/18627 [7:27:20<2:55:52,  1.99s/it]
 71%|███████▏  | 13312/18627 [7:27:22<2:59:37,  2.03s/it]
 71%|███████▏  | 13313/18627 [7:27:24<3:03:20,  2.07s/it]
 71%|███████▏  | 13314/18627 [7:27:25<2:31:00,  1.71s/it]
 71%|███████▏  | 13315/18627 [7:27:27<2:42:52,  1.84s/it]
 71%|███████▏  | 13316/18627 [7:27:29<2:51:20,  1.94s/it]
 71%|███████▏  | 13317/18627 [7:27:30<2:19:23,  1.57s/it]
 71%|███████▏  | 13318/18627 [7:27:32<2:34:06,  1.74s/it]
 72%|███████▏  | 13319/18627 [7:27:34<2:45:02,  1.87s/it]
 72%|███████▏  | 13320/18627 [7:27:36<2:51:45,  1.94s/it]
                                                         
{'loss': 0.5749, 'grad_norm': 5.675798416137695, 'learning_rate': 9.909455212310426e-07, 'epoch': 0.72}
+
 72%|███████▏  | 13320/18627 [7:27:36<2:51:45,  1.94s/it]
 72%|███████▏  | 13321/18627 [7:27:37<2:31:56,  1.72s/it]
 72%|███████▏  | 13322/18627 [7:27:39<2:43:02,  1.84s/it]
 72%|███████▏  | 13323/18627 [7:27:42<2:51:29,  1.94s/it]
 72%|███████▏  | 13324/18627 [7:27:44<2:55:59,  1.99s/it]
 72%|███████▏  | 13325/18627 [7:27:46<2:59:16,  2.03s/it]
 72%|███████▏  | 13326/18627 [7:27:48<3:01:35,  2.06s/it]
 72%|███████▏  | 13327/18627 [7:27:50<3:04:49,  2.09s/it]
 72%|███████▏  | 13328/18627 [7:27:52<3:05:52,  2.10s/it]
 72%|███████▏  | 13329/18627 [7:27:54<3:07:55,  2.13s/it]
 72%|███████▏  | 13330/18627 [7:27:55<2:38:42,  1.80s/it]
                                                         
{'loss': 0.9684, 'grad_norm': 15.930012702941895, 'learning_rate': 9.87482146480446e-07, 'epoch': 0.72}
+
 72%|███████▏  | 13330/18627 [7:27:55<2:38:42,  1.80s/it]
 72%|███████▏  | 13331/18627 [7:27:57<2:22:41,  1.62s/it]
 72%|███████▏  | 13332/18627 [7:27:59<2:36:33,  1.77s/it]
 72%|███████▏  | 13333/18627 [7:28:01<2:45:23,  1.87s/it]
 72%|███████▏  | 13334/18627 [7:28:03<2:52:29,  1.96s/it]
 72%|███████▏  | 13335/18627 [7:28:05<2:56:46,  2.00s/it]
 72%|███████▏  | 13336/18627 [7:28:06<2:35:06,  1.76s/it]
 72%|███████▏  | 13337/18627 [7:28:08<2:44:26,  1.87s/it]
 72%|███████▏  | 13338/18627 [7:28:11<2:50:42,  1.94s/it]
 72%|███████▏  | 13339/18627 [7:28:13<2:57:00,  2.01s/it]
 72%|███████▏  | 13340/18627 [7:28:15<3:00:14,  2.05s/it]
                                                         
{'loss': 0.9736, 'grad_norm': 8.702911376953125, 'learning_rate': 9.840233445064213e-07, 'epoch': 0.72}
+
 72%|███████▏  | 13340/18627 [7:28:15<3:00:14,  2.05s/it]
 72%|███████▏  | 13341/18627 [7:28:17<3:01:53,  2.06s/it]
 72%|███████▏  | 13342/18627 [7:28:18<2:38:35,  1.80s/it]
 72%|███████▏  | 13343/18627 [7:28:20<2:47:32,  1.90s/it]
 72%|███████▏  | 13344/18627 [7:28:22<2:53:28,  1.97s/it]
 72%|███████▏  | 13345/18627 [7:28:25<2:57:40,  2.02s/it]
 72%|███████▏  | 13346/18627 [7:28:27<3:00:13,  2.05s/it]
 72%|███████▏  | 13347/18627 [7:28:29<3:02:29,  2.07s/it]
 72%|███████▏  | 13348/18627 [7:28:31<3:04:56,  2.10s/it]
 72%|███████▏  | 13349/18627 [7:28:33<3:05:06,  2.10s/it]
 72%|███████▏  | 13350/18627 [7:28:35<3:05:21,  2.11s/it]
                                                         
{'loss': 0.827, 'grad_norm': 6.550909519195557, 'learning_rate': 9.805691257659188e-07, 'epoch': 0.72}
+
 72%|███████▏  | 13350/18627 [7:28:35<3:05:21,  2.11s/it]
 72%|███████▏  | 13351/18627 [7:28:36<2:41:24,  1.84s/it]
 72%|███████▏  | 13352/18627 [7:28:38<2:24:19,  1.64s/it]
 72%|███████▏  | 13353/18627 [7:28:40<2:37:14,  1.79s/it]
 72%|███████▏  | 13354/18627 [7:28:40<2:08:42,  1.46s/it]
 72%|███████▏  | 13355/18627 [7:28:42<2:01:49,  1.39s/it]
 72%|███████▏  | 13356/18627 [7:28:44<2:20:55,  1.60s/it]
 72%|███████▏  | 13357/18627 [7:28:46<2:33:48,  1.75s/it]
 72%|███████▏  | 13358/18627 [7:28:48<2:44:02,  1.87s/it]
 72%|███████▏  | 13359/18627 [7:28:49<2:23:23,  1.63s/it]
 72%|███████▏  | 13360/18627 [7:28:52<2:53:55,  1.98s/it]
                                                         
{'loss': 1.3828, 'grad_norm': 6.246655464172363, 'learning_rate': 9.771195007020374e-07, 'epoch': 0.72}
+
 72%|███████▏  | 13360/18627 [7:28:52<2:53:55,  1.98s/it]
 72%|███████▏  | 13361/18627 [7:28:53<2:32:53,  1.74s/it]
 72%|███████▏  | 13362/18627 [7:28:55<2:42:53,  1.86s/it]
 72%|███████▏  | 13363/18627 [7:28:56<2:25:17,  1.66s/it]
 72%|███████▏  | 13364/18627 [7:28:58<2:36:56,  1.79s/it]
 72%|███████▏  | 13365/18627 [7:29:01<2:46:16,  1.90s/it]
 72%|███████▏  | 13366/18627 [7:29:03<2:53:34,  1.98s/it]
 72%|███████▏  | 13367/18627 [7:29:05<2:58:11,  2.03s/it]
 72%|███████▏  | 13368/18627 [7:29:07<3:01:09,  2.07s/it]
 72%|███████▏  | 13369/18627 [7:29:09<3:03:52,  2.10s/it]
 72%|███████▏  | 13370/18627 [7:29:11<3:04:22,  2.10s/it]
                                                         
{'loss': 1.0558, 'grad_norm': 11.373625755310059, 'learning_rate': 9.736744797439848e-07, 'epoch': 0.72}
+
 72%|███████▏  | 13370/18627 [7:29:11<3:04:22,  2.10s/it]
 72%|███████▏  | 13371/18627 [7:29:14<3:04:35,  2.11s/it]
 72%|███████▏  | 13372/18627 [7:29:16<3:05:26,  2.12s/it]
 72%|███████▏  | 13373/18627 [7:29:17<2:40:54,  1.84s/it]
 72%|███████▏  | 13374/18627 [7:29:19<2:48:36,  1.93s/it]
 72%|███████▏  | 13375/18627 [7:29:21<2:53:45,  1.99s/it]
 72%|███████▏  | 13376/18627 [7:29:22<2:32:56,  1.75s/it]
 72%|███████▏  | 13377/18627 [7:29:24<2:43:40,  1.87s/it]
 72%|███████▏  | 13378/18627 [7:29:26<2:25:34,  1.66s/it]
 72%|███████▏  | 13379/18627 [7:29:27<2:13:15,  1.52s/it]
 72%|███████▏  | 13380/18627 [7:29:29<2:28:44,  1.70s/it]
                                                         
{'loss': 1.4408, 'grad_norm': 5.425200939178467, 'learning_rate': 9.702340733070508e-07, 'epoch': 0.72}
+
 72%|███████▏  | 13380/18627 [7:29:29<2:28:44,  1.70s/it]
 72%|███████▏  | 13381/18627 [7:29:31<2:39:46,  1.83s/it]
 72%|███████▏  | 13382/18627 [7:29:33<2:47:14,  1.91s/it]
 72%|███████▏  | 13383/18627 [7:29:34<2:28:32,  1.70s/it]
 72%|███████▏  | 13384/18627 [7:29:37<2:40:35,  1.84s/it]
 72%|███████▏  | 13385/18627 [7:29:38<2:19:21,  1.60s/it]
 72%|███████▏  | 13386/18627 [7:29:40<2:33:27,  1.76s/it]
 72%|███████▏  | 13387/18627 [7:29:41<2:18:26,  1.59s/it]
 72%|███████▏  | 13388/18627 [7:29:43<2:32:00,  1.74s/it]
 72%|███████▏  | 13389/18627 [7:29:45<2:41:22,  1.85s/it]
 72%|███████▏  | 13390/18627 [7:29:47<2:48:34,  1.93s/it]
                                                         
{'loss': 1.1383, 'grad_norm': 8.932554244995117, 'learning_rate': 9.66798291792574e-07, 'epoch': 0.72}
+
 72%|███████▏  | 13390/18627 [7:29:47<2:48:34,  1.93s/it]
 72%|███████▏  | 13391/18627 [7:29:49<2:53:37,  1.99s/it]
 72%|███████▏  | 13392/18627 [7:29:51<2:56:17,  2.02s/it]
 72%|███████▏  | 13393/18627 [7:29:52<2:31:18,  1.73s/it]
 72%|███████▏  | 13394/18627 [7:29:55<2:42:52,  1.87s/it]
 72%|███████▏  | 13395/18627 [7:29:56<2:25:19,  1.67s/it]
 72%|███████▏  | 13396/18627 [7:29:58<2:36:42,  1.80s/it]
 72%|███████▏  | 13397/18627 [7:29:59<2:16:16,  1.56s/it]
 72%|███████▏  | 13398/18627 [7:30:00<2:06:37,  1.45s/it]
 72%|███████▏  | 13399/18627 [7:30:01<1:59:45,  1.37s/it]
 72%|███████▏  | 13400/18627 [7:30:04<2:19:15,  1.60s/it]
                                                         
{'loss': 1.7011, 'grad_norm': 4.846194744110107, 'learning_rate': 9.633671455879108e-07, 'epoch': 0.72}
+
 72%|███████▏  | 13400/18627 [7:30:04<2:19:15,  1.60s/it]
 72%|███████▏  | 13401/18627 [7:30:06<2:32:16,  1.75s/it]
 72%|███████▏  | 13402/18627 [7:30:08<2:42:12,  1.86s/it]
 72%|███████▏  | 13403/18627 [7:30:09<2:24:52,  1.66s/it]
 72%|███████▏  | 13404/18627 [7:30:11<2:36:49,  1.80s/it]
 72%|███████▏  | 13405/18627 [7:30:12<2:20:53,  1.62s/it]
 72%|███████▏  | 13406/18627 [7:30:13<2:09:37,  1.49s/it]
 72%|███████▏  | 13407/18627 [7:30:16<2:26:02,  1.68s/it]
 72%|███████▏  | 13408/18627 [7:30:18<2:36:56,  1.80s/it]
 72%|███████▏  | 13409/18627 [7:30:20<2:46:17,  1.91s/it]
 72%|███████▏  | 13410/18627 [7:30:21<2:25:05,  1.67s/it]
                                                         
{'loss': 1.4429, 'grad_norm': 15.551196098327637, 'learning_rate': 9.59940645066401e-07, 'epoch': 0.72}
+
 72%|███████▏  | 13410/18627 [7:30:21<2:25:05,  1.67s/it]
 72%|███████▏  | 13411/18627 [7:30:23<2:37:01,  1.81s/it]
 72%|███████▏  | 13412/18627 [7:30:25<2:45:16,  1.90s/it]
 72%|███████▏  | 13413/18627 [7:30:27<2:51:14,  1.97s/it]
 72%|███████▏  | 13414/18627 [7:30:31<3:32:38,  2.45s/it]
 72%|███████▏  | 13415/18627 [7:30:32<2:55:07,  2.02s/it]
 72%|███████▏  | 13416/18627 [7:30:34<2:57:25,  2.04s/it]
 72%|███████▏  | 13417/18627 [7:30:36<2:59:24,  2.07s/it]
 72%|███████▏  | 13418/18627 [7:30:38<3:00:44,  2.08s/it]
 72%|███████▏  | 13419/18627 [7:30:40<3:01:45,  2.09s/it]
 72%|███████▏  | 13420/18627 [7:30:42<2:38:07,  1.82s/it]
                                                         
{'loss': 1.0166, 'grad_norm': 13.631563186645508, 'learning_rate': 9.565188005873446e-07, 'epoch': 0.72}
+
 72%|███████▏  | 13420/18627 [7:30:42<2:38:07,  1.82s/it]
 72%|███████▏  | 13421/18627 [7:30:44<2:45:59,  1.91s/it]
 72%|███████▏  | 13422/18627 [7:30:46<2:50:32,  1.97s/it]
 72%|███████▏  | 13423/18627 [7:30:48<2:54:22,  2.01s/it]
 72%|███████▏  | 13424/18627 [7:30:49<2:33:11,  1.77s/it]
 72%|███████▏  | 13425/18627 [7:30:50<2:18:27,  1.60s/it]
 72%|███████▏  | 13426/18627 [7:30:52<2:32:10,  1.76s/it]
 72%|███████▏  | 13427/18627 [7:30:55<2:42:00,  1.87s/it]
 72%|███████▏  | 13428/18627 [7:30:57<2:48:21,  1.94s/it]
 72%|███████▏  | 13429/18627 [7:30:59<2:53:09,  2.00s/it]
 72%|███████▏  | 13430/18627 [7:31:01<2:57:04,  2.04s/it]
                                                         
{'loss': 1.0459, 'grad_norm': 6.831662654876709, 'learning_rate': 9.531016224959583e-07, 'epoch': 0.72}
+
 72%|███████▏  | 13430/18627 [7:31:01<2:57:04,  2.04s/it]
 72%|███████▏  | 13431/18627 [7:31:02<2:35:01,  1.79s/it]
 72%|███████▏  | 13432/18627 [7:31:04<2:43:58,  1.89s/it]
 72%|███████▏  | 13433/18627 [7:31:06<2:49:48,  1.96s/it]
 72%|███████▏  | 13434/18627 [7:31:08<2:53:39,  2.01s/it]
 72%|███████▏  | 13435/18627 [7:31:11<2:56:19,  2.04s/it]
 72%|███████▏  | 13436/18627 [7:31:12<2:34:35,  1.79s/it]
 72%|███████▏  | 13437/18627 [7:31:14<2:43:16,  1.89s/it]
 72%|███████▏  | 13438/18627 [7:31:16<2:48:57,  1.95s/it]
 72%|███████▏  | 13439/18627 [7:31:17<2:27:01,  1.70s/it]
 72%|███████▏  | 13440/18627 [7:31:19<2:38:10,  1.83s/it]
                                                         
{'loss': 1.1572, 'grad_norm': 7.9199604988098145, 'learning_rate': 9.496891211233578e-07, 'epoch': 0.72}
+
 72%|███████▏  | 13440/18627 [7:31:19<2:38:10,  1.83s/it]
 72%|███████▏  | 13441/18627 [7:31:21<2:45:45,  1.92s/it]
 72%|███████▏  | 13442/18627 [7:31:23<2:50:32,  1.97s/it]
 72%|███████▏  | 13443/18627 [7:31:26<2:54:24,  2.02s/it]
 72%|███████▏  | 13444/18627 [7:31:28<2:56:14,  2.04s/it]
 72%|███████▏  | 13445/18627 [7:31:30<2:58:41,  2.07s/it]
 72%|███████▏  | 13446/18627 [7:31:32<3:00:21,  2.09s/it]
 72%|███████▏  | 13447/18627 [7:31:34<3:01:02,  2.10s/it]
 72%|███████▏  | 13448/18627 [7:31:36<3:01:30,  2.10s/it]
 72%|███████▏  | 13449/18627 [7:31:37<2:38:00,  1.83s/it]
 72%|███████▏  | 13450/18627 [7:31:39<2:44:59,  1.91s/it]
                                                         
{'loss': 0.8202, 'grad_norm': 5.234699249267578, 'learning_rate': 9.462813067865139e-07, 'epoch': 0.72}
+
 72%|███████▏  | 13450/18627 [7:31:39<2:44:59,  1.91s/it]
 72%|███████▏  | 13451/18627 [7:31:41<2:26:28,  1.70s/it]
 72%|███████▏  | 13452/18627 [7:31:42<2:13:15,  1.55s/it]
 72%|███████▏  | 13453/18627 [7:31:43<2:04:13,  1.44s/it]
 72%|███████▏  | 13454/18627 [7:31:45<2:21:36,  1.64s/it]
 72%|███████▏  | 13455/18627 [7:31:47<2:34:19,  1.79s/it]
 72%|███████▏  | 13456/18627 [7:31:49<2:42:21,  1.88s/it]
 72%|███████▏  | 13457/18627 [7:31:52<2:48:27,  1.95s/it]
 72%|███████▏  | 13458/18627 [7:31:54<2:52:19,  2.00s/it]
 72%|███████▏  | 13459/18627 [7:31:56<2:55:15,  2.03s/it]
 72%|███████▏  | 13460/18627 [7:31:58<2:57:53,  2.07s/it]
                                                         
{'loss': 1.2445, 'grad_norm': 7.433228492736816, 'learning_rate': 9.428781897882308e-07, 'epoch': 0.72}
+
 72%|███████▏  | 13460/18627 [7:31:58<2:57:53,  2.07s/it]
 72%|███████▏  | 13461/18627 [7:32:00<2:59:28,  2.08s/it]
 72%|███████▏  | 13462/18627 [7:32:01<2:36:16,  1.82s/it]
 72%|███████▏  | 13463/18627 [7:32:03<2:43:57,  1.91s/it]
 72%|███████▏  | 13464/18627 [7:32:05<2:49:09,  1.97s/it]
 72%|███████▏  | 13465/18627 [7:32:07<2:29:03,  1.73s/it]
 72%|███████▏  | 13466/18627 [7:32:09<2:38:36,  1.84s/it]
 72%|███████▏  | 13467/18627 [7:32:11<2:45:32,  1.92s/it]
 72%|███████▏  | 13468/18627 [7:32:13<2:50:53,  1.99s/it]
 72%|███████▏  | 13469/18627 [7:32:14<2:30:14,  1.75s/it]
 72%|███████▏  | 13470/18627 [7:32:16<2:41:32,  1.88s/it]
                                                         
{'loss': 1.1995, 'grad_norm': 11.753120422363281, 'learning_rate': 9.394797804171096e-07, 'epoch': 0.72}
+
 72%|███████▏  | 13470/18627 [7:32:16<2:41:32,  1.88s/it]
 72%|███████▏  | 13471/18627 [7:32:18<2:47:46,  1.95s/it]
 72%|███████▏  | 13472/18627 [7:32:20<2:26:35,  1.71s/it]
 72%|███████▏  | 13473/18627 [7:32:22<2:37:08,  1.83s/it]
 72%|███████▏  | 13474/18627 [7:32:24<2:44:49,  1.92s/it]
 72%|███████▏  | 13475/18627 [7:32:26<2:50:13,  1.98s/it]
 72%|███████▏  | 13476/18627 [7:32:28<2:54:02,  2.03s/it]
 72%|███████▏  | 13477/18627 [7:32:29<2:32:36,  1.78s/it]
 72%|███████▏  | 13478/18627 [7:32:31<2:41:33,  1.88s/it]
 72%|███████▏  | 13479/18627 [7:32:34<2:48:32,  1.96s/it]
 72%|███████▏  | 13480/18627 [7:32:36<2:53:23,  2.02s/it]
                                                         
{'loss': 0.9782, 'grad_norm': 5.27089786529541, 'learning_rate': 9.360860889475201e-07, 'epoch': 0.72}
+
 72%|███████▏  | 13480/18627 [7:32:36<2:53:23,  2.02s/it]
 72%|███████▏  | 13481/18627 [7:32:38<2:57:13,  2.07s/it]
 72%|███████▏  | 13482/18627 [7:32:40<2:59:21,  2.09s/it]
 72%|███████▏  | 13483/18627 [7:32:42<3:00:23,  2.10s/it]
 72%|███████▏  | 13484/18627 [7:32:44<3:00:34,  2.11s/it]
 72%|███████▏  | 13485/18627 [7:32:46<3:01:00,  2.11s/it]
 72%|███████▏  | 13486/18627 [7:32:49<3:00:51,  2.11s/it]
 72%|███████▏  | 13487/18627 [7:32:50<2:34:25,  1.80s/it]
 72%|███████▏  | 13488/18627 [7:32:52<2:42:39,  1.90s/it]
 72%|███████▏  | 13489/18627 [7:32:54<2:48:08,  1.96s/it]
 72%|███████▏  | 13490/18627 [7:32:56<2:52:06,  2.01s/it]
                                                         
{'loss': 0.7745, 'grad_norm': 4.742655277252197, 'learning_rate': 9.326971256395675e-07, 'epoch': 0.72}
+
 72%|███████▏  | 13490/18627 [7:32:56<2:52:06,  2.01s/it]
 72%|███████▏  | 13491/18627 [7:32:58<2:54:15,  2.04s/it]
 72%|███████▏  | 13492/18627 [7:33:00<2:56:30,  2.06s/it]
 72%|███████▏  | 13493/18627 [7:33:01<2:34:15,  1.80s/it]
 72%|███████▏  | 13494/18627 [7:33:04<2:42:14,  1.90s/it]
 72%|███████▏  | 13495/18627 [7:33:06<2:48:12,  1.97s/it]
 72%|███████▏  | 13496/18627 [7:33:07<2:28:16,  1.73s/it]
 72%|███████▏  | 13497/18627 [7:33:08<2:02:28,  1.43s/it]
 72%|███████▏  | 13498/18627 [7:33:10<2:19:19,  1.63s/it]
 72%|███████▏  | 13499/18627 [7:33:12<2:31:36,  1.77s/it]
 72%|███████▏  | 13500/18627 [7:33:14<2:40:14,  1.88s/it]
                                                         
{'loss': 1.0859, 'grad_norm': 6.478848934173584, 'learning_rate': 9.293129007390636e-07, 'epoch': 0.72}
+
 72%|███████▏  | 13500/18627 [7:33:14<2:40:14,  1.88s/it]
 72%|███████▏  | 13501/18627 [7:33:15<2:22:33,  1.67s/it]
 72%|███████▏  | 13502/18627 [7:33:16<2:10:17,  1.53s/it]
 72%|███████▏  | 13503/18627 [7:33:17<1:52:14,  1.31s/it]
 72%|███████▏  | 13504/18627 [7:33:19<2:13:04,  1.56s/it]
 73%|███████▎  | 13505/18627 [7:33:21<2:26:58,  1.72s/it]
 73%|███████▎  | 13506/18627 [7:33:23<2:37:12,  1.84s/it]
 73%|███████▎  | 13507/18627 [7:33:26<2:44:07,  1.92s/it]
 73%|███████▎  | 13508/18627 [7:33:28<2:48:57,  1.98s/it]
 73%|███████▎  | 13509/18627 [7:33:30<2:53:18,  2.03s/it]
 73%|███████▎  | 13510/18627 [7:33:31<2:31:53,  1.78s/it]
                                                         
{'loss': 1.2472, 'grad_norm': 14.039655685424805, 'learning_rate': 9.259334244774912e-07, 'epoch': 0.73}
+
 73%|███████▎  | 13510/18627 [7:33:31<2:31:53,  1.78s/it]
 73%|███████▎  | 13511/18627 [7:33:33<2:40:14,  1.88s/it]
 73%|███████▎  | 13512/18627 [7:33:36<3:17:06,  2.31s/it]
 73%|███████▎  | 13513/18627 [7:33:39<3:12:33,  2.26s/it]
 73%|███████▎  | 13514/18627 [7:33:41<3:09:28,  2.22s/it]
 73%|███████▎  | 13515/18627 [7:33:43<3:07:10,  2.20s/it]
 73%|███████▎  | 13516/18627 [7:33:45<3:05:12,  2.17s/it]
 73%|███████▎  | 13517/18627 [7:33:46<2:40:29,  1.88s/it]
 73%|███████▎  | 13518/18627 [7:33:48<2:45:53,  1.95s/it]
 73%|███████▎  | 13519/18627 [7:33:50<2:50:32,  2.00s/it]
 73%|███████▎  | 13520/18627 [7:33:52<2:29:51,  1.76s/it]
                                                         
{'loss': 0.9831, 'grad_norm': 17.909832000732422, 'learning_rate': 9.225587070719827e-07, 'epoch': 0.73}
+
 73%|███████▎  | 13520/18627 [7:33:52<2:29:51,  1.76s/it]
 73%|███████▎  | 13521/18627 [7:33:53<2:13:06,  1.56s/it]
 73%|███████▎  | 13522/18627 [7:33:55<2:28:25,  1.74s/it]
 73%|███████▎  | 13523/18627 [7:33:56<2:14:01,  1.58s/it]
 73%|███████▎  | 13524/18627 [7:33:58<2:27:34,  1.74s/it]
 73%|███████▎  | 13525/18627 [7:34:00<2:37:13,  1.85s/it]
 73%|███████▎  | 13526/18627 [7:34:01<2:20:15,  1.65s/it]
 73%|███████▎  | 13527/18627 [7:34:04<2:31:32,  1.78s/it]
 73%|███████▎  | 13528/18627 [7:34:05<2:16:24,  1.61s/it]
 73%|███████▎  | 13529/18627 [7:34:07<2:29:37,  1.76s/it]
 73%|███████▎  | 13530/18627 [7:34:09<2:38:33,  1.87s/it]
                                                         
{'loss': 1.3642, 'grad_norm': 7.570437908172607, 'learning_rate': 9.19188758725277e-07, 'epoch': 0.73}
+
 73%|███████▎  | 13530/18627 [7:34:09<2:38:33,  1.87s/it]
 73%|███████▎  | 13531/18627 [7:34:11<2:45:05,  1.94s/it]
 73%|███████▎  | 13532/18627 [7:34:12<2:25:40,  1.72s/it]
 73%|███████▎  | 13533/18627 [7:34:14<2:35:56,  1.84s/it]
 73%|███████▎  | 13534/18627 [7:34:16<2:19:22,  1.64s/it]
 73%|███████▎  | 13535/18627 [7:34:17<2:07:46,  1.51s/it]
 73%|███████▎  | 13536/18627 [7:34:19<2:23:21,  1.69s/it]
 73%|███████▎  | 13537/18627 [7:34:21<2:34:14,  1.82s/it]
 73%|███████▎  | 13538/18627 [7:34:23<2:41:39,  1.91s/it]
 73%|███████▎  | 13539/18627 [7:34:25<2:47:25,  1.97s/it]
 73%|███████▎  | 13540/18627 [7:34:27<2:50:35,  2.01s/it]
                                                         
{'loss': 1.2551, 'grad_norm': 5.7500529289245605, 'learning_rate': 9.158235896256984e-07, 'epoch': 0.73}
+
 73%|███████▎  | 13540/18627 [7:34:27<2:50:35,  2.01s/it]
 73%|███████▎  | 13541/18627 [7:34:30<2:53:27,  2.05s/it]
 73%|███████▎  | 13542/18627 [7:34:32<2:56:02,  2.08s/it]
 73%|███████▎  | 13543/18627 [7:34:33<2:33:25,  1.81s/it]
 73%|███████▎  | 13544/18627 [7:34:35<2:41:48,  1.91s/it]
 73%|███████▎  | 13545/18627 [7:34:37<2:47:16,  1.97s/it]
 73%|███████▎  | 13546/18627 [7:34:39<2:52:51,  2.04s/it]
 73%|███████▎  | 13547/18627 [7:34:41<2:55:15,  2.07s/it]
 73%|███████▎  | 13548/18627 [7:34:44<2:56:23,  2.08s/it]
 73%|███████▎  | 13549/18627 [7:34:45<2:34:04,  1.82s/it]
 73%|███████▎  | 13550/18627 [7:34:47<2:40:59,  1.90s/it]
                                                         
{'loss': 1.0367, 'grad_norm': 4.77415657043457, 'learning_rate': 9.124632099471209e-07, 'epoch': 0.73}
+
 73%|███████▎  | 13550/18627 [7:34:47<2:40:59,  1.90s/it]
 73%|███████▎  | 13551/18627 [7:34:49<2:46:50,  1.97s/it]
 73%|███████▎  | 13552/18627 [7:34:51<2:50:41,  2.02s/it]
 73%|███████▎  | 13553/18627 [7:34:53<2:52:55,  2.04s/it]
 73%|███████▎  | 13554/18627 [7:34:55<2:54:55,  2.07s/it]
 73%|███████▎  | 13555/18627 [7:34:57<2:56:25,  2.09s/it]
 73%|███████▎  | 13556/18627 [7:35:00<2:56:41,  2.09s/it]
 73%|███████▎  | 13557/18627 [7:35:02<2:57:13,  2.10s/it]
 73%|███████▎  | 13558/18627 [7:35:03<2:34:00,  1.82s/it]
 73%|███████▎  | 13559/18627 [7:35:04<2:18:03,  1.63s/it]
 73%|███████▎  | 13560/18627 [7:35:06<2:30:22,  1.78s/it]
                                                         
{'loss': 1.0457, 'grad_norm': 5.784523010253906, 'learning_rate': 9.09107629848939e-07, 'epoch': 0.73}
+
 73%|███████▎  | 13560/18627 [7:35:06<2:30:22,  1.78s/it]
 73%|███████▎  | 13561/18627 [7:35:08<2:39:00,  1.88s/it]
 73%|███████▎  | 13562/18627 [7:35:10<2:45:10,  1.96s/it]
 73%|███████▎  | 13563/18627 [7:35:13<2:48:38,  2.00s/it]
 73%|███████▎  | 13564/18627 [7:35:14<2:28:11,  1.76s/it]
 73%|███████▎  | 13565/18627 [7:35:16<2:37:08,  1.86s/it]
 73%|███████▎  | 13566/18627 [7:35:18<2:43:49,  1.94s/it]
 73%|███████▎  | 13567/18627 [7:35:20<2:47:45,  1.99s/it]
 73%|███████▎  | 13568/18627 [7:35:21<2:27:24,  1.75s/it]
 73%|███████▎  | 13569/18627 [7:35:23<2:37:13,  1.87s/it]
 73%|███████▎  | 13570/18627 [7:35:25<2:43:12,  1.94s/it]
                                                         
{'loss': 1.026, 'grad_norm': 6.024356365203857, 'learning_rate': 9.057568594760366e-07, 'epoch': 0.73}
+
 73%|███████▎  | 13570/18627 [7:35:25<2:43:12,  1.94s/it]
 73%|███████▎  | 13571/18627 [7:35:28<2:47:30,  1.99s/it]
 73%|███████▎  | 13572/18627 [7:35:29<2:27:12,  1.75s/it]
 73%|███████▎  | 13573/18627 [7:35:30<2:11:38,  1.56s/it]
 73%|███████▎  | 13574/18627 [7:35:32<2:25:28,  1.73s/it]
 73%|███████▎  | 13575/18627 [7:35:33<2:12:01,  1.57s/it]
 73%|███████▎  | 13576/18627 [7:35:34<2:02:34,  1.46s/it]
 73%|███████▎  | 13577/18627 [7:35:37<2:18:58,  1.65s/it]
 73%|███████▎  | 13578/18627 [7:35:38<2:07:50,  1.52s/it]
 73%|███████▎  | 13579/18627 [7:35:40<2:23:49,  1.71s/it]
 73%|███████▎  | 13580/18627 [7:35:42<2:35:32,  1.85s/it]
                                                         
{'loss': 1.4477, 'grad_norm': 7.0575151443481445, 'learning_rate': 9.024109089587566e-07, 'epoch': 0.73}
+
 73%|███████▎  | 13580/18627 [7:35:42<2:35:32,  1.85s/it]
 73%|███████▎  | 13581/18627 [7:35:44<2:43:59,  1.95s/it]
 73%|███████▎  | 13582/18627 [7:35:46<2:48:42,  2.01s/it]
 73%|███████▎  | 13583/18627 [7:35:49<2:51:44,  2.04s/it]
 73%|███████▎  | 13584/18627 [7:35:51<2:53:59,  2.07s/it]
 73%|███████▎  | 13585/18627 [7:35:53<2:55:11,  2.08s/it]
 73%|███████▎  | 13586/18627 [7:35:55<2:55:25,  2.09s/it]
 73%|███████▎  | 13587/18627 [7:35:56<2:32:51,  1.82s/it]
 73%|███████▎  | 13588/18627 [7:35:58<2:40:13,  1.91s/it]
 73%|███████▎  | 13589/18627 [7:36:00<2:45:07,  1.97s/it]
 73%|███████▎  | 13590/18627 [7:36:01<2:23:21,  1.71s/it]
                                                         
{'loss': 1.0003, 'grad_norm': 14.014503479003906, 'learning_rate': 8.990697884128696e-07, 'epoch': 0.73}
+
 73%|███████▎  | 13590/18627 [7:36:01<2:23:21,  1.71s/it]
 73%|███████▎  | 13591/18627 [7:36:03<2:10:17,  1.55s/it]
 73%|███████▎  | 13592/18627 [7:36:05<2:24:11,  1.72s/it]
 73%|███████▎  | 13593/18627 [7:36:07<2:33:37,  1.83s/it]
 73%|███████▎  | 13594/18627 [7:36:09<2:40:50,  1.92s/it]
 73%|███████▎  | 13595/18627 [7:36:10<2:22:24,  1.70s/it]
 73%|███████▎  | 13596/18627 [7:36:11<2:09:42,  1.55s/it]
 73%|███████▎  | 13597/18627 [7:36:13<2:23:55,  1.72s/it]
 73%|███████▎  | 13598/18627 [7:36:16<2:34:13,  1.84s/it]
 73%|███████▎  | 13599/18627 [7:36:17<2:17:42,  1.64s/it]
 73%|███████▎  | 13600/18627 [7:36:19<2:29:19,  1.78s/it]
                                                         
{'loss': 1.3623, 'grad_norm': 7.102997303009033, 'learning_rate': 8.957335079395446e-07, 'epoch': 0.73}
+
 73%|███████▎  | 13600/18627 [7:36:19<2:29:19,  1.78s/it]
 73%|███████▎  | 13601/18627 [7:36:21<2:37:57,  1.89s/it]
 73%|███████▎  | 13602/18627 [7:36:22<2:20:16,  1.67s/it]
 73%|███████▎  | 13603/18627 [7:36:24<2:31:36,  1.81s/it]
 73%|███████▎  | 13604/18627 [7:36:26<2:39:33,  1.91s/it]
 73%|███████▎  | 13605/18627 [7:36:28<2:21:43,  1.69s/it]
 73%|███████▎  | 13606/18627 [7:36:30<2:32:18,  1.82s/it]
 73%|███████▎  | 13607/18627 [7:36:32<2:39:30,  1.91s/it]
 73%|███████▎  | 13608/18627 [7:36:34<2:44:39,  1.97s/it]
 73%|███████▎  | 13609/18627 [7:36:36<2:48:08,  2.01s/it]
 73%|███████▎  | 13610/18627 [7:36:38<2:51:21,  2.05s/it]
                                                         
{'loss': 1.0217, 'grad_norm': 6.331473350524902, 'learning_rate': 8.924020776253179e-07, 'epoch': 0.73}
+
 73%|███████▎  | 13610/18627 [7:36:38<2:51:21,  2.05s/it]
 73%|███████▎  | 13611/18627 [7:36:40<2:53:06,  2.07s/it]
 73%|███████▎  | 13612/18627 [7:36:41<2:31:04,  1.81s/it]
 73%|███████▎  | 13613/18627 [7:36:43<2:15:47,  1.62s/it]
 73%|███████▎  | 13614/18627 [7:36:45<2:28:32,  1.78s/it]
 73%|███████▎  | 13615/18627 [7:36:47<2:37:22,  1.88s/it]
 73%|███████▎  | 13616/18627 [7:36:48<2:19:49,  1.67s/it]
 73%|███████▎  | 13617/18627 [7:36:50<2:31:23,  1.81s/it]
 73%|███████▎  | 13618/18627 [7:36:52<2:39:09,  1.91s/it]
 73%|███████▎  | 13619/18627 [7:36:55<2:45:51,  1.99s/it]
 73%|███████▎  | 13620/18627 [7:36:55<2:16:52,  1.64s/it]
                                                         
{'loss': 1.2782, 'grad_norm': 6.434484481811523, 'learning_rate': 8.890755075420587e-07, 'epoch': 0.73}
+
 73%|███████▎  | 13620/18627 [7:36:55<2:16:52,  1.64s/it]
 73%|███████▎  | 13621/18627 [7:36:57<2:05:58,  1.51s/it]
 73%|███████▎  | 13622/18627 [7:36:59<2:22:23,  1.71s/it]
 73%|███████▎  | 13623/18627 [7:37:01<2:34:25,  1.85s/it]
 73%|███████▎  | 13624/18627 [7:37:03<2:41:27,  1.94s/it]
 73%|███████▎  | 13625/18627 [7:37:04<2:22:55,  1.71s/it]
 73%|███████▎  | 13626/18627 [7:37:06<2:32:53,  1.83s/it]
 73%|███████▎  | 13627/18627 [7:37:08<2:39:45,  1.92s/it]
 73%|███████▎  | 13628/18627 [7:37:11<2:44:44,  1.98s/it]
 73%|███████▎  | 13629/18627 [7:37:13<2:48:01,  2.02s/it]
 73%|███████▎  | 13630/18627 [7:37:15<2:50:57,  2.05s/it]
                                                         
{'loss': 1.0443, 'grad_norm': 5.173196792602539, 'learning_rate': 8.857538077469491e-07, 'epoch': 0.73}
+
 73%|███████▎  | 13630/18627 [7:37:15<2:50:57,  2.05s/it]
 73%|███████▎  | 13631/18627 [7:37:16<2:29:32,  1.80s/it]
 73%|███████▎  | 13632/18627 [7:37:18<2:37:32,  1.89s/it]
 73%|███████▎  | 13633/18627 [7:37:20<2:43:07,  1.96s/it]
 73%|███████▎  | 13634/18627 [7:37:21<2:11:37,  1.58s/it]
 73%|███████▎  | 13635/18627 [7:37:23<2:25:06,  1.74s/it]
 73%|███████▎  | 13636/18627 [7:37:25<2:34:11,  1.85s/it]
 73%|███████▎  | 13637/18627 [7:37:27<2:40:49,  1.93s/it]
 73%|███████▎  | 13638/18627 [7:37:29<2:22:18,  1.71s/it]
 73%|███████▎  | 13639/18627 [7:37:31<2:32:09,  1.83s/it]
 73%|███████▎  | 13640/18627 [7:37:32<2:15:57,  1.64s/it]
                                                         
{'loss': 1.2161, 'grad_norm': 14.574265480041504, 'learning_rate': 8.824369882824404e-07, 'epoch': 0.73}
+
 73%|███████▎  | 13640/18627 [7:37:32<2:15:57,  1.64s/it]
 73%|███████▎  | 13641/18627 [7:37:33<2:04:55,  1.50s/it]
 73%|███████▎  | 13642/18627 [7:37:35<2:20:40,  1.69s/it]
 73%|███████▎  | 13643/18627 [7:37:36<2:08:05,  1.54s/it]
 73%|███████▎  | 13644/18627 [7:37:37<1:47:15,  1.29s/it]
 73%|███████▎  | 13645/18627 [7:37:39<2:07:23,  1.53s/it]
 73%|███████▎  | 13646/18627 [7:37:40<1:59:02,  1.43s/it]
 73%|███████▎  | 13647/18627 [7:37:42<2:15:46,  1.64s/it]
 73%|███████▎  | 13648/18627 [7:37:45<2:27:50,  1.78s/it]
 73%|███████▎  | 13649/18627 [7:37:47<2:36:13,  1.88s/it]
 73%|███████▎  | 13650/18627 [7:37:49<2:41:57,  1.95s/it]
                                                         
{'loss': 1.2387, 'grad_norm': 5.777060031890869, 'learning_rate': 8.791250591762332e-07, 'epoch': 0.73}
+
 73%|███████▎  | 13650/18627 [7:37:49<2:41:57,  1.95s/it]
 73%|███████▎  | 13651/18627 [7:37:51<2:46:10,  2.00s/it]
 73%|███████▎  | 13652/18627 [7:37:53<2:49:10,  2.04s/it]
 73%|███████▎  | 13653/18627 [7:37:55<2:51:29,  2.07s/it]
 73%|███████▎  | 13654/18627 [7:37:57<2:52:39,  2.08s/it]
 73%|███████▎  | 13655/18627 [7:37:59<2:30:30,  1.82s/it]
 73%|███████▎  | 13656/18627 [7:38:01<2:38:10,  1.91s/it]
 73%|███████▎  | 13657/18627 [7:38:03<2:43:32,  1.97s/it]
 73%|███████▎  | 13658/18627 [7:38:05<2:46:50,  2.01s/it]
 73%|███████▎  | 13659/18627 [7:38:07<2:49:20,  2.05s/it]
 73%|███████▎  | 13660/18627 [7:38:09<2:50:45,  2.06s/it]
                                                         
{'loss': 0.8216, 'grad_norm': 6.020390510559082, 'learning_rate': 8.758180304412414e-07, 'epoch': 0.73}
+
 73%|███████▎  | 13660/18627 [7:38:09<2:50:45,  2.06s/it]
 73%|███████▎  | 13661/18627 [7:38:11<2:51:48,  2.08s/it]
 73%|███████▎  | 13662/18627 [7:38:13<2:54:08,  2.10s/it]
 73%|███████▎  | 13663/18627 [7:38:16<2:54:52,  2.11s/it]
 73%|███████▎  | 13664/18627 [7:38:18<2:56:04,  2.13s/it]
 73%|███████▎  | 13665/18627 [7:38:19<2:33:00,  1.85s/it]
 73%|███████▎  | 13666/18627 [7:38:21<2:40:37,  1.94s/it]
 73%|███████▎  | 13667/18627 [7:38:23<2:44:40,  1.99s/it]
 73%|███████▎  | 13668/18627 [7:38:25<2:47:36,  2.03s/it]
 73%|███████▎  | 13669/18627 [7:38:27<2:49:39,  2.05s/it]
 73%|███████▎  | 13670/18627 [7:38:29<2:51:02,  2.07s/it]
                                                         
{'loss': 0.7959, 'grad_norm': 8.963418960571289, 'learning_rate': 8.725159120755647e-07, 'epoch': 0.73}
+
 73%|███████▎  | 13670/18627 [7:38:29<2:51:02,  2.07s/it]
 73%|███████▎  | 13671/18627 [7:38:32<2:52:38,  2.09s/it]
 73%|███████▎  | 13672/18627 [7:38:33<2:30:26,  1.82s/it]
 73%|███████▎  | 13673/18627 [7:38:35<2:37:14,  1.90s/it]
 73%|███████▎  | 13674/18627 [7:38:36<2:19:30,  1.69s/it]
 73%|███████▎  | 13675/18627 [7:38:38<2:30:11,  1.82s/it]
 73%|███████▎  | 13676/18627 [7:38:40<2:37:58,  1.91s/it]
 73%|███████▎  | 13677/18627 [7:38:42<2:20:08,  1.70s/it]
 73%|███████▎  | 13678/18627 [7:38:44<2:30:20,  1.82s/it]
 73%|███████▎  | 13679/18627 [7:38:46<2:37:39,  1.91s/it]
 73%|███████▎  | 13680/18627 [7:38:48<2:42:41,  1.97s/it]
                                                         
{'loss': 1.3077, 'grad_norm': 9.20470905303955, 'learning_rate': 8.692187140624564e-07, 'epoch': 0.73}
+
 73%|███████▎  | 13680/18627 [7:38:48<2:42:41,  1.97s/it]
 73%|███████▎  | 13681/18627 [7:38:50<2:46:27,  2.02s/it]
 73%|███████▎  | 13682/18627 [7:38:52<2:48:19,  2.04s/it]
 73%|███████▎  | 13683/18627 [7:38:54<2:50:16,  2.07s/it]
 73%|███████▎  | 13684/18627 [7:38:56<2:52:34,  2.09s/it]
 73%|███████▎  | 13685/18627 [7:38:59<2:54:00,  2.11s/it]
 73%|███████▎  | 13686/18627 [7:39:00<2:31:31,  1.84s/it]
 73%|███████▎  | 13687/18627 [7:39:02<2:38:40,  1.93s/it]
 73%|███████▎  | 13688/18627 [7:39:04<2:44:21,  2.00s/it]
 73%|███████▎  | 13689/18627 [7:39:05<2:12:34,  1.61s/it]
 73%|███████▎  | 13690/18627 [7:39:07<2:25:38,  1.77s/it]
                                                         
{'loss': 0.7919, 'grad_norm': 7.884720802307129, 'learning_rate': 8.659264463702954e-07, 'epoch': 0.73}
+
 73%|███████▎  | 13690/18627 [7:39:07<2:25:38,  1.77s/it]
 74%|███████▎  | 13691/18627 [7:39:09<2:39:00,  1.93s/it]
 74%|███████▎  | 13692/18627 [7:39:12<2:48:15,  2.05s/it]
 74%|███████▎  | 13693/18627 [7:39:14<2:56:09,  2.14s/it]
 74%|███████▎  | 13694/18627 [7:39:15<2:37:54,  1.92s/it]
 74%|███████▎  | 13695/18627 [7:39:18<2:48:40,  2.05s/it]
 74%|███████▎  | 13696/18627 [7:39:20<2:56:52,  2.15s/it]
 74%|███████▎  | 13697/18627 [7:39:21<2:38:42,  1.93s/it]
 74%|███████▎  | 13698/18627 [7:39:24<2:48:47,  2.05s/it]
 74%|███████▎  | 13699/18627 [7:39:26<2:56:11,  2.15s/it]
 74%|███████▎  | 13700/18627 [7:39:28<2:39:01,  1.94s/it]
                                                         
{'loss': 1.2739, 'grad_norm': 14.757783889770508, 'learning_rate': 8.626391189525518e-07, 'epoch': 0.74}
+
 74%|███████▎  | 13700/18627 [7:39:28<2:39:01,  1.94s/it]
 74%|███████▎  | 13701/18627 [7:39:30<2:48:26,  2.05s/it]
 74%|███████▎  | 13702/18627 [7:39:32<2:55:34,  2.14s/it]
 74%|███████▎  | 13703/18627 [7:39:34<2:38:22,  1.93s/it]
 74%|███████▎  | 13704/18627 [7:39:36<2:49:14,  2.06s/it]
 74%|███████▎  | 13705/18627 [7:39:38<2:56:02,  2.15s/it]
 74%|███████▎  | 13706/18627 [7:39:40<2:34:26,  1.88s/it]
 74%|███████▎  | 13707/18627 [7:39:42<2:45:41,  2.02s/it]
 74%|███████▎  | 13708/18627 [7:39:44<2:54:04,  2.12s/it]
 74%|███████▎  | 13709/18627 [7:39:47<3:00:32,  2.20s/it]
 74%|███████▎  | 13710/18627 [7:39:49<3:03:34,  2.24s/it]
                                                         
{'loss': 1.0657, 'grad_norm': 7.042613983154297, 'learning_rate': 8.593567417477652e-07, 'epoch': 0.74}
+
 74%|███████▎  | 13710/18627 [7:39:49<3:03:34,  2.24s/it]
 74%|███████▎  | 13711/18627 [7:39:51<3:05:36,  2.27s/it]
 74%|███████▎  | 13712/18627 [7:39:53<2:44:30,  2.01s/it]
 74%|███████▎  | 13713/18627 [7:39:54<2:29:21,  1.82s/it]
 74%|███████▎  | 13714/18627 [7:39:57<2:42:10,  1.98s/it]
 74%|███████▎  | 13715/18627 [7:39:58<2:28:37,  1.82s/it]
 74%|███████▎  | 13716/18627 [7:39:59<2:20:30,  1.72s/it]
 74%|███████▎  | 13717/18627 [7:40:02<2:37:03,  1.92s/it]
 74%|███████▎  | 13718/18627 [7:40:03<2:25:36,  1.78s/it]
 74%|███████▎  | 13719/18627 [7:40:05<2:15:08,  1.65s/it]
 74%|███████▎  | 13720/18627 [7:40:06<2:02:13,  1.49s/it]
                                                         
{'loss': 1.8561, 'grad_norm': 6.077809810638428, 'learning_rate': 8.560793246795038e-07, 'epoch': 0.74}
+
 74%|███████▎  | 13720/18627 [7:40:06<2:02:13,  1.49s/it]
 74%|███████▎  | 13721/18627 [7:40:08<2:23:25,  1.75s/it]
 74%|███████▎  | 13722/18627 [7:40:10<2:15:23,  1.66s/it]
 74%|███████▎  | 13723/18627 [7:40:11<2:10:37,  1.60s/it]
 74%|███████▎  | 13724/18627 [7:40:13<2:29:10,  1.83s/it]
 74%|███████▎  | 13725/18627 [7:40:15<2:11:12,  1.61s/it]
 74%|███████▎  | 13726/18627 [7:40:17<2:30:25,  1.84s/it]
 74%|███████▎  | 13727/18627 [7:40:19<2:42:03,  1.98s/it]
 74%|███████▎  | 13728/18627 [7:40:22<2:50:59,  2.09s/it]
 74%|███████▎  | 13729/18627 [7:40:24<2:57:09,  2.17s/it]
 74%|███████▎  | 13730/18627 [7:40:26<3:00:48,  2.22s/it]
                                                         
{'loss': 0.9913, 'grad_norm': 6.676036357879639, 'learning_rate': 8.528068776563425e-07, 'epoch': 0.74}
+
 74%|███████▎  | 13730/18627 [7:40:26<3:00:48,  2.22s/it]
 74%|███████▎  | 13731/18627 [7:40:29<3:04:14,  2.26s/it]
 74%|███████▎  | 13732/18627 [7:40:31<3:06:12,  2.28s/it]
 74%|███████▎  | 13733/18627 [7:40:33<3:06:54,  2.29s/it]
 74%|███████▎  | 13734/18627 [7:40:36<3:07:56,  2.30s/it]
 74%|███████▎  | 13735/18627 [7:40:38<3:08:16,  2.31s/it]
 74%|███████▎  | 13736/18627 [7:40:39<2:38:42,  1.95s/it]
 74%|███████▎  | 13737/18627 [7:40:41<2:48:27,  2.07s/it]
 74%|███████▍  | 13738/18627 [7:40:44<2:55:20,  2.15s/it]
 74%|███████▍  | 13739/18627 [7:40:46<2:59:44,  2.21s/it]
 74%|███████▍  | 13740/18627 [7:40:48<3:03:15,  2.25s/it]
                                                         
{'loss': 0.6377, 'grad_norm': 6.183002471923828, 'learning_rate': 8.495394105718308e-07, 'epoch': 0.74}
+
 74%|███████▍  | 13740/18627 [7:40:48<3:03:15,  2.25s/it]
 74%|███████▍  | 13741/18627 [7:40:51<3:05:41,  2.28s/it]
 74%|███████▍  | 13742/18627 [7:40:53<3:07:38,  2.30s/it]
 74%|███████▍  | 13743/18627 [7:40:55<3:08:48,  2.32s/it]
 74%|███████▍  | 13744/18627 [7:40:58<3:09:14,  2.33s/it]
 74%|███████▍  | 13745/18627 [7:40:59<2:45:42,  2.04s/it]
 74%|███████▍  | 13746/18627 [7:41:02<2:53:10,  2.13s/it]
 74%|███████▍  | 13747/18627 [7:41:03<2:35:19,  1.91s/it]
 74%|███████▍  | 13748/18627 [7:41:05<2:46:15,  2.04s/it]
 74%|███████▍  | 13749/18627 [7:41:08<2:53:47,  2.14s/it]
 74%|███████▍  | 13750/18627 [7:41:10<2:58:22,  2.19s/it]
                                                         
{'loss': 0.9959, 'grad_norm': 6.67649507522583, 'learning_rate': 8.462769333044609e-07, 'epoch': 0.74}
+
 74%|███████▍  | 13750/18627 [7:41:10<2:58:22,  2.19s/it]
 74%|███████▍  | 13751/18627 [7:41:12<3:01:47,  2.24s/it]
 74%|███████▍  | 13752/18627 [7:41:15<3:03:49,  2.26s/it]
 74%|███████▍  | 13753/18627 [7:41:16<2:41:16,  1.99s/it]
 74%|███████▍  | 13754/18627 [7:41:17<2:25:13,  1.79s/it]
 74%|███████▍  | 13755/18627 [7:41:20<2:39:27,  1.96s/it]
 74%|███████▍  | 13756/18627 [7:41:22<2:48:12,  2.07s/it]
 74%|███████▍  | 13757/18627 [7:41:24<2:54:08,  2.15s/it]
 74%|███████▍  | 13758/18627 [7:41:27<2:58:01,  2.19s/it]
 74%|███████▍  | 13759/18627 [7:41:29<3:01:10,  2.23s/it]
 74%|███████▍  | 13760/18627 [7:41:31<3:03:10,  2.26s/it]
                                                         
{'loss': 0.9962, 'grad_norm': 6.807796001434326, 'learning_rate': 8.430194557176408e-07, 'epoch': 0.74}
+
 74%|███████▍  | 13760/18627 [7:41:31<3:03:10,  2.26s/it]
 74%|███████▍  | 13761/18627 [7:41:34<3:04:50,  2.28s/it]
 74%|███████▍  | 13762/18627 [7:41:36<3:05:55,  2.29s/it]
 74%|███████▍  | 13763/18627 [7:41:38<3:07:16,  2.31s/it]
 74%|███████▍  | 13764/18627 [7:41:41<3:08:03,  2.32s/it]
 74%|███████▍  | 13765/18627 [7:41:42<2:44:00,  2.02s/it]
 74%|███████▍  | 13766/18627 [7:41:44<2:52:19,  2.13s/it]
 74%|███████▍  | 13767/18627 [7:41:47<2:56:41,  2.18s/it]
 74%|███████▍  | 13768/18627 [7:41:49<2:59:57,  2.22s/it]
 74%|███████▍  | 13769/18627 [7:41:51<3:02:05,  2.25s/it]
 74%|███████▍  | 13770/18627 [7:41:54<3:03:05,  2.26s/it]
                                                         
{'loss': 0.8119, 'grad_norm': 8.101261138916016, 'learning_rate': 8.397669876596623e-07, 'epoch': 0.74}
+
 74%|███████▍  | 13770/18627 [7:41:54<3:03:05,  2.26s/it]
 74%|███████▍  | 13771/18627 [7:41:56<3:05:21,  2.29s/it]
 74%|███████▍  | 13772/18627 [7:41:57<2:43:13,  2.02s/it]
 74%|███████▍  | 13773/18627 [7:42:00<2:50:27,  2.11s/it]
 74%|███████▍  | 13774/18627 [7:42:01<2:33:13,  1.89s/it]
 74%|███████▍  | 13775/18627 [7:42:03<2:44:39,  2.04s/it]
 74%|███████▍  | 13776/18627 [7:42:05<2:29:15,  1.85s/it]
 74%|███████▍  | 13777/18627 [7:42:07<2:41:51,  2.00s/it]
 74%|███████▍  | 13778/18627 [7:42:08<2:26:48,  1.82s/it]
 74%|███████▍  | 13779/18627 [7:42:11<2:39:17,  1.97s/it]
 74%|███████▍  | 13780/18627 [7:42:12<2:25:22,  1.80s/it]
                                                         
{'loss': 1.583, 'grad_norm': 13.647658348083496, 'learning_rate': 8.365195389636701e-07, 'epoch': 0.74}
+
 74%|███████▍  | 13780/18627 [7:42:12<2:25:22,  1.80s/it]
 74%|███████▍  | 13781/18627 [7:42:15<2:38:14,  1.96s/it]
 74%|███████▍  | 13782/18627 [7:42:17<2:48:25,  2.09s/it]
 74%|███████▍  | 13783/18627 [7:42:19<2:55:58,  2.18s/it]
 74%|███████▍  | 13784/18627 [7:42:22<3:00:04,  2.23s/it]
 74%|███████▍  | 13785/18627 [7:42:23<2:39:10,  1.97s/it]
 74%|███████▍  | 13786/18627 [7:42:25<2:47:45,  2.08s/it]
 74%|███████▍  | 13787/18627 [7:42:28<2:54:20,  2.16s/it]
 74%|███████▍  | 13788/18627 [7:42:30<2:58:42,  2.22s/it]
 74%|███████▍  | 13789/18627 [7:42:32<3:00:59,  2.24s/it]
 74%|███████▍  | 13790/18627 [7:42:34<2:39:50,  1.98s/it]
                                                         
{'loss': 0.9692, 'grad_norm': 14.911652565002441, 'learning_rate': 8.332771194476392e-07, 'epoch': 0.74}
+
 74%|███████▍  | 13790/18627 [7:42:34<2:39:50,  1.98s/it]
 74%|███████▍  | 13791/18627 [7:42:35<2:24:44,  1.80s/it]
 74%|███████▍  | 13792/18627 [7:42:37<2:37:49,  1.96s/it]
 74%|███████▍  | 13793/18627 [7:42:40<2:46:34,  2.07s/it]
 74%|███████▍  | 13794/18627 [7:42:42<2:53:07,  2.15s/it]
 74%|███████▍  | 13795/18627 [7:42:44<2:57:39,  2.21s/it]
 74%|███████▍  | 13796/18627 [7:42:46<2:37:32,  1.96s/it]
 74%|███████▍  | 13797/18627 [7:42:47<2:23:32,  1.78s/it]
 74%|███████▍  | 13798/18627 [7:42:50<2:36:41,  1.95s/it]
 74%|███████▍  | 13799/18627 [7:42:52<2:46:00,  2.06s/it]
 74%|███████▍  | 13800/18627 [7:42:54<2:52:20,  2.14s/it]
                                                         
{'loss': 1.1375, 'grad_norm': 4.764041423797607, 'learning_rate': 8.300397389143334e-07, 'epoch': 0.74}
+
 74%|███████▍  | 13800/18627 [7:42:54<2:52:20,  2.14s/it]
 74%|███████▍  | 13801/18627 [7:42:56<2:33:19,  1.91s/it]
 74%|███████▍  | 13802/18627 [7:42:58<2:43:39,  2.04s/it]
 74%|███████▍  | 13803/18627 [7:43:00<2:50:56,  2.13s/it]
 74%|███████▍  | 13804/18627 [7:43:03<2:55:40,  2.19s/it]
 74%|███████▍  | 13805/18627 [7:43:04<2:28:39,  1.85s/it]
 74%|███████▍  | 13806/18627 [7:43:05<2:17:26,  1.71s/it]
 74%|███████▍  | 13807/18627 [7:43:07<2:32:04,  1.89s/it]
 74%|███████▍  | 13808/18627 [7:43:10<2:42:29,  2.02s/it]
 74%|███████▍  | 13809/18627 [7:43:11<2:26:51,  1.83s/it]
 74%|███████▍  | 13810/18627 [7:43:13<2:39:24,  1.99s/it]
                                                         
{'loss': 1.2378, 'grad_norm': 5.9352192878723145, 'learning_rate': 8.268074071512866e-07, 'epoch': 0.74}
+
 74%|███████▍  | 13810/18627 [7:43:13<2:39:24,  1.99s/it]
 74%|███████▍  | 13811/18627 [7:43:16<2:48:03,  2.09s/it]
 74%|███████▍  | 13812/18627 [7:43:17<2:30:55,  1.88s/it]
 74%|███████▍  | 13813/18627 [7:43:19<2:42:02,  2.02s/it]
 74%|███████▍  | 13814/18627 [7:43:22<2:49:06,  2.11s/it]
 74%|███████▍  | 13815/18627 [7:43:24<2:54:25,  2.17s/it]
 74%|███████▍  | 13816/18627 [7:43:26<2:58:05,  2.22s/it]
 74%|███████▍  | 13817/18627 [7:43:27<2:29:54,  1.87s/it]
 74%|███████▍  | 13818/18627 [7:43:30<2:41:01,  2.01s/it]
 74%|███████▍  | 13819/18627 [7:43:31<2:26:05,  1.82s/it]
 74%|███████▍  | 13820/18627 [7:43:35<3:10:13,  2.37s/it]
                                                         
{'loss': 1.0275, 'grad_norm': 9.963265419006348, 'learning_rate': 8.235801339307675e-07, 'epoch': 0.74}
+
 74%|███████▍  | 13820/18627 [7:43:35<3:10:13,  2.37s/it]
 74%|███████▍  | 13821/18627 [7:43:37<3:09:15,  2.36s/it]
 74%|███████▍  | 13822/18627 [7:43:40<3:08:41,  2.36s/it]
 74%|███████▍  | 13823/18627 [7:43:42<3:07:43,  2.34s/it]
 74%|███████▍  | 13824/18627 [7:43:44<3:08:46,  2.36s/it]
 74%|███████▍  | 13825/18627 [7:43:47<3:08:21,  2.35s/it]
 74%|███████▍  | 13826/18627 [7:43:49<3:09:46,  2.37s/it]
 74%|███████▍  | 13827/18627 [7:43:50<2:46:24,  2.08s/it]
 74%|███████▍  | 13828/18627 [7:43:53<2:51:59,  2.15s/it]
 74%|███████▍  | 13829/18627 [7:43:55<2:56:47,  2.21s/it]
 74%|███████▍  | 13830/18627 [7:43:57<2:58:44,  2.24s/it]
                                                         
{'loss': 0.8908, 'grad_norm': 7.309261322021484, 'learning_rate': 8.203579290097515e-07, 'epoch': 0.74}
+
 74%|███████▍  | 13830/18627 [7:43:57<2:58:44,  2.24s/it]
 74%|███████▍  | 13831/18627 [7:44:00<3:01:05,  2.27s/it]
 74%|███████▍  | 13832/18627 [7:44:01<2:39:11,  1.99s/it]
 74%|███████▍  | 13833/18627 [7:44:03<2:48:28,  2.11s/it]
 74%|███████▍  | 13834/18627 [7:44:06<2:53:18,  2.17s/it]
 74%|███████▍  | 13835/18627 [7:44:07<2:33:44,  1.93s/it]
 74%|███████▍  | 13836/18627 [7:44:09<2:43:04,  2.04s/it]
 74%|███████▍  | 13837/18627 [7:44:12<2:49:29,  2.12s/it]
 74%|███████▍  | 13838/18627 [7:44:14<2:53:48,  2.18s/it]
 74%|███████▍  | 13839/18627 [7:44:16<2:58:12,  2.23s/it]
 74%|███████▍  | 13840/18627 [7:44:19<3:00:38,  2.26s/it]
                                                         
{'loss': 1.0339, 'grad_norm': 7.186404705047607, 'learning_rate': 8.171408021298907e-07, 'epoch': 0.74}
+
 74%|███████▍  | 13840/18627 [7:44:19<3:00:38,  2.26s/it]
 74%|███████▍  | 13841/18627 [7:44:21<3:02:23,  2.29s/it]
 74%|███████▍  | 13842/18627 [7:44:22<2:40:08,  2.01s/it]
 74%|███████▍  | 13843/18627 [7:44:25<2:48:05,  2.11s/it]
 74%|███████▍  | 13844/18627 [7:44:27<2:53:37,  2.18s/it]
 74%|███████▍  | 13845/18627 [7:44:29<2:57:13,  2.22s/it]
 74%|███████▍  | 13846/18627 [7:44:32<3:00:18,  2.26s/it]
 74%|███████▍  | 13847/18627 [7:44:34<3:02:36,  2.29s/it]
 74%|███████▍  | 13848/18627 [7:44:35<2:40:09,  2.01s/it]
 74%|███████▍  | 13849/18627 [7:44:38<2:47:46,  2.11s/it]
 74%|███████▍  | 13850/18627 [7:44:40<2:54:02,  2.19s/it]
                                                         
{'loss': 0.9473, 'grad_norm': 5.620382785797119, 'learning_rate': 8.139287630174855e-07, 'epoch': 0.74}
+
 74%|███████▍  | 13850/18627 [7:44:40<2:54:02,  2.19s/it]
 74%|███████▍  | 13851/18627 [7:44:42<2:34:15,  1.94s/it]
 74%|███████▍  | 13852/18627 [7:44:44<2:44:17,  2.06s/it]
 74%|███████▍  | 13853/18627 [7:44:46<2:51:01,  2.15s/it]
 74%|███████▍  | 13854/18627 [7:44:48<2:33:25,  1.93s/it]
 74%|███████▍  | 13855/18627 [7:44:50<2:44:13,  2.06s/it]
 74%|███████▍  | 13856/18627 [7:44:52<2:50:51,  2.15s/it]
 74%|███████▍  | 13857/18627 [7:44:55<2:55:10,  2.20s/it]
 74%|███████▍  | 13858/18627 [7:44:56<2:35:24,  1.96s/it]
 74%|███████▍  | 13859/18627 [7:44:58<2:44:27,  2.07s/it]
 74%|███████▍  | 13860/18627 [7:45:01<2:51:06,  2.15s/it]
                                                         
{'loss': 1.1954, 'grad_norm': 5.416253566741943, 'learning_rate': 8.107218213834539e-07, 'epoch': 0.74}
+
 74%|███████▍  | 13860/18627 [7:45:01<2:51:06,  2.15s/it]
 74%|███████▍  | 13861/18627 [7:45:03<2:54:49,  2.20s/it]
 74%|███████▍  | 13862/18627 [7:45:04<2:34:03,  1.94s/it]
 74%|███████▍  | 13863/18627 [7:45:07<2:43:19,  2.06s/it]
 74%|███████▍  | 13864/18627 [7:45:08<2:27:20,  1.86s/it]
 74%|███████▍  | 13865/18627 [7:45:11<2:38:50,  2.00s/it]
 74%|███████▍  | 13866/18627 [7:45:13<2:45:47,  2.09s/it]
 74%|███████▍  | 13867/18627 [7:45:15<2:51:00,  2.16s/it]
 74%|███████▍  | 13868/18627 [7:45:17<2:54:40,  2.20s/it]
 74%|███████▍  | 13869/18627 [7:45:19<2:34:22,  1.95s/it]
 74%|███████▍  | 13870/18627 [7:45:21<2:43:40,  2.06s/it]
                                                         
{'loss': 1.2095, 'grad_norm': 5.165733814239502, 'learning_rate': 8.07519986923303e-07, 'epoch': 0.74}
+
 74%|███████▍  | 13870/18627 [7:45:21<2:43:40,  2.06s/it]
 74%|███████▍  | 13871/18627 [7:45:22<2:26:51,  1.85s/it]
 74%|███████▍  | 13872/18627 [7:45:25<2:38:22,  2.00s/it]
 74%|███████▍  | 13873/18627 [7:45:26<2:23:55,  1.82s/it]
 74%|███████▍  | 13874/18627 [7:45:29<2:36:26,  1.97s/it]
 74%|███████▍  | 13875/18627 [7:45:31<2:44:35,  2.08s/it]
 74%|███████▍  | 13876/18627 [7:45:33<2:50:17,  2.15s/it]
 74%|███████▍  | 13877/18627 [7:45:36<2:54:52,  2.21s/it]
 75%|███████▍  | 13878/18627 [7:45:37<2:33:03,  1.93s/it]
 75%|███████▍  | 13879/18627 [7:45:39<2:42:43,  2.06s/it]
 75%|███████▍  | 13880/18627 [7:45:41<2:49:18,  2.14s/it]
                                                         
{'loss': 1.0998, 'grad_norm': 5.741585731506348, 'learning_rate': 8.043232693171002e-07, 'epoch': 0.75}
+
 75%|███████▍  | 13880/18627 [7:45:41<2:49:18,  2.14s/it]
 75%|███████▍  | 13881/18627 [7:45:44<2:53:40,  2.20s/it]
 75%|███████▍  | 13882/18627 [7:45:46<2:57:26,  2.24s/it]
 75%|███████▍  | 13883/18627 [7:45:48<2:59:01,  2.26s/it]
 75%|███████▍  | 13884/18627 [7:45:51<2:59:46,  2.27s/it]
 75%|███████▍  | 13885/18627 [7:45:52<2:38:01,  2.00s/it]
 75%|███████▍  | 13886/18627 [7:45:54<2:45:40,  2.10s/it]
 75%|███████▍  | 13887/18627 [7:45:57<2:50:49,  2.16s/it]
 75%|███████▍  | 13888/18627 [7:45:59<2:55:05,  2.22s/it]
 75%|███████▍  | 13889/18627 [7:46:01<2:58:32,  2.26s/it]
 75%|███████▍  | 13890/18627 [7:46:04<2:59:50,  2.28s/it]
                                                         
{'loss': 0.7956, 'grad_norm': 5.891583442687988, 'learning_rate': 8.011316782294401e-07, 'epoch': 0.75}
+
 75%|███████▍  | 13890/18627 [7:46:04<2:59:50,  2.28s/it]
 75%|███████▍  | 13891/18627 [7:46:06<3:00:12,  2.28s/it]
 75%|███████▍  | 13892/18627 [7:46:08<3:01:24,  2.30s/it]
 75%|███████▍  | 13893/18627 [7:46:11<3:02:16,  2.31s/it]
 75%|███████▍  | 13894/18627 [7:46:12<2:39:46,  2.03s/it]
 75%|███████▍  | 13895/18627 [7:46:14<2:47:04,  2.12s/it]
 75%|███████▍  | 13896/18627 [7:46:16<2:28:05,  1.88s/it]
 75%|███████▍  | 13897/18627 [7:46:18<2:38:34,  2.01s/it]
 75%|███████▍  | 13898/18627 [7:46:20<2:45:52,  2.10s/it]
 75%|███████▍  | 13899/18627 [7:46:23<2:51:38,  2.18s/it]
 75%|███████▍  | 13900/18627 [7:46:24<2:33:08,  1.94s/it]
                                                         
{'loss': 1.1818, 'grad_norm': 11.913064956665039, 'learning_rate': 7.979452233094239e-07, 'epoch': 0.75}
+
 75%|███████▍  | 13900/18627 [7:46:24<2:33:08,  1.94s/it]
 75%|███████▍  | 13901/18627 [7:46:27<2:42:31,  2.06s/it]
 75%|███████▍  | 13902/18627 [7:46:29<2:48:41,  2.14s/it]
 75%|███████▍  | 13903/18627 [7:46:31<2:52:48,  2.19s/it]
 75%|███████▍  | 13904/18627 [7:46:33<2:55:44,  2.23s/it]
 75%|███████▍  | 13905/18627 [7:46:35<2:35:37,  1.98s/it]
 75%|███████▍  | 13906/18627 [7:46:37<2:43:44,  2.08s/it]
 75%|███████▍  | 13907/18627 [7:46:40<2:49:05,  2.15s/it]
 75%|███████▍  | 13908/18627 [7:46:42<2:53:23,  2.20s/it]
 75%|███████▍  | 13909/18627 [7:46:44<2:56:18,  2.24s/it]
 75%|███████▍  | 13910/18627 [7:46:46<2:36:41,  1.99s/it]
                                                         
{'loss': 1.0865, 'grad_norm': 13.882493019104004, 'learning_rate': 7.947639141906188e-07, 'epoch': 0.75}
+
 75%|███████▍  | 13910/18627 [7:46:46<2:36:41,  1.99s/it]
 75%|███████▍  | 13911/18627 [7:46:48<2:44:28,  2.09s/it]
 75%|███████▍  | 13912/18627 [7:46:49<2:24:31,  1.84s/it]
 75%|███████▍  | 13913/18627 [7:46:51<2:36:20,  1.99s/it]
 75%|███████▍  | 13914/18627 [7:46:54<2:43:46,  2.08s/it]
 75%|███████▍  | 13915/18627 [7:46:55<2:26:45,  1.87s/it]
 75%|███████▍  | 13916/18627 [7:46:57<2:37:23,  2.00s/it]
 75%|███████▍  | 13917/18627 [7:47:00<2:44:46,  2.10s/it]
 75%|███████▍  | 13918/18627 [7:47:01<2:27:42,  1.88s/it]
 75%|███████▍  | 13919/18627 [7:47:06<3:34:36,  2.73s/it]
 75%|███████▍  | 13920/18627 [7:47:08<3:25:21,  2.62s/it]
                                                         
{'loss': 1.1859, 'grad_norm': 9.192534446716309, 'learning_rate': 7.915877604910385e-07, 'epoch': 0.75}
+
 75%|███████▍  | 13920/18627 [7:47:08<3:25:21,  2.62s/it]
 75%|███████▍  | 13921/18627 [7:47:11<3:18:26,  2.53s/it]
 75%|███████▍  | 13922/18627 [7:47:13<3:12:56,  2.46s/it]
 75%|███████▍  | 13923/18627 [7:47:15<3:10:22,  2.43s/it]
 75%|███████▍  | 13924/18627 [7:47:18<3:07:58,  2.40s/it]
 75%|███████▍  | 13925/18627 [7:47:20<3:06:17,  2.38s/it]
 75%|███████▍  | 13926/18627 [7:47:22<3:04:56,  2.36s/it]
 75%|███████▍  | 13927/18627 [7:47:25<3:04:06,  2.35s/it]
 75%|███████▍  | 13928/18627 [7:47:27<3:03:31,  2.34s/it]
 75%|███████▍  | 13929/18627 [7:47:29<3:02:08,  2.33s/it]
 75%|███████▍  | 13930/18627 [7:47:31<3:01:42,  2.32s/it]
                                                         
{'loss': 0.6067, 'grad_norm': 8.157587051391602, 'learning_rate': 7.884167718131092e-07, 'epoch': 0.75}
+
 75%|███████▍  | 13930/18627 [7:47:31<3:01:42,  2.32s/it]
 75%|███████▍  | 13931/18627 [7:47:34<3:02:06,  2.33s/it]
 75%|███████▍  | 13932/18627 [7:47:36<3:02:34,  2.33s/it]
 75%|███████▍  | 13933/18627 [7:47:38<3:02:03,  2.33s/it]
 75%|███████▍  | 13934/18627 [7:47:40<2:39:23,  2.04s/it]
 75%|███████▍  | 13935/18627 [7:47:42<2:46:19,  2.13s/it]
 75%|███████▍  | 13936/18627 [7:47:44<2:51:28,  2.19s/it]
 75%|███████▍  | 13937/18627 [7:47:47<2:55:21,  2.24s/it]
 75%|███████▍  | 13938/18627 [7:47:48<2:34:25,  1.98s/it]
 75%|███████▍  | 13939/18627 [7:47:51<2:44:02,  2.10s/it]
 75%|███████▍  | 13940/18627 [7:47:53<2:49:33,  2.17s/it]
                                                         
{'loss': 1.0229, 'grad_norm': 5.338557720184326, 'learning_rate': 7.852509577436421e-07, 'epoch': 0.75}
+
 75%|███████▍  | 13940/18627 [7:47:53<2:49:33,  2.17s/it]
 75%|███████▍  | 13941/18627 [7:47:55<2:53:40,  2.22s/it]
 75%|███████▍  | 13942/18627 [7:47:57<2:33:46,  1.97s/it]
 75%|███████▍  | 13943/18627 [7:47:59<2:41:58,  2.07s/it]
 75%|███████▍  | 13944/18627 [7:48:01<2:47:57,  2.15s/it]
 75%|███████▍  | 13945/18627 [7:48:04<2:52:36,  2.21s/it]
 75%|███████▍  | 13946/18627 [7:48:06<2:54:58,  2.24s/it]
 75%|███████▍  | 13947/18627 [7:48:08<2:56:29,  2.26s/it]
 75%|███████▍  | 13948/18627 [7:48:11<2:58:34,  2.29s/it]
 75%|███████▍  | 13949/18627 [7:48:12<2:36:27,  2.01s/it]
 75%|███████▍  | 13950/18627 [7:48:14<2:45:05,  2.12s/it]
                                                         
{'loss': 1.0202, 'grad_norm': 5.674491882324219, 'learning_rate': 7.820903278538033e-07, 'epoch': 0.75}
+
 75%|███████▍  | 13950/18627 [7:48:14<2:45:05,  2.12s/it]
 75%|███████▍  | 13951/18627 [7:48:16<2:26:46,  1.88s/it]
 75%|███████▍  | 13952/18627 [7:48:18<2:36:58,  2.01s/it]
 75%|███████▍  | 13953/18627 [7:48:20<2:43:55,  2.10s/it]
 75%|███████▍  | 13954/18627 [7:48:23<2:48:22,  2.16s/it]
 75%|███████▍  | 13955/18627 [7:48:24<2:29:23,  1.92s/it]
 75%|███████▍  | 13956/18627 [7:48:26<2:38:47,  2.04s/it]
 75%|███████▍  | 13957/18627 [7:48:29<2:45:59,  2.13s/it]
 75%|███████▍  | 13958/18627 [7:48:30<2:28:02,  1.90s/it]
 75%|███████▍  | 13959/18627 [7:48:32<2:37:43,  2.03s/it]
 75%|███████▍  | 13960/18627 [7:48:35<2:44:25,  2.11s/it]
                                                         
{'loss': 1.2064, 'grad_norm': 5.5500874519348145, 'learning_rate': 7.789348916990871e-07, 'epoch': 0.75}
+
 75%|███████▍  | 13960/18627 [7:48:35<2:44:25,  2.11s/it]
 75%|███████▍  | 13961/18627 [7:48:36<2:27:22,  1.90s/it]
 75%|███████▍  | 13962/18627 [7:48:38<2:36:59,  2.02s/it]
 75%|███████▍  | 13963/18627 [7:48:41<2:44:13,  2.11s/it]
 75%|███████▍  | 13964/18627 [7:48:43<2:49:43,  2.18s/it]
 75%|███████▍  | 13965/18627 [7:48:44<2:31:21,  1.95s/it]
 75%|███████▍  | 13966/18627 [7:48:47<2:40:01,  2.06s/it]
 75%|███████▍  | 13967/18627 [7:48:49<2:45:45,  2.13s/it]
 75%|███████▍  | 13968/18627 [7:48:51<2:50:04,  2.19s/it]
 75%|███████▍  | 13969/18627 [7:48:53<2:30:23,  1.94s/it]
 75%|███████▍  | 13970/18627 [7:48:55<2:39:22,  2.05s/it]
                                                         
{'loss': 1.2032, 'grad_norm': 5.895168781280518, 'learning_rate': 7.757846588192828e-07, 'epoch': 0.75}
+
 75%|███████▍  | 13970/18627 [7:48:55<2:39:22,  2.05s/it]
 75%|███████▌  | 13971/18627 [7:48:56<2:21:34,  1.82s/it]
 75%|███████▌  | 13972/18627 [7:48:59<2:33:03,  1.97s/it]
 75%|███████▌  | 13973/18627 [7:49:01<2:41:06,  2.08s/it]
 75%|███████▌  | 13974/18627 [7:49:02<2:25:04,  1.87s/it]
 75%|███████▌  | 13975/18627 [7:49:04<2:14:18,  1.73s/it]
 75%|███████▌  | 13976/18627 [7:49:06<2:28:51,  1.92s/it]
 75%|███████▌  | 13977/18627 [7:49:08<2:38:58,  2.05s/it]
 75%|███████▌  | 13978/18627 [7:49:11<2:45:26,  2.14s/it]
 75%|███████▌  | 13979/18627 [7:49:13<2:49:34,  2.19s/it]
 75%|███████▌  | 13980/18627 [7:49:15<2:52:24,  2.23s/it]
                                                         
{'loss': 1.1921, 'grad_norm': 5.002926826477051, 'learning_rate': 7.726396387384533e-07, 'epoch': 0.75}
+
 75%|███████▌  | 13980/18627 [7:49:15<2:52:24,  2.23s/it]
 75%|███████▌  | 13981/18627 [7:49:18<2:54:35,  2.25s/it]
 75%|███████▌  | 13982/18627 [7:49:20<2:56:05,  2.27s/it]
 75%|███████▌  | 13983/18627 [7:49:22<2:57:05,  2.29s/it]
 75%|███████▌  | 13984/18627 [7:49:24<2:35:42,  2.01s/it]
 75%|███████▌  | 13985/18627 [7:49:26<2:42:45,  2.10s/it]
 75%|███████▌  | 13986/18627 [7:49:27<2:25:31,  1.88s/it]
 75%|███████▌  | 13987/18627 [7:49:29<2:13:51,  1.73s/it]
 75%|███████▌  | 13988/18627 [7:49:31<2:27:08,  1.90s/it]
 75%|███████▌  | 13989/18627 [7:49:33<2:36:17,  2.02s/it]
 75%|███████▌  | 13990/18627 [7:49:36<2:43:35,  2.12s/it]
                                                         
{'loss': 1.2462, 'grad_norm': 5.821321964263916, 'learning_rate': 7.694998409648968e-07, 'epoch': 0.75}
+
 75%|███████▌  | 13990/18627 [7:49:36<2:43:35,  2.12s/it]
 75%|███████▌  | 13991/18627 [7:49:38<2:47:51,  2.17s/it]
 75%|███████▌  | 13992/18627 [7:49:40<2:51:24,  2.22s/it]
 75%|███████▌  | 13993/18627 [7:49:42<2:32:45,  1.98s/it]
 75%|███████▌  | 13994/18627 [7:49:44<2:40:47,  2.08s/it]
 75%|███████▌  | 13995/18627 [7:49:46<2:46:16,  2.15s/it]
 75%|███████▌  | 13996/18627 [7:49:49<2:50:07,  2.20s/it]
 75%|███████▌  | 13997/18627 [7:49:51<2:52:37,  2.24s/it]
 75%|███████▌  | 13998/18627 [7:49:52<2:31:16,  1.96s/it]
 75%|███████▌  | 13999/18627 [7:49:55<2:39:26,  2.07s/it]
 75%|███████▌  | 14000/18627 [7:49:56<2:22:53,  1.85s/it]
                                                         
{'loss': 1.2219, 'grad_norm': 13.609045028686523, 'learning_rate': 7.66365274991126e-07, 'epoch': 0.75}
+
 75%|███████▌  | 14000/18627 [7:49:56<2:22:53,  1.85s/it]/home/zhengduo/workspace/LLaMA-Factory/src/qwen_vl/model/modeling_qwen2_5_vl.py:1979: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
+  with torch.cuda.amp.autocast(dtype=dtype):
+/home/zhengduo/miniconda3/envs/vgllm/lib/python3.10/site-packages/torch/utils/checkpoint.py:87: UserWarning: None of the inputs have requires_grad=True. Gradients will be None
+  warnings.warn(
+
 75%|███████▌  | 14001/18627 [7:50:59<25:47:28, 20.07s/it]
 75%|███████▌  | 14002/18627 [7:51:00<18:34:08, 14.45s/it]
 75%|███████▌  | 14003/18627 [7:51:02<13:53:07, 10.81s/it]
 75%|███████▌  | 14004/18627 [7:51:05<10:36:41,  8.26s/it]
 75%|███████▌  | 14005/18627 [7:51:07<8:19:35,  6.49s/it] 
 75%|███████▌  | 14006/18627 [7:51:08<6:19:15,  4.92s/it]
 75%|███████▌  | 14007/18627 [7:51:11<5:19:19,  4.15s/it]
 75%|███████▌  | 14008/18627 [7:51:12<4:15:41,  3.32s/it]
 75%|███████▌  | 14009/18627 [7:51:13<3:30:20,  2.73s/it]
 75%|██��████▌  | 14010/18627 [7:51:16<3:21:24,  2.62s/it]
                                                         
{'loss': 1.2953, 'grad_norm': 3.8345649242401123, 'learning_rate': 7.632359502938355e-07, 'epoch': 0.75}
+
 75%|███████▌  | 14010/18627 [7:51:16<3:21:24,  2.62s/it]
 75%|███████▌  | 14011/18627 [7:51:18<3:14:47,  2.53s/it]
 75%|███████▌  | 14012/18627 [7:51:20<3:09:58,  2.47s/it]
 75%|███████▌  | 14013/18627 [7:51:23<3:06:26,  2.42s/it]
 75%|███████▌  | 14014/18627 [7:51:25<3:03:26,  2.39s/it]
 75%|███████▌  | 14015/18627 [7:51:27<3:01:35,  2.36s/it]
 75%|███████▌  | 14016/18627 [7:51:30<3:01:04,  2.36s/it]
 75%|███████▌  | 14017/18627 [7:51:31<2:39:02,  2.07s/it]
 75%|███████▌  | 14018/18627 [7:51:33<2:45:28,  2.15s/it]
 75%|███████▌  | 14019/18627 [7:51:36<2:49:17,  2.20s/it]
 75%|███████▌  | 14020/18627 [7:51:38<2:52:14,  2.24s/it]
                                                         
{'loss': 0.8663, 'grad_norm': 12.086748123168945, 'learning_rate': 7.601118763338742e-07, 'epoch': 0.75}
+
 75%|███████▌  | 14020/18627 [7:51:38<2:52:14,  2.24s/it]
 75%|███████▌  | 14021/18627 [7:51:39<2:32:05,  1.98s/it]
 75%|███████▌  | 14022/18627 [7:51:42<2:40:29,  2.09s/it]
 75%|███████▌  | 14023/18627 [7:51:44<2:45:59,  2.16s/it]
 75%|███████▌  | 14024/18627 [7:51:46<2:49:57,  2.22s/it]
 75%|███████▌  | 14025/18627 [7:51:49<2:52:35,  2.25s/it]
 75%|███████▌  | 14026/18627 [7:51:51<2:54:48,  2.28s/it]
 75%|███████▌  | 14027/18627 [7:51:52<2:33:32,  2.00s/it]
 75%|███████▌  | 14028/18627 [7:51:55<2:40:54,  2.10s/it]
 75%|███████▌  | 14029/18627 [7:51:57<2:46:45,  2.18s/it]
 75%|███████▌  | 14030/18627 [7:51:59<2:49:53,  2.22s/it]
                                                         
{'loss': 1.0412, 'grad_norm': 5.392734527587891, 'learning_rate': 7.569930625562158e-07, 'epoch': 0.75}
+
 75%|███████▌  | 14030/18627 [7:51:59<2:49:53,  2.22s/it]
 75%|███████▌  | 14031/18627 [7:52:02<2:52:20,  2.25s/it]
 75%|███████▌  | 14032/18627 [7:52:04<2:54:03,  2.27s/it]
 75%|███████▌  | 14033/18627 [7:52:06<2:55:10,  2.29s/it]
 75%|███████▌  | 14034/18627 [7:52:09<2:55:28,  2.29s/it]
 75%|███████▌  | 14035/18627 [7:52:11<2:56:04,  2.30s/it]
 75%|███████▌  | 14036/18627 [7:52:13<2:56:26,  2.31s/it]
 75%|███████▌  | 14037/18627 [7:52:16<2:57:09,  2.32s/it]
 75%|███████▌  | 14038/18627 [7:52:18<2:56:56,  2.31s/it]
 75%|███████▌  | 14039/18627 [7:52:20<2:58:01,  2.33s/it]
 75%|███████▌  | 14040/18627 [7:52:23<2:58:08,  2.33s/it]
                                                         
{'loss': 0.6565, 'grad_norm': 5.756548881530762, 'learning_rate': 7.53879518389933e-07, 'epoch': 0.75}
+
 75%|███████▌  | 14040/18627 [7:52:23<2:58:08,  2.33s/it]
 75%|███████▌  | 14041/18627 [7:52:25<2:58:28,  2.34s/it]
 75%|███████▌  | 14042/18627 [7:52:27<2:58:57,  2.34s/it]
 75%|███████▌  | 14043/18627 [7:52:29<2:36:55,  2.05s/it]
 75%|███████▌  | 14044/18627 [7:52:31<2:43:26,  2.14s/it]
 75%|███████▌  | 14045/18627 [7:52:33<2:47:27,  2.19s/it]
 75%|███████▌  | 14046/18627 [7:52:36<2:50:11,  2.23s/it]
 75%|███████▌  | 14047/18627 [7:52:38<2:51:58,  2.25s/it]
 75%|███████▌  | 14048/18627 [7:52:39<2:31:57,  1.99s/it]
 75%|███████▌  | 14049/18627 [7:52:42<2:40:06,  2.10s/it]
 75%|███████▌  | 14050/18627 [7:52:44<2:45:27,  2.17s/it]
                                                         
{'loss': 1.0, 'grad_norm': 7.041780948638916, 'learning_rate': 7.507712532481632e-07, 'epoch': 0.75}
+
 75%|███████▌  | 14050/18627 [7:52:44<2:45:27,  2.17s/it]
 75%|███████▌  | 14051/18627 [7:52:46<2:49:10,  2.22s/it]
 75%|███████▌  | 14052/18627 [7:52:48<2:22:44,  1.87s/it]
 75%|███████▌  | 14053/18627 [7:52:50<2:33:24,  2.01s/it]
 75%|███████▌  | 14054/18627 [7:52:52<2:41:04,  2.11s/it]
 75%|███████▌  | 14055/18627 [7:52:55<2:46:16,  2.18s/it]
 75%|███████▌  | 14056/18627 [7:52:57<2:49:36,  2.23s/it]
 75%|███████▌  | 14057/18627 [7:52:59<2:52:06,  2.26s/it]
 75%|███████▌  | 14058/18627 [7:53:02<2:53:39,  2.28s/it]
 75%|███████▌  | 14059/18627 [7:53:04<2:54:48,  2.30s/it]
 75%|███████▌  | 14060/18627 [7:53:05<2:35:02,  2.04s/it]
                                                         
{'loss': 0.7847, 'grad_norm': 12.369487762451172, 'learning_rate': 7.476682765280882e-07, 'epoch': 0.75}
+
 75%|���██████▌  | 14060/18627 [7:53:05<2:35:02,  2.04s/it]
 75%|███████▌  | 14061/18627 [7:53:08<2:42:13,  2.13s/it]
 75%|███████▌  | 14062/18627 [7:53:09<2:25:59,  1.92s/it]
 75%|███████▌  | 14063/18627 [7:53:11<2:34:52,  2.04s/it]
 76%|███████▌  | 14064/18627 [7:53:13<2:20:28,  1.85s/it]
 76%|███████▌  | 14065/18627 [7:53:15<2:32:20,  2.00s/it]
 76%|███████▌  | 14066/18627 [7:53:17<2:39:20,  2.10s/it]
 76%|███████▌  | 14067/18627 [7:53:20<2:45:35,  2.18s/it]
 76%|███████▌  | 14068/18627 [7:53:21<2:27:10,  1.94s/it]
 76%|███████▌  | 14069/18627 [7:53:24<2:36:07,  2.06s/it]
 76%|███████▌  | 14070/18627 [7:53:26<2:41:49,  2.13s/it]
                                                         
{'loss': 1.2401, 'grad_norm': 5.834813594818115, 'learning_rate': 7.445705976108981e-07, 'epoch': 0.76}
+
 76%|███████▌  | 14070/18627 [7:53:26<2:41:49,  2.13s/it]
 76%|███████▌  | 14071/18627 [7:53:28<2:46:19,  2.19s/it]
 76%|███████▌  | 14072/18627 [7:53:31<2:49:42,  2.24s/it]
 76%|███████▌  | 14073/18627 [7:53:33<2:51:35,  2.26s/it]
 76%|███████▌  | 14074/18627 [7:53:35<2:53:50,  2.29s/it]
 76%|███████▌  | 14075/18627 [7:53:38<2:54:08,  2.30s/it]
 76%|███████▌  | 14076/18627 [7:53:43<3:57:47,  3.14s/it]
 76%|███████▌  | 14077/18627 [7:53:45<3:39:45,  2.90s/it]
 76%|███████▌  | 14078/18627 [7:53:47<3:27:14,  2.73s/it]
 76%|███████▌  | 14079/18627 [7:53:49<2:57:06,  2.34s/it]
 76%|███████▌  | 14080/18627 [7:53:50<2:27:46,  1.95s/it]
                                                         
{'loss': 0.7987, 'grad_norm': 7.147216796875, 'learning_rate': 7.414782258617673e-07, 'epoch': 0.76}
+
 76%|███████▌  | 14080/18627 [7:53:50<2:27:46,  1.95s/it]
 76%|███████▌  | 14081/18627 [7:53:52<2:36:43,  2.07s/it]
 76%|███████▌  | 14082/18627 [7:53:53<2:20:52,  1.86s/it]
 76%|███████▌  | 14083/18627 [7:53:56<2:32:11,  2.01s/it]
 76%|███████▌  | 14084/18627 [7:53:58<2:39:22,  2.10s/it]
 76%|███████▌  | 14085/18627 [7:54:01<2:44:09,  2.17s/it]
 76%|███████▌  | 14086/18627 [7:54:03<2:47:55,  2.22s/it]
 76%|███████▌  | 14087/18627 [7:54:05<2:50:10,  2.25s/it]
 76%|███████▌  | 14088/18627 [7:54:07<2:52:02,  2.27s/it]
 76%|███████▌  | 14089/18627 [7:54:10<2:53:07,  2.29s/it]
 76%|███████▌  | 14090/18627 [7:54:12<2:54:29,  2.31s/it]
                                                         
{'loss': 0.767, 'grad_norm': 8.170140266418457, 'learning_rate': 7.383911706298255e-07, 'epoch': 0.76}
+
 76%|███████▌  | 14090/18627 [7:54:12<2:54:29,  2.31s/it]
 76%|███████▌  | 14091/18627 [7:54:15<2:57:53,  2.35s/it]
 76%|███████▌  | 14092/18627 [7:54:19<3:52:29,  3.08s/it]
 76%|███████▌  | 14093/18627 [7:54:22<3:35:25,  2.85s/it]
 76%|███████▌  | 14094/18627 [7:54:24<3:23:10,  2.69s/it]
 76%|███████▌  | 14095/18627 [7:54:26<3:14:45,  2.58s/it]
 76%|███████▌  | 14096/18627 [7:54:28<2:48:33,  2.23s/it]
 76%|███████▌  | 14097/18627 [7:54:30<2:51:22,  2.27s/it]
 76%|███████▌  | 14098/18627 [7:54:32<2:53:02,  2.29s/it]
 76%|███████▌  | 14099/18627 [7:54:35<2:54:07,  2.31s/it]
 76%|███████▌  | 14100/18627 [7:54:37<2:54:53,  2.32s/it]
                                                         
{'loss': 0.8332, 'grad_norm': 7.053866386413574, 'learning_rate': 7.353094412481288e-07, 'epoch': 0.76}
+
 76%|███████▌  | 14100/18627 [7:54:37<2:54:53,  2.32s/it]
 76%|███████▌  | 14101/18627 [7:54:39<2:36:24,  2.07s/it]
 76%|███████▌  | 14102/18627 [7:54:41<2:42:26,  2.15s/it]
 76%|███████▌  | 14103/18627 [7:54:43<2:46:07,  2.20s/it]
 76%|███████▌  | 14104/18627 [7:54:46<2:48:54,  2.24s/it]
 76%|███████▌  | 14105/18627 [7:54:48<2:50:47,  2.27s/it]
 76%|███████▌  | 14106/18627 [7:54:50<2:51:54,  2.28s/it]
 76%|███████▌  | 14107/18627 [7:54:53<2:52:36,  2.29s/it]
 76%|███████▌  | 14108/18627 [7:54:55<2:53:18,  2.30s/it]
 76%|███████▌  | 14109/18627 [7:54:56<2:30:14,  2.00s/it]
 76%|███████▌  | 14110/18627 [7:54:59<2:37:24,  2.09s/it]
                                                         
{'loss': 0.943, 'grad_norm': 5.854894161224365, 'learning_rate': 7.322330470336314e-07, 'epoch': 0.76}
+
 76%|███████▌  | 14110/18627 [7:54:59<2:37:24,  2.09s/it]
 76%|███████▌  | 14111/18627 [7:55:01<2:42:28,  2.16s/it]
 76%|███████▌  | 14112/18627 [7:55:02<2:16:41,  1.82s/it]
 76%|████��██▌  | 14113/18627 [7:55:04<2:27:45,  1.96s/it]
 76%|███████▌  | 14114/18627 [7:55:06<2:15:13,  1.80s/it]
 76%|███████▌  | 14115/18627 [7:55:08<2:26:25,  1.95s/it]
 76%|███████▌  | 14116/18627 [7:55:10<2:34:52,  2.06s/it]
 76%|███████▌  | 14117/18627 [7:55:11<2:11:39,  1.75s/it]
 76%|███████▌  | 14118/18627 [7:55:14<2:24:20,  1.92s/it]
 76%|███████▌  | 14119/18627 [7:55:16<2:34:01,  2.05s/it]
 76%|███████▌  | 14120/18627 [7:55:17<2:17:46,  1.83s/it]
                                                         
{'loss': 1.1741, 'grad_norm': 12.823599815368652, 'learning_rate': 7.291619972871581e-07, 'epoch': 0.76}
+
 76%|███████▌  | 14120/18627 [7:55:17<2:17:46,  1.83s/it]
 76%|███████▌  | 14121/18627 [7:55:19<2:06:47,  1.69s/it]
 76%|███████▌  | 14122/18627 [7:55:20<1:54:23,  1.52s/it]
 76%|███████▌  | 14123/18627 [7:55:21<1:50:07,  1.47s/it]
 76%|███████▌  | 14124/18627 [7:55:23<2:09:00,  1.72s/it]
 76%|███████▌  | 14125/18627 [7:55:26<2:22:46,  1.90s/it]
 76%|███████▌  | 14126/18627 [7:55:28<2:33:28,  2.05s/it]
 76%|███████▌  | 14127/18627 [7:55:30<2:39:44,  2.13s/it]
 76%|███████▌  | 14128/18627 [7:55:32<2:20:51,  1.88s/it]
 76%|███████▌  | 14129/18627 [7:55:33<2:08:56,  1.72s/it]
 76%|███████▌  | 14130/18627 [7:55:35<2:22:01,  1.89s/it]
                                                         
{'loss': 1.4166, 'grad_norm': 5.5932722091674805, 'learning_rate': 7.260963012933758e-07, 'epoch': 0.76}
+
 76%|███████▌  | 14130/18627 [7:55:35<2:22:01,  1.89s/it]
 76%|███████▌  | 14131/18627 [7:55:38<2:31:44,  2.03s/it]
 76%|███████▌  | 14132/18627 [7:55:40<2:37:27,  2.10s/it]
 76%|███████▌  | 14133/18627 [7:55:42<2:42:54,  2.17s/it]
 76%|███████▌  | 14134/18627 [7:55:44<2:24:24,  1.93s/it]
 76%|███████▌  | 14135/18627 [7:55:46<2:33:03,  2.04s/it]
 76%|███████▌  | 14136/18627 [7:55:47<2:19:17,  1.86s/it]
 76%|███████▌  | 14137/18627 [7:55:49<2:07:46,  1.71s/it]
 76%|███████▌  | 14138/18627 [7:55:51<2:21:21,  1.89s/it]
 76%|███████▌  | 14139/18627 [7:55:53<2:30:46,  2.02s/it]
 76%|███████▌  | 14140/18627 [7:55:56<2:37:56,  2.11s/it]
                                                         
{'loss': 1.1078, 'grad_norm': 5.113554954528809, 'learning_rate': 7.23035968320765e-07, 'epoch': 0.76}
+
 76%|███████▌  | 14140/18627 [7:55:56<2:37:56,  2.11s/it]
 76%|███████▌  | 14141/18627 [7:55:58<2:42:45,  2.18s/it]
 76%|███████▌  | 14142/18627 [7:56:00<2:45:45,  2.22s/it]
 76%|███████▌  | 14143/18627 [7:56:02<2:26:47,  1.96s/it]
 76%|███████▌  | 14144/18627 [7:56:04<2:34:39,  2.07s/it]
 76%|███████▌  | 14145/18627 [7:56:05<2:11:33,  1.76s/it]
 76%|███████▌  | 14146/18627 [7:56:07<2:25:05,  1.94s/it]
 76%|███████▌  | 14147/18627 [7:56:09<2:12:50,  1.78s/it]
 76%|███████▌  | 14148/18627 [7:56:10<2:03:35,  1.66s/it]
 76%|███████▌  | 14149/18627 [7:56:12<2:17:09,  1.84s/it]
 76%|███████▌  | 14150/18627 [7:56:15<2:28:06,  1.98s/it]
                                                         
{'loss': 1.1827, 'grad_norm': 6.615344047546387, 'learning_rate': 7.199810076215932e-07, 'epoch': 0.76}
+
 76%|███████▌  | 14150/18627 [7:56:15<2:28:06,  1.98s/it]
 76%|███████▌  | 14151/18627 [7:56:17<2:35:24,  2.08s/it]
 76%|███████▌  | 14152/18627 [7:56:18<2:18:58,  1.86s/it]
 76%|███████▌  | 14153/18627 [7:56:20<2:07:46,  1.71s/it]
 76%|███████▌  | 14154/18627 [7:56:22<2:21:23,  1.90s/it]
 76%|███████▌  | 14155/18627 [7:56:25<2:33:52,  2.06s/it]
 76%|███████▌  | 14156/18627 [7:56:27<2:39:46,  2.14s/it]
 76%|███████▌  | 14157/18627 [7:56:28<2:12:47,  1.78s/it]
 76%|███████▌  | 14158/18627 [7:56:30<2:25:33,  1.95s/it]
 76%|███████▌  | 14159/18627 [7:56:33<2:34:14,  2.07s/it]
 76%|███████▌  | 14160/18627 [7:56:35<2:39:50,  2.15s/it]
                                                         
{'loss': 0.9641, 'grad_norm': 4.974226474761963, 'learning_rate': 7.169314284318849e-07, 'epoch': 0.76}
+
 76%|███████▌  | 14160/18627 [7:56:35<2:39:50,  2.15s/it]
 76%|███████▌  | 14161/18627 [7:56:36<2:21:54,  1.91s/it]
 76%|███████▌  | 14162/18627 [7:56:39<2:30:54,  2.03s/it]
 76%|███████▌  | 14163/18627 [7:56:41<2:37:12,  2.11s/it]
 76%|███████▌  | 14164/18627 [7:56:43<2:41:21,  2.17s/it]
 76%|███████▌  | 14165/18627 [7:56:46<2:45:05,  2.22s/it]
 76%|███████▌  | 14166/18627 [7:56:48<2:47:49,  2.26s/it]
 76%|███████▌  | 14167/18627 [7:56:50<2:49:35,  2.28s/it]
 76%|███████▌  | 14168/18627 [7:56:51<2:27:07,  1.98s/it]
 76%|███████▌  | 14169/18627 [7:56:54<2:34:24,  2.08s/it]
 76%|███████▌  | 14170/18627 [7:56:56<2:39:46,  2.15s/it]
                                                         
{'loss': 1.0313, 'grad_norm': 6.178246021270752, 'learning_rate': 7.138872399713964e-07, 'epoch': 0.76}
+
 76%|███████▌  | 14170/18627 [7:56:56<2:39:46,  2.15s/it]
 76%|███████▌  | 14171/18627 [7:56:58<2:43:31,  2.20s/it]
 76%|███████▌  | 14172/18627 [7:57:01<2:46:38,  2.24s/it]
 76%|███████▌  | 14173/18627 [7:57:03<2:47:56,  2.26s/it]
 76%|███████▌  | 14174/18627 [7:57:04<2:27:36,  1.99s/it]
 76%|███████▌  | 14175/18627 [7:57:06<2:13:27,  1.80s/it]
 76%|███████▌  | 14176/18627 [7:57:08<2:25:27,  1.96s/it]
 76%|███████▌  | 14177/18627 [7:57:10<2:33:32,  2.07s/it]
 76%|███████▌  | 14178/18627 [7:57:13<2:39:07,  2.15s/it]
 76%|███████▌  | 14179/18627 [7:57:15<2:42:58,  2.20s/it]
 76%|███████▌  | 14180/18627 [7:57:17<2:46:00,  2.24s/it]
                                                         
{'loss': 0.9752, 'grad_norm': 5.516609191894531, 'learning_rate': 7.108484514435826e-07, 'epoch': 0.76}
+
 76%|███████▌  | 14180/18627 [7:57:17<2:46:00,  2.24s/it]
 76%|███████▌  | 14181/18627 [7:57:20<2:48:17,  2.27s/it]
 76%|███████▌  | 14182/18627 [7:57:22<2:49:55,  2.29s/it]
 76%|███████▌  | 14183/18627 [7:57:23<2:29:03,  2.01s/it]
 76%|███████▌  | 14184/18627 [7:57:26<2:35:59,  2.11s/it]
 76%|███████▌  | 14185/18627 [7:57:28<2:40:17,  2.17s/it]
 76%|███████▌  | 14186/18627 [7:57:30<2:44:23,  2.22s/it]
 76%|███████▌  | 14187/18627 [7:57:33<2:45:56,  2.24s/it]
 76%|███████▌  | 14188/18627 [7:57:34<2:26:49,  1.98s/it]
 76%|███████▌  | 14189/18627 [7:57:36<2:33:49,  2.08s/it]
 76%|███████▌  | 14190/18627 [7:57:39<2:39:13,  2.15s/it]
                                                         
{'loss': 0.9818, 'grad_norm': 11.759650230407715, 'learning_rate': 7.078150720355784e-07, 'epoch': 0.76}
+
 76%|███████▌  | 14190/18627 [7:57:39<2:39:13,  2.15s/it]
 76%|███████▌  | 14191/18627 [7:57:41<2:42:51,  2.20s/it]
 76%|███████▌  | 14192/18627 [7:57:43<2:45:17,  2.24s/it]
 76%|███████▌  | 14193/18627 [7:57:46<2:47:24,  2.27s/it]
 76%|███████▌  | 14194/18627 [7:57:48<2:48:32,  2.28s/it]
 76%|███████▌  | 14195/18627 [7:57:50<2:50:08,  2.30s/it]
 76%|███████▌  | 14196/18627 [7:57:52<2:28:49,  2.02s/it]
 76%|███████▌  | 14197/18627 [7:57:54<2:36:14,  2.12s/it]
 76%|███████▌  | 14198/18627 [7:57:55<2:20:26,  1.90s/it]
 76%|███████▌  | 14199/18627 [7:57:58<2:28:28,  2.01s/it]
 76%|███████▌  | 14200/18627 [7:57:59<2:14:13,  1.82s/it]
                                                         
{'loss': 1.2276, 'grad_norm': 14.698812484741211, 'learning_rate': 7.047871109181604e-07, 'epoch': 0.76}
+
 76%|███████▌  | 14200/18627 [7:57:59<2:14:13,  1.82s/it]
 76%|███████▌  | 14201/18627 [7:58:01<2:26:35,  1.99s/it]
 76%|███████▌  | 14202/18627 [7:58:06<3:28:35,  2.83s/it]
 76%|███████▌  | 14203/18627 [7:58:09<3:17:21,  2.68s/it]
 76%|███████▋  | 14204/18627 [7:58:13<3:56:54,  3.21s/it]
 76%|███████▋  | 14205/18627 [7:58:15<3:37:14,  2.95s/it]
 76%|███████▋  | 14206/18627 [7:58:18<3:22:14,  2.74s/it]
 76%|███████▋  | 14207/18627 [7:58:20<3:12:33,  2.61s/it]
 76%|███████▋  | 14208/18627 [7:58:22<3:05:48,  2.52s/it]
 76%|███████▋  | 14209/18627 [7:58:25<3:00:34,  2.45s/it]
 76%|███████▋  | 14210/18627 [7:58:27<2:58:04,  2.42s/it]
                                                         
{'loss': 0.5772, 'grad_norm': 4.3963470458984375, 'learning_rate': 7.017645772457263e-07, 'epoch': 0.76}
+
 76%|███████▋  | 14210/18627 [7:58:27<2:58:04,  2.42s/it]
 76%|███████▋  | 14211/18627 [7:58:29<2:55:43,  2.39s/it]
 76%|███████▋  | 14212/18627 [7:58:32<2:53:38,  2.36s/it]
 76%|███████▋  | 14213/18627 [7:58:34<2:52:38,  2.35s/it]
 76%|███████▋  | 14214/18627 [7:58:35<2:30:39,  2.05s/it]
 76%|███████▋  | 14215/18627 [7:58:38<2:36:19,  2.13s/it]
 76%|███████▋  | 14216/18627 [7:58:39<2:17:17,  1.87s/it]
 76%|███████▋  | 14217/18627 [7:58:41<2:27:44,  2.01s/it]
 76%|███████▋  | 14218/18627 [7:58:43<2:34:34,  2.10s/it]
 76%|███████▋  | 14219/18627 [7:58:45<2:12:01,  1.80s/it]
 76%|███████▋  | 14220/18627 [7:58:47<2:24:11,  1.96s/it]
                                                         
{'loss': 1.0245, 'grad_norm': 6.931156635284424, 'learning_rate': 6.987474801562652e-07, 'epoch': 0.76}
+
 76%|███████▋  | 14220/18627 [7:58:47<2:24:11,  1.96s/it]
 76%|███████▋  | 14221/18627 [7:58:49<2:32:10,  2.07s/it]
 76%|███████▋  | 14222/18627 [7:58:52<2:37:37,  2.15s/it]
 76%|███████▋  | 14223/18627 [7:58:53<2:20:10,  1.91s/it]
 76%|███████▋  | 14224/18627 [7:58:55<2:29:29,  2.04s/it]
 76%|███████▋  | 14225/18627 [7:58:58<2:35:22,  2.12s/it]
 76%|███████▋  | 14226/18627 [7:59:00<2:39:55,  2.18s/it]
 76%|███████▋  | 14227/18627 [7:59:02<2:43:12,  2.23s/it]
 76%|███████▋  | 14228/18627 [7:59:04<2:45:29,  2.26s/it]
 76%|███████▋  | 14229/18627 [7:59:07<2:46:43,  2.27s/it]
 76%|███████▋  | 14230/18627 [7:59:08<2:25:05,  1.98s/it]
                                                         
{'loss': 1.0339, 'grad_norm': 13.668159484863281, 'learning_rate': 6.957358287713295e-07, 'epoch': 0.76}
+
 76%|███████▋  | 14230/18627 [7:59:08<2:25:05,  1.98s/it]
 76%|███████▋  | 14231/18627 [7:59:09<2:11:25,  1.79s/it]
 76%|███████▋  | 14232/18627 [7:59:12<2:23:01,  1.95s/it]
 76%|███████▋  | 14233/18627 [7:59:14<2:30:47,  2.06s/it]
 76%|███████▋  | 14234/18627 [7:59:16<2:36:45,  2.14s/it]
 76%|███████▋  | 14235/18627 [7:59:19<2:41:21,  2.20s/it]
 76%|███████▋  | 14236/18627 [7:59:20<2:22:31,  1.95s/it]
 76%|███████▋  | 14237/18627 [7:59:22<2:30:49,  2.06s/it]
 76%|███████▋  | 14238/18627 [7:59:25<2:36:53,  2.14s/it]
 76%|███████▋  | 14239/18627 [7:59:27<2:40:22,  2.19s/it]
 76%|███████▋  | 14240/18627 [7:59:28<2:22:30,  1.95s/it]
                                                         
{'loss': 1.1539, 'grad_norm': 15.227300643920898, 'learning_rate': 6.927296321960078e-07, 'epoch': 0.76}
+
 76%|███████▋  | 14240/18627 [7:59:28<2:22:30,  1.95s/it]
 76%|███████▋  | 14241/18627 [7:59:31<2:31:12,  2.07s/it]
 76%|███████▋  | 14242/18627 [7:59:33<2:36:50,  2.15s/it]
 76%|███████▋  | 14243/18627 [7:59:35<2:40:46,  2.20s/it]
 76%|███████▋  | 14244/18627 [7:59:38<2:43:25,  2.24s/it]
 76%|███████▋  | 14245/18627 [7:59:40<2:45:01,  2.26s/it]
 76%|███████▋  | 14246/18627 [7:59:42<2:46:26,  2.28s/it]
 76%|███████▋  | 14247/18627 [7:59:45<2:47:29,  2.29s/it]
 76%|███████▋  | 14248/18627 [7:59:47<2:49:03,  2.32s/it]
 76%|███████▋  | 14249/18627 [7:59:49<2:49:54,  2.33s/it]
 77%|███████▋  | 14250/18627 [7:59:52<2:50:35,  2.34s/it]
                                                         
{'loss': 0.5936, 'grad_norm': 5.660309791564941, 'learning_rate': 6.897288995188975e-07, 'epoch': 0.77}
+
 77%|███████▋  | 14250/18627 [7:59:52<2:50:35,  2.34s/it]
 77%|███████▋  | 14251/18627 [7:59:54<2:50:40,  2.34s/it]
 77%|███████▋  | 14252/18627 [7:59:57<2:50:01,  2.33s/it]
 77%|███████▋  | 14253/18627 [7:59:59<2:50:24,  2.34s/it]
 77%|███████▋  | 14254/18627 [8:00:01<2:51:06,  2.35s/it]
 77%|███████▋  | 14255/18627 [8:00:04<2:50:56,  2.35s/it]
 77%|███████▋  | 14256/18627 [8:00:05<2:29:30,  2.05s/it]
 77%|███████▋  | 14257/18627 [8:00:07<2:35:29,  2.13s/it]
 77%|███████▋  | 14258/18627 [8:00:10<2:39:28,  2.19s/it]
 77%|███████▋  | 14259/18627 [8:00:12<2:41:52,  2.22s/it]
 77%|███████▋  | 14260/18627 [8:00:14<2:43:51,  2.25s/it]
                                                         
{'loss': 0.765, 'grad_norm': 6.377910614013672, 'learning_rate': 6.867336398120749e-07, 'epoch': 0.77}
+
 77%|███████▋  | 14260/18627 [8:00:14<2:43:51,  2.25s/it]
 77%|███████▋  | 14261/18627 [8:00:16<2:24:46,  1.99s/it]
 77%|███████▋  | 14262/18627 [8:00:18<2:31:16,  2.08s/it]
 77%|███████▋  | 14263/18627 [8:00:20<2:37:16,  2.16s/it]
 77%|███████▋  | 14264/18627 [8:00:21<2:14:32,  1.85s/it]
 77%|███████▋  | 14265/18627 [8:00:24<2:25:17,  2.00s/it]
 77%|███████▋  | 14266/18627 [8:00:25<2:05:41,  1.73s/it]
 77%|███████▋  | 14267/18627 [8:00:27<2:18:40,  1.91s/it]
 77%|███████▋  | 14268/18627 [8:00:29<2:28:34,  2.04s/it]
 77%|███████▋  | 14269/18627 [8:00:32<2:34:35,  2.13s/it]
 77%|███████▋  | 14270/18627 [8:00:34<2:38:07,  2.18s/it]
                                                         
{'loss': 0.876, 'grad_norm': 5.526716232299805, 'learning_rate': 6.837438621310746e-07, 'epoch': 0.77}
+
 77%|███████▋  | 14270/18627 [8:00:34<2:38:07,  2.18s/it]
 77%|███████▋  | 14271/18627 [8:00:36<2:21:07,  1.94s/it]
 77%|███████▋  | 14272/18627 [8:00:38<2:29:00,  2.05s/it]
 77%|███████▋  | 14273/18627 [8:00:39<2:12:46,  1.83s/it]
 77%|███████▋  | 14274/18627 [8:00:41<2:05:37,  1.73s/it]
 77%|███████▋  | 14275/18627 [8:00:43<2:18:04,  1.90s/it]
 77%|███████▋  | 14276/18627 [8:00:45<2:27:51,  2.04s/it]
 77%|███████▋  | 14277/18627 [8:00:48<2:33:37,  2.12s/it]
 77%|███████▋  | 14278/18627 [8:00:50<2:38:55,  2.19s/it]
 77%|███████▋  | 14279/18627 [8:00:51<2:20:43,  1.94s/it]
 77%|███████▋  | 14280/18627 [8:00:54<2:29:02,  2.06s/it]
                                                         
{'loss': 1.3837, 'grad_norm': 7.58469295501709, 'learning_rate': 6.807595755148522e-07, 'epoch': 0.77}
+
 77%|███████▋  | 14280/18627 [8:00:54<2:29:02,  2.06s/it]
 77%|███████▋  | 14281/18627 [8:00:55<2:06:50,  1.75s/it]
 77%|███████▋  | 14282/18627 [8:00:57<2:19:45,  1.93s/it]
 77%|███████▋  | 14283/18627 [8:00:58<2:07:59,  1.77s/it]
 77%|███████▋  | 14284/18627 [8:01:01<2:20:35,  1.94s/it]
 77%|███████▋  | 14285/18627 [8:01:03<2:29:34,  2.07s/it]
 77%|███████▋  | 14286/18627 [8:01:05<2:35:48,  2.15s/it]
 77%|███████▋  | 14287/18627 [8:01:08<2:39:59,  2.21s/it]
 77%|███████▋  | 14288/18627 [8:01:10<2:42:38,  2.25s/it]
 77%|███████▋  | 14289/18627 [8:01:13<2:44:56,  2.28s/it]
 77%|███████▋  | 14290/18627 [8:01:14<2:18:00,  1.91s/it]
                                                         
{'loss': 0.7417, 'grad_norm': 4.648711681365967, 'learning_rate': 6.777807889857652e-07, 'epoch': 0.77}
+
 77%|███████▋  | 14290/18627 [8:01:14<2:18:00,  1.91s/it]
 77%|███████▋  | 14291/18627 [8:01:16<2:27:42,  2.04s/it]
 77%|███████▋  | 14292/18627 [8:01:18<2:34:09,  2.13s/it]
 77%|███████▋  | 14293/18627 [8:01:21<2:38:25,  2.19s/it]
 77%|███████▋  | 14294/18627 [8:01:23<2:41:38,  2.24s/it]
 77%|███████▋  | 14295/18627 [8:01:24<2:22:09,  1.97s/it]
 77%|███████▋  | 14296/18627 [8:01:27<2:29:54,  2.08s/it]
 77%|███████▋  | 14297/18627 [8:01:29<2:35:12,  2.15s/it]
 77%|███████▋  | 14298/18627 [8:01:31<2:39:33,  2.21s/it]
 77%|███████▋  | 14299/18627 [8:01:34<2:42:00,  2.25s/it]
 77%|███████▋  | 14300/18627 [8:01:36<2:44:23,  2.28s/it]
                                                         
{'loss': 0.7826, 'grad_norm': 5.295623779296875, 'learning_rate': 6.748075115495425e-07, 'epoch': 0.77}
+
 77%|███████▋  | 14300/18627 [8:01:36<2:44:23,  2.28s/it]
 77%|███████▋  | 14301/18627 [8:01:38<2:45:35,  2.30s/it]
 77%|███████▋  | 14302/18627 [8:01:40<2:25:09,  2.01s/it]
 77%|███████▋  | 14303/18627 [8:01:42<2:32:43,  2.12s/it]
 77%|███████▋  | 14304/18627 [8:01:44<2:37:05,  2.18s/it]
 77%|███████▋  | 14305/18627 [8:01:47<2:40:18,  2.23s/it]
 77%|███████▋  | 14306/18627 [8:01:49<2:41:56,  2.25s/it]
 77%|███████▋  | 14307/18627 [8:01:50<2:20:01,  1.94s/it]
 77%|███████▋  | 14308/18627 [8:01:53<2:29:05,  2.07s/it]
 77%|███████▋  | 14309/18627 [8:01:54<2:14:05,  1.86s/it]
 77%|███████▋  | 14310/18627 [8:01:56<2:24:23,  2.01s/it]
                                                         
{'loss': 1.252, 'grad_norm': 8.764469146728516, 'learning_rate': 6.718397521952568e-07, 'epoch': 0.77}
+
 77%|███████▋  | 14310/18627 [8:01:56<2:24:23,  2.01s/it]
 77%|███████▋  | 14311/18627 [8:01:58<2:11:06,  1.82s/it]
 77%|███████▋  | 14312/18627 [8:02:00<2:22:46,  1.99s/it]
 77%|███████▋  | 14313/18627 [8:02:01<2:03:07,  1.71s/it]
 77%|███████▋  | 14314/18627 [8:02:03<2:16:02,  1.89s/it]
 77%|███████▋  | 14315/18627 [8:02:06<2:25:30,  2.02s/it]
 77%|███████▋  | 14316/18627 [8:02:07<2:12:32,  1.84s/it]
 77%|███████▋  | 14317/18627 [8:02:10<2:24:44,  2.01s/it]
 77%|███████▋  | 14318/18627 [8:02:12<2:31:46,  2.11s/it]
 77%|███████▋  | 14319/18627 [8:02:14<2:36:46,  2.18s/it]
 77%|███████▋  | 14320/18627 [8:02:17<2:40:40,  2.24s/it]
                                                         
{'loss': 1.0122, 'grad_norm': 6.6944756507873535, 'learning_rate': 6.68877519895299e-07, 'epoch': 0.77}
+
 77%|███████▋  | 14320/18627 [8:02:17<2:40:40,  2.24s/it]
 77%|███████▋  | 14321/18627 [8:02:19<2:43:12,  2.27s/it]
 77%|███████▋  | 14322/18627 [8:02:21<2:44:40,  2.30s/it]
 77%|███████▋  | 14323/18627 [8:02:23<2:24:13,  2.01s/it]
 77%|███████▋  | 14324/18627 [8:02:25<2:31:01,  2.11s/it]
 77%|███████▋  | 14325/18627 [8:02:27<2:35:32,  2.17s/it]
 77%|███████▋  | 14326/18627 [8:02:30<2:39:19,  2.22s/it]
 77%|███████▋  | 14327/18627 [8:02:32<2:41:45,  2.26s/it]
 77%|███████▋  | 14328/18627 [8:02:34<2:43:19,  2.28s/it]
 77%|███████▋  | 14329/18627 [8:02:37<2:44:26,  2.30s/it]
 77%|███████▋  | 14330/18627 [8:02:39<2:44:51,  2.30s/it]
                                                         
{'loss': 0.8202, 'grad_norm': 5.427619934082031, 'learning_rate': 6.659208236053493e-07, 'epoch': 0.77}
+
 77%|███████▋  | 14330/18627 [8:02:39<2:44:51,  2.30s/it]
 77%|███████▋  | 14331/18627 [8:02:41<2:46:49,  2.33s/it]
 77%|███████▋  | 14332/18627 [8:02:44<2:47:10,  2.34s/it]
 77%|███████▋  | 14333/18627 [8:02:46<2:47:13,  2.34s/it]
 77%|███████▋  | 14334/18627 [8:02:48<2:27:27,  2.06s/it]
 77%|███████▋  | 14335/18627 [8:02:50<2:33:54,  2.15s/it]
 77%|███████▋  | 14336/18627 [8:02:52<2:37:48,  2.21s/it]
 77%|███████▋  | 14337/18627 [8:02:55<2:43:38,  2.29s/it]
 77%|███████▋  | 14338/18627 [8:02:56<2:24:14,  2.02s/it]
 77%|███████▋  | 14339/18627 [8:02:58<2:30:37,  2.11s/it]
 77%|███████▋  | 14340/18627 [8:03:01<2:35:15,  2.17s/it]
                                                         
{'loss': 0.9939, 'grad_norm': 5.072685241699219, 'learning_rate': 6.629696722643497e-07, 'epoch': 0.77}
+
 77%|███████▋  | 14340/18627 [8:03:01<2:35:15,  2.17s/it]
 77%|███████▋  | 14341/18627 [8:03:03<2:38:25,  2.22s/it]
 77%|███████▋  | 14342/18627 [8:03:05<2:40:42,  2.25s/it]
 77%|███████▋  | 14343/18627 [8:03:08<2:41:48,  2.27s/it]
 77%|███████▋  | 14344/18627 [8:03:10<2:43:03,  2.28s/it]
 77%|███████▋  | 14345/18627 [8:03:12<2:44:32,  2.31s/it]
 77%|███████▋  | 14346/18627 [8:03:15<2:45:57,  2.33s/it]
 77%|███████▋  | 14347/18627 [8:03:17<2:46:09,  2.33s/it]
 77%|███████▋  | 14348/18627 [8:03:19<2:46:18,  2.33s/it]
 77%|███████▋  | 14349/18627 [8:03:21<2:26:02,  2.05s/it]
 77%|███████▋  | 14350/18627 [8:03:23<2:31:57,  2.13s/it]
                                                         
{'loss': 0.8259, 'grad_norm': 5.672342300415039, 'learning_rate': 6.60024074794482e-07, 'epoch': 0.77}
+
 77%|███████▋  | 14350/18627 [8:03:23<2:31:57,  2.13s/it]
 77%|███████▋  | 14351/18627 [8:03:25<2:36:51,  2.20s/it]
 77%|███████▋  | 14352/18627 [8:03:27<2:19:10,  1.95s/it]
 77%|███████▋  | 14353/18627 [8:03:29<2:27:03,  2.06s/it]
 77%|███████▋  | 14354/18627 [8:03:32<2:32:17,  2.14s/it]
 77%|███████▋  | 14355/18627 [8:03:34<2:36:01,  2.19s/it]
 77%|███████▋  | 14356/18627 [8:03:36<2:39:07,  2.24s/it]
 77%|███████▋  | 14357/18627 [8:03:38<2:41:11,  2.26s/it]
 77%|███████▋  | 14358/18627 [8:03:41<2:42:43,  2.29s/it]
 77%|███████▋  | 14359/18627 [8:03:43<2:44:09,  2.31s/it]
 77%|███████▋  | 14360/18627 [8:03:46<2:44:48,  2.32s/it]
                                                         
{'loss': 0.8182, 'grad_norm': 6.412765979766846, 'learning_rate': 6.570840401011327e-07, 'epoch': 0.77}
+
 77%|███████▋  | 14360/18627 [8:03:46<2:44:48,  2.32s/it]
 77%|███████▋  | 14361/18627 [8:03:48<2:44:41,  2.32s/it]
 77%|███████▋  | 14362/18627 [8:03:50<2:45:15,  2.32s/it]
 77%|███████▋  | 14363/18627 [8:03:52<2:44:29,  2.31s/it]
 77%|███████▋  | 14364/18627 [8:03:55<2:45:16,  2.33s/it]
 77%|███████▋  | 14365/18627 [8:03:57<2:45:32,  2.33s/it]
 77%|███████▋  | 14366/18627 [8:04:00<2:48:06,  2.37s/it]
 77%|███████▋  | 14367/18627 [8:04:02<2:47:46,  2.36s/it]
 77%|███████▋  | 14368/18627 [8:04:04<2:47:11,  2.36s/it]
 77%|███████▋  | 14369/18627 [8:04:06<2:26:24,  2.06s/it]
 77%|███████▋  | 14370/18627 [8:04:08<2:32:03,  2.14s/it]
                                                         
{'loss': 0.8157, 'grad_norm': 6.315755844116211, 'learning_rate': 6.541495770728734e-07, 'epoch': 0.77}
+
 77%|███████▋  | 14370/18627 [8:04:08<2:32:03,  2.14s/it]
 77%|███████▋  | 14371/18627 [8:04:09<2:15:44,  1.91s/it]
 77%|███████▋  | 14372/18627 [8:04:12<2:25:13,  2.05s/it]
 77%|███████▋  | 14373/18627 [8:04:14<2:30:57,  2.13s/it]
 77%|███████▋  | 14374/18627 [8:04:16<2:35:09,  2.19s/it]
 77%|███████▋  | 14375/18627 [8:04:19<2:38:24,  2.24s/it]
 77%|███████▋  | 14376/18627 [8:04:21<2:40:56,  2.27s/it]
 77%|███████▋  | 14377/18627 [8:04:23<2:42:00,  2.29s/it]
 77%|███████▋  | 14378/18627 [8:04:26<2:43:24,  2.31s/it]
 77%|███████▋  | 14379/18627 [8:04:28<2:44:29,  2.32s/it]
 77%|███████▋  | 14380/18627 [8:04:29<2:23:10,  2.02s/it]
                                                         
{'loss': 0.963, 'grad_norm': 13.682597160339355, 'learning_rate': 6.512206945814298e-07, 'epoch': 0.77}
+
 77%|███████▋  | 14380/18627 [8:04:29<2:23:10,  2.02s/it]
 77%|███████▋  | 14381/18627 [8:04:32<2:30:36,  2.13s/it]
 77%|███████▋  | 14382/18627 [8:04:34<2:36:30,  2.21s/it]
 77%|███████▋  | 14383/18627 [8:04:37<2:39:20,  2.25s/it]
 77%|███████▋  | 14384/18627 [8:04:39<2:40:41,  2.27s/it]
 77%|███████▋  | 14385/18627 [8:04:41<2:41:42,  2.29s/it]
 77%|███████▋  | 14386/18627 [8:04:44<2:42:47,  2.30s/it]
 77%|███████▋  | 14387/18627 [8:04:46<2:43:34,  2.31s/it]
 77%|███████▋  | 14388/18627 [8:04:48<2:43:47,  2.32s/it]
 77%|███████▋  | 14389/18627 [8:04:50<2:23:49,  2.04s/it]
 77%|███████▋  | 14390/18627 [8:04:52<2:30:43,  2.13s/it]
                                                         
{'loss': 0.8044, 'grad_norm': 4.152225494384766, 'learning_rate': 6.48297401481656e-07, 'epoch': 0.77}
+
 77%|███████▋  | 14390/18627 [8:04:52<2:30:43,  2.13s/it]
 77%|███████▋  | 14391/18627 [8:04:53<2:08:01,  1.81s/it]
 77%|███████▋  | 14392/18627 [8:04:55<2:19:21,  1.97s/it]
 77%|███████▋  | 14393/18627 [8:04:58<2:27:22,  2.09s/it]
 77%|███████▋  | 14394/18627 [8:04:59<2:12:45,  1.88s/it]
 77%|███████▋  | 14395/18627 [8:05:02<2:23:43,  2.04s/it]
 77%|███████▋  | 14396/18627 [8:05:04<2:30:19,  2.13s/it]
 77%|███████▋  | 14397/18627 [8:05:05<2:14:57,  1.91s/it]
 77%|███████▋  | 14398/18627 [8:05:08<2:23:42,  2.04s/it]
 77%|███████▋  | 14399/18627 [8:05:09<2:09:25,  1.84s/it]
 77%|███████▋  | 14400/18627 [8:05:11<2:20:10,  1.99s/it]
                                                         
{'loss': 1.2063, 'grad_norm': 10.70572280883789, 'learning_rate': 6.453797066115083e-07, 'epoch': 0.77}
+
 77%|███████▋  | 14400/18627 [8:05:11<2:20:10,  1.99s/it]
 77%|███████▋  | 14401/18627 [8:05:14<2:27:18,  2.09s/it]
 77%|███████▋  | 14402/18627 [8:05:16<2:32:31,  2.17s/it]
 77%|███████▋  | 14403/18627 [8:05:17<2:16:01,  1.93s/it]
 77%|███████▋  | 14404/18627 [8:05:20<2:24:45,  2.06s/it]
 77%|███████▋  | 14405/18627 [8:05:21<2:09:36,  1.84s/it]
 77%|███████▋  | 14406/18627 [8:05:23<2:18:59,  1.98s/it]
 77%|███████▋  | 14407/18627 [8:05:26<2:26:38,  2.08s/it]
 77%|███████▋  | 14408/18627 [8:05:28<2:31:56,  2.16s/it]
 77%|███████▋  | 14409/18627 [8:05:29<2:16:20,  1.94s/it]
 77%|███████▋  | 14410/18627 [8:05:32<2:24:26,  2.06s/it]
                                                         
{'loss': 1.1985, 'grad_norm': 5.059110641479492, 'learning_rate': 6.424676187920173e-07, 'epoch': 0.77}
+
 77%|███████▋  | 14410/18627 [8:05:32<2:24:26,  2.06s/it]
 77%|███████▋  | 14411/18627 [8:05:33<2:08:56,  1.84s/it]
 77%|███████▋  | 14412/18627 [8:05:36<2:20:16,  2.00s/it]
 77%|███████▋  | 14413/18627 [8:05:37<2:00:27,  1.72s/it]
 77%|███████▋  | 14414/18627 [8:05:39<2:13:33,  1.90s/it]
 77%|███████▋  | 14415/18627 [8:05:41<2:24:01,  2.05s/it]
 77%|███████▋  | 14416/18627 [8:05:43<2:10:31,  1.86s/it]
 77%|███████▋  | 14417/18627 [8:05:45<2:21:22,  2.01s/it]
 77%|███████▋  | 14418/18627 [8:05:47<2:27:54,  2.11s/it]
 77%|███████▋  | 14419/18627 [8:05:50<2:32:46,  2.18s/it]
 77%|███████▋  | 14420/18627 [8:05:52<2:35:52,  2.22s/it]
                                                         
{'loss': 1.0035, 'grad_norm': 6.054802417755127, 'learning_rate': 6.395611468272625e-07, 'epoch': 0.77}
+
 77%|███████▋  | 14420/18627 [8:05:52<2:35:52,  2.22s/it]
 77%|███████▋  | 14421/18627 [8:05:55<2:41:21,  2.30s/it]
 77%|███████▋  | 14422/18627 [8:05:57<2:41:47,  2.31s/it]
 77%|███████▋  | 14423/18627 [8:05:59<2:42:13,  2.32s/it]
 77%|███████▋  | 14424/18627 [8:06:01<2:21:20,  2.02s/it]
 77%|███████▋  | 14425/18627 [8:06:03<2:27:45,  2.11s/it]
 77%|███████▋  | 14426/18627 [8:06:05<2:32:11,  2.17s/it]
 77%|███████▋  | 14427/18627 [8:06:08<2:35:15,  2.22s/it]
 77%|███████▋  | 14428/18627 [8:06:10<2:37:25,  2.25s/it]
 77%|███████▋  | 14429/18627 [8:06:11<2:19:56,  2.00s/it]
 77%|███████▋  | 14430/18627 [8:06:14<2:27:35,  2.11s/it]
                                                         
{'loss': 1.0446, 'grad_norm': 12.596382141113281, 'learning_rate': 6.366602995043448e-07, 'epoch': 0.77}
+
 77%|███████▋  | 14430/18627 [8:06:14<2:27:35,  2.11s/it]
 77%|███████▋  | 14431/18627 [8:06:16<2:33:20,  2.19s/it]
 77%|███████▋  | 14432/18627 [8:06:17<2:16:50,  1.96s/it]
 77%|███████▋  | 14433/18627 [8:06:20<2:25:04,  2.08s/it]
 77%|███████▋  | 14434/18627 [8:06:22<2:31:07,  2.16s/it]
 77%|███████▋  | 14435/18627 [8:06:25<2:38:14,  2.26s/it]
 78%|███████▊  | 14436/18627 [8:06:27<2:39:51,  2.29s/it]
 78%|███████▊  | 14437/18627 [8:06:29<2:40:24,  2.30s/it]
 78%|███████▊  | 14438/18627 [8:06:31<2:21:33,  2.03s/it]
 78%|███████▊  | 14439/18627 [8:06:33<2:28:22,  2.13s/it]
 78%|███████▊  | 14440/18627 [8:06:35<2:33:28,  2.20s/it]
                                                         
{'loss': 1.0262, 'grad_norm': 5.147801399230957, 'learning_rate': 6.337650855933603e-07, 'epoch': 0.78}
+
 78%|███████▊  | 14440/18627 [8:06:35<2:33:28,  2.20s/it]
 78%|███████▊  | 14441/18627 [8:06:38<2:35:55,  2.23s/it]
 78%|███████▊  | 14442/18627 [8:06:40<2:37:57,  2.26s/it]
 78%|███████▊  | 14443/18627 [8:06:41<2:20:01,  2.01s/it]
 78%|███████▊  | 14444/18627 [8:06:44<2:27:12,  2.11s/it]
 78%|███████▊  | 14445/18627 [8:06:46<2:31:49,  2.18s/it]
 78%|███████▊  | 14446/18627 [8:06:48<2:34:31,  2.22s/it]
 78%|███████▊  | 14447/18627 [8:06:50<2:16:46,  1.96s/it]
 78%|███████▊  | 14448/18627 [8:06:52<2:24:37,  2.08s/it]
 78%|███████▊  | 14449/18627 [8:06:55<2:30:26,  2.16s/it]
 78%|███████▊  | 14450/18627 [8:06:57<2:34:20,  2.22s/it]
                                                         
{'loss': 1.0003, 'grad_norm': 5.56813383102417, 'learning_rate': 6.308755138473718e-07, 'epoch': 0.78}
+
 78%|███████▊  | 14450/18627 [8:06:57<2:34:20,  2.22s/it]
 78%|███████▊  | 14451/18627 [8:06:59<2:37:04,  2.26s/it]
 78%|███████▊  | 14452/18627 [8:07:02<2:38:20,  2.28s/it]
 78%|███████▊  | 14453/18627 [8:07:04<2:39:22,  2.29s/it]
 78%|███████▊  | 14454/18627 [8:07:06<2:41:58,  2.33s/it]
 78%|███████▊  | 14455/18627 [8:07:09<2:42:36,  2.34s/it]
 78%|███████▊  | 14456/18627 [8:07:11<2:42:45,  2.34s/it]
 78%|███████▊  | 14457/18627 [8:07:13<2:41:58,  2.33s/it]
 78%|███████▊  | 14458/18627 [8:07:16<2:42:18,  2.34s/it]
 78%|███████▊  | 14459/18627 [8:07:18<2:42:44,  2.34s/it]
 78%|███████▊  | 14460/18627 [8:07:20<2:43:07,  2.35s/it]
                                                         
{'loss': 0.5726, 'grad_norm': 9.963966369628906, 'learning_rate': 6.279915930023886e-07, 'epoch': 0.78}
+
 78%|███████▊  | 14460/18627 [8:07:20<2:43:07,  2.35s/it]
 78%|███████▊  | 14461/18627 [8:07:23<2:42:38,  2.34s/it]
 78%|███████▊  | 14462/18627 [8:07:25<2:42:00,  2.33s/it]
 78%|███████▊  | 14463/18627 [8:07:27<2:41:51,  2.33s/it]
 78%|███████▊  | 14464/18627 [8:07:30<2:41:33,  2.33s/it]
 78%|███████▊  | 14465/18627 [8:07:32<2:41:56,  2.33s/it]
 78%|███████▊  | 14466/18627 [8:07:34<2:42:00,  2.34s/it]
 78%|███████▊  | 14467/18627 [8:07:37<2:42:05,  2.34s/it]
 78%|███████▊  | 14468/18627 [8:07:39<2:41:53,  2.34s/it]
 78%|███████▊  | 14469/18627 [8:07:40<2:22:26,  2.06s/it]
 78%|███████▊  | 14470/18627 [8:07:43<2:28:13,  2.14s/it]
                                                         
{'loss': 0.8426, 'grad_norm': 5.578929424285889, 'learning_rate': 6.251133317773312e-07, 'epoch': 0.78}
+
 78%|███████▊  | 14470/18627 [8:07:43<2:28:13,  2.14s/it]
 78%|███████▊  | 14471/18627 [8:07:45<2:32:45,  2.21s/it]
 78%|███████▊  | 14472/18627 [8:07:47<2:35:27,  2.24s/it]
 78%|███████▊  | 14473/18627 [8:07:50<2:37:50,  2.28s/it]
 78%|███████▊  | 14474/18627 [8:07:52<2:39:23,  2.30s/it]
 78%|███████▊  | 14475/18627 [8:07:55<2:45:08,  2.39s/it]
 78%|███████▊  | 14476/18627 [8:07:56<2:24:31,  2.09s/it]
 78%|███████▊  | 14477/18627 [8:07:59<2:30:06,  2.17s/it]
 78%|███████▊  | 14478/18627 [8:08:01<2:34:42,  2.24s/it]
 78%|███████▊  | 14479/18627 [8:08:03<2:36:17,  2.26s/it]
 78%|███████▊  | 14480/18627 [8:08:05<2:17:46,  1.99s/it]
                                                         
{'loss': 1.036, 'grad_norm': 14.503446578979492, 'learning_rate': 6.222407388740115e-07, 'epoch': 0.78}
+
 78%|███████▊  | 14480/18627 [8:08:05<2:17:46,  1.99s/it]
 78%|███████▊  | 14481/18627 [8:08:07<2:25:00,  2.10s/it]
 78%|███████▊  | 14482/18627 [8:08:09<2:29:44,  2.17s/it]
 78%|███████▊  | 14483/18627 [8:08:12<2:33:20,  2.22s/it]
 78%|███████▊  | 14484/18627 [8:08:13<2:16:20,  1.97s/it]
 78%|███████▊  | 14485/18627 [8:08:15<2:24:26,  2.09s/it]
 78%|███████▊  | 14486/18627 [8:08:18<2:28:58,  2.16s/it]
 78%|███████▊  | 14487/18627 [8:08:20<2:31:47,  2.20s/it]
 78%|███████▊  | 14488/18627 [8:08:22<2:34:40,  2.24s/it]
 78%|███████▊  | 14489/18627 [8:08:25<2:36:37,  2.27s/it]
 78%|███████▊  | 14490/18627 [8:08:27<2:38:25,  2.30s/it]
                                                         
{'loss': 0.8269, 'grad_norm': 10.30693244934082, 'learning_rate': 6.193738229771043e-07, 'epoch': 0.78}
+
 78%|███████▊  | 14490/18627 [8:08:27<2:38:25,  2.30s/it]
 78%|███████▊  | 14491/18627 [8:08:29<2:39:29,  2.31s/it]
 78%|███████▊  | 14492/18627 [8:08:31<2:19:55,  2.03s/it]
 78%|███████▊  | 14493/18627 [8:08:33<2:25:32,  2.11s/it]
 78%|███████▊  | 14494/18627 [8:08:35<2:30:22,  2.18s/it]
 78%|███████▊  | 14495/18627 [8:08:37<2:13:47,  1.94s/it]
 78%|███████▊  | 14496/18627 [8:08:39<2:21:30,  2.06s/it]
 78%|███████▊  | 14497/18627 [8:08:41<2:27:20,  2.14s/it]
 78%|███████▊  | 14498/18627 [8:08:44<2:30:40,  2.19s/it]
 78%|███████▊  | 14499/18627 [8:08:45<2:14:02,  1.95s/it]
 78%|███████▊  | 14500/18627 [8:08:47<2:02:08,  1.78s/it]
                                                         
{'loss': 1.3341, 'grad_norm': 14.127557754516602, 'learning_rate': 6.165125927541216e-07, 'epoch': 0.78}
+
 78%|███████▊  | 14500/18627 [8:08:47<2:02:08,  1.78s/it]
 78%|███████▊  | 14501/18627 [8:08:49<2:14:42,  1.96s/it]
 78%|███████▊  | 14502/18627 [8:08:51<2:22:25,  2.07s/it]
 78%|███████▊  | 14503/18627 [8:08:54<2:27:31,  2.15s/it]
 78%|███████▊  | 14504/18627 [8:08:56<2:31:13,  2.20s/it]
 78%|███████▊  | 14505/18627 [8:08:58<2:33:38,  2.24s/it]
 78%|███████▊  | 14506/18627 [8:09:01<2:36:27,  2.28s/it]
 78%|███████▊  | 14507/18627 [8:09:03<2:38:30,  2.31s/it]
 78%|███████▊  | 14508/18627 [8:09:05<2:39:18,  2.32s/it]
 78%|███████▊  | 14509/18627 [8:09:08<2:39:23,  2.32s/it]
 78%|███████▊  | 14510/18627 [8:09:09<2:20:22,  2.05s/it]
                                                         
{'loss': 0.7898, 'grad_norm': 13.651432037353516, 'learning_rate': 6.136570568553848e-07, 'epoch': 0.78}
+
 78%|███████▊  | 14510/18627 [8:09:09<2:20:22,  2.05s/it]
 78%|███████▊  | 14511/18627 [8:09:11<2:27:52,  2.16s/it]
 78%|███████▊  | 14512/18627 [8:09:14<2:31:28,  2.21s/it]
 78%|███████▊  | 14513/18627 [8:09:16<2:33:42,  2.24s/it]
 78%|███████▊  | 14514/18627 [8:09:18<2:16:42,  1.99s/it]
 78%|███████▊  | 14515/18627 [8:09:20<2:24:29,  2.11s/it]
 78%|███████▊  | 14516/18627 [8:09:22<2:29:35,  2.18s/it]
 78%|███████▊  | 14517/18627 [8:09:25<2:33:14,  2.24s/it]
 78%|███████▊  | 14518/18627 [8:09:27<2:35:20,  2.27s/it]
 78%|███████▊  | 14519/18627 [8:09:29<2:36:35,  2.29s/it]
 78%|███████▊  | 14520/18627 [8:09:32<2:37:30,  2.30s/it]
                                                         
{'loss': 0.8168, 'grad_norm': 6.5036301612854, 'learning_rate': 6.108072239140012e-07, 'epoch': 0.78}
+
 78%|███████▊  | 14520/18627 [8:09:32<2:37:30,  2.30s/it]
 78%|███████▊  | 14521/18627 [8:09:34<2:37:44,  2.31s/it]
 78%|███████▊  | 14522/18627 [8:09:36<2:39:26,  2.33s/it]
 78%|███████▊  | 14523/18627 [8:09:39<2:40:31,  2.35s/it]
 78%|███████▊  | 14524/18627 [8:09:41<2:39:55,  2.34s/it]
 78%|███████▊  | 14525/18627 [8:09:43<2:40:05,  2.34s/it]
 78%|███████▊  | 14526/18627 [8:09:46<2:40:24,  2.35s/it]
 78%|███████▊  | 14527/18627 [8:09:48<2:39:59,  2.34s/it]
 78%|███████▊  | 14528/18627 [8:09:50<2:39:54,  2.34s/it]
 78%|███████▊  | 14529/18627 [8:09:52<2:20:05,  2.05s/it]
 78%|███████▊  | 14530/18627 [8:09:54<2:25:51,  2.14s/it]
                                                         
{'loss': 0.8541, 'grad_norm': 9.995062828063965, 'learning_rate': 6.079631025458338e-07, 'epoch': 0.78}
+
 78%|███████▊  | 14530/18627 [8:09:54<2:25:51,  2.14s/it]
 78%|███████▊  | 14531/18627 [8:09:56<2:12:11,  1.94s/it]
 78%|███████▊  | 14532/18627 [8:09:58<2:19:57,  2.05s/it]
 78%|███████▊  | 14533/18627 [8:10:00<2:25:47,  2.14s/it]
 78%|███████▊  | 14534/18627 [8:10:03<2:30:28,  2.21s/it]
 78%|███████▊  | 14535/18627 [8:10:05<2:33:24,  2.25s/it]
 78%|███████▊  | 14536/18627 [8:10:07<2:35:26,  2.28s/it]
 78%|███████▊  | 14537/18627 [8:10:10<2:36:43,  2.30s/it]
 78%|███████▊  | 14538/18627 [8:10:12<2:37:40,  2.31s/it]
 78%|███████▊  | 14539/18627 [8:10:14<2:38:21,  2.32s/it]
 78%|███████▊  | 14540/18627 [8:10:17<2:38:47,  2.33s/it]
                                                         
{'loss': 0.7842, 'grad_norm': 4.836038112640381, 'learning_rate': 6.05124701349482e-07, 'epoch': 0.78}
+
 78%|███████▊  | 14540/18627 [8:10:17<2:38:47,  2.33s/it]
 78%|███████▊  | 14541/18627 [8:10:19<2:38:45,  2.33s/it]
 78%|███████▊  | 14542/18627 [8:10:21<2:38:39,  2.33s/it]
 78%|███████▊  | 14543/18627 [8:10:24<2:38:21,  2.33s/it]
 78%|███████▊  | 14544/18627 [8:10:26<2:39:02,  2.34s/it]
 78%|███████▊  | 14545/18627 [8:10:28<2:39:12,  2.34s/it]
 78%|███████▊  | 14546/18627 [8:10:31<2:39:46,  2.35s/it]
 78%|███████▊  | 14547/18627 [8:10:32<2:20:06,  2.06s/it]
 78%|███████▊  | 14548/18627 [8:10:34<2:25:13,  2.14s/it]
 78%|███████▊  | 14549/18627 [8:10:36<2:09:49,  1.91s/it]
 78%|███████▊  | 14550/18627 [8:10:38<2:18:43,  2.04s/it]
                                                         
{'loss': 1.0605, 'grad_norm': 6.333088397979736, 'learning_rate': 6.022920289062472e-07, 'epoch': 0.78}
+
 78%|███████▊  | 14550/18627 [8:10:38<2:18:43,  2.04s/it]
 78%|███████▊  | 14551/18627 [8:10:41<2:24:57,  2.13s/it]
 78%|███████▊  | 14552/18627 [8:10:43<2:30:01,  2.21s/it]
 78%|███████▊  | 14553/18627 [8:10:44<2:14:38,  1.98s/it]
 78%|███████▊  | 14554/18627 [8:10:46<2:04:12,  1.83s/it]
 78%|███████▊  | 14555/18627 [8:10:47<1:55:08,  1.70s/it]
 78%|███████▊  | 14556/18627 [8:10:50<2:08:39,  1.90s/it]
 78%|███████▊  | 14557/18627 [8:10:52<2:17:39,  2.03s/it]
 78%|███████▊  | 14558/18627 [8:10:54<2:24:13,  2.13s/it]
 78%|███████▊  | 14559/18627 [8:10:57<2:28:21,  2.19s/it]
 78%|███████▊  | 14560/18627 [8:10:59<2:31:04,  2.23s/it]
                                                         
{'loss': 1.1092, 'grad_norm': 6.1778435707092285, 'learning_rate': 5.994650937801141e-07, 'epoch': 0.78}
+
 78%|███████▊  | 14560/18627 [8:10:59<2:31:04,  2.23s/it]
 78%|███████▊  | 14561/18627 [8:11:01<2:33:41,  2.27s/it]
 78%|███████▊  | 14562/18627 [8:11:04<2:35:11,  2.29s/it]
 78%|███████▊  | 14563/18627 [8:11:06<2:36:18,  2.31s/it]
 78%|███████▊  | 14564/18627 [8:11:08<2:37:01,  2.32s/it]
 78%|███████▊  | 14565/18627 [8:11:11<2:37:07,  2.32s/it]
 78%|███████▊  | 14566/18627 [8:11:13<2:36:47,  2.32s/it]
 78%|███████▊  | 14567/18627 [8:11:15<2:37:00,  2.32s/it]
 78%|███████▊  | 14568/18627 [8:11:18<2:37:51,  2.33s/it]
 78%|███████▊  | 14569/18627 [8:11:20<2:38:15,  2.34s/it]
 78%|███████▊  | 14570/18627 [8:11:22<2:37:55,  2.34s/it]
                                                         
{'loss': 0.6042, 'grad_norm': 4.797774791717529, 'learning_rate': 5.966439045177203e-07, 'epoch': 0.78}
+
 78%|███████▊  | 14570/18627 [8:11:22<2:37:55,  2.34s/it]
 78%|███████▊  | 14571/18627 [8:11:24<2:18:39,  2.05s/it]
 78%|███████▊  | 14572/18627 [8:11:25<2:04:59,  1.85s/it]
 78%|███████▊  | 14573/18627 [8:11:27<2:15:47,  2.01s/it]
 78%|███████▊  | 14574/18627 [8:11:30<2:22:36,  2.11s/it]
 78%|███████▊  | 14575/18627 [8:11:32<2:27:07,  2.18s/it]
 78%|███████▊  | 14576/18627 [8:11:35<2:30:20,  2.23s/it]
 78%|███████▊  | 14577/18627 [8:11:37<2:32:53,  2.27s/it]
 78%|███████▊  | 14578/18627 [8:11:39<2:34:40,  2.29s/it]
 78%|███████▊  | 14579/18627 [8:11:42<2:35:29,  2.30s/it]
 78%|███████▊  | 14580/18627 [8:11:44<2:35:39,  2.31s/it]
                                                         
{'loss': 1.0258, 'grad_norm': 9.564352989196777, 'learning_rate': 5.938284696483326e-07, 'epoch': 0.78}
+
 78%|███████▊  | 14580/18627 [8:11:44<2:35:39,  2.31s/it]
 78%|███████▊  | 14581/18627 [8:11:46<2:37:35,  2.34s/it]
 78%|███████▊  | 14582/18627 [8:11:49<2:37:50,  2.34s/it]
 78%|███████▊  | 14583/18627 [8:11:51<2:38:05,  2.35s/it]
 78%|███████▊  | 14584/18627 [8:11:53<2:38:08,  2.35s/it]
 78%|███████▊  | 14585/18627 [8:11:56<2:37:44,  2.34s/it]
 78%|███████▊  | 14586/18627 [8:11:58<2:37:09,  2.33s/it]
 78%|███████▊  | 14587/18627 [8:12:00<2:36:59,  2.33s/it]
 78%|███████▊  | 14588/18627 [8:12:03<2:36:42,  2.33s/it]
 78%|███████▊  | 14589/18627 [8:12:05<2:36:36,  2.33s/it]
 78%|███████▊  | 14590/18627 [8:12:07<2:37:41,  2.34s/it]
                                                         
{'loss': 0.6038, 'grad_norm': 6.39804744720459, 'learning_rate': 5.910187976838209e-07, 'epoch': 0.78}
+
 78%|███████▊  | 14590/18627 [8:12:07<2:37:41,  2.34s/it]
 78%|███████▊  | 14591/18627 [8:12:10<2:37:19,  2.34s/it]
 78%|███████▊  | 14592/18627 [8:12:12<2:36:56,  2.33s/it]
 78%|███████▊  | 14593/18627 [8:12:14<2:37:11,  2.34s/it]
 78%|███████▊  | 14594/18627 [8:12:17<2:37:01,  2.34s/it]
 78%|███████▊  | 14595/18627 [8:12:19<2:37:03,  2.34s/it]
 78%|███████▊  | 14596/18627 [8:12:21<2:36:24,  2.33s/it]
 78%|███████▊  | 14597/18627 [8:12:24<2:37:05,  2.34s/it]
 78%|███████▊  | 14598/18627 [8:12:26<2:37:21,  2.34s/it]
 78%|███████▊  | 14599/18627 [8:12:28<2:36:35,  2.33s/it]
 78%|███████▊  | 14600/18627 [8:12:31<2:37:51,  2.35s/it]
                                                         
{'loss': 0.5362, 'grad_norm': 4.713473320007324, 'learning_rate': 5.882148971186321e-07, 'epoch': 0.78}
+
 78%|███████▊  | 14600/18627 [8:12:31<2:37:51,  2.35s/it]
 78%|███████▊  | 14601/18627 [8:12:33<2:37:50,  2.35s/it]
 78%|███████▊  | 14602/18627 [8:12:35<2:38:06,  2.36s/it]
 78%|███████▊  | 14603/18627 [8:12:37<2:11:39,  1.96s/it]
 78%|███████▊  | 14604/18627 [8:12:39<2:19:11,  2.08s/it]
 78%|███████▊  | 14605/18627 [8:12:41<2:24:44,  2.16s/it]
 78%|███████▊  | 14606/18627 [8:12:44<2:28:27,  2.22s/it]
 78%|███████▊  | 14607/18627 [8:12:46<2:31:36,  2.26s/it]
 78%|███████▊  | 14608/18627 [8:12:48<2:33:37,  2.29s/it]
 78%|███████▊  | 14609/18627 [8:12:51<2:35:16,  2.32s/it]
 78%|███████▊  | 14610/18627 [8:12:53<2:35:03,  2.32s/it]
                                                         
{'loss': 0.6607, 'grad_norm': 6.555142402648926, 'learning_rate': 5.854167764297622e-07, 'epoch': 0.78}
+
 78%|███████▊  | 14610/18627 [8:12:53<2:35:03,  2.32s/it]
 78%|███████▊  | 14611/18627 [8:12:54<2:17:15,  2.05s/it]
 78%|███████▊  | 14612/18627 [8:12:56<2:04:12,  1.86s/it]
 78%|███████▊  | 14613/18627 [8:12:58<2:13:17,  1.99s/it]
 78%|███████▊  | 14614/18627 [8:12:59<2:00:51,  1.81s/it]
 78%|███████▊  | 14615/18627 [8:13:02<2:14:32,  2.01s/it]
 78%|███████▊  | 14616/18627 [8:13:04<2:20:55,  2.11s/it]
 78%|███████▊  | 14617/18627 [8:13:06<2:07:00,  1.90s/it]
 78%|███████▊  | 14618/18627 [8:13:08<2:16:14,  2.04s/it]
 78%|███████▊  | 14619/18627 [8:13:10<2:22:18,  2.13s/it]
 78%|███████▊  | 14620/18627 [8:13:13<2:25:49,  2.18s/it]
                                                         
{'loss': 1.5105, 'grad_norm': 6.934569835662842, 'learning_rate': 5.826244440767376e-07, 'epoch': 0.78}
+
 78%|███████▊  | 14620/18627 [8:13:13<2:25:49,  2.18s/it]
 78%|███████▊  | 14621/18627 [8:13:15<2:28:22,  2.22s/it]
 78%|███████▊  | 14622/18627 [8:13:17<2:30:44,  2.26s/it]
 79%|███████▊  | 14623/18627 [8:13:20<2:32:35,  2.29s/it]
 79%|███████▊  | 14624/18627 [8:13:22<2:33:07,  2.30s/it]
 79%|███████▊  | 14625/18627 [8:13:24<2:33:13,  2.30s/it]
 79%|███████▊  | 14626/18627 [8:13:26<2:15:07,  2.03s/it]
 79%|███████▊  | 14627/18627 [8:13:28<2:21:19,  2.12s/it]
 79%|███████▊  | 14628/18627 [8:13:30<2:25:58,  2.19s/it]
 79%|███████▊  | 14629/18627 [8:13:32<2:10:18,  1.96s/it]
 79%|███████▊  | 14630/18627 [8:13:34<2:19:06,  2.09s/it]
                                                         
{'loss': 1.0202, 'grad_norm': 5.729149341583252, 'learning_rate': 5.798379085015804e-07, 'epoch': 0.79}
+
 79%|███████▊  | 14630/18627 [8:13:34<2:19:06,  2.09s/it]
 79%|███████▊  | 14631/18627 [8:13:37<2:24:25,  2.17s/it]
 79%|███████▊  | 14632/18627 [8:13:39<2:27:14,  2.21s/it]
 79%|███████▊  | 14633/18627 [8:13:41<2:30:02,  2.25s/it]
 79%|███████▊  | 14634/18627 [8:13:44<2:31:49,  2.28s/it]
 79%|███████▊  | 14635/18627 [8:13:46<2:33:37,  2.31s/it]
 79%|███████▊  | 14636/18627 [8:13:48<2:34:09,  2.32s/it]
 79%|███████▊  | 14637/18627 [8:13:51<2:34:26,  2.32s/it]
 79%|███████▊  | 14638/18627 [8:13:56<3:30:36,  3.17s/it]
 79%|███████▊  | 14639/18627 [8:13:58<3:14:02,  2.92s/it]
 79%|███████▊  | 14640/18627 [8:14:00<3:02:09,  2.74s/it]
                                                         
{'loss': 0.6313, 'grad_norm': 6.059198379516602, 'learning_rate': 5.770571781287901e-07, 'epoch': 0.79}
+
 79%|███████▊  | 14640/18627 [8:14:00<3:02:09,  2.74s/it]
 79%|███████▊  | 14641/18627 [8:14:03<2:54:21,  2.62s/it]
 79%|███████▊  | 14642/18627 [8:14:05<2:48:31,  2.54s/it]
 79%|███████▊  | 14643/18627 [8:14:07<2:43:32,  2.46s/it]
 79%|███████▊  | 14644/18627 [8:14:10<2:41:09,  2.43s/it]
 79%|███████▊  | 14645/18627 [8:14:12<2:39:36,  2.40s/it]
 79%|███████▊  | 14646/18627 [8:14:15<2:42:45,  2.45s/it]
 79%|███████▊  | 14647/18627 [8:14:17<2:40:16,  2.42s/it]
 79%|███████▊  | 14648/18627 [8:14:19<2:38:43,  2.39s/it]
 79%|███████▊  | 14649/18627 [8:14:22<2:37:37,  2.38s/it]
 79%|███████▊  | 14650/18627 [8:14:24<2:36:41,  2.36s/it]
                                                         
{'loss': 0.655, 'grad_norm': 7.271918296813965, 'learning_rate': 5.742822613653141e-07, 'epoch': 0.79}
+
 79%|███████▊  | 14650/18627 [8:14:24<2:36:41,  2.36s/it]
 79%|███████▊  | 14651/18627 [8:14:25<2:17:17,  2.07s/it]
 79%|███████▊  | 14652/18627 [8:14:28<2:21:21,  2.13s/it]
 79%|███████▊  | 14653/18627 [8:14:30<2:25:11,  2.19s/it]
 79%|███████▊  | 14654/18627 [8:14:32<2:27:40,  2.23s/it]
 79%|███████▊  | 14655/18627 [8:14:35<2:29:20,  2.26s/it]
 79%|███████▊  | 14656/18627 [8:14:37<2:30:44,  2.28s/it]
 79%|███████▊  | 14657/18627 [8:14:39<2:32:31,  2.31s/it]
 79%|███████▊  | 14658/18627 [8:14:42<2:32:35,  2.31s/it]
 79%|███████▊  | 14659/18627 [8:14:43<2:13:24,  2.02s/it]
 79%|███████▊  | 14660/18627 [8:14:44<2:01:25,  1.84s/it]
                                                         
{'loss': 1.2268, 'grad_norm': 14.146615028381348, 'learning_rate': 5.715131666005242e-07, 'epoch': 0.79}
+
 79%|███████▊  | 14660/18627 [8:14:44<2:01:25,  1.84s/it]
 79%|███████▊  | 14661/18627 [8:14:47<2:12:17,  2.00s/it]
 79%|███████▊  | 14662/18627 [8:14:48<1:52:40,  1.71s/it]
 79%|███████▊  | 14663/18627 [8:14:49<1:44:19,  1.58s/it]
 79%|███████▊  | 14664/18627 [8:14:51<1:59:37,  1.81s/it]
 79%|███████▊  | 14665/18627 [8:14:54<2:09:54,  1.97s/it]
 79%|███████▊  | 14666/18627 [8:14:56<2:17:06,  2.08s/it]
 79%|███████▊  | 14667/18627 [8:14:58<2:03:51,  1.88s/it]
 79%|███████▊  | 14668/18627 [8:15:00<2:14:43,  2.04s/it]
 79%|███████▉  | 14669/18627 [8:15:02<2:20:48,  2.13s/it]
 79%|███████▉  | 14670/18627 [8:15:05<2:24:49,  2.20s/it]
                                                         
{'loss': 1.0907, 'grad_norm': 5.835081577301025, 'learning_rate': 5.687499022061904e-07, 'epoch': 0.79}
+
 79%|███████▉  | 14670/18627 [8:15:05<2:24:49,  2.20s/it]
 79%|███████▉  | 14671/18627 [8:15:07<2:27:40,  2.24s/it]
 79%|███████▉  | 14672/18627 [8:15:09<2:29:40,  2.27s/it]
 79%|███████▉  | 14673/18627 [8:15:12<2:30:48,  2.29s/it]
 79%|███████▉  | 14674/18627 [8:15:14<2:31:41,  2.30s/it]
 79%|███████▉  | 14675/18627 [8:15:16<2:32:07,  2.31s/it]
 79%|███████▉  | 14676/18627 [8:15:19<2:32:37,  2.32s/it]
 79%|███████▉  | 14677/18627 [8:15:21<2:32:53,  2.32s/it]
 79%|███████▉  | 14678/18627 [8:15:22<2:15:46,  2.06s/it]
 79%|███████▉  | 14679/18627 [8:15:25<2:21:22,  2.15s/it]
 79%|███████▉  | 14680/18627 [8:15:26<2:05:39,  1.91s/it]
                                                         
{'loss': 1.0633, 'grad_norm': 15.315571784973145, 'learning_rate': 5.65992476536456e-07, 'epoch': 0.79}
+
 79%|███████▉  | 14680/18627 [8:15:26<2:05:39,  1.91s/it]
 79%|███████▉  | 14681/18627 [8:15:29<2:13:37,  2.03s/it]
 79%|███████▉  | 14682/18627 [8:15:31<2:19:50,  2.13s/it]
 79%|███████▉  | 14683/18627 [8:15:33<2:23:29,  2.18s/it]
 79%|███████▉  | 14684/18627 [8:15:35<2:06:49,  1.93s/it]
 79%|███████▉  | 14685/18627 [8:15:36<1:56:15,  1.77s/it]
 79%|███████▉  | 14686/18627 [8:15:38<2:06:57,  1.93s/it]
 79%|███████▉  | 14687/18627 [8:15:41<2:15:19,  2.06s/it]
 79%|███████▉  | 14688/18627 [8:15:43<2:20:04,  2.13s/it]
 79%|███████▉  | 14689/18627 [8:15:45<2:24:27,  2.20s/it]
 79%|███████▉  | 14690/18627 [8:15:48<2:27:55,  2.25s/it]
                                                         
{'loss': 0.9281, 'grad_norm': 4.5801920890808105, 'learning_rate': 5.63240897927812e-07, 'epoch': 0.79}
+
 79%|███████▉  | 14690/18627 [8:15:48<2:27:55,  2.25s/it]
 79%|███████▉  | 14691/18627 [8:15:50<2:29:56,  2.29s/it]
 79%|███████▉  | 14692/18627 [8:15:52<2:31:52,  2.32s/it]
 79%|███████▉  | 14693/18627 [8:15:55<2:31:44,  2.31s/it]
 79%|███████▉  | 14694/18627 [8:15:57<2:32:12,  2.32s/it]
 79%|███████▉  | 14695/18627 [8:15:59<2:32:37,  2.33s/it]
 79%|███████▉  | 14696/18627 [8:16:01<2:14:36,  2.05s/it]
 79%|███████▉  | 14697/18627 [8:16:02<1:55:48,  1.77s/it]
 79%|███████▉  | 14698/18627 [8:16:04<2:06:13,  1.93s/it]
 79%|███████▉  | 14699/18627 [8:16:06<1:55:11,  1.76s/it]
 79%|███████▉  | 14700/18627 [8:16:07<1:47:50,  1.65s/it]
                                                         
{'loss': 1.1764, 'grad_norm': 14.99062728881836, 'learning_rate': 5.604951746990719e-07, 'epoch': 0.79}
+
 79%|███████▉  | 14700/18627 [8:16:07<1:47:50,  1.65s/it]
 79%|███████▉  | 14701/18627 [8:16:08<1:42:09,  1.56s/it]
 79%|███████▉  | 14702/18627 [8:16:11<1:56:56,  1.79s/it]
 79%|███████▉  | 14703/18627 [8:16:13<2:06:49,  1.94s/it]
 79%|███████▉  | 14704/18627 [8:16:15<2:14:56,  2.06s/it]
 79%|███████▉  | 14705/18627 [8:16:16<1:54:45,  1.76s/it]
 79%|███████▉  | 14706/18627 [8:16:19<2:06:14,  1.93s/it]
 79%|███████▉  | 14707/18627 [8:16:20<1:55:43,  1.77s/it]
 79%|███████▉  | 14708/18627 [8:16:22<2:06:32,  1.94s/it]
 79%|███████▉  | 14709/18627 [8:16:25<2:13:49,  2.05s/it]
 79%|███████▉  | 14710/18627 [8:16:27<2:19:50,  2.14s/it]
                                                         
{'loss': 0.9656, 'grad_norm': 6.696996688842773, 'learning_rate': 5.577553151513479e-07, 'epoch': 0.79}
+
 79%|███████▉  | 14710/18627 [8:16:27<2:19:50,  2.14s/it]
 79%|███████▉  | 14711/18627 [8:16:29<2:23:39,  2.20s/it]
 79%|███████▉  | 14712/18627 [8:16:32<2:26:08,  2.24s/it]
 79%|███████▉  | 14713/18627 [8:16:34<2:27:40,  2.26s/it]
 79%|███████▉  | 14714/18627 [8:16:35<2:09:43,  1.99s/it]
 79%|███████▉  | 14715/18627 [8:16:38<2:16:16,  2.09s/it]
 79%|███████▉  | 14716/18627 [8:16:40<2:20:24,  2.15s/it]
 79%|███████▉  | 14717/18627 [8:16:41<1:58:15,  1.81s/it]
 79%|███████▉  | 14718/18627 [8:16:42<1:49:24,  1.68s/it]
 79%|███████▉  | 14719/18627 [8:16:45<2:02:13,  1.88s/it]
 79%|███████▉  | 14720/18627 [8:16:47<2:11:19,  2.02s/it]
                                                         
{'loss': 1.0242, 'grad_norm': 6.302368640899658, 'learning_rate': 5.550213275680211e-07, 'epoch': 0.79}
+
 79%|███████▉  | 14720/18627 [8:16:47<2:11:19,  2.02s/it]
 79%|███████▉  | 14721/18627 [8:16:48<1:50:25,  1.70s/it]
 79%|███████▉  | 14722/18627 [8:16:50<2:03:00,  1.89s/it]
 79%|███████▉  | 14723/18627 [8:16:53<2:11:32,  2.02s/it]
 79%|███████▉  | 14724/18627 [8:16:55<2:18:00,  2.12s/it]
 79%|███████▉  | 14725/18627 [8:16:57<2:22:40,  2.19s/it]
 79%|███████▉  | 14726/18627 [8:17:00<2:26:13,  2.25s/it]
 79%|███████▉  | 14727/18627 [8:17:02<2:28:36,  2.29s/it]
 79%|███████▉  | 14728/18627 [8:17:05<2:33:24,  2.36s/it]
 79%|███████▉  | 14729/18627 [8:17:07<2:33:02,  2.36s/it]
 79%|███████▉  | 14730/18627 [8:17:09<2:33:31,  2.36s/it]
                                                         
{'loss': 0.6361, 'grad_norm': 7.835973739624023, 'learning_rate': 5.522932202147252e-07, 'epoch': 0.79}
+
 79%|███████▉  | 14730/18627 [8:17:09<2:33:31,  2.36s/it]
 79%|███████▉  | 14731/18627 [8:17:12<2:33:30,  2.36s/it]
 79%|███████▉  | 14732/18627 [8:17:14<2:33:25,  2.36s/it]
 79%|███████▉  | 14733/18627 [8:17:16<2:14:52,  2.08s/it]
 79%|███████▉  | 14734/18627 [8:17:18<2:20:11,  2.16s/it]
 79%|███████▉  | 14735/18627 [8:17:20<2:23:38,  2.21s/it]
 79%|███████▉  | 14736/18627 [8:17:23<2:24:54,  2.23s/it]
 79%|███████▉  | 14737/18627 [8:17:25<2:26:55,  2.27s/it]
 79%|███████▉  | 14738/18627 [8:17:27<2:27:59,  2.28s/it]
 79%|███████▉  | 14739/18627 [8:17:29<2:14:14,  2.07s/it]
 79%|███████▉  | 14740/18627 [8:17:31<2:19:22,  2.15s/it]
                                                         
{'loss': 0.9753, 'grad_norm': 7.245169162750244, 'learning_rate': 5.495710013393118e-07, 'epoch': 0.79}
+
 79%|███████▉  | 14740/18627 [8:17:31<2:19:22,  2.15s/it]
 79%|███████▉  | 14741/18627 [8:17:33<2:07:32,  1.97s/it]
 79%|███████▉  | 14742/18627 [8:17:34<1:55:53,  1.79s/it]
 79%|███████▉  | 14743/18627 [8:17:36<2:06:33,  1.95s/it]
 79%|███████▉  | 14744/18627 [8:17:39<2:14:29,  2.08s/it]
 79%|███████▉  | 14745/18627 [8:17:41<2:19:34,  2.16s/it]
 79%|███████▉  | 14746/18627 [8:17:43<2:22:57,  2.21s/it]
 79%|███████▉  | 14747/18627 [8:17:46<2:25:37,  2.25s/it]
 79%|███████▉  | 14748/18627 [8:17:48<2:27:16,  2.28s/it]
 79%|███████▉  | 14749/18627 [8:17:50<2:28:39,  2.30s/it]
 79%|███████▉  | 14750/18627 [8:17:53<2:27:26,  2.28s/it]
                                                         
{'loss': 1.0695, 'grad_norm': 8.01725959777832, 'learning_rate': 5.468546791718321e-07, 'epoch': 0.79}
+
 79%|███████▉  | 14750/18627 [8:17:53<2:27:26,  2.28s/it]
 79%|███████▉  | 14751/18627 [8:17:55<2:27:55,  2.29s/it]
 79%|███████▉  | 14752/18627 [8:17:57<2:28:46,  2.30s/it]
 79%|███████▉  | 14753/18627 [8:18:00<2:29:58,  2.32s/it]
 79%|███████▉  | 14754/18627 [8:18:02<2:29:19,  2.31s/it]
 79%|███████▉  | 14755/18627 [8:18:04<2:29:26,  2.32s/it]
 79%|███████▉  | 14756/18627 [8:18:07<2:30:05,  2.33s/it]
 79%|███████▉  | 14757/18627 [8:18:09<2:30:35,  2.33s/it]
 79%|███████▉  | 14758/18627 [8:18:11<2:31:16,  2.35s/it]
 79%|███████▉  | 14759/18627 [8:18:13<2:12:53,  2.06s/it]
 79%|███████▉  | 14760/18627 [8:18:15<2:18:30,  2.15s/it]
                                                         
{'loss': 0.8281, 'grad_norm': 8.593071937561035, 'learning_rate': 5.441442619245096e-07, 'epoch': 0.79}
+
 79%|███████▉  | 14760/18627 [8:18:15<2:18:30,  2.15s/it]
 79%|███████▉  | 14761/18627 [8:18:17<2:21:49,  2.20s/it]
 79%|███████▉  | 14762/18627 [8:18:20<2:24:16,  2.24s/it]
 79%|███████▉  | 14763/18627 [8:18:22<2:25:51,  2.26s/it]
 79%|███████▉  | 14764/18627 [8:18:25<2:32:49,  2.37s/it]
 79%|███████▉  | 14765/18627 [8:18:27<2:32:08,  2.36s/it]
 79%|███████▉  | 14766/18627 [8:18:28<2:12:25,  2.06s/it]
 79%|███████▉  | 14767/18627 [8:18:31<2:17:40,  2.14s/it]
 79%|███████▉  | 14768/18627 [8:18:32<2:02:33,  1.91s/it]
 79%|███████▉  | 14769/18627 [8:18:35<2:17:02,  2.13s/it]
 79%|███████▉  | 14770/18627 [8:18:37<2:20:46,  2.19s/it]
                                                         
{'loss': 0.9188, 'grad_norm': 7.386030197143555, 'learning_rate': 5.414397577917152e-07, 'epoch': 0.79}
+
 79%|███████▉  | 14770/18627 [8:18:37<2:20:46,  2.19s/it]
 79%|███████▉  | 14771/18627 [8:18:40<2:26:58,  2.29s/it]
 79%|███████▉  | 14772/18627 [8:18:42<2:27:09,  2.29s/it]
 79%|███████▉  | 14773/18627 [8:18:44<2:27:46,  2.30s/it]
 79%|███████▉  | 14774/18627 [8:18:47<2:28:18,  2.31s/it]
 79%|███████▉  | 14775/18627 [8:18:49<2:29:01,  2.32s/it]
 79%|███████▉  | 14776/18627 [8:18:51<2:29:20,  2.33s/it]
 79%|███████▉  | 14777/18627 [8:18:53<2:09:25,  2.02s/it]
 79%|███████▉  | 14778/18627 [8:18:54<1:57:22,  1.83s/it]
 79%|███████▉  | 14779/18627 [8:18:55<1:48:09,  1.69s/it]
 79%|███████▉  | 14780/18627 [8:18:58<2:00:57,  1.89s/it]
                                                         
{'loss': 1.1881, 'grad_norm': 4.840989589691162, 'learning_rate': 5.387411749499427e-07, 'epoch': 0.79}
+
 79%|███████▉  | 14780/18627 [8:18:58<2:00:57,  1.89s/it]
 79%|███████▉  | 14781/18627 [8:19:00<2:09:45,  2.02s/it]
 79%|███████▉  | 14782/18627 [8:19:02<2:16:14,  2.13s/it]
 79%|███████▉  | 14783/18627 [8:19:05<2:20:15,  2.19s/it]
 79%|███████▉  | 14784/18627 [8:19:07<2:22:32,  2.23s/it]
 79%|███████▉  | 14785/18627 [8:19:09<2:24:50,  2.26s/it]
 79%|███████▉  | 14786/18627 [8:19:12<2:26:08,  2.28s/it]
 79%|███████▉  | 14787/18627 [8:19:13<2:09:17,  2.02s/it]
 79%|███████▉  | 14788/18627 [8:19:14<1:56:52,  1.83s/it]
 79%|███████▉  | 14789/18627 [8:19:16<1:47:58,  1.69s/it]
 79%|███████▉  | 14790/18627 [8:19:18<1:59:51,  1.87s/it]
                                                         
{'loss': 1.2618, 'grad_norm': 6.873349666595459, 'learning_rate': 5.360485215577851e-07, 'epoch': 0.79}
+
 79%|███████▉  | 14790/18627 [8:19:18<1:59:51,  1.87s/it]
 79%|███████▉  | 14791/18627 [8:19:20<2:08:48,  2.01s/it]
 79%|███████▉  | 14792/18627 [8:19:23<2:14:20,  2.10s/it]
 79%|███████▉  | 14793/18627 [8:19:25<2:19:05,  2.18s/it]
 79%|███████▉  | 14794/18627 [8:19:27<2:21:38,  2.22s/it]
 79%|███████▉  | 14795/18627 [8:19:30<2:23:22,  2.24s/it]
 79%|███████▉  | 14796/18627 [8:19:31<2:00:22,  1.89s/it]
 79%|███████▉  | 14797/18627 [8:19:32<1:50:59,  1.74s/it]
 79%|███████▉  | 14798/18627 [8:19:33<1:39:42,  1.56s/it]
 79%|███████▉  | 14799/18627 [8:19:35<1:36:24,  1.51s/it]
 79%|███████▉  | 14800/18627 [8:19:36<1:34:49,  1.49s/it]
                                                         
{'loss': 1.2495, 'grad_norm': 13.610200881958008, 'learning_rate': 5.333618057559065e-07, 'epoch': 0.79}
+
 79%|███████▉  | 14800/18627 [8:19:36<1:34:49,  1.49s/it]
 79%|███████▉  | 14801/18627 [8:19:39<1:51:48,  1.75s/it]
 79%|███████▉  | 14802/18627 [8:19:41<2:03:21,  1.94s/it]
 79%|███████▉  | 14803/18627 [8:19:43<2:11:55,  2.07s/it]
 79%|███████▉  | 14804/18627 [8:19:46<2:16:39,  2.14s/it]
 79%|███████▉  | 14805/18627 [8:19:48<2:20:22,  2.20s/it]
 79%|███████▉  | 14806/18627 [8:19:50<2:23:22,  2.25s/it]
 79%|███████▉  | 14807/18627 [8:19:53<2:25:23,  2.28s/it]
 79%|███████▉  | 14808/18627 [8:19:55<2:26:52,  2.31s/it]
 80%|███████▉  | 14809/18627 [8:19:57<2:27:11,  2.31s/it]
 80%|███████▉  | 14810/18627 [8:20:00<2:31:06,  2.38s/it]
                                                         
{'loss': 0.6293, 'grad_norm': 7.336531639099121, 'learning_rate': 5.306810356670239e-07, 'epoch': 0.8}
+
 80%|███████▉  | 14810/18627 [8:20:00<2:31:06,  2.38s/it]
 80%|███████▉  | 14811/18627 [8:20:01<2:12:17,  2.08s/it]
 80%|███████▉  | 14812/18627 [8:20:04<2:16:43,  2.15s/it]
 80%|███████▉  | 14813/18627 [8:20:06<2:20:49,  2.22s/it]
 80%|███████▉  | 14814/18627 [8:20:08<2:22:23,  2.24s/it]
 80%|███████▉  | 14815/18627 [8:20:11<2:24:13,  2.27s/it]
 80%|███████▉  | 14816/18627 [8:20:12<2:07:36,  2.01s/it]
 80%|███████▉  | 14817/18627 [8:20:13<1:56:22,  1.83s/it]
 80%|███████▉  | 14818/18627 [8:20:16<2:06:05,  1.99s/it]
 80%|███████▉  | 14819/18627 [8:20:18<2:12:18,  2.08s/it]
 80%|███████▉  | 14820/18627 [8:20:19<1:59:11,  1.88s/it]
                                                         
{'loss': 1.3111, 'grad_norm': 14.844898223876953, 'learning_rate': 5.280062193958737e-07, 'epoch': 0.8}
+
 80%|███████▉  | 14820/18627 [8:20:19<1:59:11,  1.88s/it]
 80%|███████▉  | 14821/18627 [8:20:22<2:07:52,  2.02s/it]
 80%|███████▉  | 14822/18627 [8:20:24<2:13:34,  2.11s/it]
 80%|███████▉  | 14823/18627 [8:20:26<2:18:04,  2.18s/it]
 80%|███████▉  | 14824/18627 [8:20:29<2:20:58,  2.22s/it]
 80%|███████▉  | 14825/18627 [8:20:30<2:05:04,  1.97s/it]
 80%|███████▉  | 14826/18627 [8:20:32<2:11:40,  2.08s/it]
 80%|███████▉  | 14827/18627 [8:20:35<2:16:36,  2.16s/it]
 80%|███████▉  | 14828/18627 [8:20:37<2:19:40,  2.21s/it]
 80%|███████▉  | 14829/18627 [8:20:40<2:23:13,  2.26s/it]
 80%|███████▉  | 14830/18627 [8:20:41<2:00:14,  1.90s/it]
                                                         
{'loss': 0.8134, 'grad_norm': 5.348860740661621, 'learning_rate': 5.253373650291982e-07, 'epoch': 0.8}
+
 80%|███████▉  | 14830/18627 [8:20:41<2:00:14,  1.90s/it]
 80%|███████▉  | 14831/18627 [8:20:43<2:08:50,  2.04s/it]
 80%|███████▉  | 14832/18627 [8:20:45<2:14:01,  2.12s/it]
 80%|███████▉  | 14833/18627 [8:20:48<2:18:02,  2.18s/it]
 80%|███████▉  | 14834/18627 [8:20:50<2:20:48,  2.23s/it]
 80%|███████▉  | 14835/18627 [8:20:52<2:22:48,  2.26s/it]
 80%|███████▉  | 14836/18627 [8:20:54<2:05:39,  1.99s/it]
 80%|███████▉  | 14837/18627 [8:20:55<1:54:24,  1.81s/it]
 80%|███████▉  | 14838/18627 [8:20:57<2:04:28,  1.97s/it]
 80%|███████▉  | 14839/18627 [8:20:59<1:52:57,  1.79s/it]
 80%|███████▉  | 14840/18627 [8:21:01<2:03:05,  1.95s/it]
                                                         
{'loss': 1.1423, 'grad_norm': 5.151512622833252, 'learning_rate': 5.226744806357095e-07, 'epoch': 0.8}
+
 80%|███████▉  | 14840/18627 [8:21:01<2:03:05,  1.95s/it]
 80%|███████▉  | 14841/18627 [8:21:03<2:10:48,  2.07s/it]
 80%|███████▉  | 14842/18627 [8:21:06<2:15:33,  2.15s/it]
 80%|███████▉  | 14843/18627 [8:21:08<2:18:50,  2.20s/it]
 80%|███████▉  | 14844/18627 [8:21:10<2:20:52,  2.23s/it]
 80%|███████▉  | 14845/18627 [8:21:13<2:22:50,  2.27s/it]
 80%|███████▉  | 14846/18627 [8:21:14<2:04:31,  1.98s/it]
 80%|██��████▉  | 14847/18627 [8:21:15<1:52:56,  1.79s/it]
 80%|███████▉  | 14848/18627 [8:21:18<2:03:03,  1.95s/it]
 80%|███████▉  | 14849/18627 [8:21:20<2:10:10,  2.07s/it]
 80%|███████▉  | 14850/18627 [8:21:22<2:15:16,  2.15s/it]
                                                         
{'loss': 1.0752, 'grad_norm': 7.060750484466553, 'learning_rate': 5.200175742660737e-07, 'epoch': 0.8}
+
 80%|███████▉  | 14850/18627 [8:21:22<2:15:16,  2.15s/it]
 80%|███████▉  | 14851/18627 [8:21:25<2:19:11,  2.21s/it]
 80%|███████▉  | 14852/18627 [8:21:27<2:21:30,  2.25s/it]
 80%|███████▉  | 14853/18627 [8:21:29<2:23:15,  2.28s/it]
 80%|███████▉  | 14854/18627 [8:21:32<2:25:31,  2.31s/it]
 80%|███████▉  | 14855/18627 [8:21:34<2:25:37,  2.32s/it]
 80%|███████▉  | 14856/18627 [8:21:37<2:26:36,  2.33s/it]
 80%|███████▉  | 14857/18627 [8:21:38<2:09:36,  2.06s/it]
 80%|███████▉  | 14858/18627 [8:21:40<2:14:35,  2.14s/it]
 80%|███████▉  | 14859/18627 [8:21:42<2:00:24,  1.92s/it]
 80%|███████▉  | 14860/18627 [8:21:44<2:08:03,  2.04s/it]
                                                         
{'loss': 1.1241, 'grad_norm': 8.13083553314209, 'learning_rate': 5.173666539528832e-07, 'epoch': 0.8}
+
 80%|███████▉  | 14860/18627 [8:21:44<2:08:03,  2.04s/it]
 80%|███████▉  | 14861/18627 [8:21:46<2:13:37,  2.13s/it]
 80%|███████▉  | 14862/18627 [8:21:49<2:17:28,  2.19s/it]
 80%|███████▉  | 14863/18627 [8:21:51<2:20:21,  2.24s/it]
 80%|███████▉  | 14864/18627 [8:21:53<2:21:57,  2.26s/it]
 80%|███████▉  | 14865/18627 [8:21:56<2:22:59,  2.28s/it]
 80%|███████▉  | 14866/18627 [8:21:57<2:05:59,  2.01s/it]
 80%|███████▉  | 14867/18627 [8:21:59<2:11:58,  2.11s/it]
 80%|███████▉  | 14868/18627 [8:22:02<2:16:38,  2.18s/it]
 80%|███████▉  | 14869/18627 [8:22:04<2:19:36,  2.23s/it]
 80%|███████▉  | 14870/18627 [8:22:05<2:03:30,  1.97s/it]
                                                         
{'loss': 1.0418, 'grad_norm': 13.794220924377441, 'learning_rate': 5.147217277106329e-07, 'epoch': 0.8}
+
 80%|███████▉  | 14870/18627 [8:22:05<2:03:30,  1.97s/it]
 80%|███████▉  | 14871/18627 [8:22:08<2:10:40,  2.09s/it]
 80%|███████▉  | 14872/18627 [8:22:10<2:15:11,  2.16s/it]
 80%|███████▉  | 14873/18627 [8:22:12<2:18:49,  2.22s/it]
 80%|███████▉  | 14874/18627 [8:22:13<1:56:12,  1.86s/it]
 80%|███████▉  | 14875/18627 [8:22:16<2:05:11,  2.00s/it]
 80%|███████▉  | 14876/18627 [8:22:18<2:12:16,  2.12s/it]
 80%|███████▉  | 14877/18627 [8:22:21<2:16:25,  2.18s/it]
 80%|███████▉  | 14878/18627 [8:22:23<2:19:41,  2.24s/it]
 80%|███████▉  | 14879/18627 [8:22:25<2:21:57,  2.27s/it]
 80%|███████▉  | 14880/18627 [8:22:28<2:21:08,  2.26s/it]
                                                         
{'loss': 0.5808, 'grad_norm': 4.832096576690674, 'learning_rate': 5.120828035356951e-07, 'epoch': 0.8}
+
 80%|███████▉  | 14880/18627 [8:22:28<2:21:08,  2.26s/it]
 80%|███████▉  | 14881/18627 [8:22:30<2:22:50,  2.29s/it]
 80%|███████▉  | 14882/18627 [8:22:31<2:05:01,  2.00s/it]
 80%|███████▉  | 14883/18627 [8:22:34<2:12:28,  2.12s/it]
 80%|███████▉  | 14884/18627 [8:22:35<1:59:28,  1.92s/it]
 80%|███████▉  | 14885/18627 [8:22:37<2:06:49,  2.03s/it]
 80%|███████▉  | 14886/18627 [8:22:39<1:53:10,  1.82s/it]
 80%|███████▉  | 14887/18627 [8:22:41<2:03:23,  1.98s/it]
 80%|███████▉  | 14888/18627 [8:22:43<2:10:07,  2.09s/it]
 80%|███████▉  | 14889/18627 [8:22:46<2:14:47,  2.16s/it]
 80%|███████▉  | 14890/18627 [8:22:47<1:59:43,  1.92s/it]
                                                         
{'loss': 1.4165, 'grad_norm': 14.462972640991211, 'learning_rate': 5.09449889406298e-07, 'epoch': 0.8}
+
 80%|███████▉  | 14890/18627 [8:22:47<1:59:43,  1.92s/it]
 80%|███████▉  | 14891/18627 [8:22:49<2:07:30,  2.05s/it]
 80%|███████▉  | 14892/18627 [8:22:52<2:12:51,  2.13s/it]
 80%|███████▉  | 14893/18627 [8:22:53<1:56:20,  1.87s/it]
 80%|███████▉  | 14894/18627 [8:22:55<2:05:18,  2.01s/it]
 80%|███████▉  | 14895/18627 [8:22:58<2:11:58,  2.12s/it]
 80%|███████▉  | 14896/18627 [8:23:00<2:16:19,  2.19s/it]
 80%|███████▉  | 14897/18627 [8:23:01<2:01:58,  1.96s/it]
 80%|███████▉  | 14898/18627 [8:23:04<2:09:00,  2.08s/it]
 80%|███████▉  | 14899/18627 [8:23:05<1:56:11,  1.87s/it]
 80%|███████▉  | 14900/18627 [8:23:08<2:05:44,  2.02s/it]
                                                         
{'loss': 1.1924, 'grad_norm': 9.341672897338867, 'learning_rate': 5.068229932824961e-07, 'epoch': 0.8}
+
 80%|███████▉  | 14900/18627 [8:23:08<2:05:44,  2.02s/it]
 80%|███████▉  | 14901/18627 [8:23:09<1:53:13,  1.82s/it]
 80%|████████  | 14902/18627 [8:23:11<2:03:21,  1.99s/it]
 80%|████████  | 14903/18627 [8:23:14<2:09:25,  2.09s/it]
 80%|████████  | 14904/18627 [8:23:16<2:14:37,  2.17s/it]
 80%|████████  | 14905/18627 [8:23:18<2:18:01,  2.22s/it]
 80%|████████  | 14906/18627 [8:23:20<2:01:38,  1.96s/it]
 80%|████████  | 14907/18627 [8:23:22<2:08:26,  2.07s/it]
 80%|████████  | 14908/18627 [8:23:24<2:14:01,  2.16s/it]
 80%|████████  | 14909/18627 [8:23:27<2:17:30,  2.22s/it]
 80%|████████  | 14910/18627 [8:23:29<2:19:27,  2.25s/it]
                                                         
{'loss': 0.9598, 'grad_norm': 5.1106414794921875, 'learning_rate': 5.042021231061548e-07, 'epoch': 0.8}
+
 80%|████████  | 14910/18627 [8:23:29<2:19:27,  2.25s/it]
 80%|████████  | 14911/18627 [8:23:30<2:03:35,  2.00s/it]
 80%|████████  | 14912/18627 [8:23:33<2:09:11,  2.09s/it]
 80%|████████  | 14913/18627 [8:23:35<2:14:17,  2.17s/it]
 80%|████████  | 14914/18627 [8:23:38<2:18:06,  2.23s/it]
 80%|████████  | 14915/18627 [8:23:40<2:19:42,  2.26s/it]
 80%|████████  | 14916/18627 [8:23:41<2:03:37,  2.00s/it]
 80%|████████  | 14917/18627 [8:23:44<2:09:48,  2.10s/it]
 80%|████████  | 14918/18627 [8:23:46<2:14:09,  2.17s/it]
 80%|████████  | 14919/18627 [8:23:48<2:17:31,  2.23s/it]
 80%|████████  | 14920/18627 [8:23:49<1:57:21,  1.90s/it]
                                                         
{'loss': 1.0166, 'grad_norm': 5.613618850708008, 'learning_rate': 5.015872868009164e-07, 'epoch': 0.8}
+
 80%|████████  | 14920/18627 [8:23:49<1:57:21,  1.90s/it]
 80%|████████  | 14921/18627 [8:23:52<2:06:13,  2.04s/it]
 80%|████████  | 14922/18627 [8:23:54<2:11:28,  2.13s/it]
 80%|████████  | 14923/18627 [8:23:56<2:15:09,  2.19s/it]
 80%|████████  | 14924/18627 [8:23:59<2:18:02,  2.24s/it]
 80%|████████  | 14925/18627 [8:24:01<2:20:02,  2.27s/it]
 80%|████████  | 14926/18627 [8:24:03<2:21:02,  2.29s/it]
 80%|████████  | 14927/18627 [8:24:06<2:22:01,  2.30s/it]
 80%|████████  | 14928/18627 [8:24:07<2:00:02,  1.95s/it]
 80%|████████  | 14929/18627 [8:24:09<2:07:33,  2.07s/it]
 80%|████████  | 14930/18627 [8:24:12<2:12:20,  2.15s/it]
                                                         
{'loss': 0.5804, 'grad_norm': 6.023451805114746, 'learning_rate': 4.989784922721833e-07, 'epoch': 0.8}
+
 80%|████████  | 14930/18627 [8:24:12<2:12:20,  2.15s/it]
 80%|████████  | 14931/18627 [8:24:13<1:58:16,  1.92s/it]
 80%|████████  | 14932/18627 [8:24:15<2:06:35,  2.06s/it]
 80%|████████  | 14933/18627 [8:24:18<2:11:49,  2.14s/it]
 80%|████████  | 14934/18627 [8:24:19<1:58:08,  1.92s/it]
 80%|████████  | 14935/18627 [8:24:21<1:48:48,  1.77s/it]
 80%|████████  | 14936/18627 [8:24:23<1:59:30,  1.94s/it]
 80%|████████  | 14937/18627 [8:24:25<2:06:14,  2.05s/it]
 80%|████████  | 14938/18627 [8:24:28<2:11:38,  2.14s/it]
 80%|████████  | 14939/18627 [8:24:29<1:56:56,  1.90s/it]
 80%|████████  | 14940/18627 [8:24:31<2:04:29,  2.03s/it]
                                                         
{'loss': 1.4282, 'grad_norm': 7.043715476989746, 'learning_rate': 4.96375747407094e-07, 'epoch': 0.8}
+
 80%|████████  | 14940/18627 [8:24:31<2:04:29,  2.03s/it]
 80%|████████  | 14941/18627 [8:24:34<2:10:17,  2.12s/it]
 80%|████████  | 14942/18627 [8:24:36<2:13:55,  2.18s/it]
 80%|████████  | 14943/18627 [8:24:38<2:16:53,  2.23s/it]
 80%|████████  | 14944/18627 [8:24:41<2:19:15,  2.27s/it]
 80%|████████  | 14945/18627 [8:24:43<2:20:30,  2.29s/it]
 80%|████████  | 14946/18627 [8:24:44<2:03:51,  2.02s/it]
 80%|████████  | 14947/18627 [8:24:47<2:09:41,  2.11s/it]
 80%|████████  | 14948/18627 [8:24:49<2:14:06,  2.19s/it]
 80%|████████  | 14949/18627 [8:24:51<2:16:50,  2.23s/it]
 80%|████████  | 14950/18627 [8:24:54<2:18:25,  2.26s/it]
                                                         
{'loss': 0.7644, 'grad_norm': 7.3612518310546875, 'learning_rate': 4.937790600744929e-07, 'epoch': 0.8}
+
 80%|████████  | 14950/18627 [8:24:54<2:18:25,  2.26s/it]
 80%|████████  | 14951/18627 [8:24:56<2:20:07,  2.29s/it]
 80%|████████  | 14952/18627 [8:24:58<2:21:06,  2.30s/it]
 80%|████████  | 14953/18627 [8:25:01<2:21:11,  2.31s/it]
 80%|████████  | 14954/18627 [8:25:03<2:21:39,  2.31s/it]
 80%|████████  | 14955/18627 [8:25:04<2:04:20,  2.03s/it]
 80%|████████  | 14956/18627 [8:25:07<2:10:21,  2.13s/it]
 80%|████████  | 14957/18627 [8:25:09<2:14:42,  2.20s/it]
 80%|████████  | 14958/18627 [8:25:10<1:59:41,  1.96s/it]
 80%|████████  | 14959/18627 [8:25:12<1:49:09,  1.79s/it]
 80%|████████  | 14960/18627 [8:25:14<1:59:32,  1.96s/it]
                                                         
{'loss': 1.1767, 'grad_norm': 6.460395336151123, 'learning_rate': 4.911884381249135e-07, 'epoch': 0.8}
+
 80%|████████  | 14960/18627 [8:25:14<1:59:32,  1.96s/it]
 80%|████████  | 14961/18627 [8:25:16<1:49:33,  1.79s/it]
 80%|████████  | 14962/18627 [8:25:18<1:59:03,  1.95s/it]
 80%|████████  | 14963/18627 [8:25:20<2:06:13,  2.07s/it]
 80%|████████  | 14964/18627 [8:25:23<2:10:34,  2.14s/it]
 80%|████████  | 14965/18627 [8:25:24<1:57:01,  1.92s/it]
 80%|████████  | 14966/18627 [8:25:26<2:05:10,  2.05s/it]
 80%|████████  | 14967/18627 [8:25:29<2:10:29,  2.14s/it]
 80%|████████  | 14968/18627 [8:25:30<1:54:52,  1.88s/it]
 80%|████████  | 14969/18627 [8:25:32<2:02:50,  2.01s/it]
 80%|████████  | 14970/18627 [8:25:35<2:08:54,  2.11s/it]
                                                         
{'loss': 1.2538, 'grad_norm': 6.060906887054443, 'learning_rate': 4.886038893905509e-07, 'epoch': 0.8}
+
 80%|████████  | 14970/18627 [8:25:35<2:08:54,  2.11s/it]
 80%|████████  | 14971/18627 [8:25:37<2:13:39,  2.19s/it]
 80%|████████  | 14972/18627 [8:25:39<2:16:07,  2.23s/it]
 80%|████████  | 14973/18627 [8:25:41<1:59:06,  1.96s/it]
 80%|████████  | 14974/18627 [8:25:43<2:05:44,  2.07s/it]
 80%|████████  | 14975/18627 [8:25:45<2:10:39,  2.15s/it]
 80%|████████  | 14976/18627 [8:25:48<2:13:13,  2.19s/it]
 80%|████████  | 14977/18627 [8:25:49<1:58:19,  1.95s/it]
 80%|████████  | 14978/18627 [8:25:50<1:48:23,  1.78s/it]
 80%|████████  | 14979/18627 [8:25:53<1:57:31,  1.93s/it]
 80%|████████  | 14980/18627 [8:25:55<2:05:28,  2.06s/it]
                                                         
{'loss': 1.1308, 'grad_norm': 9.795743942260742, 'learning_rate': 4.860254216852398e-07, 'epoch': 0.8}
+
 80%|████████  | 14980/18627 [8:25:55<2:05:28,  2.06s/it]
 80%|████████  | 14981/18627 [8:25:57<2:10:33,  2.15s/it]
 80%|████████  | 14982/18627 [8:25:59<1:56:08,  1.91s/it]
 80%|████████  | 14983/18627 [8:26:00<1:46:25,  1.75s/it]
 80%|████████  | 14984/18627 [8:26:01<1:39:36,  1.64s/it]
 80%|████████  | 14985/18627 [8:26:04<1:53:04,  1.86s/it]
 80%|████████  | 14986/18627 [8:26:05<1:38:35,  1.62s/it]
 80%|████████  | 14987/18627 [8:26:06<1:34:15,  1.55s/it]
 80%|████████  | 14988/18627 [8:26:09<1:48:54,  1.80s/it]
 80%|████████  | 14989/18627 [8:26:11<1:59:47,  1.98s/it]
 80%|████████  | 14990/18627 [8:26:12<1:48:23,  1.79s/it]
                                                         
{'loss': 1.5139, 'grad_norm': 14.42686653137207, 'learning_rate': 4.834530428044293e-07, 'epoch': 0.8}
+
 80%|████████  | 14990/18627 [8:26:12<1:48:23,  1.79s/it]
 80%|████████  | 14991/18627 [8:26:15<1:58:39,  1.96s/it]
 80%|████████  | 14992/18627 [8:26:16<1:41:57,  1.68s/it]
 80%|████████  | 14993/18627 [8:26:18<1:54:02,  1.88s/it]
 80%|████████  | 14994/18627 [8:26:20<2:01:53,  2.01s/it]
 81%|████████  | 14995/18627 [8:26:23<2:07:51,  2.11s/it]
 81%|████████  | 14996/18627 [8:26:25<2:12:55,  2.20s/it]
 81%|████████  | 14997/18627 [8:26:27<1:57:59,  1.95s/it]
 81%|████████  | 14998/18627 [8:26:29<2:05:04,  2.07s/it]
 81%|████████  | 14999/18627 [8:26:30<1:53:08,  1.87s/it]
 81%|████████  | 15000/18627 [8:26:33<2:01:33,  2.01s/it]
                                                         
{'loss': 1.0009, 'grad_norm': 8.141839981079102, 'learning_rate': 4.808867605251619e-07, 'epoch': 0.81}
+
 81%|████████  | 15000/18627 [8:26:33<2:01:33,  2.01s/it]/home/zhengduo/workspace/LLaMA-Factory/src/qwen_vl/model/modeling_qwen2_5_vl.py:1979: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
+  with torch.cuda.amp.autocast(dtype=dtype):
+/home/zhengduo/miniconda3/envs/vgllm/lib/python3.10/site-packages/torch/utils/checkpoint.py:87: UserWarning: None of the inputs have requires_grad=True. Gradients will be None
+  warnings.warn(
+
 81%|████████  | 15001/18627 [8:27:34<19:54:43, 19.77s/it]
 81%|████████  | 15002/18627 [8:27:35<14:21:59, 14.27s/it]
 81%|████████  | 15003/18627 [8:27:38<10:45:07, 10.68s/it]
 81%|████████  | 15004/18627 [8:27:40<8:14:08,  8.18s/it] 
 81%|████████  | 15005/18627 [8:27:42<6:28:17,  6.43s/it]
 81%|████████  | 15006/18627 [8:27:44<4:57:06,  4.92s/it]
 81%|████████  | 15007/18627 [8:27:45<3:53:21,  3.87s/it]
 81%|████████  | 15008/18627 [8:27:46<3:02:48,  3.03s/it]
 81%|████████  | 15009/18627 [8:27:49<2:50:32,  2.83s/it]
 81%|████████  | 15010/18627 [8:27:51<2:42:02,  2.69s/it]
                                                         
{'loss': 1.1594, 'grad_norm': 4.5711212158203125, 'learning_rate': 4.783265826060451e-07, 'epoch': 0.81}
+
 81%|████████  | 15010/18627 [8:27:51<2:42:02,  2.69s/it]
 81%|████████  | 15011/18627 [8:27:53<2:35:38,  2.58s/it]
 81%|████████  | 15012/18627 [8:27:54<2:09:31,  2.15s/it]
 81%|████████  | 15013/18627 [8:27:57<2:13:42,  2.22s/it]
 81%|████████  | 15014/18627 [8:27:59<2:16:23,  2.26s/it]
 81%|████████  | 15015/18627 [8:28:02<2:18:09,  2.29s/it]
 81%|████████  | 15016/18627 [8:28:04<2:19:09,  2.31s/it]
 81%|████████  | 15017/18627 [8:28:06<2:19:59,  2.33s/it]
 81%|████████  | 15018/18627 [8:28:08<2:01:18,  2.02s/it]
 81%|████████  | 15019/18627 [8:28:10<2:07:39,  2.12s/it]
 81%|████████  | 15020/18627 [8:28:15<3:01:40,  3.02s/it]
                                                         
{'loss': 0.7674, 'grad_norm': 6.025176048278809, 'learning_rate': 4.757725167872354e-07, 'epoch': 0.81}
+
 81%|████████  | 15020/18627 [8:28:15<3:01:40,  3.02s/it]
 81%|████████  | 15021/18627 [8:28:17<2:49:38,  2.82s/it]
 81%|████████  | 15022/18627 [8:28:20<2:41:17,  2.68s/it]
 81%|████████  | 15023/18627 [8:28:21<2:20:58,  2.35s/it]
 81%|████████  | 15024/18627 [8:28:24<2:21:06,  2.35s/it]
 81%|████████  | 15025/18627 [8:28:26<2:21:05,  2.35s/it]
 81%|████████  | 15026/18627 [8:28:28<2:21:01,  2.35s/it]
 81%|████████  | 15027/18627 [8:28:31<2:21:36,  2.36s/it]
 81%|████████  | 15028/18627 [8:28:33<2:21:00,  2.35s/it]
 81%|████████  | 15029/18627 [8:28:35<2:21:32,  2.36s/it]
 81%|████████  | 15030/18627 [8:28:38<2:20:38,  2.35s/it]
                                                         
{'loss': 0.8334, 'grad_norm': 5.193818092346191, 'learning_rate': 4.7322457079040705e-07, 'epoch': 0.81}
+
 81%|████████  | 15030/18627 [8:28:38<2:20:38,  2.35s/it]
 81%|████████  | 15031/18627 [8:28:40<2:20:45,  2.35s/it]
 81%|████████  | 15032/18627 [8:28:42<2:03:19,  2.06s/it]
 81%|████████  | 15033/18627 [8:28:44<2:07:52,  2.13s/it]
 81%|████████  | 15034/18627 [8:28:46<2:11:22,  2.19s/it]
 81%|████████  | 15035/18627 [8:28:48<1:56:50,  1.95s/it]
 81%|████████  | 15036/18627 [8:28:50<2:04:00,  2.07s/it]
 81%|████████  | 15037/18627 [8:28:52<2:09:10,  2.16s/it]
 81%|████████  | 15038/18627 [8:28:55<2:12:51,  2.22s/it]
 81%|████████  | 15039/18627 [8:28:57<2:15:13,  2.26s/it]
 81%|████████  | 15040/18627 [8:28:58<1:57:32,  1.97s/it]
                                                         
{'loss': 1.1843, 'grad_norm': 13.285975456237793, 'learning_rate': 4.706827523187343e-07, 'epoch': 0.81}
+
 81%|████████  | 15040/18627 [8:28:58<1:57:32,  1.97s/it]
 81%|████████  | 15041/18627 [8:29:01<2:04:50,  2.09s/it]
 81%|████████  | 15042/18627 [8:29:03<2:09:29,  2.17s/it]
 81%|████████  | 15043/18627 [8:29:05<2:12:43,  2.22s/it]
 81%|████████  | 15044/18627 [8:29:08<2:13:20,  2.23s/it]
 81%|████████  | 15045/18627 [8:29:10<2:15:31,  2.27s/it]
 81%|████████  | 15046/18627 [8:29:12<2:17:13,  2.30s/it]
 81%|████████  | 15047/18627 [8:29:14<2:00:40,  2.02s/it]
 81%|████████  | 15048/18627 [8:29:15<1:49:27,  1.83s/it]
 81%|████████  | 15049/18627 [8:29:17<1:58:46,  1.99s/it]
 81%|████████  | 15050/18627 [8:29:20<2:04:54,  2.10s/it]
                                                         
{'loss': 0.9891, 'grad_norm': 5.036655902862549, 'learning_rate': 4.6814706905686543e-07, 'epoch': 0.81}
+
 81%|███████��  | 15050/18627 [8:29:20<2:04:54,  2.10s/it]
 81%|████████  | 15051/18627 [8:29:22<2:09:41,  2.18s/it]
 81%|████████  | 15052/18627 [8:29:25<2:17:49,  2.31s/it]
 81%|████████  | 15053/18627 [8:29:27<2:17:57,  2.32s/it]
 81%|████████  | 15054/18627 [8:29:30<2:21:21,  2.37s/it]
 81%|████████  | 15055/18627 [8:29:32<2:20:53,  2.37s/it]
 81%|████████  | 15056/18627 [8:29:34<2:20:29,  2.36s/it]
 81%|████████  | 15057/18627 [8:29:37<2:20:58,  2.37s/it]
 81%|████████  | 15058/18627 [8:29:38<2:02:48,  2.06s/it]
 81%|████████  | 15059/18627 [8:29:40<2:08:30,  2.16s/it]
 81%|████████  | 15060/18627 [8:29:42<1:56:20,  1.96s/it]
                                                         
{'loss': 1.0376, 'grad_norm': 15.112704277038574, 'learning_rate': 4.6561752867090053e-07, 'epoch': 0.81}
+
 81%|████████  | 15060/18627 [8:29:42<1:56:20,  1.96s/it]
 81%|████████  | 15061/18627 [8:29:44<2:03:19,  2.08s/it]
 81%|████████  | 15062/18627 [8:29:47<2:07:52,  2.15s/it]
 81%|████████  | 15063/18627 [8:29:49<2:11:00,  2.21s/it]
 81%|████████  | 15064/18627 [8:29:51<2:13:47,  2.25s/it]
 81%|████████  | 15065/18627 [8:29:54<2:15:12,  2.28s/it]
 81%|████████  | 15066/18627 [8:29:56<2:16:56,  2.31s/it]
 81%|████████  | 15067/18627 [8:29:57<2:00:31,  2.03s/it]
 81%|████████  | 15068/18627 [8:29:59<1:49:13,  1.84s/it]
 81%|████████  | 15069/18627 [8:30:01<1:58:40,  2.00s/it]
 81%|████████  | 15070/18627 [8:30:03<2:04:10,  2.09s/it]
                                                         
{'loss': 0.9921, 'grad_norm': 9.768606185913086, 'learning_rate': 4.630941388083676e-07, 'epoch': 0.81}
+
 81%|████████  | 15070/18627 [8:30:03<2:04:10,  2.09s/it]
 81%|████████  | 15071/18627 [8:30:06<2:09:02,  2.18s/it]
 81%|████████  | 15072/18627 [8:30:08<2:12:09,  2.23s/it]
 81%|████████  | 15073/18627 [8:30:11<2:14:41,  2.27s/it]
 81%|████████  | 15074/18627 [8:30:12<2:00:06,  2.03s/it]
 81%|████████  | 15075/18627 [8:30:14<2:06:24,  2.14s/it]
 81%|████████  | 15076/18627 [8:30:17<2:10:02,  2.20s/it]
 81%|████████  | 15077/18627 [8:30:19<2:13:13,  2.25s/it]
 81%|████████  | 15078/18627 [8:30:22<2:15:23,  2.29s/it]
 81%|████████  | 15079/18627 [8:30:23<1:58:57,  2.01s/it]
 81%|████████  | 15080/18627 [8:30:25<2:04:24,  2.10s/it]
                                                         
{'loss': 1.0634, 'grad_norm': 5.773665904998779, 'learning_rate': 4.605769070982011e-07, 'epoch': 0.81}
+
 81%|████████  | 15080/18627 [8:30:25<2:04:24,  2.10s/it]
 81%|████████  | 15081/18627 [8:30:27<1:51:12,  1.88s/it]
 81%|████████  | 15082/18627 [8:30:29<1:59:02,  2.01s/it]
 81%|████████  | 15083/18627 [8:30:31<2:04:35,  2.11s/it]
 81%|████████  | 15084/18627 [8:30:33<1:51:04,  1.88s/it]
 81%|████████  | 15085/18627 [8:30:35<1:59:21,  2.02s/it]
 81%|████████  | 15086/18627 [8:30:37<2:05:07,  2.12s/it]
 81%|████████  | 15087/18627 [8:30:39<1:50:20,  1.87s/it]
 81%|████████  | 15088/18627 [8:30:41<1:58:13,  2.00s/it]
 81%|████████  | 15089/18627 [8:30:43<2:04:07,  2.11s/it]
 81%|████████  | 15090/18627 [8:30:46<2:08:43,  2.18s/it]
                                                         
{'loss': 1.1296, 'grad_norm': 8.623888969421387, 'learning_rate': 4.580658411507144e-07, 'epoch': 0.81}
+
 81%|████████  | 15090/18627 [8:30:46<2:08:43,  2.18s/it]
 81%|████████  | 15091/18627 [8:30:47<1:48:41,  1.84s/it]
 81%|████████  | 15092/18627 [8:30:48<1:40:19,  1.70s/it]
 81%|████████  | 15093/18627 [8:30:50<1:52:15,  1.91s/it]
 81%|████████  | 15094/18627 [8:30:53<1:58:56,  2.02s/it]
 81%|████████  | 15095/18627 [8:30:55<2:04:13,  2.11s/it]
 81%|████████  | 15096/18627 [8:30:57<2:07:57,  2.17s/it]
 81%|████████  | 15097/18627 [8:31:00<2:13:27,  2.27s/it]
 81%|████████  | 15098/18627 [8:31:01<1:56:41,  1.98s/it]
 81%|████████  | 15099/18627 [8:31:02<1:44:05,  1.77s/it]
 81%|████████  | 15100/18627 [8:31:05<1:54:04,  1.94s/it]
                                                         
{'loss': 1.1278, 'grad_norm': 6.4867777824401855, 'learning_rate': 4.555609485575849e-07, 'epoch': 0.81}
+
 81%|████████  | 15100/18627 [8:31:05<1:54:04,  1.94s/it]
 81%|████████  | 15101/18627 [8:31:06<1:43:57,  1.77s/it]
 81%|████████  | 15102/18627 [8:31:08<1:53:43,  1.94s/it]
 81%|████████  | 15103/18627 [8:31:11<2:01:00,  2.06s/it]
 81%|████████  | 15104/18627 [8:31:12<1:43:23,  1.76s/it]
 81%|████████  | 15105/18627 [8:31:14<1:53:01,  1.93s/it]
 81%|████████  | 15106/18627 [8:31:17<2:00:51,  2.06s/it]
 81%|████████  | 15107/18627 [8:31:19<2:06:00,  2.15s/it]
 81%|████████  | 15108/18627 [8:31:21<2:09:28,  2.21s/it]
 81%|████████  | 15109/18627 [8:31:23<1:54:21,  1.95s/it]
 81%|████████  | 15110/18627 [8:31:25<2:02:07,  2.08s/it]
                                                         
{'loss': 0.9839, 'grad_norm': 8.722004890441895, 'learning_rate': 4.5306223689182137e-07, 'epoch': 0.81}
+
 81%|████████  | 15110/18627 [8:31:25<2:02:07,  2.08s/it]
 81%|████████  | 15111/18627 [8:31:27<2:07:14,  2.17s/it]
 81%|████████  | 15112/18627 [8:31:29<1:53:27,  1.94s/it]
 81%|████████  | 15113/18627 [8:31:31<2:00:36,  2.06s/it]
 81%|████████  | 15114/18627 [8:31:33<2:05:47,  2.15s/it]
 81%|████████  | 15115/18627 [8:31:35<1:52:24,  1.92s/it]
 81%|████████  | 15116/18627 [8:31:37<1:59:32,  2.04s/it]
 81%|████████  | 15117/18627 [8:31:39<1:48:04,  1.85s/it]
 81%|████████  | 15118/18627 [8:31:41<1:56:20,  1.99s/it]
 81%|████████  | 15119/18627 [8:31:43<2:02:41,  2.10s/it]
 81%|████████  | 15120/18627 [8:31:46<2:06:39,  2.17s/it]
                                                         
{'loss': 1.1946, 'grad_norm': 7.843863487243652, 'learning_rate': 4.505697137077486e-07, 'epoch': 0.81}
+
 81%|████████  | 15120/18627 [8:31:46<2:06:39,  2.17s/it]
 81%|████████  | 15121/18627 [8:31:48<2:09:15,  2.21s/it]
 81%|████████  | 15122/18627 [8:31:50<2:11:36,  2.25s/it]
 81%|████████  | 15123/18627 [8:31:53<2:12:53,  2.28s/it]
 81%|████████  | 15124/18627 [8:31:55<2:13:58,  2.29s/it]
 81%|████████  | 15125/18627 [8:31:56<1:57:56,  2.02s/it]
 81%|████████  | 15126/18627 [8:31:59<2:03:16,  2.11s/it]
 81%|████████  | 15127/18627 [8:32:00<1:50:31,  1.89s/it]
 81%|████████  | 15128/18627 [8:32:01<1:41:52,  1.75s/it]
 81%|████████  | 15129/18627 [8:32:04<1:52:21,  1.93s/it]
 81%|████████  | 15130/18627 [8:32:06<1:59:38,  2.05s/it]
                                                         
{'loss': 1.1836, 'grad_norm': 5.4133453369140625, 'learning_rate': 4.480833865409817e-07, 'epoch': 0.81}
+
 81%|████████  | 15130/18627 [8:32:06<1:59:38,  2.05s/it]
 81%|████████  | 15131/18627 [8:32:08<2:04:32,  2.14s/it]
 81%|████████  | 15132/18627 [8:32:11<2:07:55,  2.20s/it]
 81%|████████  | 15133/18627 [8:32:12<1:48:00,  1.85s/it]
 81%|████████  | 15134/18627 [8:32:14<1:56:56,  2.01s/it]
 81%|████████▏ | 15135/18627 [8:32:17<2:03:08,  2.12s/it]
 81%|████████▏ | 15136/18627 [8:32:19<2:07:51,  2.20s/it]
 81%|████████▏ | 15137/18627 [8:32:20<1:53:24,  1.95s/it]
 81%|████████▏ | 15138/18627 [8:32:23<2:00:06,  2.07s/it]
 81%|████████▏ | 15139/18627 [8:32:25<2:05:57,  2.17s/it]
 81%|████████▏ | 15140/18627 [8:32:27<2:09:10,  2.22s/it]
                                                         
{'loss': 0.7816, 'grad_norm': 7.0167436599731445, 'learning_rate': 4.456032629084023e-07, 'epoch': 0.81}
+
 81%|████████▏ | 15140/18627 [8:32:27<2:09:10,  2.22s/it]
 81%|████████▏ | 15141/18627 [8:32:30<2:12:00,  2.27s/it]
 81%|████████▏ | 15142/18627 [8:32:32<2:13:21,  2.30s/it]
 81%|████████▏ | 15143/18627 [8:32:34<1:59:30,  2.06s/it]
 81%|████████▏ | 15144/18627 [8:32:36<2:04:56,  2.15s/it]
 81%|████████▏ | 15145/18627 [8:32:38<2:08:22,  2.21s/it]
 81%|████████▏ | 15146/18627 [8:32:41<2:10:46,  2.25s/it]
 81%|████████▏ | 15147/18627 [8:32:43<2:12:17,  2.28s/it]
 81%|████████▏ | 15148/18627 [8:32:45<1:57:59,  2.03s/it]
 81%|████████▏ | 15149/18627 [8:32:47<2:03:43,  2.13s/it]
 81%|████████▏ | 15150/18627 [8:32:49<2:07:49,  2.21s/it]
                                                         
{'loss': 0.9915, 'grad_norm': 5.609349727630615, 'learning_rate': 4.43129350308138e-07, 'epoch': 0.81}
+
 81%|████████▏ | 15150/18627 [8:32:49<2:07:49,  2.21s/it]
 81%|████████▏ | 15151/18627 [8:32:52<2:11:19,  2.27s/it]
 81%|████████▏ | 15152/18627 [8:32:54<2:12:59,  2.30s/it]
 81%|████████▏ | 15153/18627 [8:32:58<2:37:09,  2.71s/it]
 81%|████████▏ | 15154/18627 [8:33:00<2:30:53,  2.61s/it]
 81%|████████▏ | 15155/18627 [8:33:02<2:26:13,  2.53s/it]
 81%|████████▏ | 15156/18627 [8:33:05<2:23:21,  2.48s/it]
 81%|████████▏ | 15157/18627 [8:33:07<2:20:58,  2.44s/it]
 81%|████████▏ | 15158/18627 [8:33:10<2:21:30,  2.45s/it]
 81%|████████▏ | 15159/18627 [8:33:12<2:19:16,  2.41s/it]
 81%|████████▏ | 15160/18627 [8:33:13<2:01:31,  2.10s/it]
                                                         
{'loss': 0.7721, 'grad_norm': 14.107940673828125, 'learning_rate': 4.4066165621953845e-07, 'epoch': 0.81}
+
 81%|████████▏ | 15160/18627 [8:33:13<2:01:31,  2.10s/it]
 81%|████████▏ | 15161/18627 [8:33:15<1:49:05,  1.89s/it]
 81%|████████▏ | 15162/18627 [8:33:17<1:56:34,  2.02s/it]
 81%|████████▏ | 15163/18627 [8:33:18<1:45:03,  1.82s/it]
 81%|████████▏ | 15164/18627 [8:33:21<1:54:01,  1.98s/it]
 81%|████████▏ | 15165/18627 [8:33:22<1:43:34,  1.80s/it]
 81%|████████▏ | 15166/18627 [8:33:24<1:52:54,  1.96s/it]
 81%|████████▏ | 15167/18627 [8:33:27<1:59:13,  2.07s/it]
 81%|████████▏ | 15168/18627 [8:33:29<2:04:17,  2.16s/it]
 81%|████████▏ | 15169/18627 [8:33:31<2:07:37,  2.21s/it]
 81%|████████▏ | 15170/18627 [8:33:34<2:10:26,  2.26s/it]
                                                         
{'loss': 1.1434, 'grad_norm': 6.951791286468506, 'learning_rate': 4.382001881031514e-07, 'epoch': 0.81}
+
 81%|████████▏ | 15170/18627 [8:33:34<2:10:26,  2.26s/it]
 81%|████████▏ | 15171/18627 [8:33:36<2:11:47,  2.29s/it]
 81%|████████▏ | 15172/18627 [8:33:38<1:55:20,  2.00s/it]
 81%|████████▏ | 15173/18627 [8:33:40<2:01:41,  2.11s/it]
 81%|████████▏ | 15174/18627 [8:33:42<2:06:07,  2.19s/it]
 81%|████████▏ | 15175/18627 [8:33:45<2:08:58,  2.24s/it]
 81%|████████▏ | 15176/18627 [8:33:47<2:10:42,  2.27s/it]
 81%|████████▏ | 15177/18627 [8:33:49<2:11:23,  2.29s/it]
 81%|████████▏ | 15178/18627 [8:33:52<2:12:32,  2.31s/it]
 81%|████████▏ | 15179/18627 [8:33:54<2:13:24,  2.32s/it]
 81%|████████▏ | 15180/18627 [8:33:56<2:12:58,  2.31s/it]
                                                         
{'loss': 0.7286, 'grad_norm': 5.524426460266113, 'learning_rate': 4.3574495340070437e-07, 'epoch': 0.81}
+
 81%|████████▏ | 15180/18627 [8:33:56<2:12:58,  2.31s/it]
 81%|████████▏ | 15181/18627 [8:33:58<1:56:45,  2.03s/it]
 82%|████████▏ | 15182/18627 [8:34:00<2:01:31,  2.12s/it]
 82%|████████▏ | 15183/18627 [8:34:02<2:05:14,  2.18s/it]
 82%|████████▏ | 15184/18627 [8:34:05<2:08:04,  2.23s/it]
 82%|████████▏ | 15185/18627 [8:34:06<1:48:42,  1.89s/it]
 82%|████████▏ | 15186/18627 [8:34:07<1:38:08,  1.71s/it]
 82%|████████▏ | 15187/18627 [8:34:09<1:48:40,  1.90s/it]
 82%|████████▏ | 15188/18627 [8:34:12<1:56:08,  2.03s/it]
 82%|████████▏ | 15189/18627 [8:34:14<2:02:01,  2.13s/it]
 82%|████████▏ | 15190/18627 [8:34:16<2:05:47,  2.20s/it]
                                                         
{'loss': 0.9624, 'grad_norm': 7.650599956512451, 'learning_rate': 4.3329595953507887e-07, 'epoch': 0.82}
+
 82%|████████▏ | 15190/18627 [8:34:16<2:05:47,  2.20s/it]
 82%|████████▏ | 15191/18627 [8:34:19<2:08:14,  2.24s/it]
 82%|████████▏ | 15192/18627 [8:34:21<2:10:02,  2.27s/it]
 82%|████████▏ | 15193/18627 [8:34:23<2:11:47,  2.30s/it]
 82%|████████▏ | 15194/18627 [8:34:26<2:12:38,  2.32s/it]
 82%|████████▏ | 15195/18627 [8:34:27<1:54:18,  2.00s/it]
 82%|████████▏ | 15196/18627 [8:34:29<2:00:08,  2.10s/it]
 82%|████████▏ | 15197/18627 [8:34:32<2:03:55,  2.17s/it]
 82%|████████▏ | 15198/18627 [8:34:34<2:06:35,  2.22s/it]
 82%|████████▏ | 15199/18627 [8:34:36<2:08:14,  2.24s/it]
 82%|████████▏ | 15200/18627 [8:34:39<2:09:27,  2.27s/it]
                                                         
{'loss': 0.8099, 'grad_norm': 11.388049125671387, 'learning_rate': 4.3085321391028527e-07, 'epoch': 0.82}
+
 82%|████████▏ | 15200/18627 [8:34:39<2:09:27,  2.27s/it]
 82%|████████▏ | 15201/18627 [8:34:41<2:10:34,  2.29s/it]
 82%|████████▏ | 15202/18627 [8:34:42<1:49:50,  1.92s/it]
 82%|████████▏ | 15203/18627 [8:34:44<1:57:26,  2.06s/it]
 82%|████████▏ | 15204/18627 [8:34:47<2:02:08,  2.14s/it]
 82%|████████▏ | 15205/18627 [8:34:48<1:48:42,  1.91s/it]
 82%|████████▏ | 15206/18627 [8:34:51<1:55:42,  2.03s/it]
 82%|████████�� | 15207/18627 [8:34:53<2:00:25,  2.11s/it]
 82%|████████▏ | 15208/18627 [8:34:54<1:50:00,  1.93s/it]
 82%|████████▏ | 15209/18627 [8:34:55<1:35:01,  1.67s/it]
 82%|████████▏ | 15210/18627 [8:34:58<1:47:21,  1.89s/it]
                                                         
{'loss': 1.1217, 'grad_norm': 7.6944756507873535, 'learning_rate': 4.2841672391144895e-07, 'epoch': 0.82}
+
 82%|████████▏ | 15210/18627 [8:34:58<1:47:21,  1.89s/it]
 82%|████████▏ | 15211/18627 [8:35:00<1:55:07,  2.02s/it]
 82%|████████▏ | 15212/18627 [8:35:02<2:00:31,  2.12s/it]
 82%|████████▏ | 15213/18627 [8:35:05<2:05:22,  2.20s/it]
 82%|████████▏ | 15214/18627 [8:35:07<2:08:07,  2.25s/it]
 82%|████████▏ | 15215/18627 [8:35:10<2:13:20,  2.34s/it]
 82%|████████▏ | 15216/18627 [8:35:12<2:14:20,  2.36s/it]
 82%|████████▏ | 15217/18627 [8:35:14<2:01:13,  2.13s/it]
 82%|████████▏ | 15218/18627 [8:35:15<1:48:30,  1.91s/it]
 82%|████████▏ | 15219/18627 [8:35:18<1:56:10,  2.05s/it]
 82%|████████▏ | 15220/18627 [8:35:20<2:01:01,  2.13s/it]
                                                         
{'loss': 0.9964, 'grad_norm': 5.906994342803955, 'learning_rate': 4.2598649690477864e-07, 'epoch': 0.82}
+
 82%|████████▏ | 15220/18627 [8:35:20<2:01:01,  2.13s/it]
 82%|████████▏ | 15221/18627 [8:35:22<2:04:45,  2.20s/it]
 82%|████████▏ | 15222/18627 [8:35:24<1:51:03,  1.96s/it]
 82%|████████▏ | 15223/18627 [8:35:26<1:57:31,  2.07s/it]
 82%|████████▏ | 15224/18627 [8:35:27<1:40:54,  1.78s/it]
 82%|████████▏ | 15225/18627 [8:35:29<1:50:28,  1.95s/it]
 82%|████████▏ | 15226/18627 [8:35:32<1:57:23,  2.07s/it]
 82%|████████▏ | 15227/18627 [8:35:34<2:01:28,  2.14s/it]
 82%|████████▏ | 15228/18627 [8:35:36<2:05:20,  2.21s/it]
 82%|████████▏ | 15229/18627 [8:35:39<2:07:32,  2.25s/it]
 82%|████████▏ | 15230/18627 [8:35:40<1:51:31,  1.97s/it]
                                                         
{'loss': 0.94, 'grad_norm': 13.138711929321289, 'learning_rate': 4.2356254023754956e-07, 'epoch': 0.82}
+
 82%|████████▏ | 15230/18627 [8:35:40<1:51:31,  1.97s/it]
 82%|████████▏ | 15231/18627 [8:35:42<1:57:54,  2.08s/it]
 82%|████████▏ | 15232/18627 [8:35:45<2:02:10,  2.16s/it]
 82%|████████▏ | 15233/18627 [8:35:47<2:04:51,  2.21s/it]
 82%|████████▏ | 15234/18627 [8:35:49<2:06:47,  2.24s/it]
 82%|████████▏ | 15235/18627 [8:35:52<2:09:28,  2.29s/it]
 82%|████████▏ | 15236/18627 [8:35:54<2:09:59,  2.30s/it]
 82%|████████▏ | 15237/18627 [8:35:56<2:10:52,  2.32s/it]
 82%|████████▏ | 15238/18627 [8:35:59<2:11:50,  2.33s/it]
 82%|████████▏ | 15239/18627 [8:36:01<2:12:45,  2.35s/it]
 82%|████████▏ | 15240/18627 [8:36:04<2:13:02,  2.36s/it]
                                                         
{'loss': 0.5789, 'grad_norm': 4.798529624938965, 'learning_rate': 4.2114486123808087e-07, 'epoch': 0.82}
+
 82%|████████▏ | 15240/18627 [8:36:04<2:13:02,  2.36s/it]
 82%|████████▏ | 15241/18627 [8:36:05<1:58:22,  2.10s/it]
 82%|████████▏ | 15242/18627 [8:36:07<2:02:52,  2.18s/it]
 82%|████████▏ | 15243/18627 [8:36:10<2:05:52,  2.23s/it]
 82%|████████▏ | 15244/18627 [8:36:12<2:08:15,  2.27s/it]
 82%|████████▏ | 15245/18627 [8:36:15<2:12:41,  2.35s/it]
 82%|████████▏ | 15246/18627 [8:36:17<2:12:09,  2.35s/it]
 82%|████████▏ | 15247/18627 [8:36:19<2:11:42,  2.34s/it]
 82%|████████▏ | 15248/18627 [8:36:22<2:11:36,  2.34s/it]
 82%|████████▏ | 15249/18627 [8:36:24<2:11:52,  2.34s/it]
 82%|████████▏ | 15250/18627 [8:36:26<2:11:24,  2.33s/it]
                                                         
{'loss': 0.7818, 'grad_norm': 8.891928672790527, 'learning_rate': 4.187334672157109e-07, 'epoch': 0.82}
+
 82%|████████▏ | 15250/18627 [8:36:26<2:11:24,  2.33s/it]
 82%|████████▏ | 15251/18627 [8:36:29<2:11:30,  2.34s/it]
 82%|████████▏ | 15252/18627 [8:36:31<2:11:28,  2.34s/it]
 82%|████████▏ | 15253/18627 [8:36:33<2:11:23,  2.34s/it]
 82%|████████▏ | 15254/18627 [8:36:36<2:11:22,  2.34s/it]
 82%|████████▏ | 15255/18627 [8:36:37<1:53:52,  2.03s/it]
 82%|████████▏ | 15256/18627 [8:36:39<1:59:03,  2.12s/it]
 82%|████████▏ | 15257/18627 [8:36:42<2:03:25,  2.20s/it]
 82%|████████▏ | 15258/18627 [8:36:44<2:05:35,  2.24s/it]
 82%|████████▏ | 15259/18627 [8:36:46<2:07:16,  2.27s/it]
 82%|████████▏ | 15260/18627 [8:36:49<2:08:41,  2.29s/it]
                                                         
{'loss': 0.7588, 'grad_norm': 7.39893913269043, 'learning_rate': 4.1632836546077774e-07, 'epoch': 0.82}
+
 82%|████████▏ | 15260/18627 [8:36:49<2:08:41,  2.29s/it]
 82%|████████▏ | 15261/18627 [8:36:51<2:09:49,  2.31s/it]
 82%|████████▏ | 15262/18627 [8:36:54<2:10:30,  2.33s/it]
 82%|████████▏ | 15263/18627 [8:36:56<2:10:21,  2.32s/it]
 82%|████████▏ | 15264/18627 [8:36:57<1:54:35,  2.04s/it]
 82%|████████▏ | 15265/18627 [8:37:00<1:59:46,  2.14s/it]
 82%|████████▏ | 15266/18627 [8:37:01<1:46:14,  1.90s/it]
 82%|████████▏ | 15267/18627 [8:37:02<1:32:23,  1.65s/it]
 82%|████████▏ | 15268/18627 [8:37:03<1:28:23,  1.58s/it]
 82%|████████▏ | 15269/18627 [8:37:06<1:41:42,  1.82s/it]
 82%|████████▏ | 15270/18627 [8:37:08<1:50:04,  1.97s/it]
                                                         
{'loss': 1.1293, 'grad_norm': 8.864723205566406, 'learning_rate': 4.1392956324459665e-07, 'epoch': 0.82}
+
 82%|████████▏ | 15270/18627 [8:37:08<1:50:04,  1.97s/it]
 82%|████████▏ | 15271/18627 [8:37:09<1:39:43,  1.78s/it]
 82%|████████▏ | 15272/18627 [8:37:12<1:49:33,  1.96s/it]
 82%|████████▏ | 15273/18627 [8:37:13<1:41:59,  1.82s/it]
 82%|████████▏ | 15274/18627 [8:37:15<1:35:00,  1.70s/it]
 82%|████████▏ | 15275/18627 [8:37:16<1:30:18,  1.62s/it]
 82%|████████▏ | 15276/18627 [8:37:19<1:42:39,  1.84s/it]
 82%|████████▏ | 15277/18627 [8:37:21<1:50:56,  1.99s/it]
 82%|████████▏ | 15278/18627 [8:37:23<1:56:35,  2.09s/it]
 82%|████████▏ | 15279/18627 [8:37:26<2:00:51,  2.17s/it]
 82%|████████▏ | 15280/18627 [8:37:27<1:47:17,  1.92s/it]
                                                         
{'loss': 1.5874, 'grad_norm': 13.912874221801758, 'learning_rate': 4.115370678194347e-07, 'epoch': 0.82}
+
 82%|████████▏ | 15280/18627 [8:37:27<1:47:17,  1.92s/it]
 82%|████████▏ | 15281/18627 [8:37:29<1:55:20,  2.07s/it]
 82%|████████▏ | 15282/18627 [8:37:31<1:44:43,  1.88s/it]
 82%|████████▏ | 15283/18627 [8:37:33<1:52:02,  2.01s/it]
 82%|████████▏ | 15284/18627 [8:37:35<1:57:48,  2.11s/it]
 82%|████████▏ | 15285/18627 [8:37:38<2:01:41,  2.18s/it]
 82%|████████▏ | 15286/18627 [8:37:39<1:48:44,  1.95s/it]
 82%|████████▏ | 15287/18627 [8:37:41<1:39:24,  1.79s/it]
 82%|████████▏ | 15288/18627 [8:37:43<1:48:15,  1.95s/it]
 82%|████████▏ | 15289/18627 [8:37:45<1:54:43,  2.06s/it]
 82%|████████▏ | 15290/18627 [8:37:48<1:58:27,  2.13s/it]
                                                         
{'loss': 1.219, 'grad_norm': 9.129638671875, 'learning_rate': 4.091508864184962e-07, 'epoch': 0.82}
+
 82%|████████▏ | 15290/18627 [8:37:48<1:58:27,  2.13s/it]
 82%|████████▏ | 15291/18627 [8:37:49<1:40:07,  1.80s/it]
 82%|████████▏ | 15292/18627 [8:37:51<1:49:21,  1.97s/it]
 82%|████████▏ | 15293/18627 [8:37:53<1:55:48,  2.08s/it]
 82%|████████▏ | 15294/18627 [8:37:56<1:59:41,  2.15s/it]
 82%|████████▏ | 15295/18627 [8:37:58<2:02:51,  2.21s/it]
 82%|████████▏ | 15296/18627 [8:38:00<2:04:33,  2.24s/it]
 82%|████████▏ | 15297/18627 [8:38:03<2:06:45,  2.28s/it]
 82%|████████▏ | 15298/18627 [8:38:05<2:07:13,  2.29s/it]
 82%|████████▏ | 15299/18627 [8:38:07<2:08:30,  2.32s/it]
 82%|████████▏ | 15300/18627 [8:38:09<1:52:55,  2.04s/it]
                                                         
{'loss': 0.8687, 'grad_norm': 14.170513153076172, 'learning_rate': 4.067710262558924e-07, 'epoch': 0.82}
+
 82%|████████▏ | 15300/18627 [8:38:09<1:52:55,  2.04s/it]
 82%|████████▏ | 15301/18627 [8:38:11<1:57:35,  2.12s/it]
 82%|████████▏ | 15302/18627 [8:38:13<2:01:32,  2.19s/it]
 82%|████████▏ | 15303/18627 [8:38:16<2:03:56,  2.24s/it]
 82%|████████▏ | 15304/18627 [8:38:18<2:05:56,  2.27s/it]
 82%|████████▏ | 15305/18627 [8:38:20<2:07:46,  2.31s/it]
 82%|████████▏ | 15306/18627 [8:38:22<1:52:18,  2.03s/it]
 82%|████████▏ | 15307/18627 [8:38:24<1:57:20,  2.12s/it]
 82%|████████▏ | 15308/18627 [8:38:26<2:00:54,  2.19s/it]
 82%|████████▏ | 15309/18627 [8:38:29<2:04:46,  2.26s/it]
 82%|████████▏ | 15310/18627 [8:38:31<2:06:18,  2.28s/it]
                                                         
{'loss': 0.8751, 'grad_norm': 5.297933101654053, 'learning_rate': 4.0439749452662575e-07, 'epoch': 0.82}
+
 82%|████████▏ | 15310/18627 [8:38:31<2:06:18,  2.28s/it]
 82%|████████▏ | 15311/18627 [8:38:34<2:07:46,  2.31s/it]
 82%|████████▏ | 15312/18627 [8:38:36<2:07:43,  2.31s/it]
 82%|████████▏ | 15313/18627 [8:38:38<2:08:24,  2.32s/it]
 82%|████████▏ | 15314/18627 [8:38:41<2:08:16,  2.32s/it]
 82%|████████▏ | 15315/18627 [8:38:43<2:08:16,  2.32s/it]
 82%|████████▏ | 15316/18627 [8:38:45<2:08:23,  2.33s/it]
 82%|████████▏ | 15317/18627 [8:38:48<2:09:10,  2.34s/it]
 82%|████████▏ | 15318/18627 [8:38:50<2:09:24,  2.35s/it]
 82%|████████▏ | 15319/18627 [8:38:52<2:09:17,  2.35s/it]
 82%|████████▏ | 15320/18627 [8:38:54<1:53:22,  2.06s/it]
                                                         
{'loss': 0.8279, 'grad_norm': 14.562607765197754, 'learning_rate': 4.020302984065652e-07, 'epoch': 0.82}
+
 82%|████████▏ | 15320/18627 [8:38:54<1:53:22,  2.06s/it]
 82%|████████▏ | 15321/18627 [8:38:56<1:58:02,  2.14s/it]
 82%|████████▏ | 15322/18627 [8:38:59<2:02:37,  2.23s/it]
 82%|████████▏ | 15323/18627 [8:39:01<2:04:20,  2.26s/it]
 82%|████████▏ | 15324/18627 [8:39:03<2:05:22,  2.28s/it]
 82%|████████▏ | 15325/18627 [8:39:06<2:06:45,  2.30s/it]
 82%|████████▏ | 15326/18627 [8:39:07<1:50:15,  2.00s/it]
 82%|████████▏ | 15327/18627 [8:39:09<1:55:28,  2.10s/it]
 82%|████████▏ | 15328/18627 [8:39:12<1:59:44,  2.18s/it]
 82%|████████▏ | 15329/18627 [8:39:14<2:02:57,  2.24s/it]
 82%|████████▏ | 15330/18627 [8:39:16<2:05:53,  2.29s/it]
                                                         
{'loss': 0.8008, 'grad_norm': 6.039121150970459, 'learning_rate': 3.9966944505242576e-07, 'epoch': 0.82}
+
 82%|████████▏ | 15330/18627 [8:39:16<2:05:53,  2.29s/it]
 82%|████████▏ | 15331/18627 [8:39:19<2:07:17,  2.32s/it]
 82%|████████▏ | 15332/18627 [8:39:20<1:52:09,  2.04s/it]
 82%|████████▏ | 15333/18627 [8:39:22<1:57:16,  2.14s/it]
 82%|████████▏ | 15334/18627 [8:39:24<1:45:31,  1.92s/it]
 82%|████████▏ | 15335/18627 [8:39:25<1:36:56,  1.77s/it]
 82%|████████▏ | 15336/18627 [8:39:28<1:47:07,  1.95s/it]
 82%|████████▏ | 15337/18627 [8:39:30<1:53:35,  2.07s/it]
 82%|████████▏ | 15338/18627 [8:39:32<1:58:15,  2.16s/it]
 82%|████████▏ | 15339/18627 [8:39:35<2:00:55,  2.21s/it]
 82%|████████▏ | 15340/18627 [8:39:36<1:47:08,  1.96s/it]
                                                         
{'loss': 1.3814, 'grad_norm': 12.649148941040039, 'learning_rate': 3.9731494160174605e-07, 'epoch': 0.82}
+
 82%|████████▏ | 15340/18627 [8:39:36<1:47:08,  1.96s/it]
 82%|████████▏ | 15341/18627 [8:39:37<1:36:45,  1.77s/it]
 82%|████████▏ | 15342/18627 [8:39:40<1:46:14,  1.94s/it]
 82%|████████▏ | 15343/18627 [8:39:42<1:52:14,  2.05s/it]
 82%|████████▏ | 15344/18627 [8:39:44<1:56:31,  2.13s/it]
 82%|████████▏ | 15345/18627 [8:39:47<1:59:28,  2.18s/it]
 82%|████████▏ | 15346/18627 [8:39:49<2:01:59,  2.23s/it]
 82%|████████▏ | 15347/18627 [8:39:51<2:03:50,  2.27s/it]
 82%|████████▏ | 15348/18627 [8:39:53<1:48:34,  1.99s/it]
 82%|████████▏ | 15349/18627 [8:39:55<1:54:34,  2.10s/it]
 82%|████████▏ | 15350/18627 [8:39:57<1:58:37,  2.17s/it]
                                                         
{'loss': 0.9989, 'grad_norm': 5.797545433044434, 'learning_rate': 3.9496679517286806e-07, 'epoch': 0.82}
+
 82%|████████▏ | 15350/18627 [8:39:57<1:58:37,  2.17s/it]
 82%|████████▏ | 15351/18627 [8:40:00<2:01:22,  2.22s/it]
 82%|████████▏ | 15352/18627 [8:40:02<2:03:36,  2.26s/it]
 82%|████████▏ | 15353/18627 [8:40:03<1:47:23,  1.97s/it]
 82%|████████▏ | 15354/18627 [8:40:06<1:54:40,  2.10s/it]
 82%|████████▏ | 15355/18627 [8:40:08<1:59:43,  2.20s/it]
 82%|████████▏ | 15356/18627 [8:40:11<2:02:44,  2.25s/it]
 82%|████████▏ | 15357/18627 [8:40:13<2:04:25,  2.28s/it]
 82%|████████▏ | 15358/18627 [8:40:15<2:05:51,  2.31s/it]
 82%|████████▏ | 15359/18627 [8:40:18<2:06:23,  2.32s/it]
 82%|████████▏ | 15360/18627 [8:40:20<2:06:51,  2.33s/it]
                                                         
{'loss': 0.7848, 'grad_norm': 5.674045085906982, 'learning_rate': 3.926250128649123e-07, 'epoch': 0.82}
+
 82%|████████▏ | 15360/18627 [8:40:20<2:06:51,  2.33s/it]
 82%|████████▏ | 15361/18627 [8:40:22<2:07:57,  2.35s/it]
 82%|████████▏ | 15362/18627 [8:40:25<2:07:49,  2.35s/it]
 82%|████████▏ | 15363/18627 [8:40:27<2:08:37,  2.36s/it]
 82%|████████▏ | 15364/18627 [8:40:30<2:09:27,  2.38s/it]
 82%|████████▏ | 15365/18627 [8:40:32<2:08:45,  2.37s/it]
 82%|████████▏ | 15366/18627 [8:40:34<2:08:04,  2.36s/it]
 82%|████████▏ | 15367/18627 [8:40:37<2:07:27,  2.35s/it]
 83%|████████▎ | 15368/18627 [8:40:39<2:07:03,  2.34s/it]
 83%|████████▎ | 15369/18627 [8:40:41<2:07:36,  2.35s/it]
 83%|████████▎ | 15370/18627 [8:40:43<1:52:08,  2.07s/it]
                                                         
{'loss': 0.8516, 'grad_norm': 14.328997611999512, 'learning_rate': 3.902896017577626e-07, 'epoch': 0.83}
+
 83%|████████▎ | 15370/18627 [8:40:43<1:52:08,  2.07s/it]
 83%|████████▎ | 15371/18627 [8:40:45<1:56:43,  2.15s/it]
 83%|████████▎ | 15372/18627 [8:40:46<1:44:28,  1.93s/it]
 83%|████████▎ | 15373/18627 [8:40:49<1:52:21,  2.07s/it]
 83%|████████▎ | 15374/18627 [8:40:51<1:57:41,  2.17s/it]
 83%|████████▎ | 15375/18627 [8:40:54<2:00:37,  2.23s/it]
 83%|████████▎ | 15376/18627 [8:40:55<1:47:11,  1.98s/it]
 83%|████████▎ | 15377/18627 [8:40:57<1:52:59,  2.09s/it]
 83%|████████▎ | 15378/18627 [8:40:58<1:36:18,  1.78s/it]
 83%|████████▎ | 15379/18627 [8:41:01<1:45:15,  1.94s/it]
 83%|████████▎ | 15380/18627 [8:41:03<1:50:56,  2.05s/it]
                                                         
{'loss': 1.0528, 'grad_norm': 5.823481559753418, 'learning_rate': 3.879605689120367e-07, 'epoch': 0.83}
+
 83%|████████▎ | 15380/18627 [8:41:03<1:50:56,  2.05s/it]
 83%|████████▎ | 15381/18627 [8:41:04<1:40:02,  1.85s/it]
 83%|████████▎ | 15382/18627 [8:41:07<1:48:09,  2.00s/it]
 83%|████████▎ | 15383/18627 [8:41:08<1:33:34,  1.73s/it]
 83%|████████▎ | 15384/18627 [8:41:09<1:28:42,  1.64s/it]
 83%|████████▎ | 15385/18627 [8:41:12<1:39:51,  1.85s/it]
 83%|████████▎ | 15386/18627 [8:41:14<1:48:37,  2.01s/it]
 83%|████████▎ | 15387/18627 [8:41:16<1:53:36,  2.10s/it]
 83%|████████▎ | 15388/18627 [8:41:18<1:42:08,  1.89s/it]
 83%|████████▎ | 15389/18627 [8:41:20<1:49:53,  2.04s/it]
 83%|████████▎ | 15390/18627 [8:41:22<1:55:13,  2.14s/it]
                                                         
{'loss': 1.2037, 'grad_norm': 6.061529159545898, 'learning_rate': 3.856379213690714e-07, 'epoch': 0.83}
+
 83%|████████▎ | 15390/18627 [8:41:22<1:55:13,  2.14s/it]
 83%|████████▎ | 15391/18627 [8:41:25<1:58:14,  2.19s/it]
 83%|████████▎ | 15392/18627 [8:41:27<2:00:49,  2.24s/it]
 83%|████████▎ | 15393/18627 [8:41:30<2:04:51,  2.32s/it]
 83%|████████▎ | 15394/18627 [8:41:32<2:05:30,  2.33s/it]
 83%|████████▎ | 15395/18627 [8:41:34<2:06:21,  2.35s/it]
 83%|████████▎ | 15396/18627 [8:41:37<2:06:01,  2.34s/it]
 83%|████████▎ | 15397/18627 [8:41:39<2:06:15,  2.35s/it]
 83%|████████▎ | 15398/18627 [8:41:41<2:06:31,  2.35s/it]
 83%|████████▎ | 15399/18627 [8:41:44<2:07:03,  2.36s/it]
 83%|████████▎ | 15400/18627 [8:41:46<2:07:12,  2.37s/it]
                                                         
{'loss': 0.5627, 'grad_norm': 7.215034008026123, 'learning_rate': 3.833216661508984e-07, 'epoch': 0.83}
+
 83%|████████▎ | 15400/18627 [8:41:46<2:07:12,  2.37s/it]
 83%|████████▎ | 15401/18627 [8:41:48<1:50:55,  2.06s/it]
 83%|████████▎ | 15402/18627 [8:41:49<1:39:44,  1.86s/it]
 83%|████████▎ | 15403/18627 [8:41:51<1:48:02,  2.01s/it]
 83%|████████▎ | 15404/18627 [8:41:54<1:53:02,  2.10s/it]
 83%|████████▎ | 15405/18627 [8:41:55<1:41:33,  1.89s/it]
 83%|████████▎ | 15406/18627 [8:41:57<1:48:42,  2.03s/it]
 83%|████████▎ | 15407/18627 [8:42:00<1:53:15,  2.11s/it]
 83%|████████▎ | 15408/18627 [8:42:01<1:41:26,  1.89s/it]
 83%|████████▎ | 15409/18627 [8:42:03<1:49:13,  2.04s/it]
 83%|████████▎ | 15410/18627 [8:42:06<1:54:45,  2.14s/it]
                                                         
{'loss': 1.3848, 'grad_norm': 5.339962005615234, 'learning_rate': 3.8101181026022366e-07, 'epoch': 0.83}
+
 83%|████████▎ | 15410/18627 [8:42:06<1:54:45,  2.14s/it]
 83%|████████▎ | 15411/18627 [8:42:08<1:58:00,  2.20s/it]
 83%|████████▎ | 15412/18627 [8:42:10<2:00:07,  2.24s/it]
 83%|████████▎ | 15413/18627 [8:42:12<1:46:14,  1.98s/it]
 83%|████████▎ | 15414/18627 [8:42:14<1:52:19,  2.10s/it]
 83%|████████▎ | 15415/18627 [8:42:17<1:55:54,  2.17s/it]
 83%|████████▎ | 15416/18627 [8:42:19<1:58:14,  2.21s/it]
 83%|████████▎ | 15417/18627 [8:42:21<2:00:36,  2.25s/it]
 83%|████████▎ | 15418/18627 [8:42:24<2:02:39,  2.29s/it]
 83%|████████▎ | 15419/18627 [8:42:26<2:03:20,  2.31s/it]
 83%|████████▎ | 15420/18627 [8:42:28<2:03:42,  2.31s/it]
                                                         
{'loss': 0.7389, 'grad_norm': 5.421991348266602, 'learning_rate': 3.7870836068040596e-07, 'epoch': 0.83}
+
 83%|████████▎ | 15420/18627 [8:42:28<2:03:42,  2.31s/it]
 83%|████████▎ | 15421/18627 [8:42:31<2:04:58,  2.34s/it]
 83%|████████▎ | 15422/18627 [8:42:33<2:05:44,  2.35s/it]
 83%|████████▎ | 15423/18627 [8:42:35<2:06:11,  2.36s/it]
 83%|████████▎ | 15424/18627 [8:42:38<2:06:11,  2.36s/it]
 83%|████████▎ | 15425/18627 [8:42:39<1:51:03,  2.08s/it]
 83%|████████▎ | 15426/18627 [8:42:42<1:55:24,  2.16s/it]
 83%|████████▎ | 15427/18627 [8:42:44<1:57:53,  2.21s/it]
 83%|████████▎ | 15428/18627 [8:42:46<1:59:55,  2.25s/it]
 83%|████████▎ | 15429/18627 [8:42:48<1:46:23,  2.00s/it]
 83%|████████▎ | 15430/18627 [8:42:50<1:52:20,  2.11s/it]
                                                         
{'loss': 0.9713, 'grad_norm': 5.2090654373168945, 'learning_rate': 3.7641132437543667e-07, 'epoch': 0.83}
+
 83%|████████▎ | 15430/18627 [8:42:50<1:52:20,  2.11s/it]
 83%|████████▎ | 15431/18627 [8:42:52<1:55:56,  2.18s/it]
 83%|████████▎ | 15432/18627 [8:42:55<1:59:56,  2.25s/it]
 83%|████████▎ | 15433/18627 [8:42:57<2:01:21,  2.28s/it]
 83%|████████▎ | 15434/18627 [8:42:59<2:01:53,  2.29s/it]
 83%|████████▎ | 15435/18627 [8:43:02<2:03:04,  2.31s/it]
 83%|████████▎ | 15436/18627 [8:43:04<2:03:32,  2.32s/it]
 83%|████████▎ | 15437/18627 [8:43:06<2:03:42,  2.33s/it]
 83%|████████▎ | 15438/18627 [8:43:08<1:49:24,  2.06s/it]
 83%|████████▎ | 15439/18627 [8:43:10<1:54:26,  2.15s/it]
 83%|████████▎ | 15440/18627 [8:43:12<1:42:35,  1.93s/it]
                                                         
{'loss': 1.03, 'grad_norm': 13.75158977508545, 'learning_rate': 3.741207082899154e-07, 'epoch': 0.83}
+
 83%|████████▎ | 15440/18627 [8:43:12<1:42:35,  1.93s/it]
 83%|████████▎ | 15441/18627 [8:43:14<1:48:56,  2.05s/it]
 83%|████████▎ | 15442/18627 [8:43:16<1:53:55,  2.15s/it]
 83%|████████▎ | 15443/18627 [8:43:19<1:58:01,  2.22s/it]
 83%|████████▎ | 15444/18627 [8:43:20<1:43:57,  1.96s/it]
 83%|████████▎ | 15445/18627 [8:43:22<1:49:40,  2.07s/it]
 83%|████████▎ | 15446/18627 [8:43:25<1:53:51,  2.15s/it]
 83%|████████▎ | 15447/18627 [8:43:27<1:57:15,  2.21s/it]
 83%|████████▎ | 15448/18627 [8:43:30<2:01:47,  2.30s/it]
 83%|████████▎ | 15449/18627 [8:43:32<2:02:04,  2.30s/it]
 83%|████████▎ | 15450/18627 [8:43:34<2:02:19,  2.31s/it]
                                                         
{'loss': 0.8533, 'grad_norm': 7.3717360496521, 'learning_rate': 3.71836519349035e-07, 'epoch': 0.83}
+
 83%|████████▎ | 15450/18627 [8:43:34<2:02:19,  2.31s/it]
 83%|████████▎ | 15451/18627 [8:43:37<2:03:40,  2.34s/it]
 83%|████████▎ | 15452/18627 [8:43:39<2:03:33,  2.33s/it]
 83%|████████▎ | 15453/18627 [8:43:41<2:03:15,  2.33s/it]
 83%|████████▎ | 15454/18627 [8:43:43<1:49:06,  2.06s/it]
 83%|████████▎ | 15455/18627 [8:43:45<1:52:56,  2.14s/it]
 83%|████████▎ | 15456/18627 [8:43:47<1:56:12,  2.20s/it]
 83%|████████▎ | 15457/18627 [8:43:50<1:58:45,  2.25s/it]
 83%|████████▎ | 15458/18627 [8:43:52<1:59:41,  2.27s/it]
 83%|████████▎ | 15459/18627 [8:43:55<2:01:57,  2.31s/it]
 83%|████████▎ | 15460/18627 [8:43:57<2:02:50,  2.33s/it]
                                                         
{'loss': 0.73, 'grad_norm': 7.324709892272949, 'learning_rate': 3.695587644585566e-07, 'epoch': 0.83}
+
 83%|████████▎ | 15460/18627 [8:43:57<2:02:50,  2.33s/it]
 83%|████████▎ | 15461/18627 [8:43:59<2:03:30,  2.34s/it]
 83%|████████▎ | 15462/18627 [8:44:02<2:03:22,  2.34s/it]
 83%|████████▎ | 15463/18627 [8:44:04<2:04:01,  2.35s/it]
 83%|████████▎ | 15464/18627 [8:44:06<2:04:04,  2.35s/it]
 83%|████████▎ | 15465/18627 [8:44:09<2:04:15,  2.36s/it]
 83%|████████▎ | 15466/18627 [8:44:10<1:49:13,  2.07s/it]
 83%|████████▎ | 15467/18627 [8:44:13<1:54:25,  2.17s/it]
 83%|████████▎ | 15468/18627 [8:44:14<1:40:48,  1.91s/it]
 83%|████████▎ | 15469/18627 [8:44:16<1:47:55,  2.05s/it]
 83%|████████▎ | 15470/18627 [8:44:19<1:52:09,  2.13s/it]
                                                         
{'loss': 1.0157, 'grad_norm': 5.253335475921631, 'learning_rate': 3.6728745050478575e-07, 'epoch': 0.83}
+
 83%|████████▎ | 15470/18627 [8:44:19<1:52:09,  2.13s/it]
 83%|████████▎ | 15471/18627 [8:44:21<1:55:49,  2.20s/it]
 83%|████████▎ | 15472/18627 [8:44:23<1:57:37,  2.24s/it]
 83%|████████▎ | 15473/18627 [8:44:25<1:45:11,  2.00s/it]
 83%|████████▎ | 15474/18627 [8:44:26<1:35:26,  1.82s/it]
 83%|████████▎ | 15475/18627 [8:44:27<1:28:42,  1.69s/it]
 83%|████████▎ | 15476/18627 [8:44:30<1:38:58,  1.88s/it]
 83%|████████▎ | 15477/18627 [8:44:32<1:46:03,  2.02s/it]
 83%|████████▎ | 15478/18627 [8:44:34<1:50:58,  2.11s/it]
 83%|████████▎ | 15479/18627 [8:44:37<1:54:36,  2.18s/it]
 83%|████████▎ | 15480/18627 [8:44:39<1:57:57,  2.25s/it]
                                                         
{'loss': 1.1621, 'grad_norm': 6.440526962280273, 'learning_rate': 3.650225843545607e-07, 'epoch': 0.83}
+
 83%|████████▎ | 15480/18627 [8:44:39<1:57:57,  2.25s/it]
 83%|████████▎ | 15481/18627 [8:44:42<1:59:29,  2.28s/it]
 83%|████████▎ | 15482/18627 [8:44:44<2:00:15,  2.29s/it]
 83%|████████▎ | 15483/18627 [8:44:46<2:01:34,  2.32s/it]
 83%|████████▎ | 15484/18627 [8:44:49<2:01:54,  2.33s/it]
 83%|████████▎ | 15485/18627 [8:44:50<1:42:48,  1.96s/it]
 83%|████████▎ | 15486/18627 [8:44:52<1:48:59,  2.08s/it]
 83%|████████▎ | 15487/18627 [8:44:54<1:53:48,  2.17s/it]
 83%|████████▎ | 15488/18627 [8:44:56<1:41:55,  1.95s/it]
 83%|████████▎ | 15489/18627 [8:44:58<1:48:21,  2.07s/it]
 83%|████████▎ | 15490/18627 [8:45:00<1:37:07,  1.86s/it]
                                                         
{'loss': 0.976, 'grad_norm': 14.390092849731445, 'learning_rate': 3.6276417285522116e-07, 'epoch': 0.83}
+
 83%|████████▎ | 15490/18627 [8:45:00<1:37:07,  1.86s/it]
 83%|████████▎ | 15491/18627 [8:45:02<1:45:57,  2.03s/it]
 83%|████████▎ | 15492/18627 [8:45:03<1:35:42,  1.83s/it]
 83%|████████▎ | 15493/18627 [8:45:06<1:44:27,  2.00s/it]
 83%|████████▎ | 15494/18627 [8:45:08<1:50:19,  2.11s/it]
 83%|████████▎ | 15495/18627 [8:45:11<1:54:10,  2.19s/it]
 83%|████████▎ | 15496/18627 [8:45:13<1:55:57,  2.22s/it]
 83%|████████▎ | 15497/18627 [8:45:15<1:58:28,  2.27s/it]
 83%|████████▎ | 15498/18627 [8:45:18<1:59:07,  2.28s/it]
 83%|████████▎ | 15499/18627 [8:45:20<2:00:30,  2.31s/it]
 83%|████████▎ | 15500/18627 [8:45:21<1:45:08,  2.02s/it]
                                                         
{'loss': 0.9741, 'grad_norm': 15.713860511779785, 'learning_rate': 3.605122228345967e-07, 'epoch': 0.83}
+
 83%|████████▎ | 15500/18627 [8:45:21<1:45:08,  2.02s/it]
 83%|████████▎ | 15501/18627 [8:45:24<1:49:59,  2.11s/it]
 83%|████████▎ | 15502/18627 [8:45:26<1:53:33,  2.18s/it]
 83%|████████▎ | 15503/18627 [8:45:28<1:56:11,  2.23s/it]
 83%|████████▎ | 15504/18627 [8:45:29<1:37:44,  1.88s/it]
 83%|████████▎ | 15505/18627 [8:45:31<1:29:56,  1.73s/it]
 83%|████████▎ | 15506/18627 [8:45:33<1:40:10,  1.93s/it]
 83%|████████▎ | 15507/18627 [8:45:35<1:46:38,  2.05s/it]
 83%|████████▎ | 15508/18627 [8:45:38<1:50:48,  2.13s/it]
 83%|████████▎ | 15509/18627 [8:45:40<1:54:15,  2.20s/it]
 83%|████████▎ | 15510/18627 [8:45:42<1:42:19,  1.97s/it]
                                                         
{'loss': 1.0934, 'grad_norm': 14.099813461303711, 'learning_rate': 3.5826674110097853e-07, 'epoch': 0.83}
+
 83%|████████▎ | 15510/18627 [8:45:42<1:42:19,  1.97s/it]
 83%|████████▎ | 15511/18627 [8:45:44<1:48:13,  2.08s/it]
 83%|████████▎ | 15512/18627 [8:45:45<1:32:26,  1.78s/it]
 83%|████████▎ | 15513/18627 [8:45:46<1:26:01,  1.66s/it]
 83%|████████▎ | 15514/18627 [8:45:48<1:21:14,  1.57s/it]
 83%|████████▎ | 15515/18627 [8:45:49<1:18:19,  1.51s/it]
 83%|████████▎ | 15516/18627 [8:45:51<1:31:23,  1.76s/it]
 83%|████████▎ | 15517/18627 [8:45:54<1:41:02,  1.95s/it]
 83%|████████▎ | 15518/18627 [8:45:56<1:46:50,  2.06s/it]
 83%|████████▎ | 15519/18627 [8:45:57<1:35:34,  1.85s/it]
 83%|████████▎ | 15520/18627 [8:45:59<1:27:47,  1.70s/it]
                                                         
{'loss': 1.5931, 'grad_norm': 12.254583358764648, 'learning_rate': 3.5602773444310457e-07, 'epoch': 0.83}
+
 83%|████████▎ | 15520/18627 [8:45:59<1:27:47,  1.70s/it]
 83%|████████▎ | 15521/18627 [8:46:01<1:37:48,  1.89s/it]
 83%|████████▎ | 15522/18627 [8:46:03<1:44:18,  2.02s/it]
 83%|████████▎ | 15523/18627 [8:46:06<1:49:01,  2.11s/it]
 83%|████████▎ | 15524/18627 [8:46:08<1:52:29,  2.18s/it]
 83%|████████▎ | 15525/18627 [8:46:09<1:40:04,  1.94s/it]
 83%|████████▎ | 15526/18627 [8:46:12<1:45:34,  2.04s/it]
 83%|████████▎ | 15527/18627 [8:46:13<1:35:10,  1.84s/it]
 83%|████████▎ | 15528/18627 [8:46:15<1:42:37,  1.99s/it]
 83%|████████▎ | 15529/18627 [8:46:18<1:48:02,  2.09s/it]
 83%|████████▎ | 15530/18627 [8:46:19<1:36:39,  1.87s/it]
                                                         
{'loss': 1.2173, 'grad_norm': 16.35249137878418, 'learning_rate': 3.537952096301356e-07, 'epoch': 0.83}
+
 83%|████████▎ | 15530/18627 [8:46:19<1:36:39,  1.87s/it]
 83%|████████▎ | 15531/18627 [8:46:21<1:29:58,  1.74s/it]
 83%|████████▎ | 15532/18627 [8:46:23<1:38:35,  1.91s/it]
 83%|████████▎ | 15533/18627 [8:46:25<1:44:29,  2.03s/it]
 83%|████████▎ | 15534/18627 [8:46:28<1:49:21,  2.12s/it]
 83%|████████▎ | 15535/18627 [8:46:29<1:37:29,  1.89s/it]
 83%|████████▎ | 15536/18627 [8:46:30<1:29:55,  1.75s/it]
 83%|████████▎ | 15537/18627 [8:46:33<1:38:35,  1.91s/it]
 83%|████████▎ | 15538/18627 [8:46:35<1:45:55,  2.06s/it]
 83%|████████▎ | 15539/18627 [8:46:37<1:39:34,  1.93s/it]
 83%|████████▎ | 15540/18627 [8:46:39<1:50:41,  2.15s/it]
                                                         
{'loss': 1.3068, 'grad_norm': 8.78498649597168, 'learning_rate': 3.5156917341163666e-07, 'epoch': 0.83}
+
 83%|████████▎ | 15540/18627 [8:46:39<1:50:41,  2.15s/it]
 83%|████████▎ | 15541/18627 [8:46:40<1:33:53,  1.83s/it]
 83%|████████▎ | 15542/18627 [8:46:43<1:41:42,  1.98s/it]
 83%|████████▎ | 15543/18627 [8:46:45<1:47:25,  2.09s/it]
 83%|████████▎ | 15544/18627 [8:46:46<1:36:36,  1.88s/it]
 83%|████████▎ | 15545/18627 [8:46:49<1:44:23,  2.03s/it]
 83%|████████▎ | 15546/18627 [8:46:50<1:35:02,  1.85s/it]
 83%|████████▎ | 15547/18627 [8:46:52<1:27:34,  1.71s/it]
 83%|████████▎ | 15548/18627 [8:46:54<1:37:31,  1.90s/it]
 83%|████████▎ | 15549/18627 [8:46:55<1:26:52,  1.69s/it]
 83%|████████▎ | 15550/18627 [8:46:58<1:38:05,  1.91s/it]
                                                         
{'loss': 1.4443, 'grad_norm': 6.873340129852295, 'learning_rate': 3.493496325175558e-07, 'epoch': 0.83}
+
 83%|████████▎ | 15550/18627 [8:46:58<1:38:05,  1.91s/it]
 83%|████████▎ | 15551/18627 [8:47:00<1:45:40,  2.06s/it]
 83%|████████▎ | 15552/18627 [8:47:01<1:35:36,  1.87s/it]
 83%|████████▎ | 15553/18627 [8:47:04<1:42:51,  2.01s/it]
 84%|████████▎ | 15554/18627 [8:47:05<1:32:46,  1.81s/it]
 84%|████████▎ | 15555/18627 [8:47:07<1:40:44,  1.97s/it]
 84%|████████▎ | 15556/18627 [8:47:10<1:46:10,  2.07s/it]
 84%|████████▎ | 15557/18627 [8:47:11<1:35:20,  1.86s/it]
 84%|████████▎ | 15558/18627 [8:47:14<1:42:37,  2.01s/it]
 84%|████████▎ | 15559/18627 [8:47:16<1:47:42,  2.11s/it]
 84%|████████▎ | 15560/18627 [8:47:18<1:50:55,  2.17s/it]
                                                         
{'loss': 1.092, 'grad_norm': 5.45495080947876, 'learning_rate': 3.4713659365820384e-07, 'epoch': 0.84}
+
 84%|████████▎ | 15560/18627 [8:47:18<1:50:55,  2.17s/it]
 84%|████████▎ | 15561/18627 [8:47:19<1:33:51,  1.84s/it]
 84%|████████▎ | 15562/18627 [8:47:21<1:26:23,  1.69s/it]
 84%|████████▎ | 15563/18627 [8:47:22<1:22:09,  1.61s/it]
 84%|████████▎ | 15564/18627 [8:47:24<1:33:21,  1.83s/it]
 84%|█��██████▎ | 15565/18627 [8:47:27<1:40:29,  1.97s/it]
 84%|████████▎ | 15566/18627 [8:47:29<1:45:31,  2.07s/it]
 84%|████████▎ | 15567/18627 [8:47:31<1:49:08,  2.14s/it]
 84%|████████▎ | 15568/18627 [8:47:34<1:52:26,  2.21s/it]
 84%|████████▎ | 15569/18627 [8:47:36<1:55:11,  2.26s/it]
 84%|████████▎ | 15570/18627 [8:47:38<1:56:27,  2.29s/it]
                                                         
{'loss': 0.9729, 'grad_norm': 6.163602828979492, 'learning_rate': 3.4493006352423226e-07, 'epoch': 0.84}
+
 84%|████████▎ | 15570/18627 [8:47:38<1:56:27,  2.29s/it]
 84%|████████▎ | 15571/18627 [8:47:41<1:57:03,  2.30s/it]
 84%|████████▎ | 15572/18627 [8:47:43<1:58:33,  2.33s/it]
 84%|████████▎ | 15573/18627 [8:47:45<1:59:19,  2.34s/it]
 84%|████████▎ | 15574/18627 [8:47:48<1:59:33,  2.35s/it]
 84%|████████▎ | 15575/18627 [8:47:50<1:59:37,  2.35s/it]
 84%|████████▎ | 15576/18627 [8:47:52<1:59:17,  2.35s/it]
 84%|████████▎ | 15577/18627 [8:47:55<2:00:08,  2.36s/it]
 84%|████████▎ | 15578/18627 [8:47:57<1:59:55,  2.36s/it]
 84%|████████▎ | 15579/18627 [8:48:00<1:59:30,  2.35s/it]
 84%|████████▎ | 15580/18627 [8:48:02<1:59:59,  2.36s/it]
                                                         
{'loss': 0.596, 'grad_norm': 5.356025695800781, 'learning_rate': 3.4273004878661865e-07, 'epoch': 0.84}
+
 84%|████████▎ | 15580/18627 [8:48:02<1:59:59,  2.36s/it]
 84%|████████▎ | 15581/18627 [8:48:04<1:59:35,  2.36s/it]
 84%|████████▎ | 15582/18627 [8:48:07<2:00:01,  2.37s/it]
 84%|████████▎ | 15583/18627 [8:48:08<1:45:05,  2.07s/it]
 84%|████████▎ | 15584/18627 [8:48:10<1:48:52,  2.15s/it]
 84%|████████▎ | 15585/18627 [8:48:13<1:51:40,  2.20s/it]
 84%|████████▎ | 15586/18627 [8:48:15<1:54:34,  2.26s/it]
 84%|████████▎ | 15587/18627 [8:48:17<1:55:46,  2.29s/it]
 84%|████████▎ | 15588/18627 [8:48:19<1:36:49,  1.91s/it]
 84%|████████▎ | 15589/18627 [8:48:21<1:42:57,  2.03s/it]
 84%|████████▎ | 15590/18627 [8:48:23<1:47:03,  2.12s/it]
                                                         
{'loss': 0.803, 'grad_norm': 4.77785062789917, 'learning_rate': 3.405365560966387e-07, 'epoch': 0.84}
+
 84%|████████▎ | 15590/18627 [8:48:23<1:47:03,  2.12s/it]
 84%|████████▎ | 15591/18627 [8:48:24<1:34:35,  1.87s/it]
 84%|████████▎ | 15592/18627 [8:48:27<1:42:01,  2.02s/it]
 84%|████████▎ | 15593/18627 [8:48:29<1:46:49,  2.11s/it]
 84%|████████▎ | 15594/18627 [8:48:30<1:34:42,  1.87s/it]
 84%|████████▎ | 15595/18627 [8:48:33<1:40:32,  1.99s/it]
 84%|████████▎ | 15596/18627 [8:48:35<1:45:53,  2.10s/it]
 84%|████████▎ | 15597/18627 [8:48:37<1:49:17,  2.16s/it]
 84%|████████▎ | 15598/18627 [8:48:40<1:51:51,  2.22s/it]
 84%|████████▎ | 15599/18627 [8:48:41<1:38:02,  1.94s/it]
 84%|████████▎ | 15600/18627 [8:48:42<1:29:52,  1.78s/it]
                                                         
{'loss': 1.3778, 'grad_norm': 14.395994186401367, 'learning_rate': 3.383495920858526e-07, 'epoch': 0.84}
+
 84%|████████▎ | 15600/18627 [8:48:42<1:29:52,  1.78s/it]
 84%|████████▍ | 15601/18627 [8:48:45<1:38:40,  1.96s/it]
 84%|████████▍ | 15602/18627 [8:48:47<1:44:08,  2.07s/it]
 84%|████████▍ | 15603/18627 [8:48:49<1:47:49,  2.14s/it]
 84%|████████▍ | 15604/18627 [8:48:51<1:36:28,  1.91s/it]
 84%|████████▍ | 15605/18627 [8:48:53<1:42:34,  2.04s/it]
 84%|████████▍ | 15606/18627 [8:48:56<1:47:48,  2.14s/it]
 84%|████████▍ | 15607/18627 [8:48:57<1:31:16,  1.81s/it]
 84%|████████▍ | 15608/18627 [8:48:59<1:39:05,  1.97s/it]
 84%|████████▍ | 15609/18627 [8:49:00<1:30:08,  1.79s/it]
 84%|████████▍ | 15610/18627 [8:49:03<1:38:38,  1.96s/it]
                                                         
{'loss': 0.9509, 'grad_norm': 5.17941427230835, 'learning_rate': 3.361691633660813e-07, 'epoch': 0.84}
+
 84%|████████▍ | 15610/18627 [8:49:03<1:38:38,  1.96s/it]
 84%|████████▍ | 15611/18627 [8:49:05<1:44:20,  2.08s/it]
 84%|████████▍ | 15612/18627 [8:49:07<1:47:54,  2.15s/it]
 84%|████████▍ | 15613/18627 [8:49:10<1:50:40,  2.20s/it]
 84%|████████▍ | 15614/18627 [8:49:12<1:53:15,  2.26s/it]
 84%|████████▍ | 15615/18627 [8:49:14<1:54:39,  2.28s/it]
 84%|████████▍ | 15616/18627 [8:49:17<1:55:50,  2.31s/it]
 84%|████████▍ | 15617/18627 [8:49:19<1:56:02,  2.31s/it]
 84%|████████▍ | 15618/18627 [8:49:20<1:41:48,  2.03s/it]
 84%|████████▍ | 15619/18627 [8:49:23<1:46:14,  2.12s/it]
 84%|████████▍ | 15620/18627 [8:49:25<1:49:35,  2.19s/it]
                                                         
{'loss': 0.7782, 'grad_norm': 12.606101036071777, 'learning_rate': 3.339952765293883e-07, 'epoch': 0.84}
+
 84%|████████▍ | 15620/18627 [8:49:25<1:49:35,  2.19s/it]
 84%|████████▍ | 15621/18627 [8:49:27<1:52:32,  2.25s/it]
 84%|████████▍ | 15622/18627 [8:49:30<1:53:43,  2.27s/it]
 84%|████████▍ | 15623/18627 [8:49:32<1:55:18,  2.30s/it]
 84%|████████▍ | 15624/18627 [8:49:35<1:58:39,  2.37s/it]
 84%|████████▍ | 15625/18627 [8:49:36<1:45:55,  2.12s/it]
 84%|████████▍ | 15626/18627 [8:49:39<1:48:42,  2.17s/it]
 84%|████████▍ | 15627/18627 [8:49:40<1:37:56,  1.96s/it]
 84%|████████▍ | 15628/18627 [8:49:42<1:43:13,  2.07s/it]
 84%|████████▍ | 15629/18627 [8:49:45<1:47:35,  2.15s/it]
 84%|████████▍ | 15630/18627 [8:49:47<1:50:37,  2.21s/it]
                                                         
{'loss': 1.035, 'grad_norm': 11.191023826599121, 'learning_rate': 3.318279381480591e-07, 'epoch': 0.84}
+
 84%|████████▍ | 15630/18627 [8:49:47<1:50:37,  2.21s/it]
 84%|████████▍ | 15631/18627 [8:49:49<1:52:39,  2.26s/it]
 84%|████████▍ | 15632/18627 [8:49:51<1:39:16,  1.99s/it]
 84%|████████▍ | 15633/18627 [8:49:52<1:30:06,  1.81s/it]
 84%|████████▍ | 15634/18627 [8:49:54<1:38:08,  1.97s/it]
 84%|████████▍ | 15635/18627 [8:49:57<1:43:25,  2.07s/it]
 84%|████████▍ | 15636/18627 [8:49:59<1:47:27,  2.16s/it]
 84%|████████▍ | 15637/18627 [8:50:01<1:50:16,  2.21s/it]
 84%|████████▍ | 15638/18627 [8:50:04<1:51:45,  2.24s/it]
 84%|████████▍ | 15639/18627 [8:50:05<1:34:02,  1.89s/it]
 84%|████████▍ | 15640/18627 [8:50:06<1:25:25,  1.72s/it]
                                                         
{'loss': 1.1408, 'grad_norm': 14.106231689453125, 'learning_rate': 3.296671547745811e-07, 'epoch': 0.84}
+
 84%|████████▍ | 15640/18627 [8:50:06<1:25:25,  1.72s/it]
 84%|████████▍ | 15641/18627 [8:50:09<1:34:54,  1.91s/it]
 84%|████████▍ | 15642/18627 [8:50:10<1:26:36,  1.74s/it]
 84%|████████▍ | 15643/18627 [8:50:12<1:35:10,  1.91s/it]
 84%|████████▍ | 15644/18627 [8:50:15<1:42:47,  2.07s/it]
 84%|████████▍ | 15645/18627 [8:50:17<1:47:13,  2.16s/it]
 84%|████████▍ | 15646/18627 [8:50:19<1:49:51,  2.21s/it]
 84%|████████▍ | 15647/18627 [8:50:22<1:51:55,  2.25s/it]
 84%|████████▍ | 15648/18627 [8:50:23<1:38:47,  1.99s/it]
 84%|████████▍ | 15649/18627 [8:50:25<1:44:21,  2.10s/it]
 84%|████████▍ | 15650/18627 [8:50:28<1:47:25,  2.17s/it]
                                                         
{'loss': 0.9647, 'grad_norm': 8.368142127990723, 'learning_rate': 3.275129329416227e-07, 'epoch': 0.84}
+
 84%|████████▍ | 15650/18627 [8:50:28<1:47:25,  2.17s/it]
 84%|████████▍ | 15651/18627 [8:50:29<1:35:26,  1.92s/it]
 84%|████████▍ | 15652/18627 [8:50:31<1:41:08,  2.04s/it]
 84%|████████▍ | 15653/18627 [8:50:34<1:45:08,  2.12s/it]
 84%|████████▍ | 15654/18627 [8:50:36<1:48:01,  2.18s/it]
 84%|████████▍ | 15655/18627 [8:50:37<1:36:17,  1.94s/it]
 84%|████████▍ | 15656/18627 [8:50:39<1:28:37,  1.79s/it]
 84%|████████▍ | 15657/18627 [8:50:40<1:18:20,  1.58s/it]
 84%|████████▍ | 15658/18627 [8:50:42<1:30:36,  1.83s/it]
 84%|████████▍ | 15659/18627 [8:50:45<1:38:09,  1.98s/it]
 84%|████████▍ | 15660/18627 [8:50:47<1:43:49,  2.10s/it]
                                                         
{'loss': 1.0934, 'grad_norm': 6.511770248413086, 'learning_rate': 3.253652791620182e-07, 'epoch': 0.84}
+
 84%|████████▍ | 15660/18627 [8:50:47<1:43:49,  2.10s/it]
 84%|████████▍ | 15661/18627 [8:50:49<1:46:52,  2.16s/it]
 84%|████████▍ | 15662/18627 [8:50:52<1:49:05,  2.21s/it]
 84%|████████▍ | 15663/18627 [8:50:54<1:51:42,  2.26s/it]
 84%|████████▍ | 15664/18627 [8:50:55<1:38:20,  1.99s/it]
 84%|████████▍ | 15665/18627 [8:50:58<1:42:43,  2.08s/it]
 84%|████████▍ | 15666/18627 [8:51:00<1:46:11,  2.15s/it]
 84%|████████▍ | 15667/18627 [8:51:01<1:34:27,  1.91s/it]
 84%|█████���██▍ | 15668/18627 [8:51:03<1:26:18,  1.75s/it]
 84%|████████▍ | 15669/18627 [8:51:04<1:20:39,  1.64s/it]
 84%|████████▍ | 15670/18627 [8:51:06<1:31:16,  1.85s/it]
                                                         
{'loss': 1.5033, 'grad_norm': 6.934089660644531, 'learning_rate': 3.232241999287414e-07, 'epoch': 0.84}
+
 84%|████████▍ | 15670/18627 [8:51:07<1:31:16,  1.85s/it]
 84%|████████▍ | 15671/18627 [8:51:08<1:23:53,  1.70s/it]
 84%|████████▍ | 15672/18627 [8:51:09<1:14:10,  1.51s/it]
 84%|████████▍ | 15673/18627 [8:51:11<1:26:31,  1.76s/it]
 84%|████████▍ | 15674/18627 [8:51:12<1:18:58,  1.60s/it]
 84%|████████▍ | 15675/18627 [8:51:15<1:29:47,  1.83s/it]
 84%|████████▍ | 15676/18627 [8:51:17<1:37:06,  1.97s/it]
 84%|████████▍ | 15677/18627 [8:51:20<1:44:29,  2.13s/it]
 84%|████████▍ | 15678/18627 [8:51:22<1:47:50,  2.19s/it]
 84%|████████▍ | 15679/18627 [8:51:24<1:49:53,  2.24s/it]
 84%|████████▍ | 15680/18627 [8:51:27<1:51:38,  2.27s/it]
                                                         
{'loss': 0.9953, 'grad_norm': 5.094871997833252, 'learning_rate': 3.210897017148909e-07, 'epoch': 0.84}
+
 84%|████████▍ | 15680/18627 [8:51:27<1:51:38,  2.27s/it]
 84%|████████▍ | 15681/18627 [8:51:28<1:38:22,  2.00s/it]
 84%|████████▍ | 15682/18627 [8:51:29<1:29:30,  1.82s/it]
 84%|████████▍ | 15683/18627 [8:51:32<1:37:25,  1.99s/it]
 84%|████████▍ | 15684/18627 [8:51:34<1:42:40,  2.09s/it]
 84%|████████▍ | 15685/18627 [8:51:36<1:32:32,  1.89s/it]
 84%|████████▍ | 15686/18627 [8:51:38<1:38:30,  2.01s/it]
 84%|████████▍ | 15687/18627 [8:51:40<1:43:30,  2.11s/it]
 84%|████████▍ | 15688/18627 [8:51:42<1:45:53,  2.16s/it]
 84%|████████▍ | 15689/18627 [8:51:45<1:49:13,  2.23s/it]
 84%|████████▍ | 15690/18627 [8:51:47<1:50:30,  2.26s/it]
                                                         
{'loss': 1.2537, 'grad_norm': 6.364528179168701, 'learning_rate': 3.189617909736695e-07, 'epoch': 0.84}
+
 84%|████████▍ | 15690/18627 [8:51:47<1:50:30,  2.26s/it]
 84%|████████▍ | 15691/18627 [8:51:49<1:38:39,  2.02s/it]
 84%|████████▍ | 15692/18627 [8:51:50<1:24:59,  1.74s/it]
 84%|████████▍ | 15693/18627 [8:51:52<1:34:03,  1.92s/it]
 84%|████████▍ | 15694/18627 [8:51:54<1:39:54,  2.04s/it]
 84%|████████▍ | 15695/18627 [8:51:57<1:43:52,  2.13s/it]
 84%|████████▍ | 15696/18627 [8:51:58<1:31:43,  1.88s/it]
 84%|████████▍ | 15697/18627 [8:52:00<1:38:33,  2.02s/it]
 84%|████████▍ | 15698/18627 [8:52:03<1:42:50,  2.11s/it]
 84%|████████▍ | 15699/18627 [8:52:05<1:45:45,  2.17s/it]
 84%|████████▍ | 15700/18627 [8:52:06<1:34:02,  1.93s/it]
                                                         
{'loss': 1.1445, 'grad_norm': 13.439284324645996, 'learning_rate': 3.1684047413836267e-07, 'epoch': 0.84}
+
 84%|████████▍ | 15700/18627 [8:52:06<1:34:02,  1.93s/it]
 84%|████████▍ | 15701/18627 [8:52:09<1:39:41,  2.04s/it]
 84%|████████▍ | 15702/18627 [8:52:11<1:44:03,  2.13s/it]
 84%|████████▍ | 15703/18627 [8:52:13<1:46:25,  2.18s/it]
 84%|████████▍ | 15704/18627 [8:52:16<1:48:29,  2.23s/it]
 84%|████████▍ | 15705/18627 [8:52:18<1:50:42,  2.27s/it]
 84%|████████▍ | 15706/18627 [8:52:20<1:52:39,  2.31s/it]
 84%|████████▍ | 15707/18627 [8:52:22<1:39:06,  2.04s/it]
 84%|████████▍ | 15708/18627 [8:52:23<1:28:21,  1.82s/it]
 84%|████████▍ | 15709/18627 [8:52:25<1:35:57,  1.97s/it]
 84%|████████▍ | 15710/18627 [8:52:28<1:40:50,  2.07s/it]
                                                         
{'loss': 0.9951, 'grad_norm': 5.618056774139404, 'learning_rate': 3.14725757622322e-07, 'epoch': 0.84}
+
 84%|████████▍ | 15710/18627 [8:52:28<1:40:50,  2.07s/it]
 84%|████████▍ | 15711/18627 [8:52:30<1:44:46,  2.16s/it]
 84%|████████▍ | 15712/18627 [8:52:32<1:47:12,  2.21s/it]
 84%|████████▍ | 15713/18627 [8:52:35<1:49:22,  2.25s/it]
 84%|████████▍ | 15714/18627 [8:52:36<1:36:47,  1.99s/it]
 84%|████████▍ | 15715/18627 [8:52:39<1:41:43,  2.10s/it]
 84%|████████▍ | 15716/18627 [8:52:40<1:31:01,  1.88s/it]
 84%|████████▍ | 15717/18627 [8:52:42<1:37:35,  2.01s/it]
 84%|████████▍ | 15718/18627 [8:52:45<1:43:48,  2.14s/it]
 84%|████████▍ | 15719/18627 [8:52:47<1:46:32,  2.20s/it]
 84%|████████▍ | 15720/18627 [8:52:49<1:48:10,  2.23s/it]
                                                         
{'loss': 0.9883, 'grad_norm': 6.045928001403809, 'learning_rate': 3.126176478189435e-07, 'epoch': 0.84}
+
 84%|████████▍ | 15720/18627 [8:52:49<1:48:10,  2.23s/it]
 84%|████████▍ | 15721/18627 [8:52:52<1:49:43,  2.27s/it]
 84%|████████▍ | 15722/18627 [8:52:54<1:50:23,  2.28s/it]
 84%|████████▍ | 15723/18627 [8:52:55<1:36:21,  1.99s/it]
 84%|████████▍ | 15724/18627 [8:52:58<1:41:45,  2.10s/it]
 84%|████████▍ | 15725/18627 [8:53:00<1:44:57,  2.17s/it]
 84%|████████▍ | 15726/18627 [8:53:02<1:47:08,  2.22s/it]
 84%|████████▍ | 15727/18627 [8:53:05<1:49:56,  2.27s/it]
 84%|████████▍ | 15728/18627 [8:53:07<1:50:43,  2.29s/it]
 84%|████████▍ | 15729/18627 [8:53:09<1:51:46,  2.31s/it]
 84%|████████▍ | 15730/18627 [8:53:12<1:52:37,  2.33s/it]
                                                         
{'loss': 0.8264, 'grad_norm': 5.2609543800354, 'learning_rate': 3.1051615110164995e-07, 'epoch': 0.84}
+
 84%|████████▍ | 15730/18627 [8:53:12<1:52:37,  2.33s/it]
 84%|████████▍ | 15731/18627 [8:53:13<1:39:29,  2.06s/it]
 84%|████████▍ | 15732/18627 [8:53:15<1:28:59,  1.84s/it]
 84%|████████▍ | 15733/18627 [8:53:17<1:36:36,  2.00s/it]
 84%|████████▍ | 15734/18627 [8:53:19<1:41:16,  2.10s/it]
 84%|████████▍ | 15735/18627 [8:53:22<1:44:08,  2.16s/it]
 84%|████████▍ | 15736/18627 [8:53:23<1:27:54,  1.82s/it]
 84%|████████▍ | 15737/18627 [8:53:25<1:35:13,  1.98s/it]
 84%|████████▍ | 15738/18627 [8:53:27<1:40:19,  2.08s/it]
 84%|████████▍ | 15739/18627 [8:53:29<1:30:20,  1.88s/it]
 85%|████████▍ | 15740/18627 [8:53:31<1:36:47,  2.01s/it]
                                                         
{'loss': 1.0839, 'grad_norm': 6.589994430541992, 'learning_rate': 3.084212738238701e-07, 'epoch': 0.85}
+
 85%|████████▍ | 15740/18627 [8:53:31<1:36:47,  2.01s/it]
 85%|████████▍ | 15741/18627 [8:53:33<1:41:22,  2.11s/it]
 85%|████████▍ | 15742/18627 [8:53:36<1:45:00,  2.18s/it]
 85%|████████▍ | 15743/18627 [8:53:38<1:46:46,  2.22s/it]
 85%|████████▍ | 15744/18627 [8:53:40<1:48:06,  2.25s/it]
 85%|████████▍ | 15745/18627 [8:53:43<1:49:10,  2.27s/it]
 85%|████████▍ | 15746/18627 [8:53:45<1:50:16,  2.30s/it]
 85%|████████▍ | 15747/18627 [8:53:47<1:50:53,  2.31s/it]
 85%|████████▍ | 15748/18627 [8:53:50<1:51:22,  2.32s/it]
 85%|████████▍ | 15749/18627 [8:53:52<1:51:10,  2.32s/it]
 85%|████████▍ | 15750/18627 [8:53:54<1:51:22,  2.32s/it]
                                                         
{'loss': 0.6038, 'grad_norm': 4.5332794189453125, 'learning_rate': 3.063330223190211e-07, 'epoch': 0.85}
+
 85%|████████▍ | 15750/18627 [8:53:54<1:51:22,  2.32s/it]
 85%|████████▍ | 15751/18627 [8:53:57<1:51:42,  2.33s/it]
 85%|████████▍ | 15752/18627 [8:53:59<1:51:40,  2.33s/it]
 85%|████████▍ | 15753/18627 [8:54:03<2:10:52,  2.73s/it]
 85%|████████▍ | 15754/18627 [8:54:05<2:05:05,  2.61s/it]
 85%|████████▍ | 15755/18627 [8:54:06<1:47:09,  2.24s/it]
 85%|████████▍ | 15756/18627 [8:54:09<1:48:35,  2.27s/it]
 85%|████████▍ | 15757/18627 [8:54:11<1:49:30,  2.29s/it]
 85%|████████▍ | 15758/18627 [8:54:13<1:49:58,  2.30s/it]
 85%|████████▍ | 15759/18627 [8:54:16<1:50:07,  2.30s/it]
 85%|████████▍ | 15760/18627 [8:54:17<1:36:31,  2.02s/it]
                                                         
{'loss': 1.1868, 'grad_norm': 13.612586975097656, 'learning_rate': 3.042514029004867e-07, 'epoch': 0.85}
+
 85%|████████▍ | 15760/18627 [8:54:17<1:36:31,  2.02s/it]
 85%|████████▍ | 15761/18627 [8:54:19<1:41:12,  2.12s/it]
 85%|████████▍ | 15762/18627 [8:54:22<1:44:10,  2.18s/it]
 85%|████████▍ | 15763/18627 [8:54:24<1:46:21,  2.23s/it]
 85%|████████▍ | 15764/18627 [8:54:26<1:48:06,  2.27s/it]
 85%|████████▍ | 15765/18627 [8:54:29<1:48:47,  2.28s/it]
 85%|████████▍ | 15766/18627 [8:54:30<1:35:42,  2.01s/it]
 85%|████████▍ | 15767/18627 [8:54:31<1:25:56,  1.80s/it]
 85%|████████▍ | 15768/18627 [8:54:34<1:34:04,  1.97s/it]
 85%|████████▍ | 15769/18627 [8:54:36<1:39:03,  2.08s/it]
 85%|████████▍ | 15770/18627 [8:54:38<1:42:32,  2.15s/it]
                                                         
{'loss': 0.9685, 'grad_norm': 5.75744104385376, 'learning_rate': 3.02176421861603e-07, 'epoch': 0.85}
+
 85%|████████▍ | 15770/18627 [8:54:38<1:42:32,  2.15s/it]
 85%|████████▍ | 15771/18627 [8:54:41<1:45:30,  2.22s/it]
 85%|████████▍ | 15772/18627 [8:54:43<1:46:55,  2.25s/it]
 85%|████████▍ | 15773/18627 [8:54:45<1:47:47,  2.27s/it]
 85%|████████▍ | 15774/18627 [8:54:48<1:48:36,  2.28s/it]
 85%|████████▍ | 15775/18627 [8:54:49<1:31:38,  1.93s/it]
 85%|████████▍ | 15776/18627 [8:54:51<1:37:49,  2.06s/it]
 85%|████████▍ | 15777/18627 [8:54:54<1:41:45,  2.14s/it]
 85%|████████▍ | 15778/18627 [8:54:55<1:31:12,  1.92s/it]
 85%|████████▍ | 15779/18627 [8:54:57<1:36:59,  2.04s/it]
 85%|████████▍ | 15780/18627 [8:55:00<1:41:25,  2.14s/it]
                                                         
{'loss': 0.8696, 'grad_norm': 6.578497409820557, 'learning_rate': 3.001080854756333e-07, 'epoch': 0.85}
+
 85%|████████▍ | 15780/18627 [8:55:00<1:41:25,  2.14s/it]
 85%|████████▍ | 15781/18627 [8:55:02<1:44:37,  2.21s/it]
 85%|████████▍ | 15782/18627 [8:55:04<1:46:44,  2.25s/it]
 85%|████████▍ | 15783/18627 [8:55:07<1:47:54,  2.28s/it]
 85%|████████▍ | 15784/18627 [8:55:09<1:48:24,  2.29s/it]
 85%|████████▍ | 15785/18627 [8:55:11<1:49:10,  2.30s/it]
 85%|████████▍ | 15786/18627 [8:55:13<1:36:05,  2.03s/it]
 85%|████████▍ | 15787/18627 [8:55:15<1:40:19,  2.12s/it]
 85%|████████▍ | 15788/18627 [8:55:17<1:43:18,  2.18s/it]
 85%|████████▍ | 15789/18627 [8:55:19<1:31:41,  1.94s/it]
 85%|████████▍ | 15790/18627 [8:55:20<1:23:23,  1.76s/it]
                                                         
{'loss': 1.2128, 'grad_norm': 14.110395431518555, 'learning_rate': 2.9804639999575363e-07, 'epoch': 0.85}
+
 85%|████████▍ | 15790/18627 [8:55:20<1:23:23,  1.76s/it]
 85%|████████▍ | 15791/18627 [8:55:22<1:31:21,  1.93s/it]
 85%|████████▍ | 15792/18627 [8:55:25<1:36:27,  2.04s/it]
 85%|████████▍ | 15793/18627 [8:55:27<1:40:34,  2.13s/it]
 85%|████████▍ | 15794/18627 [8:55:29<1:30:25,  1.92s/it]
 85%|████████▍ | 15795/18627 [8:55:31<1:37:12,  2.06s/it]
 85%|████████▍ | 15796/18627 [8:55:33<1:40:54,  2.14s/it]
 85%|████████▍ | 15797/18627 [8:55:36<1:43:27,  2.19s/it]
 85%|████████▍ | 15798/18627 [8:55:38<1:45:23,  2.24s/it]
 85%|████████▍ | 15799/18627 [8:55:40<1:46:42,  2.26s/it]
 85%|████████▍ | 15800/18627 [8:55:41<1:32:35,  1.97s/it]
                                                         
{'loss': 0.9753, 'grad_norm': 13.504613876342773, 'learning_rate': 2.959913716550328e-07, 'epoch': 0.85}
+
 85%|████████▍ | 15800/18627 [8:55:42<1:32:35,  1.97s/it]
 85%|████████▍ | 15801/18627 [8:55:43<1:23:59,  1.78s/it]
 85%|████████▍ | 15802/18627 [8:55:45<1:31:58,  1.95s/it]
 85%|████████▍ | 15803/18627 [8:55:48<1:37:32,  2.07s/it]
 85%|████████▍ | 15804/18627 [8:55:49<1:27:43,  1.86s/it]
 85%|████████▍ | 15805/18627 [8:55:51<1:34:10,  2.00s/it]
 85%|████████▍ | 15806/18627 [8:55:54<1:38:42,  2.10s/it]
 85%|████████▍ | 15807/18627 [8:55:56<1:41:56,  2.17s/it]
 85%|████████▍ | 15808/18627 [8:55:57<1:30:40,  1.93s/it]
 85%|████████▍ | 15809/18627 [8:56:00<1:36:43,  2.06s/it]
 85%|████████▍ | 15810/18627 [8:56:01<1:27:11,  1.86s/it]
                                                         
{'loss': 1.4672, 'grad_norm': 14.940838813781738, 'learning_rate': 2.93943006666412e-07, 'epoch': 0.85}
+
 85%|████████▍ | 15810/18627 [8:56:01<1:27:11,  1.86s/it]
 85%|████████▍ | 15811/18627 [8:56:03<1:33:51,  2.00s/it]
 85%|████████▍ | 15812/18627 [8:56:06<1:38:36,  2.10s/it]
 85%|████████▍ | 15813/18627 [8:56:08<1:42:00,  2.17s/it]
 85%|████████▍ | 15814/18627 [8:56:10<1:44:10,  2.22s/it]
 85%|████████▍ | 15815/18627 [8:56:13<1:45:34,  2.25s/it]
 85%|████████▍ | 15816/18627 [8:56:15<1:46:23,  2.27s/it]
 85%|████████▍ | 15817/18627 [8:56:17<1:47:03,  2.29s/it]
 85%|████████▍ | 15818/18627 [8:56:20<1:48:11,  2.31s/it]
 85%|████████▍ | 15819/18627 [8:56:22<1:49:01,  2.33s/it]
 85%|████████▍ | 15820/18627 [8:56:24<1:49:21,  2.34s/it]
                                                         
{'loss': 0.5676, 'grad_norm': 5.956108093261719, 'learning_rate': 2.9190131122268814e-07, 'epoch': 0.85}
+
 85%|████████▍ | 15820/18627 [8:56:24<1:49:21,  2.34s/it]
 85%|████████▍ | 15821/18627 [8:56:27<1:49:39,  2.34s/it]
 85%|████████▍ | 15822/18627 [8:56:28<1:36:45,  2.07s/it]
 85%|████████▍ | 15823/18627 [8:56:31<1:40:40,  2.15s/it]
 85%|████████▍ | 15824/18627 [8:56:33<1:43:13,  2.21s/it]
 85%|████████▍ | 15825/18627 [8:56:35<1:45:12,  2.25s/it]
 85%|████████▍ | 15826/18627 [8:56:38<1:46:50,  2.29s/it]
 85%|████████▍ | 15827/18627 [8:56:40<1:48:03,  2.32s/it]
 85%|████████▍ | 15828/18627 [8:56:42<1:48:34,  2.33s/it]
 85%|████████▍ | 15829/18627 [8:56:45<1:48:20,  2.32s/it]
 85%|████████▍ | 15830/18627 [8:56:47<1:48:30,  2.33s/it]
                                                         
{'loss': 0.8479, 'grad_norm': 9.678078651428223, 'learning_rate': 2.8986629149649475e-07, 'epoch': 0.85}
+
 85%|████████▍ | 15830/18627 [8:56:47<1:48:30,  2.33s/it]
 85%|████████▍ | 15831/18627 [8:56:49<1:48:41,  2.33s/it]
 85%|████████▍ | 15832/18627 [8:56:52<1:48:30,  2.33s/it]
 85%|████████▌ | 15833/18627 [8:56:54<1:48:29,  2.33s/it]
 85%|████████▌ | 15834/18627 [8:56:56<1:48:24,  2.33s/it]
 85%|████████▌ | 15835/18627 [8:56:59<1:47:59,  2.32s/it]
 85%|████████▌ | 15836/18627 [8:57:01<1:48:30,  2.33s/it]
 85%|████████▌ | 15837/18627 [8:57:03<1:48:16,  2.33s/it]
 85%|████████▌ | 15838/18627 [8:57:06<1:48:07,  2.33s/it]
 85%|████████▌ | 15839/18627 [8:57:08<1:48:35,  2.34s/it]
 85%|████████▌ | 15840/18627 [8:57:10<1:48:27,  2.33s/it]
                                                         
{'loss': 0.554, 'grad_norm': 6.044682502746582, 'learning_rate': 2.878379536402798e-07, 'epoch': 0.85}
+
 85%|████████▌ | 15840/18627 [8:57:10<1:48:27,  2.33s/it]
 85%|████████▌ | 15841/18627 [8:57:13<1:48:26,  2.34s/it]
 85%|████████▌ | 15842/18627 [8:57:14<1:34:32,  2.04s/it]
 85%|████████▌ | 15843/18627 [8:57:15<1:21:36,  1.76s/it]
 85%|████████▌ | 15844/18627 [8:57:17<1:29:14,  1.92s/it]
 85%|████████▌ | 15845/18627 [8:57:19<1:21:29,  1.76s/it]
 85%|████████▌ | 15846/18627 [8:57:21<1:29:47,  1.94s/it]
 85%|████████▌ | 15847/18627 [8:57:23<1:34:58,  2.05s/it]
 85%|████████▌ | 15848/18627 [8:57:26<1:38:41,  2.13s/it]
 85%|████████▌ | 15849/18627 [8:57:28<1:41:02,  2.18s/it]
 85%|████████▌ | 15850/18627 [8:57:30<1:42:46,  2.22s/it]
                                                         
{'loss': 1.0309, 'grad_norm': 4.474028587341309, 'learning_rate': 2.8581630378629405e-07, 'epoch': 0.85}
+
 85%|████████▌ | 15850/18627 [8:57:30<1:42:46,  2.22s/it]
 85%|████████▌ | 15851/18627 [8:57:33<1:43:14,  2.23s/it]
 85%|████████▌ | 15852/18627 [8:57:35<1:44:35,  2.26s/it]
 85%|████████▌ | 15853/18627 [8:57:37<1:45:39,  2.29s/it]
 85%|████████▌ | 15854/18627 [8:57:40<1:46:23,  2.30s/it]
 85%|████████▌ | 15855/18627 [8:57:42<1:46:54,  2.31s/it]
 85%|████████▌ | 15856/18627 [8:57:44<1:47:30,  2.33s/it]
 85%|████████▌ | 15857/18627 [8:57:46<1:34:04,  2.04s/it]
 85%|████████▌ | 15858/18627 [8:57:48<1:40:27,  2.18s/it]
 85%|████████▌ | 15859/18627 [8:57:51<1:42:26,  2.22s/it]
 85%|████████▌ | 15860/18627 [8:57:53<1:43:39,  2.25s/it]
                                                         
{'loss': 0.8343, 'grad_norm': 4.543635845184326, 'learning_rate': 2.838013480465657e-07, 'epoch': 0.85}
+
 85%|████████▌ | 15860/18627 [8:57:53<1:43:39,  2.25s/it]
 85%|████████▌ | 15861/18627 [8:57:55<1:44:47,  2.27s/it]
 85%|████████▌ | 15862/18627 [8:57:58<1:45:56,  2.30s/it]
 85%|████████▌ | 15863/18627 [8:57:59<1:31:29,  1.99s/it]
 85%|████████▌ | 15864/18627 [8:58:00<1:22:54,  1.80s/it]
 85%|████████▌ | 15865/18627 [8:58:03<1:29:58,  1.95s/it]
 85%|████████▌ | 15866/18627 [8:58:05<1:34:52,  2.06s/it]
 85%|████████▌ | 15867/18627 [8:58:06<1:25:05,  1.85s/it]
 85%|████████▌ | 15868/18627 [8:58:08<1:18:55,  1.72s/it]
 85%|████████▌ | 15869/18627 [8:58:10<1:30:17,  1.96s/it]
 85%|████████▌ | 15870/18627 [8:58:12<1:34:56,  2.07s/it]
                                                         
{'loss': 1.4364, 'grad_norm': 6.112476825714111, 'learning_rate': 2.8179309251288563e-07, 'epoch': 0.85}
+
 85%|████████▌ | 15870/18627 [8:58:12<1:34:56,  2.07s/it]
 85%|███████���▌ | 15871/18627 [8:58:14<1:25:00,  1.85s/it]
 85%|████████▌ | 15872/18627 [8:58:18<1:50:54,  2.42s/it]
 85%|████████▌ | 15873/18627 [8:58:20<1:49:52,  2.39s/it]
 85%|████████▌ | 15874/18627 [8:58:22<1:48:35,  2.37s/it]
 85%|████████▌ | 15875/18627 [8:58:24<1:47:40,  2.35s/it]
 85%|████████▌ | 15876/18627 [8:58:27<1:47:16,  2.34s/it]
 85%|████████▌ | 15877/18627 [8:58:28<1:33:18,  2.04s/it]
 85%|████████▌ | 15878/18627 [8:58:30<1:37:18,  2.12s/it]
 85%|████████▌ | 15879/18627 [8:58:32<1:27:16,  1.91s/it]
 85%|████████▌ | 15880/18627 [8:58:34<1:33:20,  2.04s/it]
                                                         
{'loss': 1.1432, 'grad_norm': 8.480449676513672, 'learning_rate': 2.7979154325678797e-07, 'epoch': 0.85}
+
 85%|████████▌ | 15880/18627 [8:58:34<1:33:20,  2.04s/it]
 85%|████████▌ | 15881/18627 [8:58:37<1:37:31,  2.13s/it]
 85%|████████▌ | 15882/18627 [8:58:39<1:40:05,  2.19s/it]
 85%|████████▌ | 15883/18627 [8:58:41<1:41:48,  2.23s/it]
 85%|████████▌ | 15884/18627 [8:58:44<1:43:59,  2.27s/it]
 85%|████████▌ | 15885/18627 [8:58:46<1:44:14,  2.28s/it]
 85%|████████▌ | 15886/18627 [8:58:48<1:44:33,  2.29s/it]
 85%|████████▌ | 15887/18627 [8:58:50<1:44:47,  2.29s/it]
 85%|████████▌ | 15888/18627 [8:58:53<1:45:16,  2.31s/it]
 85%|████████▌ | 15889/18627 [8:58:55<1:45:47,  2.32s/it]
 85%|████████▌ | 15890/18627 [8:58:57<1:46:01,  2.32s/it]
                                                         
{'loss': 0.5996, 'grad_norm': 5.434619426727295, 'learning_rate': 2.7779670632953227e-07, 'epoch': 0.85}
+
 85%|████████▌ | 15890/18627 [8:58:57<1:46:01,  2.32s/it]
 85%|████████▌ | 15891/18627 [8:59:00<1:46:03,  2.33s/it]
 85%|████████▌ | 15892/18627 [8:59:02<1:46:15,  2.33s/it]
 85%|████████▌ | 15893/18627 [8:59:04<1:46:15,  2.33s/it]
 85%|████████▌ | 15894/18627 [8:59:07<1:46:18,  2.33s/it]
 85%|████████▌ | 15895/18627 [8:59:09<1:45:45,  2.32s/it]
 85%|████████▌ | 15896/18627 [8:59:11<1:33:05,  2.05s/it]
 85%|████████▌ | 15897/18627 [8:59:13<1:36:58,  2.13s/it]
 85%|████████▌ | 15898/18627 [8:59:15<1:39:17,  2.18s/it]
 85%|████████▌ | 15899/18627 [8:59:17<1:27:51,  1.93s/it]
 85%|████████▌ | 15900/18627 [8:59:19<1:35:11,  2.09s/it]
                                                         
{'loss': 0.9732, 'grad_norm': 5.943763732910156, 'learning_rate': 2.758085877620839e-07, 'epoch': 0.85}
+
 85%|████████▌ | 15900/18627 [8:59:19<1:35:11,  2.09s/it]
 85%|████████▌ | 15901/18627 [8:59:21<1:38:06,  2.16s/it]
 85%|████████▌ | 15902/18627 [8:59:24<1:40:17,  2.21s/it]
 85%|████████▌ | 15903/18627 [8:59:26<1:42:02,  2.25s/it]
 85%|████████▌ | 15904/18627 [8:59:28<1:43:11,  2.27s/it]
 85%|████████▌ | 15905/18627 [8:59:31<1:44:01,  2.29s/it]
 85%|████████▌ | 15906/18627 [8:59:33<1:44:31,  2.30s/it]
 85%|████████▌ | 15907/18627 [8:59:35<1:44:38,  2.31s/it]
 85%|████████▌ | 15908/18627 [8:59:38<1:45:13,  2.32s/it]
 85%|████████▌ | 15909/18627 [8:59:39<1:32:59,  2.05s/it]
 85%|████████▌ | 15910/18627 [8:59:41<1:36:45,  2.14s/it]
                                                         
{'loss': 0.8181, 'grad_norm': 6.547821998596191, 'learning_rate': 2.7382719356509824e-07, 'epoch': 0.85}
+
 85%|████████▌ | 15910/18627 [8:59:41<1:36:45,  2.14s/it]
 85%|████████▌ | 15911/18627 [8:59:44<1:39:12,  2.19s/it]
 85%|████████▌ | 15912/18627 [8:59:46<1:40:36,  2.22s/it]
 85%|████████▌ | 15913/18627 [8:59:48<1:42:09,  2.26s/it]
 85%|████████▌ | 15914/18627 [8:59:50<1:29:42,  1.98s/it]
 85%|████████▌ | 15915/18627 [8:59:52<1:34:28,  2.09s/it]
 85%|████████▌ | 15916/18627 [8:59:54<1:37:43,  2.16s/it]
 85%|████████▌ | 15917/18627 [8:59:57<1:40:12,  2.22s/it]
 85%|████████▌ | 15918/18627 [8:59:58<1:28:20,  1.96s/it]
 85%|████████▌ | 15919/18627 [8:59:59<1:20:55,  1.79s/it]
 85%|████████▌ | 15920/18627 [9:00:02<1:28:21,  1.96s/it]
                                                         
{'loss': 1.1697, 'grad_norm': 7.002528190612793, 'learning_rate': 2.718525297288979e-07, 'epoch': 0.85}
+
 85%|████████▌ | 15920/18627 [9:00:02<1:28:21,  1.96s/it]
 85%|████████▌ | 15921/18627 [9:00:04<1:32:59,  2.06s/it]
 85%|████████▌ | 15922/18627 [9:00:06<1:36:38,  2.14s/it]
 85%|████████▌ | 15923/18627 [9:00:09<1:39:29,  2.21s/it]
 85%|████████▌ | 15924/18627 [9:00:10<1:27:27,  1.94s/it]
 85%|████████▌ | 15925/18627 [9:00:11<1:19:35,  1.77s/it]
 85%|████████▌ | 15926/18627 [9:00:13<1:13:39,  1.64s/it]
 86%|████████▌ | 15927/18627 [9:00:15<1:23:55,  1.86s/it]
 86%|████████▌ | 15928/18627 [9:00:18<1:30:03,  2.00s/it]
 86%|████████▌ | 15929/18627 [9:00:20<1:34:18,  2.10s/it]
 86%|████████▌ | 15930/18627 [9:00:22<1:37:24,  2.17s/it]
                                                         
{'loss': 1.2268, 'grad_norm': 5.954078674316406, 'learning_rate': 2.6988460222346216e-07, 'epoch': 0.86}
+
 86%|████████▌ | 15930/18627 [9:00:22<1:37:24,  2.17s/it]
 86%|████████▌ | 15931/18627 [9:00:25<1:41:17,  2.25s/it]
 86%|████████▌ | 15932/18627 [9:00:27<1:42:44,  2.29s/it]
 86%|████████▌ | 15933/18627 [9:00:29<1:43:03,  2.30s/it]
 86%|████████▌ | 15934/18627 [9:00:32<1:43:39,  2.31s/it]
 86%|████████▌ | 15935/18627 [9:00:33<1:29:34,  2.00s/it]
 86%|████████▌ | 15936/18627 [9:00:35<1:34:06,  2.10s/it]
 86%|████████▌ | 15937/18627 [9:00:38<1:36:47,  2.16s/it]
 86%|████████▌ | 15938/18627 [9:00:40<1:39:48,  2.23s/it]
 86%|████████▌ | 15939/18627 [9:00:42<1:41:55,  2.28s/it]
 86%|████████▌ | 15940/18627 [9:00:45<1:43:07,  2.30s/it]
                                                         
{'loss': 0.8354, 'grad_norm': 5.344947814941406, 'learning_rate': 2.6792341699840047e-07, 'epoch': 0.86}
+
 86%|████████▌ | 15940/18627 [9:00:45<1:43:07,  2.30s/it]
 86%|████████▌ | 15941/18627 [9:00:47<1:43:47,  2.32s/it]
 86%|████████▌ | 15942/18627 [9:00:48<1:29:51,  2.01s/it]
 86%|████████▌ | 15943/18627 [9:00:51<1:34:13,  2.11s/it]
 86%|████████▌ | 15944/18627 [9:00:53<1:37:23,  2.18s/it]
 86%|████████▌ | 15945/18627 [9:00:56<1:43:23,  2.31s/it]
 86%|████████▌ | 15946/18627 [9:00:58<1:43:26,  2.32s/it]
 86%|████████▌ | 15947/18627 [9:00:59<1:30:32,  2.03s/it]
 86%|████████▌ | 15948/18627 [9:01:02<1:34:25,  2.11s/it]
 86%|████████▌ | 15949/18627 [9:01:03<1:24:42,  1.90s/it]
 86%|████████▌ | 15950/18627 [9:01:05<1:30:27,  2.03s/it]
                                                         
{'loss': 1.2116, 'grad_norm': 9.842594146728516, 'learning_rate': 2.6596897998293985e-07, 'epoch': 0.86}
+
 86%|████████▌ | 15950/18627 [9:01:05<1:30:27,  2.03s/it]
 86%|████████▌ | 15951/18627 [9:01:08<1:34:11,  2.11s/it]
 86%|████████▌ | 15952/18627 [9:01:10<1:37:02,  2.18s/it]
 86%|████████▌ | 15953/18627 [9:01:12<1:39:00,  2.22s/it]
 86%|████████▌ | 15954/18627 [9:01:15<1:41:04,  2.27s/it]
 86%|████████▌ | 15955/18627 [9:01:17<1:45:15,  2.36s/it]
 86%|████████▌ | 15956/18627 [9:01:20<1:45:11,  2.36s/it]
 86%|████████▌ | 15957/18627 [9:01:22<1:44:36,  2.35s/it]
 86%|████████▌ | 15958/18627 [9:01:24<1:44:04,  2.34s/it]
 86%|████████▌ | 15959/18627 [9:01:26<1:29:04,  2.00s/it]
 86%|████████▌ | 15960/18627 [9:01:28<1:33:17,  2.10s/it]
                                                         
{'loss': 0.6259, 'grad_norm': 9.079874992370605, 'learning_rate': 2.640212970859063e-07, 'epoch': 0.86}
+
 86%|████████▌ | 15960/18627 [9:01:28<1:33:17,  2.10s/it]
 86%|████████▌ | 15961/18627 [9:01:29<1:24:02,  1.89s/it]
 86%|████████▌ | 15962/18627 [9:01:32<1:29:48,  2.02s/it]
 86%|████████▌ | 15963/18627 [9:01:34<1:33:59,  2.12s/it]
 86%|████████▌ | 15964/18627 [9:01:36<1:36:40,  2.18s/it]
 86%|████████▌ | 15965/18627 [9:01:38<1:26:03,  1.94s/it]
 86%|████████▌ | 15966/18627 [9:01:40<1:31:11,  2.06s/it]
 86%|████████▌ | 15967/18627 [9:01:42<1:34:57,  2.14s/it]
 86%|████████▌ | 15968/18627 [9:01:45<1:40:45,  2.27s/it]
 86%|████████▌ | 15969/18627 [9:01:47<1:41:15,  2.29s/it]
 86%|████████▌ | 15970/18627 [9:01:50<1:42:02,  2.30s/it]
                                                         
{'loss': 0.9541, 'grad_norm': 6.48187780380249, 'learning_rate': 2.6208037419570506e-07, 'epoch': 0.86}
+
 86%|████████▌ | 15970/18627 [9:01:50<1:42:02,  2.30s/it]
 86%|████████▌ | 15971/18627 [9:01:52<1:42:27,  2.31s/it]
 86%|████████▌ | 15972/18627 [9:01:53<1:28:18,  2.00s/it]
 86%|████████▌ | 15973/18627 [9:01:55<1:20:41,  1.82s/it]
 86%|████████▌ | 15974/18627 [9:01:56<1:14:30,  1.68s/it]
 86%|████████▌ | 15975/18627 [9:01:58<1:22:35,  1.87s/it]
 86%|████████▌ | 15976/18627 [9:02:01<1:28:55,  2.01s/it]
 86%|████████▌ | 15977/18627 [9:02:02<1:20:47,  1.83s/it]
 86%|████████▌ | 15978/18627 [9:02:04<1:27:37,  1.98s/it]
 86%|████████▌ | 15979/18627 [9:02:06<1:19:00,  1.79s/it]
 86%|████████▌ | 15980/18627 [9:02:07<1:12:29,  1.64s/it]
                                                         
{'loss': 1.7646, 'grad_norm': 13.5930814743042, 'learning_rate': 2.601462171803043e-07, 'epoch': 0.86}
+
 86%|████████▌ | 15980/18627 [9:02:07<1:12:29,  1.64s/it]
 86%|████████▌ | 15981/18627 [9:02:09<1:21:56,  1.86s/it]
 86%|████████▌ | 15982/18627 [9:02:12<1:28:10,  2.00s/it]
 86%|████████▌ | 15983/18627 [9:02:14<1:32:18,  2.09s/it]
 86%|████████▌ | 15984/18627 [9:02:16<1:35:31,  2.17s/it]
 86%|████████▌ | 15985/18627 [9:02:19<1:37:22,  2.21s/it]
 86%|████████▌ | 15986/18627 [9:02:21<1:39:06,  2.25s/it]
 86%|████████▌ | 15987/18627 [9:02:23<1:40:29,  2.28s/it]
 86%|████████▌ | 15988/18627 [9:02:26<1:41:19,  2.30s/it]
 86%|████████▌ | 15989/18627 [9:02:28<1:42:02,  2.32s/it]
 86%|████████▌ | 15990/18627 [9:02:30<1:42:30,  2.33s/it]
                                                         
{'loss': 0.5917, 'grad_norm': 8.183120727539062, 'learning_rate': 2.582188318872175e-07, 'epoch': 0.86}
+
 86%|████████▌ | 15990/18627 [9:02:30<1:42:30,  2.33s/it]
 86%|████████▌ | 15991/18627 [9:02:33<1:42:37,  2.34s/it]
 86%|████████▌ | 15992/18627 [9:02:34<1:29:54,  2.05s/it]
 86%|████████▌ | 15993/18627 [9:02:36<1:33:57,  2.14s/it]
 86%|████████▌ | 15994/18627 [9:02:39<1:36:10,  2.19s/it]
 86%|████████▌ | 15995/18627 [9:02:41<1:38:02,  2.24s/it]
 86%|████████▌ | 15996/18627 [9:02:43<1:38:46,  2.25s/it]
 86%|████████▌ | 15997/18627 [9:02:45<1:28:16,  2.01s/it]
 86%|████████▌ | 15998/18627 [9:02:47<1:33:14,  2.13s/it]
 86%|████████▌ | 15999/18627 [9:02:50<1:38:50,  2.26s/it]
 86%|████████▌ | 16000/18627 [9:02:52<1:39:16,  2.27s/it]
                                                         
{'loss': 0.9252, 'grad_norm': 6.542034149169922, 'learning_rate': 2.562982241434847e-07, 'epoch': 0.86}
+
 86%|████████▌ | 16000/18627 [9:02:52<1:39:16,  2.27s/it]/home/zhengduo/workspace/LLaMA-Factory/src/qwen_vl/model/modeling_qwen2_5_vl.py:1979: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
+  with torch.cuda.amp.autocast(dtype=dtype):
+/home/zhengduo/miniconda3/envs/vgllm/lib/python3.10/site-packages/torch/utils/checkpoint.py:87: UserWarning: None of the inputs have requires_grad=True. Gradients will be None
+  warnings.warn(
+
 86%|████████▌ | 16001/18627 [9:04:01<16:12:55, 22.23s/it]
 86%|████████▌ | 16002/18627 [9:04:03<11:51:06, 16.25s/it]
 86%|████████▌ | 16003/18627 [9:04:06<8:48:07, 12.08s/it] 
 86%|████████▌ | 16004/18627 [9:04:08<6:39:55,  9.15s/it]
 86%|████████▌ | 16005/18627 [9:04:10<5:10:04,  7.10s/it]
 86%|████████▌ | 16006/18627 [9:04:11<3:54:15,  5.36s/it]
 86%|████████▌ | 16007/18627 [9:04:13<3:01:52,  4.17s/it]
 86%|████████▌ | 16008/18627 [9:04:15<2:37:52,  3.62s/it]
 86%|████████▌ | 16009/18627 [9:04:17<2:08:19,  2.94s/it]
 86%|████████▌ | 16010/18627 [9:04:19<2:00:02,  2.75s/it]
                                                         
{'loss': 1.297, 'grad_norm': 8.35802936553955, 'learning_rate': 2.5438439975565556e-07, 'epoch': 0.86}
+
 86%|████████▌ | 16010/18627 [9:04:19<2:00:02,  2.75s/it]
 86%|████████▌ | 16011/18627 [9:04:20<1:41:49,  2.34s/it]
 86%|████████▌ | 16012/18627 [9:04:23<1:41:45,  2.33s/it]
 86%|████████▌ | 16013/18627 [9:04:25<1:46:08,  2.44s/it]
 86%|████████▌ | 16014/18627 [9:04:28<1:45:54,  2.43s/it]
 86%|████████▌ | 16015/18627 [9:04:30<1:44:53,  2.41s/it]
 86%|████████▌ | 16016/18627 [9:04:32<1:44:40,  2.41s/it]
 86%|████████▌ | 16017/18627 [9:04:35<1:44:56,  2.41s/it]
 86%|████████▌ | 16018/18627 [9:04:36<1:27:41,  2.02s/it]
 86%|████████▌ | 16019/18627 [9:04:38<1:31:29,  2.11s/it]
 86%|████████▌ | 16020/18627 [9:04:40<1:21:53,  1.88s/it]
                                                         
{'loss': 0.9472, 'grad_norm': 13.594679832458496, 'learning_rate': 2.5247736450977197e-07, 'epoch': 0.86}
+
 86%|████████▌ | 16020/18627 [9:04:40<1:21:53,  1.88s/it]
 86%|████████▌ | 16021/18627 [9:04:42<1:27:41,  2.02s/it]
 86%|████████▌ | 16022/18627 [9:04:44<1:31:28,  2.11s/it]
 86%|████████▌ | 16023/18627 [9:04:47<1:34:07,  2.17s/it]
 86%|████████▌ | 16024/18627 [9:04:48<1:22:54,  1.91s/it]
 86%|████████▌ | 16025/18627 [9:04:49<1:15:44,  1.75s/it]
 86%|████████▌ | 16026/18627 [9:04:52<1:23:14,  1.92s/it]
 86%|████████▌ | 16027/18627 [9:04:53<1:16:06,  1.76s/it]
 86%|████████▌ | 16028/18627 [9:04:55<1:23:35,  1.93s/it]
 86%|████████▌ | 16029/18627 [9:04:58<1:29:06,  2.06s/it]
 86%|████████▌ | 16030/18627 [9:04:59<1:20:09,  1.85s/it]
                                                         
{'loss': 1.3473, 'grad_norm': 13.287386894226074, 'learning_rate': 2.5057712417134895e-07, 'epoch': 0.86}
+
 86%|████████▌ | 16030/18627 [9:04:59<1:20:09,  1.85s/it]
 86%|████████▌ | 16031/18627 [9:05:01<1:26:53,  2.01s/it]
 86%|████████▌ | 16032/18627 [9:05:04<1:31:24,  2.11s/it]
 86%|████████▌ | 16033/18627 [9:05:05<1:21:41,  1.89s/it]
 86%|████████▌ | 16034/18627 [9:05:07<1:27:23,  2.02s/it]
 86%|████████▌ | 16035/18627 [9:05:10<1:33:14,  2.16s/it]
 86%|████████▌ | 16036/18627 [9:05:12<1:35:10,  2.20s/it]
 86%|████████▌ | 16037/18627 [9:05:14<1:23:33,  1.94s/it]
 86%|████████▌ | 16038/18627 [9:05:15<1:17:22,  1.79s/it]
 86%|████████▌ | 16039/18627 [9:05:17<1:23:59,  1.95s/it]
 86%|████████▌ | 16040/18627 [9:05:20<1:28:30,  2.05s/it]
                                                         
{'loss': 1.1956, 'grad_norm': 6.062356948852539, 'learning_rate': 2.486836844853616e-07, 'epoch': 0.86}
+
 86%|████████▌ | 16040/18627 [9:05:20<1:28:30,  2.05s/it]
 86%|████████▌ | 16041/18627 [9:05:22<1:32:03,  2.14s/it]
 86%|████████▌ | 16042/18627 [9:05:23<1:21:45,  1.90s/it]
 86%|████████▌ | 16043/18627 [9:05:26<1:27:07,  2.02s/it]
 86%|████████▌ | 16044/18627 [9:05:28<1:30:10,  2.09s/it]
 86%|████████▌ | 16045/18627 [9:05:30<1:33:12,  2.17s/it]
 86%|████████▌ | 16046/18627 [9:05:32<1:35:07,  2.21s/it]
 86%|████████▌ | 16047/18627 [9:05:34<1:24:04,  1.96s/it]
 86%|████████▌ | 16048/18627 [9:05:36<1:28:56,  2.07s/it]
 86%|████████▌ | 16049/18627 [9:05:38<1:20:17,  1.87s/it]
 86%|████████▌ | 16050/18627 [9:05:40<1:26:45,  2.02s/it]
                                                         
{'loss': 1.1925, 'grad_norm': 4.892768383026123, 'learning_rate': 2.4679705117622103e-07, 'epoch': 0.86}
+
 86%|████████▌ | 16050/18627 [9:05:40<1:26:45,  2.02s/it]
 86%|████████▌ | 16051/18627 [9:05:42<1:30:40,  2.11s/it]
 86%|████████▌ | 16052/18627 [9:05:45<1:33:04,  2.17s/it]
 86%|████████▌ | 16053/18627 [9:05:47<1:34:43,  2.21s/it]
 86%|████████▌ | 16054/18627 [9:05:49<1:36:29,  2.25s/it]
 86%|████████▌ | 16055/18627 [9:05:52<1:37:11,  2.27s/it]
 86%|████████▌ | 16056/18627 [9:05:53<1:25:14,  1.99s/it]
 86%|████████▌ | 16057/18627 [9:05:55<1:29:32,  2.09s/it]
 86%|████████▌ | 16058/18627 [9:05:57<1:20:25,  1.88s/it]
 86%|████████▌ | 16059/18627 [9:05:59<1:26:29,  2.02s/it]
 86%|████████▌ | 16060/18627 [9:06:00<1:18:22,  1.83s/it]
                                                         
{'loss': 1.1784, 'grad_norm': 13.090235710144043, 'learning_rate': 2.449172299477634e-07, 'epoch': 0.86}
+
 86%|████████▌ | 16060/18627 [9:06:00<1:18:22,  1.83s/it]
 86%|████████▌ | 16061/18627 [9:06:02<1:13:16,  1.71s/it]
 86%|████████▌ | 16062/18627 [9:06:03<1:05:11,  1.53s/it]
 86%|████████▌ | 16063/18627 [9:06:05<1:15:34,  1.77s/it]
 86%|████████▌ | 16064/18627 [9:06:08<1:22:40,  1.94s/it]
 86%|████████▌ | 16065/18627 [9:06:10<1:27:34,  2.05s/it]
 86%|████████▋ | 16066/18627 [9:06:11<1:18:50,  1.85s/it]
 86%|████████▋ | 16067/18627 [9:06:14<1:25:32,  2.01s/it]
 86%|████████▋ | 16068/18627 [9:06:16<1:29:50,  2.11s/it]
 86%|████████▋ | 16069/18627 [9:06:17<1:20:19,  1.88s/it]
 86%|████████▋ | 16070/18627 [9:06:20<1:25:49,  2.01s/it]
                                                         
{'loss': 1.2096, 'grad_norm': 17.01757049560547, 'learning_rate': 2.430442264832286e-07, 'epoch': 0.86}
+
 86%|████████▋ | 16070/18627 [9:06:20<1:25:49,  2.01s/it]
 86%|████████▋ | 16071/18627 [9:06:22<1:30:03,  2.11s/it]
 86%|████████▋ | 16072/18627 [9:06:24<1:33:07,  2.19s/it]
 86%|████████▋ | 16073/18627 [9:06:26<1:22:58,  1.95s/it]
 86%|████████▋ | 16074/18627 [9:06:27<1:16:00,  1.79s/it]
 86%|████████▋ | 16075/18627 [9:06:28<1:10:04,  1.65s/it]
 86%|████████▋ | 16076/18627 [9:06:31<1:18:44,  1.85s/it]
 86%|████████▋ | 16077/18627 [9:06:33<1:24:31,  1.99s/it]
 86%|████████▋ | 16078/18627 [9:06:35<1:28:40,  2.09s/it]
 86%|████████▋ | 16079/18627 [9:06:38<1:32:06,  2.17s/it]
 86%|████████▋ | 16080/18627 [9:06:39<1:21:44,  1.93s/it]
                                                         
{'loss': 1.4189, 'grad_norm': 13.214481353759766, 'learning_rate': 2.411780464452448e-07, 'epoch': 0.86}
+
 86%|████████▋ | 16080/18627 [9:06:39<1:21:44,  1.93s/it]
 86%|████████▋ | 16081/18627 [9:06:41<1:27:08,  2.05s/it]
 86%|████████▋ | 16082/18627 [9:06:43<1:18:12,  1.84s/it]
 86%|████████▋ | 16083/18627 [9:06:45<1:24:17,  1.99s/it]
 86%|████████▋ | 16084/18627 [9:06:47<1:28:46,  2.09s/it]
 86%|████████▋ | 16085/18627 [9:06:50<1:31:29,  2.16s/it]
 86%|████████▋ | 16086/18627 [9:06:52<1:33:51,  2.22s/it]
 86%|████████▋ | 16087/18627 [9:06:54<1:35:05,  2.25s/it]
 86%|████████▋ | 16088/18627 [9:06:57<1:36:19,  2.28s/it]
 86%|████████▋ | 16089/18627 [9:06:59<1:36:54,  2.29s/it]
 86%|████████▋ | 16090/18627 [9:07:01<1:37:33,  2.31s/it]
                                                         
{'loss': 0.822, 'grad_norm': 5.731315612792969, 'learning_rate': 2.3931869547581174e-07, 'epoch': 0.86}
+
 86%|████████▋ | 16090/18627 [9:07:01<1:37:33,  2.31s/it]
 86%|████████▋ | 16091/18627 [9:07:04<1:37:53,  2.32s/it]
 86%|████████▋ | 16092/18627 [9:07:06<1:37:55,  2.32s/it]
 86%|████████▋ | 16093/18627 [9:07:08<1:37:58,  2.32s/it]
 86%|████████▋ | 16094/18627 [9:07:11<1:37:50,  2.32s/it]
 86%|████████▋ | 16095/18627 [9:07:13<1:38:21,  2.33s/it]
 86%|████████▋ | 16096/18627 [9:07:15<1:38:41,  2.34s/it]
 86%|████████▋ | 16097/18627 [9:07:17<1:26:02,  2.04s/it]
 86%|████████▋ | 16098/18627 [9:07:18<1:17:47,  1.85s/it]
 86%|████████▋ | 16099/18627 [9:07:20<1:12:04,  1.71s/it]
 86%|████████▋ | 16100/18627 [9:07:22<1:20:38,  1.91s/it]
                                                         
{'loss': 1.1854, 'grad_norm': 8.962678909301758, 'learning_rate': 2.3746617919628234e-07, 'epoch': 0.86}
+
 86%|████████▋ | 16100/18627 [9:07:22<1:20:38,  1.91s/it]
 86%|████████▋ | 16101/18627 [9:07:24<1:25:38,  2.03s/it]
 86%|████████▋ | 16102/18627 [9:07:27<1:29:30,  2.13s/it]
 86%|████████▋ | 16103/18627 [9:07:28<1:19:22,  1.89s/it]
 86%|████████▋ | 16104/18627 [9:07:30<1:24:56,  2.02s/it]
 86%|████████▋ | 16105/18627 [9:07:33<1:28:38,  2.11s/it]
 86%|████████▋ | 16106/18627 [9:07:35<1:31:53,  2.19s/it]
 86%|████████▋ | 16107/18627 [9:07:37<1:33:41,  2.23s/it]
 86%|████████▋ | 16108/18627 [9:07:40<1:35:09,  2.27s/it]
 86%|████████▋ | 16109/18627 [9:07:41<1:23:44,  2.00s/it]
 86%|████████▋ | 16110/18627 [9:07:43<1:27:52,  2.09s/it]
                                                         
{'loss': 0.964, 'grad_norm': 6.1976118087768555, 'learning_rate': 2.356205032073447e-07, 'epoch': 0.86}
+
 86%|████████▋ | 16110/18627 [9:07:43<1:27:52,  2.09s/it]
 86%|████████▋ | 16111/18627 [9:07:45<1:19:23,  1.89s/it]
 86%|████████▋ | 16112/18627 [9:07:47<1:25:11,  2.03s/it]
 87%|████████▋ | 16113/18627 [9:07:50<1:29:22,  2.13s/it]
 87%|████████▋ | 16114/18627 [9:07:52<1:32:11,  2.20s/it]
 87%|████████▋ | 16115/18627 [9:07:54<1:33:44,  2.24s/it]
 87%|████████▋ | 16116/18627 [9:07:57<1:34:49,  2.27s/it]
 87%|████████▋ | 16117/18627 [9:07:59<1:35:57,  2.29s/it]
 87%|████████▋ | 16118/18627 [9:08:01<1:36:16,  2.30s/it]
 87%|████████▋ | 16119/18627 [9:08:04<1:36:45,  2.31s/it]
 87%|████████▋ | 16120/18627 [9:08:05<1:23:40,  2.00s/it]
                                                         
{'loss': 0.9556, 'grad_norm': 13.814313888549805, 'learning_rate': 2.3378167308901022e-07, 'epoch': 0.87}
+
 87%|████████▋ | 16120/18627 [9:08:05<1:23:40,  2.00s/it]
 87%|████████▋ | 16121/18627 [9:08:07<1:27:50,  2.10s/it]
 87%|████████▋ | 16122/18627 [9:08:10<1:30:39,  2.17s/it]
 87%|��███████▋ | 16123/18627 [9:08:12<1:32:37,  2.22s/it]
 87%|████████▋ | 16124/18627 [9:08:14<1:33:30,  2.24s/it]
 87%|████████▋ | 16125/18627 [9:08:16<1:34:22,  2.26s/it]
 87%|████████▋ | 16126/18627 [9:08:19<1:35:07,  2.28s/it]
 87%|████████▋ | 16127/18627 [9:08:21<1:35:47,  2.30s/it]
 87%|████████▋ | 16128/18627 [9:08:23<1:35:34,  2.29s/it]
 87%|████████▋ | 16129/18627 [9:08:25<1:23:56,  2.02s/it]
 87%|████████▋ | 16130/18627 [9:08:26<1:16:05,  1.83s/it]
                                                         
{'loss': 0.9886, 'grad_norm': 11.780576705932617, 'learning_rate': 2.3194969440059e-07, 'epoch': 0.87}
+
 87%|████████▋ | 16130/18627 [9:08:26<1:16:05,  1.83s/it]
 87%|████████▋ | 16131/18627 [9:08:28<1:22:09,  1.98s/it]
 87%|████████▋ | 16132/18627 [9:08:30<1:14:25,  1.79s/it]
 87%|████████▋ | 16133/18627 [9:08:31<1:05:12,  1.57s/it]
 87%|████████▋ | 16134/18627 [9:08:33<1:14:25,  1.79s/it]
 87%|████████▋ | 16135/18627 [9:08:35<1:08:35,  1.65s/it]
 87%|████████▋ | 16136/18627 [9:08:36<1:05:20,  1.57s/it]
 87%|████████▋ | 16137/18627 [9:08:38<1:14:51,  1.80s/it]
 87%|████████▋ | 16138/18627 [9:08:41<1:21:43,  1.97s/it]
 87%|████████▋ | 16139/18627 [9:08:43<1:26:10,  2.08s/it]
 87%|████████▋ | 16140/18627 [9:08:45<1:29:49,  2.17s/it]
                                                         
{'loss': 1.1546, 'grad_norm': 5.901388168334961, 'learning_rate': 2.3012457268068305e-07, 'epoch': 0.87}
+
 87%|████████▋ | 16140/18627 [9:08:45<1:29:49,  2.17s/it]
 87%|████████▋ | 16141/18627 [9:08:48<1:32:11,  2.22s/it]
 87%|████████▋ | 16142/18627 [9:08:50<1:33:15,  2.25s/it]
 87%|████████▋ | 16143/18627 [9:08:52<1:34:20,  2.28s/it]
 87%|████████▋ | 16144/18627 [9:08:55<1:34:44,  2.29s/it]
 87%|████████▋ | 16145/18627 [9:08:57<1:35:08,  2.30s/it]
 87%|████████▋ | 16146/18627 [9:08:58<1:23:57,  2.03s/it]
 87%|████████▋ | 16147/18627 [9:09:01<1:27:34,  2.12s/it]
 87%|████████▋ | 16148/18627 [9:09:03<1:29:45,  2.17s/it]
 87%|████████▋ | 16149/18627 [9:09:05<1:31:11,  2.21s/it]
 87%|████████▋ | 16150/18627 [9:09:08<1:32:04,  2.23s/it]
                                                         
{'loss': 0.7811, 'grad_norm': 6.536515712738037, 'learning_rate': 2.283063134471572e-07, 'epoch': 0.87}
+
 87%|████████▋ | 16150/18627 [9:09:08<1:32:04,  2.23s/it]
 87%|████████▋ | 16151/18627 [9:09:10<1:33:18,  2.26s/it]
 87%|████████▋ | 16152/18627 [9:09:12<1:33:40,  2.27s/it]
 87%|████████▋ | 16153/18627 [9:09:14<1:23:17,  2.02s/it]
 87%|████████▋ | 16154/18627 [9:09:15<1:15:30,  1.83s/it]
 87%|████████▋ | 16155/18627 [9:09:16<1:09:47,  1.69s/it]
 87%|████████▋ | 16156/18627 [9:09:19<1:17:50,  1.89s/it]
 87%|████████▋ | 16157/18627 [9:09:20<1:11:49,  1.74s/it]
 87%|████████▋ | 16158/18627 [9:09:23<1:19:22,  1.93s/it]
 87%|████████▋ | 16159/18627 [9:09:25<1:24:34,  2.06s/it]
 87%|████████▋ | 16160/18627 [9:09:27<1:27:44,  2.13s/it]
                                                         
{'loss': 1.4084, 'grad_norm': 6.261140823364258, 'learning_rate': 2.2649492219713355e-07, 'epoch': 0.87}
+
 87%|████████▋ | 16160/18627 [9:09:27<1:27:44,  2.13s/it]
 87%|████████▋ | 16161/18627 [9:09:30<1:30:09,  2.19s/it]
 87%|████████▋ | 16162/18627 [9:09:31<1:20:56,  1.97s/it]
 87%|████████▋ | 16163/18627 [9:09:33<1:25:26,  2.08s/it]
 87%|████████▋ | 16164/18627 [9:09:36<1:28:18,  2.15s/it]
 87%|████████▋ | 16165/18627 [9:09:37<1:18:32,  1.91s/it]
 87%|████████▋ | 16166/18627 [9:09:39<1:23:14,  2.03s/it]
 87%|████████▋ | 16167/18627 [9:09:42<1:26:46,  2.12s/it]
 87%|████████▋ | 16168/18627 [9:09:44<1:29:14,  2.18s/it]
 87%|████████▋ | 16169/18627 [9:09:46<1:31:12,  2.23s/it]
 87%|████████▋ | 16170/18627 [9:09:49<1:32:34,  2.26s/it]
                                                         
{'loss': 0.9362, 'grad_norm': 7.525829315185547, 'learning_rate': 2.2469040440696864e-07, 'epoch': 0.87}
+
 87%|████████▋ | 16170/18627 [9:09:49<1:32:34,  2.26s/it]
 87%|████████▋ | 16171/18627 [9:09:50<1:21:31,  1.99s/it]
 87%|████████▋ | 16172/18627 [9:09:51<1:14:05,  1.81s/it]
 87%|████████▋ | 16173/18627 [9:09:54<1:20:07,  1.96s/it]
 87%|████████▋ | 16174/18627 [9:09:55<1:11:57,  1.76s/it]
 87%|████████▋ | 16175/18627 [9:09:56<1:03:26,  1.55s/it]
 87%|████████▋ | 16176/18627 [9:09:57<1:01:30,  1.51s/it]
 87%|████████▋ | 16177/18627 [9:10:00<1:11:29,  1.75s/it]
 87%|████████▋ | 16178/18627 [9:10:01<1:06:56,  1.64s/it]
 87%|████████▋ | 16179/18627 [9:10:02<1:03:36,  1.56s/it]
 87%|████████▋ | 16180/18627 [9:10:05<1:12:58,  1.79s/it]
                                                         
{'loss': 1.8306, 'grad_norm': 4.603457450866699, 'learning_rate': 2.2289276553223938e-07, 'epoch': 0.87}
+
 87%|████████▋ | 16180/18627 [9:10:05<1:12:58,  1.79s/it]
 87%|████████▋ | 16181/18627 [9:10:07<1:20:04,  1.96s/it]
 87%|████████▋ | 16182/18627 [9:10:10<1:24:49,  2.08s/it]
 87%|████████▋ | 16183/18627 [9:10:12<1:27:47,  2.16s/it]
 87%|████████▋ | 16184/18627 [9:10:14<1:30:01,  2.21s/it]
 87%|████████▋ | 16185/18627 [9:10:17<1:31:36,  2.25s/it]
 87%|████████▋ | 16186/18627 [9:10:19<1:32:52,  2.28s/it]
 87%|████████▋ | 16187/18627 [9:10:21<1:33:32,  2.30s/it]
 87%|████████▋ | 16188/18627 [9:10:24<1:33:59,  2.31s/it]
 87%|████████▋ | 16189/18627 [9:10:26<1:34:07,  2.32s/it]
 87%|████████▋ | 16190/18627 [9:10:28<1:34:14,  2.32s/it]
                                                         
{'loss': 0.5669, 'grad_norm': 4.993289947509766, 'learning_rate': 2.211020110077236e-07, 'epoch': 0.87}
+
 87%|████████▋ | 16190/18627 [9:10:28<1:34:14,  2.32s/it]
 87%|████████▋ | 16191/18627 [9:10:31<1:34:42,  2.33s/it]
 87%|████████▋ | 16192/18627 [9:10:33<1:34:54,  2.34s/it]
 87%|████████▋ | 16193/18627 [9:10:35<1:34:43,  2.34s/it]
 87%|████████▋ | 16194/18627 [9:10:38<1:34:54,  2.34s/it]
 87%|████████▋ | 16195/18627 [9:10:42<1:59:20,  2.94s/it]
 87%|████████▋ | 16196/18627 [9:10:44<1:51:42,  2.76s/it]
 87%|████████▋ | 16197/18627 [9:10:46<1:34:48,  2.34s/it]
 87%|████████▋ | 16198/18627 [9:10:48<1:33:36,  2.31s/it]
 87%|████████▋ | 16199/18627 [9:10:50<1:33:41,  2.32s/it]
 87%|████████▋ | 16200/18627 [9:10:53<1:34:21,  2.33s/it]
                                                         
{'loss': 0.8243, 'grad_norm': 4.902647972106934, 'learning_rate': 2.193181462473895e-07, 'epoch': 0.87}
+
 87%|████████▋ | 16200/18627 [9:10:53<1:34:21,  2.33s/it]
 87%|████████▋ | 16201/18627 [9:10:54<1:23:31,  2.07s/it]
 87%|████████▋ | 16202/18627 [9:10:55<1:15:10,  1.86s/it]
 87%|████████▋ | 16203/18627 [9:10:57<1:08:33,  1.70s/it]
 87%|████████▋ | 16204/18627 [9:10:59<1:16:02,  1.88s/it]
 87%|████████▋ | 16205/18627 [9:11:01<1:21:48,  2.03s/it]
 87%|████████▋ | 16206/18627 [9:11:04<1:25:15,  2.11s/it]
 87%|████████▋ | 16207/18627 [9:11:06<1:27:33,  2.17s/it]
 87%|████████▋ | 16208/18627 [9:11:07<1:17:01,  1.91s/it]
 87%|████████▋ | 16209/18627 [9:11:10<1:21:50,  2.03s/it]
 87%|████████▋ | 16210/18627 [9:11:12<1:25:07,  2.11s/it]
                                                         
{'loss': 1.3468, 'grad_norm': 5.600671291351318, 'learning_rate': 2.1754117664437164e-07, 'epoch': 0.87}
+
 87%|████████▋ | 16210/18627 [9:11:12<1:25:07,  2.11s/it]
 87%|████████▋ | 16211/18627 [9:11:14<1:27:40,  2.18s/it]
 87%|████████▋ | 16212/18627 [9:11:16<1:17:45,  1.93s/it]
 87%|████████▋ | 16213/18627 [9:11:18<1:22:26,  2.05s/it]
 87%|████████▋ | 16214/18627 [9:11:20<1:25:28,  2.13s/it]
 87%|████████▋ | 16215/18627 [9:11:23<1:27:46,  2.18s/it]
 87%|████████▋ | 16216/18627 [9:11:24<1:17:50,  1.94s/it]
 87%|████████▋ | 16217/18627 [9:11:26<1:22:22,  2.05s/it]
 87%|████████▋ | 16218/18627 [9:11:29<1:25:41,  2.13s/it]
 87%|████████▋ | 16219/18627 [9:11:31<1:27:56,  2.19s/it]
 87%|████████▋ | 16220/18627 [9:11:33<1:29:26,  2.23s/it]
                                                         
{'loss': 1.0147, 'grad_norm': 4.874847412109375, 'learning_rate': 2.1577110757096175e-07, 'epoch': 0.87}
+
 87%|████████▋ | 16220/18627 [9:11:33<1:29:26,  2.23s/it]
 87%|████████▋ | 16221/18627 [9:11:36<1:30:26,  2.26s/it]
 87%|████████▋ | 16222/18627 [9:11:38<1:30:50,  2.27s/it]
 87%|████████▋ | 16223/18627 [9:11:39<1:16:03,  1.90s/it]
 87%|████████▋ | 16224/18627 [9:11:41<1:21:19,  2.03s/it]
 87%|████████▋ | 16225/18627 [9:11:43<1:12:59,  1.82s/it]
 87%|████████▋ | 16226/18627 [9:11:44<1:07:11,  1.68s/it]
 87%|████████▋ | 16227/18627 [9:11:46<1:15:13,  1.88s/it]
 87%|████████▋ | 16228/18627 [9:11:49<1:20:32,  2.01s/it]
 87%|████████▋ | 16229/18627 [9:11:50<1:12:58,  1.83s/it]
 87%|████████▋ | 16230/18627 [9:11:52<1:19:04,  1.98s/it]
                                                         
{'loss': 1.2434, 'grad_norm': 14.312248229980469, 'learning_rate': 2.1400794437858696e-07, 'epoch': 0.87}
+
 87%|████████▋ | 16230/18627 [9:11:52<1:19:04,  1.98s/it]
 87%|████████▋ | 16231/18627 [9:11:55<1:23:09,  2.08s/it]
 87%|████████▋ | 16232/18627 [9:11:57<1:26:08,  2.16s/it]
 87%|████████▋ | 16233/18627 [9:11:59<1:27:57,  2.20s/it]
 87%|████████▋ | 16234/18627 [9:12:02<1:29:20,  2.24s/it]
 87%|████████▋ | 16235/18627 [9:12:04<1:30:00,  2.26s/it]
 87%|████████▋ | 16236/18627 [9:12:05<1:19:02,  1.98s/it]
 87%|████████▋ | 16237/18627 [9:12:08<1:23:14,  2.09s/it]
 87%|████████▋ | 16238/18627 [9:12:10<1:26:52,  2.18s/it]
 87%|████████▋ | 16239/18627 [9:12:12<1:28:47,  2.23s/it]
 87%|████████▋ | 16240/18627 [9:12:14<1:18:23,  1.97s/it]
                                                         
{'loss': 0.8378, 'grad_norm': 13.422438621520996, 'learning_rate': 2.122516923977966e-07, 'epoch': 0.87}
+
 87%|████████▋ | 16240/18627 [9:12:14<1:18:23,  1.97s/it]
 87%|████████▋ | 16241/18627 [9:12:16<1:22:49,  2.08s/it]
 87%|████████▋ | 16242/18627 [9:12:18<1:25:40,  2.16s/it]
 87%|████████▋ | 16243/18627 [9:12:21<1:27:54,  2.21s/it]
 87%|████████▋ | 16244/18627 [9:12:23<1:29:08,  2.24s/it]
 87%|████████▋ | 16245/18627 [9:12:25<1:30:03,  2.27s/it]
 87%|████████▋ | 16246/18627 [9:12:28<1:30:56,  2.29s/it]
 87%|████████▋ | 16247/18627 [9:12:29<1:20:02,  2.02s/it]
 87%|████████▋ | 16248/18627 [9:12:31<1:23:36,  2.11s/it]
 87%|████████▋ | 16249/18627 [9:12:33<1:14:31,  1.88s/it]
 87%|████████▋ | 16250/18627 [9:12:35<1:19:28,  2.01s/it]
                                                         
{'loss': 0.9621, 'grad_norm': 5.652050018310547, 'learning_rate': 2.1050235693824612e-07, 'epoch': 0.87}
+
 87%|████████▋ | 16250/18627 [9:12:35<1:19:28,  2.01s/it]
 87%|████████▋ | 16251/18627 [9:12:37<1:23:21,  2.11s/it]
 87%|████████▋ | 16252/18627 [9:12:40<1:26:14,  2.18s/it]
 87%|████████▋ | 16253/18627 [9:12:41<1:14:56,  1.89s/it]
 87%|████████▋ | 16254/18627 [9:12:43<1:21:32,  2.06s/it]
 87%|████████▋ | 16255/18627 [9:12:46<1:25:04,  2.15s/it]
 87%|████████▋ | 16256/18627 [9:12:47<1:11:36,  1.81s/it]
 87%|████████▋ | 16257/18627 [9:12:49<1:17:39,  1.97s/it]
 87%|████████▋ | 16258/18627 [9:12:51<1:21:51,  2.07s/it]
 87%|████████▋ | 16259/18627 [9:12:54<1:24:59,  2.15s/it]
 87%|████████▋ | 16260/18627 [9:12:55<1:14:15,  1.88s/it]
                                                         
{'loss': 1.0221, 'grad_norm': 14.596415519714355, 'learning_rate': 2.0875994328868e-07, 'epoch': 0.87}
+
 87%|████████▋ | 16260/18627 [9:12:55<1:14:15,  1.88s/it]
 87%|████████▋ | 16261/18627 [9:12:57<1:19:34,  2.02s/it]
 87%|████████▋ | 16262/18627 [9:13:00<1:22:58,  2.11s/it]
 87%|████████▋ | 16263/18627 [9:13:02<1:25:43,  2.18s/it]
 87%|████████▋ | 16264/18627 [9:13:04<1:27:32,  2.22s/it]
 87%|████████▋ | 16265/18627 [9:13:07<1:28:58,  2.26s/it]
 87%|████████▋ | 16266/18627 [9:13:09<1:29:28,  2.27s/it]
 87%|████████▋ | 16267/18627 [9:13:11<1:30:04,  2.29s/it]
 87%|████████▋ | 16268/18627 [9:13:14<1:30:08,  2.29s/it]
 87%|████████▋ | 16269/18627 [9:13:16<1:30:21,  2.30s/it]
 87%|████████▋ | 16270/18627 [9:13:18<1:29:37,  2.28s/it]
                                                         
{'loss': 0.5787, 'grad_norm': 6.0642242431640625, 'learning_rate': 2.0702445671691524e-07, 'epoch': 0.87}
+
 87%|████████▋ | 16270/18627 [9:13:18<1:29:37,  2.28s/it]
 87%|████████▋ | 16271/18627 [9:13:21<1:31:12,  2.32s/it]
 87%|████████▋ | 16272/18627 [9:13:23<1:31:02,  2.32s/it]
 87%|████████▋ | 16273/18627 [9:13:24<1:19:43,  2.03s/it]
 87%|████████▋ | 16274/18627 [9:13:27<1:22:59,  2.12s/it]
 87%|████████▋ | 16275/18627 [9:13:29<1:25:09,  2.17s/it]
 87%|████████▋ | 16276/18627 [9:13:31<1:26:35,  2.21s/it]
 87%|████████▋ | 16277/18627 [9:13:34<1:27:50,  2.24s/it]
 87%|████████▋ | 16278/18627 [9:13:35<1:17:15,  1.97s/it]
 87%|████████▋ | 16279/18627 [9:13:37<1:21:24,  2.08s/it]
 87%|████████▋ | 16280/18627 [9:13:40<1:24:21,  2.16s/it]
                                                         
{'loss': 1.0479, 'grad_norm': 4.171885013580322, 'learning_rate': 2.0529590246982755e-07, 'epoch': 0.87}
+
 87%|████████▋ | 16280/18627 [9:13:40<1:24:21,  2.16s/it]
 87%|████████▋ | 16281/18627 [9:13:42<1:26:15,  2.21s/it]
 87%|████████▋ | 16282/18627 [9:13:44<1:27:54,  2.25s/it]
 87%|████████▋ | 16283/18627 [9:13:47<1:28:45,  2.27s/it]
 87%|████████▋ | 16284/18627 [9:13:49<1:29:34,  2.29s/it]
 87%|████████▋ | 16285/18627 [9:13:51<1:29:46,  2.30s/it]
 87%|████████▋ | 16286/18627 [9:13:54<1:30:08,  2.31s/it]
 87%|████████▋ | 16287/18627 [9:13:56<1:30:23,  2.32s/it]
 87%|████████▋ | 16288/18627 [9:13:58<1:28:53,  2.28s/it]
 87%|████████▋ | 16289/18627 [9:13:59<1:16:38,  1.97s/it]
 87%|████████▋ | 16290/18627 [9:14:01<1:09:47,  1.79s/it]
                                                         
{'loss': 1.0218, 'grad_norm': 14.880964279174805, 'learning_rate': 2.035742857733336e-07, 'epoch': 0.87}
+
 87%|████████▋ | 16290/18627 [9:14:01<1:09:47,  1.79s/it]
 87%|████████▋ | 16291/18627 [9:14:03<1:15:53,  1.95s/it]
 87%|████████▋ | 16292/18627 [9:14:04<1:08:37,  1.76s/it]
 87%|████████▋ | 16293/18627 [9:14:07<1:15:05,  1.93s/it]
 87%|████████▋ | 16294/18627 [9:14:08<1:08:07,  1.75s/it]
 87%|████████▋ | 16295/18627 [9:14:10<1:14:48,  1.92s/it]
 87%|████████▋ | 16296/18627 [9:14:13<1:19:23,  2.04s/it]
 87%|████████▋ | 16297/18627 [9:14:14<1:11:43,  1.85s/it]
 87%|████████▋ | 16298/18627 [9:14:16<1:18:15,  2.02s/it]
 88%|████████▊ | 16299/18627 [9:14:19<1:22:16,  2.12s/it]
 88%|████████▊ | 16300/18627 [9:14:21<1:24:41,  2.18s/it]
                                                         
{'loss': 1.2075, 'grad_norm': 8.987008094787598, 'learning_rate': 2.0185961183237568e-07, 'epoch': 0.88}
+
 88%|████████▊ | 16300/18627 [9:14:21<1:24:41,  2.18s/it]
 88%|████████▊ | 16301/18627 [9:14:23<1:26:31,  2.23s/it]
 88%|████████▊ | 16302/18627 [9:14:26<1:27:34,  2.26s/it]
 88%|████████▊ | 16303/18627 [9:14:28<1:28:49,  2.29s/it]
 88%|████████▊ | 16304/18627 [9:14:30<1:28:59,  2.30s/it]
 88%|████████▊ | 16305/18627 [9:14:33<1:29:06,  2.30s/it]
 88%|████████▊ | 16306/18627 [9:14:35<1:29:18,  2.31s/it]
 88%|████████▊ | 16307/18627 [9:14:37<1:29:29,  2.31s/it]
 88%|████████▊ | 16308/18627 [9:14:40<1:29:30,  2.32s/it]
 88%|████████▊ | 16309/18627 [9:14:42<1:29:47,  2.32s/it]
 88%|████████▊ | 16310/18627 [9:14:44<1:29:45,  2.32s/it]
                                                         
{'loss': 0.6197, 'grad_norm': 5.683349609375, 'learning_rate': 2.0015188583090707e-07, 'epoch': 0.88}
+
 88%|████████▊ | 16310/18627 [9:14:44<1:29:45,  2.32s/it]
 88%|████████▊ | 16311/18627 [9:14:47<1:29:49,  2.33s/it]
 88%|████████▊ | 16312/18627 [9:14:49<1:29:28,  2.32s/it]
 88%|████████▊ | 16313/18627 [9:14:50<1:18:11,  2.03s/it]
 88%|████████▊ | 16314/18627 [9:14:52<1:10:32,  1.83s/it]
 88%|████████▊ | 16315/18627 [9:14:54<1:16:09,  1.98s/it]
 88%|████████▊ | 16316/18627 [9:14:56<1:20:08,  2.08s/it]
 88%|████████▊ | 16317/18627 [9:14:59<1:22:57,  2.15s/it]
 88%|████████▊ | 16318/18627 [9:15:01<1:24:45,  2.20s/it]
 88%|████████▊ | 16319/18627 [9:15:03<1:26:03,  2.24s/it]
 88%|████████▊ | 16320/18627 [9:15:06<1:26:58,  2.26s/it]
                                                         
{'loss': 1.0677, 'grad_norm': 4.731996059417725, 'learning_rate': 1.9845111293187308e-07, 'epoch': 0.88}
+
 88%|████████▊ | 16320/18627 [9:15:06<1:26:58,  2.26s/it]
 88%|████████▊ | 16321/18627 [9:15:08<1:27:09,  2.27s/it]
 88%|████████▊ | 16322/18627 [9:15:10<1:27:51,  2.29s/it]
 88%|████████▊ | 16323/18627 [9:15:11<1:13:34,  1.92s/it]
 88%|████████▊ | 16324/18627 [9:15:14<1:18:41,  2.05s/it]
 88%|████████▊ | 16325/18627 [9:15:15<1:10:10,  1.83s/it]
 88%|████████▊ | 16326/18627 [9:15:17<1:15:47,  1.98s/it]
 88%|████████▊ | 16327/18627 [9:15:20<1:19:42,  2.08s/it]
 88%|████████▊ | 16328/18627 [9:15:21<1:12:30,  1.89s/it]
 88%|████��███▊ | 16329/18627 [9:15:23<1:17:11,  2.02s/it]
 88%|████████▊ | 16330/18627 [9:15:26<1:20:42,  2.11s/it]
                                                         
{'loss': 0.9701, 'grad_norm': 10.180704116821289, 'learning_rate': 1.9675729827720147e-07, 'epoch': 0.88}
+
 88%|████████▊ | 16330/18627 [9:15:26<1:20:42,  2.11s/it]
 88%|████████▊ | 16331/18627 [9:15:27<1:12:12,  1.89s/it]
 88%|████████▊ | 16332/18627 [9:15:29<1:16:50,  2.01s/it]
 88%|████████▊ | 16333/18627 [9:15:32<1:20:29,  2.11s/it]
 88%|████████▊ | 16334/18627 [9:15:34<1:23:09,  2.18s/it]
 88%|████████▊ | 16335/18627 [9:15:36<1:24:48,  2.22s/it]
 88%|████████▊ | 16336/18627 [9:15:39<1:25:47,  2.25s/it]
 88%|████████▊ | 16337/18627 [9:15:41<1:26:43,  2.27s/it]
 88%|████████▊ | 16338/18627 [9:15:43<1:27:37,  2.30s/it]
 88%|████████▊ | 16339/18627 [9:15:46<1:27:49,  2.30s/it]
 88%|████████▊ | 16340/18627 [9:15:48<1:28:09,  2.31s/it]
                                                         
{'loss': 0.753, 'grad_norm': 6.053878307342529, 'learning_rate': 1.9507044698777982e-07, 'epoch': 0.88}
+
 88%|████████▊ | 16340/18627 [9:15:48<1:28:09,  2.31s/it]
 88%|████████▊ | 16341/18627 [9:15:50<1:28:17,  2.32s/it]
 88%|████████▊ | 16342/18627 [9:15:53<1:28:43,  2.33s/it]
 88%|████████▊ | 16343/18627 [9:15:55<1:28:46,  2.33s/it]
 88%|████████▊ | 16344/18627 [9:15:57<1:28:51,  2.34s/it]
 88%|████████▊ | 16345/18627 [9:16:00<1:29:02,  2.34s/it]
 88%|████████▊ | 16346/18627 [9:16:01<1:17:58,  2.05s/it]
 88%|████████▊ | 16347/18627 [9:16:03<1:21:09,  2.14s/it]
 88%|████████▊ | 16348/18627 [9:16:05<1:12:55,  1.92s/it]
 88%|████████▊ | 16349/18627 [9:16:06<1:07:13,  1.77s/it]
 88%|████████▊ | 16350/18627 [9:16:09<1:13:39,  1.94s/it]
                                                         
{'loss': 1.24, 'grad_norm': 4.995319366455078, 'learning_rate': 1.9339056416344538e-07, 'epoch': 0.88}
+
 88%|████████▊ | 16350/18627 [9:16:09<1:13:39,  1.94s/it]
 88%|████████▊ | 16351/18627 [9:16:11<1:18:02,  2.06s/it]
 88%|████████▊ | 16352/18627 [9:16:13<1:20:48,  2.13s/it]
 88%|████████▊ | 16353/18627 [9:16:16<1:23:23,  2.20s/it]
 88%|████████▊ | 16354/18627 [9:16:18<1:24:18,  2.23s/it]
 88%|████████▊ | 16355/18627 [9:16:20<1:25:01,  2.25s/it]
 88%|████████▊ | 16356/18627 [9:16:23<1:25:44,  2.27s/it]
 88%|████████▊ | 16357/18627 [9:16:24<1:15:39,  2.00s/it]
 88%|████████▊ | 16358/18627 [9:16:26<1:19:21,  2.10s/it]
 88%|████████▊ | 16359/18627 [9:16:28<1:11:01,  1.88s/it]
 88%|████████▊ | 16360/18627 [9:16:30<1:16:44,  2.03s/it]
                                                         
{'loss': 1.0195, 'grad_norm': 5.0205912590026855, 'learning_rate': 1.917176548829669e-07, 'epoch': 0.88}
+
 88%|████████▊ | 16360/18627 [9:16:30<1:16:44,  2.03s/it]
 88%|████████▊ | 16361/18627 [9:16:31<1:05:26,  1.73s/it]
 88%|████████▊ | 16362/18627 [9:16:32<1:01:17,  1.62s/it]
 88%|████████▊ | 16363/18627 [9:16:35<1:09:14,  1.84s/it]
 88%|████████▊ | 16364/18627 [9:16:37<1:14:51,  1.98s/it]
 88%|████████▊ | 16365/18627 [9:16:39<1:18:29,  2.08s/it]
 88%|████████▊ | 16366/18627 [9:16:42<1:21:02,  2.15s/it]
 88%|████████▊ | 16367/18627 [9:16:44<1:22:42,  2.20s/it]
 88%|████████▊ | 16368/18627 [9:16:46<1:23:58,  2.23s/it]
 88%|████████▊ | 16369/18627 [9:16:48<1:13:38,  1.96s/it]
 88%|████████▊ | 16370/18627 [9:16:50<1:18:13,  2.08s/it]
                                                         
{'loss': 1.0292, 'grad_norm': 5.405204772949219, 'learning_rate': 1.9005172420403035e-07, 'epoch': 0.88}
+
 88%|████████▊ | 16370/18627 [9:16:50<1:18:13,  2.08s/it]
 88%|████████▊ | 16371/18627 [9:16:52<1:21:05,  2.16s/it]
 88%|████████▊ | 16372/18627 [9:16:54<1:12:01,  1.92s/it]
 88%|████████▊ | 16373/18627 [9:16:56<1:16:26,  2.03s/it]
 88%|████████▊ | 16374/18627 [9:16:57<1:09:00,  1.84s/it]
 88%|████████▊ | 16375/18627 [9:16:59<1:03:46,  1.70s/it]
 88%|████████▊ | 16376/18627 [9:17:00<59:59,  1.60s/it]  
 88%|████████▊ | 16377/18627 [9:17:02<1:08:28,  1.83s/it]
 88%|████████▊ | 16378/18627 [9:17:05<1:14:20,  1.98s/it]
 88%|████████▊ | 16379/18627 [9:17:06<1:07:37,  1.81s/it]
 88%|████████▊ | 16380/18627 [9:17:09<1:13:48,  1.97s/it]
                                                         
{'loss': 1.6501, 'grad_norm': 6.854423999786377, 'learning_rate': 1.883927771632238e-07, 'epoch': 0.88}
+
 88%|████████▊ | 16380/18627 [9:17:09<1:13:48,  1.97s/it]
 88%|████████▊ | 16381/18627 [9:17:10<1:07:24,  1.80s/it]
 88%|████████▊ | 16382/18627 [9:17:12<1:13:41,  1.97s/it]
 88%|████████▊ | 16383/18627 [9:17:15<1:17:43,  2.08s/it]
 88%|████████▊ | 16384/18627 [9:17:17<1:20:33,  2.15s/it]
 88%|████████▊ | 16385/18627 [9:17:19<1:22:29,  2.21s/it]
 88%|████████▊ | 16386/18627 [9:17:22<1:24:05,  2.25s/it]
 88%|████████▊ | 16387/18627 [9:17:24<1:25:02,  2.28s/it]
 88%|████████▊ | 16388/18627 [9:17:25<1:15:21,  2.02s/it]
 88%|████████▊ | 16389/18627 [9:17:28<1:18:32,  2.11s/it]
 88%|████████▊ | 16390/18627 [9:17:30<1:21:06,  2.18s/it]
                                                         
{'loss': 1.1233, 'grad_norm': 9.502025604248047, 'learning_rate': 1.867408187760214e-07, 'epoch': 0.88}
+
 88%|████████▊ | 16390/18627 [9:17:30<1:21:06,  2.18s/it]
 88%|████████▊ | 16391/18627 [9:17:31<1:12:11,  1.94s/it]
 88%|████████▊ | 16392/18627 [9:17:34<1:16:19,  2.05s/it]
 88%|████████▊ | 16393/18627 [9:17:36<1:19:15,  2.13s/it]
 88%|████████▊ | 16394/18627 [9:17:38<1:21:22,  2.19s/it]
 88%|████████▊ | 16395/18627 [9:17:41<1:22:56,  2.23s/it]
 88%|████████▊ | 16396/18627 [9:17:42<1:13:20,  1.97s/it]
 88%|████████▊ | 16397/18627 [9:17:47<1:47:55,  2.90s/it]
 88%|████████▊ | 16398/18627 [9:17:49<1:30:38,  2.44s/it]
 88%|████████▊ | 16399/18627 [9:17:51<1:29:19,  2.41s/it]
 88%|████████▊ | 16400/18627 [9:17:53<1:28:14,  2.38s/it]
                                                         
{'loss': 1.2102, 'grad_norm': 6.623039722442627, 'learning_rate': 1.8509585403676756e-07, 'epoch': 0.88}
+
 88%|████████▊ | 16400/18627 [9:17:53<1:28:14,  2.38s/it]
 88%|████████▊ | 16401/18627 [9:17:56<1:27:56,  2.37s/it]
 88%|████████▊ | 16402/18627 [9:17:58<1:27:05,  2.35s/it]
 88%|████████▊ | 16403/18627 [9:18:00<1:26:38,  2.34s/it]
 88%|████████▊ | 16404/18627 [9:18:02<1:26:19,  2.33s/it]
 88%|████████▊ | 16405/18627 [9:18:05<1:26:09,  2.33s/it]
 88%|████████▊ | 16406/18627 [9:18:06<1:11:49,  1.94s/it]
 88%|████████▊ | 16407/18627 [9:18:08<1:16:14,  2.06s/it]
 88%|████████▊ | 16408/18627 [9:18:11<1:19:19,  2.14s/it]
 88%|████████▊ | 16409/18627 [9:18:13<1:21:38,  2.21s/it]
 88%|████████▊ | 16410/18627 [9:18:15<1:23:02,  2.25s/it]
                                                         
{'loss': 0.563, 'grad_norm': 6.781182289123535, 'learning_rate': 1.8345788791866548e-07, 'epoch': 0.88}
+
 88%|████████▊ | 16410/18627 [9:18:15<1:23:02,  2.25s/it]
 88%|████████▊ | 16411/18627 [9:18:20<1:49:02,  2.95s/it]
 88%|████████▊ | 16412/18627 [9:18:22<1:41:57,  2.76s/it]
 88%|████████▊ | 16413/18627 [9:18:23<1:26:16,  2.34s/it]
 88%|████████▊ | 16414/18627 [9:18:26<1:27:12,  2.36s/it]
 88%|████████▊ | 16415/18627 [9:18:28<1:26:43,  2.35s/it]
 88%|████████▊ | 16416/18627 [9:18:31<1:26:18,  2.34s/it]
 88%|████████▊ | 16417/18627 [9:18:33<1:26:02,  2.34s/it]
 88%|████████▊ | 16418/18627 [9:18:35<1:25:48,  2.33s/it]
 88%|████████▊ | 16419/18627 [9:18:38<1:25:55,  2.34s/it]
 88%|████████▊ | 16420/18627 [9:18:40<1:26:00,  2.34s/it]
                                                         
{'loss': 0.7479, 'grad_norm': 7.516454219818115, 'learning_rate': 1.8182692537375673e-07, 'epoch': 0.88}
+
 88%|████████▊ | 16420/18627 [9:18:40<1:26:00,  2.34s/it]
 88%|████████▊ | 16421/18627 [9:18:42<1:25:36,  2.33s/it]
 88%|████████▊ | 16422/18627 [9:18:45<1:25:52,  2.34s/it]
 88%|████████▊ | 16423/18627 [9:18:47<1:25:36,  2.33s/it]
 88%|████████▊ | 16424/18627 [9:18:49<1:25:43,  2.33s/it]
 88%|████████▊ | 16425/18627 [9:18:52<1:25:39,  2.33s/it]
 88%|████████▊ | 16426/18627 [9:18:53<1:11:56,  1.96s/it]
 88%|████████▊ | 16427/18627 [9:18:55<1:16:31,  2.09s/it]
 88%|████████▊ | 16428/18627 [9:18:56<1:08:42,  1.87s/it]
 88%|████████▊ | 16429/18627 [9:18:59<1:13:38,  2.01s/it]
 88%|████████▊ | 16430/18627 [9:19:01<1:17:04,  2.10s/it]
                                                         
{'loss': 0.7812, 'grad_norm': 5.878048896789551, 'learning_rate': 1.8020297133291038e-07, 'epoch': 0.88}
+
 88%|████████▊ | 16430/18627 [9:19:01<1:17:04,  2.10s/it]
 88%|████████▊ | 16431/18627 [9:19:02<1:08:49,  1.88s/it]
 88%|████████▊ | 16432/18627 [9:19:05<1:13:27,  2.01s/it]
 88%|████████▊ | 16433/18627 [9:19:07<1:16:33,  2.09s/it]
 88%|████████▊ | 16434/18627 [9:19:09<1:19:04,  2.16s/it]
 88%|████████▊ | 16435/18627 [9:19:12<1:20:49,  2.21s/it]
 88%|████████▊ | 16436/18627 [9:19:14<1:21:58,  2.25s/it]
 88%|████████▊ | 16437/18627 [9:19:15<1:12:04,  1.97s/it]
 88%|████████▊ | 16438/18627 [9:19:17<1:05:38,  1.80s/it]
 88%|████████▊ | 16439/18627 [9:19:18<1:00:54,  1.67s/it]
 88%|████████▊ | 16440/18627 [9:19:23<1:35:29,  2.62s/it]
                                                         
{'loss': 1.3748, 'grad_norm': 4.865749835968018, 'learning_rate': 1.785860307058071e-07, 'epoch': 0.88}
+
 88%|████████▊ | 16440/18627 [9:19:23<1:35:29,  2.62s/it]
 88%|████████▊ | 16441/18627 [9:19:25<1:32:11,  2.53s/it]
 88%|████████▊ | 16442/18627 [9:19:28<1:29:58,  2.47s/it]
 88%|████████▊ | 16443/18627 [9:19:29<1:19:36,  2.19s/it]
 88%|████████▊ | 16444/18627 [9:19:31<1:11:28,  1.96s/it]
 88%|████████▊ | 16445/18627 [9:19:33<1:15:05,  2.06s/it]
 88%|████████▊ | 16446/18627 [9:19:34<1:07:07,  1.85s/it]
 88%|████████▊ | 16447/18627 [9:19:36<1:12:09,  1.99s/it]
 88%|████████▊ | 16448/18627 [9:19:39<1:15:49,  2.09s/it]
 88%|████████▊ | 16449/18627 [9:19:41<1:18:54,  2.17s/it]
 88%|████████▊ | 16450/18627 [9:19:43<1:20:27,  2.22s/it]
                                                         
{'loss': 1.1744, 'grad_norm': 5.623748302459717, 'learning_rate': 1.7697610838092321e-07, 'epoch': 0.88}
+
 88%|████████▊ | 16450/18627 [9:19:43<1:20:27,  2.22s/it]
 88%|████████▊ | 16451/18627 [9:19:46<1:21:44,  2.25s/it]
 88%|████████▊ | 16452/18627 [9:19:48<1:22:05,  2.26s/it]
 88%|████████▊ | 16453/18627 [9:19:49<1:12:22,  2.00s/it]
 88%|████████▊ | 16454/18627 [9:19:52<1:16:00,  2.10s/it]
 88%|████████▊ | 16455/18627 [9:19:54<1:18:31,  2.17s/it]
 88%|████████▊ | 16456/18627 [9:19:57<1:21:06,  2.24s/it]
 88%|████████▊ | 16457/18627 [9:19:59<1:21:54,  2.26s/it]
 88%|████████▊ | 16458/18627 [9:20:01<1:21:44,  2.26s/it]
 88%|████████▊ | 16459/18627 [9:20:03<1:22:11,  2.27s/it]
 88%|████████▊ | 16460/18627 [9:20:06<1:22:19,  2.28s/it]
                                                         
{'loss': 0.7838, 'grad_norm': 7.565742015838623, 'learning_rate': 1.7537320922551697e-07, 'epoch': 0.88}
+
 88%|████████▊ | 16460/18627 [9:20:06<1:22:19,  2.28s/it]
 88%|████████▊ | 16461/18627 [9:20:08<1:22:21,  2.28s/it]
 88%|████████▊ | 16462/18627 [9:20:10<1:22:26,  2.28s/it]
 88%|████████▊ | 16463/18627 [9:20:12<1:11:42,  1.99s/it]
 88%|████████▊ | 16464/18627 [9:20:13<1:04:22,  1.79s/it]
 88%|████████▊ | 16465/18627 [9:20:15<1:09:36,  1.93s/it]
 88%|████████▊ | 16466/18627 [9:20:17<1:13:10,  2.03s/it]
 88%|████████▊ | 16467/18627 [9:20:19<1:05:16,  1.81s/it]
 88%|████████▊ | 16468/18627 [9:20:21<1:10:06,  1.95s/it]
 88%|████████▊ | 16469/18627 [9:20:23<1:13:26,  2.04s/it]
 88%|████████▊ | 16470/18627 [9:20:26<1:16:41,  2.13s/it]
                                                         
{'loss': 1.2782, 'grad_norm': 7.496234893798828, 'learning_rate': 1.737773380856142e-07, 'epoch': 0.88}
+
 88%|████████▊ | 16470/18627 [9:20:26<1:16:41,  2.13s/it]
 88%|████████▊ | 16471/18627 [9:20:28<1:18:14,  2.18s/it]
 88%|████████▊ | 16472/18627 [9:20:29<1:09:00,  1.92s/it]
 88%|████████▊ | 16473/18627 [9:20:32<1:13:06,  2.04s/it]
 88%|████████▊ | 16474/18627 [9:20:34<1:15:39,  2.11s/it]
 88%|████████▊ | 16475/18627 [9:20:36<1:17:47,  2.17s/it]
 88%|████████▊ | 16476/18627 [9:20:40<1:37:21,  2.72s/it]
 88%|████████▊ | 16477/18627 [9:20:42<1:32:36,  2.58s/it]
 88%|████████▊ | 16478/18627 [9:20:45<1:29:08,  2.49s/it]
 88%|████████▊ | 16479/18627 [9:20:47<1:26:46,  2.42s/it]
 88%|████████▊ | 16480/18627 [9:20:49<1:25:21,  2.39s/it]
                                                         
{'loss': 0.7694, 'grad_norm': 8.178376197814941, 'learning_rate': 1.7218849978599127e-07, 'epoch': 0.88}
+
 88%|████████▊ | 16480/18627 [9:20:49<1:25:21,  2.39s/it]
 88%|████████▊ | 16481/18627 [9:20:51<1:13:51,  2.06s/it]
 88%|████████▊ | 16482/18627 [9:20:52<1:05:38,  1.84s/it]
 88%|████████▊ | 16483/18627 [9:20:54<1:10:17,  1.97s/it]
 88%|████████▊ | 16484/18627 [9:20:56<1:13:50,  2.07s/it]
 89%|████████▊ | 16485/18627 [9:20:59<1:17:18,  2.17s/it]
 89%|████████▊ | 16486/18627 [9:21:01<1:19:08,  2.22s/it]
 89%|████████▊ | 16487/18627 [9:21:03<1:19:56,  2.24s/it]
 89%|████████▊ | 16488/18627 [9:21:05<1:10:43,  1.98s/it]
 89%|████████▊ | 16489/18627 [9:21:06<1:04:45,  1.82s/it]
 89%|████████▊ | 16490/18627 [9:21:09<1:10:25,  1.98s/it]
                                                         
{'loss': 1.4795, 'grad_norm': 7.756636142730713, 'learning_rate': 1.70606699130165e-07, 'epoch': 0.89}
+
 89%|████████▊ | 16490/18627 [9:21:09<1:10:25,  1.98s/it]
 89%|████████▊ | 16491/18627 [9:21:11<1:14:09,  2.08s/it]
 89%|████████▊ | 16492/18627 [9:21:12<1:06:24,  1.87s/it]
 89%|████████▊ | 16493/18627 [9:21:15<1:11:00,  2.00s/it]
 89%|████████▊ | 16494/18627 [9:21:17<1:13:59,  2.08s/it]
 89%|████████▊ | 16495/18627 [9:21:19<1:16:43,  2.16s/it]
 89%|████████▊ | 16496/18627 [9:21:21<1:08:44,  1.94s/it]
 89%|████████▊ | 16497/18627 [9:21:23<1:12:24,  2.04s/it]
 89%|████████▊ | 16498/18627 [9:21:24<1:05:46,  1.85s/it]
 89%|████████▊ | 16499/18627 [9:21:26<1:01:20,  1.73s/it]
 89%|████████▊ | 16500/18627 [9:21:28<1:07:47,  1.91s/it]
                                                         
{'loss': 1.4667, 'grad_norm': 6.48458194732666, 'learning_rate': 1.6903194090037267e-07, 'epoch': 0.89}
+
 89%|████████▊ | 16500/18627 [9:21:28<1:07:47,  1.91s/it]
 89%|████████▊ | 16501/18627 [9:21:29<1:01:57,  1.75s/it]
 89%|████████▊ | 16502/18627 [9:21:31<58:06,  1.64s/it]  
 89%|████████▊ | 16503/18627 [9:21:33<1:05:13,  1.84s/it]
 89%|████████▊ | 16504/18627 [9:21:36<1:11:07,  2.01s/it]
 89%|████████▊ | 16505/18627 [9:21:37<1:04:19,  1.82s/it]
 89%|████████▊ | 16506/18627 [9:21:39<1:10:21,  1.99s/it]
 89%|████████▊ | 16507/18627 [9:21:42<1:14:16,  2.10s/it]
 89%|████████▊ | 16508/18627 [9:21:44<1:16:59,  2.18s/it]
 89%|████████▊ | 16509/18627 [9:21:46<1:18:48,  2.23s/it]
 89%|████████▊ | 16510/18627 [9:21:48<1:10:10,  1.99s/it]
                                                         
{'loss': 1.3209, 'grad_norm': 14.310111045837402, 'learning_rate': 1.6746422985756155e-07, 'epoch': 0.89}
+
 89%|████████▊ | 16510/18627 [9:21:48<1:10:10,  1.99s/it]
 89%|████████▊ | 16511/18627 [9:21:50<1:14:08,  2.10s/it]
 89%|████████▊ | 16512/18627 [9:21:52<1:06:30,  1.89s/it]
 89%|████████▊ | 16513/18627 [9:21:54<1:11:34,  2.03s/it]
 89%|████████▊ | 16514/18627 [9:21:56<1:15:14,  2.14s/it]
 89%|████████▊ | 16515/18627 [9:21:59<1:17:27,  2.20s/it]
 89%|████████▊ | 16516/18627 [9:22:01<1:18:43,  2.24s/it]
 89%|████████▊ | 16517/18627 [9:22:03<1:19:46,  2.27s/it]
 89%|████████▊ | 16518/18627 [9:22:06<1:20:07,  2.28s/it]
 89%|████████▊ | 16519/18627 [9:22:08<1:20:13,  2.28s/it]
 89%|████████▊ | 16520/18627 [9:22:10<1:20:51,  2.30s/it]
                                                         
{'loss': 0.7611, 'grad_norm': 7.360464096069336, 'learning_rate': 1.6590357074137308e-07, 'epoch': 0.89}
+
 89%|████████▊ | 16520/18627 [9:22:10<1:20:51,  2.30s/it]
 89%|████████▊ | 16521/18627 [9:22:13<1:21:31,  2.32s/it]
 89%|████████▊ | 16522/18627 [9:22:14<1:11:26,  2.04s/it]
 89%|████████▊ | 16523/18627 [9:22:15<1:05:03,  1.86s/it]
 89%|████████▊ | 16524/18627 [9:22:18<1:09:36,  1.99s/it]
 89%|████████▊ | 16525/18627 [9:22:20<1:13:38,  2.10s/it]
 89%|████████▊ | 16526/18627 [9:22:23<1:16:11,  2.18s/it]
 89%|████████▊ | 16527/18627 [9:22:25<1:18:00,  2.23s/it]
 89%|████████▊ | 16528/18627 [9:22:26<1:09:15,  1.98s/it]
 89%|████████▊ | 16529/18627 [9:22:29<1:13:03,  2.09s/it]
 89%|████████▊ | 16530/18627 [9:22:31<1:15:44,  2.17s/it]
                                                         
{'loss': 1.2491, 'grad_norm': 5.160701751708984, 'learning_rate': 1.643499682701291e-07, 'epoch': 0.89}
+
 89%|████████▊ | 16530/18627 [9:22:31<1:15:44,  2.17s/it]
 89%|████████▊ | 16531/18627 [9:22:33<1:17:48,  2.23s/it]
 89%|████████▉ | 16532/18627 [9:22:36<1:19:18,  2.27s/it]
 89%|████████▉ | 16533/18627 [9:22:38<1:19:25,  2.28s/it]
 89%|████████▉ | 16534/18627 [9:22:40<1:19:50,  2.29s/it]
 89%|████████▉ | 16535/18627 [9:22:43<1:19:40,  2.29s/it]
 89%|████████▉ | 16536/18627 [9:22:45<1:20:46,  2.32s/it]
 89%|████████▉ | 16537/18627 [9:22:47<1:21:20,  2.33s/it]
 89%|████████▉ | 16538/18627 [9:22:50<1:21:01,  2.33s/it]
 89%|████████▉ | 16539/18627 [9:22:51<1:10:50,  2.04s/it]
 89%|████████▉ | 16540/18627 [9:22:52<1:03:59,  1.84s/it]
                                                         
{'loss': 0.9898, 'grad_norm': 13.49434757232666, 'learning_rate': 1.628034271408166e-07, 'epoch': 0.89}
+
 89%|████████▉ | 16540/18627 [9:22:52<1:03:59,  1.84s/it]
 89%|████████▉ | 16541/18627 [9:22:55<1:09:14,  1.99s/it]
 89%|████████▉ | 16542/18627 [9:22:56<1:02:02,  1.79s/it]
 89%|████████▉ | 16543/18627 [9:22:58<1:07:30,  1.94s/it]
 89%|████████▉ | 16544/18627 [9:23:00<1:01:04,  1.76s/it]
 89%|████████▉ | 16545/18627 [9:23:02<1:06:42,  1.92s/it]
 89%|████████▉ | 16546/18627 [9:23:04<1:10:57,  2.05s/it]
 89%|████████▉ | 16547/18627 [9:23:07<1:13:25,  2.12s/it]
 89%|████████▉ | 16548/18627 [9:23:09<1:15:59,  2.19s/it]
 89%|████████▉ | 16549/18627 [9:23:11<1:17:42,  2.24s/it]
 89%|████████▉ | 16550/18627 [9:23:14<1:18:33,  2.27s/it]
                                                         
{'loss': 0.9863, 'grad_norm': 11.073318481445312, 'learning_rate': 1.612639520290743e-07, 'epoch': 0.89}
+
 89%|████████▉ | 16550/18627 [9:23:14<1:18:33,  2.27s/it]
 89%|████████▉ | 16551/18627 [9:23:16<1:19:27,  2.30s/it]
 89%|████████▉ | 16552/18627 [9:23:17<1:09:54,  2.02s/it]
 89%|████████▉ | 16553/18627 [9:23:20<1:13:21,  2.12s/it]
 89%|████████▉ | 16554/18627 [9:23:22<1:15:38,  2.19s/it]
 89%|████████▉ | 16555/18627 [9:23:23<1:07:23,  1.95s/it]
 89%|████████▉ | 16556/18627 [9:23:26<1:11:13,  2.06s/it]
 89%|████████▉ | 16557/18627 [9:23:28<1:13:38,  2.13s/it]
 89%|████████▉ | 16558/18627 [9:23:30<1:06:21,  1.92s/it]
 89%|████████▉ | 16559/18627 [9:23:32<1:10:50,  2.06s/it]
 89%|████████▉ | 16560/18627 [9:23:34<1:13:55,  2.15s/it]
                                                         
{'loss': 1.1984, 'grad_norm': 9.222113609313965, 'learning_rate': 1.5973154758917808e-07, 'epoch': 0.89}
+
 89%|████████▉ | 16560/18627 [9:23:34<1:13:55,  2.15s/it]
 89%|████████▉ | 16561/18627 [9:23:36<1:06:36,  1.93s/it]
 89%|████████▉ | 16562/18627 [9:23:37<1:00:54,  1.77s/it]
 89%|████████▉ | 16563/18627 [9:23:39<1:06:59,  1.95s/it]
 89%|████████▉ | 16564/18627 [9:23:41<1:00:55,  1.77s/it]
 89%|████████▉ | 16565/18627 [9:23:42<57:15,  1.67s/it]  
 89%|████████▉ | 16566/18627 [9:23:45<1:04:01,  1.86s/it]
 89%|████████▉ | 16567/18627 [9:23:47<1:08:51,  2.01s/it]
 89%|████████▉ | 16568/18627 [9:23:49<1:12:16,  2.11s/it]
 89%|████████▉ | 16569/18627 [9:23:52<1:14:46,  2.18s/it]
 89%|████████▉ | 16570/18627 [9:23:54<1:16:35,  2.23s/it]
                                                         
{'loss': 1.428, 'grad_norm': 4.880134582519531, 'learning_rate': 1.5820621845402756e-07, 'epoch': 0.89}
+
 89%|████████▉ | 16570/18627 [9:23:54<1:16:35,  2.23s/it]
 89%|████████▉ | 16571/18627 [9:23:55<1:08:09,  1.99s/it]
 89%|████████▉ | 16572/18627 [9:23:58<1:11:35,  2.09s/it]
 89%|████████▉ | 16573/18627 [9:23:59<1:04:21,  1.88s/it]
 89%|████████▉ | 16574/18627 [9:24:01<1:09:27,  2.03s/it]
 89%|████████▉ | 16575/18627 [9:24:04<1:12:45,  2.13s/it]
 89%|████████▉ | 16576/18627 [9:24:05<1:04:54,  1.90s/it]
 89%|████████▉ | 16577/18627 [9:24:08<1:09:32,  2.04s/it]
 89%|████████▉ | 16578/18627 [9:24:09<1:00:30,  1.77s/it]
 89%|████████▉ | 16579/18627 [9:24:10<56:57,  1.67s/it]  
 89%|████████▉ | 16580/18627 [9:24:13<1:04:20,  1.89s/it]
                                                         
{'loss': 1.5253, 'grad_norm': 6.376257419586182, 'learning_rate': 1.5668796923513224e-07, 'epoch': 0.89}
+
 89%|████████▉ | 16580/18627 [9:24:13<1:04:20,  1.89s/it]
 89%|████████▉ | 16581/18627 [9:24:14<59:10,  1.74s/it]  
 89%|████████▉ | 16582/18627 [9:24:16<1:05:11,  1.91s/it]
 89%|████████▉ | 16583/18627 [9:24:19<1:09:53,  2.05s/it]
 89%|████████▉ | 16584/18627 [9:24:21<1:13:03,  2.15s/it]
 89%|████████▉ | 16585/18627 [9:24:22<1:05:04,  1.91s/it]
 89%|████████▉ | 16586/18627 [9:24:25<1:08:58,  2.03s/it]
 89%|████████▉ | 16587/18627 [9:24:27<1:11:59,  2.12s/it]
 89%|████████▉ | 16588/18627 [9:24:29<1:14:13,  2.18s/it]
 89%|████████▉ | 16589/18627 [9:24:30<1:02:56,  1.85s/it]
 89%|████████▉ | 16590/18627 [9:24:33<1:07:57,  2.00s/it]
                                                         
{'loss': 0.9487, 'grad_norm': 6.526719093322754, 'learning_rate': 1.551768045225946e-07, 'epoch': 0.89}
+
 89%|████████▉ | 16590/18627 [9:24:33<1:07:57,  2.00s/it]
 89%|████████▉ | 16591/18627 [9:24:35<1:11:55,  2.12s/it]
 89%|████████▉ | 16592/18627 [9:24:37<1:04:19,  1.90s/it]
 89%|████████▉ | 16593/18627 [9:24:38<59:04,  1.74s/it]  
 89%|████████▉ | 16594/18627 [9:24:40<1:05:15,  1.93s/it]
 89%|████████▉ | 16595/18627 [9:24:42<59:56,  1.77s/it]  
 89%|████████▉ | 16596/18627 [9:24:44<1:06:09,  1.95s/it]
 89%|████████▉ | 16597/18627 [9:24:46<1:10:20,  2.08s/it]
 89%|████████▉ | 16598/18627 [9:24:49<1:13:25,  2.17s/it]
 89%|████████▉ | 16599/18627 [9:24:50<1:05:59,  1.95s/it]
 89%|████████▉ | 16600/18627 [9:24:53<1:09:57,  2.07s/it]
                                                         
{'loss': 1.4526, 'grad_norm': 5.535374164581299, 'learning_rate': 1.53672728885102e-07, 'epoch': 0.89}
+
 89%|████████▉ | 16600/18627 [9:24:53<1:09:57,  2.07s/it]
 89%|████████▉ | 16601/18627 [9:24:55<1:13:28,  2.18s/it]
 89%|████████▉ | 16602/18627 [9:24:57<1:15:08,  2.23s/it]
 89%|████████▉ | 16603/18627 [9:25:00<1:16:14,  2.26s/it]
 89%|████████▉ | 16604/18627 [9:25:01<1:07:26,  2.00s/it]
 89%|████████▉ | 16605/18627 [9:25:03<1:10:16,  2.09s/it]
 89%|████████▉ | 16606/18627 [9:25:05<1:02:35,  1.86s/it]
 89%|████████▉ | 16607/18627 [9:25:07<1:07:08,  1.99s/it]
 89%|████████▉ | 16608/18627 [9:25:09<1:10:31,  2.10s/it]
 89%|████████▉ | 16609/18627 [9:25:11<1:03:16,  1.88s/it]
 89%|████████▉ | 16610/18627 [9:25:13<1:08:40,  2.04s/it]
                                                         
{'loss': 1.1913, 'grad_norm': 5.568209171295166, 'learning_rate': 1.5217574686990644e-07, 'epoch': 0.89}
+
 89%|████████▉ | 16610/18627 [9:25:13<1:08:40,  2.04s/it]
 89%|████████▉ | 16611/18627 [9:25:15<1:11:19,  2.12s/it]
 89%|████████▉ | 16612/18627 [9:25:18<1:13:13,  2.18s/it]
 89%|████████▉ | 16613/18627 [9:25:20<1:14:45,  2.23s/it]
 89%|████████▉ | 16614/18627 [9:25:22<1:15:38,  2.25s/it]
 89%|████████▉ | 16615/18627 [9:25:25<1:16:11,  2.27s/it]
 89%|████████▉ | 16616/18627 [9:25:27<1:16:07,  2.27s/it]
 89%|████████▉ | 16617/18627 [9:25:29<1:16:16,  2.28s/it]
 89%|████████▉ | 16618/18627 [9:25:32<1:16:28,  2.28s/it]
 89%|████████▉ | 16619/18627 [9:25:34<1:16:31,  2.29s/it]
 89%|████████▉ | 16620/18627 [9:25:36<1:16:24,  2.28s/it]
                                                         
{'loss': 0.5533, 'grad_norm': 5.700722694396973, 'learning_rate': 1.5068586300281573e-07, 'epoch': 0.89}
+
 89%|████████▉ | 16620/18627 [9:25:36<1:16:24,  2.28s/it]
 89%|████████▉ | 16621/18627 [9:25:38<1:16:25,  2.29s/it]
 89%|████████▉ | 16622/18627 [9:25:41<1:16:21,  2.29s/it]
 89%|████████▉ | 16623/18627 [9:25:43<1:16:56,  2.30s/it]
 89%|████████▉ | 16624/18627 [9:25:45<1:17:25,  2.32s/it]
 89%|████████▉ | 16625/18627 [9:25:48<1:17:45,  2.33s/it]
 89%|████████▉ | 16626/18627 [9:25:50<1:18:02,  2.34s/it]
 89%|████████▉ | 16627/18627 [9:25:53<1:18:34,  2.36s/it]
 89%|████████▉ | 16628/18627 [9:25:55<1:19:44,  2.39s/it]
 89%|████████▉ | 16629/18627 [9:25:57<1:19:24,  2.38s/it]
 89%|████████▉ | 16630/18627 [9:26:00<1:18:59,  2.37s/it]
                                                         
{'loss': 0.6182, 'grad_norm': 7.555735111236572, 'learning_rate': 1.492030817881776e-07, 'epoch': 0.89}
+
 89%|████████▉ | 16630/18627 [9:26:00<1:18:59,  2.37s/it]
 89%|████████▉ | 16631/18627 [9:26:02<1:19:21,  2.39s/it]
 89%|████████▉ | 16632/18627 [9:26:05<1:19:17,  2.38s/it]
 89%|████████▉ | 16633/18627 [9:26:07<1:18:55,  2.37s/it]
 89%|████████▉ | 16634/18627 [9:26:08<1:09:57,  2.11s/it]
 89%|███████��▉ | 16635/18627 [9:26:11<1:12:37,  2.19s/it]
 89%|████████▉ | 16636/18627 [9:26:13<1:15:26,  2.27s/it]
 89%|████████▉ | 16637/18627 [9:26:16<1:16:16,  2.30s/it]
 89%|████████▉ | 16638/18627 [9:26:17<1:06:34,  2.01s/it]
 89%|████████▉ | 16639/18627 [9:26:19<1:10:42,  2.13s/it]
 89%|████████▉ | 16640/18627 [9:26:21<1:02:43,  1.89s/it]
                                                         
{'loss': 1.2319, 'grad_norm': 12.773331642150879, 'learning_rate': 1.4772740770886584e-07, 'epoch': 0.89}
+
 89%|████████▉ | 16640/18627 [9:26:21<1:02:43,  1.89s/it]
 89%|████████▉ | 16641/18627 [9:26:23<1:07:33,  2.04s/it]
 89%|████████▉ | 16642/18627 [9:26:25<1:10:34,  2.13s/it]
 89%|████████▉ | 16643/18627 [9:26:28<1:12:19,  2.19s/it]
 89%|████████▉ | 16644/18627 [9:26:30<1:13:20,  2.22s/it]
 89%|████████▉ | 16645/18627 [9:26:32<1:14:21,  2.25s/it]
 89%|████████▉ | 16646/18627 [9:26:35<1:15:13,  2.28s/it]
 89%|████████▉ | 16647/18627 [9:26:37<1:15:39,  2.29s/it]
 89%|████████▉ | 16648/18627 [9:26:38<1:05:53,  2.00s/it]
 89%|████████▉ | 16649/18627 [9:26:40<58:33,  1.78s/it]  
 89%|████████▉ | 16650/18627 [9:26:42<1:03:41,  1.93s/it]
                                                         
{'loss': 0.9853, 'grad_norm': 4.722620964050293, 'learning_rate': 1.462588452262681e-07, 'epoch': 0.89}
+
 89%|████████▉ | 16650/18627 [9:26:42<1:03:41,  1.93s/it]
 89%|████████▉ | 16651/18627 [9:26:44<1:07:25,  2.05s/it]
 89%|████████▉ | 16652/18627 [9:26:46<1:09:54,  2.12s/it]
 89%|████████▉ | 16653/18627 [9:26:49<1:11:48,  2.18s/it]
 89%|████████▉ | 16654/18627 [9:26:50<1:03:30,  1.93s/it]
 89%|████████▉ | 16655/18627 [9:26:52<1:07:08,  2.04s/it]
 89%|████████▉ | 16656/18627 [9:26:54<1:00:21,  1.84s/it]
 89%|████████▉ | 16657/18627 [9:26:56<1:05:02,  1.98s/it]
 89%|████████▉ | 16658/18627 [9:26:58<1:08:41,  2.09s/it]
 89%|████████▉ | 16659/18627 [9:27:01<1:10:59,  2.16s/it]
 89%|████████▉ | 16660/18627 [9:27:03<1:12:42,  2.22s/it]
                                                         
{'loss': 1.0407, 'grad_norm': 5.152465343475342, 'learning_rate': 1.4479739878027117e-07, 'epoch': 0.89}
+
 89%|████████▉ | 16660/18627 [9:27:03<1:12:42,  2.22s/it]
 89%|████████▉ | 16661/18627 [9:27:04<1:03:52,  1.95s/it]
 89%|████████▉ | 16662/18627 [9:27:07<1:07:59,  2.08s/it]
 89%|████████▉ | 16663/18627 [9:27:09<1:10:43,  2.16s/it]
 89%|████████▉ | 16664/18627 [9:27:11<1:02:57,  1.92s/it]
 89%|████████▉ | 16665/18627 [9:27:13<1:06:59,  2.05s/it]
 89%|████████▉ | 16666/18627 [9:27:15<1:09:52,  2.14s/it]
 89%|████████▉ | 16667/18627 [9:27:18<1:11:57,  2.20s/it]
 89%|████████▉ | 16668/18627 [9:27:20<1:15:13,  2.30s/it]
 89%|████████▉ | 16669/18627 [9:27:22<1:06:30,  2.04s/it]
 89%|████████▉ | 16670/18627 [9:27:23<1:00:04,  1.84s/it]
                                                         
{'loss': 1.3438, 'grad_norm': 12.55517292022705, 'learning_rate': 1.4334307278924707e-07, 'epoch': 0.89}
+
 89%|████████▉ | 16670/18627 [9:27:23<1:00:04,  1.84s/it]
 89%|████████▉ | 16671/18627 [9:27:24<56:17,  1.73s/it]  
 90%|████████▉ | 16672/18627 [9:27:25<49:47,  1.53s/it]
 90%|████████▉ | 16673/18627 [9:27:28<57:31,  1.77s/it]
 90%|████████▉ | 16674/18627 [9:27:30<1:03:07,  1.94s/it]
 90%|████████▉ | 16675/18627 [9:27:33<1:07:13,  2.07s/it]
 90%|████████▉ | 16676/18627 [9:27:35<1:10:15,  2.16s/it]
 90%|████████▉ | 16677/18627 [9:27:37<1:12:14,  2.22s/it]
 90%|████████▉ | 16678/18627 [9:27:40<1:13:30,  2.26s/it]
 90%|████████▉ | 16679/18627 [9:27:42<1:14:19,  2.29s/it]
 90%|████████▉ | 16680/18627 [9:27:43<1:01:42,  1.90s/it]
                                                         
{'loss': 0.9041, 'grad_norm': 6.65220308303833, 'learning_rate': 1.418958716500432e-07, 'epoch': 0.9}
+
 90%|████████▉ | 16680/18627 [9:27:43<1:01:42,  1.90s/it]
 90%|████████▉ | 16681/18627 [9:27:45<1:06:00,  2.04s/it]
 90%|████████▉ | 16682/18627 [9:27:47<59:46,  1.84s/it]  
 90%|████████▉ | 16683/18627 [9:27:49<1:04:35,  1.99s/it]
 90%|████████▉ | 16684/18627 [9:27:51<1:07:40,  2.09s/it]
 90%|████████▉ | 16685/18627 [9:27:54<1:09:36,  2.15s/it]
 90%|████████▉ | 16686/18627 [9:27:56<1:11:45,  2.22s/it]
 90%|████████▉ | 16687/18627 [9:27:58<1:13:05,  2.26s/it]
 90%|████████▉ | 16688/18627 [9:28:01<1:13:59,  2.29s/it]
 90%|████████▉ | 16689/18627 [9:28:02<1:02:55,  1.95s/it]
 90%|████████▉ | 16690/18627 [9:28:03<57:03,  1.77s/it]  
                                                       
{'loss': 0.9812, 'grad_norm': 13.85601806640625, 'learning_rate': 1.404557997379638e-07, 'epoch': 0.9}
+
 90%|████████▉ | 16690/18627 [9:28:03<57:03,  1.77s/it]
 90%|████████▉ | 16691/18627 [9:28:06<1:02:45,  1.94s/it]
 90%|████████▉ | 16692/18627 [9:28:08<1:06:34,  2.06s/it]
 90%|████████▉ | 16693/18627 [9:28:10<1:08:59,  2.14s/it]
 90%|████████▉ | 16694/18627 [9:28:13<1:11:12,  2.21s/it]
 90%|████████▉ | 16695/18627 [9:28:14<1:03:00,  1.96s/it]
 90%|████████▉ | 16696/18627 [9:28:16<1:06:31,  2.07s/it]
 90%|████████▉ | 16697/18627 [9:28:19<1:08:50,  2.14s/it]
 90%|████████▉ | 16698/18627 [9:28:21<1:10:17,  2.19s/it]
 90%|████████▉ | 16699/18627 [9:28:23<1:11:44,  2.23s/it]
 90%|████████▉ | 16700/18627 [9:28:26<1:13:09,  2.28s/it]
                                                         
{'loss': 0.7958, 'grad_norm': 5.970524311065674, 'learning_rate': 1.390228614067607e-07, 'epoch': 0.9}
+
 90%|████████▉ | 16700/18627 [9:28:26<1:13:09,  2.28s/it]
 90%|████████▉ | 16701/18627 [9:28:28<1:13:46,  2.30s/it]
 90%|████████▉ | 16702/18627 [9:28:29<1:04:44,  2.02s/it]
 90%|████████▉ | 16703/18627 [9:28:32<1:07:55,  2.12s/it]
 90%|████████▉ | 16704/18627 [9:28:34<1:10:33,  2.20s/it]
 90%|████████▉ | 16705/18627 [9:28:36<1:11:50,  2.24s/it]
 90%|████████▉ | 16706/18627 [9:28:39<1:12:09,  2.25s/it]
 90%|████████▉ | 16707/18627 [9:28:41<1:12:48,  2.28s/it]
 90%|████████▉ | 16708/18627 [9:28:42<1:03:49,  2.00s/it]
 90%|████████▉ | 16709/18627 [9:28:45<1:06:22,  2.08s/it]
 90%|████████▉ | 16710/18627 [9:28:47<1:09:04,  2.16s/it]
                                                         
{'loss': 1.0571, 'grad_norm': 7.2943339347839355, 'learning_rate': 1.3759706098861902e-07, 'epoch': 0.9}
+
 90%|████████▉ | 16710/18627 [9:28:47<1:09:04,  2.16s/it]
 90%|████████▉ | 16711/18627 [9:28:48<1:00:12,  1.89s/it]
 90%|████████▉ | 16712/18627 [9:28:51<1:03:52,  2.00s/it]
 90%|████████▉ | 16713/18627 [9:28:53<1:06:08,  2.07s/it]
 90%|████████▉ | 16714/18627 [9:28:55<1:08:18,  2.14s/it]
 90%|████████▉ | 16715/18627 [9:28:57<1:09:58,  2.20s/it]
 90%|████████▉ | 16716/18627 [9:28:59<1:03:01,  1.98s/it]
 90%|████████▉ | 16717/18627 [9:29:00<54:10,  1.70s/it]  
 90%|████████▉ | 16718/18627 [9:29:02<1:00:09,  1.89s/it]
 90%|████████▉ | 16719/18627 [9:29:05<1:04:35,  2.03s/it]
 90%|████████▉ | 16720/18627 [9:29:07<1:07:44,  2.13s/it]
                                                         
{'loss': 0.9845, 'grad_norm': 9.76077938079834, 'learning_rate': 1.3617840279414317e-07, 'epoch': 0.9}
+
 90%|████████▉ | 16720/18627 [9:29:07<1:07:44,  2.13s/it]
 90%|████████▉ | 16721/18627 [9:29:09<1:10:01,  2.20s/it]
 90%|████████▉ | 16722/18627 [9:29:11<1:02:29,  1.97s/it]
 90%|████████▉ | 16723/18627 [9:29:13<1:05:51,  2.08s/it]
 90%|████████▉ | 16724/18627 [9:29:15<1:07:54,  2.14s/it]
 90%|████████▉ | 16725/18627 [9:29:18<1:09:24,  2.19s/it]
 90%|████████▉ | 16726/18627 [9:29:20<1:10:58,  2.24s/it]
 90%|████████▉ | 16727/18627 [9:29:22<1:12:07,  2.28s/it]
 90%|████████▉ | 16728/18627 [9:29:25<1:12:03,  2.28s/it]
 90%|████████▉ | 16729/18627 [9:29:26<1:04:35,  2.04s/it]
 90%|████████▉ | 16730/18627 [9:29:29<1:08:01,  2.15s/it]
                                                         
{'loss': 1.0341, 'grad_norm': 6.444101333618164, 'learning_rate': 1.3476689111234537e-07, 'epoch': 0.9}
+
 90%|████████▉ | 16730/18627 [9:29:29<1:08:01,  2.15s/it]
 90%|████████▉ | 16731/18627 [9:29:31<1:09:18,  2.19s/it]
 90%|████████▉ | 16732/18627 [9:29:33<1:10:34,  2.23s/it]
 90%|████████▉ | 16733/18627 [9:29:35<1:02:40,  1.99s/it]
 90%|████████▉ | 16734/18627 [9:29:37<1:07:32,  2.14s/it]
 90%|████████▉ | 16735/18627 [9:29:39<1:09:29,  2.20s/it]
 90%|████████▉ | 16736/18627 [9:29:42<1:10:55,  2.25s/it]
 90%|████████▉ | 16737/18627 [9:29:44<1:12:15,  2.29s/it]
 90%|████████▉ | 16738/18627 [9:29:47<1:13:07,  2.32s/it]
 90%|████████▉ | 16739/18627 [9:29:49<1:13:20,  2.33s/it]
 90%|████████▉ | 16740/18627 [9:29:51<1:13:34,  2.34s/it]
                                                         
{'loss': 0.8978, 'grad_norm': 5.951274394989014, 'learning_rate': 1.3336253021063173e-07, 'epoch': 0.9}
+
 90%|████████▉ | 16740/18627 [9:29:51<1:13:34,  2.34s/it]
 90%|████████▉ | 16741/18627 [9:29:54<1:13:42,  2.34s/it]
 90%|████████▉ | 16742/18627 [9:29:56<1:13:35,  2.34s/it]
 90%|████████▉ | 16743/18627 [9:29:58<1:12:23,  2.31s/it]
 90%|████████▉ | 16744/18627 [9:30:00<1:02:55,  2.01s/it]
 90%|████████▉ | 16745/18627 [9:30:02<1:05:57,  2.10s/it]
 90%|████████▉ | 16746/18627 [9:30:04<1:07:41,  2.16s/it]
 90%|████████▉ | 16747/18627 [9:30:06<1:09:04,  2.20s/it]
 90%|████████▉ | 16748/18627 [9:30:09<1:09:48,  2.23s/it]
 90%|████████▉ | 16749/18627 [9:30:11<1:11:15,  2.28s/it]
 90%|████████▉ | 16750/18627 [9:30:14<1:11:58,  2.30s/it]
                                                         
{'loss': 0.7903, 'grad_norm': 5.252955913543701, 'learning_rate': 1.3196532433478793e-07, 'epoch': 0.9}
+
 90%|████████▉ | 16750/18627 [9:30:14<1:11:58,  2.30s/it]
 90%|████████▉ | 16751/18627 [9:30:16<1:12:16,  2.31s/it]
 90%|████████▉ | 16752/18627 [9:30:18<1:11:52,  2.30s/it]
 90%|████████▉ | 16753/18627 [9:30:20<1:11:56,  2.30s/it]
 90%|████████▉ | 16754/18627 [9:30:23<1:11:48,  2.30s/it]
 90%|████████▉ | 16755/18627 [9:30:25<1:11:47,  2.30s/it]
 90%|████████▉ | 16756/18627 [9:30:27<1:12:00,  2.31s/it]
 90%|████████▉ | 16757/18627 [9:30:29<1:03:10,  2.03s/it]
 90%|████████▉ | 16758/18627 [9:30:31<1:06:02,  2.12s/it]
 90%|████████▉ | 16759/18627 [9:30:33<1:08:01,  2.18s/it]
 90%|████████▉ | 16760/18627 [9:30:36<1:09:26,  2.23s/it]
                                                         
{'loss': 0.8711, 'grad_norm': 5.719615936279297, 'learning_rate': 1.3057527770897083e-07, 'epoch': 0.9}
+
 90%|████████▉ | 16760/18627 [9:30:36<1:09:26,  2.23s/it]
 90%|████████▉ | 16761/18627 [9:30:38<1:10:00,  2.25s/it]
 90%|████████▉ | 16762/18627 [9:30:40<1:10:02,  2.25s/it]
 90%|████████▉ | 16763/18627 [9:30:43<1:10:47,  2.28s/it]
 90%|████████▉ | 16764/18627 [9:30:44<1:01:29,  1.98s/it]
 90%|█████████ | 16765/18627 [9:30:45<55:38,  1.79s/it]  
 90%|█████████ | 16766/18627 [9:30:48<1:00:25,  1.95s/it]
 90%|█████████ | 16767/18627 [9:30:50<1:04:51,  2.09s/it]
 90%|█████████ | 16768/18627 [9:30:52<1:07:30,  2.18s/it]
 90%|█████████ | 16769/18627 [9:30:55<1:08:32,  2.21s/it]
 90%|█████████ | 16770/18627 [9:30:57<1:10:19,  2.27s/it]
                                                         
{'loss': 1.0752, 'grad_norm': 7.622428894042969, 'learning_rate': 1.2919239453569043e-07, 'epoch': 0.9}
+
 90%|█████████ | 16770/18627 [9:30:57<1:10:19,  2.27s/it]
 90%|█████████ | 16771/18627 [9:31:00<1:11:43,  2.32s/it]
 90%|█████████ | 16772/18627 [9:31:02<1:12:39,  2.35s/it]
 90%|█████████ | 16773/18627 [9:31:04<1:13:03,  2.36s/it]
 90%|█████████ | 16774/18627 [9:31:06<1:05:12,  2.11s/it]
 90%|█████████ | 16775/18627 [9:31:08<1:06:50,  2.17s/it]
 90%|█████████ | 16776/18627 [9:31:10<1:08:13,  2.21s/it]
 90%|█████████ | 16777/18627 [9:31:12<58:33,  1.90s/it]  
 90%|█████████ | 16778/18627 [9:31:13<50:06,  1.63s/it]
 90%|█████████ | 16779/18627 [9:31:14<48:14,  1.57s/it]
 90%|█████████ | 16780/18627 [9:31:16<55:29,  1.80s/it]
                                                       
{'loss': 1.2476, 'grad_norm': 9.959822654724121, 'learning_rate': 1.2781667899580018e-07, 'epoch': 0.9}
+
 90%|█████████ | 16780/18627 [9:31:16<55:29,  1.80s/it]
 90%|█████████ | 16781/18627 [9:31:19<1:00:21,  1.96s/it]
 90%|█████████ | 16782/18627 [9:31:21<1:03:33,  2.07s/it]
 90%|█████████ | 16783/18627 [9:31:23<1:06:07,  2.15s/it]
 90%|█████████ | 16784/18627 [9:31:25<58:51,  1.92s/it]  
 90%|█████████ | 16785/18627 [9:31:27<1:02:19,  2.03s/it]
 90%|█████████ | 16786/18627 [9:31:29<1:05:07,  2.12s/it]
 90%|█████████ | 16787/18627 [9:31:32<1:06:32,  2.17s/it]
 90%|█████████ | 16788/18627 [9:31:34<1:08:17,  2.23s/it]
 90%|█████████ | 16789/18627 [9:31:36<1:10:00,  2.29s/it]
 90%|█████████ | 16790/18627 [9:31:39<1:11:19,  2.33s/it]
                                                         
{'loss': 0.7926, 'grad_norm': 5.833847522735596, 'learning_rate': 1.2644813524848448e-07, 'epoch': 0.9}
+
 90%|█████████ | 16790/18627 [9:31:39<1:11:19,  2.33s/it]
 90%|█████████ | 16791/18627 [9:31:41<1:12:16,  2.36s/it]
 90%|█████████ | 16792/18627 [9:31:43<1:01:18,  2.00s/it]
 90%|█████████ | 16793/18627 [9:31:45<1:04:34,  2.11s/it]
 90%|█████████ | 16794/18627 [9:31:46<58:58,  1.93s/it]  
 90%|█████████ | 16795/18627 [9:31:49<1:03:34,  2.08s/it]
 90%|█████████ | 16796/18627 [9:31:50<57:47,  1.89s/it]  
 90%|█████████ | 16797/18627 [9:31:53<1:02:54,  2.06s/it]
 90%|█████████ | 16798/18627 [9:31:54<57:44,  1.89s/it]  
 90%|█████████ | 16799/18627 [9:31:57<1:02:53,  2.06s/it]
 90%|█████████ | 16800/18627 [9:31:59<1:06:31,  2.18s/it]
                                                         
{'loss': 1.211, 'grad_norm': 7.675579071044922, 'learning_rate': 1.2508676743124447e-07, 'epoch': 0.9}
+
 90%|█████████ | 16800/18627 [9:31:59<1:06:31,  2.18s/it]
 90%|█████████ | 16801/18627 [9:32:01<1:06:22,  2.18s/it]
 90%|█████████ | 16802/18627 [9:32:03<1:05:50,  2.16s/it]
 90%|█████████ | 16803/18627 [9:32:06<1:05:35,  2.16s/it]
 90%|█████████ | 16804/18627 [9:32:08<1:05:22,  2.15s/it]
 90%|█████████ | 16805/18627 [9:32:09<56:30,  1.86s/it]  
 90%|█████████ | 16806/18627 [9:32:10<50:33,  1.67s/it]
 90%|█████████ | 16807/18627 [9:32:12<54:48,  1.81s/it]
 90%|█████████ | 16808/18627 [9:32:13<49:10,  1.62s/it]
 90%|█████████ | 16809/18627 [9:32:16<53:49,  1.78s/it]
 90%|█████████ | 16810/18627 [9:32:18<57:02,  1.88s/it]
                                                       
{'loss': 1.1985, 'grad_norm': 5.106683731079102, 'learning_rate': 1.237325796598865e-07, 'epoch': 0.9}
+
 90%|█████████ | 16810/18627 [9:32:18<57:02,  1.88s/it]
 90%|█████████ | 16811/18627 [9:32:20<59:24,  1.96s/it]
 90%|█████████ | 16812/18627 [9:32:22<1:00:54,  2.01s/it]
 90%|█████████ | 16813/18627 [9:32:23<53:28,  1.77s/it]  
 90%|█████████ | 16814/18627 [9:32:25<56:32,  1.87s/it]
 90%|█████████ | 16815/18627 [9:32:27<59:10,  1.96s/it]
 90%|█████████ | 16816/18627 [9:32:30<1:00:43,  2.01s/it]
 90%|█████████ | 16817/18627 [9:32:31<53:17,  1.77s/it]  
 90%|█████████ | 16818/18627 [9:32:32<48:09,  1.60s/it]
 90%|█████████ | 16819/18627 [9:32:34<52:48,  1.75s/it]
 90%|█████████ | 16820/18627 [9:32:36<55:51,  1.85s/it]
                                                       
{'loss': 1.1216, 'grad_norm': 9.795063972473145, 'learning_rate': 1.223855760285103e-07, 'epoch': 0.9}
+
 90%|█████████ | 16820/18627 [9:32:36<55:51,  1.85s/it]
 90%|█████████ | 16821/18627 [9:32:38<58:12,  1.93s/it]
 90%|█████████ | 16822/18627 [9:32:40<59:55,  1.99s/it]
 90%|█████████ | 16823/18627 [9:32:43<1:01:07,  2.03s/it]
 90%|█████████ | 16824/18627 [9:32:45<1:01:47,  2.06s/it]
 90%|█████████ | 16825/18627 [9:32:47<1:02:10,  2.07s/it]
 90%|█████████ | 16826/18627 [9:32:48<54:11,  1.81s/it]  
 90%|█████████ | 16827/18627 [9:32:50<57:07,  1.90s/it]
 90%|█████████ | 16828/18627 [9:32:51<49:05,  1.64s/it]
 90%|█████████ | 16829/18627 [9:32:52<44:09,  1.47s/it]
 90%|█████████ | 16830/18627 [9:32:54<49:58,  1.67s/it]
                                                       
{'loss': 1.2195, 'grad_norm': 10.792073249816895, 'learning_rate': 1.2104576060949475e-07, 'epoch': 0.9}
+
 90%|█████████ | 16830/18627 [9:32:54<49:58,  1.67s/it]
 90%|█████████ | 16831/18627 [9:32:56<53:49,  1.80s/it]
 90%|█████████ | 16832/18627 [9:32:59<56:44,  1.90s/it]
 90%|█████████ | 16833/18627 [9:33:01<58:46,  1.97s/it]
 90%|█████████ | 16834/18627 [9:33:03<1:00:15,  2.02s/it]
 90%|█████████ | 16835/18627 [9:33:05<1:01:03,  2.04s/it]
 90%|█████████ | 16836/18627 [9:33:06<53:22,  1.79s/it]  
 90%|█████████ | 16837/18627 [9:33:08<56:28,  1.89s/it]
 90%|█████████ | 16838/18627 [9:33:10<58:40,  1.97s/it]
 90%|█████████ | 16839/18627 [9:33:13<1:00:10,  2.02s/it]
 90%|█████████ | 16840/18627 [9:33:15<1:01:11,  2.05s/it]
                                                         
{'loss': 0.7844, 'grad_norm': 11.519238471984863, 'learning_rate': 1.1971313745348768e-07, 'epoch': 0.9}
+
 90%|█████████ | 16840/18627 [9:33:15<1:01:11,  2.05s/it]
 90%|█████████ | 16841/18627 [9:33:16<53:27,  1.80s/it]  
 90%|█████████ | 16842/18627 [9:33:18<56:27,  1.90s/it]
 90%|█████████ | 16843/18627 [9:33:19<49:21,  1.66s/it]
 90%|█████████ | 16844/18627 [9:33:20<45:05,  1.52s/it]
 90%|█████████ | 16845/18627 [9:33:22<50:25,  1.70s/it]
 90%|█████████ | 16846/18627 [9:33:25<54:12,  1.83s/it]
 90%|█████████ | 16847/18627 [9:33:27<56:55,  1.92s/it]
 90%|█████████ | 16848/18627 [9:33:27<45:58,  1.55s/it]
 90%|█████████ | 16849/18627 [9:33:30<51:31,  1.74s/it]
 90%|█████████ | 16850/18627 [9:33:32<54:49,  1.85s/it]
                                                       
{'loss': 1.1751, 'grad_norm': 6.861754894256592, 'learning_rate': 1.1838771058939213e-07, 'epoch': 0.9}
+
 90%|█████████ | 16850/18627 [9:33:32<54:49,  1.85s/it]
 90%|█████████ | 16851/18627 [9:33:34<57:08,  1.93s/it]
 90%|█████████ | 16852/18627 [9:33:36<58:55,  1.99s/it]
 90%|█████████ | 16853/18627 [9:33:38<1:00:09,  2.03s/it]
 90%|█████████ | 16854/18627 [9:33:39<52:41,  1.78s/it]  
 90%|█████████ | 16855/18627 [9:33:41<55:38,  1.88s/it]
 90%|█████████ | 16856/18627 [9:33:43<49:24,  1.67s/it]
 90%|█████████ | 16857/18627 [9:33:45<53:07,  1.80s/it]
 91%|█████████ | 16858/18627 [9:33:47<56:05,  1.90s/it]
 91%|█████████ | 16859/18627 [9:33:49<58:22,  1.98s/it]
 91%|█████████ | 16860/18627 [9:33:50<47:43,  1.62s/it]
                                                       
{'loss': 0.9749, 'grad_norm': 6.551353454589844, 'learning_rate': 1.1706948402435442e-07, 'epoch': 0.91}
+
 91%|█████████ | 16860/18627 [9:33:50<47:43,  1.62s/it]
 91%|█████████ | 16861/18627 [9:33:52<52:23,  1.78s/it]
 91%|█████████ | 16862/18627 [9:33:54<55:41,  1.89s/it]
 91%|█████████ | 16863/18627 [9:33:56<58:07,  1.98s/it]
 91%|█████████ | 16864/18627 [9:33:57<51:11,  1.74s/it]
 91%|█████████ | 16865/18627 [9:34:00<54:24,  1.85s/it]
 91%|█████████ | 16866/18627 [9:34:02<56:44,  1.93s/it]
 91%|█████████ | 16867/18627 [9:34:04<58:21,  1.99s/it]
 91%|█████████ | 16868/18627 [9:34:05<51:19,  1.75s/it]
 91%|█████████ | 16869/18627 [9:34:07<54:28,  1.86s/it]
 91%|█████████ | 16870/18627 [9:34:09<56:45,  1.94s/it]
                                                       
{'loss': 1.0021, 'grad_norm': 6.385002613067627, 'learning_rate': 1.157584617437535e-07, 'epoch': 0.91}
+
 91%|█████████ | 16870/18627 [9:34:09<56:45,  1.94s/it]
 91%|█████████ | 16871/18627 [9:34:11<58:22,  1.99s/it]
 91%|█████████ | 16872/18627 [9:34:13<59:33,  2.04s/it]
 91%|█████████ | 16873/18627 [9:34:16<1:00:18,  2.06s/it]
 91%|█████████ | 16874/18627 [9:34:18<1:00:43,  2.08s/it]
 91%|█████████ | 16875/18627 [9:34:20<1:01:39,  2.11s/it]
 91%|█████████ | 16876/18627 [9:34:22<1:01:45,  2.12s/it]
 91%|█████████ | 16877/18627 [9:34:24<1:01:35,  2.11s/it]
 91%|█████████ | 16878/18627 [9:34:26<1:01:41,  2.12s/it]
 91%|█████████ | 16879/18627 [9:34:28<1:01:50,  2.12s/it]
 91%|█████████ | 16880/18627 [9:34:31<1:01:49,  2.12s/it]
                                                         
{'loss': 0.5233, 'grad_norm': 12.106761932373047, 'learning_rate': 1.1445464771118576e-07, 'epoch': 0.91}
+
 91%|█████████ | 16880/18627 [9:34:31<1:01:49,  2.12s/it]
 91%|█████████ | 16881/18627 [9:34:33<1:01:44,  2.12s/it]
 91%|█████████ | 16882/18627 [9:34:35<1:01:48,  2.12s/it]
 91%|█████████ | 16883/18627 [9:34:37<1:01:48,  2.13s/it]
 91%|█████████ | 16884/18627 [9:34:39<1:01:39,  2.12s/it]
 91%|█████████ | 16885/18627 [9:34:41<1:01:43,  2.13s/it]
 91%|█████████ | 16886/18627 [9:34:43<1:01:26,  2.12s/it]
 91%|█████████ | 16887/18627 [9:34:45<1:01:20,  2.12s/it]
 91%|█████████ | 16888/18627 [9:34:47<1:01:16,  2.11s/it]
 91%|█████████ | 16889/18627 [9:34:50<1:01:06,  2.11s/it]
 91%|█████████ | 16890/18627 [9:34:52<1:01:09,  2.11s/it]
                                                         
{'loss': 0.531, 'grad_norm': 5.353480339050293, 'learning_rate': 1.1315804586845725e-07, 'epoch': 0.91}
+
 91%|█████████ | 16890/18627 [9:34:52<1:01:09,  2.11s/it]
 91%|█████████ | 16891/18627 [9:34:54<1:01:05,  2.11s/it]
 91%|█████████ | 16892/18627 [9:34:56<1:01:09,  2.11s/it]
 91%|█████████ | 16893/18627 [9:34:58<1:01:12,  2.12s/it]
 91%|█████████ | 16894/18627 [9:34:59<53:12,  1.84s/it]  
 91%|█████████ | 16895/18627 [9:35:01<55:39,  1.93s/it]
 91%|█████████ | 16896/18627 [9:35:02<48:10,  1.67s/it]
 91%|█████████ | 16897/18627 [9:35:05<52:10,  1.81s/it]
 91%|█████████ | 16898/18627 [9:35:07<54:49,  1.90s/it]
 91%|█████████ | 16899/18627 [9:35:09<56:42,  1.97s/it]
 91%|█████████ | 16900/18627 [9:35:10<49:59,  1.74s/it]
                                                       
{'loss': 1.2271, 'grad_norm': 15.447254180908203, 'learning_rate': 1.1186866013556758e-07, 'epoch': 0.91}
+
 91%|█████████ | 16900/18627 [9:35:10<49:59,  1.74s/it]
 91%|█████████ | 16901/18627 [9:35:12<53:29,  1.86s/it]
 91%|█████████ | 16902/18627 [9:35:14<55:44,  1.94s/it]
 91%|█████████ | 16903/18627 [9:35:16<57:08,  1.99s/it]
 91%|█████████ | 16904/18627 [9:35:18<58:06,  2.02s/it]
 91%|█████████ | 16905/18627 [9:35:20<51:00,  1.78s/it]
 91%|█████████ | 16906/18627 [9:35:22<53:53,  1.88s/it]
 91%|█████████ | 16907/18627 [9:35:24<55:52,  1.95s/it]
 91%|█████████ | 16908/18627 [9:35:25<47:57,  1.67s/it]
 91%|█████████ | 16909/18627 [9:35:27<51:46,  1.81s/it]
 91%|█████████ | 16910/18627 [9:35:29<54:35,  1.91s/it]
                                                       
{'loss': 0.9594, 'grad_norm': 6.8522539138793945, 'learning_rate': 1.1058649441070075e-07, 'epoch': 0.91}
+
 91%|█████████ | 16910/18627 [9:35:29<54:35,  1.91s/it]
 91%|█████████ | 16911/18627 [9:35:31<56:31,  1.98s/it]
 91%|█████████ | 16912/18627 [9:35:33<57:34,  2.01s/it]
 91%|█████████ | 16913/18627 [9:35:36<58:42,  2.06s/it]
 91%|█████████ | 16914/18627 [9:35:38<59:20,  2.08s/it]
 91%|█████████ | 16915/18627 [9:35:40<1:01:12,  2.15s/it]
 91%|█████████ | 16916/18627 [9:35:41<53:00,  1.86s/it]  
 91%|█████████ | 16917/18627 [9:35:43<55:32,  1.95s/it]
 91%|█████████ | 16918/18627 [9:35:46<57:04,  2.00s/it]
 91%|█████████ | 16919/18627 [9:35:48<57:51,  2.03s/it]
 91%|█████████ | 16920/18627 [9:35:50<58:30,  2.06s/it]
                                                       
{'loss': 0.8343, 'grad_norm': 9.474037170410156, 'learning_rate': 1.0931155257021291e-07, 'epoch': 0.91}
+
 91%|█████████ | 16920/18627 [9:35:50<58:30,  2.06s/it]
 91%|█████████ | 16921/18627 [9:35:51<51:11,  1.80s/it]
 91%|█████████ | 16922/18627 [9:35:53<54:00,  1.90s/it]
 91%|█████████ | 16923/18627 [9:35:55<55:45,  1.96s/it]
 91%|█████████ | 16924/18627 [9:35:57<56:49,  2.00s/it]
 91%|█████████ | 16925/18627 [9:35:58<45:56,  1.62s/it]
 91%|█████████ | 16926/18627 [9:35:59<42:22,  1.49s/it]
 91%|█████████ | 16927/18627 [9:36:01<47:39,  1.68s/it]
 91%|█████████ | 16928/18627 [9:36:03<51:12,  1.81s/it]
 91%|█████████ | 16929/18627 [9:36:05<45:55,  1.62s/it]
 91%|█████████ | 16930/18627 [9:36:06<42:12,  1.49s/it]
                                                       
{'loss': 1.3551, 'grad_norm': 12.29819107055664, 'learning_rate': 1.0804383846861971e-07, 'epoch': 0.91}
+
 91%|█████████ | 16930/18627 [9:36:06<42:12,  1.49s/it]
 91%|█████████ | 16931/18627 [9:36:07<39:38,  1.40s/it]
 91%|█████████ | 16932/18627 [9:36:09<45:50,  1.62s/it]
 91%|█████████ | 16933/18627 [9:36:11<50:03,  1.77s/it]
 91%|█████████ | 16934/18627 [9:36:13<53:16,  1.89s/it]
 91%|█████████ | 16935/18627 [9:36:16<55:20,  1.96s/it]
 91%|█████████ | 16936/18627 [9:36:18<56:45,  2.01s/it]
 91%|█████████ | 16937/18627 [9:36:19<49:45,  1.77s/it]
 91%|█████████ | 16938/18627 [9:36:21<52:35,  1.87s/it]
 91%|█████████ | 16939/18627 [9:36:23<54:49,  1.95s/it]
 91%|█████████ | 16940/18627 [9:36:25<56:12,  2.00s/it]
                                                       
{'loss': 0.9949, 'grad_norm': 5.700606346130371, 'learning_rate': 1.0678335593858558e-07, 'epoch': 0.91}
+
 91%|█████████ | 16940/18627 [9:36:25<56:12,  2.00s/it]
 91%|█████████ | 16941/18627 [9:36:26<49:21,  1.76s/it]
 91%|█████████ | 16942/18627 [9:36:29<52:14,  1.86s/it]
 91%|█████████ | 16943/18627 [9:36:30<46:35,  1.66s/it]
 91%|█████████ | 16944/18627 [9:36:32<50:19,  1.79s/it]
 91%|█████████ | 16945/18627 [9:36:34<53:16,  1.90s/it]
 91%|█████████ | 16946/18627 [9:36:36<54:58,  1.96s/it]
 91%|█████████ | 16947/18627 [9:36:38<56:44,  2.03s/it]
 91%|█████████ | 16948/18627 [9:36:40<57:32,  2.06s/it]
 91%|█████████ | 16949/18627 [9:36:43<58:12,  2.08s/it]
 91%|█████████ | 16950/18627 [9:36:45<58:37,  2.10s/it]
                                                       
{'loss': 1.0281, 'grad_norm': 4.340880393981934, 'learning_rate': 1.0553010879091246e-07, 'epoch': 0.91}
+
 91%|█████████ | 16950/18627 [9:36:45<58:37,  2.10s/it]
 91%|█████████ | 16951/18627 [9:36:47<58:40,  2.10s/it]
 91%|█████████ | 16952/18627 [9:36:49<58:41,  2.10s/it]
 91%|█████████ | 16953/18627 [9:36:51<58:49,  2.11s/it]
 91%|█████████ | 16954/18627 [9:36:53<58:49,  2.11s/it]
 91%|█████████ | 16955/18627 [9:36:55<59:18,  2.13s/it]
 91%|█████████ | 16956/18627 [9:36:56<51:30,  1.85s/it]
 91%|█████████ | 16957/18627 [9:36:59<53:42,  1.93s/it]
 91%|█████████ | 16958/18627 [9:37:01<55:41,  2.00s/it]
 91%|█████████ | 16959/18627 [9:37:02<48:09,  1.73s/it]
 91%|█████████ | 16960/18627 [9:37:04<51:21,  1.85s/it]
                                                       
{'loss': 0.9805, 'grad_norm': 9.698758125305176, 'learning_rate': 1.0428410081452566e-07, 'epoch': 0.91}
+
 91%|█████████ | 16960/18627 [9:37:04<51:21,  1.85s/it]
 91%|█████████ | 16961/18627 [9:37:06<53:56,  1.94s/it]
 91%|█████████ | 16962/18627 [9:37:07<43:55,  1.58s/it]
 91%|█████████ | 16963/18627 [9:37:09<48:22,  1.74s/it]
 91%|█████████ | 16964/18627 [9:37:10<43:46,  1.58s/it]
 91%|█████████ | 16965/18627 [9:37:12<48:09,  1.74s/it]
 91%|█████████ | 16966/18627 [9:37:14<51:26,  1.86s/it]
 91%|█████████ | 16967/18627 [9:37:17<53:46,  1.94s/it]
 91%|█████████ | 16968/18627 [9:37:19<55:07,  1.99s/it]
 91%|█████████ | 16969/18627 [9:37:21<56:03,  2.03s/it]
 91%|█████████ | 16970/18627 [9:37:23<57:06,  2.07s/it]
                                                       
{'loss': 0.7785, 'grad_norm': 7.991297245025635, 'learning_rate': 1.0304533577646714e-07, 'epoch': 0.91}
+
 91%|█████████ | 16970/18627 [9:37:23<57:06,  2.07s/it]
 91%|█████████ | 16971/18627 [9:37:25<57:46,  2.09s/it]
 91%|█████████ | 16972/18627 [9:37:27<58:08,  2.11s/it]
 91%|█████████ | 16973/18627 [9:37:29<58:16,  2.11s/it]
 91%|█████████ | 16974/18627 [9:37:32<58:22,  2.12s/it]
 91%|█████████ | 16975/18627 [9:37:34<58:21,  2.12s/it]
 91%|█████████ | 16976/18627 [9:37:36<58:19,  2.12s/it]
 91%|█████████ | 16977/18627 [9:37:38<58:07,  2.11s/it]
 91%|█████████ | 16978/18627 [9:37:40<58:10,  2.12s/it]
 91%|█████████ | 16979/18627 [9:37:41<49:36,  1.81s/it]
 91%|█████████ | 16980/18627 [9:37:43<52:02,  1.90s/it]
                                                       
{'loss': 0.7986, 'grad_norm': 5.511057376861572, 'learning_rate': 1.0181381742187884e-07, 'epoch': 0.91}
+
 91%|█████████ | 16980/18627 [9:37:43<52:02,  1.90s/it]
 91%|█████████ | 16981/18627 [9:37:45<53:51,  1.96s/it]
 91%|█████████ | 16982/18627 [9:37:46<47:28,  1.73s/it]
 91%|█████████ | 16983/18627 [9:37:49<50:41,  1.85s/it]
 91%|█████████ | 16984/18627 [9:37:50<45:11,  1.65s/it]
 91%|█████████ | 16985/18627 [9:37:52<48:56,  1.79s/it]
 91%|█████████ | 16986/18627 [9:37:53<43:19,  1.58s/it]
 91%|█████████ | 16987/18627 [9:37:55<47:47,  1.75s/it]
 91%|█████████ | 16988/18627 [9:37:57<50:47,  1.86s/it]
 91%|█████████ | 16989/18627 [9:37:59<52:48,  1.93s/it]
 91%|█████████ | 16990/18627 [9:38:01<54:08,  1.98s/it]
                                                       
{'loss': 1.1077, 'grad_norm': 6.462248802185059, 'learning_rate': 1.0058954947399557e-07, 'epoch': 0.91}
+
 91%|█████████ | 16990/18627 [9:38:01<54:08,  1.98s/it]
 91%|█████████ | 16991/18627 [9:38:03<47:47,  1.75s/it]
 91%|█████████ | 16992/18627 [9:38:04<43:08,  1.58s/it]
 91%|█████████ | 16993/18627 [9:38:05<36:05,  1.33s/it]
 91%|█████████ | 16994/18627 [9:38:06<34:59,  1.29s/it]
 91%|█████████ | 16995/18627 [9:38:08<41:46,  1.54s/it]
 91%|█████████ | 16996/18627 [9:38:09<38:53,  1.43s/it]
 91%|█████████ | 16997/18627 [9:38:11<44:22,  1.63s/it]
 91%|██���██████▏| 16998/18627 [9:38:13<48:12,  1.78s/it]
 91%|█████████▏| 16999/18627 [9:38:15<50:48,  1.87s/it]
 91%|█████████▏| 17000/18627 [9:38:18<52:41,  1.94s/it]
                                                       
{'loss': 1.3605, 'grad_norm': 4.6878862380981445, 'learning_rate': 9.937253563413157e-08, 'epoch': 0.91}
+
 91%|█████████▏| 17000/18627 [9:38:18<52:41,  1.94s/it]/home/zhengduo/workspace/LLaMA-Factory/src/qwen_vl/model/modeling_qwen2_5_vl.py:1979: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
+  with torch.cuda.amp.autocast(dtype=dtype):
+/home/zhengduo/miniconda3/envs/vgllm/lib/python3.10/site-packages/torch/utils/checkpoint.py:87: UserWarning: None of the inputs have requires_grad=True. Gradients will be None
+  warnings.warn(
+
 91%|█████████▏| 17001/18627 [9:39:06<7:11:02, 15.91s/it]
 91%|█████████▏| 17002/18627 [9:39:07<5:10:36, 11.47s/it]
 91%|█████████▏| 17003/18627 [9:39:08<3:47:07,  8.39s/it]
 91%|█████████▏| 17004/18627 [9:39:11<2:56:25,  6.52s/it]
 91%|█████████▏| 17005/18627 [9:39:13<2:20:51,  5.21s/it]
 91%|█████████▏| 17006/18627 [9:39:15<1:56:06,  4.30s/it]
 91%|█████████▏| 17007/18627 [9:39:16<1:31:02,  3.37s/it]
 91%|█████████▏| 17008/18627 [9:39:18<1:21:01,  3.00s/it]
 91%|█████████▏| 17009/18627 [9:39:20<1:14:15,  2.75s/it]
 91%|█████████▏| 17010/18627 [9:39:22<1:09:03,  2.56s/it]
                                                         
{'loss': 1.2065, 'grad_norm': 5.9183220863342285, 'learning_rate': 9.816277958166975e-08, 'epoch': 0.91}
+
 91%|█████████▏| 17010/18627 [9:39:22<1:09:03,  2.56s/it]
 91%|█████████▏| 17011/18627 [9:39:26<1:14:37,  2.77s/it]
 91%|█████████▏| 17012/18627 [9:39:28<1:09:17,  2.57s/it]
 91%|█████████▏| 17013/18627 [9:39:30<1:05:31,  2.44s/it]
 91%|█████████▏| 17014/18627 [9:39:32<1:03:19,  2.36s/it]
 91%|█████████▏| 17015/18627 [9:39:34<1:01:43,  2.30s/it]
 91%|█████████▏| 17016/18627 [9:39:35<49:38,  1.85s/it]  
 91%|█████████▏| 17017/18627 [9:39:37<51:56,  1.94s/it]
 91%|█████████▏| 17018/18627 [9:39:38<45:16,  1.69s/it]
 91%|█████████▏| 17019/18627 [9:39:39<37:18,  1.39s/it]
 91%|█████████▏| 17020/18627 [9:39:40<31:50,  1.19s/it]
                                                       
{'loss': 0.7966, 'grad_norm': 5.923278331756592, 'learning_rate': 9.69602849740503e-08, 'epoch': 0.91}
+
 91%|█████████▏| 17020/18627 [9:39:40<31:50,  1.19s/it]
 91%|█████████▏| 17021/18627 [9:39:42<39:35,  1.48s/it]
 91%|█████████▏| 17022/18627 [9:39:44<45:11,  1.69s/it]
 91%|█████████▏| 17023/18627 [9:39:45<37:59,  1.42s/it]
 91%|█████████▏| 17024/18627 [9:39:47<43:47,  1.64s/it]
 91%|█████████▏| 17025/18627 [9:39:49<47:38,  1.78s/it]
 91%|█████████▏| 17026/18627 [9:39:51<50:36,  1.90s/it]
 91%|█████████▏| 17027/18627 [9:39:53<44:59,  1.69s/it]
 91%|█████████▏| 17028/18627 [9:39:54<40:59,  1.54s/it]
 91%|█████████▏| 17029/18627 [9:39:56<45:40,  1.71s/it]
 91%|█████████▏| 17030/18627 [9:39:58<48:35,  1.83s/it]
                                                       
{'loss': 1.004, 'grad_norm': 7.234816074371338, 'learning_rate': 9.576505544676096e-08, 'epoch': 0.91}
+
 91%|█████████▏| 17030/18627 [9:39:58<48:35,  1.83s/it]
 91%|█████████▏| 17031/18627 [9:40:00<51:03,  1.92s/it]
 91%|█████████▏| 17032/18627 [9:40:01<43:48,  1.65s/it]
 91%|█████████▏| 17033/18627 [9:40:03<47:29,  1.79s/it]
 91%|█████████▏| 17034/18627 [9:40:05<49:58,  1.88s/it]
 91%|█████████▏| 17035/18627 [9:40:06<44:30,  1.68s/it]
 91%|█████████▏| 17036/18627 [9:40:09<48:03,  1.81s/it]
 91%|█████████▏| 17037/18627 [9:40:11<50:23,  1.90s/it]
 91%|█████████▏| 17038/18627 [9:40:13<52:19,  1.98s/it]
 91%|█████████▏| 17039/18627 [9:40:14<46:09,  1.74s/it]
 91%|█████████▏| 17040/18627 [9:40:15<41:11,  1.56s/it]
                                                       
{'loss': 1.2928, 'grad_norm': 13.594586372375488, 'learning_rate': 9.457709461332343e-08, 'epoch': 0.91}
+
 91%|█████████▏| 17040/18627 [9:40:15<41:11,  1.56s/it]
 91%|█████████▏| 17041/18627 [9:40:17<45:43,  1.73s/it]
 91%|█████████▏| 17042/18627 [9:40:19<48:50,  1.85s/it]
 91%|█████████��| 17043/18627 [9:40:22<51:00,  1.93s/it]
 92%|█████████▏| 17044/18627 [9:40:24<52:28,  1.99s/it]
 92%|█████████▏| 17045/18627 [9:40:26<53:32,  2.03s/it]
 92%|█████████▏| 17046/18627 [9:40:28<54:09,  2.06s/it]
 92%|█████████▏| 17047/18627 [9:40:30<54:38,  2.08s/it]
 92%|█████████▏| 17048/18627 [9:40:32<55:00,  2.09s/it]
 92%|█████████▏| 17049/18627 [9:40:34<55:24,  2.11s/it]
 92%|█████████▏| 17050/18627 [9:40:36<55:25,  2.11s/it]
                                                       
{'loss': 0.5624, 'grad_norm': 5.343970775604248, 'learning_rate': 9.339640606528644e-08, 'epoch': 0.92}
+
 92%|█████████▏| 17050/18627 [9:40:36<55:25,  2.11s/it]
 92%|█████████▏| 17051/18627 [9:40:39<55:35,  2.12s/it]
 92%|█████████▏| 17052/18627 [9:40:41<55:35,  2.12s/it]
 92%|█████████▏| 17053/18627 [9:40:43<55:32,  2.12s/it]
 92%|█████████▏| 17054/18627 [9:40:45<55:31,  2.12s/it]
 92%|█████████▏| 17055/18627 [9:40:46<48:12,  1.84s/it]
 92%|█████████▏| 17056/18627 [9:40:48<50:29,  1.93s/it]
 92%|█████████▏| 17057/18627 [9:40:50<51:59,  1.99s/it]
 92%|█████████▏| 17058/18627 [9:40:53<53:03,  2.03s/it]
 92%|█████████▏| 17059/18627 [9:40:54<46:31,  1.78s/it]
 92%|█████████▏| 17060/18627 [9:40:56<49:08,  1.88s/it]
                                                       
{'loss': 1.0104, 'grad_norm': 6.650351047515869, 'learning_rate': 9.222299337220963e-08, 'epoch': 0.92}
+
 92%|█████████▏| 17060/18627 [9:40:56<49:08,  1.88s/it]
 92%|█████████▏| 17061/18627 [9:40:58<51:01,  1.96s/it]
 92%|█████████▏| 17062/18627 [9:41:00<52:18,  2.01s/it]
 92%|█████████▏| 17063/18627 [9:41:02<53:06,  2.04s/it]
 92%|█████████▏| 17064/18627 [9:41:04<53:42,  2.06s/it]
 92%|█████████▏| 17065/18627 [9:41:06<54:08,  2.08s/it]
 92%|█████████▏| 17066/18627 [9:41:09<54:33,  2.10s/it]
 92%|█████████▏| 17067/18627 [9:41:11<54:46,  2.11s/it]
 92%|█████████▏| 17068/18627 [9:41:13<54:58,  2.12s/it]
 92%|█████████▏| 17069/18627 [9:41:15<54:56,  2.12s/it]
 92%|█████████▏| 17070/18627 [9:41:17<54:58,  2.12s/it]
                                                       
{'loss': 0.6376, 'grad_norm': 5.70102596282959, 'learning_rate': 9.105686008165776e-08, 'epoch': 0.92}
+
 92%|█████████▏| 17070/18627 [9:41:17<54:58,  2.12s/it]
 92%|█████████▏| 17071/18627 [9:41:19<54:52,  2.12s/it]
 92%|█████████▏| 17072/18627 [9:41:21<54:48,  2.11s/it]
 92%|█████████▏| 17073/18627 [9:41:23<54:46,  2.11s/it]
 92%|█████████▏| 17074/18627 [9:41:26<54:49,  2.12s/it]
 92%|█████████▏| 17075/18627 [9:41:27<47:36,  1.84s/it]
 92%|█████████▏| 17076/18627 [9:41:29<49:44,  1.92s/it]
 92%|█████████▏| 17077/18627 [9:41:31<51:16,  1.98s/it]
 92%|█████████▏| 17078/18627 [9:41:32<45:07,  1.75s/it]
 92%|█████████▏| 17079/18627 [9:41:34<48:12,  1.87s/it]
 92%|█████████▏| 17080/18627 [9:41:36<50:08,  1.94s/it]
                                                       
{'loss': 0.9912, 'grad_norm': 6.201600551605225, 'learning_rate': 8.989800971918789e-08, 'epoch': 0.92}
+
 92%|█████████▏| 17080/18627 [9:41:36<50:08,  1.94s/it]
 92%|█████████▏| 17081/18627 [9:41:38<44:22,  1.72s/it]
 92%|█████████▏| 17082/18627 [9:41:40<47:29,  1.84s/it]
 92%|█████████▏| 17083/18627 [9:41:41<42:29,  1.65s/it]
 92%|█████████▏| 17084/18627 [9:41:43<46:10,  1.80s/it]
 92%|█████████▏| 17085/18627 [9:41:45<48:48,  1.90s/it]
 92%|█████████▏| 17086/18627 [9:41:47<50:30,  1.97s/it]
 92%|█████████▏| 17087/18627 [9:41:50<51:47,  2.02s/it]
 92%|█████████▏| 17088/18627 [9:41:51<45:23,  1.77s/it]
 92%|█████████▏| 17089/18627 [9:41:53<47:49,  1.87s/it]
 92%|█████████▏| 17090/18627 [9:41:55<49:48,  1.94s/it]
                                                       
{'loss': 1.2591, 'grad_norm': 6.082885265350342, 'learning_rate': 8.874644578833718e-08, 'epoch': 0.92}
+
 92%|█████████▏| 17090/18627 [9:41:55<49:48,  1.94s/it]
 92%|█████████▏| 17091/18627 [9:41:57<50:58,  1.99s/it]
 92%|█████████▏| 17092/18627 [9:41:59<51:50,  2.03s/it]
 92%|█████████▏| 17093/18627 [9:42:01<52:26,  2.05s/it]
 92%|█████████▏| 17094/18627 [9:42:03<52:52,  2.07s/it]
 92%|██��██████▏| 17095/18627 [9:42:05<53:07,  2.08s/it]
 92%|█████████▏| 17096/18627 [9:42:08<53:18,  2.09s/it]
 92%|█████████▏| 17097/18627 [9:42:10<53:20,  2.09s/it]
 92%|█████████▏| 17098/18627 [9:42:11<46:23,  1.82s/it]
 92%|█████████▏| 17099/18627 [9:42:13<48:39,  1.91s/it]
 92%|█████████▏| 17100/18627 [9:42:15<50:10,  1.97s/it]
                                                       
{'loss': 0.7846, 'grad_norm': 7.98124885559082, 'learning_rate': 8.760217177061492e-08, 'epoch': 0.92}
+
 92%|█████████▏| 17100/18627 [9:42:15<50:10,  1.97s/it]
 92%|█████████▏| 17101/18627 [9:42:17<51:28,  2.02s/it]
 92%|█████████▏| 17102/18627 [9:42:19<52:13,  2.05s/it]
 92%|█████████▏| 17103/18627 [9:42:21<52:46,  2.08s/it]
 92%|█████████▏| 17104/18627 [9:42:24<53:12,  2.10s/it]
 92%|█████████▏| 17105/18627 [9:42:25<46:16,  1.82s/it]
 92%|█████████▏| 17106/18627 [9:42:27<48:33,  1.92s/it]
 92%|█████████▏| 17107/18627 [9:42:29<50:03,  1.98s/it]
 92%|█████████▏| 17108/18627 [9:42:31<51:04,  2.02s/it]
 92%|█████████▏| 17109/18627 [9:42:33<51:46,  2.05s/it]
 92%|█████████▏| 17110/18627 [9:42:35<52:19,  2.07s/it]
                                                       
{'loss': 0.8477, 'grad_norm': 6.164219856262207, 'learning_rate': 8.646519112549045e-08, 'epoch': 0.92}
+
 92%|█████████▏| 17110/18627 [9:42:35<52:19,  2.07s/it]
 92%|█████████▏| 17111/18627 [9:42:37<45:41,  1.81s/it]
 92%|█████████▏| 17112/18627 [9:42:39<48:01,  1.90s/it]
 92%|█████████▏| 17113/18627 [9:42:41<49:36,  1.97s/it]
 92%|█████████▏| 17114/18627 [9:42:43<50:51,  2.02s/it]
 92%|█████████▏| 17115/18627 [9:42:45<51:30,  2.04s/it]
 92%|█████████▏| 17116/18627 [9:42:47<52:08,  2.07s/it]
 92%|█████████▏| 17117/18627 [9:42:49<52:27,  2.08s/it]
 92%|█████████▏| 17118/18627 [9:42:51<52:46,  2.10s/it]
 92%|█████████▏| 17119/18627 [9:42:54<52:50,  2.10s/it]
 92%|█████████▏| 17120/18627 [9:42:56<52:52,  2.10s/it]
                                                       
{'loss': 0.7715, 'grad_norm': 5.993988037109375, 'learning_rate': 8.53355072903836e-08, 'epoch': 0.92}
+
 92%|█████████▏| 17120/18627 [9:42:56<52:52,  2.10s/it]
 92%|█████████▏| 17121/18627 [9:42:58<53:00,  2.11s/it]
 92%|█████████▏| 17122/18627 [9:43:00<53:00,  2.11s/it]
 92%|█████████▏| 17123/18627 [9:43:02<53:04,  2.12s/it]
 92%|█████████▏| 17124/18627 [9:43:03<46:09,  1.84s/it]
 92%|█████████▏| 17125/18627 [9:43:04<41:22,  1.65s/it]
 92%|█████████▏| 17126/18627 [9:43:07<44:53,  1.79s/it]
 92%|█████████▏| 17127/18627 [9:43:08<40:22,  1.61s/it]
 92%|█████████▏| 17128/18627 [9:43:10<44:09,  1.77s/it]
 92%|█████████▏| 17129/18627 [9:43:11<39:49,  1.60s/it]
 92%|█████████▏| 17130/18627 [9:43:13<43:43,  1.75s/it]
                                                       
{'loss': 1.3903, 'grad_norm': 7.52805757522583, 'learning_rate': 8.421312368065343e-08, 'epoch': 0.92}
+
 92%|█████████▏| 17130/18627 [9:43:13<43:43,  1.75s/it]
 92%|█████████▏| 17131/18627 [9:43:15<46:31,  1.87s/it]
 92%|█████████▏| 17132/18627 [9:43:17<48:28,  1.95s/it]
 92%|█████████▏| 17133/18627 [9:43:20<49:46,  2.00s/it]
 92%|█████████▏| 17134/18627 [9:43:22<50:27,  2.03s/it]
 92%|█████████▏| 17135/18627 [9:43:24<51:09,  2.06s/it]
 92%|█████████▏| 17136/18627 [9:43:26<51:35,  2.08s/it]
 92%|█████████▏| 17137/18627 [9:43:28<51:56,  2.09s/it]
 92%|█████████▏| 17138/18627 [9:43:30<52:06,  2.10s/it]
 92%|█████████▏| 17139/18627 [9:43:32<52:10,  2.10s/it]
 92%|█████████▏| 17140/18627 [9:43:34<52:14,  2.11s/it]
                                                       
{'loss': 0.6142, 'grad_norm': 6.513925075531006, 'learning_rate': 8.309804368958868e-08, 'epoch': 0.92}
+
 92%|█████████▏| 17140/18627 [9:43:34<52:14,  2.11s/it]
 92%|█████████▏| 17141/18627 [9:43:37<52:25,  2.12s/it]
 92%|█████████▏| 17142/18627 [9:43:39<52:29,  2.12s/it]
 92%|█████████▏| 17143/18627 [9:43:41<52:18,  2.11s/it]
 92%|█████████▏| 17144/18627 [9:43:43<52:17,  2.12s/it]
 92%|█████████▏| 17145/18627 [9:43:44<45:25,  1.84s/it]
 92%|█████████▏| 17146/18627 [9:43:46<47:29,  1.92s/it]
 92%|█████████▏| 17147/18627 [9:43:48<48:54,  1.98s/it]
 92%|█████████▏| 17148/18627 [9:43:50<49:47,  2.02s/it]
 92%|█████████▏| 17149/18627 [9:43:53<50:37,  2.06s/it]
 92%|█████████▏| 17150/18627 [9:43:54<44:12,  1.80s/it]
                                                       
{'loss': 0.9503, 'grad_norm': 13.960413932800293, 'learning_rate': 8.199027068839627e-08, 'epoch': 0.92}
+
 92%|█████████▏| 17150/18627 [9:43:54<44:12,  1.80s/it]
 92%|█████████▏| 17151/18627 [9:43:56<46:45,  1.90s/it]
 92%|█████████▏| 17152/18627 [9:43:57<41:04,  1.67s/it]
 92%|█████████▏| 17153/18627 [9:43:58<37:36,  1.53s/it]
 92%|█████████▏| 17154/18627 [9:43:59<35:06,  1.43s/it]
 92%|█████████▏| 17155/18627 [9:44:02<40:15,  1.64s/it]
 92%|█████████▏| 17156/18627 [9:44:04<43:46,  1.79s/it]
 92%|█████████▏| 17157/18627 [9:44:06<46:48,  1.91s/it]
 92%|█████████▏| 17158/18627 [9:44:07<41:41,  1.70s/it]
 92%|█████████▏| 17159/18627 [9:44:08<36:35,  1.50s/it]
 92%|█████████▏| 17160/18627 [9:44:10<41:05,  1.68s/it]
                                                       
{'loss': 1.4947, 'grad_norm': 6.463510513305664, 'learning_rate': 8.088980802619384e-08, 'epoch': 0.92}
+
 92%|█████████▏| 17160/18627 [9:44:10<41:05,  1.68s/it]
 92%|█████████▏| 17161/18627 [9:44:12<44:10,  1.81s/it]
 92%|█████████▏| 17162/18627 [9:44:15<46:25,  1.90s/it]
 92%|█████████▏| 17163/18627 [9:44:17<47:50,  1.96s/it]
 92%|█████████▏| 17164/18627 [9:44:19<48:49,  2.00s/it]
 92%|█████████▏| 17165/18627 [9:44:21<49:28,  2.03s/it]
 92%|█████████▏| 17166/18627 [9:44:22<43:17,  1.78s/it]
 92%|█████████▏| 17167/18627 [9:44:24<45:50,  1.88s/it]
 92%|█████████▏| 17168/18627 [9:44:26<47:32,  1.96s/it]
 92%|█████████▏| 17169/18627 [9:44:28<48:43,  2.01s/it]
 92%|█████████▏| 17170/18627 [9:44:29<41:28,  1.71s/it]
                                                       
{'loss': 1.0333, 'grad_norm': 12.759188652038574, 'learning_rate': 7.979665902999533e-08, 'epoch': 0.92}
+
 92%|█████████▏| 17170/18627 [9:44:29<41:28,  1.71s/it]
 92%|█████████▏| 17171/18627 [9:44:31<37:40,  1.55s/it]
 92%|█████████▏| 17172/18627 [9:44:33<41:55,  1.73s/it]
 92%|█████████▏| 17173/18627 [9:44:34<38:02,  1.57s/it]
 92%|█████████▏| 17174/18627 [9:44:36<42:04,  1.74s/it]
 92%|█████████▏| 17175/18627 [9:44:37<38:08,  1.58s/it]
 92%|█████████▏| 17176/18627 [9:44:38<35:21,  1.46s/it]
 92%|█████████▏| 17177/18627 [9:44:41<40:05,  1.66s/it]
 92%|█████████▏| 17178/18627 [9:44:43<43:21,  1.80s/it]
 92%|█████████▏| 17179/18627 [9:44:45<45:38,  1.89s/it]
 92%|█████████▏| 17180/18627 [9:44:47<47:12,  1.96s/it]
                                                       
{'loss': 1.3297, 'grad_norm': 6.954909801483154, 'learning_rate': 7.871082700470544e-08, 'epoch': 0.92}
+
 92%|█████████▏| 17180/18627 [9:44:47<47:12,  1.96s/it]
 92%|█████████▏| 17181/18627 [9:44:49<48:17,  2.00s/it]
 92%|█████████▏| 17182/18627 [9:44:51<49:08,  2.04s/it]
 92%|█████████▏| 17183/18627 [9:44:53<49:52,  2.07s/it]
 92%|█████████▏| 17184/18627 [9:44:55<49:59,  2.08s/it]
 92%|█████████▏| 17185/18627 [9:44:58<50:26,  2.10s/it]
 92%|█████████▏| 17186/18627 [9:44:58<40:19,  1.68s/it]
 92%|█████████▏| 17187/18627 [9:44:59<36:46,  1.53s/it]
 92%|█████████▏| 17188/18627 [9:45:02<41:13,  1.72s/it]
 92%|█████████▏| 17189/18627 [9:45:03<37:23,  1.56s/it]
 92%|█████████▏| 17190/18627 [9:45:05<41:17,  1.72s/it]
                                                       
{'loss': 1.0929, 'grad_norm': 4.726139068603516, 'learning_rate': 7.763231523310627e-08, 'epoch': 0.92}
+
 92%|█████████▏| 17190/18627 [9:45:05<41:17,  1.72s/it]
 92%|█████████▏| 17191/18627 [9:45:07<44:07,  1.84s/it]
 92%|█████████▏| 17192/18627 [9:45:09<46:02,  1.93s/it]
 92%|█████████▏| 17193/18627 [9:45:11<47:26,  1.99s/it]
 92%|█████████▏| 17194/18627 [9:45:13<48:12,  2.02s/it]
 92%|█████████▏| 17195/18627 [9:45:16<49:44,  2.08s/it]
 92%|█████████▏| 17196/18627 [9:45:18<51:03,  2.14s/it]
 92%|█████████▏| 17197/18627 [9:45:19<43:00,  1.80s/it]
 92%|█████████▏| 17198/18627 [9:45:21<46:19,  1.94s/it]
 92%|█████████▏| 17199/18627 [9:45:25<1:02:37,  2.63s/it]
 92%|█████████▏| 17200/18627 [9:45:28<1:00:14,  2.53s/it]
                                                         
{'loss': 0.6638, 'grad_norm': 5.912487983703613, 'learning_rate': 7.656112697584928e-08, 'epoch': 0.92}
+
 92%|█████████▏| 17200/18627 [9:45:28<1:00:14,  2.53s/it]
 92%|█████████▏| 17201/18627 [9:45:30<58:22,  2.46s/it]  
 92%|█████████▏| 17202/18627 [9:45:32<57:00,  2.40s/it]
 92%|█████████▏| 17203/18627 [9:45:34<55:59,  2.36s/it]
 92%|█████████▏| 17204/18627 [9:45:37<55:27,  2.34s/it]
 92%|█████████▏| 17205/18627 [9:45:38<47:54,  2.02s/it]
 92%|█████████▏| 17206/18627 [9:45:40<49:41,  2.10s/it]
 92%|█████████▏| 17207/18627 [9:45:43<50:48,  2.15s/it]
 92%|█████████▏| 17208/18627 [9:45:45<51:34,  2.18s/it]
 92%|█████████▏| 17209/18627 [9:45:47<52:07,  2.21s/it]
 92%|█████████▏| 17210/18627 [9:45:49<52:42,  2.23s/it]
                                                       
{'loss': 0.8416, 'grad_norm': 5.926290035247803, 'learning_rate': 7.549726547144504e-08, 'epoch': 0.92}
+
 92%|█████████▏| 17210/18627 [9:45:49<52:42,  2.23s/it]
 92%|█████████▏| 17211/18627 [9:45:51<46:08,  1.96s/it]
 92%|█████████▏| 17212/18627 [9:45:53<48:33,  2.06s/it]
 92%|█████████▏| 17213/18627 [9:45:55<50:08,  2.13s/it]
 92%|█████████▏| 17214/18627 [9:45:58<51:08,  2.17s/it]
 92%|█████████▏| 17215/18627 [9:45:59<44:41,  1.90s/it]
 92%|█████████▏| 17216/18627 [9:46:01<47:24,  2.02s/it]
 92%|█████████▏| 17217/18627 [9:46:03<49:10,  2.09s/it]
 92%|█████████▏| 17218/18627 [9:46:06<50:28,  2.15s/it]
 92%|█████████▏| 17219/18627 [9:46:08<51:25,  2.19s/it]
 92%|█████████▏| 17220/18627 [9:46:10<51:51,  2.21s/it]
                                                       
{'loss': 0.9909, 'grad_norm': 3.708927869796753, 'learning_rate': 7.444073393625267e-08, 'epoch': 0.92}
+
 92%|█████████▏| 17220/18627 [9:46:10<51:51,  2.21s/it]
 92%|█████████▏| 17221/18627 [9:46:11<45:00,  1.92s/it]
 92%|█████████▏| 17222/18627 [9:46:14<47:34,  2.03s/it]
 92%|█████████▏| 17223/18627 [9:46:15<42:34,  1.82s/it]
 92%|█████████▏| 17224/18627 [9:46:17<45:41,  1.95s/it]
 92%|█████████▏| 17225/18627 [9:46:20<48:03,  2.06s/it]
 92%|█████████▏| 17226/18627 [9:46:22<49:29,  2.12s/it]
 92%|█████████▏| 17227/18627 [9:46:24<50:34,  2.17s/it]
 92%|█████████▏| 17228/18627 [9:46:26<44:48,  1.92s/it]
 92%|█████████▏| 17229/18627 [9:46:28<47:14,  2.03s/it]
 93%|█████████▎| 17230/18627 [9:46:30<49:03,  2.11s/it]
                                                       
{'loss': 1.2265, 'grad_norm': 6.825359344482422, 'learning_rate': 7.33915355644707e-08, 'epoch': 0.93}
+
 93%|█████████▎| 17230/18627 [9:46:30<49:03,  2.11s/it]
 93%|█████████▎| 17231/18627 [9:46:32<50:19,  2.16s/it]
 93%|█████████▎| 17232/18627 [9:46:35<51:12,  2.20s/it]
 93%|█████████▎| 17233/18627 [9:46:37<51:48,  2.23s/it]
 93%|█████████▎| 17234/18627 [9:46:38<44:46,  1.93s/it]
 93%|█████████▎| 17235/18627 [9:46:40<47:14,  2.04s/it]
 93%|█████████▎| 17236/18627 [9:46:43<49:03,  2.12s/it]
 93%|█████████▎| 17237/18627 [9:46:44<42:35,  1.84s/it]
 93%|█████████▎| 17238/18627 [9:46:46<45:39,  1.97s/it]
 93%|█████████▎| 17239/18627 [9:46:49<47:49,  2.07s/it]
 93%|█████████▎| 17240/18627 [9:46:51<49:16,  2.13s/it]
                                                       
{'loss': 0.9078, 'grad_norm': 8.064277648925781, 'learning_rate': 7.234967352812817e-08, 'epoch': 0.93}
+
 93%|█████████▎| 17240/18627 [9:46:51<49:16,  2.13s/it]
 93%|█████████▎| 17241/18627 [9:46:52<43:07,  1.87s/it]
 93%|█████████▎| 17242/18627 [9:46:54<45:47,  1.98s/it]
 93%|█████████▎| 17243/18627 [9:46:57<47:36,  2.06s/it]
 93%|█████████▎| 17244/18627 [9:46:59<49:26,  2.15s/it]
 93%|█████████▎| 17245/18627 [9:47:01<50:27,  2.19s/it]
 93%|█████████▎| 17246/18627 [9:47:03<44:09,  1.92s/it]
 93%|█████████▎| 17247/18627 [9:47:05<46:42,  2.03s/it]
 93%|█████████▎| 17248/18627 [9:47:07<48:26,  2.11s/it]
 93%|█████████▎| 17249/18627 [9:47:09<49:49,  2.17s/it]
 93%|███���█████▎| 17250/18627 [9:47:12<50:39,  2.21s/it]
                                                       
{'loss': 1.0521, 'grad_norm': 11.741791725158691, 'learning_rate': 7.131515097707293e-08, 'epoch': 0.93}
+
 93%|█████████▎| 17250/18627 [9:47:12<50:39,  2.21s/it]
 93%|█████████▎| 17251/18627 [9:47:13<44:16,  1.93s/it]
 93%|█████████▎| 17252/18627 [9:47:15<46:29,  2.03s/it]
 93%|█████████▎| 17253/18627 [9:47:17<47:54,  2.09s/it]
 93%|█████████▎| 17254/18627 [9:47:20<49:08,  2.15s/it]
 93%|█████████▎| 17255/18627 [9:47:21<43:14,  1.89s/it]
 93%|█████████▎| 17256/18627 [9:47:23<45:48,  2.00s/it]
 93%|█████████▎| 17257/18627 [9:47:26<47:30,  2.08s/it]
 93%|█████████▎| 17258/18627 [9:47:27<42:16,  1.85s/it]
 93%|█████████▎| 17259/18627 [9:47:29<45:22,  1.99s/it]
 93%|█████████▎| 17260/18627 [9:47:32<47:25,  2.08s/it]
                                                       
{'loss': 1.1996, 'grad_norm': 6.170499324798584, 'learning_rate': 7.028797103896507e-08, 'epoch': 0.93}
+
 93%|█████████▎| 17260/18627 [9:47:32<47:25,  2.08s/it]
 93%|█████████▎| 17261/18627 [9:47:33<42:00,  1.85s/it]
 93%|█████████▎| 17262/18627 [9:47:35<44:56,  1.98s/it]
 93%|█████████▎| 17263/18627 [9:47:37<46:43,  2.06s/it]
 93%|█████████▎| 17264/18627 [9:47:40<48:23,  2.13s/it]
 93%|█████████▎| 17265/18627 [9:47:41<42:35,  1.88s/it]
 93%|█████████▎| 17266/18627 [9:47:42<36:17,  1.60s/it]
 93%|█████████▎| 17267/18627 [9:47:44<40:48,  1.80s/it]
 93%|█████████▎| 17268/18627 [9:47:46<43:57,  1.94s/it]
 93%|█████████▎| 17269/18627 [9:47:48<39:23,  1.74s/it]
 93%|█████████▎| 17270/18627 [9:47:50<42:51,  1.89s/it]
                                                       
{'loss': 1.1815, 'grad_norm': 8.637792587280273, 'learning_rate': 6.926813681926436e-08, 'epoch': 0.93}
+
 93%|█████████▎| 17270/18627 [9:47:50<42:51,  1.89s/it]
 93%|█████████▎| 17271/18627 [9:47:52<45:29,  2.01s/it]
 93%|█████████▎| 17272/18627 [9:47:53<40:23,  1.79s/it]
 93%|█████████▎| 17273/18627 [9:47:56<43:40,  1.94s/it]
 93%|█████████▎| 17274/18627 [9:47:57<39:05,  1.73s/it]
 93%|█████████▎| 17275/18627 [9:47:59<42:34,  1.89s/it]
 93%|█████████▎| 17276/18627 [9:48:02<45:15,  2.01s/it]
 93%|█████████▎| 17277/18627 [9:48:04<47:09,  2.10s/it]
 93%|█████████▎| 17278/18627 [9:48:05<41:29,  1.85s/it]
 93%|█████████▎| 17279/18627 [9:48:07<44:14,  1.97s/it]
 93%|█████████▎| 17280/18627 [9:48:10<46:13,  2.06s/it]
                                                       
{'loss': 1.1182, 'grad_norm': 3.948028564453125, 'learning_rate': 6.825565140122364e-08, 'epoch': 0.93}
+
 93%|█████████▎| 17280/18627 [9:48:10<46:13,  2.06s/it]
 93%|█████████▎| 17281/18627 [9:48:12<47:44,  2.13s/it]
 93%|█████████▎| 17282/18627 [9:48:14<48:35,  2.17s/it]
 93%|█████████▎| 17283/18627 [9:48:16<49:19,  2.20s/it]
 93%|█████████▎| 17284/18627 [9:48:20<58:45,  2.62s/it]
 93%|█████████▎| 17285/18627 [9:48:22<56:26,  2.52s/it]
 93%|█████████▎| 17286/18627 [9:48:25<54:41,  2.45s/it]
 93%|█████████▎| 17287/18627 [9:48:27<53:25,  2.39s/it]
 93%|█████████▎| 17288/18627 [9:48:29<52:31,  2.35s/it]
 93%|█████████▎| 17289/18627 [9:48:31<52:06,  2.34s/it]
 93%|█████████▎| 17290/18627 [9:48:34<51:30,  2.31s/it]
                                                       
{'loss': 0.6015, 'grad_norm': 7.411802291870117, 'learning_rate': 6.725051784587711e-08, 'epoch': 0.93}
+
 93%|█████████▎| 17290/18627 [9:48:34<51:30,  2.31s/it]
 93%|█████████▎| 17291/18627 [9:48:36<51:08,  2.30s/it]
 93%|█████████▎| 17292/18627 [9:48:38<50:50,  2.29s/it]
 93%|█████████▎| 17293/18627 [9:48:40<44:04,  1.98s/it]
 93%|█████████▎| 17294/18627 [9:48:42<46:00,  2.07s/it]
 93%|█████████▎| 17295/18627 [9:48:44<47:18,  2.13s/it]
 93%|█████████▎| 17296/18627 [9:48:46<48:07,  2.17s/it]
 93%|█████████▎| 17297/18627 [9:48:49<48:49,  2.20s/it]
 93%|█████████▎| 17298/18627 [9:48:50<43:00,  1.94s/it]
 93%|█████████▎| 17299/18627 [9:48:52<45:16,  2.05s/it]
 93%|█████████▎| 17300/18627 [9:48:54<46:36,  2.11s/it]
                                                       
{'loss': 1.0073, 'grad_norm': 5.4390177726745605, 'learning_rate': 6.625273919203313e-08, 'epoch': 0.93}
+
 93%|█████████▎| 17300/18627 [9:48:54<46:36,  2.11s/it]
 93%|█████████▎| 17301/18627 [9:48:57<47:43,  2.16s/it]
 93%|█████████▎| 17302/18627 [9:48:59<48:20,  2.19s/it]
 93%|█████████▎| 17303/18627 [9:49:00<42:19,  1.92s/it]
 93%|█████████▎| 17304/18627 [9:49:02<37:32,  1.70s/it]
 93%|█████████▎| 17305/18627 [9:49:03<34:46,  1.58s/it]
 93%|█████████▎| 17306/18627 [9:49:04<33:02,  1.50s/it]
 93%|█████████▎| 17307/18627 [9:49:06<38:18,  1.74s/it]
 93%|█████████▎| 17308/18627 [9:49:08<35:21,  1.61s/it]
 93%|█████████▎| 17309/18627 [9:49:10<39:54,  1.82s/it]
 93%|█████████▎| 17310/18627 [9:49:11<36:18,  1.65s/it]
                                                       
{'loss': 1.8452, 'grad_norm': 13.381534576416016, 'learning_rate': 6.526231845626258e-08, 'epoch': 0.93}
+
 93%|█████████▎| 17310/18627 [9:49:11<36:18,  1.65s/it]
 93%|█████████▎| 17311/18627 [9:49:13<33:46,  1.54s/it]
 93%|█████████▎| 17312/18627 [9:49:15<38:39,  1.76s/it]
 93%|█████████▎| 17313/18627 [9:49:17<41:54,  1.91s/it]
 93%|█████████▎| 17314/18627 [9:49:19<44:17,  2.02s/it]
 93%|█████████▎| 17315/18627 [9:49:22<46:09,  2.11s/it]
 93%|█████████▎| 17316/18627 [9:49:24<47:22,  2.17s/it]
 93%|█████████▎| 17317/18627 [9:49:26<48:07,  2.20s/it]
 93%|█████████▎| 17318/18627 [9:49:29<48:38,  2.23s/it]
 93%|█████████▎| 17319/18627 [9:49:30<40:27,  1.86s/it]
 93%|█████████▎| 17320/18627 [9:49:32<43:08,  1.98s/it]
                                                       
{'loss': 0.7463, 'grad_norm': 6.753538608551025, 'learning_rate': 6.42792586328933e-08, 'epoch': 0.93}
+
 93%|█████████▎| 17320/18627 [9:49:32<43:08,  1.98s/it]
 93%|█████████▎| 17321/18627 [9:49:34<44:57,  2.07s/it]
 93%|█████████▎| 17322/18627 [9:49:35<39:52,  1.83s/it]
 93%|█████████▎| 17323/18627 [9:49:37<36:20,  1.67s/it]
 93%|█████████▎| 17324/18627 [9:49:39<40:20,  1.86s/it]
 93%|█████████▎| 17325/18627 [9:49:40<36:46,  1.69s/it]
 93%|█████████▎| 17326/18627 [9:49:43<40:35,  1.87s/it]
 93%|█████████▎| 17327/18627 [9:49:44<36:42,  1.69s/it]
 93%|█████████▎| 17328/18627 [9:49:45<34:05,  1.57s/it]
 93%|█████████▎| 17329/18627 [9:49:47<38:31,  1.78s/it]
 93%|█████████▎| 17330/18627 [9:49:50<41:40,  1.93s/it]
                                                       
{'loss': 1.5172, 'grad_norm': 4.439027309417725, 'learning_rate': 6.33035626939979e-08, 'epoch': 0.93}
+
 93%|█████████▎| 17330/18627 [9:49:50<41:40,  1.93s/it]
 93%|█████████▎| 17331/18627 [9:49:52<43:52,  2.03s/it]
 93%|█████████▎| 17332/18627 [9:49:53<38:48,  1.80s/it]
 93%|█████████▎| 17333/18627 [9:49:55<35:42,  1.66s/it]
 93%|█████████▎| 17334/18627 [9:49:56<31:17,  1.45s/it]
 93%|█████████▎| 17335/18627 [9:49:58<36:55,  1.71s/it]
 93%|█████████▎| 17336/18627 [9:49:59<34:04,  1.58s/it]
 93%|█████████▎| 17337/18627 [9:50:01<38:33,  1.79s/it]
 93%|█████████▎| 17338/18627 [9:50:04<41:37,  1.94s/it]
 93%|█████████▎| 17339/18627 [9:50:05<35:12,  1.64s/it]
 93%|█████████▎| 17340/18627 [9:50:06<33:01,  1.54s/it]
                                                       
{'loss': 1.5219, 'grad_norm': 15.756540298461914, 'learning_rate': 6.233523358938509e-08, 'epoch': 0.93}
+
 93%|█████████▎| 17340/18627 [9:50:06<33:01,  1.54s/it]
 93%|█████████▎| 17341/18627 [9:50:08<37:52,  1.77s/it]
 93%|█████████▎| 17342/18627 [9:50:10<41:01,  1.92s/it]
 93%|█████████▎| 17343/18627 [9:50:13<43:24,  2.03s/it]
 93%|█████████▎| 17344/18627 [9:50:14<38:10,  1.79s/it]
 93%|█████████▎| 17345/18627 [9:50:16<41:24,  1.94s/it]
 93%|█████████▎| 17346/18627 [9:50:19<43:27,  2.04s/it]
 93%|█████████▎| 17347/18627 [9:50:21<44:49,  2.10s/it]
 93%|█████████▎| 17348/18627 [9:50:22<39:37,  1.86s/it]
 93%|█████████▎| 17349/18627 [9:50:24<42:09,  1.98s/it]
 93%|█████████▎| 17350/18627 [9:50:27<44:07,  2.07s/it]
                                                       
{'loss': 1.0129, 'grad_norm': 7.112362861633301, 'learning_rate': 6.13742742465931e-08, 'epoch': 0.93}
+
 93%|█████████▎| 17350/18627 [9:50:27<44:07,  2.07s/it]
 93%|█████████▎| 17351/18627 [9:50:29<45:21,  2.13s/it]
 93%|█████████▎| 17352/18627 [9:50:30<39:48,  1.87s/it]
 93%|█████████▎| 17353/18627 [9:50:32<42:26,  2.00s/it]
 93%|█████████▎| 17354/18627 [9:50:35<44:11,  2.08s/it]
 93%|█████████▎| 17355/18627 [9:50:37<45:25,  2.14s/it]
 93%|█████████▎| 17356/18627 [9:50:39<46:02,  2.17s/it]
 93%|█████████▎| 17357/18627 [9:50:42<46:35,  2.20s/it]
 93%|█████████▎| 17358/18627 [9:50:44<47:16,  2.24s/it]
 93%|█████████▎| 17359/18627 [9:50:46<47:22,  2.24s/it]
 93%|█████████▎| 17360/18627 [9:50:48<47:30,  2.25s/it]
                                                       
{'loss': 0.7846, 'grad_norm': 4.9493608474731445, 'learning_rate': 6.042068757087793e-08, 'epoch': 0.93}
+
 93%|█████████▎| 17360/18627 [9:50:48<47:30,  2.25s/it]
 93%|█████████▎| 17361/18627 [9:50:51<47:36,  2.26s/it]
 93%|█████████▎| 17362/18627 [9:50:52<41:28,  1.97s/it]
 93%|█████████▎| 17363/18627 [9:50:54<43:21,  2.06s/it]
 93%|█████████▎| 17364/18627 [9:50:56<38:20,  1.82s/it]
 93%|█████████▎| 17365/18627 [9:50:58<41:03,  1.95s/it]
 93%|█████████▎| 17366/18627 [9:51:00<42:57,  2.04s/it]
 93%|█████████▎| 17367/18627 [9:51:01<38:24,  1.83s/it]
 93%|█████████▎| 17368/18627 [9:51:04<41:24,  1.97s/it]
 93%|█████████▎| 17369/18627 [9:51:06<43:12,  2.06s/it]
 93%|█████████▎| 17370/18627 [9:51:08<44:28,  2.12s/it]
                                                       
{'loss': 1.2661, 'grad_norm': 13.696374893188477, 'learning_rate': 5.947447644520682e-08, 'epoch': 0.93}
+
 93%|█████████▎| 17370/18627 [9:51:08<44:28,  2.12s/it]
 93%|█████████▎| 17371/18627 [9:51:09<38:36,  1.84s/it]
 93%|█████████▎| 17372/18627 [9:51:12<41:17,  1.97s/it]
 93%|█████████▎| 17373/18627 [9:51:13<36:51,  1.76s/it]
 93%|█████████▎| 17374/18627 [9:51:15<40:00,  1.92s/it]
 93%|█████████▎| 17375/18627 [9:51:17<42:11,  2.02s/it]
 93%|█████████▎| 17376/18627 [9:51:20<43:32,  2.09s/it]
 93%|█████████▎| 17377/18627 [9:51:22<44:55,  2.16s/it]
 93%|█████████▎| 17378/18627 [9:51:24<45:42,  2.20s/it]
 93%|█████████▎| 17379/18627 [9:51:27<46:09,  2.22s/it]
 93%|█████████▎| 17380/18627 [9:51:28<39:47,  1.91s/it]
                                                       
{'loss': 1.129, 'grad_norm': 14.819186210632324, 'learning_rate': 5.853564373024784e-08, 'epoch': 0.93}
+
 93%|█████████▎| 17380/18627 [9:51:28<39:47,  1.91s/it]
 93%|█████████▎| 17381/18627 [9:51:30<42:01,  2.02s/it]
 93%|█████████▎| 17382/18627 [9:51:32<43:29,  2.10s/it]
 93%|█████████▎| 17383/18627 [9:51:34<38:17,  1.85s/it]
 93%|█████████▎| 17384/18627 [9:51:36<40:45,  1.97s/it]
 93%|█████████▎| 17385/18627 [9:51:38<42:25,  2.05s/it]
 93%|█████████▎| 17386/18627 [9:51:39<37:30,  1.81s/it]
 93%|█████████▎| 17387/18627 [9:51:42<40:20,  1.95s/it]
 93%|█████████▎| 17388/18627 [9:51:44<42:10,  2.04s/it]
 93%|█████████▎| 17389/18627 [9:51:45<37:41,  1.83s/it]
 93%|█████████▎| 17390/18627 [9:51:47<40:25,  1.96s/it]
                                                       
{'loss': 1.2165, 'grad_norm': 7.000197887420654, 'learning_rate': 5.760419226436248e-08, 'epoch': 0.93}
+
 93%|█████████▎| 17390/18627 [9:51:47<40:25,  1.96s/it]
 93%|█████████▎| 17391/18627 [9:51:50<42:16,  2.05s/it]
 93%|█████████▎| 17392/18627 [9:51:51<37:18,  1.81s/it]
 93%|█████████▎| 17393/18627 [9:51:53<40:05,  1.95s/it]
 93%|█████████▎| 17394/18627 [9:51:55<35:52,  1.75s/it]
 93%|█████████▎| 17395/18627 [9:51:57<38:59,  1.90s/it]
 93%|█████████▎| 17396/18627 [9:51:58<34:58,  1.70s/it]
 93%|█████████▎| 17397/18627 [9:52:00<38:24,  1.87s/it]
 93%|█████████▎| 17398/18627 [9:52:03<40:49,  1.99s/it]
 93%|█████████▎| 17399/18627 [9:52:05<42:37,  2.08s/it]
 93%|█████████▎| 17400/18627 [9:52:07<43:49,  2.14s/it]
                                                       
{'loss': 1.2956, 'grad_norm': 7.475527286529541, 'learning_rate': 5.668012486359675e-08, 'epoch': 0.93}
+
 93%|█████████▎| 17400/18627 [9:52:07<43:49,  2.14s/it]
 93%|█████████▎| 17401/18627 [9:52:08<38:29,  1.88s/it]
 93%|██���██████▎| 17402/18627 [9:52:11<40:45,  2.00s/it]
 93%|█████████▎| 17403/18627 [9:52:13<42:19,  2.07s/it]
 93%|█████████▎| 17404/18627 [9:52:14<36:51,  1.81s/it]
 93%|█████████▎| 17405/18627 [9:52:15<33:39,  1.65s/it]
 93%|█████████▎| 17406/18627 [9:52:18<37:31,  1.84s/it]
 93%|█████████▎| 17407/18627 [9:52:20<40:12,  1.98s/it]
 93%|█████████▎| 17408/18627 [9:52:21<36:01,  1.77s/it]
 93%|█████████▎| 17409/18627 [9:52:24<38:49,  1.91s/it]
 93%|█████████▎| 17410/18627 [9:52:26<40:55,  2.02s/it]
                                                       
{'loss': 1.3628, 'grad_norm': 7.747924327850342, 'learning_rate': 5.5763444321671744e-08, 'epoch': 0.93}
+
 93%|█████████▎| 17410/18627 [9:52:26<40:55,  2.02s/it]
 93%|█████████▎| 17411/18627 [9:52:28<42:30,  2.10s/it]
 93%|█████████▎| 17412/18627 [9:52:30<43:34,  2.15s/it]
 93%|█████████▎| 17413/18627 [9:52:33<44:14,  2.19s/it]
 93%|█████████▎| 17414/18627 [9:52:34<38:48,  1.92s/it]
 93%|█████████▎| 17415/18627 [9:52:36<40:58,  2.03s/it]
 93%|█████████▎| 17416/18627 [9:52:38<42:21,  2.10s/it]
 94%|█████████▎| 17417/18627 [9:52:41<43:34,  2.16s/it]
 94%|█████████▎| 17418/18627 [9:52:43<44:19,  2.20s/it]
 94%|█████████▎| 17419/18627 [9:52:44<38:57,  1.93s/it]
 94%|█████████▎| 17420/18627 [9:52:46<35:01,  1.74s/it]
                                                       
{'loss': 1.1587, 'grad_norm': 13.559918403625488, 'learning_rate': 5.485415340997669e-08, 'epoch': 0.94}
+
 94%|█████████▎| 17420/18627 [9:52:46<35:01,  1.74s/it]
 94%|█████████▎| 17421/18627 [9:52:48<38:14,  1.90s/it]
 94%|█████████▎| 17422/18627 [9:52:49<34:14,  1.71s/it]
 94%|█████████▎| 17423/18627 [9:52:51<37:33,  1.87s/it]
 94%|█████████▎| 17424/18627 [9:52:54<39:46,  1.98s/it]
 94%|█████████▎| 17425/18627 [9:52:56<41:14,  2.06s/it]
 94%|█████████▎| 17426/18627 [9:52:58<42:32,  2.12s/it]
 94%|█████████▎| 17427/18627 [9:53:00<43:13,  2.16s/it]
 94%|█████████▎| 17428/18627 [9:53:03<43:38,  2.18s/it]
 94%|█████████▎| 17429/18627 [9:53:05<44:06,  2.21s/it]
 94%|█████████▎| 17430/18627 [9:53:06<38:38,  1.94s/it]
                                                       
{'loss': 1.0131, 'grad_norm': 13.788677215576172, 'learning_rate': 5.3952254877559784e-08, 'epoch': 0.94}
+
 94%|█████████▎| 17430/18627 [9:53:06<38:38,  1.94s/it]
 94%|█████████▎| 17431/18627 [9:53:08<34:31,  1.73s/it]
 94%|█████████▎| 17432/18627 [9:53:10<37:42,  1.89s/it]
 94%|█████████▎| 17433/18627 [9:53:12<39:47,  2.00s/it]
 94%|█████████▎| 17434/18627 [9:53:14<41:17,  2.08s/it]
 94%|█████████▎| 17435/18627 [9:53:16<36:31,  1.84s/it]
 94%|█████████▎| 17436/18627 [9:53:18<39:03,  1.97s/it]
 94%|█████████▎| 17437/18627 [9:53:20<40:52,  2.06s/it]
 94%|█████████▎| 17438/18627 [9:53:22<42:06,  2.12s/it]
 94%|█████████▎| 17439/18627 [9:53:24<37:08,  1.88s/it]
 94%|█████████▎| 17440/18627 [9:53:26<39:22,  1.99s/it]
                                                       
{'loss': 1.1329, 'grad_norm': 4.872044086456299, 'learning_rate': 5.3057751451119075e-08, 'epoch': 0.94}
+
 94%|█████████▎| 17440/18627 [9:53:26<39:22,  1.99s/it]
 94%|█████████▎| 17441/18627 [9:53:28<41:06,  2.08s/it]
 94%|█████████▎| 17442/18627 [9:53:31<42:21,  2.15s/it]
 94%|█████████▎| 17443/18627 [9:53:32<36:41,  1.86s/it]
 94%|█████████▎| 17444/18627 [9:53:34<39:04,  1.98s/it]
 94%|█████████▎| 17445/18627 [9:53:36<40:39,  2.06s/it]
 94%|█████████▎| 17446/18627 [9:53:39<41:44,  2.12s/it]
 94%|█████████▎| 17447/18627 [9:53:40<36:34,  1.86s/it]
 94%|█████████▎| 17448/18627 [9:53:42<39:01,  1.99s/it]
 94%|█████████▎| 17449/18627 [9:53:44<40:41,  2.07s/it]
 94%|█████████▎| 17450/18627 [9:53:47<41:36,  2.12s/it]
                                                       
{'loss': 0.97, 'grad_norm': 7.38754415512085, 'learning_rate': 5.217064583499659e-08, 'epoch': 0.94}
+
 94%|█████████▎| 17450/18627 [9:53:47<41:36,  2.12s/it]
 94%|█████████▎| 17451/18627 [9:53:49<42:26,  2.17s/it]
 94%|█████████▎| 17452/18627 [9:53:51<43:57,  2.24s/it]
 94%|█████████▎| 17453/18627 [9:53:52<36:19,  1.86s/it]
 94%|█████████▎| 17454/18627 [9:53:53<32:37,  1.67s/it]
 94%|█████████▎| 17455/18627 [9:53:56<36:10,  1.85s/it]
 94%|█████████▎| 17456/18627 [9:53:58<38:31,  1.97s/it]
 94%|█████████▎| 17457/18627 [9:54:00<40:07,  2.06s/it]
 94%|█████████▎| 17458/18627 [9:54:01<35:25,  1.82s/it]
 94%|█████████▎| 17459/18627 [9:54:03<32:09,  1.65s/it]
 94%|█████████▎| 17460/18627 [9:54:04<30:01,  1.54s/it]
                                                       
{'loss': 1.4056, 'grad_norm': 14.111727714538574, 'learning_rate': 5.129094071116725e-08, 'epoch': 0.94}
+
 94%|█████████▎| 17460/18627 [9:54:04<30:01,  1.54s/it]
 94%|█████████▎| 17461/18627 [9:54:06<34:13,  1.76s/it]
 94%|█████████▎| 17462/18627 [9:54:09<37:08,  1.91s/it]
 94%|█████████▍| 17463/18627 [9:54:11<38:59,  2.01s/it]
 94%|█████████▍| 17464/18627 [9:54:12<34:46,  1.79s/it]
 94%|█████████▍| 17465/18627 [9:54:14<37:33,  1.94s/it]
 94%|█████████▍| 17466/18627 [9:54:17<39:24,  2.04s/it]
 94%|█████████▍| 17467/18627 [9:54:19<40:41,  2.10s/it]
 94%|█████████▍| 17468/18627 [9:54:21<41:30,  2.15s/it]
 94%|█████████▍| 17469/18627 [9:54:23<42:10,  2.19s/it]
 94%|█████████▍| 17470/18627 [9:54:26<42:31,  2.21s/it]
                                                       
{'loss': 0.8585, 'grad_norm': 7.140087604522705, 'learning_rate': 5.041863873923275e-08, 'epoch': 0.94}
+
 94%|█████████▍| 17470/18627 [9:54:26<42:31,  2.21s/it]
 94%|█████████▍| 17471/18627 [9:54:27<36:39,  1.90s/it]
 94%|█████████▍| 17472/18627 [9:54:29<38:35,  2.01s/it]
 94%|█████████▍| 17473/18627 [9:54:31<39:56,  2.08s/it]
 94%|█████████▍| 17474/18627 [9:54:34<40:53,  2.13s/it]
 94%|█████████▍| 17475/18627 [9:54:35<35:49,  1.87s/it]
 94%|█████████▍| 17476/18627 [9:54:37<38:10,  1.99s/it]
 94%|█████████▍| 17477/18627 [9:54:38<33:46,  1.76s/it]
 94%|█████████▍| 17478/18627 [9:54:41<37:04,  1.94s/it]
 94%|█████████▍| 17479/18627 [9:54:42<33:08,  1.73s/it]
 94%|█████████▍| 17480/18627 [9:54:44<36:09,  1.89s/it]
                                                       
{'loss': 1.3199, 'grad_norm': 9.142769813537598, 'learning_rate': 4.9553742556412984e-08, 'epoch': 0.94}
+
 94%|█████████▍| 17480/18627 [9:54:44<36:09,  1.89s/it]
 94%|█████████▍| 17481/18627 [9:54:45<32:27,  1.70s/it]
 94%|█████████▍| 17482/18627 [9:54:48<35:35,  1.87s/it]
 94%|█████████▍| 17483/18627 [9:54:50<37:56,  1.99s/it]
 94%|█████████▍| 17484/18627 [9:54:52<39:33,  2.08s/it]
 94%|█████████▍| 17485/18627 [9:54:55<40:38,  2.14s/it]
 94%|█████████▍| 17486/18627 [9:54:57<41:17,  2.17s/it]
 94%|█████████▍| 17487/18627 [9:54:59<41:49,  2.20s/it]
 94%|█████████▍| 17488/18627 [9:55:01<42:17,  2.23s/it]
 94%|█████████▍| 17489/18627 [9:55:04<42:32,  2.24s/it]
 94%|█████████▍| 17490/18627 [9:55:06<42:29,  2.24s/it]
                                                       
{'loss': 0.8005, 'grad_norm': 4.277403354644775, 'learning_rate': 4.869625477753798e-08, 'epoch': 0.94}
+
 94%|█████████▍| 17490/18627 [9:55:06<42:29,  2.24s/it]
 94%|█████████▍| 17491/18627 [9:55:08<42:35,  2.25s/it]
 94%|█████████▍| 17492/18627 [9:55:10<42:40,  2.26s/it]
 94%|█████████▍| 17493/18627 [9:55:13<42:27,  2.25s/it]
 94%|█████████▍| 17494/18627 [9:55:15<42:25,  2.25s/it]
 94%|█████████▍| 17495/18627 [9:55:17<42:25,  2.25s/it]
 94%|█████████▍| 17496/18627 [9:55:19<42:29,  2.25s/it]
 94%|█████████▍| 17497/18627 [9:55:21<36:36,  1.94s/it]
 94%|█████████▍| 17498/18627 [9:55:23<38:05,  2.02s/it]
 94%|█████████▍| 17499/18627 [9:55:25<39:20,  2.09s/it]
 94%|█████████▍| 17500/18627 [9:55:27<40:13,  2.14s/it]
                                                       
{'loss': 0.7965, 'grad_norm': 5.613715171813965, 'learning_rate': 4.7846177995039544e-08, 'epoch': 0.94}
+
 94%|█████████▍| 17500/18627 [9:55:27<40:13,  2.14s/it]
 94%|█████████▍| 17501/18627 [9:55:30<40:52,  2.18s/it]
 94%|█████████▍| 17502/18627 [9:55:32<41:08,  2.19s/it]
 94%|█████████▍| 17503/18627 [9:55:34<41:22,  2.21s/it]
 94%|█████████▍| 17504/18627 [9:55:36<41:34,  2.22s/it]
 94%|█████████▍| 17505/18627 [9:55:39<41:35,  2.22s/it]
 94%|█████████▍| 17506/18627 [9:55:41<41:37,  2.23s/it]
 94%|█████████▍| 17507/18627 [9:55:43<41:43,  2.23s/it]
 94%|█████████▍| 17508/18627 [9:55:45<41:42,  2.24s/it]
 94%|█████████▍| 17509/18627 [9:55:48<41:50,  2.25s/it]
 94%|█████████▍| 17510/18627 [9:55:50<41:55,  2.25s/it]
                                                       
{'loss': 0.5945, 'grad_norm': 5.491856098175049, 'learning_rate': 4.7003514778945215e-08, 'epoch': 0.94}
+
 94%|█████████▍| 17510/18627 [9:55:50<41:55,  2.25s/it]
 94%|█████████▍| 17511/18627 [9:55:52<41:53,  2.25s/it]
 94%|█████████▍| 17512/18627 [9:55:54<42:02,  2.26s/it]
 94%|█████████▍| 17513/18627 [9:55:57<41:54,  2.26s/it]
 94%|█████████▍| 17514/18627 [9:55:59<42:02,  2.27s/it]
 94%|█████████▍| 17515/18627 [9:56:00<36:37,  1.98s/it]
 94%|█████████▍| 17516/18627 [9:56:01<32:30,  1.76s/it]
 94%|█████████▍| 17517/18627 [9:56:03<29:57,  1.62s/it]
 94%|█████████▍| 17518/18627 [9:56:05<33:42,  1.82s/it]
 94%|█████████▍| 17519/18627 [9:56:07<36:05,  1.95s/it]
 94%|█████████▍| 17520/18627 [9:56:09<32:04,  1.74s/it]
                                                       
{'loss': 1.4097, 'grad_norm': 15.6240816116333, 'learning_rate': 4.6168267676867386e-08, 'epoch': 0.94}
+
 94%|█████████▍| 17520/18627 [9:56:09<32:04,  1.74s/it]
 94%|█████████▍| 17521/18627 [9:56:11<35:01,  1.90s/it]
 94%|█████████▍| 17522/18627 [9:56:13<37:12,  2.02s/it]
 94%|█████████▍| 17523/18627 [9:56:15<38:37,  2.10s/it]
 94%|█████████▍| 17524/18627 [9:56:16<32:14,  1.75s/it]
 94%|█████████▍| 17525/18627 [9:56:19<34:55,  1.90s/it]
 94%|█████████▍| 17526/18627 [9:56:21<37:03,  2.02s/it]
 94%|█████████▍| 17527/18627 [9:56:23<38:18,  2.09s/it]
 94%|█████████▍| 17528/18627 [9:56:25<39:12,  2.14s/it]
 94%|█████████▍| 17529/18627 [9:56:27<34:20,  1.88s/it]
 94%|█████████▍| 17530/18627 [9:56:29<36:24,  1.99s/it]
                                                       
{'loss': 0.8255, 'grad_norm': 5.749660491943359, 'learning_rate': 4.534043921399889e-08, 'epoch': 0.94}
+
 94%|█████████▍| 17530/18627 [9:56:29<36:24,  1.99s/it]
 94%|█████████▍| 17531/18627 [9:56:31<38:01,  2.08s/it]
 94%|█████████▍| 17532/18627 [9:56:32<31:38,  1.73s/it]
 94%|█████████▍| 17533/18627 [9:56:34<34:35,  1.90s/it]
 94%|█████████▍| 17534/18627 [9:56:37<36:32,  2.01s/it]
 94%|█████████▍| 17535/18627 [9:56:38<32:27,  1.78s/it]
 94%|█████████▍| 17536/18627 [9:56:40<34:58,  1.92s/it]
 94%|█████████▍| 17537/18627 [9:56:43<36:52,  2.03s/it]
 94%|█████████▍| 17538/18627 [9:56:45<38:08,  2.10s/it]
 94%|█████████▍| 17539/18627 [9:56:47<39:01,  2.15s/it]
 94%|█████████▍| 17540/18627 [9:56:49<39:50,  2.20s/it]
                                                       
{'loss': 0.9009, 'grad_norm': 5.679696559906006, 'learning_rate': 4.4520031893103e-08, 'epoch': 0.94}
+
 94%|█████████▍| 17540/18627 [9:56:49<39:50,  2.20s/it]
 94%|█████████▍| 17541/18627 [9:56:52<40:11,  2.22s/it]
 94%|█████████▍| 17542/18627 [9:56:53<35:10,  1.95s/it]
 94%|█████████▍| 17543/18627 [9:56:55<36:44,  2.03s/it]
 94%|█████████▍| 17544/18627 [9:56:57<38:01,  2.11s/it]
 94%|█████████▍| 17545/18627 [9:57:00<39:04,  2.17s/it]
 94%|█████████▍| 17546/18627 [9:57:02<39:35,  2.20s/it]
 94%|█████████▍| 17547/18627 [9:57:04<39:49,  2.21s/it]
 94%|█████████▍| 17548/18627 [9:57:07<40:15,  2.24s/it]
 94%|█████████▍| 17549/18627 [9:57:09<40:16,  2.24s/it]
 94%|█████████▍| 17550/18627 [9:57:11<40:22,  2.25s/it]
                                                       
{'loss': 0.7823, 'grad_norm': 6.706498622894287, 'learning_rate': 4.370704819450705e-08, 'epoch': 0.94}
+
 94%|█████████▍| 17550/18627 [9:57:11<40:22,  2.25s/it]
 94%|█████████▍| 17551/18627 [9:57:13<40:27,  2.26s/it]
 94%|█████████▍| 17552/18627 [9:57:16<40:19,  2.25s/it]
 94%|█████████▍| 17553/18627 [9:57:18<40:26,  2.26s/it]
 94%|█████████▍| 17554/18627 [9:57:19<34:56,  1.95s/it]
 94%|█████████▍| 17555/18627 [9:57:21<36:37,  2.05s/it]
 94%|█████████▍| 17556/18627 [9:57:24<37:47,  2.12s/it]
 94%|████���████▍| 17557/18627 [9:57:26<38:34,  2.16s/it]
 94%|█████████▍| 17558/18627 [9:57:28<38:57,  2.19s/it]
 94%|█████████▍| 17559/18627 [9:57:30<39:23,  2.21s/it]
 94%|█████████▍| 17560/18627 [9:57:33<39:52,  2.24s/it]
                                                       
{'loss': 0.8138, 'grad_norm': 8.199049949645996, 'learning_rate': 4.290149057609494e-08, 'epoch': 0.94}
+
 94%|█████████▍| 17560/18627 [9:57:33<39:52,  2.24s/it]
 94%|█████████▍| 17561/18627 [9:57:35<39:56,  2.25s/it]
 94%|█████████▍| 17562/18627 [9:57:37<39:51,  2.25s/it]
 94%|█████████▍| 17563/18627 [9:57:39<39:47,  2.24s/it]
 94%|█████████▍| 17564/18627 [9:57:42<39:37,  2.24s/it]
 94%|█████████▍| 17565/18627 [9:57:44<39:59,  2.26s/it]
 94%|█████████▍| 17566/18627 [9:57:46<39:55,  2.26s/it]
 94%|█████████▍| 17567/18627 [9:57:49<39:59,  2.26s/it]
 94%|█████████▍| 17568/18627 [9:57:51<39:43,  2.25s/it]
 94%|█████████▍| 17569/18627 [9:57:52<34:19,  1.95s/it]
 94%|█████████▍| 17570/18627 [9:57:54<35:53,  2.04s/it]
                                                       
{'loss': 0.8175, 'grad_norm': 5.3225860595703125, 'learning_rate': 4.21033614732988e-08, 'epoch': 0.94}
+
 94%|█████████▍| 17570/18627 [9:57:54<35:53,  2.04s/it]
 94%|█████████▍| 17571/18627 [9:57:55<31:36,  1.80s/it]
 94%|█████████▍| 17572/18627 [9:57:58<33:58,  1.93s/it]
 94%|█████████▍| 17573/18627 [9:57:59<28:46,  1.64s/it]
 94%|█████████▍| 17574/18627 [9:58:01<31:51,  1.81s/it]
 94%|█████████▍| 17575/18627 [9:58:02<28:47,  1.64s/it]
 94%|█████████▍| 17576/18627 [9:58:04<31:56,  1.82s/it]
 94%|█████████▍| 17577/18627 [9:58:07<34:12,  1.95s/it]
 94%|█████████▍| 17578/18627 [9:58:09<35:47,  2.05s/it]
 94%|█████████▍| 17579/18627 [9:58:11<36:56,  2.11s/it]
 94%|█████████▍| 17580/18627 [9:58:13<37:40,  2.16s/it]
                                                       
{'loss': 1.0212, 'grad_norm': 6.2288103103637695, 'learning_rate': 4.131266329909234e-08, 'epoch': 0.94}
+
 94%|█████████▍| 17580/18627 [9:58:13<37:40,  2.16s/it]
 94%|█████████▍| 17581/18627 [9:58:16<38:00,  2.18s/it]
 94%|█████████▍| 17582/18627 [9:58:18<38:22,  2.20s/it]
 94%|█████████▍| 17583/18627 [9:58:20<38:48,  2.23s/it]
 94%|█████████▍| 17584/18627 [9:58:23<38:59,  2.24s/it]
 94%|█████████▍| 17585/18627 [9:58:24<33:49,  1.95s/it]
 94%|█████████▍| 17586/18627 [9:58:25<30:07,  1.74s/it]
 94%|█████████▍| 17587/18627 [9:58:27<32:53,  1.90s/it]
 94%|█████████▍| 17588/18627 [9:58:30<34:43,  2.01s/it]
 94%|█████████▍| 17589/18627 [9:58:32<35:57,  2.08s/it]
 94%|█████████▍| 17590/18627 [9:58:34<36:48,  2.13s/it]
                                                       
{'loss': 0.9169, 'grad_norm': 5.749591827392578, 'learning_rate': 4.0529398443983906e-08, 'epoch': 0.94}
+
 94%|█████████▍| 17590/18627 [9:58:34<36:48,  2.13s/it]
 94%|█████████▍| 17591/18627 [9:58:35<32:10,  1.86s/it]
 94%|█████████▍| 17592/18627 [9:58:38<34:22,  1.99s/it]
 94%|█████████▍| 17593/18627 [9:58:40<35:36,  2.07s/it]
 94%|█████████▍| 17594/18627 [9:58:42<36:29,  2.12s/it]
 94%|█████████▍| 17595/18627 [9:58:46<44:07,  2.56s/it]
 94%|█████████▍| 17596/18627 [9:58:48<42:26,  2.47s/it]
 94%|█████████▍| 17597/18627 [9:58:50<41:25,  2.41s/it]
 94%|█████████▍| 17598/18627 [9:58:52<40:37,  2.37s/it]
 94%|█████████▍| 17599/18627 [9:58:54<35:01,  2.04s/it]
 94%|█████████▍| 17600/18627 [9:58:56<36:03,  2.11s/it]
                                                       
{'loss': 0.9355, 'grad_norm': 7.289031505584717, 'learning_rate': 3.975356927600843e-08, 'epoch': 0.94}
+
 94%|█████████▍| 17600/18627 [9:58:56<36:03,  2.11s/it]
 94%|█████████▍| 17601/18627 [9:58:57<31:08,  1.82s/it]
 94%|█████████▍| 17602/18627 [9:58:59<33:30,  1.96s/it]
 95%|█████████▍| 17603/18627 [9:59:02<35:00,  2.05s/it]
 95%|█████████▍| 17604/18627 [9:59:04<36:03,  2.12s/it]
 95%|█████████▍| 17605/18627 [9:59:06<36:45,  2.16s/it]
 95%|█████████▍| 17606/18627 [9:59:08<37:13,  2.19s/it]
 95%|█████████▍| 17607/18627 [9:59:10<32:21,  1.90s/it]
 95%|█████████▍| 17608/18627 [9:59:12<34:11,  2.01s/it]
 95%|█████████▍| 17609/18627 [9:59:14<35:26,  2.09s/it]
 95%|█████████▍| 17610/18627 [9:59:16<31:04,  1.83s/it]
                                                       
{'loss': 1.3169, 'grad_norm': 16.4879093170166, 'learning_rate': 3.898517814072078e-08, 'epoch': 0.95}
+
 95%|█████████▍| 17610/18627 [9:59:16<31:04,  1.83s/it]
 95%|█████████▍| 17611/18627 [9:59:18<33:05,  1.95s/it]
 95%|█████████▍| 17612/18627 [9:59:20<34:35,  2.04s/it]
 95%|█████████▍| 17613/18627 [9:59:22<35:28,  2.10s/it]
 95%|█████████▍| 17614/18627 [9:59:24<36:08,  2.14s/it]
 95%|█████████▍| 17615/18627 [9:59:27<36:37,  2.17s/it]
 95%|█████████▍| 17616/18627 [9:59:28<31:53,  1.89s/it]
 95%|█████████▍| 17617/18627 [9:59:29<28:49,  1.71s/it]
 95%|█████████▍| 17618/18627 [9:59:32<31:33,  1.88s/it]
 95%|█████████▍| 17619/18627 [9:59:33<28:28,  1.70s/it]
 95%|█████████▍| 17620/18627 [9:59:34<25:57,  1.55s/it]
                                                       
{'loss': 1.4638, 'grad_norm': 13.460213661193848, 'learning_rate': 3.822422736118825e-08, 'epoch': 0.95}
+
 95%|█████████▍| 17620/18627 [9:59:34<25:57,  1.55s/it]
 95%|█████████▍| 17621/18627 [9:59:36<29:33,  1.76s/it]
 95%|█████████▍| 17622/18627 [9:59:39<32:04,  1.92s/it]
 95%|█████████▍| 17623/18627 [9:59:40<29:03,  1.74s/it]
 95%|█████████▍| 17624/18627 [9:59:42<31:46,  1.90s/it]
 95%|█████████▍| 17625/18627 [9:59:44<33:30,  2.01s/it]
 95%|█████████▍| 17626/18627 [9:59:46<29:41,  1.78s/it]
 95%|█████████▍| 17627/18627 [9:59:48<32:11,  1.93s/it]
 95%|█████████▍| 17628/18627 [9:59:50<33:52,  2.04s/it]
 95%|█████████▍| 17629/18627 [9:59:52<35:11,  2.12s/it]
 95%|█████████▍| 17630/18627 [9:59:55<35:54,  2.16s/it]
                                                       
{'loss': 1.059, 'grad_norm': 5.899116039276123, 'learning_rate': 3.74707192379839e-08, 'epoch': 0.95}
+
 95%|█████████▍| 17630/18627 [9:59:55<35:54,  2.16s/it]
 95%|█████████▍| 17631/18627 [9:59:57<36:28,  2.20s/it]
 95%|█████████▍| 17632/18627 [9:59:59<36:53,  2.22s/it]
 95%|█████████▍| 17633/18627 [10:00:02<37:11,  2.25s/it]
 95%|█████████▍| 17634/18627 [10:00:03<32:20,  1.95s/it]
 95%|█████████▍| 17635/18627 [10:00:04<28:50,  1.74s/it]
 95%|█████████▍| 17636/18627 [10:00:06<31:32,  1.91s/it]
 95%|█████████▍| 17637/18627 [10:00:08<28:36,  1.73s/it]
 95%|█████████▍| 17638/18627 [10:00:09<26:14,  1.59s/it]
 95%|█████████▍| 17639/18627 [10:00:11<29:31,  1.79s/it]
 95%|█████████▍| 17640/18627 [10:00:12<26:31,  1.61s/it]
                                                        
{'loss': 1.658, 'grad_norm': 15.04935359954834, 'learning_rate': 3.672465604917963e-08, 'epoch': 0.95}
+
 95%|█████████▍| 17640/18627 [10:00:12<26:31,  1.61s/it]
 95%|█████████▍| 17641/18627 [10:00:15<29:41,  1.81s/it]
 95%|█████████▍| 17642/18627 [10:00:18<38:27,  2.34s/it]
 95%|█████████▍| 17643/18627 [10:00:21<37:53,  2.31s/it]
 95%|█████████▍| 17644/18627 [10:00:23<37:33,  2.29s/it]
 95%|█████████▍| 17645/18627 [10:00:25<37:19,  2.28s/it]
 95%|█████████▍| 17646/18627 [10:00:27<37:05,  2.27s/it]
 95%|█████████▍| 17647/18627 [10:00:30<36:55,  2.26s/it]
 95%|█████████▍| 17648/18627 [10:00:32<36:58,  2.27s/it]
 95%|█████████▍| 17649/18627 [10:00:36<44:50,  2.75s/it]
 95%|█████████▍| 17650/18627 [10:00:37<37:39,  2.31s/it]
                                                        
{'loss': 0.809, 'grad_norm': 14.351346015930176, 'learning_rate': 3.598604005033896e-08, 'epoch': 0.95}
+
 95%|█████████▍| 17650/18627 [10:00:37<37:39,  2.31s/it]
 95%|█████████▍| 17651/18627 [10:00:39<37:24,  2.30s/it]
 95%|█████████▍| 17652/18627 [10:00:42<37:11,  2.29s/it]
 95%|█████████▍| 17653/18627 [10:00:44<37:05,  2.29s/it]
 95%|█████████▍| 17654/18627 [10:00:46<36:51,  2.27s/it]
 95%|█████████▍| 17655/18627 [10:00:48<36:45,  2.27s/it]
 95%|█████████▍| 17656/18627 [10:00:51<36:39,  2.26s/it]
 95%|█████████▍| 17657/18627 [10:00:53<36:34,  2.26s/it]
 95%|█████████▍| 17658/18627 [10:00:55<36:25,  2.26s/it]
 95%|█████████▍| 17659/18627 [10:00:57<36:25,  2.26s/it]
 95%|█████████▍| 17660/18627 [10:01:00<36:15,  2.25s/it]
                                                        
{'loss': 0.5764, 'grad_norm': 5.682928562164307, 'learning_rate': 3.525487347451062e-08, 'epoch': 0.95}
+
 95%|█████████▍| 17660/18627 [10:01:00<36:15,  2.25s/it]
 95%|█████████▍| 17661/18627 [10:01:02<36:10,  2.25s/it]
 95%|█████████▍| 17662/18627 [10:01:04<36:02,  2.24s/it]
 95%|█████████▍| 17663/18627 [10:01:06<36:05,  2.25s/it]
 95%|█████████▍| 17664/18627 [10:01:08<31:14,  1.95s/it]
 95%|█████████▍| 17665/18627 [10:01:09<27:49,  1.74s/it]
 95%|█████████▍| 17666/18627 [10:01:10<25:32,  1.59s/it]
 95%|█████████▍| 17667/18627 [10:01:12<28:34,  1.79s/it]
 95%|█████████▍| 17668/18627 [10:01:14<25:58,  1.63s/it]
 95%|█████████▍| 17669/18627 [10:01:16<28:53,  1.81s/it]
 95%|█████████▍| 17670/18627 [10:01:18<30:53,  1.94s/it]
                                                        
{'loss': 1.407, 'grad_norm': 6.558468341827393, 'learning_rate': 3.4531158532221384e-08, 'epoch': 0.95}
+
 95%|█████████▍| 17670/18627 [10:01:18<30:53,  1.94s/it]
 95%|█████████▍| 17671/18627 [10:01:20<32:23,  2.03s/it]
 95%|█████████▍| 17672/18627 [10:01:23<33:28,  2.10s/it]
 95%|█████████▍| 17673/18627 [10:01:25<34:06,  2.14s/it]
 95%|█████████▍| 17674/18627 [10:01:26<28:16,  1.78s/it]
 95%|█████████▍| 17675/18627 [10:01:28<30:28,  1.92s/it]
 95%|█████████▍| 17676/18627 [10:01:30<31:58,  2.02s/it]
 95%|█████████▍| 17677/18627 [10:01:31<28:17,  1.79s/it]
 95%|█████████▍| 17678/18627 [10:01:33<25:43,  1.63s/it]
 95%|█████████▍| 17679/18627 [10:01:35<28:30,  1.80s/it]
 95%|█████████▍| 17680/18627 [10:01:37<30:28,  1.93s/it]
                                                        
{'loss': 0.9844, 'grad_norm': 5.316196918487549, 'learning_rate': 3.3814897411469924e-08, 'epoch': 0.95}
+
 95%|█████████▍| 17680/18627 [10:01:37<30:28,  1.93s/it]
 95%|█████████▍| 17681/18627 [10:01:38<26:17,  1.67s/it]
 95%|█████████▍| 17682/18627 [10:01:40<28:58,  1.84s/it]
 95%|█████████▍| 17683/18627 [10:01:43<30:48,  1.96s/it]
 95%|█████████▍| 17684/18627 [10:01:45<32:06,  2.04s/it]
 95%|█████████▍| 17685/18627 [10:01:47<33:07,  2.11s/it]
 95%|█████████▍| 17686/18627 [10:01:48<29:12,  1.86s/it]
 95%|█████████▍| 17687/18627 [10:01:49<24:50,  1.59s/it]
 95%|█████████▍| 17688/18627 [10:01:52<27:55,  1.78s/it]
 95%|█████████▍| 17689/18627 [10:01:54<30:07,  1.93s/it]
 95%|█████████▍| 17690/18627 [10:01:56<31:36,  2.02s/it]
                                                        
{'loss': 0.955, 'grad_norm': 9.460941314697266, 'learning_rate': 3.310609227771933e-08, 'epoch': 0.95}
+
 95%|█████████▍| 17690/18627 [10:01:56<31:36,  2.02s/it]
 95%|█████████▍| 17691/18627 [10:01:58<32:43,  2.10s/it]
 95%|█████████▍| 17692/18627 [10:02:00<28:52,  1.85s/it]
 95%|█████████▍| 17693/18627 [10:02:02<30:45,  1.98s/it]
 95%|█████████▍| 17694/18627 [10:02:04<32:00,  2.06s/it]
 95%|█████████▍| 17695/18627 [10:02:05<28:14,  1.82s/it]
 95%|█████████▌| 17696/18627 [10:02:08<30:26,  1.96s/it]
 95%|█████████▌| 17697/18627 [10:02:10<31:38,  2.04s/it]
 95%|█████████▌| 17698/18627 [10:02:12<32:40,  2.11s/it]
 95%|█████████▌| 17699/18627 [10:02:15<33:32,  2.17s/it]
 95%|█████████▌| 17700/18627 [10:02:17<33:45,  2.19s/it]
                                                        
{'loss': 1.0483, 'grad_norm': 5.473784446716309, 'learning_rate': 3.240474527389209e-08, 'epoch': 0.95}
+
 95%|█████████▌| 17700/18627 [10:02:17<33:45,  2.19s/it]
 95%|█████████▌| 17701/18627 [10:02:19<34:05,  2.21s/it]
 95%|█████████▌| 17702/18627 [10:02:21<34:30,  2.24s/it]
 95%|█████████▌| 17703/18627 [10:02:23<29:35,  1.92s/it]
 95%|█████████▌| 17704/18627 [10:02:24<26:23,  1.72s/it]
 95%|█████████▌| 17705/18627 [10:02:26<28:49,  1.88s/it]
 95%|█████████▌| 17706/18627 [10:02:27<24:36,  1.60s/it]
 95%|█████████▌| 17707/18627 [10:02:29<27:31,  1.79s/it]
 95%|█████████▌| 17708/18627 [10:02:31<25:03,  1.64s/it]
 95%|█████████▌| 17709/18627 [10:02:32<23:23,  1.53s/it]
 95%|█████████▌| 17710/18627 [10:02:34<26:39,  1.74s/it]
                                                        
{'loss': 1.4778, 'grad_norm': 6.276250839233398, 'learning_rate': 3.1710858520360986e-08, 'epoch': 0.95}
+
 95%|█████████▌| 17710/18627 [10:02:34<26:39,  1.74s/it]
 95%|█████████▌| 17711/18627 [10:02:36<28:55,  1.89s/it]
 95%|█████████▌| 17712/18627 [10:02:40<36:43,  2.41s/it]
 95%|█████████▌| 17713/18627 [10:02:42<35:51,  2.35s/it]
 95%|█████████▌| 17714/18627 [10:02:43<30:51,  2.03s/it]
 95%|█████████▌| 17715/18627 [10:02:46<31:44,  2.09s/it]
 95%|█████████▌| 17716/18627 [10:02:48<32:26,  2.14s/it]
 95%|█████████▌| 17717/18627 [10:02:50<32:56,  2.17s/it]
 95%|█████████▌| 17718/18627 [10:02:52<33:27,  2.21s/it]
 95%|█████████▌| 17719/18627 [10:02:55<33:33,  2.22s/it]
 95%|█████████▌| 17720/18627 [10:02:57<33:41,  2.23s/it]
                                                        
{'loss': 0.8079, 'grad_norm': 5.623984336853027, 'learning_rate': 3.102443411494627e-08, 'epoch': 0.95}
+
 95%|█████████▌| 17720/18627 [10:02:57<33:41,  2.23s/it]
 95%|█████████▌| 17721/18627 [10:02:58<29:08,  1.93s/it]
 95%|█████████▌| 17722/18627 [10:03:00<30:38,  2.03s/it]
 95%|█████████▌| 17723/18627 [10:03:03<31:47,  2.11s/it]
 95%|█████████▌| 17724/18627 [10:03:05<32:16,  2.14s/it]
 95%|█████████▌| 17725/18627 [10:03:07<32:37,  2.17s/it]
 95%|█████████▌| 17726/18627 [10:03:09<32:57,  2.19s/it]
 95%|█████████▌| 17727/18627 [10:03:12<33:15,  2.22s/it]
 95%|█████████▌| 17728/18627 [10:03:14<33:27,  2.23s/it]
 95%|█████████▌| 17729/18627 [10:03:15<29:06,  1.94s/it]
 95%|█████████▌| 17730/18627 [10:03:17<30:25,  2.03s/it]
                                                        
{'loss': 1.1052, 'grad_norm': 7.380721092224121, 'learning_rate': 3.0345474132905396e-08, 'epoch': 0.95}
+
 95%|█████████▌| 17730/18627 [10:03:17<30:25,  2.03s/it]
 95%|█████████▌| 17731/18627 [10:03:20<31:27,  2.11s/it]
 95%|█████████▌| 17732/18627 [10:03:21<26:06,  1.75s/it]
 95%|█████████▌| 17733/18627 [10:03:22<23:56,  1.61s/it]
 95%|█████████▌| 17734/18627 [10:03:24<26:48,  1.80s/it]
 95%|█████████▌| 17735/18627 [10:03:26<28:50,  1.94s/it]
 95%|█████████▌| 17736/18627 [10:03:29<30:11,  2.03s/it]
 95%|█████████▌| 17737/18627 [10:03:31<31:11,  2.10s/it]
 95%|█████████▌| 17738/18627 [10:03:33<31:47,  2.15s/it]
 95%|█████████▌| 17739/18627 [10:03:36<32:23,  2.19s/it]
 95%|█████████▌| 17740/18627 [10:03:38<32:47,  2.22s/it]
                                                        
{'loss': 0.8392, 'grad_norm': 4.901718616485596, 'learning_rate': 2.9673980626930554e-08, 'epoch': 0.95}
+
 95%|█████████▌| 17740/18627 [10:03:38<32:47,  2.22s/it]
 95%|█████████▌| 17741/18627 [10:03:40<32:58,  2.23s/it]
 95%|█████████▌| 17742/18627 [10:03:41<28:37,  1.94s/it]
 95%|█████████▌| 17743/18627 [10:03:43<25:30,  1.73s/it]
 95%|█████████▌| 17744/18627 [10:03:45<27:47,  1.89s/it]
 95%|█████████▌| 17745/18627 [10:03:47<29:29,  2.01s/it]
 95%|█████████▌| 17746/18627 [10:03:49<30:31,  2.08s/it]
 95%|█████████▌| 17747/18627 [10:03:52<31:14,  2.13s/it]
 95%|█████████▌| 17748/18627 [10:03:54<31:47,  2.17s/it]
 95%|█████████▌| 17749/18627 [10:03:55<27:46,  1.90s/it]
 95%|█████████▌| 17750/18627 [10:03:57<29:22,  2.01s/it]
                                                        
{'loss': 1.2105, 'grad_norm': 10.777400016784668, 'learning_rate': 2.900995562713921e-08, 'epoch': 0.95}
+
 95%|█████████▌| 17750/18627 [10:03:57<29:22,  2.01s/it]
 95%|█████████▌| 17751/18627 [10:04:00<30:27,  2.09s/it]
 95%|█████████▌| 17752/18627 [10:04:01<26:54,  1.84s/it]
 95%|█████████▌| 17753/18627 [10:04:03<28:39,  1.97s/it]
 95%|█████████▌| 17754/18627 [10:04:04<24:11,  1.66s/it]
 95%|█████████▌| 17755/18627 [10:04:06<26:46,  1.84s/it]
 95%|█████████▌| 17756/18627 [10:04:07<22:32,  1.55s/it]
 95%|█████████▌| 17757/18627 [10:04:10<25:31,  1.76s/it]
 95%|█████████▌| 17758/18627 [10:04:11<23:24,  1.62s/it]
 95%|█████████▌| 17759/18627 [10:04:13<26:12,  1.81s/it]
 95%|█████████▌| 17760/18627 [10:04:15<28:10,  1.95s/it]
                                                        
{'loss': 0.9428, 'grad_norm': 4.757724761962891, 'learning_rate': 2.8353401141070215e-08, 'epoch': 0.95}
+
 95%|█████████▌| 17760/18627 [10:04:15<28:10,  1.95s/it]
 95%|█████████▌| 17761/18627 [10:04:18<29:29,  2.04s/it]
 95%|█████████▌| 17762/18627 [10:04:20<30:23,  2.11s/it]
 95%|█████████▌| 17763/18627 [10:04:21<26:36,  1.85s/it]
 95%|█████████▌| 17764/18627 [10:04:23<28:16,  1.97s/it]
 95%|█████████▌| 17765/18627 [10:04:26<29:26,  2.05s/it]
 95%|█████████▌| 17766/18627 [10:04:28<30:17,  2.11s/it]
 95%|█████████▌| 17767/18627 [10:04:29<26:38,  1.86s/it]
 95%|█████████▌| 17768/18627 [10:04:30<23:55,  1.67s/it]
 95%|█████████▌| 17769/18627 [10:04:33<26:24,  1.85s/it]
 95%|█████████▌| 17770/18627 [10:04:35<28:08,  1.97s/it]
                                                        
{'loss': 1.1475, 'grad_norm': 9.286770820617676, 'learning_rate': 2.7704319153676606e-08, 'epoch': 0.95}
+
 95%|█████████▌| 17770/18627 [10:04:35<28:08,  1.97s/it]
 95%|█████████▌| 17771/18627 [10:04:37<29:23,  2.06s/it]
 95%|█████████▌| 17772/18627 [10:04:39<30:05,  2.11s/it]
 95%|█████████▌| 17773/18627 [10:04:41<26:20,  1.85s/it]
 95%|█████████▌| 17774/18627 [10:04:43<28:18,  1.99s/it]
 95%|█████████▌| 17775/18627 [10:04:45<29:25,  2.07s/it]
 95%|█████████▌| 17776/18627 [10:04:47<30:07,  2.12s/it]
 95%|█████████▌| 17777/18627 [10:04:49<26:23,  1.86s/it]
 95%|█████████▌| 17778/18627 [10:04:51<28:03,  1.98s/it]
 95%|█████████▌| 17779/18627 [10:04:53<29:16,  2.07s/it]
 95%|█████████▌| 17780/18627 [10:04:55<30:00,  2.13s/it]
                                                        
{'loss': 0.9576, 'grad_norm': 9.04317855834961, 'learning_rate': 2.7062711627320048e-08, 'epoch': 0.95}
+
 95%|█████████▌| 17780/18627 [10:04:55<30:00,  2.13s/it]
 95%|█████████▌| 17781/18627 [10:04:58<30:41,  2.18s/it]
 95%|█████████▌| 17782/18627 [10:05:00<31:07,  2.21s/it]
 95%|█████████▌| 17783/18627 [10:05:01<27:11,  1.93s/it]
 95%|█████████▌| 17784/18627 [10:05:03<24:38,  1.75s/it]
 95%|█████████▌| 17785/18627 [10:05:05<26:46,  1.91s/it]
 95%|█████████▌| 17786/18627 [10:05:07<28:18,  2.02s/it]
 95%|█████████▌| 17787/18627 [10:05:09<25:09,  1.80s/it]
 95%|█████████▌| 17788/18627 [10:05:11<27:06,  1.94s/it]
 96%|█████████▌| 17789/18627 [10:05:13<28:31,  2.04s/it]
 96%|█████████▌| 17790/18627 [10:05:15<29:23,  2.11s/it]
                                                        
{'loss': 1.2747, 'grad_norm': 10.708128929138184, 'learning_rate': 2.6428580501764444e-08, 'epoch': 0.96}
+
 96%|█████████▌| 17790/18627 [10:05:15<29:23,  2.11s/it]
 96%|█████████▌| 17791/18627 [10:05:17<25:48,  1.85s/it]
 96%|█████████▌| 17792/18627 [10:05:19<27:30,  1.98s/it]
 96%|█████████▌| 17793/18627 [10:05:21<28:40,  2.06s/it]
 96%|█████████▌| 17794/18627 [10:05:23<29:23,  2.12s/it]
 96%|█████████▌| 17795/18627 [10:05:25<26:00,  1.88s/it]
 96%|█████████▌| 17796/18627 [10:05:27<27:38,  2.00s/it]
 96%|█████████▌| 17797/18627 [10:05:29<28:44,  2.08s/it]
 96%|█████████▌| 17798/18627 [10:05:31<29:21,  2.13s/it]
 96%|█████████▌| 17799/18627 [10:05:34<29:48,  2.16s/it]
 96%|█████████▌| 17800/18627 [10:05:35<25:58,  1.88s/it]
                                                        
{'loss': 1.181, 'grad_norm': 14.917594909667969, 'learning_rate': 2.5801927694170948e-08, 'epoch': 0.96}
+
 96%|█████████▌| 17800/18627 [10:05:35<25:58,  1.88s/it]
 96%|█████████▌| 17801/18627 [10:05:36<23:21,  1.70s/it]
 96%|█████████▌| 17802/18627 [10:05:38<25:45,  1.87s/it]
 96%|█████████▌| 17803/18627 [10:05:41<27:17,  1.99s/it]
 96%|█████████▌| 17804/18627 [10:05:43<28:32,  2.08s/it]
 96%|█████████▌| 17805/18627 [10:05:45<29:09,  2.13s/it]
 96%|█████████▌| 17806/18627 [10:05:47<29:33,  2.16s/it]
 96%|█████████▌| 17807/18627 [10:05:50<29:48,  2.18s/it]
 96%|█████████▌| 17808/18627 [10:05:52<29:59,  2.20s/it]
 96%|█████████▌| 17809/18627 [10:05:54<30:07,  2.21s/it]
 96%|█████████▌| 17810/18627 [10:05:56<30:11,  2.22s/it]
                                                        
{'loss': 0.8406, 'grad_norm': 6.624945163726807, 'learning_rate': 2.5182755099090738e-08, 'epoch': 0.96}
+
 96%|█████████▌| 17810/18627 [10:05:56<30:11,  2.22s/it]
 96%|█████████▌| 17811/18627 [10:05:58<26:14,  1.93s/it]
 96%|█████████▌| 17812/18627 [10:06:00<27:43,  2.04s/it]
 96%|█████████▌| 17813/18627 [10:06:01<24:31,  1.81s/it]
 96%|█████████▌| 17814/18627 [10:06:02<22:11,  1.64s/it]
 96%|█████████▌| 17815/18627 [10:06:05<24:41,  1.82s/it]
 96%|█████████▌| 17816/18627 [10:06:06<22:26,  1.66s/it]
 96%|█████████▌| 17817/18627 [10:06:08<24:59,  1.85s/it]
 96%|█████████▌| 17818/18627 [10:06:10<22:35,  1.68s/it]
 96%|█████████▌| 17819/18627 [10:06:12<25:00,  1.86s/it]
 96%|█████████▌| 17820/18627 [10:06:14<26:35,  1.98s/it]
                                                        
{'loss': 1.6098, 'grad_norm': 8.624831199645996, 'learning_rate': 2.4571064588461148e-08, 'epoch': 0.96}
+
 96%|█████████▌| 17820/18627 [10:06:14<26:35,  1.98s/it]
 96%|█████████▌| 17821/18627 [10:06:16<27:46,  2.07s/it]
 96%|█████████▌| 17822/18627 [10:06:19<28:30,  2.12s/it]
 96%|█████████▌| 17823/18627 [10:06:20<25:01,  1.87s/it]
 96%|█████████▌| 17824/18627 [10:06:22<26:42,  2.00s/it]
 96%|█████████▌| 17825/18627 [10:06:24<27:42,  2.07s/it]
 96%|█████████▌| 17826/18627 [10:06:27<28:24,  2.13s/it]
 96%|█████████▌| 17827/18627 [10:06:29<28:49,  2.16s/it]
 96%|█████████▌| 17828/18627 [10:06:31<29:07,  2.19s/it]
 96%|█████████▌| 17829/18627 [10:06:32<25:24,  1.91s/it]
 96%|█████████▌| 17830/18627 [10:06:34<22:56,  1.73s/it]
                                                        
{'loss': 1.3032, 'grad_norm': 14.403082847595215, 'learning_rate': 2.3966858011598425e-08, 'epoch': 0.96}
+
 96%|█████████▌| 17830/18627 [10:06:34<22:56,  1.73s/it]
 96%|█████████▌| 17831/18627 [10:06:36<26:10,  1.97s/it]
 96%|█████████▌| 17832/18627 [10:06:39<27:19,  2.06s/it]
 96%|█████████▌| 17833/18627 [10:06:40<24:04,  1.82s/it]
 96%|█████████▌| 17834/18627 [10:06:42<25:45,  1.95s/it]
 96%|█████████▌| 17835/18627 [10:06:44<26:58,  2.04s/it]
 96%|█████████▌| 17836/18627 [10:06:47<27:41,  2.10s/it]
 96%|█████████▌| 17837/18627 [10:06:49<28:15,  2.15s/it]
 96%|█████████▌| 17838/18627 [10:06:50<24:38,  1.87s/it]
 96%|█████████▌| 17839/18627 [10:06:52<26:02,  1.98s/it]
 96%|█████████▌| 17840/18627 [10:06:55<27:03,  2.06s/it]
                                                        
{'loss': 1.043, 'grad_norm': 5.488682746887207, 'learning_rate': 2.3370137195192487e-08, 'epoch': 0.96}
+
 96%|█████████▌| 17840/18627 [10:06:55<27:03,  2.06s/it]
 96%|█████████▌| 17841/18627 [10:06:57<27:48,  2.12s/it]
 96%|█████████▌| 17842/18627 [10:06:58<24:20,  1.86s/it]
 96%|█████████▌| 17843/18627 [10:07:00<25:56,  1.99s/it]
 96%|█████████▌| 17844/18627 [10:07:03<26:55,  2.06s/it]
 96%|█████████▌| 17845/18627 [10:07:05<27:33,  2.11s/it]
 96%|█████████▌| 17846/18627 [10:07:07<28:08,  2.16s/it]
 96%|█████████▌| 17847/18627 [10:07:09<28:20,  2.18s/it]
 96%|█████████▌| 17848/18627 [10:07:12<28:35,  2.20s/it]
 96%|█████████▌| 17849/18627 [10:07:14<28:49,  2.22s/it]
 96%|█████████▌| 17850/18627 [10:07:15<25:00,  1.93s/it]
                                                        
{'loss': 0.9343, 'grad_norm': 13.663739204406738, 'learning_rate': 2.2780903943302734e-08, 'epoch': 0.96}
+
 96%|█████████▌| 17850/18627 [10:07:15<25:00,  1.93s/it]
 96%|█████████▌| 17851/18627 [10:07:16<22:28,  1.74s/it]
 96%|█████████▌| 17852/18627 [10:07:19<24:30,  1.90s/it]
 96%|█████████▌| 17853/18627 [10:07:21<25:56,  2.01s/it]
 96%|█████████▌| 17854/18627 [10:07:23<26:59,  2.10s/it]
 96%|█████████▌| 17855/18627 [10:07:26<27:33,  2.14s/it]
 96%|█████████▌| 17856/18627 [10:07:28<27:51,  2.17s/it]
 96%|█████████▌| 17857/18627 [10:07:30<28:12,  2.20s/it]
 96%|█████████▌| 17858/18627 [10:07:32<28:30,  2.22s/it]
 96%|█████████▌| 17859/18627 [10:07:35<28:52,  2.26s/it]
 96%|█████████▌| 17860/18627 [10:07:36<25:03,  1.96s/it]
                                                        
{'loss': 0.9745, 'grad_norm': 13.979706764221191, 'learning_rate': 2.2199160037350286e-08, 'epoch': 0.96}
+
 96%|█████████▌| 17860/18627 [10:07:36<25:03,  1.96s/it]
 96%|█████████▌| 17861/18627 [10:07:38<26:12,  2.05s/it]
 96%|█████████▌| 17862/18627 [10:07:39<23:06,  1.81s/it]
 96%|█████████▌| 17863/18627 [10:07:42<24:39,  1.94s/it]
 96%|█████████▌| 17864/18627 [10:07:44<25:57,  2.04s/it]
 96%|█████████▌| 17865/18627 [10:07:45<22:53,  1.80s/it]
 96%|█████████▌| 17866/18627 [10:07:47<24:34,  1.94s/it]
 96%|█████████▌| 17867/18627 [10:07:49<21:58,  1.73s/it]
 96%|█████████▌| 17868/18627 [10:07:50<20:12,  1.60s/it]
 96%|█████████▌| 17869/18627 [10:07:52<22:43,  1.80s/it]
 96%|█████████▌| 17870/18627 [10:07:54<20:48,  1.65s/it]
                                                        
{'loss': 1.6075, 'grad_norm': 13.1190185546875, 'learning_rate': 2.1624907236114378e-08, 'epoch': 0.96}
+
 96%|█████████▌| 17870/18627 [10:07:54<20:48,  1.65s/it]
 96%|█████████▌| 17871/18627 [10:07:56<22:55,  1.82s/it]
 96%|█████████▌| 17872/18627 [10:07:58<24:26,  1.94s/it]
 96%|█████████▌| 17873/18627 [10:08:00<25:33,  2.03s/it]
 96%|█████████▌| 17874/18627 [10:08:02<26:20,  2.10s/it]
 96%|█████████▌| 17875/18627 [10:08:04<23:11,  1.85s/it]
 96%|█████████▌| 17876/18627 [10:08:06<25:42,  2.05s/it]
 96%|█████████▌| 17877/18627 [10:08:09<26:27,  2.12s/it]
 96%|█████████▌| 17878/18627 [10:08:11<27:09,  2.18s/it]
 96%|█████████▌| 17879/18627 [10:08:13<27:27,  2.20s/it]
 96%|█████████▌| 17880/18627 [10:08:15<27:42,  2.23s/it]
                                                        
{'loss': 0.8294, 'grad_norm': 4.753443241119385, 'learning_rate': 2.1058147275726525e-08, 'epoch': 0.96}
+
 96%|█████████▌| 17880/18627 [10:08:15<27:42,  2.23s/it]
 96%|█████████▌| 17881/18627 [10:08:18<27:59,  2.25s/it]
 96%|█████████▌| 17882/18627 [10:08:20<28:02,  2.26s/it]
 96%|█████████▌| 17883/18627 [10:08:22<28:08,  2.27s/it]
 96%|█████████▌| 17884/18627 [10:08:25<28:09,  2.27s/it]
 96%|█████████▌| 17885/18627 [10:08:27<28:08,  2.28s/it]
 96%|█████████▌| 17886/18627 [10:08:29<28:00,  2.27s/it]
 96%|█████████▌| 17887/18627 [10:08:31<28:03,  2.28s/it]
 96%|█████████▌| 17888/18627 [10:08:34<27:58,  2.27s/it]
 96%|█████████▌| 17889/18627 [10:08:36<27:48,  2.26s/it]
 96%|█████████▌| 17890/18627 [10:08:37<24:08,  1.97s/it]
                                                        
{'loss': 0.7947, 'grad_norm': 14.212505340576172, 'learning_rate': 2.0498881869665256e-08, 'epoch': 0.96}
+
 96%|█████████▌| 17890/18627 [10:08:37<24:08,  1.97s/it]
 96%|█████████▌| 17891/18627 [10:08:39<25:05,  2.05s/it]
 96%|█████████▌| 17892/18627 [10:08:42<25:44,  2.10s/it]
 96%|█████████▌| 17893/18627 [10:08:43<22:40,  1.85s/it]
 96%|█████████▌| 17894/18627 [10:08:45<24:26,  2.00s/it]
 96%|█████████▌| 17895/18627 [10:08:48<26:32,  2.18s/it]
 96%|█████████▌| 17896/18627 [10:08:49<23:17,  1.91s/it]
 96%|█████████▌| 17897/18627 [10:08:51<24:55,  2.05s/it]
 96%|█████████▌| 17898/18627 [10:08:54<25:39,  2.11s/it]
 96%|█████████▌| 17899/18627 [10:08:56<26:07,  2.15s/it]
 96%|█████████▌| 17900/18627 [10:08:58<26:26,  2.18s/it]
                                                        
{'loss': 0.9522, 'grad_norm': 8.319589614868164, 'learning_rate': 1.994711270875027e-08, 'epoch': 0.96}
+
 96%|█████████▌| 17900/18627 [10:08:58<26:26,  2.18s/it]
 96%|█████████▌| 17901/18627 [10:08:59<22:57,  1.90s/it]
 96%|█████████▌| 17902/18627 [10:09:02<24:28,  2.03s/it]
 96%|█████████▌| 17903/18627 [10:09:04<25:15,  2.09s/it]
 96%|█████████▌| 17904/18627 [10:09:06<25:46,  2.14s/it]
 96%|█████████▌| 17905/18627 [10:09:07<22:17,  1.85s/it]
 96%|█████████▌| 17906/18627 [10:09:10<23:37,  1.97s/it]
 96%|█████████▌| 17907/18627 [10:09:12<24:34,  2.05s/it]
 96%|█████████▌| 17908/18627 [10:09:13<21:24,  1.79s/it]
 96%|█████████▌| 17909/18627 [10:09:15<22:59,  1.92s/it]
 96%|█████████▌| 17910/18627 [10:09:18<24:04,  2.01s/it]
                                                        
{'loss': 1.1906, 'grad_norm': 5.447271823883057, 'learning_rate': 1.9402841461138856e-08, 'epoch': 0.96}
+
 96%|█████████▌| 17910/18627 [10:09:18<24:04,  2.01s/it]
 96%|█████████▌| 17911/18627 [10:09:20<24:53,  2.09s/it]
 96%|█████████▌| 17912/18627 [10:09:22<25:26,  2.14s/it]
 96%|█████████▌| 17913/18627 [10:09:24<25:48,  2.17s/it]
 96%|█████████▌| 17914/18627 [10:09:27<25:58,  2.19s/it]
 96%|█████████▌| 17915/18627 [10:09:29<26:06,  2.20s/it]
 96%|█████████▌| 17916/18627 [10:09:31<26:15,  2.22s/it]
 96%|█████████▌| 17917/18627 [10:09:32<22:56,  1.94s/it]
 96%|█████████▌| 17918/18627 [10:09:35<24:03,  2.04s/it]
 96%|█████████▌| 17919/18627 [10:09:37<24:44,  2.10s/it]
 96%|█████████▌| 17920/18627 [10:09:39<25:19,  2.15s/it]
                                                        
{'loss': 0.7939, 'grad_norm': 7.951906204223633, 'learning_rate': 1.886606977231975e-08, 'epoch': 0.96}
+
 96%|█████████▌| 17920/18627 [10:09:39<25:19,  2.15s/it]
 96%|█████████▌| 17921/18627 [10:09:41<25:39,  2.18s/it]
 96%|█████████▌| 17922/18627 [10:09:44<25:52,  2.20s/it]
 96%|█████████▌| 17923/18627 [10:09:46<25:55,  2.21s/it]
 96%|█████████▌| 17924/18627 [10:09:48<25:57,  2.22s/it]
 96%|█████████▌| 17925/18627 [10:09:50<26:02,  2.23s/it]
 96%|█████████▌| 17926/18627 [10:09:52<22:30,  1.93s/it]
 96%|█████████▌| 17927/18627 [10:09:54<23:31,  2.02s/it]
 96%|█████████▌| 17928/18627 [10:09:56<24:21,  2.09s/it]
 96%|█████████▋| 17929/18627 [10:09:58<24:51,  2.14s/it]
 96%|█████████▋| 17930/18627 [10:10:01<25:09,  2.17s/it]
                                                        
{'loss': 0.7441, 'grad_norm': 7.3565592765808105, 'learning_rate': 1.8336799265107617e-08, 'epoch': 0.96}
+
 96%|█████████▋| 17930/18627 [10:10:01<25:09,  2.17s/it]
 96%|█████████▋| 17931/18627 [10:10:03<25:25,  2.19s/it]
 96%|█████████▋| 17932/18627 [10:10:05<25:34,  2.21s/it]
 96%|█████████▋| 17933/18627 [10:10:07<25:41,  2.22s/it]
 96%|█████████▋| 17934/18627 [10:10:10<25:47,  2.23s/it]
 96%|█████████▋| 17935/18627 [10:10:12<25:48,  2.24s/it]
 96%|█████████▋| 17936/18627 [10:10:14<25:45,  2.24s/it]
 96%|█████████▋| 17937/18627 [10:10:16<25:46,  2.24s/it]
 96%|█████████▋| 17938/18627 [10:10:19<25:49,  2.25s/it]
 96%|█████████▋| 17939/18627 [10:10:21<25:44,  2.24s/it]
 96%|█████████▋| 17940/18627 [10:10:22<22:22,  1.95s/it]
                                                        
{'loss': 0.8173, 'grad_norm': 14.070670127868652, 'learning_rate': 1.7815031539639983e-08, 'epoch': 0.96}
+
 96%|█████████▋| 17940/18627 [10:10:22<22:22,  1.95s/it]
 96%|█████████▋| 17941/18627 [10:10:24<23:22,  2.04s/it]
 96%|█████████▋| 17942/18627 [10:10:27<24:00,  2.10s/it]
 96%|█████████▋| 17943/18627 [10:10:29<24:32,  2.15s/it]
 96%|█████████▋| 17944/18627 [10:10:31<24:58,  2.19s/it]
 96%|█████████▋| 17945/18627 [10:10:33<25:07,  2.21s/it]
 96%|█████████▋| 17946/18627 [10:10:35<21:49,  1.92s/it]
 96%|█████████▋| 17947/18627 [10:10:36<19:31,  1.72s/it]
 96%|█████████▋| 17948/18627 [10:10:38<21:14,  1.88s/it]
 96%|█████████▋| 17949/18627 [10:10:40<22:32,  1.99s/it]
 96%|█████████▋| 17950/18627 [10:10:42<20:09,  1.79s/it]
                                                        
{'loss': 1.1088, 'grad_norm': 13.530094146728516, 'learning_rate': 1.7300768173370564e-08, 'epoch': 0.96}
+
 96%|█████████▋| 17950/18627 [10:10:42<20:09,  1.79s/it]
 96%|█████████▋| 17951/18627 [10:10:44<21:43,  1.93s/it]
 96%|█████████▋| 17952/18627 [10:10:46<22:48,  2.03s/it]
 96%|█████████▋| 17953/18627 [10:10:48<23:34,  2.10s/it]
 96%|█████████▋| 17954/18627 [10:10:51<24:05,  2.15s/it]
 96%|█████████▋| 17955/18627 [10:10:53<24:33,  2.19s/it]
 96%|█████████▋| 17956/18627 [10:10:55<24:45,  2.21s/it]
 96%|█████████▋| 17957/18627 [10:10:58<24:50,  2.22s/it]
 96%|█████████▋| 17958/18627 [10:11:00<24:46,  2.22s/it]
 96%|█████████▋| 17959/18627 [10:11:02<24:48,  2.23s/it]
 96%|█████████▋| 17960/18627 [10:11:03<20:17,  1.83s/it]
                                                        
{'loss': 0.6264, 'grad_norm': 5.844667911529541, 'learning_rate': 1.67940107210654e-08, 'epoch': 0.96}
+
 96%|█████████▋| 17960/18627 [10:11:03<20:17,  1.83s/it]
 96%|█████████▋| 17961/18627 [10:11:05<21:45,  1.96s/it]
 96%|█████████▋| 17962/18627 [10:11:07<22:40,  2.05s/it]
 96%|█████████▋| 17963/18627 [10:11:09<20:03,  1.81s/it]
 96%|█████████▋| 17964/18627 [10:11:10<18:08,  1.64s/it]
 96%|█████████▋| 17965/18627 [10:11:12<20:04,  1.82s/it]
 96%|█████████▋| 17966/18627 [10:11:13<18:12,  1.65s/it]
 96%|█████████▋| 17967/18627 [10:11:16<20:04,  1.82s/it]
 96%|█████████▋| 17968/18627 [10:11:18<21:22,  1.95s/it]
 96%|█████████▋| 17969/18627 [10:11:19<19:02,  1.74s/it]
 96%|█████████▋| 17970/18627 [10:11:21<20:40,  1.89s/it]
                                                        
{'loss': 1.2993, 'grad_norm': 5.290414333343506, 'learning_rate': 1.629476071479813e-08, 'epoch': 0.96}
+
 96%|█████████▋| 17970/18627 [10:11:21<20:40,  1.89s/it]
 96%|█████████▋| 17971/18627 [10:11:24<22:03,  2.02s/it]
 96%|█████████▋| 17972/18627 [10:11:26<22:43,  2.08s/it]
 96%|█████████▋| 17973/18627 [10:11:28<23:13,  2.13s/it]
 96%|█████████▋| 17974/18627 [10:11:30<23:34,  2.17s/it]
 96%|█████████▋| 17975/18627 [10:11:33<23:49,  2.19s/it]
 97%|█████████▋| 17976/18627 [10:11:35<23:56,  2.21s/it]
 97%|█████████▋| 17977/18627 [10:11:37<24:02,  2.22s/it]
 97%|█████████▋| 17978/18627 [10:11:39<24:06,  2.23s/it]
 97%|█████████▋| 17979/18627 [10:11:42<24:37,  2.28s/it]
 97%|█████████▋| 17980/18627 [10:11:44<24:28,  2.27s/it]
                                                        
{'loss': 0.5587, 'grad_norm': 5.902510643005371, 'learning_rate': 1.5803019663944984e-08, 'epoch': 0.97}
+
 97%|█████████▋| 17980/18627 [10:11:44<24:28,  2.27s/it]
 97%|█████████▋| 17981/18627 [10:11:45<20:56,  1.94s/it]
 97%|█████████▋| 17982/18627 [10:11:47<21:45,  2.02s/it]
 97%|█████████▋| 17983/18627 [10:11:50<22:26,  2.09s/it]
 97%|█████████▋| 17984/18627 [10:11:52<22:54,  2.14s/it]
 97%|█████████▋| 17985/18627 [10:11:54<23:16,  2.18s/it]
 97%|█████████▋| 17986/18627 [10:11:56<23:23,  2.19s/it]
 97%|█████████▋| 17987/18627 [10:11:58<20:17,  1.90s/it]
 97%|█████████▋| 17988/18627 [10:12:00<21:23,  2.01s/it]
 97%|█████████▋| 17989/18627 [10:12:02<22:09,  2.08s/it]
 97%|█████████▋| 17990/18627 [10:12:04<22:38,  2.13s/it]
                                                        
{'loss': 1.0156, 'grad_norm': 5.548087120056152, 'learning_rate': 1.5318789055180916e-08, 'epoch': 0.97}
+
 97%|█████████▋| 17990/18627 [10:12:04<22:38,  2.13s/it]
 97%|█████████▋| 17991/18627 [10:12:06<19:49,  1.87s/it]
 97%|█████████▋| 17992/18627 [10:12:08<21:15,  2.01s/it]
 97%|█████████▋| 17993/18627 [10:12:10<22:03,  2.09s/it]
 97%|█████████▋| 17994/18627 [10:12:13<22:32,  2.14s/it]
 97%|█████████▋| 17995/18627 [10:12:15<22:53,  2.17s/it]
 97%|█████████▋| 17996/18627 [10:12:17<23:03,  2.19s/it]
 97%|█████████▋| 17997/18627 [10:12:19<23:13,  2.21s/it]
 97%|█████████▋| 17998/18627 [10:12:22<23:21,  2.23s/it]
 97%|█████████▋| 17999/18627 [10:12:25<28:42,  2.74s/it]
 97%|█████████▋| 18000/18627 [10:12:28<27:09,  2.60s/it]
                                                        
{'loss': 0.7524, 'grad_norm': 4.363626956939697, 'learning_rate': 1.4842070352474036e-08, 'epoch': 0.97}
+
 97%|█████████▋| 18000/18627 [10:12:28<27:09,  2.60s/it]/home/zhengduo/workspace/LLaMA-Factory/src/qwen_vl/model/modeling_qwen2_5_vl.py:1979: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
+  with torch.cuda.amp.autocast(dtype=dtype):
+/home/zhengduo/miniconda3/envs/vgllm/lib/python3.10/site-packages/torch/utils/checkpoint.py:87: UserWarning: None of the inputs have requires_grad=True. Gradients will be None
+  warnings.warn(
+
 97%|█████████▋| 18001/18627 [10:13:25<3:19:18, 19.10s/it]
 97%|█████████▋| 18002/18627 [10:13:28<2:26:29, 14.06s/it]
 97%|█████████▋| 18003/18627 [10:13:30<1:49:35, 10.54s/it]
 97%|█████████▋| 18004/18627 [10:13:32<1:23:44,  8.07s/it]
 97%|█████████▋| 18005/18627 [10:13:35<1:05:39,  6.33s/it]
 97%|█████████▋| 18006/18627 [10:13:37<52:58,  5.12s/it]  
 97%|█████████▋| 18007/18627 [10:13:38<41:05,  3.98s/it]
 97%|█████████▋| 18008/18627 [10:13:40<32:58,  3.20s/it]
 97%|█████████▋| 18009/18627 [10:13:42<30:02,  2.92s/it]
 97%|████���████▋| 18010/18627 [10:13:44<27:57,  2.72s/it]
                                                        
{'loss': 1.0264, 'grad_norm': 7.808571815490723, 'learning_rate': 1.4372864997082291e-08, 'epoch': 0.97}
+
 97%|█████████▋| 18010/18627 [10:13:44<27:57,  2.72s/it]
 97%|█████████▋| 18011/18627 [10:13:46<26:31,  2.58s/it]
 97%|█████████▋| 18012/18627 [10:13:49<25:26,  2.48s/it]
 97%|█████████▋| 18013/18627 [10:13:51<24:39,  2.41s/it]
 97%|█████████▋| 18014/18627 [10:13:53<24:01,  2.35s/it]
 97%|█████████▋| 18015/18627 [10:13:55<23:38,  2.32s/it]
 97%|█████████▋| 18016/18627 [10:13:58<23:21,  2.29s/it]
 97%|█████████▋| 18017/18627 [10:14:00<23:06,  2.27s/it]
 97%|█████████▋| 18018/18627 [10:14:02<22:58,  2.26s/it]
 97%|█████████▋| 18019/18627 [10:14:04<22:50,  2.25s/it]
 97%|█████████▋| 18020/18627 [10:14:06<22:43,  2.25s/it]
                                                        
{'loss': 0.627, 'grad_norm': 7.280130386352539, 'learning_rate': 1.3911174407548189e-08, 'epoch': 0.97}
+
 97%|█████████▋| 18020/18627 [10:14:06<22:43,  2.25s/it]
 97%|█████████▋| 18021/18627 [10:14:09<22:37,  2.24s/it]
 97%|█████████▋| 18022/18627 [10:14:11<22:33,  2.24s/it]
 97%|█████████▋| 18023/18627 [10:14:13<22:30,  2.24s/it]
 97%|█████████▋| 18024/18627 [10:14:15<22:30,  2.24s/it]
 97%|█████████▋| 18025/18627 [10:14:18<22:26,  2.24s/it]
 97%|█████████▋| 18026/18627 [10:14:20<22:20,  2.23s/it]
 97%|█████████▋| 18027/18627 [10:14:22<22:24,  2.24s/it]
 97%|█████████▋| 18028/18627 [10:14:24<22:32,  2.26s/it]
 97%|█████████▋| 18029/18627 [10:14:27<22:47,  2.29s/it]
 97%|█████████▋| 18030/18627 [10:14:29<22:52,  2.30s/it]
                                                        
{'loss': 0.5966, 'grad_norm': 4.784152984619141, 'learning_rate': 1.345699997969574e-08, 'epoch': 0.97}
+
 97%|█████████▋| 18030/18627 [10:14:29<22:52,  2.30s/it]
 97%|█████████▋| 18031/18627 [10:14:30<19:51,  2.00s/it]
 97%|█████████▋| 18032/18627 [10:14:33<20:41,  2.09s/it]
 97%|█████████▋| 18033/18627 [10:14:35<21:17,  2.15s/it]
 97%|█████████▋| 18034/18627 [10:14:37<21:47,  2.20s/it]
 97%|█████████▋| 18035/18627 [10:14:39<19:05,  1.94s/it]
 97%|█████████▋| 18036/18627 [10:14:41<20:12,  2.05s/it]
 97%|█████████▋| 18037/18627 [10:14:42<18:05,  1.84s/it]
 97%|█████████▋| 18038/18627 [10:14:43<16:17,  1.66s/it]
 97%|█████████▋| 18039/18627 [10:14:46<18:09,  1.85s/it]
 97%|█████████▋| 18040/18627 [10:14:47<16:45,  1.71s/it]
                                                        
{'loss': 1.6275, 'grad_norm': 15.725899696350098, 'learning_rate': 1.3010343086624354e-08, 'epoch': 0.97}
+
 97%|█████████▋| 18040/18627 [10:14:47<16:45,  1.71s/it]
 97%|█████████▋| 18041/18627 [10:14:50<18:37,  1.91s/it]
 97%|█████████▋| 18042/18627 [10:14:51<16:50,  1.73s/it]
 97%|█████████▋| 18043/18627 [10:14:53<18:29,  1.90s/it]
 97%|█████████▋| 18044/18627 [10:14:55<19:44,  2.03s/it]
 97%|█████████▋| 18045/18627 [10:14:56<16:42,  1.72s/it]
 97%|█████████▋| 18046/18627 [10:14:59<18:23,  1.90s/it]
 97%|█████████▋| 18047/18627 [10:15:01<19:37,  2.03s/it]
 97%|█████████▋| 18048/18627 [10:15:03<20:26,  2.12s/it]
 97%|█████████▋| 18049/18627 [10:15:06<20:53,  2.17s/it]
 97%|█████████▋| 18050/18627 [10:15:08<21:05,  2.19s/it]
                                                        
{'loss': 0.8308, 'grad_norm': 5.423874855041504, 'learning_rate': 1.2571205078706627e-08, 'epoch': 0.97}
+
 97%|█████████▋| 18050/18627 [10:15:08<21:05,  2.19s/it]
 97%|█████████▋| 18051/18627 [10:15:09<18:19,  1.91s/it]
 97%|█████████▋| 18052/18627 [10:15:11<19:13,  2.01s/it]
 97%|█████████▋| 18053/18627 [10:15:14<20:00,  2.09s/it]
 97%|█████████▋| 18054/18627 [10:15:16<20:23,  2.14s/it]
 97%|█████████▋| 18055/18627 [10:15:18<20:38,  2.17s/it]
 97%|█████████▋| 18056/18627 [10:15:21<20:54,  2.20s/it]
 97%|█████████▋| 18057/18627 [10:15:23<21:00,  2.21s/it]
 97%|█████████▋| 18058/18627 [10:15:25<21:05,  2.22s/it]
 97%|█████████▋| 18059/18627 [10:15:27<21:07,  2.23s/it]
 97%|█████████▋| 18060/18627 [10:15:29<21:03,  2.23s/it]
                                                        
{'loss': 0.7535, 'grad_norm': 4.604104518890381, 'learning_rate': 1.2139587283583055e-08, 'epoch': 0.97}
+
 97%|█████████▋| 18060/18627 [10:15:29<21:03,  2.23s/it]
 97%|█████████▋| 18061/18627 [10:15:32<21:01,  2.23s/it]
 97%|█████████▋| 18062/18627 [10:15:33<18:04,  1.92s/it]
 97%|█████████▋| 18063/18627 [10:15:35<18:54,  2.01s/it]
 97%|█████████▋| 18064/18627 [10:15:37<19:28,  2.08s/it]
 97%|█████████▋| 18065/18627 [10:15:40<20:00,  2.14s/it]
 97%|█████████▋| 18066/18627 [10:15:42<20:24,  2.18s/it]
 97%|█████████▋| 18067/18627 [10:15:43<18:10,  1.95s/it]
 97%|█████████▋| 18068/18627 [10:15:46<19:09,  2.06s/it]
 97%|█████████▋| 18069/18627 [10:15:47<17:16,  1.86s/it]
 97%|█████████▋| 18070/18627 [10:15:48<15:49,  1.70s/it]
                                                        
{'loss': 1.4176, 'grad_norm': 16.02294158935547, 'learning_rate': 1.1715491006158153e-08, 'epoch': 0.97}
+
 97%|█████████▋| 18070/18627 [10:15:48<15:49,  1.70s/it]
 97%|█████████▋| 18071/18627 [10:15:51<17:30,  1.89s/it]
 97%|█████████▋| 18072/18627 [10:15:52<15:56,  1.72s/it]
 97%|█████████▋| 18073/18627 [10:15:54<17:33,  1.90s/it]
 97%|█████████▋| 18074/18627 [10:15:57<18:38,  2.02s/it]
 97%|█████████▋| 18075/18627 [10:15:58<15:50,  1.72s/it]
 97%|█████████▋| 18076/18627 [10:16:00<17:16,  1.88s/it]
 97%|█████████▋| 18077/18627 [10:16:02<18:15,  1.99s/it]
 97%|█████████▋| 18078/18627 [10:16:04<18:58,  2.07s/it]
 97%|█████████▋| 18079/18627 [10:16:07<19:22,  2.12s/it]
 97%|█████████▋| 18080/18627 [10:16:09<19:39,  2.16s/it]
                                                        
{'loss': 0.7895, 'grad_norm': 8.886614799499512, 'learning_rate': 1.129891752859713e-08, 'epoch': 0.97}
+
 97%|█████████▋| 18080/18627 [10:16:09<19:39,  2.16s/it]
 97%|█████████▋| 18081/18627 [10:16:11<19:51,  2.18s/it]
 97%|█████████▋| 18082/18627 [10:16:13<20:00,  2.20s/it]
 97%|█████████▋| 18083/18627 [10:16:16<20:04,  2.21s/it]
 97%|█████████▋| 18084/18627 [10:16:17<17:18,  1.91s/it]
 97%|█████████▋| 18085/18627 [10:16:19<18:08,  2.01s/it]
 97%|█████████▋| 18086/18627 [10:16:21<18:43,  2.08s/it]
 97%|█████████▋| 18087/18627 [10:16:22<16:11,  1.80s/it]
 97%|█████████▋| 18088/18627 [10:16:25<17:18,  1.93s/it]
 97%|█████████▋| 18089/18627 [10:16:27<18:10,  2.03s/it]
 97%|█████████▋| 18090/18627 [10:16:28<16:04,  1.80s/it]
                                                        
{'loss': 1.1152, 'grad_norm': 14.897608757019043, 'learning_rate': 1.0889868110321445e-08, 'epoch': 0.97}
+
 97%|█████████▋| 18090/18627 [10:16:28<16:04,  1.80s/it]
 97%|█████████▋| 18091/18627 [10:16:30<17:15,  1.93s/it]
 97%|█████████▋| 18092/18627 [10:16:33<18:09,  2.04s/it]
 97%|█████████▋| 18093/18627 [10:16:35<18:53,  2.12s/it]
 97%|█████████▋| 18094/18627 [10:16:37<19:30,  2.20s/it]
 97%|█████████▋| 18095/18627 [10:16:40<19:49,  2.24s/it]
 97%|█████████▋| 18096/18627 [10:16:42<20:00,  2.26s/it]
 97%|█████████▋| 18097/18627 [10:16:43<17:24,  1.97s/it]
 97%|█████████▋| 18098/18627 [10:16:45<15:33,  1.76s/it]
 97%|█████████▋| 18099/18627 [10:16:47<16:55,  1.92s/it]
 97%|█████████▋| 18100/18627 [10:16:49<17:53,  2.04s/it]
                                                        
{'loss': 1.0447, 'grad_norm': 9.89549732208252, 'learning_rate': 1.048834398800519e-08, 'epoch': 0.97}
+
 97%|█████████▋| 18100/18627 [10:16:49<17:53,  2.04s/it]
 97%|█████████▋| 18101/18627 [10:16:51<16:00,  1.83s/it]
 97%|█████████▋| 18102/18627 [10:16:53<17:12,  1.97s/it]
 97%|█████████▋| 18103/18627 [10:16:54<15:32,  1.78s/it]
 97%|█████████▋| 18104/18627 [10:16:56<14:16,  1.64s/it]
 97%|█████████▋| 18105/18627 [10:16:58<16:06,  1.85s/it]
 97%|█████████▋| 18106/18627 [10:17:00<17:23,  2.00s/it]
 97%|█████████▋| 18107/18627 [10:17:03<18:11,  2.10s/it]
 97%|█████████▋| 18108/18627 [10:17:05<18:43,  2.16s/it]
 97%|█████████▋| 18109/18627 [10:17:06<16:34,  1.92s/it]
 97%|█████████▋| 18110/18627 [10:17:08<14:59,  1.74s/it]
                                                        
{'loss': 1.559, 'grad_norm': 14.480354309082031, 'learning_rate': 1.0094346375571218e-08, 'epoch': 0.97}
+
 97%|█████████▋| 18110/18627 [10:17:08<14:59,  1.74s/it]
 97%|█████████▋| 18111/18627 [10:17:10<16:19,  1.90s/it]
 97%|█████████▋| 18112/18627 [10:17:12<17:10,  2.00s/it]
 97%|█████████▋| 18113/18627 [10:17:14<17:48,  2.08s/it]
 97%|█████████▋| 18114/18627 [10:17:17<18:15,  2.14s/it]
 97%|█████████▋| 18115/18627 [10:17:18<15:55,  1.87s/it]
 97%|█████████▋| 18116/18627 [10:17:20<16:55,  1.99s/it]
 97%|█████████▋| 18117/18627 [10:17:22<17:35,  2.07s/it]
 97%|█████████▋| 18118/18627 [10:17:25<18:02,  2.13s/it]
 97%|█████████▋| 18119/18627 [10:17:27<18:20,  2.17s/it]
 97%|█████████▋| 18120/18627 [10:17:29<18:28,  2.19s/it]
                                                        
{'loss': 0.7892, 'grad_norm': 5.865367412567139, 'learning_rate': 9.707876464187805e-09, 'epoch': 0.97}
+
 97%|█████████▋| 18120/18627 [10:17:29<18:28,  2.19s/it]
 97%|█████████▋| 18121/18627 [10:17:31<18:32,  2.20s/it]
 97%|█████████▋| 18122/18627 [10:17:34<18:34,  2.21s/it]
 97%|█████████▋| 18123/18627 [10:17:36<18:38,  2.22s/it]
 97%|█████████▋| 18124/18627 [10:17:38<18:41,  2.23s/it]
 97%|█████████▋| 18125/18627 [10:17:39<16:08,  1.93s/it]
 97%|█████████▋| 18126/18627 [10:17:42<16:49,  2.01s/it]
 97%|█████████▋| 18127/18627 [10:17:44<17:20,  2.08s/it]
 97%|█████████▋| 18128/18627 [10:17:45<14:55,  1.79s/it]
 97%|█████████▋| 18129/18627 [10:17:47<16:00,  1.93s/it]
 97%|█████████▋| 18130/18627 [10:17:49<16:43,  2.02s/it]
                                                        
{'loss': 0.9339, 'grad_norm': 7.592047214508057, 'learning_rate': 9.328935422264208e-09, 'epoch': 0.97}
+
 97%|█████████▋| 18130/18627 [10:17:49<16:43,  2.02s/it]
 97%|█████████▋| 18131/18627 [10:17:52<17:14,  2.09s/it]
 97%|█████████▋| 18132/18627 [10:17:54<17:33,  2.13s/it]
 97%|█████████▋| 18133/18627 [10:17:56<17:34,  2.13s/it]
 97%|█████████▋| 18134/18627 [10:17:58<17:35,  2.14s/it]
 97%|█████████▋| 18135/18627 [10:17:59<15:19,  1.87s/it]
 97%|█████████▋| 18136/18627 [10:18:02<16:04,  1.96s/it]
 97%|█████████▋| 18137/18627 [10:18:04<16:34,  2.03s/it]
 97%|█████████▋| 18138/18627 [10:18:06<16:53,  2.07s/it]
 97%|█████████▋| 18139/18627 [10:18:07<14:42,  1.81s/it]
 97%|█████████▋| 18140/18627 [10:18:09<15:29,  1.91s/it]
                                                        
{'loss': 0.9784, 'grad_norm': 10.322150230407715, 'learning_rate': 8.957524395449002e-09, 'epoch': 0.97}
+
 97%|█████████▋| 18140/18627 [10:18:09<15:29,  1.91s/it]
 97%|█████████▋| 18141/18627 [10:18:11<15:56,  1.97s/it]
 97%|█████████▋| 18142/18627 [10:18:13<16:15,  2.01s/it]
 97%|█████████▋| 18143/18627 [10:18:16<16:27,  2.04s/it]
 97%|█████████▋| 18144/18627 [10:18:18<16:36,  2.06s/it]
 97%|█████████▋| 18145/18627 [10:18:20<16:44,  2.08s/it]
 97%|█████████▋| 18146/18627 [10:18:21<13:30,  1.68s/it]
 97%|█████████▋| 18147/18627 [10:18:22<11:51,  1.48s/it]
 97%|█████████▋| 18148/18627 [10:18:24<13:26,  1.68s/it]
 97%|█████████▋| 18149/18627 [10:18:26<14:28,  1.82s/it]
 97%|█████████▋| 18150/18627 [10:18:28<15:09,  1.91s/it]
                                                        
{'loss': 0.8782, 'grad_norm': 5.849143028259277, 'learning_rate': 8.5936445066237e-09, 'epoch': 0.97}
+
 97%|█████████▋| 18150/18627 [10:18:28<15:09,  1.91s/it]
 97%|█████████▋| 18151/18627 [10:18:29<12:15,  1.54s/it]
 97%|█████████▋| 18152/18627 [10:18:29<10:13,  1.29s/it]
 97%|█████████▋| 18153/18627 [10:18:32<12:19,  1.56s/it]
 97%|█████████▋| 18154/18627 [10:18:34<13:37,  1.73s/it]
 97%|█████████▋| 18155/18627 [10:18:35<12:19,  1.57s/it]
 97%|█████████▋| 18156/18627 [10:18:37<13:42,  1.75s/it]
 97%|█████████▋| 18157/18627 [10:18:39<14:34,  1.86s/it]
 97%|█████████▋| 18158/18627 [10:18:40<12:46,  1.63s/it]
 97%|█████████▋| 18159/18627 [10:18:42<13:56,  1.79s/it]
 97%|█████████▋| 18160/18627 [10:18:45<14:50,  1.91s/it]
                                                        
{'loss': 1.0272, 'grad_norm': 6.506575107574463, 'learning_rate': 8.237296855902465e-09, 'epoch': 0.97}
+
 97%|█████████▋| 18160/18627 [10:18:45<14:50,  1.91s/it]
 97%|█████████▋| 18161/18627 [10:18:47<15:31,  2.00s/it]
 98%|█████████▊| 18162/18627 [10:18:49<15:58,  2.06s/it]
 98%|█████████▊| 18163/18627 [10:18:51<16:08,  2.09s/it]
 98%|█████████▊| 18164/18627 [10:18:53<16:11,  2.10s/it]
 98%|█████████▊| 18165/18627 [10:18:55<16:12,  2.11s/it]
 98%|█████████▊| 18166/18627 [10:18:57<14:04,  1.83s/it]
 98%|█████████▊| 18167/18627 [10:18:59<14:41,  1.92s/it]
 98%|█████████▊| 18168/18627 [10:19:00<12:58,  1.70s/it]
 98%|█████████▊| 18169/18627 [10:19:02<13:56,  1.83s/it]
 98%|█████████▊| 18170/18627 [10:19:04<14:36,  1.92s/it]
                                                        
{'loss': 0.9683, 'grad_norm': 4.64422082901001, 'learning_rate': 7.888482520626573e-09, 'epoch': 0.98}
+
 98%|█████████▊| 18170/18627 [10:19:04<14:36,  1.92s/it]
 98%|█████████▊| 18171/18627 [10:19:06<15:01,  1.98s/it]
 98%|█████████▊| 18172/18627 [10:19:08<15:20,  2.02s/it]
 98%|█████████▊| 18173/18627 [10:19:10<13:26,  1.78s/it]
 98%|█████████▊| 18174/18627 [10:19:11<12:03,  1.60s/it]
 98%|█████████▊| 18175/18627 [10:19:13<13:12,  1.75s/it]
 98%|█████████▊| 18176/18627 [10:19:15<14:00,  1.86s/it]
 98%|█████████▊| 18177/18627 [10:19:17<14:32,  1.94s/it]
 98%|█████████▊| 18178/18627 [10:19:19<14:51,  1.99s/it]
 98%|█████████▊| 18179/18627 [10:19:20<13:02,  1.75s/it]
 98%|█████████▊| 18180/18627 [10:19:23<13:48,  1.85s/it]
                                                        
{'loss': 1.2065, 'grad_norm': 6.020284652709961, 'learning_rate': 7.547202555362176e-09, 'epoch': 0.98}
+
 98%|█████████▊| 18180/18627 [10:19:23<13:48,  1.85s/it]
 98%|█████████▊| 18181/18627 [10:19:25<14:23,  1.94s/it]
 98%|█████████▊| 18182/18627 [10:19:27<14:50,  2.00s/it]
 98%|█████████▊| 18183/18627 [10:19:29<15:15,  2.06s/it]
 98%|█████████▊| 18184/18627 [10:19:31<15:27,  2.09s/it]
 98%|█████████▊| 18185/18627 [10:19:33<15:36,  2.12s/it]
 98%|█████████▊| 18186/18627 [10:19:35<15:35,  2.12s/it]
 98%|█████████▊| 18187/18627 [10:19:38<15:35,  2.13s/it]
 98%|█████████▊| 18188/18627 [10:19:40<15:33,  2.13s/it]
 98%|█████████▊| 18189/18627 [10:19:40<12:24,  1.70s/it]
 98%|█████████▊| 18190/18627 [10:19:43<13:26,  1.85s/it]
                                                        
{'loss': 0.615, 'grad_norm': 6.799893856048584, 'learning_rate': 7.213457991896711e-09, 'epoch': 0.98}
+
 98%|█████████▊| 18190/18627 [10:19:43<13:26,  1.85s/it]
 98%|█████████▊| 18191/18627 [10:19:45<13:58,  1.92s/it]
 98%|█████████▊| 18192/18627 [10:19:47<14:22,  1.98s/it]
 98%|█████████▊| 18193/18627 [10:19:48<11:33,  1.60s/it]
 98%|█████████▊| 18194/18627 [10:19:49<10:38,  1.47s/it]
 98%|█████████▊| 18195/18627 [10:19:50<10:01,  1.39s/it]
 98%|█████████▊| 18196/18627 [10:19:52<11:32,  1.61s/it]
 98%|█████████▊| 18197/18627 [10:19:54<12:37,  1.76s/it]
 98%|█████████▊| 18198/18627 [10:19:56<13:20,  1.87s/it]
 98%|█████████▊| 18199/18627 [10:19:57<11:51,  1.66s/it]
 98%|█████████▊| 18200/18627 [10:20:00<13:00,  1.83s/it]
                                                        
{'loss': 1.1637, 'grad_norm': 6.635623455047607, 'learning_rate': 6.887249839235833e-09, 'epoch': 0.98}
+
 98%|█████████▊| 18200/18627 [10:20:00<13:00,  1.83s/it]
 98%|█████████▊| 18201/18627 [10:20:02<13:44,  1.93s/it]
 98%|█████████▊| 18202/18627 [10:20:03<12:05,  1.71s/it]
 98%|█████████▊| 18203/18627 [10:20:05<12:59,  1.84s/it]
 98%|█████████▊| 18204/18627 [10:20:07<13:35,  1.93s/it]
 98%|█████████▊| 18205/18627 [10:20:10<14:07,  2.01s/it]
 98%|█████████▊| 18206/18627 [10:20:12<14:27,  2.06s/it]
 98%|█████████▊| 18207/18627 [10:20:14<14:43,  2.10s/it]
 98%|█████████▊| 18208/18627 [10:20:16<14:48,  2.12s/it]
 98%|█████████▊| 18209/18627 [10:20:18<14:56,  2.14s/it]
 98%|█████████▊| 18210/18627 [10:20:20<14:54,  2.15s/it]
                                                        
{'loss': 0.8002, 'grad_norm': 7.470244407653809, 'learning_rate': 6.568579083600646e-09, 'epoch': 0.98}
+
 98%|█████████▊| 18210/18627 [10:20:20<14:54,  2.15s/it]
 98%|█████████▊| 18211/18627 [10:20:22<12:53,  1.86s/it]
 98%|█████████▊| 18212/18627 [10:20:24<13:26,  1.94s/it]
 98%|█████████▊| 18213/18627 [10:20:25<11:50,  1.72s/it]
 98%|█████████▊| 18214/18627 [10:20:27<12:40,  1.84s/it]
 98%|█████████▊| 18215/18627 [10:20:29<13:14,  1.93s/it]
 98%|█████████▊| 18216/18627 [10:20:30<11:42,  1.71s/it]
 98%|█████████▊| 18217/18627 [10:20:33<12:30,  1.83s/it]
 98%|█████████▊| 18218/18627 [10:20:35<13:05,  1.92s/it]
 98%|█████████▊| 18219/18627 [10:20:37<13:25,  1.97s/it]
 98%|█████████▊| 18220/18627 [10:20:39<13:42,  2.02s/it]
                                                        
{'loss': 1.2426, 'grad_norm': 8.707500457763672, 'learning_rate': 6.257446688424929e-09, 'epoch': 0.98}
+
 98%|█████████▊| 18220/18627 [10:20:39<13:42,  2.02s/it]
 98%|█████████▊| 18221/18627 [10:20:41<13:51,  2.05s/it]
 98%|█████████▊| 18222/18627 [10:20:43<13:57,  2.07s/it]
 98%|█████████▊| 18223/18627 [10:20:45<14:03,  2.09s/it]
 98%|█████████▊| 18224/18627 [10:20:47<14:05,  2.10s/it]
 98%|█████████▊| 18225/18627 [10:20:49<14:06,  2.11s/it]
 98%|█████████▊| 18226/18627 [10:20:51<12:14,  1.83s/it]
 98%|█████████▊| 18227/18627 [10:20:52<11:00,  1.65s/it]
 98%|█████████▊| 18228/18627 [10:20:54<12:02,  1.81s/it]
 98%|█████████▊| 18229/18627 [10:20:55<10:39,  1.61s/it]
 98%|█████████▊| 18230/18627 [10:20:56<09:49,  1.49s/it]
                                                        
{'loss': 1.3893, 'grad_norm': 15.11837100982666, 'learning_rate': 5.953853594351244e-09, 'epoch': 0.98}
+
 98%|█████████▊| 18230/18627 [10:20:56<09:49,  1.49s/it]
 98%|█████████▊| 18231/18627 [10:20:59<11:13,  1.70s/it]
 98%|█████████▊| 18232/18627 [10:21:01<12:06,  1.84s/it]
 98%|█████████▊| 18233/18627 [10:21:03<12:37,  1.92s/it]
 98%|█████████▊| 18234/18627 [10:21:04<11:10,  1.70s/it]
 98%|█████████▊| 18235/18627 [10:21:06<11:57,  1.83s/it]
 98%|█████████▊| 18236/18627 [10:21:08<12:30,  1.92s/it]
 98%|█████████▊| 18237/18627 [10:21:10<12:48,  1.97s/it]
 98%|█████████▊| 18238/18627 [10:21:13<13:04,  2.02s/it]
 98%|█████████▊| 18239/18627 [10:21:15<13:14,  2.05s/it]
 98%|█████████▊| 18240/18627 [10:21:17<13:23,  2.08s/it]
                                                        
{'loss': 0.7658, 'grad_norm': 4.197880268096924, 'learning_rate': 5.6578007192284456e-09, 'epoch': 0.98}
+
 98%|█████████▊| 18240/18627 [10:21:17<13:23,  2.08s/it]
 98%|█████████▊| 18241/18627 [10:21:19<13:27,  2.09s/it]
 98%|█████████▊| 18242/18627 [10:21:20<11:41,  1.82s/it]
 98%|█████████▊| 18243/18627 [10:21:21<10:27,  1.63s/it]
 98%|█████████▊| 18244/18627 [10:21:23<11:20,  1.78s/it]
 98%|█████████▊| 18245/18627 [10:21:26<11:55,  1.87s/it]
 98%|█████████▊| 18246/18627 [10:21:26<09:41,  1.53s/it]
 98%|█████████▊| 18247/18627 [10:21:27<09:02,  1.43s/it]
 98%|█████████▊| 18248/18627 [10:21:30<10:19,  1.63s/it]
 98%|█████████▊| 18249/18627 [10:21:31<09:27,  1.50s/it]
 98%|█████████▊| 18250/18627 [10:21:31<07:55,  1.26s/it]
                                                        
{'loss': 1.4916, 'grad_norm': 6.866054534912109, 'learning_rate': 5.369288958110008e-09, 'epoch': 0.98}
+
 98%|█████████▊| 18250/18627 [10:21:31<07:55,  1.26s/it]
 98%|█████████▊| 18251/18627 [10:21:34<09:31,  1.52s/it]
 98%|█████████▊| 18252/18627 [10:21:36<10:39,  1.71s/it]
 98%|█████████▊| 18253/18627 [10:21:38<11:32,  1.85s/it]
 98%|█████████▊| 18254/18627 [10:21:40<12:06,  1.95s/it]
 98%|█████████▊| 18255/18627 [10:21:42<12:30,  2.02s/it]
 98%|█████████▊| 18256/18627 [10:21:44<12:46,  2.07s/it]
 98%|█████████▊| 18257/18627 [10:21:47<12:50,  2.08s/it]
 98%|█████████▊| 18258/18627 [10:21:49<12:53,  2.10s/it]
 98%|█████████▊| 18259/18627 [10:21:51<12:54,  2.11s/it]
 98%|█████████▊| 18260/18627 [10:21:52<11:11,  1.83s/it]
                                                        
{'loss': 0.7967, 'grad_norm': 13.23369312286377, 'learning_rate': 5.088319183249868e-09, 'epoch': 0.98}
+
 98%|█████████▊| 18260/18627 [10:21:52<11:11,  1.83s/it]
 98%|█████████▊| 18261/18627 [10:21:53<09:59,  1.64s/it]
 98%|█████████▊| 18262/18627 [10:21:55<10:49,  1.78s/it]
 98%|█████████▊| 18263/18627 [10:21:57<11:22,  1.87s/it]
 98%|█████████▊| 18264/18627 [10:22:00<11:46,  1.95s/it]
 98%|█████████▊| 18265/18627 [10:22:01<10:21,  1.72s/it]
 98%|█████████▊| 18266/18627 [10:22:03<11:02,  1.83s/it]
 98%|█████████▊| 18267/18627 [10:22:04<09:50,  1.64s/it]
 98%|█████████▊| 18268/18627 [10:22:06<10:40,  1.79s/it]
 98%|█████████▊| 18269/18627 [10:22:08<11:13,  1.88s/it]
 98%|█████████▊| 18270/18627 [10:22:09<09:56,  1.67s/it]
                                                        
{'loss': 1.3152, 'grad_norm': 14.598579406738281, 'learning_rate': 4.8148922441002e-09, 'epoch': 0.98}
+
 98%|█████████▊| 18270/18627 [10:22:09<09:56,  1.67s/it]
 98%|█████████▊| 18271/18627 [10:22:12<10:43,  1.81s/it]
 98%|█████████▊| 18272/18627 [10:22:13<09:35,  1.62s/it]
 98%|█████████▊| 18273/18627 [10:22:15<10:27,  1.77s/it]
 98%|█████████▊| 18274/18627 [10:22:17<11:00,  1.87s/it]
 98%|█████████▊| 18275/18627 [10:22:18<09:40,  1.65s/it]
 98%|█████████▊| 18276/18627 [10:22:20<10:33,  1.80s/it]
 98%|█████████▊| 18277/18627 [10:22:22<11:11,  1.92s/it]
 98%|█████████▊| 18278/18627 [10:22:25<11:35,  1.99s/it]
 98%|█████████▊| 18279/18627 [10:22:27<11:51,  2.04s/it]
 98%|█████████▊| 18280/18627 [10:22:29<11:56,  2.06s/it]
                                                        
{'loss': 1.0615, 'grad_norm': 4.72528600692749, 'learning_rate': 4.549008967309754e-09, 'epoch': 0.98}
+
 98%|█████████▊| 18280/18627 [10:22:29<11:56,  2.06s/it]
 98%|█████████▊| 18281/18627 [10:22:30<10:14,  1.78s/it]
 98%|█████████▊| 18282/18627 [10:22:32<10:48,  1.88s/it]
 98%|█████████▊| 18283/18627 [10:22:33<09:35,  1.67s/it]
 98%|█████████▊| 18284/18627 [10:22:35<10:19,  1.81s/it]
 98%|█████████▊| 18285/18627 [10:22:37<09:14,  1.62s/it]
 98%|█████████▊| 18286/18627 [10:22:38<08:28,  1.49s/it]
 98%|█████████▊| 18287/18627 [10:22:40<09:31,  1.68s/it]
 98%|█████████▊| 18288/18627 [10:22:42<10:15,  1.81s/it]
 98%|█████████▊| 18289/18627 [10:22:43<09:10,  1.63s/it]
 98%|█████████▊| 18290/18627 [10:22:45<09:59,  1.78s/it]
                                                        
{'loss': 1.617, 'grad_norm': 6.146447658538818, 'learning_rate': 4.290670156719967e-09, 'epoch': 0.98}
+
 98%|█████████▊| 18290/18627 [10:22:45<09:59,  1.78s/it]
 98%|█████████▊| 18291/18627 [10:22:47<08:58,  1.60s/it]
 98%|█████████▊| 18292/18627 [10:22:49<09:47,  1.75s/it]
 98%|█████████▊| 18293/18627 [10:22:51<10:23,  1.87s/it]
 98%|█████████▊| 18294/18627 [10:22:53<10:46,  1.94s/it]
 98%|█████████▊| 18295/18627 [10:22:55<11:01,  1.99s/it]
 98%|█████████▊| 18296/18627 [10:22:57<11:11,  2.03s/it]
 98%|█████████▊| 18297/18627 [10:22:59<11:17,  2.05s/it]
 98%|█████████▊| 18298/18627 [10:23:01<11:26,  2.09s/it]
 98%|█████████▊| 18299/18627 [10:23:04<11:33,  2.11s/it]
 98%|█████████▊| 18300/18627 [10:23:06<11:38,  2.14s/it]
                                                        
{'loss': 0.8482, 'grad_norm': 6.958872318267822, 'learning_rate': 4.039876593363023e-09, 'epoch': 0.98}
+
 98%|█████████▊| 18300/18627 [10:23:06<11:38,  2.14s/it]
 98%|█████████▊| 18301/18627 [10:23:08<11:39,  2.15s/it]
 98%|█████████▊| 18302/18627 [10:23:10<11:38,  2.15s/it]
 98%|█████████▊| 18303/18627 [10:23:12<11:34,  2.14s/it]
 98%|█████████▊| 18304/18627 [10:23:14<11:30,  2.14s/it]
 98%|█████████▊| 18305/18627 [10:23:16<09:56,  1.85s/it]
 98%|█████████▊| 18306/18627 [10:23:18<10:21,  1.93s/it]
 98%|█████████▊| 18307/18627 [10:23:20<10:37,  1.99s/it]
 98%|█████████▊| 18308/18627 [10:23:22<10:48,  2.03s/it]
 98%|█████████▊| 18309/18627 [10:23:23<09:26,  1.78s/it]
 98%|█████████▊| 18310/18627 [10:23:25<09:56,  1.88s/it]
                                                        
{'loss': 0.9607, 'grad_norm': 6.924693584442139, 'learning_rate': 3.7966290354607416e-09, 'epoch': 0.98}
+
 98%|█████████▊| 18310/18627 [10:23:25<09:56,  1.88s/it]
 98%|█████████▊| 18311/18627 [10:23:27<10:17,  1.95s/it]
 98%|█████████▊| 18312/18627 [10:23:29<10:31,  2.00s/it]
 98%|█████████▊| 18313/18627 [10:23:32<10:39,  2.04s/it]
 98%|█████████▊| 18314/18627 [10:23:34<10:46,  2.06s/it]
 98%|█████████▊| 18315/18627 [10:23:35<09:22,  1.80s/it]
 98%|█████████▊| 18316/18627 [10:23:37<09:51,  1.90s/it]
 98%|█████████▊| 18317/18627 [10:23:39<10:09,  1.97s/it]
 98%|█████████▊| 18318/18627 [10:23:41<10:21,  2.01s/it]
 98%|█████████▊| 18319/18627 [10:23:43<10:31,  2.05s/it]
 98%|█████████▊| 18320/18627 [10:23:45<09:17,  1.82s/it]
                                                        
{'loss': 1.009, 'grad_norm': 15.666465759277344, 'learning_rate': 3.5609282184198567e-09, 'epoch': 0.98}
+
 98%|█████████▊| 18320/18627 [10:23:45<09:17,  1.82s/it]
 98%|█████████▊| 18321/18627 [10:23:47<09:47,  1.92s/it]
 98%|█████████▊| 18322/18627 [10:23:48<08:40,  1.71s/it]
 98%|█████████▊| 18323/18627 [10:23:49<07:52,  1.55s/it]
 98%|█████████▊| 18324/18627 [10:23:51<08:46,  1.74s/it]
 98%|█████████▊| 18325/18627 [10:23:53<07:56,  1.58s/it]
 98%|█████████▊| 18326/18627 [10:23:55<08:46,  1.75s/it]
 98%|█████████▊| 18327/18627 [10:23:57<09:19,  1.86s/it]
 98%|█████████▊| 18328/18627 [10:23:59<09:40,  1.94s/it]
 98%|█████████▊| 18329/18627 [10:24:00<08:30,  1.71s/it]
 98%|█████████▊| 18330/18627 [10:24:02<09:05,  1.84s/it]
                                                        
{'loss': 1.428, 'grad_norm': 5.486618995666504, 'learning_rate': 3.332774854831744e-09, 'epoch': 0.98}
+
 98%|█████████▊| 18330/18627 [10:24:02<09:05,  1.84s/it]
 98%|█████████▊| 18331/18627 [10:24:04<09:26,  1.92s/it]
 98%|█████████▊| 18332/18627 [10:24:07<09:41,  1.97s/it]
 98%|█████████▊| 18333/18627 [10:24:09<09:51,  2.01s/it]
 98%|█████████▊| 18334/18627 [10:24:10<08:37,  1.77s/it]
 98%|█████████▊| 18335/18627 [10:24:12<09:05,  1.87s/it]
 98%|█████████▊| 18336/18627 [10:24:13<07:58,  1.65s/it]
 98%|█████████▊| 18337/18627 [10:24:15<08:38,  1.79s/it]
 98%|█████████▊| 18338/18627 [10:24:16<07:44,  1.61s/it]
 98%|█████████▊| 18339/18627 [10:24:18<08:26,  1.76s/it]
 98%|█████████▊| 18340/18627 [10:24:21<08:55,  1.87s/it]
                                                        
{'loss': 1.1813, 'grad_norm': 5.269192695617676, 'learning_rate': 3.1121696344701968e-09, 'epoch': 0.98}
+
 98%|█████████▊| 18340/18627 [10:24:21<08:55,  1.87s/it]
 98%|█████████▊| 18341/18627 [10:24:22<07:56,  1.67s/it]
 98%|█████████▊| 18342/18627 [10:24:24<08:34,  1.81s/it]
 98%|█████████▊| 18343/18627 [10:24:26<08:58,  1.90s/it]
 98%|█████████▊| 18344/18627 [10:24:27<07:58,  1.69s/it]
 98%|█████████▊| 18345/18627 [10:24:28<07:16,  1.55s/it]
 98%|█████████▊| 18346/18627 [10:24:30<06:45,  1.44s/it]
 98%|█████████▊| 18347/18627 [10:24:32<07:43,  1.66s/it]
 99%|█████████▊| 18348/18627 [10:24:34<08:25,  1.81s/it]
 99%|█████████▊| 18349/18627 [10:24:36<08:52,  1.92s/it]
 99%|█████████▊| 18350/18627 [10:24:37<07:51,  1.70s/it]
                                                        
{'loss': 1.6131, 'grad_norm': 14.692547798156738, 'learning_rate': 2.899113224287542e-09, 'epoch': 0.99}
+
 99%|█████████▊| 18350/18627 [10:24:37<07:51,  1.70s/it]
 99%|█████████▊| 18351/18627 [10:24:39<07:06,  1.55s/it]
 99%|█████████▊| 18352/18627 [10:24:41<07:52,  1.72s/it]
 99%|█████████▊| 18353/18627 [10:24:43<08:22,  1.83s/it]
 99%|█████████▊| 18354/18627 [10:24:45<08:44,  1.92s/it]
 99%|█████████▊| 18355/18627 [10:24:47<08:57,  1.98s/it]
 99%|█████████▊| 18356/18627 [10:24:48<07:51,  1.74s/it]
 99%|█████████▊| 18357/18627 [10:24:50<08:20,  1.85s/it]
 99%|█████████▊| 18358/18627 [10:24:52<08:40,  1.93s/it]
 99%|█████████▊| 18359/18627 [10:24:55<08:53,  1.99s/it]
 99%|█████████▊| 18360/18627 [10:24:57<09:03,  2.03s/it]
                                                        
{'loss': 0.9784, 'grad_norm': 4.017879962921143, 'learning_rate': 2.6936062684151942e-09, 'epoch': 0.99}
+
 99%|█████████▊| 18360/18627 [10:24:57<09:03,  2.03s/it]
 99%|█████████▊| 18361/18627 [10:24:58<07:53,  1.78s/it]
 99%|█████████▊| 18362/18627 [10:25:00<08:19,  1.88s/it]
 99%|████████���▊| 18363/18627 [10:25:01<06:44,  1.53s/it]
 99%|█████████▊| 18364/18627 [10:25:02<06:17,  1.43s/it]
 99%|█████████▊| 18365/18627 [10:25:04<07:09,  1.64s/it]
 99%|█████████▊| 18366/18627 [10:25:06<07:46,  1.79s/it]
 99%|█████████▊| 18367/18627 [10:25:08<08:11,  1.89s/it]
 99%|█████████▊| 18368/18627 [10:25:10<07:16,  1.69s/it]
 99%|█████████▊| 18369/18627 [10:25:12<07:52,  1.83s/it]
 99%|█████████▊| 18370/18627 [10:25:14<08:17,  1.94s/it]
                                                        
{'loss': 1.1613, 'grad_norm': 5.900289058685303, 'learning_rate': 2.4956493881594935e-09, 'epoch': 0.99}
+
 99%|█████████▊| 18370/18627 [10:25:14<08:17,  1.94s/it]
 99%|█████████▊| 18371/18627 [10:25:16<08:35,  2.01s/it]
 99%|█████████▊| 18372/18627 [10:25:18<08:46,  2.06s/it]
 99%|█████████▊| 18373/18627 [10:25:20<08:48,  2.08s/it]
 99%|█████████▊| 18374/18627 [10:25:22<08:48,  2.09s/it]
 99%|█████████▊| 18375/18627 [10:25:25<08:47,  2.09s/it]
 99%|█████████▊| 18376/18627 [10:25:27<08:46,  2.10s/it]
 99%|█████████▊| 18377/18627 [10:25:29<08:46,  2.10s/it]
 99%|█████████▊| 18378/18627 [10:25:31<08:46,  2.12s/it]
 99%|█████████▊| 18379/18627 [10:25:33<08:44,  2.11s/it]
 99%|█████████▊| 18380/18627 [10:25:35<08:41,  2.11s/it]
                                                        
{'loss': 0.6572, 'grad_norm': 5.436233043670654, 'learning_rate': 2.305243182001149e-09, 'epoch': 0.99}
+
 99%|█████████▊| 18380/18627 [10:25:35<08:41,  2.11s/it]
 99%|█████████▊| 18381/18627 [10:25:36<07:31,  1.84s/it]
 99%|█████████▊| 18382/18627 [10:25:38<07:49,  1.92s/it]
 99%|█████████▊| 18383/18627 [10:25:41<08:02,  1.98s/it]
 99%|█████████▊| 18384/18627 [10:25:43<08:11,  2.02s/it]
 99%|█████████▊| 18385/18627 [10:25:44<07:10,  1.78s/it]
 99%|█████████▊| 18386/18627 [10:25:46<07:32,  1.88s/it]
 99%|█████████▊| 18387/18627 [10:25:48<07:47,  1.95s/it]
 99%|█████████▊| 18388/18627 [10:25:50<07:58,  2.00s/it]
 99%|█████████▊| 18389/18627 [10:25:51<06:58,  1.76s/it]
 99%|█████████▊| 18390/18627 [10:25:54<07:23,  1.87s/it]
                                                        
{'loss': 1.1928, 'grad_norm': 5.662302494049072, 'learning_rate': 2.122388225592742e-09, 'epoch': 0.99}
+
 99%|█████████▊| 18390/18627 [10:25:54<07:23,  1.87s/it]
 99%|█████████▊| 18391/18627 [10:25:56<07:42,  1.96s/it]
 99%|█████████▊| 18392/18627 [10:25:58<07:56,  2.03s/it]
 99%|█████████▊| 18393/18627 [10:26:00<08:02,  2.06s/it]
 99%|█████████▊| 18394/18627 [10:26:02<08:07,  2.09s/it]
 99%|█████████▉| 18395/18627 [10:26:04<08:07,  2.10s/it]
 99%|█████████▉| 18396/18627 [10:26:06<08:07,  2.11s/it]
 99%|█████████▉| 18397/18627 [10:26:09<08:06,  2.12s/it]
 99%|█████████▉| 18398/18627 [10:26:11<08:04,  2.11s/it]
 99%|█████████▉| 18399/18627 [10:26:12<06:58,  1.84s/it]
 99%|█████████▉| 18400/18627 [10:26:13<06:09,  1.63s/it]
                                                        
{'loss': 0.9659, 'grad_norm': 13.394158363342285, 'learning_rate': 1.947085071757615e-09, 'epoch': 0.99}
+
 99%|█████████▉| 18400/18627 [10:26:13<06:09,  1.63s/it]
 99%|█████████▉| 18401/18627 [10:26:15<06:40,  1.77s/it]
 99%|█████████▉| 18402/18627 [10:26:17<07:02,  1.88s/it]
 99%|█████████▉| 18403/18627 [10:26:18<06:13,  1.67s/it]
 99%|█████████▉| 18404/18627 [10:26:21<06:42,  1.81s/it]
 99%|█████████▉| 18405/18627 [10:26:23<07:02,  1.90s/it]
 99%|█████████▉| 18406/18627 [10:26:25<07:15,  1.97s/it]
 99%|█████████▉| 18407/18627 [10:26:27<07:23,  2.02s/it]
 99%|█████████▉| 18408/18627 [10:26:28<06:26,  1.77s/it]
 99%|█████████▉| 18409/18627 [10:26:30<06:47,  1.87s/it]
 99%|█████████▉| 18410/18627 [10:26:32<07:02,  1.95s/it]
                                                        
{'loss': 0.9742, 'grad_norm': 5.0899505615234375, 'learning_rate': 1.7793342504876521e-09, 'epoch': 0.99}
+
 99%|█████████▉| 18410/18627 [10:26:32<07:02,  1.95s/it]
 99%|█████████▉| 18411/18627 [10:26:35<07:11,  2.00s/it]
 99%|█████████▉| 18412/18627 [10:26:37<07:19,  2.05s/it]
 99%|█████████▉| 18413/18627 [10:26:39<07:27,  2.09s/it]
 99%|████��████▉| 18414/18627 [10:26:40<06:04,  1.71s/it]
 99%|█████████▉| 18415/18627 [10:26:42<06:33,  1.86s/it]
 99%|█████████▉| 18416/18627 [10:26:44<06:52,  1.96s/it]
 99%|█████████▉| 18417/18627 [10:26:46<07:02,  2.01s/it]
 99%|█████████▉| 18418/18627 [10:26:48<07:07,  2.05s/it]
 99%|█████████▉| 18419/18627 [10:26:50<06:11,  1.79s/it]
 99%|█████████▉| 18420/18627 [10:26:52<06:31,  1.89s/it]
                                                        
{'loss': 0.7846, 'grad_norm': 7.820817947387695, 'learning_rate': 1.6191362689413347e-09, 'epoch': 0.99}
+
 99%|█████████▉| 18420/18627 [10:26:52<06:31,  1.89s/it]
 99%|█████████▉| 18421/18627 [10:26:54<06:42,  1.96s/it]
 99%|█████████▉| 18422/18627 [10:26:56<06:51,  2.01s/it]
 99%|█████████▉| 18423/18627 [10:26:58<06:56,  2.04s/it]
 99%|█████████▉| 18424/18627 [10:27:00<06:58,  2.06s/it]
 99%|█████████▉| 18425/18627 [10:27:02<07:01,  2.09s/it]
 99%|█████████▉| 18426/18627 [10:27:04<07:01,  2.10s/it]
 99%|█████████▉| 18427/18627 [10:27:07<07:01,  2.11s/it]
 99%|█████████▉| 18428/18627 [10:27:08<06:04,  1.83s/it]
 99%|█████████▉| 18429/18627 [10:27:10<06:19,  1.92s/it]
 99%|█████████▉| 18430/18627 [10:27:12<06:28,  1.97s/it]
                                                        
{'loss': 0.7813, 'grad_norm': 13.910109519958496, 'learning_rate': 1.4664916114440209e-09, 'epoch': 0.99}
+
 99%|█████████▉| 18430/18627 [10:27:12<06:28,  1.97s/it]
 99%|█████████▉| 18431/18627 [10:27:13<05:40,  1.74s/it]
 99%|█████████▉| 18432/18627 [10:27:15<06:01,  1.85s/it]
 99%|█████████▉| 18433/18627 [10:27:17<06:15,  1.94s/it]
 99%|█████████▉| 18434/18627 [10:27:20<06:25,  2.00s/it]
 99%|█████████▉| 18435/18627 [10:27:21<05:39,  1.77s/it]
 99%|█████████▉| 18436/18627 [10:27:23<06:03,  1.90s/it]
 99%|█████████▉| 18437/18627 [10:27:24<05:15,  1.66s/it]
 99%|█████████▉| 18438/18627 [10:27:26<05:43,  1.82s/it]
 99%|█████████▉| 18439/18627 [10:27:28<06:01,  1.92s/it]
 99%|█████████▉| 18440/18627 [10:27:32<07:30,  2.41s/it]
                                                        
{'loss': 1.1285, 'grad_norm': 8.051785469055176, 'learning_rate': 1.3214007394837801e-09, 'epoch': 0.99}
+
 99%|█████████▉| 18440/18627 [10:27:32<07:30,  2.41s/it]
 99%|█████████▉| 18441/18627 [10:27:34<07:10,  2.32s/it]
 99%|█████████▉| 18442/18627 [10:27:36<06:57,  2.26s/it]
 99%|█████████▉| 18443/18627 [10:27:38<06:47,  2.22s/it]
 99%|█████████▉| 18444/18627 [10:27:40<06:40,  2.19s/it]
 99%|█████████▉| 18445/18627 [10:27:42<05:43,  1.89s/it]
 99%|█████████▉| 18446/18627 [10:27:44<05:55,  1.96s/it]
 99%|█████████▉| 18447/18627 [10:27:45<05:11,  1.73s/it]
 99%|█████████▉| 18448/18627 [10:27:47<05:30,  1.85s/it]
 99%|█████████▉| 18449/18627 [10:27:49<05:43,  1.93s/it]
 99%|█████████▉| 18450/18627 [10:27:50<05:02,  1.71s/it]
                                                        
{'loss': 1.2327, 'grad_norm': 14.118709564208984, 'learning_rate': 1.1838640917125055e-09, 'epoch': 0.99}
+
 99%|█████████▉| 18450/18627 [10:27:50<05:02,  1.71s/it]
 99%|█████████▉| 18451/18627 [10:27:52<04:33,  1.56s/it]
 99%|█████████▉| 18452/18627 [10:27:54<05:02,  1.73s/it]
 99%|█████████▉| 18453/18627 [10:27:56<05:20,  1.84s/it]
 99%|█████████▉| 18454/18627 [10:27:58<05:33,  1.93s/it]
 99%|█████████▉| 18455/18627 [10:28:00<05:40,  1.98s/it]
 99%|█████████▉| 18456/18627 [10:28:02<05:47,  2.03s/it]
 99%|█████████▉| 18457/18627 [10:28:04<05:52,  2.07s/it]
 99%|█████████▉| 18458/18627 [10:28:07<05:56,  2.11s/it]
 99%|█████████▉| 18459/18627 [10:28:09<05:57,  2.13s/it]
 99%|█████████▉| 18460/18627 [10:28:11<05:57,  2.14s/it]
                                                        
{'loss': 0.8404, 'grad_norm': 5.370258331298828, 'learning_rate': 1.0538820839425813e-09, 'epoch': 0.99}
+
 99%|█████████▉| 18460/18627 [10:28:11<05:57,  2.14s/it]
 99%|█████████▉| 18461/18627 [10:28:12<05:08,  1.86s/it]
 99%|█████████▉| 18462/18627 [10:28:13<04:34,  1.66s/it]
 99%|█████████▉| 18463/18627 [10:28:15<04:54,  1.80s/it]
 99%|█████████▉| 18464/18627 [10:28:18<05:08,  1.90s/it]
 99%|█████████▉| 18465/18627 [10:28:20<05:16,  1.96s/it]
 99%|█████████▉| 18466/18627 [10:28:21<04:37,  1.73s/it]
 99%|█████████▉| 18467/18627 [10:28:23<04:55,  1.85s/it]
 99%|█████████▉| 18468/18627 [10:28:24<04:22,  1.65s/it]
 99%|█████████▉| 18469/18627 [10:28:26<04:42,  1.79s/it]
 99%|█████████▉| 18470/18627 [10:28:28<04:56,  1.89s/it]
                                                        
{'loss': 1.3685, 'grad_norm': 7.795014381408691, 'learning_rate': 9.31455109147439e-10, 'epoch': 0.99}
+
 99%|█████████▉| 18470/18627 [10:28:28<04:56,  1.89s/it]
 99%|█████████▉| 18471/18627 [10:28:30<04:21,  1.68s/it]
 99%|█████████▉| 18472/18627 [10:28:32<04:41,  1.82s/it]
 99%|█████████▉| 18473/18627 [10:28:34<04:52,  1.90s/it]
 99%|█████████▉| 18474/18627 [10:28:36<05:01,  1.97s/it]
 99%|█████████▉| 18475/18627 [10:28:38<05:06,  2.02s/it]
 99%|█████████▉| 18476/18627 [10:28:39<04:27,  1.77s/it]
 99%|█████████▉| 18477/18627 [10:28:41<04:42,  1.88s/it]
 99%|█████████▉| 18478/18627 [10:28:44<04:50,  1.95s/it]
 99%|█████████▉| 18479/18627 [10:28:46<04:58,  2.01s/it]
 99%|█████████▉| 18480/18627 [10:28:47<04:18,  1.76s/it]
                                                        
{'loss': 1.1518, 'grad_norm': 14.720968246459961, 'learning_rate': 8.165835374590592e-10, 'epoch': 0.99}
+
 99%|█████████▉| 18480/18627 [10:28:47<04:18,  1.76s/it]
 99%|█████████▉| 18481/18627 [10:28:49<04:35,  1.89s/it]
 99%|█████████▉| 18482/18627 [10:28:51<04:46,  1.98s/it]
 99%|█████████▉| 18483/18627 [10:28:53<04:52,  2.03s/it]
 99%|█████████▉| 18484/18627 [10:28:55<04:53,  2.06s/it]
 99%|█████████▉| 18485/18627 [10:28:57<04:14,  1.80s/it]
 99%|█████████▉| 18486/18627 [10:28:58<03:47,  1.61s/it]
 99%|█████████▉| 18487/18627 [10:28:59<03:28,  1.49s/it]
 99%|█████████▉| 18488/18627 [10:29:00<03:11,  1.38s/it]
 99%|█████████▉| 18489/18627 [10:29:02<03:40,  1.60s/it]
 99%|█████████▉| 18490/18627 [10:29:04<04:01,  1.77s/it]
                                                        
{'loss': 1.3963, 'grad_norm': 6.658933162689209, 'learning_rate': 7.092677161668615e-10, 'epoch': 0.99}
+
 99%|█████████▉| 18490/18627 [10:29:04<04:01,  1.77s/it]
 99%|█████████▉| 18491/18627 [10:29:06<03:36,  1.59s/it]
 99%|█████████▉| 18492/18627 [10:29:08<03:56,  1.75s/it]
 99%|█████████▉| 18493/18627 [10:29:10<04:10,  1.87s/it]
 99%|█████████▉| 18494/18627 [10:29:12<04:18,  1.95s/it]
 99%|█████████▉| 18495/18627 [10:29:14<04:23,  2.00s/it]
 99%|█████████▉| 18496/18627 [10:29:18<05:15,  2.41s/it]
 99%|█████████▉| 18497/18627 [10:29:20<05:01,  2.32s/it]
 99%|█████████▉| 18498/18627 [10:29:22<04:51,  2.26s/it]
 99%|█████████▉| 18499/18627 [10:29:23<04:08,  1.94s/it]
 99%|█████████▉| 18500/18627 [10:29:25<04:12,  1.99s/it]
                                                        
{'loss': 1.0098, 'grad_norm': 6.427835941314697, 'learning_rate': 6.095079697174267e-10, 'epoch': 0.99}
+
 99%|█████████▉| 18500/18627 [10:29:25<04:12,  1.99s/it]
 99%|█████████▉| 18501/18627 [10:29:27<04:16,  2.03s/it]
 99%|█████████▉| 18502/18627 [10:29:29<04:18,  2.07s/it]
 99%|█████████▉| 18503/18627 [10:29:30<03:41,  1.79s/it]
 99%|█████████▉| 18504/18627 [10:29:33<03:53,  1.90s/it]
 99%|█████████▉| 18505/18627 [10:29:34<03:22,  1.66s/it]
 99%|█████████▉| 18506/18627 [10:29:36<03:39,  1.81s/it]
 99%|█████████▉| 18507/18627 [10:29:38<03:48,  1.91s/it]
 99%|█████████▉| 18508/18627 [10:29:40<03:54,  1.97s/it]
 99%|█████████▉| 18509/18627 [10:29:42<03:57,  2.01s/it]
 99%|█████████▉| 18510/18627 [10:29:44<03:58,  2.04s/it]
                                                        
{'loss': 0.9866, 'grad_norm': 4.885433673858643, 'learning_rate': 5.173045997136639e-10, 'epoch': 0.99}
+
 99%|█████████▉| 18510/18627 [10:29:44<03:58,  2.04s/it]
 99%|█████████▉| 18511/18627 [10:29:47<04:00,  2.07s/it]
 99%|█████████▉| 18512/18627 [10:29:49<04:00,  2.09s/it]
 99%|█████████▉| 18513/18627 [10:29:51<03:58,  2.09s/it]
 99%|█████████▉| 18514/18627 [10:29:53<03:57,  2.10s/it]
 99%|█████████▉| 18515/18627 [10:29:55<03:55,  2.10s/it]
 99%|█████████▉| 18516/18627 [10:29:57<03:54,  2.11s/it]
 99%|█████████▉| 18517/18627 [10:29:59<03:52,  2.12s/it]
 99%|█████████▉| 18518/18627 [10:30:01<03:50,  2.12s/it]
 99%|█████████▉| 18519/18627 [10:30:03<03:18,  1.84s/it]
 99%|█████████▉| 18520/18627 [10:30:05<03:26,  1.93s/it]
                                                        
{'loss': 0.7839, 'grad_norm': 6.107245445251465, 'learning_rate': 4.3265788491259066e-10, 'epoch': 0.99}
+
 99%|█████████▉| 18520/18627 [10:30:05<03:26,  1.93s/it]
 99%|█████████▉| 18521/18627 [10:30:06<03:01,  1.71s/it]
 99%|█████████▉| 18522/18627 [10:30:08<03:13,  1.84s/it]
 99%|█████████▉| 18523/18627 [10:30:10<03:20,  1.93s/it]
 99%|█████████▉| 18524/18627 [10:30:12<03:25,  1.99s/it]
 99%|█████████▉| 18525/18627 [10:30:15<03:29,  2.05s/it]
 99%|█████████▉| 18526/18627 [10:30:17<03:30,  2.08s/it]
 99%|█████████▉| 18527/18627 [10:30:19<03:31,  2.11s/it]
 99%|█████████▉| 18528/18627 [10:30:21<03:28,  2.11s/it]
 99%|█████████▉| 18529/18627 [10:30:23<03:26,  2.11s/it]
 99%|█████████▉| 18530/18627 [10:30:25<03:25,  2.11s/it]
                                                        
{'loss': 0.8508, 'grad_norm': 6.574924945831299, 'learning_rate': 3.5556808122561017e-10, 'epoch': 0.99}
+
 99%|█████████▉| 18530/18627 [10:30:25<03:25,  2.11s/it]
 99%|█████████▉| 18531/18627 [10:30:27<03:23,  2.12s/it]
 99%|█████████▉| 18532/18627 [10:30:29<03:21,  2.12s/it]
 99%|█████████▉| 18533/18627 [10:30:32<03:18,  2.11s/it]
100%|█████████▉| 18534/18627 [10:30:34<03:16,  2.11s/it]
100%|█████████▉| 18535/18627 [10:30:34<02:35,  1.69s/it]
100%|█████████▉| 18536/18627 [10:30:36<02:45,  1.82s/it]
100%|█████████▉| 18537/18627 [10:30:39<02:51,  1.91s/it]
100%|█████████▉| 18538/18627 [10:30:41<02:55,  1.98s/it]
100%|█████████▉| 18539/18627 [10:30:41<02:20,  1.59s/it]
100%|█████████▉| 18540/18627 [10:30:44<02:32,  1.75s/it]
                                                        
{'loss': 0.5612, 'grad_norm': 5.311215877532959, 'learning_rate': 2.8603542171740107e-10, 'epoch': 1.0}
+
100%|█████████▉| 18540/18627 [10:30:44<02:32,  1.75s/it]
100%|█████████▉| 18541/18627 [10:30:44<02:03,  1.44s/it]
100%|█████████▉| 18542/18627 [10:30:46<02:19,  1.64s/it]
100%|█████████▉| 18543/18627 [10:30:48<02:30,  1.79s/it]
100%|█████████▉| 18544/18627 [10:30:51<02:36,  1.89s/it]
100%|█████████▉| 18545/18627 [10:30:52<02:17,  1.68s/it]
100%|█████████▉| 18546/18627 [10:30:54<02:27,  1.82s/it]
100%|█████████▉| 18547/18627 [10:30:56<02:34,  1.93s/it]
100%|█████████▉| 18548/18627 [10:30:58<02:39,  2.02s/it]
100%|█████████▉| 18549/18627 [10:31:01<02:40,  2.06s/it]
100%|█████████▉| 18550/18627 [10:31:03<02:41,  2.10s/it]
                                                        
{'loss': 0.746, 'grad_norm': 7.114765644073486, 'learning_rate': 2.2406011660508486e-10, 'epoch': 1.0}
+
100%|█████████▉| 18550/18627 [10:31:03<02:41,  2.10s/it]
100%|█████████▉| 18551/18627 [10:31:04<02:19,  1.83s/it]
100%|█████████▉| 18552/18627 [10:31:06<02:23,  1.92s/it]
100%|█████████▉| 18553/18627 [10:31:08<02:26,  1.98s/it]
100%|█████████▉| 18554/18627 [10:31:10<02:27,  2.02s/it]
100%|█████████▉| 18555/18627 [10:31:12<02:27,  2.05s/it]
100%|█████████▉| 18556/18627 [10:31:14<02:27,  2.07s/it]
100%|█████████▉| 18557/18627 [10:31:17<02:25,  2.08s/it]
100%|█████████▉| 18558/18627 [10:31:19<02:24,  2.10s/it]
100%|█████████▉| 18559/18627 [10:31:21<02:23,  2.11s/it]
100%|█████████▉| 18560/18627 [10:31:23<02:21,  2.11s/it]
                                                        
{'loss': 0.7767, 'grad_norm': 5.029722213745117, 'learning_rate': 1.6964235325794832e-10, 'epoch': 1.0}
+
100%|█████████▉| 18560/18627 [10:31:23<02:21,  2.11s/it]
100%|█████████▉| 18561/18627 [10:31:24<02:00,  1.83s/it]
100%|█████████▉| 18562/18627 [10:31:26<02:04,  1.91s/it]
100%|█████████▉| 18563/18627 [10:31:27<01:48,  1.70s/it]
100%|█████████▉| 18564/18627 [10:31:30<01:54,  1.82s/it]
100%|█████████▉| 18565/18627 [10:31:32<01:58,  1.91s/it]
100%|█████████▉| 18566/18627 [10:31:34<02:00,  1.97s/it]
100%|█████████▉| 18567/18627 [10:31:36<02:01,  2.02s/it]
100%|█████████▉| 18568/18627 [10:31:38<02:01,  2.06s/it]
100%|█████████▉| 18569/18627 [10:31:40<02:01,  2.10s/it]
100%|█████████▉| 18570/18627 [10:31:41<01:43,  1.81s/it]
                                                        
{'loss': 1.2298, 'grad_norm': 15.190839767456055, 'learning_rate': 1.227822961963332e-10, 'epoch': 1.0}
+
100%|█████████▉| 18570/18627 [10:31:41<01:43,  1.81s/it]
100%|█████████▉| 18571/18627 [10:31:44<01:47,  1.93s/it]
100%|█████████▉| 18572/18627 [10:31:46<01:49,  2.00s/it]
100%|█████████▉| 18573/18627 [10:31:48<01:49,  2.03s/it]
100%|█████████▉| 18574/18627 [10:31:49<01:34,  1.78s/it]
100%|█████████▉| 18575/18627 [10:31:51<01:37,  1.88s/it]
100%|█████████▉| 18576/18627 [10:31:53<01:39,  1.95s/it]
100%|█████████▉| 18577/18627 [10:31:55<01:40,  2.01s/it]
100%|█████████▉| 18578/18627 [10:31:58<01:39,  2.04s/it]
100%|█████████▉| 18579/18627 [10:32:00<01:39,  2.06s/it]
100%|█████████▉| 18580/18627 [10:32:02<01:37,  2.07s/it]
                                                        
{'loss': 0.7517, 'grad_norm': 5.3325066566467285, 'learning_rate': 8.34800870919139e-11, 'epoch': 1.0}
+
100%|█████████▉| 18580/18627 [10:32:02<01:37,  2.07s/it]
100%|█████████▉| 18581/18627 [10:32:04<01:36,  2.09s/it]
100%|█████████▉| 18582/18627 [10:32:06<01:34,  2.10s/it]
100%|█████████▉| 18583/18627 [10:32:08<01:32,  2.11s/it]
100%|█████████▉| 18584/18627 [10:32:10<01:31,  2.12s/it]
100%|█████████▉| 18585/18627 [10:32:12<01:28,  2.12s/it]
100%|█████████▉| 18586/18627 [10:32:15<01:26,  2.12s/it]
100%|█████████▉| 18587/18627 [10:32:17<01:24,  2.12s/it]
100%|█████████▉| 18588/18627 [10:32:19<01:22,  2.12s/it]
100%|█████████▉| 18589/18627 [10:32:21<01:21,  2.14s/it]
100%|█████████▉| 18590/18627 [10:32:23<01:19,  2.16s/it]
                                                        
{'loss': 0.6129, 'grad_norm': 6.228683948516846, 'learning_rate': 5.1735844766587215e-11, 'epoch': 1.0}
+
100%|█████████▉| 18590/18627 [10:32:23<01:19,  2.16s/it]
100%|█████████▉| 18591/18627 [10:32:24<01:07,  1.88s/it]
100%|█████████▉| 18592/18627 [10:32:27<01:09,  1.97s/it]
100%|█████████▉| 18593/18627 [10:32:29<01:09,  2.03s/it]
100%|█████████▉| 18594/18627 [10:32:31<01:07,  2.05s/it]
100%|█████████▉| 18595/18627 [10:32:33<01:06,  2.07s/it]
100%|█████████▉| 18596/18627 [10:32:35<01:04,  2.09s/it]
100%|█████████▉| 18597/18627 [10:32:37<01:02,  2.10s/it]
100%|█████████▉| 18598/18627 [10:32:39<01:01,  2.10s/it]
100%|█████████▉| 18599/18627 [10:32:41<00:58,  2.10s/it]
100%|█████████▉| 18600/18627 [10:32:44<00:56,  2.10s/it]
                                                        
{'loss': 0.7911, 'grad_norm': 6.248929500579834, 'learning_rate': 2.754966519219471e-11, 'epoch': 1.0}
+
100%|█████████▉| 18600/18627 [10:32:44<00:56,  2.10s/it]
100%|█████████▉| 18601/18627 [10:32:45<00:46,  1.79s/it]
100%|█████████▉| 18602/18627 [10:32:47<00:47,  1.89s/it]
100%|█████████▉| 18603/18627 [10:32:49<00:47,  1.96s/it]
100%|█████████▉| 18604/18627 [10:32:51<00:46,  2.01s/it]
100%|█████████▉| 18605/18627 [10:32:52<00:35,  1.61s/it]
100%|█████████▉| 18606/18627 [10:32:53<00:31,  1.49s/it]
100%|█████████▉| 18607/18627 [10:32:54<00:27,  1.40s/it]
100%|█████████▉| 18608/18627 [10:32:56<00:30,  1.61s/it]
100%|█████████▉| 18609/18627 [10:32:57<00:26,  1.48s/it]
100%|█████████▉| 18610/18627 [10:32:58<00:21,  1.25s/it]
                                                        
{'loss': 1.4155, 'grad_norm': 7.371974468231201, 'learning_rate': 1.0921621490522782e-11, 'epoch': 1.0}
+
100%|█████████▉| 18610/18627 [10:32:58<00:21,  1.25s/it]
100%|█████████▉| 18611/18627 [10:33:00<00:24,  1.51s/it]
100%|█████████▉| 18612/18627 [10:33:01<00:21,  1.41s/it]
100%|█████████▉| 18613/18627 [10:33:04<00:22,  1.63s/it]
100%|█████████▉| 18614/18627 [10:33:06<00:23,  1.79s/it]
100%|█████████▉| 18615/18627 [10:33:08<00:22,  1.90s/it]
100%|█████████▉| 18616/18627 [10:33:10<00:21,  1.98s/it]
100%|█████████▉| 18617/18627 [10:33:12<00:20,  2.03s/it]
100%|█████████▉| 18618/18627 [10:33:13<00:16,  1.78s/it]
100%|█████████▉| 18619/18627 [10:33:15<00:15,  1.89s/it]
100%|█████████▉| 18620/18627 [10:33:18<00:13,  1.96s/it]
                                                        
{'loss': 0.9206, 'grad_norm': 5.834037780761719, 'learning_rate': 1.851763933025108e-12, 'epoch': 1.0}
+
100%|█████████▉| 18620/18627 [10:33:18<00:13,  1.96s/it]
100%|█████████▉| 18621/18627 [10:33:20<00:12,  2.01s/it]
100%|█████████▉| 18622/18627 [10:33:22<00:10,  2.04s/it]
100%|█████████▉| 18623/18627 [10:33:23<00:07,  1.78s/it]
100%|█████████▉| 18624/18627 [10:33:25<00:05,  1.88s/it]
100%|█████████▉| 18625/18627 [10:33:27<00:03,  1.95s/it]
100%|█████████▉| 18626/18627 [10:33:32<00:02,  2.71s/it]
                                                        
{'train_runtime': 38012.227, 'train_samples_per_second': 7.84, 'train_steps_per_second': 0.49, 'train_loss': 1.1301016090994784, 'epoch': 1.0}
+
100%|█████████▉| 18626/18627 [10:33:32<00:02,  2.71s/it]
100%|█████████▉| 18626/18627 [10:33:32<00:02,  2.04s/it]
+[rank0]:[W531 03:45:02.977345473 ProcessGroupNCCL.cpp:1496] Warning: WARNING: destroy_process_group() was not called before program exit, which can leak resources. For more info, please see https://pytorch.org/docs/stable/distributed.html#shutdown (function operator())