| Collecting git+https://github.com/cheind/py-thin-plate-spline |
| Cloning https://github.com/cheind/py-thin-plate-spline to /tmp/pip-req-build-lr87xsyc |
| Resolved https://github.com/cheind/py-thin-plate-spline to commit f6995795397118b7d0ac01aecd3f39ffbfad9dee |
| Preparing metadata (setup.py): started |
| Preparing metadata (setup.py): finished with status 'done' |
| Collecting absl-py==1.4.0 |
| Downloading absl_py-1.4.0-py3-none-any.whl.metadata (2.3 kB) |
| Collecting appdirs==1.4.4 |
| Downloading appdirs-1.4.4-py2.py3-none-any.whl.metadata (9.0 kB) |
| Collecting cachetools==5.3.1 |
| Downloading cachetools-5.3.1-py3-none-any.whl.metadata (5.2 kB) |
| Collecting certifi==2023.5.7 |
| Downloading certifi-2023.5.7-py3-none-any.whl.metadata (2.2 kB) |
| Collecting charset-normalizer==3.1.0 |
| Downloading charset_normalizer-3.1.0-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (30 kB) |
| Collecting click==8.1.4 |
| Downloading click-8.1.4-py3-none-any.whl.metadata (3.0 kB) |
| Collecting cmake==3.26.4 |
| Downloading cmake-3.26.4-py2.py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl.metadata (6.7 kB) |
| Collecting contourpy==1.1.0 |
| Downloading contourpy-1.1.0-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (5.7 kB) |
| Collecting cycler==0.11.0 |
| Downloading cycler-0.11.0-py3-none-any.whl.metadata (785 bytes) |
| Collecting debugpy==1.5.1 |
| Downloading debugpy-1.5.1-py2.py3-none-any.whl.metadata (1.2 kB) |
| Collecting decord==0.6.0 |
| Downloading decord-0.6.0-py3-none-manylinux2010_x86_64.whl.metadata (422 bytes) |
| Collecting docker-pycreds==0.4.0 |
| Downloading docker_pycreds-0.4.0-py2.py3-none-any.whl.metadata (1.8 kB) |
| Collecting filelock==3.12.2 |
| Downloading filelock-3.12.2-py3-none-any.whl.metadata (2.7 kB) |
| Collecting fonttools==4.40.0 |
| Downloading fonttools-4.40.0-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (148 kB) |
| Collecting gdown |
| Downloading gdown-5.2.0-py3-none-any.whl.metadata (5.8 kB) |
| Collecting gitdb==4.0.10 |
| Downloading gitdb-4.0.10-py3-none-any.whl.metadata (1.1 kB) |
| Collecting gitpython==3.1.31 |
| Downloading GitPython-3.1.31-py3-none-any.whl.metadata (1.3 kB) |
| Collecting google-auth-oauthlib==1.0.0 |
| Downloading google_auth_oauthlib-1.0.0-py2.py3-none-any.whl.metadata (2.7 kB) |
| Collecting google-auth==2.22.0 |
| Downloading google_auth-2.22.0-py2.py3-none-any.whl.metadata (4.2 kB) |
| Collecting grpcio==1.56.2 |
| Downloading grpcio-1.56.2-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (4.0 kB) |
| Collecting hickle |
| Downloading hickle-5.0.3-py3-none-any.whl.metadata (22 kB) |
| Collecting idna==3.4 |
| Downloading idna-3.4-py3-none-any.whl.metadata (9.8 kB) |
| Collecting imageio==2.31.5 |
| Downloading imageio-2.31.5-py3-none-any.whl.metadata (4.6 kB) |
| Collecting jinja2==3.1.2 |
| Downloading Jinja2-3.1.2-py3-none-any.whl.metadata (3.5 kB) |
| Collecting joblib==1.2.0 |
| Downloading joblib-1.2.0-py3-none-any.whl.metadata (5.3 kB) |
| Collecting jupyter-core==5.3.1 |
| Downloading jupyter_core-5.3.1-py3-none-any.whl.metadata (3.4 kB) |
| Collecting kiwisolver==1.4.4 |
| Downloading kiwisolver-1.4.4-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (6.3 kB) |
| Collecting kornia==0.6.12 |
| Downloading kornia-0.6.12-py2.py3-none-any.whl.metadata (12 kB) |
| Collecting lazy-loader==0.3 |
| Downloading lazy_loader-0.3-py3-none-any.whl.metadata (4.3 kB) |
| Collecting lit==16.0.6 |
| Downloading lit-16.0.6.tar.gz (153 kB) |
| Installing build dependencies: started |
| Installing build dependencies: finished with status 'done' |
| Getting requirements to build wheel: started |
| Getting requirements to build wheel: finished with status 'done' |
| Preparing metadata (pyproject.toml): started |
| Preparing metadata (pyproject.toml): finished with status 'done' |
| Collecting lzstring |
| Downloading lzstring-1.0.4.tar.gz (4.3 kB) |
| Preparing metadata (setup.py): started |
| Preparing metadata (setup.py): finished with status 'done' |
| Collecting markdown==3.4.3 |
| Downloading Markdown-3.4.3-py3-none-any.whl.metadata (6.7 kB) |
| Collecting markupsafe==2.1.3 |
| Downloading MarkupSafe-2.1.3-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (3.0 kB) |
| Collecting matplotlib==3.7.1 |
| Downloading matplotlib-3.7.1-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (5.6 kB) |
| Collecting mpmath==1.3.0 |
| Downloading mpmath-1.3.0-py3-none-any.whl.metadata (8.6 kB) |
| Collecting natsort==8.4.0 |
| Downloading natsort-8.4.0-py3-none-any.whl.metadata (21 kB) |
| Collecting networkx==3.1 |
| Downloading networkx-3.1-py3-none-any.whl.metadata (5.3 kB) |
| Collecting numpy==1.25.0 |
| Downloading numpy-1.25.0-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (5.6 kB) |
| Collecting nvidia-cublas-cu11==11.10.3.66 |
| Downloading nvidia_cublas_cu11-11.10.3.66-py3-none-manylinux1_x86_64.whl.metadata (1.6 kB) |
| Collecting nvidia-cuda-cupti-cu11==11.7.101 |
| Downloading nvidia_cuda_cupti_cu11-11.7.101-py3-none-manylinux1_x86_64.whl.metadata (1.6 kB) |
| Collecting nvidia-cuda-nvrtc-cu11==11.7.99 |
| Downloading nvidia_cuda_nvrtc_cu11-11.7.99-2-py3-none-manylinux1_x86_64.whl.metadata (1.5 kB) |
| Collecting nvidia-cuda-runtime-cu11==11.7.99 |
| Downloading nvidia_cuda_runtime_cu11-11.7.99-py3-none-manylinux1_x86_64.whl.metadata (1.6 kB) |
| Collecting nvidia-cudnn-cu11==8.5.0.96 |
| Downloading nvidia_cudnn_cu11-8.5.0.96-2-py3-none-manylinux1_x86_64.whl.metadata (1.6 kB) |
| Collecting nvidia-cufft-cu11==10.9.0.58 |
| Downloading nvidia_cufft_cu11-10.9.0.58-py3-none-manylinux2014_x86_64.whl.metadata (1.5 kB) |
| Collecting nvidia-curand-cu11==10.2.10.91 |
| Downloading nvidia_curand_cu11-10.2.10.91-py3-none-manylinux1_x86_64.whl.metadata (1.6 kB) |
| Collecting nvidia-cusolver-cu11==11.4.0.1 |
| Downloading nvidia_cusolver_cu11-11.4.0.1-2-py3-none-manylinux1_x86_64.whl.metadata (1.6 kB) |
| Collecting nvidia-cusparse-cu11==11.7.4.91 |
| Downloading nvidia_cusparse_cu11-11.7.4.91-py3-none-manylinux1_x86_64.whl.metadata (1.6 kB) |
| Collecting nvidia-nccl-cu11==2.14.3 |
| Downloading nvidia_nccl_cu11-2.14.3-py3-none-manylinux1_x86_64.whl.metadata (1.8 kB) |
| Collecting nvidia-nvtx-cu11==11.7.91 |
| Downloading nvidia_nvtx_cu11-11.7.91-py3-none-manylinux1_x86_64.whl.metadata (1.7 kB) |
| Collecting oauthlib==3.2.2 |
| Downloading oauthlib-3.2.2-py3-none-any.whl.metadata (7.5 kB) |
| Collecting opencv-python-headless |
| Downloading opencv_python_headless-4.10.0.84-cp37-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (20 kB) |
| Collecting opencv-python==4.7.0.72 |
| Downloading opencv_python-4.7.0.72-cp37-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (18 kB) |
| Collecting pandas==2.0.3 |
| Downloading pandas-2.0.3-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (18 kB) |
| Collecting pathtools==0.1.2 |
| Downloading pathtools-0.1.2.tar.gz (11 kB) |
| Preparing metadata (setup.py): started |
| Preparing metadata (setup.py): finished with status 'done' |
| Collecting pillow==9.5.0 |
| Downloading Pillow-9.5.0-cp311-cp311-manylinux_2_28_x86_64.whl.metadata (9.5 kB) |
| Collecting pip==23.1.2 |
| Downloading pip-23.1.2-py3-none-any.whl.metadata (4.1 kB) |
| Collecting progressbar2 |
| Downloading progressbar2-4.5.0-py3-none-any.whl.metadata (16 kB) |
| Collecting protobuf==4.23.4 |
| Downloading protobuf-4.23.4-cp37-abi3-manylinux2014_x86_64.whl.metadata (540 bytes) |
| Collecting psutil==5.9.0 |
| Downloading psutil-5.9.0.tar.gz (478 kB) |
| Preparing metadata (setup.py): started |
| Preparing metadata (setup.py): finished with status 'done' |
| Collecting pyasn1-modules==0.3.0 |
| Downloading pyasn1_modules-0.3.0-py2.py3-none-any.whl.metadata (3.6 kB) |
| Collecting pyasn1==0.5.0 |
| Downloading pyasn1-0.5.0-py2.py3-none-any.whl.metadata (8.5 kB) |
| Collecting pycocotools==2.0.6 |
| Downloading pycocotools-2.0.6.tar.gz (24 kB) |
| Installing build dependencies: started |
| Installing build dependencies: finished with status 'done' |
| Getting requirements to build wheel: started |
| Getting requirements to build wheel: finished with status 'done' |
| Preparing metadata (pyproject.toml): started |
| Preparing metadata (pyproject.toml): finished with status 'done' |
| Collecting pyparsing==3.1.0 |
| Downloading pyparsing-3.1.0-py3-none-any.whl.metadata (4.9 kB) |
| Collecting pytz==2023.3 |
| Downloading pytz-2023.3-py2.py3-none-any.whl.metadata (22 kB) |
| Collecting pyyaml==6.0 |
| Downloading PyYAML-6.0-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (2.1 kB) |
| Collecting pyzmq==25.1.0 |
| Downloading pyzmq-25.1.0-cp311-cp311-manylinux_2_28_x86_64.whl.metadata (4.8 kB) |
| Collecting requests-oauthlib==1.3.1 |
| Downloading requests_oauthlib-1.3.1-py2.py3-none-any.whl.metadata (10 kB) |
| Collecting requests==2.31.0 |
| Downloading requests-2.31.0-py3-none-any.whl.metadata (4.6 kB) |
| Collecting rsa==4.9 |
| Downloading rsa-4.9-py3-none-any.whl.metadata (4.2 kB) |
| Collecting scikit-image==0.22.0 |
| Downloading scikit_image-0.22.0-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (13 kB) |
| Collecting scikit-learn==1.2.2 |
| Downloading scikit_learn-1.2.2-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (11 kB) |
| Collecting scipy==1.11.0 |
| Downloading scipy-1.11.0-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (59 kB) |
| Collecting sentry-sdk==1.27.1 |
| Downloading sentry_sdk-1.27.1-py2.py3-none-any.whl.metadata (8.8 kB) |
| Collecting setproctitle==1.3.2 |
| Downloading setproctitle-1.3.2-cp311-cp311-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (9.8 kB) |
| Collecting setuptools==67.8.0 |
| Downloading setuptools-67.8.0-py3-none-any.whl.metadata (6.2 kB) |
| Collecting smmap==5.0.0 |
| Downloading smmap-5.0.0-py3-none-any.whl.metadata (4.2 kB) |
| Collecting sympy==1.12 |
| Downloading sympy-1.12-py3-none-any.whl.metadata (12 kB) |
| Collecting tensorboard-data-server==0.7.1 |
| Downloading tensorboard_data_server-0.7.1-py3-none-manylinux2014_x86_64.whl.metadata (1.1 kB) |
| Collecting tensorboard==2.13.0 |
| Downloading tensorboard-2.13.0-py3-none-any.whl.metadata (1.8 kB) |
| Collecting tensorboardx==2.6.1 |
| Downloading tensorboardX-2.6.1-py2.py3-none-any.whl.metadata (5.6 kB) |
| Collecting threadpoolctl==3.1.0 |
| Downloading threadpoolctl-3.1.0-py3-none-any.whl.metadata (9.2 kB) |
| Collecting tifffile==2023.9.26 |
| Downloading tifffile-2023.9.26-py3-none-any.whl.metadata (30 kB) |
| Collecting torch==2.0.1 |
| Downloading torch-2.0.1-cp311-cp311-manylinux1_x86_64.whl.metadata (24 kB) |
| Collecting torchvision==0.15.2 |
| Downloading torchvision-0.15.2-cp311-cp311-manylinux1_x86_64.whl.metadata (11 kB) |
| Collecting tornado==6.1 |
| Downloading tornado-6.1.tar.gz (497 kB) |
| Preparing metadata (setup.py): started |
| Preparing metadata (setup.py): finished with status 'done' |
| Collecting tqdm==4.65.0 |
| Downloading tqdm-4.65.0-py3-none-any.whl.metadata (56 kB) |
| Collecting triton==2.0.0 |
| Downloading triton-2.0.0-1-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.whl.metadata (1.0 kB) |
| Collecting tzdata==2023.3 |
| Downloading tzdata-2023.3-py2.py3-none-any.whl.metadata (1.4 kB) |
| Collecting urllib3==1.26.16 |
| Downloading urllib3-1.26.16-py2.py3-none-any.whl.metadata (48 kB) |
| Collecting wandb==0.15.5 |
| Downloading wandb-0.15.5-py3-none-any.whl.metadata (8.2 kB) |
| Collecting werkzeug==2.3.6 |
| Downloading Werkzeug-2.3.6-py3-none-any.whl.metadata (4.1 kB) |
| Collecting wheel==0.38.4 |
| Downloading wheel-0.38.4-py3-none-any.whl.metadata (2.1 kB) |
| Collecting six>=1.4.0 (from docker-pycreds==0.4.0) |
| Downloading six-1.16.0-py2.py3-none-any.whl.metadata (1.8 kB) |
| Collecting platformdirs>=2.5 (from jupyter-core==5.3.1) |
| Downloading platformdirs-4.3.2-py3-none-any.whl.metadata (11 kB) |
| Collecting traitlets>=5.3 (from jupyter-core==5.3.1) |
| Downloading traitlets-5.14.3-py3-none-any.whl.metadata (10 kB) |
| Collecting packaging (from kornia==0.6.12) |
| Downloading packaging-24.1-py3-none-any.whl.metadata (3.2 kB) |
| Collecting python-dateutil>=2.7 (from matplotlib==3.7.1) |
| Downloading python_dateutil-2.9.0.post0-py2.py3-none-any.whl.metadata (8.4 kB) |
| Collecting typing-extensions (from torch==2.0.1) |
| Downloading typing_extensions-4.12.2-py3-none-any.whl.metadata (3.0 kB) |
| Collecting beautifulsoup4 (from gdown) |
| Downloading beautifulsoup4-4.12.3-py3-none-any.whl.metadata (3.8 kB) |
| Collecting h5py>=2.10.0 (from hickle) |
| Downloading h5py-3.11.0-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (2.5 kB) |
| Collecting future>=0.14.0 (from lzstring) |
| Downloading future-1.0.0-py3-none-any.whl.metadata (4.0 kB) |
| Collecting python-utils>=3.8.1 (from progressbar2) |
| Downloading python_utils-3.8.2-py2.py3-none-any.whl.metadata (9.7 kB) |
| Collecting soupsieve>1.2 (from beautifulsoup4->gdown) |
| Downloading soupsieve-2.6-py3-none-any.whl.metadata (4.6 kB) |
| Collecting PySocks!=1.5.7,>=1.5.6 (from requests[socks]->gdown) |
| Downloading PySocks-1.7.1-py3-none-any.whl.metadata (13 kB) |
| Downloading absl_py-1.4.0-py3-none-any.whl (126 kB) |
| Downloading appdirs-1.4.4-py2.py3-none-any.whl (9.6 kB) |
| Downloading cachetools-5.3.1-py3-none-any.whl (9.3 kB) |
| Downloading certifi-2023.5.7-py3-none-any.whl (156 kB) |
| Downloading charset_normalizer-3.1.0-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (197 kB) |
| Downloading click-8.1.4-py3-none-any.whl (98 kB) |
| Downloading cmake-3.26.4-py2.py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl (24.0 MB) |
| ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 24.0/24.0 MB 131.2 MB/s eta 0:00:00 |
| Downloading contourpy-1.1.0-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (300 kB) |
| Downloading cycler-0.11.0-py3-none-any.whl (6.4 kB) |
| Downloading debugpy-1.5.1-py2.py3-none-any.whl (4.1 MB) |
| ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 4.1/4.1 MB 96.8 MB/s eta 0:00:00 |
| Downloading decord-0.6.0-py3-none-manylinux2010_x86_64.whl (13.6 MB) |
| ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 13.6/13.6 MB 193.3 MB/s eta 0:00:00 |
| Downloading docker_pycreds-0.4.0-py2.py3-none-any.whl (9.0 kB) |
| Downloading filelock-3.12.2-py3-none-any.whl (10 kB) |
| Downloading fonttools-4.40.0-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (4.3 MB) |
| ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 4.3/4.3 MB 33.8 MB/s eta 0:00:00 |
| Downloading gitdb-4.0.10-py3-none-any.whl (62 kB) |
| Downloading GitPython-3.1.31-py3-none-any.whl (184 kB) |
| Downloading google_auth_oauthlib-1.0.0-py2.py3-none-any.whl (18 kB) |
| Downloading google_auth-2.22.0-py2.py3-none-any.whl (181 kB) |
| Downloading grpcio-1.56.2-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (5.2 MB) |
| ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 5.2/5.2 MB 181.4 MB/s eta 0:00:00 |
| Downloading idna-3.4-py3-none-any.whl (61 kB) |
| Downloading imageio-2.31.5-py3-none-any.whl (313 kB) |
| Downloading Jinja2-3.1.2-py3-none-any.whl (133 kB) |
| Downloading joblib-1.2.0-py3-none-any.whl (297 kB) |
| Downloading jupyter_core-5.3.1-py3-none-any.whl (93 kB) |
| Downloading kiwisolver-1.4.4-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.4 MB) |
| ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.4/1.4 MB 90.4 MB/s eta 0:00:00 |
| Downloading kornia-0.6.12-py2.py3-none-any.whl (653 kB) |
| ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 653.4/653.4 kB 73.4 MB/s eta 0:00:00 |
| Downloading lazy_loader-0.3-py3-none-any.whl (9.1 kB) |
| Downloading Markdown-3.4.3-py3-none-any.whl (93 kB) |
| Downloading MarkupSafe-2.1.3-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (28 kB) |
| Downloading matplotlib-3.7.1-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (11.6 MB) |
| ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 11.6/11.6 MB 177.9 MB/s eta 0:00:00 |
| Downloading mpmath-1.3.0-py3-none-any.whl (536 kB) |
| ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 536.2/536.2 kB 101.5 MB/s eta 0:00:00 |
| Downloading natsort-8.4.0-py3-none-any.whl (38 kB) |
| Downloading networkx-3.1-py3-none-any.whl (2.1 MB) |
| ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 2.1/2.1 MB 221.3 MB/s eta 0:00:00 |
| Downloading numpy-1.25.0-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (17.6 MB) |
| ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 17.6/17.6 MB 152.1 MB/s eta 0:00:00 |
| Downloading nvidia_cublas_cu11-11.10.3.66-py3-none-manylinux1_x86_64.whl (317.1 MB) |
| ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 317.1/317.1 MB 192.1 MB/s eta 0:00:00 |
| Downloading nvidia_cuda_cupti_cu11-11.7.101-py3-none-manylinux1_x86_64.whl (11.8 MB) |
| ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 11.8/11.8 MB 189.1 MB/s eta 0:00:00 |
| Downloading nvidia_cuda_nvrtc_cu11-11.7.99-2-py3-none-manylinux1_x86_64.whl (21.0 MB) |
| ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 21.0/21.0 MB 149.4 MB/s eta 0:00:00 |
| Downloading nvidia_cuda_runtime_cu11-11.7.99-py3-none-manylinux1_x86_64.whl (849 kB) |
| ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 849.3/849.3 kB 166.1 MB/s eta 0:00:00 |
| Downloading nvidia_cudnn_cu11-8.5.0.96-2-py3-none-manylinux1_x86_64.whl (557.1 MB) |
| ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 557.1/557.1 MB 156.8 MB/s eta 0:00:00 |
| Downloading nvidia_cufft_cu11-10.9.0.58-py3-none-manylinux2014_x86_64.whl (168.4 MB) |
| ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 168.4/168.4 MB 173.3 MB/s eta 0:00:00 |
| Downloading nvidia_curand_cu11-10.2.10.91-py3-none-manylinux1_x86_64.whl (54.6 MB) |
| ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 54.6/54.6 MB 176.2 MB/s eta 0:00:00 |
| Downloading nvidia_cusolver_cu11-11.4.0.1-2-py3-none-manylinux1_x86_64.whl (102.6 MB) |
| ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 102.6/102.6 MB 191.9 MB/s eta 0:00:00 |
| Downloading nvidia_cusparse_cu11-11.7.4.91-py3-none-manylinux1_x86_64.whl (173.2 MB) |
| ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 173.2/173.2 MB 186.8 MB/s eta 0:00:00 |
| Downloading nvidia_nccl_cu11-2.14.3-py3-none-manylinux1_x86_64.whl (177.1 MB) |
| ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 177.1/177.1 MB 190.2 MB/s eta 0:00:00 |
| Downloading nvidia_nvtx_cu11-11.7.91-py3-none-manylinux1_x86_64.whl (98 kB) |
| Downloading oauthlib-3.2.2-py3-none-any.whl (151 kB) |
| Downloading opencv_python-4.7.0.72-cp37-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (61.8 MB) |
| ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 61.8/61.8 MB 152.7 MB/s eta 0:00:00 |
| Downloading pandas-2.0.3-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (12.2 MB) |
| ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 12.2/12.2 MB 194.8 MB/s eta 0:00:00 |
| Downloading Pillow-9.5.0-cp311-cp311-manylinux_2_28_x86_64.whl (3.4 MB) |
| ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 3.4/3.4 MB 173.3 MB/s eta 0:00:00 |
| Downloading pip-23.1.2-py3-none-any.whl (2.1 MB) |
| ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 2.1/2.1 MB 200.3 MB/s eta 0:00:00 |
| Downloading protobuf-4.23.4-cp37-abi3-manylinux2014_x86_64.whl (304 kB) |
| Downloading pyasn1_modules-0.3.0-py2.py3-none-any.whl (181 kB) |
| Downloading pyasn1-0.5.0-py2.py3-none-any.whl (83 kB) |
| Downloading pyparsing-3.1.0-py3-none-any.whl (102 kB) |
| Downloading pytz-2023.3-py2.py3-none-any.whl (502 kB) |
| Downloading PyYAML-6.0-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (757 kB) |
| ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 757.9/757.9 kB 158.3 MB/s eta 0:00:00 |
| Downloading pyzmq-25.1.0-cp311-cp311-manylinux_2_28_x86_64.whl (1.1 MB) |
| ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.1/1.1 MB 138.8 MB/s eta 0:00:00 |
| Downloading requests_oauthlib-1.3.1-py2.py3-none-any.whl (23 kB) |
| Downloading requests-2.31.0-py3-none-any.whl (62 kB) |
| Downloading rsa-4.9-py3-none-any.whl (34 kB) |
| Downloading scikit_image-0.22.0-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (14.7 MB) |
| ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 14.7/14.7 MB 129.8 MB/s eta 0:00:00 |
| Downloading scikit_learn-1.2.2-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (9.6 MB) |
| ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 9.6/9.6 MB 188.8 MB/s eta 0:00:00 |
| Downloading scipy-1.11.0-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (36.2 MB) |
| ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 36.2/36.2 MB 38.2 MB/s eta 0:00:00 |
| Downloading sentry_sdk-1.27.1-py2.py3-none-any.whl (211 kB) |
| Downloading setproctitle-1.3.2-cp311-cp311-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (31 kB) |
| Downloading setuptools-67.8.0-py3-none-any.whl (1.1 MB) |
| ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.1/1.1 MB 124.7 MB/s eta 0:00:00 |
| Downloading smmap-5.0.0-py3-none-any.whl (24 kB) |
| Downloading sympy-1.12-py3-none-any.whl (5.7 MB) |
| ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 5.7/5.7 MB 169.5 MB/s eta 0:00:00 |
| Downloading tensorboard_data_server-0.7.1-py3-none-manylinux2014_x86_64.whl (6.6 MB) |
| ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 6.6/6.6 MB 153.4 MB/s eta 0:00:00 |
| Downloading tensorboard-2.13.0-py3-none-any.whl (5.6 MB) |
| ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 5.6/5.6 MB 211.7 MB/s eta 0:00:00 |
| Downloading tensorboardX-2.6.1-py2.py3-none-any.whl (101 kB) |
| Downloading threadpoolctl-3.1.0-py3-none-any.whl (14 kB) |
| Downloading tifffile-2023.9.26-py3-none-any.whl (222 kB) |
| Downloading torch-2.0.1-cp311-cp311-manylinux1_x86_64.whl (619.9 MB) |
| ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 619.9/619.9 MB 116.6 MB/s eta 0:00:00 |
| Downloading torchvision-0.15.2-cp311-cp311-manylinux1_x86_64.whl (6.0 MB) |
| ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 6.0/6.0 MB 140.7 MB/s eta 0:00:00 |
| Downloading tqdm-4.65.0-py3-none-any.whl (77 kB) |
| Downloading triton-2.0.0-1-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.whl (63.3 MB) |
| ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 63.3/63.3 MB 146.3 MB/s eta 0:00:00 |
| Downloading tzdata-2023.3-py2.py3-none-any.whl (341 kB) |
| Downloading urllib3-1.26.16-py2.py3-none-any.whl (143 kB) |
| Downloading wandb-0.15.5-py3-none-any.whl (2.1 MB) |
| ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 2.1/2.1 MB 130.5 MB/s eta 0:00:00 |
| Downloading Werkzeug-2.3.6-py3-none-any.whl (242 kB) |
| Downloading wheel-0.38.4-py3-none-any.whl (36 kB) |
| Downloading gdown-5.2.0-py3-none-any.whl (18 kB) |
| Downloading hickle-5.0.3-py3-none-any.whl (107 kB) |
| Downloading opencv_python_headless-4.10.0.84-cp37-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (49.9 MB) |
| ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 49.9/49.9 MB 192.3 MB/s eta 0:00:00 |
| Downloading progressbar2-4.5.0-py3-none-any.whl (57 kB) |
| Downloading future-1.0.0-py3-none-any.whl (491 kB) |
| Downloading h5py-3.11.0-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (5.4 MB) |
| ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 5.4/5.4 MB 209.9 MB/s eta 0:00:00 |
| Downloading packaging-24.1-py3-none-any.whl (53 kB) |
| Downloading platformdirs-4.3.2-py3-none-any.whl (18 kB) |
| Downloading python_dateutil-2.9.0.post0-py2.py3-none-any.whl (229 kB) |
| Downloading python_utils-3.8.2-py2.py3-none-any.whl (27 kB) |
| Downloading six-1.16.0-py2.py3-none-any.whl (11 kB) |
| Downloading traitlets-5.14.3-py3-none-any.whl (85 kB) |
| Downloading typing_extensions-4.12.2-py3-none-any.whl (37 kB) |
| Downloading beautifulsoup4-4.12.3-py3-none-any.whl (147 kB) |
| Downloading PySocks-1.7.1-py3-none-any.whl (16 kB) |
| Downloading soupsieve-2.6-py3-none-any.whl (36 kB) |
| Building wheels for collected packages: lit, pathtools, psutil, pycocotools, tornado, thinplate, lzstring |
| Building wheel for lit (pyproject.toml): started |
| Building wheel for lit (pyproject.toml): finished with status 'done' |
| Created wheel for lit: filename=lit-16.0.6-py3-none-any.whl size=93587 sha256=516307eb365c7a34c36ad29221b78af5c0ef46efb81fd376ca67440e71d8cb97 |
| Stored in directory: /scratch/yuqian_fu/.cache/pip/wheels/ab/84/e4/5af8c76af9e5bee472e825f1451c18bb3b261d80a7b3ec7f8a |
| Building wheel for pathtools (setup.py): started |
| Building wheel for pathtools (setup.py): finished with status 'done' |
| Created wheel for pathtools: filename=pathtools-0.1.2-py3-none-any.whl size=8791 sha256=0326b7279d7b87442eed741583d4bc390bd0e03d78541d1e4fc3097353aaeaa1 |
| Stored in directory: /scratch/yuqian_fu/.cache/pip/wheels/ea/b7/8b/84e94095ea418b9442f5abeba4ca7b0ad52d3fe7b69d6238a6 |
| Building wheel for psutil (setup.py): started |
| Building wheel for psutil (setup.py): finished with status 'done' |
| Created wheel for psutil: filename=psutil-5.9.0-cp311-cp311-linux_x86_64.whl size=230265 sha256=4baf93c0dfeed31b573e43dbfbfb59ef540a2e421cc8a92f3aeadb5e2aeb2711 |
| Stored in directory: /scratch/yuqian_fu/.cache/pip/wheels/b7/3f/a5/6da60cf045cc7d875f3f6746c27e9afa159642fc5acaa5ae08 |
| Building wheel for pycocotools (pyproject.toml): started |
| Building wheel for pycocotools (pyproject.toml): finished with status 'done' |
| Created wheel for pycocotools: filename=pycocotools-2.0.6-cp311-cp311-linux_x86_64.whl size=104817 sha256=be529d436a996e27c495e5386d7db3901cf44a266f3cb026b9a4e11c6d5b52d4 |
| Stored in directory: /scratch/yuqian_fu/.cache/pip/wheels/ad/ca/ea/fb115e04c841c3f71fd369b7d9805a43a5193f4f9251bed0ec |
| Building wheel for tornado (setup.py): started |
| Building wheel for tornado (setup.py): finished with status 'done' |
| Created wheel for tornado: filename=tornado-6.1-cp311-cp311-linux_x86_64.whl size=416972 sha256=f476e7f7a7d846a6eca8249c3363f64c2a5235972b195b59903afdd2b1d09a3f |
| Stored in directory: /scratch/yuqian_fu/.cache/pip/wheels/f2/59/06/a9c85c7b17ec0fc9b1e2ae0c59e3d39255d5c0a38492e33fea |
| Building wheel for thinplate (setup.py): started |
| Building wheel for thinplate (setup.py): finished with status 'done' |
| Created wheel for thinplate: filename=thinplate-1.0.0-py3-none-any.whl size=6699 sha256=71cbe9f9b9d8a98e0c3ee3d7acbd0b07fce9df2a2193e3a45d78b1139af842f0 |
| Stored in directory: /tmp/pip-ephem-wheel-cache-78ai6_13/wheels/8d/a9/10/26074e234ee6057094ebcf080ff2f898ab16f418672fab12ad |
| Building wheel for lzstring (setup.py): started |
| Building wheel for lzstring (setup.py): finished with status 'done' |
| Created wheel for lzstring: filename=lzstring-1.0.4-py2.py3-none-any.whl size=4568 sha256=6589aea31ca6756c59667ce84b666cf31546bb97732885657a3e4fa038594290 |
| Stored in directory: /scratch/yuqian_fu/.cache/pip/wheels/a2/68/ae/ab2520f1ce001476a22df0405a1066359d6c4f14aad5f6aae1 |
| Successfully built lit pathtools psutil pycocotools tornado thinplate lzstring |
| Installing collected packages: pytz, pathtools, mpmath, lit, cmake, appdirs, wheel, urllib3, tzdata, typing-extensions, traitlets, tqdm, tornado, threadpoolctl, tensorboard-data-server, sympy, soupsieve, smmap, six, setuptools, setproctitle, pyzmq, pyyaml, PySocks, pyparsing, pyasn1, psutil, protobuf, platformdirs, pip, pillow, packaging, oauthlib, nvidia-nccl-cu11, nvidia-cufft-cu11, nvidia-cuda-nvrtc-cu11, numpy, networkx, natsort, markupsafe, markdown, lazy-loader, kiwisolver, joblib, idna, grpcio, future, fonttools, filelock, debugpy, cycler, click, charset-normalizer, certifi, cachetools, absl-py, werkzeug, tifffile, tensorboardx, sentry-sdk, scipy, rsa, requests, python-utils, python-dateutil, pyasn1-modules, opencv-python-headless, opencv-python, nvidia-nvtx-cu11, nvidia-cusparse-cu11, nvidia-curand-cu11, nvidia-cuda-runtime-cu11, nvidia-cuda-cupti-cu11, nvidia-cublas-cu11, lzstring, jupyter-core, jinja2, imageio, h5py, gitdb, docker-pycreds, decord, contourpy, beautifulsoup4, scikit-learn, scikit-image, requests-oauthlib, progressbar2, pandas, nvidia-cusolver-cu11, nvidia-cudnn-cu11, matplotlib, hickle, google-auth, gitpython, wandb, pycocotools, google-auth-oauthlib, gdown, tensorboard, triton, torch, torchvision, thinplate, kornia |
| Attempting uninstall: wheel |
| Found existing installation: wheel 0.44.0 |
| Uninstalling wheel-0.44.0: |
| Successfully uninstalled wheel-0.44.0 |
| Attempting uninstall: setuptools |
| Found existing installation: setuptools 73.0.1 |
| Uninstalling setuptools-73.0.1: |
| Successfully uninstalled setuptools-73.0.1 |
| Attempting uninstall: pip |
| Found existing installation: pip 24.2 |
| Uninstalling pip-24.2: |
| Successfully uninstalled pip-24.2 |
| Successfully installed PySocks-1.7.1 absl-py-1.4.0 appdirs-1.4.4 beautifulsoup4-4.12.3 cachetools-5.3.1 certifi-2023.5.7 charset-normalizer-3.1.0 click-8.1.4 cmake-3.26.4 contourpy-1.1.0 cycler-0.11.0 debugpy-1.5.1 decord-0.6.0 docker-pycreds-0.4.0 filelock-3.12.2 fonttools-4.40.0 future-1.0.0 gdown-5.2.0 gitdb-4.0.10 gitpython-3.1.31 google-auth-2.22.0 google-auth-oauthlib-1.0.0 grpcio-1.56.2 h5py-3.11.0 hickle-5.0.3 idna-3.4 imageio-2.31.5 jinja2-3.1.2 joblib-1.2.0 jupyter-core-5.3.1 kiwisolver-1.4.4 kornia-0.6.12 lazy-loader-0.3 lit-16.0.6 lzstring-1.0.4 markdown-3.4.3 markupsafe-2.1.3 matplotlib-3.7.1 mpmath-1.3.0 natsort-8.4.0 networkx-3.1 numpy-1.25.0 nvidia-cublas-cu11-11.10.3.66 nvidia-cuda-cupti-cu11-11.7.101 nvidia-cuda-nvrtc-cu11-11.7.99 nvidia-cuda-runtime-cu11-11.7.99 nvidia-cudnn-cu11-8.5.0.96 nvidia-cufft-cu11-10.9.0.58 nvidia-curand-cu11-10.2.10.91 nvidia-cusolver-cu11-11.4.0.1 nvidia-cusparse-cu11-11.7.4.91 nvidia-nccl-cu11-2.14.3 nvidia-nvtx-cu11-11.7.91 oauthlib-3.2.2 opencv-python-4.7.0.72 opencv-python-headless-4.10.0.84 packaging-24.1 pandas-2.0.3 pathtools-0.1.2 pillow-9.5.0 pip-23.1.2 platformdirs-4.3.2 progressbar2-4.5.0 protobuf-4.23.4 psutil-5.9.0 pyasn1-0.5.0 pyasn1-modules-0.3.0 pycocotools-2.0.6 pyparsing-3.1.0 python-dateutil-2.9.0.post0 python-utils-3.8.2 pytz-2023.3 pyyaml-6.0 pyzmq-25.1.0 requests-2.31.0 requests-oauthlib-1.3.1 rsa-4.9 scikit-image-0.22.0 scikit-learn-1.2.2 scipy-1.11.0 sentry-sdk-1.27.1 setproctitle-1.3.2 setuptools-67.8.0 six-1.16.0 smmap-5.0.0 soupsieve-2.6 sympy-1.12 tensorboard-2.13.0 tensorboard-data-server-0.7.1 tensorboardx-2.6.1 thinplate-1.0.0 threadpoolctl-3.1.0 tifffile-2023.9.26 torch-2.0.1 torchvision-0.15.2 tornado-6.1 tqdm-4.65.0 traitlets-5.14.3 triton-2.0.0 typing-extensions-4.12.2 tzdata-2023.3 urllib3-1.26.16 wandb-0.15.5 werkzeug-2.3.6 wheel-0.38.4 |
| CUDA Device count: 2 |
| CUDA Device count: 2 |
| CUDA Device count: 2 |
| CUDA Device count: 2 |
| I am rank 0 in this world of size 2! |
| We are assuming 2 GPUs. |
| We are now starting stage 3 |
| I will take the role of logging! |
| I am rank 1 in this world of size 2! |
| We are assuming 2 GPUs. |
| We are now starting stage 3 |
| Single object mode: False |
| I am rank 1 in this world of size 2! |
| We are assuming 2 GPUs. |
| We are now starting stage 3 |
| Single object mode: False |
| I am rank 0 in this world of size 2! |
| We are assuming 2 GPUs. |
| We are now starting stage 3 |
| I will take the role of logging! |
| git No Git Info |
| hyperpara {'benchmark': False, 'no_amp': False, 'swap': False, 'enable_segswap': False, 'segswap_model': None, 'static_root': '../static', 'bl_root': '../BL30K', 'yv_root': '../YouTube', 'egoexo_root': '/data/work-gcp-europe-west4-a/yuqian_fu/Ego/data_segswap', 'davis_root': '../DAVIS', 'num_workers': 4, 'key_dim': 64, 'value_dim': 512, 'hidden_dim': 64, 'deep_update_prob': 0.2, 'stages': '3', 's0_batch_size': 16, 's0_iterations': 150000, 's0_finetune': 0, 's0_steps': [], 's0_lr': 1e-05, 's0_num_ref_frames': 2, 's0_num_frames': 3, 's0_start_warm': 20000, 's0_end_warm': 70000, 's1_batch_size': 8, 's1_iterations': 250000, 's1_finetune': 0, 's1_steps': [200000], 's1_lr': 1e-05, 's1_num_ref_frames': 3, 's1_num_frames': 8, 's1_start_warm': 20000, 's1_end_warm': 70000, 's2_batch_size': 8, 's2_iterations': 150000, 's2_finetune': 10000, 's2_steps': [120000], 's2_lr': 1e-05, 's2_num_ref_frames': 3, 's2_num_frames': 8, 's2_start_warm': 20000, 's2_end_warm': 70000, 's3_batch_size': 8, 's3_iterations': 100000, 's3_finetune': 10000, 's3_steps': [80000], 's3_lr': 1e-05, 's3_num_ref_frames': 3, 's3_num_frames': 4, 's3_start_warm': 20000, 's3_end_warm': 70000, 'gamma': 0.1, 'weight_decay': 0.05, 'load_network': '/data/work-gcp-europe-west4-a/yuqian_fu/Ego/models/XMem.pth', 'load_checkpoint': None, 'log_text_interval': 100, 'log_image_interval': 1000, 'save_network_interval': 1000, 'save_checkpoint_interval': 2000, 'exp_id': 'xmem_egoexo_tune_s3', 'debug': False, 'local_rank': 0, 'amp': True, 'batch_size': 4, 'iterations': 100000, 'finetune': 10000, 'steps': [80000], 'lr': 1e-05, 'num_ref_frames': 3, 'num_frames': 4, 'start_warm': 20000, 'end_warm': 70000, 'single_object': False, 'num_gpus': 2} |
| Single object mode: False |
| git No Git Info |
| hyperpara {'benchmark': False, 'no_amp': False, 'swap': False, 'enable_segswap': False, 'segswap_model': None, 'static_root': '../static', 'bl_root': '../BL30K', 'yv_root': '../YouTube', 'egoexo_root': '/data/work-gcp-europe-west4-a/yuqian_fu/Ego/data_segswap', 'davis_root': '../DAVIS', 'num_workers': 4, 'key_dim': 64, 'value_dim': 512, 'hidden_dim': 64, 'deep_update_prob': 0.2, 'stages': '3', 's0_batch_size': 16, 's0_iterations': 150000, 's0_finetune': 0, 's0_steps': [], 's0_lr': 1e-05, 's0_num_ref_frames': 2, 's0_num_frames': 3, 's0_start_warm': 20000, 's0_end_warm': 70000, 's1_batch_size': 8, 's1_iterations': 250000, 's1_finetune': 0, 's1_steps': [200000], 's1_lr': 1e-05, 's1_num_ref_frames': 3, 's1_num_frames': 8, 's1_start_warm': 20000, 's1_end_warm': 70000, 's2_batch_size': 8, 's2_iterations': 150000, 's2_finetune': 10000, 's2_steps': [120000], 's2_lr': 1e-05, 's2_num_ref_frames': 3, 's2_num_frames': 8, 's2_start_warm': 20000, 's2_end_warm': 70000, 's3_batch_size': 8, 's3_iterations': 100000, 's3_finetune': 10000, 's3_steps': [80000], 's3_lr': 1e-05, 's3_num_ref_frames': 3, 's3_num_frames': 4, 's3_start_warm': 20000, 's3_end_warm': 70000, 'gamma': 0.1, 'weight_decay': 0.05, 'load_network': '/data/work-gcp-europe-west4-a/yuqian_fu/Ego/models/XMem.pth', 'load_checkpoint': None, 'log_text_interval': 100, 'log_image_interval': 1000, 'save_network_interval': 1000, 'save_checkpoint_interval': 2000, 'exp_id': 'xmem_egoexo_tune_s3', 'debug': False, 'local_rank': 0, 'amp': True, 'batch_size': 4, 'iterations': 100000, 'finetune': 10000, 'steps': [80000], 'lr': 1e-05, 'num_ref_frames': 3, 'num_frames': 4, 'start_warm': 20000, 'end_warm': 70000, 'single_object': False, 'num_gpus': 2} |
| Single object mode: False |
| model_size 62185224 |
| I am loading network from a disk, as listed in configuration |
| I am loading network from a disk, as listed in configuration |
| model_size 62185224 |
| I am loading network from a disk, as listed in configuration |
| I am loading network from a disk, as listed in configuration |
| Network weight loaded from memory.Network weight loaded from memory. |
| |
| Network weight loaded from /data/work-gcp-europe-west4-a/yuqian_fu/Ego/models/XMem.pth |
| Network weight loaded from /data/work-gcp-europe-west4-a/yuqian_fu/Ego/models/XMem.pth |
| Network weight loaded from memory. |
| Network weight loaded from /data/work-gcp-europe-west4-a/yuqian_fu/Ego/models/XMem.pth |
| Network weight loaded from memory. |
| Network weight loaded from /data/work-gcp-europe-west4-a/yuqian_fu/Ego/models/XMem.pth |
| 3090 out of 755 videos accepted in /data/work-gcp-europe-west4-a/yuqian_fu/Ego/data_segswap. |
| 3090 out of 755 videos accepted in /data/work-gcp-europe-west4-a/yuqian_fu/Ego/data_segswap. |
| 3090 out of 755 videos accepted in /data/work-gcp-europe-west4-a/yuqian_fu/Ego/data_segswap.3090 out of 755 videos accepted in /data/work-gcp-europe-west4-a/yuqian_fu/Ego/data_segswap. |
| |
| EgoExo dataset size: 3090EgoExo dataset size: 3090 |
| |
| Renewed with max_skip=5 |
| Renewed with max_skip=5 |
| EgoExo dataset size: 3090 |
| Renewed with max_skip=5 |
| EgoExo dataset size: 3090 |
| Renewed with max_skip=5 |
| We approximately use 260 epochs.We approximately use 260 epochs. |
| |
| We approximately use 260 epochs.We approximately use 260 epochs. |
| |
| The skip value will change approximately at the following iterations: [10000, 30000, 90000]The skip value will change approximately at the following iterations: [10000, 30000, 90000] |
|
|
| The skip value will change approximately at the following iterations: [10000, 30000, 90000]The skip value will change approximately at the following iterations: [10000, 30000, 90000] |
|
|
| Current epoch: 1 |
| Current epoch: 1Current epoch: 1 |
|
|
| Current epoch: 1 |
| xmem_egoexo_tune_s3 - It 385 [TRAIN] [time ]: 26.326623 |
| xmem_egoexo_tune_s3 - It 385 [TRAIN] [time ]: 26.327072 |
| xmem_egoexo_tune_s3 - It 385 [TRAIN] [total_loss ]: 1.9630768 |
| xmem_egoexo_tune_s3 - It 385 [TRAIN] [p ]: 1.0000000 |
| xmem_egoexo_tune_s3 - It 385 [TRAIN] [ce_loss_1 ]: 0.0207262 |
| xmem_egoexo_tune_s3 - It 385 [TRAIN] [dice_loss_1 ]: 0.5997622 |
| xmem_egoexo_tune_s3 - It 385 [TRAIN] [ce_loss_2 ]: 0.0239976 |
| xmem_egoexo_tune_s3 - It 385 [TRAIN] [dice_loss_2 ]: 0.6391031 |
| xmem_egoexo_tune_s3 - It 385 [TRAIN] [ce_loss_3 ]: 0.0246612 |
| xmem_egoexo_tune_s3 - It 385 [TRAIN] [dice_loss_3 ]: 0.6548261 |
| xmem_egoexo_tune_s3 - It 385 [TRAIN] [iou ]: 0.3001694 |
| xmem_egoexo_tune_s3 - It 385 [TRAIN] [total_loss ]: 1.9271796 |
| xmem_egoexo_tune_s3 - It 385 [TRAIN] [p ]: 1.0000000 |
| xmem_egoexo_tune_s3 - It 385 [TRAIN] [ce_loss_1 ]: 0.0202294 |
| xmem_egoexo_tune_s3 - It 385 [TRAIN] [dice_loss_1 ]: 0.5880547 |
| xmem_egoexo_tune_s3 - It 385 [TRAIN] [ce_loss_2 ]: 0.0235891 |
| xmem_egoexo_tune_s3 - It 385 [TRAIN] [dice_loss_2 ]: 0.6265825 |
| xmem_egoexo_tune_s3 - It 385 [TRAIN] [ce_loss_3 ]: 0.0244976 |
| xmem_egoexo_tune_s3 - It 385 [TRAIN] [dice_loss_3 ]: 0.6442261 |
| xmem_egoexo_tune_s3 - It 385 [TRAIN] [iou ]: 0.3108642 |
| Current epoch: 2 |
| Current epoch: 2 |
| Current epoch: 2 |
| Current epoch: 2 |
| xmem_egoexo_tune_s3 - It 771 [TRAIN] [time ]: 17.366277 |
| xmem_egoexo_tune_s3 - It 771 [TRAIN] [time ]: 17.366685 |
| xmem_egoexo_tune_s3 - It 771 [TRAIN] [total_loss ]: 1.4374012 |
| xmem_egoexo_tune_s3 - It 771 [TRAIN] [p ]: 1.0000000 |
| xmem_egoexo_tune_s3 - It 771 [TRAIN] [ce_loss_1 ]: 0.0130019 |
| xmem_egoexo_tune_s3 - It 771 [TRAIN] [dice_loss_1 ]: 0.4321885 |
| xmem_egoexo_tune_s3 - It 771 [TRAIN] [ce_loss_2 ]: 0.0153394 |
| xmem_egoexo_tune_s3 - It 771 [TRAIN] [dice_loss_2 ]: 0.4708838 |
| xmem_egoexo_tune_s3 - It 771 [TRAIN] [ce_loss_3 ]: 0.0167058 |
| xmem_egoexo_tune_s3 - It 771 [TRAIN] [dice_loss_3 ]: 0.4892815 |
| xmem_egoexo_tune_s3 - It 771 [TRAIN] [iou ]: 0.4583819 |
| xmem_egoexo_tune_s3 - It 771 [TRAIN] [total_loss ]: 1.4412586 |
| xmem_egoexo_tune_s3 - It 771 [TRAIN] [p ]: 1.0000000 |
| xmem_egoexo_tune_s3 - It 771 [TRAIN] [ce_loss_1 ]: 0.0129108 |
| xmem_egoexo_tune_s3 - It 771 [TRAIN] [dice_loss_1 ]: 0.4336698 |
| xmem_egoexo_tune_s3 - It 771 [TRAIN] [ce_loss_2 ]: 0.0150062 |
| xmem_egoexo_tune_s3 - It 771 [TRAIN] [dice_loss_2 ]: 0.4728092 |
| xmem_egoexo_tune_s3 - It 771 [TRAIN] [ce_loss_3 ]: 0.0162552 |
| xmem_egoexo_tune_s3 - It 771 [TRAIN] [dice_loss_3 ]: 0.4906072 |
| xmem_egoexo_tune_s3 - It 771 [TRAIN] [iou ]: 0.4570977 |
| Current epoch: 3 |
| Current epoch: 3 |
| Current epoch: 3 |
| Current epoch: 3 |
| Network saved to /data/work-gcp-europe-west4-a/yuqian_fu/Ego/saves_xmem2/Sep10_20.14.00_xmem_egoexo_tune_s3/Sep10_20.14.00_xmem_egoexo_tune_s3_1000.pth. |
| Network saved to /data/work-gcp-europe-west4-a/yuqian_fu/Ego/saves_xmem2/Sep10_20.14.00_xmem_egoexo_tune_s3/Sep10_20.14.00_xmem_egoexo_tune_s3_1000.pth. |
| xmem_egoexo_tune_s3 - It 1157 [TRAIN] [time ]: 17.287787 |
| xmem_egoexo_tune_s3 - It 1157 [TRAIN] [time ]: 17.289621 |
| xmem_egoexo_tune_s3 - It 1157 [TRAIN] [total_loss ]: 1.3348642 |
| xmem_egoexo_tune_s3 - It 1157 [TRAIN] [p ]: 1.0000000 |
| xmem_egoexo_tune_s3 - It 1157 [TRAIN] [ce_loss_1 ]: 0.0118151 |
| xmem_egoexo_tune_s3 - It 1157 [TRAIN] [dice_loss_1 ]: 0.4028752 |
| xmem_egoexo_tune_s3 - It 1157 [TRAIN] [ce_loss_2 ]: 0.0124762 |
| xmem_egoexo_tune_s3 - It 1157 [TRAIN] [dice_loss_2 ]: 0.4339893 |
| xmem_egoexo_tune_s3 - It 1157 [TRAIN] [ce_loss_3 ]: 0.0148608 |
| xmem_egoexo_tune_s3 - It 1157 [TRAIN] [dice_loss_3 ]: 0.4588474 |
| xmem_egoexo_tune_s3 - It 1157 [TRAIN] [iou ]: 0.4914688 |
| xmem_egoexo_tune_s3 - It 1157 [TRAIN] [total_loss ]: 1.3443501 |
| xmem_egoexo_tune_s3 - It 1157 [TRAIN] [p ]: 1.0000000 |
| xmem_egoexo_tune_s3 - It 1157 [TRAIN] [ce_loss_1 ]: 0.0117923 |
| xmem_egoexo_tune_s3 - It 1157 [TRAIN] [dice_loss_1 ]: 0.4054139 |
| xmem_egoexo_tune_s3 - It 1157 [TRAIN] [ce_loss_2 ]: 0.0124050 |
| xmem_egoexo_tune_s3 - It 1157 [TRAIN] [dice_loss_2 ]: 0.4374599 |
| xmem_egoexo_tune_s3 - It 1157 [TRAIN] [ce_loss_3 ]: 0.0147124 |
| xmem_egoexo_tune_s3 - It 1157 [TRAIN] [dice_loss_3 ]: 0.4625663 |
| xmem_egoexo_tune_s3 - It 1157 [TRAIN] [iou ]: 0.4891421 |
| Current epoch: 4 |
| Current epoch: 4 |
| Current epoch: 4 |
| Current epoch: 4 |
| xmem_egoexo_tune_s3 - It 1543 [TRAIN] [time ]: 17.336733 |
| xmem_egoexo_tune_s3 - It 1543 [TRAIN] [total_loss ]: 1.2890151 |
| xmem_egoexo_tune_s3 - It 1543 [TRAIN] [p ]: 1.0000000 |
| xmem_egoexo_tune_s3 - It 1543 [TRAIN] [ce_loss_1 ]: 0.0118880 |
| xmem_egoexo_tune_s3 - It 1543 [TRAIN] [dice_loss_1 ]: 0.3835255 |
| xmem_egoexo_tune_s3 - It 1543 [TRAIN] [ce_loss_2 ]: 0.0127974 |
| xmem_egoexo_tune_s3 - It 1543 [TRAIN] [dice_loss_2 ]: 0.4164177 |
| xmem_egoexo_tune_s3 - It 1543 [TRAIN] [ce_loss_3 ]: 0.0194035 |
| xmem_egoexo_tune_s3 - It 1543 [TRAIN] [dice_loss_3 ]: 0.4449827 |
| xmem_egoexo_tune_s3 - It 1543 [TRAIN] [iou ]: 0.5070235 |
| xmem_egoexo_tune_s3 - It 1543 [TRAIN] [time ]: 17.339390 |
| xmem_egoexo_tune_s3 - It 1543 [TRAIN] [total_loss ]: 1.3309963 |
| xmem_egoexo_tune_s3 - It 1543 [TRAIN] [p ]: 1.0000000 |
| xmem_egoexo_tune_s3 - It 1543 [TRAIN] [ce_loss_1 ]: 0.0118243 |
| xmem_egoexo_tune_s3 - It 1543 [TRAIN] [dice_loss_1 ]: 0.3959992 |
| xmem_egoexo_tune_s3 - It 1543 [TRAIN] [ce_loss_2 ]: 0.0134215 |
| xmem_egoexo_tune_s3 - It 1543 [TRAIN] [dice_loss_2 ]: 0.4298661 |
| xmem_egoexo_tune_s3 - It 1543 [TRAIN] [ce_loss_3 ]: 0.0200917 |
| xmem_egoexo_tune_s3 - It 1543 [TRAIN] [dice_loss_3 ]: 0.4597933 |
| xmem_egoexo_tune_s3 - It 1543 [TRAIN] [iou ]: 0.4978615 |
| Current epoch: 5 |
| Current epoch: 5 |
| Current epoch: 5 |
| Current epoch: 5 |
| xmem_egoexo_tune_s3 - It 1929 [TRAIN] [time ]: 17.204149 |
| xmem_egoexo_tune_s3 - It 1929 [TRAIN] [total_loss ]: 1.2134232 |
| xmem_egoexo_tune_s3 - It 1929 [TRAIN] [p ]: 1.0000000 |
| xmem_egoexo_tune_s3 - It 1929 [TRAIN] [ce_loss_1 ]: 0.0100513 |
| xmem_egoexo_tune_s3 - It 1929 [TRAIN] [dice_loss_1 ]: 0.3656548 |
| xmem_egoexo_tune_s3 - It 1929 [TRAIN] [ce_loss_2 ]: 0.0117032 |
| xmem_egoexo_tune_s3 - It 1929 [TRAIN] [dice_loss_2 ]: 0.3945764 |
| xmem_egoexo_tune_s3 - It 1929 [TRAIN] [ce_loss_3 ]: 0.0125156 |
| xmem_egoexo_tune_s3 - It 1929 [TRAIN] [dice_loss_3 ]: 0.4189217 |
| xmem_egoexo_tune_s3 - It 1929 [TRAIN] [iou ]: 0.5264138 |
| xmem_egoexo_tune_s3 - It 1929 [TRAIN] [time ]: 17.211286 |
| xmem_egoexo_tune_s3 - It 1929 [TRAIN] [total_loss ]: 1.1659077 |
| xmem_egoexo_tune_s3 - It 1929 [TRAIN] [p ]: 1.0000000 |
| xmem_egoexo_tune_s3 - It 1929 [TRAIN] [ce_loss_1 ]: 0.0096625 |
| xmem_egoexo_tune_s3 - It 1929 [TRAIN] [dice_loss_1 ]: 0.3509466 |
| xmem_egoexo_tune_s3 - It 1929 [TRAIN] [ce_loss_2 ]: 0.0115409 |
| xmem_egoexo_tune_s3 - It 1929 [TRAIN] [dice_loss_2 ]: 0.3813270 |
| xmem_egoexo_tune_s3 - It 1929 [TRAIN] [ce_loss_3 ]: 0.0121315 |
| xmem_egoexo_tune_s3 - It 1929 [TRAIN] [dice_loss_3 ]: 0.4002988 |
| xmem_egoexo_tune_s3 - It 1929 [TRAIN] [iou ]: 0.5411596 |
| Current epoch: 6 |
| Current epoch: 6 |
| Current epoch: 6 |
| Current epoch: 6 |
| Network saved to /data/work-gcp-europe-west4-a/yuqian_fu/Ego/saves_xmem2/Sep10_20.14.00_xmem_egoexo_tune_s3/Sep10_20.14.00_xmem_egoexo_tune_s3_2000.pth. |
| Network saved to /data/work-gcp-europe-west4-a/yuqian_fu/Ego/saves_xmem2/Sep10_20.14.00_xmem_egoexo_tune_s3/Sep10_20.14.00_xmem_egoexo_tune_s3_2000.pth. |
| Checkpoint saved to /data/work-gcp-europe-west4-a/yuqian_fu/Ego/saves_xmem2/Sep10_20.14.00_xmem_egoexo_tune_s3/Sep10_20.14.00_xmem_egoexo_tune_s3_checkpoint_2000.pth. |
| Checkpoint saved to /data/work-gcp-europe-west4-a/yuqian_fu/Ego/saves_xmem2/Sep10_20.14.00_xmem_egoexo_tune_s3/Sep10_20.14.00_xmem_egoexo_tune_s3_checkpoint_2000.pth. |
| xmem_egoexo_tune_s3 - It 2315 [TRAIN] [time ]: 17.227245 |
| xmem_egoexo_tune_s3 - It 2315 [TRAIN] [total_loss ]: 1.1449062 |
| xmem_egoexo_tune_s3 - It 2315 [TRAIN] [p ]: 1.0000000 |
| xmem_egoexo_tune_s3 - It 2315 [TRAIN] [ce_loss_1 ]: 0.0105613 |
| xmem_egoexo_tune_s3 - It 2315 [TRAIN] [dice_loss_1 ]: 0.3416202 |
| xmem_egoexo_tune_s3 - It 2315 [TRAIN] [ce_loss_2 ]: 0.0111439 |
| xmem_egoexo_tune_s3 - It 2315 [TRAIN] [dice_loss_2 ]: 0.3721945 |
| xmem_egoexo_tune_s3 - It 2315 [TRAIN] [ce_loss_3 ]: 0.0129656 |
| xmem_egoexo_tune_s3 - It 2315 [TRAIN] [dice_loss_3 ]: 0.3964205 |
| xmem_egoexo_tune_s3 - It 2315 [TRAIN] [iou ]: 0.5485376 |
| xmem_egoexo_tune_s3 - It 2315 [TRAIN] [time ]: 17.230772 |
| xmem_egoexo_tune_s3 - It 2315 [TRAIN] [total_loss ]: 1.1638792 |
| xmem_egoexo_tune_s3 - It 2315 [TRAIN] [p ]: 1.0000000 |
| xmem_egoexo_tune_s3 - It 2315 [TRAIN] [ce_loss_1 ]: 0.0108808 |
| xmem_egoexo_tune_s3 - It 2315 [TRAIN] [dice_loss_1 ]: 0.3479303 |
| xmem_egoexo_tune_s3 - It 2315 [TRAIN] [ce_loss_2 ]: 0.0112283 |
| xmem_egoexo_tune_s3 - It 2315 [TRAIN] [dice_loss_2 ]: 0.3786448 |
| xmem_egoexo_tune_s3 - It 2315 [TRAIN] [ce_loss_3 ]: 0.0129068 |
| xmem_egoexo_tune_s3 - It 2315 [TRAIN] [dice_loss_3 ]: 0.4022880 |
| xmem_egoexo_tune_s3 - It 2315 [TRAIN] [iou ]: 0.5420842 |
| Current epoch: 7 |
| Current epoch: 7 |
| Current epoch: 7 |
| Current epoch: 7 |
| xmem_egoexo_tune_s3 - It 2701 [TRAIN] [time ]: 17.145550 |
| xmem_egoexo_tune_s3 - It 2701 [TRAIN] [time ]: 17.144847 |
| xmem_egoexo_tune_s3 - It 2701 [TRAIN] [total_loss ]: 1.1077121 |
| xmem_egoexo_tune_s3 - It 2701 [TRAIN] [p ]: 1.0000000 |
| xmem_egoexo_tune_s3 - It 2701 [TRAIN] [ce_loss_1 ]: 0.0097082 |
| xmem_egoexo_tune_s3 - It 2701 [TRAIN] [dice_loss_1 ]: 0.3342654 |
| xmem_egoexo_tune_s3 - It 2701 [TRAIN] [ce_loss_2 ]: 0.0108173 |
| xmem_egoexo_tune_s3 - It 2701 [TRAIN] [dice_loss_2 ]: 0.3612714 |
| xmem_egoexo_tune_s3 - It 2701 [TRAIN] [ce_loss_3 ]: 0.0114320 |
| xmem_egoexo_tune_s3 - It 2701 [TRAIN] [dice_loss_3 ]: 0.3802175 |
| xmem_egoexo_tune_s3 - It 2701 [TRAIN] [iou ]: 0.5592049 |
| xmem_egoexo_tune_s3 - It 2701 [TRAIN] [total_loss ]: 1.0899891 |
| xmem_egoexo_tune_s3 - It 2701 [TRAIN] [p ]: 1.0000000 |
| xmem_egoexo_tune_s3 - It 2701 [TRAIN] [ce_loss_1 ]: 0.0096085 |
| xmem_egoexo_tune_s3 - It 2701 [TRAIN] [dice_loss_1 ]: 0.3278658 |
| xmem_egoexo_tune_s3 - It 2701 [TRAIN] [ce_loss_2 ]: 0.0105472 |
| xmem_egoexo_tune_s3 - It 2701 [TRAIN] [dice_loss_2 ]: 0.3538666 |
| xmem_egoexo_tune_s3 - It 2701 [TRAIN] [ce_loss_3 ]: 0.0112839 |
| xmem_egoexo_tune_s3 - It 2701 [TRAIN] [dice_loss_3 ]: 0.3768168 |
| xmem_egoexo_tune_s3 - It 2701 [TRAIN] [iou ]: 0.5658308 |
| Current epoch: 8 |
| Current epoch: 8 |
| Current epoch: 8 |
| Current epoch: 8 |
| Network saved to /data/work-gcp-europe-west4-a/yuqian_fu/Ego/saves_xmem2/Sep10_20.14.00_xmem_egoexo_tune_s3/Sep10_20.14.00_xmem_egoexo_tune_s3_3000.pth. |
| Network saved to /data/work-gcp-europe-west4-a/yuqian_fu/Ego/saves_xmem2/Sep10_20.14.00_xmem_egoexo_tune_s3/Sep10_20.14.00_xmem_egoexo_tune_s3_3000.pth. |
| xmem_egoexo_tune_s3 - It 3087 [TRAIN] [time ]: 17.110273 |
| xmem_egoexo_tune_s3 - It 3087 [TRAIN] [total_loss ]: 1.0893781 |
| xmem_egoexo_tune_s3 - It 3087 [TRAIN] [p ]: 1.0000000 |
| xmem_egoexo_tune_s3 - It 3087 [TRAIN] [ce_loss_1 ]: 0.0088374 |
| xmem_egoexo_tune_s3 - It 3087 [TRAIN] [dice_loss_1 ]: 0.3230016 |
| xmem_egoexo_tune_s3 - It 3087 [TRAIN] [ce_loss_2 ]: 0.0107509 |
| xmem_egoexo_tune_s3 - It 3087 [TRAIN] [dice_loss_2 ]: 0.3609083 |
| xmem_egoexo_tune_s3 - It 3087 [TRAIN] [ce_loss_3 ]: 0.0114650 |
| xmem_egoexo_tune_s3 - It 3087 [TRAIN] [dice_loss_3 ]: 0.3744145 |
| xmem_egoexo_tune_s3 - It 3087 [TRAIN] [iou ]: 0.5650082 |
| xmem_egoexo_tune_s3 - It 3087 [TRAIN] [time ]: 17.110378 |
| xmem_egoexo_tune_s3 - It 3087 [TRAIN] [total_loss ]: 1.1071137 |
| xmem_egoexo_tune_s3 - It 3087 [TRAIN] [p ]: 1.0000000 |
| xmem_egoexo_tune_s3 - It 3087 [TRAIN] [ce_loss_1 ]: 0.0091798 |
| xmem_egoexo_tune_s3 - It 3087 [TRAIN] [dice_loss_1 ]: 0.3276070 |
| xmem_egoexo_tune_s3 - It 3087 [TRAIN] [ce_loss_2 ]: 0.0112980 |
| xmem_egoexo_tune_s3 - It 3087 [TRAIN] [dice_loss_2 ]: 0.3671102 |
| xmem_egoexo_tune_s3 - It 3087 [TRAIN] [ce_loss_3 ]: 0.0119131 |
| xmem_egoexo_tune_s3 - It 3087 [TRAIN] [dice_loss_3 ]: 0.3800052 |
| xmem_egoexo_tune_s3 - It 3087 [TRAIN] [iou ]: 0.5601372 |
| Current epoch: 9 |
| Current epoch: 9 |
| Current epoch: 9 |
| Current epoch: 9 |
| xmem_egoexo_tune_s3 - It 3473 [TRAIN] [time ]: 17.252347 |
| xmem_egoexo_tune_s3 - It 3473 [TRAIN] [total_loss ]: 1.0498688 |
| xmem_egoexo_tune_s3 - It 3473 [TRAIN] [p ]: 1.0000000 |
| xmem_egoexo_tune_s3 - It 3473 [TRAIN] [ce_loss_1 ]: 0.0088701 |
| xmem_egoexo_tune_s3 - It 3473 [TRAIN] [dice_loss_1 ]: 0.3123125 |
| xmem_egoexo_tune_s3 - It 3473 [TRAIN] [ce_loss_2 ]: 0.0097084 |
| xmem_egoexo_tune_s3 - It 3473 [TRAIN] [dice_loss_2 ]: 0.3419464 |
| xmem_egoexo_tune_s3 - It 3473 [TRAIN] [ce_loss_3 ]: 0.0115497 |
| xmem_egoexo_tune_s3 - It 3473 [TRAIN] [dice_loss_3 ]: 0.3654813 |
| xmem_egoexo_tune_s3 - It 3473 [TRAIN] [iou ]: 0.5789119 |
| xmem_egoexo_tune_s3 - It 3473 [TRAIN] [time ]: 17.252557 |
| xmem_egoexo_tune_s3 - It 3473 [TRAIN] [total_loss ]: 1.0376553 |
| xmem_egoexo_tune_s3 - It 3473 [TRAIN] [p ]: 1.0000000 |
| xmem_egoexo_tune_s3 - It 3473 [TRAIN] [ce_loss_1 ]: 0.0089046 |
| xmem_egoexo_tune_s3 - It 3473 [TRAIN] [dice_loss_1 ]: 0.3102215 |
| xmem_egoexo_tune_s3 - It 3473 [TRAIN] [ce_loss_2 ]: 0.0094815 |
| xmem_egoexo_tune_s3 - It 3473 [TRAIN] [dice_loss_2 ]: 0.3370008 |
| xmem_egoexo_tune_s3 - It 3473 [TRAIN] [ce_loss_3 ]: 0.0110340 |
| xmem_egoexo_tune_s3 - It 3473 [TRAIN] [dice_loss_3 ]: 0.3610128 |
| xmem_egoexo_tune_s3 - It 3473 [TRAIN] [iou ]: 0.5831646 |
| Current epoch: 10 |
| Current epoch: 10 |
| Current epoch: 10 |
| Current epoch: 10 |
| xmem_egoexo_tune_s3 - It 3859 [TRAIN] [time ]: 17.228024 |
| xmem_egoexo_tune_s3 - It 3859 [TRAIN] [time ]: 17.228022 |
| xmem_egoexo_tune_s3 - It 3859 [TRAIN] [total_loss ]: 1.0244812 |
| xmem_egoexo_tune_s3 - It 3859 [TRAIN] [p ]: 1.0000000 |
| xmem_egoexo_tune_s3 - It 3859 [TRAIN] [ce_loss_1 ]: 0.0076633 |
| xmem_egoexo_tune_s3 - It 3859 [TRAIN] [dice_loss_1 ]: 0.2986138 |
| xmem_egoexo_tune_s3 - It 3859 [TRAIN] [ce_loss_2 ]: 0.0150291 |
| xmem_egoexo_tune_s3 - It 3859 [TRAIN] [dice_loss_2 ]: 0.3366727 |
| xmem_egoexo_tune_s3 - It 3859 [TRAIN] [ce_loss_3 ]: 0.0102654 |
| xmem_egoexo_tune_s3 - It 3859 [TRAIN] [dice_loss_3 ]: 0.3562366 |
| xmem_egoexo_tune_s3 - It 3859 [TRAIN] [iou ]: 0.5877245 |
| xmem_egoexo_tune_s3 - It 3859 [TRAIN] [total_loss ]: 1.0189120 |
| xmem_egoexo_tune_s3 - It 3859 [TRAIN] [p ]: 1.0000000 |
| xmem_egoexo_tune_s3 - It 3859 [TRAIN] [ce_loss_1 ]: 0.0078511 |
| xmem_egoexo_tune_s3 - It 3859 [TRAIN] [dice_loss_1 ]: 0.2971227 |
| xmem_egoexo_tune_s3 - It 3859 [TRAIN] [ce_loss_2 ]: 0.0146185 |
| xmem_egoexo_tune_s3 - It 3859 [TRAIN] [dice_loss_2 ]: 0.3348797 |
| xmem_egoexo_tune_s3 - It 3859 [TRAIN] [ce_loss_3 ]: 0.0105526 |
| xmem_egoexo_tune_s3 - It 3859 [TRAIN] [dice_loss_3 ]: 0.3538872 |
| xmem_egoexo_tune_s3 - It 3859 [TRAIN] [iou ]: 0.5899406 |
| Current epoch: 11 |
| Current epoch: 11 |
| Current epoch: 11 |
| Current epoch: 11 |
| Network saved to /data/work-gcp-europe-west4-a/yuqian_fu/Ego/saves_xmem2/Sep10_20.14.00_xmem_egoexo_tune_s3/Sep10_20.14.00_xmem_egoexo_tune_s3_4000.pth. |
| Network saved to /data/work-gcp-europe-west4-a/yuqian_fu/Ego/saves_xmem2/Sep10_20.14.00_xmem_egoexo_tune_s3/Sep10_20.14.00_xmem_egoexo_tune_s3_4000.pth. |
| Checkpoint saved to /data/work-gcp-europe-west4-a/yuqian_fu/Ego/saves_xmem2/Sep10_20.14.00_xmem_egoexo_tune_s3/Sep10_20.14.00_xmem_egoexo_tune_s3_checkpoint_4000.pth. |
| Checkpoint saved to /data/work-gcp-europe-west4-a/yuqian_fu/Ego/saves_xmem2/Sep10_20.14.00_xmem_egoexo_tune_s3/Sep10_20.14.00_xmem_egoexo_tune_s3_checkpoint_4000.pth. |
| xmem_egoexo_tune_s3 - It 4245 [TRAIN] [time ]: 17.444423 |
| xmem_egoexo_tune_s3 - It 4245 [TRAIN] [total_loss ]: 1.0209610 |
| xmem_egoexo_tune_s3 - It 4245 [TRAIN] [p ]: 1.0000000 |
| xmem_egoexo_tune_s3 - It 4245 [TRAIN] [ce_loss_1 ]: 0.0082090 |
| xmem_egoexo_tune_s3 - It 4245 [TRAIN] [dice_loss_1 ]: 0.3054210 |
| xmem_egoexo_tune_s3 - It 4245 [TRAIN] [ce_loss_2 ]: 0.0096977 |
| xmem_egoexo_tune_s3 - It 4245 [TRAIN] [dice_loss_2 ]: 0.3346187 |
| xmem_egoexo_tune_s3 - It 4245 [TRAIN] [ce_loss_3 ]: 0.0104663 |
| xmem_egoexo_tune_s3 - It 4245 [TRAIN] [dice_loss_3 ]: 0.3525480 |
| xmem_egoexo_tune_s3 - It 4245 [TRAIN] [iou ]: 0.5883526 |
| xmem_egoexo_tune_s3 - It 4245 [TRAIN] [time ]: 17.447627 |
| xmem_egoexo_tune_s3 - It 4245 [TRAIN] [total_loss ]: 1.0277469 |
| xmem_egoexo_tune_s3 - It 4245 [TRAIN] [p ]: 1.0000000 |
| xmem_egoexo_tune_s3 - It 4245 [TRAIN] [ce_loss_1 ]: 0.0081935 |
| xmem_egoexo_tune_s3 - It 4245 [TRAIN] [dice_loss_1 ]: 0.3080472 |
| xmem_egoexo_tune_s3 - It 4245 [TRAIN] [ce_loss_2 ]: 0.0098983 |
| xmem_egoexo_tune_s3 - It 4245 [TRAIN] [dice_loss_2 ]: 0.3349435 |
| xmem_egoexo_tune_s3 - It 4245 [TRAIN] [ce_loss_3 ]: 0.0106891 |
| xmem_egoexo_tune_s3 - It 4245 [TRAIN] [dice_loss_3 ]: 0.3559750 |
| xmem_egoexo_tune_s3 - It 4245 [TRAIN] [iou ]: 0.5859799 |
| Current epoch: 12 |
| Current epoch: 12 |
| Current epoch: 12 |
| Current epoch: 12 |
| xmem_egoexo_tune_s3 - It 4631 [TRAIN] [time ]: 17.212758 |
| xmem_egoexo_tune_s3 - It 4631 [TRAIN] [time ]: 17.209659 |
| xmem_egoexo_tune_s3 - It 4631 [TRAIN] [total_loss ]: 1.0198221 |
| xmem_egoexo_tune_s3 - It 4631 [TRAIN] [total_loss ]: 1.0112140 |
| xmem_egoexo_tune_s3 - It 4631 [TRAIN] [p ]: 1.0000000 |
| xmem_egoexo_tune_s3 - It 4631 [TRAIN] [p ]: 1.0000000 |
| xmem_egoexo_tune_s3 - It 4631 [TRAIN] [ce_loss_1 ]: 0.0078050 |
| xmem_egoexo_tune_s3 - It 4631 [TRAIN] [ce_loss_1 ]: 0.0076612 |
| xmem_egoexo_tune_s3 - It 4631 [TRAIN] [dice_loss_1 ]: 0.3041178 |
| xmem_egoexo_tune_s3 - It 4631 [TRAIN] [dice_loss_1 ]: 0.3006271 |
| xmem_egoexo_tune_s3 - It 4631 [TRAIN] [ce_loss_2 ]: 0.0097256 |
| xmem_egoexo_tune_s3 - It 4631 [TRAIN] [ce_loss_2 ]: 0.0095511 |
| xmem_egoexo_tune_s3 - It 4631 [TRAIN] [dice_loss_2 ]: 0.3330118 |
| xmem_egoexo_tune_s3 - It 4631 [TRAIN] [dice_loss_2 ]: 0.3293591 |
| xmem_egoexo_tune_s3 - It 4631 [TRAIN] [ce_loss_3 ]: 0.0103395 |
| xmem_egoexo_tune_s3 - It 4631 [TRAIN] [ce_loss_3 ]: 0.0102189 |
| xmem_egoexo_tune_s3 - It 4631 [TRAIN] [dice_loss_3 ]: 0.3548221 |
| xmem_egoexo_tune_s3 - It 4631 [TRAIN] [dice_loss_3 ]: 0.3537962 |
| xmem_egoexo_tune_s3 - It 4631 [TRAIN] [iou ]: 0.5879908 |
| xmem_egoexo_tune_s3 - It 4631 [TRAIN] [iou ]: 0.5903824 |
| Current epoch: 13 |
| Current epoch: 13 |
| Current epoch: 13 |
| Current epoch: 13 |
| Network saved to /data/work-gcp-europe-west4-a/yuqian_fu/Ego/saves_xmem2/Sep10_20.14.00_xmem_egoexo_tune_s3/Sep10_20.14.00_xmem_egoexo_tune_s3_5000.pth. |
| Network saved to /data/work-gcp-europe-west4-a/yuqian_fu/Ego/saves_xmem2/Sep10_20.14.00_xmem_egoexo_tune_s3/Sep10_20.14.00_xmem_egoexo_tune_s3_5000.pth. |
| xmem_egoexo_tune_s3 - It 5017 [TRAIN] [time ]: 17.236133 |
| xmem_egoexo_tune_s3 - It 5017 [TRAIN] [time ]: 17.236131 |
| xmem_egoexo_tune_s3 - It 5017 [TRAIN] [total_loss ]: 0.9725446 |
| xmem_egoexo_tune_s3 - It 5017 [TRAIN] [total_loss ]: 0.9790159 |
| xmem_egoexo_tune_s3 - It 5017 [TRAIN] [p ]: 1.0000000 |
| xmem_egoexo_tune_s3 - It 5017 [TRAIN] [p ]: 1.0000000 |
| xmem_egoexo_tune_s3 - It 5017 [TRAIN] [ce_loss_1 ]: 0.0078427 |
| xmem_egoexo_tune_s3 - It 5017 [TRAIN] [ce_loss_1 ]: 0.0078013 |
| xmem_egoexo_tune_s3 - It 5017 [TRAIN] [dice_loss_1 ]: 0.2916321 |
| xmem_egoexo_tune_s3 - It 5017 [TRAIN] [dice_loss_1 ]: 0.2934574 |
| xmem_egoexo_tune_s3 - It 5017 [TRAIN] [ce_loss_2 ]: 0.0092198 |
| xmem_egoexo_tune_s3 - It 5017 [TRAIN] [ce_loss_2 ]: 0.0093610 |
| xmem_egoexo_tune_s3 - It 5017 [TRAIN] [dice_loss_2 ]: 0.3205496 |
| xmem_egoexo_tune_s3 - It 5017 [TRAIN] [dice_loss_2 ]: 0.3233856 |
| xmem_egoexo_tune_s3 - It 5017 [TRAIN] [ce_loss_3 ]: 0.0096537 |
| xmem_egoexo_tune_s3 - It 5017 [TRAIN] [ce_loss_3 ]: 0.0097087 |
| xmem_egoexo_tune_s3 - It 5017 [TRAIN] [dice_loss_3 ]: 0.3336465 |
| xmem_egoexo_tune_s3 - It 5017 [TRAIN] [dice_loss_3 ]: 0.3353017 |
| xmem_egoexo_tune_s3 - It 5017 [TRAIN] [iou ]: 0.6030849 |
| xmem_egoexo_tune_s3 - It 5017 [TRAIN] [iou ]: 0.6010079 |
| Current epoch: 14 |
| Current epoch: 14 |
| Current epoch: 14 |
| Current epoch: 14 |
| xmem_egoexo_tune_s3 - It 5403 [TRAIN] [time ]: 17.332608 |
| xmem_egoexo_tune_s3 - It 5403 [TRAIN] [total_loss ]: 0.9919451 |
| xmem_egoexo_tune_s3 - It 5403 [TRAIN] [p ]: 1.0000000 |
| xmem_egoexo_tune_s3 - It 5403 [TRAIN] [ce_loss_1 ]: 0.0075405 |
| xmem_egoexo_tune_s3 - It 5403 [TRAIN] [dice_loss_1 ]: 0.2944352 |
| xmem_egoexo_tune_s3 - It 5403 [TRAIN] [ce_loss_2 ]: 0.0087055 |
| xmem_egoexo_tune_s3 - It 5403 [TRAIN] [dice_loss_2 ]: 0.3211776 |
| xmem_egoexo_tune_s3 - It 5403 [TRAIN] [ce_loss_3 ]: 0.0147405 |
| xmem_egoexo_tune_s3 - It 5403 [TRAIN] [dice_loss_3 ]: 0.3453456 |
| xmem_egoexo_tune_s3 - It 5403 [TRAIN] [iou ]: 0.5987336 |
| xmem_egoexo_tune_s3 - It 5403 [TRAIN] [time ]: 17.333155 |
| xmem_egoexo_tune_s3 - It 5403 [TRAIN] [total_loss ]: 0.9791336 |
| xmem_egoexo_tune_s3 - It 5403 [TRAIN] [p ]: 1.0000000 |
| xmem_egoexo_tune_s3 - It 5403 [TRAIN] [ce_loss_1 ]: 0.0074494 |
| xmem_egoexo_tune_s3 - It 5403 [TRAIN] [dice_loss_1 ]: 0.2902823 |
| xmem_egoexo_tune_s3 - It 5403 [TRAIN] [ce_loss_2 ]: 0.0084842 |
| xmem_egoexo_tune_s3 - It 5403 [TRAIN] [dice_loss_2 ]: 0.3167376 |
| xmem_egoexo_tune_s3 - It 5403 [TRAIN] [ce_loss_3 ]: 0.0152566 |
| xmem_egoexo_tune_s3 - It 5403 [TRAIN] [dice_loss_3 ]: 0.3409231 |
| xmem_egoexo_tune_s3 - It 5403 [TRAIN] [iou ]: 0.6022546 |
| Current epoch: 15 |
| Current epoch: 15 |
| Current epoch: 15 |
| Current epoch: 15 |
| xmem_egoexo_tune_s3 - It 5789 [TRAIN] [time ]: 17.190541 |
| xmem_egoexo_tune_s3 - It 5789 [TRAIN] [time ]: 17.190270 |
| xmem_egoexo_tune_s3 - It 5789 [TRAIN] [total_loss ]: 0.9875407 |
| xmem_egoexo_tune_s3 - It 5789 [TRAIN] [p ]: 1.0000000 |
| xmem_egoexo_tune_s3 - It 5789 [TRAIN] [ce_loss_1 ]: 0.0088650 |
| xmem_egoexo_tune_s3 - It 5789 [TRAIN] [dice_loss_1 ]: 0.2939115 |
| xmem_egoexo_tune_s3 - It 5789 [TRAIN] [ce_loss_2 ]: 0.0100317 |
| xmem_egoexo_tune_s3 - It 5789 [TRAIN] [dice_loss_2 ]: 0.3249244 |
| xmem_egoexo_tune_s3 - It 5789 [TRAIN] [ce_loss_3 ]: 0.0106102 |
| xmem_egoexo_tune_s3 - It 5789 [TRAIN] [dice_loss_3 ]: 0.3391976 |
| xmem_egoexo_tune_s3 - It 5789 [TRAIN] [iou ]: 0.5994688 |
| xmem_egoexo_tune_s3 - It 5789 [TRAIN] [total_loss ]: 0.9745652 |
| xmem_egoexo_tune_s3 - It 5789 [TRAIN] [p ]: 1.0000000 |
| xmem_egoexo_tune_s3 - It 5789 [TRAIN] [ce_loss_1 ]: 0.0087583 |
| xmem_egoexo_tune_s3 - It 5789 [TRAIN] [dice_loss_1 ]: 0.2891781 |
| xmem_egoexo_tune_s3 - It 5789 [TRAIN] [ce_loss_2 ]: 0.0100356 |
| xmem_egoexo_tune_s3 - It 5789 [TRAIN] [dice_loss_2 ]: 0.3222396 |
| xmem_egoexo_tune_s3 - It 5789 [TRAIN] [ce_loss_3 ]: 0.0103378 |
| xmem_egoexo_tune_s3 - It 5789 [TRAIN] [dice_loss_3 ]: 0.3340156 |
| xmem_egoexo_tune_s3 - It 5789 [TRAIN] [iou ]: 0.6039028 |
| Current epoch: 16 |
| Current epoch: 16 |
| Current epoch: 16 |
| Current epoch: 16 |
| Network saved to /data/work-gcp-europe-west4-a/yuqian_fu/Ego/saves_xmem2/Sep10_20.14.00_xmem_egoexo_tune_s3/Sep10_20.14.00_xmem_egoexo_tune_s3_6000.pth. |
| Network saved to /data/work-gcp-europe-west4-a/yuqian_fu/Ego/saves_xmem2/Sep10_20.14.00_xmem_egoexo_tune_s3/Sep10_20.14.00_xmem_egoexo_tune_s3_6000.pth. |
| Checkpoint saved to /data/work-gcp-europe-west4-a/yuqian_fu/Ego/saves_xmem2/Sep10_20.14.00_xmem_egoexo_tune_s3/Sep10_20.14.00_xmem_egoexo_tune_s3_checkpoint_6000.pth. |
| Checkpoint saved to /data/work-gcp-europe-west4-a/yuqian_fu/Ego/saves_xmem2/Sep10_20.14.00_xmem_egoexo_tune_s3/Sep10_20.14.00_xmem_egoexo_tune_s3_checkpoint_6000.pth. |
| xmem_egoexo_tune_s3 - It 6175 [TRAIN] [time ]: 17.369582 |
| xmem_egoexo_tune_s3 - It 6175 [TRAIN] [total_loss ]: 0.9500146 |
| xmem_egoexo_tune_s3 - It 6175 [TRAIN] [p ]: 1.0000000 |
| xmem_egoexo_tune_s3 - It 6175 [TRAIN] [ce_loss_1 ]: 0.0078877 |
| xmem_egoexo_tune_s3 - It 6175 [TRAIN] [dice_loss_1 ]: 0.2873166 |
| xmem_egoexo_tune_s3 - It 6175 [TRAIN] [ce_loss_2 ]: 0.0081164 |
| xmem_egoexo_tune_s3 - It 6175 [TRAIN] [dice_loss_2 ]: 0.3080097 |
| xmem_egoexo_tune_s3 - It 6175 [TRAIN] [ce_loss_3 ]: 0.0094818 |
| xmem_egoexo_tune_s3 - It 6175 [TRAIN] [dice_loss_3 ]: 0.3292023 |
| xmem_egoexo_tune_s3 - It 6175 [TRAIN] [iou ]: 0.6107876 |
| xmem_egoexo_tune_s3 - It 6175 [TRAIN] [time ]: 17.369395 |
| xmem_egoexo_tune_s3 - It 6175 [TRAIN] [total_loss ]: 0.9569437 |
| xmem_egoexo_tune_s3 - It 6175 [TRAIN] [p ]: 1.0000000 |
| xmem_egoexo_tune_s3 - It 6175 [TRAIN] [ce_loss_1 ]: 0.0076051 |
| xmem_egoexo_tune_s3 - It 6175 [TRAIN] [dice_loss_1 ]: 0.2878702 |
| xmem_egoexo_tune_s3 - It 6175 [TRAIN] [ce_loss_2 ]: 0.0081069 |
| xmem_egoexo_tune_s3 - It 6175 [TRAIN] [dice_loss_2 ]: 0.3118155 |
| xmem_egoexo_tune_s3 - It 6175 [TRAIN] [ce_loss_3 ]: 0.0092721 |
| xmem_egoexo_tune_s3 - It 6175 [TRAIN] [dice_loss_3 ]: 0.3322737 |
| xmem_egoexo_tune_s3 - It 6175 [TRAIN] [iou ]: 0.6087510 |
| Current epoch: 17 |
| Current epoch: 17 |
| Current epoch: 17 |
| Current epoch: 17 |
| xmem_egoexo_tune_s3 - It 6561 [TRAIN] [time ]: 17.351912 |
| xmem_egoexo_tune_s3 - It 6561 [TRAIN] [total_loss ]: 0.9523862 |
| xmem_egoexo_tune_s3 - It 6561 [TRAIN] [p ]: 1.0000000 |
| xmem_egoexo_tune_s3 - It 6561 [TRAIN] [ce_loss_1 ]: 0.0076718 |
| xmem_egoexo_tune_s3 - It 6561 [TRAIN] [dice_loss_1 ]: 0.2852446 |
| xmem_egoexo_tune_s3 - It 6561 [TRAIN] [ce_loss_2 ]: 0.0089150 |
| xmem_egoexo_tune_s3 - It 6561 [TRAIN] [dice_loss_2 ]: 0.3128220 |
| xmem_egoexo_tune_s3 - It 6561 [TRAIN] [ce_loss_3 ]: 0.0095505 |
| xmem_egoexo_tune_s3 - It 6561 [TRAIN] [dice_loss_3 ]: 0.3281820 |
| xmem_egoexo_tune_s3 - It 6561 [TRAIN] [iou ]: 0.6118655 |
| xmem_egoexo_tune_s3 - It 6561 [TRAIN] [time ]: 17.352903 |
| xmem_egoexo_tune_s3 - It 6561 [TRAIN] [total_loss ]: 0.9521752 |
| xmem_egoexo_tune_s3 - It 6561 [TRAIN] [p ]: 1.0000000 |
| xmem_egoexo_tune_s3 - It 6561 [TRAIN] [ce_loss_1 ]: 0.0076367 |
| xmem_egoexo_tune_s3 - It 6561 [TRAIN] [dice_loss_1 ]: 0.2870542 |
| xmem_egoexo_tune_s3 - It 6561 [TRAIN] [ce_loss_2 ]: 0.0088502 |
| xmem_egoexo_tune_s3 - It 6561 [TRAIN] [dice_loss_2 ]: 0.3129493 |
| xmem_egoexo_tune_s3 - It 6561 [TRAIN] [ce_loss_3 ]: 0.0092352 |
| xmem_egoexo_tune_s3 - It 6561 [TRAIN] [dice_loss_3 ]: 0.3264493 |
| xmem_egoexo_tune_s3 - It 6561 [TRAIN] [iou ]: 0.6118615 |
| Current epoch: 18 |
| Current epoch: 18 |
| Current epoch: 18 |
| Current epoch: 18 |
| xmem_egoexo_tune_s3 - It 6947 [TRAIN] [time ]: 17.271393 |
| xmem_egoexo_tune_s3 - It 6947 [TRAIN] [total_loss ]: 0.9221727 |
| xmem_egoexo_tune_s3 - It 6947 [TRAIN] [p ]: 1.0000000 |
| xmem_egoexo_tune_s3 - It 6947 [TRAIN] [ce_loss_1 ]: 0.0070969 |
| xmem_egoexo_tune_s3 - It 6947 [TRAIN] [dice_loss_1 ]: 0.2733548 |
| xmem_egoexo_tune_s3 - It 6947 [TRAIN] [ce_loss_2 ]: 0.0082470 |
| xmem_egoexo_tune_s3 - It 6947 [TRAIN] [dice_loss_2 ]: 0.3040221 |
| xmem_egoexo_tune_s3 - It 6947 [TRAIN] [ce_loss_3 ]: 0.0091545 |
| xmem_egoexo_tune_s3 - It 6947 [TRAIN] [dice_loss_3 ]: 0.3202972 |
| xmem_egoexo_tune_s3 - It 6947 [TRAIN] [iou ]: 0.6206735 |
| xmem_egoexo_tune_s3 - It 6947 [TRAIN] [time ]: 17.275092 |
| xmem_egoexo_tune_s3 - It 6947 [TRAIN] [total_loss ]: 0.9187072 |
| xmem_egoexo_tune_s3 - It 6947 [TRAIN] [p ]: 1.0000000 |
| xmem_egoexo_tune_s3 - It 6947 [TRAIN] [ce_loss_1 ]: 0.0070232 |
| xmem_egoexo_tune_s3 - It 6947 [TRAIN] [dice_loss_1 ]: 0.2725345 |
| xmem_egoexo_tune_s3 - It 6947 [TRAIN] [ce_loss_2 ]: 0.0080588 |
| xmem_egoexo_tune_s3 - It 6947 [TRAIN] [dice_loss_2 ]: 0.3028546 |
| xmem_egoexo_tune_s3 - It 6947 [TRAIN] [ce_loss_3 ]: 0.0090213 |
| xmem_egoexo_tune_s3 - It 6947 [TRAIN] [dice_loss_3 ]: 0.3192144 |
| xmem_egoexo_tune_s3 - It 6947 [TRAIN] [iou ]: 0.6212548 |
| Current epoch: 19 |
| Current epoch: 19 |
| Current epoch: 19 |
| Current epoch: 19 |
| Network saved to /data/work-gcp-europe-west4-a/yuqian_fu/Ego/saves_xmem2/Sep10_20.14.00_xmem_egoexo_tune_s3/Sep10_20.14.00_xmem_egoexo_tune_s3_7000.pth. |
| Network saved to /data/work-gcp-europe-west4-a/yuqian_fu/Ego/saves_xmem2/Sep10_20.14.00_xmem_egoexo_tune_s3/Sep10_20.14.00_xmem_egoexo_tune_s3_7000.pth. |
| xmem_egoexo_tune_s3 - It 7333 [TRAIN] [time ]: 17.291757 |
| xmem_egoexo_tune_s3 - It 7333 [TRAIN] [total_loss ]: 0.9398555 |
| xmem_egoexo_tune_s3 - It 7333 [TRAIN] [p ]: 1.0000000 |
| xmem_egoexo_tune_s3 - It 7333 [TRAIN] [ce_loss_1 ]: 0.0073838 |
| xmem_egoexo_tune_s3 - It 7333 [TRAIN] [dice_loss_1 ]: 0.2806870 |
| xmem_egoexo_tune_s3 - It 7333 [TRAIN] [ce_loss_2 ]: 0.0083207 |
| xmem_egoexo_tune_s3 - It 7333 [TRAIN] [dice_loss_2 ]: 0.3062314 |
| xmem_egoexo_tune_s3 - It 7333 [TRAIN] [time ]: 17.295014 |
| xmem_egoexo_tune_s3 - It 7333 [TRAIN] [ce_loss_3 ]: 0.0094011 |
| xmem_egoexo_tune_s3 - It 7333 [TRAIN] [dice_loss_3 ]: 0.3278312 |
| xmem_egoexo_tune_s3 - It 7333 [TRAIN] [iou ]: 0.6150705 |
| xmem_egoexo_tune_s3 - It 7333 [TRAIN] [total_loss ]: 0.9332901 |
| xmem_egoexo_tune_s3 - It 7333 [TRAIN] [p ]: 1.0000000 |
| xmem_egoexo_tune_s3 - It 7333 [TRAIN] [ce_loss_1 ]: 0.0075171 |
| xmem_egoexo_tune_s3 - It 7333 [TRAIN] [dice_loss_1 ]: 0.2785396 |
| xmem_egoexo_tune_s3 - It 7333 [TRAIN] [ce_loss_2 ]: 0.0081513 |
| xmem_egoexo_tune_s3 - It 7333 [TRAIN] [dice_loss_2 ]: 0.3033610 |
| xmem_egoexo_tune_s3 - It 7333 [TRAIN] [ce_loss_3 ]: 0.0095471 |
| xmem_egoexo_tune_s3 - It 7333 [TRAIN] [dice_loss_3 ]: 0.3261737 |
| xmem_egoexo_tune_s3 - It 7333 [TRAIN] [iou ]: 0.6177871 |
| Current epoch: 20Current epoch: 20 |
| |
| Current epoch: 20 |
| Current epoch: 20 |
| xmem_egoexo_tune_s3 - It 7719 [TRAIN] [time ]: 17.338561 |
| xmem_egoexo_tune_s3 - It 7719 [TRAIN] [time ]: 17.338123 |
| xmem_egoexo_tune_s3 - It 7719 [TRAIN] [total_loss ]: 0.9337726 |
| xmem_egoexo_tune_s3 - It 7719 [TRAIN] [p ]: 1.0000000 |
| xmem_egoexo_tune_s3 - It 7719 [TRAIN] [ce_loss_1 ]: 0.0067985 |
| xmem_egoexo_tune_s3 - It 7719 [TRAIN] [dice_loss_1 ]: 0.2782503 |
| xmem_egoexo_tune_s3 - It 7719 [TRAIN] [ce_loss_2 ]: 0.0075532 |
| xmem_egoexo_tune_s3 - It 7719 [TRAIN] [dice_loss_2 ]: 0.3073752 |
| xmem_egoexo_tune_s3 - It 7719 [TRAIN] [ce_loss_3 ]: 0.0086785 |
| xmem_egoexo_tune_s3 - It 7719 [TRAIN] [dice_loss_3 ]: 0.3251167 |
| xmem_egoexo_tune_s3 - It 7719 [TRAIN] [iou ]: 0.6163989 |
| xmem_egoexo_tune_s3 - It 7719 [TRAIN] [total_loss ]: 0.9252947 |
| xmem_egoexo_tune_s3 - It 7719 [TRAIN] [p ]: 1.0000000 |
| xmem_egoexo_tune_s3 - It 7719 [TRAIN] [ce_loss_1 ]: 0.0068075 |
| xmem_egoexo_tune_s3 - It 7719 [TRAIN] [dice_loss_1 ]: 0.2747072 |
| xmem_egoexo_tune_s3 - It 7719 [TRAIN] [ce_loss_2 ]: 0.0075102 |
| xmem_egoexo_tune_s3 - It 7719 [TRAIN] [dice_loss_2 ]: 0.3034705 |
| xmem_egoexo_tune_s3 - It 7719 [TRAIN] [ce_loss_3 ]: 0.0088450 |
| xmem_egoexo_tune_s3 - It 7719 [TRAIN] [dice_loss_3 ]: 0.3239540 |
| xmem_egoexo_tune_s3 - It 7719 [TRAIN] [iou ]: 0.6197115 |
| Current epoch: 21 |
| Current epoch: 21 |
| Current epoch: 21 |
| Current epoch: 21 |
| Network saved to /data/work-gcp-europe-west4-a/yuqian_fu/Ego/saves_xmem2/Sep10_20.14.00_xmem_egoexo_tune_s3/Sep10_20.14.00_xmem_egoexo_tune_s3_8000.pth. |
| Network saved to /data/work-gcp-europe-west4-a/yuqian_fu/Ego/saves_xmem2/Sep10_20.14.00_xmem_egoexo_tune_s3/Sep10_20.14.00_xmem_egoexo_tune_s3_8000.pth. |
| Checkpoint saved to /data/work-gcp-europe-west4-a/yuqian_fu/Ego/saves_xmem2/Sep10_20.14.00_xmem_egoexo_tune_s3/Sep10_20.14.00_xmem_egoexo_tune_s3_checkpoint_8000.pth. |
| Checkpoint saved to /data/work-gcp-europe-west4-a/yuqian_fu/Ego/saves_xmem2/Sep10_20.14.00_xmem_egoexo_tune_s3/Sep10_20.14.00_xmem_egoexo_tune_s3_checkpoint_8000.pth. |
| xmem_egoexo_tune_s3 - It 8105 [TRAIN] [time ]: 17.293388 |
| xmem_egoexo_tune_s3 - It 8105 [TRAIN] [time ]: 17.294382 |
| xmem_egoexo_tune_s3 - It 8105 [TRAIN] [total_loss ]: 0.9025317 |
| xmem_egoexo_tune_s3 - It 8105 [TRAIN] [p ]: 1.0000000 |
| xmem_egoexo_tune_s3 - It 8105 [TRAIN] [ce_loss_1 ]: 0.0072689 |
| xmem_egoexo_tune_s3 - It 8105 [TRAIN] [dice_loss_1 ]: 0.2667175 |
| xmem_egoexo_tune_s3 - It 8105 [TRAIN] [ce_loss_2 ]: 0.0079754 |
| xmem_egoexo_tune_s3 - It 8105 [TRAIN] [dice_loss_2 ]: 0.2949270 |
| xmem_egoexo_tune_s3 - It 8105 [TRAIN] [ce_loss_3 ]: 0.0091379 |
| xmem_egoexo_tune_s3 - It 8105 [TRAIN] [dice_loss_3 ]: 0.3165048 |
| xmem_egoexo_tune_s3 - It 8105 [TRAIN] [iou ]: 0.6277425 |
| xmem_egoexo_tune_s3 - It 8105 [TRAIN] [total_loss ]: 0.8911731 |
| xmem_egoexo_tune_s3 - It 8105 [TRAIN] [p ]: 1.0000000 |
| xmem_egoexo_tune_s3 - It 8105 [TRAIN] [ce_loss_1 ]: 0.0070024 |
| xmem_egoexo_tune_s3 - It 8105 [TRAIN] [dice_loss_1 ]: 0.2632348 |
| xmem_egoexo_tune_s3 - It 8105 [TRAIN] [ce_loss_2 ]: 0.0076277 |
| xmem_egoexo_tune_s3 - It 8105 [TRAIN] [dice_loss_2 ]: 0.2903712 |
| xmem_egoexo_tune_s3 - It 8105 [TRAIN] [ce_loss_3 ]: 0.0088465 |
| xmem_egoexo_tune_s3 - It 8105 [TRAIN] [dice_loss_3 ]: 0.3140904 |
| xmem_egoexo_tune_s3 - It 8105 [TRAIN] [iou ]: 0.6315896 |
| Current epoch: 22 |
| Current epoch: 22 |
| Current epoch: 22 |
| Current epoch: 22 |
| xmem_egoexo_tune_s3 - It 8491 [TRAIN] [time ]: 17.366709 |
| xmem_egoexo_tune_s3 - It 8491 [TRAIN] [time ]: 17.365849 |
| xmem_egoexo_tune_s3 - It 8491 [TRAIN] [total_loss ]: 0.8835418 |
| xmem_egoexo_tune_s3 - It 8491 [TRAIN] [p ]: 1.0000000 |
| xmem_egoexo_tune_s3 - It 8491 [TRAIN] [ce_loss_1 ]: 0.0068425 |
| xmem_egoexo_tune_s3 - It 8491 [TRAIN] [dice_loss_1 ]: 0.2662362 |
| xmem_egoexo_tune_s3 - It 8491 [TRAIN] [ce_loss_2 ]: 0.0077776 |
| xmem_egoexo_tune_s3 - It 8491 [TRAIN] [dice_loss_2 ]: 0.2858447 |
| xmem_egoexo_tune_s3 - It 8491 [TRAIN] [ce_loss_3 ]: 0.0085905 |
| xmem_egoexo_tune_s3 - It 8491 [TRAIN] [dice_loss_3 ]: 0.3082501 |
| xmem_egoexo_tune_s3 - It 8491 [TRAIN] [iou ]: 0.6323812 |
| xmem_egoexo_tune_s3 - It 8491 [TRAIN] [total_loss ]: 0.8946050 |
| xmem_egoexo_tune_s3 - It 8491 [TRAIN] [p ]: 1.0000000 |
| xmem_egoexo_tune_s3 - It 8491 [TRAIN] [ce_loss_1 ]: 0.0070856 |
| xmem_egoexo_tune_s3 - It 8491 [TRAIN] [dice_loss_1 ]: 0.2696720 |
| xmem_egoexo_tune_s3 - It 8491 [TRAIN] [ce_loss_2 ]: 0.0081599 |
| xmem_egoexo_tune_s3 - It 8491 [TRAIN] [dice_loss_2 ]: 0.2905781 |
| xmem_egoexo_tune_s3 - It 8491 [TRAIN] [ce_loss_3 ]: 0.0089441 |
| xmem_egoexo_tune_s3 - It 8491 [TRAIN] [dice_loss_3 ]: 0.3101651 |
| xmem_egoexo_tune_s3 - It 8491 [TRAIN] [iou ]: 0.6298400 |
| Current epoch: 23 |
| Current epoch: 23 |
| Current epoch: 23 |
| Current epoch: 23 |
| xmem_egoexo_tune_s3 - It 8877 [TRAIN] [time ]: 17.288112 |
| xmem_egoexo_tune_s3 - It 8877 [TRAIN] [time ]: 17.288003 |
| xmem_egoexo_tune_s3 - It 8877 [TRAIN] [total_loss ]: 0.9026874 |
| xmem_egoexo_tune_s3 - It 8877 [TRAIN] [total_loss ]: 0.9120978 |
| xmem_egoexo_tune_s3 - It 8877 [TRAIN] [p ]: 1.0000000 |
| xmem_egoexo_tune_s3 - It 8877 [TRAIN] [p ]: 1.0000000 |
| xmem_egoexo_tune_s3 - It 8877 [TRAIN] [ce_loss_1 ]: 0.0069208 |
| xmem_egoexo_tune_s3 - It 8877 [TRAIN] [ce_loss_1 ]: 0.0068392 |
| xmem_egoexo_tune_s3 - It 8877 [TRAIN] [dice_loss_1 ]: 0.2748566 |
| xmem_egoexo_tune_s3 - It 8877 [TRAIN] [dice_loss_1 ]: 0.2763745 |
| xmem_egoexo_tune_s3 - It 8877 [TRAIN] [ce_loss_2 ]: 0.0075416 |
| xmem_egoexo_tune_s3 - It 8877 [TRAIN] [ce_loss_2 ]: 0.0078342 |
| xmem_egoexo_tune_s3 - It 8877 [TRAIN] [dice_loss_2 ]: 0.2907021 |
| xmem_egoexo_tune_s3 - It 8877 [TRAIN] [dice_loss_2 ]: 0.2957014 |
| xmem_egoexo_tune_s3 - It 8877 [TRAIN] [ce_loss_3 ]: 0.0084969 |
| xmem_egoexo_tune_s3 - It 8877 [TRAIN] [ce_loss_3 ]: 0.0089048 |
| xmem_egoexo_tune_s3 - It 8877 [TRAIN] [dice_loss_3 ]: 0.3141690 |
| xmem_egoexo_tune_s3 - It 8877 [TRAIN] [dice_loss_3 ]: 0.3164434 |
| xmem_egoexo_tune_s3 - It 8877 [TRAIN] [iou ]: 0.6271451 |
| xmem_egoexo_tune_s3 - It 8877 [TRAIN] [iou ]: 0.6230235 |
| Current epoch: 24 |
| Current epoch: 24 |
| Current epoch: 24 |
| Current epoch: 24 |
| Network saved to /data/work-gcp-europe-west4-a/yuqian_fu/Ego/saves_xmem2/Sep10_20.14.00_xmem_egoexo_tune_s3/Sep10_20.14.00_xmem_egoexo_tune_s3_9000.pth. |
| Network saved to /data/work-gcp-europe-west4-a/yuqian_fu/Ego/saves_xmem2/Sep10_20.14.00_xmem_egoexo_tune_s3/Sep10_20.14.00_xmem_egoexo_tune_s3_9000.pth. |
| xmem_egoexo_tune_s3 - It 9263 [TRAIN] [time ]: 17.375791 |
| xmem_egoexo_tune_s3 - It 9263 [TRAIN] [total_loss ]: 0.8893340 |
| xmem_egoexo_tune_s3 - It 9263 [TRAIN] [p ]: 1.0000000 |
| xmem_egoexo_tune_s3 - It 9263 [TRAIN] [ce_loss_1 ]: 0.0064838 |
| xmem_egoexo_tune_s3 - It 9263 [TRAIN] [dice_loss_1 ]: 0.2624540 |
| xmem_egoexo_tune_s3 - It 9263 [TRAIN] [ce_loss_2 ]: 0.0077439 |
| xmem_egoexo_tune_s3 - It 9263 [TRAIN] [dice_loss_2 ]: 0.2887411 |
| xmem_egoexo_tune_s3 - It 9263 [TRAIN] [ce_loss_3 ]: 0.0090447 |
| xmem_egoexo_tune_s3 - It 9263 [TRAIN] [dice_loss_3 ]: 0.3148663 |
| xmem_egoexo_tune_s3 - It 9263 [TRAIN] [iou ]: 0.6324592 |
| xmem_egoexo_tune_s3 - It 9263 [TRAIN] [time ]: 17.375872 |
| xmem_egoexo_tune_s3 - It 9263 [TRAIN] [total_loss ]: 0.8806600 |
| xmem_egoexo_tune_s3 - It 9263 [TRAIN] [p ]: 1.0000000 |
| xmem_egoexo_tune_s3 - It 9263 [TRAIN] [ce_loss_1 ]: 0.0065300 |
| xmem_egoexo_tune_s3 - It 9263 [TRAIN] [dice_loss_1 ]: 0.2602019 |
| xmem_egoexo_tune_s3 - It 9263 [TRAIN] [ce_loss_2 ]: 0.0075648 |
| xmem_egoexo_tune_s3 - It 9263 [TRAIN] [dice_loss_2 ]: 0.2865942 |
| xmem_egoexo_tune_s3 - It 9263 [TRAIN] [ce_loss_3 ]: 0.0087087 |
| xmem_egoexo_tune_s3 - It 9263 [TRAIN] [dice_loss_3 ]: 0.3110603 |
| xmem_egoexo_tune_s3 - It 9263 [TRAIN] [iou ]: 0.6349420 |
| Current epoch: 25 |
| Current epoch: 25 |
| Current epoch: 25 |
| Current epoch: 25 |
| xmem_egoexo_tune_s3 - It 9649 [TRAIN] [time ]: 17.327534 |
| xmem_egoexo_tune_s3 - It 9649 [TRAIN] [time ]: 17.329286 |
| xmem_egoexo_tune_s3 - It 9649 [TRAIN] [total_loss ]: 0.9022024 |
| xmem_egoexo_tune_s3 - It 9649 [TRAIN] [p ]: 1.0000000 |
| xmem_egoexo_tune_s3 - It 9649 [TRAIN] [ce_loss_1 ]: 0.0126726 |
| xmem_egoexo_tune_s3 - It 9649 [TRAIN] [dice_loss_1 ]: 0.2675598 |
| xmem_egoexo_tune_s3 - It 9649 [TRAIN] [ce_loss_2 ]: 0.0085049 |
| xmem_egoexo_tune_s3 - It 9649 [TRAIN] [dice_loss_2 ]: 0.2902843 |
| xmem_egoexo_tune_s3 - It 9649 [TRAIN] [ce_loss_3 ]: 0.0096246 |
| xmem_egoexo_tune_s3 - It 9649 [TRAIN] [dice_loss_3 ]: 0.3135561 |
| xmem_egoexo_tune_s3 - It 9649 [TRAIN] [iou ]: 0.6308362 |
| xmem_egoexo_tune_s3 - It 9649 [TRAIN] [total_loss ]: 0.9116822 |
| xmem_egoexo_tune_s3 - It 9649 [TRAIN] [p ]: 1.0000000 |
| xmem_egoexo_tune_s3 - It 9649 [TRAIN] [ce_loss_1 ]: 0.0141976 |
| xmem_egoexo_tune_s3 - It 9649 [TRAIN] [dice_loss_1 ]: 0.2684851 |
| xmem_egoexo_tune_s3 - It 9649 [TRAIN] [ce_loss_2 ]: 0.0089191 |
| xmem_egoexo_tune_s3 - It 9649 [TRAIN] [dice_loss_2 ]: 0.2945180 |
| xmem_egoexo_tune_s3 - It 9649 [TRAIN] [ce_loss_3 ]: 0.0098886 |
| xmem_egoexo_tune_s3 - It 9649 [TRAIN] [dice_loss_3 ]: 0.3156735 |
| xmem_egoexo_tune_s3 - It 9649 [TRAIN] [iou ]: 0.6276664 |
| Current epoch: 26 |
| Current epoch: 26 |
| Current epoch: 26 |
| Current epoch: 26 |
| Changing skip to cur_skip=10 |
| Changing skip to cur_skip=10 |
| Changing skip to cur_skip=10Changing skip to cur_skip=10 |
| |
| 3090 out of 755 videos accepted in /data/work-gcp-europe-west4-a/yuqian_fu/Ego/data_segswap. |
| 3090 out of 755 videos accepted in /data/work-gcp-europe-west4-a/yuqian_fu/Ego/data_segswap. |
| EgoExo dataset size: 3090 |
| Renewed with max_skip=10 |
| EgoExo dataset size: 3090 |
| Renewed with max_skip=10 |
| 3090 out of 755 videos accepted in /data/work-gcp-europe-west4-a/yuqian_fu/Ego/data_segswap. |
| 3090 out of 755 videos accepted in /data/work-gcp-europe-west4-a/yuqian_fu/Ego/data_segswap. |
| EgoExo dataset size: 3090 |
| Renewed with max_skip=10 |
| EgoExo dataset size: 3090 |
| Renewed with max_skip=10 |
| Current epoch: 27 |
| Current epoch: 27 |
| Current epoch: 27 |
| Current epoch: 27 |
| Network saved to /data/work-gcp-europe-west4-a/yuqian_fu/Ego/saves_xmem2/Sep10_20.14.00_xmem_egoexo_tune_s3/Sep10_20.14.00_xmem_egoexo_tune_s3_10000.pth. |
| Network saved to /data/work-gcp-europe-west4-a/yuqian_fu/Ego/saves_xmem2/Sep10_20.14.00_xmem_egoexo_tune_s3/Sep10_20.14.00_xmem_egoexo_tune_s3_10000.pth. |
| Checkpoint saved to /data/work-gcp-europe-west4-a/yuqian_fu/Ego/saves_xmem2/Sep10_20.14.00_xmem_egoexo_tune_s3/Sep10_20.14.00_xmem_egoexo_tune_s3_checkpoint_10000.pth. |
| Checkpoint saved to /data/work-gcp-europe-west4-a/yuqian_fu/Ego/saves_xmem2/Sep10_20.14.00_xmem_egoexo_tune_s3/Sep10_20.14.00_xmem_egoexo_tune_s3_checkpoint_10000.pth. |
| xmem_egoexo_tune_s3 - It 10385 [TRAIN] [time ]: 37.606128 |
| xmem_egoexo_tune_s3 - It 10385 [TRAIN] [time ]: 37.607799 |
| xmem_egoexo_tune_s3 - It 10385 [TRAIN] [total_loss ]: 0.8968966 |
| xmem_egoexo_tune_s3 - It 10385 [TRAIN] [total_loss ]: 0.9004717 |
| xmem_egoexo_tune_s3 - It 10385 [TRAIN] [p ]: 1.0000000 |
| xmem_egoexo_tune_s3 - It 10385 [TRAIN] [p ]: 1.0000000 |
| xmem_egoexo_tune_s3 - It 10385 [TRAIN] [ce_loss_1 ]: 0.0075924 |
| xmem_egoexo_tune_s3 - It 10385 [TRAIN] [ce_loss_1 ]: 0.0073729 |
| xmem_egoexo_tune_s3 - It 10385 [TRAIN] [dice_loss_1 ]: 0.2688172 |
| xmem_egoexo_tune_s3 - It 10385 [TRAIN] [dice_loss_1 ]: 0.2685364 |
| xmem_egoexo_tune_s3 - It 10385 [TRAIN] [ce_loss_2 ]: 0.0082116 |
| xmem_egoexo_tune_s3 - It 10385 [TRAIN] [ce_loss_2 ]: 0.0081231 |
| xmem_egoexo_tune_s3 - It 10385 [TRAIN] [dice_loss_2 ]: 0.2930835 |
| xmem_egoexo_tune_s3 - It 10385 [TRAIN] [dice_loss_2 ]: 0.2938465 |
| xmem_egoexo_tune_s3 - It 10385 [TRAIN] [ce_loss_3 ]: 0.0096118 |
| xmem_egoexo_tune_s3 - It 10385 [TRAIN] [ce_loss_3 ]: 0.0095025 |
| xmem_egoexo_tune_s3 - It 10385 [TRAIN] [dice_loss_3 ]: 0.3095797 |
| xmem_egoexo_tune_s3 - It 10385 [TRAIN] [dice_loss_3 ]: 0.3130901 |
| xmem_egoexo_tune_s3 - It 10385 [TRAIN] [iou ]: 0.6301156 |
| xmem_egoexo_tune_s3 - It 10385 [TRAIN] [iou ]: 0.6286453 |
| Current epoch: 28 |
| Current epoch: 28 |
| Current epoch: 28 |
| Current epoch: 28 |
| xmem_egoexo_tune_s3 - It 10771 [TRAIN] [time ]: 17.332059 |
| xmem_egoexo_tune_s3 - It 10771 [TRAIN] [time ]: 17.333113 |
| xmem_egoexo_tune_s3 - It 10771 [TRAIN] [total_loss ]: 0.8909333 |
| xmem_egoexo_tune_s3 - It 10771 [TRAIN] [p ]: 1.0000000 |
| xmem_egoexo_tune_s3 - It 10771 [TRAIN] [ce_loss_1 ]: 0.0066483 |
| xmem_egoexo_tune_s3 - It 10771 [TRAIN] [dice_loss_1 ]: 0.2692261 |
| xmem_egoexo_tune_s3 - It 10771 [TRAIN] [total_loss ]: 0.8855675 |
| xmem_egoexo_tune_s3 - It 10771 [TRAIN] [ce_loss_2 ]: 0.0079389 |
| xmem_egoexo_tune_s3 - It 10771 [TRAIN] [p ]: 1.0000000 |
| xmem_egoexo_tune_s3 - It 10771 [TRAIN] [dice_loss_2 ]: 0.2860629 |
| xmem_egoexo_tune_s3 - It 10771 [TRAIN] [ce_loss_1 ]: 0.0062845 |
| xmem_egoexo_tune_s3 - It 10771 [TRAIN] [ce_loss_3 ]: 0.0088396 |
| xmem_egoexo_tune_s3 - It 10771 [TRAIN] [dice_loss_1 ]: 0.2669663 |
| xmem_egoexo_tune_s3 - It 10771 [TRAIN] [dice_loss_3 ]: 0.3122173 |
| xmem_egoexo_tune_s3 - It 10771 [TRAIN] [ce_loss_2 ]: 0.0076876 |
| xmem_egoexo_tune_s3 - It 10771 [TRAIN] [iou ]: 0.6317957 |
| xmem_egoexo_tune_s3 - It 10771 [TRAIN] [dice_loss_2 ]: 0.2850447 |
| xmem_egoexo_tune_s3 - It 10771 [TRAIN] [ce_loss_3 ]: 0.0085677 |
| xmem_egoexo_tune_s3 - It 10771 [TRAIN] [dice_loss_3 ]: 0.3110165 |
| xmem_egoexo_tune_s3 - It 10771 [TRAIN] [iou ]: 0.6333277 |
| Current epoch: 29 |
| Current epoch: 29 |
| Current epoch: 29 |
| Current epoch: 29 |
| Network saved to /data/work-gcp-europe-west4-a/yuqian_fu/Ego/saves_xmem2/Sep10_20.14.00_xmem_egoexo_tune_s3/Sep10_20.14.00_xmem_egoexo_tune_s3_11000.pth. |
| Network saved to /data/work-gcp-europe-west4-a/yuqian_fu/Ego/saves_xmem2/Sep10_20.14.00_xmem_egoexo_tune_s3/Sep10_20.14.00_xmem_egoexo_tune_s3_11000.pth. |
| xmem_egoexo_tune_s3 - It 11157 [TRAIN] [time ]: 17.338269 |
| xmem_egoexo_tune_s3 - It 11157 [TRAIN] [total_loss ]: 0.8926560 |
| xmem_egoexo_tune_s3 - It 11157 [TRAIN] [p ]: 1.0000000 |
| xmem_egoexo_tune_s3 - It 11157 [TRAIN] [ce_loss_1 ]: 0.0070202 |
| xmem_egoexo_tune_s3 - It 11157 [TRAIN] [dice_loss_1 ]: 0.2672836 |
| xmem_egoexo_tune_s3 - It 11157 [TRAIN] [ce_loss_2 ]: 0.0075956 |
| xmem_egoexo_tune_s3 - It 11157 [TRAIN] [dice_loss_2 ]: 0.2908360 |
| xmem_egoexo_tune_s3 - It 11157 [TRAIN] [ce_loss_3 ]: 0.0085418 |
| xmem_egoexo_tune_s3 - It 11157 [TRAIN] [dice_loss_3 ]: 0.3113786 |
| xmem_egoexo_tune_s3 - It 11157 [TRAIN] [iou ]: 0.6315774 |
| xmem_egoexo_tune_s3 - It 11157 [TRAIN] [time ]: 17.338725 |
| xmem_egoexo_tune_s3 - It 11157 [TRAIN] [total_loss ]: 0.9025907 |
| xmem_egoexo_tune_s3 - It 11157 [TRAIN] [p ]: 1.0000000 |
| xmem_egoexo_tune_s3 - It 11157 [TRAIN] [ce_loss_1 ]: 0.0071095 |
| xmem_egoexo_tune_s3 - It 11157 [TRAIN] [dice_loss_1 ]: 0.2691220 |
| xmem_egoexo_tune_s3 - It 11157 [TRAIN] [ce_loss_2 ]: 0.0077298 |
| xmem_egoexo_tune_s3 - It 11157 [TRAIN] [dice_loss_2 ]: 0.2932221 |
| xmem_egoexo_tune_s3 - It 11157 [TRAIN] [ce_loss_3 ]: 0.0088132 |
| xmem_egoexo_tune_s3 - It 11157 [TRAIN] [dice_loss_3 ]: 0.3165939 |
| xmem_egoexo_tune_s3 - It 11157 [TRAIN] [iou ]: 0.6280782 |
| Current epoch: 30 |
| Current epoch: 30 |
| Current epoch: 30 |
| Current epoch: 30 |
| xmem_egoexo_tune_s3 - It 11543 [TRAIN] [time ]: 17.262498 |
| xmem_egoexo_tune_s3 - It 11543 [TRAIN] [total_loss ]: 0.8832973 |
| xmem_egoexo_tune_s3 - It 11543 [TRAIN] [p ]: 1.0000000 |
| xmem_egoexo_tune_s3 - It 11543 [TRAIN] [ce_loss_1 ]: 0.0062756 |
| xmem_egoexo_tune_s3 - It 11543 [TRAIN] [time ]: 17.264037 |
| xmem_egoexo_tune_s3 - It 11543 [TRAIN] [dice_loss_1 ]: 0.2669585 |
| xmem_egoexo_tune_s3 - It 11543 [TRAIN] [total_loss ]: 0.8847644 |
| xmem_egoexo_tune_s3 - It 11543 [TRAIN] [ce_loss_2 ]: 0.0074671 |
| xmem_egoexo_tune_s3 - It 11543 [TRAIN] [p ]: 1.0000000 |
| xmem_egoexo_tune_s3 - It 11543 [TRAIN] [dice_loss_2 ]: 0.2860944 |
| xmem_egoexo_tune_s3 - It 11543 [TRAIN] [ce_loss_1 ]: 0.0064912 |
| xmem_egoexo_tune_s3 - It 11543 [TRAIN] [ce_loss_3 ]: 0.0082449 |
| xmem_egoexo_tune_s3 - It 11543 [TRAIN] [dice_loss_1 ]: 0.2660557 |
| xmem_egoexo_tune_s3 - It 11543 [TRAIN] [dice_loss_3 ]: 0.3082567 |
| xmem_egoexo_tune_s3 - It 11543 [TRAIN] [ce_loss_2 ]: 0.0075159 |
| xmem_egoexo_tune_s3 - It 11543 [TRAIN] [iou ]: 0.6337507 |
| xmem_egoexo_tune_s3 - It 11543 [TRAIN] [dice_loss_2 ]: 0.2877334 |
| xmem_egoexo_tune_s3 - It 11543 [TRAIN] [ce_loss_3 ]: 0.0084380 |
| xmem_egoexo_tune_s3 - It 11543 [TRAIN] [dice_loss_3 ]: 0.3085299 |
| xmem_egoexo_tune_s3 - It 11543 [TRAIN] [iou ]: 0.6330494 |
| Current epoch: 31 |
| Current epoch: 31 |
| Current epoch: 31 |
| Current epoch: 31 |
| xmem_egoexo_tune_s3 - It 11929 [TRAIN] [time ]: 17.410457 |
| xmem_egoexo_tune_s3 - It 11929 [TRAIN] [time ]: 17.411159 |
| xmem_egoexo_tune_s3 - It 11929 [TRAIN] [total_loss ]: 0.8920884 |
| xmem_egoexo_tune_s3 - It 11929 [TRAIN] [p ]: 1.0000000 |
| xmem_egoexo_tune_s3 - It 11929 [TRAIN] [ce_loss_1 ]: 0.0069544 |
| xmem_egoexo_tune_s3 - It 11929 [TRAIN] [dice_loss_1 ]: 0.2640838 |
| xmem_egoexo_tune_s3 - It 11929 [TRAIN] [ce_loss_2 ]: 0.0080713 |
| xmem_egoexo_tune_s3 - It 11929 [TRAIN] [dice_loss_2 ]: 0.2870348 |
| xmem_egoexo_tune_s3 - It 11929 [TRAIN] [ce_loss_3 ]: 0.0089972 |
| xmem_egoexo_tune_s3 - It 11929 [TRAIN] [dice_loss_3 ]: 0.3169466 |
| xmem_egoexo_tune_s3 - It 11929 [TRAIN] [iou ]: 0.6314011 |
| xmem_egoexo_tune_s3 - It 11929 [TRAIN] [total_loss ]: 0.9182960 |
| xmem_egoexo_tune_s3 - It 11929 [TRAIN] [p ]: 1.0000000 |
| xmem_egoexo_tune_s3 - It 11929 [TRAIN] [ce_loss_1 ]: 0.0077385 |
| xmem_egoexo_tune_s3 - It 11929 [TRAIN] [dice_loss_1 ]: 0.2709398 |
| xmem_egoexo_tune_s3 - It 11929 [TRAIN] [ce_loss_2 ]: 0.0086057 |
| xmem_egoexo_tune_s3 - It 11929 [TRAIN] [dice_loss_2 ]: 0.2952569 |
| xmem_egoexo_tune_s3 - It 11929 [TRAIN] [ce_loss_3 ]: 0.0094601 |
| xmem_egoexo_tune_s3 - It 11929 [TRAIN] [dice_loss_3 ]: 0.3262949 |
| xmem_egoexo_tune_s3 - It 11929 [TRAIN] [iou ]: 0.6229287 |
| Current epoch: 32 |
| Current epoch: 32 |
| Current epoch: 32 |
| Current epoch: 32 |
| Network saved to /data/work-gcp-europe-west4-a/yuqian_fu/Ego/saves_xmem2/Sep10_20.14.00_xmem_egoexo_tune_s3/Sep10_20.14.00_xmem_egoexo_tune_s3_12000.pth. |
| Network saved to /data/work-gcp-europe-west4-a/yuqian_fu/Ego/saves_xmem2/Sep10_20.14.00_xmem_egoexo_tune_s3/Sep10_20.14.00_xmem_egoexo_tune_s3_12000.pth. |
| Checkpoint saved to /data/work-gcp-europe-west4-a/yuqian_fu/Ego/saves_xmem2/Sep10_20.14.00_xmem_egoexo_tune_s3/Sep10_20.14.00_xmem_egoexo_tune_s3_checkpoint_12000.pth. |
| Checkpoint saved to /data/work-gcp-europe-west4-a/yuqian_fu/Ego/saves_xmem2/Sep10_20.14.00_xmem_egoexo_tune_s3/Sep10_20.14.00_xmem_egoexo_tune_s3_checkpoint_12000.pth. |
| xmem_egoexo_tune_s3 - It 12315 [TRAIN] [time ]: 17.551562 |
| xmem_egoexo_tune_s3 - It 12315 [TRAIN] [total_loss ]: 0.8985861 |
| xmem_egoexo_tune_s3 - It 12315 [TRAIN] [p ]: 1.0000000 |
| xmem_egoexo_tune_s3 - It 12315 [TRAIN] [ce_loss_1 ]: 0.0127216 |
| xmem_egoexo_tune_s3 - It 12315 [TRAIN] [dice_loss_1 ]: 0.2670189 |
| xmem_egoexo_tune_s3 - It 12315 [TRAIN] [ce_loss_2 ]: 0.0078074 |
| xmem_egoexo_tune_s3 - It 12315 [TRAIN] [dice_loss_2 ]: 0.2907140 |
| xmem_egoexo_tune_s3 - It 12315 [TRAIN] [ce_loss_3 ]: 0.0092428 |
| xmem_egoexo_tune_s3 - It 12315 [TRAIN] [dice_loss_3 ]: 0.3110812 |
| xmem_egoexo_tune_s3 - It 12315 [TRAIN] [iou ]: 0.6314867 |
| xmem_egoexo_tune_s3 - It 12315 [TRAIN] [time ]: 17.554930 |
| xmem_egoexo_tune_s3 - It 12315 [TRAIN] [total_loss ]: 0.8912219 |
| xmem_egoexo_tune_s3 - It 12315 [TRAIN] [p ]: 1.0000000 |
| xmem_egoexo_tune_s3 - It 12315 [TRAIN] [ce_loss_1 ]: 0.0126399 |
| xmem_egoexo_tune_s3 - It 12315 [TRAIN] [dice_loss_1 ]: 0.2651752 |
| xmem_egoexo_tune_s3 - It 12315 [TRAIN] [ce_loss_2 ]: 0.0077405 |
| xmem_egoexo_tune_s3 - It 12315 [TRAIN] [dice_loss_2 ]: 0.2871891 |
| xmem_egoexo_tune_s3 - It 12315 [TRAIN] [ce_loss_3 ]: 0.0088884 |
| xmem_egoexo_tune_s3 - It 12315 [TRAIN] [dice_loss_3 ]: 0.3095886 |
| xmem_egoexo_tune_s3 - It 12315 [TRAIN] [iou ]: 0.6338956 |
| Current epoch: 33 |
| Current epoch: 33 |
| Current epoch: 33 |
| Current epoch: 33 |
| xmem_egoexo_tune_s3 - It 12701 [TRAIN] [time ]: 17.428569 |
| xmem_egoexo_tune_s3 - It 12701 [TRAIN] [total_loss ]: 0.8843774 |
| xmem_egoexo_tune_s3 - It 12701 [TRAIN] [p ]: 1.0000000 |
| xmem_egoexo_tune_s3 - It 12701 [TRAIN] [ce_loss_1 ]: 0.0069388 |
| xmem_egoexo_tune_s3 - It 12701 [TRAIN] [dice_loss_1 ]: 0.2623208 |
| xmem_egoexo_tune_s3 - It 12701 [TRAIN] [ce_loss_2 ]: 0.0075816 |
| xmem_egoexo_tune_s3 - It 12701 [TRAIN] [dice_loss_2 ]: 0.2845015 |
| xmem_egoexo_tune_s3 - It 12701 [TRAIN] [ce_loss_3 ]: 0.0089643 |
| xmem_egoexo_tune_s3 - It 12701 [TRAIN] [dice_loss_3 ]: 0.3140702 |
| xmem_egoexo_tune_s3 - It 12701 [TRAIN] [iou ]: 0.6333131 |
| xmem_egoexo_tune_s3 - It 12701 [TRAIN] [time ]: 17.431356 |
| xmem_egoexo_tune_s3 - It 12701 [TRAIN] [total_loss ]: 0.9122688 |
| xmem_egoexo_tune_s3 - It 12701 [TRAIN] [p ]: 1.0000000 |
| xmem_egoexo_tune_s3 - It 12701 [TRAIN] [ce_loss_1 ]: 0.0073122 |
| xmem_egoexo_tune_s3 - It 12701 [TRAIN] [dice_loss_1 ]: 0.2708262 |
| xmem_egoexo_tune_s3 - It 12701 [TRAIN] [ce_loss_2 ]: 0.0080022 |
| xmem_egoexo_tune_s3 - It 12701 [TRAIN] [dice_loss_2 ]: 0.2907660 |
| xmem_egoexo_tune_s3 - It 12701 [TRAIN] [ce_loss_3 ]: 0.0094269 |
| xmem_egoexo_tune_s3 - It 12701 [TRAIN] [dice_loss_3 ]: 0.3259351 |
| xmem_egoexo_tune_s3 - It 12701 [TRAIN] [iou ]: 0.6240681 |
| Current epoch: 34 |
| Current epoch: 34 |
| Current epoch: 34 |
| Current epoch: 34 |
| Network saved to /data/work-gcp-europe-west4-a/yuqian_fu/Ego/saves_xmem2/Sep10_20.14.00_xmem_egoexo_tune_s3/Sep10_20.14.00_xmem_egoexo_tune_s3_13000.pth. |
| Network saved to /data/work-gcp-europe-west4-a/yuqian_fu/Ego/saves_xmem2/Sep10_20.14.00_xmem_egoexo_tune_s3/Sep10_20.14.00_xmem_egoexo_tune_s3_13000.pth. |
| xmem_egoexo_tune_s3 - It 13087 [TRAIN] [time ]: 17.381110 |
| xmem_egoexo_tune_s3 - It 13087 [TRAIN] [time ]: 17.381239 |
| xmem_egoexo_tune_s3 - It 13087 [TRAIN] [total_loss ]: 0.8734003 |
| xmem_egoexo_tune_s3 - It 13087 [TRAIN] [p ]: 1.0000000 |
| xmem_egoexo_tune_s3 - It 13087 [TRAIN] [ce_loss_1 ]: 0.0068424 |
| xmem_egoexo_tune_s3 - It 13087 [TRAIN] [dice_loss_1 ]: 0.2669465 |
| xmem_egoexo_tune_s3 - It 13087 [TRAIN] [ce_loss_2 ]: 0.0074295 |
| xmem_egoexo_tune_s3 - It 13087 [TRAIN] [dice_loss_2 ]: 0.2844884 |
| xmem_egoexo_tune_s3 - It 13087 [TRAIN] [ce_loss_3 ]: 0.0080797 |
| xmem_egoexo_tune_s3 - It 13087 [TRAIN] [dice_loss_3 ]: 0.2996137 |
| xmem_egoexo_tune_s3 - It 13087 [TRAIN] [iou ]: 0.6380248 |
| xmem_egoexo_tune_s3 - It 13087 [TRAIN] [total_loss ]: 0.8735018 |
| xmem_egoexo_tune_s3 - It 13087 [TRAIN] [p ]: 1.0000000 |
| xmem_egoexo_tune_s3 - It 13087 [TRAIN] [ce_loss_1 ]: 0.0066665 |
| xmem_egoexo_tune_s3 - It 13087 [TRAIN] [dice_loss_1 ]: 0.2660036 |
| xmem_egoexo_tune_s3 - It 13087 [TRAIN] [ce_loss_2 ]: 0.0075407 |
| xmem_egoexo_tune_s3 - It 13087 [TRAIN] [dice_loss_2 ]: 0.2849388 |
| xmem_egoexo_tune_s3 - It 13087 [TRAIN] [ce_loss_3 ]: 0.0080456 |
| xmem_egoexo_tune_s3 - It 13087 [TRAIN] [dice_loss_3 ]: 0.3003064 |
| xmem_egoexo_tune_s3 - It 13087 [TRAIN] [iou ]: 0.6376935 |
| Current epoch: 35 |
| Current epoch: 35 |
| Current epoch: 35 |
| Current epoch: 35 |
| xmem_egoexo_tune_s3 - It 13473 [TRAIN] [time ]: 17.380188 |
| xmem_egoexo_tune_s3 - It 13473 [TRAIN] [time ]: 17.380282 |
| xmem_egoexo_tune_s3 - It 13473 [TRAIN] [total_loss ]: 0.8632630 |
| xmem_egoexo_tune_s3 - It 13473 [TRAIN] [p ]: 1.0000000 |
| xmem_egoexo_tune_s3 - It 13473 [TRAIN] [ce_loss_1 ]: 0.0067582 |
| xmem_egoexo_tune_s3 - It 13473 [TRAIN] [dice_loss_1 ]: 0.2560612 |
| xmem_egoexo_tune_s3 - It 13473 [TRAIN] [ce_loss_2 ]: 0.0076919 |
| xmem_egoexo_tune_s3 - It 13473 [TRAIN] [dice_loss_2 ]: 0.2808583 |
| xmem_egoexo_tune_s3 - It 13473 [TRAIN] [ce_loss_3 ]: 0.0085491 |
| xmem_egoexo_tune_s3 - It 13473 [TRAIN] [dice_loss_3 ]: 0.3033441 |
| xmem_egoexo_tune_s3 - It 13473 [TRAIN] [iou ]: 0.6409144 |
| xmem_egoexo_tune_s3 - It 13473 [TRAIN] [total_loss ]: 0.8679157 |
| xmem_egoexo_tune_s3 - It 13473 [TRAIN] [p ]: 1.0000000 |
| xmem_egoexo_tune_s3 - It 13473 [TRAIN] [ce_loss_1 ]: 0.0068524 |
| xmem_egoexo_tune_s3 - It 13473 [TRAIN] [dice_loss_1 ]: 0.2556495 |
| xmem_egoexo_tune_s3 - It 13473 [TRAIN] [ce_loss_2 ]: 0.0076188 |
| xmem_egoexo_tune_s3 - It 13473 [TRAIN] [dice_loss_2 ]: 0.2825295 |
| xmem_egoexo_tune_s3 - It 13473 [TRAIN] [ce_loss_3 ]: 0.0083468 |
| xmem_egoexo_tune_s3 - It 13473 [TRAIN] [dice_loss_3 ]: 0.3069186 |
| xmem_egoexo_tune_s3 - It 13473 [TRAIN] [iou ]: 0.6399687 |
| Current epoch: 36 |
| Current epoch: 36 |
| Current epoch: 36 |
| Current epoch: 36 |
| xmem_egoexo_tune_s3 - It 13859 [TRAIN] [time ]: 17.373065 |
| xmem_egoexo_tune_s3 - It 13859 [TRAIN] [total_loss ]: 0.8572934 |
| xmem_egoexo_tune_s3 - It 13859 [TRAIN] [p ]: 1.0000000 |
| xmem_egoexo_tune_s3 - It 13859 [TRAIN] [ce_loss_1 ]: 0.0062161 |
| xmem_egoexo_tune_s3 - It 13859 [TRAIN] [dice_loss_1 ]: 0.2554022 |
| xmem_egoexo_tune_s3 - It 13859 [TRAIN] [ce_loss_2 ]: 0.0071576 |
| xmem_egoexo_tune_s3 - It 13859 [TRAIN] [dice_loss_2 ]: 0.2819637 |
| xmem_egoexo_tune_s3 - It 13859 [TRAIN] [ce_loss_3 ]: 0.0082561 |
| xmem_egoexo_tune_s3 - It 13859 [TRAIN] [dice_loss_3 ]: 0.2982974 |
| xmem_egoexo_tune_s3 - It 13859 [TRAIN] [iou ]: 0.6438627 |
| xmem_egoexo_tune_s3 - It 13859 [TRAIN] [time ]: 17.377288 |
| xmem_egoexo_tune_s3 - It 13859 [TRAIN] [total_loss ]: 0.8558124 |
| xmem_egoexo_tune_s3 - It 13859 [TRAIN] [p ]: 1.0000000 |
| xmem_egoexo_tune_s3 - It 13859 [TRAIN] [ce_loss_1 ]: 0.0064624 |
| xmem_egoexo_tune_s3 - It 13859 [TRAIN] [dice_loss_1 ]: 0.2554910 |
| xmem_egoexo_tune_s3 - It 13859 [TRAIN] [ce_loss_2 ]: 0.0072938 |
| xmem_egoexo_tune_s3 - It 13859 [TRAIN] [dice_loss_2 ]: 0.2810275 |
| xmem_egoexo_tune_s3 - It 13859 [TRAIN] [ce_loss_3 ]: 0.0081051 |
| xmem_egoexo_tune_s3 - It 13859 [TRAIN] [dice_loss_3 ]: 0.2974323 |
| xmem_egoexo_tune_s3 - It 13859 [TRAIN] [iou ]: 0.6442252 |
| Current epoch: 37 |
| Current epoch: 37 |
| Current epoch: 37 |
| Current epoch: 37 |
| Network saved to /data/work-gcp-europe-west4-a/yuqian_fu/Ego/saves_xmem2/Sep10_20.14.00_xmem_egoexo_tune_s3/Sep10_20.14.00_xmem_egoexo_tune_s3_14000.pth. |
| Network saved to /data/work-gcp-europe-west4-a/yuqian_fu/Ego/saves_xmem2/Sep10_20.14.00_xmem_egoexo_tune_s3/Sep10_20.14.00_xmem_egoexo_tune_s3_14000.pth. |
| Checkpoint saved to /data/work-gcp-europe-west4-a/yuqian_fu/Ego/saves_xmem2/Sep10_20.14.00_xmem_egoexo_tune_s3/Sep10_20.14.00_xmem_egoexo_tune_s3_checkpoint_14000.pth. |
| Checkpoint saved to /data/work-gcp-europe-west4-a/yuqian_fu/Ego/saves_xmem2/Sep10_20.14.00_xmem_egoexo_tune_s3/Sep10_20.14.00_xmem_egoexo_tune_s3_checkpoint_14000.pth. |
| xmem_egoexo_tune_s3 - It 14245 [TRAIN] [time ]: 17.387759 |
| xmem_egoexo_tune_s3 - It 14245 [TRAIN] [time ]: 17.391952 |
| xmem_egoexo_tune_s3 - It 14245 [TRAIN] [total_loss ]: 0.8605376 |
| xmem_egoexo_tune_s3 - It 14245 [TRAIN] [p ]: 1.0000000 |
| xmem_egoexo_tune_s3 - It 14245 [TRAIN] [ce_loss_1 ]: 0.0069811 |
| xmem_egoexo_tune_s3 - It 14245 [TRAIN] [dice_loss_1 ]: 0.2600437 |
| xmem_egoexo_tune_s3 - It 14245 [TRAIN] [ce_loss_2 ]: 0.0076526 |
| xmem_egoexo_tune_s3 - It 14245 [TRAIN] [dice_loss_2 ]: 0.2791029 |
| xmem_egoexo_tune_s3 - It 14245 [TRAIN] [ce_loss_3 ]: 0.0081993 |
| xmem_egoexo_tune_s3 - It 14245 [TRAIN] [dice_loss_3 ]: 0.2985578 |
| xmem_egoexo_tune_s3 - It 14245 [TRAIN] [iou ]: 0.6412630 |
| xmem_egoexo_tune_s3 - It 14245 [TRAIN] [total_loss ]: 0.8589408 |
| xmem_egoexo_tune_s3 - It 14245 [TRAIN] [p ]: 1.0000000 |
| xmem_egoexo_tune_s3 - It 14245 [TRAIN] [ce_loss_1 ]: 0.0068052 |
| xmem_egoexo_tune_s3 - It 14245 [TRAIN] [dice_loss_1 ]: 0.2595748 |
| xmem_egoexo_tune_s3 - It 14245 [TRAIN] [ce_loss_2 ]: 0.0074011 |
| xmem_egoexo_tune_s3 - It 14245 [TRAIN] [dice_loss_2 ]: 0.2790147 |
| xmem_egoexo_tune_s3 - It 14245 [TRAIN] [ce_loss_3 ]: 0.0078318 |
| xmem_egoexo_tune_s3 - It 14245 [TRAIN] [dice_loss_3 ]: 0.2983131 |
| xmem_egoexo_tune_s3 - It 14245 [TRAIN] [iou ]: 0.6423543 |
| Current epoch: 38 |
| Current epoch: 38 |
| Current epoch: 38 |
| Current epoch: 38 |
| xmem_egoexo_tune_s3 - It 14631 [TRAIN] [time ]: 17.505573 |
| xmem_egoexo_tune_s3 - It 14631 [TRAIN] [total_loss ]: 0.8550047 |
| xmem_egoexo_tune_s3 - It 14631 [TRAIN] [p ]: 1.0000000 |
| xmem_egoexo_tune_s3 - It 14631 [TRAIN] [ce_loss_1 ]: 0.0065407 |
| xmem_egoexo_tune_s3 - It 14631 [TRAIN] [dice_loss_1 ]: 0.2592743 |
| xmem_egoexo_tune_s3 - It 14631 [TRAIN] [ce_loss_2 ]: 0.0072192 |
| xmem_egoexo_tune_s3 - It 14631 [TRAIN] [dice_loss_2 ]: 0.2746483 |
| xmem_egoexo_tune_s3 - It 14631 [TRAIN] [ce_loss_3 ]: 0.0086312 |
| xmem_egoexo_tune_s3 - It 14631 [TRAIN] [dice_loss_3 ]: 0.2986906 |
| xmem_egoexo_tune_s3 - It 14631 [TRAIN] [iou ]: 0.6422417 |
| xmem_egoexo_tune_s3 - It 14631 [TRAIN] [time ]: 17.505635 |
| xmem_egoexo_tune_s3 - It 14631 [TRAIN] [total_loss ]: 0.8510866 |
| xmem_egoexo_tune_s3 - It 14631 [TRAIN] [p ]: 1.0000000 |
| xmem_egoexo_tune_s3 - It 14631 [TRAIN] [ce_loss_1 ]: 0.0065495 |
| xmem_egoexo_tune_s3 - It 14631 [TRAIN] [dice_loss_1 ]: 0.2581148 |
| xmem_egoexo_tune_s3 - It 14631 [TRAIN] [ce_loss_2 ]: 0.0068900 |
| xmem_egoexo_tune_s3 - It 14631 [TRAIN] [dice_loss_2 ]: 0.2736590 |
| xmem_egoexo_tune_s3 - It 14631 [TRAIN] [ce_loss_3 ]: 0.0083359 |
| xmem_egoexo_tune_s3 - It 14631 [TRAIN] [dice_loss_3 ]: 0.2975371 |
| xmem_egoexo_tune_s3 - It 14631 [TRAIN] [iou ]: 0.6442621 |
| Current epoch: 39 |
| Current epoch: 39 |
| Current epoch: 39 |
| Current epoch: 39 |
| Network saved to /data/work-gcp-europe-west4-a/yuqian_fu/Ego/saves_xmem2/Sep10_20.14.00_xmem_egoexo_tune_s3/Sep10_20.14.00_xmem_egoexo_tune_s3_15000.pth. |
| Network saved to /data/work-gcp-europe-west4-a/yuqian_fu/Ego/saves_xmem2/Sep10_20.14.00_xmem_egoexo_tune_s3/Sep10_20.14.00_xmem_egoexo_tune_s3_15000.pth. |
| xmem_egoexo_tune_s3 - It 15017 [TRAIN] [time ]: 17.392473 |
| xmem_egoexo_tune_s3 - It 15017 [TRAIN] [total_loss ]: 0.8403213 |
| xmem_egoexo_tune_s3 - It 15017 [TRAIN] [p ]: 1.0000000 |
| xmem_egoexo_tune_s3 - It 15017 [TRAIN] [ce_loss_1 ]: 0.0122812 |
| xmem_egoexo_tune_s3 - It 15017 [TRAIN] [dice_loss_1 ]: 0.2467458 |
| xmem_egoexo_tune_s3 - It 15017 [TRAIN] [ce_loss_2 ]: 0.0074194 |
| xmem_egoexo_tune_s3 - It 15017 [TRAIN] [dice_loss_2 ]: 0.2715044 |
| xmem_egoexo_tune_s3 - It 15017 [TRAIN] [ce_loss_3 ]: 0.0083516 |
| xmem_egoexo_tune_s3 - It 15017 [TRAIN] [dice_loss_3 ]: 0.2940184 |
| xmem_egoexo_tune_s3 - It 15017 [TRAIN] [iou ]: 0.6500924 |
| xmem_egoexo_tune_s3 - It 15017 [TRAIN] [time ]: 17.395948 |
| xmem_egoexo_tune_s3 - It 15017 [TRAIN] [total_loss ]: 0.8498401 |
| xmem_egoexo_tune_s3 - It 15017 [TRAIN] [p ]: 1.0000000 |
| xmem_egoexo_tune_s3 - It 15017 [TRAIN] [ce_loss_1 ]: 0.0123704 |
| xmem_egoexo_tune_s3 - It 15017 [TRAIN] [dice_loss_1 ]: 0.2505768 |
| xmem_egoexo_tune_s3 - It 15017 [TRAIN] [ce_loss_2 ]: 0.0075758 |
| xmem_egoexo_tune_s3 - It 15017 [TRAIN] [dice_loss_2 ]: 0.2751543 |
| xmem_egoexo_tune_s3 - It 15017 [TRAIN] [ce_loss_3 ]: 0.0084607 |
| xmem_egoexo_tune_s3 - It 15017 [TRAIN] [dice_loss_3 ]: 0.2957019 |
| xmem_egoexo_tune_s3 - It 15017 [TRAIN] [iou ]: 0.6467585 |
| Current epoch: 40 |
| Current epoch: 40 |
| Current epoch: 40 |
| Current epoch: 40 |
| xmem_egoexo_tune_s3 - It 15403 [TRAIN] [time ]: 17.462990 |
| xmem_egoexo_tune_s3 - It 15403 [TRAIN] [total_loss ]: 0.8326903 |
| xmem_egoexo_tune_s3 - It 15403 [TRAIN] [p ]: 1.0000000 |
| xmem_egoexo_tune_s3 - It 15403 [TRAIN] [ce_loss_1 ]: 0.0065377 |
| xmem_egoexo_tune_s3 - It 15403 [TRAIN] [dice_loss_1 ]: 0.2536860 |
| xmem_egoexo_tune_s3 - It 15403 [TRAIN] [ce_loss_2 ]: 0.0072689 |
| xmem_egoexo_tune_s3 - It 15403 [TRAIN] [dice_loss_2 ]: 0.2706836 |
| xmem_egoexo_tune_s3 - It 15403 [TRAIN] [ce_loss_3 ]: 0.0083825 |
| xmem_egoexo_tune_s3 - It 15403 [TRAIN] [dice_loss_3 ]: 0.2861313 |
| xmem_egoexo_tune_s3 - It 15403 [TRAIN] [iou ]: 0.6511020 |
| xmem_egoexo_tune_s3 - It 15403 [TRAIN] [time ]: 17.466975 |
| xmem_egoexo_tune_s3 - It 15403 [TRAIN] [total_loss ]: 0.8311821 |
| xmem_egoexo_tune_s3 - It 15403 [TRAIN] [p ]: 1.0000000 |
| xmem_egoexo_tune_s3 - It 15403 [TRAIN] [ce_loss_1 ]: 0.0065873 |
| xmem_egoexo_tune_s3 - It 15403 [TRAIN] [dice_loss_1 ]: 0.2509352 |
| xmem_egoexo_tune_s3 - It 15403 [TRAIN] [ce_loss_2 ]: 0.0070231 |
| xmem_egoexo_tune_s3 - It 15403 [TRAIN] [dice_loss_2 ]: 0.2696689 |
| xmem_egoexo_tune_s3 - It 15403 [TRAIN] [ce_loss_3 ]: 0.0083325 |
| xmem_egoexo_tune_s3 - It 15403 [TRAIN] [dice_loss_3 ]: 0.2886348 |
| xmem_egoexo_tune_s3 - It 15403 [TRAIN] [iou ]: 0.6520369 |
| Current epoch: 41 |
| Current epoch: 41 |
| Current epoch: 41 |
| Current epoch: 41 |
| xmem_egoexo_tune_s3 - It 15789 [TRAIN] [time ]: 17.342170 |
| xmem_egoexo_tune_s3 - It 15789 [TRAIN] [total_loss ]: 0.8344208 |
| xmem_egoexo_tune_s3 - It 15789 [TRAIN] [p ]: 1.0000000 |
| xmem_egoexo_tune_s3 - It 15789 [TRAIN] [ce_loss_1 ]: 0.0060872 |
| xmem_egoexo_tune_s3 - It 15789 [TRAIN] [dice_loss_1 ]: 0.2524626 |
| xmem_egoexo_tune_s3 - It 15789 [TRAIN] [ce_loss_2 ]: 0.0068084 |
| xmem_egoexo_tune_s3 - It 15789 [TRAIN] [dice_loss_2 ]: 0.2707974 |
| xmem_egoexo_tune_s3 - It 15789 [TRAIN] [ce_loss_3 ]: 0.0075980 |
| xmem_egoexo_tune_s3 - It 15789 [TRAIN] [dice_loss_3 ]: 0.2906669 |
| xmem_egoexo_tune_s3 - It 15789 [TRAIN] [iou ]: 0.6495666 |
| xmem_egoexo_tune_s3 - It 15789 [TRAIN] [time ]: 17.343367 |
| xmem_egoexo_tune_s3 - It 15789 [TRAIN] [total_loss ]: 0.8341128 |
| xmem_egoexo_tune_s3 - It 15789 [TRAIN] [p ]: 1.0000000 |
| xmem_egoexo_tune_s3 - It 15789 [TRAIN] [ce_loss_1 ]: 0.0061940 |
| xmem_egoexo_tune_s3 - It 15789 [TRAIN] [dice_loss_1 ]: 0.2517146 |
| xmem_egoexo_tune_s3 - It 15789 [TRAIN] [ce_loss_2 ]: 0.0069687 |
| xmem_egoexo_tune_s3 - It 15789 [TRAIN] [dice_loss_2 ]: 0.2717986 |
| xmem_egoexo_tune_s3 - It 15789 [TRAIN] [ce_loss_3 ]: 0.0077662 |
| xmem_egoexo_tune_s3 - It 15789 [TRAIN] [dice_loss_3 ]: 0.2896704 |
| xmem_egoexo_tune_s3 - It 15789 [TRAIN] [iou ]: 0.6495385 |
| Current epoch: 42 |
| Current epoch: 42 |
| Current epoch: 42 |
| Current epoch: 42 |
| Network saved to /data/work-gcp-europe-west4-a/yuqian_fu/Ego/saves_xmem2/Sep10_20.14.00_xmem_egoexo_tune_s3/Sep10_20.14.00_xmem_egoexo_tune_s3_16000.pth. |
| Network saved to /data/work-gcp-europe-west4-a/yuqian_fu/Ego/saves_xmem2/Sep10_20.14.00_xmem_egoexo_tune_s3/Sep10_20.14.00_xmem_egoexo_tune_s3_16000.pth. |
| Checkpoint saved to /data/work-gcp-europe-west4-a/yuqian_fu/Ego/saves_xmem2/Sep10_20.14.00_xmem_egoexo_tune_s3/Sep10_20.14.00_xmem_egoexo_tune_s3_checkpoint_16000.pth. |
| Checkpoint saved to /data/work-gcp-europe-west4-a/yuqian_fu/Ego/saves_xmem2/Sep10_20.14.00_xmem_egoexo_tune_s3/Sep10_20.14.00_xmem_egoexo_tune_s3_checkpoint_16000.pth. |
| xmem_egoexo_tune_s3 - It 16175 [TRAIN] [time ]: 17.384726 |
| xmem_egoexo_tune_s3 - It 16175 [TRAIN] [total_loss ]: 0.8507321 |
| xmem_egoexo_tune_s3 - It 16175 [TRAIN] [p ]: 1.0000000 |
| xmem_egoexo_tune_s3 - It 16175 [TRAIN] [ce_loss_1 ]: 0.0067487 |
| xmem_egoexo_tune_s3 - It 16175 [TRAIN] [dice_loss_1 ]: 0.2535491 |
| xmem_egoexo_tune_s3 - It 16175 [TRAIN] [ce_loss_2 ]: 0.0125046 |
| xmem_egoexo_tune_s3 - It 16175 [TRAIN] [dice_loss_2 ]: 0.2745267 |
| xmem_egoexo_tune_s3 - It 16175 [TRAIN] [ce_loss_3 ]: 0.0082572 |
| xmem_egoexo_tune_s3 - It 16175 [TRAIN] [dice_loss_3 ]: 0.2951455 |
| xmem_egoexo_tune_s3 - It 16175 [TRAIN] [iou ]: 0.6466380 |
| xmem_egoexo_tune_s3 - It 16175 [TRAIN] [time ]: 17.384257 |
| xmem_egoexo_tune_s3 - It 16175 [TRAIN] [total_loss ]: 0.8446232 |
| xmem_egoexo_tune_s3 - It 16175 [TRAIN] [p ]: 1.0000000 |
| xmem_egoexo_tune_s3 - It 16175 [TRAIN] [ce_loss_1 ]: 0.0067624 |
| xmem_egoexo_tune_s3 - It 16175 [TRAIN] [dice_loss_1 ]: 0.2507897 |
| xmem_egoexo_tune_s3 - It 16175 [TRAIN] [ce_loss_2 ]: 0.0122698 |
| xmem_egoexo_tune_s3 - It 16175 [TRAIN] [dice_loss_2 ]: 0.2736256 |
| xmem_egoexo_tune_s3 - It 16175 [TRAIN] [ce_loss_3 ]: 0.0079309 |
| xmem_egoexo_tune_s3 - It 16175 [TRAIN] [dice_loss_3 ]: 0.2932447 |
| xmem_egoexo_tune_s3 - It 16175 [TRAIN] [iou ]: 0.6485030 |
| Current epoch: 43 |
| Current epoch: 43 |
| Current epoch: 43 |
| Current epoch: 43 |
| xmem_egoexo_tune_s3 - It 16561 [TRAIN] [time ]: 17.044893 |
| xmem_egoexo_tune_s3 - It 16561 [TRAIN] [time ]: 17.044896 |
| xmem_egoexo_tune_s3 - It 16561 [TRAIN] [total_loss ]: 0.8351644 |
| xmem_egoexo_tune_s3 - It 16561 [TRAIN] [p ]: 1.0000000 |
| xmem_egoexo_tune_s3 - It 16561 [TRAIN] [ce_loss_1 ]: 0.0068107 |
| xmem_egoexo_tune_s3 - It 16561 [TRAIN] [dice_loss_1 ]: 0.2459699 |
| xmem_egoexo_tune_s3 - It 16561 [TRAIN] [ce_loss_2 ]: 0.0078508 |
| xmem_egoexo_tune_s3 - It 16561 [TRAIN] [dice_loss_2 ]: 0.2729964 |
| xmem_egoexo_tune_s3 - It 16561 [TRAIN] [ce_loss_3 ]: 0.0079226 |
| xmem_egoexo_tune_s3 - It 16561 [TRAIN] [dice_loss_3 ]: 0.2936137 |
| xmem_egoexo_tune_s3 - It 16561 [TRAIN] [iou ]: 0.6504556 |
| xmem_egoexo_tune_s3 - It 16561 [TRAIN] [total_loss ]: 0.8326444 |
| xmem_egoexo_tune_s3 - It 16561 [TRAIN] [p ]: 1.0000000 |
| xmem_egoexo_tune_s3 - It 16561 [TRAIN] [ce_loss_1 ]: 0.0067791 |
| xmem_egoexo_tune_s3 - It 16561 [TRAIN] [dice_loss_1 ]: 0.2449742 |
| xmem_egoexo_tune_s3 - It 16561 [TRAIN] [ce_loss_2 ]: 0.0078945 |
| xmem_egoexo_tune_s3 - It 16561 [TRAIN] [dice_loss_2 ]: 0.2727874 |
| xmem_egoexo_tune_s3 - It 16561 [TRAIN] [ce_loss_3 ]: 0.0080288 |
| xmem_egoexo_tune_s3 - It 16561 [TRAIN] [dice_loss_3 ]: 0.2921802 |
| xmem_egoexo_tune_s3 - It 16561 [TRAIN] [iou ]: 0.6511928 |
| Current epoch: 44 |
| Current epoch: 44 |
| Current epoch: 44 |
| Current epoch: 44 |
| xmem_egoexo_tune_s3 - It 16947 [TRAIN] [time ]: 16.981041 |
| xmem_egoexo_tune_s3 - It 16947 [TRAIN] [total_loss ]: 0.8291943 |
| xmem_egoexo_tune_s3 - It 16947 [TRAIN] [p ]: 1.0000000 |
| xmem_egoexo_tune_s3 - It 16947 [TRAIN] [ce_loss_1 ]: 0.0063714 |
| xmem_egoexo_tune_s3 - It 16947 [TRAIN] [dice_loss_1 ]: 0.2492881 |
| xmem_egoexo_tune_s3 - It 16947 [TRAIN] [ce_loss_2 ]: 0.0076460 |
| xmem_egoexo_tune_s3 - It 16947 [TRAIN] [dice_loss_2 ]: 0.2677584 |
| xmem_egoexo_tune_s3 - It 16947 [TRAIN] [ce_loss_3 ]: 0.0082860 |
| xmem_egoexo_tune_s3 - It 16947 [TRAIN] [dice_loss_3 ]: 0.2898441 |
| xmem_egoexo_tune_s3 - It 16947 [TRAIN] [iou ]: 0.6518330 |
| xmem_egoexo_tune_s3 - It 16947 [TRAIN] [time ]: 16.983659 |
| xmem_egoexo_tune_s3 - It 16947 [TRAIN] [total_loss ]: 0.8270978 |
| xmem_egoexo_tune_s3 - It 16947 [TRAIN] [p ]: 1.0000000 |
| xmem_egoexo_tune_s3 - It 16947 [TRAIN] [ce_loss_1 ]: 0.0064444 |
| xmem_egoexo_tune_s3 - It 16947 [TRAIN] [dice_loss_1 ]: 0.2481723 |
| xmem_egoexo_tune_s3 - It 16947 [TRAIN] [ce_loss_2 ]: 0.0075056 |
| xmem_egoexo_tune_s3 - It 16947 [TRAIN] [dice_loss_2 ]: 0.2679721 |
| xmem_egoexo_tune_s3 - It 16947 [TRAIN] [ce_loss_3 ]: 0.0081363 |
| xmem_egoexo_tune_s3 - It 16947 [TRAIN] [dice_loss_3 ]: 0.2888669 |
| xmem_egoexo_tune_s3 - It 16947 [TRAIN] [iou ]: 0.6524211 |
| Current epoch: 45 |
| Current epoch: 45 |
| Current epoch: 45 |
| Current epoch: 45 |
| Network saved to /data/work-gcp-europe-west4-a/yuqian_fu/Ego/saves_xmem2/Sep10_20.14.00_xmem_egoexo_tune_s3/Sep10_20.14.00_xmem_egoexo_tune_s3_17000.pth. |
| Network saved to /data/work-gcp-europe-west4-a/yuqian_fu/Ego/saves_xmem2/Sep10_20.14.00_xmem_egoexo_tune_s3/Sep10_20.14.00_xmem_egoexo_tune_s3_17000.pth. |
| xmem_egoexo_tune_s3 - It 17333 [TRAIN] [time ]: 16.886557 |
| xmem_egoexo_tune_s3 - It 17333 [TRAIN] [time ]: 16.889348 |
| xmem_egoexo_tune_s3 - It 17333 [TRAIN] [total_loss ]: 0.8156712 |
| xmem_egoexo_tune_s3 - It 17333 [TRAIN] [total_loss ]: 0.8147275 |
| xmem_egoexo_tune_s3 - It 17333 [TRAIN] [p ]: 1.0000000 |
| xmem_egoexo_tune_s3 - It 17333 [TRAIN] [p ]: 1.0000000 |
| xmem_egoexo_tune_s3 - It 17333 [TRAIN] [ce_loss_1 ]: 0.0063811 |
| xmem_egoexo_tune_s3 - It 17333 [TRAIN] [ce_loss_1 ]: 0.0063769 |
| xmem_egoexo_tune_s3 - It 17333 [TRAIN] [dice_loss_1 ]: 0.2488764 |
| xmem_egoexo_tune_s3 - It 17333 [TRAIN] [dice_loss_1 ]: 0.2482083 |
| xmem_egoexo_tune_s3 - It 17333 [TRAIN] [ce_loss_2 ]: 0.0069957 |
| xmem_egoexo_tune_s3 - It 17333 [TRAIN] [ce_loss_2 ]: 0.0072116 |
| xmem_egoexo_tune_s3 - It 17333 [TRAIN] [dice_loss_2 ]: 0.2627047 |
| xmem_egoexo_tune_s3 - It 17333 [TRAIN] [dice_loss_2 ]: 0.2611292 |
| xmem_egoexo_tune_s3 - It 17333 [TRAIN] [ce_loss_3 ]: 0.0077042 |
| xmem_egoexo_tune_s3 - It 17333 [TRAIN] [ce_loss_3 ]: 0.0081443 |
| xmem_egoexo_tune_s3 - It 17333 [TRAIN] [dice_loss_3 ]: 0.2830089 |
| xmem_egoexo_tune_s3 - It 17333 [TRAIN] [dice_loss_3 ]: 0.2836570 |
| xmem_egoexo_tune_s3 - It 17333 [TRAIN] [iou ]: 0.6568626 |
| xmem_egoexo_tune_s3 - It 17333 [TRAIN] [iou ]: 0.6576206 |
| Current epoch: 46 |
| Current epoch: 46 |
| Current epoch: 46 |
| Current epoch: 46 |
| xmem_egoexo_tune_s3 - It 17719 [TRAIN] [time ]: 16.871353 |
| xmem_egoexo_tune_s3 - It 17719 [TRAIN] [time ]: 16.871421 |
| xmem_egoexo_tune_s3 - It 17719 [TRAIN] [total_loss ]: 0.8236466 |
| xmem_egoexo_tune_s3 - It 17719 [TRAIN] [p ]: 1.0000000 |
| xmem_egoexo_tune_s3 - It 17719 [TRAIN] [ce_loss_1 ]: 0.0066831 |
| xmem_egoexo_tune_s3 - It 17719 [TRAIN] [dice_loss_1 ]: 0.2499457 |
| xmem_egoexo_tune_s3 - It 17719 [TRAIN] [ce_loss_2 ]: 0.0074611 |
| xmem_egoexo_tune_s3 - It 17719 [TRAIN] [dice_loss_2 ]: 0.2669204 |
| xmem_egoexo_tune_s3 - It 17719 [TRAIN] [ce_loss_3 ]: 0.0078014 |
| xmem_egoexo_tune_s3 - It 17719 [TRAIN] [dice_loss_3 ]: 0.2848347 |
| xmem_egoexo_tune_s3 - It 17719 [TRAIN] [iou ]: 0.6534544 |
| xmem_egoexo_tune_s3 - It 17719 [TRAIN] [total_loss ]: 0.8166371 |
| xmem_egoexo_tune_s3 - It 17719 [TRAIN] [p ]: 1.0000000 |
| xmem_egoexo_tune_s3 - It 17719 [TRAIN] [ce_loss_1 ]: 0.0065673 |
| xmem_egoexo_tune_s3 - It 17719 [TRAIN] [dice_loss_1 ]: 0.2464687 |
| xmem_egoexo_tune_s3 - It 17719 [TRAIN] [ce_loss_2 ]: 0.0072480 |
| xmem_egoexo_tune_s3 - It 17719 [TRAIN] [dice_loss_2 ]: 0.2649348 |
| xmem_egoexo_tune_s3 - It 17719 [TRAIN] [ce_loss_3 ]: 0.0078146 |
| xmem_egoexo_tune_s3 - It 17719 [TRAIN] [dice_loss_3 ]: 0.2836035 |
| xmem_egoexo_tune_s3 - It 17719 [TRAIN] [iou ]: 0.6555474 |
| Current epoch: 47 |
| Current epoch: 47 |
| Current epoch: 47 |
| Current epoch: 47 |
| Network saved to /data/work-gcp-europe-west4-a/yuqian_fu/Ego/saves_xmem2/Sep10_20.14.00_xmem_egoexo_tune_s3/Sep10_20.14.00_xmem_egoexo_tune_s3_18000.pth. |
| Network saved to /data/work-gcp-europe-west4-a/yuqian_fu/Ego/saves_xmem2/Sep10_20.14.00_xmem_egoexo_tune_s3/Sep10_20.14.00_xmem_egoexo_tune_s3_18000.pth. |
| Checkpoint saved to /data/work-gcp-europe-west4-a/yuqian_fu/Ego/saves_xmem2/Sep10_20.14.00_xmem_egoexo_tune_s3/Sep10_20.14.00_xmem_egoexo_tune_s3_checkpoint_18000.pth. |
| Checkpoint saved to /data/work-gcp-europe-west4-a/yuqian_fu/Ego/saves_xmem2/Sep10_20.14.00_xmem_egoexo_tune_s3/Sep10_20.14.00_xmem_egoexo_tune_s3_checkpoint_18000.pth. |
| xmem_egoexo_tune_s3 - It 18105 [TRAIN] [time ]: 17.220408 |
| xmem_egoexo_tune_s3 - It 18105 [TRAIN] [time ]: 17.220765 |
| xmem_egoexo_tune_s3 - It 18105 [TRAIN] [total_loss ]: 0.8378596 |
| xmem_egoexo_tune_s3 - It 18105 [TRAIN] [p ]: 1.0000000 |
| xmem_egoexo_tune_s3 - It 18105 [TRAIN] [ce_loss_1 ]: 0.0067535 |
| xmem_egoexo_tune_s3 - It 18105 [TRAIN] [dice_loss_1 ]: 0.2496168 |
| xmem_egoexo_tune_s3 - It 18105 [TRAIN] [ce_loss_2 ]: 0.0073217 |
| xmem_egoexo_tune_s3 - It 18105 [TRAIN] [dice_loss_2 ]: 0.2688065 |
| xmem_egoexo_tune_s3 - It 18105 [TRAIN] [ce_loss_3 ]: 0.0130253 |
| xmem_egoexo_tune_s3 - It 18105 [TRAIN] [dice_loss_3 ]: 0.2923357 |
| xmem_egoexo_tune_s3 - It 18105 [TRAIN] [iou ]: 0.6506015 |
| xmem_egoexo_tune_s3 - It 18105 [TRAIN] [total_loss ]: 0.8378151 |
| xmem_egoexo_tune_s3 - It 18105 [TRAIN] [p ]: 1.0000000 |
| xmem_egoexo_tune_s3 - It 18105 [TRAIN] [ce_loss_1 ]: 0.0067258 |
| xmem_egoexo_tune_s3 - It 18105 [TRAIN] [dice_loss_1 ]: 0.2505472 |
| xmem_egoexo_tune_s3 - It 18105 [TRAIN] [ce_loss_2 ]: 0.0072380 |
| xmem_egoexo_tune_s3 - It 18105 [TRAIN] [dice_loss_2 ]: 0.2686536 |
| xmem_egoexo_tune_s3 - It 18105 [TRAIN] [ce_loss_3 ]: 0.0132604 |
| xmem_egoexo_tune_s3 - It 18105 [TRAIN] [dice_loss_3 ]: 0.2913897 |
| xmem_egoexo_tune_s3 - It 18105 [TRAIN] [iou ]: 0.6498680 |
| Current epoch: 48 |
| Current epoch: 48 |
| Current epoch: 48 |
| Current epoch: 48 |
| xmem_egoexo_tune_s3 - It 18491 [TRAIN] [time ]: 16.984346 |
| xmem_egoexo_tune_s3 - It 18491 [TRAIN] [time ]: 16.984014 |
| xmem_egoexo_tune_s3 - It 18491 [TRAIN] [total_loss ]: 0.7987242 |
| xmem_egoexo_tune_s3 - It 18491 [TRAIN] [p ]: 1.0000000 |
| xmem_egoexo_tune_s3 - It 18491 [TRAIN] [ce_loss_1 ]: 0.0065657 |
| xmem_egoexo_tune_s3 - It 18491 [TRAIN] [dice_loss_1 ]: 0.2441459 |
| xmem_egoexo_tune_s3 - It 18491 [TRAIN] [ce_loss_2 ]: 0.0068764 |
| xmem_egoexo_tune_s3 - It 18491 [TRAIN] [dice_loss_2 ]: 0.2577467 |
| xmem_egoexo_tune_s3 - It 18491 [TRAIN] [ce_loss_3 ]: 0.0075023 |
| xmem_egoexo_tune_s3 - It 18491 [TRAIN] [dice_loss_3 ]: 0.2758870 |
| xmem_egoexo_tune_s3 - It 18491 [TRAIN] [iou ]: 0.6619547 |
| xmem_egoexo_tune_s3 - It 18491 [TRAIN] [total_loss ]: 0.8002581 |
| xmem_egoexo_tune_s3 - It 18491 [TRAIN] [p ]: 1.0000000 |
| xmem_egoexo_tune_s3 - It 18491 [TRAIN] [ce_loss_1 ]: 0.0062727 |
| xmem_egoexo_tune_s3 - It 18491 [TRAIN] [dice_loss_1 ]: 0.2438019 |
| xmem_egoexo_tune_s3 - It 18491 [TRAIN] [ce_loss_2 ]: 0.0069484 |
| xmem_egoexo_tune_s3 - It 18491 [TRAIN] [dice_loss_2 ]: 0.2587081 |
| xmem_egoexo_tune_s3 - It 18491 [TRAIN] [ce_loss_3 ]: 0.0076217 |
| xmem_egoexo_tune_s3 - It 18491 [TRAIN] [dice_loss_3 ]: 0.2769050 |
| xmem_egoexo_tune_s3 - It 18491 [TRAIN] [iou ]: 0.6616296 |
| Current epoch: 49Current epoch: 49 |
| |
| Current epoch: 49 |
| Current epoch: 49 |
| xmem_egoexo_tune_s3 - It 18877 [TRAIN] [time ]: 16.990756 |
| xmem_egoexo_tune_s3 - It 18877 [TRAIN] [total_loss ]: 0.8132756 |
| xmem_egoexo_tune_s3 - It 18877 [TRAIN] [p ]: 1.0000000 |
| xmem_egoexo_tune_s3 - It 18877 [TRAIN] [ce_loss_1 ]: 0.0064399 |
| xmem_egoexo_tune_s3 - It 18877 [TRAIN] [dice_loss_1 ]: 0.2420967 |
| xmem_egoexo_tune_s3 - It 18877 [TRAIN] [ce_loss_2 ]: 0.0068741 |
| xmem_egoexo_tune_s3 - It 18877 [TRAIN] [dice_loss_2 ]: 0.2613349 |
| xmem_egoexo_tune_s3 - It 18877 [TRAIN] [ce_loss_3 ]: 0.0130399 |
| xmem_egoexo_tune_s3 - It 18877 [TRAIN] [dice_loss_3 ]: 0.2834899 |
| xmem_egoexo_tune_s3 - It 18877 [TRAIN] [iou ]: 0.6585729 |
| xmem_egoexo_tune_s3 - It 18877 [TRAIN] [time ]: 16.993828 |
| xmem_egoexo_tune_s3 - It 18877 [TRAIN] [total_loss ]: 0.8065046 |
| xmem_egoexo_tune_s3 - It 18877 [TRAIN] [p ]: 1.0000000 |
| xmem_egoexo_tune_s3 - It 18877 [TRAIN] [ce_loss_1 ]: 0.0062062 |
| xmem_egoexo_tune_s3 - It 18877 [TRAIN] [dice_loss_1 ]: 0.2409506 |
| xmem_egoexo_tune_s3 - It 18877 [TRAIN] [ce_loss_2 ]: 0.0068489 |
| xmem_egoexo_tune_s3 - It 18877 [TRAIN] [dice_loss_2 ]: 0.2581268 |
| xmem_egoexo_tune_s3 - It 18877 [TRAIN] [ce_loss_3 ]: 0.0134968 |
| xmem_egoexo_tune_s3 - It 18877 [TRAIN] [dice_loss_3 ]: 0.2808751 |
| xmem_egoexo_tune_s3 - It 18877 [TRAIN] [iou ]: 0.6615135 |
| Current epoch: 50 |
| Current epoch: 50 |
| Current epoch: 50 |
| Current epoch: 50 |
| Network saved to /data/work-gcp-europe-west4-a/yuqian_fu/Ego/saves_xmem2/Sep10_20.14.00_xmem_egoexo_tune_s3/Sep10_20.14.00_xmem_egoexo_tune_s3_19000.pth. |
| Network saved to /data/work-gcp-europe-west4-a/yuqian_fu/Ego/saves_xmem2/Sep10_20.14.00_xmem_egoexo_tune_s3/Sep10_20.14.00_xmem_egoexo_tune_s3_19000.pth. |
| xmem_egoexo_tune_s3 - It 19263 [TRAIN] [time ]: 16.855857 |
| xmem_egoexo_tune_s3 - It 19263 [TRAIN] [total_loss ]: 0.8071635 |
| xmem_egoexo_tune_s3 - It 19263 [TRAIN] [p ]: 1.0000000 |
| xmem_egoexo_tune_s3 - It 19263 [TRAIN] [ce_loss_1 ]: 0.0061802 |
| xmem_egoexo_tune_s3 - It 19263 [TRAIN] [dice_loss_1 ]: 0.2418832 |
| xmem_egoexo_tune_s3 - It 19263 [TRAIN] [ce_loss_2 ]: 0.0120591 |
| xmem_egoexo_tune_s3 - It 19263 [TRAIN] [dice_loss_2 ]: 0.2608682 |
| xmem_egoexo_tune_s3 - It 19263 [TRAIN] [ce_loss_3 ]: 0.0074340 |
| xmem_egoexo_tune_s3 - It 19263 [TRAIN] [dice_loss_3 ]: 0.2787386 |
| xmem_egoexo_tune_s3 - It 19263 [TRAIN] [iou ]: 0.6610227 |
| xmem_egoexo_tune_s3 - It 19263 [TRAIN] [time ]: 16.859606 |
| xmem_egoexo_tune_s3 - It 19263 [TRAIN] [total_loss ]: 0.8089929 |
| xmem_egoexo_tune_s3 - It 19263 [TRAIN] [p ]: 1.0000000 |
| xmem_egoexo_tune_s3 - It 19263 [TRAIN] [ce_loss_1 ]: 0.0059275 |
| xmem_egoexo_tune_s3 - It 19263 [TRAIN] [dice_loss_1 ]: 0.2410689 |
| xmem_egoexo_tune_s3 - It 19263 [TRAIN] [ce_loss_2 ]: 0.0120066 |
| xmem_egoexo_tune_s3 - It 19263 [TRAIN] [dice_loss_2 ]: 0.2630042 |
| xmem_egoexo_tune_s3 - It 19263 [TRAIN] [ce_loss_3 ]: 0.0074829 |
| xmem_egoexo_tune_s3 - It 19263 [TRAIN] [dice_loss_3 ]: 0.2795025 |
| xmem_egoexo_tune_s3 - It 19263 [TRAIN] [iou ]: 0.6600888 |
| Current epoch: 51 |
| Current epoch: 51 |
| Current epoch: 51 |
| Current epoch: 51 |
| xmem_egoexo_tune_s3 - It 19649 [TRAIN] [time ]: 17.028869 |
| xmem_egoexo_tune_s3 - It 19649 [TRAIN] [time ]: 17.029547 |
| xmem_egoexo_tune_s3 - It 19649 [TRAIN] [total_loss ]: 0.8094299 |
| xmem_egoexo_tune_s3 - It 19649 [TRAIN] [total_loss ]: 0.7995327 |
| xmem_egoexo_tune_s3 - It 19649 [TRAIN] [p ]: 1.0000000 |
| xmem_egoexo_tune_s3 - It 19649 [TRAIN] [p ]: 1.0000000 |
| xmem_egoexo_tune_s3 - It 19649 [TRAIN] [ce_loss_1 ]: 0.0063089 |
| xmem_egoexo_tune_s3 - It 19649 [TRAIN] [ce_loss_1 ]: 0.0063057 |
| xmem_egoexo_tune_s3 - It 19649 [TRAIN] [dice_loss_1 ]: 0.2464801 |
| xmem_egoexo_tune_s3 - It 19649 [TRAIN] [dice_loss_1 ]: 0.2436901 |
| xmem_egoexo_tune_s3 - It 19649 [TRAIN] [ce_loss_2 ]: 0.0069060 |
| xmem_egoexo_tune_s3 - It 19649 [TRAIN] [ce_loss_2 ]: 0.0069589 |
| xmem_egoexo_tune_s3 - It 19649 [TRAIN] [dice_loss_2 ]: 0.2625196 |
| xmem_egoexo_tune_s3 - It 19649 [TRAIN] [dice_loss_2 ]: 0.2600436 |
| xmem_egoexo_tune_s3 - It 19649 [TRAIN] [ce_loss_3 ]: 0.0075266 |
| xmem_egoexo_tune_s3 - It 19649 [TRAIN] [ce_loss_3 ]: 0.0077608 |
| xmem_egoexo_tune_s3 - It 19649 [TRAIN] [dice_loss_3 ]: 0.2796884 |
| xmem_egoexo_tune_s3 - It 19649 [TRAIN] [dice_loss_3 ]: 0.2747733 |
| xmem_egoexo_tune_s3 - It 19649 [TRAIN] [iou ]: 0.6585171 |
| xmem_egoexo_tune_s3 - It 19649 [TRAIN] [iou ]: 0.6617586 |
| Current epoch: 52 |
| Current epoch: 52 |
| Current epoch: 52 |
| Current epoch: 52 |
| Network saved to /data/work-gcp-europe-west4-a/yuqian_fu/Ego/saves_xmem2/Sep10_20.14.00_xmem_egoexo_tune_s3/Sep10_20.14.00_xmem_egoexo_tune_s3_20000.pth. |
| Network saved to /data/work-gcp-europe-west4-a/yuqian_fu/Ego/saves_xmem2/Sep10_20.14.00_xmem_egoexo_tune_s3/Sep10_20.14.00_xmem_egoexo_tune_s3_20000.pth. |
| Checkpoint saved to /data/work-gcp-europe-west4-a/yuqian_fu/Ego/saves_xmem2/Sep10_20.14.00_xmem_egoexo_tune_s3/Sep10_20.14.00_xmem_egoexo_tune_s3_checkpoint_20000.pth. |
| Checkpoint saved to /data/work-gcp-europe-west4-a/yuqian_fu/Ego/saves_xmem2/Sep10_20.14.00_xmem_egoexo_tune_s3/Sep10_20.14.00_xmem_egoexo_tune_s3_checkpoint_20000.pth. |
| xmem_egoexo_tune_s3 - It 20035 [TRAIN] [time ]: 17.059396 |
| xmem_egoexo_tune_s3 - It 20035 [TRAIN] [time ]: 17.059396 |
| xmem_egoexo_tune_s3 - It 20035 [TRAIN] [total_loss ]: 0.7849532 |
| xmem_egoexo_tune_s3 - It 20035 [TRAIN] [total_loss ]: 0.7928102 |
| xmem_egoexo_tune_s3 - It 20035 [TRAIN] [p ]: 0.9999722 |
| xmem_egoexo_tune_s3 - It 20035 [TRAIN] [p ]: 0.9999722 |
| xmem_egoexo_tune_s3 - It 20035 [TRAIN] [ce_loss_1 ]: 0.0109384 |
| xmem_egoexo_tune_s3 - It 20035 [TRAIN] [ce_loss_1 ]: 0.0111236 |
| xmem_egoexo_tune_s3 - It 20035 [TRAIN] [dice_loss_1 ]: 0.2331203 |
| xmem_egoexo_tune_s3 - It 20035 [TRAIN] [dice_loss_1 ]: 0.2361585 |
| xmem_egoexo_tune_s3 - It 20035 [TRAIN] [ce_loss_2 ]: 0.0068829 |
| xmem_egoexo_tune_s3 - It 20035 [TRAIN] [ce_loss_2 ]: 0.0070630 |
| xmem_egoexo_tune_s3 - It 20035 [TRAIN] [dice_loss_2 ]: 0.2546201 |
| xmem_egoexo_tune_s3 - It 20035 [TRAIN] [dice_loss_2 ]: 0.2562985 |
| xmem_egoexo_tune_s3 - It 20035 [TRAIN] [ce_loss_3 ]: 0.0070030 |
| xmem_egoexo_tune_s3 - It 20035 [TRAIN] [ce_loss_3 ]: 0.0072043 |
| xmem_egoexo_tune_s3 - It 20035 [TRAIN] [dice_loss_3 ]: 0.2723882 |
| xmem_egoexo_tune_s3 - It 20035 [TRAIN] [dice_loss_3 ]: 0.2749620 |
| xmem_egoexo_tune_s3 - It 20035 [TRAIN] [iou ]: 0.6671682 |
| xmem_egoexo_tune_s3 - It 20035 [TRAIN] [iou ]: 0.6641426 |
| Current epoch: 53 |
| Current epoch: 53 |
| Current epoch: 53 |
| Current epoch: 53 |
| xmem_egoexo_tune_s3 - It 20421 [TRAIN] [time ]: 16.933678 |
| xmem_egoexo_tune_s3 - It 20421 [TRAIN] [total_loss ]: 0.7927576 |
| xmem_egoexo_tune_s3 - It 20421 [TRAIN] [p ]: 0.9961155 |
| xmem_egoexo_tune_s3 - It 20421 [TRAIN] [time ]: 16.934002 |
| xmem_egoexo_tune_s3 - It 20421 [TRAIN] [ce_loss_1 ]: 0.0056667 |
| xmem_egoexo_tune_s3 - It 20421 [TRAIN] [dice_loss_1 ]: 0.2359274 |
| xmem_egoexo_tune_s3 - It 20421 [TRAIN] [ce_loss_2 ]: 0.0114078 |
| xmem_egoexo_tune_s3 - It 20421 [TRAIN] [dice_loss_2 ]: 0.2539464 |
| xmem_egoexo_tune_s3 - It 20421 [TRAIN] [ce_loss_3 ]: 0.0068604 |
| xmem_egoexo_tune_s3 - It 20421 [TRAIN] [dice_loss_3 ]: 0.2789489 |
| xmem_egoexo_tune_s3 - It 20421 [TRAIN] [iou ]: 0.6659321 |
| xmem_egoexo_tune_s3 - It 20421 [TRAIN] [total_loss ]: 0.7879611 |
| xmem_egoexo_tune_s3 - It 20421 [TRAIN] [p ]: 0.9961155 |
| xmem_egoexo_tune_s3 - It 20421 [TRAIN] [ce_loss_1 ]: 0.0056933 |
| xmem_egoexo_tune_s3 - It 20421 [TRAIN] [dice_loss_1 ]: 0.2354602 |
| xmem_egoexo_tune_s3 - It 20421 [TRAIN] [ce_loss_2 ]: 0.0114280 |
| xmem_egoexo_tune_s3 - It 20421 [TRAIN] [dice_loss_2 ]: 0.2527546 |
| xmem_egoexo_tune_s3 - It 20421 [TRAIN] [ce_loss_3 ]: 0.0067787 |
| xmem_egoexo_tune_s3 - It 20421 [TRAIN] [dice_loss_3 ]: 0.2758458 |
| xmem_egoexo_tune_s3 - It 20421 [TRAIN] [iou ]: 0.6671259 |
| Current epoch: 54 |
| Current epoch: 54 |
| Current epoch: 54 |
| Current epoch: 54 |
| xmem_egoexo_tune_s3 - It 20807 [TRAIN] [time ]: 16.955894 |
| xmem_egoexo_tune_s3 - It 20807 [TRAIN] [total_loss ]: 0.7715119 |
| xmem_egoexo_tune_s3 - It 20807 [TRAIN] [p ]: 0.9895535 |
| xmem_egoexo_tune_s3 - It 20807 [TRAIN] [ce_loss_1 ]: 0.0058580 |
| xmem_egoexo_tune_s3 - It 20807 [TRAIN] [dice_loss_1 ]: 0.2332636 |
| xmem_egoexo_tune_s3 - It 20807 [TRAIN] [ce_loss_2 ]: 0.0067686 |
| xmem_egoexo_tune_s3 - It 20807 [TRAIN] [dice_loss_2 ]: 0.2482347 |
| xmem_egoexo_tune_s3 - It 20807 [TRAIN] [ce_loss_3 ]: 0.0072223 |
| xmem_egoexo_tune_s3 - It 20807 [TRAIN] [dice_loss_3 ]: 0.2701644 |
| xmem_egoexo_tune_s3 - It 20807 [TRAIN] [iou ]: 0.6702619 |
| xmem_egoexo_tune_s3 - It 20807 [TRAIN] [time ]: 16.956345 |
| xmem_egoexo_tune_s3 - It 20807 [TRAIN] [total_loss ]: 0.7912991 |
| xmem_egoexo_tune_s3 - It 20807 [TRAIN] [p ]: 0.9895535 |
| xmem_egoexo_tune_s3 - It 20807 [TRAIN] [ce_loss_1 ]: 0.0061997 |
| xmem_egoexo_tune_s3 - It 20807 [TRAIN] [dice_loss_1 ]: 0.2408392 |
| xmem_egoexo_tune_s3 - It 20807 [TRAIN] [ce_loss_2 ]: 0.0069549 |
| xmem_egoexo_tune_s3 - It 20807 [TRAIN] [dice_loss_2 ]: 0.2558301 |
| xmem_egoexo_tune_s3 - It 20807 [TRAIN] [ce_loss_3 ]: 0.0074044 |
| xmem_egoexo_tune_s3 - It 20807 [TRAIN] [dice_loss_3 ]: 0.2740705 |
| xmem_egoexo_tune_s3 - It 20807 [TRAIN] [iou ]: 0.6633548 |
| Current epoch: 55 |
| Current epoch: 55 |
| Current epoch: 55 |
| Current epoch: 55 |
| Network saved to /data/work-gcp-europe-west4-a/yuqian_fu/Ego/saves_xmem2/Sep10_20.14.00_xmem_egoexo_tune_s3/Sep10_20.14.00_xmem_egoexo_tune_s3_21000.pth. |
| Network saved to /data/work-gcp-europe-west4-a/yuqian_fu/Ego/saves_xmem2/Sep10_20.14.00_xmem_egoexo_tune_s3/Sep10_20.14.00_xmem_egoexo_tune_s3_21000.pth. |
| xmem_egoexo_tune_s3 - It 21193 [TRAIN] [time ]: 17.069115 |
| xmem_egoexo_tune_s3 - It 21193 [TRAIN] [time ]: 17.070121 |
| xmem_egoexo_tune_s3 - It 21193 [TRAIN] [total_loss ]: 0.7811578 |
| xmem_egoexo_tune_s3 - It 21193 [TRAIN] [p ]: 0.9829915 |
| xmem_egoexo_tune_s3 - It 21193 [TRAIN] [ce_loss_1 ]: 0.0063082 |
| xmem_egoexo_tune_s3 - It 21193 [TRAIN] [dice_loss_1 ]: 0.2338694 |
| xmem_egoexo_tune_s3 - It 21193 [TRAIN] [ce_loss_2 ]: 0.0065945 |
| xmem_egoexo_tune_s3 - It 21193 [TRAIN] [dice_loss_2 ]: 0.2526202 |
| xmem_egoexo_tune_s3 - It 21193 [TRAIN] [ce_loss_3 ]: 0.0076206 |
| xmem_egoexo_tune_s3 - It 21193 [TRAIN] [dice_loss_3 ]: 0.2741446 |
| xmem_egoexo_tune_s3 - It 21193 [TRAIN] [iou ]: 0.6686309 |
| xmem_egoexo_tune_s3 - It 21193 [TRAIN] [total_loss ]: 0.7828432 |
| xmem_egoexo_tune_s3 - It 21193 [TRAIN] [p ]: 0.9829915 |
| xmem_egoexo_tune_s3 - It 21193 [TRAIN] [ce_loss_1 ]: 0.0061940 |
| xmem_egoexo_tune_s3 - It 21193 [TRAIN] [dice_loss_1 ]: 0.2343612 |
| xmem_egoexo_tune_s3 - It 21193 [TRAIN] [ce_loss_2 ]: 0.0068351 |
| xmem_egoexo_tune_s3 - It 21193 [TRAIN] [dice_loss_2 ]: 0.2523067 |
| xmem_egoexo_tune_s3 - It 21193 [TRAIN] [ce_loss_3 ]: 0.0078755 |
| xmem_egoexo_tune_s3 - It 21193 [TRAIN] [dice_loss_3 ]: 0.2752704 |
| xmem_egoexo_tune_s3 - It 21193 [TRAIN] [iou ]: 0.6681686 |
| Current epoch: 56 |
| Current epoch: 56 |
| Current epoch: 56 |
| Current epoch: 56 |
| xmem_egoexo_tune_s3 - It 21579 [TRAIN] [time ]: 16.884066 |
| xmem_egoexo_tune_s3 - It 21579 [TRAIN] [total_loss ]: 0.7690761 |
| xmem_egoexo_tune_s3 - It 21579 [TRAIN] [p ]: 0.9764295 |
| xmem_egoexo_tune_s3 - It 21579 [TRAIN] [ce_loss_1 ]: 0.0062169 |
| xmem_egoexo_tune_s3 - It 21579 [TRAIN] [dice_loss_1 ]: 0.2310277 |
| xmem_egoexo_tune_s3 - It 21579 [TRAIN] [ce_loss_2 ]: 0.0072477 |
| xmem_egoexo_tune_s3 - It 21579 [TRAIN] [dice_loss_2 ]: 0.2474456 |
| xmem_egoexo_tune_s3 - It 21579 [TRAIN] [ce_loss_3 ]: 0.0076502 |
| xmem_egoexo_tune_s3 - It 21579 [TRAIN] [dice_loss_3 ]: 0.2694877 |
| xmem_egoexo_tune_s3 - It 21579 [TRAIN] [iou ]: 0.6727124 |
| xmem_egoexo_tune_s3 - It 21579 [TRAIN] [time ]: 16.883947 |
| xmem_egoexo_tune_s3 - It 21579 [TRAIN] [total_loss ]: 0.7774159 |
| xmem_egoexo_tune_s3 - It 21579 [TRAIN] [p ]: 0.9764295 |
| xmem_egoexo_tune_s3 - It 21579 [TRAIN] [ce_loss_1 ]: 0.0062290 |
| xmem_egoexo_tune_s3 - It 21579 [TRAIN] [dice_loss_1 ]: 0.2320011 |
| xmem_egoexo_tune_s3 - It 21579 [TRAIN] [ce_loss_2 ]: 0.0074546 |
| xmem_egoexo_tune_s3 - It 21579 [TRAIN] [dice_loss_2 ]: 0.2507418 |
| xmem_egoexo_tune_s3 - It 21579 [TRAIN] [ce_loss_3 ]: 0.0078640 |
| xmem_egoexo_tune_s3 - It 21579 [TRAIN] [dice_loss_3 ]: 0.2731252 |
| xmem_egoexo_tune_s3 - It 21579 [TRAIN] [iou ]: 0.6701972 |
| Current epoch: 57 |
| Current epoch: 57 |
| Current epoch: 57 |
| Current epoch: 57 |
| xmem_egoexo_tune_s3 - It 21965 [TRAIN] [time ]: 16.833691 |
| xmem_egoexo_tune_s3 - It 21965 [TRAIN] [time ]: 16.833580 |
| xmem_egoexo_tune_s3 - It 21965 [TRAIN] [total_loss ]: 0.7795912 |
| xmem_egoexo_tune_s3 - It 21965 [TRAIN] [p ]: 0.9698675 |
| xmem_egoexo_tune_s3 - It 21965 [TRAIN] [ce_loss_1 ]: 0.0064473 |
| xmem_egoexo_tune_s3 - It 21965 [TRAIN] [dice_loss_1 ]: 0.2381542 |
| xmem_egoexo_tune_s3 - It 21965 [TRAIN] [ce_loss_2 ]: 0.0067417 |
| xmem_egoexo_tune_s3 - It 21965 [TRAIN] [dice_loss_2 ]: 0.2522888 |
| xmem_egoexo_tune_s3 - It 21965 [TRAIN] [ce_loss_3 ]: 0.0074158 |
| xmem_egoexo_tune_s3 - It 21965 [TRAIN] [dice_loss_3 ]: 0.2685431 |
| xmem_egoexo_tune_s3 - It 21965 [TRAIN] [iou ]: 0.6672483 |
| xmem_egoexo_tune_s3 - It 21965 [TRAIN] [total_loss ]: 0.7862516 |
| xmem_egoexo_tune_s3 - It 21965 [TRAIN] [p ]: 0.9698675 |
| xmem_egoexo_tune_s3 - It 21965 [TRAIN] [ce_loss_1 ]: 0.0064232 |
| xmem_egoexo_tune_s3 - It 21965 [TRAIN] [dice_loss_1 ]: 0.2387557 |
| xmem_egoexo_tune_s3 - It 21965 [TRAIN] [ce_loss_2 ]: 0.0066341 |
| xmem_egoexo_tune_s3 - It 21965 [TRAIN] [dice_loss_2 ]: 0.2541351 |
| xmem_egoexo_tune_s3 - It 21965 [TRAIN] [ce_loss_3 ]: 0.0076347 |
| xmem_egoexo_tune_s3 - It 21965 [TRAIN] [dice_loss_3 ]: 0.2726687 |
| xmem_egoexo_tune_s3 - It 21965 [TRAIN] [iou ]: 0.6655315 |
| Current epoch: 58 |
| Current epoch: 58 |
| Current epoch: 58 |
| Current epoch: 58 |
| Network saved to /data/work-gcp-europe-west4-a/yuqian_fu/Ego/saves_xmem2/Sep10_20.14.00_xmem_egoexo_tune_s3/Sep10_20.14.00_xmem_egoexo_tune_s3_22000.pth. |
| Network saved to /data/work-gcp-europe-west4-a/yuqian_fu/Ego/saves_xmem2/Sep10_20.14.00_xmem_egoexo_tune_s3/Sep10_20.14.00_xmem_egoexo_tune_s3_22000.pth. |
| Checkpoint saved to /data/work-gcp-europe-west4-a/yuqian_fu/Ego/saves_xmem2/Sep10_20.14.00_xmem_egoexo_tune_s3/Sep10_20.14.00_xmem_egoexo_tune_s3_checkpoint_22000.pth. |
| Checkpoint saved to /data/work-gcp-europe-west4-a/yuqian_fu/Ego/saves_xmem2/Sep10_20.14.00_xmem_egoexo_tune_s3/Sep10_20.14.00_xmem_egoexo_tune_s3_checkpoint_22000.pth. |
| xmem_egoexo_tune_s3 - It 22351 [TRAIN] [time ]: 16.850860 |
| xmem_egoexo_tune_s3 - It 22351 [TRAIN] [total_loss ]: 0.7661235 |
| xmem_egoexo_tune_s3 - It 22351 [TRAIN] [p ]: 0.9633054 |
| xmem_egoexo_tune_s3 - It 22351 [TRAIN] [ce_loss_1 ]: 0.0060732 |
| xmem_egoexo_tune_s3 - It 22351 [TRAIN] [dice_loss_1 ]: 0.2316508 |
| xmem_egoexo_tune_s3 - It 22351 [TRAIN] [time ]: 16.850896 |
| xmem_egoexo_tune_s3 - It 22351 [TRAIN] [ce_loss_2 ]: 0.0066037 |
| xmem_egoexo_tune_s3 - It 22351 [TRAIN] [total_loss ]: 0.7606599 |
| xmem_egoexo_tune_s3 - It 22351 [TRAIN] [dice_loss_2 ]: 0.2511224 |
| xmem_egoexo_tune_s3 - It 22351 [TRAIN] [p ]: 0.9633054 |
| xmem_egoexo_tune_s3 - It 22351 [TRAIN] [ce_loss_3 ]: 0.0075463 |
| xmem_egoexo_tune_s3 - It 22351 [TRAIN] [ce_loss_1 ]: 0.0059265 |
| xmem_egoexo_tune_s3 - It 22351 [TRAIN] [dice_loss_3 ]: 0.2631269 |
| xmem_egoexo_tune_s3 - It 22351 [TRAIN] [dice_loss_1 ]: 0.2303472 |
| xmem_egoexo_tune_s3 - It 22351 [TRAIN] [iou ]: 0.6721277 |
| xmem_egoexo_tune_s3 - It 22351 [TRAIN] [ce_loss_2 ]: 0.0066669 |
| xmem_egoexo_tune_s3 - It 22351 [TRAIN] [dice_loss_2 ]: 0.2481356 |
| xmem_egoexo_tune_s3 - It 22351 [TRAIN] [ce_loss_3 ]: 0.0073094 |
| xmem_egoexo_tune_s3 - It 22351 [TRAIN] [dice_loss_3 ]: 0.2622741 |
| xmem_egoexo_tune_s3 - It 22351 [TRAIN] [iou ]: 0.6737014 |
| Current epoch: 59 |
| Current epoch: 59 |
| Current epoch: 59 |
| Current epoch: 59 |
| xmem_egoexo_tune_s3 - It 22737 [TRAIN] [time ]: 16.904457 |
| xmem_egoexo_tune_s3 - It 22737 [TRAIN] [total_loss ]: 0.7566398 |
| xmem_egoexo_tune_s3 - It 22737 [TRAIN] [p ]: 0.9567434 |
| xmem_egoexo_tune_s3 - It 22737 [TRAIN] [ce_loss_1 ]: 0.0059653 |
| xmem_egoexo_tune_s3 - It 22737 [TRAIN] [dice_loss_1 ]: 0.2271350 |
| xmem_egoexo_tune_s3 - It 22737 [TRAIN] [time ]: 16.905410 |
| xmem_egoexo_tune_s3 - It 22737 [TRAIN] [ce_loss_2 ]: 0.0060731 |
| xmem_egoexo_tune_s3 - It 22737 [TRAIN] [dice_loss_2 ]: 0.2439093 |
| xmem_egoexo_tune_s3 - It 22737 [TRAIN] [ce_loss_3 ]: 0.0072936 |
| xmem_egoexo_tune_s3 - It 22737 [TRAIN] [dice_loss_3 ]: 0.2662633 |
| xmem_egoexo_tune_s3 - It 22737 [TRAIN] [iou ]: 0.6751478 |
| xmem_egoexo_tune_s3 - It 22737 [TRAIN] [total_loss ]: 0.7557411 |
| xmem_egoexo_tune_s3 - It 22737 [TRAIN] [p ]: 0.9567434 |
| xmem_egoexo_tune_s3 - It 22737 [TRAIN] [ce_loss_1 ]: 0.0059866 |
| xmem_egoexo_tune_s3 - It 22737 [TRAIN] [dice_loss_1 ]: 0.2256823 |
| xmem_egoexo_tune_s3 - It 22737 [TRAIN] [ce_loss_2 ]: 0.0061928 |
| xmem_egoexo_tune_s3 - It 22737 [TRAIN] [dice_loss_2 ]: 0.2438903 |
| xmem_egoexo_tune_s3 - It 22737 [TRAIN] [ce_loss_3 ]: 0.0074230 |
| xmem_egoexo_tune_s3 - It 22737 [TRAIN] [dice_loss_3 ]: 0.2665659 |
| xmem_egoexo_tune_s3 - It 22737 [TRAIN] [iou ]: 0.6760287 |
| Current epoch: 60 |
| Current epoch: 60 |
| Current epoch: 60 |
| Current epoch: 60 |
| Network saved to /data/work-gcp-europe-west4-a/yuqian_fu/Ego/saves_xmem2/Sep10_20.14.00_xmem_egoexo_tune_s3/Sep10_20.14.00_xmem_egoexo_tune_s3_23000.pth. |
| Network saved to /data/work-gcp-europe-west4-a/yuqian_fu/Ego/saves_xmem2/Sep10_20.14.00_xmem_egoexo_tune_s3/Sep10_20.14.00_xmem_egoexo_tune_s3_23000.pth. |
| xmem_egoexo_tune_s3 - It 23123 [TRAIN] [time ]: 16.942766 |
| xmem_egoexo_tune_s3 - It 23123 [TRAIN] [total_loss ]: 0.7593171 |
| xmem_egoexo_tune_s3 - It 23123 [TRAIN] [p ]: 0.9501814 |
| xmem_egoexo_tune_s3 - It 23123 [TRAIN] [ce_loss_1 ]: 0.0057755 |
| xmem_egoexo_tune_s3 - It 23123 [TRAIN] [dice_loss_1 ]: 0.2302028 |
| xmem_egoexo_tune_s3 - It 23123 [TRAIN] [ce_loss_2 ]: 0.0064475 |
| xmem_egoexo_tune_s3 - It 23123 [TRAIN] [dice_loss_2 ]: 0.2440165 |
| xmem_egoexo_tune_s3 - It 23123 [TRAIN] [ce_loss_3 ]: 0.0074902 |
| xmem_egoexo_tune_s3 - It 23123 [TRAIN] [dice_loss_3 ]: 0.2653844 |
| xmem_egoexo_tune_s3 - It 23123 [TRAIN] [iou ]: 0.6742948 |
| xmem_egoexo_tune_s3 - It 23123 [TRAIN] [time ]: 16.942224 |
| xmem_egoexo_tune_s3 - It 23123 [TRAIN] [total_loss ]: 0.7534502 |
| xmem_egoexo_tune_s3 - It 23123 [TRAIN] [p ]: 0.9501814 |
| xmem_egoexo_tune_s3 - It 23123 [TRAIN] [ce_loss_1 ]: 0.0057074 |
| xmem_egoexo_tune_s3 - It 23123 [TRAIN] [dice_loss_1 ]: 0.2277640 |
| xmem_egoexo_tune_s3 - It 23123 [TRAIN] [ce_loss_2 ]: 0.0063321 |
| xmem_egoexo_tune_s3 - It 23123 [TRAIN] [dice_loss_2 ]: 0.2418989 |
| xmem_egoexo_tune_s3 - It 23123 [TRAIN] [ce_loss_3 ]: 0.0074343 |
| xmem_egoexo_tune_s3 - It 23123 [TRAIN] [dice_loss_3 ]: 0.2643132 |
| xmem_egoexo_tune_s3 - It 23123 [TRAIN] [iou ]: 0.6775321 |
| Current epoch: 61 |
| Current epoch: 61 |
| Current epoch: 61 |
| Current epoch: 61 |
| xmem_egoexo_tune_s3 - It 23509 [TRAIN] [time ]: 16.852195 |
| xmem_egoexo_tune_s3 - It 23509 [TRAIN] [time ]: 16.852569 |
| xmem_egoexo_tune_s3 - It 23509 [TRAIN] [total_loss ]: 0.7645732 |
| xmem_egoexo_tune_s3 - It 23509 [TRAIN] [p ]: 0.9436194 |
| xmem_egoexo_tune_s3 - It 23509 [TRAIN] [ce_loss_1 ]: 0.0115897 |
| xmem_egoexo_tune_s3 - It 23509 [TRAIN] [dice_loss_1 ]: 0.2302781 |
| xmem_egoexo_tune_s3 - It 23509 [TRAIN] [ce_loss_2 ]: 0.0070134 |
| xmem_egoexo_tune_s3 - It 23509 [TRAIN] [dice_loss_2 ]: 0.2448334 |
| xmem_egoexo_tune_s3 - It 23509 [TRAIN] [ce_loss_3 ]: 0.0076209 |
| xmem_egoexo_tune_s3 - It 23509 [TRAIN] [dice_loss_3 ]: 0.2632375 |
| xmem_egoexo_tune_s3 - It 23509 [TRAIN] [iou ]: 0.6754506 |
| xmem_egoexo_tune_s3 - It 23509 [TRAIN] [total_loss ]: 0.7616592 |
| xmem_egoexo_tune_s3 - It 23509 [TRAIN] [p ]: 0.9436194 |
| xmem_egoexo_tune_s3 - It 23509 [TRAIN] [ce_loss_1 ]: 0.0115464 |
| xmem_egoexo_tune_s3 - It 23509 [TRAIN] [dice_loss_1 ]: 0.2291763 |
| xmem_egoexo_tune_s3 - It 23509 [TRAIN] [ce_loss_2 ]: 0.0068969 |
| xmem_egoexo_tune_s3 - It 23509 [TRAIN] [dice_loss_2 ]: 0.2439368 |
| xmem_egoexo_tune_s3 - It 23509 [TRAIN] [ce_loss_3 ]: 0.0074955 |
| xmem_egoexo_tune_s3 - It 23509 [TRAIN] [dice_loss_3 ]: 0.2626070 |
| xmem_egoexo_tune_s3 - It 23509 [TRAIN] [iou ]: 0.6761837 |
| Current epoch: 62 |
| Current epoch: 62 |
| Current epoch: 62 |
| Current epoch: 62 |
| xmem_egoexo_tune_s3 - It 23895 [TRAIN] [time ]: 16.970785 |
| xmem_egoexo_tune_s3 - It 23895 [TRAIN] [total_loss ]: 0.7745357 |
| xmem_egoexo_tune_s3 - It 23895 [TRAIN] [p ]: 0.9370574 |
| xmem_egoexo_tune_s3 - It 23895 [TRAIN] [ce_loss_1 ]: 0.0063166 |
| xmem_egoexo_tune_s3 - It 23895 [TRAIN] [dice_loss_1 ]: 0.2349413 |
| xmem_egoexo_tune_s3 - It 23895 [TRAIN] [ce_loss_2 ]: 0.0070301 |
| xmem_egoexo_tune_s3 - It 23895 [TRAIN] [dice_loss_2 ]: 0.2507512 |
| xmem_egoexo_tune_s3 - It 23895 [TRAIN] [ce_loss_3 ]: 0.0073763 |
| xmem_egoexo_tune_s3 - It 23895 [TRAIN] [dice_loss_3 ]: 0.2681200 |
| xmem_egoexo_tune_s3 - It 23895 [TRAIN] [iou ]: 0.6693774 |
| xmem_egoexo_tune_s3 - It 23895 [TRAIN] [time ]: 16.973737 |
| xmem_egoexo_tune_s3 - It 23895 [TRAIN] [total_loss ]: 0.7580872 |
| xmem_egoexo_tune_s3 - It 23895 [TRAIN] [p ]: 0.9370574 |
| xmem_egoexo_tune_s3 - It 23895 [TRAIN] [ce_loss_1 ]: 0.0061296 |
| xmem_egoexo_tune_s3 - It 23895 [TRAIN] [dice_loss_1 ]: 0.2323501 |
| xmem_egoexo_tune_s3 - It 23895 [TRAIN] [ce_loss_2 ]: 0.0066621 |
| xmem_egoexo_tune_s3 - It 23895 [TRAIN] [dice_loss_2 ]: 0.2432634 |
| xmem_egoexo_tune_s3 - It 23895 [TRAIN] [ce_loss_3 ]: 0.0072889 |
| xmem_egoexo_tune_s3 - It 23895 [TRAIN] [dice_loss_3 ]: 0.2623928 |
| xmem_egoexo_tune_s3 - It 23895 [TRAIN] [iou ]: 0.6748253 |
| Current epoch: 63 |
| Current epoch: 63 |
| Current epoch: 63 |
| Current epoch: 63 |
| Network saved to /data/work-gcp-europe-west4-a/yuqian_fu/Ego/saves_xmem2/Sep10_20.14.00_xmem_egoexo_tune_s3/Sep10_20.14.00_xmem_egoexo_tune_s3_24000.pth. |
| Network saved to /data/work-gcp-europe-west4-a/yuqian_fu/Ego/saves_xmem2/Sep10_20.14.00_xmem_egoexo_tune_s3/Sep10_20.14.00_xmem_egoexo_tune_s3_24000.pth. |
| Checkpoint saved to /data/work-gcp-europe-west4-a/yuqian_fu/Ego/saves_xmem2/Sep10_20.14.00_xmem_egoexo_tune_s3/Sep10_20.14.00_xmem_egoexo_tune_s3_checkpoint_24000.pth. |
| Checkpoint saved to /data/work-gcp-europe-west4-a/yuqian_fu/Ego/saves_xmem2/Sep10_20.14.00_xmem_egoexo_tune_s3/Sep10_20.14.00_xmem_egoexo_tune_s3_checkpoint_24000.pth. |
| xmem_egoexo_tune_s3 - It 24281 [TRAIN] [time ]: 16.928785 |
| xmem_egoexo_tune_s3 - It 24281 [TRAIN] [time ]: 16.927549 |
| xmem_egoexo_tune_s3 - It 24281 [TRAIN] [total_loss ]: 0.7628237 |
| xmem_egoexo_tune_s3 - It 24281 [TRAIN] [p ]: 0.9304955 |
| xmem_egoexo_tune_s3 - It 24281 [TRAIN] [ce_loss_1 ]: 0.0060249 |
| xmem_egoexo_tune_s3 - It 24281 [TRAIN] [dice_loss_1 ]: 0.2306462 |
| xmem_egoexo_tune_s3 - It 24281 [TRAIN] [ce_loss_2 ]: 0.0069551 |
| xmem_egoexo_tune_s3 - It 24281 [TRAIN] [dice_loss_2 ]: 0.2479067 |
| xmem_egoexo_tune_s3 - It 24281 [TRAIN] [ce_loss_3 ]: 0.0073719 |
| xmem_egoexo_tune_s3 - It 24281 [TRAIN] [dice_loss_3 ]: 0.2639186 |
| xmem_egoexo_tune_s3 - It 24281 [TRAIN] [iou ]: 0.6743069 |
| xmem_egoexo_tune_s3 - It 24281 [TRAIN] [total_loss ]: 0.7657988 |
| xmem_egoexo_tune_s3 - It 24281 [TRAIN] [p ]: 0.9304955 |
| xmem_egoexo_tune_s3 - It 24281 [TRAIN] [ce_loss_1 ]: 0.0060303 |
| xmem_egoexo_tune_s3 - It 24281 [TRAIN] [dice_loss_1 ]: 0.2298003 |
| xmem_egoexo_tune_s3 - It 24281 [TRAIN] [ce_loss_2 ]: 0.0068656 |
| xmem_egoexo_tune_s3 - It 24281 [TRAIN] [dice_loss_2 ]: 0.2496981 |
| xmem_egoexo_tune_s3 - It 24281 [TRAIN] [ce_loss_3 ]: 0.0073500 |
| xmem_egoexo_tune_s3 - It 24281 [TRAIN] [dice_loss_3 ]: 0.2660542 |
| xmem_egoexo_tune_s3 - It 24281 [TRAIN] [iou ]: 0.6730639 |
| Current epoch: 64 |
| Current epoch: 64 |
| Current epoch: 64 |
| Current epoch: 64 |
| xmem_egoexo_tune_s3 - It 24667 [TRAIN] [time ]: 16.806159 |
| xmem_egoexo_tune_s3 - It 24667 [TRAIN] [total_loss ]: 0.7428306 |
| xmem_egoexo_tune_s3 - It 24667 [TRAIN] [p ]: 0.9239335 |
| xmem_egoexo_tune_s3 - It 24667 [TRAIN] [ce_loss_1 ]: 0.0059108 |
| xmem_egoexo_tune_s3 - It 24667 [TRAIN] [dice_loss_1 ]: 0.2244736 |
| xmem_egoexo_tune_s3 - It 24667 [TRAIN] [ce_loss_2 ]: 0.0066578 |
| xmem_egoexo_tune_s3 - It 24667 [TRAIN] [dice_loss_2 ]: 0.2400009 |
| xmem_egoexo_tune_s3 - It 24667 [TRAIN] [ce_loss_3 ]: 0.0075489 |
| xmem_egoexo_tune_s3 - It 24667 [TRAIN] [dice_loss_3 ]: 0.2582384 |
| xmem_egoexo_tune_s3 - It 24667 [TRAIN] [iou ]: 0.6801959 |
| xmem_egoexo_tune_s3 - It 24667 [TRAIN] [time ]: 16.808001 |
| xmem_egoexo_tune_s3 - It 24667 [TRAIN] [total_loss ]: 0.7387712 |
| xmem_egoexo_tune_s3 - It 24667 [TRAIN] [p ]: 0.9239335 |
| xmem_egoexo_tune_s3 - It 24667 [TRAIN] [ce_loss_1 ]: 0.0061783 |
| xmem_egoexo_tune_s3 - It 24667 [TRAIN] [dice_loss_1 ]: 0.2236614 |
| xmem_egoexo_tune_s3 - It 24667 [TRAIN] [ce_loss_2 ]: 0.0067622 |
| xmem_egoexo_tune_s3 - It 24667 [TRAIN] [dice_loss_2 ]: 0.2379652 |
| xmem_egoexo_tune_s3 - It 24667 [TRAIN] [ce_loss_3 ]: 0.0077736 |
| xmem_egoexo_tune_s3 - It 24667 [TRAIN] [dice_loss_3 ]: 0.2564301 |
| xmem_egoexo_tune_s3 - It 24667 [TRAIN] [iou ]: 0.6815653 |
| Current epoch: 65 |
| Current epoch: 65 |
| Current epoch: 65 |
| Current epoch: 65 |
| Network saved to /data/work-gcp-europe-west4-a/yuqian_fu/Ego/saves_xmem2/Sep10_20.14.00_xmem_egoexo_tune_s3/Sep10_20.14.00_xmem_egoexo_tune_s3_25000.pth. |
| Network saved to /data/work-gcp-europe-west4-a/yuqian_fu/Ego/saves_xmem2/Sep10_20.14.00_xmem_egoexo_tune_s3/Sep10_20.14.00_xmem_egoexo_tune_s3_25000.pth. |
| xmem_egoexo_tune_s3 - It 25053 [TRAIN] [time ]: 16.896312 |
| xmem_egoexo_tune_s3 - It 25053 [TRAIN] [total_loss ]: 0.7656065 |
| xmem_egoexo_tune_s3 - It 25053 [TRAIN] [p ]: 0.9173715 |
| xmem_egoexo_tune_s3 - It 25053 [TRAIN] [ce_loss_1 ]: 0.0065365 |
| xmem_egoexo_tune_s3 - It 25053 [TRAIN] [time ]: 16.896469 |
| xmem_egoexo_tune_s3 - It 25053 [TRAIN] [dice_loss_1 ]: 0.2300922 |
| xmem_egoexo_tune_s3 - It 25053 [TRAIN] [ce_loss_2 ]: 0.0069247 |
| xmem_egoexo_tune_s3 - It 25053 [TRAIN] [total_loss ]: 0.7631714 |
| xmem_egoexo_tune_s3 - It 25053 [TRAIN] [dice_loss_2 ]: 0.2496877 |
| xmem_egoexo_tune_s3 - It 25053 [TRAIN] [p ]: 0.9173715 |
| xmem_egoexo_tune_s3 - It 25053 [TRAIN] [ce_loss_3 ]: 0.0133216 |
| xmem_egoexo_tune_s3 - It 25053 [TRAIN] [ce_loss_1 ]: 0.0066013 |
| xmem_egoexo_tune_s3 - It 25053 [TRAIN] [dice_loss_3 ]: 0.2590435 |
| xmem_egoexo_tune_s3 - It 25053 [TRAIN] [dice_loss_1 ]: 0.2305060 |
| xmem_egoexo_tune_s3 - It 25053 [TRAIN] [iou ]: 0.6761045 |
| xmem_egoexo_tune_s3 - It 25053 [TRAIN] [ce_loss_2 ]: 0.0071111 |
| xmem_egoexo_tune_s3 - It 25053 [TRAIN] [dice_loss_2 ]: 0.2473267 |
| xmem_egoexo_tune_s3 - It 25053 [TRAIN] [ce_loss_3 ]: 0.0138397 |
| xmem_egoexo_tune_s3 - It 25053 [TRAIN] [dice_loss_3 ]: 0.2577865 |
| xmem_egoexo_tune_s3 - It 25053 [TRAIN] [iou ]: 0.6766639 |
| Current epoch: 66 |
| Current epoch: 66 |
| Current epoch: 66 |
| Current epoch: 66 |
| xmem_egoexo_tune_s3 - It 25439 [TRAIN] [time ]: 16.818547 |
| xmem_egoexo_tune_s3 - It 25439 [TRAIN] [time ]: 16.818726 |
| xmem_egoexo_tune_s3 - It 25439 [TRAIN] [total_loss ]: 0.7374533 |
| xmem_egoexo_tune_s3 - It 25439 [TRAIN] [p ]: 0.9108095 |
| xmem_egoexo_tune_s3 - It 25439 [TRAIN] [ce_loss_1 ]: 0.0063437 |
| xmem_egoexo_tune_s3 - It 25439 [TRAIN] [dice_loss_1 ]: 0.2220008 |
| xmem_egoexo_tune_s3 - It 25439 [TRAIN] [ce_loss_2 ]: 0.0070467 |
| xmem_egoexo_tune_s3 - It 25439 [TRAIN] [dice_loss_2 ]: 0.2388959 |
| xmem_egoexo_tune_s3 - It 25439 [TRAIN] [ce_loss_3 ]: 0.0077397 |
| xmem_egoexo_tune_s3 - It 25439 [TRAIN] [dice_loss_3 ]: 0.2554261 |
| xmem_egoexo_tune_s3 - It 25439 [TRAIN] [iou ]: 0.6821396 |
| xmem_egoexo_tune_s3 - It 25439 [TRAIN] [total_loss ]: 0.7379600 |
| xmem_egoexo_tune_s3 - It 25439 [TRAIN] [p ]: 0.9108095 |
| xmem_egoexo_tune_s3 - It 25439 [TRAIN] [ce_loss_1 ]: 0.0064504 |
| xmem_egoexo_tune_s3 - It 25439 [TRAIN] [dice_loss_1 ]: 0.2226556 |
| xmem_egoexo_tune_s3 - It 25439 [TRAIN] [ce_loss_2 ]: 0.0069225 |
| xmem_egoexo_tune_s3 - It 25439 [TRAIN] [dice_loss_2 ]: 0.2380406 |
| xmem_egoexo_tune_s3 - It 25439 [TRAIN] [ce_loss_3 ]: 0.0078005 |
| xmem_egoexo_tune_s3 - It 25439 [TRAIN] [dice_loss_3 ]: 0.2560902 |
| xmem_egoexo_tune_s3 - It 25439 [TRAIN] [iou ]: 0.6822787 |
| Current epoch: 67 |
| Current epoch: 67 |
| Current epoch: 67 |
| Current epoch: 67 |
| xmem_egoexo_tune_s3 - It 25825 [TRAIN] [time ]: 16.958708 |
| xmem_egoexo_tune_s3 - It 25825 [TRAIN] [total_loss ]: 0.7728401 |
| xmem_egoexo_tune_s3 - It 25825 [TRAIN] [p ]: 0.9042475 |
| xmem_egoexo_tune_s3 - It 25825 [TRAIN] [ce_loss_1 ]: 0.0069302 |
| xmem_egoexo_tune_s3 - It 25825 [TRAIN] [dice_loss_1 ]: 0.2359781 |
| xmem_egoexo_tune_s3 - It 25825 [TRAIN] [ce_loss_2 ]: 0.0074527 |
| xmem_egoexo_tune_s3 - It 25825 [TRAIN] [dice_loss_2 ]: 0.2489103 |
| xmem_egoexo_tune_s3 - It 25825 [TRAIN] [ce_loss_3 ]: 0.0082568 |
| xmem_egoexo_tune_s3 - It 25825 [TRAIN] [dice_loss_3 ]: 0.2653118 |
| xmem_egoexo_tune_s3 - It 25825 [TRAIN] [iou ]: 0.6697497 |
| xmem_egoexo_tune_s3 - It 25825 [TRAIN] [time ]: 16.959360 |
| xmem_egoexo_tune_s3 - It 25825 [TRAIN] [total_loss ]: 0.7626206 |
| xmem_egoexo_tune_s3 - It 25825 [TRAIN] [p ]: 0.9042475 |
| xmem_egoexo_tune_s3 - It 25825 [TRAIN] [ce_loss_1 ]: 0.0069642 |
| xmem_egoexo_tune_s3 - It 25825 [TRAIN] [dice_loss_1 ]: 0.2338329 |
| xmem_egoexo_tune_s3 - It 25825 [TRAIN] [ce_loss_2 ]: 0.0072702 |
| xmem_egoexo_tune_s3 - It 25825 [TRAIN] [dice_loss_2 ]: 0.2449728 |
| xmem_egoexo_tune_s3 - It 25825 [TRAIN] [ce_loss_3 ]: 0.0078781 |
| xmem_egoexo_tune_s3 - It 25825 [TRAIN] [dice_loss_3 ]: 0.2617021 |
| xmem_egoexo_tune_s3 - It 25825 [TRAIN] [iou ]: 0.6735110 |
| Current epoch: 68 |
| Current epoch: 68 |
| Current epoch: 68 |
| Current epoch: 68 |
| Network saved to /data/work-gcp-europe-west4-a/yuqian_fu/Ego/saves_xmem2/Sep10_20.14.00_xmem_egoexo_tune_s3/Sep10_20.14.00_xmem_egoexo_tune_s3_26000.pth. |
| Network saved to /data/work-gcp-europe-west4-a/yuqian_fu/Ego/saves_xmem2/Sep10_20.14.00_xmem_egoexo_tune_s3/Sep10_20.14.00_xmem_egoexo_tune_s3_26000.pth. |
| Checkpoint saved to /data/work-gcp-europe-west4-a/yuqian_fu/Ego/saves_xmem2/Sep10_20.14.00_xmem_egoexo_tune_s3/Sep10_20.14.00_xmem_egoexo_tune_s3_checkpoint_26000.pth. |
| Checkpoint saved to /data/work-gcp-europe-west4-a/yuqian_fu/Ego/saves_xmem2/Sep10_20.14.00_xmem_egoexo_tune_s3/Sep10_20.14.00_xmem_egoexo_tune_s3_checkpoint_26000.pth. |
| xmem_egoexo_tune_s3 - It 26211 [TRAIN] [time ]: 16.920635 |
| xmem_egoexo_tune_s3 - It 26211 [TRAIN] [total_loss ]: 0.7464336 |
| xmem_egoexo_tune_s3 - It 26211 [TRAIN] [p ]: 0.8976855 |
| xmem_egoexo_tune_s3 - It 26211 [TRAIN] [ce_loss_1 ]: 0.0069429 |
| xmem_egoexo_tune_s3 - It 26211 [TRAIN] [dice_loss_1 ]: 0.2279829 |
| xmem_egoexo_tune_s3 - It 26211 [TRAIN] [ce_loss_2 ]: 0.0072193 |
| xmem_egoexo_tune_s3 - It 26211 [TRAIN] [dice_loss_2 ]: 0.2399151 |
| xmem_egoexo_tune_s3 - It 26211 [TRAIN] [ce_loss_3 ]: 0.0079147 |
| xmem_egoexo_tune_s3 - It 26211 [TRAIN] [dice_loss_3 ]: 0.2564584 |
| xmem_egoexo_tune_s3 - It 26211 [TRAIN] [iou ]: 0.6789016 |
| xmem_egoexo_tune_s3 - It 26211 [TRAIN] [time ]: 16.921716 |
| xmem_egoexo_tune_s3 - It 26211 [TRAIN] [total_loss ]: 0.7396110 |
| xmem_egoexo_tune_s3 - It 26211 [TRAIN] [p ]: 0.8976855 |
| xmem_egoexo_tune_s3 - It 26211 [TRAIN] [ce_loss_1 ]: 0.0066543 |
| xmem_egoexo_tune_s3 - It 26211 [TRAIN] [dice_loss_1 ]: 0.2264101 |
| xmem_egoexo_tune_s3 - It 26211 [TRAIN] [ce_loss_2 ]: 0.0071844 |
| xmem_egoexo_tune_s3 - It 26211 [TRAIN] [dice_loss_2 ]: 0.2368784 |
| xmem_egoexo_tune_s3 - It 26211 [TRAIN] [ce_loss_3 ]: 0.0079335 |
| xmem_egoexo_tune_s3 - It 26211 [TRAIN] [dice_loss_3 ]: 0.2545499 |
| xmem_egoexo_tune_s3 - It 26211 [TRAIN] [iou ]: 0.6821120 |
| Current epoch: 69 |
| Current epoch: 69 |
| Current epoch: 69 |
| Current epoch: 69 |
| xmem_egoexo_tune_s3 - It 26597 [TRAIN] [time ]: 16.897996 |
| xmem_egoexo_tune_s3 - It 26597 [TRAIN] [total_loss ]: 0.7389112 |
| xmem_egoexo_tune_s3 - It 26597 [TRAIN] [p ]: 0.8911234 |
| xmem_egoexo_tune_s3 - It 26597 [TRAIN] [ce_loss_1 ]: 0.0061165 |
| xmem_egoexo_tune_s3 - It 26597 [TRAIN] [dice_loss_1 ]: 0.2243166 |
| xmem_egoexo_tune_s3 - It 26597 [TRAIN] [ce_loss_2 ]: 0.0066586 |
| xmem_egoexo_tune_s3 - It 26597 [TRAIN] [dice_loss_2 ]: 0.2411262 |
| xmem_egoexo_tune_s3 - It 26597 [TRAIN] [ce_loss_3 ]: 0.0077244 |
| xmem_egoexo_tune_s3 - It 26597 [TRAIN] [dice_loss_3 ]: 0.2529686 |
| xmem_egoexo_tune_s3 - It 26597 [TRAIN] [iou ]: 0.6825784 |
| xmem_egoexo_tune_s3 - It 26597 [TRAIN] [time ]: 16.899075 |
| xmem_egoexo_tune_s3 - It 26597 [TRAIN] [total_loss ]: 0.7455435 |
| xmem_egoexo_tune_s3 - It 26597 [TRAIN] [p ]: 0.8911234 |
| xmem_egoexo_tune_s3 - It 26597 [TRAIN] [ce_loss_1 ]: 0.0063390 |
| xmem_egoexo_tune_s3 - It 26597 [TRAIN] [dice_loss_1 ]: 0.2249234 |
| xmem_egoexo_tune_s3 - It 26597 [TRAIN] [ce_loss_2 ]: 0.0068298 |
| xmem_egoexo_tune_s3 - It 26597 [TRAIN] [dice_loss_2 ]: 0.2441990 |
| xmem_egoexo_tune_s3 - It 26597 [TRAIN] [ce_loss_3 ]: 0.0077721 |
| xmem_egoexo_tune_s3 - It 26597 [TRAIN] [dice_loss_3 ]: 0.2554800 |
| xmem_egoexo_tune_s3 - It 26597 [TRAIN] [iou ]: 0.6804224 |
| Current epoch: 70 |
| Current epoch: 70 |
| Current epoch: 70 |
| Current epoch: 70 |
| xmem_egoexo_tune_s3 - It 26983 [TRAIN] [time ]: 16.842526 |
| xmem_egoexo_tune_s3 - It 26983 [TRAIN] [time ]: 16.844989 |
| xmem_egoexo_tune_s3 - It 26983 [TRAIN] [total_loss ]: 0.7540530 |
| xmem_egoexo_tune_s3 - It 26983 [TRAIN] [p ]: 0.8845614 |
| xmem_egoexo_tune_s3 - It 26983 [TRAIN] [ce_loss_1 ]: 0.0069732 |
| xmem_egoexo_tune_s3 - It 26983 [TRAIN] [dice_loss_1 ]: 0.2308949 |
| xmem_egoexo_tune_s3 - It 26983 [TRAIN] [ce_loss_2 ]: 0.0070168 |
| xmem_egoexo_tune_s3 - It 26983 [TRAIN] [dice_loss_2 ]: 0.2405101 |
| xmem_egoexo_tune_s3 - It 26983 [TRAIN] [ce_loss_3 ]: 0.0078191 |
| xmem_egoexo_tune_s3 - It 26983 [TRAIN] [dice_loss_3 ]: 0.2608387 |
| xmem_egoexo_tune_s3 - It 26983 [TRAIN] [iou ]: 0.6758179 |
| xmem_egoexo_tune_s3 - It 26983 [TRAIN] [total_loss ]: 0.7574735 |
| xmem_egoexo_tune_s3 - It 26983 [TRAIN] [p ]: 0.8845614 |
| xmem_egoexo_tune_s3 - It 26983 [TRAIN] [ce_loss_1 ]: 0.0069425 |
| xmem_egoexo_tune_s3 - It 26983 [TRAIN] [dice_loss_1 ]: 0.2303962 |
| xmem_egoexo_tune_s3 - It 26983 [TRAIN] [ce_loss_2 ]: 0.0071715 |
| xmem_egoexo_tune_s3 - It 26983 [TRAIN] [dice_loss_2 ]: 0.2424767 |
| xmem_egoexo_tune_s3 - It 26983 [TRAIN] [ce_loss_3 ]: 0.0079503 |
| xmem_egoexo_tune_s3 - It 26983 [TRAIN] [dice_loss_3 ]: 0.2625361 |
| xmem_egoexo_tune_s3 - It 26983 [TRAIN] [iou ]: 0.6754094 |
| Current epoch: 71 |
| Current epoch: 71 |
| Current epoch: 71 |
| Current epoch: 71 |
| Network saved to /data/work-gcp-europe-west4-a/yuqian_fu/Ego/saves_xmem2/Sep10_20.14.00_xmem_egoexo_tune_s3/Sep10_20.14.00_xmem_egoexo_tune_s3_27000.pth. |
| Network saved to /data/work-gcp-europe-west4-a/yuqian_fu/Ego/saves_xmem2/Sep10_20.14.00_xmem_egoexo_tune_s3/Sep10_20.14.00_xmem_egoexo_tune_s3_27000.pth. |
| xmem_egoexo_tune_s3 - It 27369 [TRAIN] [time ]: 17.081352 |
| xmem_egoexo_tune_s3 - It 27369 [TRAIN] [total_loss ]: 0.7305466 |
| xmem_egoexo_tune_s3 - It 27369 [TRAIN] [p ]: 0.8779994 |
| xmem_egoexo_tune_s3 - It 27369 [TRAIN] [ce_loss_1 ]: 0.0061732 |
| xmem_egoexo_tune_s3 - It 27369 [TRAIN] [dice_loss_1 ]: 0.2203456 |
| xmem_egoexo_tune_s3 - It 27369 [TRAIN] [ce_loss_2 ]: 0.0067867 |
| xmem_egoexo_tune_s3 - It 27369 [TRAIN] [dice_loss_2 ]: 0.2355684 |
| xmem_egoexo_tune_s3 - It 27369 [TRAIN] [ce_loss_3 ]: 0.0076096 |
| xmem_egoexo_tune_s3 - It 27369 [TRAIN] [dice_loss_3 ]: 0.2540628 |
| xmem_egoexo_tune_s3 - It 27369 [TRAIN] [iou ]: 0.6851359 |
| xmem_egoexo_tune_s3 - It 27369 [TRAIN] [time ]: 17.085025 |
| xmem_egoexo_tune_s3 - It 27369 [TRAIN] [total_loss ]: 0.7377119 |
| xmem_egoexo_tune_s3 - It 27369 [TRAIN] [p ]: 0.8779994 |
| xmem_egoexo_tune_s3 - It 27369 [TRAIN] [ce_loss_1 ]: 0.0063677 |
| xmem_egoexo_tune_s3 - It 27369 [TRAIN] [dice_loss_1 ]: 0.2236063 |
| xmem_egoexo_tune_s3 - It 27369 [TRAIN] [ce_loss_2 ]: 0.0068159 |
| xmem_egoexo_tune_s3 - It 27369 [TRAIN] [dice_loss_2 ]: 0.2379536 |
| xmem_egoexo_tune_s3 - It 27369 [TRAIN] [ce_loss_3 ]: 0.0075003 |
| xmem_egoexo_tune_s3 - It 27369 [TRAIN] [dice_loss_3 ]: 0.2554678 |
| xmem_egoexo_tune_s3 - It 27369 [TRAIN] [iou ]: 0.6825662 |
| Current epoch: 72 |
| Current epoch: 72 |
| Current epoch: 72 |
| Current epoch: 72 |
| xmem_egoexo_tune_s3 - It 27755 [TRAIN] [time ]: 16.814409 |
| xmem_egoexo_tune_s3 - It 27755 [TRAIN] [time ]: 16.811547 |
| xmem_egoexo_tune_s3 - It 27755 [TRAIN] [total_loss ]: 0.7259706 |
| xmem_egoexo_tune_s3 - It 27755 [TRAIN] [p ]: 0.8714374 |
| xmem_egoexo_tune_s3 - It 27755 [TRAIN] [ce_loss_1 ]: 0.0066570 |
| xmem_egoexo_tune_s3 - It 27755 [TRAIN] [dice_loss_1 ]: 0.2219487 |
| xmem_egoexo_tune_s3 - It 27755 [TRAIN] [ce_loss_2 ]: 0.0064330 |
| xmem_egoexo_tune_s3 - It 27755 [TRAIN] [dice_loss_2 ]: 0.2359448 |
| xmem_egoexo_tune_s3 - It 27755 [TRAIN] [ce_loss_3 ]: 0.0066966 |
| xmem_egoexo_tune_s3 - It 27755 [TRAIN] [dice_loss_3 ]: 0.2482903 |
| xmem_egoexo_tune_s3 - It 27755 [TRAIN] [iou ]: 0.6863460 |
| xmem_egoexo_tune_s3 - It 27755 [TRAIN] [total_loss ]: 0.7325468 |
| xmem_egoexo_tune_s3 - It 27755 [TRAIN] [p ]: 0.8714374 |
| xmem_egoexo_tune_s3 - It 27755 [TRAIN] [ce_loss_1 ]: 0.0068521 |
| xmem_egoexo_tune_s3 - It 27755 [TRAIN] [dice_loss_1 ]: 0.2234779 |
| xmem_egoexo_tune_s3 - It 27755 [TRAIN] [ce_loss_2 ]: 0.0064180 |
| xmem_egoexo_tune_s3 - It 27755 [TRAIN] [dice_loss_2 ]: 0.2390369 |
| xmem_egoexo_tune_s3 - It 27755 [TRAIN] [ce_loss_3 ]: 0.0068021 |
| xmem_egoexo_tune_s3 - It 27755 [TRAIN] [dice_loss_3 ]: 0.2499595 |
| xmem_egoexo_tune_s3 - It 27755 [TRAIN] [iou ]: 0.6846106 |
| Current epoch: 73 |
| Current epoch: 73 |
| Current epoch: 73 |
| Current epoch: 73 |
| Network saved to /data/work-gcp-europe-west4-a/yuqian_fu/Ego/saves_xmem2/Sep10_20.14.00_xmem_egoexo_tune_s3/Sep10_20.14.00_xmem_egoexo_tune_s3_28000.pth. |
| Network saved to /data/work-gcp-europe-west4-a/yuqian_fu/Ego/saves_xmem2/Sep10_20.14.00_xmem_egoexo_tune_s3/Sep10_20.14.00_xmem_egoexo_tune_s3_28000.pth. |
| Checkpoint saved to /data/work-gcp-europe-west4-a/yuqian_fu/Ego/saves_xmem2/Sep10_20.14.00_xmem_egoexo_tune_s3/Sep10_20.14.00_xmem_egoexo_tune_s3_checkpoint_28000.pth. |
| Checkpoint saved to /data/work-gcp-europe-west4-a/yuqian_fu/Ego/saves_xmem2/Sep10_20.14.00_xmem_egoexo_tune_s3/Sep10_20.14.00_xmem_egoexo_tune_s3_checkpoint_28000.pth. |
| xmem_egoexo_tune_s3 - It 28141 [TRAIN] [time ]: 17.109271 |
| xmem_egoexo_tune_s3 - It 28141 [TRAIN] [total_loss ]: 0.7368962 |
| xmem_egoexo_tune_s3 - It 28141 [TRAIN] [p ]: 0.8648754 |
| xmem_egoexo_tune_s3 - It 28141 [TRAIN] [ce_loss_1 ]: 0.0114771 |
| xmem_egoexo_tune_s3 - It 28141 [TRAIN] [dice_loss_1 ]: 0.2203911 |
| xmem_egoexo_tune_s3 - It 28141 [TRAIN] [ce_loss_2 ]: 0.0067007 |
| xmem_egoexo_tune_s3 - It 28141 [TRAIN] [dice_loss_2 ]: 0.2331992 |
| xmem_egoexo_tune_s3 - It 28141 [TRAIN] [ce_loss_3 ]: 0.0078127 |
| xmem_egoexo_tune_s3 - It 28141 [TRAIN] [dice_loss_3 ]: 0.2573152 |
| xmem_egoexo_tune_s3 - It 28141 [TRAIN] [iou ]: 0.6848143 |
| xmem_egoexo_tune_s3 - It 28141 [TRAIN] [time ]: 17.112152 |
| xmem_egoexo_tune_s3 - It 28141 [TRAIN] [total_loss ]: 0.7415327 |
| xmem_egoexo_tune_s3 - It 28141 [TRAIN] [p ]: 0.8648754 |
| xmem_egoexo_tune_s3 - It 28141 [TRAIN] [ce_loss_1 ]: 0.0115729 |
| xmem_egoexo_tune_s3 - It 28141 [TRAIN] [dice_loss_1 ]: 0.2221260 |
| xmem_egoexo_tune_s3 - It 28141 [TRAIN] [ce_loss_2 ]: 0.0067121 |
| xmem_egoexo_tune_s3 - It 28141 [TRAIN] [dice_loss_2 ]: 0.2337847 |
| xmem_egoexo_tune_s3 - It 28141 [TRAIN] [ce_loss_3 ]: 0.0081684 |
| xmem_egoexo_tune_s3 - It 28141 [TRAIN] [dice_loss_3 ]: 0.2591684 |
| xmem_egoexo_tune_s3 - It 28141 [TRAIN] [iou ]: 0.6832495 |
| Current epoch: 74 |
| Current epoch: 74 |
| Current epoch: 74 |
| Current epoch: 74 |
| xmem_egoexo_tune_s3 - It 28527 [TRAIN] [time ]: 16.900995 |
| xmem_egoexo_tune_s3 - It 28527 [TRAIN] [total_loss ]: 0.7280824 |
| xmem_egoexo_tune_s3 - It 28527 [TRAIN] [p ]: 0.8583135 |
| xmem_egoexo_tune_s3 - It 28527 [TRAIN] [ce_loss_1 ]: 0.0067516 |
| xmem_egoexo_tune_s3 - It 28527 [TRAIN] [dice_loss_1 ]: 0.2179532 |
| xmem_egoexo_tune_s3 - It 28527 [TRAIN] [ce_loss_2 ]: 0.0076983 |
| xmem_egoexo_tune_s3 - It 28527 [TRAIN] [dice_loss_2 ]: 0.2339553 |
| xmem_egoexo_tune_s3 - It 28527 [TRAIN] [ce_loss_3 ]: 0.0085826 |
| xmem_egoexo_tune_s3 - It 28527 [TRAIN] [dice_loss_3 ]: 0.2531412 |
| xmem_egoexo_tune_s3 - It 28527 [TRAIN] [iou ]: 0.6870473 |
| Current epoch: 75 |
| Current epoch: 75 |
| xmem_egoexo_tune_s3 - It 28527 [TRAIN] [time ]: 16.910204 |
| xmem_egoexo_tune_s3 - It 28527 [TRAIN] [total_loss ]: 0.7174422 |
| xmem_egoexo_tune_s3 - It 28527 [TRAIN] [p ]: 0.8583135 |
| xmem_egoexo_tune_s3 - It 28527 [TRAIN] [ce_loss_1 ]: 0.0065214 |
| xmem_egoexo_tune_s3 - It 28527 [TRAIN] [dice_loss_1 ]: 0.2140328 |
| xmem_egoexo_tune_s3 - It 28527 [TRAIN] [ce_loss_2 ]: 0.0075497 |
| xmem_egoexo_tune_s3 - It 28527 [TRAIN] [dice_loss_2 ]: 0.2308405 |
| xmem_egoexo_tune_s3 - It 28527 [TRAIN] [ce_loss_3 ]: 0.0079729 |
| xmem_egoexo_tune_s3 - It 28527 [TRAIN] [dice_loss_3 ]: 0.2505248 |
| xmem_egoexo_tune_s3 - It 28527 [TRAIN] [iou ]: 0.6909179 |
| Current epoch: 75 |
| Current epoch: 75 |
| xmem_egoexo_tune_s3 - It 28913 [TRAIN] [time ]: 16.752555 |
| xmem_egoexo_tune_s3 - It 28913 [TRAIN] [total_loss ]: 0.7282465 |
| xmem_egoexo_tune_s3 - It 28913 [TRAIN] [p ]: 0.8517515 |
| xmem_egoexo_tune_s3 - It 28913 [TRAIN] [ce_loss_1 ]: 0.0065608 |
| xmem_egoexo_tune_s3 - It 28913 [TRAIN] [dice_loss_1 ]: 0.2179933 |
| xmem_egoexo_tune_s3 - It 28913 [TRAIN] [ce_loss_2 ]: 0.0072960 |
| xmem_egoexo_tune_s3 - It 28913 [TRAIN] [dice_loss_2 ]: 0.2361968 |
| xmem_egoexo_tune_s3 - It 28913 [TRAIN] [ce_loss_3 ]: 0.0080212 |
| xmem_egoexo_tune_s3 - It 28913 [TRAIN] [dice_loss_3 ]: 0.2521782 |
| xmem_egoexo_tune_s3 - It 28913 [TRAIN] [time ]: 16.740463 |
| xmem_egoexo_tune_s3 - It 28913 [TRAIN] [iou ]: 0.6860311 |
| xmem_egoexo_tune_s3 - It 28913 [TRAIN] [total_loss ]: 0.7295508 |
| xmem_egoexo_tune_s3 - It 28913 [TRAIN] [p ]: 0.8517515 |
| xmem_egoexo_tune_s3 - It 28913 [TRAIN] [ce_loss_1 ]: 0.0065604 |
| xmem_egoexo_tune_s3 - It 28913 [TRAIN] [dice_loss_1 ]: 0.2183952 |
| xmem_egoexo_tune_s3 - It 28913 [TRAIN] [ce_loss_2 ]: 0.0072505 |
| xmem_egoexo_tune_s3 - It 28913 [TRAIN] [dice_loss_2 ]: 0.2356391 |
| xmem_egoexo_tune_s3 - It 28913 [TRAIN] [ce_loss_3 ]: 0.0079770 |
| xmem_egoexo_tune_s3 - It 28913 [TRAIN] [dice_loss_3 ]: 0.2537285 |
| xmem_egoexo_tune_s3 - It 28913 [TRAIN] [iou ]: 0.6864302 |
| Current epoch: 76 |
| Current epoch: 76 |
| Current epoch: 76 |
| Current epoch: 76 |
| Network saved to /data/work-gcp-europe-west4-a/yuqian_fu/Ego/saves_xmem2/Sep10_20.14.00_xmem_egoexo_tune_s3/Sep10_20.14.00_xmem_egoexo_tune_s3_29000.pth. |
| Network saved to /data/work-gcp-europe-west4-a/yuqian_fu/Ego/saves_xmem2/Sep10_20.14.00_xmem_egoexo_tune_s3/Sep10_20.14.00_xmem_egoexo_tune_s3_29000.pth. |
| xmem_egoexo_tune_s3 - It 29299 [TRAIN] [time ]: 16.883767 |
| xmem_egoexo_tune_s3 - It 29299 [TRAIN] [total_loss ]: 0.7177404 |
| xmem_egoexo_tune_s3 - It 29299 [TRAIN] [p ]: 0.8451895 |
| xmem_egoexo_tune_s3 - It 29299 [TRAIN] [ce_loss_1 ]: 0.0067320 |
| xmem_egoexo_tune_s3 - It 29299 [TRAIN] [dice_loss_1 ]: 0.2160204 |
| xmem_egoexo_tune_s3 - It 29299 [TRAIN] [ce_loss_2 ]: 0.0069974 |
| xmem_egoexo_tune_s3 - It 29299 [TRAIN] [dice_loss_2 ]: 0.2312829 |
| xmem_egoexo_tune_s3 - It 29299 [TRAIN] [ce_loss_3 ]: 0.0074773 |
| xmem_egoexo_tune_s3 - It 29299 [TRAIN] [dice_loss_3 ]: 0.2492301 |
| xmem_egoexo_tune_s3 - It 29299 [TRAIN] [time ]: 16.883904 |
| xmem_egoexo_tune_s3 - It 29299 [TRAIN] [iou ]: 0.6899383 |
| xmem_egoexo_tune_s3 - It 29299 [TRAIN] [total_loss ]: 0.7179332 |
| xmem_egoexo_tune_s3 - It 29299 [TRAIN] [p ]: 0.8451895 |
| xmem_egoexo_tune_s3 - It 29299 [TRAIN] [ce_loss_1 ]: 0.0066769 |
| xmem_egoexo_tune_s3 - It 29299 [TRAIN] [dice_loss_1 ]: 0.2168870 |
| xmem_egoexo_tune_s3 - It 29299 [TRAIN] [ce_loss_2 ]: 0.0069594 |
| xmem_egoexo_tune_s3 - It 29299 [TRAIN] [dice_loss_2 ]: 0.2306177 |
| xmem_egoexo_tune_s3 - It 29299 [TRAIN] [ce_loss_3 ]: 0.0075843 |
| xmem_egoexo_tune_s3 - It 29299 [TRAIN] [dice_loss_3 ]: 0.2492077 |
| xmem_egoexo_tune_s3 - It 29299 [TRAIN] [iou ]: 0.6909154 |
| Current epoch: 77 |
| Current epoch: 77 |
| Current epoch: 77 |
| Current epoch: 77 |
| xmem_egoexo_tune_s3 - It 29685 [TRAIN] [time ]: 17.076282 |
| xmem_egoexo_tune_s3 - It 29685 [TRAIN] [total_loss ]: 0.7205634 |
| xmem_egoexo_tune_s3 - It 29685 [TRAIN] [p ]: 0.8386275 |
| xmem_egoexo_tune_s3 - It 29685 [TRAIN] [ce_loss_1 ]: 0.0063977 |
| xmem_egoexo_tune_s3 - It 29685 [TRAIN] [dice_loss_1 ]: 0.2184614 |
| xmem_egoexo_tune_s3 - It 29685 [TRAIN] [ce_loss_2 ]: 0.0072168 |
| xmem_egoexo_tune_s3 - It 29685 [TRAIN] [dice_loss_2 ]: 0.2309952 |
| xmem_egoexo_tune_s3 - It 29685 [TRAIN] [ce_loss_3 ]: 0.0077749 |
| xmem_egoexo_tune_s3 - It 29685 [TRAIN] [dice_loss_3 ]: 0.2497171 |
| xmem_egoexo_tune_s3 - It 29685 [TRAIN] [iou ]: 0.6887178 |
| xmem_egoexo_tune_s3 - It 29685 [TRAIN] [time ]: 17.076308 |
| xmem_egoexo_tune_s3 - It 29685 [TRAIN] [total_loss ]: 0.7126493 |
| xmem_egoexo_tune_s3 - It 29685 [TRAIN] [p ]: 0.8386275 |
| xmem_egoexo_tune_s3 - It 29685 [TRAIN] [ce_loss_1 ]: 0.0064139 |
| xmem_egoexo_tune_s3 - It 29685 [TRAIN] [dice_loss_1 ]: 0.2159481 |
| xmem_egoexo_tune_s3 - It 29685 [TRAIN] [ce_loss_2 ]: 0.0073055 |
| xmem_egoexo_tune_s3 - It 29685 [TRAIN] [dice_loss_2 ]: 0.2290429 |
| xmem_egoexo_tune_s3 - It 29685 [TRAIN] [ce_loss_3 ]: 0.0077182 |
| xmem_egoexo_tune_s3 - It 29685 [TRAIN] [dice_loss_3 ]: 0.2462205 |
| xmem_egoexo_tune_s3 - It 29685 [TRAIN] [iou ]: 0.6918876 |
| Current epoch: 78 |
| Current epoch: 78 |
| Current epoch: 78 |
| Current epoch: 78 |
| Changing skip to cur_skip=15 |
| Changing skip to cur_skip=15 |
| Changing skip to cur_skip=15 |
| Changing skip to cur_skip=15 |
| 3090 out of 755 videos accepted in /data/work-gcp-europe-west4-a/yuqian_fu/Ego/data_segswap. |
| EgoExo dataset size: 3090 |
| Renewed with max_skip=15 |
| 3090 out of 755 videos accepted in /data/work-gcp-europe-west4-a/yuqian_fu/Ego/data_segswap. |
| EgoExo dataset size: 3090 |
| Renewed with max_skip=15 |
| 3090 out of 755 videos accepted in /data/work-gcp-europe-west4-a/yuqian_fu/Ego/data_segswap. |
| EgoExo dataset size: 3090 |
| Renewed with max_skip=15 |
| 3090 out of 755 videos accepted in /data/work-gcp-europe-west4-a/yuqian_fu/Ego/data_segswap. |
| EgoExo dataset size: 3090 |
| Renewed with max_skip=15 |
| Current epoch: 79 |
| Current epoch: 79 |
| Current epoch: 79 |
| Current epoch: 79 |
| Network saved to /data/work-gcp-europe-west4-a/yuqian_fu/Ego/saves_xmem2/Sep10_20.14.00_xmem_egoexo_tune_s3/Sep10_20.14.00_xmem_egoexo_tune_s3_30000.pth. |
| Network saved to /data/work-gcp-europe-west4-a/yuqian_fu/Ego/saves_xmem2/Sep10_20.14.00_xmem_egoexo_tune_s3/Sep10_20.14.00_xmem_egoexo_tune_s3_30000.pth. |
| Checkpoint saved to /data/work-gcp-europe-west4-a/yuqian_fu/Ego/saves_xmem2/Sep10_20.14.00_xmem_egoexo_tune_s3/Sep10_20.14.00_xmem_egoexo_tune_s3_checkpoint_30000.pth. |
| Checkpoint saved to /data/work-gcp-europe-west4-a/yuqian_fu/Ego/saves_xmem2/Sep10_20.14.00_xmem_egoexo_tune_s3/Sep10_20.14.00_xmem_egoexo_tune_s3_checkpoint_30000.pth. |
| xmem_egoexo_tune_s3 - It 30385 [TRAIN] [time ]: 34.591435 |
| xmem_egoexo_tune_s3 - It 30385 [TRAIN] [time ]: 34.591535 |
| xmem_egoexo_tune_s3 - It 30385 [TRAIN] [total_loss ]: 0.7203772 |
| xmem_egoexo_tune_s3 - It 30385 [TRAIN] [p ]: 0.8293964 |
| xmem_egoexo_tune_s3 - It 30385 [TRAIN] [ce_loss_1 ]: 0.0064424 |
| xmem_egoexo_tune_s3 - It 30385 [TRAIN] [dice_loss_1 ]: 0.2180129 |
| xmem_egoexo_tune_s3 - It 30385 [TRAIN] [ce_loss_2 ]: 0.0069214 |
| xmem_egoexo_tune_s3 - It 30385 [TRAIN] [dice_loss_2 ]: 0.2300248 |
| xmem_egoexo_tune_s3 - It 30385 [TRAIN] [ce_loss_3 ]: 0.0081353 |
| xmem_egoexo_tune_s3 - It 30385 [TRAIN] [dice_loss_3 ]: 0.2508402 |
| xmem_egoexo_tune_s3 - It 30385 [TRAIN] [iou ]: 0.6894856 |
| xmem_egoexo_tune_s3 - It 30385 [TRAIN] [total_loss ]: 0.7256650 |
| xmem_egoexo_tune_s3 - It 30385 [TRAIN] [p ]: 0.8293964 |
| xmem_egoexo_tune_s3 - It 30385 [TRAIN] [ce_loss_1 ]: 0.0065485 |
| xmem_egoexo_tune_s3 - It 30385 [TRAIN] [dice_loss_1 ]: 0.2188454 |
| xmem_egoexo_tune_s3 - It 30385 [TRAIN] [ce_loss_2 ]: 0.0069807 |
| xmem_egoexo_tune_s3 - It 30385 [TRAIN] [dice_loss_2 ]: 0.2312897 |
| xmem_egoexo_tune_s3 - It 30385 [TRAIN] [ce_loss_3 ]: 0.0082924 |
| xmem_egoexo_tune_s3 - It 30385 [TRAIN] [dice_loss_3 ]: 0.2537081 |
| xmem_egoexo_tune_s3 - It 30385 [TRAIN] [iou ]: 0.6874651 |
| Current epoch: 80 |
| Current epoch: 80 |
| Current epoch: 80 |
| Current epoch: 80 |
| xmem_egoexo_tune_s3 - It 30771 [TRAIN] [time ]: 17.003964 |
| xmem_egoexo_tune_s3 - It 30771 [TRAIN] [time ]: 17.003968 |
| xmem_egoexo_tune_s3 - It 30771 [TRAIN] [total_loss ]: 0.7388233 |
| xmem_egoexo_tune_s3 - It 30771 [TRAIN] [p ]: 0.8201655 |
| xmem_egoexo_tune_s3 - It 30771 [TRAIN] [total_loss ]: 0.7405031 |
| xmem_egoexo_tune_s3 - It 30771 [TRAIN] [ce_loss_1 ]: 0.0070559 |
| xmem_egoexo_tune_s3 - It 30771 [TRAIN] [p ]: 0.8201655 |
| xmem_egoexo_tune_s3 - It 30771 [TRAIN] [dice_loss_1 ]: 0.2242802 |
| xmem_egoexo_tune_s3 - It 30771 [TRAIN] [ce_loss_1 ]: 0.0069919 |
| xmem_egoexo_tune_s3 - It 30771 [TRAIN] [ce_loss_2 ]: 0.0076611 |
| xmem_egoexo_tune_s3 - It 30771 [TRAIN] [dice_loss_1 ]: 0.2237696 |
| xmem_egoexo_tune_s3 - It 30771 [TRAIN] [dice_loss_2 ]: 0.2342321 |
| xmem_egoexo_tune_s3 - It 30771 [TRAIN] [ce_loss_2 ]: 0.0075826 |
| xmem_egoexo_tune_s3 - It 30771 [TRAIN] [ce_loss_3 ]: 0.0081660 |
| xmem_egoexo_tune_s3 - It 30771 [TRAIN] [dice_loss_2 ]: 0.2373372 |
| xmem_egoexo_tune_s3 - It 30771 [TRAIN] [dice_loss_3 ]: 0.2574277 |
| xmem_egoexo_tune_s3 - It 30771 [TRAIN] [ce_loss_3 ]: 0.0081471 |
| xmem_egoexo_tune_s3 - It 30771 [TRAIN] [iou ]: 0.6831134 |
| xmem_egoexo_tune_s3 - It 30771 [TRAIN] [dice_loss_3 ]: 0.2566745 |
| xmem_egoexo_tune_s3 - It 30771 [TRAIN] [iou ]: 0.6825498 |
| Current epoch: 81Current epoch: 81 |
| |
| Current epoch: 81 |
| Current epoch: 81 |
| Network saved to /data/work-gcp-europe-west4-a/yuqian_fu/Ego/saves_xmem2/Sep10_20.14.00_xmem_egoexo_tune_s3/Sep10_20.14.00_xmem_egoexo_tune_s3_31000.pth. |
| Network saved to /data/work-gcp-europe-west4-a/yuqian_fu/Ego/saves_xmem2/Sep10_20.14.00_xmem_egoexo_tune_s3/Sep10_20.14.00_xmem_egoexo_tune_s3_31000.pth. |
| xmem_egoexo_tune_s3 - It 31157 [TRAIN] [time ]: 16.932354 |
| xmem_egoexo_tune_s3 - It 31157 [TRAIN] [time ]: 16.932367 |
| xmem_egoexo_tune_s3 - It 31157 [TRAIN] [total_loss ]: 0.7370492 |
| xmem_egoexo_tune_s3 - It 31157 [TRAIN] [p ]: 0.8136035 |
| xmem_egoexo_tune_s3 - It 31157 [TRAIN] [total_loss ]: 0.7262853 |
| xmem_egoexo_tune_s3 - It 31157 [TRAIN] [p ]: 0.8136035 |
| xmem_egoexo_tune_s3 - It 31157 [TRAIN] [ce_loss_1 ]: 0.0063331 |
| xmem_egoexo_tune_s3 - It 31157 [TRAIN] [dice_loss_1 ]: 0.2172900 |
| xmem_egoexo_tune_s3 - It 31157 [TRAIN] [ce_loss_2 ]: 0.0074341 |
| xmem_egoexo_tune_s3 - It 31157 [TRAIN] [dice_loss_2 ]: 0.2343358 |
| xmem_egoexo_tune_s3 - It 31157 [TRAIN] [ce_loss_3 ]: 0.0079044 |
| xmem_egoexo_tune_s3 - It 31157 [TRAIN] [dice_loss_3 ]: 0.2529877 |
| xmem_egoexo_tune_s3 - It 31157 [TRAIN] [ce_loss_1 ]: 0.0067603 |
| xmem_egoexo_tune_s3 - It 31157 [TRAIN] [iou ]: 0.6873129 |
| xmem_egoexo_tune_s3 - It 31157 [TRAIN] [dice_loss_1 ]: 0.2198815 |
| xmem_egoexo_tune_s3 - It 31157 [TRAIN] [ce_loss_2 ]: 0.0077065 |
| xmem_egoexo_tune_s3 - It 31157 [TRAIN] [dice_loss_2 ]: 0.2361822 |
| xmem_egoexo_tune_s3 - It 31157 [TRAIN] [ce_loss_3 ]: 0.0082516 |
| xmem_egoexo_tune_s3 - It 31157 [TRAIN] [dice_loss_3 ]: 0.2582669 |
| xmem_egoexo_tune_s3 - It 31157 [TRAIN] [iou ]: 0.6830811 |
| Current epoch: 82 |
| Current epoch: 82 |
| Current epoch: 82 |
| Current epoch: 82 |
| xmem_egoexo_tune_s3 - It 31543 [TRAIN] [time ]: 16.906514 |
| xmem_egoexo_tune_s3 - It 31543 [TRAIN] [total_loss ]: 0.7328574 |
| xmem_egoexo_tune_s3 - It 31543 [TRAIN] [p ]: 0.8070415 |
| xmem_egoexo_tune_s3 - It 31543 [TRAIN] [ce_loss_1 ]: 0.0070210 |
| xmem_egoexo_tune_s3 - It 31543 [TRAIN] [dice_loss_1 ]: 0.2191975 |
| xmem_egoexo_tune_s3 - It 31543 [TRAIN] [ce_loss_2 ]: 0.0072246 |
| xmem_egoexo_tune_s3 - It 31543 [TRAIN] [dice_loss_2 ]: 0.2396803 |
| xmem_egoexo_tune_s3 - It 31543 [TRAIN] [ce_loss_3 ]: 0.0079302 |
| xmem_egoexo_tune_s3 - It 31543 [TRAIN] [dice_loss_3 ]: 0.2518034 |
| xmem_egoexo_tune_s3 - It 31543 [TRAIN] [iou ]: 0.6847114 |
| xmem_egoexo_tune_s3 - It 31543 [TRAIN] [time ]: 16.909655 |
| xmem_egoexo_tune_s3 - It 31543 [TRAIN] [total_loss ]: 0.7253355 |
| xmem_egoexo_tune_s3 - It 31543 [TRAIN] [p ]: 0.8070415 |
| xmem_egoexo_tune_s3 - It 31543 [TRAIN] [ce_loss_1 ]: 0.0069149 |
| xmem_egoexo_tune_s3 - It 31543 [TRAIN] [dice_loss_1 ]: 0.2176194 |
| xmem_egoexo_tune_s3 - It 31543 [TRAIN] [ce_loss_2 ]: 0.0070192 |
| xmem_egoexo_tune_s3 - It 31543 [TRAIN] [dice_loss_2 ]: 0.2354178 |
| xmem_egoexo_tune_s3 - It 31543 [TRAIN] [ce_loss_3 ]: 0.0080274 |
| xmem_egoexo_tune_s3 - It 31543 [TRAIN] [dice_loss_3 ]: 0.2503367 |
| xmem_egoexo_tune_s3 - It 31543 [TRAIN] [iou ]: 0.6864065 |
| Current epoch: 83 |
| Current epoch: 83 |
| Current epoch: 83 |
| Current epoch: 83 |
| xmem_egoexo_tune_s3 - It 31929 [TRAIN] [time ]: 16.820898 |
| xmem_egoexo_tune_s3 - It 31929 [TRAIN] [total_loss ]: 0.7292677 |
| xmem_egoexo_tune_s3 - It 31929 [TRAIN] [p ]: 0.8004794 |
| xmem_egoexo_tune_s3 - It 31929 [TRAIN] [ce_loss_1 ]: 0.0071211 |
| xmem_egoexo_tune_s3 - It 31929 [TRAIN] [dice_loss_1 ]: 0.2245860 |
| xmem_egoexo_tune_s3 - It 31929 [TRAIN] [ce_loss_2 ]: 0.0075658 |
| xmem_egoexo_tune_s3 - It 31929 [TRAIN] [dice_loss_2 ]: 0.2363723 |
| xmem_egoexo_tune_s3 - It 31929 [TRAIN] [ce_loss_3 ]: 0.0086829 |
| xmem_egoexo_tune_s3 - It 31929 [TRAIN] [dice_loss_3 ]: 0.2449393 |
| xmem_egoexo_tune_s3 - It 31929 [TRAIN] [iou ]: 0.6855429 |
| xmem_egoexo_tune_s3 - It 31929 [TRAIN] [time ]: 16.826633 |
| xmem_egoexo_tune_s3 - It 31929 [TRAIN] [total_loss ]: 0.7367951 |
| xmem_egoexo_tune_s3 - It 31929 [TRAIN] [p ]: 0.8004794 |
| xmem_egoexo_tune_s3 - It 31929 [TRAIN] [ce_loss_1 ]: 0.0074273 |
| xmem_egoexo_tune_s3 - It 31929 [TRAIN] [dice_loss_1 ]: 0.2262122 |
| xmem_egoexo_tune_s3 - It 31929 [TRAIN] [ce_loss_2 ]: 0.0078517 |
| xmem_egoexo_tune_s3 - It 31929 [TRAIN] [dice_loss_2 ]: 0.2367237 |
| xmem_egoexo_tune_s3 - It 31929 [TRAIN] [ce_loss_3 ]: 0.0090400 |
| xmem_egoexo_tune_s3 - It 31929 [TRAIN] [dice_loss_3 ]: 0.2495399 |
| xmem_egoexo_tune_s3 - It 31929 [TRAIN] [iou ]: 0.6828955 |
| Current epoch: 84 |
| Current epoch: 84 |
| Current epoch: 84 |
| Current epoch: 84 |
| Network saved to /data/work-gcp-europe-west4-a/yuqian_fu/Ego/saves_xmem2/Sep10_20.14.00_xmem_egoexo_tune_s3/Sep10_20.14.00_xmem_egoexo_tune_s3_32000.pth. |
| Network saved to /data/work-gcp-europe-west4-a/yuqian_fu/Ego/saves_xmem2/Sep10_20.14.00_xmem_egoexo_tune_s3/Sep10_20.14.00_xmem_egoexo_tune_s3_32000.pth. |
| Checkpoint saved to /data/work-gcp-europe-west4-a/yuqian_fu/Ego/saves_xmem2/Sep10_20.14.00_xmem_egoexo_tune_s3/Sep10_20.14.00_xmem_egoexo_tune_s3_checkpoint_32000.pth. |
| Checkpoint saved to /data/work-gcp-europe-west4-a/yuqian_fu/Ego/saves_xmem2/Sep10_20.14.00_xmem_egoexo_tune_s3/Sep10_20.14.00_xmem_egoexo_tune_s3_checkpoint_32000.pth. |
| xmem_egoexo_tune_s3 - It 32315 [TRAIN] [time ]: 17.253016 |
| xmem_egoexo_tune_s3 - It 32315 [TRAIN] [time ]: 17.255813 |
| xmem_egoexo_tune_s3 - It 32315 [TRAIN] [total_loss ]: 0.7452034 |
| xmem_egoexo_tune_s3 - It 32315 [TRAIN] [p ]: 0.7939174 |
| xmem_egoexo_tune_s3 - It 32315 [TRAIN] [ce_loss_1 ]: 0.0074952 |
| xmem_egoexo_tune_s3 - It 32315 [TRAIN] [dice_loss_1 ]: 0.2239264 |
| xmem_egoexo_tune_s3 - It 32315 [TRAIN] [ce_loss_2 ]: 0.0075536 |
| xmem_egoexo_tune_s3 - It 32315 [TRAIN] [dice_loss_2 ]: 0.2361655 |
| xmem_egoexo_tune_s3 - It 32315 [TRAIN] [ce_loss_3 ]: 0.0136193 |
| xmem_egoexo_tune_s3 - It 32315 [TRAIN] [dice_loss_3 ]: 0.2564431 |
| xmem_egoexo_tune_s3 - It 32315 [TRAIN] [iou ]: 0.6834129 |
| xmem_egoexo_tune_s3 - It 32315 [TRAIN] [total_loss ]: 0.7349876 |
| xmem_egoexo_tune_s3 - It 32315 [TRAIN] [p ]: 0.7939174 |
| xmem_egoexo_tune_s3 - It 32315 [TRAIN] [ce_loss_1 ]: 0.0072449 |
| xmem_egoexo_tune_s3 - It 32315 [TRAIN] [dice_loss_1 ]: 0.2217353 |
| xmem_egoexo_tune_s3 - It 32315 [TRAIN] [ce_loss_2 ]: 0.0073934 |
| xmem_egoexo_tune_s3 - It 32315 [TRAIN] [dice_loss_2 ]: 0.2329052 |
| xmem_egoexo_tune_s3 - It 32315 [TRAIN] [ce_loss_3 ]: 0.0135323 |
| xmem_egoexo_tune_s3 - It 32315 [TRAIN] [dice_loss_3 ]: 0.2521763 |
| xmem_egoexo_tune_s3 - It 32315 [TRAIN] [iou ]: 0.6864967 |
| Current epoch: 85 |
| Current epoch: 85 |
| Current epoch: 85 |
| Current epoch: 85 |
| xmem_egoexo_tune_s3 - It 32701 [TRAIN] [time ]: 16.889503 |
| xmem_egoexo_tune_s3 - It 32701 [TRAIN] [total_loss ]: 0.7167245 |
| xmem_egoexo_tune_s3 - It 32701 [TRAIN] [p ]: 0.7873554 |
| xmem_egoexo_tune_s3 - It 32701 [TRAIN] [ce_loss_1 ]: 0.0071121 |
| xmem_egoexo_tune_s3 - It 32701 [TRAIN] [dice_loss_1 ]: 0.2164873 |
| xmem_egoexo_tune_s3 - It 32701 [TRAIN] [ce_loss_2 ]: 0.0073439 |
| xmem_egoexo_tune_s3 - It 32701 [TRAIN] [dice_loss_2 ]: 0.2286943 |
| xmem_egoexo_tune_s3 - It 32701 [TRAIN] [ce_loss_3 ]: 0.0078112 |
| xmem_egoexo_tune_s3 - It 32701 [TRAIN] [dice_loss_3 ]: 0.2492753 |
| xmem_egoexo_tune_s3 - It 32701 [TRAIN] [iou ]: 0.6903032 |
| xmem_egoexo_tune_s3 - It 32701 [TRAIN] [time ]: 16.892296 |
| xmem_egoexo_tune_s3 - It 32701 [TRAIN] [total_loss ]: 0.7134227 |
| xmem_egoexo_tune_s3 - It 32701 [TRAIN] [p ]: 0.7873554 |
| xmem_egoexo_tune_s3 - It 32701 [TRAIN] [ce_loss_1 ]: 0.0070659 |
| xmem_egoexo_tune_s3 - It 32701 [TRAIN] [dice_loss_1 ]: 0.2167871 |
| xmem_egoexo_tune_s3 - It 32701 [TRAIN] [ce_loss_2 ]: 0.0071261 |
| xmem_egoexo_tune_s3 - It 32701 [TRAIN] [dice_loss_2 ]: 0.2261749 |
| xmem_egoexo_tune_s3 - It 32701 [TRAIN] [ce_loss_3 ]: 0.0074379 |
| xmem_egoexo_tune_s3 - It 32701 [TRAIN] [dice_loss_3 ]: 0.2488306 |
| xmem_egoexo_tune_s3 - It 32701 [TRAIN] [iou ]: 0.6912838 |
| Current epoch: 86 |
| Current epoch: 86 |
| Current epoch: 86 |
| Current epoch: 86 |
| Network saved to /data/work-gcp-europe-west4-a/yuqian_fu/Ego/saves_xmem2/Sep10_20.14.00_xmem_egoexo_tune_s3/Sep10_20.14.00_xmem_egoexo_tune_s3_33000.pth. |
| Network saved to /data/work-gcp-europe-west4-a/yuqian_fu/Ego/saves_xmem2/Sep10_20.14.00_xmem_egoexo_tune_s3/Sep10_20.14.00_xmem_egoexo_tune_s3_33000.pth. |
| xmem_egoexo_tune_s3 - It 33087 [TRAIN] [time ]: 16.884986 |
| xmem_egoexo_tune_s3 - It 33087 [TRAIN] [total_loss ]: 0.7188795 |
| xmem_egoexo_tune_s3 - It 33087 [TRAIN] [p ]: 0.7807934 |
| xmem_egoexo_tune_s3 - It 33087 [TRAIN] [ce_loss_1 ]: 0.0067885 |
| xmem_egoexo_tune_s3 - It 33087 [TRAIN] [dice_loss_1 ]: 0.2202007 |
| xmem_egoexo_tune_s3 - It 33087 [TRAIN] [ce_loss_2 ]: 0.0071143 |
| xmem_egoexo_tune_s3 - It 33087 [TRAIN] [dice_loss_2 ]: 0.2259153 |
| xmem_egoexo_tune_s3 - It 33087 [TRAIN] [ce_loss_3 ]: 0.0134524 |
| xmem_egoexo_tune_s3 - It 33087 [TRAIN] [dice_loss_3 ]: 0.2454080 |
| xmem_egoexo_tune_s3 - It 33087 [TRAIN] [iou ]: 0.6919472 |
| xmem_egoexo_tune_s3 - It 33087 [TRAIN] [time ]: 16.885021 |
| xmem_egoexo_tune_s3 - It 33087 [TRAIN] [total_loss ]: 0.7144316 |
| xmem_egoexo_tune_s3 - It 33087 [TRAIN] [p ]: 0.7807934 |
| xmem_egoexo_tune_s3 - It 33087 [TRAIN] [ce_loss_1 ]: 0.0069258 |
| xmem_egoexo_tune_s3 - It 33087 [TRAIN] [dice_loss_1 ]: 0.2196897 |
| xmem_egoexo_tune_s3 - It 33087 [TRAIN] [ce_loss_2 ]: 0.0070720 |
| xmem_egoexo_tune_s3 - It 33087 [TRAIN] [dice_loss_2 ]: 0.2245064 |
| xmem_egoexo_tune_s3 - It 33087 [TRAIN] [ce_loss_3 ]: 0.0133706 |
| xmem_egoexo_tune_s3 - It 33087 [TRAIN] [dice_loss_3 ]: 0.2428668 |
| xmem_egoexo_tune_s3 - It 33087 [TRAIN] [iou ]: 0.6929430 |
| Current epoch: 87 |
| Current epoch: 87 |
| Current epoch: 87 |
| Current epoch: 87 |
| xmem_egoexo_tune_s3 - It 33473 [TRAIN] [time ]: 16.879993 |
| xmem_egoexo_tune_s3 - It 33473 [TRAIN] [time ]: 16.883088 |
| xmem_egoexo_tune_s3 - It 33473 [TRAIN] [total_loss ]: 0.7198020 |
| xmem_egoexo_tune_s3 - It 33473 [TRAIN] [total_loss ]: 0.7195267 |
| xmem_egoexo_tune_s3 - It 33473 [TRAIN] [p ]: 0.7742314 |
| xmem_egoexo_tune_s3 - It 33473 [TRAIN] [p ]: 0.7742314 |
| xmem_egoexo_tune_s3 - It 33473 [TRAIN] [ce_loss_1 ]: 0.0068691 |
| xmem_egoexo_tune_s3 - It 33473 [TRAIN] [ce_loss_1 ]: 0.0070196 |
| xmem_egoexo_tune_s3 - It 33473 [TRAIN] [dice_loss_1 ]: 0.2176893 |
| xmem_egoexo_tune_s3 - It 33473 [TRAIN] [dice_loss_1 ]: 0.2178285 |
| xmem_egoexo_tune_s3 - It 33473 [TRAIN] [ce_loss_2 ]: 0.0074412 |
| xmem_egoexo_tune_s3 - It 33473 [TRAIN] [ce_loss_2 ]: 0.0075750 |
| xmem_egoexo_tune_s3 - It 33473 [TRAIN] [dice_loss_2 ]: 0.2289078 |
| xmem_egoexo_tune_s3 - It 33473 [TRAIN] [dice_loss_2 ]: 0.2301875 |
| xmem_egoexo_tune_s3 - It 33473 [TRAIN] [ce_loss_3 ]: 0.0082084 |
| xmem_egoexo_tune_s3 - It 33473 [TRAIN] [ce_loss_3 ]: 0.0079552 |
| xmem_egoexo_tune_s3 - It 33473 [TRAIN] [dice_loss_3 ]: 0.2506859 |
| xmem_egoexo_tune_s3 - It 33473 [TRAIN] [dice_loss_3 ]: 0.2489606 |
| xmem_egoexo_tune_s3 - It 33473 [TRAIN] [iou ]: 0.6902055 |
| xmem_egoexo_tune_s3 - It 33473 [TRAIN] [iou ]: 0.6897718 |
| Current epoch: 88 |
| Current epoch: 88 |
| Current epoch: 88 |
| Current epoch: 88 |
| xmem_egoexo_tune_s3 - It 33859 [TRAIN] [time ]: 16.894239 |
| xmem_egoexo_tune_s3 - It 33859 [TRAIN] [time ]: 16.895670 |
| xmem_egoexo_tune_s3 - It 33859 [TRAIN] [total_loss ]: 0.7230765 |
| xmem_egoexo_tune_s3 - It 33859 [TRAIN] [p ]: 0.7676694 |
| xmem_egoexo_tune_s3 - It 33859 [TRAIN] [ce_loss_1 ]: 0.0066122 |
| xmem_egoexo_tune_s3 - It 33859 [TRAIN] [dice_loss_1 ]: 0.2204187 |
| xmem_egoexo_tune_s3 - It 33859 [TRAIN] [ce_loss_2 ]: 0.0072593 |
| xmem_egoexo_tune_s3 - It 33859 [TRAIN] [dice_loss_2 ]: 0.2297100 |
| xmem_egoexo_tune_s3 - It 33859 [TRAIN] [ce_loss_3 ]: 0.0083074 |
| xmem_egoexo_tune_s3 - It 33859 [TRAIN] [dice_loss_3 ]: 0.2507687 |
| xmem_egoexo_tune_s3 - It 33859 [TRAIN] [iou ]: 0.6885107 |
| xmem_egoexo_tune_s3 - It 33859 [TRAIN] [total_loss ]: 0.7248624 |
| xmem_egoexo_tune_s3 - It 33859 [TRAIN] [p ]: 0.7676694 |
| xmem_egoexo_tune_s3 - It 33859 [TRAIN] [ce_loss_1 ]: 0.0067556 |
| xmem_egoexo_tune_s3 - It 33859 [TRAIN] [dice_loss_1 ]: 0.2195470 |
| xmem_egoexo_tune_s3 - It 33859 [TRAIN] [ce_loss_2 ]: 0.0075319 |
| xmem_egoexo_tune_s3 - It 33859 [TRAIN] [dice_loss_2 ]: 0.2314421 |
| xmem_egoexo_tune_s3 - It 33859 [TRAIN] [ce_loss_3 ]: 0.0085812 |
| xmem_egoexo_tune_s3 - It 33859 [TRAIN] [dice_loss_3 ]: 0.2510045 |
| xmem_egoexo_tune_s3 - It 33859 [TRAIN] [iou ]: 0.6878029 |
| Current epoch: 89 |
| Current epoch: 89 |
| Current epoch: 89 |
| Current epoch: 89 |
| Network saved to /data/work-gcp-europe-west4-a/yuqian_fu/Ego/saves_xmem2/Sep10_20.14.00_xmem_egoexo_tune_s3/Sep10_20.14.00_xmem_egoexo_tune_s3_34000.pth. |
| Network saved to /data/work-gcp-europe-west4-a/yuqian_fu/Ego/saves_xmem2/Sep10_20.14.00_xmem_egoexo_tune_s3/Sep10_20.14.00_xmem_egoexo_tune_s3_34000.pth. |
| Checkpoint saved to /data/work-gcp-europe-west4-a/yuqian_fu/Ego/saves_xmem2/Sep10_20.14.00_xmem_egoexo_tune_s3/Sep10_20.14.00_xmem_egoexo_tune_s3_checkpoint_34000.pth. |
| Checkpoint saved to /data/work-gcp-europe-west4-a/yuqian_fu/Ego/saves_xmem2/Sep10_20.14.00_xmem_egoexo_tune_s3/Sep10_20.14.00_xmem_egoexo_tune_s3_checkpoint_34000.pth. |
| xmem_egoexo_tune_s3 - It 34245 [TRAIN] [time ]: 17.057199 |
| xmem_egoexo_tune_s3 - It 34245 [TRAIN] [time ]: 17.061018 |
| xmem_egoexo_tune_s3 - It 34245 [TRAIN] [total_loss ]: 0.7128356 |
| xmem_egoexo_tune_s3 - It 34245 [TRAIN] [p ]: 0.7611075 |
| xmem_egoexo_tune_s3 - It 34245 [TRAIN] [ce_loss_1 ]: 0.0071037 |
| xmem_egoexo_tune_s3 - It 34245 [TRAIN] [dice_loss_1 ]: 0.2167633 |
| xmem_egoexo_tune_s3 - It 34245 [TRAIN] [ce_loss_2 ]: 0.0076147 |
| xmem_egoexo_tune_s3 - It 34245 [TRAIN] [dice_loss_2 ]: 0.2274196 |
| xmem_egoexo_tune_s3 - It 34245 [TRAIN] [ce_loss_3 ]: 0.0081923 |
| xmem_egoexo_tune_s3 - It 34245 [TRAIN] [dice_loss_3 ]: 0.2457417 |
| xmem_egoexo_tune_s3 - It 34245 [TRAIN] [iou ]: 0.6928477 |
| xmem_egoexo_tune_s3 - It 34245 [TRAIN] [total_loss ]: 0.7139327 |
| xmem_egoexo_tune_s3 - It 34245 [TRAIN] [p ]: 0.7611075 |
| xmem_egoexo_tune_s3 - It 34245 [TRAIN] [ce_loss_1 ]: 0.0073022 |
| xmem_egoexo_tune_s3 - It 34245 [TRAIN] [dice_loss_1 ]: 0.2159749 |
| xmem_egoexo_tune_s3 - It 34245 [TRAIN] [ce_loss_2 ]: 0.0078741 |
| xmem_egoexo_tune_s3 - It 34245 [TRAIN] [dice_loss_2 ]: 0.2273064 |
| xmem_egoexo_tune_s3 - It 34245 [TRAIN] [ce_loss_3 ]: 0.0088552 |
| xmem_egoexo_tune_s3 - It 34245 [TRAIN] [dice_loss_3 ]: 0.2466197 |
| xmem_egoexo_tune_s3 - It 34245 [TRAIN] [iou ]: 0.6927101 |
| Current epoch: 90 |
| Current epoch: 90 |
| Current epoch: 90 |
| Current epoch: 90 |
| xmem_egoexo_tune_s3 - It 34631 [TRAIN] [time ]: 17.081700 |
| xmem_egoexo_tune_s3 - It 34631 [TRAIN] [total_loss ]: 0.7170739 |
| xmem_egoexo_tune_s3 - It 34631 [TRAIN] [p ]: 0.7545455 |
| xmem_egoexo_tune_s3 - It 34631 [TRAIN] [ce_loss_1 ]: 0.0065594 |
| xmem_egoexo_tune_s3 - It 34631 [TRAIN] [dice_loss_1 ]: 0.2149927 |
| xmem_egoexo_tune_s3 - It 34631 [TRAIN] [ce_loss_2 ]: 0.0073524 |
| xmem_egoexo_tune_s3 - It 34631 [TRAIN] [dice_loss_2 ]: 0.2312512 |
| xmem_egoexo_tune_s3 - It 34631 [TRAIN] [ce_loss_3 ]: 0.0079484 |
| xmem_egoexo_tune_s3 - It 34631 [TRAIN] [dice_loss_3 ]: 0.2489694 |
| xmem_egoexo_tune_s3 - It 34631 [TRAIN] [time ]: 17.084083 |
| xmem_egoexo_tune_s3 - It 34631 [TRAIN] [iou ]: 0.6907858 |
| xmem_egoexo_tune_s3 - It 34631 [TRAIN] [total_loss ]: 0.7363997 |
| xmem_egoexo_tune_s3 - It 34631 [TRAIN] [p ]: 0.7545455 |
| xmem_egoexo_tune_s3 - It 34631 [TRAIN] [ce_loss_1 ]: 0.0070197 |
| xmem_egoexo_tune_s3 - It 34631 [TRAIN] [dice_loss_1 ]: 0.2186283 |
| xmem_egoexo_tune_s3 - It 34631 [TRAIN] [ce_loss_2 ]: 0.0081672 |
| xmem_egoexo_tune_s3 - It 34631 [TRAIN] [dice_loss_2 ]: 0.2372156 |
| xmem_egoexo_tune_s3 - It 34631 [TRAIN] [ce_loss_3 ]: 0.0089045 |
| xmem_egoexo_tune_s3 - It 34631 [TRAIN] [dice_loss_3 ]: 0.2564642 |
| xmem_egoexo_tune_s3 - It 34631 [TRAIN] [iou ]: 0.6846171 |
| Current epoch: 91 |
| Current epoch: 91 |
| Current epoch: 91 |
| Current epoch: 91 |
| Network saved to /data/work-gcp-europe-west4-a/yuqian_fu/Ego/saves_xmem2/Sep10_20.14.00_xmem_egoexo_tune_s3/Sep10_20.14.00_xmem_egoexo_tune_s3_35000.pth. |
| Network saved to /data/work-gcp-europe-west4-a/yuqian_fu/Ego/saves_xmem2/Sep10_20.14.00_xmem_egoexo_tune_s3/Sep10_20.14.00_xmem_egoexo_tune_s3_35000.pth. |
| xmem_egoexo_tune_s3 - It 35017 [TRAIN] [time ]: 16.896046 |
| xmem_egoexo_tune_s3 - It 35017 [TRAIN] [total_loss ]: 0.7184497 |
| xmem_egoexo_tune_s3 - It 35017 [TRAIN] [p ]: 0.7479835 |
| xmem_egoexo_tune_s3 - It 35017 [TRAIN] [ce_loss_1 ]: 0.0121178 |
| xmem_egoexo_tune_s3 - It 35017 [TRAIN] [dice_loss_1 ]: 0.2133018 |
| xmem_egoexo_tune_s3 - It 35017 [TRAIN] [ce_loss_2 ]: 0.0077874 |
| xmem_egoexo_tune_s3 - It 35017 [TRAIN] [dice_loss_2 ]: 0.2312983 |
| xmem_egoexo_tune_s3 - It 35017 [TRAIN] [ce_loss_3 ]: 0.0086422 |
| xmem_egoexo_tune_s3 - It 35017 [TRAIN] [dice_loss_3 ]: 0.2453018 |
| xmem_egoexo_tune_s3 - It 35017 [TRAIN] [iou ]: 0.6921067 |
| Current epoch: 92 |
| xmem_egoexo_tune_s3 - It 35017 [TRAIN] [time ]: 16.899202 |
| xmem_egoexo_tune_s3 - It 35017 [TRAIN] [total_loss ]: 0.7204416 |
| xmem_egoexo_tune_s3 - It 35017 [TRAIN] [p ]: 0.7479835 |
| xmem_egoexo_tune_s3 - It 35017 [TRAIN] [ce_loss_1 ]: 0.0120012 |
| xmem_egoexo_tune_s3 - It 35017 [TRAIN] [dice_loss_1 ]: 0.2132704 |
| xmem_egoexo_tune_s3 - It 35017 [TRAIN] [ce_loss_2 ]: 0.0077901 |
| xmem_egoexo_tune_s3 - It 35017 [TRAIN] [dice_loss_2 ]: 0.2301416 |
| xmem_egoexo_tune_s3 - It 35017 [TRAIN] [ce_loss_3 ]: 0.0087225 |
| xmem_egoexo_tune_s3 - It 35017 [TRAIN] [dice_loss_3 ]: 0.2485154 |
| xmem_egoexo_tune_s3 - It 35017 [TRAIN] [iou ]: 0.6906215 |
| Current epoch: 92 |
| Current epoch: 92 |
| Current epoch: 92 |
| xmem_egoexo_tune_s3 - It 35403 [TRAIN] [time ]: 16.803997 |
| xmem_egoexo_tune_s3 - It 35403 [TRAIN] [time ]: 16.801140 |
| xmem_egoexo_tune_s3 - It 35403 [TRAIN] [total_loss ]: 0.7139157 |
| xmem_egoexo_tune_s3 - It 35403 [TRAIN] [p ]: 0.7414215 |
| xmem_egoexo_tune_s3 - It 35403 [TRAIN] [ce_loss_1 ]: 0.0071821 |
| xmem_egoexo_tune_s3 - It 35403 [TRAIN] [dice_loss_1 ]: 0.2131087 |
| xmem_egoexo_tune_s3 - It 35403 [TRAIN] [ce_loss_2 ]: 0.0077725 |
| xmem_egoexo_tune_s3 - It 35403 [TRAIN] [dice_loss_2 ]: 0.2290824 |
| xmem_egoexo_tune_s3 - It 35403 [TRAIN] [ce_loss_3 ]: 0.0083091 |
| xmem_egoexo_tune_s3 - It 35403 [TRAIN] [dice_loss_3 ]: 0.2484605 |
| xmem_egoexo_tune_s3 - It 35403 [TRAIN] [iou ]: 0.6915472 |
| xmem_egoexo_tune_s3 - It 35403 [TRAIN] [total_loss ]: 0.7033817 |
| xmem_egoexo_tune_s3 - It 35403 [TRAIN] [p ]: 0.7414215 |
| xmem_egoexo_tune_s3 - It 35403 [TRAIN] [ce_loss_1 ]: 0.0069035 |
| xmem_egoexo_tune_s3 - It 35403 [TRAIN] [dice_loss_1 ]: 0.2107927 |
| xmem_egoexo_tune_s3 - It 35403 [TRAIN] [ce_loss_2 ]: 0.0076894 |
| xmem_egoexo_tune_s3 - It 35403 [TRAIN] [dice_loss_2 ]: 0.2276278 |
| xmem_egoexo_tune_s3 - It 35403 [TRAIN] [ce_loss_3 ]: 0.0081372 |
| xmem_egoexo_tune_s3 - It 35403 [TRAIN] [dice_loss_3 ]: 0.2422308 |
| xmem_egoexo_tune_s3 - It 35403 [TRAIN] [iou ]: 0.6959824 |
| Current epoch: 93 |
| Current epoch: 93 |
| Current epoch: 93 |
| Current epoch: 93 |
| xmem_egoexo_tune_s3 - It 35789 [TRAIN] [time ]: 16.931748 |
| xmem_egoexo_tune_s3 - It 35789 [TRAIN] [time ]: 16.931517 |
| xmem_egoexo_tune_s3 - It 35789 [TRAIN] [total_loss ]: 0.7195883 |
| xmem_egoexo_tune_s3 - It 35789 [TRAIN] [total_loss ]: 0.7187386 |
| xmem_egoexo_tune_s3 - It 35789 [TRAIN] [p ]: 0.7348595 |
| xmem_egoexo_tune_s3 - It 35789 [TRAIN] [p ]: 0.7348595 |
| xmem_egoexo_tune_s3 - It 35789 [TRAIN] [ce_loss_1 ]: 0.0077133 |
| xmem_egoexo_tune_s3 - It 35789 [TRAIN] [ce_loss_1 ]: 0.0073851 |
| xmem_egoexo_tune_s3 - It 35789 [TRAIN] [dice_loss_1 ]: 0.2165393 |
| xmem_egoexo_tune_s3 - It 35789 [TRAIN] [dice_loss_1 ]: 0.2164635 |
| xmem_egoexo_tune_s3 - It 35789 [TRAIN] [ce_loss_2 ]: 0.0084547 |
| xmem_egoexo_tune_s3 - It 35789 [TRAIN] [ce_loss_2 ]: 0.0080464 |
| xmem_egoexo_tune_s3 - It 35789 [TRAIN] [dice_loss_2 ]: 0.2299124 |
| xmem_egoexo_tune_s3 - It 35789 [TRAIN] [dice_loss_2 ]: 0.2302547 |
| xmem_egoexo_tune_s3 - It 35789 [TRAIN] [ce_loss_3 ]: 0.0088462 |
| xmem_egoexo_tune_s3 - It 35789 [TRAIN] [ce_loss_3 ]: 0.0085321 |
| xmem_egoexo_tune_s3 - It 35789 [TRAIN] [dice_loss_3 ]: 0.2481221 |
| xmem_egoexo_tune_s3 - It 35789 [TRAIN] [dice_loss_3 ]: 0.2480566 |
| xmem_egoexo_tune_s3 - It 35789 [TRAIN] [iou ]: 0.6914079 |
| xmem_egoexo_tune_s3 - It 35789 [TRAIN] [iou ]: 0.6911691 |
| Current epoch: 94 |
| Current epoch: 94 |
| Current epoch: 94 |
| Current epoch: 94 |
| Network saved to /data/work-gcp-europe-west4-a/yuqian_fu/Ego/saves_xmem2/Sep10_20.14.00_xmem_egoexo_tune_s3/Sep10_20.14.00_xmem_egoexo_tune_s3_36000.pth. |
| Network saved to /data/work-gcp-europe-west4-a/yuqian_fu/Ego/saves_xmem2/Sep10_20.14.00_xmem_egoexo_tune_s3/Sep10_20.14.00_xmem_egoexo_tune_s3_36000.pth. |
| Checkpoint saved to /data/work-gcp-europe-west4-a/yuqian_fu/Ego/saves_xmem2/Sep10_20.14.00_xmem_egoexo_tune_s3/Sep10_20.14.00_xmem_egoexo_tune_s3_checkpoint_36000.pth. |
| Checkpoint saved to /data/work-gcp-europe-west4-a/yuqian_fu/Ego/saves_xmem2/Sep10_20.14.00_xmem_egoexo_tune_s3/Sep10_20.14.00_xmem_egoexo_tune_s3_checkpoint_36000.pth. |
| xmem_egoexo_tune_s3 - It 36175 [TRAIN] [time ]: 17.178650 |
| xmem_egoexo_tune_s3 - It 36175 [TRAIN] [total_loss ]: 0.7070585 |
| xmem_egoexo_tune_s3 - It 36175 [TRAIN] [p ]: 0.7282974 |
| xmem_egoexo_tune_s3 - It 36175 [TRAIN] [ce_loss_1 ]: 0.0070096 |
| xmem_egoexo_tune_s3 - It 36175 [TRAIN] [dice_loss_1 ]: 0.2095819 |
| xmem_egoexo_tune_s3 - It 36175 [TRAIN] [ce_loss_2 ]: 0.0076734 |
| xmem_egoexo_tune_s3 - It 36175 [TRAIN] [dice_loss_2 ]: 0.2275309 |
| xmem_egoexo_tune_s3 - It 36175 [TRAIN] [ce_loss_3 ]: 0.0129356 |
| xmem_egoexo_tune_s3 - It 36175 [TRAIN] [dice_loss_3 ]: 0.2423269 |
| xmem_egoexo_tune_s3 - It 36175 [TRAIN] [iou ]: 0.6959402 |
| xmem_egoexo_tune_s3 - It 36175 [TRAIN] [time ]: 17.181632 |
| xmem_egoexo_tune_s3 - It 36175 [TRAIN] [total_loss ]: 0.7041671 |
| xmem_egoexo_tune_s3 - It 36175 [TRAIN] [p ]: 0.7282974 |
| xmem_egoexo_tune_s3 - It 36175 [TRAIN] [ce_loss_1 ]: 0.0067406 |
| xmem_egoexo_tune_s3 - It 36175 [TRAIN] [dice_loss_1 ]: 0.2086100 |
| xmem_egoexo_tune_s3 - It 36175 [TRAIN] [ce_loss_2 ]: 0.0078553 |
| xmem_egoexo_tune_s3 - It 36175 [TRAIN] [dice_loss_2 ]: 0.2274529 |
| xmem_egoexo_tune_s3 - It 36175 [TRAIN] [ce_loss_3 ]: 0.0130686 |
| xmem_egoexo_tune_s3 - It 36175 [TRAIN] [dice_loss_3 ]: 0.2404394 |
| xmem_egoexo_tune_s3 - It 36175 [TRAIN] [iou ]: 0.6972770 |
| Current epoch: 95 |
| Current epoch: 95 |
| Current epoch: 95 |
| Current epoch: 95 |
| xmem_egoexo_tune_s3 - It 36561 [TRAIN] [time ]: 16.879543 |
| xmem_egoexo_tune_s3 - It 36561 [TRAIN] [total_loss ]: 0.6983845 |
| xmem_egoexo_tune_s3 - It 36561 [TRAIN] [p ]: 0.7217354 |
| xmem_egoexo_tune_s3 - It 36561 [TRAIN] [ce_loss_1 ]: 0.0066293 |
| xmem_egoexo_tune_s3 - It 36561 [TRAIN] [dice_loss_1 ]: 0.2018671 |
| xmem_egoexo_tune_s3 - It 36561 [TRAIN] [ce_loss_2 ]: 0.0076838 |
| xmem_egoexo_tune_s3 - It 36561 [TRAIN] [dice_loss_2 ]: 0.2280383 |
| xmem_egoexo_tune_s3 - It 36561 [TRAIN] [time ]: 16.877153 |
| xmem_egoexo_tune_s3 - It 36561 [TRAIN] [ce_loss_3 ]: 0.0082014 |
| xmem_egoexo_tune_s3 - It 36561 [TRAIN] [dice_loss_3 ]: 0.2459644 |
| xmem_egoexo_tune_s3 - It 36561 [TRAIN] [iou ]: 0.6968249 |
| xmem_egoexo_tune_s3 - It 36561 [TRAIN] [total_loss ]: 0.7048027 |
| xmem_egoexo_tune_s3 - It 36561 [TRAIN] [p ]: 0.7217354 |
| xmem_egoexo_tune_s3 - It 36561 [TRAIN] [ce_loss_1 ]: 0.0070354 |
| xmem_egoexo_tune_s3 - It 36561 [TRAIN] [dice_loss_1 ]: 0.2040018 |
| xmem_egoexo_tune_s3 - It 36561 [TRAIN] [ce_loss_2 ]: 0.0079187 |
| xmem_egoexo_tune_s3 - It 36561 [TRAIN] [dice_loss_2 ]: 0.2307261 |
| xmem_egoexo_tune_s3 - It 36561 [TRAIN] [ce_loss_3 ]: 0.0084236 |
| xmem_egoexo_tune_s3 - It 36561 [TRAIN] [dice_loss_3 ]: 0.2466970 |
| xmem_egoexo_tune_s3 - It 36561 [TRAIN] [iou ]: 0.6946719 |
| Current epoch: 96 |
| Current epoch: 96 |
| Current epoch: 96 |
| Current epoch: 96 |
| xmem_egoexo_tune_s3 - It 36947 [TRAIN] [time ]: 16.895552 |
| xmem_egoexo_tune_s3 - It 36947 [TRAIN] [total_loss ]: 0.7063536 |
| xmem_egoexo_tune_s3 - It 36947 [TRAIN] [p ]: 0.7151734 |
| xmem_egoexo_tune_s3 - It 36947 [TRAIN] [ce_loss_1 ]: 0.0072472 |
| xmem_egoexo_tune_s3 - It 36947 [TRAIN] [dice_loss_1 ]: 0.2119061 |
| xmem_egoexo_tune_s3 - It 36947 [TRAIN] [ce_loss_2 ]: 0.0080302 |
| xmem_egoexo_tune_s3 - It 36947 [TRAIN] [dice_loss_2 ]: 0.2249939 |
| xmem_egoexo_tune_s3 - It 36947 [TRAIN] [ce_loss_3 ]: 0.0086273 |
| xmem_egoexo_tune_s3 - It 36947 [TRAIN] [dice_loss_3 ]: 0.2455487 |
| xmem_egoexo_tune_s3 - It 36947 [TRAIN] [iou ]: 0.6948809 |
| xmem_egoexo_tune_s3 - It 36947 [TRAIN] [time ]: 16.896545 |
| xmem_egoexo_tune_s3 - It 36947 [TRAIN] [total_loss ]: 0.7027585 |
| xmem_egoexo_tune_s3 - It 36947 [TRAIN] [p ]: 0.7151734 |
| xmem_egoexo_tune_s3 - It 36947 [TRAIN] [ce_loss_1 ]: 0.0072401 |
| xmem_egoexo_tune_s3 - It 36947 [TRAIN] [dice_loss_1 ]: 0.2112388 |
| xmem_egoexo_tune_s3 - It 36947 [TRAIN] [ce_loss_2 ]: 0.0077474 |
| xmem_egoexo_tune_s3 - It 36947 [TRAIN] [dice_loss_2 ]: 0.2253507 |
| xmem_egoexo_tune_s3 - It 36947 [TRAIN] [ce_loss_3 ]: 0.0082423 |
| xmem_egoexo_tune_s3 - It 36947 [TRAIN] [dice_loss_3 ]: 0.2429390 |
| xmem_egoexo_tune_s3 - It 36947 [TRAIN] [iou ]: 0.6955306 |
| Current epoch: 97 |
| Current epoch: 97 |
| Current epoch: 97 |
| Current epoch: 97 |
| Network saved to /data/work-gcp-europe-west4-a/yuqian_fu/Ego/saves_xmem2/Sep10_20.14.00_xmem_egoexo_tune_s3/Sep10_20.14.00_xmem_egoexo_tune_s3_37000.pth. |
| Network saved to /data/work-gcp-europe-west4-a/yuqian_fu/Ego/saves_xmem2/Sep10_20.14.00_xmem_egoexo_tune_s3/Sep10_20.14.00_xmem_egoexo_tune_s3_37000.pth. |
| xmem_egoexo_tune_s3 - It 37333 [TRAIN] [time ]: 16.859145 |
| xmem_egoexo_tune_s3 - It 37333 [TRAIN] [total_loss ]: 0.7189514 |
| xmem_egoexo_tune_s3 - It 37333 [TRAIN] [p ]: 0.7086114 |
| xmem_egoexo_tune_s3 - It 37333 [TRAIN] [ce_loss_1 ]: 0.0071597 |
| xmem_egoexo_tune_s3 - It 37333 [TRAIN] [dice_loss_1 ]: 0.2164450 |
| xmem_egoexo_tune_s3 - It 37333 [TRAIN] [ce_loss_2 ]: 0.0082170 |
| xmem_egoexo_tune_s3 - It 37333 [TRAIN] [dice_loss_2 ]: 0.2325320 |
| xmem_egoexo_tune_s3 - It 37333 [TRAIN] [ce_loss_3 ]: 0.0089160 |
| xmem_egoexo_tune_s3 - It 37333 [TRAIN] [dice_loss_3 ]: 0.2456814 |
| xmem_egoexo_tune_s3 - It 37333 [TRAIN] [iou ]: 0.6908400 |
| xmem_egoexo_tune_s3 - It 37333 [TRAIN] [time ]: 16.858824 |
| xmem_egoexo_tune_s3 - It 37333 [TRAIN] [total_loss ]: 0.7068181 |
| xmem_egoexo_tune_s3 - It 37333 [TRAIN] [p ]: 0.7086114 |
| xmem_egoexo_tune_s3 - It 37333 [TRAIN] [ce_loss_1 ]: 0.0072582 |
| xmem_egoexo_tune_s3 - It 37333 [TRAIN] [dice_loss_1 ]: 0.2145711 |
| xmem_egoexo_tune_s3 - It 37333 [TRAIN] [ce_loss_2 ]: 0.0078027 |
| xmem_egoexo_tune_s3 - It 37333 [TRAIN] [dice_loss_2 ]: 0.2298393 |
| xmem_egoexo_tune_s3 - It 37333 [TRAIN] [ce_loss_3 ]: 0.0082132 |
| xmem_egoexo_tune_s3 - It 37333 [TRAIN] [dice_loss_3 ]: 0.2391332 |
| xmem_egoexo_tune_s3 - It 37333 [TRAIN] [iou ]: 0.6952108 |
| Current epoch: 98 |
| Current epoch: 98 |
| Current epoch: 98 |
| Current epoch: 98 |
| xmem_egoexo_tune_s3 - It 37719 [TRAIN] [time ]: 17.023203 |
| xmem_egoexo_tune_s3 - It 37719 [TRAIN] [total_loss ]: 0.6935734 |
| xmem_egoexo_tune_s3 - It 37719 [TRAIN] [p ]: 0.7020494 |
| xmem_egoexo_tune_s3 - It 37719 [TRAIN] [ce_loss_1 ]: 0.0069194 |
| xmem_egoexo_tune_s3 - It 37719 [TRAIN] [dice_loss_1 ]: 0.2082904 |
| xmem_egoexo_tune_s3 - It 37719 [TRAIN] [ce_loss_2 ]: 0.0074664 |
| xmem_egoexo_tune_s3 - It 37719 [TRAIN] [dice_loss_2 ]: 0.2235138 |
| xmem_egoexo_tune_s3 - It 37719 [TRAIN] [ce_loss_3 ]: 0.0086311 |
| xmem_egoexo_tune_s3 - It 37719 [TRAIN] [dice_loss_3 ]: 0.2387521 |
| xmem_egoexo_tune_s3 - It 37719 [TRAIN] [iou ]: 0.6988391 |
| xmem_egoexo_tune_s3 - It 37719 [TRAIN] [time ]: 17.023860 |
| xmem_egoexo_tune_s3 - It 37719 [TRAIN] [total_loss ]: 0.6951618 |
| xmem_egoexo_tune_s3 - It 37719 [TRAIN] [p ]: 0.7020494 |
| xmem_egoexo_tune_s3 - It 37719 [TRAIN] [ce_loss_1 ]: 0.0070391 |
| xmem_egoexo_tune_s3 - It 37719 [TRAIN] [dice_loss_1 ]: 0.2077032 |
| xmem_egoexo_tune_s3 - It 37719 [TRAIN] [ce_loss_2 ]: 0.0078862 |
| xmem_egoexo_tune_s3 - It 37719 [TRAIN] [dice_loss_2 ]: 0.2228186 |
| xmem_egoexo_tune_s3 - It 37719 [TRAIN] [ce_loss_3 ]: 0.0092261 |
| xmem_egoexo_tune_s3 - It 37719 [TRAIN] [dice_loss_3 ]: 0.2404884 |
| xmem_egoexo_tune_s3 - It 37719 [TRAIN] [iou ]: 0.6995283 |
| Current epoch: 99 |
| Current epoch: 99 |
| Current epoch: 99 |
| Current epoch: 99 |
| Network saved to /data/work-gcp-europe-west4-a/yuqian_fu/Ego/saves_xmem2/Sep10_20.14.00_xmem_egoexo_tune_s3/Sep10_20.14.00_xmem_egoexo_tune_s3_38000.pth. |
| Network saved to /data/work-gcp-europe-west4-a/yuqian_fu/Ego/saves_xmem2/Sep10_20.14.00_xmem_egoexo_tune_s3/Sep10_20.14.00_xmem_egoexo_tune_s3_38000.pth. |
| Checkpoint saved to /data/work-gcp-europe-west4-a/yuqian_fu/Ego/saves_xmem2/Sep10_20.14.00_xmem_egoexo_tune_s3/Sep10_20.14.00_xmem_egoexo_tune_s3_checkpoint_38000.pth. |
| Checkpoint saved to /data/work-gcp-europe-west4-a/yuqian_fu/Ego/saves_xmem2/Sep10_20.14.00_xmem_egoexo_tune_s3/Sep10_20.14.00_xmem_egoexo_tune_s3_checkpoint_38000.pth. |
| xmem_egoexo_tune_s3 - It 38105 [TRAIN] [time ]: 16.943846 |
| xmem_egoexo_tune_s3 - It 38105 [TRAIN] [time ]: 16.943384 |
| xmem_egoexo_tune_s3 - It 38105 [TRAIN] [total_loss ]: 0.6864870 |
| xmem_egoexo_tune_s3 - It 38105 [TRAIN] [p ]: 0.6954874 |
| xmem_egoexo_tune_s3 - It 38105 [TRAIN] [ce_loss_1 ]: 0.0068392 |
| xmem_egoexo_tune_s3 - It 38105 [TRAIN] [dice_loss_1 ]: 0.2078383 |
| xmem_egoexo_tune_s3 - It 38105 [TRAIN] [ce_loss_2 ]: 0.0074637 |
| xmem_egoexo_tune_s3 - It 38105 [TRAIN] [dice_loss_2 ]: 0.2210043 |
| xmem_egoexo_tune_s3 - It 38105 [TRAIN] [ce_loss_3 ]: 0.0076704 |
| xmem_egoexo_tune_s3 - It 38105 [TRAIN] [dice_loss_3 ]: 0.2356708 |
| xmem_egoexo_tune_s3 - It 38105 [TRAIN] [iou ]: 0.7008177 |
| xmem_egoexo_tune_s3 - It 38105 [TRAIN] [total_loss ]: 0.6934680 |
| xmem_egoexo_tune_s3 - It 38105 [TRAIN] [p ]: 0.6954874 |
| xmem_egoexo_tune_s3 - It 38105 [TRAIN] [ce_loss_1 ]: 0.0071880 |
| xmem_egoexo_tune_s3 - It 38105 [TRAIN] [dice_loss_1 ]: 0.2115061 |
| xmem_egoexo_tune_s3 - It 38105 [TRAIN] [ce_loss_2 ]: 0.0075660 |
| xmem_egoexo_tune_s3 - It 38105 [TRAIN] [dice_loss_2 ]: 0.2228371 |
| xmem_egoexo_tune_s3 - It 38105 [TRAIN] [ce_loss_3 ]: 0.0077206 |
| xmem_egoexo_tune_s3 - It 38105 [TRAIN] [dice_loss_3 ]: 0.2366499 |
| xmem_egoexo_tune_s3 - It 38105 [TRAIN] [iou ]: 0.6984140 |
| Current epoch: 100 |
| Current epoch: 100 |
| Current epoch: 100 |
| Current epoch: 100 |
| xmem_egoexo_tune_s3 - It 38491 [TRAIN] [time ]: 17.081790 |
| xmem_egoexo_tune_s3 - It 38491 [TRAIN] [time ]: 17.081702 |
| xmem_egoexo_tune_s3 - It 38491 [TRAIN] [total_loss ]: 0.6833270 |
| xmem_egoexo_tune_s3 - It 38491 [TRAIN] [p ]: 0.6889255 |
| xmem_egoexo_tune_s3 - It 38491 [TRAIN] [ce_loss_1 ]: 0.0071634 |
| xmem_egoexo_tune_s3 - It 38491 [TRAIN] [dice_loss_1 ]: 0.2051358 |
| xmem_egoexo_tune_s3 - It 38491 [TRAIN] [ce_loss_2 ]: 0.0076076 |
| xmem_egoexo_tune_s3 - It 38491 [TRAIN] [dice_loss_2 ]: 0.2163749 |
| xmem_egoexo_tune_s3 - It 38491 [TRAIN] [ce_loss_3 ]: 0.0085877 |
| xmem_egoexo_tune_s3 - It 38491 [TRAIN] [dice_loss_3 ]: 0.2384575 |
| xmem_egoexo_tune_s3 - It 38491 [TRAIN] [iou ]: 0.7044566 |
| xmem_egoexo_tune_s3 - It 38491 [TRAIN] [total_loss ]: 0.6899236 |
| xmem_egoexo_tune_s3 - It 38491 [TRAIN] [p ]: 0.6889255 |
| xmem_egoexo_tune_s3 - It 38491 [TRAIN] [ce_loss_1 ]: 0.0073217 |
| xmem_egoexo_tune_s3 - It 38491 [TRAIN] [dice_loss_1 ]: 0.2065342 |
| xmem_egoexo_tune_s3 - It 38491 [TRAIN] [ce_loss_2 ]: 0.0075793 |
| xmem_egoexo_tune_s3 - It 38491 [TRAIN] [dice_loss_2 ]: 0.2177637 |
| xmem_egoexo_tune_s3 - It 38491 [TRAIN] [ce_loss_3 ]: 0.0086224 |
| xmem_egoexo_tune_s3 - It 38491 [TRAIN] [dice_loss_3 ]: 0.2421021 |
| xmem_egoexo_tune_s3 - It 38491 [TRAIN] [iou ]: 0.7020434 |
| Current epoch: 101 |
| Current epoch: 101 |
| Current epoch: 101 |
| Current epoch: 101 |
| |
