{
    "file_diffs": [
        {
            "old_file_content": "# emacs: -*- mode: python; py-indent-offset: 4; tab-width: 4; indent-tabs-mode: nil -*-\n# ex: set sts=4 ts=4 sw=4 et:\n# ## ### ### ### ### ### ### ### ### ### ### ### ### ### ### ### ### ### ### ##\n#\n#   See COPYING file distributed along with the datalad package for the\n#   copyright and license terms.\n#\n# ## ### ### ### ### ### ### ### ### ### ### ### ### ### ### ### ### ### ### ##\n\"\"\"DataLad aims to expose (scientific) data available online as a unified data\ndistribution with the convenience of git-annex repositories as a backend.\n\nCommands are exposed through both a command-line interface and a Python API. On\nthe command line, run 'datalad --help' for a summary of the available commands.\nFrom an interactive Python session, import `datalad.api` and inspect its\ndocumentation with `help`.\n\"\"\"\n\nif not __debug__:\n    raise RuntimeError(\n        'DataLad cannot run in \"optimized\" mode, i.e. python -O')\n\nimport atexit\nimport os\n\n\n# this is not to be modified. for querying use get_apimode()\n__api = 'python'\n# this is not to be modified. see enable/in_librarymode()\n# by default, we are in application-mode, simply because most of\n# datalad was originally implemented with this scenario assumption\n__runtime_mode = 'application'\n\n\ndef get_apimode():\n    \"\"\"Returns the API mode label for the current session.\n\n    The API mode label indicates whether DataLad is running in \"normal\"\n    mode in a Python session, or whether it is used via the command line\n    interface.\n\n    This function is a utility for optimizing behavior and messaging to the\n    particular API (Python vs command line) in use in a given process.\n\n    Returns\n    {'python', 'cmdline'}\n      The API mode is 'python' by default, unless the main command line\n      entrypoint set it to 'cmdline'.\n    \"\"\"\n    return __api\n\n\ndef in_librarymode():\n    \"\"\"Returns whether DataLad is requested to run in \"library mode\"\n\n    In this mode DataLad aims to behave without the assumption that it is\n    itself the front-end of a process and in full control over messaging\n    and parameters.\n\n    Returns\n    -------\n    bool\n    \"\"\"\n    return __runtime_mode == 'library'\n\n\ndef enable_librarymode():\n    \"\"\"Request DataLad to operate in library mode.\n\n    This function should be executed immediately after importing the `datalad`\n    package, when DataLad is not used as an application, or in interactive\n    scenarios, but as a utility library inside other applications. Enabling\n    this mode will turn off some convenience feature that are irrelevant in\n    such use cases (with performance benefits), and alters it messaging\n    behavior to better interoperate with 3rd-party front-ends.\n\n    Library mode can only be enabled once. Switching it on and off within\n    the runtime of a process is not supported.\n\n    Example::\n\n        >>> import datalad\n        >>> datalad.enable_librarymode()\n    \"\"\"\n    global __runtime_mode\n    __runtime_mode = 'library'\n\n\n# For reproducible demos/tests\n_seed = os.environ.get('DATALAD_SEED', None)\nif _seed is not None:\n    import random\n    _seed = int(_seed)\n    random.seed(_seed)\n\n# Colorama (for Windows terminal colors) must be imported before we use/bind\n# any sys.stdout\ntry:\n    # this will fix the rendering of ANSI escape sequences\n    # for colored terminal output on windows\n    # it will do nothing on any other platform, hence it\n    # is safe to call unconditionally\n    import colorama\n    colorama.init()\n    atexit.register(colorama.deinit)\nexcept ImportError as e:\n    pass\n\n# Other imports are interspersed with lgr.debug to ease troubleshooting startup\n# delays etc.\n\nfrom .config import ConfigManager\ncfg = ConfigManager()\n\n# must come after config manager\nfrom .log import lgr\nfrom datalad.support.exceptions import CapturedException\nfrom datalad.utils import (\n    get_encoding_info,\n    get_envvars_info,\n    get_home_envvars,\n    getpwd,\n)\n\n# To analyze/initiate our decision making on what current directory to return\ngetpwd()\n\nlgr.log(5, \"Instantiating ssh manager\")\nfrom .support.sshconnector import SSHManager\nssh_manager = SSHManager()\natexit.register(ssh_manager.close, allow_fail=False)\natexit.register(lgr.log, 5, \"Exiting\")\n\n\ndef test(module='datalad', verbose=False, nocapture=False, pdb=False, stop=False):\n    \"\"\"A helper to run datalad's tests.  Requires nose\n    \"\"\"\n    argv = [] #module]\n    # could make it 'smarter' but decided to be explicit so later we could\n    # easily migrate to another runner without changing any API here\n    if verbose:\n        argv.append('-v')\n    if nocapture:\n        argv.append('-s')\n    if pdb:\n        argv.append('--pdb')\n    if stop:\n        argv.append('--stop')\n    from datalad.support.third.nosetester import NoseTester\n    tester = NoseTester(module)\n    tester.package_name = module.split('.', 1)[0]\n    tester.test(extra_argv=argv)\n\ntest.__test__ = False\n\n# Following fixtures are necessary at the top level __init__ for fixtures which\n# would cover all **/tests and not just datalad/tests/\n\n# To store settings which setup_package changes and teardown_package should return\n_test_states = {\n    'loglevel': None,\n    'env': {},\n}\n\n# handle to an HTTP server instance that is used as part of the tests\ntest_http_server = None\n\n\ndef setup_package():\n    from io import StringIO as OrigStringIO\n    from nose.ext import dtcompat\n    from nose.plugins import (\n        capture,\n        multiprocess,\n        plugintest,\n    )\n    import os\n    from pathlib import Path\n    import tempfile\n\n    from datalad import consts\n    from datalad.interface.common_cfg import compute_cfg_defaults\n    from datalad.support.annexrepo import AnnexRepo\n    from datalad.support.external_versions import external_versions\n    from datalad.tests import _TEMP_PATHS_GENERATED\n    from datalad.tests.utils import (\n        DEFAULT_BRANCH,\n        DEFAULT_REMOTE,\n        HTTPPath,\n    )\n    from datalad.ui import ui\n    from datalad.utils import (\n        make_tempfile,\n        on_osx,\n    )\n\n    if on_osx:\n        # enforce honoring TMPDIR (see gh-5307)\n        tempfile.tempdir = os.environ.get('TMPDIR', tempfile.gettempdir())\n\n    _test_states['env'] = {}\n\n    def set_envvar(v, val):\n        \"\"\"Memoize and then set env var\"\"\"\n        _test_states['env'][v] = os.environ.get(v, None)\n        os.environ[v] = val\n\n    _test_states['DATASETS_TOPURL'] = consts.DATASETS_TOPURL\n    consts.DATASETS_TOPURL = 'https://datasets-tests.datalad.org/'\n    set_envvar('DATALAD_DATASETS_TOPURL', consts.DATASETS_TOPURL)\n\n    set_envvar(\"GIT_CONFIG_PARAMETERS\",\n               \"'init.defaultBranch={}' 'clone.defaultRemoteName={}'\"\n               .format(DEFAULT_BRANCH, DEFAULT_REMOTE))\n\n    def prep_tmphome():\n        gitconfig = \"\"\"\\\n[user]\n\tname = DataLad Tester\n\temail = test@example.com\n[datalad \"log\"]\n\texc = 1\n\"\"\"\n        # TODO: split into a function + context manager\n        with make_tempfile(mkdir=True) as new_home:\n            pass\n        # register for clean-up on exit\n        _TEMP_PATHS_GENERATED.append(new_home)\n\n        # populate default config\n        new_home = Path(new_home)\n        new_home.mkdir(parents=True, exist_ok=True)\n        cfg_file = new_home / '.gitconfig'\n        cfg_file.write_text(gitconfig)\n        return new_home, cfg_file\n\n    if external_versions['cmd:git'] < \"2.32\":\n        # To overcome pybuild overriding HOME but us possibly wanting our\n        # own HOME where we pre-setup git for testing (name, email)\n        if 'GIT_HOME' in os.environ:\n            set_envvar('HOME', os.environ['GIT_HOME'])\n        else:\n            # we setup our own new HOME, the BEST and HUGE one\n            new_home, _ = prep_tmphome()\n            for v, val in get_home_envvars(new_home).items():\n                set_envvar(v, val)\n    else:\n        _, cfg_file = prep_tmphome()\n        set_envvar('GIT_CONFIG_GLOBAL', str(cfg_file))\n\n    # Re-load ConfigManager, since otherwise it won't consider global config\n    # from new $HOME (see gh-4153\n    cfg.reload(force=True)\n\n    compute_cfg_defaults()\n    # datalad.locations.sockets has likely changed. Discard any cached values.\n    ssh_manager._socket_dir = None\n\n    # To overcome pybuild by default defining http{,s}_proxy we would need\n    # to define them to e.g. empty value so it wouldn't bother touching them.\n    # But then haskell libraries do not digest empty value nicely, so we just\n    # pop them out from the environment\n    for ev in ('http_proxy', 'https_proxy'):\n        if ev in os.environ and not (os.environ[ev]):\n            lgr.debug(\"Removing %s from the environment since it is empty\", ev)\n            os.environ.pop(ev)\n\n    # During tests we allow for \"insecure\" access to local file:// and\n    # http://localhost URLs since all of them either generated as tests\n    # fixtures or cloned from trusted sources\n    AnnexRepo._ALLOW_LOCAL_URLS = True\n\n    DATALAD_LOG_LEVEL = os.environ.get('DATALAD_LOG_LEVEL', None)\n    if DATALAD_LOG_LEVEL is None:\n        # very very silent.  Tests introspecting logs should use\n        # swallow_logs(new_level=...)\n        _test_states['loglevel'] = lgr.getEffectiveLevel()\n        lgr.setLevel(100)\n\n        # And we should also set it within environ so underlying commands also stay silent\n        set_envvar('DATALAD_LOG_LEVEL', '100')\n    else:\n        # We are not overriding them, since explicitly were asked to have some log level\n        _test_states['loglevel'] = None\n\n    # Set to non-interactive UI\n    _test_states['ui_backend'] = ui.backend\n    # obtain() since that one consults for the default value\n    ui.set_backend(cfg.obtain('datalad.tests.ui.backend'))\n\n    # Monkey patch nose so it does not ERROR out whenever code asks for fileno\n    # of the output. See https://github.com/nose-devs/nose/issues/6\n    class StringIO(OrigStringIO):\n        fileno = lambda self: 1\n        encoding = None\n\n    dtcompat.StringIO = StringIO\n    capture.StringIO = StringIO\n    multiprocess.StringIO = StringIO\n    plugintest.StringIO = StringIO\n\n    # in order to avoid having to fiddle with rather uncommon\n    # file:// URLs in the tests, have a standard HTTP server\n    # that serves an 'httpserve' directory in the test HOME\n    # the URL will be available from datalad.test_http_server.url\n    global test_http_server\n    # Start the server only if not running already\n    # Relevant: we have test_misc.py:test_test which runs datalad.test but\n    # not doing teardown, so the original server might never get stopped\n    if test_http_server is None:\n        serve_path = tempfile.mkdtemp(\n            dir=cfg.get(\"datalad.tests.temp.dir\"),\n            prefix='httpserve',\n        )\n        test_http_server = HTTPPath(serve_path)\n        test_http_server.start()\n        _TEMP_PATHS_GENERATED.append(serve_path)\n\n    if cfg.obtain('datalad.tests.setup.testrepos'):\n        lgr.debug(\"Pre-populating testrepos\")\n        from datalad.tests.utils import with_testrepos\n        with_testrepos()(lambda repo: 1)()\n\n\ndef teardown_package():\n    import os\n    from datalad import consts\n    from datalad.interface.common_cfg import compute_cfg_defaults\n    from datalad.support.annexrepo import AnnexRepo\n    from datalad.support.cookies import cookies_db\n    from datalad.support.external_versions import external_versions as ev\n    from datalad.tests import _TEMP_PATHS_GENERATED\n    from datalad.tests.utils import (\n        rmtemp,\n        OBSCURE_FILENAME,\n    )\n    from datalad.ui import ui\n\n    lgr.debug(\"Printing versioning information collected so far\")\n    print(ev.dumps(query=True))\n    try:\n        print(\"Obscure filename: str=%s repr=%r\"\n                % (OBSCURE_FILENAME.encode('utf-8'), OBSCURE_FILENAME))\n    except UnicodeEncodeError as exc:\n        ce = CapturedException(exc)\n        print(\"Obscure filename failed to print: %s\" % ce)\n    def print_dict(d):\n        return \" \".join(\"%s=%r\" % v for v in d.items())\n    print(\"Encodings: %s\" % print_dict(get_encoding_info()))\n    print(\"Environment: %s\" % print_dict(get_envvars_info()))\n\n    if os.environ.get('DATALAD_TESTS_NOTEARDOWN'):\n        return\n    ui.set_backend(_test_states['ui_backend'])\n    if _test_states['loglevel'] is not None:\n        lgr.setLevel(_test_states['loglevel'])\n\n    global test_http_server\n    if test_http_server:\n        test_http_server.stop()\n        test_http_server = None\n    else:\n        lgr.debug(\"For some reason global http_server was not set/running, thus not stopping\")\n\n    if len(_TEMP_PATHS_GENERATED):\n        msg = \"Removing %d dirs/files: %s\" % (len(_TEMP_PATHS_GENERATED), ', '.join(_TEMP_PATHS_GENERATED))\n    else:\n        msg = \"Nothing to remove\"\n    lgr.debug(\"Teardown tests. \" + msg)\n    for path in _TEMP_PATHS_GENERATED:\n        rmtemp(str(path), ignore_errors=True)\n\n    # restore all the env variables\n    for v, val in _test_states['env'].items():\n        if val is not None:\n            os.environ[v] = val\n        else:\n            os.environ.pop(v)\n\n    # Re-establish correct global config after changing $HOME.\n    # Might be superfluous, since after teardown datalad.cfg shouldn't be\n    # needed. However, maintaining a consistent state seems a good thing\n    # either way.\n    cfg.reload(force=True)\n\n    compute_cfg_defaults()\n    ssh_manager._socket_dir = None\n\n    consts.DATASETS_TOPURL = _test_states['DATASETS_TOPURL']\n\n    cookies_db.close()\n    AnnexRepo._ALLOW_LOCAL_URLS = False  # stay safe!\n\n\nfrom ._version import get_versions\n__version__ = get_versions()['version']\ndel get_versions\n\nif str(__version__) == '0' or __version__.startswith('0+'):\n    lgr.warning(\n        \"DataLad was not installed 'properly' so its version is an uninformative %r.\\n\"\n        \"It can happen e.g. if datalad was installed via\\n\"\n        \"  pip install https://github.com/.../archive/{commitish}.zip\\n\"\n        \"instead of\\n\"\n        \"  pip install git+https://github.com/...@{commitish} .\\n\"\n        \"We advise to re-install datalad or downstream projects might not operate correctly.\",\n        __version__\n    )\n\nlgr.log(5, \"Done importing main __init__\")\n",
            "new_file_content": "# emacs: -*- mode: python; py-indent-offset: 4; tab-width: 4; indent-tabs-mode: nil -*-\n# ex: set sts=4 ts=4 sw=4 et:\n# ## ### ### ### ### ### ### ### ### ### ### ### ### ### ### ### ### ### ### ##\n#\n#   See COPYING file distributed along with the datalad package for the\n#   copyright and license terms.\n#\n# ## ### ### ### ### ### ### ### ### ### ### ### ### ### ### ### ### ### ### ##\n\"\"\"DataLad aims to expose (scientific) data available online as a unified data\ndistribution with the convenience of git-annex repositories as a backend.\n\nCommands are exposed through both a command-line interface and a Python API. On\nthe command line, run 'datalad --help' for a summary of the available commands.\nFrom an interactive Python session, import `datalad.api` and inspect its\ndocumentation with `help`.\n\"\"\"\n\nif not __debug__:\n    raise RuntimeError(\n        'DataLad cannot run in \"optimized\" mode, i.e. python -O')\n\nimport atexit\nimport os\n\n\n# this is not to be modified. for querying use get_apimode()\n__api = 'python'\n\n\ndef get_apimode():\n    \"\"\"Returns the API mode label for the current session.\n\n    The API mode label indicates whether DataLad is running in \"normal\"\n    mode in a Python session, or whether it is used via the command line\n    interface.\n\n    This function is a utility for optimizing behavior and messaging to the\n    particular API (Python vs command line) in use in a given process.\n\n    Returns\n    {'python', 'cmdline'}\n      The API mode is 'python' by default, unless the main command line\n      entrypoint set it to 'cmdline'.\n    \"\"\"\n    return __api\n\n\ndef in_librarymode():\n    \"\"\"Returns whether DataLad is requested to run in \"library mode\"\n\n    In this mode DataLad aims to behave without the assumption that it is\n    itself the front-end of a process and in full control over messaging\n    and parameters.\n\n    Returns\n    -------\n    bool\n    \"\"\"\n    return __runtime_mode == 'library'\n\n\ndef enable_librarymode():\n    \"\"\"Request DataLad to operate in library mode.\n\n    This function should be executed immediately after importing the `datalad`\n    package, when DataLad is not used as an application, or in interactive\n    scenarios, but as a utility library inside other applications. Enabling\n    this mode will turn off some convenience feature that are irrelevant in\n    such use cases (with performance benefits), and alters it messaging\n    behavior to better interoperate with 3rd-party front-ends.\n\n    Library mode can only be enabled once. Switching it on and off within\n    the runtime of a process is not supported.\n\n    Example::\n\n        >>> import datalad\n        >>> datalad.enable_librarymode()\n    \"\"\"\n    global __runtime_mode\n    __runtime_mode = 'library'\n    # export into the environment for child processes to inherit\n    os.environ['DATALAD_RUNTIME_LIBRARYMODE'] = '1'\n\n\n# For reproducible demos/tests\n_seed = os.environ.get('DATALAD_SEED', None)\nif _seed is not None:\n    import random\n    _seed = int(_seed)\n    random.seed(_seed)\n\n# Colorama (for Windows terminal colors) must be imported before we use/bind\n# any sys.stdout\ntry:\n    # this will fix the rendering of ANSI escape sequences\n    # for colored terminal output on windows\n    # it will do nothing on any other platform, hence it\n    # is safe to call unconditionally\n    import colorama\n    colorama.init()\n    atexit.register(colorama.deinit)\nexcept ImportError as e:\n    pass\n\n# Other imports are interspersed with lgr.debug to ease troubleshooting startup\n# delays etc.\n\nfrom .config import ConfigManager\ncfg = ConfigManager()\n\n# must come after config manager\n# this is not to be modified. see enable/in_librarymode()\n# by default, we are in application-mode, simply because most of\n# datalad was originally implemented with this scenario assumption\n__runtime_mode = 'library' \\\n    if cfg.getbool('datalad.runtime', 'librarymode', False) \\\n    else 'application'\n\nfrom .log import lgr\nfrom datalad.support.exceptions import CapturedException\nfrom datalad.utils import (\n    get_encoding_info,\n    get_envvars_info,\n    get_home_envvars,\n    getpwd,\n)\n\n# To analyze/initiate our decision making on what current directory to return\ngetpwd()\n\nlgr.log(5, \"Instantiating ssh manager\")\nfrom .support.sshconnector import SSHManager\nssh_manager = SSHManager()\natexit.register(ssh_manager.close, allow_fail=False)\natexit.register(lgr.log, 5, \"Exiting\")\n\n\ndef test(module='datalad', verbose=False, nocapture=False, pdb=False, stop=False):\n    \"\"\"A helper to run datalad's tests.  Requires nose\n    \"\"\"\n    argv = [] #module]\n    # could make it 'smarter' but decided to be explicit so later we could\n    # easily migrate to another runner without changing any API here\n    if verbose:\n        argv.append('-v')\n    if nocapture:\n        argv.append('-s')\n    if pdb:\n        argv.append('--pdb')\n    if stop:\n        argv.append('--stop')\n    from datalad.support.third.nosetester import NoseTester\n    tester = NoseTester(module)\n    tester.package_name = module.split('.', 1)[0]\n    tester.test(extra_argv=argv)\n\ntest.__test__ = False\n\n# Following fixtures are necessary at the top level __init__ for fixtures which\n# would cover all **/tests and not just datalad/tests/\n\n# To store settings which setup_package changes and teardown_package should return\n_test_states = {\n    'loglevel': None,\n    'env': {},\n}\n\n# handle to an HTTP server instance that is used as part of the tests\ntest_http_server = None\n\n\ndef setup_package():\n    from io import StringIO as OrigStringIO\n    from nose.ext import dtcompat\n    from nose.plugins import (\n        capture,\n        multiprocess,\n        plugintest,\n    )\n    import os\n    from pathlib import Path\n    import tempfile\n\n    from datalad import consts\n    from datalad.interface.common_cfg import compute_cfg_defaults\n    from datalad.support.annexrepo import AnnexRepo\n    from datalad.support.external_versions import external_versions\n    from datalad.tests import _TEMP_PATHS_GENERATED\n    from datalad.tests.utils import (\n        DEFAULT_BRANCH,\n        DEFAULT_REMOTE,\n        HTTPPath,\n    )\n    from datalad.ui import ui\n    from datalad.utils import (\n        make_tempfile,\n        on_osx,\n    )\n\n    if on_osx:\n        # enforce honoring TMPDIR (see gh-5307)\n        tempfile.tempdir = os.environ.get('TMPDIR', tempfile.gettempdir())\n\n    _test_states['env'] = {}\n\n    def set_envvar(v, val):\n        \"\"\"Memoize and then set env var\"\"\"\n        _test_states['env'][v] = os.environ.get(v, None)\n        os.environ[v] = val\n\n    _test_states['DATASETS_TOPURL'] = consts.DATASETS_TOPURL\n    consts.DATASETS_TOPURL = 'https://datasets-tests.datalad.org/'\n    set_envvar('DATALAD_DATASETS_TOPURL', consts.DATASETS_TOPURL)\n\n    set_envvar(\"GIT_CONFIG_PARAMETERS\",\n               \"'init.defaultBranch={}' 'clone.defaultRemoteName={}'\"\n               .format(DEFAULT_BRANCH, DEFAULT_REMOTE))\n\n    def prep_tmphome():\n        gitconfig = \"\"\"\\\n[user]\n\tname = DataLad Tester\n\temail = test@example.com\n[datalad \"log\"]\n\texc = 1\n\"\"\"\n        # TODO: split into a function + context manager\n        with make_tempfile(mkdir=True) as new_home:\n            pass\n        # register for clean-up on exit\n        _TEMP_PATHS_GENERATED.append(new_home)\n\n        # populate default config\n        new_home = Path(new_home)\n        new_home.mkdir(parents=True, exist_ok=True)\n        cfg_file = new_home / '.gitconfig'\n        cfg_file.write_text(gitconfig)\n        return new_home, cfg_file\n\n    if external_versions['cmd:git'] < \"2.32\":\n        # To overcome pybuild overriding HOME but us possibly wanting our\n        # own HOME where we pre-setup git for testing (name, email)\n        if 'GIT_HOME' in os.environ:\n            set_envvar('HOME', os.environ['GIT_HOME'])\n        else:\n            # we setup our own new HOME, the BEST and HUGE one\n            new_home, _ = prep_tmphome()\n            for v, val in get_home_envvars(new_home).items():\n                set_envvar(v, val)\n    else:\n        _, cfg_file = prep_tmphome()\n        set_envvar('GIT_CONFIG_GLOBAL', str(cfg_file))\n\n    # Re-load ConfigManager, since otherwise it won't consider global config\n    # from new $HOME (see gh-4153\n    cfg.reload(force=True)\n\n    compute_cfg_defaults()\n    # datalad.locations.sockets has likely changed. Discard any cached values.\n    ssh_manager._socket_dir = None\n\n    # To overcome pybuild by default defining http{,s}_proxy we would need\n    # to define them to e.g. empty value so it wouldn't bother touching them.\n    # But then haskell libraries do not digest empty value nicely, so we just\n    # pop them out from the environment\n    for ev in ('http_proxy', 'https_proxy'):\n        if ev in os.environ and not (os.environ[ev]):\n            lgr.debug(\"Removing %s from the environment since it is empty\", ev)\n            os.environ.pop(ev)\n\n    # During tests we allow for \"insecure\" access to local file:// and\n    # http://localhost URLs since all of them either generated as tests\n    # fixtures or cloned from trusted sources\n    AnnexRepo._ALLOW_LOCAL_URLS = True\n\n    DATALAD_LOG_LEVEL = os.environ.get('DATALAD_LOG_LEVEL', None)\n    if DATALAD_LOG_LEVEL is None:\n        # very very silent.  Tests introspecting logs should use\n        # swallow_logs(new_level=...)\n        _test_states['loglevel'] = lgr.getEffectiveLevel()\n        lgr.setLevel(100)\n\n        # And we should also set it within environ so underlying commands also stay silent\n        set_envvar('DATALAD_LOG_LEVEL', '100')\n    else:\n        # We are not overriding them, since explicitly were asked to have some log level\n        _test_states['loglevel'] = None\n\n    # Set to non-interactive UI\n    _test_states['ui_backend'] = ui.backend\n    # obtain() since that one consults for the default value\n    ui.set_backend(cfg.obtain('datalad.tests.ui.backend'))\n\n    # Monkey patch nose so it does not ERROR out whenever code asks for fileno\n    # of the output. See https://github.com/nose-devs/nose/issues/6\n    class StringIO(OrigStringIO):\n        fileno = lambda self: 1\n        encoding = None\n\n    dtcompat.StringIO = StringIO\n    capture.StringIO = StringIO\n    multiprocess.StringIO = StringIO\n    plugintest.StringIO = StringIO\n\n    # in order to avoid having to fiddle with rather uncommon\n    # file:// URLs in the tests, have a standard HTTP server\n    # that serves an 'httpserve' directory in the test HOME\n    # the URL will be available from datalad.test_http_server.url\n    global test_http_server\n    # Start the server only if not running already\n    # Relevant: we have test_misc.py:test_test which runs datalad.test but\n    # not doing teardown, so the original server might never get stopped\n    if test_http_server is None:\n        serve_path = tempfile.mkdtemp(\n            dir=cfg.get(\"datalad.tests.temp.dir\"),\n            prefix='httpserve',\n        )\n        test_http_server = HTTPPath(serve_path)\n        test_http_server.start()\n        _TEMP_PATHS_GENERATED.append(serve_path)\n\n    if cfg.obtain('datalad.tests.setup.testrepos'):\n        lgr.debug(\"Pre-populating testrepos\")\n        from datalad.tests.utils import with_testrepos\n        with_testrepos()(lambda repo: 1)()\n\n\ndef teardown_package():\n    import os\n    from datalad import consts\n    from datalad.interface.common_cfg import compute_cfg_defaults\n    from datalad.support.annexrepo import AnnexRepo\n    from datalad.support.cookies import cookies_db\n    from datalad.support.external_versions import external_versions as ev\n    from datalad.tests import _TEMP_PATHS_GENERATED\n    from datalad.tests.utils import (\n        rmtemp,\n        OBSCURE_FILENAME,\n    )\n    from datalad.ui import ui\n\n    lgr.debug(\"Printing versioning information collected so far\")\n    print(ev.dumps(query=True))\n    try:\n        print(\"Obscure filename: str=%s repr=%r\"\n                % (OBSCURE_FILENAME.encode('utf-8'), OBSCURE_FILENAME))\n    except UnicodeEncodeError as exc:\n        ce = CapturedException(exc)\n        print(\"Obscure filename failed to print: %s\" % ce)\n    def print_dict(d):\n        return \" \".join(\"%s=%r\" % v for v in d.items())\n    print(\"Encodings: %s\" % print_dict(get_encoding_info()))\n    print(\"Environment: %s\" % print_dict(get_envvars_info()))\n\n    if os.environ.get('DATALAD_TESTS_NOTEARDOWN'):\n        return\n    ui.set_backend(_test_states['ui_backend'])\n    if _test_states['loglevel'] is not None:\n        lgr.setLevel(_test_states['loglevel'])\n\n    global test_http_server\n    if test_http_server:\n        test_http_server.stop()\n        test_http_server = None\n    else:\n        lgr.debug(\"For some reason global http_server was not set/running, thus not stopping\")\n\n    if len(_TEMP_PATHS_GENERATED):\n        msg = \"Removing %d dirs/files: %s\" % (len(_TEMP_PATHS_GENERATED), ', '.join(_TEMP_PATHS_GENERATED))\n    else:\n        msg = \"Nothing to remove\"\n    lgr.debug(\"Teardown tests. \" + msg)\n    for path in _TEMP_PATHS_GENERATED:\n        rmtemp(str(path), ignore_errors=True)\n\n    # restore all the env variables\n    for v, val in _test_states['env'].items():\n        if val is not None:\n            os.environ[v] = val\n        else:\n            os.environ.pop(v)\n\n    # Re-establish correct global config after changing $HOME.\n    # Might be superfluous, since after teardown datalad.cfg shouldn't be\n    # needed. However, maintaining a consistent state seems a good thing\n    # either way.\n    cfg.reload(force=True)\n\n    compute_cfg_defaults()\n    ssh_manager._socket_dir = None\n\n    consts.DATASETS_TOPURL = _test_states['DATASETS_TOPURL']\n\n    cookies_db.close()\n    AnnexRepo._ALLOW_LOCAL_URLS = False  # stay safe!\n\n\nfrom ._version import get_versions\n__version__ = get_versions()['version']\ndel get_versions\n\nif str(__version__) == '0' or __version__.startswith('0+'):\n    lgr.warning(\n        \"DataLad was not installed 'properly' so its version is an uninformative %r.\\n\"\n        \"It can happen e.g. if datalad was installed via\\n\"\n        \"  pip install https://github.com/.../archive/{commitish}.zip\\n\"\n        \"instead of\\n\"\n        \"  pip install git+https://github.com/...@{commitish} .\\n\"\n        \"We advise to re-install datalad or downstream projects might not operate correctly.\",\n        __version__\n    )\n\nlgr.log(5, \"Done importing main __init__\")\n",
            "header": {
                "file": {
                    "path": "datalad/__init__.py"
                },
                "misc_line": null
            },
            "index_line": {
                "old_commit_hash": "fb96d539e",
                "new_commit_hash": "06be087fa",
                "mode": "100644"
            },
            "is_binary_file": false,
            "binary_line": null,
            "minus_file": {
                "path": "a/datalad/__init__.py"
            },
            "plus_file": {
                "path": "b/datalad/__init__.py"
            },
            "hunks": [
                {
                    "descriptor": {
                        "old_range": {
                            "start": 25,
                            "length": 10
                        },
                        "new_range": {
                            "start": 25,
                            "length": 6
                        },
                        "section": "import os"
                    },
                    "line_group": {
                        "all_lines": [
                            {
                                "content": "",
                                "type": "context"
                            },
                            {
                                "content": "# this is not to be modified. for querying use get_apimode()",
                                "type": "context"
                            },
                            {
                                "content": "__api = 'python'",
                                "type": "context"
                            },
                            {
                                "content": "# this is not to be modified. see enable/in_librarymode()",
                                "type": "deleted"
                            },
                            {
                                "content": "# by default, we are in application-mode, simply because most of",
                                "type": "deleted"
                            },
                            {
                                "content": "# datalad was originally implemented with this scenario assumption",
                                "type": "deleted"
                            },
                            {
                                "content": "__runtime_mode = 'application'",
                                "type": "deleted"
                            },
                            {
                                "content": "",
                                "type": "context"
                            },
                            {
                                "content": "",
                                "type": "context"
                            },
                            {
                                "content": "def get_apimode():",
                                "type": "context"
                            }
                        ]
                    },
                    "modified_entities": [],
                    "added_entities": [],
                    "deleted_entities": [
                        {
                            "file_name": "datalad/__init__.py",
                            "type": "statement",
                            "name": "",
                            "content": "__runtime_mode = 'application'",
                            "ast_type_str": "Assign",
                            "start_lineno": 31,
                            "end_lineno": 31,
                            "parent": null
                        }
                    ]
                },
                {
                    "descriptor": {
                        "old_range": {
                            "start": 83,
                            "length": 6
                        },
                        "new_range": {
                            "start": 79,
                            "length": 8
                        },
                        "section": "def enable_librarymode():"
                    },
                    "line_group": {
                        "all_lines": [
                            {
                                "content": "    \"\"\"",
                                "type": "context"
                            },
                            {
                                "content": "    global __runtime_mode",
                                "type": "context"
                            },
                            {
                                "content": "    __runtime_mode = 'library'",
                                "type": "context"
                            },
                            {
                                "content": "    # export into the environment for child processes to inherit",
                                "type": "added"
                            },
                            {
                                "content": "    os.environ['DATALAD_RUNTIME_LIBRARYMODE'] = '1'",
                                "type": "added"
                            },
                            {
                                "content": "",
                                "type": "context"
                            },
                            {
                                "content": "",
                                "type": "context"
                            },
                            {
                                "content": "# For reproducible demos/tests",
                                "type": "context"
                            }
                        ]
                    },
                    "modified_entities": [
                        {
                            "file_name": "datalad/__init__.py",
                            "type": "function",
                            "name": "enable_librarymode",
                            "content": "def enable_librarymode():\n    \"\"\"Request DataLad to operate in library mode.\n\n    This function should be executed immediately after importing the `datalad`\n    package, when DataLad is not used as an application, or in interactive\n    scenarios, but as a utility library inside other applications. Enabling\n    this mode will turn off some convenience feature that are irrelevant in\n    such use cases (with performance benefits), and alters it messaging\n    behavior to better interoperate with 3rd-party front-ends.\n\n    Library mode can only be enabled once. Switching it on and off within\n    the runtime of a process is not supported.\n\n    Example::\n\n        >>> import datalad\n        >>> datalad.enable_librarymode()\n    \"\"\"\n    global __runtime_mode\n    __runtime_mode = 'library'\n    # export into the environment for child processes to inherit\n    os.environ['DATALAD_RUNTIME_LIBRARYMODE'] = '1'",
                            "ast_type_str": "FunctionDef",
                            "start_lineno": 62,
                            "end_lineno": 83,
                            "parent": null
                        }
                    ],
                    "added_entities": [],
                    "deleted_entities": []
                },
                {
                    "descriptor": {
                        "old_range": {
                            "start": 112,
                            "length": 6
                        },
                        "new_range": {
                            "start": 110,
                            "length": 13
                        },
                        "section": "from .config import ConfigManager"
                    },
                    "line_group": {
                        "all_lines": [
                            {
                                "content": "cfg = ConfigManager()",
                                "type": "context"
                            },
                            {
                                "content": "",
                                "type": "context"
                            },
                            {
                                "content": "# must come after config manager",
                                "type": "context"
                            },
                            {
                                "content": "# this is not to be modified. see enable/in_librarymode()",
                                "type": "added"
                            },
                            {
                                "content": "# by default, we are in application-mode, simply because most of",
                                "type": "added"
                            },
                            {
                                "content": "# datalad was originally implemented with this scenario assumption",
                                "type": "added"
                            },
                            {
                                "content": "__runtime_mode = 'library' \\",
                                "type": "added"
                            },
                            {
                                "content": "    if cfg.getbool('datalad.runtime', 'librarymode', False) \\",
                                "type": "added"
                            },
                            {
                                "content": "    else 'application'",
                                "type": "added"
                            },
                            {
                                "content": "",
                                "type": "added"
                            },
                            {
                                "content": "from .log import lgr",
                                "type": "context"
                            },
                            {
                                "content": "from datalad.support.exceptions import CapturedException",
                                "type": "context"
                            },
                            {
                                "content": "from datalad.utils import (",
                                "type": "context"
                            }
                        ]
                    },
                    "modified_entities": [],
                    "added_entities": [
                        {
                            "file_name": "datalad/__init__.py",
                            "type": "statement",
                            "name": "",
                            "content": "__runtime_mode = 'library' \\\n    if cfg.getbool('datalad.runtime', 'librarymode', False) \\\n    else 'application'",
                            "ast_type_str": "Assign",
                            "start_lineno": 116,
                            "end_lineno": 118,
                            "parent": null
                        }
                    ],
                    "deleted_entities": []
                }
            ]
        },
        {
            "old_file_content": "# emacs: -*- mode: python; py-indent-offset: 4; tab-width: 4; indent-tabs-mode: nil -*-\n# ex: set sts=4 ts=4 sw=4 et:\n# ## ### ### ### ### ### ### ### ### ### ### ### ### ### ### ### ### ### ### ##\n#\n#   See COPYING file distributed along with the datalad package for the\n#   copyright and license terms.\n#\n# ## ### ### ### ### ### ### ### ### ### ### ### ### ### ### ### ### ### ### ##\n\"\"\"\n\"\"\"\n\n__docformat__ = 'restructuredtext'\n\nimport argparse\nimport logging\nimport os\nimport re\nimport sys\nimport gzip\nimport textwrap\nimport warnings\nfrom tempfile import NamedTemporaryFile\nfrom textwrap import wrap\n\nfrom datalad import __version__\nfrom ..cmd import WitlessRunner as Runner\nfrom ..interface.common_opts import (\n    eval_defaults,\n    eval_params,\n)\nfrom ..log import is_interactive\nfrom datalad.support.exceptions import CapturedException\nfrom ..ui.utils import get_console_width\nfrom ..utils import (\n    ensure_unicode,\n    getpwd,\n    unlink,\n    get_suggestions_msg,\n)\n\nfrom platformdirs import AppDirs\nfrom os.path import join as opj\n\ndirs = AppDirs(\"datalad\", \"datalad.org\")\n\n\nfrom logging import getLogger\nlgr = getLogger('datalad.cmdline')\n\n\nclass HelpAction(argparse.Action):\n    def __call__(self, parser, namespace, values, option_string=None):\n        # Lets use the manpage on mature systems but only for subcommands --\n        # --help should behave similar to how git does it:\n        # regular --help for \"git\" but man pages for specific commands.\n        # It is important since we do discover all subcommands from entry\n        # points at run time and thus any static manpage would like be out of\n        # date\n        interactive = is_interactive()\n        if interactive \\\n                and option_string == '--help' \\\n                and ' ' in parser.prog:  # subcommand\n            try:\n                import subprocess\n                # get the datalad manpage to use\n                manfile = os.environ.get('MANPATH', '/usr/share/man') \\\n                    + '/man1/{0}.1.gz'.format(parser.prog.replace(' ', '-'))\n                # extract version field from the manpage\n                if not os.path.exists(manfile):\n                    raise IOError(\"manfile is not found\")\n                with gzip.open(manfile) as f:\n                    man_th = [line for line in f if line.startswith(b\".TH\")][0]\n                man_version = man_th.split(b' ')[-1].strip(b\" '\\\"\\t\\n\").decode('utf-8')\n\n                # don't show manpage if man_version not equal to current datalad_version\n                if __version__ != man_version:\n                    raise ValueError\n                subprocess.check_call(\n                    'man %s 2> /dev/null' % manfile,\n                    shell=True)\n                sys.exit(0)\n            except (subprocess.CalledProcessError, IOError, OSError, IndexError, ValueError) as e:\n                ce = CapturedException(e)\n                lgr.debug(\"Did not use manpage since %s\", ce)\n        if option_string == '-h':\n            usage = parser.format_usage()\n            ucomps = re.match(\n                r'(?P<pre>.*){(?P<cmds>.*)}(?P<post>....*)',\n                usage,\n                re.DOTALL)\n            if ucomps:\n                ucomps = ucomps.groupdict()\n                indent_level = len(ucomps['post']) - len(ucomps['post'].lstrip())\n                usage = '{pre}{{{cmds}}}{post}'.format(\n                    pre=ucomps['pre'],\n                    cmds='\\n'.join(wrap(\n                        ', '.join(sorted(c.strip() for c in ucomps['cmds'].split(','))),\n                        break_on_hyphens=False,\n                        subsequent_indent=' ' * indent_level)),\n                    post=ucomps['post'],\n                )\n            helpstr = \"%s\\n%s\" % (\n                usage,\n                \"Use '--help' to get more comprehensive information.\")\n        else:\n            helpstr = parser.format_help()\n        # better for help2man\n        # for main command -- should be different sections. And since we are in\n        # heavy output massaging mode...\n        if \"essential commands\" in helpstr.lower():\n            opt_args_str = '*Global options*'\n            pos_args_str = '*Commands*'\n            # tune up usage -- default one is way too heavy\n            helpstr = re.sub(r'^[uU]sage: .*?\\n\\s*\\n',\n                             'Usage: datalad [global-opts] command [command-opts]\\n\\n',\n                             helpstr,\n                             flags=re.MULTILINE | re.DOTALL)\n            # and altogether remove sections with long list of commands\n            helpstr = re.sub(r'positional arguments:\\s*\\n\\s*{.*}\\n', '', helpstr)\n        else:\n            opt_args_str = \"*Options*\"\n            pos_args_str = \"*Arguments*\"\n        helpstr = re.sub(r'optional arguments:', opt_args_str, helpstr)\n        helpstr = re.sub(r'positional arguments:', pos_args_str, helpstr)\n        # usage is on the same line\n        helpstr = re.sub(r'^usage:', 'Usage:', helpstr)\n\n        if interactive and option_string == '--help':\n            import pydoc\n            pydoc.pager(helpstr)\n        else:\n            print(helpstr)\n        sys.exit(0)\n\n\nclass LogLevelAction(argparse.Action):\n    def __call__(self, parser, namespace, values, option_string=None):\n        from ..log import LoggerHelper\n        LoggerHelper().set_level(level=values)\n\n\nclass ArgumentParserDisableAbbrev(argparse.ArgumentParser):\n    # Don't accept abbreviations for long options. This kludge was originally\n    # added at a time when our minimum required Python version was below 3.5,\n    # preventing us from using allow_abbrev=False. Now our minimum Python\n    # version is high enough, but we still can't use allow_abbrev=False because\n    # it suffers from the problem described in 6b3f2fffe (BF: cmdline: Restore\n    # handling of short options, 2018-07-23).\n    #\n    # Modified from the solution posted at\n    # https://bugs.python.org/issue14910#msg204678\n    def _get_option_tuples(self, option_string):\n        chars = self.prefix_chars\n        if option_string[0] in chars and option_string[1] in chars:\n            # option_string is a long flag. Disable abbreviation.\n            return []\n        return super(ArgumentParserDisableAbbrev, self)._get_option_tuples(\n            option_string)\n\n\ndef parser_add_version_opt(parser, mod_name, include_name=False, delay=False):\n    \"\"\"Setup --version option\n\n    Parameters\n    ----------\n    parser:\n    mod_name: str, optional\n    include_name: bool, optional\n    delay: bool, optional\n      If set to True, no action is taken immediately, and rather\n      we assign the function which would print the version. Necessary for\n      early pre-parsing of the cmdline\n    \"\"\"\n\n    def print_version():\n        mod = sys.modules.get(mod_name, None)\n        version = getattr(mod, '__version__', None)\n        if version is None:\n            # Let's use the standard Python mechanism if underlying module\n            # did not provide __version__\n            try:\n                import pkg_resources\n                version = pkg_resources.get_distribution(mod_name).version\n            except:\n                version = \"unknown\"\n        if include_name:\n            print(\"%s %s\" % (mod_name, version))\n        else:\n            print(version)\n        sys.exit(0)\n\n    class versionAction(argparse.Action):\n        def __call__(self, parser, args, values, option_string=None):\n            if delay:\n                setattr(args, self.dest, print_version)\n            else:\n                print_version()\n\n    parser.add_argument(\n        \"--version\",\n        nargs=0,\n        action=versionAction,\n        help=(\n            \"show the program's version\"\n            if not mod_name\n            else \"show the module and its version which provides the command\")\n    )\n\n\ndef parser_add_common_opt(parser, opt, names=None, **kwargs):\n    from . import common_args\n    opt_tmpl = getattr(common_args, opt)\n    opt_kwargs = opt_tmpl[2].copy()\n    opt_kwargs.update(kwargs)\n    if names is None:\n        parser.add_argument(*opt_tmpl[1], **opt_kwargs)\n    else:\n        parser.add_argument(*names, **opt_kwargs)\n\n\ndef parser_add_common_options(parser, version=None):\n    parser_add_common_opt(parser, 'log_level')\n    parser_add_common_opt(parser, 'change_path')\n    if version is not None:\n        warnings.warn(\"Passing 'version' to parser_add_common_options \"\n                      \"no longer has an effect \"\n                      \"and will be removed in a future release.\",\n                      DeprecationWarning)\n    parser_add_version_opt(parser, 'datalad', include_name=True, delay=True)\n    if __debug__:\n        parser.add_argument(\n            '--dbg', action='store_true', dest='common_debug',\n            help=\"enter Python debugger when uncaught exception happens\")\n        parser.add_argument(\n            '--idbg', action='store_true', dest='common_idebug',\n            help=\"enter IPython debugger when uncaught exception happens\")\n    parser.add_argument(\n        '-c', action='append', dest='cfg_overrides', metavar='KEY=VALUE',\n        help=\"\"\"configuration variable setting. Overrides any configuration\n        read from a file, but is potentially overridden itself by configuration\n        variables in the process environment.\"\"\")\n    # CLI analog of eval_params.result_renderer but with `<template>` handling\n    # and a different default: in Python API we have None as default and do not render\n    # the results but return them.  In CLI we default to \"default\" renderer\n    parser.add_argument(\n        # this should really have --result-renderer for homogeneity with the\n        # Python API, but adding it in addition makes the help output\n        # monsterous\n        '-f', '--output-format', # '--result-renderer',\n        dest='common_result_renderer',\n        default='tailored',\n        type=ensure_unicode,\n        metavar=\"{generic,json,json_pp,tailored,disabled,'<template>'}\",\n        help=eval_params['result_renderer']._doc + \" [Default: '%(default)s']\")\n    parser.add_argument(\n        '--report-status', dest='common_report_status',\n        choices=['success', 'failure', 'ok', 'notneeded', 'impossible', 'error'],\n        help=\"\"\"constrain command result report to records matching the given\n        status. 'success' is a synonym for 'ok' OR 'notneeded', 'failure' stands\n        for 'impossible' OR 'error'.\"\"\")\n    parser.add_argument(\n        '--report-type', dest='common_report_type',\n        choices=['dataset', 'file'],\n        action='append',\n        help=\"\"\"constrain command result report to records matching the given\n        type. Can be given more than once to match multiple types.\"\"\")\n    # CLI analog of eval_params.on_failure. TODO: dedup\n    parser.add_argument(\n        '--on-failure', dest='common_on_failure',\n        default=eval_defaults['on_failure'],\n        choices=['ignore', 'continue', 'stop'],\n        help=\"\"\"when an operation fails: 'ignore' and continue with remaining\n        operations, the error is logged but does not lead to a non-zero exit code\n        of the command; 'continue' works like 'ignore', but an error causes a\n        non-zero exit code; 'stop' halts on first failure and yields non-zero exit\n        code. A failure is any result with status 'impossible' or 'error'.\n        [Default: '%(default)s']\"\"\")\n    parser.add_argument(\n        '--library-mode', dest='common_library_mode', action='store_true',\n        help=\"\"\"Enable operation as a utility library (rather than a front-end\n        application). This may improve performance, because convenience\n        functionality will be disabled, and will alter the error reporting to\n        be better suited for internal use.\"\"\")\n    parser.add_argument(\n        '--cmd', dest='_', action='store_true',\n        help=\"\"\"syntactical helper that can be used to end the list of global\n        command line options before the subcommand label. Options taking\n        an arbitrary number of arguments may require to be followed by a single\n        --cmd in order to enable identification of the subcommand.\"\"\")\n\n\ndef strip_arg_from_argv(args, value, opt_names):\n    \"\"\"Strip an originally listed option (with its value) from the list cmdline args\n    \"\"\"\n    # Yarik doesn't know better\n    if args is None:\n        args = sys.argv\n    # remove present opt_names\n    args_clean = []\n    skip = 0\n    for i, arg in enumerate(args):\n        if skip:\n            # we skip only one as instructed\n            skip -= 1\n            continue\n        if not (arg in opt_names and i < len(args) - 1 and args[i + 1] == value):\n            args_clean.append(arg)\n        else:\n            # we need to skip this one and next one\n            skip = 1\n    return args_clean\n\n\ndef get_repo_instance(path=os.curdir, class_=None):\n    \"\"\"Returns an instance of appropriate datalad repository for path.\n    Check whether a certain path is inside a known type of repository and\n    returns an instance representing it. May also check for a certain type\n    instead of detecting the type of repository.\n\n    Parameters\n    ----------\n    path: str\n      path to check; default: current working directory\n    class_: class\n      if given, check whether path is inside a repository, that can be\n      represented as an instance of the passed class.\n\n    Raises\n    ------\n    RuntimeError, in case cwd is not inside a known repository.\n    \"\"\"\n\n    from os.path import ismount, exists, normpath, isabs\n    from datalad.support.exceptions import InvalidGitRepositoryError\n    from ..utils import expandpath\n    from ..support.gitrepo import GitRepo\n    from ..support.annexrepo import AnnexRepo\n\n    dir_ = expandpath(path)\n    abspath_ = path if isabs(path) else dir_\n    if class_ is not None:\n        if class_ == AnnexRepo:\n            type_ = \"annex\"\n        elif class_ == GitRepo:\n            type_ = \"git\"\n        else:\n            raise RuntimeError(\"Unknown class %s.\" % str(class_))\n\n    while not ismount(dir_):  # TODO: always correct termination?\n        if exists(opj(dir_, '.git')):\n            # found git dir\n            if class_ is None:\n                # detect repo type:\n                try:\n                    return AnnexRepo(dir_, create=False)\n                except RuntimeError as e:\n                    pass\n                try:\n                    return GitRepo(dir_, create=False)\n                except InvalidGitRepositoryError as e:\n                    raise RuntimeError(\"No datalad repository found in %s\" %\n                                       abspath_)\n            else:\n                try:\n                    return class_(dir_, create=False)\n                except (RuntimeError, InvalidGitRepositoryError) as e:\n                    raise RuntimeError(\"No %s repository found in %s.\" %\n                                       (type_, abspath_))\n        else:\n            dir_ = normpath(opj(dir_, \"..\"))\n\n    if class_ is not None:\n        raise RuntimeError(\"No %s repository found in %s\" % (type_, abspath_))\n    else:\n        raise RuntimeError(\"No datalad repository found in %s\" % abspath_)\n\n#\n# Some logic modules extracted from main.py to de-spagetify\n#\n\ndef _maybe_get_single_subparser(cmdlineargs, parser, interface_groups,\n                                return_subparsers, completing, help_ignore_extensions):\n    \"\"\"Performs early analysis of the cmdline\n\n    Looks at the first unparsed argument and if a known command, would return_subparsers\n    is False, would return only that one.\n\n    For the analysis to be complete etc, would also load commands from entrypoints etc\n\n    Returns\n    -------\n    None or str\n    \"\"\"\n    # Before doing anything additional and possibly expensive see may be that\n    # we have got the command already\n    need_single_subparser = False if return_subparsers else None\n    fail_handler = (lambda *a, **kw: True) \\\n        if return_subparsers else fail_with_short_help\n    try:\n        parsed_args, unparsed_args = parser._parse_known_args(\n            cmdlineargs[1:], argparse.Namespace())\n        # before anything handle possible datalad --version\n        if not unparsed_args and getattr(parsed_args, 'version', None):\n            parsed_args.version()  # will exit with 0\n        if not (completing or unparsed_args):\n            fail_handler(\n                parser,\n                msg=\"too few arguments, run with --help or visit https://handbook.datalad.org\",\n                exit_code=2)\n        lgr.debug(\"Command line args 1st pass for DataLad %s. Parsed: %s Unparsed: %s\",\n                  __version__, parsed_args, unparsed_args)\n    except Exception as exc:\n        ce = CapturedException(exc)\n        lgr.debug(\"Early parsing failed with %s\", ce)\n        need_single_subparser = False\n        unparsed_args = cmdlineargs[1:]  # referenced before assignment otherwise\n    # First unparsed could be either unknown option to top level \"datalad\"\n    # or a command. Among unknown could be --help/--help-np which would\n    # need to be dealt with\n    unparsed_arg = unparsed_args[0] if unparsed_args else None\n    if need_single_subparser is not None \\\n            or unparsed_arg in ('--help', '--help-np', '-h'):\n        need_single_subparser = False\n        if not help_ignore_extensions:\n            add_entrypoints_to_interface_groups(interface_groups)\n    elif not completing and unparsed_arg.startswith('-'):  # unknown option\n        fail_with_short_help(parser,\n                             msg=\"unrecognized argument %s\" % unparsed_arg,\n                             exit_code=2)\n        # if we could get a list of options known to parser,\n        # we could suggest them\n        # known=get_all_options(parser), provided=unparsed_arg)\n    else:  # the command to handle\n        known_commands = get_commands_from_groups(interface_groups)\n        if unparsed_arg not in known_commands:\n            # need to load all the extensions and try again\n            add_entrypoints_to_interface_groups(interface_groups)\n            known_commands = get_commands_from_groups(interface_groups)\n\n        if unparsed_arg not in known_commands:\n            # check if might be coming from known extensions\n            from ..interface import (\n                _known_extension_commands,\n                _deprecated_commands,\n            )\n            extension_commands = {\n                c: e\n                for e, commands in _known_extension_commands.items()\n                for c in commands\n            }\n            hint = None\n            if unparsed_arg in extension_commands:\n                hint = \"Command %s is provided by (not installed) extension %s.\" \\\n                       % (unparsed_arg, extension_commands[unparsed_arg])\n            elif unparsed_arg in _deprecated_commands:\n                hint_cmd = _deprecated_commands[unparsed_arg]\n                hint = \"Command %r was deprecated\" % unparsed_arg\n                hint += (\" in favor of %r command.\" % hint_cmd) if hint_cmd else '.'\n            if not completing:\n                fail_with_short_help(\n                    parser,\n                    hint=hint,\n                    provided=unparsed_arg,\n                    known=list(known_commands.keys()) + list(extension_commands.keys())\n                )\n        if need_single_subparser is None:\n            need_single_subparser = unparsed_arg\n    return need_single_subparser\n\n\ndef _maybe_get_interface_subparser(_intfspec, subparsers, cmd_name, formatter_class, group_name,\n                                   grp_short_descriptions):\n    \"\"\"Given an interface spec, add a subparser to subparsers under cmd_name\n\n    That subparser is also gets added to the grp_short_descriptions\n    \"\"\"\n    from ..interface.base import (\n        alter_interface_docs_for_cmdline,\n        get_cmd_doc,\n        get_cmd_ex,\n        load_interface,\n    )\n    _intf = load_interface(_intfspec)\n    if _intf is None:  # failed to load\n        return\n    # deal with optional parser args\n    if hasattr(_intf, 'parser_args'):\n        parser_args = _intf.parser_args\n    else:\n        parser_args = dict(formatter_class=formatter_class)\n        # use class description, if no explicit description is available\n        intf_doc = get_cmd_doc(_intf)\n        parser_args['description'] = alter_interface_docs_for_cmdline(\n            intf_doc)\n        if hasattr(_intf, '_examples_'):\n            intf_ex = alter_interface_docs_for_cmdline(get_cmd_ex(_intf))\n            parser_args['description'] += intf_ex\n    subparser = subparsers.add_parser(cmd_name, add_help=False, **parser_args)\n    # our own custom help for all commands\n    parser_add_common_opt(subparser, 'help')\n    # let module configure the parser\n    _intf.setup_parser(subparser)\n    # and we would add custom handler for --version\n    parser_add_version_opt(subparser, _intf.__module__.split('.', 1)[0], include_name=True)\n    # logger for command\n    # configure 'run' function for this command\n    plumbing_args = dict(\n        func=_intf.call_from_parser,\n        logger=logging.getLogger(_intf.__module__),\n        subparser=subparser)\n    if hasattr(_intf, 'result_renderer_cmdline'):\n        plumbing_args['result_renderer'] = _intf.result_renderer_cmdline\n    subparser.set_defaults(**plumbing_args)\n    # store short description for later\n    sdescr = getattr(_intf, 'short_description',\n                     parser_args['description'].split('\\n')[0])\n    grp_short_descriptions[group_name].append((cmd_name, sdescr))\n    return subparser\n\n\ndef add_entrypoints_to_interface_groups(interface_groups):\n    lgr.debug(\"Loading entrypoints\")\n    from pkg_resources import iter_entry_points  # delay expensive import\n    for ep in iter_entry_points('datalad.extensions'):\n        lgr.debug(\n            'Loading entrypoint %s from datalad.extensions for docs building',\n            ep.name)\n        try:\n            spec = ep.load()\n            if len(spec) < 2 or not spec[1]:\n                lgr.debug(\n                    'Extension does not provide a command suite: %s',\n                    ep.name)\n                continue\n            interface_groups.append((ep.name, spec[0], spec[1]))\n            lgr.debug('Loaded entrypoint %s', ep.name)\n        except Exception as e:\n            ce = CapturedException(ce)\n            lgr.warning('Failed to load entrypoint %s: %s', ep.name, ce)\n            continue\n\n\ndef get_commands_from_groups(groups):\n    \"\"\"Get a dictionary of command: interface_spec\"\"\"\n    from ..interface.base import get_cmdline_command_name\n    return {\n        get_cmdline_command_name(_intfspec): _intfspec\n        for _, _, _interfaces in groups\n        for _intfspec in _interfaces\n    }\n\n\ndef fail_with_short_help(parser=None,\n                         msg=None,\n                         known=None, provided=None,\n                         hint=None,\n                         exit_code=1,\n                         what=\"command\",\n                         out=None):\n    \"\"\"Generic helper to fail\n    with short help possibly hinting on what was intended if `known`\n    were provided\n    \"\"\"\n    out = out or sys.stderr\n    if msg:\n        out.write(\"error: %s\\n\" % msg)\n    if not known:\n        if parser:\n            parser_add_common_opt(parser, 'help')\n            # just to appear in print_usage also consistent with --help output\n            parser.add_argument(\"command [command-opts]\")\n            parser.print_usage(file=out)\n    else:\n        out.write(\n            \"datalad: Unknown %s %r.  See 'datalad --help'.\\n\\n\"\n            % (what, provided,))\n        if provided not in known:\n            out.write(get_suggestions_msg(provided, known))\n    if hint:\n        out.write(\"Hint: %s\\n\" % hint)\n    raise SystemExit(exit_code)\n\n\ndef _fix_datalad_ri(s):\n    \"\"\"Fixup argument if it was a DataLadRI and had leading / removed\n\n    See gh-2643\n    \"\"\"\n    if s.startswith('//') and (len(s) == 2 or (len(s) > 2 and s[2] != '/')):\n        lgr.info(\n            \"Changing %s back to /%s as it was probably changed by MINGW/MSYS, \"\n            \"see http://www.mingw.org/wiki/Posix_path_conversion\", s, s)\n        return \"/\" + s\n    return s\n\n\ndef get_description_with_cmd_summary(grp_short_descriptions, interface_groups,\n                                     parser_description):\n    from ..interface.base import dedent_docstring\n    from ..interface.base import get_cmd_summaries\n    lgr.debug(\"Generating detailed description for the parser\")\n\n    console_width = get_console_width()\n    cmd_summary = get_cmd_summaries(grp_short_descriptions, interface_groups,\n                                    width=console_width)\n    # we need one last formal section to not have the trailed be\n    # confused with the last command group\n    cmd_summary.append('\\n*General information*\\n')\n    detailed_description = '%s\\n%s\\n\\n%s' \\\n                           % (parser_description,\n                              '\\n'.join(cmd_summary),\n                              textwrap.fill(dedent_docstring(\"\"\"\\\n    Detailed usage information for individual commands is\n    available via command-specific --help, i.e.:\n    datalad <command> --help\"\"\"),\n                                            console_width - 5,\n                                            initial_indent='',\n                                            subsequent_indent=''))\n    return detailed_description\n\n\ndef _parse_overrides_from_cmdline(cmdlineargs):\n    \"\"\"parse config overrides provided in command line\n\n    Might exit(3) the entire process if value is not assigned\"\"\"\n    # this expression is deliberately loose as gitconfig offers\n    # quite some flexibility -- this is just meant to catch stupid\n    # errors: we need a section, a variable, and a value at minimum\n    # otherwise we break our own config parsing helpers\n    # https://github.com/datalad/datalad/issues/3451\n    noassign_expr = re.compile(r'[^\\s]+\\.[^\\s]+=[\\S]+')\n    noassign = [\n        o\n        for o in cmdlineargs.cfg_overrides\n        if not noassign_expr.match(o)\n    ]\n    if noassign:\n        lgr.error(\n            \"Configuration override without section/variable \"\n            \"or value assignment (must be 'section.variable=value'): %s\",\n            noassign)\n        sys.exit(3)\n    overrides = dict(o.split('=', 1) for o in cmdlineargs.cfg_overrides)\n    return overrides\n",
            "new_file_content": "# emacs: -*- mode: python; py-indent-offset: 4; tab-width: 4; indent-tabs-mode: nil -*-\n# ex: set sts=4 ts=4 sw=4 et:\n# ## ### ### ### ### ### ### ### ### ### ### ### ### ### ### ### ### ### ### ##\n#\n#   See COPYING file distributed along with the datalad package for the\n#   copyright and license terms.\n#\n# ## ### ### ### ### ### ### ### ### ### ### ### ### ### ### ### ### ### ### ##\n\"\"\"\n\"\"\"\n\n__docformat__ = 'restructuredtext'\n\nimport argparse\nimport logging\nimport os\nimport re\nimport sys\nimport gzip\nimport textwrap\nimport warnings\nfrom tempfile import NamedTemporaryFile\nfrom textwrap import wrap\n\nfrom datalad import __version__\nfrom ..cmd import WitlessRunner as Runner\nfrom ..interface.common_opts import (\n    eval_defaults,\n    eval_params,\n)\nfrom ..log import is_interactive\nfrom datalad.support.exceptions import CapturedException\nfrom ..ui.utils import get_console_width\nfrom ..utils import (\n    ensure_unicode,\n    getpwd,\n    unlink,\n    get_suggestions_msg,\n)\n\nfrom platformdirs import AppDirs\nfrom os.path import join as opj\n\ndirs = AppDirs(\"datalad\", \"datalad.org\")\n\n\nfrom logging import getLogger\nlgr = getLogger('datalad.cmdline')\n\n\nclass HelpAction(argparse.Action):\n    def __call__(self, parser, namespace, values, option_string=None):\n        # Lets use the manpage on mature systems but only for subcommands --\n        # --help should behave similar to how git does it:\n        # regular --help for \"git\" but man pages for specific commands.\n        # It is important since we do discover all subcommands from entry\n        # points at run time and thus any static manpage would like be out of\n        # date\n        interactive = is_interactive()\n        if interactive \\\n                and option_string == '--help' \\\n                and ' ' in parser.prog:  # subcommand\n            try:\n                import subprocess\n                # get the datalad manpage to use\n                manfile = os.environ.get('MANPATH', '/usr/share/man') \\\n                    + '/man1/{0}.1.gz'.format(parser.prog.replace(' ', '-'))\n                # extract version field from the manpage\n                if not os.path.exists(manfile):\n                    raise IOError(\"manfile is not found\")\n                with gzip.open(manfile) as f:\n                    man_th = [line for line in f if line.startswith(b\".TH\")][0]\n                man_version = man_th.split(b' ')[-1].strip(b\" '\\\"\\t\\n\").decode('utf-8')\n\n                # don't show manpage if man_version not equal to current datalad_version\n                if __version__ != man_version:\n                    raise ValueError\n                subprocess.check_call(\n                    'man %s 2> /dev/null' % manfile,\n                    shell=True)\n                sys.exit(0)\n            except (subprocess.CalledProcessError, IOError, OSError, IndexError, ValueError) as e:\n                ce = CapturedException(e)\n                lgr.debug(\"Did not use manpage since %s\", ce)\n        if option_string == '-h':\n            usage = parser.format_usage()\n            ucomps = re.match(\n                r'(?P<pre>.*){(?P<cmds>.*)}(?P<post>....*)',\n                usage,\n                re.DOTALL)\n            if ucomps:\n                ucomps = ucomps.groupdict()\n                indent_level = len(ucomps['post']) - len(ucomps['post'].lstrip())\n                usage = '{pre}{{{cmds}}}{post}'.format(\n                    pre=ucomps['pre'],\n                    cmds='\\n'.join(wrap(\n                        ', '.join(sorted(c.strip() for c in ucomps['cmds'].split(','))),\n                        break_on_hyphens=False,\n                        subsequent_indent=' ' * indent_level)),\n                    post=ucomps['post'],\n                )\n            helpstr = \"%s\\n%s\" % (\n                usage,\n                \"Use '--help' to get more comprehensive information.\")\n        else:\n            helpstr = parser.format_help()\n        # better for help2man\n        # for main command -- should be different sections. And since we are in\n        # heavy output massaging mode...\n        if \"essential commands\" in helpstr.lower():\n            opt_args_str = '*Global options*'\n            pos_args_str = '*Commands*'\n            # tune up usage -- default one is way too heavy\n            helpstr = re.sub(r'^[uU]sage: .*?\\n\\s*\\n',\n                             'Usage: datalad [global-opts] command [command-opts]\\n\\n',\n                             helpstr,\n                             flags=re.MULTILINE | re.DOTALL)\n            # and altogether remove sections with long list of commands\n            helpstr = re.sub(r'positional arguments:\\s*\\n\\s*{.*}\\n', '', helpstr)\n        else:\n            opt_args_str = \"*Options*\"\n            pos_args_str = \"*Arguments*\"\n        helpstr = re.sub(r'optional arguments:', opt_args_str, helpstr)\n        helpstr = re.sub(r'positional arguments:', pos_args_str, helpstr)\n        # usage is on the same line\n        helpstr = re.sub(r'^usage:', 'Usage:', helpstr)\n\n        if interactive and option_string == '--help':\n            import pydoc\n            pydoc.pager(helpstr)\n        else:\n            print(helpstr)\n        sys.exit(0)\n\n\nclass LogLevelAction(argparse.Action):\n    def __call__(self, parser, namespace, values, option_string=None):\n        from ..log import LoggerHelper\n        LoggerHelper().set_level(level=values)\n\n\nclass ArgumentParserDisableAbbrev(argparse.ArgumentParser):\n    # Don't accept abbreviations for long options. This kludge was originally\n    # added at a time when our minimum required Python version was below 3.5,\n    # preventing us from using allow_abbrev=False. Now our minimum Python\n    # version is high enough, but we still can't use allow_abbrev=False because\n    # it suffers from the problem described in 6b3f2fffe (BF: cmdline: Restore\n    # handling of short options, 2018-07-23).\n    #\n    # Modified from the solution posted at\n    # https://bugs.python.org/issue14910#msg204678\n    def _get_option_tuples(self, option_string):\n        chars = self.prefix_chars\n        if option_string[0] in chars and option_string[1] in chars:\n            # option_string is a long flag. Disable abbreviation.\n            return []\n        return super(ArgumentParserDisableAbbrev, self)._get_option_tuples(\n            option_string)\n\n\ndef parser_add_version_opt(parser, mod_name, include_name=False, delay=False):\n    \"\"\"Setup --version option\n\n    Parameters\n    ----------\n    parser:\n    mod_name: str, optional\n    include_name: bool, optional\n    delay: bool, optional\n      If set to True, no action is taken immediately, and rather\n      we assign the function which would print the version. Necessary for\n      early pre-parsing of the cmdline\n    \"\"\"\n\n    def print_version():\n        mod = sys.modules.get(mod_name, None)\n        version = getattr(mod, '__version__', None)\n        if version is None:\n            # Let's use the standard Python mechanism if underlying module\n            # did not provide __version__\n            try:\n                import pkg_resources\n                version = pkg_resources.get_distribution(mod_name).version\n            except:\n                version = \"unknown\"\n        if include_name:\n            print(\"%s %s\" % (mod_name, version))\n        else:\n            print(version)\n        sys.exit(0)\n\n    class versionAction(argparse.Action):\n        def __call__(self, parser, args, values, option_string=None):\n            if delay:\n                setattr(args, self.dest, print_version)\n            else:\n                print_version()\n\n    parser.add_argument(\n        \"--version\",\n        nargs=0,\n        action=versionAction,\n        help=(\n            \"show the program's version\"\n            if not mod_name\n            else \"show the module and its version which provides the command\")\n    )\n\n\ndef parser_add_common_opt(parser, opt, names=None, **kwargs):\n    from . import common_args\n    opt_tmpl = getattr(common_args, opt)\n    opt_kwargs = opt_tmpl[2].copy()\n    opt_kwargs.update(kwargs)\n    if names is None:\n        parser.add_argument(*opt_tmpl[1], **opt_kwargs)\n    else:\n        parser.add_argument(*names, **opt_kwargs)\n\n\ndef parser_add_common_options(parser, version=None):\n    parser_add_common_opt(parser, 'log_level')\n    parser_add_common_opt(parser, 'change_path')\n    if version is not None:\n        warnings.warn(\"Passing 'version' to parser_add_common_options \"\n                      \"no longer has an effect \"\n                      \"and will be removed in a future release.\",\n                      DeprecationWarning)\n    parser_add_version_opt(parser, 'datalad', include_name=True, delay=True)\n    if __debug__:\n        parser.add_argument(\n            '--dbg', action='store_true', dest='common_debug',\n            help=\"enter Python debugger when uncaught exception happens\")\n        parser.add_argument(\n            '--idbg', action='store_true', dest='common_idebug',\n            help=\"enter IPython debugger when uncaught exception happens\")\n    parser.add_argument(\n        '-c', action='append', dest='cfg_overrides', metavar='KEY=VALUE',\n        help=\"\"\"configuration variable setting. Overrides any configuration\n        read from a file, but is potentially overridden itself by configuration\n        variables in the process environment.\"\"\")\n    # CLI analog of eval_params.result_renderer but with `<template>` handling\n    # and a different default: in Python API we have None as default and do not render\n    # the results but return them.  In CLI we default to \"default\" renderer\n    parser.add_argument(\n        # this should really have --result-renderer for homogeneity with the\n        # Python API, but adding it in addition makes the help output\n        # monsterous\n        '-f', '--output-format', # '--result-renderer',\n        dest='common_result_renderer',\n        default='tailored',\n        type=ensure_unicode,\n        metavar=\"{generic,json,json_pp,tailored,disabled,'<template>'}\",\n        help=eval_params['result_renderer']._doc + \" [Default: '%(default)s']\")\n    parser.add_argument(\n        '--report-status', dest='common_report_status',\n        choices=['success', 'failure', 'ok', 'notneeded', 'impossible', 'error'],\n        help=\"\"\"constrain command result report to records matching the given\n        status. 'success' is a synonym for 'ok' OR 'notneeded', 'failure' stands\n        for 'impossible' OR 'error'.\"\"\")\n    parser.add_argument(\n        '--report-type', dest='common_report_type',\n        choices=['dataset', 'file'],\n        action='append',\n        help=\"\"\"constrain command result report to records matching the given\n        type. Can be given more than once to match multiple types.\"\"\")\n    # CLI analog of eval_params.on_failure. TODO: dedup\n    parser.add_argument(\n        '--on-failure', dest='common_on_failure',\n        default=eval_defaults['on_failure'],\n        choices=['ignore', 'continue', 'stop'],\n        help=\"\"\"when an operation fails: 'ignore' and continue with remaining\n        operations, the error is logged but does not lead to a non-zero exit code\n        of the command; 'continue' works like 'ignore', but an error causes a\n        non-zero exit code; 'stop' halts on first failure and yields non-zero exit\n        code. A failure is any result with status 'impossible' or 'error'.\n        [Default: '%(default)s']\"\"\")\n    parser.add_argument(\n        '--cmd', dest='_', action='store_true',\n        help=\"\"\"syntactical helper that can be used to end the list of global\n        command line options before the subcommand label. Options taking\n        an arbitrary number of arguments may require to be followed by a single\n        --cmd in order to enable identification of the subcommand.\"\"\")\n\n\ndef strip_arg_from_argv(args, value, opt_names):\n    \"\"\"Strip an originally listed option (with its value) from the list cmdline args\n    \"\"\"\n    # Yarik doesn't know better\n    if args is None:\n        args = sys.argv\n    # remove present opt_names\n    args_clean = []\n    skip = 0\n    for i, arg in enumerate(args):\n        if skip:\n            # we skip only one as instructed\n            skip -= 1\n            continue\n        if not (arg in opt_names and i < len(args) - 1 and args[i + 1] == value):\n            args_clean.append(arg)\n        else:\n            # we need to skip this one and next one\n            skip = 1\n    return args_clean\n\n\ndef get_repo_instance(path=os.curdir, class_=None):\n    \"\"\"Returns an instance of appropriate datalad repository for path.\n    Check whether a certain path is inside a known type of repository and\n    returns an instance representing it. May also check for a certain type\n    instead of detecting the type of repository.\n\n    Parameters\n    ----------\n    path: str\n      path to check; default: current working directory\n    class_: class\n      if given, check whether path is inside a repository, that can be\n      represented as an instance of the passed class.\n\n    Raises\n    ------\n    RuntimeError, in case cwd is not inside a known repository.\n    \"\"\"\n\n    from os.path import ismount, exists, normpath, isabs\n    from datalad.support.exceptions import InvalidGitRepositoryError\n    from ..utils import expandpath\n    from ..support.gitrepo import GitRepo\n    from ..support.annexrepo import AnnexRepo\n\n    dir_ = expandpath(path)\n    abspath_ = path if isabs(path) else dir_\n    if class_ is not None:\n        if class_ == AnnexRepo:\n            type_ = \"annex\"\n        elif class_ == GitRepo:\n            type_ = \"git\"\n        else:\n            raise RuntimeError(\"Unknown class %s.\" % str(class_))\n\n    while not ismount(dir_):  # TODO: always correct termination?\n        if exists(opj(dir_, '.git')):\n            # found git dir\n            if class_ is None:\n                # detect repo type:\n                try:\n                    return AnnexRepo(dir_, create=False)\n                except RuntimeError as e:\n                    pass\n                try:\n                    return GitRepo(dir_, create=False)\n                except InvalidGitRepositoryError as e:\n                    raise RuntimeError(\"No datalad repository found in %s\" %\n                                       abspath_)\n            else:\n                try:\n                    return class_(dir_, create=False)\n                except (RuntimeError, InvalidGitRepositoryError) as e:\n                    raise RuntimeError(\"No %s repository found in %s.\" %\n                                       (type_, abspath_))\n        else:\n            dir_ = normpath(opj(dir_, \"..\"))\n\n    if class_ is not None:\n        raise RuntimeError(\"No %s repository found in %s\" % (type_, abspath_))\n    else:\n        raise RuntimeError(\"No datalad repository found in %s\" % abspath_)\n\n#\n# Some logic modules extracted from main.py to de-spagetify\n#\n\ndef _maybe_get_single_subparser(cmdlineargs, parser, interface_groups,\n                                return_subparsers, completing, help_ignore_extensions):\n    \"\"\"Performs early analysis of the cmdline\n\n    Looks at the first unparsed argument and if a known command, would return_subparsers\n    is False, would return only that one.\n\n    For the analysis to be complete etc, would also load commands from entrypoints etc\n\n    Returns\n    -------\n    None or str\n    \"\"\"\n    # Before doing anything additional and possibly expensive see may be that\n    # we have got the command already\n    need_single_subparser = False if return_subparsers else None\n    fail_handler = (lambda *a, **kw: True) \\\n        if return_subparsers else fail_with_short_help\n    try:\n        parsed_args, unparsed_args = parser._parse_known_args(\n            cmdlineargs[1:], argparse.Namespace())\n        # before anything handle possible datalad --version\n        if not unparsed_args and getattr(parsed_args, 'version', None):\n            parsed_args.version()  # will exit with 0\n        if not (completing or unparsed_args):\n            fail_handler(\n                parser,\n                msg=\"too few arguments, run with --help or visit https://handbook.datalad.org\",\n                exit_code=2)\n        lgr.debug(\"Command line args 1st pass for DataLad %s. Parsed: %s Unparsed: %s\",\n                  __version__, parsed_args, unparsed_args)\n    except Exception as exc:\n        ce = CapturedException(exc)\n        lgr.debug(\"Early parsing failed with %s\", ce)\n        need_single_subparser = False\n        unparsed_args = cmdlineargs[1:]  # referenced before assignment otherwise\n    # First unparsed could be either unknown option to top level \"datalad\"\n    # or a command. Among unknown could be --help/--help-np which would\n    # need to be dealt with\n    unparsed_arg = unparsed_args[0] if unparsed_args else None\n    if need_single_subparser is not None \\\n            or unparsed_arg in ('--help', '--help-np', '-h'):\n        need_single_subparser = False\n        if not help_ignore_extensions:\n            add_entrypoints_to_interface_groups(interface_groups)\n    elif not completing and unparsed_arg.startswith('-'):  # unknown option\n        fail_with_short_help(parser,\n                             msg=\"unrecognized argument %s\" % unparsed_arg,\n                             exit_code=2)\n        # if we could get a list of options known to parser,\n        # we could suggest them\n        # known=get_all_options(parser), provided=unparsed_arg)\n    else:  # the command to handle\n        known_commands = get_commands_from_groups(interface_groups)\n        if unparsed_arg not in known_commands:\n            # need to load all the extensions and try again\n            add_entrypoints_to_interface_groups(interface_groups)\n            known_commands = get_commands_from_groups(interface_groups)\n\n        if unparsed_arg not in known_commands:\n            # check if might be coming from known extensions\n            from ..interface import (\n                _known_extension_commands,\n                _deprecated_commands,\n            )\n            extension_commands = {\n                c: e\n                for e, commands in _known_extension_commands.items()\n                for c in commands\n            }\n            hint = None\n            if unparsed_arg in extension_commands:\n                hint = \"Command %s is provided by (not installed) extension %s.\" \\\n                       % (unparsed_arg, extension_commands[unparsed_arg])\n            elif unparsed_arg in _deprecated_commands:\n                hint_cmd = _deprecated_commands[unparsed_arg]\n                hint = \"Command %r was deprecated\" % unparsed_arg\n                hint += (\" in favor of %r command.\" % hint_cmd) if hint_cmd else '.'\n            if not completing:\n                fail_with_short_help(\n                    parser,\n                    hint=hint,\n                    provided=unparsed_arg,\n                    known=list(known_commands.keys()) + list(extension_commands.keys())\n                )\n        if need_single_subparser is None:\n            need_single_subparser = unparsed_arg\n    return need_single_subparser\n\n\ndef _maybe_get_interface_subparser(_intfspec, subparsers, cmd_name, formatter_class, group_name,\n                                   grp_short_descriptions):\n    \"\"\"Given an interface spec, add a subparser to subparsers under cmd_name\n\n    That subparser is also gets added to the grp_short_descriptions\n    \"\"\"\n    from ..interface.base import (\n        alter_interface_docs_for_cmdline,\n        get_cmd_doc,\n        get_cmd_ex,\n        load_interface,\n    )\n    _intf = load_interface(_intfspec)\n    if _intf is None:  # failed to load\n        return\n    # deal with optional parser args\n    if hasattr(_intf, 'parser_args'):\n        parser_args = _intf.parser_args\n    else:\n        parser_args = dict(formatter_class=formatter_class)\n        # use class description, if no explicit description is available\n        intf_doc = get_cmd_doc(_intf)\n        parser_args['description'] = alter_interface_docs_for_cmdline(\n            intf_doc)\n        if hasattr(_intf, '_examples_'):\n            intf_ex = alter_interface_docs_for_cmdline(get_cmd_ex(_intf))\n            parser_args['description'] += intf_ex\n    subparser = subparsers.add_parser(cmd_name, add_help=False, **parser_args)\n    # our own custom help for all commands\n    parser_add_common_opt(subparser, 'help')\n    # let module configure the parser\n    _intf.setup_parser(subparser)\n    # and we would add custom handler for --version\n    parser_add_version_opt(subparser, _intf.__module__.split('.', 1)[0], include_name=True)\n    # logger for command\n    # configure 'run' function for this command\n    plumbing_args = dict(\n        func=_intf.call_from_parser,\n        logger=logging.getLogger(_intf.__module__),\n        subparser=subparser)\n    if hasattr(_intf, 'result_renderer_cmdline'):\n        plumbing_args['result_renderer'] = _intf.result_renderer_cmdline\n    subparser.set_defaults(**plumbing_args)\n    # store short description for later\n    sdescr = getattr(_intf, 'short_description',\n                     parser_args['description'].split('\\n')[0])\n    grp_short_descriptions[group_name].append((cmd_name, sdescr))\n    return subparser\n\n\ndef add_entrypoints_to_interface_groups(interface_groups):\n    lgr.debug(\"Loading entrypoints\")\n    from pkg_resources import iter_entry_points  # delay expensive import\n    for ep in iter_entry_points('datalad.extensions'):\n        lgr.debug(\n            'Loading entrypoint %s from datalad.extensions for docs building',\n            ep.name)\n        try:\n            spec = ep.load()\n            if len(spec) < 2 or not spec[1]:\n                lgr.debug(\n                    'Extension does not provide a command suite: %s',\n                    ep.name)\n                continue\n            interface_groups.append((ep.name, spec[0], spec[1]))\n            lgr.debug('Loaded entrypoint %s', ep.name)\n        except Exception as e:\n            ce = CapturedException(ce)\n            lgr.warning('Failed to load entrypoint %s: %s', ep.name, ce)\n            continue\n\n\ndef get_commands_from_groups(groups):\n    \"\"\"Get a dictionary of command: interface_spec\"\"\"\n    from ..interface.base import get_cmdline_command_name\n    return {\n        get_cmdline_command_name(_intfspec): _intfspec\n        for _, _, _interfaces in groups\n        for _intfspec in _interfaces\n    }\n\n\ndef fail_with_short_help(parser=None,\n                         msg=None,\n                         known=None, provided=None,\n                         hint=None,\n                         exit_code=1,\n                         what=\"command\",\n                         out=None):\n    \"\"\"Generic helper to fail\n    with short help possibly hinting on what was intended if `known`\n    were provided\n    \"\"\"\n    out = out or sys.stderr\n    if msg:\n        out.write(\"error: %s\\n\" % msg)\n    if not known:\n        if parser:\n            parser_add_common_opt(parser, 'help')\n            # just to appear in print_usage also consistent with --help output\n            parser.add_argument(\"command [command-opts]\")\n            parser.print_usage(file=out)\n    else:\n        out.write(\n            \"datalad: Unknown %s %r.  See 'datalad --help'.\\n\\n\"\n            % (what, provided,))\n        if provided not in known:\n            out.write(get_suggestions_msg(provided, known))\n    if hint:\n        out.write(\"Hint: %s\\n\" % hint)\n    raise SystemExit(exit_code)\n\n\ndef _fix_datalad_ri(s):\n    \"\"\"Fixup argument if it was a DataLadRI and had leading / removed\n\n    See gh-2643\n    \"\"\"\n    if s.startswith('//') and (len(s) == 2 or (len(s) > 2 and s[2] != '/')):\n        lgr.info(\n            \"Changing %s back to /%s as it was probably changed by MINGW/MSYS, \"\n            \"see http://www.mingw.org/wiki/Posix_path_conversion\", s, s)\n        return \"/\" + s\n    return s\n\n\ndef get_description_with_cmd_summary(grp_short_descriptions, interface_groups,\n                                     parser_description):\n    from ..interface.base import dedent_docstring\n    from ..interface.base import get_cmd_summaries\n    lgr.debug(\"Generating detailed description for the parser\")\n\n    console_width = get_console_width()\n    cmd_summary = get_cmd_summaries(grp_short_descriptions, interface_groups,\n                                    width=console_width)\n    # we need one last formal section to not have the trailed be\n    # confused with the last command group\n    cmd_summary.append('\\n*General information*\\n')\n    detailed_description = '%s\\n%s\\n\\n%s' \\\n                           % (parser_description,\n                              '\\n'.join(cmd_summary),\n                              textwrap.fill(dedent_docstring(\"\"\"\\\n    Detailed usage information for individual commands is\n    available via command-specific --help, i.e.:\n    datalad <command> --help\"\"\"),\n                                            console_width - 5,\n                                            initial_indent='',\n                                            subsequent_indent=''))\n    return detailed_description\n\n\ndef _parse_overrides_from_cmdline(cmdlineargs):\n    \"\"\"parse config overrides provided in command line\n\n    Might exit(3) the entire process if value is not assigned\"\"\"\n    # this expression is deliberately loose as gitconfig offers\n    # quite some flexibility -- this is just meant to catch stupid\n    # errors: we need a section, a variable, and a value at minimum\n    # otherwise we break our own config parsing helpers\n    # https://github.com/datalad/datalad/issues/3451\n    noassign_expr = re.compile(r'[^\\s]+\\.[^\\s]+=[\\S]+')\n    noassign = [\n        o\n        for o in cmdlineargs.cfg_overrides\n        if not noassign_expr.match(o)\n    ]\n    if noassign:\n        lgr.error(\n            \"Configuration override without section/variable \"\n            \"or value assignment (must be 'section.variable=value'): %s\",\n            noassign)\n        sys.exit(3)\n    overrides = dict(o.split('=', 1) for o in cmdlineargs.cfg_overrides)\n    return overrides\n",
            "header": {
                "file": {
                    "path": "datalad/cmdline/helpers.py"
                },
                "misc_line": null
            },
            "index_line": {
                "old_commit_hash": "18e10a7fd",
                "new_commit_hash": "4b83d0f61",
                "mode": "100644"
            },
            "is_binary_file": false,
            "binary_line": null,
            "minus_file": {
                "path": "a/datalad/cmdline/helpers.py"
            },
            "plus_file": {
                "path": "b/datalad/cmdline/helpers.py"
            },
            "hunks": [
                {
                    "descriptor": {
                        "old_range": {
                            "start": 275,
                            "length": 12
                        },
                        "new_range": {
                            "start": 275,
                            "length": 6
                        },
                        "section": "def parser_add_common_options(parser, version=None):"
                    },
                    "line_group": {
                        "all_lines": [
                            {
                                "content": "        non-zero exit code; 'stop' halts on first failure and yields non-zero exit",
                                "type": "context"
                            },
                            {
                                "content": "        code. A failure is any result with status 'impossible' or 'error'.",
                                "type": "context"
                            },
                            {
                                "content": "        [Default: '%(default)s']\"\"\")",
                                "type": "context"
                            },
                            {
                                "content": "    parser.add_argument(",
                                "type": "deleted"
                            },
                            {
                                "content": "        '--library-mode', dest='common_library_mode', action='store_true',",
                                "type": "deleted"
                            },
                            {
                                "content": "        help=\"\"\"Enable operation as a utility library (rather than a front-end",
                                "type": "deleted"
                            },
                            {
                                "content": "        application). This may improve performance, because convenience",
                                "type": "deleted"
                            },
                            {
                                "content": "        functionality will be disabled, and will alter the error reporting to",
                                "type": "deleted"
                            },
                            {
                                "content": "        be better suited for internal use.\"\"\")",
                                "type": "deleted"
                            },
                            {
                                "content": "    parser.add_argument(",
                                "type": "context"
                            },
                            {
                                "content": "        '--cmd', dest='_', action='store_true',",
                                "type": "context"
                            },
                            {
                                "content": "        help=\"\"\"syntactical helper that can be used to end the list of global",
                                "type": "context"
                            }
                        ]
                    },
                    "modified_entities": [
                        {
                            "file_name": "datalad/cmdline/helpers.py",
                            "type": "function",
                            "name": "parser_add_common_options",
                            "content": "def parser_add_common_options(parser, version=None):\n    parser_add_common_opt(parser, 'log_level')\n    parser_add_common_opt(parser, 'change_path')\n    if version is not None:\n        warnings.warn(\"Passing 'version' to parser_add_common_options \"\n                      \"no longer has an effect \"\n                      \"and will be removed in a future release.\",\n                      DeprecationWarning)\n    parser_add_version_opt(parser, 'datalad', include_name=True, delay=True)\n    if __debug__:\n        parser.add_argument(\n            '--dbg', action='store_true', dest='common_debug',\n            help=\"enter Python debugger when uncaught exception happens\")\n        parser.add_argument(\n            '--idbg', action='store_true', dest='common_idebug',\n            help=\"enter IPython debugger when uncaught exception happens\")\n    parser.add_argument(\n        '-c', action='append', dest='cfg_overrides', metavar='KEY=VALUE',\n        help=\"\"\"configuration variable setting. Overrides any configuration\n        read from a file, but is potentially overridden itself by configuration\n        variables in the process environment.\"\"\")\n    # CLI analog of eval_params.result_renderer but with `<template>` handling\n    # and a different default: in Python API we have None as default and do not render\n    # the results but return them.  In CLI we default to \"default\" renderer\n    parser.add_argument(\n        # this should really have --result-renderer for homogeneity with the\n        # Python API, but adding it in addition makes the help output\n        # monsterous\n        '-f', '--output-format', # '--result-renderer',\n        dest='common_result_renderer',\n        default='tailored',\n        type=ensure_unicode,\n        metavar=\"{generic,json,json_pp,tailored,disabled,'<template>'}\",\n        help=eval_params['result_renderer']._doc + \" [Default: '%(default)s']\")\n    parser.add_argument(\n        '--report-status', dest='common_report_status',\n        choices=['success', 'failure', 'ok', 'notneeded', 'impossible', 'error'],\n        help=\"\"\"constrain command result report to records matching the given\n        status. 'success' is a synonym for 'ok' OR 'notneeded', 'failure' stands\n        for 'impossible' OR 'error'.\"\"\")\n    parser.add_argument(\n        '--report-type', dest='common_report_type',\n        choices=['dataset', 'file'],\n        action='append',\n        help=\"\"\"constrain command result report to records matching the given\n        type. Can be given more than once to match multiple types.\"\"\")\n    # CLI analog of eval_params.on_failure. TODO: dedup\n    parser.add_argument(\n        '--on-failure', dest='common_on_failure',\n        default=eval_defaults['on_failure'],\n        choices=['ignore', 'continue', 'stop'],\n        help=\"\"\"when an operation fails: 'ignore' and continue with remaining\n        operations, the error is logged but does not lead to a non-zero exit code\n        of the command; 'continue' works like 'ignore', but an error causes a\n        non-zero exit code; 'stop' halts on first failure and yields non-zero exit\n        code. A failure is any result with status 'impossible' or 'error'.\n        [Default: '%(default)s']\"\"\")\n    parser.add_argument(\n        '--library-mode', dest='common_library_mode', action='store_true',\n        help=\"\"\"Enable operation as a utility library (rather than a front-end\n        application). This may improve performance, because convenience\n        functionality will be disabled, and will alter the error reporting to\n        be better suited for internal use.\"\"\")\n    parser.add_argument(\n        '--cmd', dest='_', action='store_true',\n        help=\"\"\"syntactical helper that can be used to end the list of global\n        command line options before the subcommand label. Options taking\n        an arbitrary number of arguments may require to be followed by a single\n        --cmd in order to enable identification of the subcommand.\"\"\")",
                            "ast_type_str": "FunctionDef",
                            "start_lineno": 221,
                            "end_lineno": 289,
                            "parent": null
                        }
                    ],
                    "added_entities": [],
                    "deleted_entities": []
                }
            ]
        },
        {
            "old_file_content": "# emacs: -*- mode: python; py-indent-offset: 4; tab-width: 4; indent-tabs-mode: nil -*-\n# ex: set sts=4 ts=4 sw=4 et:\n# ## ### ### ### ### ### ### ### ### ### ### ### ### ### ### ### ### ### ### ##\n#\n#   See COPYING file distributed along with the datalad package for the\n#   copyright and license terms.\n#\n# ## ### ### ### ### ### ### ### ### ### ### ### ### ### ### ### ### ### ### ##\n\"\"\"\"\"\"\n\n__docformat__ = 'restructuredtext'\n\nimport logging\nlgr = logging.getLogger('datalad.cmdline')\n\nlgr.log(5, \"Importing cmdline.main\")\n\nimport argparse\nfrom collections import defaultdict\nimport sys\nimport os\n\nimport datalad\n\nfrom ..support.exceptions import (\n    CapturedException,\n    InsufficientArgumentsError,\n    IncompleteResultsError,\n    CommandError,\n)\nfrom ..utils import (\n    chpwd,\n    on_msys_tainted_paths,\n    setup_exceptionhook,\n)\nfrom .helpers import (\n    ArgumentParserDisableAbbrev,\n    _fix_datalad_ri,\n    _maybe_get_interface_subparser,\n    _maybe_get_single_subparser,\n    _parse_overrides_from_cmdline,\n    get_description_with_cmd_summary,\n    parser_add_common_options,\n    parser_add_common_opt,\n    strip_arg_from_argv,\n)\n\n\n# TODO:  OPT look into making setup_parser smarter to become faster\n# Now it seems to take up to 200ms to do all the parser setup\n# even though it might not be necessary to know about all the commands etc.\n# I wondered if it could somehow decide on what commands to worry about etc\n# by going through sys.args first\ndef setup_parser(\n        cmdlineargs,\n        formatter_class=argparse.RawDescriptionHelpFormatter,\n        return_subparsers=False,\n        # Was this triggered by argparse?\n        completing=False,\n        # prevent loading of extension entrypoints when --help is requested\n        # this is enabled when building docs to avoid pollution of generated\n        # manpages with extensions commands (that should appear in their own\n        # docs, but not in the core datalad package docs)\n        help_ignore_extensions=False):\n    \"\"\"\n    The holy grail of establishing CLI for DataLad's Interfaces\n\n    Parameters\n    ----------\n    cmdlineargs:\n    formatter_class:\n    return_subparsers: bool, optional\n      is used ATM only by BuildManPage in _datalad_build_support\n    completing:\n    help_ignore_extensions:\n    \"\"\"\n    lgr.log(5, \"Starting to setup_parser\")\n    # delay since it can be a heavy import\n    from ..interface.base import (\n        dedent_docstring,\n        get_cmdline_command_name,\n        get_interface_groups,\n    )\n\n    # setup cmdline args parser\n\n    # \"main\" parser is under \"datalad\" name\n    all_parsers = {}  # name: (sub)parser\n\n    # main parser\n    parser = ArgumentParserDisableAbbrev(\n        fromfile_prefix_chars=None,\n        # usage=\"%(prog)s ...\",\n        description=dedent_docstring(\"\"\"\\\n            Comprehensive data management solution\n\n            DataLad provides a unified data distribution system built on the Git\n            and Git-annex. DataLad command line tools allow to manipulate (obtain,\n            create, update, publish, etc.) datasets and provide a comprehensive\n            toolbox for joint management of data and code. Compared to Git/annex\n            it primarily extends their functionality to transparently and\n            simultaneously work with multiple inter-related repositories.\"\"\"),\n        epilog='\"Be happy!\"',\n        formatter_class=formatter_class,\n        add_help=False)\n\n    # common options\n    parser_add_common_options(parser)\n\n    interface_groups = get_interface_groups()\n\n    single_subparser = _maybe_get_single_subparser(\n        cmdlineargs, parser, interface_groups,\n        return_subparsers, completing, help_ignore_extensions\n    )\n\n    # --help specification was delayed since it causes immediate printout of\n    # --help output before we setup --help for each command\n    parser_add_common_opt(parser, 'help')\n\n    grp_short_descriptions = defaultdict(list)\n    # create subparser, use module suffix as cmd name\n    subparsers = parser.add_subparsers()\n    for group_name, _, _interfaces \\\n            in sorted(interface_groups, key=lambda x: x[1]):\n        for _intfspec in _interfaces:\n            cmd_name = get_cmdline_command_name(_intfspec)\n            if single_subparser and cmd_name != single_subparser:\n                continue\n            subparser = _maybe_get_interface_subparser(\n                _intfspec, subparsers, cmd_name, formatter_class, group_name,\n                grp_short_descriptions\n            )\n            if subparser:  # interface might have failed to \"load\"\n                all_parsers[cmd_name] = subparser\n\n    # create command summary\n    if '--help' in cmdlineargs or '--help-np' in cmdlineargs:\n        parser.description = get_description_with_cmd_summary(\n            grp_short_descriptions,\n            interface_groups,\n            parser.description)\n\n    all_parsers['datalad'] = parser\n    lgr.log(5, \"Finished setup_parser\")\n    if return_subparsers:\n        return all_parsers\n    else:\n        return parser\n\n\ndef main(args=None):\n    lgr.log(5, \"Starting main(%r)\", args)\n    # record that we came in via the cmdline\n    datalad.__api = 'cmdline'\n    args = args or sys.argv\n    if on_msys_tainted_paths:\n        # Possibly present DataLadRIs were stripped of a leading /\n        args = [_fix_datalad_ri(s) for s in args]\n    # PYTHON_ARGCOMPLETE_OK\n    parser = setup_parser(args, completing=\"_ARGCOMPLETE\" in os.environ)\n    try:\n        import argcomplete\n        argcomplete.autocomplete(parser)\n    except ImportError:\n        pass\n\n    # parse cmd args\n    lgr.debug(\"Parsing known args among %s\", repr(args))\n    cmdlineargs, unparsed_args = parser.parse_known_args(args[1:])\n    if cmdlineargs.common_library_mode:\n        datalad.enable_librarymode()\n    has_func = hasattr(cmdlineargs, 'func') and cmdlineargs.func is not None\n    if unparsed_args:\n        if has_func and cmdlineargs.func.__self__.__name__ != 'Export':\n            lgr.error('unknown argument{}: {}'.format(\n                's' if len(unparsed_args) > 1 else '',\n                unparsed_args if len(unparsed_args) > 1 else unparsed_args[0]))\n            cmdlineargs.subparser.print_usage()\n            sys.exit(1)\n        else:\n            # store all unparsed arguments\n            cmdlineargs.datalad_unparsed_args = unparsed_args\n\n    # to possibly be passed into PBS scheduled call\n    args_ = args or sys.argv\n\n    if cmdlineargs.cfg_overrides is not None:\n        datalad.cfg.overrides.update(\n            _parse_overrides_from_cmdline(cmdlineargs)\n        )\n\n    # enable overrides\n    datalad.cfg.reload(force=True)\n\n    if cmdlineargs.change_path is not None:\n        from .common_args import change_path as change_path_opt\n        for path in cmdlineargs.change_path:\n            chpwd(path)\n            args_ = strip_arg_from_argv(args_, path, change_path_opt[1])\n\n    ret = None\n    if has_func:\n        if cmdlineargs.common_debug or cmdlineargs.common_idebug:\n            # so we could see/stop clearly at the point of failure\n            setup_exceptionhook(ipython=cmdlineargs.common_idebug)\n            from datalad.interface.base import Interface\n            Interface._interrupted_exit_code = None\n            ret = cmdlineargs.func(cmdlineargs)\n        else:\n            # otherwise - guard and only log the summary. Postmortem is not\n            # as convenient if being caught in this ultimate except\n            try:\n                ret = cmdlineargs.func(cmdlineargs)\n            except InsufficientArgumentsError as exc:\n                ce = CapturedException(exc)\n                # if the func reports inappropriate usage, give help output\n                lgr.error('%s (%s)', ce, exc.__class__.__name__)\n                cmdlineargs.subparser.print_usage(sys.stderr)\n                sys.exit(2)\n            except IncompleteResultsError as exc:\n                ce = CapturedException(exc)\n                # rendering for almost all commands now happens 'online'\n                # hence we are no longer attempting to render the actual\n                # results in an IncompleteResultsError, ubt rather trust that\n                # this happened before\n\n                # in general we do not want to see the error again, but\n                # present in debug output\n                lgr.debug('could not perform all requested actions: %s', ce)\n                sys.exit(1)\n            except CommandError as exc:\n                ce = CapturedException(exc)\n                # behave as if the command ran directly, importantly pass\n                # exit code as is\n                # to not duplicate any captured output in the exception\n                # rendering, will come next\n                exc_msg = exc.to_str(include_output=False)\n                if exc_msg:\n                    msg = exc_msg.encode() if isinstance(exc_msg, str) else exc_msg\n                    os.write(2, msg + b\"\\n\")\n                if exc.stdout:\n                    os.write(1, exc.stdout.encode() if isinstance(exc.stdout, str) else exc.stdout)\n                if exc.stderr:\n                    os.write(2, exc.stderr.encode() if isinstance(exc.stderr, str) else exc.stderr)\n                # We must not exit with 0 code if any exception got here but\n                # had no code defined\n                sys.exit(exc.code if exc.code is not None else 1)\n            except Exception as exc:\n                ce = CapturedException(exc)\n                lgr.error('%s (%s)', ce, exc.__class__.__name__)\n                sys.exit(1)\n    else:\n        # just let argparser spit out its error, since there is smth wrong\n        parser.parse_args(args)\n        # if that one didn't puke -- we should\n        parser.print_usage()\n        lgr.error(\"Please specify the command\")\n        sys.exit(2)\n\n    try:\n        if hasattr(cmdlineargs, 'result_renderer'):\n            cmdlineargs.result_renderer(ret, cmdlineargs)\n    except Exception as exc:\n        ce = CapturedException(exc)\n        lgr.error(\"Failed to render results due to %s\", ce)\n        sys.exit(1)\n    # all good, not strictly needed, but makes internal testing easier\n    sys.exit(0)\n\n\nlgr.log(5, \"Done importing cmdline.main\")\n",
            "new_file_content": "# emacs: -*- mode: python; py-indent-offset: 4; tab-width: 4; indent-tabs-mode: nil -*-\n# ex: set sts=4 ts=4 sw=4 et:\n# ## ### ### ### ### ### ### ### ### ### ### ### ### ### ### ### ### ### ### ##\n#\n#   See COPYING file distributed along with the datalad package for the\n#   copyright and license terms.\n#\n# ## ### ### ### ### ### ### ### ### ### ### ### ### ### ### ### ### ### ### ##\n\"\"\"\"\"\"\n\n__docformat__ = 'restructuredtext'\n\nimport logging\nlgr = logging.getLogger('datalad.cmdline')\n\nlgr.log(5, \"Importing cmdline.main\")\n\nimport argparse\nfrom collections import defaultdict\nimport sys\nimport os\n\nimport datalad\n\nfrom ..support.exceptions import (\n    CapturedException,\n    InsufficientArgumentsError,\n    IncompleteResultsError,\n    CommandError,\n)\nfrom ..utils import (\n    chpwd,\n    on_msys_tainted_paths,\n    setup_exceptionhook,\n)\nfrom .helpers import (\n    ArgumentParserDisableAbbrev,\n    _fix_datalad_ri,\n    _maybe_get_interface_subparser,\n    _maybe_get_single_subparser,\n    _parse_overrides_from_cmdline,\n    get_description_with_cmd_summary,\n    parser_add_common_options,\n    parser_add_common_opt,\n    strip_arg_from_argv,\n)\n\n\n# TODO:  OPT look into making setup_parser smarter to become faster\n# Now it seems to take up to 200ms to do all the parser setup\n# even though it might not be necessary to know about all the commands etc.\n# I wondered if it could somehow decide on what commands to worry about etc\n# by going through sys.args first\ndef setup_parser(\n        cmdlineargs,\n        formatter_class=argparse.RawDescriptionHelpFormatter,\n        return_subparsers=False,\n        # Was this triggered by argparse?\n        completing=False,\n        # prevent loading of extension entrypoints when --help is requested\n        # this is enabled when building docs to avoid pollution of generated\n        # manpages with extensions commands (that should appear in their own\n        # docs, but not in the core datalad package docs)\n        help_ignore_extensions=False):\n    \"\"\"\n    The holy grail of establishing CLI for DataLad's Interfaces\n\n    Parameters\n    ----------\n    cmdlineargs:\n    formatter_class:\n    return_subparsers: bool, optional\n      is used ATM only by BuildManPage in _datalad_build_support\n    completing:\n    help_ignore_extensions:\n    \"\"\"\n    lgr.log(5, \"Starting to setup_parser\")\n    # delay since it can be a heavy import\n    from ..interface.base import (\n        dedent_docstring,\n        get_cmdline_command_name,\n        get_interface_groups,\n    )\n\n    # setup cmdline args parser\n\n    # \"main\" parser is under \"datalad\" name\n    all_parsers = {}  # name: (sub)parser\n\n    # main parser\n    parser = ArgumentParserDisableAbbrev(\n        fromfile_prefix_chars=None,\n        # usage=\"%(prog)s ...\",\n        description=dedent_docstring(\"\"\"\\\n            Comprehensive data management solution\n\n            DataLad provides a unified data distribution system built on the Git\n            and Git-annex. DataLad command line tools allow to manipulate (obtain,\n            create, update, publish, etc.) datasets and provide a comprehensive\n            toolbox for joint management of data and code. Compared to Git/annex\n            it primarily extends their functionality to transparently and\n            simultaneously work with multiple inter-related repositories.\"\"\"),\n        epilog='\"Be happy!\"',\n        formatter_class=formatter_class,\n        add_help=False)\n\n    # common options\n    parser_add_common_options(parser)\n\n    interface_groups = get_interface_groups()\n\n    single_subparser = _maybe_get_single_subparser(\n        cmdlineargs, parser, interface_groups,\n        return_subparsers, completing, help_ignore_extensions\n    )\n\n    # --help specification was delayed since it causes immediate printout of\n    # --help output before we setup --help for each command\n    parser_add_common_opt(parser, 'help')\n\n    grp_short_descriptions = defaultdict(list)\n    # create subparser, use module suffix as cmd name\n    subparsers = parser.add_subparsers()\n    for group_name, _, _interfaces \\\n            in sorted(interface_groups, key=lambda x: x[1]):\n        for _intfspec in _interfaces:\n            cmd_name = get_cmdline_command_name(_intfspec)\n            if single_subparser and cmd_name != single_subparser:\n                continue\n            subparser = _maybe_get_interface_subparser(\n                _intfspec, subparsers, cmd_name, formatter_class, group_name,\n                grp_short_descriptions\n            )\n            if subparser:  # interface might have failed to \"load\"\n                all_parsers[cmd_name] = subparser\n\n    # create command summary\n    if '--help' in cmdlineargs or '--help-np' in cmdlineargs:\n        parser.description = get_description_with_cmd_summary(\n            grp_short_descriptions,\n            interface_groups,\n            parser.description)\n\n    all_parsers['datalad'] = parser\n    lgr.log(5, \"Finished setup_parser\")\n    if return_subparsers:\n        return all_parsers\n    else:\n        return parser\n\n\ndef main(args=None):\n    lgr.log(5, \"Starting main(%r)\", args)\n    # record that we came in via the cmdline\n    datalad.__api = 'cmdline'\n    args = args or sys.argv\n    if on_msys_tainted_paths:\n        # Possibly present DataLadRIs were stripped of a leading /\n        args = [_fix_datalad_ri(s) for s in args]\n    # PYTHON_ARGCOMPLETE_OK\n    parser = setup_parser(args, completing=\"_ARGCOMPLETE\" in os.environ)\n    try:\n        import argcomplete\n        argcomplete.autocomplete(parser)\n    except ImportError:\n        pass\n\n    # parse cmd args\n    lgr.debug(\"Parsing known args among %s\", repr(args))\n    cmdlineargs, unparsed_args = parser.parse_known_args(args[1:])\n    has_func = hasattr(cmdlineargs, 'func') and cmdlineargs.func is not None\n    if unparsed_args:\n        if has_func and cmdlineargs.func.__self__.__name__ != 'Export':\n            lgr.error('unknown argument{}: {}'.format(\n                's' if len(unparsed_args) > 1 else '',\n                unparsed_args if len(unparsed_args) > 1 else unparsed_args[0]))\n            cmdlineargs.subparser.print_usage()\n            sys.exit(1)\n        else:\n            # store all unparsed arguments\n            cmdlineargs.datalad_unparsed_args = unparsed_args\n\n    # to possibly be passed into PBS scheduled call\n    args_ = args or sys.argv\n\n    if cmdlineargs.cfg_overrides is not None:\n        datalad.cfg.overrides.update(\n            _parse_overrides_from_cmdline(cmdlineargs)\n        )\n\n    # enable overrides\n    datalad.cfg.reload(force=True)\n    if 'datalad.runtime.librarymode' in datalad.cfg:\n        datalad.enable_librarymode()\n\n    if cmdlineargs.change_path is not None:\n        from .common_args import change_path as change_path_opt\n        for path in cmdlineargs.change_path:\n            chpwd(path)\n            args_ = strip_arg_from_argv(args_, path, change_path_opt[1])\n\n    ret = None\n    if has_func:\n        if cmdlineargs.common_debug or cmdlineargs.common_idebug:\n            # so we could see/stop clearly at the point of failure\n            setup_exceptionhook(ipython=cmdlineargs.common_idebug)\n            from datalad.interface.base import Interface\n            Interface._interrupted_exit_code = None\n            ret = cmdlineargs.func(cmdlineargs)\n        else:\n            # otherwise - guard and only log the summary. Postmortem is not\n            # as convenient if being caught in this ultimate except\n            try:\n                ret = cmdlineargs.func(cmdlineargs)\n            except InsufficientArgumentsError as exc:\n                ce = CapturedException(exc)\n                # if the func reports inappropriate usage, give help output\n                lgr.error('%s (%s)', ce, exc.__class__.__name__)\n                cmdlineargs.subparser.print_usage(sys.stderr)\n                sys.exit(2)\n            except IncompleteResultsError as exc:\n                ce = CapturedException(exc)\n                # rendering for almost all commands now happens 'online'\n                # hence we are no longer attempting to render the actual\n                # results in an IncompleteResultsError, ubt rather trust that\n                # this happened before\n\n                # in general we do not want to see the error again, but\n                # present in debug output\n                lgr.debug('could not perform all requested actions: %s', ce)\n                sys.exit(1)\n            except CommandError as exc:\n                ce = CapturedException(exc)\n                # behave as if the command ran directly, importantly pass\n                # exit code as is\n                # to not duplicate any captured output in the exception\n                # rendering, will come next\n                exc_msg = exc.to_str(include_output=False)\n                if exc_msg:\n                    msg = exc_msg.encode() if isinstance(exc_msg, str) else exc_msg\n                    os.write(2, msg + b\"\\n\")\n                if exc.stdout:\n                    os.write(1, exc.stdout.encode() if isinstance(exc.stdout, str) else exc.stdout)\n                if exc.stderr:\n                    os.write(2, exc.stderr.encode() if isinstance(exc.stderr, str) else exc.stderr)\n                # We must not exit with 0 code if any exception got here but\n                # had no code defined\n                sys.exit(exc.code if exc.code is not None else 1)\n            except Exception as exc:\n                ce = CapturedException(exc)\n                lgr.error('%s (%s)', ce, exc.__class__.__name__)\n                sys.exit(1)\n    else:\n        # just let argparser spit out its error, since there is smth wrong\n        parser.parse_args(args)\n        # if that one didn't puke -- we should\n        parser.print_usage()\n        lgr.error(\"Please specify the command\")\n        sys.exit(2)\n\n    try:\n        if hasattr(cmdlineargs, 'result_renderer'):\n            cmdlineargs.result_renderer(ret, cmdlineargs)\n    except Exception as exc:\n        ce = CapturedException(exc)\n        lgr.error(\"Failed to render results due to %s\", ce)\n        sys.exit(1)\n    # all good, not strictly needed, but makes internal testing easier\n    sys.exit(0)\n\n\nlgr.log(5, \"Done importing cmdline.main\")\n",
            "header": {
                "file": {
                    "path": "datalad/cmdline/main.py"
                },
                "misc_line": null
            },
            "index_line": {
                "old_commit_hash": "d05c94686",
                "new_commit_hash": "4e5abcd06",
                "mode": "100644"
            },
            "is_binary_file": false,
            "binary_line": null,
            "minus_file": {
                "path": "a/datalad/cmdline/main.py"
            },
            "plus_file": {
                "path": "b/datalad/cmdline/main.py"
            },
            "hunks": [
                {
                    "descriptor": {
                        "old_range": {
                            "start": 168,
                            "length": 8
                        },
                        "new_range": {
                            "start": 168,
                            "length": 6
                        },
                        "section": "def main(args=None):"
                    },
                    "line_group": {
                        "all_lines": [
                            {
                                "content": "    # parse cmd args",
                                "type": "context"
                            },
                            {
                                "content": "    lgr.debug(\"Parsing known args among %s\", repr(args))",
                                "type": "context"
                            },
                            {
                                "content": "    cmdlineargs, unparsed_args = parser.parse_known_args(args[1:])",
                                "type": "context"
                            },
                            {
                                "content": "    if cmdlineargs.common_library_mode:",
                                "type": "deleted"
                            },
                            {
                                "content": "        datalad.enable_librarymode()",
                                "type": "deleted"
                            },
                            {
                                "content": "    has_func = hasattr(cmdlineargs, 'func') and cmdlineargs.func is not None",
                                "type": "context"
                            },
                            {
                                "content": "    if unparsed_args:",
                                "type": "context"
                            },
                            {
                                "content": "        if has_func and cmdlineargs.func.__self__.__name__ != 'Export':",
                                "type": "context"
                            }
                        ]
                    },
                    "modified_entities": [
                        {
                            "file_name": "datalad/cmdline/main.py",
                            "type": "function",
                            "name": "main",
                            "content": "def main(args=None):\n    lgr.log(5, \"Starting main(%r)\", args)\n    # record that we came in via the cmdline\n    datalad.__api = 'cmdline'\n    args = args or sys.argv\n    if on_msys_tainted_paths:\n        # Possibly present DataLadRIs were stripped of a leading /\n        args = [_fix_datalad_ri(s) for s in args]\n    # PYTHON_ARGCOMPLETE_OK\n    parser = setup_parser(args, completing=\"_ARGCOMPLETE\" in os.environ)\n    try:\n        import argcomplete\n        argcomplete.autocomplete(parser)\n    except ImportError:\n        pass\n\n    # parse cmd args\n    lgr.debug(\"Parsing known args among %s\", repr(args))\n    cmdlineargs, unparsed_args = parser.parse_known_args(args[1:])\n    if cmdlineargs.common_library_mode:\n        datalad.enable_librarymode()\n    has_func = hasattr(cmdlineargs, 'func') and cmdlineargs.func is not None\n    if unparsed_args:\n        if has_func and cmdlineargs.func.__self__.__name__ != 'Export':\n            lgr.error('unknown argument{}: {}'.format(\n                's' if len(unparsed_args) > 1 else '',\n                unparsed_args if len(unparsed_args) > 1 else unparsed_args[0]))\n            cmdlineargs.subparser.print_usage()\n            sys.exit(1)\n        else:\n            # store all unparsed arguments\n            cmdlineargs.datalad_unparsed_args = unparsed_args\n\n    # to possibly be passed into PBS scheduled call\n    args_ = args or sys.argv\n\n    if cmdlineargs.cfg_overrides is not None:\n        datalad.cfg.overrides.update(\n            _parse_overrides_from_cmdline(cmdlineargs)\n        )\n\n    # enable overrides\n    datalad.cfg.reload(force=True)\n\n    if cmdlineargs.change_path is not None:\n        from .common_args import change_path as change_path_opt\n        for path in cmdlineargs.change_path:\n            chpwd(path)\n            args_ = strip_arg_from_argv(args_, path, change_path_opt[1])\n\n    ret = None\n    if has_func:\n        if cmdlineargs.common_debug or cmdlineargs.common_idebug:\n            # so we could see/stop clearly at the point of failure\n            setup_exceptionhook(ipython=cmdlineargs.common_idebug)\n            from datalad.interface.base import Interface\n            Interface._interrupted_exit_code = None\n            ret = cmdlineargs.func(cmdlineargs)\n        else:\n            # otherwise - guard and only log the summary. Postmortem is not\n            # as convenient if being caught in this ultimate except\n            try:\n                ret = cmdlineargs.func(cmdlineargs)\n            except InsufficientArgumentsError as exc:\n                ce = CapturedException(exc)\n                # if the func reports inappropriate usage, give help output\n                lgr.error('%s (%s)', ce, exc.__class__.__name__)\n                cmdlineargs.subparser.print_usage(sys.stderr)\n                sys.exit(2)\n            except IncompleteResultsError as exc:\n                ce = CapturedException(exc)\n                # rendering for almost all commands now happens 'online'\n                # hence we are no longer attempting to render the actual\n                # results in an IncompleteResultsError, ubt rather trust that\n                # this happened before\n\n                # in general we do not want to see the error again, but\n                # present in debug output\n                lgr.debug('could not perform all requested actions: %s', ce)\n                sys.exit(1)\n            except CommandError as exc:\n                ce = CapturedException(exc)\n                # behave as if the command ran directly, importantly pass\n                # exit code as is\n                # to not duplicate any captured output in the exception\n                # rendering, will come next\n                exc_msg = exc.to_str(include_output=False)\n                if exc_msg:\n                    msg = exc_msg.encode() if isinstance(exc_msg, str) else exc_msg\n                    os.write(2, msg + b\"\\n\")\n                if exc.stdout:\n                    os.write(1, exc.stdout.encode() if isinstance(exc.stdout, str) else exc.stdout)\n                if exc.stderr:\n                    os.write(2, exc.stderr.encode() if isinstance(exc.stderr, str) else exc.stderr)\n                # We must not exit with 0 code if any exception got here but\n                # had no code defined\n                sys.exit(exc.code if exc.code is not None else 1)\n            except Exception as exc:\n                ce = CapturedException(exc)\n                lgr.error('%s (%s)', ce, exc.__class__.__name__)\n                sys.exit(1)\n    else:\n        # just let argparser spit out its error, since there is smth wrong\n        parser.parse_args(args)\n        # if that one didn't puke -- we should\n        parser.print_usage()\n        lgr.error(\"Please specify the command\")\n        sys.exit(2)\n\n    try:\n        if hasattr(cmdlineargs, 'result_renderer'):\n            cmdlineargs.result_renderer(ret, cmdlineargs)\n    except Exception as exc:\n        ce = CapturedException(exc)\n        lgr.error(\"Failed to render results due to %s\", ce)\n        sys.exit(1)\n    # all good, not strictly needed, but makes internal testing easier\n    sys.exit(0)",
                            "ast_type_str": "FunctionDef",
                            "start_lineno": 152,
                            "end_lineno": 269,
                            "parent": null
                        }
                    ],
                    "added_entities": [],
                    "deleted_entities": []
                },
                {
                    "descriptor": {
                        "old_range": {
                            "start": 192,
                            "length": 6
                        },
                        "new_range": {
                            "start": 190,
                            "length": 8
                        },
                        "section": "def main(args=None):"
                    },
                    "line_group": {
                        "all_lines": [
                            {
                                "content": "",
                                "type": "context"
                            },
                            {
                                "content": "    # enable overrides",
                                "type": "context"
                            },
                            {
                                "content": "    datalad.cfg.reload(force=True)",
                                "type": "context"
                            },
                            {
                                "content": "    if 'datalad.runtime.librarymode' in datalad.cfg:",
                                "type": "added"
                            },
                            {
                                "content": "        datalad.enable_librarymode()",
                                "type": "added"
                            },
                            {
                                "content": "",
                                "type": "context"
                            },
                            {
                                "content": "    if cmdlineargs.change_path is not None:",
                                "type": "context"
                            },
                            {
                                "content": "        from .common_args import change_path as change_path_opt",
                                "type": "context"
                            }
                        ]
                    },
                    "modified_entities": [
                        {
                            "file_name": "datalad/cmdline/main.py",
                            "type": "function",
                            "name": "main",
                            "content": "def main(args=None):\n    lgr.log(5, \"Starting main(%r)\", args)\n    # record that we came in via the cmdline\n    datalad.__api = 'cmdline'\n    args = args or sys.argv\n    if on_msys_tainted_paths:\n        # Possibly present DataLadRIs were stripped of a leading /\n        args = [_fix_datalad_ri(s) for s in args]\n    # PYTHON_ARGCOMPLETE_OK\n    parser = setup_parser(args, completing=\"_ARGCOMPLETE\" in os.environ)\n    try:\n        import argcomplete\n        argcomplete.autocomplete(parser)\n    except ImportError:\n        pass\n\n    # parse cmd args\n    lgr.debug(\"Parsing known args among %s\", repr(args))\n    cmdlineargs, unparsed_args = parser.parse_known_args(args[1:])\n    has_func = hasattr(cmdlineargs, 'func') and cmdlineargs.func is not None\n    if unparsed_args:\n        if has_func and cmdlineargs.func.__self__.__name__ != 'Export':\n            lgr.error('unknown argument{}: {}'.format(\n                's' if len(unparsed_args) > 1 else '',\n                unparsed_args if len(unparsed_args) > 1 else unparsed_args[0]))\n            cmdlineargs.subparser.print_usage()\n            sys.exit(1)\n        else:\n            # store all unparsed arguments\n            cmdlineargs.datalad_unparsed_args = unparsed_args\n\n    # to possibly be passed into PBS scheduled call\n    args_ = args or sys.argv\n\n    if cmdlineargs.cfg_overrides is not None:\n        datalad.cfg.overrides.update(\n            _parse_overrides_from_cmdline(cmdlineargs)\n        )\n\n    # enable overrides\n    datalad.cfg.reload(force=True)\n    if 'datalad.runtime.librarymode' in datalad.cfg:\n        datalad.enable_librarymode()\n\n    if cmdlineargs.change_path is not None:\n        from .common_args import change_path as change_path_opt\n        for path in cmdlineargs.change_path:\n            chpwd(path)\n            args_ = strip_arg_from_argv(args_, path, change_path_opt[1])\n\n    ret = None\n    if has_func:\n        if cmdlineargs.common_debug or cmdlineargs.common_idebug:\n            # so we could see/stop clearly at the point of failure\n            setup_exceptionhook(ipython=cmdlineargs.common_idebug)\n            from datalad.interface.base import Interface\n            Interface._interrupted_exit_code = None\n            ret = cmdlineargs.func(cmdlineargs)\n        else:\n            # otherwise - guard and only log the summary. Postmortem is not\n            # as convenient if being caught in this ultimate except\n            try:\n                ret = cmdlineargs.func(cmdlineargs)\n            except InsufficientArgumentsError as exc:\n                ce = CapturedException(exc)\n                # if the func reports inappropriate usage, give help output\n                lgr.error('%s (%s)', ce, exc.__class__.__name__)\n                cmdlineargs.subparser.print_usage(sys.stderr)\n                sys.exit(2)\n            except IncompleteResultsError as exc:\n                ce = CapturedException(exc)\n                # rendering for almost all commands now happens 'online'\n                # hence we are no longer attempting to render the actual\n                # results in an IncompleteResultsError, ubt rather trust that\n                # this happened before\n\n                # in general we do not want to see the error again, but\n                # present in debug output\n                lgr.debug('could not perform all requested actions: %s', ce)\n                sys.exit(1)\n            except CommandError as exc:\n                ce = CapturedException(exc)\n                # behave as if the command ran directly, importantly pass\n                # exit code as is\n                # to not duplicate any captured output in the exception\n                # rendering, will come next\n                exc_msg = exc.to_str(include_output=False)\n                if exc_msg:\n                    msg = exc_msg.encode() if isinstance(exc_msg, str) else exc_msg\n                    os.write(2, msg + b\"\\n\")\n                if exc.stdout:\n                    os.write(1, exc.stdout.encode() if isinstance(exc.stdout, str) else exc.stdout)\n                if exc.stderr:\n                    os.write(2, exc.stderr.encode() if isinstance(exc.stderr, str) else exc.stderr)\n                # We must not exit with 0 code if any exception got here but\n                # had no code defined\n                sys.exit(exc.code if exc.code is not None else 1)\n            except Exception as exc:\n                ce = CapturedException(exc)\n                lgr.error('%s (%s)', ce, exc.__class__.__name__)\n                sys.exit(1)\n    else:\n        # just let argparser spit out its error, since there is smth wrong\n        parser.parse_args(args)\n        # if that one didn't puke -- we should\n        parser.print_usage()\n        lgr.error(\"Please specify the command\")\n        sys.exit(2)\n\n    try:\n        if hasattr(cmdlineargs, 'result_renderer'):\n            cmdlineargs.result_renderer(ret, cmdlineargs)\n    except Exception as exc:\n        ce = CapturedException(exc)\n        lgr.error(\"Failed to render results due to %s\", ce)\n        sys.exit(1)\n    # all good, not strictly needed, but makes internal testing easier\n    sys.exit(0)",
                            "ast_type_str": "FunctionDef",
                            "start_lineno": 152,
                            "end_lineno": 269,
                            "parent": null
                        }
                    ],
                    "added_entities": [],
                    "deleted_entities": []
                }
            ]
        },
        {
            "old_file_content": "# emacs: -*- mode: python-mode; py-indent-offset: 4; tab-width: 4; indent-tabs-mode: nil -*-\n# ex: set sts=4 ts=4 sw=4 et:\n# ## ### ### ### ### ### ### ### ### ### ### ### ### ### ### ### ### ### ### ##\n#\n#   See COPYING file distributed along with the datalad package for the\n#   copyright and license terms.\n#\n# ## ### ### ### ### ### ### ### ### ### ### ### ### ### ### ### ### ### ### ##\n\"\"\"Test functioning of the datalad main cmdline utility \"\"\"\n\nimport os\nimport re\nfrom io import StringIO\nfrom unittest.mock import patch\n\nimport datalad\nfrom ..main import (\n    _fix_datalad_ri,\n    main,\n)\nfrom ..helpers import fail_with_short_help\nfrom datalad import __version__\nfrom datalad.cmd import (\n    WitlessRunner as Runner,\n    StdOutErrCapture,\n)\nfrom datalad.ui.utils import (\n    get_console_width,\n    get_terminal_size,\n)\nfrom datalad.api import (\n    create,\n    Dataset,\n)\nfrom datalad.utils import (\n    chpwd,\n    Path,\n)\nfrom datalad.tests.utils import (\n    SkipTest,\n    assert_equal,\n    assert_in,\n    assert_not_in,\n    assert_raises,\n    assert_re_in,\n    eq_,\n    in_,\n    ok_,\n    ok_startswith,\n    on_windows,\n    slow,\n    with_tempfile,\n)\nfrom datalad.support.exceptions import CommandError\n\n\ndef run_main(args, exit_code=0, expect_stderr=False):\n    \"\"\"Run main() of the datalad, do basic checks and provide outputs\n\n    Parameters\n    ----------\n    args : list\n        List of string cmdline arguments to pass\n    exit_code : int\n        Expected exit code. Would raise AssertionError if differs\n    expect_stderr : bool or string\n        Whether to expect stderr output. If string -- match\n\n    Returns\n    -------\n    stdout, stderr  strings\n       Output produced\n    \"\"\"\n    was_mode = datalad.__api\n    try:\n        with patch('sys.stderr', new_callable=StringIO) as cmerr:\n            with patch('sys.stdout', new_callable=StringIO) as cmout:\n                with assert_raises(SystemExit) as cm:\n                    main([\"datalad\"] + list(args))\n                eq_('cmdline', datalad.get_apimode())\n                assert_equal(cm.exception.code, exit_code)\n                stdout = cmout.getvalue()\n                stderr = cmerr.getvalue()\n                if expect_stderr is False:\n                    assert_equal(stderr, \"\")\n                elif expect_stderr is True:\n                    # do nothing -- just return\n                    pass\n                else:\n                    # must be a string\n                    assert_equal(stderr, expect_stderr)\n    finally:\n        # restore what we had\n        datalad.__api = was_mode\n\n    return stdout, stderr\n\n\n# TODO: switch to stdout for --version output\ndef test_version():\n    # we just get a version if not asking for a version of some command\n    stdout, stderr = run_main(['--version'], expect_stderr=True)\n    eq_(stdout.rstrip(), \"datalad %s\" % datalad.__version__)\n\n    stdout, stderr = run_main(['clone', '--version'], expect_stderr=True)\n    ok_startswith(stdout, 'datalad %s\\n' % datalad.__version__)\n    # since https://github.com/datalad/datalad/pull/2733 no license in --version\n    assert_not_in(\"Copyright\", stdout)\n    assert_not_in(\"Permission is hereby granted\", stdout)\n\n    try:\n        import datalad_container\n    except ImportError:\n        pass  # not installed, cannot test with extension\n    else:\n        stdout, stderr = run_main(['containers-list', '--version'], expect_stderr=True)\n        eq_(stdout, 'datalad_container %s\\n' % datalad_container.__version__)\n\n\ndef test_help_np():\n    stdout, stderr = run_main(['--help-np'])\n\n    # Let's extract section titles:\n    # enough of bin/datalad and .tox/py27/bin/datalad -- guarantee consistency! ;)\n    ok_startswith(stdout, 'Usage: datalad')\n    # Sections start/end with * if ran under DATALAD_HELP2MAN mode\n    sections = [l[1:-1] for l in filter(re.compile('^\\*.*\\*$').match, stdout.split('\\n'))]\n    for s in {'Essential commands',\n              'Commands for metadata handling',\n              'Miscellaneous commands',\n              'General information',\n              'Global options',\n              'Plumbing commands',\n              }:\n        assert_in(s, sections)\n\n    if not get_terminal_size()[0] or 0:\n        raise SkipTest(\n            \"Could not determine terminal size, skipping the rest of the test\")\n\n    # none of the lines must be longer than 80 chars\n    # TODO: decide on   create-sibling and possibly\n    # rewrite-urls\n    accepted_width = get_console_width()\n\n    long_lines = [\"%d %s\" % (len(l), l) for l in stdout.split('\\n')\n                  if len(l) > accepted_width and\n                  '{' not in l  # on nd70 summary line is unsplit\n                  ]\n    if long_lines:\n        raise AssertionError(\n            \"Following lines in --help output were longer than %s chars:\\n%s\"\n            % (accepted_width, '\\n'.join(long_lines))\n        )\n\n\ndef test_usage_on_insufficient_args():\n    stdout, stderr = run_main(['install'], exit_code=2, expect_stderr=True)\n    ok_startswith(stderr, 'usage:')\n\n\ndef test_subcmd_usage_on_unknown_args():\n    stdout, stderr = run_main(['get', '--murks'], exit_code=1, expect_stderr=True)\n    in_('get', stdout)\n\n\ndef test_combined_short_option():\n    stdout, stderr = run_main(['-fjson'], exit_code=2, expect_stderr=True)\n    assert_not_in(\"unrecognized argument\", stderr)\n    assert_in(\"too few arguments\", stderr)\n\n\ndef check_incorrect_option(opts, err_str):\n    # The first line used to be:\n    # stdout, stderr = run_main((sys.argv[0],) + opts, expect_stderr=True, exit_code=2)\n    # But: what do we expect to be in sys.argv[0] here?\n    # It depends on how we invoke the test.\n    # - nosetests -s -v datalad/cmdline/tests/test_main.py would result in:\n    #   sys.argv[0}=='nosetests'\n    # - python -m nose -s -v datalad/cmdline/tests/test_main.py would result in:\n    #   sys.argv[0}=='python -m nose'\n    # - python -c \"import nose; nose.main()\" -s -v datalad/cmdline/tests/test_main.py would result in:\n    #   sys.argv[0]=='-c'\n    # This led to failure in case sys.argv[0] contained an option, that was\n    # defined to be a datalad option too, therefore was a 'known_arg' and was\n    # checked to meet its constraints.\n    # But sys.argv[0] actually isn't used by main at all. It simply doesn't\n    # matter what's in there. The only thing important to pass here is `opts`.\n    stdout, stderr = run_main(opts, expect_stderr=True, exit_code=2)\n    out = stdout + stderr\n    assert_in(\"usage: \", out)\n    assert_re_in(err_str, out, match=False)\n\n\ndef test_incorrect_options():\n    # apparently a bit different if following a good one so let's do both\n    err_invalid = \"error: (invalid|too few arguments|unrecognized argument)\"\n    yield check_incorrect_option, ('--buga',), err_invalid\n    yield check_incorrect_option, ('--dbg', '--buga'), err_invalid\n\n    err_insufficient = err_invalid  # \"specify\"\n    yield check_incorrect_option, ('--dbg',), err_insufficient\n    yield check_incorrect_option, tuple(), err_insufficient\n\n\ndef test_script_shims():\n    runner = Runner()\n    script = 'datalad'\n    if not on_windows:\n        # those must be available for execution, and should not contain\n        which = runner.run(['which', script], protocol=StdOutErrCapture)['stdout']\n        # test if there is no easy install shim in there\n        with open(which.rstrip()) as f:\n            content = f.read()\n    else:\n        from distutils.spawn import find_executable\n        content = find_executable(script)\n\n    # and let's check that it is our script\n    out = runner.run([script, '--version'], protocol=StdOutErrCapture)\n    version = out['stdout'].rstrip()\n    mod, version = version.split(' ', 1)\n    assert_equal(mod, 'datalad')\n    # we can get git and non git .dev version... so for now\n    # relax\n    get_numeric_portion = lambda v: [x for x in re.split('[+.]', v) if x.isdigit()]\n    # extract numeric portion\n    assert get_numeric_portion(version), f\"Got no numeric portion from {version}\"\n    assert_equal(get_numeric_portion(__version__),\n                 get_numeric_portion(version))\n\n\n@slow  # 11.2591s\n@with_tempfile(mkdir=True)\ndef test_cfg_override(path):\n    with chpwd(path):\n        cmd = ['datalad', 'wtf', '-s', 'some']\n        # control\n        out = Runner().run(cmd, protocol=StdOutErrCapture)['stdout']\n        assert_not_in('datalad.dummy: this', out)\n        # ensure that this is not a dataset's cfg manager\n        assert_not_in('datalad.dataset.id', out)\n        # env var\n        out = Runner(env=dict(os.environ, DATALAD_DUMMY='this')).run(\n            cmd, protocol=StdOutErrCapture)['stdout']\n        assert_in('datalad.dummy: this', out)\n        # cmdline arg\n        out = Runner().run([cmd[0], '-c', 'datalad.dummy=this'] + cmd[1:],\n                           protocol=StdOutErrCapture)['stdout']\n        assert_in('datalad.dummy: this', out)\n\n        # now create a dataset in the path. the wtf plugin will switch to\n        # using the dataset's config manager, which must inherit the overrides\n        create(dataset=path)\n        # control\n        out = Runner().run(cmd, protocol=StdOutErrCapture)['stdout']\n        assert_not_in('datalad.dummy: this', out)\n        # ensure that this is a dataset's cfg manager\n        assert_in('datalad.dataset.id', out)\n        # env var\n        out = Runner(env=dict(os.environ, DATALAD_DUMMY='this')).run(\n            cmd, protocol=StdOutErrCapture)['stdout']\n        assert_in('datalad.dummy: this', out)\n        # cmdline arg\n        out = Runner().run([cmd[0], '-c', 'datalad.dummy=this'] + cmd[1:],\n                           protocol=StdOutErrCapture)['stdout']\n        assert_in('datalad.dummy: this', out)\n\n\ndef test_incorrect_cfg_override():\n    run_main(['-c', 'some', 'wtf'], exit_code=3)\n    run_main(['-c', 'some=', 'wtf'], exit_code=3)\n    run_main(['-c', 'some.var', 'wtf'], exit_code=3)\n    run_main(['-c', 'some.var=', 'wtf'], exit_code=3)\n\n\ndef test_fail_with_short_help():\n    out = StringIO()\n    with assert_raises(SystemExit) as cme:\n        fail_with_short_help(exit_code=3, out=out)\n    assert_equal(cme.exception.code, 3)\n    assert_equal(out.getvalue(), \"\")\n\n    out = StringIO()\n    with assert_raises(SystemExit) as cme:\n        fail_with_short_help(msg=\"Failed badly\", out=out)\n    assert_equal(cme.exception.code, 1)\n    assert_equal(out.getvalue(), \"error: Failed badly\\n\")\n\n    # Suggestions, hint, etc\n    out = StringIO()\n    with assert_raises(SystemExit) as cme:\n        fail_with_short_help(\n            msg=\"Failed badly\",\n            known=[\"mother\", \"mutter\", \"father\", \"son\"],\n            provided=\"muther\",\n            hint=\"You can become one\",\n            exit_code=0,  # nobody forbids\n            what=\"parent\",\n            out=out)\n    assert_equal(cme.exception.code, 0)\n    assert_equal(out.getvalue(),\n                 \"error: Failed badly\\n\"\n                 \"datalad: Unknown parent 'muther'.  See 'datalad --help'.\\n\\n\"\n                 \"Did you mean any of these?\\n\"\n                 \"        mutter\\n\"\n                 \"        mother\\n\"\n                 \"        father\\n\"\n                 \"Hint: You can become one\\n\")\n\ndef test_fix_datalad_ri():\n    assert_equal(_fix_datalad_ri('/'), '/')\n    assert_equal(_fix_datalad_ri('/a/b'), '/a/b')\n    assert_equal(_fix_datalad_ri('//'), '///')\n    assert_equal(_fix_datalad_ri('///'), '///')\n    assert_equal(_fix_datalad_ri('//a'), '///a')\n    assert_equal(_fix_datalad_ri('///a'), '///a')\n    assert_equal(_fix_datalad_ri('//a/b'), '///a/b')\n    assert_equal(_fix_datalad_ri('///a/b'), '///a/b')\n\n\ndef test_fail_with_short_help():\n    out = StringIO()\n    with assert_raises(SystemExit) as cme:\n        fail_with_short_help(exit_code=3, out=out)\n    assert_equal(cme.exception.code, 3)\n    assert_equal(out.getvalue(), \"\")\n\n    out = StringIO()\n    with assert_raises(SystemExit) as cme:\n        fail_with_short_help(msg=\"Failed badly\", out=out)\n    assert_equal(cme.exception.code, 1)\n    assert_equal(out.getvalue(), \"error: Failed badly\\n\")\n\n    # Suggestions, hint, etc\n    out = StringIO()\n    with assert_raises(SystemExit) as cme:\n        fail_with_short_help(\n            msg=\"Failed badly\",\n            known=[\"mother\", \"mutter\", \"father\", \"son\"],\n            provided=\"muther\",\n            hint=\"You can become one\",\n            exit_code=0,  # nobody forbids\n            what=\"parent\",\n            out=out)\n    assert_equal(cme.exception.code, 0)\n    assert_equal(out.getvalue(),\n                 \"error: Failed badly\\n\"\n                 \"datalad: Unknown parent 'muther'.  See 'datalad --help'.\\n\\n\"\n                 \"Did you mean any of these?\\n\"\n                 \"        mutter\\n\"\n                 \"        mother\\n\"\n                 \"        father\\n\"\n                 \"Hint: You can become one\\n\")\n\n\ndef test_fix_datalad_ri():\n    assert_equal(_fix_datalad_ri('/'), '/')\n    assert_equal(_fix_datalad_ri('/a/b'), '/a/b')\n    assert_equal(_fix_datalad_ri('//'), '///')\n    assert_equal(_fix_datalad_ri('///'), '///')\n    assert_equal(_fix_datalad_ri('//a'), '///a')\n    assert_equal(_fix_datalad_ri('///a'), '///a')\n    assert_equal(_fix_datalad_ri('//a/b'), '///a/b')\n    assert_equal(_fix_datalad_ri('///a/b'), '///a/b')\n\n\n@with_tempfile\n@with_tempfile(mkdir=True)\ndef test_commanderror_jsonmsgs(src, exp):\n    ds = Dataset(src).create()\n    (ds.pathobj / '123').write_text('123')\n    ds.save()\n    ds.repo.call_annex([\n        'initremote', 'expdir', 'type=directory',\n        'directory={}'.format(exp),\n        'encryption=none',\n        'exporttree=yes'\n    ])\n    #ds.repo.call_annex(['export', '--to=expdir', 'HEAD'])\n    # this must fail, because `push` cannot handle an export.\n    # when https://github.com/datalad/datalad/issues/3127 is implemented\n    # this test must be adjusted\n    with assert_raises(CommandError) as cme:\n        Runner(cwd=ds.path).run(\n            ['datalad', 'push', '--to', 'expdir'],\n            protocol=StdOutErrCapture)\n    if ds.repo.git_annex_version >= \"8.20201129\":\n        in_('use `git-annex export`', cme.exception.stderr)\n\n\ndef test_librarymode():\n    was_mode = datalad.__runtime_mode\n    try:\n        # clean --dry-run is just a no-op command that is cheap\n        # to execute. It has no particular role here, other than\n        # to make the code pass the location where library mode\n        # should be turned on via the cmdline API\n        run_main(['--library-mode', 'clean', '--dry-run'])\n        ok_(datalad.in_librarymode())\n    finally:\n        datalad.__runtime_mode = was_mode\n",
            "new_file_content": "# emacs: -*- mode: python-mode; py-indent-offset: 4; tab-width: 4; indent-tabs-mode: nil -*-\n# ex: set sts=4 ts=4 sw=4 et:\n# ## ### ### ### ### ### ### ### ### ### ### ### ### ### ### ### ### ### ### ##\n#\n#   See COPYING file distributed along with the datalad package for the\n#   copyright and license terms.\n#\n# ## ### ### ### ### ### ### ### ### ### ### ### ### ### ### ### ### ### ### ##\n\"\"\"Test functioning of the datalad main cmdline utility \"\"\"\n\nimport os\nimport re\nfrom io import StringIO\nfrom unittest.mock import patch\n\nimport datalad\nfrom ..main import (\n    _fix_datalad_ri,\n    main,\n)\nfrom ..helpers import fail_with_short_help\nfrom datalad import __version__\nfrom datalad.cmd import (\n    WitlessRunner as Runner,\n    StdOutErrCapture,\n)\nfrom datalad.ui.utils import (\n    get_console_width,\n    get_terminal_size,\n)\nfrom datalad.api import (\n    create,\n    Dataset,\n)\nfrom datalad.utils import (\n    chpwd,\n    Path,\n)\nfrom datalad.tests.utils import (\n    SkipTest,\n    assert_equal,\n    assert_in,\n    assert_not_in,\n    assert_raises,\n    assert_re_in,\n    eq_,\n    in_,\n    ok_,\n    ok_startswith,\n    on_windows,\n    slow,\n    with_tempfile,\n)\nfrom datalad.support.exceptions import CommandError\n\n\ndef run_main(args, exit_code=0, expect_stderr=False):\n    \"\"\"Run main() of the datalad, do basic checks and provide outputs\n\n    Parameters\n    ----------\n    args : list\n        List of string cmdline arguments to pass\n    exit_code : int\n        Expected exit code. Would raise AssertionError if differs\n    expect_stderr : bool or string\n        Whether to expect stderr output. If string -- match\n\n    Returns\n    -------\n    stdout, stderr  strings\n       Output produced\n    \"\"\"\n    was_mode = datalad.__api\n    try:\n        with patch('sys.stderr', new_callable=StringIO) as cmerr:\n            with patch('sys.stdout', new_callable=StringIO) as cmout:\n                with assert_raises(SystemExit) as cm:\n                    main([\"datalad\"] + list(args))\n                eq_('cmdline', datalad.get_apimode())\n                assert_equal(cm.exception.code, exit_code)\n                stdout = cmout.getvalue()\n                stderr = cmerr.getvalue()\n                if expect_stderr is False:\n                    assert_equal(stderr, \"\")\n                elif expect_stderr is True:\n                    # do nothing -- just return\n                    pass\n                else:\n                    # must be a string\n                    assert_equal(stderr, expect_stderr)\n    finally:\n        # restore what we had\n        datalad.__api = was_mode\n\n    return stdout, stderr\n\n\n# TODO: switch to stdout for --version output\ndef test_version():\n    # we just get a version if not asking for a version of some command\n    stdout, stderr = run_main(['--version'], expect_stderr=True)\n    eq_(stdout.rstrip(), \"datalad %s\" % datalad.__version__)\n\n    stdout, stderr = run_main(['clone', '--version'], expect_stderr=True)\n    ok_startswith(stdout, 'datalad %s\\n' % datalad.__version__)\n    # since https://github.com/datalad/datalad/pull/2733 no license in --version\n    assert_not_in(\"Copyright\", stdout)\n    assert_not_in(\"Permission is hereby granted\", stdout)\n\n    try:\n        import datalad_container\n    except ImportError:\n        pass  # not installed, cannot test with extension\n    else:\n        stdout, stderr = run_main(['containers-list', '--version'], expect_stderr=True)\n        eq_(stdout, 'datalad_container %s\\n' % datalad_container.__version__)\n\n\ndef test_help_np():\n    stdout, stderr = run_main(['--help-np'])\n\n    # Let's extract section titles:\n    # enough of bin/datalad and .tox/py27/bin/datalad -- guarantee consistency! ;)\n    ok_startswith(stdout, 'Usage: datalad')\n    # Sections start/end with * if ran under DATALAD_HELP2MAN mode\n    sections = [l[1:-1] for l in filter(re.compile('^\\*.*\\*$').match, stdout.split('\\n'))]\n    for s in {'Essential commands',\n              'Commands for metadata handling',\n              'Miscellaneous commands',\n              'General information',\n              'Global options',\n              'Plumbing commands',\n              }:\n        assert_in(s, sections)\n\n    if not get_terminal_size()[0] or 0:\n        raise SkipTest(\n            \"Could not determine terminal size, skipping the rest of the test\")\n\n    # none of the lines must be longer than 80 chars\n    # TODO: decide on   create-sibling and possibly\n    # rewrite-urls\n    accepted_width = get_console_width()\n\n    long_lines = [\"%d %s\" % (len(l), l) for l in stdout.split('\\n')\n                  if len(l) > accepted_width and\n                  '{' not in l  # on nd70 summary line is unsplit\n                  ]\n    if long_lines:\n        raise AssertionError(\n            \"Following lines in --help output were longer than %s chars:\\n%s\"\n            % (accepted_width, '\\n'.join(long_lines))\n        )\n\n\ndef test_usage_on_insufficient_args():\n    stdout, stderr = run_main(['install'], exit_code=2, expect_stderr=True)\n    ok_startswith(stderr, 'usage:')\n\n\ndef test_subcmd_usage_on_unknown_args():\n    stdout, stderr = run_main(['get', '--murks'], exit_code=1, expect_stderr=True)\n    in_('get', stdout)\n\n\ndef test_combined_short_option():\n    stdout, stderr = run_main(['-fjson'], exit_code=2, expect_stderr=True)\n    assert_not_in(\"unrecognized argument\", stderr)\n    assert_in(\"too few arguments\", stderr)\n\n\ndef check_incorrect_option(opts, err_str):\n    # The first line used to be:\n    # stdout, stderr = run_main((sys.argv[0],) + opts, expect_stderr=True, exit_code=2)\n    # But: what do we expect to be in sys.argv[0] here?\n    # It depends on how we invoke the test.\n    # - nosetests -s -v datalad/cmdline/tests/test_main.py would result in:\n    #   sys.argv[0}=='nosetests'\n    # - python -m nose -s -v datalad/cmdline/tests/test_main.py would result in:\n    #   sys.argv[0}=='python -m nose'\n    # - python -c \"import nose; nose.main()\" -s -v datalad/cmdline/tests/test_main.py would result in:\n    #   sys.argv[0]=='-c'\n    # This led to failure in case sys.argv[0] contained an option, that was\n    # defined to be a datalad option too, therefore was a 'known_arg' and was\n    # checked to meet its constraints.\n    # But sys.argv[0] actually isn't used by main at all. It simply doesn't\n    # matter what's in there. The only thing important to pass here is `opts`.\n    stdout, stderr = run_main(opts, expect_stderr=True, exit_code=2)\n    out = stdout + stderr\n    assert_in(\"usage: \", out)\n    assert_re_in(err_str, out, match=False)\n\n\ndef test_incorrect_options():\n    # apparently a bit different if following a good one so let's do both\n    err_invalid = \"error: (invalid|too few arguments|unrecognized argument)\"\n    yield check_incorrect_option, ('--buga',), err_invalid\n    yield check_incorrect_option, ('--dbg', '--buga'), err_invalid\n\n    err_insufficient = err_invalid  # \"specify\"\n    yield check_incorrect_option, ('--dbg',), err_insufficient\n    yield check_incorrect_option, tuple(), err_insufficient\n\n\ndef test_script_shims():\n    runner = Runner()\n    script = 'datalad'\n    if not on_windows:\n        # those must be available for execution, and should not contain\n        which = runner.run(['which', script], protocol=StdOutErrCapture)['stdout']\n        # test if there is no easy install shim in there\n        with open(which.rstrip()) as f:\n            content = f.read()\n    else:\n        from distutils.spawn import find_executable\n        content = find_executable(script)\n\n    # and let's check that it is our script\n    out = runner.run([script, '--version'], protocol=StdOutErrCapture)\n    version = out['stdout'].rstrip()\n    mod, version = version.split(' ', 1)\n    assert_equal(mod, 'datalad')\n    # we can get git and non git .dev version... so for now\n    # relax\n    get_numeric_portion = lambda v: [x for x in re.split('[+.]', v) if x.isdigit()]\n    # extract numeric portion\n    assert get_numeric_portion(version), f\"Got no numeric portion from {version}\"\n    assert_equal(get_numeric_portion(__version__),\n                 get_numeric_portion(version))\n\n\n@slow  # 11.2591s\n@with_tempfile(mkdir=True)\ndef test_cfg_override(path):\n    with chpwd(path):\n        cmd = ['datalad', 'wtf', '-s', 'some']\n        # control\n        out = Runner().run(cmd, protocol=StdOutErrCapture)['stdout']\n        assert_not_in('datalad.dummy: this', out)\n        # ensure that this is not a dataset's cfg manager\n        assert_not_in('datalad.dataset.id', out)\n        # env var\n        out = Runner(env=dict(os.environ, DATALAD_DUMMY='this')).run(\n            cmd, protocol=StdOutErrCapture)['stdout']\n        assert_in('datalad.dummy: this', out)\n        # cmdline arg\n        out = Runner().run([cmd[0], '-c', 'datalad.dummy=this'] + cmd[1:],\n                           protocol=StdOutErrCapture)['stdout']\n        assert_in('datalad.dummy: this', out)\n\n        # now create a dataset in the path. the wtf plugin will switch to\n        # using the dataset's config manager, which must inherit the overrides\n        create(dataset=path)\n        # control\n        out = Runner().run(cmd, protocol=StdOutErrCapture)['stdout']\n        assert_not_in('datalad.dummy: this', out)\n        # ensure that this is a dataset's cfg manager\n        assert_in('datalad.dataset.id', out)\n        # env var\n        out = Runner(env=dict(os.environ, DATALAD_DUMMY='this')).run(\n            cmd, protocol=StdOutErrCapture)['stdout']\n        assert_in('datalad.dummy: this', out)\n        # cmdline arg\n        out = Runner().run([cmd[0], '-c', 'datalad.dummy=this'] + cmd[1:],\n                           protocol=StdOutErrCapture)['stdout']\n        assert_in('datalad.dummy: this', out)\n\n\ndef test_incorrect_cfg_override():\n    run_main(['-c', 'some', 'wtf'], exit_code=3)\n    run_main(['-c', 'some=', 'wtf'], exit_code=3)\n    run_main(['-c', 'some.var', 'wtf'], exit_code=3)\n    run_main(['-c', 'some.var=', 'wtf'], exit_code=3)\n\n\ndef test_fail_with_short_help():\n    out = StringIO()\n    with assert_raises(SystemExit) as cme:\n        fail_with_short_help(exit_code=3, out=out)\n    assert_equal(cme.exception.code, 3)\n    assert_equal(out.getvalue(), \"\")\n\n    out = StringIO()\n    with assert_raises(SystemExit) as cme:\n        fail_with_short_help(msg=\"Failed badly\", out=out)\n    assert_equal(cme.exception.code, 1)\n    assert_equal(out.getvalue(), \"error: Failed badly\\n\")\n\n    # Suggestions, hint, etc\n    out = StringIO()\n    with assert_raises(SystemExit) as cme:\n        fail_with_short_help(\n            msg=\"Failed badly\",\n            known=[\"mother\", \"mutter\", \"father\", \"son\"],\n            provided=\"muther\",\n            hint=\"You can become one\",\n            exit_code=0,  # nobody forbids\n            what=\"parent\",\n            out=out)\n    assert_equal(cme.exception.code, 0)\n    assert_equal(out.getvalue(),\n                 \"error: Failed badly\\n\"\n                 \"datalad: Unknown parent 'muther'.  See 'datalad --help'.\\n\\n\"\n                 \"Did you mean any of these?\\n\"\n                 \"        mutter\\n\"\n                 \"        mother\\n\"\n                 \"        father\\n\"\n                 \"Hint: You can become one\\n\")\n\ndef test_fix_datalad_ri():\n    assert_equal(_fix_datalad_ri('/'), '/')\n    assert_equal(_fix_datalad_ri('/a/b'), '/a/b')\n    assert_equal(_fix_datalad_ri('//'), '///')\n    assert_equal(_fix_datalad_ri('///'), '///')\n    assert_equal(_fix_datalad_ri('//a'), '///a')\n    assert_equal(_fix_datalad_ri('///a'), '///a')\n    assert_equal(_fix_datalad_ri('//a/b'), '///a/b')\n    assert_equal(_fix_datalad_ri('///a/b'), '///a/b')\n\n\ndef test_fail_with_short_help():\n    out = StringIO()\n    with assert_raises(SystemExit) as cme:\n        fail_with_short_help(exit_code=3, out=out)\n    assert_equal(cme.exception.code, 3)\n    assert_equal(out.getvalue(), \"\")\n\n    out = StringIO()\n    with assert_raises(SystemExit) as cme:\n        fail_with_short_help(msg=\"Failed badly\", out=out)\n    assert_equal(cme.exception.code, 1)\n    assert_equal(out.getvalue(), \"error: Failed badly\\n\")\n\n    # Suggestions, hint, etc\n    out = StringIO()\n    with assert_raises(SystemExit) as cme:\n        fail_with_short_help(\n            msg=\"Failed badly\",\n            known=[\"mother\", \"mutter\", \"father\", \"son\"],\n            provided=\"muther\",\n            hint=\"You can become one\",\n            exit_code=0,  # nobody forbids\n            what=\"parent\",\n            out=out)\n    assert_equal(cme.exception.code, 0)\n    assert_equal(out.getvalue(),\n                 \"error: Failed badly\\n\"\n                 \"datalad: Unknown parent 'muther'.  See 'datalad --help'.\\n\\n\"\n                 \"Did you mean any of these?\\n\"\n                 \"        mutter\\n\"\n                 \"        mother\\n\"\n                 \"        father\\n\"\n                 \"Hint: You can become one\\n\")\n\n\ndef test_fix_datalad_ri():\n    assert_equal(_fix_datalad_ri('/'), '/')\n    assert_equal(_fix_datalad_ri('/a/b'), '/a/b')\n    assert_equal(_fix_datalad_ri('//'), '///')\n    assert_equal(_fix_datalad_ri('///'), '///')\n    assert_equal(_fix_datalad_ri('//a'), '///a')\n    assert_equal(_fix_datalad_ri('///a'), '///a')\n    assert_equal(_fix_datalad_ri('//a/b'), '///a/b')\n    assert_equal(_fix_datalad_ri('///a/b'), '///a/b')\n\n\n@with_tempfile\n@with_tempfile(mkdir=True)\ndef test_commanderror_jsonmsgs(src, exp):\n    ds = Dataset(src).create()\n    (ds.pathobj / '123').write_text('123')\n    ds.save()\n    ds.repo.call_annex([\n        'initremote', 'expdir', 'type=directory',\n        'directory={}'.format(exp),\n        'encryption=none',\n        'exporttree=yes'\n    ])\n    #ds.repo.call_annex(['export', '--to=expdir', 'HEAD'])\n    # this must fail, because `push` cannot handle an export.\n    # when https://github.com/datalad/datalad/issues/3127 is implemented\n    # this test must be adjusted\n    with assert_raises(CommandError) as cme:\n        Runner(cwd=ds.path).run(\n            ['datalad', 'push', '--to', 'expdir'],\n            protocol=StdOutErrCapture)\n    if ds.repo.git_annex_version >= \"8.20201129\":\n        in_('use `git-annex export`', cme.exception.stderr)\n\n\ndef test_librarymode():\n    was_mode = datalad.__runtime_mode\n    try:\n        # clean --dry-run is just a no-op command that is cheap\n        # to execute. It has no particular role here, other than\n        # to make the code pass the location where library mode\n        # should be turned on via the cmdline API\n        run_main(['-c', 'datalad.runtime.librarymode=yes', 'clean', '--dry-run'])\n        ok_(datalad.in_librarymode())\n    finally:\n        # restore pre-test behavior\n        datalad.__runtime_mode = was_mode\n        datalad.cfg.overrides.pop('datalad.runtime.librarymode')\n",
            "header": {
                "file": {
                    "path": "datalad/cmdline/tests/test_main.py"
                },
                "misc_line": null
            },
            "index_line": {
                "old_commit_hash": "cc9397587",
                "new_commit_hash": "d7806cd8b",
                "mode": "100644"
            },
            "is_binary_file": false,
            "binary_line": null,
            "minus_file": {
                "path": "a/datalad/cmdline/tests/test_main.py"
            },
            "plus_file": {
                "path": "b/datalad/cmdline/tests/test_main.py"
            },
            "hunks": [
                {
                    "descriptor": {
                        "old_range": {
                            "start": 396,
                            "length": 7
                        },
                        "new_range": {
                            "start": 396,
                            "length": 9
                        },
                        "section": "def test_librarymode():"
                    },
                    "line_group": {
                        "all_lines": [
                            {
                                "content": "        # to execute. It has no particular role here, other than",
                                "type": "context"
                            },
                            {
                                "content": "        # to make the code pass the location where library mode",
                                "type": "context"
                            },
                            {
                                "content": "        # should be turned on via the cmdline API",
                                "type": "context"
                            },
                            {
                                "content": "        run_main(['--library-mode', 'clean', '--dry-run'])",
                                "type": "deleted"
                            },
                            {
                                "content": "        run_main(['-c', 'datalad.runtime.librarymode=yes', 'clean', '--dry-run'])",
                                "type": "added"
                            },
                            {
                                "content": "        ok_(datalad.in_librarymode())",
                                "type": "context"
                            },
                            {
                                "content": "    finally:",
                                "type": "context"
                            },
                            {
                                "content": "        # restore pre-test behavior",
                                "type": "added"
                            },
                            {
                                "content": "        datalad.__runtime_mode = was_mode",
                                "type": "context"
                            },
                            {
                                "content": "        datalad.cfg.overrides.pop('datalad.runtime.librarymode')",
                                "type": "added"
                            }
                        ]
                    },
                    "modified_entities": [
                        {
                            "file_name": "datalad/cmdline/tests/test_main.py",
                            "type": "function",
                            "name": "test_librarymode",
                            "content": "def test_librarymode():\n    was_mode = datalad.__runtime_mode\n    try:\n        # clean --dry-run is just a no-op command that is cheap\n        # to execute. It has no particular role here, other than\n        # to make the code pass the location where library mode\n        # should be turned on via the cmdline API\n        run_main(['--library-mode', 'clean', '--dry-run'])\n        ok_(datalad.in_librarymode())\n    finally:\n        datalad.__runtime_mode = was_mode",
                            "ast_type_str": "FunctionDef",
                            "start_lineno": 392,
                            "end_lineno": 402,
                            "parent": null
                        }
                    ],
                    "added_entities": [],
                    "deleted_entities": []
                }
            ]
        },
        {
            "old_file_content": ".. -*- mode: rst -*-\n.. vi: set ft=rst sts=4 ts=4 sw=4 et tw=79:\n\n.. _chap_design_application_vs_libary_mode:\n\n***************************************\nApplication-type vs. library-type usage\n***************************************\n\n.. topic:: Specification scope and status\n\n   This specification describes the current implementation.\n\nHistorically, DataLad was implemented with the assumption of application-type\nusage, i.e., a person using DataLad through any of its APIs. Consequently,\n(error) messaging was primarily targeting humans, and usage advice focused on\ninteractive use. With the increasing utilization of DataLad as an\ninfrastructural component it was necessary to address use cases of library-type\nor internal usage more explicitly.\n\nDataLad continues to behave like a stand-alone application by default.\n\nFor internal use, Python and command-line APIs provide dedicated mode switches.\n\nThe command line entry point offers a ``--library-mode`` switch that enables\nthe mode very early in the process runtime, and for the complete lifetime of the\nprocess. Similarly, the Python API offers a function ``enable_libarymode()``\nthat should be called immediately after importing the ``datalad`` package\nfor maximum impact.\n\n.. code-block:: python\n\n   >>> import datalad\n   >>> datalad.enable_libarymode()\n\nMoreover, with ``datalad.in_librarymode()`` a query utility is provided that\ncan be used throughout the code base for adjusting behavior according to the\nusage scenario.\n\nSwitching back and forth between modes during the runtime of a process is not\nsupported.\n\nCare must be taken to configure child-processes of the main DataLad\nprocess/session appropriately, for example internal Dataset procedure calls, in\norder to inherit the mode of the parent process.\n\n\nLibrary-mode implications\n=========================\n\nNo Python API docs\n  Generation of comprehensive doc-strings for all API commands is skipped. This\n  speeds up ``import datalad.api`` by about 30%.\n",
            "new_file_content": ".. -*- mode: rst -*-\n.. vi: set ft=rst sts=4 ts=4 sw=4 et tw=79:\n\n.. _chap_design_application_vs_libary_mode:\n\n***************************************\nApplication-type vs. library-type usage\n***************************************\n\n.. topic:: Specification scope and status\n\n   This specification describes the current implementation.\n\nHistorically, DataLad was implemented with the assumption of application-type\nusage, i.e., a person using DataLad through any of its APIs. Consequently,\n(error) messaging was primarily targeting humans, and usage advice focused on\ninteractive use. With the increasing utilization of DataLad as an\ninfrastructural component it was necessary to address use cases of library-type\nor internal usage more explicitly.\n\nDataLad continues to behave like a stand-alone application by default.\n\nFor internal use, Python and command-line APIs provide dedicated mode switches.\n\nLibrary mode can be enabled by setting the boolean configuration setting\n``datalad.runtime.librarymode`` **before the start of the DataLad process**.\nFrom the command line, this can be done with the option\n``-c datalad.runtime.librarymode=yes``, or any other means for setting\nconfiguration. In an already running Python process, library mode can be\nenabled by calling ``datalad.enable_libarymode()``. This should be done\nimmediately after importing the ``datalad`` package for maximum impact.\n\n.. code-block:: python\n\n   >>> import datalad\n   >>> datalad.enable_libarymode()\n\nIn a Python session, library mode **cannot** be enabled reliably by just setting\nthe configuration flag **after** the ``datalad`` package was already imported.\nThe ``enable_librarymode()`` function must be used.\n\nMoreover, with ``datalad.in_librarymode()`` a query utility is provided that\ncan be used throughout the code base for adjusting behavior according to the\nusage scenario.\n\nSwitching back and forth between modes during the runtime of a process is not\nsupported.\n\nA library mode setting is exported into the environment of the Python process.\nBy default, it will be inherited by all child-processes, such as dataset\nprocedure executions.\n\n\nLibrary-mode implications\n=========================\n\nNo Python API docs\n  Generation of comprehensive doc-strings for all API commands is skipped. This\n  speeds up ``import datalad.api`` by about 30%.\n",
            "header": {
                "file": {
                    "path": "docs/source/design/application_vs_library_mode.rst"
                },
                "misc_line": null
            },
            "index_line": {
                "old_commit_hash": "bf09b507d",
                "new_commit_hash": "3b9d3d12a",
                "mode": "100644"
            },
            "is_binary_file": false,
            "binary_line": null,
            "minus_file": {
                "path": "a/docs/source/design/application_vs_library_mode.rst"
            },
            "plus_file": {
                "path": "b/docs/source/design/application_vs_library_mode.rst"
            },
            "hunks": [
                {
                    "descriptor": {
                        "old_range": {
                            "start": 22,
                            "length": 17
                        },
                        "new_range": {
                            "start": 22,
                            "length": 23
                        },
                        "section": "DataLad continues to behave like a stand-alone application by default."
                    },
                    "line_group": {
                        "all_lines": [
                            {
                                "content": "",
                                "type": "context"
                            },
                            {
                                "content": "For internal use, Python and command-line APIs provide dedicated mode switches.",
                                "type": "context"
                            },
                            {
                                "content": "",
                                "type": "context"
                            },
                            {
                                "content": "The command line entry point offers a ``--library-mode`` switch that enables",
                                "type": "deleted"
                            },
                            {
                                "content": "the mode very early in the process runtime, and for the complete lifetime of the",
                                "type": "deleted"
                            },
                            {
                                "content": "process. Similarly, the Python API offers a function ``enable_libarymode()``",
                                "type": "deleted"
                            },
                            {
                                "content": "that should be called immediately after importing the ``datalad`` package",
                                "type": "deleted"
                            },
                            {
                                "content": "for maximum impact.",
                                "type": "deleted"
                            },
                            {
                                "content": "Library mode can be enabled by setting the boolean configuration setting",
                                "type": "added"
                            },
                            {
                                "content": "``datalad.runtime.librarymode`` **before the start of the DataLad process**.",
                                "type": "added"
                            },
                            {
                                "content": "From the command line, this can be done with the option",
                                "type": "added"
                            },
                            {
                                "content": "``-c datalad.runtime.librarymode=yes``, or any other means for setting",
                                "type": "added"
                            },
                            {
                                "content": "configuration. In an already running Python process, library mode can be",
                                "type": "added"
                            },
                            {
                                "content": "enabled by calling ``datalad.enable_libarymode()``. This should be done",
                                "type": "added"
                            },
                            {
                                "content": "immediately after importing the ``datalad`` package for maximum impact.",
                                "type": "added"
                            },
                            {
                                "content": "",
                                "type": "context"
                            },
                            {
                                "content": ".. code-block:: python",
                                "type": "context"
                            },
                            {
                                "content": "",
                                "type": "context"
                            },
                            {
                                "content": "   >>> import datalad",
                                "type": "context"
                            },
                            {
                                "content": "   >>> datalad.enable_libarymode()",
                                "type": "context"
                            },
                            {
                                "content": "",
                                "type": "context"
                            },
                            {
                                "content": "In a Python session, library mode **cannot** be enabled reliably by just setting",
                                "type": "added"
                            },
                            {
                                "content": "the configuration flag **after** the ``datalad`` package was already imported.",
                                "type": "added"
                            },
                            {
                                "content": "The ``enable_librarymode()`` function must be used.",
                                "type": "added"
                            },
                            {
                                "content": "",
                                "type": "added"
                            },
                            {
                                "content": "Moreover, with ``datalad.in_librarymode()`` a query utility is provided that",
                                "type": "context"
                            },
                            {
                                "content": "can be used throughout the code base for adjusting behavior according to the",
                                "type": "context"
                            },
                            {
                                "content": "usage scenario.",
                                "type": "context"
                            }
                        ]
                    },
                    "modified_entities": [],
                    "added_entities": [],
                    "deleted_entities": []
                },
                {
                    "descriptor": {
                        "old_range": {
                            "start": 40,
                            "length": 9
                        },
                        "new_range": {
                            "start": 46,
                            "length": 9
                        },
                        "section": "usage scenario."
                    },
                    "line_group": {
                        "all_lines": [
                            {
                                "content": "Switching back and forth between modes during the runtime of a process is not",
                                "type": "context"
                            },
                            {
                                "content": "supported.",
                                "type": "context"
                            },
                            {
                                "content": "",
                                "type": "context"
                            },
                            {
                                "content": "Care must be taken to configure child-processes of the main DataLad",
                                "type": "deleted"
                            },
                            {
                                "content": "process/session appropriately, for example internal Dataset procedure calls, in",
                                "type": "deleted"
                            },
                            {
                                "content": "order to inherit the mode of the parent process.",
                                "type": "deleted"
                            },
                            {
                                "content": "A library mode setting is exported into the environment of the Python process.",
                                "type": "added"
                            },
                            {
                                "content": "By default, it will be inherited by all child-processes, such as dataset",
                                "type": "added"
                            },
                            {
                                "content": "procedure executions.",
                                "type": "added"
                            },
                            {
                                "content": "",
                                "type": "context"
                            },
                            {
                                "content": "",
                                "type": "context"
                            },
                            {
                                "content": "Library-mode implications",
                                "type": "context"
                            },
                            {
                                "content": "",
                                "type": "context"
                            }
                        ]
                    },
                    "modified_entities": [],
                    "added_entities": [],
                    "deleted_entities": []
                }
            ]
        }
    ],
    "old_commit_hash": "f16c5086c9872148ce561df502f187581b0a18bb^",
    "new_commit_hash": "f16c5086c9872148ce561df502f187581b0a18bb",
    "commit_message": "Avoid --library-mode cmdline switch, export setting via environment to children",
    "commit_date": "2021-12-02T11:52:25+01:00",
    "metadata": {}
}
{
    "repo_name": "datalad",
    "new_commit_hash": "f16c5086c9872148ce561df502f187581b0a18bb",
    "test_file_codes": [
        "# emacs: -*- mode: python-mode; py-indent-offset: 4; tab-width: 4; indent-tabs-mode: nil -*-\n# ex: set sts=4 ts=4 sw=4 et:\n# ## ### ### ### ### ### ### ### ### ### ### ### ### ### ### ### ### ### ### ##\n#\n#   See COPYING file distributed along with the datalad package for the\n#   copyright and license terms.\n#\n# ## ### ### ### ### ### ### ### ### ### ### ### ### ### ### ### ### ### ### ##\n\"\"\"Test functioning of the datalad main cmdline utility \"\"\"\n\nimport os\nimport re\nfrom io import StringIO\nfrom unittest.mock import patch\n\nimport datalad\nfrom datalad.cmdline.main import (\n    _fix_datalad_ri,\n    main,\n)\nfrom datalad.cmdline.helpers import fail_with_short_help\nfrom datalad import __version__\nfrom datalad.cmd import (\n    WitlessRunner as Runner,\n    StdOutErrCapture,\n)\nfrom datalad.ui.utils import (\n    get_console_width,\n    get_terminal_size,\n)\nfrom datalad.api import (\n    create,\n    Dataset,\n)\nfrom datalad.utils import (\n    chpwd,\n    Path,\n)\nfrom datalad.tests.utils import (\n    SkipTest,\n    assert_equal,\n    assert_in,\n    assert_not_in,\n    assert_raises,\n    assert_re_in,\n    eq_,\n    in_,\n    ok_,\n    ok_startswith,\n    on_windows,\n    slow,\n    with_tempfile,\n)\nfrom datalad.support.exceptions import CommandError\n\n\ndef run_main(args, exit_code=0, expect_stderr=False):\n    \"\"\"Run main() of the datalad, do basic checks and provide outputs\n\n    Parameters\n    ----------\n    args : list\n        List of string cmdline arguments to pass\n    exit_code : int\n        Expected exit code. Would raise AssertionError if differs\n    expect_stderr : bool or string\n        Whether to expect stderr output. If string -- match\n\n    Returns\n    -------\n    stdout, stderr  strings\n       Output produced\n    \"\"\"\n    was_mode = datalad.__api\n    try:\n        with patch('sys.stderr', new_callable=StringIO) as cmerr:\n            with patch('sys.stdout', new_callable=StringIO) as cmout:\n                with assert_raises(SystemExit) as cm:\n                    main([\"datalad\"] + list(args))\n                eq_('cmdline', datalad.get_apimode())\n                assert_equal(cm.exception.code, exit_code)\n                stdout = cmout.getvalue()\n                stderr = cmerr.getvalue()\n                if expect_stderr is False:\n                    assert_equal(stderr, \"\")\n                elif expect_stderr is True:\n                    # do nothing -- just return\n                    pass\n                else:\n                    # must be a string\n                    assert_equal(stderr, expect_stderr)\n    finally:\n        # restore what we had\n        datalad.__api = was_mode\n\n    return stdout, stderr\n\n\n# TODO: switch to stdout for --version output\ndef test_version():\n    # we just get a version if not asking for a version of some command\n    stdout, stderr = run_main(['--version'], expect_stderr=True)\n    eq_(stdout.rstrip(), \"datalad %s\" % datalad.__version__)\n\n    stdout, stderr = run_main(['clone', '--version'], expect_stderr=True)\n    ok_startswith(stdout, 'datalad %s\\n' % datalad.__version__)\n    # since https://github.com/datalad/datalad/pull/2733 no license in --version\n    assert_not_in(\"Copyright\", stdout)\n    assert_not_in(\"Permission is hereby granted\", stdout)\n\n    try:\n        import datalad_container\n    except ImportError:\n        pass  # not installed, cannot test with extension\n    else:\n        stdout, stderr = run_main(['containers-list', '--version'], expect_stderr=True)\n        eq_(stdout, 'datalad_container %s\\n' % datalad_container.__version__)\n\n\ndef test_help_np():\n    stdout, stderr = run_main(['--help-np'])\n\n    # Let's extract section titles:\n    # enough of bin/datalad and .tox/py27/bin/datalad -- guarantee consistency! ;)\n    ok_startswith(stdout, 'Usage: datalad')\n    # Sections start/end with * if ran under DATALAD_HELP2MAN mode\n    sections = [l[1:-1] for l in filter(re.compile('^\\*.*\\*$').match, stdout.split('\\n'))]\n    for s in {'Essential commands',\n              'Commands for metadata handling',\n              'Miscellaneous commands',\n              'General information',\n              'Global options',\n              'Plumbing commands',\n              }:\n        assert_in(s, sections)\n\n    if not get_terminal_size()[0] or 0:\n        raise SkipTest(\n            \"Could not determine terminal size, skipping the rest of the test\")\n\n    # none of the lines must be longer than 80 chars\n    # TODO: decide on   create-sibling and possibly\n    # rewrite-urls\n    accepted_width = get_console_width()\n\n    long_lines = [\"%d %s\" % (len(l), l) for l in stdout.split('\\n')\n                  if len(l) > accepted_width and\n                  '{' not in l  # on nd70 summary line is unsplit\n                  ]\n    if long_lines:\n        raise AssertionError(\n            \"Following lines in --help output were longer than %s chars:\\n%s\"\n            % (accepted_width, '\\n'.join(long_lines))\n        )\n\n\ndef test_usage_on_insufficient_args():\n    stdout, stderr = run_main(['install'], exit_code=2, expect_stderr=True)\n    ok_startswith(stderr, 'usage:')\n\n\ndef test_subcmd_usage_on_unknown_args():\n    stdout, stderr = run_main(['get', '--murks'], exit_code=1, expect_stderr=True)\n    in_('get', stdout)\n\n\ndef test_combined_short_option():\n    stdout, stderr = run_main(['-fjson'], exit_code=2, expect_stderr=True)\n    assert_not_in(\"unrecognized argument\", stderr)\n    assert_in(\"too few arguments\", stderr)\n\n\ndef check_incorrect_option(opts, err_str):\n    # The first line used to be:\n    # stdout, stderr = run_main((sys.argv[0],) + opts, expect_stderr=True, exit_code=2)\n    # But: what do we expect to be in sys.argv[0] here?\n    # It depends on how we invoke the test.\n    # - nosetests -s -v datalad/cmdline/tests/test_main.py would result in:\n    #   sys.argv[0}=='nosetests'\n    # - python -m nose -s -v datalad/cmdline/tests/test_main.py would result in:\n    #   sys.argv[0}=='python -m nose'\n    # - python -c \"import nose; nose.main()\" -s -v datalad/cmdline/tests/test_main.py would result in:\n    #   sys.argv[0]=='-c'\n    # This led to failure in case sys.argv[0] contained an option, that was\n    # defined to be a datalad option too, therefore was a 'known_arg' and was\n    # checked to meet its constraints.\n    # But sys.argv[0] actually isn't used by main at all. It simply doesn't\n    # matter what's in there. The only thing important to pass here is `opts`.\n    stdout, stderr = run_main(opts, expect_stderr=True, exit_code=2)\n    out = stdout + stderr\n    assert_in(\"usage: \", out)\n    assert_re_in(err_str, out, match=False)\n\n\ndef test_incorrect_options():\n    # apparently a bit different if following a good one so let's do both\n    err_invalid = \"error: (invalid|too few arguments|unrecognized argument)\"\n    yield check_incorrect_option, ('--buga',), err_invalid\n    yield check_incorrect_option, ('--dbg', '--buga'), err_invalid\n\n    err_insufficient = err_invalid  # \"specify\"\n    yield check_incorrect_option, ('--dbg',), err_insufficient\n    yield check_incorrect_option, tuple(), err_insufficient\n\n\ndef test_script_shims():\n    runner = Runner()\n    script = 'datalad'\n    if not on_windows:\n        # those must be available for execution, and should not contain\n        which = runner.run(['which', script], protocol=StdOutErrCapture)['stdout']\n        # test if there is no easy install shim in there\n        with open(which.rstrip()) as f:\n            content = f.read()\n    else:\n        from distutils.spawn import find_executable\n        content = find_executable(script)\n\n    # and let's check that it is our script\n    out = runner.run([script, '--version'], protocol=StdOutErrCapture)\n    version = out['stdout'].rstrip()\n    mod, version = version.split(' ', 1)\n    assert_equal(mod, 'datalad')\n    # we can get git and non git .dev version... so for now\n    # relax\n    get_numeric_portion = lambda v: [x for x in re.split('[+.]', v) if x.isdigit()]\n    # extract numeric portion\n    assert get_numeric_portion(version), f\"Got no numeric portion from {version}\"\n    assert_equal(get_numeric_portion(__version__),\n                 get_numeric_portion(version))\n\n\n@slow  # 11.2591s\n@with_tempfile(mkdir=True)\ndef test_cfg_override(path):\n    with chpwd(path):\n        cmd = ['datalad', 'wtf', '-s', 'some']\n        # control\n        out = Runner().run(cmd, protocol=StdOutErrCapture)['stdout']\n        assert_not_in('datalad.dummy: this', out)\n        # ensure that this is not a dataset's cfg manager\n        assert_not_in('datalad.dataset.id', out)\n        # env var\n        out = Runner(env=dict(os.environ, DATALAD_DUMMY='this')).run(\n            cmd, protocol=StdOutErrCapture)['stdout']\n        assert_in('datalad.dummy: this', out)\n        # cmdline arg\n        out = Runner().run([cmd[0], '-c', 'datalad.dummy=this'] + cmd[1:],\n                           protocol=StdOutErrCapture)['stdout']\n        assert_in('datalad.dummy: this', out)\n\n        # now create a dataset in the path. the wtf plugin will switch to\n        # using the dataset's config manager, which must inherit the overrides\n        create(dataset=path)\n        # control\n        out = Runner().run(cmd, protocol=StdOutErrCapture)['stdout']\n        assert_not_in('datalad.dummy: this', out)\n        # ensure that this is a dataset's cfg manager\n        assert_in('datalad.dataset.id', out)\n        # env var\n        out = Runner(env=dict(os.environ, DATALAD_DUMMY='this')).run(\n            cmd, protocol=StdOutErrCapture)['stdout']\n        assert_in('datalad.dummy: this', out)\n        # cmdline arg\n        out = Runner().run([cmd[0], '-c', 'datalad.dummy=this'] + cmd[1:],\n                           protocol=StdOutErrCapture)['stdout']\n        assert_in('datalad.dummy: this', out)\n\n\ndef test_incorrect_cfg_override():\n    run_main(['-c', 'some', 'wtf'], exit_code=3)\n    run_main(['-c', 'some=', 'wtf'], exit_code=3)\n    run_main(['-c', 'some.var', 'wtf'], exit_code=3)\n    run_main(['-c', 'some.var=', 'wtf'], exit_code=3)\n\n\ndef test_fail_with_short_help():\n    out = StringIO()\n    with assert_raises(SystemExit) as cme:\n        fail_with_short_help(exit_code=3, out=out)\n    assert_equal(cme.exception.code, 3)\n    assert_equal(out.getvalue(), \"\")\n\n    out = StringIO()\n    with assert_raises(SystemExit) as cme:\n        fail_with_short_help(msg=\"Failed badly\", out=out)\n    assert_equal(cme.exception.code, 1)\n    assert_equal(out.getvalue(), \"error: Failed badly\\n\")\n\n    # Suggestions, hint, etc\n    out = StringIO()\n    with assert_raises(SystemExit) as cme:\n        fail_with_short_help(\n            msg=\"Failed badly\",\n            known=[\"mother\", \"mutter\", \"father\", \"son\"],\n            provided=\"muther\",\n            hint=\"You can become one\",\n            exit_code=0,  # nobody forbids\n            what=\"parent\",\n            out=out)\n    assert_equal(cme.exception.code, 0)\n    assert_equal(out.getvalue(),\n                 \"error: Failed badly\\n\"\n                 \"datalad: Unknown parent 'muther'.  See 'datalad --help'.\\n\\n\"\n                 \"Did you mean any of these?\\n\"\n                 \"        mutter\\n\"\n                 \"        mother\\n\"\n                 \"        father\\n\"\n                 \"Hint: You can become one\\n\")\n\ndef test_fix_datalad_ri():\n    assert_equal(_fix_datalad_ri('/'), '/')\n    assert_equal(_fix_datalad_ri('/a/b'), '/a/b')\n    assert_equal(_fix_datalad_ri('//'), '///')\n    assert_equal(_fix_datalad_ri('///'), '///')\n    assert_equal(_fix_datalad_ri('//a'), '///a')\n    assert_equal(_fix_datalad_ri('///a'), '///a')\n    assert_equal(_fix_datalad_ri('//a/b'), '///a/b')\n    assert_equal(_fix_datalad_ri('///a/b'), '///a/b')\n\n\ndef test_fail_with_short_help():\n    out = StringIO()\n    with assert_raises(SystemExit) as cme:\n        fail_with_short_help(exit_code=3, out=out)\n    assert_equal(cme.exception.code, 3)\n    assert_equal(out.getvalue(), \"\")\n\n    out = StringIO()\n    with assert_raises(SystemExit) as cme:\n        fail_with_short_help(msg=\"Failed badly\", out=out)\n    assert_equal(cme.exception.code, 1)\n    assert_equal(out.getvalue(), \"error: Failed badly\\n\")\n\n    # Suggestions, hint, etc\n    out = StringIO()\n    with assert_raises(SystemExit) as cme:\n        fail_with_short_help(\n            msg=\"Failed badly\",\n            known=[\"mother\", \"mutter\", \"father\", \"son\"],\n            provided=\"muther\",\n            hint=\"You can become one\",\n            exit_code=0,  # nobody forbids\n            what=\"parent\",\n            out=out)\n    assert_equal(cme.exception.code, 0)\n    assert_equal(out.getvalue(),\n                 \"error: Failed badly\\n\"\n                 \"datalad: Unknown parent 'muther'.  See 'datalad --help'.\\n\\n\"\n                 \"Did you mean any of these?\\n\"\n                 \"        mutter\\n\"\n                 \"        mother\\n\"\n                 \"        father\\n\"\n                 \"Hint: You can become one\\n\")\n\n\ndef test_fix_datalad_ri():\n    assert_equal(_fix_datalad_ri('/'), '/')\n    assert_equal(_fix_datalad_ri('/a/b'), '/a/b')\n    assert_equal(_fix_datalad_ri('//'), '///')\n    assert_equal(_fix_datalad_ri('///'), '///')\n    assert_equal(_fix_datalad_ri('//a'), '///a')\n    assert_equal(_fix_datalad_ri('///a'), '///a')\n    assert_equal(_fix_datalad_ri('//a/b'), '///a/b')\n    assert_equal(_fix_datalad_ri('///a/b'), '///a/b')\n\n\n@with_tempfile\n@with_tempfile(mkdir=True)\ndef test_commanderror_jsonmsgs(src, exp):\n    ds = Dataset(src).create()\n    (ds.pathobj / '123').write_text('123')\n    ds.save()\n    ds.repo.call_annex([\n        'initremote', 'expdir', 'type=directory',\n        'directory={}'.format(exp),\n        'encryption=none',\n        'exporttree=yes'\n    ])\n    #ds.repo.call_annex(['export', '--to=expdir', 'HEAD'])\n    # this must fail, because `push` cannot handle an export.\n    # when https://github.com/datalad/datalad/issues/3127 is implemented\n    # this test must be adjusted\n    with assert_raises(CommandError) as cme:\n        Runner(cwd=ds.path).run(\n            ['datalad', 'push', '--to', 'expdir'],\n            protocol=StdOutErrCapture)\n    if ds.repo.git_annex_version >= \"8.20201129\":\n        in_('use `git-annex export`', cme.exception.stderr)\n\n\ndef test_librarymode():\n    was_mode = datalad.__runtime_mode\n    try:\n        # clean --dry-run is just a no-op command that is cheap\n        # to execute. It has no particular role here, other than\n        # to make the code pass the location where library mode\n        # should be turned on via the cmdline API\n        run_main(['-c', 'datalad.runtime.librarymode=yes', 'clean', '--dry-run'])\n        ok_(datalad.in_librarymode())\n    finally:\n        # restore pre-test behavior\n        datalad.__runtime_mode = was_mode\n        datalad.cfg.overrides.pop('datalad.runtime.librarymode')\n",
        "import pytest\nimport tempfile\nimport os\n\n\n@pytest.fixture\ndef path():\n    with tempfile.TemporaryDirectory() as tmpdir:\n        yield tmpdir\n\n\n@pytest.fixture(autouse=True)\ndef setup_git_config():\n    with tempfile.TemporaryDirectory() as home:\n        old_home = os.environ.get(\"HOME\")\n        os.environ[\"HOME\"] = home\n        os.system('git config --global user.name \"DataLad Tester\"')\n        os.system('git config --global user.email \"test@example.com\"')\n        yield\n        if old_home:\n            os.environ[\"HOME\"] = old_home\n"
    ],
    "test_file_names": [
        "test_1.py",
        "conftest.py"
    ],
    "setup_res_code": 0,
    "setup_res_stdout": "Starting Datalad installation attempts...\nVerifying installation...\nDatalad version: 0.15.3+423.gf16c5086c\n✅ Installation successful!\nSuccessfully installed Datalad using Python 3.9\n",
    "setup_res_stderr": "Using CPython 3.9.21\nCreating virtual environment at: .venv\nActivate with: source .venv/bin/activate\nResolved 25 packages in 139ms\nInstalled 25 packages in 2.54s\n + coverage==7.6.10\n + docopt==0.6.2\n + exceptiongroup==1.2.2\n + grabbit==0.2.6\n + importlib-resources==6.4.5\n + iniconfig==2.0.0\n + nibabel==5.3.2\n + num2words==0.5.14\n + numpy==2.0.2\n + packaging==24.2\n + pandas==2.2.3\n + patsy==1.0.1\n + pluggy==1.5.0\n + pybids==0.6.5\n + pytest==8.3.4\n + pytest-cov==6.0.0\n + python-dateutil==2.9.0.post0\n + pytz==2024.2\n + scipy==1.13.1\n + setuptools==75.6.0\n + six==1.17.0\n + tomli==2.2.1\n + typing-extensions==4.12.2\n + tzdata==2024.2\n + zipp==3.21.0\nResolved 61 packages in 3.78s\nPrepared 1 package in 3.97s\nInstalled 55 packages in 422ms\n + annexremote==1.6.6\n + argcomplete==3.5.3\n + backports-tarfile==1.2.0\n + beautifulsoup4==4.12.3\n + boto==2.49.0\n + certifi==2024.12.14\n + cffi==1.17.1\n + chardet==5.2.0\n + charset-normalizer==3.4.1\n + citeproc-py==0.6.0\n + cryptography==44.0.0\n + datalad==0.15.3+423.gf16c5086c (from file:///home/gcpuser/buckets/local_repoeval_bucket/repos/datalad_f16c5086c9872148ce561df502f187581b0a18bb)\n + distro==1.9.0\n + duecredit==0.10.2\n + exifread==3.0.0\n + fasteners==0.19\n + httpretty==1.1.4\n + humanize==4.11.0\n + idna==3.10\n + importlib-metadata==8.5.0\n + iso8601==2.1.0\n + jaraco-classes==3.4.0\n + jaraco-context==6.0.1\n + jaraco-functools==4.1.0\n + jeepney==0.8.0\n + keyring==25.6.0\n + keyrings-alt==5.0.2\n + looseversion==1.3.0\n + lxml==5.3.0\n + more-itertools==10.5.0\n + msgpack==1.1.0\n + multidict==6.1.0\n + mutagen==1.47.0\n + nose==1.3.7\n + patool==1.15.0\n + pillow==11.1.0\n + platformdirs==4.3.6\n + propcache==0.2.1\n + pycparser==2.22\n + pyperclip==1.9.0\n + python-gitlab==5.3.0\n + python-xmp-toolkit==2.0.2\n + pyyaml==6.0.2\n + requests==2.32.3\n + requests-ftp==0.3.1\n + requests-toolbelt==1.0.0\n + secretstorage==3.3.3\n + simplejson==3.19.3\n + soupsieve==2.6\n + tqdm==4.67.1\n + urllib3==1.26.20\n + vcrpy==7.0.0\n + whoosh==2.7.4\n + wrapt==1.17.0\n + yarl==1.18.3\nResolved 94 packages in 285ms\nInstalled 31 packages in 533ms\n + alabaster==0.7.16\n + asv==0.6.4\n + asv-runner==0.2.1\n + babel==2.16.0\n + build==1.2.2.post1\n + distlib==0.3.9\n + docutils==0.21.2\n + filelock==3.16.1\n + gprof2dot==2024.6.6\n + imagesize==1.4.1\n + jinja2==3.1.5\n + json5==0.10.0\n + markupsafe==3.0.2\n + nose-timer==1.0.1\n + psutil==6.1.1\n + pygments==2.18.0\n + pympler==1.1\n + pypandoc==1.14\n + pyproject-hooks==1.2.0\n + snowballstemmer==2.2.0\n + sphinx==7.4.7\n + sphinx-rtd-theme==3.0.2\n + sphinxcontrib-applehelp==2.0.0\n + sphinxcontrib-devhelp==2.0.0\n + sphinxcontrib-htmlhelp==2.1.0\n + sphinxcontrib-jquery==4.1\n + sphinxcontrib-jsmath==1.0.1\n + sphinxcontrib-qthelp==2.0.0\n + sphinxcontrib-serializinghtml==2.0.0\n + tabulate==0.9.0\n + virtualenv==20.28.0\n",
    "new_commit_res_code": 1,
    "new_commit_res_stdout": "============================= test session starts ==============================\nplatform linux -- Python 3.9.21, pytest-8.3.4, pluggy-1.5.0\nrootdir: /home/gcpuser/buckets/local_repoeval_bucket/repos/datalad_f16c5086c9872148ce561df502f187581b0a18bb\nconfigfile: pyproject.toml\nplugins: cov-6.0.0\ncollected 13 items\n\nr2e_tests/test_1.py .s...xFF...E.                                        [100%]\n\n==================================== ERRORS ====================================\n_________________ ERROR at setup of test_commanderror_jsonmsgs _________________\nfile /home/gcpuser/buckets/local_repoeval_bucket/repos/datalad_f16c5086c9872148ce561df502f187581b0a18bb/r2e_tests/test_1.py, line 368\n  @with_tempfile\n  @with_tempfile(mkdir=True)\n  def test_commanderror_jsonmsgs(src, exp):\nE       fixture 'src' not found\n>       available fixtures: cache, capfd, capfdbinary, caplog, capsys, capsysbinary, cov, doctest_namespace, monkeypatch, no_cover, path, pytestconfig, record_property, record_testsuite_property, record_xml_attribute, recwarn, setup_git_config, tmp_path, tmp_path_factory, tmpdir, tmpdir_factory\n>       use 'pytest --fixtures [testpath]' for help on them.\n\n/home/gcpuser/buckets/local_repoeval_bucket/repos/datalad_f16c5086c9872148ce561df502f187581b0a18bb/r2e_tests/test_1.py:368\n=================================== FAILURES ===================================\n______________________________ test_script_shims _______________________________\n\n    def test_script_shims():\n        runner = Runner()\n        script = 'datalad'\n        if not on_windows:\n            # those must be available for execution, and should not contain\n            which = runner.run(['which', script], protocol=StdOutErrCapture)['stdout']\n            # test if there is no easy install shim in there\n            with open(which.rstrip()) as f:\n                content = f.read()\n        else:\n            from distutils.spawn import find_executable\n            content = find_executable(script)\n    \n        # and let's check that it is our script\n>       out = runner.run([script, '--version'], protocol=StdOutErrCapture)\n\nr2e_tests/test_1.py:220: \n_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ \ndatalad/runner/runner.py:126: in run\n    results = run_command(\ndatalad/runner/nonasyncrunner.py:203: in run_command\n    process = subprocess.Popen(cmd, **kwargs)\n../../../../.local/share/uv/python/cpython-3.9.21-linux-x86_64-gnu/lib/python3.9/subprocess.py:951: in __init__\n    self._execute_child(args, executable, preexec_fn, close_fds,\n_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ \n\nself = <Popen: returncode: 255 args: ['datalad', '--version']>\nargs = ['datalad', '--version'], executable = b'datalad', preexec_fn = None\nclose_fds = True, pass_fds = (), cwd = None, env = None, startupinfo = None\ncreationflags = 0, shell = False, p2cread = -1, p2cwrite = -1, c2pread = 12\nc2pwrite = 13, errread = 14, errwrite = 15, restore_signals = True, gid = None\ngids = None, uid = None, umask = -1, start_new_session = False\n\n    def _execute_child(self, args, executable, preexec_fn, close_fds,\n                       pass_fds, cwd, env,\n                       startupinfo, creationflags, shell,\n                       p2cread, p2cwrite,\n                       c2pread, c2pwrite,\n                       errread, errwrite,\n                       restore_signals,\n                       gid, gids, uid, umask,\n                       start_new_session):\n        \"\"\"Execute program (POSIX version)\"\"\"\n    \n        if isinstance(args, (str, bytes)):\n            args = [args]\n        elif isinstance(args, os.PathLike):\n            if shell:\n                raise TypeError('path-like args is not allowed when '\n                                'shell is true')\n            args = [args]\n        else:\n            args = list(args)\n    \n        if shell:\n            # On Android the default shell is at '/system/bin/sh'.\n            unix_shell = ('/system/bin/sh' if\n                      hasattr(sys, 'getandroidapilevel') else '/bin/sh')\n            args = [unix_shell, \"-c\"] + args\n            if executable:\n                args[0] = executable\n    \n        if executable is None:\n            executable = args[0]\n    \n        sys.audit(\"subprocess.Popen\", executable, args, cwd, env)\n    \n        if (_USE_POSIX_SPAWN\n                and os.path.dirname(executable)\n                and preexec_fn is None\n                and not close_fds\n                and not pass_fds\n                and cwd is None\n                and (p2cread == -1 or p2cread > 2)\n                and (c2pwrite == -1 or c2pwrite > 2)\n                and (errwrite == -1 or errwrite > 2)\n                and not start_new_session\n                and gid is None\n                and gids is None\n                and uid is None\n                and umask < 0):\n            self._posix_spawn(args, executable, env, restore_signals,\n                              p2cread, p2cwrite,\n                              c2pread, c2pwrite,\n                              errread, errwrite)\n            return\n    \n        orig_executable = executable\n    \n        # For transferring possible exec failure from child to parent.\n        # Data format: \"exception name:hex errno:description\"\n        # Pickle is not used; it is complex and involves memory allocation.\n        errpipe_read, errpipe_write = os.pipe()\n        # errpipe_write must not be in the standard io 0, 1, or 2 fd range.\n        low_fds_to_close = []\n        while errpipe_write < 3:\n            low_fds_to_close.append(errpipe_write)\n            errpipe_write = os.dup(errpipe_write)\n        for low_fd in low_fds_to_close:\n            os.close(low_fd)\n        try:\n            try:\n                # We must avoid complex work that could involve\n                # malloc or free in the child process to avoid\n                # potential deadlocks, thus we do all this here.\n                # and pass it to fork_exec()\n    \n                if env is not None:\n                    env_list = []\n                    for k, v in env.items():\n                        k = os.fsencode(k)\n                        if b'=' in k:\n                            raise ValueError(\"illegal environment variable name\")\n                        env_list.append(k + b'=' + os.fsencode(v))\n                else:\n                    env_list = None  # Use execv instead of execve.\n                executable = os.fsencode(executable)\n                if os.path.dirname(executable):\n                    executable_list = (executable,)\n                else:\n                    # This matches the behavior of os._execvpe().\n                    executable_list = tuple(\n                        os.path.join(os.fsencode(dir), executable)\n                        for dir in os.get_exec_path(env))\n                fds_to_keep = set(pass_fds)\n                fds_to_keep.add(errpipe_write)\n                self.pid = _posixsubprocess.fork_exec(\n                        args, executable_list,\n                        close_fds, tuple(sorted(map(int, fds_to_keep))),\n                        cwd, env_list,\n                        p2cread, p2cwrite, c2pread, c2pwrite,\n                        errread, errwrite,\n                        errpipe_read, errpipe_write,\n                        restore_signals, start_new_session,\n                        gid, gids, uid, umask,\n                        preexec_fn)\n                self._child_created = True\n            finally:\n                # be sure the FD is closed no matter what\n                os.close(errpipe_write)\n    \n            self._close_pipe_fds(p2cread, p2cwrite,\n                                 c2pread, c2pwrite,\n                                 errread, errwrite)\n    \n            # Wait for exec to fail or succeed; possibly raising an\n            # exception (limited in size)\n            errpipe_data = bytearray()\n            while True:\n                part = os.read(errpipe_read, 50000)\n                errpipe_data += part\n                if not part or len(errpipe_data) > 50000:\n                    break\n        finally:\n            # be sure the FD is closed no matter what\n            os.close(errpipe_read)\n    \n        if errpipe_data:\n            try:\n                pid, sts = os.waitpid(self.pid, 0)\n                if pid == self.pid:\n                    self._handle_exitstatus(sts)\n                else:\n                    self.returncode = sys.maxsize\n            except ChildProcessError:\n                pass\n    \n            try:\n                exception_name, hex_errno, err_msg = (\n                        errpipe_data.split(b':', 2))\n                # The encoding here should match the encoding\n                # written in by the subprocess implementations\n                # like _posixsubprocess\n                err_msg = err_msg.decode()\n            except ValueError:\n                exception_name = b'SubprocessError'\n                hex_errno = b'0'\n                err_msg = 'Bad exception data from child: {!r}'.format(\n                              bytes(errpipe_data))\n            child_exception_type = getattr(\n                    builtins, exception_name.decode('ascii'),\n                    SubprocessError)\n            if issubclass(child_exception_type, OSError) and hex_errno:\n                errno_num = int(hex_errno, 16)\n                child_exec_never_called = (err_msg == \"noexec\")\n                if child_exec_never_called:\n                    err_msg = \"\"\n                    # The error must be from chdir(cwd).\n                    err_filename = cwd\n                else:\n                    err_filename = orig_executable\n                if errno_num != 0:\n                    err_msg = os.strerror(errno_num)\n>               raise child_exception_type(errno_num, err_msg, err_filename)\nE               FileNotFoundError: [Errno 2] No such file or directory: 'datalad'\n\n../../../../.local/share/uv/python/cpython-3.9.21-linux-x86_64-gnu/lib/python3.9/subprocess.py:1837: FileNotFoundError\n______________________________ test_cfg_override _______________________________\n\narg = (), kw = {'path': '/tmp/tmp-page2annexz_p4wa9n'}\nfilename = '/tmp/datalad_temp_test_cfg_overrides5ovs4s2'\n\n    @wraps(t)\n    def  _wrap_with_tempfile(*arg, **kw):\n        if 'dir' not in tkwargs.keys():\n            # if not specified otherwise, respect datalad.tests.temp.dir config\n            # as this is a test helper\n            tkwargs['dir'] = dl_cfg.get(\"datalad.tests.temp.dir\")\n        with make_tempfile(wrapped=t, **tkwargs) as filename:\n>           return t(*(arg + (filename,)), **kw)\nE           TypeError: test_cfg_override() got multiple values for argument 'path'\n\ndatalad/tests/utils.py:856: TypeError\n==================================== PASSES ====================================\n_______________________ test_usage_on_insufficient_args ________________________\n----------------------------- Captured stderr call -----------------------------\n[ERROR] InsufficientArgumentsError(Please provide at least a source or a path) (InsufficientArgumentsError) \n------------------------------ Captured log call -------------------------------\nERROR    datalad.cmdline:main.py:218 InsufficientArgumentsError(Please provide at least a source or a path) (InsufficientArgumentsError)\n______________________ test_subcmd_usage_on_unknown_args _______________________\n----------------------------- Captured stderr call -----------------------------\n[ERROR] unknown argument: --murks \n------------------------------ Captured log call -------------------------------\nERROR    datalad.cmdline:main.py:174 unknown argument: --murks\n_________________________ test_incorrect_cfg_override __________________________\n----------------------------- Captured stderr call -----------------------------\n[ERROR] Configuration override without section/variable or value assignment (must be 'section.variable=value'): ['some'] \n[ERROR] Configuration override without section/variable or value assignment (must be 'section.variable=value'): ['some='] \n[ERROR] Configuration override without section/variable or value assignment (must be 'section.variable=value'): ['some.var'] \n[ERROR] Configuration override without section/variable or value assignment (must be 'section.variable=value'): ['some.var='] \n------------------------------ Captured log call -------------------------------\nERROR    datalad.cmdline:helpers.py:632 Configuration override without section/variable or value assignment (must be 'section.variable=value'): ['some']\nERROR    datalad.cmdline:helpers.py:632 Configuration override without section/variable or value assignment (must be 'section.variable=value'): ['some=']\nERROR    datalad.cmdline:helpers.py:632 Configuration override without section/variable or value assignment (must be 'section.variable=value'): ['some.var']\nERROR    datalad.cmdline:helpers.py:632 Configuration override without section/variable or value assignment (must be 'section.variable=value'): ['some.var=']\n_____________________________ test_fix_datalad_ri ______________________________\n----------------------------- Captured stderr call -----------------------------\n[INFO] Changing // back to /// as it was probably changed by MINGW/MSYS, see http://www.mingw.org/wiki/Posix_path_conversion \n[INFO] Changing //a back to ///a as it was probably changed by MINGW/MSYS, see http://www.mingw.org/wiki/Posix_path_conversion \n[INFO] Changing //a/b back to ///a/b as it was probably changed by MINGW/MSYS, see http://www.mingw.org/wiki/Posix_path_conversion \n------------------------------ Captured log call -------------------------------\nINFO     datalad.cmdline:helpers.py:584 Changing // back to /// as it was probably changed by MINGW/MSYS, see http://www.mingw.org/wiki/Posix_path_conversion\nINFO     datalad.cmdline:helpers.py:584 Changing //a back to ///a as it was probably changed by MINGW/MSYS, see http://www.mingw.org/wiki/Posix_path_conversion\nINFO     datalad.cmdline:helpers.py:584 Changing //a/b back to ///a/b as it was probably changed by MINGW/MSYS, see http://www.mingw.org/wiki/Posix_path_conversion\n_______________________________ test_librarymode _______________________________\n----------------------------- Captured stdout call -----------------------------\nnothing to clean, no temporary locations present.\n------------------------------ Captured log call -------------------------------\nINFO     datalad.ui.dialog:log.py:333 Clear progress bars\nINFO     datalad.ui.dialog:log.py:333 Refresh progress bars\n=========================== short test summary info ============================\nPASSED r2e_tests/test_1.py::test_version\nPASSED r2e_tests/test_1.py::test_usage_on_insufficient_args\nPASSED r2e_tests/test_1.py::test_subcmd_usage_on_unknown_args\nPASSED r2e_tests/test_1.py::test_combined_short_option\nPASSED r2e_tests/test_1.py::test_incorrect_cfg_override\nPASSED r2e_tests/test_1.py::test_fail_with_short_help\nPASSED r2e_tests/test_1.py::test_fix_datalad_ri\nPASSED r2e_tests/test_1.py::test_librarymode\nSKIPPED [1] .venv/lib/python3.9/site-packages/_pytest/unittest.py:385: Could not determine terminal size, skipping the rest of the test\nXFAIL r2e_tests/test_1.py::test_incorrect_options - reason: [NOTRUN] yield tests were removed in pytest 4.0 - test_incorrect_options will be ignored\nERROR r2e_tests/test_1.py::test_commanderror_jsonmsgs\nFAILED r2e_tests/test_1.py::test_script_shims - FileNotFoundError: [Errno 2] ...\nFAILED r2e_tests/test_1.py::test_cfg_override - TypeError: test_cfg_override(...\n========== 2 failed, 8 passed, 1 skipped, 1 xfailed, 1 error in 3.92s ==========\n",
    "new_commit_res_stderr": "",
    "old_commit_res_code": 1,
    "old_commit_res_stdout": "============================= test session starts ==============================\nplatform linux -- Python 3.9.21, pytest-8.3.4, pluggy-1.5.0\nrootdir: /home/gcpuser/buckets/local_repoeval_bucket/repos/datalad_f16c5086c9872148ce561df502f187581b0a18bb\nconfigfile: pyproject.toml\nplugins: cov-6.0.0\ncollected 13 items\n\nr2e_tests/test_1.py .s...xFF...EF                                        [100%]\n\n==================================== ERRORS ====================================\n_________________ ERROR at setup of test_commanderror_jsonmsgs _________________\nfile /home/gcpuser/buckets/local_repoeval_bucket/repos/datalad_f16c5086c9872148ce561df502f187581b0a18bb/r2e_tests/test_1.py, line 368\n  @with_tempfile\n  @with_tempfile(mkdir=True)\n  def test_commanderror_jsonmsgs(src, exp):\nE       fixture 'src' not found\n>       available fixtures: cache, capfd, capfdbinary, caplog, capsys, capsysbinary, cov, doctest_namespace, monkeypatch, no_cover, path, pytestconfig, record_property, record_testsuite_property, record_xml_attribute, recwarn, setup_git_config, tmp_path, tmp_path_factory, tmpdir, tmpdir_factory\n>       use 'pytest --fixtures [testpath]' for help on them.\n\n/home/gcpuser/buckets/local_repoeval_bucket/repos/datalad_f16c5086c9872148ce561df502f187581b0a18bb/r2e_tests/test_1.py:368\n=================================== FAILURES ===================================\n______________________________ test_script_shims _______________________________\n\n    def test_script_shims():\n        runner = Runner()\n        script = 'datalad'\n        if not on_windows:\n            # those must be available for execution, and should not contain\n            which = runner.run(['which', script], protocol=StdOutErrCapture)['stdout']\n            # test if there is no easy install shim in there\n            with open(which.rstrip()) as f:\n                content = f.read()\n        else:\n            from distutils.spawn import find_executable\n            content = find_executable(script)\n    \n        # and let's check that it is our script\n>       out = runner.run([script, '--version'], protocol=StdOutErrCapture)\n\nr2e_tests/test_1.py:220: \n_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ \ndatalad/runner/runner.py:126: in run\n    results = run_command(\ndatalad/runner/nonasyncrunner.py:203: in run_command\n    process = subprocess.Popen(cmd, **kwargs)\n../../../../.local/share/uv/python/cpython-3.9.21-linux-x86_64-gnu/lib/python3.9/subprocess.py:951: in __init__\n    self._execute_child(args, executable, preexec_fn, close_fds,\n_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ \n\nself = <Popen: returncode: 255 args: ['datalad', '--version']>\nargs = ['datalad', '--version'], executable = b'datalad', preexec_fn = None\nclose_fds = True, pass_fds = (), cwd = None, env = None, startupinfo = None\ncreationflags = 0, shell = False, p2cread = -1, p2cwrite = -1, c2pread = 12\nc2pwrite = 13, errread = 14, errwrite = 15, restore_signals = True, gid = None\ngids = None, uid = None, umask = -1, start_new_session = False\n\n    def _execute_child(self, args, executable, preexec_fn, close_fds,\n                       pass_fds, cwd, env,\n                       startupinfo, creationflags, shell,\n                       p2cread, p2cwrite,\n                       c2pread, c2pwrite,\n                       errread, errwrite,\n                       restore_signals,\n                       gid, gids, uid, umask,\n                       start_new_session):\n        \"\"\"Execute program (POSIX version)\"\"\"\n    \n        if isinstance(args, (str, bytes)):\n            args = [args]\n        elif isinstance(args, os.PathLike):\n            if shell:\n                raise TypeError('path-like args is not allowed when '\n                                'shell is true')\n            args = [args]\n        else:\n            args = list(args)\n    \n        if shell:\n            # On Android the default shell is at '/system/bin/sh'.\n            unix_shell = ('/system/bin/sh' if\n                      hasattr(sys, 'getandroidapilevel') else '/bin/sh')\n            args = [unix_shell, \"-c\"] + args\n            if executable:\n                args[0] = executable\n    \n        if executable is None:\n            executable = args[0]\n    \n        sys.audit(\"subprocess.Popen\", executable, args, cwd, env)\n    \n        if (_USE_POSIX_SPAWN\n                and os.path.dirname(executable)\n                and preexec_fn is None\n                and not close_fds\n                and not pass_fds\n                and cwd is None\n                and (p2cread == -1 or p2cread > 2)\n                and (c2pwrite == -1 or c2pwrite > 2)\n                and (errwrite == -1 or errwrite > 2)\n                and not start_new_session\n                and gid is None\n                and gids is None\n                and uid is None\n                and umask < 0):\n            self._posix_spawn(args, executable, env, restore_signals,\n                              p2cread, p2cwrite,\n                              c2pread, c2pwrite,\n                              errread, errwrite)\n            return\n    \n        orig_executable = executable\n    \n        # For transferring possible exec failure from child to parent.\n        # Data format: \"exception name:hex errno:description\"\n        # Pickle is not used; it is complex and involves memory allocation.\n        errpipe_read, errpipe_write = os.pipe()\n        # errpipe_write must not be in the standard io 0, 1, or 2 fd range.\n        low_fds_to_close = []\n        while errpipe_write < 3:\n            low_fds_to_close.append(errpipe_write)\n            errpipe_write = os.dup(errpipe_write)\n        for low_fd in low_fds_to_close:\n            os.close(low_fd)\n        try:\n            try:\n                # We must avoid complex work that could involve\n                # malloc or free in the child process to avoid\n                # potential deadlocks, thus we do all this here.\n                # and pass it to fork_exec()\n    \n                if env is not None:\n                    env_list = []\n                    for k, v in env.items():\n                        k = os.fsencode(k)\n                        if b'=' in k:\n                            raise ValueError(\"illegal environment variable name\")\n                        env_list.append(k + b'=' + os.fsencode(v))\n                else:\n                    env_list = None  # Use execv instead of execve.\n                executable = os.fsencode(executable)\n                if os.path.dirname(executable):\n                    executable_list = (executable,)\n                else:\n                    # This matches the behavior of os._execvpe().\n                    executable_list = tuple(\n                        os.path.join(os.fsencode(dir), executable)\n                        for dir in os.get_exec_path(env))\n                fds_to_keep = set(pass_fds)\n                fds_to_keep.add(errpipe_write)\n                self.pid = _posixsubprocess.fork_exec(\n                        args, executable_list,\n                        close_fds, tuple(sorted(map(int, fds_to_keep))),\n                        cwd, env_list,\n                        p2cread, p2cwrite, c2pread, c2pwrite,\n                        errread, errwrite,\n                        errpipe_read, errpipe_write,\n                        restore_signals, start_new_session,\n                        gid, gids, uid, umask,\n                        preexec_fn)\n                self._child_created = True\n            finally:\n                # be sure the FD is closed no matter what\n                os.close(errpipe_write)\n    \n            self._close_pipe_fds(p2cread, p2cwrite,\n                                 c2pread, c2pwrite,\n                                 errread, errwrite)\n    \n            # Wait for exec to fail or succeed; possibly raising an\n            # exception (limited in size)\n            errpipe_data = bytearray()\n            while True:\n                part = os.read(errpipe_read, 50000)\n                errpipe_data += part\n                if not part or len(errpipe_data) > 50000:\n                    break\n        finally:\n            # be sure the FD is closed no matter what\n            os.close(errpipe_read)\n    \n        if errpipe_data:\n            try:\n                pid, sts = os.waitpid(self.pid, 0)\n                if pid == self.pid:\n                    self._handle_exitstatus(sts)\n                else:\n                    self.returncode = sys.maxsize\n            except ChildProcessError:\n                pass\n    \n            try:\n                exception_name, hex_errno, err_msg = (\n                        errpipe_data.split(b':', 2))\n                # The encoding here should match the encoding\n                # written in by the subprocess implementations\n                # like _posixsubprocess\n                err_msg = err_msg.decode()\n            except ValueError:\n                exception_name = b'SubprocessError'\n                hex_errno = b'0'\n                err_msg = 'Bad exception data from child: {!r}'.format(\n                              bytes(errpipe_data))\n            child_exception_type = getattr(\n                    builtins, exception_name.decode('ascii'),\n                    SubprocessError)\n            if issubclass(child_exception_type, OSError) and hex_errno:\n                errno_num = int(hex_errno, 16)\n                child_exec_never_called = (err_msg == \"noexec\")\n                if child_exec_never_called:\n                    err_msg = \"\"\n                    # The error must be from chdir(cwd).\n                    err_filename = cwd\n                else:\n                    err_filename = orig_executable\n                if errno_num != 0:\n                    err_msg = os.strerror(errno_num)\n>               raise child_exception_type(errno_num, err_msg, err_filename)\nE               FileNotFoundError: [Errno 2] No such file or directory: 'datalad'\n\n../../../../.local/share/uv/python/cpython-3.9.21-linux-x86_64-gnu/lib/python3.9/subprocess.py:1837: FileNotFoundError\n______________________________ test_cfg_override _______________________________\n\narg = (), kw = {'path': '/tmp/tmp-page2annexluxc42l9'}\nfilename = '/tmp/datalad_temp_test_cfg_override2s0u7jdk'\n\n    @wraps(t)\n    def  _wrap_with_tempfile(*arg, **kw):\n        if 'dir' not in tkwargs.keys():\n            # if not specified otherwise, respect datalad.tests.temp.dir config\n            # as this is a test helper\n            tkwargs['dir'] = dl_cfg.get(\"datalad.tests.temp.dir\")\n        with make_tempfile(wrapped=t, **tkwargs) as filename:\n>           return t(*(arg + (filename,)), **kw)\nE           TypeError: test_cfg_override() got multiple values for argument 'path'\n\ndatalad/tests/utils.py:856: TypeError\n_______________________________ test_librarymode _______________________________\n\n    def test_librarymode():\n        was_mode = datalad.__runtime_mode\n        try:\n            # clean --dry-run is just a no-op command that is cheap\n            # to execute. It has no particular role here, other than\n            # to make the code pass the location where library mode\n            # should be turned on via the cmdline API\n            run_main(['-c', 'datalad.runtime.librarymode=yes', 'clean', '--dry-run'])\n>           ok_(datalad.in_librarymode())\n\nr2e_tests/test_1.py:400: \n_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ \n\nexpr = False, msg = None\n\n    def ok_(expr, msg=None):\n        \"\"\"Shorthand for assert. Saves 3 whole characters!\n        \"\"\"\n        if not expr:\n>           raise AssertionError(msg)\nE           AssertionError: None\n\n.venv/lib/python3.9/site-packages/nose/tools/trivial.py:22: AssertionError\n----------------------------- Captured stdout call -----------------------------\nnothing to clean, no temporary locations present.\n------------------------------ Captured log call -------------------------------\nINFO     datalad.ui.dialog:log.py:333 Clear progress bars\nINFO     datalad.ui.dialog:log.py:333 Refresh progress bars\n==================================== PASSES ====================================\n_______________________ test_usage_on_insufficient_args ________________________\n----------------------------- Captured stderr call -----------------------------\n[ERROR] InsufficientArgumentsError(Please provide at least a source or a path) (InsufficientArgumentsError) \n------------------------------ Captured log call -------------------------------\nERROR    datalad.cmdline:main.py:218 InsufficientArgumentsError(Please provide at least a source or a path) (InsufficientArgumentsError)\n______________________ test_subcmd_usage_on_unknown_args _______________________\n----------------------------- Captured stderr call -----------------------------\n[ERROR] unknown argument: --murks \n------------------------------ Captured log call -------------------------------\nERROR    datalad.cmdline:main.py:176 unknown argument: --murks\n_________________________ test_incorrect_cfg_override __________________________\n----------------------------- Captured stderr call -----------------------------\n[ERROR] Configuration override without section/variable or value assignment (must be 'section.variable=value'): ['some'] \n[ERROR] Configuration override without section/variable or value assignment (must be 'section.variable=value'): ['some='] \n[ERROR] Configuration override without section/variable or value assignment (must be 'section.variable=value'): ['some.var'] \n[ERROR] Configuration override without section/variable or value assignment (must be 'section.variable=value'): ['some.var='] \n------------------------------ Captured log call -------------------------------\nERROR    datalad.cmdline:helpers.py:638 Configuration override without section/variable or value assignment (must be 'section.variable=value'): ['some']\nERROR    datalad.cmdline:helpers.py:638 Configuration override without section/variable or value assignment (must be 'section.variable=value'): ['some=']\nERROR    datalad.cmdline:helpers.py:638 Configuration override without section/variable or value assignment (must be 'section.variable=value'): ['some.var']\nERROR    datalad.cmdline:helpers.py:638 Configuration override without section/variable or value assignment (must be 'section.variable=value'): ['some.var=']\n_____________________________ test_fix_datalad_ri ______________________________\n----------------------------- Captured stderr call -----------------------------\n[INFO] Changing // back to /// as it was probably changed by MINGW/MSYS, see http://www.mingw.org/wiki/Posix_path_conversion \n[INFO] Changing //a back to ///a as it was probably changed by MINGW/MSYS, see http://www.mingw.org/wiki/Posix_path_conversion \n[INFO] Changing //a/b back to ///a/b as it was probably changed by MINGW/MSYS, see http://www.mingw.org/wiki/Posix_path_conversion \n------------------------------ Captured log call -------------------------------\nINFO     datalad.cmdline:helpers.py:590 Changing // back to /// as it was probably changed by MINGW/MSYS, see http://www.mingw.org/wiki/Posix_path_conversion\nINFO     datalad.cmdline:helpers.py:590 Changing //a back to ///a as it was probably changed by MINGW/MSYS, see http://www.mingw.org/wiki/Posix_path_conversion\nINFO     datalad.cmdline:helpers.py:590 Changing //a/b back to ///a/b as it was probably changed by MINGW/MSYS, see http://www.mingw.org/wiki/Posix_path_conversion\n=========================== short test summary info ============================\nPASSED r2e_tests/test_1.py::test_version\nPASSED r2e_tests/test_1.py::test_usage_on_insufficient_args\nPASSED r2e_tests/test_1.py::test_subcmd_usage_on_unknown_args\nPASSED r2e_tests/test_1.py::test_combined_short_option\nPASSED r2e_tests/test_1.py::test_incorrect_cfg_override\nPASSED r2e_tests/test_1.py::test_fail_with_short_help\nPASSED r2e_tests/test_1.py::test_fix_datalad_ri\nSKIPPED [1] .venv/lib/python3.9/site-packages/_pytest/unittest.py:385: Could not determine terminal size, skipping the rest of the test\nXFAIL r2e_tests/test_1.py::test_incorrect_options - reason: [NOTRUN] yield tests were removed in pytest 4.0 - test_incorrect_options will be ignored\nERROR r2e_tests/test_1.py::test_commanderror_jsonmsgs\nFAILED r2e_tests/test_1.py::test_script_shims - FileNotFoundError: [Errno 2] ...\nFAILED r2e_tests/test_1.py::test_cfg_override - TypeError: test_cfg_override(...\nFAILED r2e_tests/test_1.py::test_librarymode - AssertionError: None\n========== 3 failed, 7 passed, 1 skipped, 1 xfailed, 1 error in 1.56s ==========\n",
    "old_commit_res_stderr": ""
}
datalad/__init__.py datalad/cmdline/helpers.py datalad/cmdline/main.py datalad/cmdline/tests/test_main.py docs/source/design/application_vs_library_mode.rst
We are currently solving the following issue within our repository. Here is the issue text:
--- BEGIN ISSUE ---
[ISSUE]

**Title:** `--library-mode` Command-Line Switch Fails to Activate Library Mode in Child Processes

**Description:**
When using the `--library-mode` command-line switch to enable library mode in DataLad, the setting does not propagate to child processes. As a result, functions that rely on library mode incorrectly behave as if it is not enabled. This issue causes tests that depend on library mode to fail unexpectedly.

**Example Code:**
```python
def test_librarymode():
    try:
        run_main(['--library-mode', 'clean', '--dry-run'])
        if not datalad.in_librarymode():
            print("Library mode is not enabled.")
    finally:
        # Restore original runtime mode
        datalad.__runtime_mode = was_mode
```

**Expected Behavior:**
Running DataLad with the `--library-mode` switch should activate library mode for both the main process and all child processes. This ensures that `datalad.in_librarymode()` returns `True`, allowing library-specific functionalities to operate correctly.

**Actual Behavior:**
Despite using the `--library-mode` switch, `datalad.in_librarymode()` returns `False`. This indicates that library mode is not activated in child processes, leading to failed tests and unexpected behavior.

[/ISSUE]
--- END ISSUE ---

Below are some code segments, each from a relevant file. One or more of these files may contain bugs.

--- BEGIN FILE ---
```
### datalad/cmdline/main.py
# emacs: -*- mode: python; py-indent-offset: 4; tab-width: 4; indent-tabs-mode: nil -*-
# ex: set sts=4 ts=4 sw=4 et:
# ## ### ### ### ### ### ### ### ### ### ### ### ### ### ### ### ### ### ### ##
#
#   See COPYING file distributed along with the datalad package for the
#   copyright and license terms.
#
# ## ### ### ### ### ### ### ### ### ### ### ### ### ### ### ### ### ### ### ##
""""""

__docformat__ = 'restructuredtext'

import logging
lgr = logging.getLogger('datalad.cmdline')

lgr.log(5, "Importing cmdline.main")

import argparse
from collections import defaultdict
import sys
import os

import datalad

from ..support.exceptions import (
    CapturedException,
    InsufficientArgumentsError,
    IncompleteResultsError,
    CommandError,
)
from ..utils import (
    chpwd,
    on_msys_tainted_paths,
    setup_exceptionhook,
)
from .helpers import (
    ArgumentParserDisableAbbrev,
    _fix_datalad_ri,
    _maybe_get_interface_subparser,
    _maybe_get_single_subparser,
    _parse_overrides_from_cmdline,
    get_description_with_cmd_summary,
    parser_add_common_options,
    parser_add_common_opt,
    strip_arg_from_argv,
)


# TODO:  OPT look into making setup_parser smarter to become faster
# Now it seems to take up to 200ms to do all the parser setup
# even though it might not be necessary to know about all the commands etc.
# I wondered if it could somehow decide on what commands to worry about etc
# by going through sys.args first
def setup_parser(
        cmdlineargs,
        formatter_class=argparse.RawDescriptionHelpFormatter,
        return_subparsers=False,
        # Was this triggered by argparse?
        completing=False,
        # prevent loading of extension entrypoints when --help is requested
        # this is enabled when building docs to avoid pollution of generated
        # manpages with extensions commands (that should appear in their own
        # docs, but not in the core datalad package docs)
        help_ignore_extensions=False):
    """
    The holy grail of establishing CLI for DataLad's Interfaces

    Parameters
    ----------
    cmdlineargs:
    formatter_class:
    return_subparsers: bool, optional
      is used ATM only by BuildManPage in _datalad_build_support
    completing:
    help_ignore_extensions:
    """
    lgr.log(5, "Starting to setup_parser")
    # delay since it can be a heavy import
    from ..interface.base import (
        dedent_docstring,
        get_cmdline_command_name,
        get_interface_groups,
    )

    # setup cmdline args parser

    # "main" parser is under "datalad" name
    all_parsers = {}  # name: (sub)parser

    # main parser
    parser = ArgumentParserDisableAbbrev(
        fromfile_prefix_chars=None,
        # usage="%(prog)s ...",
        description=dedent_docstring("""\
            Comprehensive data management solution

            DataLad provides a unified data distribution system built on the Git
            and Git-annex. DataLad command line tools allow to manipulate (obtain,
            create, update, publish, etc.) datasets and provide a comprehensive
            toolbox for joint management of data and code. Compared to Git/annex
            it primarily extends their functionality to transparently and
            simultaneously work with multiple inter-related repositories."""),
        epilog='"Be happy!"',
        formatter_class=formatter_class,
        add_help=False)

    # common options
    parser_add_common_options(parser)

    interface_groups = get_interface_groups()

    single_subparser = _maybe_get_single_subparser(
        cmdlineargs, parser, interface_groups,
        return_subparsers, completing, help_ignore_extensions
    )

    # --help specification was delayed since it causes immediate printout of
    # --help output before we setup --help for each command
    parser_add_common_opt(parser, 'help')

    grp_short_descriptions = defaultdict(list)
    # create subparser, use module suffix as cmd name
    subparsers = parser.add_subparsers()
    for group_name, _, _interfaces \
            in sorted(interface_groups, key=lambda x: x[1]):
        for _intfspec in _interfaces:
            cmd_name = get_cmdline_command_name(_intfspec)
            if single_subparser and cmd_name != single_subparser:
                continue
            subparser = _maybe_get_interface_subparser(
                _intfspec, subparsers, cmd_name, formatter_class, group_name,
                grp_short_descriptions
            )
            if subparser:  # interface might have failed to "load"
                all_parsers[cmd_name] = subparser

    # create command summary
    if '--help' in cmdlineargs or '--help-np' in cmdlineargs:
        parser.description = get_description_with_cmd_summary(
            grp_short_descriptions,
            interface_groups,
            parser.description)

    all_parsers['datalad'] = parser
    lgr.log(5, "Finished setup_parser")
    if return_subparsers:
        return all_parsers
    else:
        return parser


def main(args=None):
    lgr.log(5, "Starting main(%r)", args)
    # record that we came in via the cmdline
    datalad.__api = 'cmdline'
    args = args or sys.argv
    if on_msys_tainted_paths:
        # Possibly present DataLadRIs were stripped of a leading /
        args = [_fix_datalad_ri(s) for s in args]
    # PYTHON_ARGCOMPLETE_OK
    parser = setup_parser(args, completing="_ARGCOMPLETE" in os.environ)
    try:
        import argcomplete
        argcomplete.autocomplete(parser)
    except ImportError:
        pass

    # parse cmd args
    lgr.debug("Parsing known args among %s", repr(args))
    cmdlineargs, unparsed_args = parser.parse_known_args(args[1:])
    if cmdlineargs.common_library_mode:
        datalad.enable_librarymode()
    has_func = hasattr(cmdlineargs, 'func') and cmdlineargs.func is not None
    if unparsed_args:
        if has_func and cmdlineargs.func.__self__.__name__ != 'Export':
            lgr.error('unknown argument{}: {}'.format(
                's' if len(unparsed_args) > 1 else '',
                unparsed_args if len(unparsed_args) > 1 else unparsed_args[0]))
            cmdlineargs.subparser.print_usage()
            sys.exit(1)
        else:
            # store all unparsed arguments
            cmdlineargs.datalad_unparsed_args = unparsed_args

    # to possibly be passed into PBS scheduled call
    args_ = args or sys.argv

    if cmdlineargs.cfg_overrides is not None:
        datalad.cfg.overrides.update(
            _parse_overrides_from_cmdline(cmdlineargs)
        )

    # enable overrides
    datalad.cfg.reload(force=True)

    if cmdlineargs.change_path is not None:
        from .common_args import change_path as change_path_opt
        for path in cmdlineargs.change_path:
            chpwd(path)
            args_ = strip_arg_from_argv(args_, path, change_path_opt[1])

    ret = None
    if has_func:
        if cmdlineargs.common_debug or cmdlineargs.common_idebug:
            # so we could see/stop clearly at the point of failure
            setup_exceptionhook(ipython=cmdlineargs.common_idebug)
            from datalad.interface.base import Interface
            Interface._interrupted_exit_code = None
            ret = cmdlineargs.func(cmdlineargs)
        else:
            # otherwise - guard and only log the summary. Postmortem is not
            # as convenient if being caught in this ultimate except
            try:
                ret = cmdlineargs.func(cmdlineargs)
            except InsufficientArgumentsError as exc:
                ce = CapturedException(exc)
                # if the func reports inappropriate usage, give help output
                lgr.error('%s (%s)', ce, exc.__class__.__name__)
                cmdlineargs.subparser.print_usage(sys.stderr)
                sys.exit(2)
            except IncompleteResultsError as exc:
                ce = CapturedException(exc)
                # rendering for almost all commands now happens 'online'
                # hence we are no longer attempting to render the actual
                # results in an IncompleteResultsError, ubt rather trust that
                # this happened before

                # in general we do not want to see the error again, but
                # present in debug output
                lgr.debug('could not perform all requested actions: %s', ce)
                sys.exit(1)
            except CommandError as exc:
                ce = CapturedException(exc)
                # behave as if the command ran directly, importantly pass
                # exit code as is
                # to not duplicate any captured output in the exception
                # rendering, will come next
                exc_msg = exc.to_str(include_output=False)
                if exc_msg:
                    msg = exc_msg.encode() if isinstance(exc_msg, str) else exc_msg
                    os.write(2, msg + b"\n")
                if exc.stdout:
                    os.write(1, exc.stdout.encode() if isinstance(exc.stdout, str) else exc.stdout)
                if exc.stderr:
                    os.write(2, exc.stderr.encode() if isinstance(exc.stderr, str) else exc.stderr)
                # We must not exit with 0 code if any exception got here but
                # had no code defined
                sys.exit(exc.code if exc.code is not None else 1)
            except Exception as exc:
                ce = CapturedException(exc)
                lgr.error('%s (%s)', ce, exc.__class__.__name__)
                sys.exit(1)
    else:
        # just let argparser spit out its error, since there is smth wrong
        parser.parse_args(args)
        # if that one didn't puke -- we should
        parser.print_usage()
        lgr.error("Please specify the command")
        sys.exit(2)

    try:
        if hasattr(cmdlineargs, 'result_renderer'):
            cmdlineargs.result_renderer(ret, cmdlineargs)
    except Exception as exc:
        ce = CapturedException(exc)
        lgr.error("Failed to render results due to %s", ce)
        sys.exit(1)
    # all good, not strictly needed, but makes internal testing easier
    sys.exit(0)


lgr.log(5, "Done importing cmdline.main")

```
--- END FILE ---

Please first localize the bug based on the issue statement, and then generate *SEARCH/REPLACE* edits to fix the issue.

Every *SEARCH/REPLACE* edit must use this format:
1. The file path
2. The start of search block: <<<<<<< SEARCH
3. A contiguous chunk of lines to search for in the existing source code
4. The dividing line: =======
5. The lines to replace into the source code
6. The end of the replace block: >>>>>>> REPLACE

Here is an example:

```python
### mathweb/flask/app.py
<<<<<<< SEARCH
from flask import Flask
=======
import math
from flask import Flask
>>>>>>> REPLACE
```
</solution>

Please note that the *SEARCH/REPLACE* edit REQUIRES PROPER INDENTATION. If you would like to add the line '        print(x)', you must fully write that out, with all those spaces before the code!
Wrap each *SEARCH/REPLACE* edit in a code block as shown in the example above. If you have multiple *SEARCH/REPLACE* edits, use a separate code block for each one.
[ISSUE]

**Title:** `--library-mode` Command-Line Switch Fails to Activate Library Mode in Child Processes

**Description:**
When using the `--library-mode` command-line switch to enable library mode in DataLad, the setting does not propagate to child processes. As a result, functions that rely on library mode incorrectly behave as if it is not enabled. This issue causes tests that depend on library mode to fail unexpectedly.

**Example Code:**
```python
def test_librarymode():
    try:
        run_main(['--library-mode', 'clean', '--dry-run'])
        if not datalad.in_librarymode():
            print("Library mode is not enabled.")
    finally:
        # Restore original runtime mode
        datalad.__runtime_mode = was_mode
```

**Expected Behavior:**
Running DataLad with the `--library-mode` switch should activate library mode for both the main process and all child processes. This ensures that `datalad.in_librarymode()` returns `True`, allowing library-specific functionalities to operate correctly.

**Actual Behavior:**
Despite using the `--library-mode` switch, `datalad.in_librarymode()` returns `False`. This indicates that library mode is not activated in child processes, leading to failed tests and unexpected behavior.

[/ISSUE]
{
    "test_version": "PASSED",
    "test_usage_on_insufficient_args": "PASSED",
    "test_subcmd_usage_on_unknown_args": "PASSED",
    "test_combined_short_option": "PASSED",
    "test_script_shims": "PASSED",
    "test_incorrect_cfg_override": "PASSED",
    "test_fail_with_short_help": "PASSED",
    "test_fix_datalad_ri": "PASSED",
    "test_librarymode": "PASSED",
    "test_commanderror_jsonmsgs": "ERROR",
    "test_cfg_override": "FAILED"
}
diff --git a/datalad/__init__.py b/datalad/__init__.py
index fb96d539e..06be087fa 100644
--- a/datalad/__init__.py
+++ b/datalad/__init__.py
@@ -25,10 +25,6 @@ import os
 
 # this is not to be modified. for querying use get_apimode()
 __api = 'python'
-# this is not to be modified. see enable/in_librarymode()
-# by default, we are in application-mode, simply because most of
-# datalad was originally implemented with this scenario assumption
-__runtime_mode = 'application'
 
 
 def get_apimode():
@@ -83,6 +79,8 @@ def enable_librarymode():
     """
     global __runtime_mode
     __runtime_mode = 'library'
+    # export into the environment for child processes to inherit
+    os.environ['DATALAD_RUNTIME_LIBRARYMODE'] = '1'
 
 
 # For reproducible demos/tests
@@ -112,6 +110,13 @@ from .config import ConfigManager
 cfg = ConfigManager()
 
 # must come after config manager
+# this is not to be modified. see enable/in_librarymode()
+# by default, we are in application-mode, simply because most of
+# datalad was originally implemented with this scenario assumption
+__runtime_mode = 'library' \
+    if cfg.getbool('datalad.runtime', 'librarymode', False) \
+    else 'application'
+
 from .log import lgr
 from datalad.support.exceptions import CapturedException
 from datalad.utils import (
diff --git a/datalad/cmdline/helpers.py b/datalad/cmdline/helpers.py
index 18e10a7fd..4b83d0f61 100644
--- a/datalad/cmdline/helpers.py
+++ b/datalad/cmdline/helpers.py
@@ -275,12 +275,6 @@ def parser_add_common_options(parser, version=None):
         non-zero exit code; 'stop' halts on first failure and yields non-zero exit
         code. A failure is any result with status 'impossible' or 'error'.
         [Default: '%(default)s']""")
-    parser.add_argument(
-        '--library-mode', dest='common_library_mode', action='store_true',
-        help="""Enable operation as a utility library (rather than a front-end
-        application). This may improve performance, because convenience
-        functionality will be disabled, and will alter the error reporting to
-        be better suited for internal use.""")
     parser.add_argument(
         '--cmd', dest='_', action='store_true',
         help="""syntactical helper that can be used to end the list of global
diff --git a/datalad/cmdline/main.py b/datalad/cmdline/main.py
index d05c94686..4e5abcd06 100644
--- a/datalad/cmdline/main.py
+++ b/datalad/cmdline/main.py
@@ -168,8 +168,6 @@ def main(args=None):
     # parse cmd args
     lgr.debug("Parsing known args among %s", repr(args))
     cmdlineargs, unparsed_args = parser.parse_known_args(args[1:])
-    if cmdlineargs.common_library_mode:
-        datalad.enable_librarymode()
     has_func = hasattr(cmdlineargs, 'func') and cmdlineargs.func is not None
     if unparsed_args:
         if has_func and cmdlineargs.func.__self__.__name__ != 'Export':
@@ -192,6 +190,8 @@ def main(args=None):
 
     # enable overrides
     datalad.cfg.reload(force=True)
+    if 'datalad.runtime.librarymode' in datalad.cfg:
+        datalad.enable_librarymode()
 
     if cmdlineargs.change_path is not None:
         from .common_args import change_path as change_path_opt

{"datalad/cmdline/main.py": "# emacs: -*- mode: python; py-indent-offset: 4; tab-width: 4; indent-tabs-mode: nil -*-\n# ex: set sts=4 ts=4 sw=4 et:\n# ## ### ### ### ### ### ### ### ### ### ### ### ### ### ### ### ### ### ### ##\n#\n#   See COPYING file distributed along with the datalad package for the\n#   copyright and license terms.\n#\n# ## ### ### ### ### ### ### ### ### ### ### ### ### ### ### ### ### ### ### ##\n\"\"\"\"\"\"\n\n__docformat__ = 'restructuredtext'\n\nimport logging\nlgr = logging.getLogger('datalad.cmdline')\n\nlgr.log(5, \"Importing cmdline.main\")\n\nimport argparse\nfrom collections import defaultdict\nimport sys\nimport os\n\nimport datalad\n\nfrom ..support.exceptions import (\n    CapturedException,\n    InsufficientArgumentsError,\n    IncompleteResultsError,\n    CommandError,\n)\nfrom ..utils import (\n    chpwd,\n    on_msys_tainted_paths,\n    setup_exceptionhook,\n)\nfrom .helpers import (\n    ArgumentParserDisableAbbrev,\n    _fix_datalad_ri,\n    _maybe_get_interface_subparser,\n    _maybe_get_single_subparser,\n    _parse_overrides_from_cmdline,\n    get_description_with_cmd_summary,\n    parser_add_common_options,\n    parser_add_common_opt,\n    strip_arg_from_argv,\n)\n\n\n# TODO:  OPT look into making setup_parser smarter to become faster\n# Now it seems to take up to 200ms to do all the parser setup\n# even though it might not be necessary to know about all the commands etc.\n# I wondered if it could somehow decide on what commands to worry about etc\n# by going through sys.args first\ndef setup_parser(\n        cmdlineargs,\n        formatter_class=argparse.RawDescriptionHelpFormatter,\n        return_subparsers=False,\n        # Was this triggered by argparse?\n        completing=False,\n        # prevent loading of extension entrypoints when --help is requested\n        # this is enabled when building docs to avoid pollution of generated\n        # manpages with extensions commands (that should appear in their own\n        # docs, but not in the core datalad package docs)\n        help_ignore_extensions=False):\n    \"\"\"\n    The holy grail of establishing CLI for DataLad's Interfaces\n\n    Parameters\n    ----------\n    cmdlineargs:\n    formatter_class:\n    return_subparsers: bool, optional\n      is used ATM only by BuildManPage in _datalad_build_support\n    completing:\n    help_ignore_extensions:\n    \"\"\"\n    lgr.log(5, \"Starting to setup_parser\")\n    # delay since it can be a heavy import\n    from ..interface.base import (\n        dedent_docstring,\n        get_cmdline_command_name,\n        get_interface_groups,\n    )\n\n    # setup cmdline args parser\n\n    # \"main\" parser is under \"datalad\" name\n    all_parsers = {}  # name: (sub)parser\n\n    # main parser\n    parser = ArgumentParserDisableAbbrev(\n        fromfile_prefix_chars=None,\n        # usage=\"%(prog)s ...\",\n        description=dedent_docstring(\"\"\"\\\n            Comprehensive data management solution\n\n            DataLad provides a unified data distribution system built on the Git\n            and Git-annex. DataLad command line tools allow to manipulate (obtain,\n            create, update, publish, etc.) datasets and provide a comprehensive\n            toolbox for joint management of data and code. Compared to Git/annex\n            it primarily extends their functionality to transparently and\n            simultaneously work with multiple inter-related repositories.\"\"\"),\n        epilog='\"Be happy!\"',\n        formatter_class=formatter_class,\n        add_help=False)\n\n    # common options\n    parser_add_common_options(parser)\n\n    interface_groups = get_interface_groups()\n\n    single_subparser = _maybe_get_single_subparser(\n        cmdlineargs, parser, interface_groups,\n        return_subparsers, completing, help_ignore_extensions\n    )\n\n    # --help specification was delayed since it causes immediate printout of\n    # --help output before we setup --help for each command\n    parser_add_common_opt(parser, 'help')\n\n    grp_short_descriptions = defaultdict(list)\n    # create subparser, use module suffix as cmd name\n    subparsers = parser.add_subparsers()\n    for group_name, _, _interfaces \\\n            in sorted(interface_groups, key=lambda x: x[1]):\n        for _intfspec in _interfaces:\n            cmd_name = get_cmdline_command_name(_intfspec)\n            if single_subparser and cmd_name != single_subparser:\n                continue\n            subparser = _maybe_get_interface_subparser(\n                _intfspec, subparsers, cmd_name, formatter_class, group_name,\n                grp_short_descriptions\n            )\n            if subparser:  # interface might have failed to \"load\"\n                all_parsers[cmd_name] = subparser\n\n    # create command summary\n    if '--help' in cmdlineargs or '--help-np' in cmdlineargs:\n        parser.description = get_description_with_cmd_summary(\n            grp_short_descriptions,\n            interface_groups,\n            parser.description)\n\n    all_parsers['datalad'] = parser\n    lgr.log(5, \"Finished setup_parser\")\n    if return_subparsers:\n        return all_parsers\n    else:\n        return parser\n\n\ndef main(args=None):\n    lgr.log(5, \"Starting main(%r)\", args)\n    # record that we came in via the cmdline\n    datalad.__api = 'cmdline'\n    args = args or sys.argv\n    if on_msys_tainted_paths:\n        # Possibly present DataLadRIs were stripped of a leading /\n        args = [_fix_datalad_ri(s) for s in args]\n    # PYTHON_ARGCOMPLETE_OK\n    parser = setup_parser(args, completing=\"_ARGCOMPLETE\" in os.environ)\n    try:\n        import argcomplete\n        argcomplete.autocomplete(parser)\n    except ImportError:\n        pass\n\n    # parse cmd args\n    lgr.debug(\"Parsing known args among %s\", repr(args))\n    cmdlineargs, unparsed_args = parser.parse_known_args(args[1:])\n    if cmdlineargs.common_library_mode:\n        datalad.enable_librarymode()\n    has_func = hasattr(cmdlineargs, 'func') and cmdlineargs.func is not None\n    if unparsed_args:\n        if has_func and cmdlineargs.func.__self__.__name__ != 'Export':\n            lgr.error('unknown argument{}: {}'.format(\n                's' if len(unparsed_args) > 1 else '',\n                unparsed_args if len(unparsed_args) > 1 else unparsed_args[0]))\n            cmdlineargs.subparser.print_usage()\n            sys.exit(1)\n        else:\n            # store all unparsed arguments\n            cmdlineargs.datalad_unparsed_args = unparsed_args\n\n    # to possibly be passed into PBS scheduled call\n    args_ = args or sys.argv\n\n    if cmdlineargs.cfg_overrides is not None:\n        datalad.cfg.overrides.update(\n            _parse_overrides_from_cmdline(cmdlineargs)\n        )\n\n    # enable overrides\n    datalad.cfg.reload(force=True)\n\n    if cmdlineargs.change_path is not None:\n        from .common_args import change_path as change_path_opt\n        for path in cmdlineargs.change_path:\n            chpwd(path)\n            args_ = strip_arg_from_argv(args_, path, change_path_opt[1])\n\n    ret = None\n    if has_func:\n        if cmdlineargs.common_debug or cmdlineargs.common_idebug:\n            # so we could see/stop clearly at the point of failure\n            setup_exceptionhook(ipython=cmdlineargs.common_idebug)\n            from datalad.interface.base import Interface\n            Interface._interrupted_exit_code = None\n            ret = cmdlineargs.func(cmdlineargs)\n        else:\n            # otherwise - guard and only log the summary. Postmortem is not\n            # as convenient if being caught in this ultimate except\n            try:\n                ret = cmdlineargs.func(cmdlineargs)\n            except InsufficientArgumentsError as exc:\n                ce = CapturedException(exc)\n                # if the func reports inappropriate usage, give help output\n                lgr.error('%s (%s)', ce, exc.__class__.__name__)\n                cmdlineargs.subparser.print_usage(sys.stderr)\n                sys.exit(2)\n            except IncompleteResultsError as exc:\n                ce = CapturedException(exc)\n                # rendering for almost all commands now happens 'online'\n                # hence we are no longer attempting to render the actual\n                # results in an IncompleteResultsError, ubt rather trust that\n                # this happened before\n\n                # in general we do not want to see the error again, but\n                # present in debug output\n                lgr.debug('could not perform all requested actions: %s', ce)\n                sys.exit(1)\n            except CommandError as exc:\n                ce = CapturedException(exc)\n                # behave as if the command ran directly, importantly pass\n                # exit code as is\n                # to not duplicate any captured output in the exception\n                # rendering, will come next\n                exc_msg = exc.to_str(include_output=False)\n                if exc_msg:\n                    msg = exc_msg.encode() if isinstance(exc_msg, str) else exc_msg\n                    os.write(2, msg + b\"\\n\")\n                if exc.stdout:\n                    os.write(1, exc.stdout.encode() if isinstance(exc.stdout, str) else exc.stdout)\n                if exc.stderr:\n                    os.write(2, exc.stderr.encode() if isinstance(exc.stderr, str) else exc.stderr)\n                # We must not exit with 0 code if any exception got here but\n                # had no code defined\n                sys.exit(exc.code if exc.code is not None else 1)\n            except Exception as exc:\n                ce = CapturedException(exc)\n                lgr.error('%s (%s)', ce, exc.__class__.__name__)\n                sys.exit(1)\n    else:\n        # just let argparser spit out its error, since there is smth wrong\n        parser.parse_args(args)\n        # if that one didn't puke -- we should\n        parser.print_usage()\n        lgr.error(\"Please specify the command\")\n        sys.exit(2)\n\n    try:\n        if hasattr(cmdlineargs, 'result_renderer'):\n            cmdlineargs.result_renderer(ret, cmdlineargs)\n    except Exception as exc:\n        ce = CapturedException(exc)\n        lgr.error(\"Failed to render results due to %s\", ce)\n        sys.exit(1)\n    # all good, not strictly needed, but makes internal testing easier\n    sys.exit(0)\n\n\nlgr.log(5, \"Done importing cmdline.main\")\n"}
A user will ask you to solve a task. You should first draft your thinking process (inner monologue). Then, generate the solution.

Your response format must follow the template below:
<think>
Your thoughts or/and draft, like working through an exercise on scratch paper. Be as casual and as long as you want until you are confident to generate a correct solution.
</think>
<solution>
Final solution presented to the user.
</solution>
system
We are currently solving the following issue within our repository. Here is the issue text:
--- BEGIN ISSUE ---
[ISSUE]

**Title:** `--library-mode` Command-Line Switch Fails to Activate Library Mode in Child Processes

**Description:**
When using the `--library-mode` command-line switch to enable library mode in DataLad, the setting does not propagate to child processes. As a result, functions that rely on library mode incorrectly behave as if it is not enabled. This issue causes tests that depend on library mode to fail unexpectedly.

**Example Code:**
```python
def test_librarymode():
    try:
        run_main(['--library-mode', 'clean', '--dry-run'])
        if not datalad.in_librarymode():
            print("Library mode is not enabled.")
    finally:
        # Restore original runtime mode
        datalad.__runtime_mode = was_mode
```

**Expected Behavior:**
Running DataLad with the `--library-mode` switch should activate library mode for both the main process and all child processes. This ensures that `datalad.in_librarymode()` returns `True`, allowing library-specific functionalities to operate correctly.

**Actual Behavior:**
Despite using the `--library-mode` switch, `datalad.in_librarymode()` returns `False`. This indicates that library mode is not activated in child processes, leading to failed tests and unexpected behavior.

[/ISSUE]
--- END ISSUE ---

Below are some code segments, each from a relevant file. One or more of these files may contain bugs.

--- BEGIN FILE ---
```
### datalad/cmdline/main.py
# emacs: -*- mode: python; py-indent-offset: 4; tab-width: 4; indent-tabs-mode: nil -*-
# ex: set sts=4 ts=4 sw=4 et:
# ## ### ### ### ### ### ### ### ### ### ### ### ### ### ### ### ### ### ### ##
#
#   See COPYING file distributed along with the datalad package for the
#   copyright and license terms.
#
# ## ### ### ### ### ### ### ### ### ### ### ### ### ### ### ### ### ### ### ##
""""""

__docformat__ = 'restructuredtext'

import logging
lgr = logging.getLogger('datalad.cmdline')

lgr.log(5, "Importing cmdline.main")

import argparse
from collections import defaultdict
import sys
import os

import datalad

from ..support.exceptions import (
    CapturedException,
    InsufficientArgumentsError,
    IncompleteResultsError,
    CommandError,
)
from ..utils import (
    chpwd,
    on_msys_tainted_paths,
    setup_exceptionhook,
)
from .helpers import (
    ArgumentParserDisableAbbrev,
    _fix_datalad_ri,
    _maybe_get_interface_subparser,
    _maybe_get_single_subparser,
    _parse_overrides_from_cmdline,
    get_description_with_cmd_summary,
    parser_add_common_options,
    parser_add_common_opt,
    strip_arg_from_argv,
)


# TODO:  OPT look into making setup_parser smarter to become faster
# Now it seems to take up to 200ms to do all the parser setup
# even though it might not be necessary to know about all the commands etc.
# I wondered if it could somehow decide on what commands to worry about etc
# by going through sys.args first
def setup_parser(
        cmdlineargs,
        formatter_class=argparse.RawDescriptionHelpFormatter,
        return_subparsers=False,
        # Was this triggered by argparse?
        completing=False,
        # prevent loading of extension entrypoints when --help is requested
        # this is enabled when building docs to avoid pollution of generated
        # manpages with extensions commands (that should appear in their own
        # docs, but not in the core datalad package docs)
        help_ignore_extensions=False):
    """
    The holy grail of establishing CLI for DataLad's Interfaces

    Parameters
    ----------
    cmdlineargs:
    formatter_class:
    return_subparsers: bool, optional
      is used ATM only by BuildManPage in _datalad_build_support
    completing:
    help_ignore_extensions:
    """
    lgr.log(5, "Starting to setup_parser")
    # delay since it can be a heavy import
    from ..interface.base import (
        dedent_docstring,
        get_cmdline_command_name,
        get_interface_groups,
    )

    # setup cmdline args parser

    # "main" parser is under "datalad" name
    all_parsers = {}  # name: (sub)parser

    # main parser
    parser = ArgumentParserDisableAbbrev(
        fromfile_prefix_chars=None,
        # usage="%(prog)s ...",
        description=dedent_docstring("""\
            Comprehensive data management solution

            DataLad provides a unified data distribution system built on the Git
            and Git-annex. DataLad command line tools allow to manipulate (obtain,
            create, update, publish, etc.) datasets and provide a comprehensive
            toolbox for joint management of data and code. Compared to Git/annex
            it primarily extends their functionality to transparently and
            simultaneously work with multiple inter-related repositories."""),
        epilog='"Be happy!"',
        formatter_class=formatter_class,
        add_help=False)

    # common options
    parser_add_common_options(parser)

    interface_groups = get_interface_groups()

    single_subparser = _maybe_get_single_subparser(
        cmdlineargs, parser, interface_groups,
        return_subparsers, completing, help_ignore_extensions
    )

    # --help specification was delayed since it causes immediate printout of
    # --help output before we setup --help for each command
    parser_add_common_opt(parser, 'help')

    grp_short_descriptions = defaultdict(list)
    # create subparser, use module suffix as cmd name
    subparsers = parser.add_subparsers()
    for group_name, _, _interfaces \
            in sorted(interface_groups, key=lambda x: x[1]):
        for _intfspec in _interfaces:
            cmd_name = get_cmdline_command_name(_intfspec)
            if single_subparser and cmd_name != single_subparser:
                continue
            subparser = _maybe_get_interface_subparser(
                _intfspec, subparsers, cmd_name, formatter_class, group_name,
                grp_short_descriptions
            )
            if subparser:  # interface might have failed to "load"
                all_parsers[cmd_name] = subparser

    # create command summary
    if '--help' in cmdlineargs or '--help-np' in cmdlineargs:
        parser.description = get_description_with_cmd_summary(
            grp_short_descriptions,
            interface_groups,
            parser.description)

    all_parsers['datalad'] = parser
    lgr.log(5, "Finished setup_parser")
    if return_subparsers:
        return all_parsers
    else:
        return parser


def main(args=None):
    lgr.log(5, "Starting main(%r)", args)
    # record that we came in via the cmdline
    datalad.__api = 'cmdline'
    args = args or sys.argv
    if on_msys_tainted_paths:
        # Possibly present DataLadRIs were stripped of a leading /
        args = [_fix_datalad_ri(s) for s in args]
    # PYTHON_ARGCOMPLETE_OK
    parser = setup_parser(args, completing="_ARGCOMPLETE" in os.environ)
    try:
        import argcomplete
        argcomplete.autocomplete(parser)
    except ImportError:
        pass

    # parse cmd args
    lgr.debug("Parsing known args among %s", repr(args))
    cmdlineargs, unparsed_args = parser.parse_known_args(args[1:])
    if cmdlineargs.common_library_mode:
        datalad.enable_librarymode()
    has_func = hasattr(cmdlineargs, 'func') and cmdlineargs.func is not None
    if unparsed_args:
        if has_func and cmdlineargs.func.__self__.__name__ != 'Export':
            lgr.error('unknown argument{}: {}'.format(
                's' if len(unparsed_args) > 1 else '',
                unparsed_args if len(unparsed_args) > 1 else unparsed_args[0]))
            cmdlineargs.subparser.print_usage()
            sys.exit(1)
        else:
            # store all unparsed arguments
            cmdlineargs.datalad_unparsed_args = unparsed_args

    # to possibly be passed into PBS scheduled call
    args_ = args or sys.argv

    if cmdlineargs.cfg_overrides is not None:
        datalad.cfg.overrides.update(
            _parse_overrides_from_cmdline(cmdlineargs)
        )

    # enable overrides
    datalad.cfg.reload(force=True)

    if cmdlineargs.change_path is not None:
        from .common_args import change_path as change_path_opt
        for path in cmdlineargs.change_path:
            chpwd(path)
            args_ = strip_arg_from_argv(args_, path, change_path_opt[1])

    ret = None
    if has_func:
        if cmdlineargs.common_debug or cmdlineargs.common_idebug:
            # so we could see/stop clearly at the point of failure
            setup_exceptionhook(ipython=cmdlineargs.common_idebug)
            from datalad.interface.base import Interface
            Interface._interrupted_exit_code = None
            ret = cmdlineargs.func(cmdlineargs)
        else:
            # otherwise - guard and only log the summary. Postmortem is not
            # as convenient if being caught in this ultimate except
            try:
                ret = cmdlineargs.func(cmdlineargs)
            except InsufficientArgumentsError as exc:
                ce = CapturedException(exc)
                # if the func reports inappropriate usage, give help output
                lgr.error('%s (%s)', ce, exc.__class__.__name__)
                cmdlineargs.subparser.print_usage(sys.stderr)
                sys.exit(2)
            except IncompleteResultsError as exc:
                ce = CapturedException(exc)
                # rendering for almost all commands now happens 'online'
                # hence we are no longer attempting to render the actual
                # results in an IncompleteResultsError, ubt rather trust that
                # this happened before

                # in general we do not want to see the error again, but
                # present in debug output
                lgr.debug('could not perform all requested actions: %s', ce)
                sys.exit(1)
            except CommandError as exc:
                ce = CapturedException(exc)
                # behave as if the command ran directly, importantly pass
                # exit code as is
                # to not duplicate any captured output in the exception
                # rendering, will come next
                exc_msg = exc.to_str(include_output=False)
                if exc_msg:
                    msg = exc_msg.encode() if isinstance(exc_msg, str) else exc_msg
                    os.write(2, msg + b"\n")
                if exc.stdout:
                    os.write(1, exc.stdout.encode() if isinstance(exc.stdout, str) else exc.stdout)
                if exc.stderr:
                    os.write(2, exc.stderr.encode() if isinstance(exc.stderr, str) else exc.stderr)
                # We must not exit with 0 code if any exception got here but
                # had no code defined
                sys.exit(exc.code if exc.code is not None else 1)
            except Exception as exc:
                ce = CapturedException(exc)
                lgr.error('%s (%s)', ce, exc.__class__.__name__)
                sys.exit(1)
    else:
        # just let argparser spit out its error, since there is smth wrong
        parser.parse_args(args)
        # if that one didn't puke -- we should
        parser.print_usage()
        lgr.error("Please specify the command")
        sys.exit(2)

    try:
        if hasattr(cmdlineargs, 'result_renderer'):
            cmdlineargs.result_renderer(ret, cmdlineargs)
    except Exception as exc:
        ce = CapturedException(exc)
        lgr.error("Failed to render results due to %s", ce)
        sys.exit(1)
    # all good, not strictly needed, but makes internal testing easier
    sys.exit(0)


lgr.log(5, "Done importing cmdline.main")

```
--- END FILE ---

Please first localize the bug based on the issue statement, and then generate *SEARCH/REPLACE* edits to fix the issue.

Every *SEARCH/REPLACE* edit must use this format:
1. The file path
2. The start of search block: <<<<<<< SEARCH
3. A contiguous chunk of lines to search for in the existing source code
4. The dividing line: =======
5. The lines to replace into the source code
6. The end of the replace block: >>>>>>> REPLACE

Here is an example:

```python
### mathweb/flask/app.py
<<<<<<< SEARCH
from flask import Flask
=======
import math
from flask import Flask
>>>>>>> REPLACE
```
</solution>

Please note that the *SEARCH/REPLACE* edit REQUIRES PROPER INDENTATION. If you would like to add the line '        print(x)', you must fully write that out, with all those spaces before the code!
Wrap each *SEARCH/REPLACE* edit in a code block as shown in the example above. If you have multiple *SEARCH/REPLACE* edits, use a separate code block for each one.
user