We are currently solving the following issue within our repository. Here is the issue text:
--- BEGIN ISSUE ---
Thread should be created after deamonizing
There is a long existing bug related to `Task Queue has died unexpectedly` in daemon mode. #1947 #2656 The cause of this bug is really simple. **Do not** create thread before `os.fork()` (demonize). Here is a small code snippet that demonstrates the problem:
```python
from threading import Thread
import os
import time
import sys


def func():
    for _ in range(5):
        print('running in thread')
        time.sleep(0.5)

t = Thread(target=func)

pid = os.fork()
if pid > 0:
    sys.exit(0)

t.start()
print("pid:", os.getpid())
print('is thread alive after fork:', t.is_alive())
t.join()
```
The running result is as follows:
```
# lym @ mig186 in ~ [11:46:22]
$ python main.py
running in thread
pid: 122253
is thread alive after fork: False
# lym @ mig186 in ~ [11:46:23]
$ running in thread
running in thread
running in thread
running in thread
```
The thread resource created before `os.fork()` is only alive in the parent process (which is exited to daemonize). In the child process, the `t.is_alive()` always return False. Here is the right way to do:
```python
from threading import Thread
import os
import time
import sys


def func():
    for _ in range(5):
        print('running in thread')
        time.sleep(0.5)


pid = os.fork()
if pid > 0:
    sys.exit(0)

t = Thread(target=func)
t.start()
print("pid:", os.getpid())
print('is thread alive after fork:', t.is_alive())
t.join()
```
By the way, the process of daemonize usually involves double fork, in which case we should create all the threading after the second fork.

And here is the cause of bug in `flexget/manager.py`, the task queue is initialized in line 231 in the `def initialize(self)` function, and the damonize happens afterwards in line 476 `self.daemonize()`, which involves double forking:
```python
    def daemonize(self) -> None:
        """Daemonizes the current process. Returns the new pid"""
        if sys.platform.startswith('win'):
            logger.error('Cannot daemonize on windows')
            return
        if threading.active_count() != 1:
            logger.critical(
                'There are {!r} active threads. Daemonizing now may cause strange failures.',
                threading.enumerate(),
            )

        logger.info('Daemonizing...')

        try:
            pid = os.fork()
            if pid > 0:
                # Don't run the exit handlers on the parent
                atexit._exithandlers = []
                # exit first parent
                sys.exit(0)
        except OSError as e:
            sys.stderr.write(f'fork #1 failed: {e.errno} ({e.strerror})\n')
            sys.exit(1)

        # decouple from parent environment
        os.chdir('/')
        os.setsid()
        os.umask(0)

        # do second fork
        try:
            pid = os.fork()
            if pid > 0:
                # Don't run the exit handlers on the parent
                atexit._exithandlers = []
                # exit from second parent
                sys.exit(0)
        except OSError as e:
            sys.stderr.write(f'fork #2 failed: {e.errno} ({e.strerror})\n')
            sys.exit(1)
```

Therefore after daemonizing, when we try to execute the command, this function always shows the not alive error (line 415):
```python
    def execute_command(self, options: argparse.Namespace) -> None:
        """
        Handles the 'execute' CLI command.
        If there is already a task queue running in this process, adds the execution to the queue.
        If FlexGet is being invoked with this command, starts up a task queue and runs the execution.
        Fires events:
        * manager.execute.started
        * manager.execute.completed
        :param options: argparse options
        """
        fire_event('manager.execute.started', self, options)
        if self.task_queue.is_alive() or self.is_daemon:
            if not self.task_queue.is_alive():
                logger.error(
                    'Task queue has died unexpectedly. Restarting it. Please open an issue on Github and include'
                    ' any previous error logs.'
                )
```
And it creates a new task queue and leaving the old one hanging, which causes other strange problems like the flexget daemon can not be stopped.




--- END ISSUE ---

Below are some code segments, each from a relevant file. One or more of these files may contain bugs.
--- BEGIN FILES ---
Path: `flexget/plugins/input/filesystem.py`
Content:
```
1 from __future__ import unicode_literals, division, absolute_import
2 import logging
3 import re
4 import os
5 
6 from path import Path
7 
8 from flexget import plugin
9 from flexget.config_schema import one_or_more
10 from flexget.event import event
11 from flexget.entry import Entry
12 
13 log = logging.getLogger('filesystem')
14 
15 
16 class Filesystem(object):
17     """
18     Uses local path content as an input. Can use recursion if configured.
19     Recursion is False by default. Can be configured to true or get integer that will specify max depth in relation to
20         base folder.
21     All files/dir/symlinks are retrieved by default. Can be changed by using the 'retrieve' property.
22 
23     Example 1:: Single path
24 
25       filesystem: /storage/movies/
26 
27     Example 2:: List of paths
28 
29       filesystem:
30          - /storage/movies/
31          - /storage/tv/
32 
33     Example 3:: Object with list of paths
34 
35       filesystem:
36         path:
37           - /storage/movies/
38           - /storage/tv/
39         mask: '*.mkv'
40 
41     Example 4::
42 
43       filesystem:
44         path:
45           - /storage/movies/
46           - /storage/tv/
47         recursive: 4  # 4 levels deep from each base folder
48         retrieve: files  # Only files will be retrieved
49 
50     Example 5::
51 
52       filesystem:
53         path:
54           - /storage/movies/
55           - /storage/tv/
56         recursive: yes  # No limit to depth, all sub dirs will be accessed
57         retrieve:  # Only files and dirs will be retrieved
58           - files
59           - dirs
60 
61     """
62     retrieval_options = ['files', 'dirs', 'symlinks']
63     paths = one_or_more({'type': 'string', 'format': 'path'}, unique_items=True)
64 
65     schema = {
66         'oneOf': [
67             paths,
68             {'type': 'object',
69              'properties': {
70                  'path': paths,
71                  'mask': {'type': 'string'},
72                  'regexp': {'type': 'string', 'format': 'regex'},
73                  'recursive': {'oneOf': [{'type': 'integer', 'minimum': 2}, {'type': 'boolean'}]},
74                  'retrieve': one_or_more({'type': 'string', 'enum': retrieval_options}, unique_items=True)
75              },
76              'required': ['path'],
77              'additionalProperties': False}]
78     }
79 
80     def prepare_config(self, config):
81         from fnmatch import translate
82         config = config
83 
84         # Converts config to a dict with a list of paths
85         if not isinstance(config, dict):
86             config = {'path': config}
87         if not isinstance(config['path'], list):
88             config['path'] = [config['path']]
89 
90         config.setdefault('recursive', False)
91         # If mask was specified, turn it in to a regexp
92         if config.get('mask'):
93             config['regexp'] = translate(config['mask'])
94         # If no mask or regexp specified, accept all files
95         config.setdefault('regexp', '.')
96         # Sets the default retrieval option to files
97         config.setdefault('retrieve', self.retrieval_options)
98 
99         return config
100 
101     def create_entry(self, filepath, test_mode):
102         """
103         Creates a single entry using a filepath and a type (file/dir)
104         """
105         entry = Entry()
106         entry['location'] = filepath
107         entry['url'] = 'file://{}'.format(filepath)
108         entry['filename'] = filepath.name
109         if filepath.isfile():
110             entry['title'] = filepath.namebase
111         else:
112             entry['title'] = filepath.name
113         try:
114             entry['timestamp'] = os.path.getmtime(filepath)
115         except Exception as e:
116             log.warning('Error setting timestamp for %s: %s' % (filepath, e))
117             entry['timestamp'] = None
118         if entry.isvalid():
119             if test_mode:
120                 log.info("Test mode. Entry includes:")
121                 log.info("    Title: %s" % entry["title"])
122                 log.info("    URL: %s" % entry["url"])
123                 log.info("    Filename: %s" % entry["filename"])
124                 log.info("    Location: %s" % entry["location"])
125                 log.info("    Timestamp: %s" % entry["timestamp"])
126             return entry
127         else:
128             log.error('Non valid entry created: {}'.format(entry))
129             return
130 
131     def get_max_depth(self, recursion, base_depth):
132         if recursion is False:
133             return base_depth + 1
134         elif recursion is True:
135             return float('inf')
136         else:
137             return base_depth + recursion
138 
139     def get_folder_objects(self, folder, recursion):
140         if recursion is False:
141             return folder.listdir()
142         else:
143             return folder.walk(errors='ignore')
144 
145     def get_entries_from_path(self, path_list, match, recursion, test_mode, get_files, get_dirs, get_symlinks):
146         entries = []
147 
148         for folder in path_list:
149             log.verbose('Scanning folder {}. Recursion is set to {}.'.format(folder, recursion))
150             folder = Path(folder).expanduser()
151             log.debug('Scanning %s' % folder)
152             base_depth = len(folder.splitall())
153             max_depth = self.get_max_depth(recursion, base_depth)
154             folder_objects = self.get_folder_objects(folder, recursion)
155             for path_object in folder_objects:
156                 log.debug('Checking if {} qualifies to be added as an entry.'.format(path_object))
157                 try:
158                     path_object.exists()
159                 except UnicodeError:
160                     log.error('File %s not decodable with filesystem encoding: {}'.format(path_object))
161                     continue
162                 entry = None
163                 object_depth = len(path_object.splitall())
164                 if object_depth <= max_depth:
165                     if match(path_object):
166                         if (path_object.isdir() and get_dirs) or (
167                             path_object.islink() and get_symlinks) or (
168                             path_object.isfile() and get_files):
169                             entry = self.create_entry(path_object, test_mode)
170                         else:
171                             log.debug("Path object's {} type doesn't match requested object types.".format(path_object))
172                         if entry and entry not in entries:
173                             entries.append(entry)
174 
175         return entries
176 
177     def on_task_input(self, task, config):
178         config = self.prepare_config(config)
179 
180         path_list = config['path']
181         test_mode = task.options.test
182         match = re.compile(config['regexp'], re.IGNORECASE).match
183         recursive = config['recursive']
184         get_files = 'files' in config['retrieve']
185         get_dirs = 'dirs' in config['retrieve']
186         get_symlinks = 'symlinks' in config['retrieve']
187 
188         log.info('Starting to scan folders.')
189         return self.get_entries_from_path(path_list, match, recursive, test_mode, get_files, get_dirs, get_symlinks)
190 
191 
192 @event('plugin.register')
193 def register_plugin():
194     plugin.register(Filesystem, 'filesystem', api_ver=2)
195
```
--- END FILES ---

Please first localize the bug based on the issue statement, and then generate a patch according to the `git diff` format fenced by three backticks.

Here is an example:

```diff
diff --git a/examples/server_async.py b/examples/server_async.py
--- a/examples/server_async.py
+++ b/examples/server_async.py
@@ -313,4 +313,4 @@
 
 
 if __name__ == "__main__":
-    asyncio.run(run_async_server("."), debug=True)
+    asyncio.run(run_async_server(), debug=True)
diff --git a/examples/server_sync.py b/examples/server_sync.py
--- a/examples/server_sync.py
+++ b/examples/server_sync.py
@@ -313,5 +313,5 @@
 
 
 if __name__ == "__main__":
-    server = run_sync_server(".")
+    server = run_sync_server()
     server.shutdown()

```

diff --git a/flexget/plugins/input/filesystem.py b/flexget/plugins/input/filesystem.py
--- a/flexget/plugins/input/filesystem.py
+++ b/flexget/plugins/input/filesystem.py
@@ -1,7 +1,7 @@
 from __future__ import unicode_literals, division, absolute_import
 import logging
 import re
-import os
+from datetime import datetime
 
 from path import Path
 
@@ -104,14 +104,14 @@
         """
         entry = Entry()
         entry['location'] = filepath
-        entry['url'] = 'file://{}'.format(filepath)
+        entry['url'] = 'file://%s' % filepath
         entry['filename'] = filepath.name
         if filepath.isfile():
             entry['title'] = filepath.namebase
         else:
             entry['title'] = filepath.name
         try:
-            entry['timestamp'] = os.path.getmtime(filepath)
+            entry['timestamp'] = datetime.fromtimestamp(filepath.getmtime())
         except Exception as e:
             log.warning('Error setting timestamp for %s: %s' % (filepath, e))
             entry['timestamp'] = None
@@ -125,7 +125,7 @@
                 log.info("    Timestamp: %s" % entry["timestamp"])
             return entry
         else:
-            log.error('Non valid entry created: {}'.format(entry))
+            log.error('Non valid entry created: %s '% entry)
             return
 
     def get_max_depth(self, recursion, base_depth):
@@ -146,18 +146,18 @@
         entries = []
 
         for folder in path_list:
-            log.verbose('Scanning folder {}. Recursion is set to {}.'.format(folder, recursion))
+            log.verbose('Scanning folder %s. Recursion is set to %s.' % (folder, recursion))
             folder = Path(folder).expanduser()
             log.debug('Scanning %s' % folder)
             base_depth = len(folder.splitall())
             max_depth = self.get_max_depth(recursion, base_depth)
             folder_objects = self.get_folder_objects(folder, recursion)
             for path_object in folder_objects:
-                log.debug('Checking if {} qualifies to be added as an entry.'.format(path_object))
+                log.debug('Checking if %s qualifies to be added as an entry.' % path_object)
                 try:
                     path_object.exists()
                 except UnicodeError:
-                    log.error('File %s not decodable with filesystem encoding: {}'.format(path_object))
+                    log.error('File %s not decodable with filesystem encoding: %s' % path_object)
                     continue
                 entry = None
                 object_depth = len(path_object.splitall())
@@ -168,7 +168,7 @@
                             path_object.isfile() and get_files):
                             entry = self.create_entry(path_object, test_mode)
                         else:
-                            log.debug("Path object's {} type doesn't match requested object types.".format(path_object))
+                            log.debug("Path object's %s type doesn't match requested object types." % path_object)
                         if entry and entry not in entries:
                             entries.append(entry)

{"golden_diff": "diff --git a/flexget/plugins/input/filesystem.py b/flexget/plugins/input/filesystem.py\n--- a/flexget/plugins/input/filesystem.py\n+++ b/flexget/plugins/input/filesystem.py\n@@ -1,7 +1,7 @@\n from __future__ import unicode_literals, division, absolute_import\n import logging\n import re\n-import os\n+from datetime import datetime\n \n from path import Path\n \n@@ -104,14 +104,14 @@\n         \"\"\"\n         entry = Entry()\n         entry['location'] = filepath\n-        entry['url'] = 'file://{}'.format(filepath)\n+        entry['url'] = 'file://%s' % filepath\n         entry['filename'] = filepath.name\n         if filepath.isfile():\n             entry['title'] = filepath.namebase\n         else:\n             entry['title'] = filepath.name\n         try:\n-            entry['timestamp'] = os.path.getmtime(filepath)\n+            entry['timestamp'] = datetime.fromtimestamp(filepath.getmtime())\n         except Exception as e:\n             log.warning('Error setting timestamp for %s: %s' % (filepath, e))\n             entry['timestamp'] = None\n@@ -125,7 +125,7 @@\n                 log.info(\"    Timestamp: %s\" % entry[\"timestamp\"])\n             return entry\n         else:\n-            log.error('Non valid entry created: {}'.format(entry))\n+            log.error('Non valid entry created: %s '% entry)\n             return\n \n     def get_max_depth(self, recursion, base_depth):\n@@ -146,18 +146,18 @@\n         entries = []\n \n         for folder in path_list:\n-            log.verbose('Scanning folder {}. Recursion is set to {}.'.format(folder, recursion))\n+            log.verbose('Scanning folder %s. Recursion is set to %s.' % (folder, recursion))\n             folder = Path(folder).expanduser()\n             log.debug('Scanning %s' % folder)\n             base_depth = len(folder.splitall())\n             max_depth = self.get_max_depth(recursion, base_depth)\n             folder_objects = self.get_folder_objects(folder, recursion)\n             for path_object in folder_objects:\n-                log.debug('Checking if {} qualifies to be added as an entry.'.format(path_object))\n+                log.debug('Checking if %s qualifies to be added as an entry.' % path_object)\n                 try:\n                     path_object.exists()\n                 except UnicodeError:\n-                    log.error('File %s not decodable with filesystem encoding: {}'.format(path_object))\n+                    log.error('File %s not decodable with filesystem encoding: %s' % path_object)\n                     continue\n                 entry = None\n                 object_depth = len(path_object.splitall())\n@@ -168,7 +168,7 @@\n                             path_object.isfile() and get_files):\n                             entry = self.create_entry(path_object, test_mode)\n                         else:\n-                            log.debug(\"Path object's {} type doesn't match requested object types.\".format(path_object))\n+                            log.debug(\"Path object's %s type doesn't match requested object types.\" % path_object)\n                         if entry and entry not in entries:\n                             entries.append(entry)\n", "issue": "Thread should be created after deamonizing\nThere is a long existing bug related to `Task Queue has died unexpectedly` in daemon mode. #1947 #2656 The cause of this bug is really simple. **Do not** create thread before `os.fork()` (demonize). Here is a small code snippet that demonstrates the problem:\r\n```python\r\nfrom threading import Thread\r\nimport os\r\nimport time\r\nimport sys\r\n\r\n\r\ndef func():\r\n    for _ in range(5):\r\n        print('running in thread')\r\n        time.sleep(0.5)\r\n\r\nt = Thread(target=func)\r\n\r\npid = os.fork()\r\nif pid > 0:\r\n    sys.exit(0)\r\n\r\nt.start()\r\nprint(\"pid:\", os.getpid())\r\nprint('is thread alive after fork:', t.is_alive())\r\nt.join()\r\n```\r\nThe running result is as follows:\r\n```\r\n# lym @ mig186 in ~ [11:46:22]\r\n$ python main.py\r\nrunning in thread\r\npid: 122253\r\nis thread alive after fork: False\r\n# lym @ mig186 in ~ [11:46:23]\r\n$ running in thread\r\nrunning in thread\r\nrunning in thread\r\nrunning in thread\r\n```\r\nThe thread resource created before `os.fork()` is only alive in the parent process (which is exited to daemonize). In the child process, the `t.is_alive()` always return False. Here is the right way to do:\r\n```python\r\nfrom threading import Thread\r\nimport os\r\nimport time\r\nimport sys\r\n\r\n\r\ndef func():\r\n    for _ in range(5):\r\n        print('running in thread')\r\n        time.sleep(0.5)\r\n\r\n\r\npid = os.fork()\r\nif pid > 0:\r\n    sys.exit(0)\r\n\r\nt = Thread(target=func)\r\nt.start()\r\nprint(\"pid:\", os.getpid())\r\nprint('is thread alive after fork:', t.is_alive())\r\nt.join()\r\n```\r\nBy the way, the process of daemonize usually involves double fork, in which case we should create all the threading after the second fork.\r\n\r\nAnd here is the cause of bug in `flexget/manager.py`, the task queue is initialized in line 231 in the `def initialize(self)` function, and the damonize happens afterwards in line 476 `self.daemonize()`, which involves double forking:\r\n```python\r\n    def daemonize(self) -> None:\r\n        \"\"\"Daemonizes the current process. Returns the new pid\"\"\"\r\n        if sys.platform.startswith('win'):\r\n            logger.error('Cannot daemonize on windows')\r\n            return\r\n        if threading.active_count() != 1:\r\n            logger.critical(\r\n                'There are {!r} active threads. Daemonizing now may cause strange failures.',\r\n                threading.enumerate(),\r\n            )\r\n\r\n        logger.info('Daemonizing...')\r\n\r\n        try:\r\n            pid = os.fork()\r\n            if pid > 0:\r\n                # Don't run the exit handlers on the parent\r\n                atexit._exithandlers = []\r\n                # exit first parent\r\n                sys.exit(0)\r\n        except OSError as e:\r\n            sys.stderr.write(f'fork #1 failed: {e.errno} ({e.strerror})\\n')\r\n            sys.exit(1)\r\n\r\n        # decouple from parent environment\r\n        os.chdir('/')\r\n        os.setsid()\r\n        os.umask(0)\r\n\r\n        # do second fork\r\n        try:\r\n            pid = os.fork()\r\n            if pid > 0:\r\n                # Don't run the exit handlers on the parent\r\n                atexit._exithandlers = []\r\n                # exit from second parent\r\n                sys.exit(0)\r\n        except OSError as e:\r\n            sys.stderr.write(f'fork #2 failed: {e.errno} ({e.strerror})\\n')\r\n            sys.exit(1)\r\n```\r\n\r\nTherefore after daemonizing, when we try to execute the command, this function always shows the not alive error (line 415):\r\n```python\r\n    def execute_command(self, options: argparse.Namespace) -> None:\r\n        \"\"\"\r\n        Handles the 'execute' CLI command.\r\n        If there is already a task queue running in this process, adds the execution to the queue.\r\n        If FlexGet is being invoked with this command, starts up a task queue and runs the execution.\r\n        Fires events:\r\n        * manager.execute.started\r\n        * manager.execute.completed\r\n        :param options: argparse options\r\n        \"\"\"\r\n        fire_event('manager.execute.started', self, options)\r\n        if self.task_queue.is_alive() or self.is_daemon:\r\n            if not self.task_queue.is_alive():\r\n                logger.error(\r\n                    'Task queue has died unexpectedly. Restarting it. Please open an issue on Github and include'\r\n                    ' any previous error logs.'\r\n                )\r\n```\r\nAnd it creates a new task queue and leaving the old one hanging, which causes other strange problems like the flexget daemon can not be stopped.\r\n\r\n\r\n\n", "before_files": [{"content": "from __future__ import unicode_literals, division, absolute_import\nimport logging\nimport re\nimport os\n\nfrom path import Path\n\nfrom flexget import plugin\nfrom flexget.config_schema import one_or_more\nfrom flexget.event import event\nfrom flexget.entry import Entry\n\nlog = logging.getLogger('filesystem')\n\n\nclass Filesystem(object):\n    \"\"\"\n    Uses local path content as an input. Can use recursion if configured.\n    Recursion is False by default. Can be configured to true or get integer that will specify max depth in relation to\n        base folder.\n    All files/dir/symlinks are retrieved by default. Can be changed by using the 'retrieve' property.\n\n    Example 1:: Single path\n\n      filesystem: /storage/movies/\n\n    Example 2:: List of paths\n\n      filesystem:\n         - /storage/movies/\n         - /storage/tv/\n\n    Example 3:: Object with list of paths\n\n      filesystem:\n        path:\n          - /storage/movies/\n          - /storage/tv/\n        mask: '*.mkv'\n\n    Example 4::\n\n      filesystem:\n        path:\n          - /storage/movies/\n          - /storage/tv/\n        recursive: 4  # 4 levels deep from each base folder\n        retrieve: files  # Only files will be retrieved\n\n    Example 5::\n\n      filesystem:\n        path:\n          - /storage/movies/\n          - /storage/tv/\n        recursive: yes  # No limit to depth, all sub dirs will be accessed\n        retrieve:  # Only files and dirs will be retrieved\n          - files\n          - dirs\n\n    \"\"\"\n    retrieval_options = ['files', 'dirs', 'symlinks']\n    paths = one_or_more({'type': 'string', 'format': 'path'}, unique_items=True)\n\n    schema = {\n        'oneOf': [\n            paths,\n            {'type': 'object',\n             'properties': {\n                 'path': paths,\n                 'mask': {'type': 'string'},\n                 'regexp': {'type': 'string', 'format': 'regex'},\n                 'recursive': {'oneOf': [{'type': 'integer', 'minimum': 2}, {'type': 'boolean'}]},\n                 'retrieve': one_or_more({'type': 'string', 'enum': retrieval_options}, unique_items=True)\n             },\n             'required': ['path'],\n             'additionalProperties': False}]\n    }\n\n    def prepare_config(self, config):\n        from fnmatch import translate\n        config = config\n\n        # Converts config to a dict with a list of paths\n        if not isinstance(config, dict):\n            config = {'path': config}\n        if not isinstance(config['path'], list):\n            config['path'] = [config['path']]\n\n        config.setdefault('recursive', False)\n        # If mask was specified, turn it in to a regexp\n        if config.get('mask'):\n            config['regexp'] = translate(config['mask'])\n        # If no mask or regexp specified, accept all files\n        config.setdefault('regexp', '.')\n        # Sets the default retrieval option to files\n        config.setdefault('retrieve', self.retrieval_options)\n\n        return config\n\n    def create_entry(self, filepath, test_mode):\n        \"\"\"\n        Creates a single entry using a filepath and a type (file/dir)\n        \"\"\"\n        entry = Entry()\n        entry['location'] = filepath\n        entry['url'] = 'file://{}'.format(filepath)\n        entry['filename'] = filepath.name\n        if filepath.isfile():\n            entry['title'] = filepath.namebase\n        else:\n            entry['title'] = filepath.name\n        try:\n            entry['timestamp'] = os.path.getmtime(filepath)\n        except Exception as e:\n            log.warning('Error setting timestamp for %s: %s' % (filepath, e))\n            entry['timestamp'] = None\n        if entry.isvalid():\n            if test_mode:\n                log.info(\"Test mode. Entry includes:\")\n                log.info(\"    Title: %s\" % entry[\"title\"])\n                log.info(\"    URL: %s\" % entry[\"url\"])\n                log.info(\"    Filename: %s\" % entry[\"filename\"])\n                log.info(\"    Location: %s\" % entry[\"location\"])\n                log.info(\"    Timestamp: %s\" % entry[\"timestamp\"])\n            return entry\n        else:\n            log.error('Non valid entry created: {}'.format(entry))\n            return\n\n    def get_max_depth(self, recursion, base_depth):\n        if recursion is False:\n            return base_depth + 1\n        elif recursion is True:\n            return float('inf')\n        else:\n            return base_depth + recursion\n\n    def get_folder_objects(self, folder, recursion):\n        if recursion is False:\n            return folder.listdir()\n        else:\n            return folder.walk(errors='ignore')\n\n    def get_entries_from_path(self, path_list, match, recursion, test_mode, get_files, get_dirs, get_symlinks):\n        entries = []\n\n        for folder in path_list:\n            log.verbose('Scanning folder {}. Recursion is set to {}.'.format(folder, recursion))\n            folder = Path(folder).expanduser()\n            log.debug('Scanning %s' % folder)\n            base_depth = len(folder.splitall())\n            max_depth = self.get_max_depth(recursion, base_depth)\n            folder_objects = self.get_folder_objects(folder, recursion)\n            for path_object in folder_objects:\n                log.debug('Checking if {} qualifies to be added as an entry.'.format(path_object))\n                try:\n                    path_object.exists()\n                except UnicodeError:\n                    log.error('File %s not decodable with filesystem encoding: {}'.format(path_object))\n                    continue\n                entry = None\n                object_depth = len(path_object.splitall())\n                if object_depth <= max_depth:\n                    if match(path_object):\n                        if (path_object.isdir() and get_dirs) or (\n                            path_object.islink() and get_symlinks) or (\n                            path_object.isfile() and get_files):\n                            entry = self.create_entry(path_object, test_mode)\n                        else:\n                            log.debug(\"Path object's {} type doesn't match requested object types.\".format(path_object))\n                        if entry and entry not in entries:\n                            entries.append(entry)\n\n        return entries\n\n    def on_task_input(self, task, config):\n        config = self.prepare_config(config)\n\n        path_list = config['path']\n        test_mode = task.options.test\n        match = re.compile(config['regexp'], re.IGNORECASE).match\n        recursive = config['recursive']\n        get_files = 'files' in config['retrieve']\n        get_dirs = 'dirs' in config['retrieve']\n        get_symlinks = 'symlinks' in config['retrieve']\n\n        log.info('Starting to scan folders.')\n        return self.get_entries_from_path(path_list, match, recursive, test_mode, get_files, get_dirs, get_symlinks)\n\n\n@event('plugin.register')\ndef register_plugin():\n    plugin.register(Filesystem, 'filesystem', api_ver=2)\n", "path": "flexget/plugins/input/filesystem.py"}], "after_files": [{"content": "from __future__ import unicode_literals, division, absolute_import\nimport logging\nimport re\nfrom datetime import datetime\n\nfrom path import Path\n\nfrom flexget import plugin\nfrom flexget.config_schema import one_or_more\nfrom flexget.event import event\nfrom flexget.entry import Entry\n\nlog = logging.getLogger('filesystem')\n\n\nclass Filesystem(object):\n    \"\"\"\n    Uses local path content as an input. Can use recursion if configured.\n    Recursion is False by default. Can be configured to true or get integer that will specify max depth in relation to\n        base folder.\n    All files/dir/symlinks are retrieved by default. Can be changed by using the 'retrieve' property.\n\n    Example 1:: Single path\n\n      filesystem: /storage/movies/\n\n    Example 2:: List of paths\n\n      filesystem:\n         - /storage/movies/\n         - /storage/tv/\n\n    Example 3:: Object with list of paths\n\n      filesystem:\n        path:\n          - /storage/movies/\n          - /storage/tv/\n        mask: '*.mkv'\n\n    Example 4::\n\n      filesystem:\n        path:\n          - /storage/movies/\n          - /storage/tv/\n        recursive: 4  # 4 levels deep from each base folder\n        retrieve: files  # Only files will be retrieved\n\n    Example 5::\n\n      filesystem:\n        path:\n          - /storage/movies/\n          - /storage/tv/\n        recursive: yes  # No limit to depth, all sub dirs will be accessed\n        retrieve:  # Only files and dirs will be retrieved\n          - files\n          - dirs\n\n    \"\"\"\n    retrieval_options = ['files', 'dirs', 'symlinks']\n    paths = one_or_more({'type': 'string', 'format': 'path'}, unique_items=True)\n\n    schema = {\n        'oneOf': [\n            paths,\n            {'type': 'object',\n             'properties': {\n                 'path': paths,\n                 'mask': {'type': 'string'},\n                 'regexp': {'type': 'string', 'format': 'regex'},\n                 'recursive': {'oneOf': [{'type': 'integer', 'minimum': 2}, {'type': 'boolean'}]},\n                 'retrieve': one_or_more({'type': 'string', 'enum': retrieval_options}, unique_items=True)\n             },\n             'required': ['path'],\n             'additionalProperties': False}]\n    }\n\n    def prepare_config(self, config):\n        from fnmatch import translate\n        config = config\n\n        # Converts config to a dict with a list of paths\n        if not isinstance(config, dict):\n            config = {'path': config}\n        if not isinstance(config['path'], list):\n            config['path'] = [config['path']]\n\n        config.setdefault('recursive', False)\n        # If mask was specified, turn it in to a regexp\n        if config.get('mask'):\n            config['regexp'] = translate(config['mask'])\n        # If no mask or regexp specified, accept all files\n        config.setdefault('regexp', '.')\n        # Sets the default retrieval option to files\n        config.setdefault('retrieve', self.retrieval_options)\n\n        return config\n\n    def create_entry(self, filepath, test_mode):\n        \"\"\"\n        Creates a single entry using a filepath and a type (file/dir)\n        \"\"\"\n        entry = Entry()\n        entry['location'] = filepath\n        entry['url'] = 'file://%s' % filepath\n        entry['filename'] = filepath.name\n        if filepath.isfile():\n            entry['title'] = filepath.namebase\n        else:\n            entry['title'] = filepath.name\n        try:\n            entry['timestamp'] = datetime.fromtimestamp(filepath.getmtime())\n        except Exception as e:\n            log.warning('Error setting timestamp for %s: %s' % (filepath, e))\n            entry['timestamp'] = None\n        if entry.isvalid():\n            if test_mode:\n                log.info(\"Test mode. Entry includes:\")\n                log.info(\"    Title: %s\" % entry[\"title\"])\n                log.info(\"    URL: %s\" % entry[\"url\"])\n                log.info(\"    Filename: %s\" % entry[\"filename\"])\n                log.info(\"    Location: %s\" % entry[\"location\"])\n                log.info(\"    Timestamp: %s\" % entry[\"timestamp\"])\n            return entry\n        else:\n            log.error('Non valid entry created: %s '% entry)\n            return\n\n    def get_max_depth(self, recursion, base_depth):\n        if recursion is False:\n            return base_depth + 1\n        elif recursion is True:\n            return float('inf')\n        else:\n            return base_depth + recursion\n\n    def get_folder_objects(self, folder, recursion):\n        if recursion is False:\n            return folder.listdir()\n        else:\n            return folder.walk(errors='ignore')\n\n    def get_entries_from_path(self, path_list, match, recursion, test_mode, get_files, get_dirs, get_symlinks):\n        entries = []\n\n        for folder in path_list:\n            log.verbose('Scanning folder %s. Recursion is set to %s.' % (folder, recursion))\n            folder = Path(folder).expanduser()\n            log.debug('Scanning %s' % folder)\n            base_depth = len(folder.splitall())\n            max_depth = self.get_max_depth(recursion, base_depth)\n            folder_objects = self.get_folder_objects(folder, recursion)\n            for path_object in folder_objects:\n                log.debug('Checking if %s qualifies to be added as an entry.' % path_object)\n                try:\n                    path_object.exists()\n                except UnicodeError:\n                    log.error('File %s not decodable with filesystem encoding: %s' % path_object)\n                    continue\n                entry = None\n                object_depth = len(path_object.splitall())\n                if object_depth <= max_depth:\n                    if match(path_object):\n                        if (path_object.isdir() and get_dirs) or (\n                            path_object.islink() and get_symlinks) or (\n                            path_object.isfile() and get_files):\n                            entry = self.create_entry(path_object, test_mode)\n                        else:\n                            log.debug(\"Path object's %s type doesn't match requested object types.\" % path_object)\n                        if entry and entry not in entries:\n                            entries.append(entry)\n\n        return entries\n\n    def on_task_input(self, task, config):\n        config = self.prepare_config(config)\n\n        path_list = config['path']\n        test_mode = task.options.test\n        match = re.compile(config['regexp'], re.IGNORECASE).match\n        recursive = config['recursive']\n        get_files = 'files' in config['retrieve']\n        get_dirs = 'dirs' in config['retrieve']\n        get_symlinks = 'symlinks' in config['retrieve']\n\n        log.info('Starting to scan folders.')\n        return self.get_entries_from_path(path_list, match, recursive, test_mode, get_files, get_dirs, get_symlinks)\n\n\n@event('plugin.register')\ndef register_plugin():\n    plugin.register(Filesystem, 'filesystem', api_ver=2)\n", "path": "flexget/plugins/input/filesystem.py"}]}