We are currently solving the following issue within our repository. Here is the issue text:
--- BEGIN ISSUE ---
feat: add json schema validation
Adds a JSON Schema that validates `data.json` and `removed_sites.json`. I've already validated the existing data with it and can confirm both files pass. 👍🏾 

Prepends the following property to both files:
```json
{
  "$schema": "{path/to/json-schema}"
}
```

Removes the `rank` property from `removed_sites.json` and documentation as this is no longer relevant.

Removes the "noPeriod" property from Jimdo as this appears to serve on purpose.

---

Sorry, also a question. What is the purpose of `responseUrl`? I just realized, this property is not referenced whatsoever in the actual application. 🤔 Can this property be removed in the dataset and schema?

Later I do want to make it stricter, i.e. prevent someone from using one `errorType` with the wrong additional field, i.e.
```json
{ 
  "errorMsg": "Not Found",
  "errorType": "status_code"
} 
```

I figured I'd leave it like this for now and find out what `responseUrl` is about. 🤔 

### Related
* Closes https://github.com/sherlock-project/sherlock/issues/1336

The goal of this is to improve the experience for developers and leave less room for human-error. It also enforces that we document and provide examples of each field, unlike the current Wiki which tends to get outdated.

![image](https://user-images.githubusercontent.com/22801583/167285185-11711582-a8d1-4157-a5e9-4d590d6e71b6.png)

![image](https://user-images.githubusercontent.com/22801583/167285139-15b5d326-5813-45df-8987-dfd6f7a236ad.png)

![image](https://user-images.githubusercontent.com/22801583/167285147-e759a759-2734-4811-b972-ee75ed7dd530.png)    




--- END ISSUE ---

Below are some code segments, each from a relevant file. One or more of these files may contain bugs.
--- BEGIN FILES ---
Path: `sherlock/sites.py`
Content:
```
1 """Sherlock Sites Information Module
2 
3 This module supports storing information about websites.
4 This is the raw data that will be used to search for usernames.
5 """
6 import json
7 import requests
8 import secrets
9 
10 class SiteInformation:
11     def __init__(self, name, url_home, url_username_format, username_claimed,
12                 information, is_nsfw, username_unclaimed=secrets.token_urlsafe(10)):
13         """Create Site Information Object.
14 
15         Contains information about a specific website.
16 
17         Keyword Arguments:
18         self                   -- This object.
19         name                   -- String which identifies site.
20         url_home               -- String containing URL for home of site.
21         url_username_format    -- String containing URL for Username format
22                                   on site.
23                                   NOTE:  The string should contain the
24                                          token "{}" where the username should
25                                          be substituted.  For example, a string
26                                          of "https://somesite.com/users/{}"
27                                          indicates that the individual
28                                          usernames would show up under the
29                                          "https://somesite.com/users/" area of
30                                          the website.
31         username_claimed       -- String containing username which is known
32                                   to be claimed on website.
33         username_unclaimed     -- String containing username which is known
34                                   to be unclaimed on website.
35         information            -- Dictionary containing all known information
36                                   about website.
37                                   NOTE:  Custom information about how to
38                                          actually detect the existence of the
39                                          username will be included in this
40                                          dictionary.  This information will
41                                          be needed by the detection method,
42                                          but it is only recorded in this
43                                          object for future use.
44         is_nsfw                -- Boolean indicating if site is Not Safe For Work.
45 
46         Return Value:
47         Nothing.
48         """
49 
50         self.name = name
51         self.url_home = url_home
52         self.url_username_format = url_username_format
53 
54         self.username_claimed = username_claimed
55         self.username_unclaimed = secrets.token_urlsafe(32)
56         self.information = information
57         self.is_nsfw  = is_nsfw
58 
59         return
60 
61     def __str__(self):
62         """Convert Object To String.
63 
64         Keyword Arguments:
65         self                   -- This object.
66 
67         Return Value:
68         Nicely formatted string to get information about this object.
69         """
70         
71         return f"{self.name} ({self.url_home})"
72 
73 
74 class SitesInformation:
75     def __init__(self, data_file_path=None):
76         """Create Sites Information Object.
77 
78         Contains information about all supported websites.
79 
80         Keyword Arguments:
81         self                   -- This object.
82         data_file_path         -- String which indicates path to data file.
83                                   The file name must end in ".json".
84 
85                                   There are 3 possible formats:
86                                    * Absolute File Format
87                                      For example, "c:/stuff/data.json".
88                                    * Relative File Format
89                                      The current working directory is used
90                                      as the context.
91                                      For example, "data.json".
92                                    * URL Format
93                                      For example,
94                                      "https://example.com/data.json", or
95                                      "http://example.com/data.json".
96 
97                                   An exception will be thrown if the path
98                                   to the data file is not in the expected
99                                   format, or if there was any problem loading
100                                   the file.
101 
102                                   If this option is not specified, then a
103                                   default site list will be used.
104 
105         Return Value:
106         Nothing.
107         """
108 
109         if not data_file_path:
110             # The default data file is the live data.json which is in the GitHub repo. The reason why we are using
111             # this instead of the local one is so that the user has the most up-to-date data. This prevents
112             # users from creating issue about false positives which has already been fixed or having outdated data
113             data_file_path = "https://raw.githubusercontent.com/sherlock-project/sherlock/master/sherlock/resources/data.json"
114 
115         # Ensure that specified data file has correct extension.
116         if not data_file_path.lower().endswith(".json"):
117             raise FileNotFoundError(f"Incorrect JSON file extension for data file '{data_file_path}'.")
118 
119         # if "http://"  == data_file_path[:7].lower() or "https://" == data_file_path[:8].lower():
120         if data_file_path.lower().startswith("http"):
121             # Reference is to a URL.
122             try:
123                 response = requests.get(url=data_file_path)
124             except Exception as error:
125                 raise FileNotFoundError(
126                     f"Problem while attempting to access data file URL '{data_file_path}':  {error}"
127                 )
128 
129             if response.status_code != 200:
130                 raise FileNotFoundError(f"Bad response while accessing "
131                                         f"data file URL '{data_file_path}'."
132                                         )
133             try:
134                 site_data = response.json()
135             except Exception as error:
136                 raise ValueError(
137                     f"Problem parsing json contents at '{data_file_path}':  {error}."
138                 )
139 
140         else:
141             # Reference is to a file.
142             try:
143                 with open(data_file_path, "r", encoding="utf-8") as file:
144                     try:
145                         site_data = json.load(file)
146                     except Exception as error:
147                         raise ValueError(
148                             f"Problem parsing json contents at '{data_file_path}':  {error}."
149                         )
150 
151             except FileNotFoundError:
152                 raise FileNotFoundError(f"Problem while attempting to access "
153                                         f"data file '{data_file_path}'."
154                                         )
155 
156         self.sites = {}
157 
158         # Add all site information from the json file to internal site list.
159         for site_name in site_data:
160             try:
161 
162                 self.sites[site_name] = \
163                     SiteInformation(site_name,
164                                     site_data[site_name]["urlMain"],
165                                     site_data[site_name]["url"],
166                                     site_data[site_name]["username_claimed"],
167                                     site_data[site_name],
168                                     site_data[site_name].get("isNSFW",False)
169 
170                                     )
171             except KeyError as error:
172                 raise ValueError(
173                     f"Problem parsing json contents at '{data_file_path}':  Missing attribute {error}."
174                 )
175 
176         return
177 
178     def remove_nsfw_sites(self):
179         """
180         Remove NSFW sites from the sites, if isNSFW flag is true for site
181 
182         Keyword Arguments:
183         self                   -- This object.
184 
185         Return Value:
186         None
187         """
188         sites = {}
189         for site in self.sites:
190             if self.sites[site].is_nsfw:
191                 continue
192             sites[site] = self.sites[site]  
193         self.sites =  sites
194 
195     def site_name_list(self):
196         """Get Site Name List.
197 
198         Keyword Arguments:
199         self                   -- This object.
200 
201         Return Value:
202         List of strings containing names of sites.
203         """
204 
205         return sorted([site.name for site in self], key=str.lower)
206 
207     def __iter__(self):
208         """Iterator For Object.
209 
210         Keyword Arguments:
211         self                   -- This object.
212 
213         Return Value:
214         Iterator for sites object.
215         """
216 
217         for site_name in self.sites:
218             yield self.sites[site_name]
219 
220     def __len__(self):
221         """Length For Object.
222 
223         Keyword Arguments:
224         self                   -- This object.
225 
226         Return Value:
227         Length of sites object.
228         """
229         return len(self.sites)
230
```

Path: `site_list.py`
Content:
```
1 #!/usr/bin/env python
2 # This module generates the listing of supported sites which can be found in
3 # sites.md. It also organizes all the sites in alphanumeric order
4 import json
5 
6 # Read the data.json file
7 with open("sherlock/resources/data.json", "r", encoding="utf-8") as data_file:
8     data = json.load(data_file)
9 
10 # Sort the social networks in alphanumeric order
11 social_networks = sorted(data.items())
12 
13 # Write the list of supported sites to sites.md
14 with open("sites.md", "w") as site_file:
15     site_file.write(f"## List Of Supported Sites ({len(social_networks)} Sites In Total!)\n")
16     for social_network, info in social_networks:
17         url_main = info["urlMain"]
18         is_nsfw = "**(NSFW)**" if info.get("isNSFW") else ""
19         site_file.write(f"1. ![](https://www.google.com/s2/favicons?domain={url_main}) [{social_network}]({url_main}) {is_nsfw}\n")
20 
21 # Overwrite the data.json file with sorted data
22 with open("sherlock/resources/data.json", "w") as data_file:
23     sorted_data = json.dumps(data, indent=2, sort_keys=True)
24     data_file.write(sorted_data)
25     data_file.write("\n")
26 
27 print("Finished updating supported site listing!")
28
```
--- END FILES ---

Please first localize the bug based on the issue statement, and then generate a patch according to the `git diff` format fenced by three backticks.

Here is an example:

```diff
diff --git a/examples/server_async.py b/examples/server_async.py
--- a/examples/server_async.py
+++ b/examples/server_async.py
@@ -313,4 +313,4 @@
 
 
 if __name__ == "__main__":
-    asyncio.run(run_async_server("."), debug=True)
+    asyncio.run(run_async_server(), debug=True)
diff --git a/examples/server_sync.py b/examples/server_sync.py
--- a/examples/server_sync.py
+++ b/examples/server_sync.py
@@ -313,5 +313,5 @@
 
 
 if __name__ == "__main__":
-    server = run_sync_server(".")
+    server = run_sync_server()
     server.shutdown()

```

diff --git a/sherlock/sites.py b/sherlock/sites.py
--- a/sherlock/sites.py
+++ b/sherlock/sites.py
@@ -152,6 +152,11 @@
                 raise FileNotFoundError(f"Problem while attempting to access "
                                         f"data file '{data_file_path}'."
                                         )
+        
+        try:
+            site_data.pop('$schema')
+        except:
+            pass
 
         self.sites = {}
 
diff --git a/site_list.py b/site_list.py
--- a/site_list.py
+++ b/site_list.py
@@ -5,10 +5,14 @@
 
 # Read the data.json file
 with open("sherlock/resources/data.json", "r", encoding="utf-8") as data_file:
-    data = json.load(data_file)
+    data: dict = json.load(data_file)
+
+# Removes schema-specific keywords for proper processing
+social_networks: dict = dict(data)
+social_networks.pop('$schema')
 
 # Sort the social networks in alphanumeric order
-social_networks = sorted(data.items())
+social_networks: list = sorted(social_networks.items())
 
 # Write the list of supported sites to sites.md
 with open("sites.md", "w") as site_file:

{"golden_diff": "diff --git a/sherlock/sites.py b/sherlock/sites.py\n--- a/sherlock/sites.py\n+++ b/sherlock/sites.py\n@@ -152,6 +152,11 @@\n                 raise FileNotFoundError(f\"Problem while attempting to access \"\n                                         f\"data file '{data_file_path}'.\"\n                                         )\n+        \n+        try:\n+            site_data.pop('$schema')\n+        except:\n+            pass\n \n         self.sites = {}\n \ndiff --git a/site_list.py b/site_list.py\n--- a/site_list.py\n+++ b/site_list.py\n@@ -5,10 +5,14 @@\n \n # Read the data.json file\n with open(\"sherlock/resources/data.json\", \"r\", encoding=\"utf-8\") as data_file:\n-    data = json.load(data_file)\n+    data: dict = json.load(data_file)\n+\n+# Removes schema-specific keywords for proper processing\n+social_networks: dict = dict(data)\n+social_networks.pop('$schema')\n \n # Sort the social networks in alphanumeric order\n-social_networks = sorted(data.items())\n+social_networks: list = sorted(social_networks.items())\n \n # Write the list of supported sites to sites.md\n with open(\"sites.md\", \"w\") as site_file:\n", "issue": "feat: add json schema validation\nAdds a JSON Schema that validates `data.json` and `removed_sites.json`. I've already validated the existing data with it and can confirm both files pass. \ud83d\udc4d\ud83c\udffe \r\n\r\nPrepends the following property to both files:\r\n```json\r\n{\r\n  \"$schema\": \"{path/to/json-schema}\"\r\n}\r\n```\r\n\r\nRemoves the `rank` property from `removed_sites.json` and documentation as this is no longer relevant.\r\n\r\nRemoves the \"noPeriod\" property from Jimdo as this appears to serve on purpose.\r\n\r\n---\r\n\r\nSorry, also a question. What is the purpose of `responseUrl`? I just realized, this property is not referenced whatsoever in the actual application. \ud83e\udd14 Can this property be removed in the dataset and schema?\r\n\r\nLater I do want to make it stricter, i.e. prevent someone from using one `errorType` with the wrong additional field, i.e.\r\n```json\r\n{ \r\n  \"errorMsg\": \"Not Found\",\r\n  \"errorType\": \"status_code\"\r\n} \r\n```\r\n\r\nI figured I'd leave it like this for now and find out what `responseUrl` is about. \ud83e\udd14 \r\n\r\n### Related\r\n* Closes https://github.com/sherlock-project/sherlock/issues/1336\r\n\r\nThe goal of this is to improve the experience for developers and leave less room for human-error. It also enforces that we document and provide examples of each field, unlike the current Wiki which tends to get outdated.\r\n\r\n![image](https://user-images.githubusercontent.com/22801583/167285185-11711582-a8d1-4157-a5e9-4d590d6e71b6.png)\r\n\r\n![image](https://user-images.githubusercontent.com/22801583/167285139-15b5d326-5813-45df-8987-dfd6f7a236ad.png)\r\n\r\n![image](https://user-images.githubusercontent.com/22801583/167285147-e759a759-2734-4811-b972-ee75ed7dd530.png)    \r\n\r\n\r\n\n", "before_files": [{"content": "\"\"\"Sherlock Sites Information Module\n\nThis module supports storing information about websites.\nThis is the raw data that will be used to search for usernames.\n\"\"\"\nimport json\nimport requests\nimport secrets\n\nclass SiteInformation:\n    def __init__(self, name, url_home, url_username_format, username_claimed,\n                information, is_nsfw, username_unclaimed=secrets.token_urlsafe(10)):\n        \"\"\"Create Site Information Object.\n\n        Contains information about a specific website.\n\n        Keyword Arguments:\n        self                   -- This object.\n        name                   -- String which identifies site.\n        url_home               -- String containing URL for home of site.\n        url_username_format    -- String containing URL for Username format\n                                  on site.\n                                  NOTE:  The string should contain the\n                                         token \"{}\" where the username should\n                                         be substituted.  For example, a string\n                                         of \"https://somesite.com/users/{}\"\n                                         indicates that the individual\n                                         usernames would show up under the\n                                         \"https://somesite.com/users/\" area of\n                                         the website.\n        username_claimed       -- String containing username which is known\n                                  to be claimed on website.\n        username_unclaimed     -- String containing username which is known\n                                  to be unclaimed on website.\n        information            -- Dictionary containing all known information\n                                  about website.\n                                  NOTE:  Custom information about how to\n                                         actually detect the existence of the\n                                         username will be included in this\n                                         dictionary.  This information will\n                                         be needed by the detection method,\n                                         but it is only recorded in this\n                                         object for future use.\n        is_nsfw                -- Boolean indicating if site is Not Safe For Work.\n\n        Return Value:\n        Nothing.\n        \"\"\"\n\n        self.name = name\n        self.url_home = url_home\n        self.url_username_format = url_username_format\n\n        self.username_claimed = username_claimed\n        self.username_unclaimed = secrets.token_urlsafe(32)\n        self.information = information\n        self.is_nsfw  = is_nsfw\n\n        return\n\n    def __str__(self):\n        \"\"\"Convert Object To String.\n\n        Keyword Arguments:\n        self                   -- This object.\n\n        Return Value:\n        Nicely formatted string to get information about this object.\n        \"\"\"\n        \n        return f\"{self.name} ({self.url_home})\"\n\n\nclass SitesInformation:\n    def __init__(self, data_file_path=None):\n        \"\"\"Create Sites Information Object.\n\n        Contains information about all supported websites.\n\n        Keyword Arguments:\n        self                   -- This object.\n        data_file_path         -- String which indicates path to data file.\n                                  The file name must end in \".json\".\n\n                                  There are 3 possible formats:\n                                   * Absolute File Format\n                                     For example, \"c:/stuff/data.json\".\n                                   * Relative File Format\n                                     The current working directory is used\n                                     as the context.\n                                     For example, \"data.json\".\n                                   * URL Format\n                                     For example,\n                                     \"https://example.com/data.json\", or\n                                     \"http://example.com/data.json\".\n\n                                  An exception will be thrown if the path\n                                  to the data file is not in the expected\n                                  format, or if there was any problem loading\n                                  the file.\n\n                                  If this option is not specified, then a\n                                  default site list will be used.\n\n        Return Value:\n        Nothing.\n        \"\"\"\n\n        if not data_file_path:\n            # The default data file is the live data.json which is in the GitHub repo. The reason why we are using\n            # this instead of the local one is so that the user has the most up-to-date data. This prevents\n            # users from creating issue about false positives which has already been fixed or having outdated data\n            data_file_path = \"https://raw.githubusercontent.com/sherlock-project/sherlock/master/sherlock/resources/data.json\"\n\n        # Ensure that specified data file has correct extension.\n        if not data_file_path.lower().endswith(\".json\"):\n            raise FileNotFoundError(f\"Incorrect JSON file extension for data file '{data_file_path}'.\")\n\n        # if \"http://\"  == data_file_path[:7].lower() or \"https://\" == data_file_path[:8].lower():\n        if data_file_path.lower().startswith(\"http\"):\n            # Reference is to a URL.\n            try:\n                response = requests.get(url=data_file_path)\n            except Exception as error:\n                raise FileNotFoundError(\n                    f\"Problem while attempting to access data file URL '{data_file_path}':  {error}\"\n                )\n\n            if response.status_code != 200:\n                raise FileNotFoundError(f\"Bad response while accessing \"\n                                        f\"data file URL '{data_file_path}'.\"\n                                        )\n            try:\n                site_data = response.json()\n            except Exception as error:\n                raise ValueError(\n                    f\"Problem parsing json contents at '{data_file_path}':  {error}.\"\n                )\n\n        else:\n            # Reference is to a file.\n            try:\n                with open(data_file_path, \"r\", encoding=\"utf-8\") as file:\n                    try:\n                        site_data = json.load(file)\n                    except Exception as error:\n                        raise ValueError(\n                            f\"Problem parsing json contents at '{data_file_path}':  {error}.\"\n                        )\n\n            except FileNotFoundError:\n                raise FileNotFoundError(f\"Problem while attempting to access \"\n                                        f\"data file '{data_file_path}'.\"\n                                        )\n\n        self.sites = {}\n\n        # Add all site information from the json file to internal site list.\n        for site_name in site_data:\n            try:\n\n                self.sites[site_name] = \\\n                    SiteInformation(site_name,\n                                    site_data[site_name][\"urlMain\"],\n                                    site_data[site_name][\"url\"],\n                                    site_data[site_name][\"username_claimed\"],\n                                    site_data[site_name],\n                                    site_data[site_name].get(\"isNSFW\",False)\n\n                                    )\n            except KeyError as error:\n                raise ValueError(\n                    f\"Problem parsing json contents at '{data_file_path}':  Missing attribute {error}.\"\n                )\n\n        return\n\n    def remove_nsfw_sites(self):\n        \"\"\"\n        Remove NSFW sites from the sites, if isNSFW flag is true for site\n\n        Keyword Arguments:\n        self                   -- This object.\n\n        Return Value:\n        None\n        \"\"\"\n        sites = {}\n        for site in self.sites:\n            if self.sites[site].is_nsfw:\n                continue\n            sites[site] = self.sites[site]  \n        self.sites =  sites\n\n    def site_name_list(self):\n        \"\"\"Get Site Name List.\n\n        Keyword Arguments:\n        self                   -- This object.\n\n        Return Value:\n        List of strings containing names of sites.\n        \"\"\"\n\n        return sorted([site.name for site in self], key=str.lower)\n\n    def __iter__(self):\n        \"\"\"Iterator For Object.\n\n        Keyword Arguments:\n        self                   -- This object.\n\n        Return Value:\n        Iterator for sites object.\n        \"\"\"\n\n        for site_name in self.sites:\n            yield self.sites[site_name]\n\n    def __len__(self):\n        \"\"\"Length For Object.\n\n        Keyword Arguments:\n        self                   -- This object.\n\n        Return Value:\n        Length of sites object.\n        \"\"\"\n        return len(self.sites)\n", "path": "sherlock/sites.py"}, {"content": "#!/usr/bin/env python\n# This module generates the listing of supported sites which can be found in\n# sites.md. It also organizes all the sites in alphanumeric order\nimport json\n\n# Read the data.json file\nwith open(\"sherlock/resources/data.json\", \"r\", encoding=\"utf-8\") as data_file:\n    data = json.load(data_file)\n\n# Sort the social networks in alphanumeric order\nsocial_networks = sorted(data.items())\n\n# Write the list of supported sites to sites.md\nwith open(\"sites.md\", \"w\") as site_file:\n    site_file.write(f\"## List Of Supported Sites ({len(social_networks)} Sites In Total!)\\n\")\n    for social_network, info in social_networks:\n        url_main = info[\"urlMain\"]\n        is_nsfw = \"**(NSFW)**\" if info.get(\"isNSFW\") else \"\"\n        site_file.write(f\"1. ![](https://www.google.com/s2/favicons?domain={url_main}) [{social_network}]({url_main}) {is_nsfw}\\n\")\n\n# Overwrite the data.json file with sorted data\nwith open(\"sherlock/resources/data.json\", \"w\") as data_file:\n    sorted_data = json.dumps(data, indent=2, sort_keys=True)\n    data_file.write(sorted_data)\n    data_file.write(\"\\n\")\n\nprint(\"Finished updating supported site listing!\")\n", "path": "site_list.py"}], "after_files": [{"content": "\"\"\"Sherlock Sites Information Module\n\nThis module supports storing information about websites.\nThis is the raw data that will be used to search for usernames.\n\"\"\"\nimport json\nimport requests\nimport secrets\n\nclass SiteInformation:\n    def __init__(self, name, url_home, url_username_format, username_claimed,\n                information, is_nsfw, username_unclaimed=secrets.token_urlsafe(10)):\n        \"\"\"Create Site Information Object.\n\n        Contains information about a specific website.\n\n        Keyword Arguments:\n        self                   -- This object.\n        name                   -- String which identifies site.\n        url_home               -- String containing URL for home of site.\n        url_username_format    -- String containing URL for Username format\n                                  on site.\n                                  NOTE:  The string should contain the\n                                         token \"{}\" where the username should\n                                         be substituted.  For example, a string\n                                         of \"https://somesite.com/users/{}\"\n                                         indicates that the individual\n                                         usernames would show up under the\n                                         \"https://somesite.com/users/\" area of\n                                         the website.\n        username_claimed       -- String containing username which is known\n                                  to be claimed on website.\n        username_unclaimed     -- String containing username which is known\n                                  to be unclaimed on website.\n        information            -- Dictionary containing all known information\n                                  about website.\n                                  NOTE:  Custom information about how to\n                                         actually detect the existence of the\n                                         username will be included in this\n                                         dictionary.  This information will\n                                         be needed by the detection method,\n                                         but it is only recorded in this\n                                         object for future use.\n        is_nsfw                -- Boolean indicating if site is Not Safe For Work.\n\n        Return Value:\n        Nothing.\n        \"\"\"\n\n        self.name = name\n        self.url_home = url_home\n        self.url_username_format = url_username_format\n\n        self.username_claimed = username_claimed\n        self.username_unclaimed = secrets.token_urlsafe(32)\n        self.information = information\n        self.is_nsfw  = is_nsfw\n\n        return\n\n    def __str__(self):\n        \"\"\"Convert Object To String.\n\n        Keyword Arguments:\n        self                   -- This object.\n\n        Return Value:\n        Nicely formatted string to get information about this object.\n        \"\"\"\n        \n        return f\"{self.name} ({self.url_home})\"\n\n\nclass SitesInformation:\n    def __init__(self, data_file_path=None):\n        \"\"\"Create Sites Information Object.\n\n        Contains information about all supported websites.\n\n        Keyword Arguments:\n        self                   -- This object.\n        data_file_path         -- String which indicates path to data file.\n                                  The file name must end in \".json\".\n\n                                  There are 3 possible formats:\n                                   * Absolute File Format\n                                     For example, \"c:/stuff/data.json\".\n                                   * Relative File Format\n                                     The current working directory is used\n                                     as the context.\n                                     For example, \"data.json\".\n                                   * URL Format\n                                     For example,\n                                     \"https://example.com/data.json\", or\n                                     \"http://example.com/data.json\".\n\n                                  An exception will be thrown if the path\n                                  to the data file is not in the expected\n                                  format, or if there was any problem loading\n                                  the file.\n\n                                  If this option is not specified, then a\n                                  default site list will be used.\n\n        Return Value:\n        Nothing.\n        \"\"\"\n\n        if not data_file_path:\n            # The default data file is the live data.json which is in the GitHub repo. The reason why we are using\n            # this instead of the local one is so that the user has the most up-to-date data. This prevents\n            # users from creating issue about false positives which has already been fixed or having outdated data\n            data_file_path = \"https://raw.githubusercontent.com/sherlock-project/sherlock/master/sherlock/resources/data.json\"\n\n        # Ensure that specified data file has correct extension.\n        if not data_file_path.lower().endswith(\".json\"):\n            raise FileNotFoundError(f\"Incorrect JSON file extension for data file '{data_file_path}'.\")\n\n        # if \"http://\"  == data_file_path[:7].lower() or \"https://\" == data_file_path[:8].lower():\n        if data_file_path.lower().startswith(\"http\"):\n            # Reference is to a URL.\n            try:\n                response = requests.get(url=data_file_path)\n            except Exception as error:\n                raise FileNotFoundError(\n                    f\"Problem while attempting to access data file URL '{data_file_path}':  {error}\"\n                )\n\n            if response.status_code != 200:\n                raise FileNotFoundError(f\"Bad response while accessing \"\n                                        f\"data file URL '{data_file_path}'.\"\n                                        )\n            try:\n                site_data = response.json()\n            except Exception as error:\n                raise ValueError(\n                    f\"Problem parsing json contents at '{data_file_path}':  {error}.\"\n                )\n\n        else:\n            # Reference is to a file.\n            try:\n                with open(data_file_path, \"r\", encoding=\"utf-8\") as file:\n                    try:\n                        site_data = json.load(file)\n                    except Exception as error:\n                        raise ValueError(\n                            f\"Problem parsing json contents at '{data_file_path}':  {error}.\"\n                        )\n\n            except FileNotFoundError:\n                raise FileNotFoundError(f\"Problem while attempting to access \"\n                                        f\"data file '{data_file_path}'.\"\n                                        )\n        \n        try:\n            site_data.pop('$schema')\n        except:\n            pass\n\n        self.sites = {}\n\n        # Add all site information from the json file to internal site list.\n        for site_name in site_data:\n            try:\n\n                self.sites[site_name] = \\\n                    SiteInformation(site_name,\n                                    site_data[site_name][\"urlMain\"],\n                                    site_data[site_name][\"url\"],\n                                    site_data[site_name][\"username_claimed\"],\n                                    site_data[site_name],\n                                    site_data[site_name].get(\"isNSFW\",False)\n\n                                    )\n            except KeyError as error:\n                raise ValueError(\n                    f\"Problem parsing json contents at '{data_file_path}':  Missing attribute {error}.\"\n                )\n\n        return\n\n    def remove_nsfw_sites(self):\n        \"\"\"\n        Remove NSFW sites from the sites, if isNSFW flag is true for site\n\n        Keyword Arguments:\n        self                   -- This object.\n\n        Return Value:\n        None\n        \"\"\"\n        sites = {}\n        for site in self.sites:\n            if self.sites[site].is_nsfw:\n                continue\n            sites[site] = self.sites[site]  \n        self.sites =  sites\n\n    def site_name_list(self):\n        \"\"\"Get Site Name List.\n\n        Keyword Arguments:\n        self                   -- This object.\n\n        Return Value:\n        List of strings containing names of sites.\n        \"\"\"\n\n        return sorted([site.name for site in self], key=str.lower)\n\n    def __iter__(self):\n        \"\"\"Iterator For Object.\n\n        Keyword Arguments:\n        self                   -- This object.\n\n        Return Value:\n        Iterator for sites object.\n        \"\"\"\n\n        for site_name in self.sites:\n            yield self.sites[site_name]\n\n    def __len__(self):\n        \"\"\"Length For Object.\n\n        Keyword Arguments:\n        self                   -- This object.\n\n        Return Value:\n        Length of sites object.\n        \"\"\"\n        return len(self.sites)\n", "path": "sherlock/sites.py"}, {"content": "#!/usr/bin/env python\n# This module generates the listing of supported sites which can be found in\n# sites.md. It also organizes all the sites in alphanumeric order\nimport json\n\n# Read the data.json file\nwith open(\"sherlock/resources/data.json\", \"r\", encoding=\"utf-8\") as data_file:\n    data: dict = json.load(data_file)\n\n# Removes schema-specific keywords for proper processing\nsocial_networks: dict = dict(data)\nsocial_networks.pop('$schema')\n\n# Sort the social networks in alphanumeric order\nsocial_networks: list = sorted(social_networks.items())\n\n# Write the list of supported sites to sites.md\nwith open(\"sites.md\", \"w\") as site_file:\n    site_file.write(f\"## List Of Supported Sites ({len(social_networks)} Sites In Total!)\\n\")\n    for social_network, info in social_networks:\n        url_main = info[\"urlMain\"]\n        is_nsfw = \"**(NSFW)**\" if info.get(\"isNSFW\") else \"\"\n        site_file.write(f\"1. ![](https://www.google.com/s2/favicons?domain={url_main}) [{social_network}]({url_main}) {is_nsfw}\\n\")\n\n# Overwrite the data.json file with sorted data\nwith open(\"sherlock/resources/data.json\", \"w\") as data_file:\n    sorted_data = json.dumps(data, indent=2, sort_keys=True)\n    data_file.write(sorted_data)\n    data_file.write(\"\\n\")\n\nprint(\"Finished updating supported site listing!\")\n", "path": "site_list.py"}]}