# ---
# jupyter:
#   jupytext:
#     text_representation:
#       extension: .py
#       format_name: light
#       format_version: '1.5'
#       jupytext_version: 1.14.4
#   kernelspec:
#     display_name: Python 3
#     language: python
#     name: python3
# ---

# + [markdown] slideshow={"slide_type": "slide"}
# ### What does XML stand for?

# + [markdown] slideshow={"slide_type": "fragment"}
# XML stands for Extensible Markup Language

# + [markdown] slideshow={"slide_type": "slide"}
# ### What is a markup language?

# + [markdown] slideshow={"slide_type": "fragment"}
# According to _Wikipedia_
#
# > In computer text processing, a markup language is a system for annotating a document in a way that is syntactically distinguishable from the text, meaning when the document is processed for display, the markup language is not shown, and is only used to format the text.

# + [markdown] slideshow={"slide_type": "slide"}
# ### What is the purpose of XML?

# + [markdown] slideshow={"slide_type": "slide"}
# - XML is used for sharing of data.
# - Sends data in a structure format.

# + [markdown] slideshow={"slide_type": "slide"}
# ### Basic Terminology

# + [markdown] slideshow={"slide_type": "slide"}
# #### Tag

# + [markdown] slideshow={"slide_type": "fragment"}
# Generally, strings that `<` and ends with `>`.

# + [markdown] slideshow={"slide_type": "slide"}
# #### Types of tags

# + [markdown] slideshow={"slide_type": "fragment"}
# - start-tag, such as `<section>`

# + [markdown] slideshow={"slide_type": "fragment"}
# - end-tag, such as `</section>`
#

# + [markdown] slideshow={"slide_type": "fragment"}
# - empty-element tag, such as `<line-break />` 

# + [markdown] slideshow={"slide_type": "slide"}
# #### Element

# + [markdown] slideshow={"slide_type": "fragment"}
# - Logical document component that either begins with a start-tag and ends with a matching end-tag
#
# OR
#
# - consists only of an empty-element tag. 
#

# + [markdown] slideshow={"slide_type": "fragment"}
# ##### Examples

# + [markdown] slideshow={"slide_type": "fragment"}
# - `<greeting>Hello, world!</greeting>`.

# + [markdown] slideshow={"slide_type": "fragment"}
# - `<line-break />`.

# + [markdown] slideshow={"slide_type": "slide"}
# #### Attribute

# + slideshow={"slide_type": "fragment"}
- name-value pair that exists within a start-tag or empty-element tag.

# + [markdown] slideshow={"slide_type": "slide"}
# #### Examples

# + [markdown] slideshow={"slide_type": "fragment"}
# - `<img src="foo.jpg" alt="foo" />`

# + [markdown] slideshow={"slide_type": "fragment"}
# <!-- illustrates an attribute with a list of values -->
# - `<div class="inner outer-box"> FooBar</div>`

# + [markdown] slideshow={"slide_type": "slide"}
# ### Example

# + slideshow={"slide_type": "fragment"} active=""
# <?xml version="1.0"?>
# <data>
#     <country name="Liechtenstein">
#         <rank>1</rank>
#         <year>2008</year>
#         <gdppc>141100</gdppc>
#         <neighbor name="Austria" direction="E"/>
#         <neighbor name="Switzerland" direction="W"/>
#     </country>
#     <country name="Singapore">
#         <rank>4</rank>
#         <year>2011</year>
#         <gdppc>59900</gdppc>
#         <neighbor name="Malaysia" direction="N"/>
#     </country>
#     <country name="Panama">
#         <rank>68</rank>
#         <year>2011</year>
#         <gdppc>13600</gdppc>
#         <neighbor name="<NAME>" direction="W"/>
#         <neighbor name="Colombia" direction="E"/>
#     </country>
# </data>

# + [markdown] slideshow={"slide_type": "slide"}
# ### Basic Parsing

# +
import xml.etree.ElementTree as ET

# for parsing document from string
root = ET.fromstring(val)

# for parsing from a file
root = ET.parse('file.xml').getroot()

# + slideshow={"slide_type": "slide"}


# + [markdown] slideshow={"slide_type": "slide"}
# #### Getting Interesting Elements

# + slideshow={"slide_type": "fragment"}
for child in root.iter():
    print(child.tag, child.attrib)

# + slideshow={"slide_type": "fragment"}
for neighbor in root.iter('neighbor'):
    print(neighbor.tag, neighbor.attrib)

# + slideshow={"slide_type": "slide"}


# + [markdown] slideshow={"slide_type": "slide"}
# ### More realistic example

# +
URL = 'https://www.hackadda.com/latest/feed/'

# fetch feed from this URL and extract some interesting data
# like, find urls that contain 'django' in them.


# -



# + [markdown] slideshow={"slide_type": "slide"}
# #### Iterating through every element

# + slideshow={"slide_type": "fragment"}
import xml.etree.ElementTree as ET


tree = ET.iterparse('data.xml')

# first element is event
for _, ele in tree:
    print(ele.tag, ele.attrib)

# + slideshow={"slide_type": "slide"}



# + [markdown] slideshow={"slide_type": "slide"}
# ### SAX(Simple API for XML)

# + [markdown] slideshow={"slide_type": "fragment"}
# - This can be used to parse XML element by element, line by line.

# + [markdown] slideshow={"slide_type": "fragment"}
# - It is generally slower and memory inefficient, prefer `ET.iterparse` instead.
# -



# + [markdown] slideshow={"slide_type": "slide"}
# ### DOM(Document Object Model) API

# + [markdown] slideshow={"slide_type": "fragment"}
# - DOM is a cross-language API from W3C(World Wide Web Consortium) for accessing and modifying XML documents.

# + slideshow={"slide_type": "slide"}
from xml.dom.minidom import parse, parseString


tree = parse(source)

countries = tree.getElementsByTagName('country')

for country in countries:
    tag = country.tagName
    children = country.childNodes
    print('name:', country.getAttribute('name'))

# + [markdown] slideshow={"slide_type": "slide"}
# #### SideNotes

# + [markdown] slideshow={"slide_type": "fragment"}
# - This is generally memory consuming and `xml.etree.ElementTree` is generally preferred over it.

# + slideshow={"slide_type": "slide"}


# + [markdown] slideshow={"slide_type": "slide"}
# ### Security Considerations

# + [markdown] slideshow={"slide_type": "fragment"}
# From the official `python` documentation
#
# <div class="alert alert-danger">
#     
# > Warning: The XML modules are not secure against erroneous or maliciously constructed data. If you need to parse untrusted or unauthenticated data see the [XML vulnerabilities](https://docs.python.org/3/library/xml.html#xml-vulnerabilities) and The [defusedxml](https://docs.python.org/3/library/xml.html#defusedxml-package) Package sections. 
#
# </div>

# + [markdown] slideshow={"slide_type": "slide"}
# ### Other useful parsing libraries

# + [markdown] slideshow={"slide_type": "fragment"}
# - [`xmltodict`](https://docs.python.org/3/library/xml.html#xml-vulnerabilities)
#     - convert XML to JSON like object

# + [markdown] slideshow={"slide_type": "fragment"}
# - [`untangle`](https://github.com/stchris/untangle)  
#     - converts XML to a python like object

# + slideshow={"slide_type": "slide"}


StarcoderdataJupyterScriptsDedupFiltered