<jupyter_start><jupyter_text>Creating Model Documentation Using Jupyterbook and Intake-esmA common step to any project is documenting your data and your data workflow. Fortunately, open tools in the scientific python ecosystem make that much easier! In this example, we will cover creating your github repo, creating the catalog, visualizing the catalog, and generating a static webpage you can share with collaborators! Fair WarningThis week's post is quite detailed, so just a warning! If you would like to look at the finished product, check out the following* [Github repository with the content built here](https://github.com/mgrover1/cesm-test-data)* [Finished website with content](https://mgrover1.github.io/cesm-test-data/)By the end of this post, we will cover how to build a webpage that looks like this![CESM book page](../images/cesm_book_page.png)  Create your Github RepositoryGo to [Github](https://github.com/) and select "New" in the top lefthand corner next to "Repositories" - this will pull up the following window. Once you are here, go ahead and name your repository!Be sure to add:* Repository name* Description* README* Gitignore (use the python template)* Choose a license ![screen_grab](../images/github_screen_grab.png)  Clone your RepositoryAt this point, you can go ahead and clone your repository! You can either clone to your local machine, or to some Jupyterhub (such as the [NCAR Jupyterhub](https://jupyterhub.ucar.edu)), which will do in this case. Copy the link from GithubCopy the link from Github by clicking on the green "Code" button![Github Clone Link](../images/github_clone_link.png)  Clone to your machine!We want to clone to the repository within the [Jupyterhub](https://jupyterhub.ucar.edu), so once logging on, we open a terminal and paste the link using the following syntax```bashgit clone https://github.com/mgrover1/cesm-test-data.git```  Create a docs directoryNow that you cloned the repository, move into it and create a `docs` directory using the following```bashcd cesm-test-datamkdir docs```  Build your CatalogOpen a new Jupyter Notebook called `model_documentation.ipynb` within the `docs` directory and select a development environment which includes the following:- jupyter-book- ecgtoolsIf you haven't installed these yet, you can use conda and pip (ecgtools is not yet on conda-forge)```bashconda install -c conda-forge jupyter-book intake-esm graphvizpip install ecgtools```In this case, follow the instructions in the [Building an Intake-esm catalog from CESM2 History Files](https://ncar.github.io/esds/posts/ecgtools-history-files-example/) post provides the instructions for building the data catalog  Read the Catalog and Visualize the Components and FrequencyA couple weeks ago, we covered [Creating Visualizations of Intake-ESM Catalogs](https://ncar.github.io/esds/posts/graphviz_example/) which is helpful for understanding how [`Graphviz`](https://graphviz.readthedocs.io/en/stable/manual.html) works! Imports<jupyter_code>import intake
from graphviz import Digraph<jupyter_output><empty_output><jupyter_text>Read in the Test History Catalog<jupyter_code>col = intake.open_esm_datastore('/glade/work/mgrover/cesm-hist-test.json')<jupyter_output><empty_output><jupyter_text>We will assign the dataframe from the catalog to its own variable<jupyter_code>df = col.df<jupyter_output><empty_output><jupyter_text>Visualize the CatalogUsing the `Diagraph` object from the [`Graphviz` library](https://graphviz.readthedocs.io/en/stable/manual.html), we setup a loop to create the visualization using the three categories* Case* Component* Frequency<jupyter_code># Create Digraph object - use the left to right orientation instead of vertical
dot = Digraph(graph_attr={'rankdir': 'LR'})

# Save the catalog as a pdf
dot.format = 'pdf'

# Start counting at one for node numbers
num_node = 1

# Loop through the different cases
for case in df.case.unique():
    case_i = num_node
    dot.node(str(case_i), label=case)
    num_node += 1

    # Loop through the different components in each case
    for component in df.loc[df.case == case].component.unique():
        comp_i = num_node
        dot.node(str(comp_i), label=component)
        dot.edge(str(case_i), str(comp_i))
        num_node += 1

        # Loop through the frequency in each component within each experiment
        for frequency in df.loc[(df.case == case) & (df.component == component)].frequency.unique():
            freq_i = num_node

            # Pull out the the stream information
            stream = df.loc[
                (df.case == case) & (df.component == component) & (df.frequency == frequency)
            ].stream.values[0]

            # Add both stream and frequency information to these bubbles
            dot.node(str(freq_i), label=f'stream: {stream} \n frequency: {frequency}')
            dot.edge(str(comp_i), str(freq_i))
            num_node += 1
        comp_i += 1
    case_i += 1<jupyter_output><empty_output><jupyter_text>Now visualize it in inline by running a cell with just the `dot` object<jupyter_code>dot<jupyter_output><empty_output><jupyter_text>Save the VisualizationIn the block of code above, we specified `dot.format = 'pdf'` which will ensure that when we save the graph, it is in PDF format. Other options include (but not limited to) `svg` and `png`!The `Diagraph` method for saving is `.render()` with the filename in the argument (within the parentheses)<jupyter_code>dot.render('cesm_test_catalog')<jupyter_output><empty_output>
StarcoderdataJupyterStructuredCleanDedup