<jupyter_start><jupyter_text>Calour Experiment class tutorialLearn about how calour stores the data of an experiment  Setup<jupyter_code>import calour as ca
ca.set_log_level(11)
import numpy as np
import matplotlib.pyplot as plt
%matplotlib notebook<jupyter_output>/Users/amnon/miniconda3/envs/calour/lib/python3.5/site-packages/h5py/__init__.py:36: FutureWarning: Conversion of the second argument of issubdtype from `float` to `np.floating` is deprecated. In future, it will be treated as `np.float64 == np.dtype(float).type`.
  from ._conv import register_converters as _register_converters<jupyter_text>Load the datawe use the chronic fatigue syndrome data from:<NAME>., <NAME>., <NAME>., <NAME>., <NAME>. and <NAME>., 2016.Reduced diversity and altered composition of the gut microbiome in individuals with myalgic encephalomyelitis/chronic fatigue syndrome.Microbiome, 4(1), p.30.<jupyter_code>cfs=ca.read_amplicon('data/chronic-fatigue-syndrome.biom',
                     'data/chronic-fatigue-syndrome.sample.txt',
                     normalize=10000,min_reads=1000)<jupyter_output>2018-03-04 12:36:35 INFO loaded 87 samples, 2129 features
2018-03-04 12:36:35 WARNING These have metadata but do not have data - dropped: {'ERR1331814'}
2018-03-04 12:36:35 INFO After filtering, 87 remaining<jupyter_text>The `Experiment` classCalour stores the experiment as two Pandas.DataFrame (for sample_metadata and feature_metadata) and a (sparse or dense) data matrix.The order in the dataframes and the table is synchronized, so entry number X in the sample_metadata dataframe always corresponds to row X in the data matrix (and similarily entry Y in the feature_metadata always corresponds to column Y in the data matrix).  The \__repr__ of the ExperimentContains the class (in our case - ca.AmpliconExperiment - which is derived from ca.Experiment),the original biom table filename,and how many samples and features does it have.<jupyter_code>print(cfs)<jupyter_output>AmpliconExperiment ("chronic-fatigue-syndrome.biom") with 87 samples, 2129 features<jupyter_text>The per-sample metadata (`Experiment.sample_metadata`)This is a Pandas.DataFrame, with the index being the SampleID, and columns for the sample metadata fields (loaded from the mapping file).Note that Calour also added the "_calour_original_abundance" field<jupyter_code>cfs.sample_metadata<jupyter_output><empty_output><jupyter_text>The per-feature metadata (`Experiment.feature_metadata`)This is a Pandas.DataFrame, with the index being the featureID (usually the sequence), and columns for the feature metadata (usually "taxonomy", and also additional fields added by calour following differential abundance testing)<jupyter_code>cfs.feature_metadata<jupyter_output><empty_output><jupyter_text>Reads table (`Experiment.data`)This is a numpy 2D array or a scipy.Sparse matrix containing the feature X sample reads.Rows are samples, columns are features.<jupyter_code>cfs.data<jupyter_output><empty_output><jupyter_text>Choosing sparse/dense representation of the data When loading the data, it is by default loaded as a scipy.Sparse.CSR matrix (which is more memory efficient for sparse data).We can force Calour to load the data as a dense numpy 2D array using the `sparse=False` parameter in the `read_amplicon()` function.We can also convert between sparse and dense using the `sparse` attribute of the experiment  Convert to dense<jupyter_code>cfs.sparse=False
cfs.data<jupyter_output><empty_output><jupyter_text>convert to sparse<jupyter_code>cfs.sparse=True
cfs.data<jupyter_output><empty_output><jupyter_text>Getting the dataWe can use the `Experiment.get_data()` function to obtain a copy of the data, either as sparse or dense.  sparse=None means keep the original format<jupyter_code>dat = cfs.get_data(sparse=None)
dat<jupyter_output><empty_output><jupyter_text>sparse=True returns a sparse representation of the data (copies if needed)<jupyter_code>dat = cfs.get_data(sparse=True)
dat<jupyter_output><empty_output><jupyter_text>sparse=False returns a dense representation of the data (copies if needed)<jupyter_code>dat = cfs.get_data(sparse=False)
dat<jupyter_output><empty_output><jupyter_text>we can also force copying the data using `copy=True`<jupyter_code>dat = cfs.get_data(sparse=None, copy=False)
dat is cfs.data
dat = cfs.get_data(sparse=None, copy=True)
dat is cfs.data<jupyter_output><empty_output><jupyter_text>getting a single entry based on feature and sample valuesWe can use the \__getitem(sampleid, featureid)\__ attribute.<jupyter_code>cfs['ERR1331815','TACGGAGGATCCGAGCGTTATCCGGATTTATTGGGTTTAAAGGGAGCGTAGGCGGACGCTTAAGTCAGTTGTGAAAGTTTGCGGCTCAACCGTAAAATTGCAGTTGATACTGGGTGTCTTGAGTACAGTAGAGGCAGGCGGAATTCGTGG']<jupyter_output><empty_output><jupyter_text>Saving an Experiment  Save everything (biom table+sample/feature mapping files)By default saves as an HDF5 biom table. We can save as a text biom table insteadusing `fmt="txt"` parameter<jupyter_code>cfs.save('cfs')
!ls cfs*<jupyter_output>cfs.biom        cfs_feature.txt cfs_sample.txt
StarcoderdataJupyterStructuredCleanDedup