# ---
# jupyter:
#   jupytext:
#     text_representation:
#       extension: .py
#       format_name: light
#       format_version: '1.5'
#       jupytext_version: 1.14.4
#   kernelspec:
#     display_name: Python 3
#     language: python
#     name: python3
# ---

# Naive Bayes models are a group of extremely fast and simple classification algorithms that are often suitable for very high-dimensional datasets. Because they are so fast and have so few tunable parameters, they end up being very useful as a quick-and-dirty baseline for a classification problem.

# %matplotlib inline
import numpy as np
import matplotlib.pyplot as plt
import seaborn as sns;
plt.style.use('dark_background')

# # Gaussian naive Bayes
# Perhaps the easiest naive Bayes classifier to understand is Gaussian naive Bayes. In this classifier, the assumption is that data from each label is drawn from a simple Gaussian distribution. Imagine that you have the following data

from sklearn.datasets import make_blobs
X, y = make_blobs(100, 2, centers=2, random_state=2, cluster_std=1.5)
plt.scatter(X[:, 0], X[:, 1], s=60, alpha=0.85, c=y);

# One extremely fast way to create a simple model is to assume that the data is described by a Gaussian distribution with no covariance between dimensions. This model can be fit by simply finding the mean and standard deviation of the points within each label, which is all you need to define such a distribution.
#
# Приняв гаусовские модели для каждой "кучки" данных за генеративные модели этих "кучек" мы можем быстро рассчитать posterior ratio and determine which label is the most probable for a given point.
#
# Делается это через:
# sklearn.naive_bayes.GaussianNB estimator

# +
# создадим генеративные модели для нашего набора данных
from sklearn.naive_bayes import GaussianNB
model = GaussianNB()
model.fit(X, y);

# создадим немного новых данных для теста и оценим их
rng = np.random.RandomState(0)
Xnew = [-6, -14] + [14, 18] * rng.rand(2000, 2)
ynew = model.predict(Xnew)

# визуализируем исходные данные и новые данные
plt.scatter(X[:, 0], X[:, 1], c=y, s=50)
plt.scatter(Xnew[:, 0], Xnew[:, 1], c=ynew, alpha=0.14, s=20);

# +
# мы можем посчитать вероятности попадания "точки" в одну
# из наших категорий с помощью .predict_proba()
yprob = model.predict_proba(Xnew)
yprob[-10:].round(2) # вывели 10 последних значений массива и округлили до 2х чисел после запятой
# 1-й столбец - вероятность попадания точки в группу "1"
# 2-й столбец - вероятность попадания точки в группу "2"
# # model.predict_proba?
# -

# The final classification will only be as good as the model assumptions that lead to it, which is why Gaussian naive Bayes often does not produce very good results. Still, in many cases—especially as the number of features becomes large—this assumption is not detrimental enough to prevent Gaussian naive Bayes from being a useful method.
#
#
# # Multinomial naive Bayes
#
# The Gaussian assumption just described is by no means the only simple assumption that could be used to specify the generative distribution for each label. Another useful example is multinomial naive Bayes, where the features are assumed to be generated from a simple multinomial distribution. The multinomial distribution describes the probability of observing counts among a number of categories, and thus multinomial naive Bayes is most appropriate for features that represent counts or count rates.
#
# The idea is precisely the same as before, except that instead of modeling the data distribution with the best-fit Gaussian, we model the data distribuiton with a best-fit multinomial distribution

# +
# Multinomial naive Bayes часто используется в классификации текста,
# когда features относятся к числу/частоте слов в документах которые классифицируются
# here we will use the sparse word count features from the 20 Newsgroups corpus
# to show how we might classify these short documents into categories

from sklearn.datasets import fetch_20newsgroups

data = fetch_20newsgroups()
data.target_names

# +
# для простоты выберем лишь несколько категорий и 
# загрузим тренировочный и тестовый сеты

categories = ['talk.religion.misc', 'soc.religion.christian',
              'sci.space', 'comp.graphics']
train = fetch_20newsgroups(subset='train', categories=categories)
test = fetch_20newsgroups(subset='test', categories=categories)

# вот так выглядит наш типичный entry from the data
print(train.data[5])
# -

# in order to use this data for machine learning, we need to be able to convert the content of each string into a vector of numbers. For this we will use the TF-IDF vectorizer (discussed in Feature Engineering), and create a pipeline that attaches it to a multinomial naive Bayes classifier:

# +
# pipeline - какая-то магия, надо гуглить

from sklearn.feature_extraction.text import TfidfVectorizer
from sklearn.naive_bayes import MultinomialNB
from sklearn.pipeline import make_pipeline

model = make_pipeline(TfidfVectorizer(), MultinomialNB())

# apply model to training data using the pipeline
model.fit(train.data, train.target)
labels = model.predict(test.data)

# для анализа результатов классификации выведем confusion matrix
from sklearn.metrics import confusion_matrix
mat = confusion_matrix(test.target, labels)
sns.heatmap(mat.T, square=True, annot=True, fmt='d', cbar=False,
            xticklabels=train.target_names, yticklabels=train.target_names)
plt.xlabel('true label')
plt.ylabel('predicted label');


# -

# Evidently, even this very simple classifier can successfully separate space talk from computer talk, but it gets confused between talk about religion and talk about Christianity. This is perhaps an expected area of confusion!
#
# The very cool thing here is that we now have the tools to determine the category for any string, using the predict() method of this pipeline. Here's a quick utility function that will return the prediction for a single string:
#

# +
def predict_category(s, train=train, model=model):
    pred = model.predict([s])
    return train.target_names[pred[0]]

print(predict_category('sending a payload to the ISS'))
print(predict_category('discussing islam vs atheism'))
print(predict_category('determining the screen resolution'))
# -

# Remember that this is nothing more sophisticated than a simple probability model for the (weighted) frequency of each word in the string; nevertheless, the result is striking. Even a very naive algorithm, when used carefully and trained on a large set of high-dimensional data, can be surprisingly effective.
#
#
# # When to use Naive Bayes
#
# Because naive Bayesian classifiers make such stringent assumptions about data, they will generally not perform as well as a more complicated model.
#
# But:
#
# 1) They are fast
#
# 2) They provide straitforward probabilistic prediction
#
# 3) They are easy to interpret
#
# 4) They have very few tunable parameters
#
# Это делает эти методы отличными инструментами для начала анализа и если их результаты будут хороши - мы получим простую, быструю и эффективную модель. Если они не будут хорошо работать, то мы перейдём к более сложным моделям, уже имея некоторое представление о том что хотим видеть.
#
# Лучше всего они работают когда:
#
# 1) When naive assumptions actually match the data (very rare in practice)
#
# 2) For very well-separated categories, when model complexity is less important
#
# 3) For very high-dimensional data, when model complexity is less important
#
# The last two points seem distinct, but they actually are related: as the dimension of a dataset grows, it is much less likely for any two points to be found close together (after all, they must be close in every single dimension to be close overall). This means that clusters in high dimensions tend to be more separated, on average, than clusters in low dimensions, assuming the new dimensions actually add information. For this reason, simplistic classifiers like naive Bayes tend to work as well or better than more complicated classifiers as the dimensionality grows: once you have enough data, even a simple model can be very powerful

StarcoderdataJupyterScriptsDedupFiltered