data/thb_hard/comp_sci.json

[
    {
        "theorem": "Evidence lower bound",
        "description": "The evidence lower bound (ELBO) is a lower bound on the log-evidence of a model, which is a measure of how well the model fits the data.",
        "difficulty": "Hard",
        "remark": "",
        "subfield": "Machine Learning"
    },
    {
        "theorem": "Viterbi Algorithm",
        "description": "The Viterbi Algorithm is a dynamic programming algorithm used for finding the most likely sequence of hidden states, known as the Viterbi path, in a Hidden Markov Model (HMM). It is named after its inventor, Andrew Viterbi, and is widely used in various applications such as speech recognition, natural language processing, and bioinformatics.\n\nA Hidden Markov Model (HMM) is a statistical model that represents a stochastic process involving a sequence of observable events and hidden states. In an HMM, the observable events are generated by the hidden states, which follow a Markov chain. The Markov chain is characterized by the transition probabilities between hidden states, and the emission probabilities of observable events given the hidden states.\n\nThe Viterbi Algorithm works by finding the most probable path of hidden states that generates the observed sequence of events. It does this by iteratively computing the maximum probability of reaching each state at each time step, considering all possible paths that lead to that state. The algorithm uses dynamic programming to efficiently compute these probabilities and store them in a trellis structure.\n\nHere's a high-level description of the Viterbi Algorithm:\n\n1. Initialization: Set the initial probabilities for each hidden state, considering the initial state probabilities and the emission probabilities for the first observed event.\n\n2. Recursion: For each subsequent observed event, compute the maximum probability of reaching each hidden state, considering all possible previous states and their transition probabilities. Update the emission probabilities for the current observed event.\n\n3. Termination: Identify the hidden state with the highest probability at the last time step.\n\n4. Traceback: Starting from the identified state in the termination step, backtrack through the trellis to find the most probable path of hidden states that generated the observed sequence.\n\nThe Viterbi Algorithm is an efficient and widely used method for decoding the hidden states in a Hidden Markov Model, providing valuable insights into the underlying structure of the stochastic process.",
        "difficulty": "Hard", 
        "remark": "",
        "subfield": "Dynamic Programming"
    },
    {
        "theorem": "Fano's inequality",
        "description": "In information theory, Fano's inequality relates the average information lost in a noisy channel to the probability of the categorization error.",
        "difficulty": "Hard",
        "remark": "",
        "subfield": "Information Theory"
    },
    {
        "theorem": "Message Passing algorithm", 
        "description": "Message passing algorithm is an iterative decoding algorithm factorizes the global function of many variables into product of simpler local functions, whose arguments are the subset of variables.",
        "difficulty": "Hard",
        "remark": "",
        "subfield": "Machine Learning"
    },
    {
        "theorem": "Maximal Planar Graph",
        "description": "A maximal planar graph is a graph which can be embedded in the plane such that every face of the graph is a triangle.",
        "difficulty": "Hard",
        "remark": "",
        "subfield": "Graph Theory"
    },
    {
        "theorem": "Cayley's formula",
        "description": "This formula tells how many trees can be constructed with N vertices.",
        "difficulty": "Hard",
        "remark": "",
        "subfield": "Graph Theory"
    },
    {
        "theorem": "Floyd's Cycle Finding Algorithm",
        "description": "Also known as the tortoise and the hare algorithm, it is a pointer algorithm that uses two pointers which move at different speeds to find a cycle in a sequence.",
        "difficulty": "Hard",
        "remark": "",
        "subfield": "Algorithms"
    },
    {
        "theorem": "Sigma-Delta Modulation",
        "description": "A sigma delta modulator converts this shunt voltage across the resistor, into high-frequency one-bit digital bitstream using oversampling and noise shaping.",
        "difficulty": "Hard",
        "remark": "",
        "subfield": "Digital Signal Processing"
    },
    {
        "theorem": "Kruskal's algorithm",
        "description": "greedy algorithm that sorts the list of edges in the graph by weight.",
        "difficulty": "Hard",
        "remark": "A fundamental algorithm in graph theory. It's used in network design, spanning tree construction, and various optimization problems. Requires understanding of graph theory and greedy algorithms.",
        "subfield": "Graph Theory"
    },
    {
        "theorem": "Prim's algorithm",
        "description": "greedy algorithm that maintains a priority queue of vertices in the graph ordered by connecting edge weight",
        "difficulty": "Hard",
        "remark": "",
        "subfield": "Graph Theory"
    },
    {
        "theorem": "Region growing by pixel aggregation",
        "description": "Region growing by pixel aggregation is a technique used in image processing to segment an image into regions based on the similarity of pixel values.",
        "difficulty": "Hard",
        "remark": "",
        "subfield": "Image Processing"
    },
    {
        "theorem": "Arithmetic coding",
        "description": "Arithmetic coding is a lossless data compression technique that assigns a unique code to each symbol in a message based on its probability of occurrence.",
        "difficulty": "Hard",
        "remark": "",
        "subfield": "Data Compression"
    },
    {
        "theorem": "Expectation–maximization (EM) algorithm",
        "description": "an expectation–maximization (EM) algorithm is an iterative method to find (local) maximum likelihood or maximum a posteriori (MAP) estimates of parameters in statistical models, where the model depends on unobserved latent variables.",
        "difficulty": "Hard",
        "remark": "",
        "subfield": "Machine Learning"
    },
    {
        "theorem": "Differential entropy",
        "description": "Differential entropy, also known as continuous entropy, is a concept in information theory that extends the idea of entropy from discrete random variables to continuous random variables. Entropy, in general, is a measure of the uncertainty or randomness associated with a random variable. In the context of information theory, it quantifies the average amount of information required to describe the outcome of a random variable.\n\nFor discrete random variables, entropy is well-defined using the Shannon entropy formula, which sums the product of the probability of each outcome and the logarithm of its reciprocal probability. However, for continuous random variables, the probability of any specific outcome is zero, making the Shannon entropy formula inapplicable.\n\nDifferential entropy addresses this issue by considering the probability density function (pdf) of a continuous random variable instead of the probabilities of individual outcomes. The differential entropy H(X) of a continuous random variable X with a probability density function f(x) is defined as:\n\nH(X) = - \u222b f(x) * log(f(x)) dx\n\nwhere the integral is taken over the entire range of the random variable X, and log is the logarithm base 2 (or any other base, depending on the desired unit of measurement for entropy).\n\nDifferential entropy can be interpreted as the average amount of information required to describe the outcome of a continuous random variable with a given probability density function. However, unlike the entropy of discrete random variables, differential entropy can be negative, which occurs when the probability density function is highly concentrated around certain values.\n\nIt is important to note that differential entropy is not a direct extension of discrete entropy, and some properties of discrete entropy do not hold for differential entropy. For example, differential entropy is not invariant under changes of variables or coordinate transformations, whereas discrete entropy is invariant under permutations of the outcomes.",
        "difficulty": "Hard",
        "remark": "",
        "subfield": "Information Theory"
    },
    {
        "theorem": "Kullback–Leibler divergence",
        "description": "a type of statistical distance: a measure of how much a model probability distribution Q is different from a true probability distribution P.",
        "difficulty": "Hard",
        "remark": "",
        "subfield": "Information Theory"
    },
    {
        "theorem": "Principal component analysis",
        "description": "Principal component analysis (PCA) is a statistical method that reduces the dimensions of a dataset to a smaller set of components.",
        "difficulty": "Hard",
        "remark": "",
        "subfield": "Machine Learning"
    },
    {
        "theorem": "Self-attention",
        "description": "Self-attention is a mechanism in neural networks that allows the model to focus on different parts of the input sequence when making predictions.",
        "difficulty": "Hard",
        "remark": "",
        "subfield": "Machine Learning"
    },
    {
        "theorem": "Adversarial training",
        "description": "Adversarial Training is a machine learning technique that is primarily used for improving the robustness of models. It's a process where models are trained with malicious inputs (adversarial examples) alongside the genuine data.",
        "difficulty": "Hard",
        "remark": "",
        "subfield": "Machine Learning"
    },
    {
        "theorem": "Forward-Backward Algorithm",
        "description": "The Forward-Backward Algorithm is a dynamic programming algorithm used in Hidden Markov Models (HMMs) to compute the posterior probabilities of hidden states given a sequence of observations. It is a stochastic process that combines both the forward and backward algorithms to efficiently compute these probabilities.\n\nThe algorithm consists of two main steps:\n\n1. Forward Algorithm:\nThe forward algorithm computes the probability of observing a particular sequence of observations up to a certain time step, given the hidden state at that time step. It calculates the forward probabilities, which are the joint probabilities of the observed sequence and the hidden state at each time step. The forward algorithm uses a recursive approach, where the forward probability at each time step is calculated based on the forward probabilities of the previous time step.\n\n2. Backward Algorithm:\nThe backward algorithm computes the probability of observing the remaining sequence of observations from a certain time step onwards, given the hidden state at that time step. It calculates the backward probabilities, which are the conditional probabilities of the future observations given the hidden state at each time step. Similar to the forward algorithm, the backward algorithm also uses a recursive approach, where the backward probability at each time step is calculated based on the backward probabilities of the next time step.\n\nAfter computing the forward and backward probabilities, the Forward-Backward Algorithm combines these probabilities to calculate the posterior probabilities of the hidden states at each time step. The posterior probability of a hidden state at a particular time step is the probability of that state given the entire sequence of observations. This is computed by multiplying the forward probability and the backward probability for that state at that time step and then normalizing the result.\n\nThe Forward-Backward Algorithm is widely used in various applications, such as speech recognition, natural language processing, and bioinformatics, where the goal is to infer the most likely sequence of hidden states given a sequence of observations.",
        "difficulty": "Hard",
        "remark": "",
        "subfield": "Dynamic Programming"
    },
    {
        "theorem": "Cook-Levin Theorem",
        "description": "In computational complexity theory, the Cook–Levin theorem, also known as Cook's theorem, states that the Boolean satisfiability problem is NP-complete.",
        "difficulty": "Hard",
        "remark": "",
        "subfield": "Computational Complexity"
    }
]