edu.stanford.nlp.util

Class GoogleNGramsSQLBacked

java.lang.Object

edu.stanford.nlp.util.GoogleNGramsSQLBacked

public class GoogleNGramsSQLBacked
extends java.lang.Object

To query Google Ngrams counts from SQL in a memory efficient manner. To get count of a phrase, use GoogleNGramsSQLBacked.getCount(phrase). Set this class options using Execution.fillOptions(GoogleNGramsSQLBacked.class, props);

Author:

Sonal Gupta

Constructor Summary

Constructors

Constructor and Description
GoogleNGramsSQLBacked()

Method Summary

Modifier and Type	Method and Description
static void	closeConnection()
static boolean	existsTable(java.lang.String tablename)
static int	get1GramRank(java.lang.String str) Return rank of 1 gram in google ngeams if it is less than 20k.
static long	getCount(java.lang.String str) Queries the SQL tables for the count of the phrase.
static java.util.List<Pair<java.lang.String,java.lang.Long>>	getCounts(java.util.Collection<java.lang.String> strs)
static int	getTotalCount(int ngram) Note that this is really really slow for ngram > 1 TODO: make this fast (if we had been using mysql we could have)
static void	main(java.lang.String[] args)
static void	populateTablesInSQL(java.lang.String dir, java.util.Collection<java.lang.Integer> typesOfPhrases)
static void	setDBName(java.lang.String DBName)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

GoogleNGramsSQLBacked

public GoogleNGramsSQLBacked()

Method Detail

existsTable

public static boolean existsTable(java.lang.String tablename)
                           throws java.sql.SQLException

Throws:

java.sql.SQLException

getCount

public static long getCount(java.lang.String str)

Queries the SQL tables for the count of the phrase. Returns -1 if the phrase doesn't exist

Parameters:

str - : phrase

Returns:

: count, if exists. -1 if not.

Throws:

java.sql.SQLException

getCounts

public static java.util.List<Pair<java.lang.String,java.lang.Long>> getCounts(java.util.Collection<java.lang.String> strs)
                                                                       throws java.sql.SQLException

Throws:

java.sql.SQLException

populateTablesInSQL

public static void populateTablesInSQL(java.lang.String dir,
                                       java.util.Collection<java.lang.Integer> typesOfPhrases)
                                throws java.sql.SQLException

Throws:

java.sql.SQLException

getTotalCount

public static int getTotalCount(int ngram)

Note that this is really really slow for ngram > 1 TODO: make this fast (if we had been using mysql we could have)

get1GramRank

public static int get1GramRank(java.lang.String str)

Return rank of 1 gram in google ngeams if it is less than 20k. Otherwise -1.

closeConnection

public static void closeConnection()
                            throws java.sql.SQLException

Throws:

java.sql.SQLException

main

public static void main(java.lang.String[] args)

setDBName

public static void setDBName(java.lang.String DBName)