Analyse Documents in Your Document Corpus

How do you use PoolParty's Corpus Analysis?

After you have uploaded documents to your corpus, you can start a corpus analysis.

  1. Select the Metadata & Statistics tab in the corpus Details View.



  2. Click Start Corpus Analysis in the bottom right-hand corner.

    • You can also start the corpus analysis using the respective entry in the context menu of the corpus node.

    • After the first calculation the button is renamed to Recalculate Corpus Analysis.

  3. The Analyzing Corpus dialogue opens providing information on the status and progress of the analysis process. You can stop the analysis by clicking Cancel.



  4. When the analysis is completed, the dialogue is closed and the updated statistics are displayed in the Corpus Analysis Summary.


If not available, the necessary extraction model for the project is calculated automatically when you start the corpus analysis.

In case you changed your thesaurus, you have to recalculate the extraction model first so the changes take effect in the corpus analysis.


In the Corpus Analysis Settings section of the Metadata & Statistics tab in the Corpus Details View you can enable and disable additional calculations to be performed in the course of a corpus analysis:

  • Calculate Co-Occurrences: activate this check box to find co-occurrences of terms in the corpus with concepts of your thesaurus and of concepts found in the documents. Co-occurring terms are suggested to enrich candidate concepts and thesaurus concepts as synonyms or possible related concepts. Co-occuring concepts are suggested as relations for candidate concepts. (Default: enabled)
  • Word Sense Induction: enable this to disambiguate potentially ambiguous terms. By calculating co-occurrences the context of a word will enable PoolParty to make sure, if a term means a coffee brand instead of a cocktail. The term 'Americano' would be such an example. (Default: disabled) 
  • Add to Corpus Search: activate to use the Corpus Search function in PoolParty. It allows you to quickly check on the corpus and its contents in relation to your thesaurus, as a search application would present it. (Default: disabled) 

The corpus analysis results in the following lists:

Details on the corpus quality find here: Corpus Quality

PoolParty Academy Video

(Duration: 11m09s)
https://conf-doc.poolparty.biz/pp/user-guide-for-knowledge-engineers/advanced-features/corpus-management-overview/analyse-documents-in-your-document-corpus
conf-doc.poolparty.biz
crawl=CC-MAIN-2021-43
26158
application/http; msgtype=response
sha1:ZJNLSPYDASHSRV3KMRMV3RUGDEA4MSLC
<urn:uuid:ffbc5d5b-72e8-4a4a-bacd-294e5ab4eb6c>
2021-10-22T07:45:48Z
91.209.89.15
text/html
sha1:LCDQNZYLMVIJ5W25YMCMDIJZ6U4FVZ3B
<urn:uuid:57d1c3fa-b2b3-428c-9da1-43c1480a2154>
https://conf-doc.poolparty.biz/pp/user-guide-for-knowledge-engineers/advanced-features/corpus-management-overview/analyse-documents-in-your-document-corpus
response
<urn:uuid:c4e3391a-337b-4fe2-a57a-a50a0dd42510>
isPartOf: CC-MAIN-2021-43
publisher: Common Crawl
description: Wide crawl of the web for October 2021
operator: Common Crawl Admin (info@commoncrawl.org)
hostname: ip-10-67-67-74
software: Apache Nutch 1.18 (modified, https://github.com/commoncrawl/nutch/)
robots: checked via crawler-commons 1.2-SNAPSHOT (https://github.com/crawler-commons/crawler-commons)
format: WARC File Format 1.1
conformsTo: https://iipc.github.io/warc-specifications/specifications/warc-format/warc-1.1/