Modules

Here you will find the documentation of all classes and functions in the MASSAlign library.

massalign.aligners

class massalign.aligners.ParagraphAligner

Bases: object

alignParagraphsFromDocuments()
class massalign.aligners.SentenceAligner

Bases: object

alignSentencesFromParagraphs()
class massalign.aligners.VicinityDrivenParagraphAligner(similarity_model=None, acceptable_similarity=0.3)

Bases: massalign.aligners.ParagraphAligner

Implements the vicinity-driven paragraph alignment algorithm proposed in:

Gustavo H. Paetzold and Lucia Specia. Vicinity-Driven Paragraph and Sentence Alignment for Comparable Corpora. arXiv preprint arXiv:1612.04113 (2016).
  • Parameters:
    • similarity_model: An instance of a class deriving from SimilarityModel.
    • acceptable_similarity: The minimum similarity score between two paragraphs necessary for an alignment to be considered.
alignParagraphsFromDocuments(p1s=[], p2s=[])

Finds alignments between a list of source and target paragraphs that compose a pair of comparable documents. To do so, it produces a similarity matrix between the paragraphs in the source and target list, then finds an alignment path within it using a vicinity-driven approach.

  • Parameters:
    • p1s: A list of source paragraphs. Each paragraph is a list of sentences.
    • p2s: A list of target paragraphs. Each paragraph is a list of sentences.
  • Output:
    • alignment_path: A list of coordinates in the similarity matrix that describes which paragraphs are aligned.
    • aligned_paragraphs: A list containing all pairs of aligned paragraphs.
getActualAlignedParagraphs(p1s, p2s, alignment_path)

Produces a convenient list containing the pairs of aligned paragraphs found between two lists of paragraphs. It concatenates all paragraphs in the “N” side of 1-N and N-1 alignments.

  • Parameters:
    • p1s: A list of source paragraphs. Each paragraph is a list of sentences.
    • p2s: A list of target paragraphs. Each paragraph is a list of sentences.
    • alignment_path: A list of coordinates in the similarity matrix that describes which paragraphs are aligned.
  • Output:
    • aligned_paragraphs: A list containing all pairs of aligned paragraphs.
getNextAlignment(currXY, paragraph_similarities)

Searches for the next alignment during the search for the alignment path.

  • Parameters:
    • currXY: Current coordinate in the similarity matrix from which to continue the search.
    • paragraph_similarities: A matrix containing a similarity score for each paragraph pair.
  • Output:
    • x, y: The coordinate for the next alignment.
getNextSynchronizer(currXY, paragraph_similarities)

If it was impossible to find any suitable alignments within the vicinities specified, then search for the acceptable alignment outside the vicinities that is closest to the current search coordinate.

  • Parameters:
    • currXY: Current coordinate in the similarity matrix from which to continue the search.
    • paragraph_similarities: A matrix containing a similarity score for each paragraph pair.
  • Output:
    • x, y: The coordinate for the next acceptable alignment outside the valid vicinities.
getOriginalParagraph(aligned_nodes, paragraphs)

Concatenates all paragraphs in a 1-N or N-1 alignment.

  • Parameters:
    • aligned_nodes: A list of paragraph indexes from a node in the alignment path.
    • paragraphs: A list of paragraphs. Each paragraph is a list of sentences.
  • Output:
    • text: A list of all the paragraphs in the aligned nodes.
getParagraphAlignmentPath(p1s, p2s, paragraph_similarities)

Searches for the alignment path in a matrix containing similarity scores for all paragraph pairs.

  • Parameters:
    • p1s: A list of source paragraphs. Each paragraph is a list of sentences.
    • p2s: A list of target paragraphs. Each paragraph is a list of sentences.
    • paragraph_similarities: A matrix of dimensions [length(p1s),length(p2s)] containing a similarity score for each paragraph pair.
  • Output:
    • compact_path: A list of coordinates in the similarity matrix that describes which paragraphs are aligned.
class massalign.aligners.VicinityDrivenSentenceAligner(similarity_model=None, acceptable_similarity=0.2, similarity_slack=0.05)

Bases: massalign.aligners.SentenceAligner

Implements the vicinity-driven sentence alignment algorithm proposed in:

Gustavo H. Paetzold and Lucia Specia. Vicinity-Driven Paragraph and Sentence Alignment for Comparable Corpora. arXiv preprint arXiv:1612.04113 (2016).
  • Parameters:
    • similarity_model: An instance of a class deriving from SimilarityModel.
    • acceptable_similarity: The minimum similarity score between two paragraphs necessary for an alignment to be considered.
    • similarity_slack: The maximum amount of similarity that can be lost after each step of incrementing N when finding for a 1-N or N-1 alignment.
alignSentencesFromParagraphs(p1=[], p2=[])

Finds alignments between a list of source and target sentences that compose a pair of aligned paragraphs. To do so, it produces a similarity matrix between the sentences in the source and target sentences, then finds an alignment path within it using a vicinity-driven approach.

  • Parameters:
    • p1: A source paragraph. A paragraph is a list of sentences.
    • p2: A target paragraph. A paragraph is a list of sentences.
  • Output:
    • alignment_path: A list of coordinates in the similarity matrix that describes which sentences are aligned.
    • aligned_sentences: A list containing all pairs of aligned sentences.
findStartingPoint(matrix, p1, p2, startpos)

Searches for a coordinate in the similarity matrix from which to start (or recover) the alignment path search.

  • Parameters:
    • matrix: A matrix of dimensions [length(p1),length(p2)] containing a similarity score for each sentence pair.
    • p1: A source paragraph. A paragraph is a list of sentences.
    • p2: A target paragraph. A paragraph is a list of sentences.
    • startpos: The coordinate from which to start the search for the first alignment in the matrix (usually [-1,-1]).
  • Output:
    • x, y: The coordinate of the next alignment in the similarity matrix.
getActualAlignedSentences(p1, p2, alignment_path)

Produces a convenient list containing the pairs of aligned sentences found between two paragraphs. It concatenates all sentences in the “N” side of 1-N and N-1 alignments.

  • Parameters:
    • p1: A source paragraph. A paragraph is a list of sentences.
    • p2: A target paragraph. A paragraph is a list of sentences.
    • alignment_path: A list of coordinates in the similarity matrix that describes which sentences are aligned.
  • Output:
    • aligned_sentences: A list containing all pairs of aligned sentences.
getBestNextHypothesis(matrix, p1, p2, cbuffer, sbuffer, currXY)

Searches for the next alignment in the alignment matrix.

  • Parameters:
    • matrix: A matrix of dimensions [length(p1),length(p2)] containing a similarity score for each sentence pair.
    • p1: A source paragraph. A paragraph is a list of sentences.
    • p2: A target paragraph. A paragraph is a list of sentences.
    • cbuffer: The concatenation of all aligned sentences in the source side, in case the current alignment is of N-1 kind.
    • sbuffer: The concatenation of all aligned sentences in the target side, in case the current alignment is of 1-N kind.
    • currXY: Current coordinate in the similarity matrix from which to continue the search.
  • Output:
    • x, y: A coordinate in the similarity matrix that represents the next alignment in the alignment path.
getOriginalSentence(indexes, p)

Concatenates a list of sentences.

  • Parameters:
    • indexes: A list of indexes of sentences in a paragraph
    • paragraphs: A paragraph. A paragraph is a list of sentences.
  • Output:
    • sentence: A concatenation of all sentences.
getProbabilityMatrix(p1, p2, sentence_similarities, sentence_indexes)

Produces a similarity matrix between the sentences in the aligned paragraphs

  • Parameters:
    • p1: A source paragraph. A paragraph is a list of sentences.
    • p2: A target paragraph. A paragraph is a list of sentences.
    • sentence_similarities: A matrix containing a similarity score between all possible pairs of sentences in the union of p1 and p2. The matrix’s height and width are equal and equivalent to the number of distinct sentences present in the union of p1 and p2.
    • sentence_indexes: A map connecting each sentence to its numerical index in the sentence_similarities matrix.
  • Output:
    • matrix: A similarity matrix with dimensions [length(p1),length(p2)]
getSentenceAlignmentPath(p1, p2, sentence_similarities, sentence_indexes)

Produces a similarity matrix and searches for the alignment path within it.

  • Parameters:
    • p1: A source paragraph. A paragraph is a list of sentences.
    • p2: A target paragraph. A paragraph is a list of sentences.
    • sentence_similarities: A matrix containing a similarity score between all possible pairs of sentences in the union of p1 and p2. The matrix’s height and width are equal and equivalent to the number of distinct sentences present in the union of p1 and p2.
    • sentence_indexes: A map connecting each sentence to its numerical index in the sentence_similarities matrix.
  • Output:
    • path: A list of coordinates in the similarity matrix that describes which sentences are aligned.

massalign.annotators

class massalign.annotators.SentenceAnnotator

Implements algorithms for annotating transformation operations between parallel sentences.

createConllFiles(annot_file_src, annot_file_ref, annotations, include_clauseop=True, labels_to_print=['B-A', 'B-AC', 'B-D', 'B-DC', 'B-M', 'B-MC', 'B-R', 'B-RM', 'B-RW', 'B-RWM', 'I-A', 'I-AC', 'I-D', 'I-DC', 'I-M', 'I-MC', 'I-R', 'I-RM', 'I-RW', 'I-RWM'])

Creates files in conll format for the transformation annotations of given parallel sentences.

  • Parameters:
    • annot_file_src: The file where to write the annotations for the source sentence.
    • annot_file_ref: The file where to write the annotations for the reference sentence.
    • annotations: A list of dictionaries containing the annotations for the corresponding source and reference sentences.
    • include_clauseop: Indicates whether to print labels corresponding to clause-level operations.
    • labels_to_print: Which transformation operation labels to print. By default, all are printed.
getSentenceAnnotations(src, ref, aligns, src_parse, ref_parse)

Annotates the transformation operations between a pair of aligned sentences (source and reference).

  • Parameters:
    • src: A list of words corresponding to the tokenized source sentence.
    • ref: A list of words corresponding to the tokenized reference sentence.
    • aligns: A string containing the word alignments between source and reference, in Pharaoh format.
    • src_parse: A string containing the constituent parse tree of the source sentence.
    • ref_parse: A string containing the constituent parse tree of the reference sentence.
  • Output:
    • sent_annots: A dictionary containing the token-level annotations for both source and reference sentences.
getSentenceAnnotationsForFile(sents_file, aligns_file, parse_file, verbose=True)

Annotates all the parallel sentences in a given file. Each sentence pair appears in a separate line.

  • Parameters:
    • sents_file: File containing the parallel sentences. Each line in the file contains a source-reference pair, separated by the character |||.
    • aligns_file: File containing the word alignments between each sentence pair. Each line contains the alignments in Pharaoh format.
    • parse_file: File containing the parse trees of the parallel sentences. Every two lines in the file corresponds to a sentence pair (the first is the source parse and the second the reference parse).
    • verbose: Indicates whether to print a message indicating the sentence being annotated or not.
  • Output:
    • file_annots: A list of dictionaries, each of them containing the sentence pair id and the annotations for the corresponding source and reference sentences.

massalign.core

class massalign.core.MASSAligner

A convenience class that allows you to more easily join aligners and annotators.

getParagraphAlignments(paragraphs1=[], paragraphs2=[], paragraph_aligner=None, **kwargs)

Extracts paragraph alignments from two lists of paragraphs from comparable documents.

  • Parameters:
    • paragraphs1: A list of source paragraphs. A paragraph is a list of sentences.
    • paragraphs2: A list of target paragraphs. A paragraph is a list of sentences.
    • paragraph_aligner: An instance of a class deriving from ParagraphAligner.
    • kwargs: Any complementary parameters taken as input by the paragraph aligner.
  • Output:
    • The output is the same produced by the paragraph aligner upon calling the “alignParagraphsFromDocuments” function.
getParagraphsFromDocument(document_path)

Extracts a list of paragraphs from a document.

  • Parameters:
    • document_path: A path to a document of which each line represents a sentence and paragraphs are separated by an empty line.
  • Output:
    • paragraphs: An instance of a class deriving from SimilarityModel.
getSentenceAlignments(paragraph1=[], paragraph2=[], sentence_aligner=None, **kwargs)

Extracts paragraph alignments from two lists of paragraphs from comparable documents.

  • Parameters:
    • paragraph1: A source paragraph. A paragraph is a list of sentences.
    • paragraph2: A target paragraph. A paragraph is a list of sentences.
    • sentence_aligner: An instance of a class deriving from SentenceAligner.
    • kwargs: Any complementary parameters taken as input by the sentence aligner.
  • Output:
    • The output is the same produced by the sentence aligner upon calling the “alignSentencesFromParagraphs” function.
getSentenceAnnotations(sentence1='', sentence2='', sentence_annotator=None, **kwargs)

Produces word-level annotations from two parallel sentences.

  • Parameters:
    • sentence1: A source sentence.
    • sentence2: A target sentence.
    • sentence_annotator: An instance of a class deriving from SentenceAnnotator.
    • kwargs: Any complementary parameters taken as input by the sentence annotator.
  • Output:
    • The output is the same produced by the sentence annotator upon calling the “annotate_sentence” function.
visualizeListOfParagraphAlignments(list_of_paragraph_sets1=[], list_of_paragraph_sets2=[], list_of_alignment_paths=[], **kwargs)

Displays alignments between lists of lists of paragraphs. Each list of paragraphs can represent a document, so this function allows you to see the paragraph alignments of an entire collection of documents through a single interface.

  • Parameters:
    • list_of_paragraph_sets1: A source list of paragraph lists. A paragraph is a list of sentences.
    • list_of_paragraph_sets2: A source list of paragraph lists. A paragraph is a list of sentences.
    • list_of_alignment_paths: List of alignment paths between each pair of paragraph lists.
  • Output:
    • Opens an interface showcasing the aligned paragraphs for each pair of paragraph lists.
visualizeListOfSentenceAlignments(list_of_paragraphs1=[], list_of_paragraphs2=[], list_of_alignment_paths=[], **kwargs)

Displays alignments between the sentences of each pair of paragraphs in a pair of paragraph lists. The interface will showcase the sentence-level alignments between each paragraph pair through a single interface.

  • Parameters:
    • list_of_paragraphs1: A source list of paragraphs. A paragraph is a list of sentences.
    • list_of_paragraphs2: A source list of paragraphs. A paragraph is a list of sentences.
    • list_of_alignment_paths: List of alignment paths between each pair of paragraphs.
  • Output:
    • Opens an interface showcasing the aligned sentences for each pair of paragraphs.
visualizeParagraphAlignments(paragraphs1=[], paragraphs2=[], alignments=[])

Displays alignments between lists of paragraphs.

  • Parameters:
    • paragraphs1: A list of source paragraphs. A paragraph is a list of sentences.
    • paragraphs2: A list of target paragraphs. A paragraph is a list of sentences.
    • alignments: An alignment path between the input paragraph lists.
  • Output:
    • Opens an interface showcasing aligned paragraphs.
visualizeSentenceAlignments(paragraph1=[], paragraph2=[], alignments=[], **kwargs)

Displays sentence alignments between two paragraphs.

  • Parameters:
    • paragraph1: A source paragraph. A paragraph is a list of sentences.
    • paragraph2: A target paragraph. A paragraph is a list of sentences.
    • alignments: An alignment path between the input paragraphs.
  • Output:
    • Opens an interface showcasing sentence alignments for a paragraph pair.
visualizeSentenceAnnotations(sentence1='', sentence2='', word_alignments='', annotations=[], **kwargs)

Displays word-level annotations for a pair of aligned sentences.

  • Parameters:
    • sentence1: A source sentence.
    • sentence2: A target sentence.
    • word_alignments: Word alignments in Pharaoh format.
    • annotations: Word-level annotations produced for the sentence pair.
  • Output:
    • Opens an interface showcasing the word-level annotations for the aligned sentences.

massalign.gui

class massalign.gui.BasicGUI

Bases: massalign.gui.GUI

A basic GUI to showcase alignments and annotations.

displayListOfParagraphAlignments(p1s_list, p2s_list, alignments_list)

Displays alignments between lists of lists of paragraphs. Each list of paragraphs can represent a document, so this function allows you to see the paragraph alignments of an entire collection of documents through a single interface.

  • Parameters:
    • p1s_list: A source list of paragraph lists. A paragraph is a list of sentences.
    • p2s_list: A source list of paragraph lists. A paragraph is a list of sentences.
    • alignments_list: List of alignment paths between each pair of paragraph lists.
  • Output:
    • Opens an interface showcasing the aligned paragraphs for each pair of paragraph lists.
displayListOfSentenceAlignments(p1_list, p2_list, alignments_list)

Displays alignments between the sentences of each pair of paragraphs in a pair of paragraph lists. The interface will showcase the sentence-level alignments between each paragraph pair through a single interface.

  • Parameters:
    • p1_list: A source list of paragraphs. A paragraph is a list of sentences.
    • p2_list: A source list of paragraphs. A paragraph is a list of sentences.
    • alignments_list: List of alignment paths between each pair of paragraphs.
  • Output:
    • Opens an interface showcasing the aligned sentences for each pair of paragraphs.
displayParagraphAlignments(p1s, p2s, alignments)

Displays alignments between two paragraphs.

  • Parameters:
    • p1s: A list of source paragraphs. A paragraph is a list of sentences.
    • p2s: A list of target paragraphs. A paragraph is a list of sentences.
    • alignments: An alignment path between the input paragraph lists.
  • Output:
    • Opens an interface showcasing aligned paragraphs.
displaySentenceAlignments(p1, p2, alignments)

Display sentence alignments between two paragraphs.

  • Parameters:
    • p1: A source paragraph. A paragraph is a list of sentences.
    • p2: A target paragraph. A paragraph is a list of sentences.
    • alignments: An alignment path between the input paragraphs.
  • Output:
    • Opens an interface showcasing sentence alignments for a paragraph pair.
displaySentenceAnnotations(s1, s2, alignments, annotations)

Displays word-level annotations for a pair of aligned sentences.

  • Parameters:
    • s1: A source sentence.
    • s2: A target sentence.
    • alignments: Word alignments in Pharaoh format.
    • annotations: Word-level annotations produced for the sentence pair.
  • Output:
    • Opens an interface showcasing the word-level annotations for the aligned sentences.
initializeRoot()

Initializes the GUI.

class massalign.gui.ControlFrame(parentObject, p1s_list, p2s_list, alignments_list, main_frame)

Bases: Tkinter.Frame

A frame that allows for multiple alignments to be showcase at once through left/right navigation buttons.

  • Parameters:
    • parentObject: The parent frame.
    • p1s_list: A list of source paragraph lists. A paragraph is a list of sentences.
    • p2s_list: A list of target paragraph lists. A paragraph is a list of sentences.
    • alignments_list: A list containing the alignments for each pair of paragraph lists.
    • main_frame: The frame in which to draw the alignments.
getNextAlignment()

Showcases the alignment between the next pair of paragraph lists.

getPreviousAlignment()

Showcases the alignment between the previous pair of paragraph lists.

class massalign.gui.DisplayFrame(parentObject, background)

Bases: Tkinter.Frame

A frame in which to display alignments and annotations.

  • Parameters:
    • parentObject: The parent frame.
    • background: The background object.
clearDrawingCanvas()

Clears the drawing canvas.

drawAlignments(p1s, p2s, alignments)

Draws a set of alignments between two paragraph lists.

  • Parameters:
    • p1s: A list of source paragraphs. A paragraph is a list of sentences.
    • p2s: A list of target paragraphs. A paragraph is a list of sentences.
  • Output:
    • Opens an interface showcasing the alignments between the input paragraph lists.
drawAnnotations(s1, s2, word_alignments, annotations)

Draws a set of word-level annotations between two parallel sentences.

  • Parameters:
    • s1: A source sentence.
    • s2: A target sentence.
    • word_alignments: Word alignments in Pharaoh format.
    • annotations: Word-level annotations produced for the sentence pair.
  • Output:
    • Opens an interface showcasing the word-level annotations for the aligned sentences.
formatWordAlignments(word_alignments)

Transforms Pharaoh alignments in text format onto a list of alignments.

  • Parameters:
    • word_alignments: A sentence containing the alignments in Pharaoh format.
  • Output:
    • alignment_list: A list of alignments in Pharaoh format.
getAccumulatedOffsetsAndSizes(paragraphs)

Gets offsets and sizes for a list of paragraphs.

  • Parameters:
    • paragraphs: A list of paragraphs. A paragraph is a list of sentences.
  • Output:
    • offsets: The total accumulated vertical offset of each paragraph in the input list.
    • sizes: The individual vertical size of each paragraph in the input list.
getLineSplits(sentence)

Split lines for display.

  • Parameters:
    • sentence: A sentence to split.
  • Output:
    • lines: A list of fragments of the input sentence.
onFrameConfigure(event)

Restructures underlying canvas upon update of main frame.

  • Parameters:
    • event: A frame configure event.
class massalign.gui.GUI

Bases: object

displayListOfParagraphAlignments()
displayListOfSentenceAlignments()
displayParagraphAlignments()
displaySentenceAlignments()
displaySentenceAnnotations()
class massalign.gui.ResizingCanvas(parent, **kwargs)

Bases: Tkinter.Canvas

A resizing canvas that allows for a GUI window to have a dynamic size.

on_resize(event)

Overrides the default resizing behaviour of Canvas objects.

  • Parameters:
    • event: A resizing event.

massalign.models

class massalign.models.SimilarityModel

Bases: object

getSimilarityMapBetweenParagraphsOfDocuments(ps1, ps2)
getSimilarityMapBetweenSentencesOfParagraphs(p1, p2)
class massalign.models.TFIDFModel(input_files=[], stop_list_file=None)

Bases: massalign.models.SimilarityModel

Implements a typical gensim TFIDF model for MASSAlign.

  • Parameters:
    • input_files: A set of file paths containing text from which to extract TFIDF weight values.
    • stop_list_file: A path to a file containing a list of stop-words.
getSentencesFromParagraph(p)

Extracts a set containing all unique sentences in a paragraph.

  • Parameters:
    • p: A paragraph. A paragraph is a list of sentences.
  • Output:
    • sentences: The set containing all unique sentences in the input paragraph.
getSentencesFromParagraphs(ps)

Extracts a set containing all unique sentences in a list of paragraphs.

  • Parameters:
    • ps: A list of paragraphs. A paragraph is a list of sentences.
  • Output:
    • sentences: The set containing all unique sentences in the input paragraph list.
getSimilarityMapBetweenParagraphsOfDocuments(p1s=[], p2s=[])

Produces a matrix containing similarity scores between all paragraphs in a pair of paragraph lists.

  • Parameters:
    • p1s: A list of source paragraphs. Each paragraph is a list of sentences.
    • p2s: A list of target paragraphs. Each paragraph is a list of sentences.
  • Output:
    • paragraph_similarities: A matrix containing a similarity score between all possible pairs of paragraphs in the union of p1 and p2. The matrix’s height and width are equal and equivalent to the number of distinct paragraphs present in the union of p1s and p2s.
getSimilarityMapBetweenSentencesOfParagraphs(p1, p2)

Produces a matrix containing similarity scores between all sentences in a pair of paragraphs.

  • Parameters:
    • p1: A source paragraph. A paragraph is a list of sentences.
    • p2: A target paragraph. A paragraph is a list of sentences.
  • Output:
    • sentence_similarities: A matrix containing a similarity score between all possible pairs of sentences in the union of p1 and p2. The matrix’s height and width are equal and equivalent to the number of distinct sentences present in the union of p1 and p2.
    • sentence_indexes: A map connecting each sentence to its numerical index in the sentence_similarities matrix.
getTFIDFControllers(sentences)

Produces TFIDF similarity scores between all possible pairs of sentences in a list.

  • Parameters:
    • sentences: A list of sentences.
  • Output:
    • sentence_similarities: A matrix containing a similarity score between all possible sentence pairs in the input sentence list. The matrix’s height and width are equal and equivalent to the number of distinct sentences in the input sentence list.
    • sentence_indexes: A map connecting each sentence to its numerical index in the sentence_similarities matrix.
getTFIDFmodel(input_files=[])

Trains a gensim TFIDF model.

  • Parameters:
    • input_files: A set of file paths containing text from which to extract TFIDF weight values.
  • Output:
    • tfidf: A trained gensim models.TfidfModel instance.
    • dictionary: A trained gensim.corpora.Dictionary instance.
getTextSimilarity(buffer1, buffer2)

Calculates the TFIDF similarity between two buffers containing text.

  • Parameters:
    • buffer1: A source buffer containing a block of text.
    • buffer2: A target buffer containing a block of text.
  • Output:
    • similarity: The TFIDF similarity between the two buffers of text.

massalign.util

class massalign.util.FileReader(path, stop_list=set([]))

A convenience class that allows you to more easily read local and online files.

  • Parameters:
    • path: A path to a local file or a url to an online file.
    • stop_list: A set of stop words.
getRawText()

Reads the input file and outputs the raw text in it.

  • Output:
    • text: Raw text within the file.
getSplitSentences()

Reads the input file and produces a list of split sentences.

  • Output:
    • sentences: List of sentences. Each sentence is a list of words.