Skip to content
Snippets Groups Projects
  • Kurt Partridge's avatar
    6a072047
    ResearchLogger switch word segmentation · 6a072047
    Kurt Partridge authored
    Previously, a logunit was considered a word only if it was all letters.  This is important for
    tracking bigrams correctly.
    
    Now, a logunit must have only at least one letter.  The dictionary check is still performed,
    and punctuation, etc. still comes in as separate LogUnits.  But a word can contain a space,
    which helps set up for logging words where spaces are inserted automatically, and other situations
    in which text is committed with an additional space tacked onto the end.
    
    Change-Id: Ia74094a99058890d20a9cdadf2d0989841a79a41
    6a072047
    History
    ResearchLogger switch word segmentation
    Kurt Partridge authored
    Previously, a logunit was considered a word only if it was all letters.  This is important for
    tracking bigrams correctly.
    
    Now, a logunit must have only at least one letter.  The dictionary check is still performed,
    and punctuation, etc. still comes in as separate LogUnits.  But a word can contain a space,
    which helps set up for logging words where spaces are inserted automatically, and other situations
    in which text is committed with an additional space tacked onto the end.
    
    Change-Id: Ia74094a99058890d20a9cdadf2d0989841a79a41