Commits · 6b0561f9d26215209e8e8895f5c35982af5158f0 · keyboard / LatinIME

Oct 29, 2014
- Add a class to have global counters for LanguageModelDictContent. · 6b0561f9
  Keisuke Kuroyanagi authored 10 years ago
  
  Bug: 14425059 Change-Id: I08ec19903432356b6028853fd73b4eefce20218e
  6b0561f9
Oct 22, 2014
- Support dumping ngram entries. · b5ef884f
  Keisuke Kuroyanagi authored 10 years ago
  
  Bug: 14425059 Change-Id: Ib03a0c3d166ed6f1e60c67127b28006d55143b6b
  b5ef884f
- Support ngram entry migration. · c9865785
  Keisuke Kuroyanagi authored 10 years ago
  
  Bug: 14425059 Change-Id: I98cb9fa303af2d93a0a3512e8732231c564e3c5d
  c9865785
Oct 21, 2014
- Fix debug build. · 0b8bb0c2
  Keisuke Kuroyanagi authored 10 years ago
  
  Change-Id: Id94636714d04a8828718b87741c0ee62a14cb3b4
  0b8bb0c2
- Make NgramProperty have NgramContext. · 88bb28c1
  Keisuke Kuroyanagi authored 10 years ago
  
  Bug: 14425059 Change-Id: I210acb816b122857dbbe1ee4dd6a35c5335bf2bf
  88bb28c1
- Use EntryCounters during GC. · 47fc656c
  Keisuke Kuroyanagi authored 10 years ago
  
  Bug: 14425059 Change-Id: I61eb798686dc753fb6c0fe99a0719c1732198f30
  47fc656c
- Introduce EntryCounters to count entries in a dictionary. · e8750d97
  Keisuke Kuroyanagi authored 10 years ago
  
  Bug: 14425059 Change-Id: Ic13ba827d96fa4a147485ba92fdb37e23e04e8e8
  e8750d97
- Renaming "blacklist" flag to "possibly offensive" · 05172bf1
  Adrian Velicu authored 10 years ago
  
  No behaviour changes. Unified the overloaded FusionDictionary::add method to always take an isPossiblyOffensive argument. Bug: 11031090 Change-Id: I5741a023ca1ce842d2cf10d4f6c926b0efabaa78
  05172bf1
Oct 20, 2014

Keisuke Kuroyanagi authored 10 years ago

Unigram 10K, Bigram 30K, Trigram 30K.

Change-Id: Ibd19c6a2b618499df1c70000bad7b47498187f0a

1085fef8

Oct 15, 2014
- Update useless n-gram entry detection logic during GC. · 3601c214
  Keisuke Kuroyanagi authored 10 years ago
  
  Bug: 14425059 Change-Id: Ib939deae5b60167751dee07965bb1ef1a43c4625
  3601c214
- Use better conditional probability for ngram entries. · 72d17d92
  Keisuke Kuroyanagi authored 10 years ago
  
  Old: P(W | W_prev) = f(W, W_prev) + C New: P(W | W_prev) = f(W, W_prev) / f(W_prev) Bug: 14425059 Bug: 16547409 Change-Id: I4d13be6de2c6bad6bad7fb22320a23ba4ecd361c
  72d17d92
- Move entry updating method to language model dict content. · 54007019
  Keisuke Kuroyanagi authored 10 years ago
  
  Bug: 14425059 Change-Id: I710055490d141539458cbf968adf5a7ccffd9552
  54007019
Oct 14, 2014

Quit using weightChildNode for ADDITIONAL_PROXIMITY and SUBSTITUTION. · d8ccb909

Keisuke Kuroyanagi authored 10 years ago

[Category diff]
+1       0
-1       1
+2       0
-2       0
+3       0
-3       0
+4       1
-4       1
+5       8
-5       7
+6       0
-6       1
+7       1
-7       0

[Weighted category diff]
+1       0
-1       1
+2       0
-2       0
+3       0
-3       0
+4       1
-4       1
+5       8
-5       7
+6       0
-6       1
+7       1
-7       0

Bug: 13756409
Change-Id: I6ac3567545676bbefbee3e87dda54bc083c15fb6

d8ccb909

Oct 13, 2014

Fix: BoS bigram from user history dictionary is too strong. · d70b8ff2

Keisuke Kuroyanagi authored 10 years ago

They can be always stronger than BoS predictions from the
contextual dictionary.

Bug: 17961731
Change-Id: I70297d82436c10c790bdfad6f3dfefdb4bb2f852

d70b8ff2

Oct 10, 2014
- [ML23] Introduce a different accuracy/performance tradeoff · 7d5e1cb2
  Jean Chalard authored 10 years ago
  
  Bug: 11230254 Change-Id: Ic09518c818ae7b68942b1c63160dd462e5922cb5
  7d5e1cb2
Oct 09, 2014
- Rename prev_words_info.h to ngram_context.h · 45783013
  Keisuke Kuroyanagi authored 10 years ago
  
  Bug: 14425059 Change-Id: I0e906631ecad2361a8198b3f9e3394bb22c5bf83
  45783013
- Fixing misspelled word · 44efbe64
  Adrian Velicu authored 10 years ago
  
  Change-Id: I51d77e271143d40256b39e5c60a3065d9fdf63fb
  44efbe64
- Rename PrevWordsInfo to NgramContext. · 72e2383d
  Keisuke Kuroyanagi authored 10 years ago
  
  Bug: 14425059 Change-Id: I30703fc80e9450d4e2dbfec965e7f9f4468f6a11
  72e2383d
- Rename updateCounter to updateEntriesForWordWithNgramContext. · ab4437f4
  Keisuke Kuroyanagi authored 10 years ago
  
  Bug: 14425059 Change-Id: Id9b0dd7e32c711ed4292981517c3febd5fe9e897
  ab4437f4
Oct 06, 2014

Improve bigram probability computation for decaying dicts. · aae1a062

Keisuke Kuroyanagi authored 10 years ago

Without personalization:
Total words: 1079345, Success Num: 819749, Success Percentage: 75.949%
Bad Failures, with auto-correction (typed word == expected word, output word != expected word): 1754, Bad Failure Percentage: 0.163%
Failures, with auto-correction (F-C): 28463, F-C Percentage: 2.637%
Max Keystrokes: 6074285, Min Keystrokes: 4649326, Keystroke Saving Percentage:23.459%

With current probability computing logic:
Total words: 1079382, Success Num: 838329, Success Percentage: 77.667%
Bad Failures, with auto-correction (typed word == expected word, output word != expected word): 1332, Bad Failure Percentage: 0.123%
Failures, with auto-correction (F-C): 28558, F-C Percentage: 2.646%
Max Keystrokes: 6074503, Min Keystrokes: 4474102, Keystroke Saving Percentage:26.346%
Remove isof files.

With new probability computing logic:
Total words: 1079356, Success Num: 844954, Success Percentage: 78.283%
Bad Failures, with auto-correction (typed word == expected word, output word != expected word): 1306, Bad Failure Percentage: 0.121%
Failures, with auto-correction (F-C): 27214, F-C Percentage: 2.521%
Max Keystrokes: 6074477, Min Keystrokes: 4243021, Keystroke Saving Percentage:30.150%
Remove isof files.

Bug: 16547409
Change-Id: I3d2a49c7aaa2c0f6835c52ef72d22466ee225789

aae1a062

Make sure to suppress BoS prediction until input twice. · 1c2f00f6
Keisuke Kuroyanagi authored 10 years ago
```
Change-Id: I98d91f264d5d1d3f5bcda1fd9ec885779ba2f746
```
1c2f00f6
Fix: BoS prediction is shown after inputting just once. · ca17ed7d
Keisuke Kuroyanagi authored 10 years ago
```
Change-Id: Ibba209f47cb5b1a4b08281689d607711b8dcfad4
```
ca17ed7d

Fix: ProbabilityDictContent can be wrongly large. · 16e16153

Keisuke Kuroyanagi authored 10 years ago

It can be twice as large as it should be (80KB larger).

Change-Id: If94f748f8c48a442b3c95ac989099aaed2aa2f86

16e16153

Oct 01, 2014
- Implement updateCounter() by using existing entry adding methods. · 29777e3a
  Keisuke Kuroyanagi authored 10 years ago
  
  Bug: 14425059 Change-Id: I0b6cb80e1fb8f738e9c7d9e80fbc0c479546b879
  29777e3a
- Move HistoricalInfo to property and use it in *Property. · 287e155e
  Keisuke Kuroyanagi authored 10 years ago
  
  Bug: 14425059 Change-Id: Icccccabad98fb543c6a6be2844cfc0086d80b739
  287e155e
Sep 29, 2014

Rename BigramProperty to NgramProperty. · 79bb37d4

Keisuke Kuroyanagi authored 10 years ago

Remaining work is changing bigram to ngram for supporting
ngram entry counting, dumping, and migration.

Bug: 14425059
Change-Id: Ifba288a1166996d62a5e57698f63537ea0a2a8ee

79bb37d4

Sep 25, 2014
- Quit reading unigram probability in Ver4PatriciaTrieNodeReader. · cb4f5441
  Keisuke Kuroyanagi authored 10 years ago
  
  Bug: 14425059 Change-Id: I4fc7b0e236151a2c64e7131772264024c6597633
  cb4f5441
- Use std::move for dictionary properties. · 2842e50c
  Keisuke Kuroyanagi authored 10 years ago
  
  Change-Id: I15056b36b7493f4bac1dbcbb46a0b44343ede153
  2842e50c
Sep 24, 2014
- Use CodePointArrayView in WordProperty. · 80d139a6
  Keisuke Kuroyanagi authored 10 years ago
  
  Change-Id: I45a9755c413003831788d190beb499fee8ce63aa
  80d139a6
- Refactoring method to get code points and code point count. · 65a7ccfa
  Keisuke Kuroyanagi authored 10 years ago
  
  Bug: 14425059 Change-Id: I4731bd6076d34556e46e6714180fed324fb6aba3
  65a7ccfa
- Move word flags to language model dict content. · 7d911d6f
  Keisuke Kuroyanagi authored 10 years ago
  
  Bug: 14425059 Change-Id: I64712e5c83d0bc241e6f0f16117ab47b5d75bd4b
  7d911d6f
Sep 22, 2014
- Prepare supporting n-gram for user history dictionary. · ddfaeff5
  Keisuke Kuroyanagi authored 10 years ago
  
  Bug:17097992 Change-Id: Ic8bfde3d4cc0e720bf7681e08e16fb2ad94d5670
  ddfaeff5
Sep 19, 2014
- Fix native crash. · 5a20827f
  Keisuke Kuroyanagi authored 10 years ago
  
  Change-Id: Ia8871db0248ddbc821605079e2a3e93859f4c9b7
  5a20827f
- Native side reads character table · fb2bde5a
  Akifumi Yoshimoto authored 10 years ago
  
  Bug:17097992 Change-Id: Ibcfc67833a6754fe6a2d82a3e3023b33886b9ea2
  fb2bde5a
- [ML8] Add a language weight · 6da9b211
  Jean Chalard authored 10 years ago
  
  ...and rename an improperly named normalization value Bug: 11230254 Change-Id: I0f5633148a9f66dbfd7d28540b8a8985131c4549
  6da9b211
Sep 17, 2014
- Use ReadOnlyByteArrayView in Ver2ParticiaTrieNodeReader. · 6258c57c
  Keisuke Kuroyanagi authored 10 years ago
  
  Change-Id: Ie3325bb9666c7ff6d519a1b72ed2fe9fc4f140a9
  6258c57c
- Use ReadOnlyByteArrayView in Ver2PtNodeArrayReader. · a892628f
  Keisuke Kuroyanagi authored 10 years ago
  
  Change-Id: Ia1065465abdf7c64fadb3ff1efd2a4d2df343264
  a892628f
- Use ReadOnlyByteArrayView in ShortcutListPolicy · 59ebd517
  Keisuke Kuroyanagi authored 10 years ago
  
  Change-Id: I03a6f49c9005306bcc5fce8b7e4d37b8d30b9faa
  59ebd517
- Use ReadOnlyByteArrayView in BigramListPolicy. · d01eb3c9
  Keisuke Kuroyanagi authored 10 years ago
  
  Change-Id: I0f8e8044c8803a7a542f97be5b8f08fdb5ae3926
  d01eb3c9
- Use ReadOnlyByteArrayView in PatriciaTriePolicy. · 180e7b4c
  Keisuke Kuroyanagi authored 10 years ago
  
  Change-Id: I4ef01c0af73f27c256cc1a719343be836913dc71
  180e7b4c