Unsupervised text tokenizer for Neural Network-based text generation.
https://github.com/google/sentencepiece
SentencePiece is an unsupervised text tokenizer and detokenizer mainly for
Neural Network-based text generation systems where the vocabulary size is
predetermined prior to the neural model training.
- Links to home:bird...learning / python-sentencepiece
- Has a link diff
- Download package
-
Checkout Package
osc -A https://api.opensuse.org checkout home:illuusio:python:3.11/python-sentencepiece && cd $_ - Create Badge
Refresh
Refresh
Comments 0