Natural language processing

Natural language processing (NLP) is a field of computer science that studies how computers and humans interact. In the 1950s, Alan Turing published an article that proposed a measure of intelligence, now called the Turing test. More modern techniques, such as deep learning, have produced results in the fields of language modeling, parsing, and natural-language tasks.

https://github.com/huggingface/transformers/blob/546dc24e0883e5e9f5eb06ec8060e3e6ccc5f6d7/src/transformers/models/gpt2/modeling_gpt2.py#L698

Assertions can't be relied upon for control flow because they can be disabled, as per the following:

$ python --help
usage: python [option] ... [-c cmd | -m mod | file | -] [arg] ...
...
-O     : remove assert and __debug__-dependent statem

Describe the bug
A harmless but distracting warning is raised when prefetch is passed as argument to the Flow.

Reproduce

from jina import Flow, Document
f = Flow(prefetch=10).add()
with f:
     f.index(inputs=Document())

Is your feature request related to a problem? Please describe.
I typically used compressed datasets (e.g. gzipped) to save disk space. This works fine with AllenNLP during training because I can write my dataset reader to load the compressed data. However, the predict command opens the file and reads lines for the Predictor. This fails when it tries to load data from my compressed files.

When using the SmoothingFunction's method4 with a a single word as a hypothesis I get the following error. Which is of course because math.log(1) is 0.
I ended up using just method2 or method1 which doesn't use the function that throws the error, but I guess it would be useful to report it, as some other poor soul may

Sep	OCT	Nov
	10
2020	2021	2022

Natural language processing

Here are 16,086 public repositories matching this topic...

huggingface / transformers

apachecn / AiLearning

google-research / bert

hankcs / HanLP

explosion / spaCy

oxford-cs-deepnlp-2017 / lectures

virgili0 / Virgilio

RasaHQ / rasa

RaRe-Technologies / gensim

keon / awesome-nlp

bharathgs / Awesome-pytorch-list

jina-ai / jina

flairNLP / flair

allenai / allennlp

NLP-LOVE / ML-NLP

nltk / nltk

chiphuyen / stanford-tensorflow-tutorials

huggingface / datasets

spencermountain / compromise

hanxiao / bert-as-service

graykode / nlp-tutorial

botpress / botpress

stanfordnlp / CoreNLP

sloria / TextBlob

brightmart / text_classification

PaddlePaddle / PaddleHub

brightmart / nlp_chinese_corpus

crownpku / Awesome-Chinese-NLP

dragen1860 / TensorFlow-2.x-Tutorials

ymcui / Chinese-BERT-wwm