pipe(): ValueError Error parsing doc

[blang]

I found strange behaviour using the pipe() method (only verified on german variant):

If you parse a document using pipe() you can get a ValueError, while if i use nlp(text) everything is fine. I boiled it down to single words, while german words work, english words like 'windows' don't work.

Steps to reproduce:

import spacy
nlp = spacy.load('de')
def texts():
    yield "Windows"
for doc in nlp.pipe(texts(), n_threads=16, batch_size=1000):
    print(len(doc))  # doc access -> ValueError

Trace

ValueError                                Traceback (most recent call last)
<ipython-input-2-9a095ec5505b> in <module>()
      8 def texts():
      9     yield "Windows"
---> 10 for doc in nlp.pipe(texts(), n_threads=16, batch_size=1000):
     11     print(len(doc))

.../venv/lib/python3.4/site-packages/spacy/language.py in pipe(self, texts, tag, parse, entity, n_threads, batch_size)
    254             stream = self.entity.pipe(stream,
    255                 n_threads=1, batch_size=batch_size)
--> 256         for doc in stream:
    257             yield doc
    258 
ValueError: Error parsing doc: Windows

If you use nlp("Windows") it works fine. Also if you execute nlp("Windows") before the same pipe() call, pipe() does not raise an exception (a dictionary is built?)

Versions:

Python 3.4.3 (Problem not related to ipython)
spacy 0.101.0

Maybe this is related to this region syntax/parser.pyx

if not eg.is_valid[guess]:
    # with gil:
    #     move_name = self.moves.move_name(action.move, action.label)
    #     print 'invalid action:', move_name
    return 1

[pdeburen]

Same issue here with the german model and nlp.pipe() on Amazon Linux (also on Ubuntu Server 14.04 LTS) using python 3.5.
However, blang's minimal example works on my Macbook (OSX 10.11.3) where I don't have OpenMP support in place (obviously only in single-thread). Setting n_threads=1 on Linux doesn't solve the issue for me.

[pdeburen]

I just tried this again and it seems to work now (reinstalled spaCy and the german model). @blang can you confirm? @syllog1sm Out of curiosity: Has there been an update to the German model which fixed this? Or was it a code change?

[wbartussek]

Same problem here, working on Windows 10 with German text. Thought it was German that made it break. I also reinstalled spaCy and the German model yesterday, but this din't fix the problem in my case. I then tried to break it down to a specific sentence, but even after having removed this and succesively the follwoing sentences from my texts, the problem remained the same.
As above, if I use nlp(text) everything is fine.

[elyase]

I also had this problem with english text, it looks like a parser issue. Steps to reproduce:

def texts():
    yield "11th September 11 years ago I started my first business"

for doc in nlp.pipe(texts()):
    pass

raises:

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-66-a423c90922c1> in <module>()
      3     yield "11th September 11 years ago I started my first business"
      4 
----> 5 for doc in nlp.pipe(texts()):
      6     pass

/Users/elyase/miniconda3/envs/intent/lib/python3.5/site-packages/spacy/language.py in pipe(self, texts, tag, parse, entity, n_threads, batch_size)
    254             stream = self.entity.pipe(stream,
    255                 n_threads=1, batch_size=batch_size)
--> 256         for doc in stream:
    257             yield doc
    258 

/Users/elyase/miniconda3/envs/intent/lib/python3.5/site-packages/spacy/syntax/parser.pyx in pipe (spacy/syntax/parser.cpp:7143)()

ValueError: Error parsing doc: 11th September 11 years ago I started my first business

Curious thing, If you add a comma like this:

"11th September, 11 years ago I started my first business"

the error goes away. Directly doing:

doc = nlp("11th September 11 years ago I started my first business")

is ok in both cases.

[honnibal]

I think the .pipe() method isn't actually to blame here — rather, it reports on an error condition thrown by the parseC method, while the __call__ method fails silently in the same situation. I've fixed the silent failure, so that errors will be reported more consistently, but the underlying issue is possibly tricky.

The issue is arising because the entity recogniser's push-down automaton finds itself in a state with no continuations. I haven't stepped through the automaton yet (if you want to do that, use the methodnlp.entity.step_through(doc)) to see exactly where the problem is, but I'm pretty sure the error will come from an interaction with entities pre-set by the Matcher class. In order to preserve these entities, we restrict the actions of the entity recogniser, so that it can't over-write the previous ones. There's apparently a bug in the logic to introduce this constraint, that's leaving the automaton with no available actions. This results in an invalid predicted action, leading the parser to return a status code (it can't raise an error, as it's in a nogil function). This is the status code the __call__ method was ignoring.

I'm afraid that 1.0 might paper over this problem, because the matcher won't by default set entities anymore — this will be up to the user's control (there's a better API for customising the pipeline, though).

[mattilyra]

This might have something to do with state between the documents (don't know what if any is kept). I was having this exact issue with the german model and decided to just randomly shuffle the corpus to see what happens. At first errors were still being produced, but after a few shuffles of the corpus to errors to my surprise went away.

I'll try to produce a more reliable report of the behaviour.

update so the error going away didn't have anything to do with random.shuffle. However if you take the document that produces the error, pass it through nlp(doc) and then call .pipe([doc]) again, it works.

An example document where 9/11 causes the error, removing that one token everything is fine.

doc = """Nichtsdestoweniger sagen wir Ihnen zu, dass wir genauso wie bei OEF daran arbeiten werden, dass dieses Mandat eine andere Struktur erhält, und zwar aus politischen Gründen, weil zwar nicht rechtlich, aber natürlich politisch zur Kenntnis zu nehmen ist, dass 9/11 schon einige Zeit her ist."""

Tested on 0.101 and master - the issue seems to be fixed on master not just for the example document above, but for the entire corpus where that doc came from.

[honnibal]

I think I have this taken care of, but I'm not 100% sure. Please reopen if it reoccurs.

[sadovnychyi]

FYI it did happen for me with 1.1.0, but so far I cannot provide any steps to reproduce it.

The text it tried to parse isn't relevant: Waiver of Jury Trial but I did update global nlp.matcher in a loop while parsing the doc in the same loop. I'll get back to this if I'll be able to reproduce it with specific steps.

PS: Now I'm getting Segmentation fault: 11 and I have no idea if it's relevant or not..

[honnibal]

Do you have a minute to video chat about this? If so click here:

https://appear.in/spacy_issue429