Fatal Exception

[PatrickCrenshaw]

I'm getting an error when I try to run corenlp.py

$ python  corenlp/corenlp.py 
java -Xmx3g -cp stanford-corenlp-full-2013-11-12/stanford-corenlp-3.3.0.jar:stanford-corenlp-full-2013-11-12/stanford-corenlp-3.3.0-models.jar:stanford-corenlp-full-2013-11-12/xom.jar:stanford-corenlp-full-2013-11-12/joda-time.jar:stanford-corenlp-full-2013-11-12/jollyday.jar:stanford-corenlp-full-2013-11-12/ejml-0.23.jar edu.stanford.nlp.pipeline.StanfordCoreNLP -props /home/patrick/nlp-features/corenlp-python/corenlp/default.properties
Traceback (most recent call last):                                                                                
  File "corenlp/corenlp.py", line 575, in <module>
    nlp = StanfordCoreNLP(options.corenlp, properties=options.properties, serving=True)
  File "corenlp/corenlp.py", line 430, in __init__
    self._spawn_corenlp()
  File "corenlp/corenlp.py", line 413, in _spawn_corenlp
    self.corenlp.expect("done.", timeout=timeouts[i])  # Load model
  File "/usr/lib/python2.7/dist-packages/pexpect.py", line 1316, in expect
    return self.expect_list(compiled_pattern_list, timeout, searchwindowsize)
  File "/usr/lib/python2.7/dist-packages/pexpect.py", line 1330, in expect_list
    return self.expect_loop(searcher_re(pattern_list), timeout, searchwindowsize)
  File "/usr/lib/python2.7/dist-packages/pexpect.py", line 1401, in expect_loop
    raise EOF (str(e) + '\n' + str(self))
pexpect.EOF: End Of File (EOF) in read_nonblocking(). Exception style platform.
<pexpect.spawn object at 0x8c50fac>
version: 2.4 ($Revision: 516 $)
command: /usr/bin/java
args: ['/usr/bin/java', '-Xmx3g', '-cp', 'stanford-corenlp-full-2013-11-12/stanford-corenlp-3.3.0.jar:stanford-corenlp-full-2013-11-12/stanford-corenlp-3.3.0-models.jar:stanford-corenlp-full-2013-11-12/xom.jar:stanford-corenlp-full-2013-11-12/joda-time.jar:stanford-corenlp-full-2013-11-12/jollyday.jar:stanford-corenlp-full-2013-11-12/ejml-0.23.jar', 'edu.stanford.nlp.pipeline.StanfordCoreNLP', '-props', '/home/patrick/nlp-features/corenlp-python/corenlp/default.properties']
searcher: searcher_re:
    0: re.compile("done.")
buffer (last 100 chars): 
before (last 100 chars): ld not create the Java Virtual Machine.
Error: A fatal exception has occurred. Program will exit.

after: <class 'pexpect.EOF'>
match: None
match_index: None
exitstatus: 1
flag_eof: True
pid: 18679
child_fd: 4
closed: False
timeout: 60
delimiter: <class 'pexpect.EOF'>
logfile: None
logfile_read: None
logfile_send: None
maxread: 8192
ignorecase: False
searchwindowsize: 80
delaybeforesend: 0.05
delayafterclose: 0.1
delayafterterminate: 0.1

[abendebury]

What is the output of pip freeze? How did you install the prerequisites for this package?

[PatrickCrenshaw]

I just did what the readme said to do (e.g. sudo pip install pexpect unidecode jsonrpclib ) with the modification of installing stanford-corenlp-full-2013-11-12 instead of stanford-corenlp-full-2013-06-20 because stanford-corenlp-full-2013-06-20 has different versions than what is expected. I tried both versions of stanford corenlp and they both gave this same error.

$ pip freeze
BeautifulSoup==3.2.1
GnuPGInterface==0.3.2
Mako==0.7.1
MarkupSafe==0.15
MySQL-python==1.2.3
PAM==0.4.2
PIL==1.1.7
PyICU==1.4
Pygments==1.5
Theano==0.6.0rc3
Twisted-Core==12.2.0
Twisted-Names==12.2.0
Twisted-Web==12.2.0
UniConvertor==1.1.4
Unidecode==0.04.16
apt-xapian-index==0.44
aptoncd==0.1.98-bzr117-1.2
apturl==0.4.1ubuntu4
argparse==1.2.1
chardet==2.0.1
configglue==1.0
configobj==4.7.2
debtagshw==0.1
decorator==3.3.3
defer==1.0.6
dirspec==4.0.0
distribute==0.7.3
duplicity==0.6.19
feedparser==5.1.2
gufw==12.10.0
httplib2==0.7.4
ipython==0.13.1.rc2
joblib==0.6.4
jsonrpclib==0.1.3
lxml==2.3.5
matplotlib==1.1.1
nltk==3.0.0b1
nose==1.1.2
numexpr==2.0.1
numpy==1.7.1
oauth==1.0.1
openpyxl==1.5.8
pandas==0.8.0
paramiko==1.7.7.1
pexpect==2.4
piston-mini-client==0.7.2
protobuf==2.4.1
pyOpenSSL==0.13
pycrypto==2.6
pycups==1.9.62
pycurl==7.19.0
pyinotify==0.9.3
pyparsing==1.5.6
pyserial==2.5
pysmbc==1.0.13
python-apt==0.8.7ubuntu4.1
python-dateutil==1.5
python-debian==0.1.21-nmu2ubuntu1
pytz==2011k
pyxattr==0.5.1
pyxdg==0.20
pyzmq==2.2.0
reportlab==2.5
scikit-learn==0.15.0b1
scipy==0.10.1
sessioninstaller==0.0.0
simplegeneric==0.8.1
simplejson==2.6.1
six==1.1.0
statsmodels==0.4.2
system-service==0.1.6
tables==2.3.1
tornado==2.3
uTidylib==0.2
versiontools==1.9.1
wsgiref==0.1.2
xlrd==0.6.1
xlwt==0.7.4
zope.interface==3.6.1

[abendebury]

Okay, well one thing you can do is to try to install the requirements in a virtualenv to make sure that the other python packages you have aren't conflicting.

sudo pip install virtualenv
virtualenv venv
source venv/bin/activate
pip install pexpect unidecode jsonrpclib

Then try running it again.

That said, are you sure you want to run this as a server? I'm not even sure if that functionality is bug free enough to be reliable. At least, in the wordseer project we aren't running it as a server.

[PatrickCrenshaw]

I actually get the same error when I run things like:

from corenlp import StanfordCoreNLP
corenlp_dir = "stanford-corenlp-full-2013-11-12"
corenlp = StanfordCoreNLP(corenlp_dir) # wait a few minutes...
corenlp.raw_parse("He says that Joe killed Sam.")

I'll try the vitualenv.

[abendebury]

Yeah, raw_parse should be working. Let me know how the virtualenv works out.

[PatrickCrenshaw]

I've never used virtualenv before, so I may be doing something wrong, but it failed the same way when I did (output omitted for clarity):

sudo pip install virtualenv
virtualenv venv
source venv/bin/activate
sudo venv/bin/pip install pexpect unidecode jsonrpclib
venv/bin/python test.py

[abendebury]

Try to just do this:

rm -r virtualenv
virtualenv venv
source venv/bin/activate
pip install pexpect unidecode jsonrpclib
python test.py

Notice the lack of sudo when running pip install. Virtualenv should point pip and python to the appropriate executables.

[PatrickCrenshaw]

Didn't fix it, I'm afraid.

[abendebury]

You're getting the same error as in your first post?

What about the raw_parse method?

[PatrickCrenshaw]

Sorry I wasn't clear.

Yes, I am getting the same error whether I try to run a server or I use the raw_parse method, which I am calling in a file I named "test.py".

[abendebury]

Can you paste the contents of test.py?

[PatrickCrenshaw]

from corenlp import StanfordCoreNLP
corenlp_dir = "stanford-corenlp-full-2013-11-12"
corenlp = StanfordCoreNLP(corenlp_dir) # wait a few minutes...
corenlp.raw_parse("He says that Joe killed Sam.")

[abendebury]

Do you have java installed? Which version?

[PatrickCrenshaw]

$ java -version
java version "1.7.0_51"
OpenJDK Runtime Environment (IcedTea 2.4.4) (7u51-2.4.4-0ubuntu0.12.10.2)
OpenJDK Server VM (build 24.45-b08, mixed mode)

I'll try sun java and see if that makes a difference.

[PatrickCrenshaw]

Changed java version and I still get the same error.

New java version:
$ java -version
java version "1.8.0_20"
Java(TM) SE Runtime Environment (build 1.8.0_20-b26)
Java HotSpot(TM) Server VM (build 25.20-b23, mixed mode)