Fail to test ga4gh-server installation via pip

[wdesouza]

I am trying to install the newest version of ga4gh-server package (0.3.6) via pip. After installation the server throws an error:

[Mon Mar 20 15:12:58.112647 2017] [wsgi:error] [pid 25664:tid 139725278095104] INFO:rdflib:RDFLib Version: 4.2.1
[Mon Mar 20 15:12:58.285056 2017] [wsgi:error] [pid 25664:tid 139725278095104] INFO:ga4gh.server.frontend.cors:Configuring CORS with resources: {'/*': {'automatic_options': True, 'allow_headers': [u'Content-Type'], 'send_wildcard': False, 'intercept_exceptions': True, 'origins': ['.*'], 'vary_header': True, 'supports_credentials': False, 'resources': '/*', 'methods': 'DELETE, GET, HEAD, OPTIONS, PATCH, POST, PUT'}}
[Mon Mar 20 15:12:58.291838 2017] [wsgi:error] [pid 25664:tid 139725278095104] [remote ::1:42585] mod_wsgi (pid=25664): Target WSGI script '/srv/ga4gh/application.wsgi' cannot be loaded as Python module.
[Mon Mar 20 15:12:58.291868 2017] [wsgi:error] [pid 25664:tid 139725278095104] [remote ::1:42585] mod_wsgi (pid=25664): Exception occurred processing WSGI script '/srv/ga4gh/application.wsgi'.
[Mon Mar 20 15:12:58.291904 2017] [wsgi:error] [pid 25664:tid 139725278095104] [remote ::1:42585] Traceback (most recent call last):
[Mon Mar 20 15:12:58.291933 2017] [wsgi:error] [pid 25664:tid 139725278095104] [remote ::1:42585]   File "/srv/ga4gh/application.wsgi", line 3, in <module>
[Mon Mar 20 15:12:58.291999 2017] [wsgi:error] [pid 25664:tid 139725278095104] [remote ::1:42585]     frontend.configure("/srv/ga4gh/config.py")
[Mon Mar 20 15:12:58.292015 2017] [wsgi:error] [pid 25664:tid 139725278095104] [remote ::1:42585]   File "/srv/ga4gh/ga4gh-server-env/lib/python2.7/site-packages/ga4gh/server/frontend.py", line 312, in configure
[Mon Mar 20 15:12:58.292317 2017] [wsgi:error] [pid 25664:tid 139725278095104] [remote ::1:42585]     app.logger)
[Mon Mar 20 15:12:58.292337 2017] [wsgi:error] [pid 25664:tid 139725278095104] [remote ::1:42585]   File "/srv/ga4gh/ga4gh-server-env/lib/python2.7/site-packages/ga4gh/server/network/__init__.py", line 62, in initialize
[Mon Mar 20 15:12:58.292420 2017] [wsgi:error] [pid 25664:tid 139725278095104] [remote ::1:42585]     for initialPeer in getInitialPeerList(filePath):
[Mon Mar 20 15:12:58.292437 2017] [wsgi:error] [pid 25664:tid 139725278095104] [remote ::1:42585]   File "/srv/ga4gh/ga4gh-server-env/lib/python2.7/site-packages/ga4gh/server/network/__init__.py", line 30, in getInitialPeerList
[Mon Mar 20 15:12:58.292462 2017] [wsgi:error] [pid 25664:tid 139725278095104] [remote ::1:42585]     with open(filePath) as textFile:
[Mon Mar 20 15:12:58.292490 2017] [wsgi:error] [pid 25664:tid 139725278095104] [remote ::1:42585] IOError: [Errno 2] No such file or directory: u'ga4gh/server/templates/initial_peers.txt'

I followed the instructions and did not installed the ga4gh package because it seems to be out of date (0.3.5) However, if I install this package the error changes:

[Mon Mar 20 15:20:51.626496 2017] [wsgi:error] [pid 25663:tid 139725278095104] INFO:rdflib:RDFLib Version: 4.2.1
[Mon Mar 20 15:20:51.777078 2017] [wsgi:error] [pid 25663:tid 139725278095104] INFO:ga4gh.server.frontend.cors:Configuring CORS with resources: {'/*': {'automatic_options': True, 'allow_headers': [u'Content-Type'], 'send_wildcard': False, 'intercept_exceptions': True, 'origins': ['.*'], 'vary_header': True, 'supports_credentials': False, 'resources': '/*', 'methods': 'DELETE, GET, HEAD, OPTIONS, PATCH, POST, PUT'}}
[Mon Mar 20 15:20:51.778730 2017] [wsgi:error] [pid 25663:tid 139725278095104] [remote ::1:42585] mod_wsgi (pid=25663): Target WSGI script '/srv/ga4gh/application.wsgi' cannot be loaded as Python module.
[Mon Mar 20 15:20:51.778801 2017] [wsgi:error] [pid 25663:tid 139725278095104] [remote ::1:42585] mod_wsgi (pid=25663): Exception occurred processing WSGI script '/srv/ga4gh/application.wsgi'.
[Mon Mar 20 15:20:51.778847 2017] [wsgi:error] [pid 25663:tid 139725278095104] [remote ::1:42585] Traceback (most recent call last):
[Mon Mar 20 15:20:51.778884 2017] [wsgi:error] [pid 25663:tid 139725278095104] [remote ::1:42585]   File "/srv/ga4gh/application.wsgi", line 3, in <module>
[Mon Mar 20 15:20:51.778985 2017] [wsgi:error] [pid 25663:tid 139725278095104] [remote ::1:42585]     frontend.configure("/srv/ga4gh/config.py")
[Mon Mar 20 15:20:51.778999 2017] [wsgi:error] [pid 25663:tid 139725278095104] [remote ::1:42585]   File "/srv/ga4gh/ga4gh-server-env/lib/python2.7/site-packages/ga4gh/server/frontend.py", line 289, in configure
[Mon Mar 20 15:20:51.779266 2017] [wsgi:error] [pid 25663:tid 139725278095104] [remote ::1:42585]     app.backend = _configure_backend(app)
[Mon Mar 20 15:20:51.779281 2017] [wsgi:error] [pid 25663:tid 139725278095104] [remote ::1:42585]   File "/srv/ga4gh/ga4gh-server-env/lib/python2.7/site-packages/ga4gh/server/frontend.py", line 252, in _configure_backend
[Mon Mar 20 15:20:51.779303 2017] [wsgi:error] [pid 25663:tid 139725278095104] [remote ::1:42585]     dataRepository.open(datarepo.MODE_READ)
[Mon Mar 20 15:20:51.779316 2017] [wsgi:error] [pid 25663:tid 139725278095104] [remote ::1:42585]   File "/srv/ga4gh/ga4gh-server-env/lib/python2.7/site-packages/ga4gh/server/datarepo.py", line 501, in open
[Mon Mar 20 15:20:51.779733 2017] [wsgi:error] [pid 25663:tid 139725278095104] [remote ::1:42585]     self.load()
[Mon Mar 20 15:20:51.779747 2017] [wsgi:error] [pid 25663:tid 139725278095104] [remote ::1:42585]   File "/srv/ga4gh/ga4gh-server-env/lib/python2.7/site-packages/ga4gh/server/datarepo.py", line 1520, in load
[Mon Mar 20 15:20:51.779770 2017] [wsgi:error] [pid 25663:tid 139725278095104] [remote ::1:42585]     self._readDatasetTable(cursor)
[Mon Mar 20 15:20:51.779782 2017] [wsgi:error] [pid 25663:tid 139725278095104] [remote ::1:42585]   File "/srv/ga4gh/ga4gh-server-env/lib/python2.7/site-packages/ga4gh/server/datarepo.py", line 863, in _readDatasetTable
[Mon Mar 20 15:20:51.779800 2017] [wsgi:error] [pid 25663:tid 139725278095104] [remote ::1:42585]     dataset.populateFromRow(row)
[Mon Mar 20 15:20:51.779812 2017] [wsgi:error] [pid 25663:tid 139725278095104] [remote ::1:42585]   File "/srv/ga4gh/ga4gh-server-env/lib/python2.7/site-packages/ga4gh/server/datamodel/datasets.py", line 69, in populateFromRow
[Mon Mar 20 15:20:51.779969 2017] [wsgi:error] [pid 25663:tid 139725278095104] [remote ::1:42585]     self._info = json.loads(row[b'info'])
[Mon Mar 20 15:20:51.779982 2017] [wsgi:error] [pid 25663:tid 139725278095104] [remote ::1:42585]   File "/usr/lib/python2.7/json/__init__.py", line 339, in loads
[Mon Mar 20 15:20:51.780108 2017] [wsgi:error] [pid 25663:tid 139725278095104] [remote ::1:42585]     return _default_decoder.decode(s)
[Mon Mar 20 15:20:51.780121 2017] [wsgi:error] [pid 25663:tid 139725278095104] [remote ::1:42585]   File "/usr/lib/python2.7/json/decoder.py", line 364, in decode
[Mon Mar 20 15:20:51.780240 2017] [wsgi:error] [pid 25663:tid 139725278095104] [remote ::1:42585]     obj, end = self.raw_decode(s, idx=_w(s, 0).end())
[Mon Mar 20 15:20:51.780265 2017] [wsgi:error] [pid 25663:tid 139725278095104] [remote ::1:42585] TypeError: expected string or buffer

Installing from source code (after removing all packages, then install via requirements.txt) throws another error:

[Mon Mar 20 15:20:51.626496 2017] [wsgi:error] [pid 25663:tid 139725278095104] INFO:rdflib:RDFLib Version: 4.2.1
[Mon Mar 20 15:20:51.777078 2017] [wsgi:error] [pid 25663:tid 139725278095104] INFO:ga4gh.server.frontend.cors:Configuring CORS with resources: {'/*': {'automatic_options': True, 'allow_headers': [u'Content-Type'], 'send_wildcard': False, 'intercept_exceptions': True, 'origins': ['.*'], 'vary_header': True, 'supports_credentials': False, 'resources': '/*', 'methods': 'DELETE, GET, HEAD, OPTIONS, PATCH, POST, PUT'}}
[Mon Mar 20 15:20:51.778730 2017] [wsgi:error] [pid 25663:tid 139725278095104] [remote ::1:42585] mod_wsgi (pid=25663): Target WSGI script '/srv/ga4gh/application.wsgi' cannot be loaded as Python module.
[Mon Mar 20 15:20:51.778801 2017] [wsgi:error] [pid 25663:tid 139725278095104] [remote ::1:42585] mod_wsgi (pid=25663): Exception occurred processing WSGI script '/srv/ga4gh/application.wsgi'.
[Mon Mar 20 15:20:51.778847 2017] [wsgi:error] [pid 25663:tid 139725278095104] [remote ::1:42585] Traceback (most recent call last):
[Mon Mar 20 15:20:51.778884 2017] [wsgi:error] [pid 25663:tid 139725278095104] [remote ::1:42585]   File "/srv/ga4gh/application.wsgi", line 3, in <module>
[Mon Mar 20 15:20:51.778985 2017] [wsgi:error] [pid 25663:tid 139725278095104] [remote ::1:42585]     frontend.configure("/srv/ga4gh/config.py")
[Mon Mar 20 15:20:51.778999 2017] [wsgi:error] [pid 25663:tid 139725278095104] [remote ::1:42585]   File "/srv/ga4gh/ga4gh-server-env/lib/python2.7/site-packages/ga4gh/server/frontend.py", line 289, in configure
[Mon Mar 20 15:20:51.779266 2017] [wsgi:error] [pid 25663:tid 139725278095104] [remote ::1:42585]     app.backend = _configure_backend(app)
[Mon Mar 20 15:20:51.779281 2017] [wsgi:error] [pid 25663:tid 139725278095104] [remote ::1:42585]   File "/srv/ga4gh/ga4gh-server-env/lib/python2.7/site-packages/ga4gh/server/frontend.py", line 252, in _configure_backend
[Mon Mar 20 15:20:51.779303 2017] [wsgi:error] [pid 25663:tid 139725278095104] [remote ::1:42585]     dataRepository.open(datarepo.MODE_READ)
[Mon Mar 20 15:20:51.779316 2017] [wsgi:error] [pid 25663:tid 139725278095104] [remote ::1:42585]   File "/srv/ga4gh/ga4gh-server-env/lib/python2.7/site-packages/ga4gh/server/datarepo.py", line 501, in open
[Mon Mar 20 15:20:51.779733 2017] [wsgi:error] [pid 25663:tid 139725278095104] [remote ::1:42585]     self.load()
[Mon Mar 20 15:20:51.779747 2017] [wsgi:error] [pid 25663:tid 139725278095104] [remote ::1:42585]   File "/srv/ga4gh/ga4gh-server-env/lib/python2.7/site-packages/ga4gh/server/datarepo.py", line 1520, in load
[Mon Mar 20 15:20:51.779770 2017] [wsgi:error] [pid 25663:tid 139725278095104] [remote ::1:42585]     self._readDatasetTable(cursor)
[Mon Mar 20 15:20:51.779782 2017] [wsgi:error] [pid 25663:tid 139725278095104] [remote ::1:42585]   File "/srv/ga4gh/ga4gh-server-env/lib/python2.7/site-packages/ga4gh/server/datarepo.py", line 863, in _readDatasetTable
[Mon Mar 20 15:20:51.779800 2017] [wsgi:error] [pid 25663:tid 139725278095104] [remote ::1:42585]     dataset.populateFromRow(row)
[Mon Mar 20 15:20:51.779812 2017] [wsgi:error] [pid 25663:tid 139725278095104] [remote ::1:42585]   File "/srv/ga4gh/ga4gh-server-env/lib/python2.7/site-packages/ga4gh/server/datamodel/datasets.py", line 69, in populateFromRow
[Mon Mar 20 15:20:51.779969 2017] [wsgi:error] [pid 25663:tid 139725278095104] [remote ::1:42585]     self._info = json.loads(row[b'info'])
[Mon Mar 20 15:20:51.779982 2017] [wsgi:error] [pid 25663:tid 139725278095104] [remote ::1:42585]   File "/usr/lib/python2.7/json/__init__.py", line 339, in loads
[Mon Mar 20 15:20:51.780108 2017] [wsgi:error] [pid 25663:tid 139725278095104] [remote ::1:42585]     return _default_decoder.decode(s)
[Mon Mar 20 15:20:51.780121 2017] [wsgi:error] [pid 25663:tid 139725278095104] [remote ::1:42585]   File "/usr/lib/python2.7/json/decoder.py", line 364, in decode
[Mon Mar 20 15:20:51.780240 2017] [wsgi:error] [pid 25663:tid 139725278095104] [remote ::1:42585]     obj, end = self.raw_decode(s, idx=_w(s, 0).end())
[Mon Mar 20 15:20:51.780265 2017] [wsgi:error] [pid 25663:tid 139725278095104] [remote ::1:42585] TypeError: expected string or buffer
[Mon Mar 20 15:25:43.345079 2017] [wsgi:error] [pid 25669:tid 139725278095104] [remote ::1:42585] mod_wsgi (pid=25669): Target WSGI script '/srv/ga4gh/application.wsgi' cannot be loaded as Python module.
[Mon Mar 20 15:25:43.345143 2017] [wsgi:error] [pid 25669:tid 139725278095104] [remote ::1:42585] mod_wsgi (pid=25669): Exception occurred processing WSGI script '/srv/ga4gh/application.wsgi'.
[Mon Mar 20 15:25:43.345172 2017] [wsgi:error] [pid 25669:tid 139725278095104] [remote ::1:42585] Traceback (most recent call last):
[Mon Mar 20 15:25:43.345203 2017] [wsgi:error] [pid 25669:tid 139725278095104] [remote ::1:42585]   File "/srv/ga4gh/application.wsgi", line 1, in <module>
[Mon Mar 20 15:25:43.345268 2017] [wsgi:error] [pid 25669:tid 139725278095104] [remote ::1:42585]     from ga4gh.server.frontend import app as application
[Mon Mar 20 15:25:43.345293 2017] [wsgi:error] [pid 25669:tid 139725278095104] [remote ::1:42585] ImportError: No module named server.frontend

Is there another way to install the latest version of ga4gh-server package?

[david4096]

That's a bug! The dreaded apache config directory problem again, I think. You're running behind Apache? It should fail more gracefully. You could change the apache config to be able to find that path (maybe) or set an absolute path in the configuration.

To close, catch the exception when going to grab the peer list.

[wdesouza]

I added the line below in config.py:

INITIAL_PEERS = "/srv/ga4gh/no_peers.txt"

And I created an empty file with the same name. Now the server throws the second error: TypeError: expected string or buffer. It seems to be related to JSON, right?

EDIT: I don't know why it worked after I reloaded the landing page..

[david4096]

Thanks for working with all of these new features, definitely why we do it! It sounds like specifying an absolute path to the peers list worked, but we still ought to fail more gracefully in the case that someone has not configured initial peers. Sound right @Welliton309 ?