Missing required modules for django-inventory : Forums : PythonAnywhere

Missing required modules for django-inventory

I want to try out this project: https://github.com/rosarior/django-inventory

But PAW does not have all the required modules, and when I tried to PIP the first missing one, I got the dreaded "where's Zope?" error. Then I deactivated my Virtualenv to see if it is installed in the main system.

(django15)12:46 ~/Dropbox/pythononline/django-inventory $ pip django-photologue
Traceback (most recent call last):
File "/home/rcooke/.virtualenvs/django15/bin/pip", line 5, in <module>
from pkg_resources import load_entry_point
File "/home/rcooke/.virtualenvs/django15/lib/python2.7/site-packages/setuptools-0.6c11-py2.7.egg/pkg_resources.py",  line 2623, in <module>
File "/home/rcooke/.virtualenvs/django15/lib/python2.7/site-packages/setuptools-0.6c11-py2.7.egg/pkg_resources.py", line 679, in subscribe
File "/home/rcooke/.virtualenvs/django15/lib/python2.7/site-packages/setuptools-0.6c11-py2.7.egg/pkg_resources.py", line 2623, in <lambda>
File "/home/rcooke/.virtualenvs/django15/lib/python2.7/site-packages/setuptools-0.6c11-py2.7.egg/pkg_resources.py", line 2172, in activate
File "/home/rcooke/.virtualenvs/django15/lib/python2.7/site-packages/setuptools-0.6c11-py2.7.egg/pkg_resources.py", line 1776, in declare_namespace
ImportError: No module named zope
(django15)12:50 ~/Dropbox/pythononline/django-inventory $ deactivate
12:51 ~/Dropbox/pythononline/django-inventory $ python
Python 2.7.3 (default, Mar 5 2013, 16:45:56)
[GCC 4.4.5] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import django_photologue
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ImportError: No module named django_photologue
>>> import photologue
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ImportError: No module named photologue

django-inventory production modules:

Django==1.2.4
PIL==1.1.7
South==0.7.3
distribute==0.6.10
django-photologue==2.3
django-pagination==1.0.7
wsgiref==0.1.2
yolk==0.4.1

django-inventory development modules:

Django==1.2.4
PIL==1.1.7
South==0.7.3
Werkzeug==0.6.2
distribute==0.6.10
django-extensions==0.6
django-photologue==2.3
django-rosetta==0.5.6
django-pagination==1.0.7
wsgiref==0.1.2
yolk==0.4.1

rcooke | 76 posts | April 9, 2013, 1:06 p.m. | permalink

Interesting. I created a new virtualenv, installed Django 1.5 and added a PIL .pth file to site-packages and django-photologue installed fine. import photologue then works correctly. I'm not sure what you're doing different to cause the failure you see.

glenn | 9718 posts | PythonAnywhere staff | April 9, 2013, 3:23 p.m. | permalink

ooohhh This happened before, and I commented out an installed app or something and the "zope not found" error went away!

I have to dig out my old forum posts where I mention "zope error"....

rcooke | 76 posts | April 9, 2013, 4:43 p.m. | permalink

Was it this one?

deleted-user-39880 | 669 posts | April 9, 2013, 5:05 p.m. | permalink

Yes, thanks. But "south" is not in this project. And I don't see how pip would be affected by what is in the settings.py file anyway.

Sooooo It must have been something else I did.....

Any ideas appreciated....

rcooke | 76 posts | April 9, 2013, 5:14 p.m. | permalink

Here is the traceback, maybe a clue in there..

(django15)12:46 ~/Dropbox/pythononline/django-inventory $ pip django-photologue
Traceback (most recent call last):
File "/home/rcooke/.virtualenvs/django15/bin/pip", line 5, in <module>
from pkg_resources import load_entry_point
File "/home/rcooke/.virtualenvs/django15/lib/python2.7/site-packages/setuptools-0.6c11-py2.7.egg/pkg_resources.py", line     2623, in <module>
File "/home/rcooke/.virtualenvs/django15/lib/python2.7/site-packages/setuptools-0.6c11-py2.7.egg/pkg_resources.py", line 679, in subscribe
File "/home/rcooke/.virtualenvs/django15/lib/python2.7/site-packages/setuptools-0.6c11-py2.7.egg/pkg_resources.py", line 2623, in <lambda>
File "/home/rcooke/.virtualenvs/django15/lib/python2.7/site-packages/setuptools-0.6c11-py2.7.egg/pkg_resources.py", line 2172, in activate
File "/home/rcooke/.virtualenvs/django15/lib/python2.7/site-packages/setuptools-0.6c11-py2.7.egg/pkg_resources.py", line 1776, in declare_namespace
ImportError: No module named zope

rcooke | 76 posts | April 9, 2013, 5:22 p.m. | permalink

Oh, even pip by itself gets the error:

(django15)17:27 ~/Dropbox/pythononline/django-inventory $ pip
Traceback (most recent call last):
File "/home/rcooke/.virtualenvs/django15/bin/pip", line 5, in <module>
from pkg_resources import load_entry_point
File "/home/rcooke/.virtualenvs/django15/lib/python2.7/site-packages/setuptools-0.6c11-py2.7.egg/pkg_resources.py", line 2623, in <module>
File "/home/rcooke/.virtualenvs/django15/lib/python2.7/site-packages/setuptools-0.6c11-py2.7.egg/pkg_resources.py", line 679, in subscribe
File "/home/rcooke/.virtualenvs/django15/lib/python2.7/site-packages/setuptools-0.6c11-py2.7.egg/pkg_resources.py", line 2623, in <lambda>
File "/home/rcooke/.virtualenvs/django15/lib/python2.7/site-packages/setuptools-0.6c11-py2.7.egg/pkg_resources.py", line 2172, in activate
File "/home/rcooke/.virtualenvs/django15/lib/python2.7/site-packages/setuptools-0.6c11-py2.7.egg/pkg_resources.py", line 1776, in declare_namespace
ImportError: No module named zope

rcooke | 76 posts | April 9, 2013, 5:28 p.m. | permalink

Hm. Ok. So I changed into my virtualenv top-level folder and ran grep against "zope" and only got one hit:

[19:51:08] rcooke@harry-liveconsole ~/.virtualenvs/django15
$ grep -r "zope" .
./include/python2.7/zope.proxy/proxy.h: PyObject *m = PyImport_ImportModule("zope.proxy");


:::python
$ python
Python 2.7.3 (default, Mar 5 2013, 16:37:26)
[GCC 4.4.5] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import zope                                                                                                                                        
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ImportError: No module named zope
>>>

so, its "true" - there is no zope module in my virtual env!

If I switch off virtualenv I get a different result:

[20:03:59] rcooke@harry-liveconsole ~/.virtualenvs/django15
$ deactivate
20:06 ~/.virtualenvs/django15 $ python
Python 2.7.3 (default, Mar 5 2013, 16:45:56)
[GCC 4.4.5] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import zope
>>>

No error. So, this is something I need to make a .pth for?

Why did it work for glenn???

rcooke | 76 posts | April 9, 2013, 8:08 p.m. | permalink

hm. Now I'm puzzled. while out of my Virtualenv I located where the zope module lives:

/usr/local/lib/python2.7/site-packages/zope

:::python
(django15)20:21 ~/.virtualenvs/django15/lib/python2.7/site-packages $ python
Python 2.7.3 (default, Mar 5 2013, 16:37:26)
[GCC 4.4.5] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import zope
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ImportError: No module named zope
>>> import os, sys
>>> import pprint                                                                                                                                      
>>> pprint.pprint(sys.path)
['',
'/home/rcooke/.virtualenvs/django15/lib/python2.7/site-packages/setuptools-0.6c11-py2.7.egg',
'/home/rcooke/.virtualenvs/django15/lib/python2.7/site-packages/pip-1.2.1-py2.7.egg',
'/home/rcooke/.virtualenvs/django15/lib/python27.zip',
'/home/rcooke/.virtualenvs/django15/lib/python2.7',
'/home/rcooke/.virtualenvs/django15/lib/python2.7/plat-linux2',
'/home/rcooke/.virtualenvs/django15/lib/python2.7/lib-tk',
'/home/rcooke/.virtualenvs/django15/lib/python2.7/lib-old',
'/home/rcooke/.virtualenvs/django15/lib/python2.7/lib-dynload',
'/usr/local/lib/python2.7',
'/usr/local/lib/python2.7/plat-linux2',
'/usr/local/lib/python2.7/lib-tk',
'/home/rcooke/.virtualenvs/django15/lib/python2.7/site-packages',
'/home/rcooke/Dropbox/pythononline/django-contacts-0.5/src',
'/usr/local/lib/python2.7/site-packages/PIL',
'/usr/local/lib/python2.7/site-packages']

So, it seems to have the right path on the end - why won't it import?

rcooke | 76 posts | April 9, 2013, 8:27 p.m. | permalink

hm. Now I'm puzzled. while out of my Virtualenv I located where the zope module lives: /usr/local/lib/python2.7/site-packages/zope

(django15)20:21 ~/.virtualenvs/django15/lib/python2.7/site-packages $ python
Python 2.7.3 (default, Mar 5 2013, 16:37:26)
[GCC 4.4.5] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import zope
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ImportError: No module named zope
>>> import os, sys
>>> import pprint                                                                                                                                      
>>> pprint.pprint(sys.path)
['',
'/home/rcooke/.virtualenvs/django15/lib/python2.7/site-packages/setuptools-0.6c11-py2.7.egg',
'/home/rcooke/.virtualenvs/django15/lib/python2.7/site-packages/pip-1.2.1-py2.7.egg',
'/home/rcooke/.virtualenvs/django15/lib/python27.zip',
'/home/rcooke/.virtualenvs/django15/lib/python2.7',
'/home/rcooke/.virtualenvs/django15/lib/python2.7/plat-linux2',
'/home/rcooke/.virtualenvs/django15/lib/python2.7/lib-tk',
'/home/rcooke/.virtualenvs/django15/lib/python2.7/lib-old',
'/home/rcooke/.virtualenvs/django15/lib/python2.7/lib-dynload',
'/usr/local/lib/python2.7',
'/usr/local/lib/python2.7/plat-linux2',
'/usr/local/lib/python2.7/lib-tk',
'/home/rcooke/.virtualenvs/django15/lib/python2.7/site-packages',
'/home/rcooke/Dropbox/pythononline/django-contacts-0.5/src',
'/usr/local/lib/python2.7/site-packages/PIL',
'/usr/local/lib/python2.7/site-packages']

So, it seems to have the right path on the end - why won't it import?

rcooke | 76 posts | April 9, 2013, 8:29 p.m. | permalink

So you've installed Zope in a virtualenv yourself? I thought installing Zope included some binary modules and hence required a compiler?

deleted-user-39880 | 669 posts | April 10, 2013, 11:31 a.m. | permalink

Interestingly, it looks like /usr/local/lib/python2.7/site-packages doesn't have an __init__.py. So I'd expect import zope on its own to fail -- why it doesn't when you run it from outside your virtualenv is a mystery to me. What happens if you try to import one of the zope modules? For example, from zope.interface import Interface

giles | 12095 posts | PythonAnywhere staff | April 10, 2013, 11:49 a.m. | permalink

@Cartoo - No, I just made a zope.pth file that points to the system site-packages folder. Which, up until now, has worked perfect for getting VirtualEnv access to system modules that you can't install without a compiler.

@giles - Really? Your asking me? I'm just a dumb end-user. I have no idea what black magic is "behind the scenes" to make it all work - or not....

However, if your truly interested, here is what I did/found:

$ deactivate
14:13 ~ $ python
Python 2.7.3 (default, Mar 5 2013, 16:45:56)
[GCC 4.4.5] on linux2
Type "help", "copyright", "credits" or "license" for more information.


:::python
>>> from zope.interface import Interface
>>> import zope
>>>


:::bash
14:13 ~ $ workon django15
(django15)14:14 ~ $ python
Python 2.7.3 (default, Mar 5 2013, 16:37:26)
[GCC 4.4.5] on linux2
Type "help", "copyright", "credits" or "license" for more information.


:::python
>>> from zope.interface import Interface
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ImportError: No module named zope.interface
>>> import zope
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ImportError: No module named zope
>>>

rcooke | 76 posts | April 10, 2013, 2:16 p.m. | permalink

Yes, I'm not quite sure how they get away without an __init__.py file, but I assume they're doing some nasty hackery in those *-nspkg.pth files. For example:

import sys, new, os
p = os.path.join(sys._getframe(1).f_locals['sitedir'], *('zope',))
ie = os.path.exists(os.path.join(p,'__init__.py'))
m = not ie and sys.modules.setdefault('zope',new.module('zope'))
mp = (m or []) and m.__dict__.setdefault('__path__',[])
(p not in mp) and mp.append(p)

(This was originally all one line with semicolons, I reformatted it slightly for legibility)

I assume that the call to new.module() is creating a module at runtime rather than through the standard Python import mechanism. Not quite sure why they need to use new, I thought that'd been deprecated for ages (presumably in favour of, in this example, types.ModuleType()), but maybe Zope's just been around that long.

EDIT: I was just wondering how this worked, and on double checking the documentation for the site module, I noted its specific wording:

Non-existing items are never added to sys.path, and no check is made that the item refers to a directory rather than a file. No item is added to sys.path more than once. Blank lines and lines beginning with # are skipped. Lines starting with import (followed by space or tab) are executed.

Note that "lines starting with import [...] are executed". So, this explains why all those files have a bunch of statements using semicolons to separate them - the site module quite literally runs every line starting with import, and not every statement. I tested this premise myself and it appears to be correct. What a kludge!

This probably explains why adding paths with .pth files isn't going to work for Zope, because it already relies on the site customisation hooks to get itself loaded in the first place. I don't think adding paths to sys.path is going to do any good because the site stuff only looks for its configuration files in very specific directories, not in the whole of the path.

It also explains why if you look at sys.modules in a plain Python instance on PA (or anywhere where Zope is installed) there's the zope module without you having imported it. Of course, it looks like it's just a dummy name which doesn't do anything, but I'm still fairly disgusted by it all - surely the Zope folks can come up with a cleaner solution for modern Python?

EDIT 2:

Instead of adding a zope.pth file, I wonder if it would work to simply copy all of Zope's own *.pth files into the appropriate place within the virtualenv? Can't think of much else, because the site initialisation stuff happens so early, it doesn't really give your code much chance to do anything.

deleted-user-39880 | 669 posts | April 10, 2013, 5:09 p.m. | permalink

Hm.

I just noticed that my error message says its SETUPTOOLS V 0.6c11 that is trying to load zope.

When I deactivate and check where setuptools lives, I get the newer DISTRIBUTE:

/usr/local/lib/python2.7/site-packages/distribute-0.6.35-py2.7.egg/setuptools

I'm going to try and switch my virtualenv over to distribute....

rcooke | 76 posts | April 11, 2013, 2:36 p.m. | permalink

With Zope's somewhat... Ah... Creative import system, I'm not sure that any solution which uses a site Zope package from within a virtualenv is going to fly. But by all means don't let my pessimism deter you from trying - I would love to be proved wrong in this case...

deleted-user-39880 | 669 posts | April 11, 2013, 2:45 p.m. | permalink

So much for the "calvary":

Hmmm... okay, after poking around for 15 minutes I have to say that I think you should start afresh following Glenn's instructions.

Fresh virtualenv, install of Django 1.5, add "pil.pth" into the virtualenv inside "lib\python2.7\site-packages", containing "/usr/local/lib/python2.7/site-packages/PIL"

It's not worth your time or ours trying to debug exactly what has gone wrong here I think...

Cheers

Hansel

rcooke | 76 posts | April 11, 2013, 6:57 p.m. | permalink

I had to roll-back to Django 1.2, and PAW support had to hack a couple of PIL imports for me, but its working now.

Very nice sample app. Lots of things I want to copy....

rcooke | 76 posts | May 10, 2013, 1:43 p.m. | permalink