Hi, I'm new to PA and really enjoy it. Thanks for maintaining this service! Dropbox is on the whitelist, but I still get tunneling 403 errors. (I have a free account.) Any chance they have once again changed their API, or, could I get that error for another reason more specific to my code?
Forums
White List a Moving Target?
It's hard to say -- does the 403 error say which hostname it's trying to connect to?
giles
|
12095
posts
|
PythonAnywhere staff
|
The Traceback shows, as you might expect, a long series of cascading errors. However, the code is short and there is only one URL call.
"df=pandas.read_csv("https://www.dropbox.com/s/#deleted for privacy#/Weight%20Data%202020.txt?raw=1", delimiter=' ',names=["Month","Day","Weight","BC","Systolic","Diastolic","HR"])"
yields
"OSError: Tunnel connection failed: 403 Forbidden"
The same code runs without error from my local machine.
I can see a few attempts to access www.dropbox.com in the proxy logs and they were accepted by the proxy, as expected. It's possible that the error you're seeing is actually Dropbox returning the "not authorised".
Perhaps the link is getting garbled in the parser somehow? I'll check versions and such.
This is also buried in the traceback:
File "/usr/lib/python3.7/urllib/request.py", line 755, in http_error_302 return self.parent.open(new, timeout=req.timeout)
But I assumed the fundamental error was 403. Thanks for your help. If you don't think it is an access problem, I'll keep looking.
Just for clarity -- the 403 status code is a generic way that a server using HTTP says that access to a particular resource is forbidden. So while our proxy server (which manages the whitelist) will return a 403 if you try to access a non-whitelisted site, other servers might also return it when you try to access something they don't want you to access. So, for example, if you try to access a file on www.dropbox.com that they don't want you to access, the request would pass through the proxy, get to their servers, and then Dropbox's own servers might then return a 403 error to you.
hi kokomodo & giles, I am in the same situation, the following code works fine in localhost, but got 403 error in PA, have you find solution for this problem, any help would be appreciated.
def save_thumbnail_image(request, img_url, street):
image_mapo_relative_path = ''
if 'www.dropbox.com' in img_url:
img_url = urllib.parse.unquote(img_url).replace(" ", "").split('?')[0] + '?raw=1'
try:
img_object = Image.open(urlopen(img_url))
except:
return image_mapo_relative_path
followin is traceback:
Environment:
Request Method: POST
Request URL: http://stevenliu1970.pythonanywhere.com/mapoapp/project_update/1/
Django Version: 3.0.5
Python Version: 3.7.5
Installed Applications:
['django.contrib.admin',
'django.contrib.auth',
'django.contrib.contenttypes',
'django.contrib.sessions',
'django.contrib.messages',
'django.contrib.staticfiles',
'django.contrib.sites',
'allauth',
'allauth.account',
'allauth.socialaccount',
'allauth.socialaccount.providers.facebook',
'allauth.socialaccount.providers.weixin',
'mapoapp']
Installed Middleware:
['django.middleware.security.SecurityMiddleware',
'django.contrib.sessions.middleware.SessionMiddleware',
'django.middleware.common.CommonMiddleware',
'django.middleware.csrf.CsrfViewMiddleware',
'django.contrib.auth.middleware.AuthenticationMiddleware',
'django.contrib.messages.middleware.MessageMiddleware',
'django.middleware.clickjacking.XFrameOptionsMiddleware']
Traceback (most recent call last):
File "/usr/lib/python3.7/urllib/request.py", line 1317, in do_open
encode_chunked=req.has_header('Transfer-encoding'))
File "/usr/lib/python3.7/http/client.py", line 1252, in request
self._send_request(method, url, body, headers, encode_chunked)
File "/usr/lib/python3.7/http/client.py", line 1298, in _send_request
self.endheaders(body, encode_chunked=encode_chunked)
File "/usr/lib/python3.7/http/client.py", line 1247, in endheaders
self._send_output(message_body, encode_chunked=encode_chunked)
File "/usr/lib/python3.7/http/client.py", line 1026, in _send_output
self.send(msg)
File "/usr/lib/python3.7/http/client.py", line 966, in send
self.connect()
File "/usr/lib/python3.7/http/client.py", line 1414, in connect
super().connect()
File "/usr/lib/python3.7/http/client.py", line 942, in connect
self._tunnel()
File "/usr/lib/python3.7/http/client.py", line 921, in _tunnel
message.strip()))
During handling of the above exception (Tunnel connection failed: 403 Forbidden), another exception occurred:
File "/home/Stevenliu1970/.virtualenvs/mapoenv/lib/python3.7/site-packages/django/core/handlers/exception.py", line 34, in inner
response = get_response(request)
File "/home/Stevenliu1970/.virtualenvs/mapoenv/lib/python3.7/site-packages/django/core/handlers/base.py", line 115, in _get_response
response = self.process_exception_by_middleware(e, request)
File "/home/Stevenliu1970/.virtualenvs/mapoenv/lib/python3.7/site-packages/django/core/handlers/base.py", line 113, in _get_response
response = wrapped_callback(request, *callback_args, **callback_kwargs)
File "/home/Stevenliu1970/mysite/mapoapp/views.py", line 177, in project_update
image_mapo_relative_path = save_thumbnail_image(request, project.image_link, project.street)
File "/home/Stevenliu1970/mysite/mapoapp/utilities.py", line 89, in save_thumbnail_image
img_object = Image.open(urlopen(img_url)) #'www.dropbox.com' need to be white listed by pyrhonanywhere?
File "/usr/lib/python3.7/urllib/request.py", line 222, in urlopen
return opener.open(url, data, timeout)
File "/usr/lib/python3.7/urllib/request.py", line 531, in open
response = meth(req, response)
File "/usr/lib/python3.7/urllib/request.py", line 641, in http_response
'http', request, response, code, msg, hdrs)
File "/usr/lib/python3.7/urllib/request.py", line 563, in error
result = self._call_chain(*args)
File "/usr/lib/python3.7/urllib/request.py", line 503, in _call_chain
result = func(*args)
File "/usr/lib/python3.7/urllib/request.py", line 755, in http_error_302
return self.parent.open(new, timeout=req.timeout)
File "/usr/lib/python3.7/urllib/request.py", line 531, in open
response = meth(req, response)
File "/usr/lib/python3.7/urllib/request.py", line 641, in http_response
'http', request, response, code, msg, hdrs)
File "/usr/lib/python3.7/urllib/request.py", line 563, in error
result = self._call_chain(*args)
File "/usr/lib/python3.7/urllib/request.py", line 503, in _call_chain
result = func(*args)
File "/usr/lib/python3.7/urllib/request.py", line 755, in http_error_302
return self.parent.open(new, timeout=req.timeout)
File "/usr/lib/python3.7/urllib/request.py", line 525, in open
response = self._open(req, data)
File "/usr/lib/python3.7/urllib/request.py", line 543, in _open
'_open', req)
File "/usr/lib/python3.7/urllib/request.py", line 503, in _call_chain
result = func(*args)
File "/usr/lib/python3.7/urllib/request.py", line 1360, in https_open
context=self._context, check_hostname=self._check_hostname)
File "/usr/lib/python3.7/urllib/request.py", line 1319, in do_open
raise URLError(err)
Exception Type: URLError at /mapoapp/project_update/1/
Exception Value: <urlopen error Tunnel connection failed: 403 Forbidden>
[edit by admin: formatting]
Dropbox is definitely whitelisted, so the 403 error either means that the URL you're connecting to is on some other non-whitelisted site (which seems unlikely from your code) or that Dropbox themselves are returning a 403 status code, which is the normal way that a web server would say "access denied".
Probably the first things to check are exactly what URL you're trying to access, and what the response body is; you can do that by adding an import requests to the top of your file, and then just before the Image.open:
print(f"URL is {img_url}", flush=True)
print(f"Response body is {requests.get(img_url).content}", flush=True)
The output of those prints will go into the website's server log, and hopefully will clarify what is going on here.
thanks giles, following is the server log , could you help me figure out what cause the problem, many thanks
-----------here is the code-----------------
:::python
def save_thumbnail_image(request, img_url, street):
image_mapo_relative_path = ''
if 'www.dropbox.com' in img_url:
img_url = img_url.split('?')[0] + '?raw=1'
print(f"URL is {img_url}", flush=True)
print(f"Response body is {requests.get(img_url).content}", flush=True)
img_object = Image.open(urlopen(img_url))
-----------here is the server log----------
2020-05-20 12:43:34 URL is https://www.dropbox.com/s/51y6fvxqj2swy8w/676%20Stud%20Road%20External%20Cam%20%20%20%20%20%20%20%20%20%20%204_large.jpg?raw=1
2020-05-20 12:43:35 Internal Server Error: /mapoapp/project_add/#012Traceback (most recent call last):#012 File "/home/Stevenliu1970/.virtualenvs/mapoenv/lib/python3.7/site-packages/urllib3/connectionpool.py", line 667, in urlopen#012 self._prepare_proxy(conn)#012 File "/home/Stevenliu1970/.virtualenvs/mapoenv/lib/python3.7/site-packages/urllib3/connectionpool.py", line 930, in _prepare_proxy#012 conn.connect()#012 File "/home/Stevenliu1970/.virtualenvs/mapoenv/lib/python3.7/site-packages/urllib3/connection.py", line 316, in connect#012 self._tunnel()#012 File "/usr/lib/python3.7/http/client.py", line 921, in _tunnel#012 message.strip()))#012OSError: Tunnel connection failed: 403 Forbidden#012#012During handling of the above exception, another exception occurred:#012#012Traceback (most recent call last):#012 File "/home/Stevenliu1970/.virtualenvs/mapoenv/lib/python3.7/site-packages/requests/adapters.py", line 449, in send#012 timeout=timeout#012 File "/home/Stevenliu1970/.virt
[edit by admin: formatting]
Could you post more of the server log? I think some important stuff is getting trimmed off. Also, you can format the server log when you have pasted it by selecting it and then clicking the "code" button above the input box -- the one that is two lines with "101" on the first and "010" on the second.
Thanks giles, I rerun the same code as above.
-----------here is Traceback-------------
Environment:
Request Method: POST
Request URL: http://stevenliu1970.pythonanywhere.com/mapoapp/project_add/
Django Version: 3.0.5
Python Version: 3.7.5
Installed Applications:
['django.contrib.admin',
'django.contrib.auth',
'django.contrib.contenttypes',
'django.contrib.sessions',
'django.contrib.messages',
'django.contrib.staticfiles',
'django.contrib.sites',
'allauth',
'allauth.account',
'allauth.socialaccount',
'allauth.socialaccount.providers.facebook',
'allauth.socialaccount.providers.weixin',
'mapoapp']
Installed Middleware:
['django.middleware.security.SecurityMiddleware',
'django.contrib.sessions.middleware.SessionMiddleware',
'django.middleware.common.CommonMiddleware',
'django.middleware.csrf.CsrfViewMiddleware',
'django.contrib.auth.middleware.AuthenticationMiddleware',
'django.contrib.messages.middleware.MessageMiddleware',
'django.middleware.clickjacking.XFrameOptionsMiddleware']
Traceback (most recent call last):
File "/home/Stevenliu1970/.virtualenvs/mapoenv/lib/python3.7/site-packages/urllib3/connectionpool.py", line 667, in urlopen
self._prepare_proxy(conn)
File "/home/Stevenliu1970/.virtualenvs/mapoenv/lib/python3.7/site-packages/urllib3/connectionpool.py", line 930, in _prepare_proxy
conn.connect()
File "/home/Stevenliu1970/.virtualenvs/mapoenv/lib/python3.7/site-packages/urllib3/connection.py", line 316, in connect
self._tunnel()
File "/usr/lib/python3.7/http/client.py", line 921, in _tunnel
message.strip()))
During handling of the above exception (Tunnel connection failed: 403 Forbidden), another exception occurred:
File "/home/Stevenliu1970/.virtualenvs/mapoenv/lib/python3.7/site-packages/requests/adapters.py", line 449, in send
timeout=timeout
File "/home/Stevenliu1970/.virtualenvs/mapoenv/lib/python3.7/site-packages/urllib3/connectionpool.py", line 725, in urlopen
method, url, error=e, _pool=self, _stacktrace=sys.exc_info()[2]
File "/home/Stevenliu1970/.virtualenvs/mapoenv/lib/python3.7/site-packages/urllib3/util/retry.py", line 439, in increment
raise MaxRetryError(_pool, url, error or ResponseError(cause))
During handling of the above exception (HTTPSConnectionPool(host='uc571c95203f04140859ad85daa6.dl.dropboxusercontent.com', port=443): Max retries exceeded with url: /cd/0/inline/A4EPlQIaMakQovjC5opkqlVdbegvHkYyW9b0a1ZhXS3CjtP_9K49QU5ixPmUW7FBAsLUxiQ27-2-nqcEWwLmcRsSIXK_LoNSduUt5RDPGTLkQvslxmbDYDc6-JYtqzScNr8/file (Caused by ProxyError('Cannot connect to proxy.', OSError('Tunnel connection failed: 403 Forbidden')))), another exception occurred:
File "/home/Stevenliu1970/.virtualenvs/mapoenv/lib/python3.7/site-packages/django/core/handlers/exception.py", line 34, in inner
response = get_response(request)
File "/home/Stevenliu1970/.virtualenvs/mapoenv/lib/python3.7/site-packages/django/core/handlers/base.py", line 115, in _get_response
response = self.process_exception_by_middleware(e, request)
File "/home/Stevenliu1970/.virtualenvs/mapoenv/lib/python3.7/site-packages/django/core/handlers/base.py", line 113, in _get_response
response = wrapped_callback(request, *callback_args, **callback_kwargs)
File "/home/Stevenliu1970/.virtualenvs/mapoenv/lib/python3.7/site-packages/django/contrib/auth/decorators.py", line 21, in _wrapped_view
return view_func(request, *args, **kwargs)
File "/home/Stevenliu1970/mysite/mapoapp/views.py", line 139, in project_add
image_mapo_relative_path = save_thumbnail_image(request, project.image_link, project.street)
File "/home/Stevenliu1970/mysite/mapoapp/utilities.py", line 93, in save_thumbnail_image
print(f"Response body is {requests.get(img_url).content}", flush=True)
File "/home/Stevenliu1970/.virtualenvs/mapoenv/lib/python3.7/site-packages/requests/api.py", line 76, in get
return request('get', url, params=params, **kwargs)
File "/home/Stevenliu1970/.virtualenvs/mapoenv/lib/python3.7/site-packages/requests/api.py", line 61, in request
return session.request(method=method, url=url, **kwargs)
File "/home/Stevenliu1970/.virtualenvs/mapoenv/lib/python3.7/site-packages/requests/sessions.py", line 530, in request
resp = self.send(prep, **send_kwargs)
File "/home/Stevenliu1970/.virtualenvs/mapoenv/lib/python3.7/site-packages/requests/sessions.py", line 665, in send
history = [resp for resp in gen] if allow_redirects else []
File "/home/Stevenliu1970/.virtualenvs/mapoenv/lib/python3.7/site-packages/requests/sessions.py", line 665, in <listcomp>
history = [resp for resp in gen] if allow_redirects else []
File "/home/Stevenliu1970/.virtualenvs/mapoenv/lib/python3.7/site-packages/requests/sessions.py", line 245, in resolve_redirects
**adapter_kwargs
File "/home/Stevenliu1970/.virtualenvs/mapoenv/lib/python3.7/site-packages/requests/sessions.py", line 643, in send
r = adapter.send(request, **kwargs)
File "/home/Stevenliu1970/.virtualenvs/mapoenv/lib/python3.7/site-packages/requests/adapters.py", line 510, in send
raise ProxyError(e, request=request)
Exception Type: ProxyError at /mapoapp/project_add/
Exception Value: HTTPSConnectionPool(host='uc571c95203f04140859ad85daa6.dl.dropboxusercontent.com', port=443): Max retries exceeded with url: /cd/0/inline/A4EPlQIaMakQovjC5opkqlVdbegvHkYyW9b0a1ZhXS3CjtP_9K49QU5ixPmUW7FBAsLUxiQ27-2-nqcEWwLmcRsSIXK_LoNSduUt5RDPGTLkQvslxmbDYDc6-JYtqzScNr8/file (Caused by ProxyError('Cannot connect to proxy.', OSError('Tunnel connection failed: 403 Forbidden')))`enter code here`
-------here is server log------------------
2020-05-20 21:17:26 Not Found: /favicon.ico
2020-05-20 21:17:31 Not Found: /static/facebook/js/fbconnect.js
2020-05-20 21:44:37 Wed May 20 21:44:37 2020 - received message 0 from emperor
2020-05-20 21:44:37 SIGINT/SIGQUIT received...killing workers...
2020-05-20 21:44:38 worker 1 buried after 1 seconds
2020-05-20 21:44:38 goodbye to uWSGI.
2020-05-20 21:44:38 chdir(): No such file or directory [core/uwsgi.c line 1610]
2020-05-20 21:44:38 VACUUM: unix socket /var/sockets/stevenliu1970.pythonanywhere.com/socket removed.
2020-05-20 21:44:43 *** Starting uWSGI 2.0.17.1 (64bit) on [Wed May 20 11:44:40 2020] ***
2020-05-20 21:44:43 compiled with version: 5.4.0 20160609 on 09 January 2020 17:57:39
2020-05-20 21:44:43 os: Linux-4.4.0-1100-aws #111 SMP Thu Jan 9 16:34:29 UTC 2020
2020-05-20 21:44:43 nodename: blue-liveweb23
2020-05-20 21:44:43 machine: x86_64
2020-05-20 21:44:43 clock source: unix
2020-05-20 21:44:43 pcre jit disabled
2020-05-20 21:44:43 detected number of CPU cores: 2
2020-05-20 21:44:43 current working directory: (unreachable)/etc/uwsgi/vassals
2020-05-20 21:44:43 detected binary path: /usr/local/bin/uwsgi
2020-05-20 21:44:43 *** dumping internal routing table ***
2020-05-20 21:44:43 [rule: 0] subject: path_info regexp: \.svgz$ action: addheader:Content-Encoding:gzip
2020-05-20 21:44:43 *** end of the internal routing table ***
2020-05-20 21:44:43 chdir() to /home/Stevenliu1970/
2020-05-20 21:44:43 limiting number of processes to 60...
2020-05-20 21:44:43 your processes number limit is 60
2020-05-20 21:44:43 your memory page size is 4096 bytes
2020-05-20 21:44:43 detected max file descriptor number: 123456
2020-05-20 21:44:43 building mime-types dictionary from file /etc/mime.types...
2020-05-20 21:44:43 552 entry found
2020-05-20 21:44:43 lock engine: pthread robust mutexes
2020-05-20 21:44:43 thunder lock: disabled (you can enable it with --thunder-lock)
2020-05-20 21:44:43 uwsgi socket 0 bound to UNIX address /var/sockets/stevenliu1970.pythonanywhere.com/socket fd 3
2020-05-20 21:44:43 Python version: 3.7.5 (default, Nov 14 2019, 22:26:37) [GCC 5.4.0 20160609]
2020-05-20 21:44:43 Set PythonHome to /home/Stevenliu1970/.virtualenvs/mapoenv
2020-05-20 21:44:43 *** Python threads support is disabled. You can enable it with --enable-threads ***
2020-05-20 21:44:43 Python main interpreter initialized at 0x21767e0
2020-05-20 21:44:43 your server socket listen backlog is limited to 100 connections
2020-05-20 21:44:43 your mercy for graceful operations on workers is 60 seconds
2020-05-20 21:44:43 setting request body buffering size to 65536 bytes
2020-05-20 21:44:43 mapped 334256 bytes (326 KB) for 1 cores
2020-05-20 21:44:43 *** Operational MODE: single process ***
2020-05-20 21:44:43 initialized 38 metrics
2020-05-20 21:44:43 WSGI app 0 (mountpoint='') ready in 3 seconds on interpreter 0x21767e0 pid: 1 (default app)
2020-05-20 21:44:43 *** uWSGI is running in multiple interpreter mode ***
2020-05-20 21:44:43 gracefully (RE)spawned uWSGI master process (pid: 1)
2020-05-20 21:44:43 spawned uWSGI worker 1 (pid: 2, cores: 1)
2020-05-20 21:44:43 metrics collector thread started
2020-05-20 21:44:43 spawned 2 offload threads for uWSGI worker 1
2020-05-20 21:44:45 announcing my loyalty to the Emperor...
2020-05-20 21:45:13 Not Found: /static/facebook/js/fbconnect.js
2020-05-20 21:52:37 URL is https://www.dropbox.com/s/51y6fvxqj2swy8w/676%20Stud%20Road%20External%20Cam%20%20%20%20%20%20%20%20%20%20%204_large.jpg?raw=1
2020-05-20 21:52:39 Internal Server Error: /mapoapp/project_add/#012Traceback (most recent call last):#012 File "/home/Stevenliu1970/.virtualenvs/mapoenv/lib/python3.7/site-packages/urllib3/connectionpool.py", line 667, in urlopen#012 self._prepare_proxy(conn)#012 File "/home/Stevenliu1970/.virtualenvs/mapoenv/lib/python3.7/site-packages/urllib3/connectionpool.py", line 930, in _prepare_proxy#012 conn.connect()#012 File "/home/Stevenliu1970/.virtualenvs/mapoenv/lib/python3.7/site-packages/urllib3/connection.py", line 316, in connect#012 self._tunnel()#012 File "/usr/lib/python3.7/http/client.py", line 921, in _tunnel#012 message.strip()))#012OSError: Tunnel connection failed: 403 Forbidden#012#012During handling of the above exception, another exception occurred:#012#012Traceback (most recent call last):#012 File "/home/Stevenliu1970/.virtualenvs/mapoenv/lib/python3.7/site-packages/requests/adapters.py", line 449, in send#012 timeout=timeout#012 File "/home/Stevenliu1970/.virt
The same code works fine on localhost, the URL is accessable with Chrome. the problen happened after I upload the code to Pythonanywhere.
what is urlopen? and is it obeying the proxy?
Hey Guys,
I did not pursue this further, but I am interested in a resolution. Not wanting to debug the problem for a small pet project, I continued using my local machine. I've also lately been using COLAB, but, if I could use pythonanywhere for these smaller projects, that would be great. I often do not have access to my "local" hardware during the day.
Kokomodo
This is what I get. (part of the url is deleted, but it's the correct url. I commented out everything in my code except for the two print statements Giles recommended. So, it's the requests.get().content method that is throwing the error, whatever the source.
(For what it's worth, calling requests.get(data_url).content on my phone, which also uses a non-local compiler yields the expected result, i.e., the correct URL, and the data content (text) contained in the file linked to the url.)
URL is https://www.dropbox.com/s/#deleted#/Weight%20Data%202020.txt?raw=1
Traceback (most recent call last):
File "/usr/lib/python3.7/site-packages/urllib3/connectionpool.py", line 594, in urlopen
self._prepare_proxy(conn)
File "/usr/lib/python3.7/site-packages/urllib3/connectionpool.py", line 805, in _prepare_proxy
conn.connect()
File "/usr/lib/python3.7/site-packages/urllib3/connection.py", line 308, in connect
self._tunnel()
File "/usr/lib/python3.7/http/client.py", line 921, in _tunnel
message.strip()))
OSError: Tunnel connection failed: 403 Forbidden
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
[edit by admin: formatting]
Hi conrad & Kokomodo,
Thank you very much for your response. all the best.
Hi gile, I still hope this problem could be resolved, the story is I originally create image anchers to Dropbox and google one drive on my web app, but as the original image's size too large (hundreds of MB), it is not as fast as expected then the end user brows my web. so I try to download and thumbnail it(7-9KB ONLY) plus save it on my pythonanywhere web locally. On my web app, I use this pythonanywhere local image now instead of the original ones, it is much much faster.
on localhost, the code works fine both for dropbox and google one drive. but when uploaded to pythonanywhere, google one drive still fine, the problem only happens with dropbox.
As Conrad mentioned, it might be somthing wrong with urllib3 urlopen and the proxy.servere, I tried requests and imread as well but the same error happens. I dont know how to use proxy.sever if that's the cause.
-------------the whole related code script as following---------------
#utilities.py
import requests # for here_api_location
from PIL import Image
import numpy as np
import urllib.parse
from urllib.request import urlopen
import pathlib
import os
from imageio import imread
from mysite.settings import BASE_DIR
def save_thumbnail_image(request, img_url, street):
image_mapo_relative_path = ''
if 'www.dropbox.com' in img_url:
#img_url = urllib.parse.unquote(img_url).replace(" ", "").split('?')[0] + '?raw=1'
img_url = img_url.split('?')[0] + '?raw=1'
# try:
print(f"URL is {img_url}", flush=True)
print(f"Response body is {requests.get(img_url).content}", flush=True)
img_object = Image.open(urlopen(img_url)) #'www.dropbox.com' need to be white listed by pyrhonanywhere
#response = requests.get(img_url)
#img_object = Image.open(response.raw)
#img_object = imread(img_url)
# except:
# return image_mapo_relative_path
img_array = np.array(img_object)
img_thumbnail = Image.fromarray(img_array)
size = (200, 200)
img_thumbnail.thumbnail(size)
relative_path = r'static/mapoapp/images/projects/' + request.user.username + '/'
absolute_path = os.path.join(BASE_DIR, relative_path)
if not pathlib.Path(absolute_path).exists(): # need to create a directory for the user
os.mkdir(absolute_path)
img_file_name = '_'.join(street.split())
file_type = '.jpg'
img_thumbnail.save(absolute_path + img_file_name + file_type)
image_mapo_relative_path = relative_path + img_file_name + file_type
return image_mapo_relative_path
elif 'drive.google.com' in img_url:
try:
img_object = Image.open(urlopen(img_url))
except:
return image_mapo_relative_path
img_array = np.array(img_object)
img_thumbnail = Image.fromarray(img_array)
size = (200, 200)
img_thumbnail.thumbnail(size)
relative_path = r'static/mapoapp/images/projects/' + request.user.username + '/'
absolute_path = os.path.join(BASE_DIR, relative_path)
if not pathlib.Path(absolute_path).exists(): # need to create a directory for the user
os.mkdir(absolute_path)
img_file_name = '_'.join(street.split())
file_type = '.jpg'
img_thumbnail.save(absolute_path + img_file_name + file_type)
image_mapo_relative_path = relative_path + img_file_name + file_type
return image_mapo_relative_path
else:
image_mapo_relative_path = ''
return image_mapo_relative_path
urllib does not automatically take the proxy settings from the environment. If you want to use urllib on a free account, you need to configure it to use the proxy. Check the urllib documentation for how to do that and use the settings that are on this page: http://help.pythonanywhere.com/pages/403ForbiddenError/
The below is what I'm using. It throws a 404 error, but I'm not sure I have it implemented correctly yet. Still working on it.
proxies = {"http": "proxy.server:3128", "https": "proxy.server:3128"}
print(f"URL is {data_url}", flush=True)
print(f"Response body is {requests.get(data_url, proxies=proxies).content}", flush=True)
-Kokomodo
<deleted>
OK, I think I may have worked out what the problem was. The clue was this error line:
Exception Value: HTTPSConnectionPool(host='uc571c95203f04140859ad85daa6.dl.dropboxusercontent.com', port=443): Max retries exceeded with url: /cd/0/inline/A4EPlQIaMakQovjC5opkqlVdbegvHkYyW9b0a1ZhXS3CjtP_9K49QU5ixPmUW7FBAsLUxiQ27-2-nqcEWwLmcRsSIXK_LoNSduUt5RDPGTLkQvslxmbDYDc6-JYtqzScNr8/file (Caused by ProxyError('Cannot connect to proxy.', OSError('Tunnel connection failed: 403 Forbidden')))`enter code here`
Historically, Dropbox was making files accessible on the host dl.dropboxusercontent.com. That was whitelisted, so everything worked. But now they appear to have changed so that it's on subdomains of that domain name, for example the host uc571c95203f04140859ad85daa6.dl.dropboxusercontent.com.
I've just whitelisted all hosts that end with .dl.dropboxusercontent.com, so your code should work now.
Thanks giles!!!!! My code works now, all good, thank you very much. My web like flying now.
Yes, the url call is working now, even without configuring the proxy. Thank you! Apparently my problem description ("moving target") was not far off. Thank you again for tracking down the problem.
