Forums

Getting Error Message with Proxy Server: Scrapy

Hi There,

I am new to the platform and this might sound really silly.

I am trying to run a scrapy spider script to scrape a website and I am getting error message as below.

File "/usr/local/lib/python2.7/dist-packages/scrapy/core/downloader/middleware.py", line 44, in process_request defer.returnValue((yield download_func(request=request, spider=spider))) TunnelError: Could not open CONNECT tunnel with proxy proxy.server:3128 [{'status': 403, 'reason': bytearray(b'Forbidden')}]

I hope there are some configurations/ settings that has to be changed and this should go away.

Any pointers?

Thanks! Shahbaz

Free accounts can only connect out over http(s) to a whitelist of sites: https://www.pythonanywhere.com/whitelist/. We only add sites to the whitelist if they have publicly documented APIs and if we think they will be useful to other free users.

Thats sad!