Forums

Proxy error when using requests in Pythonanywhere

Hi there, I'm using requests library (through Soundcloud module) and I'm having an error:

Caused by ProxyError('Cannot connect to proxy.', error('Tunnel connection failed: 403 Forbidden' when I try to access soundcloud API.

The strange thing was that in interactive mode I was able to connect it successfully with the API, but when I run my app it fails.

Best

That is very strange. Are the interactive mode and the web app definitely using the same API domain for soundcloud?

Disregard the interactive thing, entirely my fault. Just learned that the API endpoint that caused the failure was not the one I was testing in interactive mode.

The API endpoint that caused it is called /oembed (https://developers.soundcloud.com/docs/oembed) and its URL is:

http://soundcloud.com/oembed

I see in PythonAnywhere whitelist api.soundcloud.com but not soundcloud.com/oembed

Is it possible to white list this URL (since it is part of the Soundcloud API)?

Best piqoni

No problem. Try it now?

You guys rock!

It works.

Thanks

sorry for the trouble but could you please add nus.edu.sg on the whitelist because it is used for authentication BTW, i saw ntu on the list.....

If there's an official public API there, we can whitelist it -- just give us a link to the API docs.

https://wiki.nus.edu.sg/display/ivlelapi/Summary

https://wiki.nus.edu.sg/display/ivlelapi/Getting+Started

From that, it looks like ivle.nus.edu.sg is the correct API endpoint. Does that sound right to you?

yes
thanks

may i check when will it be added to the whiteliest. best regards

Would it be possible to add waterlevel.ie/data/ ? I'm trying to generate some graphs for our university kayak club webpage, link to the api docs: https://waterlevel.ie/page/api/

Both of those are whitelisted now.

Could you please whitelist https://api.darksky.net/forecast/ API and ip-api.com/json. I'm using it for rain prediction. Docs link: https://darksky.net/dev/docs

OK, have added those two. let us know if it works oK?

They work fantastically! Thanks!

Could you please whitelist script.googleusercontent.com ? I'm using pythonanywhere with google apps script web app.

ok. I looked into the documentation and I believe script.googleusercontent.com is used when google apps script returns content after you hit script.google.com, and is used as a one-time url.

I've whitelisted script.googleusercontent.com for you.

Thank you so much!!

Hi PythonAnywhere friends!

I am getting a proxy error on a simple subscribe request to the mailchimp api. Is it possible to get this whitelisted?

The endpoint im hitting looks like this: https://us14.api.mailchimp.com/3.0/lists/xxxxxxxxx/members/

Thanks so much and blue skies!

Hi there, I've added *.api.mailchimp.com to the whitelist now, let us know if it works OK!

Hi can anybody help to add api.hunterexpress.com.au into whitelist?

Thanks

I couldn't find any api docs there. Could you send a link to the api docs?

Hi Glenn here is API doc: http://staging.hunterexpress.com.au/apidocs/

Thanks

No problem. I've added both the live and the testing domains to the whitelist.

Work like a charm, thanks Glenn!

Hi guys, can you please white list https://app.asana.com/api ?

Thanks!

okay, we have whitelisted app.asana.com for you!

Hello guys, I see that you have added https://api.darksky.net/forecast/ API to the whitelist can you also please add 'https://darksky.net/details/' as well. I'm using it to get historical weather data.

Thanks.

Hi there, that second url looks designed for human consumption, not machine consumption, so that tends to rule it out from being added to the whitelist I'm afraid.

paid accounts start from just $5 tho!

Hello, Sir, I want to access KiiCloud for a datastore, could you please add it to whitelist? English Doc: https://en.kii.com API: https://api-jp.kii.com

Ok. I've added that to the whitelist.

Hello, Sir, thanks for your quick response. However, when I access the service via KiiCloud SDK, the error "urllib2.URLError: <urlopen error Tunnel connection failed: 403 Forbidden>" remains. (The url is https://api-jp.kii.com/api/oauth2/token) I also restarted the bash console. May you help me ?

Ah. Sorry about that - I didn't finish the process properly. It's working now.

Thanks, it worked.

Hello, Sir, After you set https://api-jp.kii.com into your whitelist, it worked well. But it suddenly does not work now. The request worked well these day, but now it took a very long time, and failed. Is there anything wrong with it? FYI, no problem for accessing the same KiiCloud from my local or from Google.

It seems proxy server is down.

Oh, now it works : )

Doesn't seem to work for me, I get the ProxyError all morning :(

Could you check again now? We just restarted it.

It works now, thank you!

Excellent, thanks for confirming!

It's working now. Thanks.

Thanks for confirming!

Hai, can you whitelist https://xtremetricks.net/ this link? im using it and im getting error('Tunnel connection failed: 403 Forbidden'). Please help

Send us a link to the API documentation and we'll consider it.

Give me your email id or something

please i need it... i use that website to collect information using my python script

Do they have API docs? Where are they?

No it is just a website

Unfortunately we can only whitelist sites with an official public API.

Hi, Sir, Do you have https://www.twilio.com in your whitelist?

FYI: https://www.twilio.com/docs/

yes. the full whitelist is here: https://www.pythonanywhere.com/whitelist/

Hi, Sir Can you add api.twilio.com to your white list? It seems twilio has changed its api server.

If .twilio.com is enough, I am sorry. I will check again the error log and my code.

let us know how you get on!

Hi there. I also get the ProxyError message. Could you please also whitelist the following api: host='api.opentransportdata.swiss'

Thank you for your help!

can do! can you send us a link to the api documentation?

more information re; requesting whitelist additions is available here, everyone:

http://help.pythonanywhere.com/pages/RequestingWhitelistAdditions/

Thank you for your reply. You can find the API documentation here: https://opentransportdata.swiss/en/dataset/fahrtprognose

Unfortunately, it's mostly in German. But the API is nothing else than train timetables etc. for the Swiss public transport system.

OK, have added that now, let us know if it works OK?

It works perfectly fine, thank you very much for your help!

Excellent, thanks for confirming!

Hi again. I got another tunnel error and kindly ask you to whitelist the following API: https://data.sbb.ch/api/records/1.0/search/?

The documentation can be accessed here: https://data.sbb.ch/explore/dataset/actual-data-sbb-previous-day/information/?timezone=Europe%2FZurich&refine.bpuic=8503011

Again, the API is for public transport information (a different one this time).

Thank you for your help!

Hi there, I've added that to the whitelist now, let us know if it works?

It works now, thank you so much!

:)

Can I request for Globe Labs API to be added to the whitelist too? Thank you very much!

API endpoint I'm accessing: https://developer.globelabs.com.ph/oauth/access_token

API Docs: http://www.globelabs.com.ph/docs/

Hello,Sir Here are several APIs of bitcoin exchanges in Japan.

Coincheck:

API:https://coincheck.com

DOC:https://coincheck.com/ja/documents/exchange/api

Bitflyer:

API:https://api.bitflyer.jp/v1/

DOC:https://bitflyer.jp/ja-jp/api

Bitbank:

API:https://public.bitbank.cc

        https://api.bitbank.cc/v1

DOC:https://docs.bitbank.cc

Zaif:

API:https://api.zaif.jp/tapi

DOC:https://corp.zaif.jp/api-docs/

Would you please add them in white list?

Thanks very much.

Regards

Those docs only refer to devapi.globelabs.com.ph, so I have whitelisted that.

Hello,

If you go to the "Opt-in via webform" section (http://www.globelabs.com.ph/docs/#getting-started-opt-in-via-webform), you'll see the endpoint I need (https://developer.globelabs.com.ph/oauth/access_token) in item number 6. Please whitelist that, thank you very much! :)

Ok. I see that. I have whitelisted the extra domain.

Alright, thank you!

No problem, glad to help!

can we get http://nodes.wavesnodes.com/api-docs/index.html whitelisted? port 443

i believe this is why i am getting the proxy error?

Sure! That's whitelisted now.

Can you please whitelist "https://www.pdfzorro.com/" API Docs here "https://www.pdfzorro.com/EDIT/api_anleitung.php" I am getting error accessing "https://edit.pdfzorro.com/pdf/***/**.pdf" Thank you in advance.

No problem, I've whitelisted www.pdfzorro.com. That won't cover edit.pdfzorro.com, though -- is there documentation that mentions that hostname?

Hello,Sir Here are several APIs of bitcoin exchanges in Japan.

Coincheck:

API:https://coincheck.com

DOC:https://coincheck.com/ja/documents/exchange/api

Bitflyer:

API:https://api.bitflyer.jp/v1/

DOC:https://bitflyer.jp/ja-jp/api

Bitbank:

API:https://public.bitbank.cc

    https://api.bitbank.cc/v1

DOC:https://docs.bitbank.cc

Zaif:

API:https://api.zaif.jp/tapi

DOC:https://corp.zaif.jp/api-docs/

Would you please add them in white list?

Thanks very much.

Regards

Sure! Those are whitelisted now.

Hello Sir, I want to use an API but I am getting error Tunnel Connection Failed.

API: https://api.stride.ai/doc_sentiment.json

API DOCS: https://stride.docs.apiary.io/#reference

Will you please add stride API to whitelist ?

Thank you very much.

no problem, have added that now, let us know if it works OK?

Hello . Can you whitelist "https://api.lendix.com" Thanks

Hi there, can you show us the api docs for api.lendix.com? Thanks

https://lendix.com/cgu-lendix-fr/

DOC "https://lendix.com/cgu-lendix-fr/"

Hmm, that page seems to be more a general terms and conditions page -- it mentioned api.lendix.com, but I can't see any documentation relating to the API specifically. Is there something that documents how to use the API?

Hi can you please whitelist "api.huobi.pro" ? this is a cryptocurrency exchange huobi.pro

Hello! could you whitelist "https://webapps2.abc.utah.gov/Production/OnlinePriceList/DisplayPriceList.aspx" please?! It is a public site from the state of Utah.

Edit: posted twice

@prateekjjw001, @JesseScraper -- could you give us links to the API documentation for those sites, please?

https://github.com/huobiapi/API_Docs_en/wiki/Huobi.pro-API there is github code and documentation. https://github.com/huobiapi/REST-API-demos

@prateekjjw001 what we need is a page on huobi.pro that says "our official API is at api.huobi.pro", or something like that.

@giles No API documentation available.

@giles No API documentation available.

Unfortunately, there is no such thing on their site. Only github api documentation is available

Then the site is not a candidate for inclusion in the whitelist.

Hi there, Could you please whitelist https://www.babelio.com/auteur/Frederic-Dard/7187/citations . I'm using this link in my Mooc to learning Python. Thank you so much.

Hi there -- could you give us a link to the API documentation that says it's OK to programmatically access that page?

Hi again, how can I get that?

If that site allows programs to scrape it, they'll say somewhere on the site.

Hello . Can you whitelist "https://api.elasticemail.com/v2/" Thanks

Hi, can you whitelist "https://playoverwatch.com" I did a simple scraper but i get the proxy error

Thanks

@ayevan and @Careerow, please post a link to the API documentation that shows the domain that the API is published on.

@glenn Unfortunately there are no API available for Overwatch, so I made a scraper of the "career" page The Url of the request is: https://playoverwatch.com/en-us/career/pc/battletag-1234

Unfortunately we can't whitelist sites without an official public API.

could you please whitelist https://weatherbit.io ?

API DOCS: https://www.weatherbit.io/api

Thanks!

Ok. I have added that to the whitelist.

Could you also please whitelist https://www.medanswering.com/Provider_API.taf? for me? API documentation here: https://www.medanswering.com/documents/Doc-MAS_Public_Site--2017-05-16-15-09-59.pdf

Thank you!

Ok. I have added that to the whitelist.

Awesome!

Can you whitelist textgain (https://api.textgain.com/), perhaps? Docs are here: https://www.textgain.com/api

Ok. I have added that to the whitelist.

@glenn

Can you add *.dol.gov ?

My app download few pdfs from efast.dol.gov (United States DEPARTMENT OF LABOR)

Could you give us a link to some kind of docs saying that automated access to the site is OK?

Also -- you have a paid account, so the whitelist won't affect outbound Internet access from your app -- is the one you're talking about running in a different, free account? If not, the problem won't be fixed by whitelisting the site.

Hi, I'm getting this same error for the endpoint https://hourlypricing.comed.com/api, docs at https://hourlypricing.comed.com/hp-api/

Ok. I have added hourlypricing.comed.com to the whitelist.

Hello, Sir, I want to access subscenes for getting movie subtitles, could you please add it to whitelist? English Doc: https://subscene.com API:sathya1124.pythonanywhere.com The sathya1124.pythonanywhere.com is working but when i type the movie name and click search it is producing forbidden error 403..

Could you give us a link to the official API documentation? This needs to be a page on the domain subscene.com that describes their API and how to connect to it.

this is not an offical api..i took that from github.. https://github.com/mamins1376/Subscene-API

If it's not an official API, unfortunately we can't whitelist it.

Hello, could you please add to whitelist the following API: Getbible (http://getbible.net/) API and Info (using JSON): https://getbible.net/api WEBSITE: https://getbible.net/about Thank you!

Ok. That's done.

Hello,could you add "adafruit.io" cloud api's in the whitelist

Best regards

endpoint is "https://io.adafruit.com/api/v2/giripranay/feeds/bin1/data?X-AIO-Key=a68f2ef379d4470780176f536af0f462"

Please provide a link to the official API documentation describing the endpoint that you are proposing to whitelist.

Wow, did not expect such a fast response. Link for API documentation:

https://io.adafruit.com/api/docs/#section/Authentication

Website : https://io.adafruit.com/

I am sending sensor data to adafruit cloud .When i want to retrieve my sensor data,i run a https request using API key provided by them.It worked well when i used it locally(using django on my PC).Now i want to host my website on pythonanywhere.It shows error when I run https request on pythonanywhere.

Best regards,

Thanks! That's whitelisted now.

Thanks a lot...!It's working now.

Excellent, thanks for confirming!

Hello, could you add a service 'pogoda.mail.ru' to the white list? End point https://pogoda.mail.ru/ Official documentation https://api.mail.ru/docs/

Hello, I create a bot for a social network that could send up-to-date weather data. For hosting decided to choose pythonanywhere. But I encountered an Proxy error. Could you add a service 'pogoda.mail.ru' to the white list? End point https://pogoda.mail.ru/ . Official documentation https://api.mail.ru/docs/. Best wishes.

it seems like their api is at www.appsmail.ru/platform/api, and not at pogoda.mail.ru, so we cannot whitelist that.

Thanks for the info and help.

Hi there, I am trying make a storage solution for my django application. Their main website is https://gofile.io and all the api calls are done using https://api.gofile.io/ Please whitelist them. Thank you.

Could you give us a link to documentation on the gofile.io site saying that automated access to the site is OK, and listing the api endpoints?

Can you whitelist https://in.bookmyshow.com?

Could you give us a link to the official API documentation? This needs to be a page on the domain bookmyshow.com that describes their API and how to connect to it.

Can you also add https://www.thestar.com.my/news/ to the whitelist? Im just scraping the website to gain data on news. Mainly for educational purpose?

If they have a publicly documented API, send us a link to the documentation and we'll consider it for the whitelist. If they do not have a publicly documented API, then it's not a candidate for the whitelist.

Can you please add http://app.neaea.gov.et/Home/result10 to whitelist? I'm trying to fetch student results from it but I can't connect since it is in blacklist.

Could you give us a link to the official API documentation? This needs to be a page on the domain app.neaea.gov.et that describes their API and how to connect to it.

.

Hello, your whitelist has the url WolframAlpha API (api.wolframalpha.com), but I can't download pictures from WolframAlpha because they have a different url: www4c.wolframalpha.com, www5b.wolframalpha.com, www5a.wolframalpha.com. (maybe there are others)

Official wolframalpha API documentation: https://products.wolframalpha.com/simple-api/documentation/

If you can provide a list of the image domains that are used, we can add them for you. Otherwise, we'll just spend ages adding one domain after another and waiting for a new one to appear.

I wrote in the previous post the domains I saw. Unfortunately, I'm not sure if this is a complete list, but I haven't come across any other domains.

I know of domains: www4c.wolframalpha.com, www5b.wolframalpha.com, www5a.wolframalpha.com.

We need public documentation for the URLs.

Is it possible to add to white list mafiaonline.ru ? It got public API (https://blog.mafiaonline.ru/api/), but its in russian...

On the API docs, I see a st.mafiaonline.ru, and a mafiaweb.ru. Can you confirm if there is anything else, or if those two would work?

Hello. I use the Yandex Disk API, but its url is not in the white list. Please add them. I use the REST API, its domain: https://cloud-api.yandex.net/ All url requests can be found on the polygon: https://yandex.ru/dev/disk/poligon/#!/%20/%20v1/disk/resources

Official documentation: https://yandex.ru/dev/disk/api/concepts/about-docpage/

No problem. I have added that to the whitelist.

Thanks

But I still get a connection error. What should I do?

Error: HTTPSConnectionPool(host='cloud-api.yandex.net', port=443): Max retries exceeded with url: /v1/disk/resources?path=temporar y_file&permanently=false&force_async=false (Caused by NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x7fcfac954d68>: Failed to estab lish a new connection: [Errno 101] Network is unreachable'))

You probably need to configure your code to use the proxy. See http://help.pythonanywhere.com/pages/403ForbiddenError/

Everything works on my home computer. Maybe you need to add all the domains provided in the documentation ?

All URLs are on the polygon: https://yandex.ru/dev/disk/poligon/#!/ / v1/disk/resources (each one starts with: https://cloud-api.yandex.net)

On your local machine, you are not using the proxy. See the link Glenn posted above.

All the subdomains of

.yandex.com
.yandex.net
.yandex.ru
.yandexcloud.net

are whitelisted

Can you whitelist https://www.manoramaonline.com and https://indianexpress.com/

We only whitelist sites with public api documentation.

Hi, coud you please whitelist https://api-covid19.rnbo.gov.ua/ ? This API currently doesn't have public documentation, but it's used by our official COVID info website https://covid19.rnbo.gov.ua/

It has to be an official documented API, unfortunately.

Hi can you please white-list https://apex.oracle.com/ Link for official documentation : https://docs.oracle.com/en/database/oracle/application-express/20.1/aeapi/index.html

I can't see anything on those pages that says that apex.oracle.com is itself an API -- the docs seem to be about an API built into Oracle databases. If there is an API on apex.oracle.com, could you give us a link to the docs specifically about that API?

Please refer section 43 : Using REST Administration Interface API

Reference : application-express-host:port is for standalone host, where as for cloud instance it is https://apex.oracle.com

Could you give us a link to the exact page where it says that?

Using REST Administration Interface API : https://docs.oracle.com/en/database/oracle/application-express/20.1/aeapi/Using-REST-Administration-Interface-API.html URL TERMS : https://docs.oracle.com/database/apex-18.1/AEUTL/about-Oracle-RESTful-services.htm#AEUTL447

http://<<server_address>>/<<schema_alias>>/hr/employees/7978

Sample cloud URL(Oracle's free Apex) https://apex.oracle.com/pls/apex/gravity/hr/employees/7654

where does it say apex.oracle.com is a free api that oracle provides? if we can't see that then you will have to upgrade your account.

https://apex.oracle.com/en/learn/getting-started/

Check the above link. It states, "Oracle APEX on Always Free Oracle Autonomous Database provides a preconfigured, fully managed and secured environment to both develop and deploy world-class applications. Your service remains free for an unlimited time as long as it is used."

And I am using it.

I am planning to upgrade my account, and before that I wanted to test and explore the abilities of pythonanywhere.

okay, I've whitelisted that for you.

https://gplinks.in/member/tools/api

Please whitelist this API

See my answer to your other forum post.

Thank you Conrad. will update you once i am done with the POC. :)