PythonAnywhere Forums

Whitelist submission

Hey guys! Not sure if there is a specific thread or not for whitelist submissions. But I am trying to create a cool recipe grabber for a project, but the Yummly API is not allowed.

I get a Forbidden error when trying to access it via my Flask app, and I'm pretty sure its because Yummly isn't "whitelisted". Any help here?

Thanks!

Here is a link to their API https://developer.yummly.com/ https://developer.yummly.com/documentation

Wow. $500/month is a pretty serious entry-level plan, respect to yummly! Will add it to the whitelist now...

OK, that's added to the whitelist now. Let us know if it all works OK.

Yeah no kidding. Luckily they have a temporary scholastic account which they let us have for a bit!

Thanks for adding it! I seem to also be running into a 403 error when I try and scrape html from specific pages. (i.e. I'm trying to get the directions from foodnetwork.com) I believe this is restricted as I'm not a premium user here?

Thanks for the help

That's correct. I don't see an API for foodnetwork.com, so they're not a candidate for whitelisting.

Would it be possible to add to the white list: http://amp.pharm.mssm.edu

They have a JSON api: http://amp.pharm.mssm.edu/Enrichr/help#api

I'm currently using a free pythonanywhere account to develop site. The group I'm developing site for is in process of paying for a paid pythonanywhere account.

Thank you.

No problem. That's on the whitelist now.

That's great. Thank you very much.

Is it possible to add http://genome.ucsc.edu/cgi-bin/das/ to the whitelist? I've an app that uses queries like "http://genome.ucsc.edu/cgi-bin/das/hg38/dna?segment=chr4:19281:20010" to access contents.

The website http://genome.ucsc.edu/ provides invaluable tools for querying the human genome.I could have got away by using a local copy of human genome, but it is slightly above the free disk-size ;)

Please whitelist... :)

We are happy to add any public api's to the whitelist if you provide us with the api documentation showing the official api endpoint.

AFAIK, they do not use APIs. They use something called Distributed Annotation System or DAS. Unfortunately, the main DAS website appears down.

My queries will be like http://genome.ucsc.edu/cgi-bin/das/hg38/dna?segment=chr4:19281:20010

Variables are hg38 and the part after segment

I'm also interested in accessing biology data. Further to karthikkn's post above, here's some more details about accessing that UCSC DAS server - the query format, and explaining that queries are permitted from interactive front-end sites (whereas programs should be limited to one hit every 15 seconds and no more than 5,000 hits per day):

http://genome.ucsc.edu/FAQ/FAQdownloads#download23

Hello,

Please add Wunderlist (documented here - https://developer.wunderlist.com/) to the whitelist. The endpoint is a.wunderlist.com

Thanks

I think, on balance, it's not a good idea for us to add the UCSC DAS server. They specifically state that they'd prefer that programs not use their site and the low usage limitation may mean that users will be trampling all over each other.

a.wunderlist.com is already on the whitelist.

Hi Glenn,

I agree about the usage limitation on the UCSC server. However I was thinking if you are using "iptables", would it be possible to add a rate limiting rule, such as:

iptables -A OUTPUT -d 128.114.119.132 -p tcp --dport 80 -o eth0 -m state --state NEW -m recent --update --seconds 60 --hitcount 10 -j REJECT

where: 128.114.119.132 is IP for genome.ucsc.edu

which would permit 10 hits within 60 seconds.

Similar to the "Example: Limit Connections Per Second" on this page: http://www.cyberciti.biz/faq/iptables-connection-limits-howto/

The whitelist is a quick way of giving some external to free users. We're not going to build out infrastructure for just one on the entries in the whitelist. Your solution also doesn't deal with one user using the entire allowance so that other users don't get to use the site.

@glenn

glenn wrote:

I think, on balance, it's not a good idea for us to add the UCSC DAS server. They specifically state that they'd prefer that programs not use their site and the low usage limitation may mean that users will be trampling all over each other.

That is why you should allow DAS, instead of the other popular method - the public MySQL server, that has more limitations (and size/CPU time issues in case of accessing from PythonAnywhere). Currently DAS is the best way to access the resource that I want in the most effective way.

I hope you reconsider this (or else I've to find some other APIs, request you to add it, and change my code).

Thanks!

It specifically says they prefer their site to be used interactively and don't want programs hitting the endpoint. As such, it is highly unlikely that we will add it to the whitelist.

OK. With great difficulty, I found an alternative source that provides similar API (http://togows.org/api/ucsc). Is is possible to whitelist it? Here's the documentation.

Okay, we have whitelisted togows.org for you.

Thank you so much conrad! :D

Would it be possible to whitelist tmi.twitch.tv ?

It allows accessing viewerlists with tmi.twitch.tv/group/user/CHANNELNAME/chatters

The channelname must be all lowercase and the site will return a json-object containg the viewerlist.

It's not part of twitch's regular api.twitch.tv, unfortunately.

Sorry, if you can't give us a link to the docs that show it's OK to use as an API, then we can't whitelist it.

I can't, it's not an official API :-(

https://horizon.stellar.org and https://horizon-testnet.stellar.org.

Public apis as described here: https://www.stellar.org/developers/horizon/reference/admin.html

I think my colleague Glenn whitelisted those yesterday after an email discussion, perhaps with you. They should work fine now.

could you please considering add nus.edu.sg onto the whitelist? it's for authentication thanks

Let's keep this all on the same thread.

I have a paid account and trying to add a website to the whitelist. It has no api and I am trying to use it with selenium. How do I do that?

If you have a paid account, you get full Internet access so whether a site is on the whitelist or not should not affect you. What happens when you try to access it using Selenium?

Could we add https://xboxapi.com/ to the whitelist? Here is the documentation https://xboxapi.com/documentation Thanks!

I've added it now -- let us know if it works OK?

Could "http://www.byond.com" be added to the whitelist?

Thanks

Hm. I'm not sure I understand exactly what byond is, but it looks like a game hosting platform, and i don't think it has a public API --- how do you imagine your code on PythonAnywhere interacting with byond?

Nevermind, I have chosen payment account because it needs more flexibility.

Thank you anyway

That works too! :-)

Hi! I need to run a script to download Boiler databases from the website 'https://rt2012-chauffage.com/public/frontend/', Could you please add it to your whitelist?

It's kind of urgent. Pls.

Thanks!

Hi there, I can't quite find the relevant documentation showing what the public api endpoint is. (also made harder because it's not in English)

Can you provide us with a link?

Alternatively, if it's very urgent, you could upgrade (paid users don't go through the proxy and get full external internet access).

Could you please add the Wikimapia API to whitelist? (http://wikimapia.org/). We are currently building a mobile application that needs processed data from this API. (http://wikimapia.org/api) The base URL is http://api.wikimapia.org/ Thank you in advance!!!

No problem. That's done.

Could you please add the easypost.com API? This service like a Stripe in shipping of packages world :) Thank you!

okay. I've add added api.easypost.com to the whitelist.

I am starting using PythonAnywhere and I would like to download the CSV with the Net Asset Value of an italian investment fund from their website, in this way:

urllib2.urlopen('https://www.animasgr.it/IT/Prodotti/Quotazioni-e-Performance/_layouts/15/GetExcel.ashx?type=QuoteFondo&code=A58A&i=01.04.2016&f=19.11.2016')

Is it possible to have the site whitelisted to avoid receiving:

urllib2.URLError: <urlopen error Tunnel connection failed: 403 Forbidden>

This would be great! Thanks in advance

Hi there,

We usually only whitelist sites that have an official, documented public API -- ie, websites that have been designed for consumption rather than machines. I'm not sure this site qualifies? It looks to me like they're intending for humans to manually download those excel files? But maybe I'm wrong -- perhaps they have some documentation somewhere?

Alternatively, premium accounts have unrestricted Internet, and start at $5/month, and there's a 30-day money back guarantee...

Hello, another whitelist request:

wyday.com/limelm/api/rest

Here is a link to their API docs

https://wyday.com/limelm/help/api/

Thanks!

Sure! That's done.

Hi,

This is a really easy easy easy image uploader:

http://uploads.im/

API info:http://uploads.im/apidocs

Could it be put on the whitelist too?

imgur is the alternative w.r.t your current image hostings but its much more complicated..

Thanks,

Fred

Ok. I've added that to the whitelist.

Thanks.

I'm having some trouble opening files, and checking file size using image.size.

Could you try this link : http://uploads.im/NfSoV.jpg I put a BMW on the site.

I get an error request saying it the website needs to be on the whitelist. I've also tried doing the same with an imgur photo but it said it isnt on the whitelist either?

It might just be something with my code.

Thanks,

Fred

Oops. Sorry, I misspelled it when I put it in the list. I've fixed it now.

Thanks for whitelisting so fast! (regarding my Jan 4th request: https://www.pythonanywhere.com/forums/topic/4177/#id_post_37663)

I'm still having a bit of trouble though, I tried another outbound call this morning on got this error:

ConnectionError: HTTPSConnectionPool(host='wyday.com', port=443): Max retries exceeded with url: /limelm/api/rest/ (Caused by ProxyError('Cannot connect to proxy.', error('Tunnel connection failed: 403 Forbidden',))

Is there a cache, or something that needs to reset after a whitelist update? This is a 443 request as well, not sure if that makes a difference.

Thanks in advance,

Neil

Sorry for the slow reply! That's done.

Hey, Can you please add https://api.line.me in whitelist? it is LINE app messaging API. Here is the link to the documentation: https://developers.line.me/messaging-api/overview.

Thanks in advance

No problem. I've added that to the whitelist.

hey, can you guys add also this site to the whitelist? https://www.mercadobitcoin.com.br/ it has a json api: https://www.mercadobitcoin.net/api/ https://www.mercadobitcoin.net/tapi/v3/ Thanks!

that's done. let us know if it works ok?

it is partially working: the application i'm developing uses two api's from the website

this one is working: https://www.mercadobitcoin.net/api/

this one is not: https://www.mercadobitcoin.net/tapi/v3/

Thanks a lot for replying so fast!!

Hm, those are the same domains right, just different URLs? our proxy doesn't distinguish between the two, so something else must be going wrong...

curl -iv https://www.mercadobitcoin.net/tapi/v3/

works ok for me, gets a 200 response...

oh, you're right.

I'm getting [errno 111]. Will look for an answer in another topic. Thanks a lot for helping :)

The 111 is what you get if you try to access a server directly, without using the proxy. Some HTTP libraries -- requests, for example -- are proxy-aware, and pick up the correct settings from the system environment. Others -- the built-in Python modules, for example -- need to be explicitly configured.

If you're writing your own code to talk to the servers, then try using requests -- it's a great library and much easier to use than the built-in modules, as well as being proxy-aware :-)

If you're using an API that someone else has written to access the sites, then they may well have a way of specifying the proxy -- most do.

Hi, could be please whitelist this url : http://edition.cnn.com/sitemaps/sitemap-news.xml

I have a paid account but I still get <Response [403]> while accessing 'http://edition.cnn.com/sitemaps/sitemap-news.xml'. Can u please check this up?

are you using a new console/did you restart your webapp etc?

yep thanks it works now :)

Hi ,

Im developing a site that will have access to weather data from tiempo. com and aemet (meteorological spanish agency). Could you add this siet to the whitelist?

tiempo.com url: api.tiempo.com api doc: www.tiempo.com/documentacion_api/manual.pdf

aemet url: opendata.aemet.es

thanks

Ok. I've added api.tiempo.com to the whitelist, but not opendata.aemet.es because I can't get a response from that address.

Hi pythonanywhere Devs,

I'm developing a Django application that uses Login with Amazon / Pay with Amazon / Amazon Product API and it seems like their api is not on the white list.

https://api.amazon.com is their endpoint, and the documentation featuring it for login is here: http://login.amazon.com/website

Can this be added?

Thank you.

Have added it now. Let us know if it works OK?

I'd like to confirm that it works!

Thanks a bunch.

:)

Hi,

Thank by api.tiempo.com

url of aemet is a ssl url (sorry , i forgot this detail) https://opendata.aemet.es

url with info info about aemet api http://www.aemet.es/es/datos_abiertos/AEMET_OpenData

Could you add it to whitelist?

Thanks

No problem -- that's whitelisted now.

Thanks . It's working :-)

Excellent, thanks for confirming!

Hi. Could you please whitelist those two api urls.

1.https://api.hunter.io

2.http://PhantomJScloud.com/api

So far i am getting this error -

File "/usr/local/lib/python2.7/dist-packages/requests/adapters.py", line 465, in send raise ProxyError(e, request=request) requests.exceptions.ProxyError: HTTPSConnectionPool(host='api.hunter.io', port=443): Max retries exceeded with url: /v2/domain-search?domain=http://aiya-ame rica.com/&api_key=83ccb74e0bce39c67e199107597b2b1adf7e6e0b (Caused by ProxyError('Cannot connect to proxy.', error('Tunnel connection failed: 403 Forbidden' ,)))

Thanks

Sure! That's done.

Can we please whitelist Santiago's two public bike systems? bikesantiago.cl and bicilascondes.cl? Cheers lads!

Do they have an official public API? Send us links to the documentation and we can probably whitelist them.

Can we please add https://api.500px.com to the free user whitelist

Done- enjoy!

Hi, I am trying to send HTTP request to my sites hosted on openshift with url *.rhcloud.com

Is it possible to add it to the white list please ? Many thanks

I don't think that would qualify, no. We only whitelist official, documented public APIs.

Please add api.yourls.org. Documentation: https://api.yourls.org/doc/. Thanks in advance!

Please whitelist this URL: https://egov.uscis.gov/casestatus/mycasestatus.do. Thank you.

Both done. Let us know if they work OK?

Hi harry, It works. thanks a lot

Hi.. I am trying to create an ebook grabber project. Can you please whitelist this URL: https://www.packtpub.com/. Thank you so much

Where is the documentation for the API?

It seems like they didn't provide API. Can't it work without API?

If there is no publicly published API, then it's not a candidate for inclusion in the whitelist.

api.yourls.org works fine, thanks!

Hey guys.

Could apiv2.pushshift.io be added to the list? The first version of the API (api.pushshift.io) is already allowed but it has been discontinued in favor of the new endpoint.

Where are the docs?

As far as I could tell, this is the available documentation.

No problem. I've added that to the whitelist.

Thanks a bunch, glenn!

Could you add this api: https://smsgateway.me/sms-api-documentation/messages/send-message-to-number

Sure! That's added.

Thank you very much, giles!

please whitelist this api url - https://api.apifier.com

No problem. I've added that to the whitelist.

Hi, I would like to ask for three websites to be whitelisted :

http://api.forismatic.com/api/1.0/

https://andruxnet-random-famous-quotes.p.mashape.com/?cat=famous

http://quotesondesign.com/wp-json/posts?filter[orderby]=rand&filter[posts_per_page]=1

which are the apis for the three following websites :

http://forismatic.com/en/

https://www.mashape.com

http://quotesondesign.com

I hope to hear from you soon, Thanks !

mashape was already on the whitelist. I've added the others.

Could you add this api: http://imlab.grid.by/appsegmxr/gallery/

http://imlab.grid.by

That looks like a website for interacting with the data. I don't see any API documentation.

T_T http://imlab.grid.by/appsegmxr/gallery/

T_T

That's just some JSON, it's not API documentation.

Can gbfs.citibikenyc.com be added? It's the general bikeshare feed for Citibike NYC data. Info here:https://www.citibikenyc.com/system-data

Thanks!

Can gbfs.citibikenyc.com be added? It's the general bikeshare feed for Citibike NYC data. Info here:https://www.citibikenyc.com/system-data

Thanks!

Sure! That's whitelisted now.

Could you please white-list https://habitica.com/apidoc/ and their API https://habitica.com/api/ .

Thanks a lot! ^_^

Sure! That's done.

Thanks a lot mate. Works like a charm!!

:-)

Sorry for bothering ya again. Could you please white-list https://www.personalityforge.com/chatbot-api-docs.php and their API https://www.personalityforge.com/api/chat/

I would love to make my agent a bit more intelligent when he doesn't have an answer ( functionality ) for what he was asked about.

Much appreciated ;P

Have added that now. Let us know if it works OK?

Perfect. Thanks a lot Harry. Works perfectly :)

Hi! Could you please whitelist these two domains?

1) rusvectores.org It provides access to distributional semantic models for the Russian language, they have API, as can be seen here http://rusvectores.org/en/

2) tesuck.eveel.ru It's a web-service for extracting key phrases from texts. They only have documentation in Russian (https://nlpub.ru/Tesuck) and their website is also fully in Russian (http://tesuck.eveel.ru/about), but they also have an API.

I teach Python to linguistics students, and my students are now working on a bunch of projects, which they are supposed to host here on pythonanywhere. The projects use those two APIs =)

Hope you can help!

No problem. I've added rusvectores.org and tesuck.eveel.ru to the whitelist.

Many thanks!