PythonAnywhere Forums

Whitelist submission

Hey guys! Not sure if there is a specific thread or not for whitelist submissions. But I am trying to create a cool recipe grabber for a project, but the Yummly API is not allowed.

I get a Forbidden error when trying to access it via my Flask app, and I'm pretty sure its because Yummly isn't "whitelisted". Any help here?

Thanks!

Here is a link to their API https://developer.yummly.com/ https://developer.yummly.com/documentation

Wow. $500/month is a pretty serious entry-level plan, respect to yummly! Will add it to the whitelist now...

OK, that's added to the whitelist now. Let us know if it all works OK.

Yeah no kidding. Luckily they have a temporary scholastic account which they let us have for a bit!

Thanks for adding it! I seem to also be running into a 403 error when I try and scrape html from specific pages. (i.e. I'm trying to get the directions from foodnetwork.com) I believe this is restricted as I'm not a premium user here?

Thanks for the help

That's correct. I don't see an API for foodnetwork.com, so they're not a candidate for whitelisting.

Would it be possible to add to the white list: http://amp.pharm.mssm.edu

They have a JSON api: http://amp.pharm.mssm.edu/Enrichr/help#api

I'm currently using a free pythonanywhere account to develop site. The group I'm developing site for is in process of paying for a paid pythonanywhere account.

Thank you.

No problem. That's on the whitelist now.

That's great. Thank you very much.

Is it possible to add http://genome.ucsc.edu/cgi-bin/das/ to the whitelist? I've an app that uses queries like "http://genome.ucsc.edu/cgi-bin/das/hg38/dna?segment=chr4:19281:20010" to access contents.

The website http://genome.ucsc.edu/ provides invaluable tools for querying the human genome.I could have got away by using a local copy of human genome, but it is slightly above the free disk-size ;)

Please whitelist... :)

We are happy to add any public api's to the whitelist if you provide us with the api documentation showing the official api endpoint.

AFAIK, they do not use APIs. They use something called Distributed Annotation System or DAS. Unfortunately, the main DAS website appears down.

My queries will be like http://genome.ucsc.edu/cgi-bin/das/hg38/dna?segment=chr4:19281:20010

Variables are hg38 and the part after segment

I'm also interested in accessing biology data. Further to karthikkn's post above, here's some more details about accessing that UCSC DAS server - the query format, and explaining that queries are permitted from interactive front-end sites (whereas programs should be limited to one hit every 15 seconds and no more than 5,000 hits per day):

http://genome.ucsc.edu/FAQ/FAQdownloads#download23

Hello,

Please add Wunderlist (documented here - https://developer.wunderlist.com/) to the whitelist. The endpoint is a.wunderlist.com

Thanks

I think, on balance, it's not a good idea for us to add the UCSC DAS server. They specifically state that they'd prefer that programs not use their site and the low usage limitation may mean that users will be trampling all over each other.

a.wunderlist.com is already on the whitelist.

Hi Glenn,

I agree about the usage limitation on the UCSC server. However I was thinking if you are using "iptables", would it be possible to add a rate limiting rule, such as:

iptables -A OUTPUT -d 128.114.119.132 -p tcp --dport 80 -o eth0 -m state --state NEW -m recent --update --seconds 60 --hitcount 10 -j REJECT

where: 128.114.119.132 is IP for genome.ucsc.edu

which would permit 10 hits within 60 seconds.

Similar to the "Example: Limit Connections Per Second" on this page: http://www.cyberciti.biz/faq/iptables-connection-limits-howto/

The whitelist is a quick way of giving some external to free users. We're not going to build out infrastructure for just one on the entries in the whitelist. Your solution also doesn't deal with one user using the entire allowance so that other users don't get to use the site.

@glenn

glenn wrote:

I think, on balance, it's not a good idea for us to add the UCSC DAS server. They specifically state that they'd prefer that programs not use their site and the low usage limitation may mean that users will be trampling all over each other.

That is why you should allow DAS, instead of the other popular method - the public MySQL server, that has more limitations (and size/CPU time issues in case of accessing from PythonAnywhere). Currently DAS is the best way to access the resource that I want in the most effective way.

I hope you reconsider this (or else I've to find some other APIs, request you to add it, and change my code).

Thanks!

It specifically says they prefer their site to be used interactively and don't want programs hitting the endpoint. As such, it is highly unlikely that we will add it to the whitelist.

OK. With great difficulty, I found an alternative source that provides similar API (http://togows.org/api/ucsc). Is is possible to whitelist it? Here's the documentation.

Okay, we have whitelisted togows.org for you.

Thank you so much conrad! :D

Would it be possible to whitelist tmi.twitch.tv ?

It allows accessing viewerlists with tmi.twitch.tv/group/user/CHANNELNAME/chatters

The channelname must be all lowercase and the site will return a json-object containg the viewerlist.

It's not part of twitch's regular api.twitch.tv, unfortunately.

Sorry, if you can't give us a link to the docs that show it's OK to use as an API, then we can't whitelist it.

I can't, it's not an official API :-(

https://horizon.stellar.org and https://horizon-testnet.stellar.org.

Public apis as described here: https://www.stellar.org/developers/horizon/reference/admin.html

I think my colleague Glenn whitelisted those yesterday after an email discussion, perhaps with you. They should work fine now.

could you please considering add nus.edu.sg onto the whitelist? it's for authentication thanks

Let's keep this all on the same thread.

I have a paid account and trying to add a website to the whitelist. It has no api and I am trying to use it with selenium. How do I do that?

If you have a paid account, you get full Internet access so whether a site is on the whitelist or not should not affect you. What happens when you try to access it using Selenium?

Could we add https://xboxapi.com/ to the whitelist? Here is the documentation https://xboxapi.com/documentation Thanks!

I've added it now -- let us know if it works OK?

Could "http://www.byond.com" be added to the whitelist?

Thanks

Hm. I'm not sure I understand exactly what byond is, but it looks like a game hosting platform, and i don't think it has a public API --- how do you imagine your code on PythonAnywhere interacting with byond?

Nevermind, I have chosen payment account because it needs more flexibility.

Thank you anyway

That works too! :-)

Hi! I need to run a script to download Boiler databases from the website 'https://rt2012-chauffage.com/public/frontend/', Could you please add it to your whitelist?

It's kind of urgent. Pls.

Thanks!

Hi there, I can't quite find the relevant documentation showing what the public api endpoint is. (also made harder because it's not in English)

Can you provide us with a link?

Alternatively, if it's very urgent, you could upgrade (paid users don't go through the proxy and get full external internet access).

Could you please add the Wikimapia API to whitelist? (http://wikimapia.org/). We are currently building a mobile application that needs processed data from this API. (http://wikimapia.org/api) The base URL is http://api.wikimapia.org/ Thank you in advance!!!

No problem. That's done.

Could you please add the easypost.com API? This service like a Stripe in shipping of packages world :) Thank you!

okay. I've add added api.easypost.com to the whitelist.

I am starting using PythonAnywhere and I would like to download the CSV with the Net Asset Value of an italian investment fund from their website, in this way:

urllib2.urlopen('https://www.animasgr.it/IT/Prodotti/Quotazioni-e-Performance/_layouts/15/GetExcel.ashx?type=QuoteFondo&code=A58A&i=01.04.2016&f=19.11.2016')

Is it possible to have the site whitelisted to avoid receiving:

urllib2.URLError: <urlopen error Tunnel connection failed: 403 Forbidden>

This would be great! Thanks in advance

Hi there,

We usually only whitelist sites that have an official, documented public API -- ie, websites that have been designed for consumption rather than machines. I'm not sure this site qualifies? It looks to me like they're intending for humans to manually download those excel files? But maybe I'm wrong -- perhaps they have some documentation somewhere?

Alternatively, premium accounts have unrestricted Internet, and start at $5/month, and there's a 30-day money back guarantee...

Hello, another whitelist request:

wyday.com/limelm/api/rest

Here is a link to their API docs

https://wyday.com/limelm/help/api/

Thanks!

Sure! That's done.

Hi,

This is a really easy easy easy image uploader:

http://uploads.im/

API info:http://uploads.im/apidocs

Could it be put on the whitelist too?

imgur is the alternative w.r.t your current image hostings but its much more complicated..

Thanks,

Fred

Ok. I've added that to the whitelist.

Thanks.

I'm having some trouble opening files, and checking file size using image.size.

Could you try this link : http://uploads.im/NfSoV.jpg I put a BMW on the site.

I get an error request saying it the website needs to be on the whitelist. I've also tried doing the same with an imgur photo but it said it isnt on the whitelist either?

It might just be something with my code.

Thanks,

Fred

Oops. Sorry, I misspelled it when I put it in the list. I've fixed it now.

Thanks for whitelisting so fast! (regarding my Jan 4th request: https://www.pythonanywhere.com/forums/topic/4177/#id_post_37663)

I'm still having a bit of trouble though, I tried another outbound call this morning on got this error:

ConnectionError: HTTPSConnectionPool(host='wyday.com', port=443): Max retries exceeded with url: /limelm/api/rest/ (Caused by ProxyError('Cannot connect to proxy.', error('Tunnel connection failed: 403 Forbidden',))

Is there a cache, or something that needs to reset after a whitelist update? This is a 443 request as well, not sure if that makes a difference.

Thanks in advance,

Neil

Sorry for the slow reply! That's done.

Hey, Can you please add https://api.line.me in whitelist? it is LINE app messaging API. Here is the link to the documentation: https://developers.line.me/messaging-api/overview.

Thanks in advance

No problem. I've added that to the whitelist.