Forums

:::python

13:54 ~ (master)$ python3
Python 3.7.0 (default, Aug 22 2018, 20:50:05) 
[GCC 5.4.0 20160609] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import pandas
>>> url = 'http://www.nseindia.com/content/historical/EQUITIES/2019/SEP/cm16SEP2019bhav.csv.zip'
>>> df = pandas.read_csv(url, compression='zip')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python3.7/site-packages/pandas/io/parsers.py", line 678, in parser_f
    return _read(filepath_or_buffer, kwds)
  File "/usr/lib/python3.7/site-packages/pandas/io/parsers.py", line 424, in _read
    filepath_or_buffer, encoding, compression)
  File "/usr/lib/python3.7/site-packages/pandas/io/common.py", line 195, in get_filepath_or_buffer
    req = _urlopen(filepath_or_buffer)
  File "/usr/lib/python3.7/urllib/request.py", line 222, in urlopen
    return opener.open(url, data, timeout)
  File "/usr/lib/python3.7/urllib/request.py", line 531, in open
    response = meth(req, response)
  File "/usr/lib/python3.7/urllib/request.py", line 641, in http_response
    'http', request, response, code, msg, hdrs)
  File "/usr/lib/python3.7/urllib/request.py", line 569, in error
    return self._call_chain(*args)
  File "/usr/lib/python3.7/urllib/request.py", line 503, in _call_chain
    result = func(*args)
  File "/usr/lib/python3.7/urllib/request.py", line 649, in http_error_default
    raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 403: Forbidden
>>>

I am trying to get something called a 'bhav_copy' from the website of National Stock Exchange of India. It is essential for my project. I wish to download this file only once everyday. Please advise how to rectify this error or a workaround. Thanks in advance.

Free accounts can only access sites on our whitelist, and that site isn't on it. I also believe that the National Stock Exchange of India block incoming connections from cloud service providers like PythonAnywhere, so even in a paid account you would not be able to connect.

Thank you for your prompt reply. After I started this thread I read through the FAQ section and i now understand the reason for the error.

However I would like to request you to consider white listing only the following URL which provides one zipped CSV file that contains End of Day prices of stocks.

https://www.nseindia.com/content/historical/EQUITIES/

This file is required only once a day and will not be a huge traffic burden for NSE. The data in this file is static in nature and as such does not change given the time of day. It is available much after the stock market as closed for trading. Also this this is the one and only source of this information and there are no services as of now that publishes this information. I therefore beg you to consider adding that url to your white list if possible.

The problem is that it's blocked from the other end -- even if we whitelisted it, you would not be able to access it from our servers.

I am also getting the same error with the url: 'https://docs.google.com/forms/d/e/1FAIpQLSeDSC6eoL9swnSzJcMHPIRMC3crefjNyr2NeSRcr6ktEMXzOw/formResponse' Since i have the "hacker" subscription, i assume python anywhere could also be blocked by google forms?

It's possible, but they might just be blocking automated access in general -- if I'm reading that URL correctly, it's one that's designed for human consumption rather than for automated access. What are you trying to do by accessing the form? Perhaps they provide a proper API for filling in forms instead of using a web-scraping solution.

Thanks for the quick reply. It's definitely for human use, it's to fill in guest information for airbnb bookings for a condo association. The weird thing is that it works intermittently. I'll look into an api.

It might also be worth looking at the response you're getting in more detail -- even if the status code is 403, it might have some details as to why they're refusing the request in the page content.

If u can add nseindia.com with whitelist atleast block from our side. We can access then via headers.

Replied to you here

I have same problem. for solving my problem I request you to add the following URL in the whitelist so I can access it. I have a very simple small project on which the user can get a message when his result appears on the website. so I need to access the URL so can you please add the following URL? https://www.gtu.ac.in/result.aspx

We can only add sites if they have an official public API -- if that site has one, please post a link to the API docs.

Hi,

Getting same error but via module, using as below

from nsetools import Nse

It does connect same link internally. It is possible to make it available.

We need an endpoint name and official api docs.

Nsetools is a package in https://pypi.org/ Which is official website. The same should be allowed right.

we allow the install of the nsetools package, but that doesn't mean that the package will work. also I believe that india nse blocks access from computers outside of india- so you wouldn't be able to use our services (which is outside of india)

Hi Admin, I don’t think 403 is blocked from their side its blocked by pythonanywhere. I don’t think their is any good site other than python anywhere when its very neat to execute and deploy. Please see if you can grant access to nseindia.com. I truely feel this fill fix above issues.

Please see to it.

Thanks

Currently as a free user you are being blocked by PythonAnywhere. You can try upgrading, but I believe you may then be blocked by NSE.

please whitelist cowin public api i can execute my code

Send us a link to the API documentation and we'll consider it for the whitelist.

from nsetools import Nse nse = Nse() print(nse) q = nse.get_quote('infy')

How to run this?

If that's to access nseiindia, then it probably will not work since it appears that they are blocking requests from PythonAnywhere.

Can we use zerodha api on Python anywhere? can i use KiteConnnect api on Python anywhere?

Possibly. We do not have any special access that allows us to know what sites blocking policies are. Send us links to the API documentation for the sites and we'll consider them for the whitelist.

https://api.kite.trade === Root Api Point https://kite.trade/docs/connect/v3/ ======API Document.

api.kite.trade is already on the whitelist; are you having any problems using it?

It is taking load Response time is somewhat wrong, Nothing Else

Could you give more details? What are you passing to it, and what are you getting back?

Hi. I'm getting Error 403 despite having a paid subscription. The site I'm trying to access is "https://committees.parliament.uk/.....". Do you know if this is blocked from their side?

If you're using a console that you've started since you upgraded your account (older ones might have kept the old account settings) then it's likely that the 403 is coming from their side --- if you print out the content of the response you're getting back, then that should give some indication of the problem.

Hey, hi. When I am trying to fetch news from forbes website, I am getting error 403. Is anything you can do for it? I dont think so forbes have any api. Can you help me in resolving this error. please

Hey sorry we only add sites to the list if they have an official, public, documented API -- ie, sites that are designed and intended for machine consumption rather than human consumption. To consume the Forbes website you'd need a paid account

If I setup a paid account, then will you all be adding forbes site into whitelist? I am little bit confused here, one side you are encouraging to get a paid account and even if some sites like forbes doesn't have official and public API. How is it possible to add the forbes site?

If you have a paid account, we do not need to add it, since you will be able to access it anyway. We do not restrict internet access for paid accounts.