Forums

When trying to receive a post request from the client I continue to get this error message in the log --

"2018-12-13 10:45:08,953: Exception on /reciever [POST] Traceback (most recent call last): File "/usr/lib/python3.6/site-packages/lxml/html/init.py", line 409, in get_element_by_id return _id_xpath(self, id=id)[0]"

Running my program works fine locally and on oceandigital vm but not on pythonanywhere. Can't figure out why.

That looks like it's just the first line of the exception stacktrace; could you give the complete trace?

BTW from the name of the function that you're calling, it looks like you might be using Selenium -- is the site you're connecting to one of the ones on the whitelist?

The rest of the errors are a result of variables not being assigned from the post request

2018-12-13 10:45:08,953: Exception on /reciever [POST]
Traceback (most recent call last):
 File "/usr/lib/python3.6/site-packages/lxml/html/__init__.py", line 409, in get_element_by_id
    return _id_xpath(self, id=id)[0]
 IndexError: list index out of range
 **NO MATCH**
 During handling of the above exception, another exception occurred:
 **NO MATCH**
 Traceback (most recent call last):
  File "/home/foosie/mysite/spotifyLyrics.py", line 86, in getLyrics
      return s.lyrics
   File "/home/foosie/.local/lib/python3.6/site-packages/tswift.py", line 76, in lyrics
      self.load()
    File "/home/foosie/.local/lib/python3.6/site-packages/tswift.py", line 68, in load
       lyric_div = tree.get_element_by_id('lyrics-body-text')
    File "/usr/lib/python3.6/site-packages/lxml/html/__init__.py", line 414, in get_element_by_id
   raise KeyError(id)
    KeyError: 'lyrics-body-text'

I am also not using selenium

Ah, sorry -- lots of other people with Selenium questions today and get_element_by_id is a method in Selenium, but of course it's also a method in lxml and you're using the latter.

I think that what's going on here is that your code is using the tswift package, which is a wrapper around the API provided by MetroLyrics. That API wasn't on our whitelist of sites that free accounts can access so when it made a request, it got an error page back. It tried to parse the error page as if it was a response from the MetroLyrics API, and generated that error.

Anyway, the MetroLyrics API is public and well-documented, so I've added it to the whitelist -- could you try again now and see if the error goes away, or at least changes?

Thank you for your help, however I'm still getting a similar error message :(

I know the function in my javascript file is obtaining the song and formatting into json properly. So I'm not sure why I'm getting a 500 server error when it gets posted to the server. Like I said the exact same code works fine on my machine and my digital ocean virtual machine.

I think the important difference is that you're accessing the site from a free PythonAnywhere account -- paid accounts on our site (like your DigitalOcean VM, which is also a paid service) get unrestricted Internet access, and of course from your own machine you have a regular Internet connection, but we have to filter the Internet access for free accounts -- we were getting too many hackers signing up and using us as a cut-out to hack into other sites before we started doing that. For paid accounts, we have payment details on file so that if people do really bad stuff we at least know who they are.

(Of course, hackers could possibly sign up for paid accounts with stolen credit cards and do bad stuff even now, but it adds enough friction to the process that in general they don't bother -- or if they do, our card processor is pretty good at spotting card fraud and letting us know before they do too much damage. Basically we're in the same situation as other hosting providers, like Digital Ocean, when it comes to that.)

Of course, the problem is that when we added filtering, code needs to work well with the filtering, and it can only access sites that are on the whitelist. I've taken a closer look at the tswift library, and it looks like instead of going to the MetroLyrics API, they're scraping the regular website. I thought that was a bit weird -- I found what appeared to be solid docs for the API -- but then I actually went to api.metrolyrics.com in my browser, and got an internal server error, so I guess they've stopped supporting it.

Unfortunately we can only whitelist official public APIs, so I can't add the regular MetroLyrics site to the whitelist. So I think the only way this will work on PythonAnywhere is if you upgrade to a paid account.

https://textgears.com/api/#documentation please add thi api to whitelist

No problem. I have added api.textgears.com to the whitelist.

Thank you for your help glenn.