Forums

Started getting the following in one of my scripts:

No problems previously - seems to have cropped up a few times recently. Anyone know what the problem is?

Nov 11 12:29:59     db=Db()
Nov 11 12:29:59   File "/home/graingerkid/ontail_staging_env/ontailapp/search_console_utils/v2_downloaders/models.py", line 141, in __init__
Nov 11 12:29:59     use_unicode=True, charset="utf8mb4"
Nov 11 12:29:59   File "/usr/local/lib/python2.7/dist-packages/MySQLdb/__init__.py", line 81, in Connect
Nov 11 12:29:59     return Connection(*args, **kwargs)
Nov 11 12:29:59   File "/usr/local/lib/python2.7/dist-packages/MySQLdb/connections.py", line 193, in __init__
Nov 11 12:29:59     super(Connection, self).__init__(*args, **kwargs2)
Nov 11 12:29:59 _mysql_exceptions.OperationalError: (2005, "Unknown MySQL server host 'graingerkid.mysql.pythonanywhere-services.com' (110)")

That's strange. Does it happen for single requests, or are there extended periods where it happens?

Seems to be mainly via the Always on tasks - when I run it in a normal console there is no issue. when it happens it happens across all processes

Happy to help debug if I have come across a bug

It looks like the DNS system on the always on servers is not as stable as it could be. I'll do a bit of experimenting and debugging today to see what we can do about it.

Ok, if you need me to do anything, just let me know

Looks like I'm getting this again - happy to discuss on email which scripts etc in my account but basically in the always on tasks

We think we know what's causing it. We're just not sure how to fix it right now. Just catch the exception and retry in a bit.

Greetings, Glenn

I am experimenting the same issue. When I first created my free mysql DB this platform gave me an address and the login info requited to access it through my WEB app. But since I tried to create my DB the bash console of python outputs the same error messages. Besides, I'd like to add to this issue that when I "ping" the address of the service where is supposed to be my DB, that address is inaccessible. So I think the only reason why is giving this problem is cause the AWS services for mysql databases is inaccessible.

Maybe, if you ask for help or technical support in amazon they could correct the error.

Are you trying to access the database and ping it through bash consoles inside your PythonAnywhere account? Or from a bash shell on your own local machine? The MySQL databases are only accessible from inside PythonAnywhere.

I'm also having this issue on my "always on task", has it been resolved?

My always on tasks also started failing with the db connections, like:

sqlalchemy.exc.OperationalError: (pymysql.err.OperationalError) (2003, "Can't connect to MySQL server on 'XXXXXX.mysql.pythonanywhere-services.com' ([Errno -3] Temporary failure in name resolution)")

It looks like the DNS server on one of the nodes in our always-on system had a problem overnight -- I've fixed it now, and hopefully your tasks are running properly now -- from the logs, it looks like they are.

So today my always on task fail again with the same name resolution error today (after they restarted on on March h23rd):

Mar 25 15:28:19 sqlalchemy.exc.OperationalError: (pymysql.err.OperationalError) (2003, "Can't connect to MySQL server on 'XXXXXXXXXXX.mysql.pythonanywhere-services.com' ([Errno -3] Temporary failure in name resolution)")

This is already a couple of hours ongoing (since around Mar 25 09:44:35 server time UTC).

restarting the tasks helped, and they are running now.

Thanks for letting us know -- we noticed the problem at around 12:45 UTC and fixed it shortly after then. We're spending today working on getting reliable alerting so that if it happens again, the on-call engineer is paged and we can fix the issue quickly. (After that, of course, we'll work on fixing the underlying problem so that these DNS issues won't happen at all, but that will be a longer process so it's better to get the alerting in place first.)

My always-on task was working fine, but now it started to get an error:

Connectiong to database failed: (2005, "Unknown MySQL server host 'Aleksey.mysql.pythonanywhere-services.com' (110)")

But I did not change any code and:

1) Console still works.

2) MySQL Workbench works too.

3) My Django website uses the same database and there are no errors.

UPD: I do not know why, but after N number of connection attempts, my script now can connect to the database. Nothing was changed from my side.

The always on task servers sometimes have DNS issues. We have systems in place to ensure that they recover from it, but it appears that they did not work well this time. I will investigate to find out why the recovery systems failed.

Hi, I started getting this error in my always on task since this morning. Is there a fix for this or would you be able to look into it please?

Seeing the same thing on my sites tasks this morning too, any update?

edit: should mention restarting the task fixed the issue no problem, does the API have a method to target always on tasks for a restart (stop/start)? If this is going to happen with any kind of frequency I'd like to build a button my client can use to restart them from the web app itself as they don't have access to PA. Perhaps a snippet of code to show this would be nice?

Take a look at our helper_scripts for some examples https://github.com/pythonanywhere/helper_scripts

This error keeps happening for my always-on task. It kind of defeats the purpose of always-on, is there a way to prevent this from happening?

Could you let us know when this last happened? Someone else reported a problem with networking from always-on tasks in another forum thread, and while we don't have any indication of errors in the always-on system right now, two people reporting similar problems sounds like there might be something going wrong.

Hi, It happened just now 4:22 PM GMT and previously this morning at 10:42 AM GMT - if it can help the log is 24070.log Thank you

Thanks! That was really useful.

It looks like the monitoring on one of the always-on machines was broken in a way we'd never seen before, so we weren't getting alerted when there were network problems, and there was nothing clear in the logs. However, with the help of the timing information from your specific tasks's logs we were able to track down the problem, and fix it.

We're still looking into what happened with the alerting; this is definitely something that we should have been paged about, and for which there should have been clear indications of the issue in the system logs.

I am glad the logs file was helpful to address the issue. Thank you, Giles.

No problem, and sorry for the slow turnaround! We've fixed the monitoring problem now, too, so someone will be paged if there's another issue.

i am using sqlalchemy for mysql database but when i hit submit button in any form it shows me internal server 500 error, and in my error logs it shows me following error (sqlalchemy.exc.OperationalError: (MySQLdb._exceptions.OperationalError) (2005, "Unknown MySQL server host 'yashvardhanv.pythonanywhere-services.com' (0)")).and when i try (sqlalchemy object)db.create_all() in console it gives me same error. thanks in advance.

[edit]:- now it shows me following error in both console and error logs :- ( sqlalchemy.exc.OperationalError: (MySQLdb._exceptions.OperationalError) (2002, "Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2)"))

Please check your database host address -- it should be rather yashvardhanv.mysql.pythonanywhere-services.com than yashvardhanv.pythonanywhere-services.com.

Thank you for your response, i've changed that but it still shows me that error.

Are you sure you're reading the most recent error? They are at the bottom of your error log file.

yes im reading most recent error logs and getting same error in console when i try db.commit()[an sqlalchemy object] and i'm also reloading website from (web dashboard /editor) .

sqlalchemy.exc.OperationalError: (MySQLdb._exceptions.OperationalError) (2002, "Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2)")

Error at the bottom of my error log file.

[EDIT]

i tried removing extra '/' and it worked!!! [app.config['SQLALCHEMY_DATABASE_URI'] = 'mysql://]

Glad to hear that you made it work!