Forums

I see a lot of "OSError: write error" message in my error log. Been there for weeks.

I'm not getting any errors in my app although I am writing data to disk to cache it.

Is this coming from the server level? It appears so.

Should I be concerned? Any insight as to the source of the error?

Thanks, Chris

We had some trouble with networking on AWS that meant that one of our fileservers was slow. That may have affected your web app. It appears to be working now, so if your web app is still having trouble then it may be something else.

The app seems completely fine and I'm not getting an exceptions, etc. This seems to be coming from the server. Really the only impact seems to be that it is cluttering the log but if it is actually coming from my app that would be great to know.

Are they appearing alongside GeneratorExit messages? If so, this help page should explain what's going on.

Thanks, no. Just the OSError: write error message. Here's an excerpt:

2018-11-26 18:58:20,040: OSError: write error
2018-11-26 19:11:08,110: OSError: write error
2018-11-26 20:04:12,727: OSError: write error
2018-11-26 21:13:56,156: OSError: write error
2018-11-26 21:13:56,156: OSError: write error
2018-11-26 21:14:21,486: OSError: write error
2018-11-26 22:03:23,262: OSError: write error
2018-11-26 22:05:18,174: OSError: write error
2018-11-27 03:25:17,531: OSError: write error
2018-11-27 03:47:52,955: OSError: write error
2018-11-27 05:41:26,548: OSError: write error
2018-11-27 11:00:31,114: OSError: write error
2018-11-27 12:10:22,251: OSError: write error
2018-11-27 12:33:31,353: OSError: write error
2018-11-27 12:33:37,890: OSError: write error
2018-11-27 15:53:30,449: OSError: write error
2018-11-27 18:59:08,227: OSError: write error

Interesting. Is it correlated with slow requests at the same time in the access log? Each request handled by your site has one line there, and at the end of the line there will be a response time.

Thanks for the suggestion. I spot checked a few from Dec 02 (errors vs. access log) and some but not all of them correlate to longer requests. I am requesting data from an API that can take up to three seconds to respond, so my requests are occasionally that long.

During the longer requests I cache the results to disk and do a bunch of stuff with zero length files to provide locking functionality. I have some exception handling in there that could be hiding some errors. They all cover pretty obscure cases (lock file didn't exist but another process created between the check and the touch with exist_ok=False). Given my traffic volume, I don't expect there would be very many collisions.

In any event I have added debug messages to those three exception handlers to see if they correlate to the messages in the error log.

If so sorry for the hassle! I will let you know either way.

Chris

OK -- looking forward to hearing the results of your investigations!

It does not appear that the failed touch nor other exceptions are causing the message. As far as I can tell this doesn't originate from my code, but I could certainly be missing something.

Here's what I did:

• Turned on a lot of log / debug messages.
• Generated 12 requests / sec of test traffic to the site.
• I can see the various scenarios getting exercised in the code, including the failed touch call and related exception handling that I was suspicious of.
• I let the test traffic run for maybe 10 minutes. This generated approximately 7,000 requests (some take longer). On a given day the site only receives 300 - 700 page views so this was a good test.
• Checked the error log and didn't find any "OSError: write error" messages. I did see one from earlier but none during the test period.

Note also that for normal users I don't do anything unique to their identity (I do basic auth for admin functions only). So generating traffic this way should be a fair simulation of normal traffic.

If it's not coming from your code, it could be coming from the uWSGI middleware -- that's certainly where the OSError: write error messages combined with GeneratorExit messages that I mentioned earlier come from.

Maybe we can see if it's just a different symptom of the same cause? If you were to make a request to the site with a super-short timeout -- perhaps pick the slowest view -- maybe we could see if the same error appears:

requests.get('https://yoursite.com/slow-view', timeout=0.1)

I created a test script with a short timeout and ran it along side my test traffic generator. I caught the timeout exceptions and printed them. I can see a lot of timeouts.

I do see a lot of " OSError: write error". Apparently they are coming from the client timeouts.

In the sever log I see the result of the timeout with errors like the ones below. I understand and expect these.

2018-12-06 05:47:44 Thu Dec  6 05:47:44 2018 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request / (ip 10.0.0.98) !!!
2018-12-06 05:47:44 Thu Dec  6 05:47:44 2018 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during GET / (10.0.0.98)

The strange thing I see a lot more 'write error's vs. the 'writing to closed pipe' errors above on my production site but now that I know what I am looking for I can find those that are there.

Thanks for your help tracking this down! I was fixated on this being a disk error. Didn't think about pipes / client disconnects.

Chris

Excellent, glad you were able to get to the bottom of it!

I have in server log:

2019-03-29 15:01:22 Fri Mar 29 15:01:22 2019 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /1b3c4b5b-c9fc-495a-9032-ffab69d2f44d/rss/ (ip 10.0.0.162) !!!
2019-03-29 15:01:22 Fri Mar 29 15:01:22 2019 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during GET /1b3c4b5b-c9fc-495a-9032-ffab69d2f44d/rss/ (10.0.0.162)

In error log:

2019-03-29 15:01:22,987: OSError: write error

Also all my SQLite databases on pythonanywhere storage are locked (return apsw.BusyError: BusyError: database is locked when my Django website or scheduled task are trying to execute sql). The only way to unlock them: delete databases' files and re-upload to the same folders again.

Everything worked fine before 28-03-2019 21:00. It was not any changes in my own code. Any suggestions?

I am seeing tons of these errors in my logs suddenly. Started around Fri Mar 29 15:27:37. Happened around the time performance issues arose on both my site and and their admin pages. Maybe networking issues on their side??

https://www.pythonanywhere.com/forums/topic/14059/

hi there, I believe we corresponded over twitter. We had some performance problems on one of our servers. It should have recovered by now. We are investigating regarding root causes, and how to monitor and be alerted sooner etc.

I am getting this, the request says pending and then times out after 4 secs (I think that it times out because the max request time is 4 secs) I have good wifi, why am I getting an OSError: write error

The only timeout we have on our side is one of several minutes for website views. What URL are you getting the four-second timeout on?

Is there any way to disable logging of OSError: write error ? The whole my log looks like this:

[....]
    2019-08-17 10:26:16,406: OSError: write error
    2019-08-17 10:27:16,391: OSError: write error
    2019-08-17 10:28:16,412: OSError: write error
    2019-08-17 10:29:16,168: OSError: write error
    2019-08-17 10:30:16,172: OSError: write error
    2019-08-17 10:31:16,387: OSError: write error
    2019-08-17 10:32:16,436: OSError: write error
    2019-08-17 10:33:16,400: OSError: write error
    2019-08-17 10:34:22,409: OSError: write error
    2019-08-17 10:35:22,410: OSError: write error
    2019-08-17 10:36:22,431: OSError: write error
[....]

And server log:

[....]
    2019-08-17 10:29:16 Sat Aug 17 10:29:16 2019 - uwsgi_response_write_body_do(): Broken pipe [core/writer.c line 429] during HEAD /25442dc2-d0e9-42e8-8296-46adc2f03008/rss/ (10.0.0.235)
    2019-08-17 10:30:16 Sat Aug 17 10:30:16 2019 - uwsgi_response_write_body_do(): Broken pipe [core/writer.c line 429] during HEAD /25442dc2-d0e9-42e8-8296-46adc2f03008/rss/ (10.0.0.235)
    2019-08-17 10:31:16 Sat Aug 17 10:31:16 2019 - uwsgi_response_write_body_do(): Broken pipe [core/writer.c line 429] during HEAD /25442dc2-d0e9-42e8-8296-46adc2f03008/rss/ (10.0.0.235)
    2019-08-17 10:32:16 Sat Aug 17 10:32:16 2019 - uwsgi_response_write_body_do(): Broken pipe [core/writer.c line 429] during HEAD /25442dc2-d0e9-42e8-8296-46adc2f03008/rss/ (10.0.0.235)
    2019-08-17 10:33:16 Sat Aug 17 10:33:16 2019 - uwsgi_response_write_body_do(): Broken pipe [core/writer.c line 429] during HEAD /25442dc2-d0e9-42e8-8296-46adc2f03008/rss/ (10.0.0.235)
    2019-08-17 10:34:22 Sat Aug 17 10:34:22 2019 - uwsgi_response_write_body_do(): Broken pipe [core/writer.c line 429] during HEAD /25442dc2-d0e9-42e8-8296-46adc2f03008/rss/ (10.0.0.235)
    2019-08-17 10:35:22 Sat Aug 17 10:35:22 2019 - uwsgi_response_write_body_do(): Broken pipe [core/writer.c line 429] during HEAD /25442dc2-d0e9-42e8-8296-46adc2f03008/rss/ (10.0.0.235)
    2019-08-17 10:36:22 Sat Aug 17 10:36:22 2019 - uwsgi_response_write_body_do(): Broken pipe [core/writer.c line 429] during HEAD /25442dc2-d0e9-42e8-8296-46adc2f03008/rss/ (10.0.0.235)
[....]

I can not see any useful information in my logs because they are totally overloaded with this error caused by one of my client's connection. Everything is okay with my website responses.

You get that error when the client that is trying to reach your web app disconnects before the response has been fully sent. We have no way to disable it.

If you're getting it after 4 seconds, then the client has a 4 second timeout set.

Same for me if i am trying to invoke any API getting "OSError: write error" in error.log, even the API is working fine from other server

You're not clear on what you mean by invoke an API. Do you mean that you're calling an API from your web app code or do you mean that you are treating your web app as an API and using a program to access it?

I had created an app for API and using the other app i am trying to call that.

Are you trying to access a URL on your site from inside a view on the same site? That will not work with a free account; you have just one process handling all requests to your site, which means that if it is busy handling one request, other requests will be queued up until it has finished handling it -- so if it makes a request back to itself, that second request will be queued and never handled.

I have the same problem "OSError: write error in error.log" during the POST request of a form. The form takes 2 files and the error occurs when I use large files. Do I need to change the UWSGI configuration or something like that?

How large are the files in question? How long is the page taking to return a response?

Are excel files with something like 100k rows each one for example. Those files may have like 15MB. I have upgraded my account, I have now 4 workers and I'm fixing the code in order to work with multiple workers expecting that it solves the problem. Am I going in a bad way?

No, writing your code so it can handle multiple workers is not a bad way to go. It may or may not solve the problem depending on what is actually causing the problem.

I have updated the code and it looks like it works better with more workers, but I have other problem now. Should I open a new thread on the forum? This is the error log:

2020-11-13 09:15:24 Fri Nov 13 09:15:23 2020 - * * * HARAKIRI ON WORKER 4 (pid: 27, try: 1) * * *

2020-11-13 09:15:24 Fri Nov 13 09:15:23 2020 - HARAKIRI !!! worker 4 status !!!

2020-11-13 09:15:24 Fri Nov 13 09:15:23 2020 - HARAKIRI [core 0] 10.0.0.124 - POST /process/start/ since 1605258322

2020-11-13 09:15:24 Fri Nov 13 09:15:23 2020 - HARAKIRI !!! end of worker 4 status !!!

I have tested it locally and it works perfectly doing the same test.

Just in case: I'm working on my other account (user: DonLarry)

Those happen when you have a request that takes more than 5 min to respond. In that case, the worker will be killed and restarted. Check the endpoint (/process/start) to see why that was taking more than 5 min and work out how you can reduce its response time to below 5 min.

I have the same error. please how can I fix it?

FileNotFoundError: [Errno 2] No such file or directory: '/home/HTSanalysis/analytics_report/complete_analytics_report.pdf' 2021-01-06 15:26:12,153: OSError: write error 2021-01-06 15:34:56,373: OSError: write error 2021-01-06 16:46:52,236: OSError: write error 2021-01-06 16:46:52,253: OSError: write error 2021-01-06 16:46:52,255: OSError: write error 2021-01-06 16:46:52,256: OSError: write error 2021-01-06 16:46:52,259: OSError: write error 2021-01-06 16:49:02,679: OSError: write error 2021-01-06 16:51:52,545: OSError: write error 2021-01-06 17:04:24,205: OSError: write error 2021-01-06 17:32:02,338: OSError: write error 2021-01-07 00:02:59,461: OSError: write error

And the path exists. please how can I resolve this?

That error is from yesterday. Are you sure that the error is still happening and the cause of your problems?

Yes the error is still happening

I need to thoroughly test it before it goes live for people testing the app

Kindly advise on how to resolve this please

Even after adding 2 more workers, I still get the error, please help out

So I tested this locally and downloaded it countless times and it works, it is therefore from the server, please can you help. it gives server error here, please

Hi, I see the web app is disabled. Is the issue still to be resolved?

its not resolved

please help

Hello again, I tested this locally, it works locally but does not fully execute here and gives an OS Write error in the process of sending the file. please help out!

What is the error that you're currently getting? The one that you posted yesterday was from the day before. Are you sure that the error that you're currently getting is the same?

OSError: [Errno 116] Stale file handle 2021-01-08 01:16:48,997: OSError: write error 2021-01-08 18:15:52,677: OSError: write error

2021-01-08 18:15:52,677: OSError: write error

The OSError: write error lines are just there when a client disconnects before the response has been fully sent to the client. They can be an indicator that your site may be a little slow or that the timeouts in the client may be too short, but they do not affect the working of the site.

The Stale file handle error, however, may be an issue. Is there a traceback that precedes it. That should help you to track down what is causing that error.

There is no trace back

And yes you are right testing it locally shows it does not affect the working of the site. How do I resolve the timeout and the other issues? Kindly advise even if I need to increase the current facility

Here is one traceback before I send the file for download: 2021-01-08 19:18:10 Ready to send file for downloading -> This was a print statement to check if the file passes all executions and yes it does because it got here.

Then this-> 2021-01-08 19:18:10 Fri Jan 8 19:18:10 2021 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /_dash-update-component (ip 10.0.0.124) !!!

2021-01-08 19:18:10 Fri Jan 8 19:18:10 2021 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during POST /_dash-update-component (10.0.0.124)

*** Python threads support is disabled. You can enable it with --enable-threads ***

Please where do I enable the threads?

Unfortunately we don't support threads in website code on PythonAnywhere.

All

I was having this error but I noticed that I was not building a DataFrame correctly and I was trying to append an empty DataFrame to an existing csv in a folder.

I was about to ask in the forum when I realised the error

This may be a note to others with this issue. Beside the above, there could be a small issue in the code triggering this error

Just to be sure -- you were getting the writing to a closed pipe/socket/fd error as a result of that? That would be strange -- I can't see any way that the bug you describe could lead to that.

I've been getting a lot of write errors too.

2021-02-28 21:57:45,631: OSError: write error
2021-03-02 13:07:45,517: OSError: write error
2021-03-02 13:26:08,959: OSError: write error
2021-03-09 20:19:29,992: OSError: write error
2021-03-09 22:38:19,130: OSError: write error
2021-03-09 22:38:22,555: OSError: write error
2021-03-10 04:33:42,871: OSError: write error
2021-03-10 19:25:22,195: OSError: write error
2021-03-13 07:01:48,436: OSError: write error
2021-03-13 07:02:33,553: OSError: write error
2021-03-13 07:03:27,987: OSError: write error
2021-03-14 11:03:49,934: OSError: write error
2021-03-14 11:06:30,467: OSError: write error
2021-03-14 11:07:20,624: OSError: write error
2021-03-14 11:08:27,702: OSError: write error
2021-03-14 11:09:31,708: OSError: write error
2021-03-14 11:10:18,614: OSError: write error
2021-03-14 11:11:02,968: OSError: write error
2021-03-14 11:14:30,490: OSError: write error
2021-03-14 18:46:21,821: OSError: write error
2021-03-15 13:31:40,861: OSError: write error
2021-03-15 13:31:40,916: OSError: write error
2021-03-15 13:31:41,120: OSError: write error
2021-03-15 13:32:35,932: OSError: write error
2021-03-15 13:35:22,459: OSError: write error
2021-03-22 21:25:24,404: OSError: write error
2021-03-22 21:25:29,396: OSError: write error
2021-03-22 21:27:33,677: OSError: write error
2021-03-24 00:53:35,447: OSError: write error
2021-03-25 15:31:37,570: OSError: write error
2021-03-26 13:35:49,061: OSError: write error
2021-03-26 15:58:33,504: OSError: write error
2021-03-26 21:54:02,503: OSError: write error

Its a total pain because my app stops the reading the database and doesn't resume until I have reloaded the site. That's not a great look for my app and it means I'm shepherding it a lot of the time. Is there any method around it? I will accept that occasionally servers throw a write error, but I need a pythonic way around this issue. Any suggestions?

Thanks.

[formatted by admin]

As Glenn mentioned before:

The OSError: write error lines are just there when a client disconnects before the response has been fully sent to the client. They can be an indicator that your site may be a little slow or that the timeouts in the client may be too short, but they do not affect the working of the site.

So that error is a symptom not a cause.

If you take a look at your error logs, you'll see bunch of errors related to database connections -- if your code fails to connect with db and eventually the request gets a timeout, the error you see may be result of that.

I think I've gotten to the bottom of the cause (fingers crossed). I was accessing the database a couple of times every run through with try & excepts, but I wasn't giving each new request a unique cursor & connection name and I believe this was causing the intermittent problem.

I hope this helps someone else. Thanks. :)

Excellent, glad you've worked that out (fingers crossed!) :)

Hi team,

I've been working on my website just fine and then something happened I started getting the "Your webapp took a long time to reload." error when trying to push the updated script onto the page. I then checked the error log and the only error it returned is the "OSError: write error" one.

To sanity check, I commented out all of my code and pasted the generic example from one of your tutorials which had worked fine for me before (https://blog.pythonanywhere.com/169/) but the problem persisted.

Could you please help me figure out what's causing the OSError?

Thanks!

Are you sure that the problem actually persisted and that you were not reading old error lines?

Thank you for your help - I think I've figured out the issue but not the solution yet. I'm importing a function from my own .py file which is also a scheduled task. The task log is not showing any issues (return code 0), however, the OSError shows in the log when I try importing a function from the scheduled .py file.

Is there a way to log the output of the scheduled task in a file and then just display the file on the webpage instead of calling the scheduled task script in the background every time? Sorry if this is basic, I'm new to this.

Btw, this was working for me just yesterday (i.e. importing and calling a function from my scheduled task script and getting the desired output), I'm not sure what happened.

Thanks!

Scheduled tasks are not limited by short timeouts like stuff running in the context of web app responses to requests.

Sorry, I'm not sure what this means for my case. Could you please explain?

My scheduled task works and there are no timeout exceptions there. It's when I try to import the results into my web app script that the issue arises.

Is there a blog post / tutorial that I could look at for how to use the output of my scheduled tasks in my web app script? Specifically, if there's a way to store the output for a day until the next script runs and output it in the flask_app scipt.

Thank you!

What kind of output is your scheduled task generating -- is it something that could be served up straight to users of your site, like a CSV or some HTML? If so, you could use the static files system to serve it up.

i am getting the same error OSError: write error. it was working just fine a couple of hours ago https://texttosql.pythonanywhere.com/

@TextToSQL as it is explained above multiple times the 'OSError: write error' message is logged when a client disconnects from the server before the server has finished sending the response. This can happen because the site is slow and a user stops the page from loading or because the timeout on a client is too small.