I created a test script with a short timeout and ran it along side my test traffic generator. I caught the timeout exceptions and printed them. I can see a lot of timeouts.
I do see a lot of " OSError: write error". Apparently they are coming from the client
In the sever log I see the result of the timeout with errors like the ones below. I understand and expect these.
2018-12-06 05:47:44 Thu Dec 6 05:47:44 2018 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request / (ip 10.0.0.98) !!!
2018-12-06 05:47:44 Thu Dec 6 05:47:44 2018 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during GET / (10.0.0.98)
The strange thing I see a lot more 'write error's vs. the 'writing to closed pipe' errors above on my production site but now that I know what I am looking for I can find those that are there.
Thanks for your help tracking this down! I was fixated on this being a disk error. Didn't think about pipes / client disconnects.