Forums

Hey I don't know if this is some change you have recently made to combat something or if this is the API I am using, but I run an hourly script which appends data from an API to a database and is run on your server. I can run this locally fine, but just recently randomly i'll have my first few 50% of users timeout using the requests.get() method with timeout=10

Oddly, if I close the console and I re-run it then some of the time it just works fine anyway and i'm stuck at what could be going on given its worked for months

Is it possible that API is rate-limiting IP address of the server your scheduled task is running at?

I don't think so because I know people who use the API for tens of thousands of requests and get about fine, I make about 225 per hour and Its the first requests that fail.

It is also odd it works all a sudden if I restart the console - although oddly sometimes restarting it the console gets closed immediately

So something else I have just discovered is that restarting the console doesn't necessarily get it to always work. If it has been a long time since I last pulled data (i.e. an hour) then I start getting timeouts, but if its a short time then it starts working almost like the IP address has been marked okay or something. It's odd since I get timeouts for the first few requests then it starts working almost like the API notices everything is fine, but thinks nothing was fine before?

Is it possible someone else is using this server and making lots of requests to the API which screws up mine when I try to use it?

A further update - by removing the timeout=10 parameter to the .get() method, it seems I was able to fix my problem

Interesting! It does sound like the API you're accessing needs to "warm up" for a particular IP in some way, which is unusual. I'd definitely recommend using some timeout or other if you're hitting an API, though, just to make sure that a problem on their side won't make your code hang indefinitely. Maybe just bump it up to 120 seconds or so and see if that's enough?

Hey giles, a 120 second timeout also fixes it and thankyou for the warning! I have changed the parameter to this