Script now working : Forums : PythonAnywhere

hey, i'm trying to run the code below at the server and works fine (at bash console).

But when i send a POST method, from my website to run this exactly same code, it doesn't work. I tried a lot of things to make work but all of them were useless... what i'm doing wrong ?

sometimes this code doesn't open the browser and others, when it opens Firefox() , the code can't open google.

explained the code: its a google web scrapping using selenium.

!python

def harvestingMain(listaEmpresa,listaTags,listaExcluir, pages): # lista que será preenchida ao final listaNoticias = []

# lista de junções de query's empresa e tags
listaEmpresaTags = listaQuery(listaEmpresa, listaTags)

with Display():

    for retry in range(3):
        try:
            browser = wb.Firefox()
            browser.set_page_load_timeout(10)
            break

        except:
            browser = None
            time.sleep(3)

    try:
        browser.get("https://www.google.com.br")

        for empTag in listaEmpresaTags:

            urlFinal = queryBrowser(browser, "q", empTag)

            # executar a query na primeira passada
            googleResearch(urlFinal, listaNoticias, empTag)

            while nextPageBrowser(browser, pages) != urlFinal:
                urlFinal = browser.current_url

                googleResearch(urlFinal, listaNoticias, empTag)

    except:
        browser.quit()

    # finalizar o browser
    finally:
        browser.quit()

# Tratar as variáveis afim de gerar um dado mais confiável
tratarLinks(listaNoticias)

listaNoticias = excluirLinks(listaNoticias, listaExcluir)

return listaNoticias

deleted-user-4570259 | 13 posts | Dec. 13, 2018, 3:02 a.m. | permalink