loop on multiple Analyze website write on same variable object about python-seo-analyzer

Same problem guyz !

I fixed the issue by doing this:
Go to the ("Manifest") class in the implementation and look for the "Analyze" method.

At the end of the method, before "return output" just write:
Manifest.clear_cache()

Everything will be cool !

[mayouf]

Hi Ghezaielm,

Thanks for your quick feedback..by the meantime, I used another workaround, see below:

import os
for website in list_of_website:
----file_name = # whatever name file you want
----command='seoanalyze {} -f json > "{}"'.format(website,file_name)
----returned_value = os.system(command)
----print(str(returned_value)+' name= '+file_name+' '+website)

And it is convenient if you want parallelize you crawl by using ThreadPoolExecutor

I have 8 cores /20 threads CPU, it is damn fast...I crawled 20k websites in few hours !!

with concurrent.futures.ThreadPoolExecutor(max_workers=80) as executor:
#48 Start the load operations and mark each future with its URL
future_to_url = {executor.submit(analyze_SEO, url): url for row in list_website}
#print(future_to_url)

for future_url in concurrent.futures.as_completed(future_to_url):
url_completed = future_to_url[future_url]

try:
data = url_completed .result()
if data!=None:
print(data)
except Exception as exc:
print('%r generated an exception: %s' % url, exc)

(PS: sorry I did not how to make the spaces on github quote for code)

[mayouf]

Did you submit the correction on github ?

[sethblack]

Ah, right. I'm putting this on my roadmap for v4.1. 👍