selenium-python-aws-lambda's People
Forkers
codybushnell shafaypro renburnett w-chase104 pranavmk98 stevezieglerva jamesd000 hoangchunghien junouyang kdw9502 andrei-dolgolev alejandroalffer orseni rahulgunkar akaashhazarika biranchi2018 victor-war kevin-ayling leeshinyook butyear rus19023 arka2010 najibkado jjang750selenium-python-aws-lambda's Issues
Zip/Unzip no longer works
I downloaded the ZIP, uploaded to AWS, and the lambda ran successfully. If I simply download the zip > extract all > send to compressed file, it no longer works when I upload it. I tracked back to this after trying upload edited code and having it fail.
Is there some step/procedure that I am missing? Some other way of extracting/zipping? I tried through direct upload and S3.
After unzipping and rezipping I get the unable to import lambda_function error (when this python file is still in the folder and unchanged)
{
"errorMessage": "Unable to import module 'lambda_function'"
}
Thanks for any help!
Returning Empty HTML
Hello,
I am able to return the html for the webpage that I am trying to scrape. However, the div that contains all of the information that is shown on the webpage is empty. When I run the code on my local machine with a different chromedriver, it returns all of the information from the page. Would you happen to know what the issue might be?
Thanks
selenium.common.exceptions.WebDriverException: Message: 'chromedriver' executable may have wrong permissions.
Hello!
When i run the functions, the following error appears:
_
selenium.common.exceptions.WebDriverException: Message: 'chromedriver' executable may have wrong permissions.
_
SAM local run Error
Starting google.com
Message: 'chromedriver' executable needs to be in PATH. Please see https://sites.google.com/a/chromium.org/chromedriver/home
: WebDriverException
Traceback (most recent call last):
File "/var/task/lambda_function.py", line 53, in lambda_handler
driver = webdriver.Chrome(chrome_options=chrome_options)
File "/var/task/selenium/webdriver/chrome/webdriver.py", line 68, in __init__
self.service.start()
File "/var/task/selenium/webdriver/common/service.py", line 83, in start
os.path.basename(self.path), self.start_error_message)
selenium.common.exceptions.WebDriverException: Message: 'chromedriver' executable needs to be in PATH. Please see https://sites.google.com/a/chromium.org/chromedriver/home
END RequestId: 32e1ecb4-3781-4c7b-8898-52a82b4c625c
REPORT RequestId: 32e1ecb4-3781-4c7b-8898-52a82b4c625c Duration: 67 ms Billed Duration: 100 ms Memory Size: 256 MB Max Memory Used: 21 MB
Won't work after change.
Hey there. I tried to make it work. If I just upload your ZIP to lambda and run the function, it works.
So I decided to change a bit of code in lambda_function.py, and that's only to edit line 49,
driver.get('http://www.geneva.info/')
after this change, I pack it back to ZIP, upload and run test. I get this error:
START RequestId: f6bd2ac2-e78c-11e8-86d6-19ee2b8cd09b Version: $LATEST
Starting google.com
Message: 'chromedriver' executable may have wrong permissions. Please see https://sites.google.com/a/chromium.org/chromedriver/home
: WebDriverException
Traceback (most recent call last):
File "/var/task/lambda_function.py", line 46, in lambda_handler
driver = webdriver.Chrome(chrome_options=chrome_options)
File "/var/task/selenium/webdriver/chrome/webdriver.py", line 68, in __init__
self.service.start()
File "/var/task/selenium/webdriver/common/service.py", line 88, in start
os.path.basename(self.path), self.start_error_message)
selenium.common.exceptions.WebDriverException: Message: 'chromedriver' executable may have wrong permissions. Please see https://sites.google.com/a/chromium.org/chromedriver/home
I use Selenium on Windows, but I am fairly new to AWS. What is a problem here and how to make it work?
Thank you.
Not Run
sir, How can i run this code, can you tell me step by step because every time this code shows me error.
How to add additional packages like boto3
I would like to copy the downloaded file into S3. How to add boto3?
Invalid URL
I'm having problems running test events. In my test event I have the following
{
"url":"www.google.com"
}
when I run test it gives me the following message
{
"errorMessage": "Message: unknown error: unhandled inspector error: {\"code\":-32000,\"message\":\"Cannot navigate to invalid URL\"}\n (Session info: headless chrome=62.0.3202.94)\n (Driver info: chromedriver=2.35.528139 (47ead77cb35ad2a9a83248b292151462a66cd881),platform=Linux 4.14.88-90.76.amzn2.x86_64 x86_64)\n",
"errorType": "WebDriverException",
"stackTrace": [
[
"/var/task/lambda_function.py",
49,
"lambda_handler",
"driver.get(event['url'])"
],
[
"/var/task/selenium/webdriver/remote/webdriver.py",
332,
"get",
"self.execute(Command.GET, {'url': url})"
],
[
"/var/task/selenium/webdriver/remote/webdriver.py",
320,
"execute",
"self.error_handler.check_response(response)"
],
[
"/var/task/selenium/webdriver/remote/errorhandler.py",
242,
"check_response",
"raise exception_class(message, screen, stacktrace)"
]
]
}
along with
START RequestId: ce5f4ee0-0263-484e-95cd-3909c4390241 Version: $LATEST
Starting google.com
Message: unknown error: unhandled inspector error: {"code":-32000,"message":"Cannot navigate to invalid URL"}
(Session info: headless chrome=62.0.3202.94)
(Driver info: chromedriver=2.35.528139 (47ead77cb35ad2a9a83248b292151462a66cd881),platform=Linux 4.14.88-90.76.amzn2.x86_64 x86_64)
: WebDriverException
Traceback (most recent call last):
File "/var/task/lambda_function.py", line 49, in lambda_handler
driver.get(event['url'])
File "/var/task/selenium/webdriver/remote/webdriver.py", line 332, in get
self.execute(Command.GET, {'url': url})
File "/var/task/selenium/webdriver/remote/webdriver.py", line 320, in execute
self.error_handler.check_response(response)
File "/var/task/selenium/webdriver/remote/errorhandler.py", line 242, in check_response
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.WebDriverException: Message: unknown error: unhandled inspector error: {"code":-32000,"message":"Cannot navigate to invalid URL"}
(Session info: headless chrome=62.0.3202.94)
(Driver info: chromedriver=2.35.528139 (47ead77cb35ad2a9a83248b292151462a66cd881),platform=Linux 4.14.88-90.76.amzn2.x86_64 x86_64)
END RequestId: ce5f4ee0-0263-484e-95cd-3909c4390241
REPORT RequestId: ce5f4ee0-0263-484e-95cd-3909c4390241 Duration: 5548.74 ms Billed Duration: 5600 ms Memory Size: 256 MB Max Memory Used: 198 MB
Any idea what could be happening here? I followed all the steps properly as far as I can. Even tried updating the bin files and changing the permission rights.
"errorMessage": "Message: 'chromedriver' executable may have wrong permissions.
Hello Manivannan,
Thanks for sharing the code, in the first instance I had no problems, but to the following code fragment of "lambda_function.py":
Original:
driver = webdriver.Chrome (chrome_options = chrome_options)
page_data = ""
if 'url' en event.keys ():
driver.get (evento ['url'])
page_data = driver.page_source
print (page_data)
driver.close ()
volver page_data
New:
driver = webdriver.Chrome (chrome_options = chrome_options)
page_data = ""
if 'url' en event.keys ():
driver.get (evento ['url'])
page_data = driver.title
print (page_data)
driver.close ()
volver page_data
When testing the lambda I am getting the following error:
{
"errorMessage": "Message: 'chromedriver' executable may have wrong permissions. Please see https://sites.google.com/a/chromium.org/chromedriver/home\n",
"errorType": "WebDriverException",
"stackTrace": [
[
"/var/task/lambda_function.py",
46,
"lambda_handler",
"driver = webdriver.Chrome(chrome_options=chrome_options)"
],
[
"/var/task/selenium/webdriver/chrome/webdriver.py",
68,
"init",
"self.service.start()"
],
[
"/var/task/selenium/webdriver/common/service.py",
88,
"start",
"os.path.basename(self.path), self.start_error_message)"
]
]
}
Any ideas? Please your help.
chromedriver' executable needs to be in PATH.
I tried uploading your ZIP in Python Lambda
Python version 3.7, 3.8, and 3.6 gives following error
{
"errorMessage": "Message: 'chromedriver' executable needs to be in PATH. Please see https://sites.google.com/a/chromium.org/chromedriver/home\n",
"errorType": "WebDriverException",
"stackTrace": [
" File "/var/task/lambda_function.py", line 46, in lambda_handler\n driver = webdriver.Chrome(chrome_options=chrome_options)\n",
" File "/var/task/selenium/webdriver/chrome/webdriver.py", line 68, in init\n self.service.start()\n",
" File "/var/task/selenium/webdriver/common/service.py", line 81, in start\n raise WebDriverException(\n"
]
}
Timed Out Error
Hi 🙂
I'm trying to run this file on lambda, but this error showed up
{ "errorMessage": "2021-02-16T01:48:40.456Z c9ee96ed-c1dd-46c5-bbed-fd12d512d186 Task timed out after 20.01 seconds" }
This is the log
START RequestId: c9ee96ed-c1dd-46c5-bbed-fd12d512d186 Version: $LATEST
Starting google.com
END RequestId: c9ee96ed-c1dd-46c5-bbed-fd12d512d186
REPORT RequestId: c9ee96ed-c1dd-46c5-bbed-fd12d512d186 Duration: 20009.43 ms Billed Duration: 20000 ms Memory Size: 256 MB Max Memory Used: 256 MB Init Duration: 56.57 ms
2021-02-16T01:48:40.456Z c9ee96ed-c1dd-46c5-bbed-fd12d512d186 Task timed out after 20.01 seconds
I didn't even unzip and re-zip the file, I uploaded the very file I downloaded here.
The runtime is Python 3.6 and I also created the PATHs.
If you have any clue, please do not hesitate to let me know. Thanks!
OSError: [Errno 8] Exec format error
Hello,
I am testing the code on my mac and it returns this information.
Is the code made for pc ?
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.