abiyani / automate-save-page-as Goto Github PK

Is it possible to perform actions that ASPA needs to do without needing this function (so without asking to focus a window or to know which one is focused)? Is it possible to create an argument to skip this step? Because I don't think X11 installed on WSL2 support EWMH. Or maybe you have an idea for this WM to support it?

when i send several Tab keys after page is loaded - it works ok. But my enter key does not work....
Anything i am missing? thanks

Hi, can this program not work with modern pages, where the pages are loaded gradually via scrolling? I would like to use this tool for a page of this type. The page reads only what is visible and scrolls down through AJAX to generate additional parts of the source code. So after loading, the page has only 40 lines, this tool sees it. Really, this page has, after scrolling down, for example, 10762 lines. How can I get a whole page of 10762 lines?

Thank you all for the tips.

The script is working perfectly fine (Opens FF, Saves, Closes FF) when I run it manually on terminal but I need it to run without human intervention, so I set a cron job to save the page everyday.

Unfortunately when the computer is locked, the script only opens the page on FF and does not save or close it. Im not sure if its an Ubuntu problem or a script problem (perhaps the keyboard shortcuts dont work when the computer is not logged in?), but I just wanted to check if anybody else has encountered and has a workaround.

Oh and I'm running Ubuntu 14.04LTS / Firefox. Thanks!

When i start the script i get then Error messages

Error: Can't open display: (null) Failed creating new xdo instance

I am running it with firefox on Ubunut 14.04

KUbuntu 18.

During the first run in a session, it appears as though the Firefox browser is first warming up. It registers the "index.html" as the name of the page to be saved, which is wrong. I click on cancel and close the browser myself. Later on, it works well automatically.

First and foremost, Thank you for writing this program. It saved my bacon when i needed to save about 250 pages and I can't thank you enough.

Secondly I use Firefox developer branch and the program didn't want to use it. It's an easy fix as i just linked firefox-developer to Firefox and noticed no issues. Personally i'd like to see this fixed from inside the program as it is more elegant.

ps: firefox branches: firefox-developer,firefox-beta,firefox-nightly,(deprecated firefox-aurora).
pps:I'm in the middle of my exams right now and don't have the time to fix it, but if there's no-one else I definitely will.

//Close the browser tab/window (Ctrl+w for KDE, Ctrl+F4 otherwise)

This is the KDE desktop, but the Firefox is never closing by itself at the end of the operation.

Other small issues.

I noticed that the script does not work for a freshly installed Firefox - only the first time.

I noticed also, that the script did not work today only on the first execution of this test run:
./save_page_as -b firefox -d tmp "https://www.bing.com"

Hi! Downloaded latest save_as... with xdotool-2.20110530.1 and my charset is UTF-8. Great tool but my first output of it is as follows:

save_page_as "www.facebook.com" --destination "/tmp/facebook_home_page.html" ERROR: Either -- destination ('/tmp/facebook_home_page.html') or --suffix ('') contains a non ascii or non-printable ascii character(s). 'xdotool' does not mingle well with non-ascii characters (https://code.google.com/p/semicomplete/issues/detail?id=14).

!!!! Will NOT proceed !!!!

is it a bug or something with my .bashrc ? Thanks

Might be helpful when used in scripting scrapes. I suspect Firefox will not hold the 250 pages I'm trying to scrape. =)

When running the script under Ubuntu 16.04 running Unity the chrome browser opens with the URL. The save dialog box appear. But then the "Type your command" popup appears as if ALT key is pressed. So the full path gets added to the command box instead of the save dialog and the dialog does not close. It's caused by the fix done for issue #1.
If the above issue is fixed, after saving the page, when the tab close (ALT+F4) is invoked, it is changing to text mode as if (CTRL+ALT+F4) is pressed.

I have fixed both the issues but tested only in Ubuntu 16.04. Will send the PR.

Hi, I am using Firefox Save as on MAC, and I was wondering how would I edit the script to implement Save As Complete Website?

thanks!
Brian

Thanks for this! Real handy tool.

It's defaulting to google-chrome, but if google-chrome isn't installed it just runs indefinitely.

Building an error message off the exit status of "which $browser" might be an easy fix.

There is a problem when the destination file already exists.
To avoid handling a popup dialog, I suggest to download to temprorary name first, then move to the destination just by mv.

Hello,
Thanks for the script :)
I use Debian 9 stretch, I have installed xdotools.
Whether with firefox or chrome, the page opens, but there are no catches.
In logs, I have this:

Aug 8 09:19:21 xxxxxxxxxxxxxxx kerrnel: [ 1258.634385] xdotool[1922]: segfault at 20 ip 00007face0fcb346 sp 00007ffd3d6ac490 error 4 in libxdo.so.3[7face0fc4000+b000]
Aug 8 09:25:08 xxxxxxxxxxxxxxxxxxxx kernel: [ 1605.317681] xdotool[2194]: segfault at 20 ip 00007f3cc379f346 sp 00007ffdbb266c20 error 4 in libxdo.so.3[7f3cc3798000+b000]
Aug 8 09:26:28 xxxxxxxxxxxxxxxxxx kernel: [ 1684.747747] xdotool[2323]: segfault at 20 ip 00007f1acba36346 sp 00007ffe012508c0 error 4 in libxdo.so.3[7f1acba2f000+b000]

Do you have any idea of ​​the problem ?

thank you

when i do ctrl+S to overwrite existing file - it fails....

thanks

what is that sync option here
browser_wid="$(xdotool search --sync --onlyvisible --class "${browser}" | head -n 1)"?
it just never worked with that option. Is it important? Please explain.

thanks for the nice tool.

After downloading the script and finding it works great,
one wants to contact the authors and congratulate them etc.

However unless one remembers where one downloaded it from,
or searches for patterns on Google, all knowledge of where the script came from is lost.

I would put a

• bug report / repository URL
• copyright date
• version #

in the --help message.

When Chrome is used on KDE, the 'save as' popup is a different file-selection-widget and doesn't have default focus on the 'filename' (like gnome/gtk)

Any pointers to make this tool work in KDE environment is appreciated.

Feature suggestion:
Add an option to save the rendered HTML from Firefox's Gecko's webengine (Inspector's "Copy Outer HTML").

I am using curl to login and then after login I am trying to save page. Since I can't get wget to replicate FireFox Save As Website Complete, I am hoping to use this Save Page As script.

thanks!

Brian

When I run:

./save_page_as 'https://www.tradingview.com/symbols/ETHUSD/' --destination /tmp/eth.html

I get:

XGetWindowProperty[_NET_WM_DESKTOP] failed (code=1)

I do have xdotool installed. Please advise...

Many thanks!

./save_page_as -b firefox --load-wait-time 240 https://trade.aliexpress.com/order_detail.htm?orderId=nnn
Page loads completely but the following message is issued
XGetWindowProperty[_NET_WM_DESKTOP] failed (code=1)
Ubuntu 16.04 MATE 1.12.1 Firefox 64.0 (64-bit) xdotool 1:3.20150503.1-2 (Ubuntu native)
I hope this can help and will repay, thanks for sharing

I added --dialog-wait-time.
(In fact you could consider not hard wiring all the --delays too...)

save_page_as.txt

This bash script does not run as it should.

Maybe add a note to --help:

Note: Save to "Webpage Complete" or "HTML Only": the browser (at least
chromium) remembers the choice made last time you saved something...
So you might need to "prep" your browser by saving something by hand first,
if you don't like its current choice.

Really wish there was a way to tell it that on Debian chromium is called chromium, without needing to edit the source.
SPA.diff.txt

In an already running browser, after the newly created window is closed, a Downloads list is still left across the bottom of the screen.
So (optionally) zapping that is also needed, for a perfect script.
The "X" on its right side needs to be clicked (in chromium at least).

I'm running the script like this:

./save_page_as -d /tmp -b chromium-browser --load-wait-time 1.5 --save-wait-time 3 "$URL"

Sometimes, while the script is saving the file, it shows &tmp on the save dialog. Sometimes it works correctly. Im also using another script to call save_page_as with several different urls.

What could be the problem?