dgets / snag-eps Goto Github PK

This one is interesting. Never bumped into this in shell script before...

When $completeURL is utilized on the command line, the entire string has had all uppercase letters shifted to lower case. Other than that it appears to be building the command line perfectly well.

There should be an option for enabling --continue for wget...

Seriously, I forgot to put a space where I split the URL, and it thought that it was in primitive mode, when it had a wrong argument count due to my type-o. Also, no warning, or error message, was displayed, so yeah--that needs to be done, and stuff.

I guess there might be some reasons to try to have a verbose mode in this script, as well...

Not sure what on earth is going on, but at line 41's conditional test of $minStartChar is refusing. The correct value is (last I checked) ending up in that variable, but the '[' test is borked. None of the comparison operators seem to work. Very strange being as other conditionals that don't seem to have any outstanding syntactical differences are working just fine...

Once the general algorithm is hashed out here, it'd be good to be able to specify a certain number of wget threads that can be pulling files concurrently. Might be useful for situations where the host is limiting each connection's bandwidth.

for cntr in $(seq $min $max) ; do
	#if num is parsed straight from the command line we may need to prune
	#the '0' that we just determined the potential leading zero on
	if [ $leading_zero -eq $TRUE ] ; then
		#so yeah, no double zeros here
		#also if $cntr >= 10, don't prepend that 0
		num="0$cntr"
	else
		num=$cntr
	fi

	completeURL="${urlString1}${num}${urlString2}"
	$WGET $completeURL
done

In the preceding code snippet there are issues with the way the prepended '0' issue has been handled. As you can see in the comments (which have not been committed, at least as of now), there are issues where the number is going to end up with an invalid '00' prefix. Also, there is no checking for whether or not $cntr is greater than ten, in which case we need to omit adding another '0' to it.

Simple fixes, just wanted to get them noted in here in case I get interrupted from coding here and forget what I was working on.

Obviously I need to brush up on sed, and do the searching & splitting of the URL within the script. Having the human parse a pasted URL example specially edited for the script is ridiculous.

Now that I see that wget supports rate limiting, it would probably be a handy feature to include variable rate limiting. Hell, maybe even time scheduled rate limiting, though that's probably overkill extreme.