I love this updater and I was wondering if there were any plans to possibly update the updater itself. Instead of just saying there is a update. Maybe give us a option of downloading and applying the update. I understand if you don't want to do this but it was a idea that allot of us would benefit from.

Exception handling could use some work to help out both the user and the developers. It's currently catching exceptions all over the place, and logging is nonexistent.

Generally, the sites should log exceptions raised during their work, and then report some more generic type of SiteException to the manager.

Thank you (and all the other contributors) for this great app!

Optionally storing the downloaded .zip files would be nice (e.g. in a subfolder of specified WoW addon folder)

PS C:\...\wow-addon-updater-retail> .\run_Windows.bat
Curse is blocking requests because it thinks you are a bot... please try later.
Failed to find addon version number for game version: GameVersion.retail, https://www.curseforge.com/wow/addons/nameplate-scrolling-combat-text

Tried updating cfscrape to latest version but didn't help. PeachTree's version using cloudscrape also didn't work.

They're onto us. :(

The example in the readme with Tukui ElvUI is not working with the current master version, it cant find the subfolder.

If the fetching of version succeeded but the zip file failed in downloading, then the version number is still saved and you have to edit installed.ini to get it try redownloading.

Hello,

just setup this program and I seem to be having an issue with one addon
https://www.curseforge.com/wow/addons/the-classic-race

when i run the program it comes up saying the addon is unavailable. but i can download it manually just fine.

Adding a command line arg like --config config-classic.ini will make it easier to operate the addon with two sets of installations. i.e. then you could have

config-retail.ini, which specifies:

• your retail install path
• addons-retail.txt
• installed-retail.ini

and keep separately a config-classic.ini which has:

• your classic path
• addons-classic.txt
• installed-classic.ini

Just started using this fork vs. the original dead one (that worked fine until today).
I figured since the updater did not work, i would pop over to a active project, but it seems either i have done something wrong, or curseforge changed something (again).

Failed to find addon version number for game version: GameVersion.retail, https://www.curseforge.com/wow/addons/bartender4
And so on and so forth..

Name                             Prev. Version                    New Version                      
────                             ─────────────                    ───────────                      
bartender4                       -----                            Unavailable                      

As i said, i might have missed some config step, but all my addons set in addons.txt are listed, and the url's seems to check out... just not the version.

I tired to follow the install guide but it makes no sense to me.

"You should already have pip included with your Python installation. This is the default package manager for Python. If not, download the latest version of Python for your platform, with pip bundled.

This utility has three Python module dependencies:

The requests module, for making HTTP requests
The BeautifulSoup4 module, for HTML document parsing
The cfscrape module, for bypassing Curse's bot-detection measures

First off i have no idea if i even have "pip" installed, because there is no clear direction on how I can check if it is.

Number 2 (from the install guide)

It's recommended you manage this with pipenv. All you need to do is run the following to install pipenv and the dependencies:

cd wow-addon-updater/
pip install pipenv
pipenv install

The packages will be automatically installed by pipenv to a local virtual environment"

Now this might make sense if your a programmer but for me, I have literally no idea what I'm supposed to do here... am i supposed to open python or cmd line and type this in ? I have no idea i wish this guide was more clear to people that are not programmers/ regular python users.

Hello good sir -- firstly thank you so much for taking the project over. You're wonderful for just that in itself. Now, I hope you can help me (and hopefully others) that come across this. I don't consider myself completely dumb when it comes to computers, but in this case I think I am in over my head a bit. I don't code, and until I installed the original updater, I didn't know there was a .py extension. Please take a moment to help, and maybe update the readme a bit once I am helped, so that others can be helped:

The beginning install seems very convoluted.

You wrote:

This utility has two external dependencies:

The requests module, for making HTTP requests
The BeautifulSoup4 module, for HTML document parsing
The install should be managed by pipenv. All you need to do is run the following:

cd wow-addon-updater/
pip install pipenv
pipenv install

So I went to those sites and gathered all the files. I had to assume you wanted the tar.gz files, and not the wheel versions. I don't know what those are, I just had to choose.
I put them all in the wow-addon-updater-master folder that I have inside my wow directory.
I then have assume that I can use a command line (I am a windows 10 user).

So I went to cmd and cd to wow-addon-updater-master (which I request you change the instructions to that instead of wow-addon-updater as your file structure has the former when directly copied)

I don't want to mix all the files unless I am supposed to do that.
No matter, I cd to the pipenv-master folder I have inside my addon-updater folder, and I try some of those command lines, but to no avail. It doesn't recognize pip or pipenv as commands. I guess that makes sense because how can windows know to run a pip command if it is not installed yet, maybe?

Anyway, that's my struggles.

I have two areas of concern:

1. Where do I go to run

cd wow-addon-updater/
pip install pipenv
pipenv install

2. after that, will the two dependencies be installed? Any more steps? Where do I put the two dependencies? Into the root wow-addon-updater-master folder?

After that, I know to change the config to do classic, get the addon list made for it to run, and go from there. Just this first part is aggravating haha.

I know you must think I am completely silly for trying to run this without knowing how to install the first part, but maybe this will make it more popular, if we get a bit more detail on the install process.

Thank you kindly either way.

Hello,

I use the script daily to update my WoW addons but today it has stopped working. I get:

Failed to find addon version number for game version: GameVersion.retail

Is this something to do with cloudflare blocking the requests? Is there any further information I can provide?

Failed to find addon version number for game version: GameVersion.retail

Name Prev. Version New Version
──── ───────────── ───────────
announce-interrupts ----- Unavailable

kuhnertdm/wow-addon-updater#42

I figured this out.
How to correctly install wow updater

1. If you have previous versions of these like i did, Uninstall all python programs you have installed
   ect and reboot computer, so we can start out fresh.

2. download current version of python for your computer. If you run 64 bit windows ensure you download 64 bit of python. (not the same as 32 bit) If 64 bit download the (Windows x86-64 executable installer)

In windows when you install python make SURE you check the box
"Add python 3.7 to Path" otherwise pip will not work within cmd prompt. (which we need later)

3. Check you have successfully installed PIP. Open cmd prompt type in
   "pip --version" (without quotes)

If PIP was successfully installed it should say something along the lines of

pip 19.0.3 from c:\users\Frank\appdata\local\programs\python\python37\lib\site-packages\pip (python 3.7)

If you messed up it will say

'pip' is not recognized as an internal or external command. Operable program or batch file.
(if it says this go back to number 1 and start over.)

4)Now open command prompt agian. type

"CD C:\Users\Frank\Desktop\Games\wow-addon-updater-master" or wherever your wowaddon updater folder is located

This must be spelt properly or else it wont work.

If you got it wrong this error will pop up.

"The system cannot find the path specified."

if you see this error do step 4 over again.

If done properly cmd prompt will show the link to you file: for me it was this:

c:\Users\Frank\Desktop\Games\wow-addon-updater-master>

5. type "pip install pipenv" press enter and wait for it to do its thing (will download a few things. wait until complete.)

6. After complete , in cmd prompt type "pipenv install" press enter (my antivirus poped up i just waited until it was all done)

then follow the rest of the guide starting at "Configuring the utility" to get it working.

What about adding a "command line parameter" (to be set within the .bat) to supress the output of "xxx version ... is up to date" (i.e. only "relevant" lines - updated addons or errors - will be shown). There's a lot of scrolling, so it would be easier to only catch the interesting actions.

addon_manager.py line 101
print(f"{addon_name} version {installed_version} is up to date.\n")

(Every update of your great code I change this line to "pass" but... you know ;-)

Had this in the addons.txt:
https://github.com/project-classic/ClassicCodex
Installs into World of Warcraft/classic/Interface/AddOns/ as ClassicCodex-master instead of ClassicCodex.
Any ideas?

Same issue from 10 days ago has come back again :(
pipenv update doesn't seem to help

Failed to find addon version number for game version: GameVersion.retail,
Name Prev. Version New Version

pipenv run pip freeze shows following
beautifulsoup4==4.8.2
certifi==2019.11.28
chardet==3.0.4
cloudscraper==1.2.28
idna==2.9
requests==2.23.0
requests-toolbelt==0.9.1
soupsieve==2.0
urllib3==1.25.8

However, when i go to https://pypi.org/project/cloudscraper/ it should be version 1.2.30
Running pipenv update cloudscraper seemed to be the solution

I'm having a bit of trouble with getting the program to download any addon from TukUI that's not ElvUI. For example, listing https://git.tukui.org/Azilroka/AddOnSkins/ in addons.txt and running the program, the add-on isn't listed as one to check for updates or install.

I've pinned it down to what I believe being the problem, the tukui.py site URL seems to have hard-listed as ElvUI's git page.

class Tukui(AbstractSite):
_URLS = [
'https://git.tukui.org/elvui/'
]

Removing elvui/ from the URL gets it to download properly. There may be a reason it's hard-coded, so I thought I'd raise a ticket rather than a pull request.

Some addons are also not listed in their git repo, but are listed on the main website, so it makes it difficult to get them updated. Some repos also house both the classic and retail versions of add-ons in one, so it makes it difficult to separate them (there's also no common folder structure).

I did notice that an add-on through their main website, https://www.tukui.org/classic-addons.php?id=2, has an extras tab which contains version information:

I can look into the possibility of using this as version information, allowing more add-ons to be installed and updated.

Hi, I recently switched to using this updater instead of the bloated twitch app, thanks!

One issue I've found is that the ElvUI_OptionsUI folder contained in the ElvUI release zip is not extracted, only the ElvUI folder ends up in WoW's addon folder. Hence ending up with an outdated or missing options UI in case of a clean install.

Sorry, I'm not aware if this already exists or not...

Also, thanks a bunch for this pure awesomeness of a project :)

This is a feature request to support Zygor Guides. While they do support Windows, Mac and Linux, having all my addons updated via a single solution automatically upon launching WoW (instead of having clients running in the background) is vastly preferred.

These are the following API calls available, from what I can tell. As these guides are not free of charge, you will need credentials (I have not tested on a free account). That said, some of the end points return data even without credentials (notably the hourly gold endpoint).

Zygor's API

AddOn and Guides

Get current addon version

GET https://data.zygorguides.com/updater/packager.php?do=query&user=${zygorUsername}&pass=${zygorPassword}

Response:

LOGIN OK
USER redacted redacted
ACCESS ELITE redacted redacted redacted redacted
VER 7.0.21656
PRODUCT sub-elite Zygor Elite Membership
PRODUCT levl-elite - Leveling & Loremaster
PRODUCT dung-elite - Dungeons & Gear
PRODUCT dair-elite - Dailies & Reputation
PRODUCT golp-elite - Gold & Professions
PRODUCT petm-elite - Pets & Mounts
PRODUCT tita-elite - Titles & Achievements
PRODUCT gold - Gold Auction Data
REM product list faked for elite
PRODUCTDIR ZygorGuidesViewer
CHG:
PROFESSIONS<br />
[B] Updated Jewelcrafting 1-300 - Fixed item ID for Delicate Copper Wire.<br />
<br />
EVENT<br />
* Set Lunar Festival quest guide as the current event trial guide.<br />

:CHG
END

The only data you need to parse is the LOGIN OK, VER and PRODUCTDIR (for the gold guide) lines.

Get addon

GET https://data.zygorguides.com/updater/packager.php?do=comp&user=${zygorUsername}&pass=${zygorPassword}

Gold realm data

Get realms

GET https://data.zygorguides.com/ahscan/realms-eu

Format seems to be:
realmID|realmHumanName|connectedRealmId1,connectedRealmId2,connectedRealmId3

Accepted regions are:

• eu
• us
• tw
• kr
• xx (with the only realm being Global), this is for all global data and should be installed regardless of the user's specified realm.

There may be more, but I'm on EU so 🤷‍♂️

Get daily(? - I think) last update time

GET https://data.zygorguides.com/ahscan/?do=check&region=eu&realm=aerie-peak

Region and realm need to be replaced with the realms from the previous command

Get daily(? - I think) data

GET https://data.zygorguides.com/ahscan/?do=dump&region=eu&realm=aerie-peak&user=${zygorUsername}&pass=${zygorPassword}

This should then be saved into the AddOn directory under:
PRODUCTDIR/Data/Gold/lowercase(regionID)_lowercase(realmID).lua

I.E.
Interface/AddOns/ZygorGuidesViewer/Data/Gold/eu_aerie-peak.lua

You should then update PRODUCTDIR/Data/Gold/files.xml to include a child element to the Ui element as such:

<Ui xmlns="http://www.blizzard.com/wow/ui/" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.blizzard.com/wow/ui/FrameXML/UI.xsd">
	<Script file="xx_global.lua" />
	<Script file="xx_global_h.lua" />
	<Script file="eu_aerie-peak.lua" />
	<Script file="eu_aerie-peak_h.lua" />
</Ui>

Get hourly last update time

Identical to the daily API command, but the URL is now https://data.zygorguides.com/ahscan/get_raw_data.php. For example:

GET https://data.zygorguides.com/ahscan/get_raw_data.php?do=check&region=eu&realm=aerie-peak

Get hourly data

Identical to the daily API command, but the URL is now https://data.zygorguides.com/ahscan/get_raw_data.php. For example:

GET https://data.zygorguides.com/ahscan/get_raw_data.php?do=dump&region=eu&realm=aerie-peak

This should then be written onto disk as:
PRODUCTDIR/Data/Gold/lowercase(regionID)_lowercase(realmID)_h.lua
`

I.E.
Interface/AddOns/ZygorGuidesViewer/Data/Gold/eu_aerie-peak_h.lua

You should then update PRODUCTDIR/Data/Gold/files.xml to include a child element to the Ui element as such:

<Ui xmlns="http://www.blizzard.com/wow/ui/" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.blizzard.com/wow/ui/FrameXML/UI.xsd">
	<Script file="xx_global.lua" />
	<Script file="xx_global_h.lua" />
	<Script file="eu_aerie-peak.lua" />
	<Script file="eu_aerie-peak_h.lua" />
</Ui>

Currently ElvUI is being updated with:
return self.url + '/-/archive/master/elvui-master.zip'
(where self.url is https://git.tukui.org/elvui/elvui)
This is good, as we always get a stable release from the master branch.

But the version fetched and stored during the update is the hash from the development branch, because it's the default branch shown at https://git.tukui.org/elvui/elvui

This causes two problems:
The version is incorrect (I false reported a bug that had already been fixed to the ElvUI team once because of this).
We get 'fake' updates caused by the latest development commit changing, but the master zip staying the same. (The same zip is downloaded and extracted, changing nothing but wasting time/bandwidth).

Recommended fix:
You could instead pull the commit hash from https://git.tukui.org/elvui/elvui/tree/master but I don't recommend this. Instead I would use the latest tag from https://git.tukui.org/elvui/elvui/-/tags which seems to always match the latest commit in the master branch anyway, but is more clear to the end user when it comes to telling them what version of the addon they just installed.

Side note, the classic version is https://git.tukui.org/elvui/elvui-classic and should work with the existing code, other than find_zip_url which would need to be changed (...elvui-classic-master.zip).

If, for some reason, unpacking does not work properly (e.g. "unexpected error unzipping xxx"), the old addon subfolder(s) should persist.

There's a severe issue at the moment causing most addons to be broken upon installation, not every directory inside the zip is being extracted.

Example: WeakAuras-2.14.1.zip contains the following four directories,
WeakAuras
WeakAurasModelPaths
WeakAurasOptions
WeakAurasTemplates

After the latest few commits I finally decided to use the fork full time (I didn't want to break what was 'working' so waited until the clean extract PR), but did backup my old AddOns directory first. The old directory had 194 directories inside it, the new, clean one, has 77.

My guess is that only the first encountered directory is being placed, then the function ends, but it's not alphabetical because...

This is a list of every directory contained within Altoholic+v8.2.001.zip:
Altoholic
Altoholic_Achievements
Altoholic_Agenda
Altoholic_Characters
Altoholic_Grids
Altoholic_Guild
Altoholic_Search
Altoholic_Summary
DataStore
DataStore_Achievements
DataStore_Agenda
DataStore_Auctions
DataStore_Characters
DataStore_Containers
DataStore_Crafts
DataStore_Currencies
DataStore_Garrisons
DataStore_Inventory
DataStore_Mails
DataStore_Pets
DataStore_Quests
DataStore_Reputations
DataStore_Spells
DataStore_Stats
DataStore_Talents

This is the only directory that actually made it to the AddOns directory:
DataStore_Stats

Download Page:

https://www.wowinterface.com/downloads/download24980-QuestLogEx

Returns with:

Exception in thread Thread-8:
Traceback (most recent call last):
  File "/usr/lib64/python3.7/threading.py", line 926, in _bootstrap_inner
    self.run()
  File "/usr/lib64/python3.7/threading.py", line 870, in run
    self._target(*self._args, **self._kwargs)
  File "/home/mishugashu/git/wow-addon-updater/updater/manager/addon_manager.py", line 75, in update_addon
    addon_name = site.get_addon_name()
  File "/home/mishugashu/git/wow-addon-updater/updater/site/wowinterface.py", line 44, in get_addon_name
    addon_name = re.search(r"info[0-9]+-(?P<name>[a-zA-z0-9]+)\.html", addon_name).group('name')
AttributeError: 'NoneType' object has no attribute 'group'

Info Page:

https://wowinterface.com/downloads/info24980-QuestLogEx.html

Returns with:

Traceback (most recent call last):
  File "/usr/lib64/python3.7/threading.py", line 926, in _bootstrap_inner
    self.run()
  File "/usr/lib64/python3.7/threading.py", line 870, in run
    self._target(*self._args, **self._kwargs)
  File "/home/mishugashu/git/wow-addon-updater/updater/manager/addon_manager.py", line 73, in update_addon
    site = site_handler.get_handler(addon_url, self.game_version)
  File "/home/mishugashu/git/wow-addon-updater/updater/site/site_handler.py", line 31, in get_handler
    raise UnknownSiteError(f"Unknown addon source: {url}")
updater.site.site_handler.UnknownSiteError: Unknown addon source: https://wowinterface.com/downloads/info24980-QuestLogEx.html

Python version: 3.7.4.

is there a way to name the folders myself ? addons need specific names for their folders and i cant seem to find a way to rename them without doing that myself manually.

Seems like cloudflare is causing issues with the requests being made. The requests eventually work in the browser, but the GET calls which this utility uses (and even something like Postman) are being blocked.

Hopefully it isn't an effort to squash bots like this, otherwise I will have to look into something like https://github.com/Anorov/cloudflare-scrape

Hello All,

Happy New Year

I am using Python 3.8 and getting the following error on Windows:

Any idea how to fix this please?

.\run_Windows.bat : Traceback (most recent call last):
At line:1 char:1

• .\run_Windows.bat

•   + CategoryInfo          : NotSpecified: (Traceback (most recent call last)::String) [], RemoteException
    + FullyQualifiedErrorId : NativeCommandError
  
  File "c:\python38\Lib\runpy.py", line 193, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "c:\python38\Lib\runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "C:\users\SteveWatson\wow-addon-updater\updater\__main__.py", line 46, in <module>
    main()
  File "C:\users\SteveWatson\wow-addon-updater\updater\__main__.py", line 39, in main
    AddonManager(args.config).update_all()
  File "C:\users\SteveWatson\wow-addon-updater\updater\manager\addon_manager.py", line 77, in update_all
    self.display_results()
  File "C:\users\SteveWatson\wow-addon-updater\updater\manager\addon_manager.py", line 188, in display_results
    print("".join(word.ljust(col_width) for word in row))
  File "C:\Users\SteveWatson\.virtualenvs\wow-addon-updater--huNV0kt\lib\encodings\cp1252.py", line 19, in encode
    return codecs.charmap_encode(input,self.errors,encoding_table)[0]
  

UnicodeEncodeError: 'charmap' codec can't encode characters in position 0-3: character maps to

It's not great to display the full 40-character hash as the version, and it makes the resulting table really wide. It's probably better to use the abbreviated hash i.e. 7741fc6.

In the situation where an addon has files A, B, C in version 1, then in version 2, it has filed B, C, D, then A will sit on the filesystem forever alone. The copy operation from zip.extract() will recursively overwrite any files with conflicts, but does not wipe the destination directory clean beforehand.

Linux user 5.4.2-1-MANJARO #1 SMP PREEMPT Thu Dec 5 09:55:57 UTC 2019 x86_64 GNU/Linux

An error occurred while installing beautifulsoup4==4.8.1 --hash=sha256:5279c36b4b2ec2cb4298d723791467e3000e5384a43ea0cdf5d45207c7e97169 --hash=sha256:6135db2ba678168c07950f9a16c4031822c6f4aec75a65e0a97bc5ca09789931 --hash=sha256:dcdef580e18a76d54002088602eba453eec38ebbcafafeaabd8cab12b6155d57! Will try again.
An error occurred while installing brotli==1.0.7 --hash=sha256:0538dc1744fd17c314d2adc409ea7d1b779783b89fd95bcfb0c2acc93a6ea5a7 --hash=sha256:0970a47f471782912d7705160b2b0a9306e68e6fadf9cffcaeb42d8f0951e26c --hash=sha256:113f51658e6fe548dce4b3749f6ef6c24de4184ba9c10a909cbee4261c2a5da0 --hash=sha256:1e1aa9c4d1558889f42749c8baf846007953bfd32c8209230cf1cd1f5ef33495 --hash=sha256:2f2f4f78f29ac4a45d15b3d9fc3fd9705e0ad313a44b129f6e1d0c6916bad0e2 --hash=sha256:3269f6de1dd150fd0cce1c158b61ff5ac06d627fd3ae9c6ea03aed26fbbff7ea --hash=sha256:3f4a1f6240916c7984c7f2542786710f622992508dafee0b1714e6d340fb9ffd --hash=sha256:50dd9ad2a2bb12da4e9002a438672d182f98e546e99952de80280a1e1729664f --hash=sha256:5519a4b01b1a4f965083cbfa2ef2b9774c5a5f352341c47b50776ad109423d72 --hash=sha256:5eb27722d320370315971c427eb8aa7cc0791f2a458840d357ac653bd0ad3a14 --hash=sha256:5f06b4d5b6f58e5b5c220c2f23cad034dc5efa51b01fde2351ced1605bd980e2 --hash=sha256:71ceee286ea7ec613f1c36f1c6181864a6ca24ebb55e371276f33d6af8742834 --hash=sha256:72848d25a5f9e736db4af4512e0c3feecc094d57d241f8f1ae959115a2c39756 --hash=sha256:743001bca75f4a6b4454be3510feca46f9d61a0c782a9bc2bc684bdb245e279e --hash=sha256:7ac98c71a15648fd11bc1f32608b6110e396121280790082e32b9a3109048bc6 --hash=sha256:9d1c2dd27a1083fefd05b1b2f8df4a6bc2aaa6c21dd82cd41c8ae5e7c23a87f8 --hash=sha256:a13ce9b419fe9f277c63f700efb0e444331509d1881b5610d2ba7e9080606967 --hash=sha256:a19ef0952b9d2803df88dff07f45a6c92d5676afb9b8d69cf32232d684036d11 --hash=sha256:ad766ca8b8c1419b71a22756b45264f45725c86133dc80a7cbe30b6b78c75620 --hash=sha256:ad7963f261988ee0883816b6b9f206f11461c9b3cb5cfbca0c9ab5adc406d395 --hash=sha256:af0451e23016631a2f52925a10d738ac4a0f794ac315c30380b22efc0c90cbc6 --hash=sha256:c16201060c5a3f8742e3deae759014251ac92f382f82bc2a41dc079ff18c3f24 --hash=sha256:c43b202f65891861a9a336984a103de25de235f756de69e32db893156f767013 --hash=sha256:c675c6cce4295cb1a692f3de7416aacace7314e064b94bc86e93aceefce7fd3e --hash=sha256:d17cec0b992b1434f5f9df9986563605a4d1b1acd5574c87fc2ac014bcbd3316 --hash=sha256:dc91f6129953861a73d9a65c52a8dd682b561a9ebaf65283541645cab6489917 --hash=sha256:e2f4cbd1760d2bf2f30e396c2301999aab0191aec031a6a8a04950b2f575a536 --hash=sha256:f192e6d3556714105c10486bbd6d045e38a0c04d9da3cef21e0a8dfd8e162df4 --hash=sha256:f775b07026af2b1b0b5a8b05e41571cdcf3a315a67df265d60af301656a5425b --hash=sha256:f969ec7f56ba9636679e69ca07fba548312ccaca37412ee823c7f413541ad7e0 --hash=sha256:f9dc52cd70907aafb99a773b66b156f2f995c7a0d284397c487c8b71ddbef2f9 --hash=sha256:f9ee88bb52352588ceb811d045b5c9bb1dc38927bc150fd156244f60ff3f59f1 --hash=sha256:fc7212e36ebeb81aebf7949c92897b622490d7c0e333a479c0395591e7994600! Will try again.
An error occurred while installing certifi==2019.11.28 --hash=sha256:017c25db2a153ce562900032d5bc68e9f191e44e9a0f762f373977de9df1fbb3 --hash=sha256:25b64c7da4cd7479594d035c08c2d809eb4aab3a26e5a990ea98cc450c320f1f! Will try again.
An error occurred while installing chardet==3.0.4 --hash=sha256:84ab92ed1c4d4f16916e05906b6b75a6c0fb5db821cc65e70cbd64a3e2a5eaae --hash=sha256:fc323ffcaeaed0e0a02bf4d117757b98aed530d9ed4531e3e15460124c106691! Will try again.
An error occurred while installing cloudscraper==1.2.16 --hash=sha256:5d809b88f163859b035fa5e0b75956eec31c40b5d187afa44ab02afd99c8422a --hash=sha256:90cb7f59e3f0e7e49064a00307c54ea6e0319afdd95c9d1e76f8c531c85b5e91! Will try again.
An error occurred while installing idna==2.8 --hash=sha256:c357b3f628cf53ae2c4c05627ecc484553142ca23264e593d327bcde5e9c3407 --hash=sha256:ea8b7f6188e6fa117537c3df7da9fc686d485087abf6ac197f9c46432f7e4a3c! Will try again.
An error occurred while installing requests==2.22.0 --hash=sha256:11e007a8a2aa0323f5a921e9e6a2d7e4e67d9877e85773fba9ba6419025cbeb4 --hash=sha256:9cf5292fcd0f598c671cfc1e0d7d1a7f13bb8085e9a590f48c010551dc6c4b31! Will try again.
An error occurred while installing requests-toolbelt==0.9.1 --hash=sha256:380606e1d10dc85c3bd47bf5a6095f815ec007be7a8b69c878507068df059e6f --hash=sha256:968089d4584ad4ad7c171454f0a5c6dac23971e9472521ea3b6d49d610aa6fc0! Will try again.
An error occurred while installing soupsieve==1.9.5 --hash=sha256:bdb0d917b03a1369ce964056fc195cfdff8819c40de04695a80bc813c3cfa1f5 --hash=sha256:e2c1c5dee4a1c36bcb790e0fabd5492d874b8ebd4617622c4f6a731701060dda! Will try again.
An error occurred while installing urllib3==1.25.7 --hash=sha256:a8a318824cc77d1fd4b2bec2ded92646630d7fe8619497b142c84a9e6f5a7293 --hash=sha256:f3c5fd51747d450d4dcf6f923c81f78f811aab8205fda64b0aba34a4e48b0745! Will try again.
🐍 ▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉ 10/10 — 00:00:00
Installing initially failed dependencies…
☤ ▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉ 10/10 — 00:00:00
[pipenv.exceptions.InstallError]: File "/usr/lib/python3.8/site-packages/pipenv/cli/command.py", line 235, in install
[pipenv.exceptions.InstallError]: retcode = do_install(
[pipenv.exceptions.InstallError]: File "/usr/lib/python3.8/site-packages/pipenv/core.py", line 1863, in do_install
[pipenv.exceptions.InstallError]: do_init(
[pipenv.exceptions.InstallError]: File "/usr/lib/python3.8/site-packages/pipenv/core.py", line 1246, in do_init
[pipenv.exceptions.InstallError]: do_install_dependencies(
[pipenv.exceptions.InstallError]: File "/usr/lib/python3.8/site-packages/pipenv/core.py", line 862, in do_install_dependencies
[pipenv.exceptions.InstallError]: _cleanup_procs(procs, False, failed_deps_queue, retry=False)
[pipenv.exceptions.InstallError]: File "/usr/lib/python3.8/site-packages/pipenv/core.py", line 681, in _cleanup_procs
[pipenv.exceptions.InstallError]: raise exceptions.InstallError(c.dep.name, extra=err_lines)
[pipenv.exceptions.InstallError]: []
[pipenv.exceptions.InstallError]: ['/home/user/.local/share/virtualenvs/wow-addon-updater-master-7UTx382O/bin/python: error while loading shared libraries: libpython3.7m.so.1.0: cannot open shared object file: No such file or directory']

Enhancement request.

Currently a copy of the program is required to update addons for more than one version of the game, adjusting the 'Game Version' and 'WoW Addon Location' variables.

In order for both versions to be updated simultaneously I suggest some combination of the following changes:
'Game Version' variable removed, or changed to 'Default Game Version'.
'addons.txt' split into two appropriate files, 'addons_retail.txt' and 'addons_classic.txt' (addons.txt could be kept as legacy and use the 'Default Game Version').
Single 'addons.txt' is kept and takes an additional, optional word input (new line format 'URL [retail|classic]', defaults to 'Default Game Version' if not supplied).
'WoW Addon Location' renamed to 'Retail Addon Location' and a second variable, 'Classic Addon Location' added (if either is left blank, appropriate subroutine is skipped).
Variable changed entirely to 'WoW Location' and takes the root directory as its value instead of the addons directory ('\_{version}_\Interface\AddOns' easily appended at runtime).

Low priority.

Some addons are releasing classic versions of themselves under the same project as their retail versions causing issues when it comes to pulling the most recent update (file).

Example: https://www.curseforge.com/wow/addons/weakauras-2
Currently pulls 2.14.0-beta5-7-g73dc607-classic when what's really wanted is the retail release, 2.14.0-beta5-7-g73dc607.
Another example: https://www.curseforge.com/wow/addons/big-wigs
Pulls v2-classic when we actually want v163.

Possible fix:
Add config.ini option 'Game Version', which takes a single number. When the /files page is checked for the latest version name, if the first character in its matching Game Version column does not match that of the config, the next line in the table should be checked. (It may be easier to use the /files/all page).
In this example, 'Game Version' should be set as 8 or 1, making a match with 8.2.0 or 1.13.2 respectively. The whole version should not be required as it is both unnecessary and could cause problems when new patches are released.
This should also help with classic addons support (copy the directory and change the config and addons list).

Though another issue, I'm not sure why weakauras-2 is getting its latest release from the top of the /all page https://www.curseforge.com/wow/addons/weakauras-2/files/all but big-wigs is getting its from the top of the non /all page https://www.curseforge.com/wow/addons/big-wigs/files (latest release on /all is v163-4-gff0f762).

Another suggestion related to both issues, consider adding an option 'Release Type' which does something similar to my fix mentioned above, except for the Type column which contains one of three letters, R, B or A. Personally I'd rather just get the latest Release, and not bother updating to new Alpha versions like 2.14.0-beta5-7-g73dc607.

pipenv run python3 -m updater
A new update is available! Check it out at https://github.com/grrttedwards/wow-addon-updater !
Failed to find addon version number for game version: GameVersion.classic, https://www.curseforge.com/wow/addons/better-vendor-price
Failed to find addon version number for game version: GameVersion.classic, https://www.curseforge.com/wow/addons/details
Failed to find addon version number for game version: GameVersion.classic, https://www.curseforge.com/wow/addons/classicthreatmeter
Failed to find addon version number for game version: GameVersion.classic, https://www.curseforge.com/wow/addons/classiccodex
Failed to find addon version number for game version: GameVersion.classic, https://www.curseforge.com/wow/addons/omni-cc
Failed to find addon version number for game version: GameVersion.classic, https://www.curseforge.com/wow/addons/nugrunning
Failed to find addon version number for game version: GameVersion.classic, https://www.curseforge.com/wow/addons/weakauras-2
Failed to find addon version number for game version: GameVersion.classic, https://www.curseforge.com/wow/addons/dejamark
Failed to find addon version number for game version: GameVersion.classic, https://www.curseforge.com/wow/addons/extended-character-stats
Failed to find addon version number for game version: GameVersion.classic, https://www.curseforge.com/wow/addons/atlaslootclassic
Failed to find addon version number for game version: GameVersion.classic, https://www.curseforge.com/wow/addons/itemrack-classic
Failed to find addon version number for game version: GameVersion.classic, https://www.curseforge.com/wow/addons/classicauradurations
Failed to find addon version number for game version: GameVersion.classic, https://www.curseforge.com/wow/addons/spy-classic
Failed to find addon version number for game version: GameVersion.classic, https://www.curseforge.com/wow/addons/bartender4
Failed to find addon version number for game version: GameVersion.classic, https://www.curseforge.com/wow/addons/classiccastbars
Failed to find addon version number for game version: GameVersion.classic, https://www.curseforge.com/wow/addons/leatrix-plus-classic
Failed to find addon version number for game version: GameVersion.classic, https://www.curseforge.com/wow/addons/pitbull-unit-frames-4-0
Failed to find addon version number for game version: GameVersion.classic, https://www.curseforge.com/wow/addons/titan-panel-classic

Name                           Prev. Version                  New Version                    
────                           ─────────────                  ───────────                    
details                        DetailsClassic.1.13.2.140.153  Unavailable                    
better-vendor-price            v1.08.00-classic               Unavailable                    
classicthreatmeter             ClassicThreatMeter-1.06.zip    Unavailable                    
classiccodex                   ClassicCodex 1.2.1             Unavailable                    
omni-cc                        8.2.4                          Unavailable                    
nugrunning                     1.13.18                        Unavailable                    
weakauras-2                    2.14.6-classic                 Unavailable                    
dejamark                       820r202                        Unavailable                    
extended-character-stats       -----                          Unavailable                    
atlaslootclassic               -----                          Unavailable                    
itemrack-classic               -----                          Unavailable                    
classicauradurations           1.13.27                        Unavailable                    
spy-classic                    Spy Classic v1.0.5             Unavailable                    
classiccastbars                v1.0.5-classic                 Unavailable                    
pitbull-unit-frames-4-0        v4.1.20-classic4               Unavailable                    
leatrix-plus-classic           1.13.29                        Unavailable                    
titan-panel-classic            1.0.5.11302                    Unavailable                    
bartender4                     4.8.7-classic                  Unavailable  

did I git pull and tried again - only change is that now all "Prev. version" is marked as "----"

I assume curse has changed something in their markup.

FAIL: test_integration_wowace_get_latest_version (test.site.test_wowace.TestWowAce)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/travis/build/grrttedwards/wow-addon-updater/test/site/test_wowace.py", line 23, in test_integration_wowace_get_latest_version
    self.assertRegex(latest_version, r"[0-9]+\.[0-9]+\.[0-9]+-[0-9]+-g[0-9a-z]{7}")
AssertionError: Regex didn't match: '[0-9]+\\.[0-9]+\\.[0-9]+-[0-9]+-g[0-9a-z]{7}' not found in '4.8.4'

This appears to be very similar to what was seen with Curse. The solution is probably to get the version from that area listing the latest releases for retail and/or classic:

Hi all,
Sorry me again, just tried to use the script and got the same message as before, tried to do a pipenv update to upgrade the dependencies but did not make a difference:

Failed to find addon version number for game version: GameVersion.retail,
Name Prev. Version New Version
──── ───────────── ───────────
garrison-mission-manager ----- Unavailable

Sometimes GitHub addons are failing to retrieve the version number and therefore failing the install. It's intermittent, and usually resolves itself the next time the code is run. A more stable or alternative approach to parsing GitHub HTML should be considered. i.e. BeautifulSoup is being used elsewhere in the project... maybe it's time to upgrade site/github.py

• Curse - #46
• WoWAce
• WoWInterface - #64

It seems that the clever multi-threading addon updating is not working as expected, possibly because of too many requests being sent to the same site at once.
When updating 10 addons or less, it is not much of a problem. But I have 60+ addons (too much, I know), and it fails every time.
You get messages like this:
Failed to find addon version number for game version: GameVersion.retail, https://www.wowinterface.com/downloads/info8533-Altoholic.html

When you run it again, it may be OK.

Or you get messages like this:

Exception in thread Thread-17:
Traceback (most recent call last):
File "C:\home\game\WowAddonUpdater\wow-addon-updater-retail\updater\site\curse.py", line 60, in _convert_old_curse_urls
page.raise_for_status()
File "C:\Users\davi.virtualenvs\wow-addon-updater-retail-4k4Ric-f\lib\site-packages\requests\models.py", line 940, in raise_for_status
raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://wow.curseforge.com/projects/deadly-boss-mods-dbm-legion

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "c:\program files\python37\Lib\threading.py", line 926, in _bootstrap_inner
self.run()
File "c:\program files\python37\Lib\threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "C:\home\game\WowAddonUpdater\wow-addon-updater-retail\updater\manager\addon_manager.py", line 73, in update_addon
site = site_handler.get_handler(addon_url, self.game_version)
File "C:\home\game\WowAddonUpdater\wow-addon-updater-retail\updater\site\site_handler.py", line 16, in get_handler
return Curse(url, game_version)
File "C:\home\game\WowAddonUpdater\wow-addon-updater-retail\updater\site\curse.py", line 15, in init
url = Curse._convert_old_curse_urls(url)
File "C:\home\game\WowAddonUpdater\wow-addon-updater-retail\updater\site\curse.py", line 63, in _convert_old_curse_urls
raise SiteError(f"Failed to find the current page for old URL: {url}") from e
updater.site.abstract_site.SiteError: Failed to find the current page for old URL: https://wow.curseforge.com/projects/deadly-boss-mods-dbm-legion

My solution has been to disable the multithreading:

Original code in addonmanager.py:

def update_all(self):
    threads = []

    with open(self.addon_list_file, 'r') as fin:
        addon_entries = fin.read().splitlines()

    # filter any blank lines or lines commented with an octothorp (#)
    addon_entries = [entry for entry in addon_entries if entry and not entry.startswith('#')]

    for addon_entry in addon_entries:
        thread = threading.Thread(target=self.update_addon, args=(addon_entry,))
        threads.append(thread)
    for thread in threads:
        thread.start()
    for thread in threads:
        thread.join()

    self.set_installed_versions()
    self.display_results()

Now just comment out the thread join loop, making all threads run in sequence:

   for addon_entry in addon_entries:
        thread = threading.Thread(target=self.update_addon, args=(addon_entry,))
        threads.append(thread)
    for thread in threads:
        thread.start()
    #for thread in threads:
        thread.join()

The code now runs without any problems, albeit slower.

/br
Dag

Hi @grrttedwards, its me. As you may or may not know pipenv has ceased development. The last supported version of python with pipenv is 3.7~. If you'd like to take advantage of later versions of python 3.8+, my advice is to migrate to poetry. Its quite similar to pipenv. Its up to you.

Versions:
OS: Kubuntu 19.04
Python: 3.7.3
Pip: 18.1
Pipenv: 2018.11.26

I've just set up the updater script for the first time. The modules and pipenv installs went fine, no errors. I populated the addons.txt file.

When I try to run the updater, it tells me it can't find the addon directory no matter what directory I try to give it in the config file.
The path to my WoW install is:
/home//Games/battlenet/drive_c/Program Files (x86)/World of Warcraft/retail/Interface/AddOns
I've tried:

• "double quoting the path"
• 'single quoting the path'
• escaping\ the\ spaces\ and\ symbols
• pointing the config -> to a cleanly-named symlink to the addons directory
• mount --binding the addons directory to a cleanly-named point
• creating a completely new directory, like /tmp/addons

In every case, I get the same error. Am I missing something? Do I need to tweak my environment somehow?

An addon with a classic-only version like https://www.curseforge.com/wow/addons/libclassicdurations will not download in classic mode.

When using https://wowinterface.com/.... instead of https://www.wowinterface.com/.... exceptions occur. This also happens for the other sites. This behavior is due to no parsing of the URL but looking only for the full URL with www prefixed.

Example can be seen in https://github.com/grrttedwards/wow-addon-updater/blob/master/updater/site/wowinterface.py#L19 and https://github.com/grrttedwards/wow-addon-updater/blob/master/updater/site/wowinterface.py#L10

    _URL = 'https://www.wowinterface.com/downloads/'

    def get_supported_urls(cls) -> [str]:
        return [cls._URL]

Very simply, if an addon only has beta releases (and as such no release on the main page) it won't download at all, throwing 'Failed to find addon version number for game version: GameVersion.'.

Examples:
Retail: https://www.curseforge.com/wow/addons/healerstatweights
Classic: https://www.curseforge.com/wow/addons/atlaslootclassic

I'm not really sure what can be done about this other than using the old download method, which would be unable to tell retail and classic versions apart, simply grabbing the latest file and putting it in the addons directory.
However, I find it unlikely that any addon with only beta releases will have both retail and classic versions. More likely it is niche and only focused on one version, so the risk with the old behavior could simply be taken.

Subfolder extraction is complicated. Non-git subfolders are not working because they follow some of the assumptions that Github addons need.

When working with non-git archives, the code does not need to figure out what the "top level folder" is because the important folders are already located at the top level.

Hi There,

I've been used the previous version of this script for a long time, I followed the setup / install instructions correctly I believe, but my Python knowledge is quite limited.

I receive the following error at the end of the process

"Failed to parse configuration file. Are you sure it is formatted correctly?"

Any assistance to point me in the right direction please ?

`C:\Games\Python\wow-addon-updater-master>pip install pipenv
Requirement already satisfied: pipenv in c:\games\python\python37\lib\site-packa
ges (2018.11.26)
Requirement already satisfied: virtualenv in c:\games\python\python37\lib\site-p
ackages (from pipenv) (16.7.4)
Requirement already satisfied: virtualenv-clone>=0.2.5 in c:\games\python\python
37\lib\site-packages (from pipenv) (0.5.3)
Requirement already satisfied: pip>=9.0.1 in c:\games\python\python37\lib\site-p
ackages (from pipenv) (19.2.3)
Requirement already satisfied: certifi in c:\games\python\python37\lib\site-pack
ages (from pipenv) (2019.6.16)
Requirement already satisfied: setuptools>=36.2.1 in c:\games\python\python37\li
b\site-packages (from pipenv) (41.2.0)

C:\Games\Python\wow-addon-updater-master>pipenv install
Creating a virtualenv for this project.
Pipfile: C:\Games\Python\wow-addon-updater-master\Pipfile
Using c:\games\python\python37\python.exe (3.7.3) to create virtualenv.
[=== ] Creating virtual environment...Already using interpreter c:\games\python
python37\python.exe
Using base prefix 'c:\games\python\python37'
New python executable in C:\Users\Law\.virtualenvs\wow-addon-updater-master-iaGb
lsuV\Scripts\python.exe
Installing setuptools, pip, wheel...
done.

Successfully created virtual environment!
Virtualenv location: C:\Users\Law\.virtualenvs\wow-addon-updater-master-iaGblsuV

Installing dependencies from Pipfile.lock (983816).
================================ 7/7 - 00:00:02
To activate this project's virtualenv, run pipenv shell.
Alternatively, run a command inside the virtualenv with pipenv run.

C:\Games\Python\wow-addon-updater-master>pipenv run python -m updater
Failed to parse configuration file. Are you sure it is formatted correctly?

C:\Games\Python\wow-addon-updater-master>`

My "config.ini"

`[WOW ADDON UPDATER]
WoW Addon Location = C:\Games\World of Warcraft_retail_\Interface\AddOnsAddon List File = addons.txt
Installed Versions File = installed.ini

retail or classic

Game Version = retail`

The updater is failing to to get version numbers for all installed addons. Curse may have changed they way they're categorizing addons or something.