Coder Social home page Coder Social logo

fedecalendino / nintendeals Goto Github PK

View Code? Open in Web Editor NEW
127.0 4.0 18.0 3.29 MB

Library with a set of tools for scraping information about Nintendo games and its prices across all regions (NA, EU and JP).

License: MIT License

Python 100.00%
reddit reddit-bot nintendo nintendo-switch deals metacritic scraping scraping-websites nintendo-eshop games

nintendeals's Introduction

nintendeals

"nintendeals was a bot, he loved learning and deals on nintendo's eshop." LetsFunHans 💬️

Version Quality Gate Status CodeCoverage


Named after the my old reddit bot, nintendeals is now a library with all the scrapers and integrations of nintendo services that I used.

Terminology

Before getting into any details first we need too get into the same page with a few terms:

Region

Here we have three regions NA, EU and JP each one corresponding to Nintendo of America (NoA), Nintendo of Europe (NoE) and Nintendo of Japan (NoJ). Each of these regions have set of countries they are "in charge of":

NoA:

  • Canada
  • Mexico
  • United Stated

NoE:

  • Every country in the European Union
  • Australia
  • New Zealand
  • South Africa

NoJ:

  • Japan

nsuid

An nsuid is a 14 digit long string which Nintendo uses to identify games on each region. Taking Breath of the Wild as an example, we have these 3 nsuids for it (one per region):

  * 70010000000025 (NA)
  * 70010000000023 (EU)
  * 70010000000026 (JP)

Product Code

The product code is another type of ID that Nintendo uses, it usually is a 8/9 character long string. Taking Splatoon 2 as an example, we have these 3 product codes for it (one per region):

  * HACPAAB6B (NA)
  * HACPAAB6C (EU)
  *  HACAAB6A (JP)

The difference with the nsuid is that the product code has a constant between all regions (AAB6 in this example), and this is what I decided to call unique_id and it is what we can you to join a game across all regions.

You can also see this code in the front of your Nintendo Switch cartridge.

Services

This library provides three types of services: Info, Listing, Searching and Pricing. Each region has a different version of Info, Listing and Searching, but Pricing is the same for all as it only requires a country and an nsuid.

Listing

Even thought there are different version for each region, they all work in the same way. Given a supported platform (for this library) they will retrieve a list games in the selected region (in the form of an iterator).

from nintendeals import noa

for game in noa.list_switch_games():
    print(game.title, "/", game.nsuid)
>> ARMS / 70010000000392
>> Astro Duel Deluxe / 70010000000301
>> Axiom Verge / 70010000000821
>> Azure Striker GUNVOLT: STRIKER PACK / 70010000000645
>> Beach Buggy Racing / 70010000000721
from nintendeals import noe

for game in noe.list_switch_games():
    print(game.title, "/", game.nsuid)
>> I and Me / 70010000000314
>> In Between / 70010000009184
>> Ghost 1.0 / 70010000001386
>> Resident Evil 0 / 70010000012848
>> 64.0 / 70010000020867

Searching

Built on top of the listing services, these provide a simple way to search for games by title or release_date:

from nintendeals import noa

for game in noa.search_switch_games(query="Zelda"):
    print(game.title, "/", game.nsuid)
>> The Legend of Zelda™: Link’s Awakening / 70010000020033
>> The Legend of Zelda™: Link's Awakening: Dreamer Edition / None
>> Cadence of Hyrule: Crypt of the NecroDancer Featuring The Legend of Zelda / 70010000021364
>> The Legend of Zelda™: Breath of the Wild / 70010000000025

Info

Once you have the nsuid of the game that you want, you can call the game_info service. And again, each region has their own version but they all work the same, but keep in mind that you need to use the correct nsuid for each region. Coming back to the nsuid of Breath of the Wild as an example:

from nintendeals import noa

game = noa.game_info("70010000000025")
print(game.title)
print(game.product_code, game.unique_id)
print(game.release_date)
print(game.players)
print(str(game.rating[0]), game.rating[1])
print(game.eshop.ca_fr)

for feature, value in game.features.items():
    print(" *", str(feature), ":", value)
>> The Legend of Zelda™: Breath of the Wild
>> HACPAAAAA AAAA
>> 2017-03-03 00:00:00
>> 1
>> ESRB Everyone 10+
>> https://www.nintendo.com/fr_CA/games/detail/the-legend-of-zelda-breath-of-the-wild-switch
>>  * Demo Available : False
>>  * DLC Available : False
>>  * Nintendo Switch Online Required : True
>>  * Save Data Cloud Supported : True
from nintendeals import noe

game = noe.game_info("70010000000023")
print(game.title)
print(game.product_code, game.unique_id)
print(game.release_date)
print(game.players)
print(str(game.rating[0]), game.rating[1])
print(game.eshop.uk_en)

for feature, value in game.features.items():
    print(" *", str(feature), ":", value)
>> The Legend of Zelda: Breath of the Wild
>> HACPAAAAA AAAA
>> 2017-03-03 00:00:00
>> 1
>> PEGI 12
>> https://www.nintendo.co.uk/Games/Nintendo-Switch/The-Legend-of-Zelda-Breath-of-the-Wild-1173609.html
>>  * Amiibo Supported : True
>>  * Demo Available : False
>>  * DLC Available : False
>>  * Nintendo Switch Online Required : False
>>  * Save Data Cloud Supported : True
>>  * Voice Chat Supported : False
from nintendeals import noj

game = noj.game_info("70010000000026")
print(game.title)
print(game.product_code, game.unique_id)
print(game.release_date)
print(game.players)
print(str(game.rating[0]), game.rating[1])
print(game.eshop.jp_jp)

for feature, value in game.features.items():
    print(" *", str(feature), ":", value)
>> ゼルダの伝説 ブレス オブ ザ ワイルド
>> HACAAAAA AAAA
>> 2017-03-03 00:00:00
>> 1
>> CERO B
>> https://store-jp.nintendo.com/list/software/70010000000026.html
>>  * Amiibo Supported : True
>>  * DLC Available : True
>>  * Nintendo Switch Online Required : False

Pricing

Given a country code (using the alpha-2 iso standard) and a game or list of games this service will fetch the current pricing of that/those games for that country. Since this service uses nsuids to fetch the price, make sure that the games that you provide have the regional nsuid that matches the country that you want. For example, only the nsuid for the American region will be able to fetch you the prices of Canada, Mexico and United Stated but not for Japan or Spain.

from nintendeals import noe
from nintendeals.api import prices

game = noe.game_info("70010000007705")
print(game.title)
print()

price = prices.get_price(game, country="CZ")  # Czech Republic
print(price.currency)
print(price.value)
print(price.sale_discount, "%")
print(price.sale_value)
print(price.sale_start)
print(price.sale_end)

# Alternatively you can do this for the same effect:
price = game.price(country="CZ") 
Dead Cells

CZK
625.0
80 %
500.0
2020-04-19 22:00:00
2020-05-03 21:59:59

To reduce the amount of call to the prices api, you can also use the get_prices service that works in a similar way but it expects a list of games instead of only one:

from nintendeals import noa
from nintendeals.api import prices

botw = noa.game_info("70010000000025")
print(botw.title)
celeste = noa.game_info("70010000006442")
print(celeste.title)

print()

prices = prices.get_prices([botw, celeste], country="US")
for nsuid, price in prices:
    print(nsuid)
    print(price.value)
    print(price.sale_value)
    print()
The Legend of Zelda™: Breath of the Wild
Celeste

70010000000025
59.99
None

70010000006442
19.99
4.99

nintendeals's People

Contributors

fedecalendino avatar mattbsg avatar nemecek-filip avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar

nintendeals's Issues

noa.game_info(nsuid) returns {KeyError}KeyError('product')

Had this problem before. Trying it again. Running the example in the readme returns {KeyError}KeyError('product'). I am on 3.0.2. Is there a workaround or do I have something misconfigured? Please let me know what other information you need.

Edit: it seems to break on data = json.loads(script.text)["props"]["pageProps"]["product"]. pageProps is there, but not product. This is what I return when I run in the debugger:

{'props': {'pageProps': {'preview': False, 'previewData': None, 'meta': {'title': 'Thief Town for Nintendo Switch - Nintendo Official Site', 'description': 'Buy Thief Town and shop other great Nintendo products online at the official My Nintendo Store.'}, 'analytics': {'pageType': 'Game Detail Page', 'pageName': 'Thief Town', 'pageCategory': 'Store:Products', 'product': {'name': 'Thief Town', 'sku': '7100026441', 'nsuid': '70010000026441'}}, 'localeAlternates': [{'locale': 'en-us', 'href': None}, {'locale': 'en-ca', 'href': 'https://www.nintendo.com/en-ca/store/products/thief-town-switch/'}, {'locale': 'fr-ca', 'href': 'https://www.nintendo.com/fr-ca/store/products/thief-town-switch/'}, {'locale': 'es-mx', 'href': 'https://www.nintendo.com/es-mx/store/products/thief-town-switch/'}, {'locale': 'pt-br', 'href': None}, {'locale': 'es-ar', 'href': None}, {'locale': 'es-cl', 'href': None}, {'locale': 'es-co', 'href': None}, {'locale': 'es-pe', 'href': None}], 'openGraph': {'image': 'https://assets.nintendo.com/image/upload/c_fill,w_1200/q_auto:best/f_auto/dpr_2.0/ncom/software/switch/70010000026441/5d287d59c0b22cfe9e5e273d5f3a6b7b34d7cb0ed5cec3a3c43bf0e044b596a4'}, 'linkedData': {'@type': 'Product', 'name': 'Thief Town', 'image': 'https://assets.nintendo.com/image/upload/c_fill,w_1200/q_auto:best/f_auto/dpr_2.0/ncom/software/switch/70010000026441/5d287d59c0b22cfe9e5e273d5f3a6b7b34d7cb0ed5cec3a3c43bf0e044b596a4', 'description': 'Up to 4 players control identical avatars (the “Thieves”) in a single-screen arena (the “Town”), also filled with other identical NPCs. Players must identify which NPCs are secretly other players and stab them for points, before being discovered and stab…', 'brand': 'Nintendo', 'category': 'Games', 'sku': '7100026441', 'offers': {'@type': 'Offer', 'url': '/us/store/products/thief-town-switch/', 'priceCurrency': 'USD', 'price': '1.99', 'availability': 'https://schema.org/InStock', 'itemCondition': 'https://schema.org/NewCondition'}, '@context': 'https://schema.org/'}, 'initialApolloState': {'AttributeOption:109:en-US': {'__typename': 'AttributeOption', 'id': '109:en-US', 'label': 'Standard Edition'}, 'ContentDescriptor:{"id":"4rXfJg3rY2e58ECl3iULqU","locale":"en_US"}': {'__typename': 'ContentDescriptor', 'id': '4rXfJg3rY2e58ECl3iULqU', 'locale': 'en_US', 'label': 'Fantasy Violence', 'type': 'CONTENT_DESCRIPTOR'}, 'ContentDescriptor:{"id":"6jE2KH8jLDQfYywkrWMKEx","locale":"en_US"}': {'__typename': 'ContentDescriptor', 'id': '6jE2KH8jLDQfYywkrWMKEx', 'locale': 'en_US', 'label': 'Mild Blood', 'type': 'CONTENT_DESCRIPTOR'}, 'ContentDescriptor:{"id":"45tDlAomCY9ukpHytMbwfl","locale":"en_US"}': {'__typename': 'ContentDescriptor', 'id': '45tDlAomCY9ukpHytMbwfl', 'locale': 'en_US', 'label': 'Use of Alcohol and Tobacco', 'type': 'CONTENT_DESCRIPTOR'}, 'ContentRating:{"id":"4QzesqRbAOTJ05zgLM8Yce","locale":"en_US"}': {'__typename': 'ContentRating', 'id': '4QzesqRbAOTJ05zgLM8Yce', 'locale': 'en_US', 'code': 'e10', 'label': 'Everyone 10+', 'requiresAgeGate': False, 'system': 'ESRB', 'order': 10}, 'GameGenre:22496:en-US': {'__typename': 'GameGenre', 'id': '22496:en-US', 'code': 'PARTY', 'label': 'Party'}, 'GameGenre:22463:en-US': {'__typename': 'GameGenre', 'id': '22463:en-US', 'code': 'ARCADE', 'label': 'Arcade'}, 'GameGenre:22487:en-US': {'__typename': 'GameGenre', 'id': '22487:en-US', 'code': 'MULTIPLAYER', 'label': 'Multiplayer'}, 'StoreProduct:{"sku":"7100026441","locale":"en_CA"}': {'__typename': 'StoreProduct', 'sku': '7100026441', 'locale': 'en_CA', 'url': 'https://www.nintendo.com/en-ca/store/products/thief-town-switch/'}, 'StoreProduct:{"sku":"7100026441","locale":"es_MX"}': {'__typename': 'StoreProduct', 'sku': '7100026441', 'locale': 'es_MX', 'url': 'https://www.nintendo.com/es-mx/store/products/thief-town-switch/'}, 'StoreProduct:{"sku":"7100026441","locale":"fr_CA"}': {'__typename': 'StoreProduct', 'sku': '7100026441', 'locale': 'fr_CA', 'url': 'https://www.nintendo.com/fr-ca/store/products/thief-town-switch/'}, 'NsoFeature:103:en-US': {'__typename': 'NsoFeature', 'id': '103:en-US', 'code': 'SAVE_DATA_CLOUD', 'label': 'Save Data Cloud'}, 'Platform:835:en-US': {'__typename': 'Platform', 'id': '835:en-US', 'label': 'Nintendo Switch', 'code': 'NINTENDO_SWITCH'}, 'PlayMode:22382:en-US': {'__typename': 'PlayMode', 'id': '22382:en-US', 'code': 'TV_MODE', 'label': 'TV mode'}, 'PlayMode:22379:en-US': {'__typename': 'PlayMode', 'id': '22379:en-US', 'code': 'TABLETOP_MODE', 'label': 'Tabletop mode'}, 'TopLevelCategory:GAMES:en-US': {'__typename': 'TopLevelCategory', 'id': 'GAMES:en-US', 'code': 'GAMES', 'label': 'Games'}, 'StoreProduct:{"sku":"7100026441","locale":"en_US"}': {'__typename': 'StoreProduct', 'ageGate': False, 'appStoreUrl': None, 'availability': ['Available now'], 'baseSoftware': None, 'bundleItems': None, 'capacity': None, 'configurableOptions': None, 'configurableProduct': None, 'containsBattery': False, 'crossSellProducts': None, 'countryOfManufacture': None, 'countryOfOrigin': None, 'demoNsuid': None, 'demoType': None, 'description': '<p>Up to 4 players control identical avatars (the “Thieves”) in a single-screen arena (the “Town”), also filled with other identical NPCs. Players must identify which NPCs are secretly other players and stab them for points, before being discovered and stabbed themselves!<br><br>Features:<br><br>Competitive 2-4 player battles – Blend in with thieves while you search for and stab your opponents <br><br>Multiple modes and items – Guns, smoke bombs, drunks, sheriffs, sandstorms, and killer tumbleweeds<br><br>Game Modes:<br><br>Thief Town – The classic backstabbing adventure. Players disguise themselves and stab their buddies in the midst of sandstorms and killer tumbleweeds. <br><br>Spy Town – Players are given items to trick and trap enemy thieves. Smoke bombs, motion detectors, and teleportation devices... Be careful, they’re one-use only! <br><br>Drunk Town – One randomly selected player (the "Sheriff") must identify and shoot the other knifeless drunks before running out of bullets.</p>', 'disclaimer': '<p><p>A Nintendo Switch Online membership (sold separately) is required for Save Data Cloud backup.</p></p><p>© Rude Ghost Games 2020. Licensed to Skymap Games.</p>', 'displayChokingHazard': False, 'displayProp65': False, 'dlcType': None, 'edition': {'__ref': 'AttributeOption:109:en-US'}, 'enableRetailCrawler': False, 'eshopDetails({"personalized":false})': {'__typename': 'EshopDetails', 'isPreorderable': False, 'isPurchasable': True, 'goldPoints': 10, 'purchaseUrl': 'https://ec.nintendo.com/title_purchase_confirm?title=70010000026441', 'discountPriceEnd': '2024-02-08T07:59:59Z'}, 'exclusive': False, 'facets': {'__typename': 'Facets', 'corePlatforms': ['Nintendo Switch']}, 'gamesShown': False, 'googlePlayUrl': None, 'isSalableQty': True, 'manufacturer': None, 'maxQtyAllowedInCart': 10000, 'metaDescription': 'Up to 4 players control identical avatars (the “Thieves”) in a single-screen arena (the “Town”), also filled with other identical NPCs. Players must identify which NPCs are secretly other players and stab them for points, before being discovered and stab…', 'metaKeyword': None, 'metaTitle': 'Thief Town', 'nsuid': '70010000026441', 'numberOfPlayers': None, 'productType': 'SIMPLE', 'platinumPoints': None, 'prePurchase': None, 'productCode': 'HACPAWMKA', 'prices({"personalized":false})': {'__typename': 'PriceRange', 'minimum': {'__typename': 'Price', 'amountOff': 6, 'currency': 'USD', 'discounted': True, 'finalPrice': 1.99, 'regularPrice': 7.99}, 'maximum': {'__typename': 'Price', 'amountOff': 0, 'currency': 'USD', 'discounted': False, 'finalPrice': 7.99, 'regularPrice': 7.99}}, 'relatedArticles': [], 'qtyAllowedPerCustomer': None, 'relatedProducts({"limit":5})': [], 'requiresLogin': None, 'requiresCoupon': None, 'requiresSubscription': None, 'romFileSize': '224395264', 'series': None, 'shipDateDisplay': None, 'size': None, 'sizeChart': None, 'soldOutTemporary': False, 'soldOutPermanent': False, 'startShippingDate': None, 'switchRequired': False, 'topLegalDisclaimer': None, 'upc': None, 'upsellProducts': None, 'variations': None, 'voucherNsuid': None, 'backgroundColor': 'f1e3c9', 'contentDescriptors': [{'__ref': 'ContentDescriptor:{"id":"4rXfJg3rY2e58ECl3iULqU","locale":"en_US"}'}, {'__ref': 'ContentDescriptor:{"id":"6jE2KH8jLDQfYywkrWMKEx","locale":"en_US"}'}, {'__ref': 'ContentDescriptor:{"id":"45tDlAomCY9ukpHytMbwfl","locale":"en_US"}'}], 'contentRating': {'__ref': 'ContentRating:{"id":"4QzesqRbAOTJ05zgLM8Yce","locale":"en_US"}'}, 'descriptionImage': None, 'genres': [{'__ref': 'GameGenre:22496:en-US'}, {'__ref': 'GameGenre:22463:en-US'}, {'__ref': 'GameGenre:22487:en-US'}], 'headline': 'Thief Town is a local multiplayer stealth action game set in a pixel-perfect rendition of the Wild West. Play with your friends and achieve backstabbing victory!', 'id': 'OTUzOTg=', 'locale': 'en_US', 'localeAlternates': [{'__ref': 'StoreProduct:{"sku":"7100026441","locale":"en_CA"}'}, {'__ref': 'StoreProduct:{"sku":"7100026441","locale":"es_MX"}'}, {'__ref': 'StoreProduct:{"sku":"7100026441","locale":"fr_CA"}'}], 'name': 'Thief Town', 'nsoFeatures': [{'__ref': 'NsoFeature:103:en-US'}], 'officialSite': 'https://skymapgames.com/games/thief-town/', 'platform': {'__ref': 'Platform:835:en-US'}, 'playersMax': 4, 'playersMaxLocal': None, 'playersMaxOnline': None, 'playersMin': 2, 'playersMinLocal': None, 'playersMinOnline': None, 'playModes': [{'__ref': 'PlayMode:22382:en-US'}, {'__ref': 'PlayMode:22379:en-US'}], 'productGallery': [{'__typename': 'CloudinaryAsset', 'publicId': 'Legacy Videos/92aHM1ajE6QFQDTNyyfz8Bq63Xb6-1j_', 'resourceType': 'video', 'type': 'upload'}, {'__typename': 'CloudinaryAsset', 'publicId': 'ncom/software/switch/70010000026441/b8a93f62f42813a4d081107c4caaf03cfe631270a321d49e40dc3b1622f5b622', 'resourceType': 'image', 'type': 'upload'}, {'__typename': 'CloudinaryAsset', 'publicId': 'ncom/software/switch/70010000026441/239e27742751c39175b0182d94827173b3981b559068fbaa81697de5a7b098b9', 'resourceType': 'image', 'type': 'upload'}, {'__typename': 'CloudinaryAsset', 'publicId': 'ncom/software/switch/70010000026441/df0eeb4ec75a8d3fe7085504cc642a9d8b37582f6dadfbc408af47bf1c232cba', 'resourceType': 'image', 'type': 'upload'}, {'__typename': 'CloudinaryAsset', 'publicId': 'ncom/software/switch/70010000026441/6bb7926850e76b45eb9bb56edd64b440d154439ff87d6bec58ef5ba7f6ea30d6', 'resourceType': 'image', 'type': 'upload'}, {'__typename': 'CloudinaryAsset', 'publicId': 'ncom/software/switch/70010000026441/1ffb28dfa3c89c72c03fc96588ce35b3f147b8ab3346fab8da8d2dcee54d6c75', 'resourceType': 'image', 'type': 'upload'}, {'__typename': 'CloudinaryAsset', 'publicId': 'ncom/software/switch/70010000026441/0914a29eb3f2bb14ae383cecf19bf96a1bfa4be7d37964064262fef8a6361342', 'resourceType': 'image', 'type': 'upload'}], 'productImage': {'__typename': 'CloudinaryAsset', 'publicId': 'ncom/software/switch/70010000026441/5d287d59c0b22cfe9e5e273d5f3a6b7b34d7cb0ed5cec3a3c43bf0e044b596a4', 'resourceType': 'image', 'type': 'upload'}, 'releaseDate': '2020-02-11T00:00:00.000Z', 'releaseDateDisplay': None, 'sku': '7100026441', 'softwareDeveloper': 'Rude Ghost', 'softwarePublisher': 'Rude Ghost', 'supportedLanguages': ['English'], 'topLevelCategory': {'__ref': 'TopLevelCategory:GAMES:en-US'}, 'url({"relative":true})': '/us/store/products/thief-town-switch/', 'urlKey': 'thief-town-switch'}, 'ROOT_QUERY': {'__typename': 'Query', 'product({"locale":"en_US","urlKey":"thief-town-switch"})': {'__ref': 'StoreProduct:{"sku":"7100026441","locale":"en_US"}'}, 'features({"ids":["wdev-1692-meta-description-update"],"project":"ncom"})': [{'__ref': 'Feature:wdev-1692-meta-description-update'}]}, 'Feature:wdev-1692-meta-description-update': {'__typename': 'Feature', 'id': 'wdev-1692-meta-description-update', 'impressionData': False, 'stale': False, 'strategies': [{'__typename': 'FeatureStrategy', 'constraints': [], 'name': 'flexibleRollout', 'parameters': {'__typename': 'FeatureStrategyParameters', 'groupId': 'wdev-1692-meta-description-update', 'hostNames': None, 'rollout': 100, 'stickiness': 'DEFAULT', 'userIds': None}}], 'type': 'RELEASE', 'variants': []}}}, '__N_SSP': True}, 'page': '/store/products/[slug]', 'query': {'slug': 'thief-town-switch'}, 'buildId': 'kL46b_cgolkQGMygqdhlD', 'assetPrefix': '/us', 'isFallback': False, 'gssp': True, 'customServer': True, 'locale': 'us', 'locales': ['default', 'us', 'en-ca', 'fr-ca', 'es-mx', 'pt-br', 'es-ar', 'es-cl', 'es-co', 'es-pe'], 'defaultLocale': 'default', 'scriptLoader': []}

The 'product' tag is hiding under 'analytics'. I think I saw this before, last summer. Do you have any grievance but adding a try, except for when product is not under page props to check if its under analytics?

`noe.game_info` fails for DLCs

Hello,

I realize that the timing to report this is quite unfortunate, so feel free to ignore this until January or later :-) Did not want to forget about this issue.

I added some code to update existing game info and noticed that the game_info for the NOE API fails to update:

a) Games no longer available - which makes sense, they aren't on the eShop

b) Lots of DLCs where it doesn't find the game by nsuid

Stuff like "Dead Cells: The Bad Seed [70050000017619]", "Splatoon 3: Expansion Pass [70070000016777]", "The Witcher 3: Wild Hunt – Blood and Wine [70050000024654]" and more. These are available on the eShop.

I think, this is due the _search is implemented in nintendo.py for NOE. Spefically this part:

params = {
        "fq": "type:GAME",
        "q": query,
        "rows": rows,
        "sort": "title asc",
        "start": -rows,
        "wt": "json",
    }

My guess is that the DLC need type:DLC for the search to work.

Happy to try to fix this with PR, I am just not sure what would be the best approach. If it is possible to specify multiple type values there, or if there should be fallback search that tries type:DLC in case the first one fails.

algoliasearch.exceptions.AlgoliaUnreachableHostException: Unreachable hosts

this my code:

from nintendeals import noa

def getUSGamesBrief():
    index = 1
    for game in noa.list_switch_games():
        print(index, "  ", game.title)
        index += 1

if __name__ == '__main__':
    getUSGamesBrief()

and it throw an exception when i run it:

/usr/local/bin/python3.9 /Users/mac/PycharmProjects/switch/venv/gamesUS.py
Traceback (most recent call last):
  File "/Users/mac/PycharmProjects/switch/venv/gamesUS.py", line 12, in <module>
    getUSGamesBrief()
  File "/Users/mac/PycharmProjects/switch/venv/gamesUS.py", line 7, in getUSGamesBrief
    for game in noa.list_switch_games():
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/nintendeals/noa/listing.py", line 52, in list_switch_games
    yield from list_games(Platforms.NINTENDO_SWITCH)
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/nintendeals/noa/listing.py", line 10, in list_games
    for data in algolia.search_by_platform(platform):
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/nintendeals/noa/api/algolia.py", line 63, in search_by_platform
    games = _search_index(query, **options)
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/nintendeals/noa/api/algolia.py", line 34, in _search_index
    response = INDEX.search(query, request_options=options)
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/algoliasearch/search_index.py", line 284, in search
    return self._transporter.read(
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/algoliasearch/http/transporter.py", line 47, in read
    return self.request(verb, hosts, path, data, request_options, timeout)
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/algoliasearch/http/transporter.py", line 72, in request
    return self.retry(hosts, request, relative_url)
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/algoliasearch/http/transporter.py", line 94, in retry
    raise AlgoliaUnreachableHostException("Unreachable hosts")
algoliasearch.exceptions.AlgoliaUnreachableHostException: Unreachable hosts

Do you know why this happened?

Issue parsing product from new Nintendo JSON format

Hello! It seems like Nintendo may have changed their game info JSON structure, I'm attaching a sample of it as it downloaded for me (Breath.of.the.Wild.data.txt). In the Nintendo.py scrap() method, there was an exception looking for the "product" element.

soup = BeautifulSoup(response.text, features="html.parser")
script = soup.find("script", id="NEXT_DATA")
data = json.loads(script.text)["props"]["pageProps"]["product"]

I locally replaced the block of code with this and it works:
soup = BeautifulSoup(response.text, features="html.parser")
script = soup.find("script", id="NEXT_DATA")
linkedData = json.loads(script.text)["props"]["pageProps"]["linkedData"]
sku = linkedData["sku"]
productElement = "StoreProduct:{"sku":"" + sku + "","locale":"en_US"}"
data = json.loads(script.text)["props"]["pageProps"]["initialApolloState"][productElement]

Thanks for this awesome library!
Breath of the Wild data.txt

NOA crash when getting game info for "Bunny Park", `platform` key issue

Hello,

I have noticed interesting issue with the noa.game_info when fetching data for this game: https://www.nintendo.com/us/store/products/bunny-park-switch/. Its nsuid is "70010000052765".

bunny_game = noa.game_info(nsuid="70010000052765")

It crashes in the build_game method when attempting to get the platform: platform=PLATFORMS[data["platform"]],.

For some reason, the platform that is included for this game is: "'Nintendo Switch – OLED Model'" and not simply "Nintendo Switch" and therefore the dictionary lookup throws KeyError.

The key corePlatforms correctly has "Nintendo Switch" entry.

I think maybe extending the PLATFORMS model like this would work?

PLATFORMS = {
    "Nintendo Switch": Platforms.NINTENDO_SWITCH,
    "Nintendo Switch – OLED Model": Platforms.NINTENDO_SWITCH
}

Nintendo API rate limits?

Hello,

today I discovered that my prices import started failling due to 403. Seems like my server IP got blocked from accessing https://api.ec.nintendo.com/v1/price. I had import for around 10 countries that run once a day with sleep sprinkled in to not make too many requests in quick succession.

Has anyone run into something similar?

Crash when i print game.slug

thanks for you job first, and i met this crash when i print game.slug, i don't know what caused this crash, hope you can fix it if it is a bug, and another problem is that some of the games' uuid is None when i call
list_switch_games() method for example Cyanide & Happiness – Freakpocalypse: Part 1, this game doesn't have a nsuid.

Traceback (most recent call last):
  File "nintendo_games.py", line 52, in <module>
    print(game.slug)
  File "/usr/local/lib/python3.8/site-packages/nintendeals/validate.py", line 173, in inner
    return func(*args, **kwargs)
  File "/usr/local/lib/python3.8/site-packages/nintendeals/noe/info.py", line 215, in game_info
    return _scrap_switch(nsuid=nsuid)
  File "/usr/local/lib/python3.8/site-packages/nintendeals/noe/info.py", line 108, in _scrap_switch
    value, unit = rom_size.split(" ")
ValueError: too many values to unpack (expected 2)

No luck getting game data from NOA for Metroid Prime Remastered or Octopath Traveler II for example

Hello,

I hoped I wouldn't have to open an issue but I cannot get NOA integration to work.

I tried it with a couple of new games Metroid Prime Remastered (NSUID: 70010000063709) and Octopath Traveler II (NSUID 70010000058128 but I always get back None.

When investigating further, I found that the issue seem to be in game_info_by_nsuid when this is called data = algolia.search_by_nsuid(nsuid)

And bellow it returns None because the data couldn't be found.

I also tried the scrap method available for NOA and this works. I can get the nsuid and then use the prices API to get the info on NOA.

Is something wrong with the algolia at the moment?

I am open to trying to fix this issue if you can point me to the right direction.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.