tweeticoats / stash-plugin-scrape_with Goto Github PK

View Code? Open in Web Editor NEW

18.0 18.0 7.0 48 KB

Python 100.00%

stash-plugin-scrape_with's People

Contributors

Stargazers

Watchers

Forkers

dullcibus stg-annon lukesal88 burkely-00orso69 elkorol sleaze

stash-plugin-scrape_with's Issues

Getting errors when running

First of all thanks for the plugin!
When I try to run it I get the following errors, perhaps they are caused by stash v0.6.0? I see v0.5.0 was the one that was released on last commit here.

DEBU[2021-04-01 02:01:17] [Plugin] Mode: scrape_all
ERRO[2021-04-01 02:01:17] [Plugin] Traceback (most recent call last):
ERRO[2021-04-01 02:01:17] [Plugin]   File "/home/usr/.stash/plugins/stash-plugin-scrape_with/scrape_with.py", line 948, in <module>
ERRO[2021-04-01 02:01:17] [Plugin]     client.update_all_scenes_with_tags()
ERRO[2021-04-01 02:01:17] [Plugin]   File "/home/usr/.stash/plugins/stash-plugin-scrape_with/scrape_with.py", line 634, in update_all_scenes_with_tags
ERRO[2021-04-01 02:01:17] [Plugin]     tags=self.listTags()
ERRO[2021-04-01 02:01:17] [Plugin]   File "/home/usr/.stash/plugins/stash-plugin-scrape_with/scrape_with.py", line 81, in listTags
ERRO[2021-04-01 02:01:17] [Plugin]     result = self.__callGraphQL(query)
ERRO[2021-04-01 02:01:17] [Plugin]   File "/home/usr/.stash/plugins/stash-plugin-scrape_with/scrape_with.py", line 70, in __callGraphQL
ERRO[2021-04-01 02:01:17] [Plugin]     query, variables))
ERRO[2021-04-01 02:01:17] [Plugin] Exception: GraphQL query failed:401 - b''. Query:                     
ERRO[2021-04-01 02:01:17] [Plugin]     query {
ERRO[2021-04-01 02:01:17] [Plugin]       allTags {
ERRO[2021-04-01 02:01:17] [Plugin]         id
ERRO[2021-04-01 02:01:17] [Plugin]         name
ERRO[2021-04-01 02:01:17] [Plugin]       }
ERRO[2021-04-01 02:01:17] [Plugin]     }. Variables: None
ERRO[2021-04-01 02:01:17] Plugin returned error: exit status 1

DEBU[2021-04-01 02:09:52] [Plugin] Mode: setup                         
ERRO[2021-04-01 02:09:52] [Plugin] Traceback (most recent call last):  
ERRO[2021-04-01 02:09:52] [Plugin]   File "/home/usr/.stash/plugins/stash-plugin-scrape_with/scrape_with.py", line 946, in <module> 
ERRO[2021-04-01 02:09:52] [Plugin]     client.setup_tags()             
ERRO[2021-04-01 02:09:52] [Plugin]   File "/home/usr.stash/plugins/stash-plugin-scrape_with/scrape_with.py", line 545, in setup_tags 
ERRO[2021-04-01 02:09:52] [Plugin]     scrapers=self.list_scrapers('FRAGMENT') 
ERRO[2021-04-01 02:09:52] [Plugin]   File "/home/usr/.stash/plugins/stash-plugin-scrape_with/scrape_with.py", line 179, in list_scrapers 
ERRO[2021-04-01 02:09:52] [Plugin]     result = self.__callGraphQL(query) 
ERRO[2021-04-01 02:09:52] [Plugin]   File "/home/usr/.stash/plugins/stash-plugin-scrape_with/scrape_with.py", line 70, in __callGraphQL 
ERRO[2021-04-01 02:09:52] [Plugin]     query, variables))              
ERRO[2021-04-01 02:09:52] [Plugin] Exception: GraphQL query failed:401 - b''. Query:  
ERRO[2021-04-01 02:09:52] [Plugin]         query listSceneScrapers {   
ERRO[2021-04-01 02:09:52] [Plugin]   listSceneScrapers {               
ERRO[2021-04-01 02:09:52] [Plugin]     id                              
ERRO[2021-04-01 02:09:52] [Plugin]     name                            
ERRO[2021-04-01 02:09:52] [Plugin]     scene{                          
ERRO[2021-04-01 02:09:52] [Plugin]       supported_scrapes             
ERRO[2021-04-01 02:09:52] [Plugin]     }                               
ERRO[2021-04-01 02:09:52] [Plugin]   }                                 
ERRO[2021-04-01 02:09:52] [Plugin] }. Variables: None                  
ERRO[2021-04-01 02:09:52] Plugin returned error: exit status 1

Asking for stash-go.squlite to be copied and placed in scrapers directory?

Hello! Wanted to first say that this is a great plugin -- I used it a few months ago and it worked really well. However, I can't seem to get it to work on a new docker container this time around. I've even tried making a new container several times because I wasn't sure if it was an error I made. Anyway, I've been consistently getting this message:

`2021/06/02 21:01:31 "POST http://localhost:9999/graphql HTTP/1.1" from 127.0.0.1:35736 - 200 116B in 3.3152ms

ERRO[2021-06-02 21:01:32] scraper: Error, the sqlite database stash-go.sqlite does not exist in the scrapers directory.

ERRO[2021-06-02 21:01:32] scraper: Copy this database from another stash instance and place this in the scrapers directory

ERRO[2021-06-02 21:01:32] could not unmarshal json: EOF

2021/06/02 21:01:32 "POST http://localhost:9999/graphql HTTP/1.1" from 127.0.0.1:35738 - 200 113B in 381.5178ms

ERRO[2021-06-02 21:01:32] scraper: input: {"name": "First Last"}'

(Obviously replace "First" and "Last" with the name of a performer. Please help?

Adding Unscrapable tag to a scene overwrites all other tags

How to reproduce:

Add any tag to a scene so it has at least one tag.
Add scrape_with_XXX to the scene tags so that the specified scraper does not return any result
Run task "Run fragment scrapers with tag"

Result: Scene has now the tag "Unscrapable"
Expected behavior: Any other tags + "Unscrapable"

Performers URL is awkward, replace with tag?

With the new bulk editing UI for performers, URL isn't available to edit, but tags are...
Plus, some scrapes (looking at you, Babepedia) put a url even if no info is really otherwise available.
And scraping performers with an existing url might be desired. So, tags make more sense.

Maybe making it like the scene scraper, and allowing scrape_with_babepedia, scrape_with_imdb, etc?
Might be better than the 'list' of scrapers in the code that must be hand edited now.

Recommend Projects

React

A declarative, efficient, and flexible JavaScript library for building user interfaces.
Vue.js

🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript

TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
TensorFlow

An Open Source Machine Learning Framework for Everyone
Django

The Web framework for perfectionists with deadlines.
Laravel

A PHP framework for web artisans
D3

Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

javascript

JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
web

Some thing interesting about web. New door for the world.
server

A server is a program made to process requests and deliver data to clients.
Machine learning

Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Visualization

Some thing interesting about visualization, use data art
Game

Some thing interesting about game, make everyone happy.

Recommend Org

Facebook

We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft

Open source projects and samples from Microsoft.
Google

Google ❤️ Open Source for everyone.
Alibaba

Alibaba Open Source for everyone
D3

Data-Driven Documents codes.
Tencent

China tencent open source team.