Comments (4)
Q1: what is page_filter(page_data){ ...}
A1: options.page_filter()
is a function to filter result pages. Return true
if you want to keep the element.
Q2 script speed limit and internet connection ?:
A2: We should call API sequential, we should not set a high options.max_threads, and your net speed is also metter. However you may set a higher options.max_threads, so it seems not a library issue.
Q4 depth: is it possible to limit category depth ? how ?
A4: Yes. I updated the document. https://kanasimi.github.io/wikiapi/Wikiapi.html#download
Q5 depth loop: is there a risk of depth infinite loop ? (Category:A contains Category:B which contains Category:A)
A5: No. The shallowest category will be selected.
Q6 header: is the xhr hearder properly set ? See meta:User-Agent_policy
A6: Use console.log(CeL.get_URL.default_user_agent);
to view.
Q7 resilience: Is the script resilient ?
- 7a) can survive poor and temporarily interrupted internet connection ?
A: Seeconsole.log(CeL.wiki.query.get_URL_options.error_retry);
- 7b) can resume where it ended
A: Yes, or equivalent. The script compare remote and local files, skip those already downloaded, and do so extremely quickly (+200 comparison/sec). See #55 .
Q8 mp3, ogg: how to download alternative file format such as .mp3
from .wav
?
A8: Yes. See https://kanasimi.github.io/wikiapi/Wikiapi.html#download
from wikiapi.
Well, I think this issue may be closed. Next time we may use "Projects" in the top to handle this kind of issues.
from wikiapi.
Sounds pretty good.
from wikiapi.
Yes, close and thanks to you. 👍🏼
We did it before the weekend eventually.
My apologizes also for moving texts, splitting issues, and editing posts all around. I go ahead wiki style to leave behind cleaner places, and more comprehensible issues. Thank you for your letting me the space to do so, I appreciate it.
Thanks to for your help on npm packages. 🙏🏼
As for coordination :
- LinguaLibre.org currently invite users to connect to a Lingualibre Discord Server, come over
😃 We have a nest of few nerds there. - Github Projects : i never used this but I want to learn. Either will follow your lead or with learn simultaneously. Let's open something there to test out. 👍🏼
from wikiapi.
Related Issues (20)
- Edit repository details to add `wikiapijs` tag. HOT 4
- Clarify naming convention (project branding) HOT 5
- Unexpected edit on bot's userpage HOT 3
- Upload upon : no edit of the page ? HOT 3
- Upload of missing file crashes script. HOT 6
- Download all files from a (Commons) category HOT 3
- Snowpack support HOT 9
- SyntaxError: Unexpected token '.' HOT 8
- Example broken on OSX HOT 5
- download is not a function HOT 4
- `.download()` scalability ? HOT 6
- "Too many values supplied for parameter \"pageids\". The limit is 50." HOT 3
- `.download()` : reduce calls to api.php, directly hit on https://upload.wikimedia.org HOT 9
- Cannot download : triggerUncaughtException HOT 6
- `.download()` : compare local and remote files by timestamps before downloading HOT 4
- `.download()`: `session.to_namespace('Category Name', 'Category')` needed HOT 1
- throws "API_URL.includes" is not a function when creating wikiapi instance HOT 5
- Please publish new version with delete
- Provide an option to set a custom User-Agent string HOT 4
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from wikiapi.