eldur / jwbf Goto Github PK
View Code? Open in Web Editor NEWJava Wiki Bot Framework is a library to maintain Wikis like Wikipedia based on MediaWiki.
Home Page: http://jwbf.sourceforge.net/
License: Apache License 2.0
Java Wiki Bot Framework is a library to maintain Wikis like Wikipedia based on MediaWiki.
Home Page: http://jwbf.sourceforge.net/
License: Apache License 2.0
Uploading my article with an empty last line, is fine, because I can see it like expected online.
But querying with jwbf for this article the last line is dropped. Any idea?
Mediawiki has released version 1.24 last month and updated it's site, now the integration test fails because it expects 1.23 as the last version
JUnit reports:
Tests run: 3, Failures: 1, Errors: 0, Skipped: 2, Time elapsed: 1.1 sec <<< FAILURE! - in net.sourceforge.jwbf.mediawiki.live.SiteinfoTest
siteInfoLastVersion(net.sourceforge.jwbf.mediawiki.live.SiteinfoTest) Time elapsed: 1.1 sec <<< FAILURE!
java.lang.AssertionError: 1.24.0<noinclude>[[Category:MediaWiki version information templates]]</noinclude> should contains 1.23
at org.junit.Assert.fail(Assert.java:88)
at org.junit.Assert.assertTrue(Assert.java:41)
at net.sourceforge.jwbf.mediawiki.live.SiteinfoTest.siteInfoLastVersion(SiteinfoTest.java:80
That's also the reason why my pull request (#40) fails in Travis.
Could you add support for Wikidata?
Related links:
I wrote an action which has an integer parameter baserevid and I got this:
Exception in thread "main" java.lang.UnsupportedOperationException: No Handler found for java.lang.Integer
at net.sourceforge.jwbf.core.actions.HttpActionClient.applyToEntityBuilder(HttpActionClient.java:205)
at net.sourceforge.jwbf.core.actions.HttpActionClient.post(HttpActionClient.java:186)
at net.sourceforge.jwbf.core.actions.HttpActionClient.processAction(HttpActionClient.java:159)
at net.sourceforge.jwbf.core.actions.HttpActionClient.performAction(HttpActionClient.java:133)
at net.sourceforge.jwbf.core.bots.HttpBot.performAction(HttpBot.java:56)
at net.sourceforge.jwbf.mediawiki.bots.MediaWikiBot.performAction(MediaWikiBot.java:261)
at net.sourceforge.jwbf.mediawiki.bots.MediaWikiBot.getPerformedAction(MediaWikiBot.java:266)
I searching a way to do custom queries. For example I want to the get the equivalent article in the English Wikipedia for a French Wikipedia article. Is there something like this:
Get query = buildQuerie();
result = getAPIResponse(query) ;
The result could be a String (the raw response of the API), a HashMap created from the JSON response of the API. Or we could pass a class to get the result in custom object that will be created from the JSON response (we will use Jackson library here).
What do you think?
Is there any function to test if an article exist or not?
I have looked to some bots code that I have found and to test if an article exist or not they do :
return !mw.getArticle(title).getText().isEmpty();
If an article is empty (but it exist) this will return false.
PS: This library has many functionalities but it's hard to figure out how to use it.
HttpActionClient.builder()
.withServerThrottlingInSeconds(5)
...
This feature have to mutate requests before sending it to the api ...
I think it would be a good idea if user-agent would be customizable. I'm actually trying to do it with your base code but I find it very hard to do without changing the code.
First of all, nice framework! It's simple and cool to use..
Yesterday I spent my whole evening figuring out why the heck I couldn't post some changes to a wiki. Today I analysed the packages sent over the line and found this:
<api><edit result="Failure"><captcha type="image" mime="image/png" id="1178352419" url="/w/index.php?title=Spezial:Captcha/image&wpCaptchaId=1178352419" /></edit></api>
It would be really helpful if jwbf would react on that with an exception or at least a log message!
Hi,
i'm trying to upload an image to my local MediWiki, but it won't work.
Here is the code i wrote:
public class Test {
static String localPathToFile = "../MediaWiki_jwbf_Test/img/schaf4.gif";
static String url = "http://localhost/wiki1.20.7/";
static String user = "wikiuser";
static String password = "mypassword";
static String descriptionText = "Schaf";
public static void main (String[]args){
MediaWikiBot b = new MediaWikiBot(url);
b.login(user, password);
SimpleFile file = new SimpleFile(localPathToFile);
FileUpload fu = new FileUpload(file, b);
b.performAction(fu);
}
}
Is there any mistake in the code?
The login and creating of a new page in the wiki work correct and $wgEnableUploads in the LocalSettings.php ist set to true. The version of the MediaWiki i work with is 1.20.7 and 1.19.8.
Could you help me with this problem?
I would like to known if your tool integrates some scripting language and some parser for the MediaWiki wikitext.
Thanks for your attention.
Defaulting to gzip compression would make it easier for users to use less bandwidth in requests.
Hi
After several days of tests, the snapshot 2.1.0 is stable for what I need to do
Therefore, is it possible to have a public release of the snapshot 2.1.0 that fix the issue #8
Thank you very much
David
Hello,
I have an issue using JWBF within JBoss.
there's an exception : URI scheme is not "file.
I tried 2 solutions:
JWBF is in the EAR
JWBF in JBoss lib
In both case, the URL starts with vfs (Virtual FS)
Is there any solution?
Thanks you
To begin using the new format of continuation data requires empty string for 'continue' parameter as provided in warning message. Currently the empty string is transformed to None which will lead to error.
Changes in token handling: http://lists.wikimedia.org/pipermail/mediawiki-api-announce/2014-August/000063.html
I wrote a post action based on your last module and other post actions. It works with smaller requests, but with larger ones I get Exception in thread "main" java.lang.IllegalStateException: invalid status: HTTP/1.1 414 Request-URI Too Large; for ... at net.sourceforge.jwbf.core.actions.HttpActionClient.execute(HttpActionClient.java:299). Could you help me?
AC:
MediaWiki 1.23.0 (download) - stable
MediaWiki 1.22.7 (download) - legacy
MediaWiki 1.19.16 (download) - legacy lts
I started a new project with Maven and copy/pasted the dependency provided in the README to my .pom. When I tried to build, I got the following error:
The POM for net.sourceforge:jwbf:jar:3.0.0-SNAPSHOT is missing, no dependency information available
When I look at Maven Central I only see versions up to 2.0.0.
Does not support Wikia. When using a page from a Wikia wiki, the Exception "java.lang.IllegalArgumentException: Invalid XML" occurs.
I notice that there has been some discussion about adding some wikidata query functionality to jwbf:
Wikidata/Wikidata-Toolkit#11
I think that sounds great, but I thought I should note that the existing jwbf API doesn't support getting the wikidata page id - which I think is the essential link between the two.
That id is displayed rather prominently in the page info of very wiki page, but I actually don't even know how to retrieve it via the mediawiki API (I don't see a prop for it). Assuming that there is a way, and I'm just missing it, I think it would be appropriate to add that to jwbf when adding wikidata support.
Otherwise one is stuck using dbpedia, or probably the wikidata API has a way to translate a wikipedia id (plus language) into its own page id - but that would have to be inefficient.
Today I was trying for the fist time use your library (2.0.0), but I'm getting the following "warning":
SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
At the page they mention I see that I could solve the problem adding one of those libraries cited there (slf4j-nop.jar, slf4j-simple.jar, slf4j-log4j12.jar, slf4j-jdk14.jar or logback-classic.jar).
We have to add one of these manually?
Once again, thank you.
feels that there is something wrong.
The API etiquette guidelines require, at a bare minimum, either an email address or a URL to the project in the user agent.
I'm trying to upload a image to Wikimedia Commons. I already edited some article with your library, works fine now. By executing the following snippet no exceptions appear after network traffic has calm down. No new image is viewable on Commons.
HttpActionClient client = HttpActionClient.builder() //
.withUrl("https://commons.wikimedia.org/w/") //
.withUserAgent("BotName", "0.1", "jotpe") //
.withRequestsPerUnit(60, TimeUnit.MINUTES) //
.build();
MediaWikiBot wikiBot = new MediaWikiBot(client);
wikiBot.login("jotpe", "yyyy");
File f = new File("/tmp/Roof terrace Hostel.JPG");
SimpleFile file = new SimpleFile(f.getAbsolutePath());
file.addText(i.generateCommonsTemplate());
wikiBot.getPerformedAction(new FileUpload(file, wikiBot));
Is this related to Token handling #28 ? The last stament was:
97371 [net.jotpe.HelperFirstUpload.main()] DEBUG org.apache.http.headers - http-outgoing-0 << MediaWiki-API-Error: mustposttoken
The JavaDoc is very sparse and has few descriptions. The library would be easier to use with more method and class descriptions.
Greetings,
Long time user of your library here; I've been using it for almost two years now, and it's absolutely great.
I'm in the process of moving several thousand pages and I thought it would be a good idea to use your library especially since it has the MovePage class.
For some reason, I can't seem to get this class to work for me. Some of the error I get include:
[Thread-3] ERROR net.sourceforge.jwbf.mapper.XmlConverter - unknownerror: Unknown error: "immobile-source-namespace"
^ Fixed when I used MediaWiki.urlEncode(oldtitle) and MediaWiki.urlEncode(newtitle) in my MovePage function call.
AND:
no attribute found for key: movetoken
^ Can't get it fixed.
I read the MovePageTest and the MovePage java doc online, but they offered no clue as to why I'm getting the above error message. I'm pretty nothing is wrong with my LocalSettings.php file...
Is this a known issue with the version of MediaWiki I'm using (1.23.2)? If so, is there any way to fix it other than upgrading my wiki to the latest version? Please help!
I recently tried to use the 4.0.0-SNAPSHOT version with IntelliJ and Maven, but it throws a NoClassDefFound exception when I try and instantiate your classes. I've pretty much copy-and-pasted the dependency tags you put in the README. This is probably not the place to put this, but I couldn't find any email associated with you. I tried asking StackOverflow, assuming it was my problem, but they can't seem to find an answer. I was hoping you could help.
Running
new MediaWikiBot("https://en.wikipedia.org/w/").getArticle("42");
I started to get
jan. 24, 2016 5:49:00 DU org.apache.http.client.protocol.ResponseProcessCookies processCookies
WARNING: Invalid cookie header: "Set-Cookie: WMF-Last-Access=24-Jan-2016;Path=/;HttpOnly;Expires=Thu, 25 Feb 2016 12:00:00 GMT". Invalid 'expires' attribute: Thu, 25 Feb 2016 12:00:00 GMT
warning.
I am using 4.0.0-SNAPSHOT with MediaWiki 1.27.0-wmf.10
Greetings,
This isn't exactly an issue, but more of a question.
Is it possible to use this library to delete category pages? If so, how?
Thanks in advance.
Hi again!
As you library seems to be the only available library for Java (there are one or two apart, but those are not being updated more) when we talk about MediaWiki API (i.e., not through database dump), I did keep testing you library.
Are you thinking to introduce a way to work with the revisions?
Search for Wikitext in various pages? Seems that now they introduced the CirrusSearch and its able to do search with regular expressions (see http://www.mediawiki.org/wiki/Thread:Help_talk:CirrusSearch/Regular_expressions), but I don't known ifs available through the rest API.
Those features would be interesting for example, in the area of anti-vandalism.
The next release (maby 4.0) should loose its xml dependecies, because mediawikiAPI prefers json.
AC
When we do a request to MediaWiki api, the library ask for 50 results. It's not configurable. MediaWiki api return up to 500 results for normal users and up to 5000 for bots.
The library will make 100 request to get all articles that are in category that contains 5000 articles. Although it's possible to get them all with just one request. MediaWiki api doc also recommend this.
One solution is to add getter and setter to be able to change it.
I'm trying (without success) to use jwbf.jar into an Eclipse plugin. Mainly the problem is Caused by: java.lang.IllegalArgumentException: URI scheme is not "file"
at java.io.File.(Unknown Source)
at net.sourceforge.jwbf.JWBF.init(JWBF.java:81)
at net.sourceforge.jwbf.JWBF.(JWBF.java:47)
Please try this on your side or tell me who can I use it in my purpose (Eclipse application integration)
I'am working behind a proxy. I have found a post on the subject http://sourceforge.net/p/jwbf/feature-requests/20/ but the code is deprecated.
Anyone has an idea ?
Thanks
Vincent Hilaire
Add 1.24 and 1.25 to the available versions and issue warning if the version is unknown.
Make efficient usage of API calls easier, possibly by creating a method that combines calls for multiple items (e.g. title1|title2|title3|...
).
New users will find it easier to get started if there are step-by-step instructions for installation for use/development.
AC:
Affected tests:
lorinczz commented #34
HttpActionClientTest.testGet_headers_customUserAgent
HttpActionClientTest.testPostParameters
because of "Version unknown" in userAgentString method.
Connection=keep-alive
Content-Length=???
Content-Type=multipart/form-data; boundary=????
Host=localhost:????
User-Agent=Unknown/Unknown JWBF/DEVEL Apache-HttpClient/4.3.4 (java 1.5)
^^^ "Version unknown" and "Version_unknown."
It is hard to find the documentation. It would be helpful to link to a download of the JavaDoc, put the JavaDoc online, or give instructions to compile it.
prepareCollection doesn't happen to use "from" when it is first called.
protected HttpAction prepareCollection() {
return generateRequest(nextPageInfoOpt(), prefix, rf,
MWAction.createNsString(namespaces));
}
Is it or will it be possible to watch articles (add them to personal watchlist) ?
Please consider implementing OAuth 1.0a authentication. OAuth will be the preferred way of authentication for bots.
Example config for Pywikibot: https://www.mediawiki.org/wiki/Manual:Pywikibot/OAuth
I can't use TemplateUserTitles with more than 50 results. It gives the same titles several times.
AC:
AC:
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.