opendatatrentino / jackan Goto Github PK

Although CKAN has a plugin to expose data as DCAT RDF, not all portals support this plugin. Moreover, the plugin is marked as work-in-progress.

Would be nice to know in tests which kind of server are we dealing with (CKAN / DKAN / other ?)

we must turn Jackan to jackan

Generated html table is very raw.
We could improve it with DataTables. Also, a CSV export would be nice.

Adding extras or others in CkanBaseResources works but they are not uploaded

Implementing file uploads doesn't seem difficult (see http://docs.ckan.org/en/ckan-2.2/filestore.html), although it's not a priority for now.

Some travis continuous integration can help with testing pull requests

Hi, I'm experiencing some issues with these domains:

1. http://www.dati.piemonte.it (API URL: http://www.dati.piemonte.it/rpapisrv/api/rest/package)
   In the developer page there's written that their API are compatible with CKAN model, but Jackan doesn't recognize the domain: Couldn't interpret json returned by the server!
   I saw that if I use as domain "http://www.dati.piemonte.it", it becomes: "http://www.dati.piemonte.it/api/3/action/package_list" and I think this is not correct.
   Can you do something?

2. http://www.dati.gov.it/ (API URL: http://www.dati.gov.it/api/3/action/package_list)
   Jackan answer: NullPointerException: Need a valid id or name!
   I think that for dati.gov the parameter "limit" is mandatory.
   Same question as above.

Thanks.

Hey guys,

I'm using Jackan 0.4.2 to connect to an SSL secured server using SNI (Server Name Identification). I think there's a problem with the httpclient version (currenlty 4.3.1, above version shouldn't be affected) included by the shade plugin.

I tried with version 4.3.3 (httpclient has a version 4.3.6 but httpcore did not so I chose the common version) and this worked out. Probably I could also use an even higher version of httpclient but I don't know if you rely on other parts of the API, though.

PR is in progress ...

BTW: Is there a reason why you include all sources and ship the stuff? This led me checkout the sources and patch the pom.xml. Wouldn't a plain dependency work out the same?

Thanks.

Some write tests don't pass with demo.ckan.org . In some cases it is just a matter of authorizations. But as a rule of thumb, we should keep demo.ckan.org as a reference and make sure most tests pass. In the wiki we will try to keep reviews of the server status

Some repository needs authorization also for get requests

Tests fails because Timestamp formatting is timezone dependant.

First spotted in #39

Exception in thread "main" eu.trentorise.opendata.jackan.JackanException: Error while getting dataset.
at eu.trentorise.opendata.jackan.ckan.CkanClient.getHttp(CkanClient.java:120)
at eu.trentorise.opendata.jackan.ckan.CkanClient.getDatasetList(CkanClient.java:130)
at eu.trentorise.opendata.ckanalyze.main.TestMain.main(TestMain.java:42)
Caused by: org.apache.http.client.HttpResponseException: Bad Request
at org.apache.http.client.fluent.ContentResponseHandler.handleResponse(ContentResponseHandler.java:47)
at org.apache.http.client.fluent.ContentResponseHandler.handleResponse(ContentResponseHandler.java:40)
at org.apache.http.client.fluent.Response.handleResponse(Response.java:85)
at org.apache.http.client.fluent.Response.returnContent(Response.java:92)
at eu.trentorise.opendata.jackan.ckan.CkanClient.getHttp(CkanClient.java:108)
... 2 more

Exceptions in HTML reported by CkanTestReporter are not properly escaped.

Hey, are you actively developing it, if not, I can help, in our work we are using ckan for a webpage and we need to simplificate the developmen with an API, we can work with this

Maybe we can implement this workaround: Pragmatists/JUnitParams#52

Cannot create a dataset with a group because Ckan complains about a mysterious '__junk' field. This is because Ckan doesn't like the passed group to have users in object format inside. If users are represented as strings or not serializaed at all it works fine.

Hi,
I'm "novice" about Jackan/Ckan technologies. During these days I created several JUnit Test
to better understand how to use Jackan with Ckan.
I'm using Jackan 0.4.2 with Ckan (2.5.1) and seems that the method CkanClient.makeResourceUrl(CKAN_CLIENT.getCatalogUrl(), resourceCreated.getPackageId(), resourceCreated.getId())) returns a broken URL.

In particular the URL returned does not contain '/dataset/' in the path.
The method returns:
http://[hostname]/[datasetID or datasetName]/resource/[resourceID] but is broken
adding the string "/dataset/" in the path after [hostname]:
http://[hostname]/dataset/[datasetID or datasetName]/resource/[resourceID] is right

Is it a bug or a misconfiguration on CKAN server? Thx a lot.

Releasing with proper IT tests causes issues because of this.
See opendatatrentino/tod-commons#1

Various methods like createCkanResource, createCkanDataSet

call printStackTrace, we need to rethrow exception wrapped in a JackanException

Query done and object structure don't actually allow to return the packages associated with an org.

Couldn't find how to configure Jackson and Joda for custom formatting of dates. Anyway this is an issue only when uploading, which we don't support yet.

Hello
I'm trying to create a new dataset with a csv file as a resource. The send process runs with no errors, but in CKAN I can't access the csv content. My code is as follows:

	dataset.setResources(new ArrayList<CkanResource>());
	String fileName = "test.csv";
	CkanResource resource = new CkanResource(CKAN.ckanDatasetUrl + "/" + fielName, fileName);
	resource.setId(UUID.randomUUID().toString());
	resource.setName(fileName);
	resource.setFormat("csv");
	resource.setMimetype("text/csv");
	StringBody body = new StringBody(csvContent, ContentType.MULTIPART_FORM_DATA);
	resource.putOthers("upload", body);

Any help will be very helpful.

Sincerely,

Alex

I have an issue reading certain records from demo.ckan.org which have "others" property populated with a simple string.

Example of such an record is:
http://demo.ckan.org/api/3/action/package_search?q=id:69c59a57-9ff3-43ff-915a-3a1c04bc4d80

You can see that there is a property others: "qui1hfv3qtm85lg2iokob6lg1m" within the very first resource entry.

It appears that Jackan fails to load this resource because it attempts to load a simple string into the private Map<String, Object> others bean property as defined in CkanResourceBase class.

Hi guys, I'm currently using the Jackan version 0.4.2 and I'm wondering if there is a way of using the Visibility/Searchable fields in order to make (make not) the dataset under creation visible and/or searchable. The extended CKanDataset class you provide, seems to have setPriv() and setOpen() methods, but I do not know if they are the ones I was looking for.

However I tried to use them, but they seem to have no effect. When creating a dataset using the CKanDataset (which extends CkanDatasetBase), the call reaches this point

` public synchronized CkanDataset createDataset(CkanDatasetBase dataset) {
checkNotNull(dataset, "Need a valid dataset!");

    checkToken("Tried to create dataset" + dataset.getName());

    String json = null;
    try {

        json = getObjectMapperForPosting(CkanDatasetBase.class).writeValueAsString(dataset);
    } catch (IOException e) {
        throw new CkanException(COULDNT_JSONIZE + dataset.getClass()
                                                         .getSimpleName(),
                this, e);
    }

    DatasetResponse response = postHttp(DatasetResponse.class, "/api/3/action/package_create", json,
            ContentType.APPLICATION_JSON);

    return response.result;
}`

Is getObjectMapperForPosting(CkanDatasetBase.class) method invocation right? Or we need to differenciate between CkanDatasetBase and CkanDataset?

Thanks :)

Dears, I am getting the following error message while accessing http://datahub.io via cc.getDatasetList().

Error message --> Exception in thread "main" eu.trentorise.opendata.jackan.exceptions.CkanException: Error while performing GET. Request url was: https://datahub.io/api/3/api/3/action/package_list?limit=1000&offset=0 CkanClient{catalogURL=https://datahub.io/api/3, ckanToken=null} Caused by: javax.net.ssl.SSLException: hostname in certificate didn't match: <datahub.io> != <.ckan.io> OR <.ckan.io> OR <ckan.io>

Is there any workaround on this problem?

Thanks a lot.

we need title (can be like 'Laghi monitorati Trento') and also license