greenbuttonalliance / openespi-datacustodian-java

Spring Framework implementation of a Green Button Data Custodian OpenESPI Application

Introduction

Swagger UI allows anyone — be it your development team or your end consumers — to visualize and interact with the API’s resources without having any of the implementation logic in place. It’s automatically generated from your OpenAPI (formerly known as Swagger) Specification, with the visual documentation making it easy for back end implementation and client side consumption.

General

👉🏼 Want to score an easy open-source contribution? Check out our Good first issue label.

🕰️ Looking for the older version of Swagger UI? Refer to the 2.x branch.

This repository publishes three different NPM modules:

• swagger-ui is a traditional npm module intended for use in single-page applications that are capable of resolving dependencies (via Webpack, Browserify, etc.).
• swagger-ui-dist is a dependency-free module that includes everything you need to serve Swagger UI in a server-side project, or a single-page application that can't resolve npm module dependencies.
• swagger-ui-react is Swagger UI packaged as a React component for use in React applications.

We strongly suggest that you use swagger-ui instead of swagger-ui-dist if you're building a single-page application, since swagger-ui-dist is significantly larger.

If you are looking for plain ol' HTML/JS/CSS, download the latest release and copy the contents of the /dist folder to your server.

Compatibility

The OpenAPI Specification has undergone 5 revisions since initial creation in 2010. Compatibility between Swagger UI and the OpenAPI Specification is as follows:

Swagger UI Version	Release Date	OpenAPI Spec compatibility	Notes
5.0.0	2023-06-12	2.0, 3.0.0, 3.0.1, 3.0.2, 3.0.3, 3.1.0	tag v5.0.0
4.0.0	2021-11-03	2.0, 3.0.0, 3.0.1, 3.0.2, 3.0.3	tag v4.0.0
3.18.3	2018-08-03	2.0, 3.0.0, 3.0.1, 3.0.2, 3.0.3	tag v3.18.3
3.0.21	2017-07-26	2.0	tag v3.0.21
2.2.10	2017-01-04	1.1, 1.2, 2.0	tag v2.2.10
2.1.5	2016-07-20	1.1, 1.2, 2.0	tag v2.1.5
2.0.24	2014-09-12	1.1, 1.2	tag v2.0.24
1.0.13	2013-03-08	1.1, 1.2	tag v1.0.13
1.0.1	2011-10-11	1.0, 1.1	tag v1.0.1

Documentation

Usage

• Installation
• Configuration
• CORS
• OAuth2
• Deep Linking
• Limitations
• Version detection

Customization

• Overview
• Plugin API
• Custom layout

Development

• Setting up
• Scripts

Contributing

• Contributing

Integration Tests

You will need JDK of version 7 or higher as instructed here https://nightwatchjs.org/guide/getting-started/installation.html#install-selenium-server

Integration tests can be run locally with npm run e2e - be sure you aren't running a dev server when testing!

Browser support

Swagger UI works in the latest versions of Chrome, Safari, Firefox, and Edge.

Known Issues

To help with the migration, here are the currently known issues with 3.X. This list will update regularly, and will not include features that were not implemented in previous versions.

• Only part of the parameters previously supported are available.
• The JSON Form Editor is not implemented.
• Support for collectionFormat is partial.
• l10n (translations) is not implemented.
• Relative path support for external files is not implemented.

Security contact

Please disclose any security-related issues or vulnerabilities by emailing [email protected], instead of using the public issue tracker.

License

SwaggerUI is licensed under Apache 2.0 license. SwaggerUI comes with an explicit NOTICE file containing additional legal notices and information.

openespi-datacustodian-java's Issues

Support of Compressed XML

Is there any plan to support Compressed XML or other data format (http://www.w3.org/XML/EXI/#Efficient_XML) to support fast data transfer and efficient way to consume by client.

Base DataCustodian URI should be stored in ApplicationInformation structure not in Spring Security XML file

datacustodian base URl in token enhancer is injected though properties. Hence it needs to be provided in application during build time. Can this be used from application information corresponds to data-custodian.

Issue was originally reported in OpenESPI-Common-java, but only applies to OpenESPI-DataCustodian-java. Therefore it has been moved to here for tracking.

Administrative Interface to Control Configuration of the DataCustodian

In some applications, an administrative dashboard and interface are desirable for the DataCustodian.

POM File Update to Common 1.2-SNAPSHOT

the datacustodian pom.xml references the wrong common version

Support Internationalization of GUI

RetailCustomer Restful Interface (API)

Putting the Retail Customer API back into the API doc and validate in the implementation

Replace XML based configuration files with Java Configuration files

Spring Framework 4.0 replaced XML based configuration files with Java Configuration files. This enhancement brings the OpenESPI-DataCustodian-java repository up-to-date with the current Spring Framework configuration technology

Long /Id comparison

File: ResourceValidationFilter
== equal operator is used to compare two Long object. This will not use value to compare.

Support for revocation of authorizations

A way for either the third party or the data custodian to revoke an authorization.

(ref: RFC 7009)

Using the Registration_Access_Token to request /espi/1_1/resource/ApplicationInformation causes an ArrayIndexOutOfBounds Exception

If the registration_access_token is used when accessing the /espi/1_1/resource/ApplicationInformation API an ArrayIndexOutOfBounds exception is generated at line 394 of the ResourceValidationFilter class.

ManageRESTController does not create a valid "Content-Type" response header

The current API definition contains:

    @RequestMapping(value = Routes.DATA_CUSTODIAN_MANAGE, method = RequestMethod.GET, produces = "text/text")

The acceptable HTTP Content-type values for "text" are "text/plain" or "text/html" which the Swagger UI enforces. The current value of "text/text" should be "text/plain".

Remove support for the Postgres Database

The current OpenESPI-DataCustodian-java repository contains dependency and references to the Postgres Database, but this database interface has never been implemented.

Manage ApplicationInformation API by TP

1. Update/PUT --- there is no validation which attribute TP can update. Current merge implementation merge all attributes, so it may override existing data.
2. Delete -Can TP delete applicationinformation record. Does this mean delete all authorization?

Java Configuration method convert from XML File

Adapt the code base to use java configuration rather then XML data files now in use.

Support the Pattern: POST /espi/1_1/resource/Batch/[RetailCustomer | Subscription]/{subscriptionId}/UsagePoint/{usagePointId}/MeterReading/{meterReadingId}/IntervalBlock

to be able to POST an IntervalBlock

Upgrade to Spring Framework 4.0.6

Spring Framework 4.0.6 was released on July 8, 2014

Selecting "Notify Third Party" on the Admin Menu causes a NULL Exception

Delete Authorization API

On DELETE rest request to authorization API, I am getting below error.
[s~londonhydro-espi-dev/api:1.378271493630962243].: org.springframework.orm.jpa.JpaSystemException: org.hibernate.exception.ConstraintViolationException: could not execute statement; nested exception is javax.persistence.PersistenceException: org.hibernate.exception.ConstraintViolationException: could not execute statement

Also, could not delete authorization record using direct delete FROM datacustodian.authorizations where id=1000000027 as subscription is linked it.

Support for revocation of ThirdParty authorization

To be able to remove an applicationInformation and all of it's associated authorizations

(ref: RFC 7009)

Refresh Token Validation

Implement refresh token support in DC. A request for a fresh access-token or refresh-token should result in an update to the OAuth token store, as well as, the corresponding Authorization object.

Support Internationalization for decoding code lists in ESPI

Many elements of the ESPI data structures are code list enumerations where the data content is an integer that corresponds to a textual name and a description. Internationalization of this data facilitates agile adaptation of GUIs to local needs.

Retail Customer Management in Web

The RetailCustomerController.java (in the datacustodian.web.package) contains the patterns:

DATA_CUSTODIAN_RETAIL_CUSTOMER_INDEX
DATA_CUSTODIAN_RETAIL_CUSTOMER_FORM
DATA_CUSTODIAN_RETAIL_CUSTOMER_CREATE
DATA_CUSTODIAN_RETAIL_CUSTOMER_SHOW

These controllers provide the basis for RetailCustomer(Account) management.

multiple retailCustomers are assigned the same access_token

Start with a clean system. Sign in to the third party and authorize allan access to the first data custodian using the first scopestring.

sign out of the third party.
sign in to the third party as charles and authorize charles access to the first data custodian using the first scopestring.

Only one authorization/subscription is created and there is no suthorization that referes to charles' retailCustomerId.

Charles is not being assigned an authorization.access_token.

Document Authorization and Subscription Table Contents and Usage

There are fields in the authorizations and subscriptions table rows that contain NULL. These should be completed at time of row creation to have suitable contents and/or any residual NULL entries well documented as to why they are not being initialized.

Support Use Case 1 -- Dynamic Registration

Reference section 2.2 of the Authorization.docx https://s3-us-west-2.amazonaws.com/technical.greenbuttonalliance.org/OpenADE/Shared+Documents/Testing+and+Certification/GreenButtonTestPlan/ReferenceMaterial/GreenButtonAuthorization.docx

Absence of access_token ignored when logged in from web-browser

1. Go to https://services.greenbuttondata.org/DataCustodian
2. Login as grace
3. open a new browser tab (from the same browser used in steps 1&2)
4. enter any of the RESTful API patterns in the browser address line and hit return

You will be presented with the XML of the requested ESPI resource.

Rather I think I should get back:

Access Not Authorized access_denied

(NOTE: that no access token is contained in the browser request).

GET /DataCustodian/espi/1_1/resource/Batch/Subscription/5/UsagePoint/1 fails with 404

GET /DataCustodian/espi/1_1/resource/Batch/Subscription/{subscriptionId}/UsagePoint/{usagePointId} should return the complete UsagePoint structure along with contained objects according to the API. The current implementation returns a 404 not available error.

A GET /DataCustodian/espi/1_1/resource/Batch/Subscription/5 successfully returns subscription including UsagePoint 1.

Change background color from Pink to White

I recommend you change the background color (currently #fff1ff) to #ffffff.

The pink is tough. especially with the green. White seems safe, especially since most of the sites you link to from that page use white.

This appears to be set in the body and hero elements in bootstrap.css

ElectricPowerUsageSummary and ElectricPowerQualitySummary APIs obtain 400 Bad Request

ElectricPowerUsageSummary and ElectricPowerQualitySummary APIs obtain a 400 Bad Request when accessed using a client_credential based access token

GET ServiceStatus not returning correct info

Currently, GET http://localhost:8080/DataCustodian/espi/1_1/resource/ReadServiceStatus HTTP/1.1 returns:

    <feed xmlns="http://www.w3.org/2005/Atom" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
    <id>a45758af-dd6f-4b84-9373-a7f1050a0153</id>
    <title>Service Status</title>
    <description>Service Status:</description>
    <updated>2014-06-04T16:16:26Z</updated>
    </feed>

The ESPI standard describes only the data structure. To emit an instance of this as XML:

    <?xml version="1.0" encoding="UTF-8"?>
    <ServiceStatus xmlns="http://naesb.org/espi" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://naesb.org/espi espiDerived.xsd">
    <currentStatus>1</currentStatus>
    </ServiceStatus>

I think we should move to the Schema compliant version.

Implement Green Button HTML CSS

Design structure to facilitate skinning of application.

Authorization authorizationPeriod and publishedPeriod Support

The authorizationPeriod and publishedPeriod must:

1. be supported correctly in the authorization class -> they be initialized from values obtained from the scope string(s).
2. must also be used properly in the exportFilter class -> publishedPeriod must be used in conjunction with publishedMin and publishedMax (and all) within the export process
3. the authorization (and associated access_tokens) must be disallowed or destroyed when the authorizationPeriod expires (possibly using RFC 7009).

"Client_Credentials" based grant_types cause a stack exception

The EspiTokenEnhancer routine causes a ClassCastException when it invokes the createSubscription(authentication) method

JavaDoc Documentation

Drive the DataCustodian .war file documentation. Focus on:

1 - Controllers
2 - Restful API - (As defined in API documentation)
3 - UX flow

Remove deprecated Profiles

The POM file contains profiles that are no longer available. These should be removed to simplify the POM file.

REST APIs assume ALL access tokens have valid SubscriptionId values in their authorization record

The current REST APIs will NOT work for client_credentials based access tokens, as they assume ALL authorization records contain valid SubscriptionId values. This affects the following Web API REST controllers:

 IntervalBlockRESTController
 ElectricPowerQualitySummaryRESTController
 RetailCustomerRESTController
 MeterReadingRESTController
 ElectricPowerUsageSummaryRESTController
 UsagePointRESTController

Support Tomcat HTTP Compression

Using:

Query parameters in data subscription service

I am raising this for further discussion on this.

Currently, as part of specification usage data though data subscription service can be controlled by published-min/published-max and updated-min/updated-max. There will be some third party who is interested on Interval Block/Reading data based on usage time. Basically, they need data from date to this data irrespective when last correction happen or when it first published though green button channel (usage time always would be less than published time... could vary one hour to days to several days.)

Support SFTP

Support SFTP in DC for Bulk retrieval. If the dataCustodianBulkRequestURI in ApplicationInformation is set to sftp://..... the DC must support a retrieval of the bulk data via SFT GETALL method.

The DC will notify the TP when data is available using the sftp:// URL.

Reset of the Repository to a known state

Reset of the Repository to a known state. The suggested API is:

POST https://services.greenbuttondata.org/DataCustodian/manage?command=resetDataCustodianDB

A new issue specifically for this RESTful api has been entered.

Setting/Initialization of Authorization authorizationPeriod and publishedPeriod from Scope String

The initial values of the authorization period and published period need to be extracted from the scope string at the time of access authorization.

Cannot find symbol

Hello,

(Beginner here)

I am trying to follow the brief instructions on the readme.

1. I cloned OpenESPI-Common-java, built it and after skipping the tests, installed it
2. I cloned OpenESPI-DataCustodian-java and run "mvn tomcat7:run". This fails with:

[ERROR] COMPILATION ERROR : 
[INFO] -------------------------------------------------------------
[ERROR] /mnt2/images/openespi/OpenESPI-DataCustodian-java/src/main/java/org/energyos/espi/datacustodian/oauth/EspiTokenEnhancer.java:[56,56] error: cannot find symbol
[INFO] 1 error

There is no mention of the missing symbol, even when supplying -e -X to the mvn command as is suggested.

What am I missing?

Thanks,

-K.

Role

I see role in sql script ROLE_TP_REGISTRATION. But in resource validation it is used as ROLE_REGISTRATION

Multiple authorizations to same third-party for different usage point.

support of multiple authorizations to same third-party for different usage point. Currently, during second authorization to another usage points returns previous authorization details instead of creating new authorization. As the Oauth scope string same for both cases, spring ouath layer returning the existing valid authorization details.

Remove unused Database population resource files

The profiles for ec2 and heroku have been removed from the POM file, but the database population resource files in the /src/main/resources/db directory still exist.

ApplicationInformation Table Population

Populate all fields in the ApplicationInformation table reflecting the contents of GreenButtonAuthorization.docx Appendix D.1

ReadMe Instructions

The ReadMe instructions say the following: git clone https://github.com/energyos/OpenESPI-DataCustodian.git

This should be corrected to say "git clone https://github.com/energyos/OpenESPI-DataCustodian-Java.git".

Private Sandbox Capability

There is a need to have individual sandbox's supported

DELETE /espi/1_1/resource/ApplicationInformation/{applicationInformationId} returns a 400 when attempted using a registration_access_token

Attempting to access the DELETE /espi/1_1/resource/ApplicationInformation/{applicationInformationId} API using the registration_access_token returns a SC of 400.

GET /DataCustodian/espi/1_1/resource/Authorization doesn't filter owned authorizations

Run SOAPUI script Run TestCase EstablishTwoSubscriptions which adds two subscriptions for "third_party". There are now 6 authorizations -- 4 for the data custodian admin, third party admin, registration admin, upload admin, and two for the two authorizations.

The API says that the Third Party, using it's client_access_token, should be able to retrieve all authorizations for that TP. It currently returns all (6) authorizations. It should return only those related to the requesting ThirdParty (4 in this case -- the two subscriptions for alan and charles, and, the authorizations reflecting it's client-access-token and registration-access-token).