ii / apisnoop Goto Github PK

When the sunburst visualisation is filtered to show only endpoints accessed by a particular app (besides the E2E test suite) it would be valuable to be able to see which of the accessed endpoints have been tested and which of those haven't.

Combined with volume of hits data this information would provide useful insights as to which endpoints are priorities for conformance testing based on their status and how they are being used.

To that end we need to introduce a selector that allows the sunburst visualisation to be updated based on whether the user wishes to see All endpoints accessed by a particular app, only the Tested endpoints or only the Untested endpoints.

This can be achieved through the addition of a radio button as below:

The existing APISnoop implementation illustrates the endpoints accessed by a single app (the e2e conformance tests), however it is possible to capture audit information for all apps that are hitting endpoints and as such we can load this data and then allow it to be filtered by user agent.

Introducing this filter will allow for more in depth data interrogation and provide valuable information to app developers about which endpoints they are hitting and their conformance status (in conjunction with additional visualisation features).

The functionality of the selector is illustrated below:

/woof

The current sunburst visualisation only shows (in colour) the endpoints that are 'hit' by an app or the E2E test suite with the remaining (non-hit) endpoints shown as solid grey blocks.

In order to allow us to filter and manipulate all of the endpoint data, each individual endpoint should be represented within the sunburst by an individual block/line that can be interacted with (and colour coded by category or other filters chosen by the user).

An example of how this could be visualised is below:

To allow others to contribute their audit data to APISnoop, we need to provide an ability to upload audit-logs for analysis. These could then be loaded into the sunburst visualization, or other APISnoop tools allowing for a deeper dive into their own data data.

It might also be possible, once test-infra jobs support storing audit.logs, to use gs://bucket/path/to/audit.log urls as as source.

To allow this I propose a drag/drop if the user wishes to upload a local file or and alternative to input a URL/Link to the data to be consumed by the APISnoop service, as illustrated below:

Our sunburst does not currently allow for the endpoint data to be filtered in any way which can make interrogating the data a challenge when dealing with over 900 potential data points. A first step to helping users focus in on areas of interest is to allow them to select which high level category of endpoints (stable, beta, alpha) they'd like to view on the sunburst.

To enable this functionality the top level categories will be shown as colour coded checkboxes underneath the sunburst allowing the user to select/unselect which ones appear on the visualisation. If an item is unchecked that endpoint data is removed from the sunburst and the rings re-displayed accordingly i.e. if alpha is unchecked then we remove this data and the inner ring will only show stable and beta with the outer rings updated accordingly.

There must be one checkbox selected at all times to ensure the sunburst has data to present.

The current sunburst visualisation does not provide any indication of the volume of hits an endpoint receives meaning there is no easy way to prioritise or make decisions about where effort needs to be spent.

Adding a volume indicator to the sunburst allows us to identify which endpoints are receiving the most hits from the test suite (and apps in the future) and will allow prioritisation of effort, in particular if an app is hitting non-tested/confirmant endpoints.

Further options once volume data can be charted could include filtering and/or sorting the sunburst by the volume information (a future consideration).

An approach for the volume bars is shown below:

Each line on the outer ring of the sunburst visualisation represents an endpoint and when you rollover each available one with your mouse we are showing the name in a box at the top left. However, the name/label box is fixed in place meaning there is a disconnect between the endpoints being rolled over and the information being shown.

In order to make it easier for the user to interrogate each endpoint on the outer ring, when they rollover them a label/name will appear connected to each in closer proximity as per the illustration below.

We collect a range of data within the audit logs that can provide further, detailed information about each of the endpoints i.e. what tests are accessing them, who the owners are and what SIGs they belong to.

In addition for the E2E tests we can present more detailed information about each of the tests that are running against the endpoint and their purpose.

All of this information can be presented back to the user when they click on an endpoint from within the sunburst visualisation and to do this we would present an overlay that loads as required. Depending on the available data we can structure the screen in a number of ways - which is being explored.

Illustration below: