Comments (14)
I can see the usefulness of this story. The implementation may become complex though. As a suggestion for a minimal implementation: have a text string/keyword that should (not) be present in the response to determine success or failure.
Example 1: testing a WMS request using XML failure responses (using EXCEPTIONS=application/vnd.ogc.se_xml
). If the response contains the string <ServiceExceptionReport>
the request has failed.
Example 2: testing WFS GetFeature URL: response should contain the keyword FeatureCollection
.
The implementation should not be too complex and would cover many use cases. Many URL monitoring services like http://uptimerobot.com/ have this generic feature. A step further is to associate these URL resource type requests to a particular OGC: resource instance within GHC.
from geohealthcheck.
This would be best applied to WWW:LINK
resources given the other types have built in checks which are based on the given OGC standards.
Options for an initial WWW:LINK
validation:
- standard HTTP error code checking is already performed as part of the existing checks
- add an
in_text
field (default=None
) to theRun
Resource
model which represents text that must exist in the response to constitute a pass - field is optionally populated in the 'Add Resource' form when
WWW:LINK
is selected - when a
Run
is executed, ifin_text
is not none then a simple'foo' in result_text
type check is done
Thoughts?
from geohealthcheck.
Yes, my proposal was for WWW:LINK
resources. The other, mainly OGC:
-types will fetch and parse a GetCapabilities
-response IFAICT. What I am seeking are health-checks beyond GeoCapabilities
, sometimes that response may even come from a static file. OGC Services can fail for many reasons, usually one notices on Get
-requests (WMS GetMap
, WFS GetFeature
etc) that the service is "unhealthy" without hard failures e.g. a blank image. I realize that generic auto-generated Get*
requests are tricky to implement, especially for WMS, maybe interactively with a map view.
Hence, having a WWW:LINK
Resource check for a keyword (in_text
) in the response text is a good first step towards more detailed health checks, so yes I agree with your proposal. My proposal above also had the reverse test: that the response should not contain the keyword like ServiceException
. This is very useful as a generic check for OGC Get*
responses, most notably, ServiceException
on a WMS GetMap
. Later we can add a "parent"-option to WWW:LINK
, such that we can couple this to a parent OGC:
service and have grouping.
As for implementation: I guess you meant the Resource
Model not the Run
Model? Adding a field/column in_text
to Resource
that would only apply to WWW:LINK
would be a bit 'specific'. Like in our discussion on Metrics
in #43, my suggestion is to add a generic field/column checklist
or test_details
as a JSON-docstring to Resource
. At a later stage we may extend/fill this with OGC-specific Get*
-checks. If we give each check-type a specific typeid
, this may be a start for a plugin-system for specific checks! Examples:
WWW:LINK
with WMS request:
checklist: [
{
type: 'hascontenttype'
properties: {
contenttype: `image/jpeg`
}
},
{
type: 'keywordnotexists'
properties: {
keyword: `ServiceException>`
}
}
]
WFS-specific (url
is e.g. WFS GetFeature
):
checklist: [
{
type: 'hascontenttype'
properties: {
contenttype: `text/xml; subtype=gml/2.1.2`
}
},
{
type: 'keywordexists'
properties: {
keyword: `FeatureCollection>`
}
}
]
Many more checks can be thought of: minimum filesize (prevent blank images), featurecount, etc For OGC specific XML-based services this would entain response parsing according to XML-schema's via OWSLib
, so less a need for regexes etc. But we can start simple with the WWW:LINK
keyword check using the checklist
-method.
Does this make sense? Still a bit new to GHC...
from geohealthcheck.
@justb4 thanks for the explanation, you're right I meant adding in_text
to Resource
, not Run
. Let's do this as the initial iteration?
Having said this, I'm thinking we can design a plugin system which allows users to:
- define/install 3rd party plugins to do specific healthchecks against the venerable
WWW:LINK
resource type - define/install 3rd party plugins to override existing healthchecks with specific tests
Putting things in plugins allows more flexibility without creating a meta-language/configuration. Thoughts?
from geohealthcheck.
An extensible plugin-system together with standard plugins for healthchecks would be valuable.
I am not sure what you mean by "without creating a meta-language/configuration". One of the reasons I was proposing a generic checklist config language has to do with the UI: a user would be able to select specific checks from available plugin-meta info and possibly parameterize these.
Thinking this a bit further and staying close to what I think was GHC's initial aim: to check the health of mainly OGC-services: I think having a "checklist" against any OGC-endpoint would be very valuable. Putting specific checks against WWW:LINKs without a reference to their (OGC) endpoint may be a bit diverting from that goal. Currently the default/minimum "checklist" for an OGC-endpoint is to fetch and parse its Capabilities
. But each OGC-service could have an additional list of specific checks. Take a WMS endpoint, via the GUI we may want to select/add checks against GetMap
for specific Layers and Bounding boxes for ServiceExceptions
.
In the end the Resources itself would be simple WWW:LINKs/checks but coupled to their parent OGC-endpoint (grouping, see #18). The UI would be a main challenge with interactive maps etc, but we could start simple: provide specific URLs for an OGC-endpoint and select checks against them.
from geohealthcheck.
Pending a re-architecture, as to support an extensive checking system (via plugins), at least our GHC instances would be very much helped with a check on any Exception
within a WWW:LINK response as we configure WWW:LINK Resources to access OGC-Layers. A simple check could be something like below on https://github.com/geopython/GeoHealthCheck/blob/master/GeoHealthCheck/healthcheck.py#L82 :
if resource_type == 'WWW:LINK':
content_type = ows.info().getheader('Content-Type')
# Check content if the response is not an image
if 'image/' not in content_type:
content = ows.read()
import re
try:
title_re = re.compile("<title>(.+?)</title>")
title = title_re.search(content).group(1)
except:
title = url
# Check for any OGC-Exceptions in response
exception = None
try:
except_re = re.compile("ServiceException>|ExceptionReport>")
exception = except_re.search(content).group(0)
except:
pass
del content
if exception:
raise Exception("Exception in response: %s" % exception)
We could even add a config setting to enable the Exception check WWW_LINK_EXCEPTION_CHECK
. I can make the above change, possibly via a new issue as this would not completely solve the initial user-story.
from geohealthcheck.
@tomkralidis should I make the above change with WWW_LINK_EXCEPTION_CHECK
default False
? For many installations, at least ours, this will be very useful as currently OGC-exceptions remain uncaught (200) for WWW-LINKs
for OGC services.
from geohealthcheck.
@justb4 sorry for the delay. When would an OGC service be registered as a WWW:LINK
? We use OWSLib for interacting with OGC services, which properly throws exception in Python that we can catch accordingly to assess a given Run.
from geohealthcheck.
@tomkralidis in our setup we register individual OGC requests like GetMap and GetFeature as WWW:LINKs. Currently many error situations are not caught. The OGC: Resources are only checked for Capabilities responses. Those can never catch all individual request-errors. Just like WWW:LINK responses are checked for a <title>, my idea is to check for any <*Exception> for none-image responses. Not ideal but for time being will catch many errors where the Caps and even WWW:LINK give 200 are ok (see presentation).
Another option is to introduce an OGC:LINK, as individual requests (GetMap, GetFeature, GetObservation, etc) and always check for Exceptions.
Yet, another is to allow single keyword yes/no match as many uptime checkers support. Problem is that that requires a DB mod.
from geohealthcheck.
Perhaps we can keep WWW:LINK
as is and introduce an in_text
field for a string to catch to throw exception? Yes, it's a DB mod/addition which can have a default of None
to preserve backwards compatibility. Thoughts or would something break?
from geohealthcheck.
Not clear to me how to silently upgrade the DB, if SQLAlchemy has some mechanism. Also in_text
is usually tagged with yes/no
whether that text should (not) be in response, e.g. Exception
with no
. For example for WMS GetMap
(or WCS GetCoverage
with TIFF expected) we expect an image but may receive an Exception string in response. With just in_text
we cannot check that condition unless we encode the condition in the column (Exception=no|yes
) but that gets somewhat hacky.
from geohealthcheck.
ah, ok. Let's go with your proposal in #19 (comment) as first pass and iterate from there?
from geohealthcheck.
We now have a generic Exception-check for WWW:LINK resources. I would like to close this issue and resume in the new issue #82 to implement not just alternative responses but generic OGC-specific checks.
from geohealthcheck.
+1
from geohealthcheck.
Related Issues (20)
- Upgrade Docker image to newer version of Python HOT 1
- update minimum jquery version
- test mailserver configuration HOT 2
- Need new Mapbox Tilejson Unit tests
- Error when adding or testing Resource in web UI on demo site HOT 1
- Support for ESRI ArcGIS Server Mapservice HOT 2
- Make 1.3.0 default WMS version
- better indication while testing
- error report drilldown-all-layers wms should indicate which layer is failing
- Slow performance in frontend
- POST requests fail with resources requiring authentication HOT 1
- Python error loading data
- Better formatting for dates HOT 2
- Filter when clicking on "operational" and "broken"
- Show more details for probes
- Show the name of the chosen tag along with "dashboard"
- Show full resport on two columns to avoid having to vertical scroll the dialog
- Doc updates for new ReadTheDocs requirements HOT 1
- Unit test failures HOT 1
- default value for webhook URL
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from geohealthcheck.