Coder Social home page Coder Social logo

stl-test's People

Contributors

adam-iris avatar

Watchers

avatar avatar

stl-test's Issues

Look into relaunching webinars topic (RM#5842)

ported from Redmine # 5842 created 2017-03-27T20:28:27Z

Proposal would be to make this more general (for non-IRIS webinars as
well as IRIS webinars), relaunch in a different area, and explicitly
fork this from the General Announcements list so it would start with
that broader subscriber base.

Software page QuerySets are broken (RM#5850)

ported from Redmine #5850 created 2017-04-04T23:29:17Z CLOSED 2017-04-10T23:26:38Z

There are two oddities here:

  • The QuerySets are defined in urls.py
  • Various views set self.queryset during handling of a request

I suspect the second is causing the pages to show stale data -- the view sets self.queryset to a "resolved" QuerySet, then on subsequent requests this QuerySet gets reused and never actually runs again.

Add feature to exclude (or perhaps show?) 'orphaned' data (RM#5851)

ported from Redmine # 5851 created 2017-04-04T23:40:10Z

Orphaned data is data in the archive where there is no metadata that
covers it.

This can result when metadata is removed or epochs changed since there's
no direct coupling
after data makes it's way into archive. Normally, at the time of
archiving, this data would
find it's way into purgatory. So orphaned data really should be in
purgatory.

Would be nice if the availability service knew about this.

Perhaps: excludeorphaned=<true|false> or something like
that.

Software page QuerySets are broken (RM#5850)

ported from Redmine #5850 created 2017-04-04T23:29:17Z CLOSED 2017-04-10T23:26:38Z

There are two oddities here:

  • The QuerySets are defined in urls.py
  • Various views set self.queryset during handling of a request

I suspect the second is causing the pages to show stale data -- the view sets self.queryset to a "resolved" QuerySet, then on subsequent requests this QuerySet gets reused and never actually runs again.

Map zoom (RM#5829)

ported from Redmine # 5829 created 2017-03-21T18:27:39Z

Allow user to zoom into map

Event table wraps awkwardly (RM#5817)

ported from Redmine # 5817 created 2017-03-20T17:31:28Z

Some lines in the event table wrap, it looks like the table will try to wrap in order to avoid horizontal scrolling, unless the last column actually begins past the right border.

response code of 404 should cause IOException and not NoDataFoundException in WaveformService.java

Created 2017-03-30T17:07:14Z by Bruce Weertman

I ran into this problem when using the WaveformService.stream() method. I used the constructor method:


	public WaveformService(String baseUrl, String version,
			String compatabilityVersion, String userAgent) {
		super(version, compatabilityVersion, userAgent);
		this.baseUrl = baseUrl;
	}

For the URL I used http://service/fdsnws/dataselect/1
This was wrong as it should have been: http://service/fdsnws/dataselect/1/

When calling waveformService.stream(fout, criteria); a NoDataFoundException is raised.
This happens at case 404 in the switch block of WaveformService.java:


			switch (responseCode) {
			case 204:
				if (logger.isLoggable(WARNING))
					logger.warning("No data Found for: " + query);
				throw new NoDataFoundException("No data found for: " + query);
			case 404:
				throw new NoDataFoundException("No data found for: " + query);
			case 401:
				throw new UnauthorizedAccessException("Unauthorized access: "
						+ query);
			case 400:
				throw new CriteriaException("Bad request parameter: "
						+ StringUtil.toString(inputStream));
			case 500:
				if (logger.isLoggable(WARNING))
					logger.warning("An error occurred while making a GET request "
							+ StringUtil.toString(inputStream));
				throw new IOException(StringUtil.toString(inputStream));
              .....

The services were designed to allow us to detect this type of error (incorrect URL).

The same is probably true of StationService etc ??

Moderator remove message from thread

Created 2017-04-03T17:54:44Z by Adam Clark

We've had a few cases where a user posts a message by replying to an earlier message, but it's not intended as a reply. (Presumably they used "reply" just to get the email address of the list.) For these cases, it would be useful for the moderator to be able to remove the message from the earlier thread before approving it. This should create a new thread in Django, and clear out the @In-Reply-To@ and @references@ headers in the email.

response code of 404 should cause IOException and not NoDataFoundException in WaveformService.java (RM#5846)

ported from Redmine # 5846 created 2017-03-30T17:07:14Z

I ran into this problem when using the WaveformService.stream() method.
I used the constructor method:

<code>
    public WaveformService(String baseUrl, String version,
            String compatabilityVersion, String userAgent) {
        super(version, compatabilityVersion, userAgent);
        this.baseUrl = baseUrl;
    }
</code>

For the URL I used http://service/fdsnws/dataselect/1\
This was wrong as it should have been:
http://service/fdsnws/dataselect/1/

When calling waveformService.stream(fout, criteria); a
NoDataFoundException is raised.
This happens at case 404 in the switch block of
WaveformService.java:

<code>
            switch (responseCode) {
            case 204:
                if (logger.isLoggable(WARNING))
                    logger.warning("No data Found for: " + query);
                throw new NoDataFoundException("No data found for: " + query);
            case 404:
                throw new NoDataFoundException("No data found for: " + query);
            case 401:
                throw new UnauthorizedAccessException("Unauthorized access: "
                        + query);
            case 400:
                throw new CriteriaException("Bad request parameter: "
                        + StringUtil.toString(inputStream));
            case 500:
                if (logger.isLoggable(WARNING))
                    logger.warning("An error occurred while making a GET request "
                            + StringUtil.toString(inputStream));
                throw new IOException(StringUtil.toString(inputStream));
              .....
</code>

The services were designed to allow us to detect this type of error
(incorrect URL).

The same is probably true of StationService etc ??

Feature: Handle 301 (redirect) in StationService fetch? (RM#5845)

ported from Redmine # 5845 created 2017-03-29T18:50:15Z

In the iterate() method could we also handle the 301 HTTP response code
case? In particular could we "try again" using the returned
redirect-URL?

We noticed this in the Federator (Harvester). One of the data centers is
now redirecting to their HTTPS version (secure). Currently the Library's
iterate() sees this and returns an IOException.

REPRO:\

  1. With a Library test app, try to fetch using the SCEDC's station
    service http://service.scedc.caltech.edu/fdsnws/station/1/ and set
    the output level to channel.\

  2. It will return with 301 and the Library throws ...

    java.io.IOException: Moved Permanently

3. By using a browser you can see that the redirect is to the secure
version (https://service.scedc.caltech.edu/fdsnws/station/1/).

Thank you!

Migrate to GitHub

Created 2017-04-10T21:06:41Z by Adam Clark

Migrate issues and history if possible

Add feature to exclude (or perhaps show?) 'orphaned' data

Created 2017-04-04T23:40:10Z by Bruce Weertman

Orphaned data is data in the archive where there is no metadata that covers it.

This can result when metadata is removed or epochs changed since there's no direct coupling
after data makes it's way into archive. Normally, at the time of archiving, this data would
find it's way into purgatory. So orphaned data really should be in purgatory.

Would be nice if the availability service knew about this.

Perhaps: excludeorphaned=<true|false> or something like that.

YD VENO 01 BHZ not fully scanned from tsindex table ??

Created 2017-04-05T21:51:07Z by Bruce Weertman

This works:

http://service.iris.edu/fdsnws/dataselect/1/query?network=YD&station=VENO&location=01&starttime=2005-02-22T02:29:08.773288&endtime=2005-02-22T02:39:08.773288&channel=BHZ

This doesn't

http://service.iris.edu/iriswsbeta/availability/1/query?start=2005-02-22T02:29:08&end=2005-02-22T02:39:08&net=YD&sta=VENO&loc=01&cha=BHZ&mergequality=true&mergesamplerate=true&mergeoverlap=true&nodata=404

Looking at extents for this I see:

http://service.iris.edu/iriswsbeta/availability/1/extent?net=YD&sta=VENO&loc=01&cha=BHZ&nodata=404

n  s     l  c   q sample-rate                    earliest                      latest               updated time-spans
YD VENO  01 BHZ D        50.0 2004-02-24T14:14:03.400000Z 2004-09-03T12:01:23.380000Z  2006-12-26T22:40:13Z         10
YD VENO  01 BHZ R        50.0 2004-09-03T12:38:57.400000Z 2005-02-08T13:02:03.859000Z  2006-12-28T22:56:28Z         13

However, looking at tsindex, it looks like there should be more ???

select network, station, location, channel, quality, samplerate, timespans, filename,
to_char(starttime at time zone 'GMT', 'YYYY-MM-DD HH24:MI:SS TZ'), 
to_char(endtime at time zone 'GMT', 'YYYY-MM-DD HH24:MI:SS TZ'), 
to_char(updated at time zone 'GMT', 'YYYY-MM-DD HH24:MI:SS TZ') from tsindex where 
network='YD' and station='VENO' and location='01' and channel='BHZ'  order by starttime

The last end time is 2005-03-06. Why does the availability service stop at 2005-02-08T13:02:03.859000Z

Moderator remove message from thread (RM#5848)

ported from Redmine # 5848 created 2017-04-03T17:54:44Z

We've had a few cases where a user posts a message by replying to an
earlier message, but it's not intended as a reply. (Presumably they used
"reply" just to get the email address of the list.) For these cases, it
would be useful for the moderator to be able to remove the message from
the earlier thread before approving it. This should create a new thread
in Django, and clear out the In-Reply-To and References headers in
the email.

Look into relaunching webinars topic

Created 2017-03-27T20:28:27Z by Adam Clark

Proposal would be to make this more general (for non-IRIS webinars as well as IRIS webinars), relaunch in a different area, and explicitly fork this from the General Announcements list so it would start with that broader subscriber base.

YD VENO 01 BHZ not fully scanned from tsindex table ?? (RM#5852)

ported from Redmine # 5852 created 2017-04-05T21:51:07Z

This works:

http://service.iris.edu/fdsnws/dataselect/1/query?network=YD&station=VENO&location=01&starttime=2005-02-22T02:29:08.773288&endtime=2005-02-22T02:39:08.773288&channel=BHZ

This doesn't

http://service.iris.edu/iriswsbeta/availability/1/query?start=2005-02-22T02:29:08&end=2005-02-22T02:39:08&net=YD&sta=VENO&loc=01&cha=BHZ&mergequality=true&mergesamplerate=true&mergeoverlap=true&nodata=404

Looking at extents for this I see:

http://service.iris.edu/iriswsbeta/availability/1/extent?net=YD&sta=VENO&loc=01&cha=BHZ&nodata=404

n  s     l  c   q sample-rate                    earliest                      latest               updated time-spans
YD VENO  01 BHZ D        50.0 2004-02-24T14:14:03.400000Z 2004-09-03T12:01:23.380000Z  2006-12-26T22:40:13Z         10
YD VENO  01 BHZ R        50.0 2004-09-03T12:38:57.400000Z 2005-02-08T13:02:03.859000Z  2006-12-28T22:56:28Z         13

However, looking at tsindex, it looks like there should be more ???

select network, station, location, channel, quality, samplerate, timespans, filename,
to_char(starttime at time zone 'GMT', 'YYYY-MM-DD HH24:MI:SS TZ'), 
to_char(endtime at time zone 'GMT', 'YYYY-MM-DD HH24:MI:SS TZ'), 
to_char(updated at time zone 'GMT', 'YYYY-MM-DD HH24:MI:SS TZ') from tsindex where 
network='YD' and station='VENO' and location='01' and channel='BHZ'  order by starttime

The last end time is 2005-03-06. Why does the availability service stop
at 2005-02-08T13:02:03.859000Z

Moderator remove message from thread (RM#5848)

ported from Redmine # 5848 created 2017-04-03T17:54:44Z

We've had a few cases where a user posts a message by replying to an earlier message, but it's not intended as a reply. (Presumably they used "reply" just to get the email address of the list.) For these cases, it would be useful for the moderator to be able to remove the message from the earlier thread before approving it. This should create a new thread in Django, and clear out the In-Reply-To and References headers in the email.

Station-level iteration: Many DC's omit the Endtime causing IO Exception (RM#5849)

ported from Redmine # 5849 created 2017-04-03T22:18:45Z

Not sure if this is our bad, or a mistake on the part of other Data
centers? Many of them (NOT IRIS) omit the endtime in their
station-level response. I guess it implies "no end". At IRIS we of
course use the "2599-" for the year indicating no end.

So not sure this is our problem - in anycase when the enddate is not
present, the Station iterator throws an IO Exception. I will note that
the FDSN Service Spec does’t explicitly say the response must include
the endtime, though it is set in each & every example in that document.
Many examples have it set to the default date in perpetude (2599- etc.)
FDSN Service Spec
http://www.fdsn.org/webservices/FDSN-WS-Specifications-1.1.pdf

I’m seeing this error in 9 Data centers at the Station level. These data
centers behave just fine at the Channel level:

ORFEUS, USPSC, BGR, IPGP, KOERI, NIEP, NOA, LMU, GEOFON

Here is an example IOE (from USPSC) while harvesting at the Station
level. In my criteria, I ask for XML if available:

USPSC: Reading and processing data.
StationTextIteratorParser: Unable to iterate lines, printing stack trace
java.io.IOException: Invalid format: [BL|RCLB|-22.4191|-47.531|650.0|Rio Claro, SP|2002-11-09T00:00:00|]
    at edu.iris.dmc.service.station.parser.StationTextIteratorParser.next(StationTextIteratorParser.java:41)
    at edu.iris.dmc.service.station.parser.StationTextIteratorParser.next(StationTextIteratorParser.java:15)
    at edu.iris.dmc.service.StationIterator.next(StationIterator.java:39)
    at edu.iris.federator.fillHoldings.FillStationData.main(FillStationData.java:149)

And just an example using your browser (BGR):
http://eida.bgr.de/fdsnws/station/1/query?net=\*&level=station&format=text

Migrate to GitHub

Created 2017-04-10T21:06:41Z by Adam Clark

Migrate issues and history if possible

Software page QuerySets are broken (RM#5850)

ported from Redmine # 5850 created 2017-04-04T23:29:17Z

There are two oddities here:

  • The QuerySets are defined in urls.py
  • Various views set self.queryset during handling of a request

I suspect the second is causing the pages to show stale data -- the view
sets self.queryset to a "resolved" QuerySet, then on subsequent
requests this QuerySet gets reused and never actually runs again.

negative magnitudes (RM#5788)

ported from Redmine # 5788 created 2017-03-16T22:16:32Z

The fdsnws-event service supports negative earthquake magnitudes, so PyWeed should also be able to handle them. Currently, the minimum magnitude "field" can support a dash "-" character but, due to character limits or other issues, it will not allow other characters to follow the dash. I suppose this support could be extended to the maximum magnitude field as well.

scrolling through date and time selections (RM#5790)

ported from Redmine # 5790 created 2017-03-17T00:12:11Z

The Date and Hour/Minute/Second entries in the starttime and endtime fields currently allow for "scrolling" between a maximum and minimum number using the keypad. It would be ideal, but not critical, if these numbers looped over back to the minimum once the maximum was reached. So, for seconds, instead of getting stuck at 59, it would scroll from 57, 58, 59, 00, 01, and so on.

Look into relaunching webinars topic (RM#5842)

ported from Redmine # 5842 created 2017-03-27T20:28:27Z

Proposal would be to make this more general (for non-IRIS webinars as well as IRIS webinars), relaunch in a different area, and explicitly fork this from the General Announcements list so it would start with that broader subscriber base.

Better progress indicator (RM#5825)

ported from Redmine # 5825 created 2017-03-21T17:14:28Z

Need a better way to signal to the user that event/station retrieval is in progress.

Software page QuerySets are broken

Created 2017-04-04T23:29:17Z by Adam Clark

There are two oddities here:

  • The QuerySets are defined in urls.py
  • Various views set @self.queryset@ during handling of a request

I suspect the second is causing the pages to show stale data -- the view sets @self.queryset@ to a "resolved" QuerySet, then on subsequent requests this QuerySet gets reused and never actually runs again.

Feature: Handle 301 (redirect) in StationService fetch?

Created 2017-03-29T18:50:15Z by Mick Van Fossen

In the iterate() method could we also handle the 301 HTTP response code case? In particular could we "try again" using the returned redirect-URL?

We noticed this in the Federator (Harvester). One of the data centers is now redirecting to their HTTPS version (secure). Currently the Library's iterate() sees this and returns an IOException.

REPRO:

  1. With a Library test app, try to fetch using the SCEDC's station service http://service.scedc.caltech.edu/fdsnws/station/1/ and set the output level to channel.
  2. It will return with 301 and the Library throws ...
java.io.IOException: Moved Permanently
  1. By using a browser you can see that the redirect is to the secure version (https://service.scedc.caltech.edu/fdsnws/station/1/).

Thank you!

Software page QuerySets are broken (RM#5850)

ported from Redmine # 5850 created 2017-04-04T23:29:17Z

There are two oddities here:

  • The QuerySets are defined in urls.py
  • Various views set self.queryset during handling of a request

I suspect the second is causing the pages to show stale data -- the view sets self.queryset to a "resolved" QuerySet, then on subsequent requests this QuerySet gets reused and never actually runs again.

Generate subscription numbers (RM#5843)

ported from Redmine # 5843 created 2017-03-27T20:29:36Z

Right now this is for Tim, but it might be useful as a regular report. For each list, show the owners and # of subscribers.

Station-level iteration: Many DC's omit the Endtime causing IO Exception

Created 2017-04-03T22:18:45Z by Mick Van Fossen

Not sure if this is our bad, or a mistake on the part of other Data centers? Many of them (NOT IRIS) omit the endtime in their station-level response. I guess it implies "no end". At IRIS we of course use the "2599-" for the year indicating no end.

So not sure this is our problem - in anycase when the enddate is not present, the Station iterator throws an IO Exception. I will note that the FDSN Service Spec does’t explicitly say the response must include the endtime, though it is set in each & every example in that document. Many examples have it set to the default date in perpetude (2599- etc.)
FDSN Service Spec
http://www.fdsn.org/webservices/FDSN-WS-Specifications-1.1.pdf

I’m seeing this error in 9 Data centers at the Station level. These data centers behave just fine at the Channel level:

ORFEUS, USPSC, BGR, IPGP, KOERI, NIEP, NOA, LMU, GEOFON

Here is an example IOE (from USPSC) while harvesting at the Station level. In my criteria, I ask for XML if available:

USPSC: Reading and processing data.
StationTextIteratorParser: Unable to iterate lines, printing stack trace
java.io.IOException: Invalid format: [BL|RCLB|-22.4191|-47.531|650.0|Rio Claro, SP|2002-11-09T00:00:00|]
	at edu.iris.dmc.service.station.parser.StationTextIteratorParser.next(StationTextIteratorParser.java:41)
	at edu.iris.dmc.service.station.parser.StationTextIteratorParser.next(StationTextIteratorParser.java:15)
	at edu.iris.dmc.service.StationIterator.next(StationIterator.java:39)
	at edu.iris.federator.fillHoldings.FillStationData.main(FillStationData.java:149)

And just an example using your browser (BGR):
http://eida.bgr.de/fdsnws/station/1/query?net=*&level=station&format=text

allow milliseconds for starttime, endtime, and display time (RM#5789)

ported from Redmine # 5789 created 2017-03-16T22:45:06Z

The "Time" listed for earthquakes could support milliseconds to ensure the most accurate information is presented to the user. The event services also support milliseconds, so it could be useful for specifying in the start/end time search windows.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.