Server Error (403: Forbidden) about siphon

[dopplershift]

Can you confirm with NCAR's HPC support that this is even expected to work? Many HPC environments disable access to external networks.

[dopplershift]

Can you test and see if this works?

import urllib.request

resp = urllib.request.urlopen('http://weather.uwyo.edu/cgi-bin/sounding?region=naconf&TYPE=TEXT%3ALIST&YEAR=2022&MONTH=08&FROM=1100&TO=1100&STNM=72632')
print(resp.read())

[rvalent]

Thank you, dopplershift -- your suggestion works! I am very grateful.

If possible, I would like to see NCAR JupyterHub be able to handle such data transfers by default.

Would this entail changes on the Unidata side or on the NCAR/UCAR side?

[dopplershift]

@rvalent Wait, are you saying that sample code worked on Casper? Because I was expecting it to fail given your previous problem.

[rvalent]

@dopplershift You are correct! I just checked it now from the JH window. That sample code fails in the same way:

Traceback (most recent call last):
File "", line 3, in
File "/glade/work/valent/conda-envs/casper-rozoff-siphon/lib/python3.7/urllib/request.py", line 222, in urlopen
return opener.open(url, data, timeout)
File "/glade/work/valent/conda-envs/casper-rozoff-siphon/lib/python3.7/urllib/request.py", line 531, in open
response = meth(req, response)
File "/glade/work/valent/conda-envs/casper-rozoff-siphon/lib/python3.7/urllib/request.py", line 641, in http_response
'http', request, response, code, msg, hdrs)
File "/glade/work/valent/conda-envs/casper-rozoff-siphon/lib/python3.7/urllib/request.py", line 569, in error
return self._call_chain(*args)
File "/glade/work/valent/conda-envs/casper-rozoff-siphon/lib/python3.7/urllib/request.py", line 503, in _call_chain
result = func(*args)
File "/glade/work/valent/conda-envs/casper-rozoff-siphon/lib/python3.7/urllib/request.py", line 649, in http_error_default
raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 403: Forbidden

[dopplershift]

@rvalent Ok, well that sample code will work on any machine with a non-impeded network configuration--and involves Siphon in no way.

In other words, the only way this works is if NCAR tweaks something, but I want to be clear: it is very common, if not expected, that you do not have arbitrary network access in an HPC environment.

Either way, there's no problem with Siphon here and there's nothing we can do to solve this on our end.

[rvalent]

@dopplershift Thank you. This is very helpful.

[jbaksta]

The network on this cluster is not bound by any outgoing rules. It's more closer architected to a cloud like element. I believe it might be a data server issue. running this several times on various machines, I would potentially get a 403 on Casper systems or sometimes a too many request or system is too busy. This suggests a rate limiter impoementation on the server side that may have blocked our system at some point (forever because of lots of requests). Hard to say without getting in touch with the data server administrator.

[dopplershift]

@jbaksta That's interesting. I regularly get a 503 from the Wyoming server when they're overloaded, but I have never experienced a 403.