ns1 / ns1-go Goto Github PK
View Code? Open in Web Editor NEWGolang API client for NS1
License: Apache License 2.0
Golang API client for NS1
License: Apache License 2.0
I'm running into an issue unmarshalling a CAA record for my domain. I think the interface is expecting an array of all strings in the JSON response, but the response has an array of mixed types, the first being an integer and followed by strings.
This is the script used:
package main
import (
"fmt"
"log"
"net/http"
"os"
"time"
api "gopkg.in/ns1/ns1-go.v2/rest"
)
func main() {
k := os.Getenv("NS1_APIKEY")
if k == "" {
fmt.Println("NS1_APIKEY environment variable is not set, giving up")
os.Exit(1)
}
httpClient := &http.Client{Timeout: time.Second * 10}
doer := api.Decorate(httpClient, api.Logging(log.New(os.Stdout, "", log.LstdFlags)))
client := api.NewClient(doer, api.SetAPIKey(k))
records, _, err := client.Records.Get("siwyd.com", "siwyd.com", "CAA")
if err != nil {
log.Fatal(err)
}
for _, a := range records.Answers {
fmt.Println(a)
}
}
Output:
2017/09/26 13:38:13 go-ns1/2.0.0: GET https://api.nsone.net/v1/zones/siwyd.com/siwyd.com/CAA
2017/09/26 13:38:14 json: cannot unmarshal number into Go struct field Answer.answer of type string
exit status 1
If I change "CAA" to "MX", I get the proper output:
2017/09/26 13:37:56 go-ns1/2.0.0: GET https://api.nsone.net/v1/zones/siwyd.com/siwyd.com/MX
10 mxb.mailgun.org
10 mxa.mailgun.org
This is the actual JSON response with curl (notice the integer in the answer array):
{
"domain": "siwyd.com",
"zone": "siwyd.com",
"use_client_subnet": true,
"answers": [
{
"answer": [
0,
"issuewild",
";"
],
"meta": {},
"id": "59a6e940c9c79d000120d3e3"
},
{
"answer": [
0,
"issue",
"letsencrypt.org"
],
"meta": {},
"id": "59a6e940c9c79d000120d3e4"
}
],
"id": "59a6e940c9c79d000120d3e5",
"regions": {},
"meta": {},
"link": null,
"filters": [],
"ttl": 3600,
"tier": 1,
"type": "CAA",
"networks": [
0
]
}
This is the response for an MX record:
{
"domain": "siwyd.com",
"zone": "siwyd.com",
"use_client_subnet": true,
"answers": [
{
"answer": [
"10",
"mxb.mailgun.org"
],
"meta": {},
"id": "59bec368a632f6000161d493"
},
{
"answer": [
"10",
"mxa.mailgun.org"
],
"meta": {},
"id": "59bec368a632f6000161d494"
}
],
"id": "59517182a632f6000189affa",
"regions": {},
"meta": {},
"link": null,
"filters": [],
"ttl": 14400,
"tier": 1,
"type": "MX",
"networks": [
0
]
}
Apply PR #7
Docs are at https://pkg.go.dev/gopkg.in/ns1/ns1-go.v2 . Could you please link to them in the README?
I'd like to use this library to pull in QPS Reports, however the QPS code seems to be commented out. Is there any particular reason for this? I've hacked up a functioning fork temporarily for prototyping. If there's interest in bringing this functionality into the official client, I'd be happy to clean up my code and help out.
For the domain property on the zone record struct, why is the JSON tag capitalized?
https://github.com/ns1/ns1-go/blob/v2/rest/model/dns/zone.go#L59
It is inconsistent with the rest of the code base, and breaks some code-gen tools when trying to integrate with the ns1-go package.
According to the API documentation here: https://developer.ibm.com/apis/catalog/ns1--ibm-ns1-connect-api/api/API--ns1--ibm-ns1-connect-api#getFeed , the feed resource has a "destinations" field. However this field is missing from the client implementation's Feed
struct.
I found the implementation for it in the same file above the Feed
:
ns1-go/rest/model/data/feed.go
Lines 3 to 23 in 72c28cb
Extending the struct like this:
type Feed struct {
...
Destinations []Destination `json:"destinations"`
}
Seems to be working properly for DataFeeds.Get
and DataSources.Get
.
Right now User.last_access
mis-typed with int type, which caused unmarshal issue.
json: cannot unmarshal number 1591550213.0 into Go struct field User.last_access of type int
https://pkg.go.dev/ is the new system for browsing package documentation.
https://pkg.go.dev/gopkg.in/ns1/ns1-go.v2 to read this package because of a discrepancy with the license. The license file is there in a file name the site expects so there must be a discrepancy with the license text.
The license policy is here: https://pkg.go.dev/license-policy
I see there is list for zones but not for records. And records has just a get function.
There is a function/rest call we can do to get record list of zone in one call?
There does not seem to be a way to filter DNS records in a zone (in DDI). for EX: to search for specific records
/v1/dns/record/search
doesn't seem to be implemented in GO SDK. (unless I'm missing something)
We are currently making use of the terraform-provider-ns1
to manage our users. The provider is configured to either set the RateLimitFunc to RateLimitStrategyConcurrent
or RateLimitStrategySleep
(see https://github.com/terraform-providers/terraform-provider-ns1/blob/master/ns1/config.go#L63-L67) for the ns1-go
lib. Unfortunately, during large refreshes of users we are still receiving 429 Rate limit exceeded
errors. After closer inspection of the code, I may not understand how rate limiting is implemented, but I don't believe it is working as expected.
When looking at the Do
method, I see that the request is first executed, and then the RateLimitFunc is executed. This seems backwards. If the client is configured to utilize the RateLimitStrategySleep
, the request will only sleep after it has been executed, making it essentially useless. I was thinking that maybe it was done this way because a retry would occur once the sleep happened, but I'm not seeing anywhere in the code where a retry might occur.
Apologies if I'm completely mis-reading the code, but ultimately, we're still having errors occur due to rate limits which I feel should be working correctly with this lib.
This issue is a continuation of PR #11.
So, I have made some breaking changes while iterating over the rest client, which I know is pretty anti-go(or any open source project for that matter).
@bobtfish had a great suggestion of utilizing gopkg:
master
into a v1
.
go get github.com/ns1/ns1-go
and go get gopkg.in/ns1/ns1-go.v1
should workdevelop
branch, we can branch that off into a v2
v1
that are still using import "github.com/ns1/ns1-go"
. but they will just need to update to import "gopkg.in/ns1/ns1-go.v1"
I could be wrong, but I believe that the disadvantages of gopkg are:
go get
command is enhanced to respect versioning.note: I made a v1
tag this morning on master
, but in hindsight, that shouldve been a branch named v1
(which I can still do).
I need to update the readme of v1
so that it states the future v2
/develop
branch version is coming, and that users should update their import paths when possible. Im thinking a time table of getting the v2
branch created by the end of the month.
Also PR #7 breaking changes are included in the develop branch. I think we can close that. Does that sound okay @josephholsten?
Regarding the corresponding terraform provider, can we wait until v2
has been created to add additional work to it?
Thanks again @bobtfish and @sarguru for the help and patience in migrating this repo to the ns1
organization.
I'm getting:
panic: interface conversion: error is *json.SyntaxError, not *rest.Error
from gopkg.in/ns1/ns1-go.v2/rest.(*RecordsService).Get()
Looks like it's happening at https://github.com/ns1/ns1-go/blob/v2/rest/record.go#L28
Hi,
I'm co-maintaining dnscontrol's ns1 support - which uses ns1-go under the hood.
It seems that 2.7.13 that is available as latest via gopkg is getting 500 from NS1's API when attempting to create or update any records. Unfortunately given that it's a 500, I don't get any more info about what's wrong.
Downgrading to 2.7.12 triggers the issue with tags/blocked_tags, so can't really say if that's an issue in 2.7.12 as well. It seems though, that 2.7.11 is working correctly against the API, so it's definitely something post 2.7.11.
Any combination of nil for tags/blocked_tags doesn't seem to help.
Not sure what more info I can provide, but happy to try anything, i can reproduce it every time in my setup.
thanks!
Hello!
I'm using ns1-go and it seems that at some recent point zone exists error handling broke - which broke our ZoneExists error handling as well :)
Digging a bit into this, it looks like NS1 API returns a different string nowadays, which breaks error matching under rest/zone.go
's Create
function.
I was able to verify with a simple curl invocation on an existing zone as well:
$ curl -X PUT -H "X-NSONE-Key: $NSONE_API_KEY" -d ' { "zone":"example.com", "networks": [0], "ttl": 3600, "refresh": 43200, "retry": 7200, "expiry": 1209600, "nx_ttl": 3600 } ' https://api.nsone.net/v1/zones/example.com
{"message":"invalid: FQDN already exists in the view"}
Seems that the fix in #162 fixes that in all test cases I was able to test.
Best,
Costas
In README.md is link to NS1 API: https://ns1.com/api/ - this is redirected to https://ns1.com/api/?docId=2393 but it ends with infinite loading screen.
Probably because there are 2 requests (https://ns1.com/api/api/data.json
and https://ns1.com/api/apidoc/data.json?docId=2393
) with 404 response
I'm working on updating to remove as many lint and vet warnings without breaking API compat.
I'd like to know if you're interested in breaking API compat to make vet happy, eg:
Id
-> ID
Dns
-> DNS
Ttl
-> TTL
_
from public fieldsWas trying to debug updating a record and I found this pattern repeated frequently:
https://github.com/ns1/ns1-go/blob/v2/rest/record.go#L89
if err != nil {
switch err.(type) {
case *Error:
switch err.(*Error).Message {
case "zone not found":
return resp, ErrZoneMissing
case "record already exists":
return resp, ErrRecordExists
}
default:
return resp, err
}
}
The inner switch for when the type is *Error
has no default. So if it is an Error
with a different message, the whole thing falls through and I get a nil error returned from the function, which is incorrect.
I'd submit a PR, but this seems to be a problem throughout the entire library, not just this one place.
We have to override this package in order to log out the response body of bad responses because it gets closed within ns1-go (and can not be read after it gets closed)
Rather than just returning an unmarshaling error, why not also pass along the response body?
Simple typo in the source, I think, and you can work around it by "manually" instantiating the proper structure.
Hi folks, thanks for the work on the client, it's appreciated.
Is this still accurate and / or relevant to this repo?
I ask because I really need to have the ability to pass in a context.Context to requests and I see #192 and #182 open for months without comment, tags or review. I'm willing to do whatever work is required to make this happen, but I don't want to go though the trouble if the PR will not be reviewed.
Also, it seems like make test
is failing in the tip of v2 and also the v2.7.12 and v2.7.13 tags:
$make test
go fmt ./...
go vet ./...
# gopkg.in/ns1/ns1-go.v2/rest/model/dns_test
vet: rest/model/dns/example_test.go:57:46: not enough arguments in call to dns.NewRecord
have (string, string, string)
want (string, string, string, map[string]string, []string)
make: *** [test] Error 1
Is this still accurate?
I totally get that there may not be enough time or people to get to PRs, I'm not looking to pressure anybody. But if that's the case, I think maybe the docs need to change to reflect that?
Thanks!
Right now, several endpoint handler methods repeat a common pattern of checking the response message for specific error messages. eg:
if err.(*Error).Message == "record not found" {
return nil, resp, ErrRecordMissing
}
case "zone not found":
return resp, ErrZoneMissing
case "record already exists":
return resp, ErrRecordExists
As proposed by @captncraig in #34 (comment), we should probably move these checks into a central place like Client.Do, or more specifically, in CheckResponse.
This issue is submitted to tie this project to the reported issue in the ns1 terraform provider where the initial problem was identified. See Initial Issue Report.
This issue is being reported to associate with a pull request shortly and is intended to close the loop between the two projects.
There appears to be no way to utilize contexts for requests. This makes it impossible to set effective timeouts for requests, as well as instrument any requests. To maintain backwards compatibility, I'd suggest making additional methods called GetWithContext
or UpdateWithContext
for example.
The equivalent of this endpoint: https://ns1.com/api#postpublish-data-from-a-data-source
stats.go example (https://github.com/ns1/ns1-go/blob/v2/rest/_examples/stats.go) fails with:
2022/07/07 19:49:33 go-ns1/2.6.5: GET https://api.nsone.net/v1/stats/qps
2022/07/07 19:49:33 json: cannot unmarshal array into Go value of type float32
API returns something like this:
{"networks":[{"network":0,"qps":106.125}],"qps":106.06666666666666}
However stat.go (https://github.com/ns1/ns1-go/blob/v2/rest/stat.go) probably expects something like this:
{"qps":105.01666666666667}
This is a test of the jira webhook
Can't seem to get the rest
sub-package. At first I thought it was likely user error on my part, but I'm not so sure anymore.
Updated description/cause:
v2.4.0 from 28 days ago (https://github.com/ns1/ns1-go/releases/tag/v2.4.0) seems to be pointing to the wrong commit:
7c167e5 which is from 2016 (really a v1 commit?)
Take a peek inside go/src/gopkg.in/ns1/ns1-go.v2 at the git repo:
Original notes:
C:\Users\REMOVED>go get gopkg.in/ns1/ns1-go.v2/rest
package gopkg.in/ns1/ns1-go.v2/rest: cannot find package "gopkg.in/ns1/ns1-go.v2/rest" in any of:
c:\go\src\gopkg.in\ns1\ns1-go.v2\rest (from $GOROOT)
C:\Users\REMOVED\go\src\gopkg.in\ns1\ns1-go.v2\rest (from $GOPATH)
And if I check in the directory, it is indeed missing the rest directory:
Tried combinations of fetching package first, then subpackage, and viceversa. It seems like the issue is actually related to release tagging.
Seems like a cool project, would be good to have something to point community members in the right direction for contributions
I'm in the middle of updating Ensighten/terraform's nsone provider branch to use v2. Feel free to chime in if I do anything dumb: Ensighten/terraform#30
To install module you now have to use go install gopkg.in/ns1/ns1-go.v2@latest
The 'cost' meta field doesn't seem to get updated when using the NS1 Terraform Provider. Is it possible that it is because it is missing in here:
ns1-go/rest/model/data/meta.go
Line 39 in b605d13
I also created a PR to support the cost meta field:
#109
dnssec
attribute.v1/zones/<zone name>/dnssec
endpointFiles in gopkg.in/ns1/ns1-go.v2/rest
are currently import
ing:
"github.com/ns1/ns1-go/rest/model/data"
"github.com/ns1/ns1-go/rest/model/account"
"github.com/ns1/ns1-go/rest/model/dns"
"github.com/ns1/ns1-go/rest/model/monitor"
Is this intentional?
If TSIG is enabled on a secondary zone, it can't be disabled through this library because the field zone.Secondary.TSIG.Enabled
is tagged with json:"omitempty"
, so it will never be serialized to JSON if it's false.
This code will reproduce:
zone := dns.NewZone("secondary.test")
zone.MakeSecondary("1.2.3.4")
zone.Secondary.TSIG = &dns.TSIG{
Enabled: true,
Key: "abc123",
Name: "test",
Hash: "hmac-sha512",
}
client.Zones.Create(zone)
zone.Secondary.TSIG.Enabled = false
client.Zones.Update(zone)
// zone still has TSIG enabled
When creating a new URLFWD record the API endpoint returns a "Invalid redirect mode for URLFWD record" error.
The Answer struct defines the RData as a string array. The API endpoint on the other hand appears to be expecting an integer for the Type, Forwarding Mode, and Query Forwarding values.
newRecord := dns.NewRecord("mydomain.com", "fwd.mydomain.com", "URLFWD")
newRecord.TTL = 3600
newRecord.AddAnswer(dns.NewAnswer([]string{"/", "https://www.google.com", "301", "2", "0"}))
Generates the following json request:
{
"meta": {},
"zone": "mydomain.com",
"domain": "fwd.mydomain.com",
"type": "URLFWD",
"ttl": 3600,
"answers": [
{
"meta": {},
"answer": [
"/",
"https://www.google.com",
"301",
"2",
"0"
]
}
],
"filters": []
}
Submitting the same request with curl, but changing the answer array as follows succeeds.
"answer": [
"/",
"https://www.google.com",
301,
2,
0
]
SDK version: v2.4.1
NS1 version: don't have access to this, but it's one of the latest ones
client Get & Delete return record doesn't not exist, even though Create, Update work as expected.
I pass zone & hostname, the same values as for Create / Update.
Anyone experienced the same issue?
Thanks
According to the API documentation here: https://developer.ibm.com/apis/catalog/ns1--ibm-ns1-connect-api/api/API--ns1--ibm-ns1-connect-api#getRecord , the answer fields of the record resource has a "feeds" field. However this field is missing from the client implementation's Answer
struct here:
ns1-go/rest/model/dns/answer.go
Lines 14 to 27 in 72c28cb
Extending the struct like this:
type AnswerFeed struct {
Feed string `json:"feed"`
Source string `json:"source"`
}
type Answer struct {
...
Feeds []AnswerFeed `json:"feeds"`
}
Seems to be working properly for Records.Get
.
This field would be very useful to have included, because feeds can not be queried only by their ID (that can be get from a metadata feed pointer) the source ID is needed as well, but there's no other way to get this info.
Hello,
Is there a plan to add the reporting and statistics API to the client?
If a resource does not exist and we receive "404" we should return an error instead of nil as we are doing it now.
in v1, nsone.APIClient
had:
GetApikey(id string) (Apikey, error)
CreateApikey(k *Apikey) error
DeleteApikey(id string) error
UpdateApikey(k *Apikey) error
in v2, rest.APIKeysService
has
List() ([]*account.APIKey, *http.Response, error)
Should we add the rest of the CRUD like the other services? I don't personally use them, but the terraform provider has a resource for managing them.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.