nbd-wtf / go-nostr Goto Github PK

Hi,

In some cases, the following code is executed after the response event at L176-L189, published status may return 0 instead of 1.

Run go test -v -race -run TestPublish (this may take many times) to reproduce it.

noscl setprivate
noscl public (outputs wrong pubkeys).

this seemed to work for my previous key but not for this new 32bit POW key?

Hm.

Running the latest code, relay.Subscribe only returns a single event. The filter set is the following

now := time.Now()
sub := relay.Subscribe(context.Background(), nostr.Filters{
        nostr.Filter{
	        Kinds: []int{nostr.KindTextNote},
	        Since: &now,
        },
})

Unfortunately I'm can't seem to drill down what is causing the issue. Anyone else having the same issue?

In file nip04/nip04.go:

panic: runtime error: slice bounds out of range [:-193]

this is the offending line

105            message := string(plaintext[0 : len(plaintext)-padding])

the encrypted string that triggers this to happen
e2FAnpPXoAyQfA26zke60w==?iv=DWb3UYIqz4U/XLaNAkMGiA==

Any more info I should provide?

Would it be possible to add the #nostr tag to this repo

I'll show up in this nice list here:

https://github.com/topics/nostr

Calling ParseMessage with invalid input triggers a panic.

import (
	"github.com/nbd-wtf/go-nostr"
	"testing"
)

func TestPanic(t *testing.T) {
	nostr.ParseMessage([]byte("invalid,input"))
}

It seems that the lib can no longer post to relay.damus.io when using commit: 0a3e898 or c42059f.

Event is simple (somewhat simplified for readability):

        ev := nostr.Event{
                PubKey:    n.public,
                CreatedAt: time.Now(),
                Kind:      nostr.KindTextNote,
                Tags:      nil,
                Content:   msg,
        }
        ev.Sign(n.private)
        result, err := r.Publish(ctx, ev)
        if err != nil {
                n.printf("publishing error %v: %v\n", r, err)
        }
        if result == nostr.PublishStatusFailed {
                n.printf("could not send: %v\n", r)
        }

This ends up traveling through nostr.PublishStatusFailed path like so: could not send: wss://relay.damus.io and the damus client does indeed not receive the message.

Running the same code (minus the time modifications required by v0.17.0) works fine.

Was hoping someone would know what is up before I spend a bunch of time digging through the code.

On shutdown I do this:

if relay != nil && relay.IsConnected() {
			err := relay.Close()
			if err != nil {
				logger.Logger.WithError(err).Error("Could not close relay connection")
			}
		}

Today I got an error:

panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0xb6e1d7]

goroutine 122 [running]:
github.com/nbd-wtf/go-nostr.(*Connection).Close(...)
        /home/roland/go/pkg/mod/github.com/nbd-wtf/[email protected]/connection.go:174
github.com/nbd-wtf/go-nostr.(*Relay).Close(0xc00032c090)
        /home/roland/go/pkg/mod/github.com/nbd-wtf/[email protected]/relay.go:484 +0xb7
github.com/getAlby/nostr-wallet-connect/service.(*service).StartNostr.func1()
        /home/roland/dev/alby/nostr-wallet-connect-next/service/start.go:104 +0x8d1
created by github.com/getAlby/nostr-wallet-connect/service.(*service).StartNostr in goroutine 53
        /home/roland/dev/alby/nostr-wallet-connect-next/service/start.go:44 +0x409

Is this the correct way to close the relay on exit? ideally it should return an error, not panic

Hi, I am trying to retrieve the last 3 notes from a single user (by modifying example.go) but I dont understand how to do this.
I am using the Author field in the filters but the results from the relay are empty.

The README.md example seems broken. There is no sub, just subId. Would submit PR but did not find a good way to unsub.

Referenced part of the README.md:

subId, events := pool.Sub(nostr.Filters{
	{
		Authors: []string{"0ded86bf80c76847320b16f22b7451c08169434837a51ad5fe3b178af6c35f5d"},
		Kinds:   []int{nostr.KindTextNote}, // or {1}
	},
})

go func() {
	for event := range nostr.Unique(events) {
		log.Print(event)
	}
}()

time.Sleep(5 * time.Second)
sub.Unsub()

Why did you opt for a custom build escapeString over the native one?
https://pkg.go.dev/strconv#AppendQuote

I tested it out and it seems to produce the same result

Test string

message := `{"msg":"Hello Alice! How are you?", /"bad/": "field"}`

ESCAPED escapeString :"{\"msg\":\"Hello Alice! How are you?\", /\"bad/\": \"field\"}"

ESCAPED strconv.AppendQuote :"{\"msg\":\"Hello Alice! How are you?\", /\"bad/\": \"field\"}"

Thanks

Valid events are being dropped here:

For some reason Signature.Verify in github.com/btcsuite/btcd/btcec/v2 returns false on some signatures that are actually valid.

Here's a smattering of event IDs that were reported as an invalid sig:

31acf56113720da6fe4aff4f2339294fa96addee134609180286c184e74641d1
180c99be0a40eee4dce93aa29cd03586d2b58468d13f7862dd403534404cd937
190c840ad07ab6cecac6cc7acb4773572da548609be817d680dd9007946113ca
b3a6b874c615bfad7b2fb207372202bf51b70cd58c9300a0c269c94fc54cec69
b1483e40f1bbaeafe88bb07ff96dfe5a0838c4adddbc2c6336b47dd4b1860cd9

Nostr army knife validates them as valid.

Today I found out that wss://relay.noswhere.com replies with the following EOSE messages:

["EOSE","01HF733A3WE8PDSY39FP5AZ206",{"debug":["filters: not planned (non-search queries)","no applicable filters"]}]

The returned array contains an extra element which specifies the reason behind this EOSE message. This causes the ParseMessage function to return nil as EOSEEnvelope.UnmarshalJSON returns an error. The question is: should we do something about this and change the code so that this message is correctly parsed?

In my opinion the situation is unclear because I'd view this as violating NIP-01 which is something I don't want to encourage.

As mentioned in NIP-05 part security-constraints, It's better to add CheckRedirect for http Client option to restrict redirect from server.

	client := &http.Client{
		CheckRedirect: func(req *http.Request, via []*http.Request) error {
			return http.ErrUseLastResponse // return error to prevent redirect
		},
	}

It is possible to obtain the messages of a specified account through a public key, how to obtain the broadcast messages of all users on the relay?

Probably needs a mutex here.
@fiatjaf if you fix it cool, if not and it keeps bugging me I'll eventually send you a pull request.

fatal error: concurrent map read and map write

goroutine 125 [running]:
runtime.throw({0x14d0672?, 0xc000225108?})
	/usr/local/Cellar/go/1.18/libexec/src/runtime/panic.go:992 +0x71 fp=0xc001021d40 sp=0xc001021d10 pc=0x1033f91
runtime.mapaccess2_faststr(0xc0010481d0?, 0x10?, {0xc0010481f0, 0xe})
	/usr/local/Cellar/go/1.18/libexec/src/runtime/map_faststr.go:117 +0x3d4 fp=0xc001021da8 sp=0xc001021d40 pc=0x1013cd4
github.com/fiatjaf/go-nostr.(*RelayPool).Add.func1()
	/Users/gareth/go/pkg/mod/github.com/fiatjaf/[email protected]/relaypool.go:155 +0x217 fp=0xc001021fe0 sp=0xc001021da8 pc=0x12a6e77
runtime.goexit()
	/usr/local/Cellar/go/1.18/libexec/src/runtime/asm_amd64.s:1571 +0x1 fp=0xc001021fe8 sp=0xc001021fe0 pc=0x1063d81
created by github.com/fiatjaf/go-nostr.(*RelayPool).Add
	/Users/gareth/go/pkg/mod/github.com/fiatjaf/[email protected]/relaypool.go:111 +0x445

I see that all JSON tags were removed here 2bf6c87

Any reason why? Could we bring them back?

Thanks.

I'm using SimplePool.SubMany and explicitly canceling the context to avoid the risk of goroutine bloat, as per the README. However I'm noticing a memory leak in my project and pprof is pointing to go-nostr.

I dug into the subMany method and have confirmed this is the point at which the context cancel is caught. The problem is, this path never invokes the defer func above so the events chan is never closed. I wonder if leaking these event chans is the problem.

Here's how I'm using it [actual source]:

// ctx is a context.WithTimeout
subscription = nostr.NewSimplePool(ctx).SubMany(ctx, someRelays, []nostr.Filter{filter})

Any ideas? Am I using the library incorrectly?

The unmarshal is handled here https://github.com/nbd-wtf/go-nostr/blob/master/event_aux.go#L71-L74. I printed the parsed json message here https://github.com/nbd-wtf/go-nostr/blob/master/event_aux.go#L13 and modified the code to be

		default:
			var anyValue any
			fmt.Println("v:", v)
			val := json.Unmarshal(v.MarshalTo(nil), anyValue)
			fmt.Printf("\nsetting key: %s to value: %s, %s\n", key, anyValue, val)
			evt.extra[key] = anyValue

Here's the logs:

parsed: {"id":"76d199c9c963de0c35aefbabffd5312d1ebde3a2c5ab945bdc587c554f0ff1da","pubkey":"73c7e463595fa9b738ae26e4ed20bacb06bcc8dfd811f9cbf41040c37c5bd231","created_at":1672764325,"kind":1,"tags":[["prev","8941e44e02a16f322613990caa71d88a63e6538f896edbf94c69e323f3566332"]],"content":"Hi l7","sig":"167f435f787d939a32004721cfd4f8bbd12ba18b53ab098698295bda759384c8f6c3c0ceb55b76461184d91de45f7ae97324f8fab969a7440fa279924f026be0","ots":"AE9wZW5UaW1lc3RhbXBzAABQcm9vZgC/ieLohOiSlAEIdtGZyclj3gw1rvur/9UxLR6946LFq5Rb3Fh8VU8P8drwEMcZCsYdJRtImWZFCxWrBIQI8CCj0EYcO5wlMl7k1kToczMt+WGg8YcVKcJQIB7H6Z07wAjxBGO0W6bwCHbs/SQSPRYNAIPf4w0u+QyOLi1odHRwczovL2FsaWNlLmJ0Yy5jYWxlbmRhci5vcGVudGltZXN0YW1wcy5vcmc="}
v: "AE9wZW5UaW1lc3RhbXBzAABQcm9vZgC/ieLohOiSlAEIdtGZyclj3gw1rvur/9UxLR6946LFq5Rb3Fh8VU8P8drwEMcZCsYdJRtImWZFCxWrBIQI8CCj0EYcO5wlMl7k1kToczMt+WGg8YcVKcJQIB7H6Z07wAjxBGO0W6bwCHbs/SQSPRYNAIPf4w0u+QyOLi1odHRwczovL2FsaWNlLmJ0Yy5jYWxlbmRhci5vcGVudGltZXN0YW1wcy5vcmc="

setting key: ots to value: %!s(<nil>), json: Unmarshal(nil)

We can see the "ots" field is a string and should get set, but I'm not sure why we're getting nil value back. I'm not familiar with fastjson lib and didn't have the time to look into debugging why exactly that's the case. I have a workaround by adding a new case for "ots" and parse a string manually like everything else so it's no rush. But it's not a clean solution.

https://github.com/nbd-wtf/go-nostr/blob/master/filter.go#L16

Is there any good reason for this? it means the filter struct cannot be easily used

Im working on trying to implement an app using nostr over the tor network. Sadly I cannot find any documentation if it is possible to ask this library to communicate through a proxy (which would be the running tor instance) for the wss. Do anyone have a clue ? I tried to dive into the low level code of this library to try to understand if it would be able to set it up but im not skilled enough to do it on my own.
Otherwise if someone could help me by giving me some insight on which object structure in the library (the connection object ?) would be the most suitable to handle the proxy thing I could try to implement it.

Thanks a lot

Hi, could you provide an example of sending and receiving Encrypted Direct Messages to a public key (NIPS-04)?

https://github.com/nostr-protocol/nips/blob/master/04.md

Hi I'm a window 11 user and golang beginner, at first I try to run
go run example/example.go
it returns the error panic: error opening websocket to 'wss://nostr.zebedee.cloud': failed to dial: dial tcp 24.199.71.187:443: i/o timeout
Then I change the 2 urls into "wss://relay.damus.io" but it runs endlessly an never terminates, am I doing wrong somewhere?

Hi there,

thanks for your hard work on this extremely useful library. I'm trying to implement nip59 and I've noticed that nip44.Encrypt() will fail unless you provide a custom salt through the applyOptions function. I'm not sure if this is intentional or not, but looking at the source code I noticed on line 56:

that the randomly generated salt is assigned to a shadow variable named salt and not the variable named salt at the top level scope. I assume this is an error because there would be no need to generate a random salt if it's never used.
It's a fairly trivial fix and if you agree that this is indeed an error, I can make a pull request.
Thanks again!

Would their be an chance an example could be made for creating, subscribing and posting to a Public Chat (NIP-28)?
Any help would be much appreciated, I've managed to create the channel but am having issues figuring out how to subscribe to the created public chat-

What would you like to be added:
Consider using https://github.com/berty/go-orbit-db as a backend to expose a nostr provider as a Kademlia service. Kademlia implemented with libp2p is one of the most mature p2p networks, designed over 20 years ago it has scaled up to hundreds of millions of nodes and is the network behind Ethereum, IPFS and Magnet BitTorrent links.

Why is this needed:
It allows nostr to be run as a federated dapp and hosted by a community.
https://www.youtube.com/watch?v=cC-tXnMyiBc

On the implementation of nip04 padding is used.

This seems to be causing incompatibility with other clients like Damus.

In the spec of nip04 there doesn't seem to be a definition of padding.

Would it be possible to remove the padding, or at least have it be optional?

If agreed, I can submit a PR with a fix

The latest xsync major version is v3, so /v3 suffix should be used when importing the library:

import (
	"github.com/puzpuzpuz/xsync/v3"
)

Note for v1 and v2 users: v1 and v2 support is discontinued, so please upgrade to v3. While the API has some breaking changes, the migration should be trivial.

https://github.com/puzpuzpuz/xsync

Identified a security flaw allowing event ID spoofing. Submitted a PR #119 with a fix and unit tests for validation.

nostr.NewRelayPool() does not seem to be a defined function?

also the added contexts everywhere... I am a bit confused, perhaps the new introductory example can explain a bit what you had in mind for those. I just use context.Background() everywhere.

Here: https://github.com/nbd-wtf/go-nostr/blob/master/tags.go#L117

Function

func (t *Tags) Scan(src any) error {
	var jtags []byte = make([]byte, 0)

	switch v := src.(type) {
	case []byte:
		jtags = v
	case string:
		jtags = []byte(v)
	default:
		return errors.New("couldn't scan tags, it's not a json string")
	}

	json.Unmarshal(jtags, &t)
	return nil
}

It's unclear how the parameter should look like because the Unmarshal error is never handled
My guess at a glance is that this function is never used or can someone help me understand?

unpaddedLen is a uint16:

if unpaddedLen holds the value 65535, adding 2 results in an overflow and a panic in line 143:

panic: runtime error: slice bounds out of range [2:1]

goroutine 1 [running]:
github.com/nbd-wtf/go-nostr/nip44.Decrypt({0x140003a8000?, 0x14000390000?}, {0x14000026180, 0x20, 0x20})
	/Users/sebdev/Documents/sebas/projects/nostr/go-nostr/nip44/nip44.go:143 +0x5d4

Failing test case provided in my fork: https://github.com/sebdeveloper6952/go-nostr/blob/de50e524b7d7e41901b9ac07e9dda991166e3030/nip44/nip44_test.go#L227

I dont understand why a channel is returned when can be returned a error. In the commit was written something about a helper function and logic. What that these refer?

https://github.com/nbd-wtf/go-nostr/blob/master/keys.go#L14C6-L14C24

If the hex starts with 00's they will be removed. Suggested solution (untested):

hexStr := hex.EncodeToString(k.Bytes())
return fmt.Sprintf("%064s", hexStr)

Supposedly this should be very rare, but I believe it's already happened to me twice.

old code

func (v CountEnvelope) MarshalJSON() ([]byte, error) {
	w := jwriter.Writer{}
	w.RawString(`["COUNT",`)
	w.RawString(`"` + v.SubscriptionID + `"`)
	if v.Count != nil {
		w.RawString(fmt.Sprintf(`{"count":%d}`, *v.Count))
	} else {
		for _, filter := range v.Filters {
			w.RawString(`,`)
			filter.MarshalEasyJSON(&w)
		}
	}
	w.RawString(`]`)
	return w.BuildBytes()
}

->

new code

func (v CountEnvelope) MarshalJSON() ([]byte, error) {
	w := jwriter.Writer{}
	w.RawString(`["COUNT",`)
	w.RawString(`"` + v.SubscriptionID + `"`)
	if v.Count != nil {
		w.RawString(fmt.Sprintf(`,{"count":%d}`, *v.Count)) // change: add ',' character 
	} else {
		for _, filter := range v.Filters {
			w.RawString(`,`)
			filter.MarshalEasyJSON(&w)
		}
	}
	w.RawString(`]`)
	return w.BuildBytes()
}

When a relay publishes an event, it immediately subscribes to the event to find out if the event was indeed sent. The subscription is not "removed" after the deed is done

// relay.go    func Publish()

sub := r.Subscribe(ctx, Filters{Filter{IDs: []string{event.ID}}})
defer sub.Unsub()   <<---- this line is missing? --------
for {
        select {
        case receivedEvent := <-sub.Events:
	        if receivedEvent.ID == event.ID {
		        // we got a success, so update our status and proceed to return
		        status = PublishStatusSucceeded
		        return status
	        }
        case <-ctx.Done():
	        // return status as it was
	        // will proceed to return status as it is
	        // e.g. if this happens because of the timeout then status will probably be "failed"
	        //      but if it happens because okCallback was called then it might be "succeeded"
	        return status
        }
}

This is probably a minor issue. I'm trying to understand the codebase better and thought I would get some clarification on this issue to make sure that my understanding is correct

Always send in a loop, Concurrent errors may occur, Causing program interruption

if len(relayConnect) != 2 {
		for _, url := range []string{"wss://relay.damus.io", "wss://nos.lol"} {
			relay, e := nostr.RelayConnect(context.Background(), url)
			if e != nil {
				fmt.Println(e)
				delete(relayConnect, url)
				continue
			}
			relayConnect[url] = relay
		}
	}

for url, relay := range relayConnect {
		var a context.Context = context.Background()
		b, e := relay.Publish(a, ev)
		fmt.Println("published to ", url, b, e)

		if e != nil {
			fmt.Println(e)
			delete(relayConnect, url)
}


panic: concurrent write to websocket connection

goroutine 241620 [running]:
github.com/gorilla/websocket.(*messageWriter).flushFrame(0xc00e0e9e60, 0x1, {0x0?, 0x0?, 0x0?})
	/root/go/pkg/mod/github.com/gorilla/websocket@v1.4.2/conn.go:610 +0x50b
github.com/gorilla/websocket.(*messageWriter).Close(0xc01275ce10?)
	/root/go/pkg/mod/github.com/gorilla/websocket@v1.4.2/conn.go:724 +0x45
github.com/gorilla/websocket.(*Conn).beginMessage(0xc0000a09a0, 0xc01275ce10, 0x1)
	/root/go/pkg/mod/github.com/gorilla/websocket@v1.4.2/conn.go:473 +0x42
github.com/gorilla/websocket.(*Conn).NextWriter(0xc0000a09a0, 0x1)
	/root/go/pkg/mod/github.com/gorilla/websocket@v1.4.2/conn.go:513 +0x45
github.com/gorilla/websocket.(*Conn).WriteJSON(0x7f40fc119418?, {0xa72a80, 0xc0044c30e0})
	/root/go/pkg/mod/github.com/gorilla/websocket@v1.4.2/json.go:24 +0x45
github.com/nbd-wtf/go-nostr.(*Connection).WriteJSON(0xc0044fe2e0?, {0xa72a80?, 0xc0044c30e0?})
	/root/go/pkg/mod/github.com/nbd-wtf/go-nostr@v0.15.3/connection.go:22 +0xb0
github.com/nbd-wtf/go-nostr.(*Subscription).Unsub(0xc014007a00)
	/root/go/pkg/mod/github.com/nbd-wtf/go-nostr@v0.15.3/subscription.go:48 +0x165
github.com/nbd-wtf/go-nostr.(*Subscription).Fire.func1()
	/root/go/pkg/mod/github.com/nbd-wtf/go-nostr@v0.15.3/subscription.go:79 +0x3c
created by github.com/nbd-wtf/go-nostr.(*Subscription).Fire
	/root/go/pkg/mod/github.com/nbd-wtf/go-nostr@v0.15.3/subscription.go:77 +0x2fc

Websocket connections to some relays are closed after a minute of inactivity(I'm seeing it with wss://nostr.oxtr.dev, wss://relay.stoner.com, etc). What should I do if I want to keep the connection alive as long as possible?

What I have in mind is to listen to the connectionError channel and call Connect again when connection breaks (and it's uninitiated by us). Is that the right approach?

An event request, sent to wss://relayer.fiatjaf.com:

["REQ","p380vv138",{"ids":["b6a04d778cf5d991583b51112b78301f5dd8fe09824a12fee2b99281ffa879d3"]}]

It returns nothing - no notice, no nothin'. The same exact request, sent to wss://nostr-pub.wellorder.net (which runs nostr-rs-relay) gets me this result:

[
  "EVENT",
  "p380vv138",
  {
    "id": "b6a04d778cf5d991583b51112b78301f5dd8fe09824a12fee2b99281ffa879d3",
    "pubkey": "3bf0c63fcb93463407af97a5e5ee64fa883d107ef9e558472c4eb9aaaefa459d",
    "created_at": 1642152719,
    "kind": 1,
    "tags": [
      [
        "p",
        "b0635d6a9851d3aed0cd6c495b282167acf761729078d975fc341b22650b07b9"
      ],
      [
        "p",
        "e668a111aa647e63ef587c17fb0e2513d5c2859cd8d389563c7640ffea1fc216"
      ],
      [
        "e",
        "195418da26999c9ea0d3b594079c897c747ca0f3f16917e699aa739abac8243e"
      ],
      [
        "e",
        "8ea3f69a20e8630fc9f346a6dc635b1c3e6765f0e2561ced3439394098c541b4"
      ]
    ],
    "content": "what do you mean by \"branle is not for blog post\"? there is a client-side character limit on every note created on branle, but that doesn't prevent branle from displaying larger notes. and I think it is better for everybody is comments on a blog post can be made from branle.",
    "sig": "2250bd3f5cbcda6b6ad123108214195b04b479b3a36ea4486f1e422af872d8df7b1d6c9ced61cfa8e02a37fdbd7f57cb96d1ad7be61e673ac638316f6c5ca9a0"
  }
]

These requests were sent using Rust CLI client (https://github.com/vinliao/clust). This bug only happens whenever client requests old event - recent events works fine.

When subscribing to a list of IDs, every second event on the list is dropped.

	ids := []string{"d6bee3593951e1d81283250f6bb4241bcf6da036b3dae1ad7aff9107bbeee2f0", "363b07a681e38063e7572e3fefe90936e0d2f743a4e9f4d582e63c33ce276b9e", "4873e66c79f31ed992d43acf0d33d52628a274379cf8e976b42e83293cfc57e8", .........}
	_, events, unsub := pool.Sub(nostr.Filters{
		{
			IDs: ids,
		},
	})

The events are being sent from the relay and are received properly by the websocket connection, but are being erroneously dropped somewhere within this library.

I'm digging into this now and will post updates as I find things.

Working on branch here to try and fix it if anyone is interested: https://github.com/stanstacks/go-nostr/tree/every2ndIDbeingDropped

Hi,
Trying to define the handlers for relayer I get stuck when
~/go/src/relayer$ go get github.com/fiatjaf/go-nostr/filter
go get: module github.com/fiatjaf/go-nostr@upgrade found (v0.6.2), but does not contain package github.com/fiatjaf/go-nostr/filter

Same goes for
~/go/src/relayer$ go get github.com/fiatjaf/go-nostr/event.

Any advice?

We noticed that the subscription will not receive new updates after running for a longer time, and will also not get an error on the error channel (we are not on latest master).

This is probably because the connection gets dropped because of inactivity by a load balancer on the relay side. In order to fix this we could add periodic ping message and return an error if we don't get a pong response in time.