One way async span is supported in brave - openzipkin/zipkin#1243

Are there plans to support this any time soon?

Currently this tracer has collectors for Kafka and Scribe but for easier adoption of Zipkin it would be good to also have an HTTP collector

I tried to install zipkin-go-opentracing with gvt, and was surprised to find the _thrift directory missing from the checkout.

After some searching, I found that gvt has some logic to ignore directories beginning with _. They justify this by linking to golang's description of package lists which states:

Directory and file names that begin with "." or "_" are ignored by the go tool, as are directories named "testdata".

Perhaps it would be a good idea to remove the underscore from the _thrift directory in this package. If you think this is something that should be allowed by gvt please let me know, and I'd be happy to open a ticket with them.

Regards,
Igor

brave and other zipkin libs can create id with minus and int.
Go zipkin use only uint type that means we can not use id with minus

We currently support logging of only IPv4 addresses. Starting with Zipkin 1.4 endpoints can omit IPv4 (by setting Endpoint.ipv4 to 0), and optionally log Endpoint.ipv6 as the raw 16byte address.
https://github.com/openzipkin/zipkin-api/blob/master/thrift/zipkinCore.thrift#L276

In json, both are the string formatting http://zipkin.io/zipkin-api

ipv4:
string
The text representation of a IPv4 address associated with this endpoint. Ex. 192.168.99.100
ipv6:
string
The text representation of a IPv6 address associated with this endpoint. Ex. 2001:db8::c001

I'm trying to build example and run. Found some errors and need help to fix.

When depends on github.com/apache/thrift @ 0.10.0 (which is the lastest version tag). The build result is below

and then goto line 333 of github.com/openzipkin/zipkin-go-opentracing/thrift/gen-go/scribe/scribe.go

the type of self2.processorMap is thrift.TProcessorFunction. And the definition of thrift.TProcessorFunction in github.com/apache/thrift @ 0.10.0 is below

But the Process function's implemention in scribeProcessorLog is defined like this

HI, I come across some error issue when compile code with go get ., details is below

# github.com/openzipkin/zipkin-go-opentracing/wire
wire/wire.pb.go:31: undefined: proto.GoGoProtoPackageIsVersion2
wire/wire.pb.go:582: undefined: proto.RegisterFile
godep: exit status 2

also proto.GoGoProtoPackageIsVersion2 can not be found in github.com/golang/protobuf project after pull all dependent codes.
any one who can help me fix this issue, thanks.

According to https://github.com/openzipkin/b3-propagation

TraceId is encoded as 32 or 16 lower hex characters
SpanId and ParentSpanId are encoded as 16 lower hex characters

We are running into issues because our go client f.e. sometimes sends X-B3-SpanId as 15 hexadecimal characters (when using tracer.inject) which generates an exception in tracer.Extract on the server because it expects 16 hexadecimal characters.

Setting

zipkin.ClientServerSameSpan(true)

in microservices-demo/user#53 caused the shared span to come up with duplicated http.method and http.url binnary annotations (fames marked with 3 in figure below):

Is there a way to make the tracer not to add annotations if their keys are already present in the span?

Thanks!

Hi there,
any reason why the generated Go code for zipkinDependencies.thrift wasn't included under thrift/gen-go/zipkincore?

If the send() method blocks (e.g. because of network), the loop() method continues to spawn goroutines indefinitely. The problem is that the currentBatchSize >= c.batchSize condition will always be true if a send() is blocked (collector-scribe.go#L147).

I think this is a broken behaviour, and could be easily fixed, probably by not allowing another send() to be spawned if one is already running. I'm up for doing a PR if the maintainers agree this should be fixed.

Hi. Seems that textMapPropagator breaks span after it being injected
https://github.com/openzipkin/zipkin-go-opentracing/blob/master/propagation_ot.go#L54

carrier.Set(zipkinTraceID, strconv.FormatUint(sc.raw.TraceID, 16))
    carrier.Set(zipkinSpanID, strconv.FormatUint(sc.raw.SpanID, 16))
    if !p.tracer.options.clientServerSameSpan {
        sc.raw.ParentSpanID = 0
    }
    carrier.Set(zipkinParentSpanID, strconv.FormatUint(sc.raw.ParentSpanID, 16))
    carrier.Set(zipkinSampled, strconv.FormatBool(sc.raw.Sampled))

sc is pointer, so if clientServerSameSpan option is equal false (it`s equal to false by default) then we remove parentSpanID from this span.

Comparing with https://github.com/openzipkin/zipkin-go-opentracing/blob/master/propagation.go#L36 there parentSpanID=0 is only passed to carrier, so it doesn`t affect span.

This error can be reproduced if request depth is more then 2:

service1 (spanID: 1, parentSpanID: 0) -> service2 (spanID: 2, parentSpanID: 1)
service1 (spanID: 1, parentSpanID: 0) -> service2 (spanID: 3, parentSpanID: 1)
service2 (spanID: 3, parentSpanID: 1) -> service3 (spanID: 4, parentSpanID: 3)

But after injecting spanID:3 it`s parentID becomes 0 and it becomes root span like as spanID.

I can send PR for fix this issue if you want.

I see there's a closed issue that looks exactly the same as this, I've literally just installed thrift and trying to go get this package but it's not working out.

~/go/src/git.neds.sh/finance/user $ go get -u github.com/openzipkin/zipkin-go-opentracing
# github.com/openzipkin/zipkin-go-opentracing/thrift/gen-go/scribe
../../../github.com/openzipkin/zipkin-go-opentracing/thrift/gen-go/scribe/scribe.go:282:14: not enough arguments in call to oprot.Flush
	have ()
	want (context.Context)
../../../github.com/openzipkin/zipkin-go-opentracing/thrift/gen-go/scribe/scribe.go:299:16: not enough arguments in call to oprot.Flush
	have ()
	want (context.Context)
../../../github.com/openzipkin/zipkin-go-opentracing/thrift/gen-go/scribe/scribe.go:312:16: not enough arguments in call to oprot.Flush
	have ()
	want (context.Context)
../../../github.com/openzipkin/zipkin-go-opentracing/thrift/gen-go/scribe/scribe.go:326:24: not enough arguments in call to oprot.Flush
	have ()
	want (context.Context)

Am I doing something wrong?

V0.2.2 of this repo is not compatible with 1.0.0 of opentracing-go because fields do not have any method named Key() and Value() which is used here https://github.com/openzipkin/zipkin-go-opentracing/blob/v0.2.2/log-materializers.go#L54.

I get following error:

go/src/github.com/openzipkin/zipkin-go-opentracing/log-materializers.go:54: field.Key undefined (type "github.com/opentracing/opentracing-go/log".Field has no field or method Key, but does have "github.com/opentracing/opentracing-go/log".key)

go/src/github.com/openzipkin/zipkin-go-opentracing/log-materializers.go:54: field.Value undefined (type "vendor/github.com/opentracing/opentracing-go/log".Field has no field or method Value)

However master of both repos are compatible with each other. As convenience methods were added back to opentracing-go in this PR

@basvanbeek @bensigelman How about releasing a minor patch for opentracing-go since the methods are added back.

Hi-
Quoting https://github.com/openzipkin/b3-propagation/#debug-flag-1

Debug is encoded as X-B3-Flags: 1. Debug implies an accept decision: don't also send X-B3-Sampled: 1.

Though https://github.com/openzipkin/zipkin-go-opentracing/blob/master/zipkin-recorder.go#L86 only checks the Sampled flag.

Should there be additional logic added to accept(sample) traces flagged as debug? I can create a PR, but maybe my understanding of this is incorrect.

Thank you!

The opentracing span tags are stored in a map. Zipkin adds them in the arbitrary order in which the iteration returns them:
https://github.com/openzipkin-contrib/zipkin-go-opentracing/blob/master/zipkin-recorder.go#L169-L171

This causes inconsistent ordering (at least in Jaeger). I propose that we sort the tags so they are added in the same order.

Would it be feasible to expose a means of waiting for the spans to finish sending? I'm experimenting with a little CLI tool I wrote, and I presently have to throw in a 1 second sleep in order for the top-most span to arrive at Zipkin.

NewRecorder() calls makeEndpoint() without checking whether it returns nil: zipkin-recorder.go#L34

Absence of nil check leads to hard to understand nil pointer panics (which can happen because of malformed hostPort, for intsance)

In my case i used malformed hostPort without host ":8080" and recieived nil pointer panic from zipkin-recorder.go#L112, which left me scrathing my head for a while

@basvanbeek I just came across this the other day -- thank you for doing the work!

This issue is really a glorified email message.

What's the state of this repo from a correctness standpoint? If it's working well, I'd like to make note of it at opentracing.io and use it as the basis for similar libs in Java, python, etc.

I will happily do the work on the opentracing.io side once you give the high sign.

Thanks again!

Hi,

So I am trying to build a multi-language tracing system that works on Java and Go. The Java API's are working and propagating trace data through HTTP headers, but the GO api is unable to get the span context from the parent Java api. The GO api is able to send basic spans to Zipkin (without propagated traceId and parentId) and I can get the HTTP header being sent to the GO API, but I can't seem to manually set the zipkintracer.Span fields for traceId/parentSpanId. Any clues?

Hello!
I've use master branch of zipkin-go-opentracing and got this message while build my project:
vendor/github.com/openzipkin/zipkin-go-opentracing/thrift/gen-go/scribe/scribe.go:210:5: undefined: thrift.TClient

In latest apache thrift release (v0.10.0) TClient was not found.

Hi ,
We are having gocli bosh and fly, currently we have got the tracing enabled by adding the library. We have got the trace_id and parent_span_id in the environment variables. Do we have option of reading those value from the env and having it added for all the HTTP request.

I have just created the span ,but not recording any events using span.logevent .Even then it is recording some events like server receive and server send .

Seems that "github.com/apache/thrift/lib/go/thrift" is the import path used, but "git.apache.org/thrift.git/lib/go/thrift" should be used instead according to this issue:

https://issues.apache.org/jira/browse/THRIFT-3131

This line looks wrong:

183		c.logger.Log("Backlog too long, disposing spans.", "count", dispose)

From the wrappedLogger implementation, it looks like the arguments should be alternating key-values (by the way, this should ideally be documented inside the Logger interface definition).

The MultiCollector aggregateErrors method returns wrong value when no error occurred, because the returned nil error is nil only as value, not as interface ("nil" Interfaces and "nil" Interfaces Values), therefore error checking will not work as expected:

err := multiCollector.Collect(span) // no errors, Collect returns nil value
if err != nil { // nil value will be != nil interface
  panic(err) // will panic
}

I see this error message when I try to compile my project: ../../openzipkin/zipkin-go-opentracing/propagation_ot.go:9:2: cannot find package "github.com/gogo/protobuf/proto" in any of:
/usr/local/Cellar/go/1.8_1/libexec/src/github.com/gogo/protobuf/proto (from $GOROOT)

and as I can see that repository github.com/gogo/protobuf/proto doesn't exists under github.

Hi, so I'm trying to implement a basic GO api using the go-opentracing, but there are like no examples that I have been able to find. Do you know of any good examples to base my API off of? I'm trying to use the scribe collector and its compiling fine, but none of my spans/traces are being sent to the Zipkin collector. I've included my code in case anybody might be able to debug any glaring mistakes. Appreciate any help!

c, err := zipkintracer.NewScribeCollector("dockerhost:9042",timeout,zipkintracer.ScribeBatchSize(1),zipkintracer.ScribeBatchInterval(batchInterval))
if err != nil {
log.Fatal(err)
}
span := zipkincore.NewSpan()
span.ID = int64(999)
span.TraceID = int64(999)
span.Name = "test"

annotate1 := zipkincore.NewAnnotation()
sr := zipkincore.NewEndpoint()
sr.Ipv4 = (127 << 24) | 1
sr.Port = 8181
annotate1.Host = sr
annotate1.Timestamp = time.Now().UnixNano() / 1e3
annotate1.Value = "test"


annotations := []*zipkincore.Annotation{
    annotate1,
}
binAnnotations := []*zipkincore.BinaryAnnotation{
    binAnnotate,
}
var dur = int64(100)
span.Duration = &dur
span.Annotations = annotations
span.BinaryAnnotations = binAnnotations

c.Collect(span)

Auto-generated Thrift code uses the official import path of git.apache.org/thrift.git/lib/go/thrift, but zipkin-go-opentracing uses the GitHub mirror at github.com/apache/thrift/lib/go/thrift. These are distinctly different packages that are similar in contents, but not actually the same: "An import path is a string that uniquely identifies a package." (emphasis mine)

For example, this means that structures and other types defined in the GitHub.com mirror are distinctly different from the official package.

I realize this has been raised and discussed in previous issues for this project, but this can lead to some interesting bugs if one is not observant of this fact, such as the example seen below:

package main

import (
	"fmt"

	"git.apache.org/thrift.git/lib/go/thrift"
	thrift2 "github.com/apache/thrift/lib/go/thrift"
)

func main() {
	bp := thrift.TBinaryProtocol{}
	var iface interface{} = bp

	_, apache := iface.(thrift.TBinaryProtocol)
	_, github := iface.(thrift2.TBinaryProtocol)

	// will print: apache: true; github: false
	fmt.Printf("apache: %v; github: %v", apache, github)
}

The example indicates the TBinaryProtocol interface exposed by GitHub is a different type from the official Apache mirror one.

Many of our developers use IntelliJ / GoLand IDE which has autocomplete features. Some of them are new to Thrift and/or Go and in the past have sometimes accidentally imported the GitHub package instead of the official Apache package. That puts us at risk of bugs like the one illustrated above. zipkin-go-opentracing is a blocker for us to purge the github.com mirror package completely, because it forces us to have both Thrift packages present (and therefore available to autocomplete, and therefore a trap waiting to ensnare the unwary user).

Given that auto-generated Thrift code uses git.apache.org/thrift.git/lib/go/thrift at https://github.com/apache/thrift/blob/88591e32e710a0524327153c8b629d5b461e35e0/compiler/cpp/src/thrift/generate/t_go_generator.cc#L64 - would zipkin-go-opentracing be open to switching back to the official package given the bugs / code correctness problems that using an alternate mirror package can raise?

Hi,

In the collector-http.go:
When zipkin is down or slow, the blocking send function can take up to c.client.Timeout seconds. in this period, the Collect function can block the application, because the select in the loop function does not extract spans from the spanc.

I think that running the code inside the collect function in a go routine might solve this problem.

Cheers,
Eyal

Hi, got this message while updating dependancies with glide:

api/vendor/github.com/openzipkin/zipkin-go-opentracing/thrift/gen-go/scribe/scribe.go:286: assignment count mismatch: 2 = 1

I am confused about what the hostPort parameter is used for when calling NewRecorder(). It would be nice if this can be better documented.

I was just browsing through bench_test.go and noticed the code in
https://github.com/openzipkin/zipkin-go-opentracing/blob/45e90b00710a4c34a1a7d8a78d90f9b010b0bd4d/bench_test.go#L58-L80

Looks to me like:

1. BenchmarkSpan_100Events
   https://github.com/openzipkin/zipkin-go-opentracing/blob/45e90b00710a4c34a1a7d8a78d90f9b010b0bd4d/bench_test.go#L62-L64
2. BenchmarkSpan_1000Events
   https://github.com/openzipkin/zipkin-go-opentracing/blob/45e90b00710a4c34a1a7d8a78d90f9b010b0bd4d/bench_test.go#L66-L68
   I don't see an obvious difference between those two despite the benchmark names differing by 100 and 1000 respectively

Same thing with
3. BenchmarkSpan_100Tags
https://github.com/openzipkin/zipkin-go-opentracing/blob/45e90b00710a4c34a1a7d8a78d90f9b010b0bd4d/bench_test.go#L70-L72
4. BenchmarkSpan_1000Tags
https://github.com/openzipkin/zipkin-go-opentracing/blob/45e90b00710a4c34a1a7d8a78d90f9b010b0bd4d/bench_test.go#L74-L76

Please advise, and I can submit a PR to fix this.

Hi,
we are experimenting go-kit micro service with openzipkin/zipkin-go-opentracing. One thing I found is that the HTTP header keys like "X-B3-TraceId, X-B3-SpanId, X-B3-ParentSpanId" according to http://zipkin.io/pages/instrumenting.html BECOMES "X-B3-Traceid, X-B3-Spanid, X-B3-Parentspanid" when the http header is sent from go-kit micro service using zipkin-go-opentracing.
While this is fine for tracing between go-kit micro services because the Extract() method of zipkin-go-opentracing/propagation_ot.go resolves the case difference.
However, when the go-kit micro service sends requests with trace to other services implemented in Java or C/C++, the canonical format could create mismatch.
I believe whoever implemented the Extract() method of zipkin-go-opentracing/propagation_ot.go understands the issue I'm talking about.
Any idea on how to resolve this discrepancy between zipkin instruments implemented by different languages?

thanks

It reports

# github.com/openzipkin/zipkin-go-opentracing/wire
../../../github.com/openzipkin/zipkin-go-opentracing/wire/wire.pb.go:31: undefined: proto.GoGoProtoPackageIsVersion2
../../../github.com/openzipkin/zipkin-go-opentracing/wire/wire.pb.go:98: undefined: proto.RegisterType
../../../github.com/openzipkin/zipkin-go-opentracing/wire/wire.pb.go:624: undefined: proto.RegisterFile

I was trying to setup zipkin in my production environment. Am using c# and golang clients of zipkin (zipkin4net & zipkin-go-opentracing).
Also have setup kafka as the collector and cassandra as the storage.
As zipkin is just used as a monitoring tool I would want it to work for small intervals.
The problem is that while testing I stopped kafka for sometime to see if the apps are working fine. The c# apps kept working properly but the one with golang became very slow after submitting some spans (probably because it could not reach kafka).

Also it occasionally gives a panic error during sending the span to zipkin in the RecordSpan function of zipkin-recorder.go file. Its the last line of the function.

_ = r.collector.Collect(span)

I think the spans should be dropped if kafka is not reachable.

Am using kafka 0.10.1

If I set tag by following code:

span.SetTag("http.status_code", 200)

then I can observe it in zipkin UI (as span tags), but search by query "http.status_code=200" find nothing, and I can not construct any query to find this (or similar) span.

If I set tag by following code

span.SetTag("http.status_code", "200")

then I can find it by in zipkin UI by query "http.status_code=200"

Via Go kit.

# github.com/openzipkin/zipkin-go-opentracing/thrift/gen-go/scribe
../../openzipkin/zipkin-go-opentracing/thrift/gen-go/scribe/scribe.go:282:14: not enough arguments in call to oprot.Flush
	have ()
	want (context.Context)
../../openzipkin/zipkin-go-opentracing/thrift/gen-go/scribe/scribe.go:299:16: not enough arguments in call to oprot.Flush
	have ()
	want (context.Context)
../../openzipkin/zipkin-go-opentracing/thrift/gen-go/scribe/scribe.go:312:16: not enough arguments in call to oprot.Flush
	have ()
	want (context.Context)
../../openzipkin/zipkin-go-opentracing/thrift/gen-go/scribe/scribe.go:326:24: not enough arguments in call to oprot.Flush
	have ()
	want (context.Context)

Currently https://github.com/apache/thrift is empty:

Please import from git.apache.org/thrift.

https://github.com/openzipkin/zipkin-go-opentracing/blob/6bb822a7f15fdc5800b9822a6ac1bfa0b7d9195d/propagation_ot.go#L29

However for PCF zipkin tracing, it only injects two headers X-B3-TraceId and X-B3-SpanId for the first hop api endpoint. [https://docs.cloudfoundry.org/concepts/http-routing.html#-http-headers-for-zipkin-tracing]

https://zipkin.io/pages/instrumenting.html indicates that true/false should be serialized as '1' and '0' respectively. Currently, they are serialized as 'true' and 'false'.

Hello,

Could you please tell me, is there any way to add autorizhation headers in http-collector. When making requests to zipkin tool I need to send 2 headers with my request, how to add them?

Given that we have a core repository zipkin-go this should be move to openzipkin-contrib in order to have clarity about what is the zipkin core and what is not.

Ping @openzipkin/core

goroutine profile: total 231
206 @ 0x4034f04 0x4034fc3 0x40464e3 0x404626d 0x406e3bd 0x4b5e5b4 0x4065341

0x404626c sync.runtime_SemacquireMutex+0x3c /usr/local/go/src/runtime/sema.go:71

0x406e3bc sync.(*Mutex).Lock+0x25c /usr/local/go/src/sync/mutex.go:134

0x4b5e5b3 github.com/openzipkin/zipkin-go-opentracing.(*HTTPCollector).send+0x53 /Users/heldiam/Developer/GO/src/github.com/openzipkin/zipkin-go-opentracing/collector-http.go:192

==================================================================

goroutine profile: total 254
229 @ 0x4034f04 0x4034fc3 0x40464e3 0x404626d 0x406e3bd 0x4b5e5b4 0x4065341

0x404626c sync.runtime_SemacquireMutex+0x3c /usr/local/go/src/runtime/sema.go:71

0x406e3bc sync.(*Mutex).Lock+0x25c /usr/local/go/src/sync/mutex.go:134

0x4b5e5b3 github.com/openzipkin/zipkin-go-opentracing.(*HTTPCollector).send+0x53 /Users/heldiam/Developer/GO/src/github.com/openzipkin/zipkin-go-opentracing/collector-http.go:192

==================================================================

goroutine profile: total 262
237 @ 0x4034f04 0x4034fc3 0x40464e3 0x404626d 0x406e3bd 0x4b5e5b4 0x4065341

0x404626c sync.runtime_SemacquireMutex+0x3c /usr/local/go/src/runtime/sema.go:71

0x406e3bc sync.(*Mutex).Lock+0x25c /usr/local/go/src/sync/mutex.go:134

0x4b5e5b3 github.com/openzipkin/zipkin-go-opentracing.(*HTTPCollector).send+0x53 /Users/heldiam/Developer/GO/src/github.com/openzipkin/zipkin-go-opentracing/collector-http.go:192

why goroutine only increase without decreasing

When upstream server is causing client errors on c.client.Do, HTTPCollector logs an error on every request that fails.

To my opinion this is too verbose. The client may have a state and can log only when the state changes.
This is clear enough and don't cause log spamming when upstream server is in error state.

Please give your opinion on this.

Cheers

project needs improved readme with examples of zipkin tracer specific settings like ClientServerSameSpan

When using zipkin.NewTracer(zipkin.NewRecorder(collector, false, hostPort, svcName))
Inside the log-materializers.go, on the following functions:

// MaterializeWithLogFmt converts log Fields into LogFmt string
func MaterializeWithLogFmt(logFields []log.Field) ([]byte, error) {
	var (
		buffer  = bytes.NewBuffer(nil)
		encoder = logfmt.NewEncoder(buffer)
	)
	for _, field := range logFields {
		if err := encoder.EncodeKeyval(field.Key(), field.Value()); err != nil {
			encoder.EncodeKeyval(field.Key(), err.Error())
		}
	}
	return buffer.Bytes(), nil
}

// StrictZipkinMaterializer will only record a log.Field of type "event".
func StrictZipkinMaterializer(logFields []log.Field) ([]byte, error) {
	for _, field := range logFields {
		if field.Key() == "event" {
			return []byte(fmt.Sprintf("%+v", field.Value())), nil
		}
	}
	return nil, errEventLogNotFound
}

I am getting the following error:
field.Key undefined (type "vendor/github.com/opentracing/opentracing-go/log".Field has no field or method Key, but does have "vendor/github.com/opentracing/opentracing-go/log".key)

This makes sense because as I look at the []log.Field, I see that the Field struct does not have a method called Key() which is exposing the key...

At the moment git.apache.org/thrift.git/lib/go/thrift seems down, while it's mirror https://github.com/apache/thrift is up. I'm not sure if the mirror is legitimate, but at least it's up.

Currently I have no way of solving a Go dependency situation with this reference in place.