scottmansfield / nanolog Goto Github PK

Let's assume that a call to log looks like nanolog.Printf(string, ....)
I assume (am I wrong?) that golang compiler allocates and inits all strings like C/C++ does.

In the implementation Printf() looks for the string address (unsafe is coming here) in a hashtable. If there is a match we know what the string is. If there is a miss we add the string to the hashtable. Lookup in a custom hashtable tuned for an integer key can take under 20ns. For example we probably can assume that all strings are placed in the same "initialized const data" segment (.text?) in the executable (runtime.moduledata?), share the same base address.

This way we can drop a call to AddLogger() and have a standard API.

The following test, for example, works on 64 bits Linux

func align(v uintptr) uintptr {
	mul := uintptr((1 << 3) - 1)
	return (v + mul) & (^uintptr(mul))
}

func getStringSize(s string) uintptr {
	return align(uintptr(len(s)))
}

func TestStringLocation(t *testing.T) {
	s0 := "Hello, world"
	s1 := "Hello, world2"
	p0 := uintptr(unsafe.Pointer(&s0))
	p1 := uintptr(unsafe.Pointer(&s1))

	p2 := p0 + align(uintptr(len(s0)))
	p3 := p1 + align(uintptr(len(s1)))
	if p1 != p2 && p0 != p3 {
		t.Fatalf("Bad locations %x %x, expected %x %x", p0, p1, p2, p3)
	}
}

var s0 string = "Hello, world"

func TestStringLocationGlobalLocalHeader(t *testing.T) {
	s1 := "Hello, world2"
	hdr0 := (*reflect.StringHeader)(unsafe.Pointer(&s0))
	hdr1 := (*reflect.StringHeader)(unsafe.Pointer(&s1))
	if hdr0.Data+0x100 > hdr1.Data {
		t.Fatalf("Bad locations %x %x", hdr0.Data, hdr1.Data)
	}
}

I need some help pls. I'm streaming data and logging it with nanolog. A bug in my code terminated the code execution, which I then restarted and naturally kept logging/append further. When I do the inflate, panics occur:

{"Name":"John","City":"SAN","t":1677767072106,"age":3,"id":127423134,"tf":5,"c":"R R","bx":"T","ax":"T","z":"A"} {"Name":"Betty","City":"DM","t":1677767072113,"age":1,"id":2423134,"tf":5,"c":"R R","bx":"P","ax":"P","z":"A"} {"Name":"John","City":"SAN","t":1677767072126,"age":3,"id":127423134,"tf":5,"c":"R R","bx":"T","ax":"T","z":"A"}panic: Bad file format

These are last three streamed messages and it seems termination append extra message/characters obscuring the json format and panics. I tried to look into the inflate code, but not much clue (ain't developer and also pretty fresh in Go). How could this be prevented in the first place if code termination happens? How can I fix the file or catch the error as the nano log file is pretty big and all is in one line?

It's really awesome logger, the fastest I've tested for my usecase. However, I'd like to write to multiple files, but I am not sure if that's even possible. Something like this:

nanoLog1, err := os.OpenFile("file1.log", os.O_CREATE|os.O_WRONLY, 0644)
nanolog.SetWriter(nanoLog1)
h1:= nanolog.AddLogger("%s")

nanoLog2, err := os.OpenFile("file2.log", os.O_CREATE|os.O_WRONLY, 0644)
nanolog.SetWriter(nanoLog2)
h2 := nanolog.AddLogger("%s")

defer nanolog.Flush()
	
// --- Logging
nanolog.Log(h1, msg1) 
nanolog.Log(h2, msg2) 

All is logged to the last file. Would it be somehow possible to write to multiple log files? Thank you 🙏🏻

Text generated from inflate -f src nanolog.txt contains alien bytes ( ÿ þ) in the very beginning, which is interfering with Windows powershell renderer.

Try this

	r := bufio.NewReader(getFileReader("nanolog.txt"))

	var count int
	for {
		count++
		b, _, err := r.ReadLine()
		if err != nil {
			break
		}
		if count == 1 {
			p("FIRST TWO BYTES > ", string(b[0]), string(b[1]), string(b[2]))
		}
		fmt.Println(string(b))
	}