Can't write 32 bit unsigned ints about nimpng

[jangko]

I don't know your programming(language) background, sorry if my explanation contains any unpleasant words.

Actually your question falls into Nim programming language domain and not specific to nimPNG, but I'll try to answer your question, hopefully thoroughly.

var 
  data1 = @[
    0xFF.uint8, 0xFF.uint8, 0xFF.uint8, 0x80.uint8, # 50% white
    0xFF.uint8, 0x00.uint8, 0x00.uint8, 0xFF.uint8, # red
    0x00.uint8, 0xFF.uint8, 0x00.uint8, 0xFF.uint8, # green
    0x00.uint8, 0x00.uint8, 0xFF.uint8, 0xFF.uint8  # blue
  ]

  data2 = @[
    0xFFFFFF80.uint32, # 50% white
    0xFF0000FF.uint32, # red
    0x00FF00FF.uint32, # green
    0x0000FFFF.uint32  # blue
  ]
  
  px1: string = cast[string](data1)
  px2: string = cast[string](data2)
  
echo px1.len
echo px2.len

the code above will produce 16 and 4, why? because you cannot cast seq to string in Nim, that is a very dangerous thing to do. The program not crash because you are lucky.

Nim manual says: Type casts are a crude mechanism to interpret the bit pattern of an expression as if it would be of another type. Type casts are only needed for low-level programming and are inherently unsafe.

You cannot force seq become string, unlike int32 become uint32 or char into uint8. your first example works because Nim compiler generate almost the same data structure for both seq and string, but the second one, the Nim compiler will generate similar but different data structure. Also, the meaning of field len of seq is different from string's. You also need to remember that both seq and string are garbage collected objects by default, cast probably will cause problems on long running program, although it works in your first example.

Another thing is endianess, 0x00FF00FF.uint32 is not the same as 0x00.uint8, 0xFF.uint8, 0x00.uint8, 0xFF.uint8 on little endian machines, for example:

import nimPNG

var 
  data = @[
    0xFFFFFF80.uint32, 
    0xFF0000FF.uint32, 
    0x00FF00FF.uint32,
    0x0000FFFF.uint32
  ]
  
  pixels = newString(data.len * sizeof(uint32))

copyMem(pixels.cstring, data[0].addr, pixels.len)
# or copyMem(pixels[0].addr, data[0].addr, pixels.len)
echo "saved: " & $savePNG32("test.png", pixels, 2, 2)

it will generate not white 50%, red, green, blue, but cyan, red, transparent, transparent.

this is the correct bit pattern if you want to use uint32

  data = @[
    0x80FFFFFF.uint32, # 50% white
    0xFF0000FF.uint32, # red
    0xFF00FF00.uint32, # green
    0xFFFF0000.uint32  # blue
  ]

although using copyMem works, remember that any operation involving raw pointers is dangerous if not handled properly. use it with cautions.

If you use js backend and not C backend, copyMem will not work.

Feel free to ask if you think my explanation is not clear enough.

[define-private-public]

Thanks for the response. I'd consider myself pretty good with C/C++, but I'm still a novice with Nim. This explains quite a bit for me.

Would you recommend then that I write all of my bytes to a dynamically allocated ptr type, then cast that to a string, write it out (using savePNG()) and then cleanup the dynamically allocated memory?

[jangko]

Would you recommend then that I write all of my bytes to a dynamically allocated ptr type, then cast that to a string, write it out (using savePNG()) and then cleanup the dynamically allocated memory?

I don't know the situation that drive you to think that ptr is necessary, are you interfacing with C library or something like that?

btw, you cannot just cast ptr into string. Nim string is a garbage collected object, more like Python string than C string. Nim string can be converted to cstring/ptr but cstring not always can be converted to Nim string

you can write the bytes directly into the string itself using escape sequence such as:

var pixels = "\xff\xaa\xbb\xcc"

or, if you do it dynamically:

# assume bytes_per_pixel is 3
# allocate string with w*h*bytes_per_pixel
var pixels = newString(w*h*bytes_per_pixel)
for i in 0.. <w*h:
  pixels[i * bytes_per_pixel + 0] = 0xFF.chr  #red
  pixels[i * bytes_per_pixel + 1] = 0x00.chr
  pixels[i * bytes_per_pixel + 2] = 0x00.chr

# using preallocated string
var pixels = newStringOfCap(w*h*bytes_per_pixel)
for i in 0.. <w*h:
   pixels.add chr(0xFF)
   pixels.add chr(0x00)
   pixels.add chr(0x00)

Nim string is a versatile container and great for text, binary content, UTF-8 encoded string, and many more. I came from C++ too, but when using Nim, allocating memory manually is a very rare occasion.

[define-private-public]

Alright, that clears things up for me. Thanks!

I'm doing some stuff with image manipulation and computer graphics. I am interfacing with a few C libraries.

I'd really recommend putting this stuff somewhere in the documentation or writing a "how to create pngs," doc for this repo.

[jangko]

I'm doing some stuff with image manipulation and computer graphics. I am interfacing with a few C libraries.

I see. If that is the case, I still recommend you to use Nim string, and if you need to pass it to C library, simply use .cstring and .len for your data.

proc some_c_func(data: cstring, len: cint) {.cdecl, importc: "some_c_func".}
var s = "some data"
some_c_func(s.cstring, s.len)

void some_c_func(char* data, int len) {
/* do something */
}

I'd really recommend putting this stuff somewhere in the documentation or writing a "how to create pngs," doc for this repo.

yeah, I agree, but unfortunately very busy. PRs are welcome.

[define-private-public]

Sure, I can do a PR. Do you think the sample I provided (if adjusted to use the proper methods) would be fine? It's simple enough, but showcases how to use it.

[jangko]

Do you think the sample I provided (if adjusted to use the proper methods) would be fine?

yes