hi-rez / satin Goto Github PK

Which versions can be used in the production environment, or how long is the production version estimated?

Is it possible to easily send textures around using Syphon with Satin? On apple sillicon, there are currently no easy options (besides unity) for a creative coding framework that both supports access to both compute shaders and syphon

Is there background rendering, not screen demo, API, and parallel processing possible at the same time?

I tried using the default PhysicalMaterial and got this error:

Physical Shader: program_source:985:5: error: no matching function for call to 'pbrDirectLighting'
    pbrDirectLighting(pixel, lights);
    ^~~~~~~~~~~~~~~~~
program_source:673:6: note: candidate function not viable: no known conversion from 'const constant Light' to 'const constant Light *' for 2nd argument; take the address of the argument with &
void pbrDirectLighting(thread PixelInfo &pixel, constant Light *lights)
     ^

There appears to be a missing ampersand on this line in the Physical Shader.

Other calls to pbrDirectLighting within this repo use an ampersand, such as this one and this one.

I cloned Stain, added an ampersand to that line, added it as a local swift package and the error went away.

combineOuterAndInnerPaths leaks?

What is triangulation algorithm using in Satin?

Implementing lights & shadows into the framework. Let me know if you want to delegate this work!

AR background is way too bright. Sorry, should be a pull request, but not sure it's perfect.

I think you're meant to change the shader in SatinSceneKitARRenderer to:

    // don't return ycbcrToRGBTransform * ycbcr;
    
    float4 color = ycbcrToRGBTransform * ycbcr;
    // Flatten Color Space to rgb?
    return float4(pow(color.rgb, float3(2,2,2)), color.a); 

Very cool project.

Why are MSL resource files not integrated into 'Group' files, but packaged in the form of resource drops? Are there any subsequent situations to be considered in the design? Or is the original intention of such a design in consideration of aspects?

It looks like there's an issue with the ffi gem not supporting the M1 chip yet, which causes an issue when you initially setup the examples and try to run

bundle exec pod install

I get the following error:

Analyzing dependencies
Pre-downloading: `Forge` from `https://github.com/Hi-Rez/Forge.git`, commit `4d1c7760cb12e129fb027e4e250037a8bed520a3`
/Users/USER/Desktop/code/Satin/Example/vendor/bundle/ruby/2.6.0/gems/ffi-1.15.0/lib/ffi/library.rb:275: [BUG] Bus Error at 0x0000000100cb4000
ruby 2.6.3p62 (2019-04-16 revision 67580) [universal.arm64e-darwin20]

And relevant crash report info:

Process:               ruby [1268]
Path:                  /System/Library/Frameworks/Ruby.framework/Versions/2.6/usr/bin/ruby
Identifier:            ruby
Version:               145.100.1
Code Type:             ARM-64 (Native)
Parent Process:        zsh [1107]
Responsible:           Terminal [519]
User ID:               501

PlugIn Path:             /Users/USER/Desktop/*/ffi_c.bundle
PlugIn Identifier:       ffi_c.bundle
PlugIn Version:          ??? (0)

Date/Time:             2021-04-29 14:16:58.436 -0500
OS Version:            macOS 11.3 (20E232)
Report Version:        12
Anonymous UUID:        72AB6060-84EC-29E6-9538-A204D5EB2E7D


Time Awake Since Boot: 450 seconds

System Integrity Protection: enabled

Crashed Thread:        0  Dispatch queue: com.apple.main-thread

Exception Type:        EXC_BAD_ACCESS (SIGABRT)
Exception Codes:       KERN_PROTECTION_FAILURE at 0x0000000100cb4000
Exception Note:        EXC_CORPSE_NOTIFY

I'm going to try the suggestions on the StackOverflow thread, but wanted to document the issue here as well.

In the actual scene, multiple mesh models are often added to the object, and multiple mesh models are exported as a whole?

Performance improvement in changing and refreshing UI interface for dynamic addition and deletion of mesh

Hi @rezaali,
I'm playing with the lookAt function and I can't get it to work as I expect it – meaning I would like the cones to point to each other by the tip. But also at the middle crossing point they "flip". Do you know how to solve this?
thx

import MetalKit
import SwiftUI

import Forge
import Satin

class MKTest1: Forge.Renderer
{
    var context: Context!
    var renderer: Satin.Renderer!
    var camera: PerspectiveCamera!
    var scene: Object = .init("Scene")

    var coneMesh1 = Mesh(geometry: ConeGeometry(size: (radius: 1, height: 3),
                                                res: (angular: 8, radial: 1, vertical: 1)),
                         material: BasicDiffuseMaterial(0.75))

    var coneMesh2 = Mesh(geometry: ConeGeometry(size: (radius: 1, height: 3),
                                                res: (angular: 8, radial: 1, vertical: 1)),
                         material: BasicDiffuseMaterial(0.5))

    var time: Float = 0.0

    override func setup()
    {
        context = Context(device, sampleCount, colorPixelFormat, depthPixelFormat, stencilPixelFormat)
        camera = PerspectiveCamera(position: [0, 0, 35], near: 0.01, far: 100.0)
        renderer = Satin.Renderer(context: context, scene: scene, camera: camera)
        renderer.setClearColor([0, 0, 0, 1])

        coneMesh1.position = [0, 0, 0]
        coneMesh2.position = [10, 0, 0]

        scene.add(coneMesh1)
        scene.add(coneMesh2)
    }

    override func update()
    {
        time += 0.01
        let sx = sin(time)
        let sy = cos(time * 5)
        let sz = sin(time)

        coneMesh2.position = [sx * 10, sy * 10, sz * 5]

        coneMesh1.material?.set("Color", [abs(sx), abs(sy), abs(sx + sy), 1.0])

        coneMesh1.lookAt(coneMesh2.position)
        coneMesh2.lookAt(coneMesh1.position)
    }

    override func draw(_ view: MTKView, _ commandBuffer: MTLCommandBuffer)
    {
        guard let renderPassDescriptor = view.currentRenderPassDescriptor else { return }
        renderer.draw(renderPassDescriptor: renderPassDescriptor, commandBuffer: commandBuffer)
    }

    override func resize(_ size: (width: Float, height: Float))
    {
        camera.aspect = size.width / size.height
        renderer.resize(size)
    }
}

struct MKTest1ContentView: View
{
    var body: some View
    {
        ForgeView(renderer: MKTest1())
    }
}

What editor is the editor for, or is it possible to develop it later?

From the Apple documentation on ARAnchorCopying:

Because anchor objects are immutable, ARKit must copy them to make changes from one ARFrame to the next.

Therefore, the transform values here in Satin are never getting updated (they stay the same):

fileprivate class ARObject: Object {
…
  override func update(camera: Camera, viewport: simd_float4) {
    guard let anchor = anchor else { return }
    self.worldMatrix = anchor.transform
    super.update(camera: camera, viewport: viewport)
  }

ARKit appears to require us to use the
ARSessionDelegate.session(_ session: ARSession, didUpdate anchors: [ARAnchor])
method to get updated transform values.

When I run the sample PBR-iOS on an iPhone 12 Pro the following error appear in the console:

Diffuse - Level: 0 - Size: 64
2022-06-07 14:59:04.084563+0200 Latest PBR[644:26983] Execution of the command buffer was aborted due to an error during execution. Internal Error (0000000e:Internal Error)
2022-06-07 14:59:04.084625+0200 Latest PBR[644:26983] Execution of the command buffer was aborted due to an error during execution. Internal Error (0000000e:Internal Error)
Diffuse - Level: 1 - Size: 32
2022-06-07 14:59:04.119867+0200 Latest PBR[644:26981] Execution of the command buffer was aborted due to an error during execution. Internal Error (0000000e:Internal Error)
2022-06-07 14:59:04.120190+0200 Latest PBR[644:26981] Execution of the command buffer was aborted due to an error during execution. Internal Error (0000000e:Internal Error)
Diffuse - Level: 2 - Size: 16
2022-06-07 14:59:04.181093+0200 Latest PBR[644:26983] Execution of the command buffer was aborted due to an error during execution. Internal Error (0000000e:Internal Error)
2022-06-07 14:59:04.181260+0200 Latest PBR[644:26983] Execution of the command buffer was aborted due to an error during execution. Internal Error (0000000e:Internal Error)
Diffuse - Level: 3 - Size: 8
Diffuse - Level: 4 - Size: 4
Diffuse - Level: 5 - Size: 2
Diffuse - Level: 6 - Size: 1

Specular - Level: 0 - Size: 512
2022-06-07 14:59:04.281598+0200 Latest PBR[644:26981] Execution of the command buffer was aborted due to an error during execution. Internal Error (0000000e:Internal Error)
2022-06-07 14:59:04.281868+0200 Latest PBR[644:26981] Execution of the command buffer was aborted due to an error during execution. Internal Error (0000000e:Internal Error)
Specular - Level: 1 - Size: 256
2022-06-07 14:59:04.315465+0200 Latest PBR[644:26983] Execution of the command buffer was aborted due to an error during execution. Internal Error (0000000e:Internal Error)
2022-06-07 14:59:04.315655+0200 Latest PBR[644:26983] Execution of the command buffer was aborted due to an error during execution. Internal Error (0000000e:Internal Error)
Specular - Level: 2 - Size: 128
2022-06-07 14:59:04.336547+0200 Latest PBR[644:26981] Execution of the command buffer was aborted due to an error during execution. Internal Error (0000000e:Internal Error)
2022-06-07 14:59:04.336725+0200 Latest PBR[644:26981] Execution of the command buffer was aborted due to an error during execution. Internal Error (0000000e:Internal Error)
Specular - Level: 3 - Size: 64
Specular - Level: 4 - Size: 32
Specular - Level: 5 - Size: 16
Specular - Level: 6 - Size: 8
Specular - Level: 7 - Size: 4
Specular - Level: 8 - Size: 2
Specular - Level: 9 - Size: 1

The metallic sphere on the top left seems to have an issue and turns dark:

It is supposed to look like this

Object gained support for observability with conformance to @ObservedObject and a couple of @Published values, which is great as it opens up for supporting a SwiftUI frontend that displays information about the scene.

But the property wrappers were later changed from @Published to a custom @PublishedDidSet:

This custom property wrapper defeats the purpose of conforming to ObservableObject, as the whole purpose of this class is:

ObservableObject
A type of object with a publisher that emits before the object has changed.

If an object does not have any @Published properties, there is no longer any effect in making it an ObservableObject (and thus breaking the ability to use directly in SwiftUI).

If possible I'd suggest to revert these back to @Published. If there is need to observe @Published values in another lib (Youi?) after they have changed, you can always use an async dispatch on the next runloop to ensure the value has been set:

    cancellable = object.$position.sink { [weak self] position in
         // Position not yet set here, self?.object.position is still old
         DispatchQueue.main.async { 
             // Position has been set here, self?.object.position is new
         }
     }

Followed the instructions carefully, but I can't compile and run macOS2D on 10.15.5 with Xcode 11.6

any ideas?

Hey Reza

Building according to instructions (using bundler) goes well:

Downloading dependencies
Installing Forge (0.1.0)
Installing Satin (0.2.0)
Generating Pods project
Integrating client project
Pod installation complete! There are 2 dependencies from the Podfile and 2 total pods installed.

...but then on building the project using xcworkspace file, I get 'No such module 'Forge'' in renderer.swift...is that happening to you too?

I think this might be a Catalina issue:

2020-06-26 10:59:30.385096-0400 LiveCode[18428:4915416] [cas] LaunchedApplication: failed with error -13052 (null){ "ApplicationType"="Foreground", "BundleIdentifierLowerCase"="com.apple.finder", "CFBundleExecutablePath"="/Users/jzting/Downloads/Satin-master/Example/Source/Shared/LiveCode/Assets/Shaders.metal/Contents/MacOS/Finder", "CFBundleExecutablePathDeviceID"=16777221, "CFBundleExecutablePathINode"=1152921500312409048, "CFBundleIdentifier"="com.apple.finder", "CFBundleName"="Finder", "CFBundlePackageType"="FNDR", "CFBundleSignature"="MACS", "LSASN"=ASN:0x0-0x206206:, "LSBundlePath"="/Users/jzting/Downloads/Satin-master/Example/Source/Shared/LiveCode/Assets/Shaders.metal", "LSBundlePathDeviceID"=16777221, "LSBundlePathINode"=8645686135, "LSDisplayName"="Shaders.metal", "LSExecutableFormat"="LSExecutableMachOFormat", "LSLaunchDLabel"="com.apple.Finder", "LSLaunchedByLaunchServices"=true, "LSLaunchedWithLaunchD"=true, "LSLaunchEventRecordTime"=85367951456787, "LSLaunchTime"=now-ish 2020/06/26 10:59:30, "LSParentASN"=ASN:0x0-0x204204:, "LSWantsToComeForwardAtRegistrationTimeKey"=true, "pid"=5092 }

This is summed up in Satin.podspec, but:

• iOS/oSX/tvOS minimum version - iOS is defaulting to 13 right now (which isn't out)
• Cocoa Pods minimum version
• Terminal commands to run/install
• Localized image assets (app icons, etc are causing issues)

I still can't get the example to work (I'm unsure if you need Xcode 11+)

Hi I opened Example.xcodeproj and i'm getting these errors. Also the Package Dependencies aren't loading. Any ideas?

Can TextGeometry support Chinese?

Log: Triangulation for 轛 FAILED!

let input = "車轛"
let geo = TextGeometry(text: input, fontName: "PingFangSC-Regular", fontSize: 8)

seams little problem with new index of triangle. happens only in PingFangSC-Light and PingFangSC-Regular, but works well in PingFangSC-Medium and PingFangSC-Semibold.

PingFangSC-Semibold is right, I don't know why

Hi Reza, thanks for sharing this amazing repo!

Have you thought of adding a ray tracer?
https://developer.apple.com/videos/play/wwdc2021/10149/

Or maybe integrate with Blender cycles Metal version?

cheers!

To compare my own engine i've used Pisa HDR and compare to real world location where pisa tower is on the right not on the left.

Same thing can be observed with brown studio hdr you are using as window is on the right side of the bed not left https://polyhaven.com/a/brown_photostudio_02