art-technologies / generative-platform-standard Goto Github PK

Given our latest discussion about custom titles for editions, I assume it would be great to send additional metadata such as output's features, e.g.:

"generativePlatformStandard": {
    "id": "1337",
    "type": "READY",
    "payload": {
        "traits": [
            {
                "name": "title",
                "value": "Random output with RED SKY #33"
            }
            {
                "name": "sunColor",
                "value": "red"
            }
        ]
    }
}

I assume there shouldn't be an API methid to fetch all possible traits, because there is a chance Project is not aware of all outputs. Therefore list of all various is usually cradfted by Platform based on all generated Project's outputs.

Had a chance to look at the current implementation! Here's some important issues I've seen:

1. Access to contentWindow
   

   generative-platform-standard/getnart-platform.js

   Lines 42 to 46 in 4cf5249

   	get implementsPreviewCaptureTrigger() {
   	return (
   	this.iframe?.contentWindow?.$implementsPreviewCaptureTrigger || false
   	);
   	}

Such code won't work when we deal with different domains (because you can't read window content of another origin in most cases). I've proposed solution to this problem in #1 (point 1).

2. Dealing with multiple projects
   

   generative-platform-standard/getnart-platform.js

   Lines 24 to 30 in 4cf5249

   	if (iframeSelector.contentWindow.$implementsDelegatedLoading) {
   	const strongRef = this;
   	window.addEventListener("message", function (event) {
   	if (event.data === "loading-complete") {
   	strongRef.onLoadingComplete();
   	}
   	});

In case there are more then one Project on page, current window will receive multiple events, and this implementation doesn't distinguish from which partiluar iframe we got the event. I proposed a solution to this problem in #1 (point 2)

Given some Platforms can use sandbox mode on their iframe we should either:

• request all Platforms to include sandbox="... allow-downloads"
• change standard to request Projects to send binary file over postMessage, so Platform can initiate download itself

The Standard looks nice! I think we should keep it simple and target to provide solutions for the most commonly used cases!

Here are some suggestions I would like to discuss:

1. Sending feature flags. The idea of feature flags is great, but the current implementation of setting window of iframe wouldn't work in case iframe has another origin (which I assume is quite a common case):

// parent won't be able to read it in case it has another origin
window.$implementsDelegatedLoading = true;

Therefore we probably want to use postMessage for this purpose:

function initialiseGenerativePlatformStandard(config) {
  const initMessage = {
    generativePlatformStandard: {
      id: "some-id",
      type: "init",
      config,
    }
  }
  window.parent.postMessage(initMessage, "*");
}

So original platform will listen to this event.

2. Sending stringified JSON instead of plain text.
   What do you think about communicating in JSON format instead of plain text? My assumption is that this way is more structured and will give the ability to add payload in the future in a better way. Also, it's easier to distinguish messages within this standard (simply check JSON.parse(event.message).generativePlatformStandard, of cause with try-catch to handle cases when a message is not stringified JSON. So my proposal is to use the format:

{
  "generativePlatformStandard": {
    "id": "SOME ID TO HANDLE MULTIPLE PROJECTS ON SAME PAGE",
    "type": "SOME_TYPE",
    // anything else needed, depending on type
  }
}

You can notice I've added "id" field as well. There should be a way to manage multiple projects in the same window. I assume we can't access the actual URL that emitted this event (we can get only the origin), as well as accessing contentWindow wouldn't work in the case of another origin. So feels like providing an "id" should be the cleanest solution.

3. Checking the origin of the message. Both Platform and Project can receive post messages, probably we should mention that it's possible to check the origin in order to process only requests from specific ones. For Project, I don't expect it to be an issue. But for Platform, I believe there could be poteniatial exploit when some embded iframe (maybe an ad) starts sending lots of such requests. Again, it's quite a corner case, but mentioning this in Docs should be great.

4. Error events. For cases when Project failed to render an output, we should ask to send some event "type": "error", so Platform can show error message.

There are some projects that takes quite long to finish "drawing" process, for example:

• https://verse.works/artworks/0ce696c3-d4e9-4745-ad37-c660600f8d86
• https://verse.works/artworks/280581b0-aaa1-4419-b390-a19a17917e7c

I do understand that sometimes it's just due to performance issues. Although would be nice to give ability for Platform to send a signal to Project indicating that we want to speed up render (if possible) to see the final version ASAP.

Actual use case: "Speed up" button on Artwork/Edition page to see final output (in case Project supports such speed up)