bug: images not displaying in front end (self hosted, kubernetes) about penpot

[suityou01]

I have managed to do some more digging and have learned the following

For clarity the load balancer for the pod is exposed on 192.168.1.178

When I upload a file from the front end, there are 3 requests made by the browser

First a POST to http://192.168.1.178/api/rpc/command/upload-file-media-object
The payload for this POST is as follows

file-id: bee42537-d365-80b3-8004-7fb77cec7be8
name: pi
is-local: true
content: (binary)

This returns an HTTP 200 and the following JSON

[
    "^ ",
    "~:file-id",
    "~ubee42537-d365-80b3-8004-7fb77cec7be8",
    "~:mtype",
    "image/jpeg",
    "~:name",
    "pi",
    "~:width",
    900,
    "~:media-id",
    "~u09dd895c-9272-497d-8836-ce7de100dc16",
    "~:is-local",
    true,
    "~:id",
    "~ubee42537-d365-80b3-8004-7fce3ddd9c22",
    "~:thumbnail-id",
    "~uf1ac2cef-d312-44c0-a1a7-0fda41f6735d",
    "~:created-at",
    "~m1718291567482",
    "~:height",
    820
]

The next request is a GET to

http://192.168.1.178/assets/by-file-media-id/bee42537-d365-80b3-8004-7fce3ddd9c22

This returns a 404

Finally a POST to

http://192.168.1.178/api/rpc/command/update-file?id=08b2f146-f406-8196-8004-7eded6a12b94

with a JSON payload like this

{"~:id":"~u08b2f146-f406-8196-8004-7eded6a12b94","~:revn":21,"~:session-id":"~u7bd03d31-2ac9-808a-8004-7faab26f527c","~:changes-with-metadata":[{"~:tags":{"~#set":[]},"~:file-id":"~u08b2f146-f406-8196-8004-7eded6a12b94","~:hint-origin":"~:app.main.data.workspace.shapes/add-shape","~:frames":{"~#set":[]},"~:hint-events":[":app.main.data.workspace.undo/check-open-transactions",":app.main.data.workspace/show-context-menu",":app.main.data.workspace/show-shape-context-menu",":app.main.data.workspace.viewport/finish-panning",":app.main.data.workspace.zoom/finish-zooming",":app.main.data.workspace.undo/check-open-transactions",":app.main.data.workspace.undo/start-undo-transaction",":app.main.data.workspace.changes/update-shapes",":app.main.data.workspace.shapes/update-shape-flags",":app.main.data.workspace.undo/start-undo-transaction",":app.main.data.comments/detach-comment-thread",":app.main.data.workspace.changes/update-indices",":app.main.data.workspace.undo/append-undo",":app.main.data.workspace.shapes/delete-shapes",":app.main.data.workspace.changes/commit-changes",":layout/update",":app.main.data.workspace.undo/start-undo-transaction",":app.main.data.workspace.guides/move-frame-guides",":app.main.data.workspace.comments/move-frame-comment-threads",":app.main.data.workspace.changes/update-shapes",":app.main.data.workspace.modifiers/clear-local-transform",":app.main.data.workspace.undo/commit-undo-transaction",":app.main.data.workspace.modifiers/apply-modifiers",":app.main.data.workspace.shape-layout/update-layout-positions",":app.main.data.workspace.undo/commit-undo-transaction",":app.main.data.workspace.shapes/delete-shapes",":app.main.data.workspace.selection/deselect-all",":app.main.data.workspace/delete-selected",":app.main.data.workspace/hide-context-menu",":app.main.data.workspace.changes/send-update-indices",":app.main.data.workspace.persistence/persist-changes",":app.main.data.workspace.changes/update-indices",":app.main.data.workspace.persistence/shapes-changes-persisted",":app.main.data.workspace.changes/update-indices",":app.main.data.workspace.persistence/shapes-changes-persisted",":app.main.data.workspace.changes/update-indices",":app.main.data.workspace.persistence/shapes-changes-persisted",":app.main.data.workspace.changes/update-indices",":app.main.data.workspace.persistence/shapes-changes-persisted",":app.main.data.workspace.persistence/shapes-changes-persisted-finished",":app.main.data.workspace.changes/send-update-indices",":app.main.data.workspace.edition/clear-edition-mode",":app.main.data.workspace/workspace-focus-lost",":app.main.data.messages/show",":app.main.data.workspace.media/process-media-objects",":app.main.data.workspace/workspace-focus-lost",":app.main.data.workspace.undo/start-undo-transaction",":app.main.data.workspace.changes/update-indices",":app.main.data.workspace.undo/append-undo",":app.main.data.workspace.shapes/add-shape",":app.main.data.workspace.changes/commit-changes"],"~:page-id":"~u08b2f146-f406-8196-8004-7eded6a16c53","~:id":"~u7bd03d31-2ac9-808a-8004-7fab4ec9a410","~:changes":[{"~:type":"~:add-obj","~:id":"~u7bd03d31-2ac9-808a-8004-7fab4ec7416a","~:page-id":"~u08b2f146-f406-8196-8004-7eded6a16c53","~:parent-id":"~u00000000-0000-0000-0000-000000000000","~:frame-id":"~u00000000-0000-0000-0000-000000000000","~:index":null,"~:ignore-touched":false,"~:obj":{"~#shape":{"~:y":-288,"~:rx":0,"~:transform":{"~#matrix":{"~:a":1,"~:b":0,"~:c":0,"~:d":1,"~:e":0,"~:f":0}},"~:rotation":0,"~:name":"pi","~:width":900,"~:type":"~:rect","~:points":[{"~#point":{"~:x":-115,"~:y":-288}},{"~#point":{"~:x":785,"~:y":-288}},{"~#point":{"~:x":785,"~:y":532}},{"~#point":{"~:x":-115,"~:y":532}}],"~:proportion-lock":true,"~:transform-inverse":{"~#matrix":{"~:a":1,"~:b":0,"~:c":0,"~:d":1,"~:e":0,"~:f":0}},"~:id":"~u7bd03d31-2ac9-808a-8004-7fab4ec7416a","~:parent-id":"~u00000000-0000-0000-0000-000000000000","~:frame-id":"~u00000000-0000-0000-0000-000000000000","~:strokes":[],"~:x":-115,"~:proportion":1.0975609756097562,"~:selrect":{"~#rect":{"~:x":-115,"~:y":-288,"~:width":900,"~:height":820,"~:x1":-115,"~:y1":-288,"~:x2":785,"~:y2":532}},"~:fills":[{"~:fill-opacity":1,"~:fill-image":{"~:name":"pi","~:width":900,"~:height":820,"~:mtype":"image/jpeg","~:id":"~u073867e1-d225-80aa-8004-7fab4ebd98a1","~:keep-aspect-ratio":true}}],"~:flip-x":null,"~:ry":0,"~:height":820,"~:flip-y":null}}},{"~:type":"~:mov-objects","~:parent-id":"~u00000000-0000-0000-0000-000000000000","~:page-id":"~u08b2f146-f406-8196-8004-7eded6a16c53","~:shapes":["~u7bd03d31-2ac9-808a-8004-7fab4ec7416a"]}],"~:undo-group":"~u7bd03d31-2ac9-808a-8004-7fab4ec86e81","~:save-undo?":true,"~:stack-undo?":false}],"~:features":{"~#set":["layout/grid","styles/v2","components/v2","fdata/shape-data-type"]}}

This returns a 200.

If I run the following query against the database

SELECT * FROM public.file
WHERE Id = 'bee42537-d365-80b3-8004-7fb77cec7be8';

I get one row exactly

So I think the form submission is working ok and storing things in the database, but the retrieval is not working.

[suityou01]

I think this is the relevant backend code for the failing api call, although I'm not much of a closure developer (do use emacs though).

penpot/backend/src/app/http/assets.clj

Line 81 in 7e87362

sobj (sto/get-object storage (kf mobj))]

(defn- generic-handler
  "A generic handler helper/common code for file-media based handlers."
  [{:keys [::sto/storage] :as cfg} request kf]
  (let [pool (::db/pool storage)
        id   (get-id request)
        mobj (get-file-media-object pool id)
        sobj (sto/get-object storage (kf mobj))]
    (if sobj
      (serve-object cfg sobj)
      {::rres/status 404})))

So it could be mobj is null or sobj is null I guess.

SELECT media_id, file_id FROM public.file_media_object
where id = '073867e1-d225-80aa-8004-7fab4ebd98a1';

This returns 1 row so it looks like it's sobj that is returning null.
The media_id is "09dd895c-9272-497d-8836-ce7de100dc16"
The file_id is "08b2f146-f406-8196-8004-7eded6a12b94"

penpot/backend/src/app/storage.clj

Line 156 in 7e87362

(defn get-object

This is where I think the storage object is failing.

Any thoughts? The file contents are stored on the disk at the following path

/opt/data/assets/09/dd/895c9272497d8836ce7de100dc16

So it looks to me like the file storage uses the first 2 bytes of the media_id for the top level folder after /opt/data/assets and the next two bytes for the subfolder then writes the file contents to the disk with the file_id as the file name.

[suityou01]

So why does http://192.168.1.178/assets/by-file-media-id/bee42537-d365-80b3-8004-7fce3ddd9c22

fail?

[suityou01]

Solved it.

The path for assets is shared by front end and back end. It is the same location.

mount this in the same place on the host > /opt/data/assets

e.g.

volumeMounts:
          - name: general-pv-storage
            subPath: penpot/penpot_assets
            mountPath: /opt/data/assets

The subPath needs to be the same for both front end and back end containers.

[suityou01]

Closed

[suityou01]

close