Coder Social home page Coder Social logo

Comments (34)

simonw avatar simonw commented on May 28, 2024 1

Turned this all into a TIL: https://til.simonwillison.net/electron/sign-notarize-electron-macos

from datasette-app.

simonw avatar simonw commented on May 28, 2024

https://kilianvalkhof.com/2019/electron/notarizing-your-electron-application/ looks useful.

from datasette-app.

simonw avatar simonw commented on May 28, 2024

Fantastic clues in this comment by @nikvdp

  • the entitlements.plist file and associated config in electron-builder.yml allow the bundled python interpreter to inherit the codesigning config of the parent electron app. This ended up being super important for us cause it fixed the main issue we ran into in our project, where macos' security prevented the electron app from launching our bundled interpreter. With these entitlements the Electron app's own signature can be inherited by the bundled python interpreter allowing the electron app to launch it succesfully.

from datasette-app.

simonw avatar simonw commented on May 28, 2024

Nik's entitlements file: https://github.com/nikvdp/datasette.app/blob/fe75729e45f91f32c85f547c477bc19a48387fbb/build/entitlements.mac.plist

And electron-builder.yml (I don't have one of those yet): https://github.com/nikvdp/datasette.app/blob/fe75729e45f91f32c85f547c477bc19a48387fbb/electron-builder.yml#L21-L34

from datasette-app.

simonw avatar simonw commented on May 28, 2024

Via https://apple.stackexchange.com/a/52680 I learned the codesign -d --entitlements :- /path/to/App.app command:

datasette-app % codesign -d --entitlements :- dist/mac/Datasette.app
Executable=/Users/simon/Dropbox/Development/datasette-app/dist/mac/Datasette.app/Contents/MacOS/Datasette
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
  <dict>
    <!-- https://github.com/electron/electron-notarize#prerequisites -->
    <key>com.apple.security.cs.allow-jit</key>
    <true/>
    <key>com.apple.security.cs.allow-unsigned-executable-memory</key>
    <true/>
    <!-- https://github.com/electron-userland/electron-builder/issues/3940 -->
    <key>com.apple.security.cs.disable-library-validation</key>
    <true/>
  </dict>
</plist>

electron-userland/electron-builder#3940 is interesting, including some debate over whether com.apple.security.cs.disable-library-validation is still the right thing to do with Big Sur. I'll cross that bridge when I come to it.

from datasette-app.

simonw avatar simonw commented on May 28, 2024

I saved the following to build/entitlements.mac.plist:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
  <dict>
    <key>com.apple.security.cs.allow-dyld-environment-variables</key>
    <true/>
    <key>com.apple.security.cs.disable-library-validation</key>
    <true/>
    <key>com.apple.security.cs.allow-jit</key>
    <true/>
    <key>com.apple.security.cs.allow-unsigned-executable-memory</key>
    <true/>
    <key>com.apple.security.cs.debugger</key>
    <true/>
    <key>com.apple.security.network.client</key>
    <true/>
    <key>com.apple.security.network.server</key>
    <true/>
    <key>com.apple.security.files.user-selected.read-only</key>
    <true/>
  </dict>
</plist>

And ran npm run dist - it asked me for my password a bunch of times (until I clicked "Always Allow"):

datasette-app_—_codesign_◂_npm_run_dist_TMPDIR__var_folders_wr_hn3206rs1yzgq3r49bz8nvnh0000gn_T__XPC_FLAGS_0x0_TERM_PROGRAM_VERSION_433_—_170×72_and_datasette-app_—_-zsh_—_151×65

The build succeeded - but I'm worried that I didn't set "hardenedRuntime": true in the "mac" section of package.json first as described in https://kilianvalkhof.com/2019/electron/notarizing-your-electron-application/

Testing the build is annoying - you have to apparently upload it and download it again, or AirDrop it to yourself! https://developer.apple.com/library/archive/documentation/Security/Conceptual/CodeSigningGuide/Procedures/Procedures.html#//apple_ref/doc/uid/TP40005929-CH4-SW26

from datasette-app.

simonw avatar simonw commented on May 28, 2024

Instead I did:

cd dist/mac
ditto -c -k --keepParent Datasette.app Datasette.app.zip

Then uploaded that zip to an S3 bucket, downloaded it over HTTPS, unzipped it and opened the app. It opened without any warnings!

Also, the entitlements seem to have stuck:

/tmp % codesign -d --entitlements :- Datasette.app 
Executable=/private/tmp/Datasette.app/Contents/MacOS/Datasette
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
  <dict>
    <key>com.apple.security.cs.allow-dyld-environment-variables</key>
    <true/>
    <key>com.apple.security.cs.disable-library-validation</key>
    <true/>
    <key>com.apple.security.cs.allow-jit</key>
    <true/>
    <key>com.apple.security.cs.allow-unsigned-executable-memory</key>
    <true/>
    <key>com.apple.security.cs.debugger</key>
    <true/>
    <key>com.apple.security.network.client</key>
    <true/>
    <key>com.apple.security.network.server</key>
    <true/>
    <key>com.apple.security.files.user-selected.read-only</key>
    <true/>
  </dict>
</plist>

Next step: see if I can run that in CI.

from datasette-app.

simonw avatar simonw commented on May 28, 2024

I tried downloading the artifact from that last commit build but got this on launch:

Add_entitlements__refs__50_·_simonw_datasette-app_b5d2d0c_and_Notarizing_your_Electron_application___Kilian_Valkhof

from datasette-app.

simonw avatar simonw commented on May 28, 2024

Based on https://kilianvalkhof.com/2019/electron/notarizing-your-electron-application/

I ran npm install electron-notarize --save-dev to install electron-notarize.

I added this as scripts/notarize.js:

/* Based on https://kilianvalkhof.com/2019/electron/notarizing-your-electron-application/ */

const { notarize } = require("electron-notarize");

exports.default = async function notarizing(context) {
  const { electronPlatformName, appOutDir } = context;
  if (electronPlatformName !== "darwin") {
    return;
  }

  const appName = context.packager.appInfo.productFilename;

  return await notarize({
    appBundleId: "io.datasette.app",
    appPath: `${appOutDir}/${appName}.app`,
    appleId: process.env.APPLEID,
    appleIdPassword: process.env.APPLEIDPASS,
  });
};

Then I ran this:

[email protected] \
   APPLEIDPASS=... \
   CSC_KEY_PASSWORD=... \
   CSC_LINK=$(openssl base64 -in /Users/simon/Dropbox/DatasetteDesktopCertificates/Developer-ID-Application-Certificates.p12) \
   npm run dist

Got this error:

  • packaging       platform=darwin arch=x64 electron=13.3.0 appOutDir=dist/mac
  • signing         file=dist/mac/Datasette.app identityName=Developer ID Application: Simon Willison (762G34JSDR) identityHash=4CBAE9A14BAE09B41C3F24895927698FB1F9A970 provisioningProfile=none
  ⨯ Failed to upload app to Apple's notarization servers

xcrun: error: unable to find utility "altool", not a developer tool or in PATH
  failedTask=build stackTrace=Error: Failed to upload app to Apple's notarization servers
                                                                                                                                             xcrun: error: unable to find utility "altool", not a developer tool or in PATH
                                                                                                                                                 at Object.<anonymous> (/Users/simon/Dropbox/Development/datasette-app/node_modules/electron-notarize/src/legacy.ts:67:13)
    at Generator.next (<anonymous>)
    at fulfilled (/Users/simon/Dropbox/Development/datasette-app/node_modules/electron-notarize/lib/legacy.js:4:58)
    at processTicksAndRejections (node:internal/process/task_queues:96:5)

from datasette-app.

simonw avatar simonw commented on May 28, 2024

https://developer.apple.com/forums/thread/118045 showed me to run this:

sudo xcode-select --reset

from datasette-app.

simonw avatar simonw commented on May 28, 2024

This time a different error:

  ⨯ Failed to upload app to Apple's notarization servers

2021-09-03 14:52:41.213 altool[16138:363548] *** Error: Unable to notarize app.
2021-09-03 14:52:41.214 altool[16138:363548] *** Error: code -1011 (Failed to authenticate for session: (
    "Error Domain=ITunesConnectionAuthenticationErrorDomain Code=-22910 \"Please sign in with an app-specific password. You can create one at appleid.apple.com.\" UserInfo={NSLocalizedRecoverySuggestion=Please sign in with an app-specific password. You can create one at appleid.apple.com., NSLocalizedDescription=Please sign in with an app-specific password. You can create one at appleid.apple.com., NSLocalizedFailureReason=App Store operation failed.}"
) Unable to upload your app for notarization.)
  failedTask=build stackTrace=Error: Failed to upload app to Apple's notarization servers

With wrapping:

"Error Domain=ITunesConnectionAuthenticationErrorDomain Code=-22910 \"Please sign in with an app-specific password. You can create one at appleid.apple.com.\" UserInfo={NSLocalizedRecoverySuggestion=Please sign in with an app-specific password. You can create one at appleid.apple.com., NSLocalizedDescription=Please sign in with an app-specific password. You can create one at appleid.apple.com., NSLocalizedFailureReason=App Store operation failed.}"

from datasette-app.

simonw avatar simonw commented on May 28, 2024

https://support.apple.com/en-us/HT204397 describes app-specific passwords:

App-specific passwords are passwords for your Apple ID that let you sign in to your account and securely access the information you store in iCloud from a third-party app.

...

How to generate an app-specific password

  1. Sign in to your Apple ID account page.
  2. In the Security section, click Generate Password below App-Specific Passwords.

I went there and created an app-specific password called "Notarize Apps" which I saved in the 1Password entry for "[email protected] AppleID".

Manage_your_Apple_ID_-_Apple

from datasette-app.

simonw avatar simonw commented on May 28, 2024

OK, running this again, this time using that new app-specific password for the APPLEIDPASS:

[email protected] \
   APPLEIDPASS=app-specific \
   CSC_KEY_PASSWORD=... \
   CSC_LINK=$(openssl base64 -in /Users/simon/Dropbox/DatasetteDesktopCertificates/Developer-ID-Application-Certificates.p12) \
   npm run dist

This time it uploaded to Apple! And then failed, but it gave me an actionable log:

  • signing         file=dist/mac/Datasette.app identityName=Developer ID Application: Simon Willison (762G34JSDR) identityHash=4CBAE9A14BAE09B41C3F24895927698FB1F9A970 provisioningProfile=none
  ⨯ Apple failed to notarize your application, check the logs for more info

Status Code: 2
Message: Package Invalid
Logs: https://osxapps-ssl.itunes.apple.com/itunes-assets/Enigma115/v4/94/45/89/94458967-e33c-bf50-01eb-5bb36c77e297/developer_log.json?accessKey=1630900936_3121281624425535927_8KqtIR999r5rjA%2FnXg5TSRkl3qaA%2FuLhTmyJPkjsW6M3wQdJk4S5MMaZ7IOI8TuGdtXKNk9KESmpR53AbRvLYdngR2fmG%2B3p%2BJUEXME5u35wNhihwGJ%2B9SkyhFFQ1NSg5Td4NqUtPYROKbURLJQEqzRgzWzfo8EixBNfTiRj6WE%3D  failedTask=build stackTrace=Error: Apple failed to notarize your application, check the logs for more info

https://osxapps-ssl.itunes.apple.com/itunes-assets/Enigma115/v4/94/45/89/94458967-e33c-bf50-01eb-5bb36c77e297/developer_log.json?accessKey=1630900936_3121281624425535927_8KqtIR999r5rjA%2FnXg5TSRkl3qaA%2FuLhTmyJPkjsW6M3wQdJk4S5MMaZ7IOI8TuGdtXKNk9KESmpR53AbRvLYdngR2fmG%2B3p%2BJUEXME5u35wNhihwGJ%2B9SkyhFFQ1NSg5Td4NqUtPYROKbURLJQEqzRgzWzfo8EixBNfTiRj6WE%3D

Here's that log file:

{
  "logFormatVersion": 1,
  "jobId": "3f934ed8-c56f-4ddd-8ee5-f75bf137dc42",
  "status": "Invalid",
  "statusSummary": "Archive contains critical validation errors",
  "statusCode": 4000,
  "archiveFilename": "Datasette.zip",
  "uploadDate": "2021-09-03T21:59:31Z",
  "sha256": "170675a919d7ba2396d933a74190a04cc5fe33a68a97cc40c909028150397d3d",
  "ticketContents": null,
  "issues": [
    {
      "severity": "error",
      "code": null,
      "path": "Datasette.zip/Datasette.app/Contents/Resources/python/bin/python3.9",
      "message": "The binary is not signed.",
      "docUrl": null,
      "architecture": "x86_64"
    },
    {
      "severity": "error",
      "code": null,
      "path": "Datasette.zip/Datasette.app/Contents/Resources/python/bin/python3.9",
      "message": "The signature does not include a secure timestamp.",
      "docUrl": null,
      "architecture": "x86_64"
    },
    {
      "severity": "error",
      "code": null,
      "path": "Datasette.zip/Datasette.app/Contents/Resources/python/bin/python3.9",
      "message": "The executable does not have the hardened runtime enabled.",
      "docUrl": null,
      "architecture": "x86_64"
    },
    {
      "severity": "error",
      "code": null,
      "path": "Datasette.zip/Datasette.app/Contents/Resources/python/lib/python3.9/lib-dynload/xxlimited.cpython-39-darwin.so",
      "message": "The binary is not signed.",
      "docUrl": null,
      "architecture": "x86_64"
    },
    {
      "severity": "error",
      "code": null,
      "path": "Datasette.zip/Datasette.app/Contents/Resources/python/lib/python3.9/lib-dynload/xxlimited.cpython-39-darwin.so",
      "message": "The signature does not include a secure timestamp.",
      "docUrl": null,
      "architecture": "x86_64"
    },
    {
      "severity": "error",
      "code": null,
      "path": "Datasette.zip/Datasette.app/Contents/Resources/python/lib/python3.9/lib-dynload/_testcapi.cpython-39-darwin.so",
      "message": "The binary is not signed.",
      "docUrl": null,
      "architecture": "x86_64"
    },
    {
      "severity": "error",
      "code": null,
      "path": "Datasette.zip/Datasette.app/Contents/Resources/python/lib/python3.9/lib-dynload/_testcapi.cpython-39-darwin.so",
      "message": "The signature does not include a secure timestamp.",
      "docUrl": null,
      "architecture": "x86_64"
    }
  ]
}

from datasette-app.

simonw avatar simonw commented on May 28, 2024

This thread is useful: electron-userland/electron-builder#4656

https://www.electron.build/configuration/mac describes "binaries" as an optional array of paths to extra binaries that need to be signed. I'll try that.

Here's an example of the "binaries" key being used: https://github.com/embtest/cypress/blob/e508af6289b8510c4bf10505120a4169b21ceaee/electron-builder.json#L13

    "binaries": [
      "./build/mac/Cypress.app/Contents/Resources/app/packages/server/node_modules/@ffmpeg-installer/darwin-x64/ffmpeg",

from datasette-app.

simonw avatar simonw commented on May 28, 2024

I think I need these two:

dist/mac/Datasette.app/Contents/Resources/python/bin/python3.9
dist/mac/Datasette.app/Contents/Resources/python/lib/python3.9/lib-dynload/xxlimited.cpython-39-darwin.so

from datasette-app.

simonw avatar simonw commented on May 28, 2024

My "build" section in package.json now looks like this:

  "build": {
    "appId": "io.datasette.app",
    "mac": {
      "category": "public.app-category.developer-tools",
      "hardenedRuntime" : true,
      "gatekeeperAssess": false,
      "entitlements": "build/entitlements.mac.plist",
      "entitlementsInherit": "build/entitlements.mac.plist",
      "binaries": [
        "./dist/mac/Datasette.app/Contents/Resources/python/bin/python3.9",
        "./dist/mac/Datasette.app/Contents/Resources/python/lib/python3.9/lib-dynload/xxlimited.cpython-39-darwin.so"
      ]
    },
    "afterSign": "scripts/notarize.js",
    "extraResources": [
      {
        "from": "python",
        "to": "python",
        "filter": [
          "**/*"
        ]
      }
    ]
  },

from datasette-app.

simonw avatar simonw commented on May 28, 2024

Failed with different (lesser) errors: https://osxapps-ssl.itunes.apple.com/itunes-assets/Enigma125/v4/eb/cb/c8/ebcbc871-e1db-b09e-f99c-42f40764c8e9/developer_log.json?accessKey=1630901822_7705680074369627131_gck5Hi5p0xJkxUUaXeS82lVuJs15nft%2BwujFyQLEjw%2F%2FOAGm7ZD5GBr2Ot16WoMk65M%2FzYSi2ViA6G3B%2FpaBgGk%2F7xw%2BQ%2BtnImNwu9PyPArNOZTq08YkijWax6Gh%2FREmXycSM%2BHldawc1OgeYWHHIb0Nz3EroDaPFAOoaD%2F0rTk%3D

{
  "logFormatVersion": 1,
  "jobId": "7dabec1f-b669-4912-8cc7-c022a51b6cdb",
  "status": "Invalid",
  "statusSummary": "Archive contains critical validation errors",
  "statusCode": 4000,
  "archiveFilename": "Datasette.zip",
  "uploadDate": "2021-09-03T22:14:47Z",
  "sha256": "d2785b0114a222d1985b362e663b6c8d71b86e429db1c90bc8daac5994ecd32f",
  "ticketContents": null,
  "issues": [
    {
      "severity": "error",
      "code": null,
      "path": "Datasette.zip/Datasette.app/Contents/Resources/python/lib/python3.9/lib-dynload/_testcapi.cpython-39-darwin.so",
      "message": "The binary is not signed.",
      "docUrl": null,
      "architecture": "x86_64"
    },
    {
      "severity": "error",
      "code": null,
      "path": "Datasette.zip/Datasette.app/Contents/Resources/python/lib/python3.9/lib-dynload/_testcapi.cpython-39-darwin.so",
      "message": "The signature does not include a secure timestamp.",
      "docUrl": null,
      "architecture": "x86_64"
    }
  ]
}

I have a hunch this is going to be a series of whack-a-mole steps.

from datasette-app.

simonw avatar simonw commented on May 28, 2024

I wonder if I'll need to sign everything in that folder that shows up as having executable permissions?

% find . -perm +111 -type f
./Contents/MacOS/Datasette
./Contents/Resources/python/bin/pip3.9
./Contents/Resources/python/bin/pip3
./Contents/Resources/python/bin/python3.9-config
./Contents/Resources/python/bin/python3.9
./Contents/Resources/python/bin/pip
./Contents/Resources/python/bin/pydoc3.9
./Contents/Resources/python/bin/idle3.9
./Contents/Resources/python/bin/2to3-3.9
./Contents/Resources/python/lib/python3.9/lib-dynload/xxlimited.cpython-39-darwin.so
./Contents/Resources/python/lib/python3.9/lib-dynload/_testcapi.cpython-39-darwin.so
./Contents/Resources/python/lib/python3.9/encodings/rot_13.py
./Contents/Resources/python/lib/python3.9/ctypes/macholib/fetch_macholib
./Contents/Resources/python/lib/python3.9/trace.py
./Contents/Resources/python/lib/python3.9/webbrowser.py
./Contents/Resources/python/lib/python3.9/tabnanny.py
./Contents/Resources/python/lib/python3.9/cProfile.py
./Contents/Resources/python/lib/python3.9/base64.py
./Contents/Resources/python/lib/python3.9/cgi.py
./Contents/Resources/python/lib/python3.9/turtledemo/tree.py
./Contents/Resources/python/lib/python3.9/turtledemo/bytedesign.py
./Contents/Resources/python/lib/python3.9/turtledemo/clock.py
./Contents/Resources/python/lib/python3.9/turtledemo/sorting_animate.py
./Contents/Resources/python/lib/python3.9/turtledemo/paint.py
./Contents/Resources/python/lib/python3.9/turtledemo/lindenmayer.py
./Contents/Resources/python/lib/python3.9/turtledemo/penrose.py
./Contents/Resources/python/lib/python3.9/turtledemo/peace.py
./Contents/Resources/python/lib/python3.9/turtledemo/yinyang.py
./Contents/Resources/python/lib/python3.9/turtledemo/fractalcurves.py
./Contents/Resources/python/lib/python3.9/turtledemo/planet_and_moon.py
./Contents/Resources/python/lib/python3.9/turtledemo/forest.py
./Contents/Resources/python/lib/python3.9/turtledemo/__main__.py
./Contents/Resources/python/lib/python3.9/turtledemo/minimal_hanoi.py
./Contents/Resources/python/lib/python3.9/profile.py
./Contents/Resources/python/lib/python3.9/uu.py
./Contents/Resources/python/lib/python3.9/pydoc.py
./Contents/Resources/python/lib/python3.9/pdb.py
./Contents/Resources/python/lib/python3.9/platform.py
./Contents/Resources/python/lib/python3.9/quopri.py
./Contents/Resources/python/lib/python3.9/config-3.9-darwin/python-config.py
./Contents/Resources/python/lib/python3.9/config-3.9-darwin/install-sh
./Contents/Resources/python/lib/python3.9/config-3.9-darwin/makesetup
./Contents/Resources/python/lib/python3.9/smtplib.py
./Contents/Resources/python/lib/python3.9/timeit.py
./Contents/Resources/python/lib/python3.9/tarfile.py
./Contents/Resources/python/lib/python3.9/lib2to3/tests/pytree_idempotency.py
./Contents/Resources/python/lib/python3.9/lib2to3/tests/data/different_encoding.py
./Contents/Resources/python/lib/python3.9/lib2to3/tests/data/false_encoding.py
./Contents/Resources/python/lib/python3.9/lib2to3/pgen2/token.py
./Contents/Resources/python/lib/python3.9/idlelib/pyshell.py
./Contents/Resources/python/lib/python3.9/socket.py
./Contents/Resources/python/lib/python3.9/smtpd.py
./Contents/Resources/python/lib/tk8.6/demos/widget
./Contents/Resources/python/lib/tk8.6/demos/timer
./Contents/Resources/python/lib/tk8.6/demos/ixset
./Contents/Resources/python/lib/tk8.6/demos/tcolor
./Contents/Resources/python/lib/tk8.6/demos/hello
./Contents/Resources/python/lib/tk8.6/demos/browse
./Contents/Resources/python/lib/tk8.6/demos/rolodex
./Contents/Resources/python/lib/tk8.6/demos/rmt
./Contents/Resources/python/lib/libpython3.9.dylib
./Contents/Frameworks/Datasette Helper (Plugin).app/Contents/MacOS/Datasette Helper (Plugin)
./Contents/Frameworks/Electron Framework.framework/Versions/A/Electron Framework
./Contents/Frameworks/Electron Framework.framework/Versions/A/Libraries/libEGL.dylib
./Contents/Frameworks/Electron Framework.framework/Versions/A/Libraries/libswiftshader_libEGL.dylib
./Contents/Frameworks/Electron Framework.framework/Versions/A/Libraries/libvk_swiftshader.dylib
./Contents/Frameworks/Electron Framework.framework/Versions/A/Libraries/libGLESv2.dylib
./Contents/Frameworks/Electron Framework.framework/Versions/A/Libraries/libswiftshader_libGLESv2.dylib
./Contents/Frameworks/Electron Framework.framework/Versions/A/Libraries/libffmpeg.dylib
./Contents/Frameworks/Electron Framework.framework/Versions/A/Helpers/chrome_crashpad_handler
./Contents/Frameworks/ReactiveObjC.framework/Versions/A/ReactiveObjC
./Contents/Frameworks/Squirrel.framework/Versions/A/Resources/ShipIt
./Contents/Frameworks/Squirrel.framework/Versions/A/Squirrel
./Contents/Frameworks/Datasette Helper.app/Contents/MacOS/Datasette Helper
./Contents/Frameworks/Mantle.framework/Versions/A/Mantle
./Contents/Frameworks/Datasette Helper (GPU).app/Contents/MacOS/Datasette Helper (GPU)
./Contents/Frameworks/Datasette Helper (Renderer).app/Contents/MacOS/Datasette Helper (Renderer)

from datasette-app.

simonw avatar simonw commented on May 28, 2024

... that seemed to work that time? With the following binaries:

      "binaries": [
        "./dist/mac/Datasette.app/Contents/Resources/python/bin/python3.9",
        "./dist/mac/Datasette.app/Contents/Resources/python/lib/python3.9/lib-dynload/xxlimited.cpython-39-darwin.so",
        "./dist/mac/Datasette.app/Contents/Resources/python/lib/python3.9/lib-dynload/_testcapi.cpython-39-darwin.so"
      ]

I'll upload it and download it to test it.

from datasette-app.

simonw avatar simonw commented on May 28, 2024

I added APPLEID and APPLEIDPASS secrets to this repo.

from datasette-app.

simonw avatar simonw commented on May 28, 2024

After downloading and unzipping it I ran the app - it got as far as the loading screen but hung there.

The /Users/simon/.datasette-app folder was created but the /Users/simon/.datasette-app/venv folder within it was not - so presumably there's some kind of permission issue running processes here:

datasette-app/main.js

Lines 86 to 107 in b5d2d0c

async ensureDatasetteInstalled() {
const datasette_app_dir = path.join(process.env.HOME, ".datasette-app");
const venv_dir = path.join(datasette_app_dir, "venv");
const datasette_binary = path.join(venv_dir, "bin", "datasette");
if (fs.existsSync(datasette_binary)) {
return datasette_binary;
}
if (!fs.existsSync(datasette_app_dir)) {
await mkdir(datasette_app_dir);
}
if (!fs.existsSync(venv_dir)) {
await execFile(findPython(), ["-m", "venv", venv_dir]);
}
const pip_path = path.join(venv_dir, "bin", "pip");
await execFile(pip_path, [
"install",
"datasette==0.59a2",
"datasette-app-support>=0.2",
]);
await new Promise((resolve) => setTimeout(resolve, 500));
return datasette_binary;
}

from datasette-app.

simonw avatar simonw commented on May 28, 2024

Hunch: the permissions are not being inherited by the child python process.

from datasette-app.

simonw avatar simonw commented on May 28, 2024

Tried adding this entitlement:

    <key>com.apple.security.inherit</key>
    <true/>

from datasette-app.

simonw avatar simonw commented on May 28, 2024

OK, this time the app installed and ran for the first time and managed to install the stuff it needs in its ~/.datasette-app/venv virtual environment AND started up!

But... attempting to open both CSV and DB files failed. Not sure why.

from datasette-app.

simonw avatar simonw commented on May 28, 2024

Found some clues in the Console.app logs while running the notarized application:

Prompting policy for hardened runtime; service: kTCCServiceCalendar requires entitlement com.apple.security.personal-information.calendars but it is missing for RESP:{ID: io.datasette.app, PID[18200], auid: 501, euid: 501, responsible path: '/private/tmp/Datasette.app/Contents/MacOS/Datasette', binary path: '/private/tmp/Datasette.app/Contents/MacOS/Datasette'}, ACC:{ID: com.apple.appkit.xpc.openAndSavePanelService, PID[18222], auid: 501, euid: 501, binary path: '/System/Library/Frameworks/AppKit.framework/Versions/C/XPCServices/com.apple.appkit.xpc.openAndSavePanelService.xpc/Contents/MacOS/com.apple.appkit.xpc.openAndSavePanelService'}, REQ:{ID: com.apple.mds, PID[116], auid: 0, euid: 0, binary path: '/System/Library/Frameworks/CoreServices.framework/Versions/A/Frameworks/Metadata.framework/Versions/A/Support/mds'}

Prompting policy for hardened runtime; service: kTCCServiceAppleEvents requires entitlement com.apple.security.automation.apple-events but it is missing for RESP:{ID: io.datasette.app, PID[18200], auid: 501, euid: 501, responsible path: '/private/tmp/Datasette.app/Contents/MacOS/Datasette', binary path: '/private/tmp/Datasette.app/Contents/MacOS/Datasette'}, ACC:{ID: com.apple.quicklook.QuickLookUIService, PID[18223], auid: 501, euid: 501, binary path: '/System/Library/Frameworks/Quartz.framework/Versions/A/Frameworks/QuickLookUI.framework/Versions/A/XPCServices/QuickLookUIService.xpc/Contents/MacOS/QuickLookUIService'}, REQ:{ID: com.apple.appleeventsd, PID[296], auid: 55, euid: 55, binary path: '/System/Library/CoreServices/appleeventsd'}

Also spotted this:

Sandbox: Datasette Helper(18204) deny(1) file-read-data /Library/Application Support/CrashReporter/SubmitDiagInfo.domains
Violation:       deny(1) file-read-data /Library/Application Support/CrashReporter/SubmitDiagInfo.domains 
Process:         Datasette Helper [18204]
Path:            /private/tmp/Datasette.app/Contents/Frameworks/Datasette Helper (GPU).app/Contents/MacOS/Datasette Helper (GPU)
Load Address:    0x1078ef000
Identifier:      io.datasette.app.helper.GPU
Version:         0.1.0 (???)
Code Type:       x86_64 (Native)
Parent Process:  Datasette [18200]
Responsible:     /private/tmp/Datasette.app/Contents/MacOS/Datasette
User ID:         501

Date/Time:       2021-09-03 15:54:35.282 PDT
OS Version:      Mac OS X 10.15.6 (19G2021)
Report Version:  8

And:

Sandbox: Datasette Helper(18204) deny(1) mach-lookup com.apple.analyticsd

No idea what this one is about:

Non-fatal error enumerating at <private>, continuing: Error Domain=NSCocoaErrorDomain Code=260 "The file “PlugIns” couldn’t be opened because there is no such file." UserInfo={NSURL=PlugIns/ -- file:///private/tmp/Datasette.app/Contents/Frameworks/Datasette%20Helper%20(GPU).app/Contents/, NSFilePath=/private/tmp/Datasette.app/Contents/Frameworks/Datasette Helper (GPU).app/Contents/PlugIns, NSUnderlyingError=0x7f9d0cb343f0 {Error Domain=NSPOSIXErrorDomain Code=2 "No such file or directory"}}

from datasette-app.

simonw avatar simonw commented on May 28, 2024

I've been running this all from /tmp/Datasette.app - I'm going to try moving it to /Applications and see if that makes a difference.

from datasette-app.

simonw avatar simonw commented on May 28, 2024

No, same exact result. The app loads OK (and installs its virtual environment) but the open CSV menu option silently fails, without writing anything to the Console app either.

from datasette-app.

simonw avatar simonw commented on May 28, 2024

I'll test installing from the latest artifact: https://github.com/simonw/datasette-app/actions/runs/1199643947

from datasette-app.

simonw avatar simonw commented on May 28, 2024

This time I got the following dialog on first launch, which is most excellent!

Datasette-macOS_and_Downloads

The app successfully installed its virtual environment and ran.

Datasette__temporary_and_Work_out_how_to_notarize_the_macOS_application_·_Issue__50_·_simonw_datasette-app

Still no luck opening a file though. I'm going to try and fix the following errors, even though they don't look like they relate to the ability to open files:

Prompting policy for hardened runtime; service: kTCCServiceAppleEvents requires entitlement com.apple.security.automation.apple-events but it is missing for RESP:{ID: io.datasette.app, PID[18731], auid: 501, euid: 501, responsible path: '/private/var/folders/wr/hn3206rs1yzgq3r49bz8nvnh0000gn/T/AppTranslocation/38993E0B-BA56-4190-A192-8119EA5C1E9C/d/Datasette.app/Contents/MacOS/Datasette', binary path: '/private/var/folders/wr/hn3206rs1yzgq3r49bz8nvnh0000gn/T/AppTranslocation/38993E0B-BA56-4190-A192-8119EA5C1E9C/d/Datasette.app/Contents/MacOS/Datasette'}, ACC:{ID: com.apple.appkit.xpc.openAndSavePanelService, PID[18790], auid: 501, euid: 501, binary path: '/System/Library/Frameworks/AppKit.framework/Versions/C/XPCServices/com.apple.appkit.xpc.openAndSavePanelService.xpc/Contents/MacOS/com.apple.appkit.xpc.openAndSavePanelService'}, REQ:{ID: com.apple.appleeventsd, PID[296], auid: 55, euid: 55, binary path: '/System/Library/CoreServices/appleeventsd'}

Prompting policy for hardened runtime; service: kTCCServiceCalendar requires entitlement com.apple.security.personal-information.calendars but it is missing for RESP:{ID: io.datasette.app, PID[18731], auid: 501, euid: 501, responsible path: '/private/var/folders/wr/hn3206rs1yzgq3r49bz8nvnh0000gn/T/AppTranslocation/38993E0B-BA56-4190-A192-8119EA5C1E9C/d/Datasette.app/Contents/MacOS/Datasette', binary path: '/private/var/folders/wr/hn3206rs1yzgq3r49bz8nvnh0000gn/T/AppTranslocation/38993E0B-BA56-4190-A192-8119EA5C1E9C/d/Datasette.app/Contents/MacOS/Datasette'}, ACC:{ID: com.apple.appkit.xpc.openAndSavePanelService, PID[18790], auid: 501, euid: 501, binary path: '/System/Library/Frameworks/AppKit.framework/Versions/C/XPCServices/com.apple.appkit.xpc.openAndSavePanelService.xpc/Contents/MacOS/com.apple.appkit.xpc.openAndSavePanelService'}, REQ:{ID: com.apple.mds, PID[116], auid: 0, euid: 0, binary path: '/System/Library/Frameworks/CoreServices.framework/Versions/A/Frameworks/Metadata.framework/Versions/A/Support/mds'}

I'll try fixing the com.apple.security.automation.apple-events one first.

from datasette-app.

simonw avatar simonw commented on May 28, 2024

I really don't understand why I would want 'com.apple.security.personal-information.calendars': https://developer.apple.com/documentation/bundleresources/entitlements/com_apple_security_personal-information_calendars

A Boolean value that indicates whether the app may have read-write access to the user's calendar.

from datasette-app.

simonw avatar simonw commented on May 28, 2024

Unsurprisingly the com.apple.security.automation.apple-events entitlement did not fix things.

from datasette-app.

simonw avatar simonw commented on May 28, 2024

Here's a guess at the problem: I have the com.apple.security.files.user-selected.read-only entitlement which should mean that, because the user selected a file using the "open" dialog, the Electron process gains the ability to read that file.

But... that's not what the Electron app does. It instead takes that file path and does this with it:

datasette-app/main.js

Lines 316 to 322 in 87038b4

const response = await request(
`http://localhost:${port}/-/open-csv-file`,
{
method: "POST",
body: JSON.stringify({ path: filepath }),
}
);

Then the Python process running that web server attempts to read the file itself.

My guess is that the Python process isn't inheriting the ability to open the file - and is then crashing without a useful error message.

from datasette-app.

simonw avatar simonw commented on May 28, 2024

I can hack on /Users/simon/.datasette-app/venv/lib/python3.9/site-packages/datasette_app_support/__init__.py to add more error handling.

from datasette-app.

simonw avatar simonw commented on May 28, 2024

Turns out the bug was that the CSV file I was trying to open had an error (see simonw/datasette-app-support#3) but I hadn't yet applied that fix in datasette-app-support!

The app I installed from the latest build worked! It looks like it is successfully notarized.

from datasette-app.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.