codemagic-ci-cd / cli-tools Goto Github PK

View Code? Open in Web Editor NEW

222.0 7.0 38.0 68.63 MB

Various utilities to managing Android and iOS app builds, code signing, and deployment.

Home Page: https://codemagic.io/start/

License: GNU General Public License v3.0

Python 98.48% Ruby 1.50% Lua 0.02%

android app-store devops google-play ios mobile testflight

cli-tools's People

Contributors

Stargazers

Watchers

Forkers

afghanistanyn pepijn-vanvlaanderen str4t3gy manggalarefactory bohdan1krokhmaliuk drewdulz esseplore-it xiaof631 wiefel dan-os jleach avivyaari vvorlov brandi1981 detachhead andreujuanc 1kirmizi danaru87 isabella232 talbekk patrickpaulin jbzd kemuelttg louvis2269 nouman-hanif jascodes altlokalt bandaoxiatian abuamro nilsreichardt nodgibarreto demolaf thomasvnn bnworkol programming086 cinarteker leehomeok oussemadaoud

cli-tools's Issues

keychain -h does not mention option -p or --path

I have the cli tools version 0.27.0 on my system running keychain -h does not mention the -p or --path option which is mentioned here in github https://github.com/codemagic-ci-cd/cli-tools/blob/master/docs/keychain/README.md. I think it would be good for other users to add this to the keychain -h

display actionable error message when xcodeproj gem is not installed

The current error message for running xcode-project use-profiles is incomprehensible:

% xcode-project use-profiles
Configure code signing settings
Searching for files matching /Users/user/Library/MobileDevice/Provisioning Profiles/*.mobileprovision
Searching for files matching /Users/user/Library/MobileDevice/Provisioning Profiles/*.provisionprofile
List available code signing certificates in keychain /private/var/folders/c9/brbhkbtd14bgt8s36glj8zhw0000gn/T/build__dnzh9z5.keychain
Searching for files matching /Users/user/nevercode/my_macos_app/**/*.xcodeproj
/Users/user/.rbenv/versions/2.7.3/lib/ruby/2.7.0/rubygems/core_ext/kernel_require.rb:83:in `require': cannot load such file -- xcodeproj (LoadError)
	from /Users/user/.rbenv/versions/2.7.3/lib/ruby/2.7.0/rubygems/core_ext/kernel_require.rb:83:in `require'
	from /Users/user/Library/Python/3.8/bin/code_signing_manager.rb:8:in `<main>'
Failed to set code signing settings for macos/Runner.xcodeproj

The solution is to run gem install xcodeproj.

Can we add a proper error message for this situation so users know they need to run this command? Maybe consider adding a setup validation script? Should we go as far as having a Bundler Gemfile (though that would feel strange in a python library)?

Feature idea: get latest build number from app store connect

looks like app store connect API has option to get list of builds associated with an app i.e. https://developer.apple.com/documentation/appstoreconnectapi/list_all_builds_of_an_app

maybe we can create some automatic numbering to always use latest build number +1 so people don't need to worry about versioning scripts?

This feature idea is not about of version name

Feature request: [app-store-connect] expire previous builds, reject build waiting for review

After implementing a "submit for review" action, the next logical steps would be to allow to expire previous builds and reject waiting review builds.

Command for macOS UI testing

When interacting with the UI while testing a macOS app, we get an alert for granting permission:

It is feasible on the local machine. However, it is not possible in the remote environment without providing the relevant permissions. It'll make it a lot easier to provide an executable similar to:

xcode-project mac-permission
        [--add-uitest-permission]

Thank you!

Create an Apple APNS key from CLI

Hello,
I’m currently working on a white-label app that I want to deploy through a CI/CD.
Each version of my white-label app would have a different Firebase project for using Firebase Messaging.

As you probably know, Apple enforces each version of a white-label app to be uploaded through the client’s developer account. That means, that for using iOS Push Notification, an APNS key/certificate would need to be generated for every version of the app through the client’s developer account, and would need to be uploaded to Firebase Cloud Messaging (please see image attached).

I've seen that your App Store Connect CLI tool has an option to create and download certificates, but does not have an option to create and download keys, which are needed to be uploaded to Firebase Console in order for push notifications to work on iOS (.p8/.p12 files).

This is something every white-label app will encounter when using iOS push notifications, so I believe it is something that is worth being added to the CLI, in order to allow automation of white-label apps deployment.

What do you think?

Generating a key through Apple's portal:

Uploading the key to Firebase Cloud Messaging (through Firebase Console):

Feature request: Add `get-latest-build-number` command

Description

There is already a get-latest-app-store-build-number and get-latest-testflight-build-number command. However, I need to get the highest build number, either from AppStore or TestFlight:

Let's look at this example:

AppStore has build number 31.
TestFlight has build number 30.

The value of get-latest-build-number would be 31.

Another case:

AppStore has build number 30.
TestFlight has build number 32.

The value of get-latest-build-number would be 32.

Get latest build number for custom track

Right now, when getting latest build number, only some tracks are showing. But, I've custom tracks, where the build number has increased. How can configure to get latest build number from custom tracks as well?

Invalid provisioning profile /xxx/xxx.mobileprovision

Hello i have an issue when command xcode-project use-profiles

any idea what's going on? I tried in my local laptop and had same issue.

Outdated bundletools

Current Bundle tools version is 0.15.0
Latest available is 1.13.1

Latest react-native abb fails to extract apk.

App Store Connect API client JWT expiration

As of version 0.14.1 the default expiration duration of App Store Connect API JSON Web Tokens used by used by AppStoreConnectApiClient is just 30 seconds [src], while it can be up to 20 minutes in case scope is not specified, and even longer if token scope is defined. See App Store Connect API docs about generating tokens for API requests.

In some cases App Store Connect API respons with 401 Unauthorized when completely valid JWT is provided in request headers, and this can make app-store-connect commands fail unexpectedly. We should reuse JWTs as long as possible to avoid such cases. In order to achieve this:

~~increase the expiration time for generated JWTs from 30 seconds to something close to 20 minutes~~,
cache generated JWTs to disk so that tokens can be reused between CLI invocations.

google-play CLI client does not detect or allow custom tracks

I've created a custom track, which are now supported in the publishing stage of the pipeline. However, the google-play cli tool does not yet support the ability to detect and/or query custom tracks.

This is problematic as publishing to a track increments the version code, but since the google-play tool doesn't detect the custom track, it cannot increment the build number, and thus the build fails due to it attempting to overwrite an existing version code.

Crash in app-store-connect on Python 3.11

We have a GitHub action that uses app-store-connect. A few days ago, the macOS 12 runner was upgraded from Python 3.10 to Python 3.11. There's apparently a problem with the paths after this update, which led me to file this bug against the runner. A workaround for that is to invoke the tool from its full path inside the Python framework:

/Library/Frameworks/Python.framework/Versions/3.11/bin/app-store-connect

When we do so, the following happens:

Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.11/bin/app-store-connect", line 5, in <module>
    from codemagic.tools import AppStoreConnect
  File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/codemagic/tools/__init__.py", line 1, in <module>
    from .android_app_bundle import AndroidAppBundle
  File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/codemagic/tools/android_app_bundle.py", line 12, in <module>
    from codemagic.models import AndroidSigningInfo
  File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/codemagic/models/__init__.py", line 12, in <module>
    from .keystore import Keystore
  File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/codemagic/models/keystore.py", line 7, in <module>
    @dataclass
     ^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/dataclasses.py", line 1221, in dataclass
    return wrap(cls)
           ^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/dataclasses.py", line 1211, in wrap
    return _process_class(cls, init, repr, eq, order, unsafe_hash,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/dataclasses.py", line 959, in _process_class
    cls_fields.append(_get_field(cls, name, type, kw_only))
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/dataclasses.py", line [81](https://github.com/movehq/hq-driver-ios/actions/runs/3379570935/jobs/5611305106#step:8:82)6, in _get_field
    raise ValueError(f'mutable default {type(f.default)} for field '
ValueError: mutable default <class 'codemagic.models.certificate_attributes.CertificateAttributes'> for field certificate_attributes is not allowed: use default_factory

I'm not sure if this is a problem with the script (incompatibility with Python 3.11?), or a problem with the fact that I'm trying to invoke it from inside the framework.

`app-store-connect apps builds <id>` fails with unexpected keyword argument 'lsMinimumSystemVersion'

When running app-store-connect apps builds <id> today with an iOS-only app we own, the tool fails with an error message like:

Executing AppStoreConnect action apps failed unexpectedly. Detailed logs are available at "/var/folders/path/to/codemagic-13-09-21.log". To see more details about the error, add `--verbose` command line option.

and the log file ends with the Python stack trace:

[14:31:36 13-09-2021] ERROR cli_app.py:113 > Exception traceback:
Traceback (most recent call last):
  File "/usr/local/lib/python3.9/site-packages/codemagic/cli/cli_app.py", line 202, in invoke_cli
    CliApp._running_app._invoke_action(args)
  File "/usr/local/lib/python3.9/site-packages/codemagic/cli/cli_app.py", line 156, in _invoke_action
    return cli_action(**action_args)
  File "/usr/local/lib/python3.9/site-packages/codemagic/cli/cli_app.py", line 455, in wrapper
    return func(*args, **kwargs)
  File "/usr/local/lib/python3.9/site-packages/codemagic/tools/_app_store_connect/action_groups/apps_action_group.py", line 72, in list_app_builds
    return self._list_related_resources(
  File "/usr/local/lib/python3.9/site-packages/codemagic/tools/_app_store_connect/resource_manager_mixin.py", line 80, in _list_related_resources
    resources = list_related_resources_method(resource_id, **kwargs)
  File "/usr/local/lib/python3.9/site-packages/codemagic/apple/app_store_connect/apps/apps.py", line 81, in list_builds
    return [Build(build) for build in self.client.paginate(url, page_size=None)]
  File "/usr/local/lib/python3.9/site-packages/codemagic/apple/app_store_connect/apps/apps.py", line 81, in <listcomp>
    return [Build(build) for build in self.client.paginate(url, page_size=None)]
  File "/usr/local/lib/python3.9/site-packages/codemagic/apple/resources/resource.py", line 196, in __init__
    self.attributes = self._create_attributes(api_response)
  File "/usr/local/lib/python3.9/site-packages/codemagic/apple/resources/resource.py", line 178, in _create_attributes
    return cls.Attributes(**api_response['attributes'])
TypeError: __init__() got an unexpected keyword argument 'lsMinimumSystemVersion'

preceded by a JSON dump where every JSON object under $.data[].attributes contains the key lsMinimumSystemVersion with a value of null (i.e. None). (I think this might be a new key Apple have introduced related to TestFlight support for macOS.)

Regardless, I think the tool shouldn't blindly pass **kwargs from remote JSON data, or else at least tolerate previously unknown keys in the parsing.

Feature request: display warnings when there is a newer version of CLI tools available

Show a warning on launch if the current version of CLI tools is older than the most recent one available. This is useful when using the CLI tools locally.

https://codemagicio.slack.com/archives/CEKE2KZ37/p1623152345437900

PyJWT 2.x support

Issue description

AppStoreConnectApiClient uses PyJWT to manage JSON Web Tokens that are used to authenticate the API calls. (See the jwt property from source.)

The API of PyJWT changed with version 2.x and now jwt.encode returns encoded string instead of bytes. As such the generation of token fails with the following stacktrace:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/codemagic/cli/cli_app.py", line 170, in invoke_cli
    CliApp._running_app._invoke_action(args)
  File "/usr/local/lib/python3.8/site-packages/codemagic/cli/cli_app.py", line 142, in _invoke_action
    return cli_action(**action_args)
  File "/usr/local/lib/python3.8/site-packages/codemagic/cli/cli_app.py", line 358, in wrapper
    return func(*args, **kwargs)
  File "/usr/local/lib/python3.8/site-packages/codemagic/tools/app_store_connect.py", line 500, in fetch_signing_files
    bundle_ids = self._get_or_create_bundle_ids(bundle_id_identifier, platform, create_resource)
  File "/usr/local/lib/python3.8/site-packages/codemagic/tools/app_store_connect.py", line 511, in _get_or_create_bundle_ids
    bundle_ids = self.list_bundle_ids(bundle_id_identifier, platform=platform, should_print=False)
  File "/usr/local/lib/python3.8/site-packages/codemagic/cli/cli_app.py", line 358, in wrapper
    return func(*args, **kwargs)
  File "/usr/local/lib/python3.8/site-packages/codemagic/tools/app_store_connect.py", line 202, in list_bundle_ids
    bundle_ids = self._list_resources(bundle_id_filter, self.api_client.bundle_ids, should_print)
  File "/usr/local/lib/python3.8/site-packages/codemagic/tools/app_store_connect.py", line 132, in _list_resources
    resources = resource_manager.list(resource_filter=resource_filter)
  File "/usr/local/lib/python3.8/site-packages/codemagic/apple/app_store_connect/provisioning/bundle_ids.py", line 93, in list
    bundle_ids = self.client.paginate(f'{self.client.API_URL}/bundleIds', params=params)
  File "/usr/local/lib/python3.8/site-packages/codemagic/apple/app_store_connect/api_client.py", line 90, in paginate
    response = self.session.get(url, params={'limit': page_size, **params}).json()
  File "/usr/local/lib/python3.8/site-packages/requests/sessions.py", line 555, in get
    return self.request('GET', url, **kwargs)
  File "/usr/local/lib/python3.8/site-packages/codemagic/apple/app_store_connect/api_session.py", line 36, in request
    headers.update(self._auth_headers_factory())
  File "/usr/local/lib/python3.8/site-packages/codemagic/apple/app_store_connect/api_client.py", line 83, in generate_auth_headers
    return {'Authorization': f'Bearer {self.jwt}'}
  File "/usr/local/lib/python3.8/site-packages/codemagic/apple/app_store_connect/api_client.py", line 61, in jwt
    self._jwt = token.decode()
AttributeError: 'str' object has no attribute 'decode'

keychain: command not found

Hi
I'm using codemagic cli tools in github workflow to build ios app. It worked fine 6 days ago, but now gives this error:

writing RSA key
/Users/runner/work/_temp/4f71b5b3-3bfa-44b6-a102-64964826b285.sh: line 5: keychain: command not found

Here is the entire workflow step:

      - name: Setup certificates
        env:
          APP_STORE_CONNECT_ISSUER_ID: ${{ secrets.APPSTORE_API_KEY_ID }}
          APP_STORE_CONNECT_KEY_IDENTIFIER: ${{ secrets.APPSTORE_ISSUER_ID }}
          P12_PASSWORD: ${{ secrets.P12_PASSWORD }}
        run: |
          pip3 install codemagic-cli-tools
          echo -n ${{ secrets.APPSTORE_API_PRIVATE_KEY }} | base64 --decode > appstore_key.p8
          echo -n ${{ secrets.BUILD_CERTIFICATE_BASE64 }} | base64 --decode > build_cert.p12
          openssl pkcs12 -in build_cert.p12 -nodes -nocerts -passin pass:$P12_PASSWORD | openssl rsa -out cert_key
          keychain initialize
          app-store-connect fetch-signing-files $(xcode-project detect-bundle-id) \
              --platform IOS \
              --type IOS_APP_STORE \
              --certificate-key=@file:cert_key \
              --private-key=@file:appstore_key.p8 \
              --create
          keychain add-certificates
          xcode-project use-profiles

Here is full log:

Run pip3 install codemagic-cli-tools
  
Collecting codemagic-cli-tools
  Downloading codemagic_cli_tools-0.36.2-py3-none-any.whl (22.3 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 22.3/22.3 MB 43.9 MB/s eta 0:00:00
Collecting cryptography!=37.0.0,<38.0.0,>=3.3
  Downloading cryptography-37.0.4-cp36-abi3-macosx_10_10_x86_64.whl (2.8 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 2.8/2.8 MB 14.7 MB/s eta 0:00:00
Collecting google-api-python-client>=1.7.12
  Downloading google_api_python_client-2.65.0-py2.py3-none-any.whl (10.4 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 10.4/10.4 MB 51.1 MB/s eta 0:00:00
Collecting httplib2>=0.19.0
  Downloading httplib2-0.21.0-py3-none-any.whl (96 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 96.8/96.8 kB 6.9 MB/s eta 0:00:00
Collecting oauth2client>=4.1.3
  Downloading oauth2client-4.1.3-py2.py3-none-any.whl (98 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 98.2/98.2 kB 13.3 MB/s eta 0:00:00
Collecting psutil>=5.8.0
  Downloading psutil-5.9.3.tar.gz (483 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 483.6/483.6 kB 35.8 MB/s eta 0:00:00
  Installing build dependencies: started
  Installing build dependencies: finished with status 'done'
  Getting requirements to build wheel: started
  Getting requirements to build wheel: finished with status 'done'
  Installing backend dependencies: started
  Installing backend dependencies: finished with status 'done'
  Preparing metadata (pyproject.toml): started
  Preparing metadata (pyproject.toml): finished with status 'done'
Collecting pyjwt<3.0.0,>=2.4.0
  Downloading PyJWT-2.6.0-py3-none-any.whl (20 kB)
Collecting requests>=2.25
  Downloading requests-2.28.1-py3-none-any.whl (62 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 62.8/62.8 kB 6.2 MB/s eta 0:00:00
Collecting cffi>=1.12
  Downloading cffi-1.15.1-cp311-cp311-macosx_10_9_x86_64.whl (179 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 179.2/179.2 kB 2.3 MB/s eta 0:00:00
Collecting google-auth<3.0.0dev,>=1.19.0
  Downloading google_auth-2.14.0-py2.py3-none-any.whl (175 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 175.0/175.0 kB 20.8 MB/s eta 0:00:00
Collecting google-auth-httplib2>=0.1.0
  Downloading google_auth_httplib2-0.1.0-py2.py3-none-any.whl (9.3 kB)
Collecting google-api-core!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.0,<3.0.0dev,>=1.31.5
  Downloading google_api_core-2.10.2-py3-none-any.whl (115 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 115.6/115.6 kB 17.4 MB/s eta 0:00:00
Collecting uritemplate<5,>=3.0.1
  Downloading uritemplate-4.1.1-py2.py3-none-any.whl (10 kB)
Collecting pyparsing!=3.0.0,!=3.0.1,!=3.0.2,!=3.0.3,<4,>=2.4.2
  Downloading pyparsing-3.0.9-py3-none-any.whl (98 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 98.3/98.3 kB 15.0 MB/s eta 0:00:00
Collecting pyasn1>=0.1.7
  Downloading pyasn1-0.4.8-py2.py3-none-any.whl (77 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 77.1/77.1 kB 7.0 MB/s eta 0:00:00
Collecting pyasn1-modules>=0.0.5
  Downloading pyasn1_modules-0.2.8-py2.py3-none-any.whl (155 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 155.3/155.3 kB 2.2 MB/s eta 0:00:00
Collecting rsa>=3.1.4
  Downloading rsa-4.9-py3-none-any.whl (34 kB)
Collecting six>=1.6.1
  Downloading six-1.16.0-py2.py3-none-any.whl (11 kB)
Collecting charset-normalizer<3,>=2
  Downloading charset_normalizer-2.1.1-py3-none-any.whl (39 kB)
Collecting idna<4,>=2.5
  Downloading idna-3.4-py3-none-any.whl (61 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 61.5/61.5 kB 10.2 MB/s eta 0:00:00
Collecting urllib3<1.27,>=1.21.1
  Downloading urllib3-1.26.12-py2.py3-none-any.whl (140 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 140.4/140.4 kB 3.4 MB/s eta 0:00:00
Requirement already satisfied: certifi>=2017.4.17 in /Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages (from requests>=2.25->codemagic-cli-tools) (2022.9.24)
Collecting pycparser
  Downloading pycparser-2.21-py2.py3-none-any.whl (118 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 118.7/118.7 kB 16.1 MB/s eta 0:00:00
Collecting googleapis-common-protos<2.0dev,>=1.56.2
  Downloading googleapis_common_protos-1.56.4-py2.py3-none-any.whl (211 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 211.7/211.7 kB 24.8 MB/s eta 0:00:00
Collecting protobuf!=3.20.0,!=3.20.1,!=4.21.0,!=4.21.1,!=4.21.2,!=4.21.3,!=4.21.4,!=4.21.5,<5.0.0dev,>=3.19.5
  Downloading protobuf-4.21.9-cp37-abi3-macosx_10_9_universal2.whl (483 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 483.8/483.8 kB 26.3 MB/s eta 0:00:00
Collecting cachetools<6.0,>=2.0.0
  Downloading cachetools-5.2.0-py3-none-any.whl (9.3 kB)
Building wheels for collected packages: psutil
  Building wheel for psutil (pyproject.toml): started
  Building wheel for psutil (pyproject.toml): finished with status 'done'
  Created wheel for psutil: filename=psutil-5.9.3-cp311-cp311-macosx_10_9_universal2.whl size=270420 sha256=aa565121ab944073a8aede614b9369a0064e6cdfcb6fc7c97cf0f9c780f1f24b
  Stored in directory: /Users/runner/Library/Caches/pip/wheels/f7/62/d7/397c42c63b442586c3667a8facad29e859c6d081bf1161a359
Successfully built psutil
Installing collected packages: pyasn1, urllib3, uritemplate, six, rsa, pyparsing, pyjwt, pycparser, pyasn1-modules, psutil, protobuf, idna, charset-normalizer, cachetools, requests, httplib2, googleapis-common-protos, google-auth, cffi, oauth2client, google-auth-httplib2, google-api-core, cryptography, google-api-python-client, codemagic-cli-tools
Successfully installed cachetools-5.2.0 cffi-1.15.1 charset-normalizer-2.1.1 codemagic-cli-tools-0.36.2 cryptography-37.0.4 google-api-core-2.10.2 google-api-python-client-2.65.0 google-auth-2.14.0 google-auth-httplib2-0.1.0 googleapis-common-protos-1.56.4 httplib2-0.21.0 idna-3.4 oauth2client-4.1.3 protobuf-4.21.9 psutil-5.9.3 pyasn1-0.4.8 pyasn1-modules-0.2.8 pycparser-2.21 pyjwt-2.6.0 pyparsing-3.0.9 requests-2.28.1 rsa-4.9 six-1.16.0 uritemplate-4.1.1 urllib3-1.26.12

Notice:  A new release of pip available: 22.3 -> 22.3.1
Notice:  To update, run: /Library/Frameworks/Python.framework/Versions/3.11/bin/python -m pip install --upgrade pip
writing RSA key
/Users/runner/work/_temp/4f71b5b3-3bfa-44b6-a102-64964826b285.sh: line 5: keychain: command not found
Error: Process completed with exit code 127.

Any idea what went wrong? I see that your tools were updated recently, as well as macos on github from 11.7 to 11.7.1. Already tried dowgrade your tools version to 0.34 (which worked before)/updated macos to 12.6.1 - all this gave no effect.

Bug: `--testflight` flag for `app-store-connect publish` action has misleading documentation

The following comment in current documentation in https://github.com/codemagic-ci-cd/cli-tools/blob/master/docs/app-store-connect/publish.md is inaccurate as it's not possible to submit an app to external testers. Using this flag only makes the build available to internal testers.

-t, --testflight
Submit an app for Testflight beta app review to allow external testing

Crash in keychain initialize on Python 3.11

Hi,

First, thanks for your great job!
This issue seems equivalent to this one.

On Python 3.11, the output of keychain initialize is the following:

Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/Current/bin/keychain", line 5, in <module>
    from codemagic.tools import Keychain
  File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/codemagic/tools/__init__.py", line 1, in <module>
    from .android_app_bundle import AndroidAppBundle
  File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/codemagic/tools/android_app_bundle.py", line 35, in <module>
    class AndroidAppBundleArgument(cli.Argument):
  File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/codemagic/tools/android_app_bundle.py", line 65, in AndroidAppBundleArgument
    KEYSTORE_PATH_REQUIRED = KEYSTORE_PATH.duplicate(argparse_kwargs={'required': True})
                             ^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: 'tuple' object has no attribute 'duplicate'

Steps to reproduce on a Github action:

on:
  pull_request:

jobs:
  ko-on-python-11:
    runs-on: macos-11
    steps:
      - uses: actions/checkout@v3
      - uses: actions/setup-python@v4
        with:
          python-version: '3.11'

      - name: Install Codemagic CLI tools
        shell: bash
        run: |
          pip3 install codemagic-cli-tools
          xcrun xcodebuild -version

      - name: Set up a temporary keychain for code signing
        shell: bash
        run: |
          keychain initialize


  ok-on-python-10:
    runs-on: macos-11
    steps:
      - uses: actions/checkout@v3
      - uses: actions/setup-python@v4
        with:
          python-version: '3.10'

      - name: Install Codemagic CLI tools
        shell: bash
        run: |
          pip3 install codemagic-cli-tools
          xcrun xcodebuild -version

      - name: Set up a temporary keychain for code signing
        shell: bash
        run: |
          keychain initialize

Thanks for your support :)

Retry app-store-connect fetch-signing-files on error?

We have the following step in our codemagic.yaml file:

- name: Fetch signing files
  script: app-store-connect fetch-signing-files "$BUNDLE_ID" --type IOS_APP_STORE --create

Sometimes the build fails with the following error message:

GET https://api.appstoreconnect.apple.com/v1/bundleIds?limit=100&sort=name&filter%5Bidentifier%5D=bundle.id.redacted&filter%5Bplatform%5D=IOS returned 401: Authentication credentials are missing or invalid. - Provide a properly configured and signed bearer token, and make sure that it has not expired. Learn more about Generating Tokens for API Requests https://developer.apple.com/go/?id=api-generating-tokens

When we start the same build it usually works again. So it seems to be a temporary issue at api.appstoreconnect.apple.com.

Is there an easy way to retry the fetch-signing-files on error?

An additional parameter could be helpful:
app-store-connect fetch-signing-files "$BUNDLE_ID" --type IOS_APP_STORE --create --retries 3

Do not create keychains to temporary directory by default

Naively using CLI tools can have undesirable consequences for the mac keychain. We should work to avoid these for default usages.

My mistake was fixed with keychain make-default ~/Library/Keychains/login.keychain-db

I think that maybe we shouldn’t generate the keychain into $TMPDIR directory by default. when default keychain is not restored back to the system default login keychain and a reboot is performed, then the previous default keychain goes missing and system OS finds itself in a odd situation where default keychain is set, but it doesn’t exist.

The code changes are drafted in: https://github.com/codemagic-ci-cd/cli-tools/pull/125/files

--private-key: Provided value not valid

Hi,
I get the error message --private-key: Provided value not valid on this command app-store-connect fetch-signing-files $BUNDLE_ID --type IOS_APP_STORE --create
I've followed this instructions https://docs.codemagic.io/code-signing-yaml/signing-ios/

The file downloaded from apple it's a p8 extension file, I've encrypted it and added to the APP_STORE_CONNECT_PRIVATE_KEY variable.

Any idea why I'm getting this error message?
thanks

Error running `keychan.initialize`

Description

Suddenly the command keychain initialize started failing in our CI. I checked the libraries versions and tried to install previous one but also noticed that the android_app_bundle.py didn't change since 2021

Might be the version of Python? Any suggestions?

keychain initialize

Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.11/bin/keychain", line 5, in <module>
    from codemagic.tools import Keychain
  File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/codemagic/tools/__init__.py", line 1, in <module>
    from .android_app_bundle import AndroidAppBundle
  File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/codemagic/tools/android_app_bundle.py", line 35, in <module>
    class AndroidAppBundleArgument(cli.Argument):
  File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/codemagic/tools/android_app_bundle.py", line 65, in AndroidAppBundleArgument
    KEYSTORE_PATH_REQUIRED = KEYSTORE_PATH.duplicate(argparse_kwargs={'required': True})
                             ^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: 'tuple' object has no attribute 'duplicate'

Feature request: Using iTMSTransporter instead of xcrun altool for `publish` subcommand

This might be a stupid question, as I don't understand the upload process to App Store Connect too deeply, but I was wondering, if it's possible to publish the ipa packages using iTMSTransporter.

This will enable use of publish subcommand on other systems and will remove the requirement of running the publish command on macOS systems, as the tool can be run on Linux and Windows systems.

The motivation is primary to "off-load" the uploading process to Linux machines, which are much cheaper to run, especially when waiting for the post-processing on Apple server. When doing iOS builds, the app could be built on macOS machine, the ipa package can be uploaded as CI artifact and the pipeline could continue on Linux machine, picking up the IPA from artifacts and performing lengthy upload at much cheaper rate.

It could be implemented as an experimental flag parameter: app-store-connect publish --use-itms-transporter. Using this flag will basically replace xcrun altool calls with iTMSTransporter (ofc with other required changes).

Relevant resources

Allow use-profiles custom export options to use @file: reference

Currently, it is only possible to pass custom export options with CLI args. For example:

--custom-export-options='{"uploadBitcode": false, "uploadSymbols": false}'

Can we make it possible to reference a JSON file that contains the options like this:

--custom-export-options=@file:/path/to/file/export_options.json

Issues with Xcode 14 update for app-store-connect CLI tool

I updated to Xcode 14 yesterday and when I try to use the app-store-connect CLI tool, I am getting this error

File "/Users/Automation01/Library/Python/3.8/bin/app-store-connect", line 5, in
from codemagic.tools import AppStoreConnect
ModuleNotFoundError: No module named 'codemagic'

This is the error I get when I try to do --help command too

**Automation01 % app-store-connect --help
Traceback (most recent call last):
File "/Users/Automation01/Library/Python/3.8/bin/app-store-connect", line 5, in
from codemagic.tools import AppStoreConnect
ModuleNotFoundError: No module named 'codemagic' **

Can anyone points if this is due to the update to Xcode 14? Any solution to fix this error?

`xcode-project use-profiles` doesn't read inherited Info.plist setting

Hello, when we added flavors to our Flutter project our iOS code signing broke. We are using automatic code signing to deploy to the App Store.

Flavors are Flutter's name for different build configurations, which create different build schemes in the Xcode project. We used flutter_flavorizr to create the schemes, but our project then failed to build on Codemagic with this error:

Error (Xcode): "Runner" requires a provisioning profile. Select a provisioning profile in the Signing & Capabilities editor.

But everything in Xcode showed that the provisioning profile was correctly set for all build schemes.

I tracked this down to the xcode-project use-profiles being unable to find the INFOPLIST_FILE build setting, with the command outputting this in verbose mode:

Build configuration Release-flavor1 INFOPLIST_FILE is ''
Could not obtain bundle id value from base configuration reference
Failed to obtain bundle id for build_configuration 'Release-flavor1'
No bundle id found for target 'Runner'

This was due to the setting being inherited, rather than specified on each scheme.

Xcode showed the setting as being correct, because it was being inherited from the project:

But we needed to change it to be specified for each scheme for xcode-project use-profiles to be able to read the value:

Ideally xcode-project use-profiles should be able to calculate the inherited value, or at least provide a good error, as this took me quite a long time to track down 😁

We're not the only ones to face this, see this Stack Overflow question.

`app-store-connect` commands fail unexpectedly in case of invalid API key

In case invalid App Store Connect API key is defined via environment variables, then app-store-connect command invocations fail unexpectedly like

$ APP_STORE_CONNECT_PRIVATE_KEY='not a valid api key'
$ app-store-connect apps list
Executing AppStoreConnect action apps failed unexpectedly. Detailed logs are available at "/var/folders/vs/tcrc5cns67zgynxt6fssjdg80000gn/T/codemagic-24-11-21.log". To see more details about the error, add `--verbose` command line option.

Instead it should show actionable error message about what went wrong.

`xcode-project build-ipa` fails unexpectedly when export options file does not exist

Stacktrace:

[08:36:04 21-09-2021] WARNING cli_app.py:111 > Executing XcodeProject action build-ipa failed unexpectedly. Detailed logs are available at "/var/folders/m7/h1mg7c7x40ddjz6mxjxm3htr0000gn/T/codemagic-21-09-21.log". To see more details about the error, add `--verbose` command line option.
[08:36:04 21-09-2021] ERROR cli_app.py:113 > Exception traceback:
Traceback (most recent call last):
  File "/Users/builder/.pyenv/versions/3.8.7/lib/python3.8/site-packages/codemagic/cli/cli_app.py", line 202, in invoke_cli
    CliApp._running_app._invoke_action(args)
  File "/Users/builder/.pyenv/versions/3.8.7/lib/python3.8/site-packages/codemagic/cli/cli_app.py", line 156, in _invoke_action
    return cli_action(**action_args)
  File "/Users/builder/.pyenv/versions/3.8.7/lib/python3.8/site-packages/codemagic/cli/cli_app.py", line 455, in wrapper
    return func(*args, **kwargs)
  File "/Users/builder/.pyenv/versions/3.8.7/lib/python3.8/site-packages/codemagic/tools/xcode_project.py", line 198, in build_ipa
    export_options = ExportOptions.from_path(export_options_plist)
  File "/Users/builder/.pyenv/versions/3.8.7/lib/python3.8/site-packages/codemagic/models/export_options.py", line 189, in from_path
    with plist_path.open('rb') as fd:
  File "/Users/builder/.pyenv/versions/3.8.7/lib/python3.8/pathlib.py", line 1221, in open
    return io.open(self, mode, buffering, encoding, errors, newline,
  File "/Users/builder/.pyenv/versions/3.8.7/lib/python3.8/pathlib.py", line 1077, in _opener
    return self._accessor.open(self, flags, mode)
FileNotFoundError: [Errno 2] No such file or directory: '/Users/builder/export_options.plist'

DOCS default value for --pre-release-version=PRE_RELEASE_VERSION is missing

It would be good to document what build number are we fetching or what is the default value for

app-store-connect get-latest-testflight-build-number --pre-release-version=PRE_RELEASE_VERSION

Users sometimes get unexpected result from this command app-store-connect get-latest-testflight-build-number.

Invalid command and action names in docs

Commands and subcommands that contain hyphen '-' are not documented correctly, such as app-store-connect create-bundle-id.

The normal hyphen symbol is replaced with the non-breaking hypen (here and here) for markdown table formatting purposes.

This however has a consequence that copy-paste of those commands do not work as expected. Command and their names should not be altered in favor of formatting.

Facing issue connecting with test flight via app-store-connect CLI

Hi team,
I am trying to connect to test flight in app store connect using the command

app-store-connect \
    get-latest-testflight-build-number '${app_id}' \
    --issuer-id "${issuer_id}" \
    --key-id "${key_id}" \
    --private-key "${private_key}" \
    --platform=IOS \
    --pre-release-version "${app_version}"

But i keep running into the error below

GET https://api.appstoreconnect.apple.com/v1/preReleaseVersions?limit=100&include=builds&filter%5Bapp%5D=${app_id}&filter%5Bplatform%5D=IOS&filter%5Bversion%5D=${app_version} returned 401: Authentication credentials are missing or invalid. - Provide a properly configured and signed bearer token, and make sure that it has not expired. Learn more about Generating Tokens for API Requests https://developer.apple.com/go/?id=api-generating-tokens

I used the same command back in May but i never faced this issue. But now I am getting this token issue which according to the link https://developer.apple.com/go/?id=api-generating-tokens is for sending CURL request but not to use with the command app-store-connect.

Did something change recently? Can someone help me debug this issue?

I have admin level access in app store connect in case it is needed

Thank you,
Aravind

Document process to generate mypy stubs

Per this discussion, it would be nice to document the process for generation of mypy stubs with stubgen.

Feature request: Submit for review with app-store-connect

Extend app-store-connect to use the App Store Connect API to submit app for App Store review as described here:

https://developer.apple.com/documentation/appstoreconnectapi/app_store_version_submissions

This would reduce manual interaction with App Store Connect UI which would be advantageous for production workflows.

errors when using the python debugger

this only seems to occur when running with the python debugger in vscode:

the error doesn't occur when clicking "Run Python File"

code

from os import environ
from typing import Final

from codemagic.apple.app_store_connect.type_declarations import IssuerId, KeyIdentifier
from codemagic.tools.app_store_connect import AppStoreConnect
from dotenv import load_dotenv

load_dotenv()


app_store_connect: Final = AppStoreConnect(
    KeyIdentifier(environ["APP_STORE_CONNECT_KEY_IDENTIFIER"]),
    IssuerId(environ["APP_STORE_CONNECT_ISSUER_ID"]),
    private_key=environ["APP_STORE_CONNECT_PRIVATE_KEY"],
)

apps = app_store_connect.list_apps()

output

Traceback (most recent call last):
  File "c:\Users\amogus\IdeaProjects\proj\.venv\lib\site-packages\codemagic\apple\app_store_connect\json_web_token_manager.py", line 148, in get_jwt
    self._jwt = self._load_jwt_from_disk()
  File "c:\Users\amogus\IdeaProjects\proj\.venv\lib\site-packages\codemagic\apple\app_store_connect\json_web_token_manager.py", line 102, in _load_jwt_from_disk
    raise JwtCacheError('Disk cache is disabled')
codemagic.apple.app_store_connect.json_web_token_manager.JwtCacheError: Disk cache is disabled

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "c:\Users\amogus\IdeaProjects\proj\.venv\lib\site-packages\jwt\algorithms.py", line 416, in prepare_key
    key = load_pem_public_key(key)
  File "c:\Users\amogus\IdeaProjects\proj\.venv\lib\site-packages\cryptography\hazmat\primitives\serialization\base.py", line 30, in load_pem_public_key
    return ossl.load_pem_public_key(data)
  File "c:\Users\amogus\IdeaProjects\proj\.venv\lib\site-packages\cryptography\hazmat\backends\openssl\backend.py", line 853, in load_pem_public_key
    self._handle_key_loading_error()
  File "c:\Users\amogus\IdeaProjects\proj\.venv\lib\site-packages\cryptography\hazmat\backends\openssl\backend.py", line 1129, in _handle_key_loading_error
    raise ValueError(
ValueError: ('Could not deserialize key data. The data may be in an incorrect format, it may be encrypted with an unsupported algorithm, or it may be an unsupported key type (e.g. EC curves with explicit parameters).', [_OpenSSLErrorWithText(code=151584876, lib=9, reason=108, reason_text=b'error:0909006C:PEM routines:get_name:no start line')])

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Users\amogus\AppData\Local\Programs\Python\Python310\lib\runpy.py", line 196, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "C:\Users\amogus\AppData\Local\Programs\Python\Python310\lib\runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "c:\Users\amogus\.vscode\extensions\ms-python.python-2022.4.1\pythonFiles\lib\python\debugpy\__main__.py", line 45, in <module>
    cli.main()
  File "c:\Users\amogus\.vscode\extensions\ms-python.python-2022.4.1\pythonFiles\lib\python\debugpy/..\debugpy\server\cli.py", line 444, in main
    run()
  File "c:\Users\amogus\.vscode\extensions\ms-python.python-2022.4.1\pythonFiles\lib\python\debugpy/..\debugpy\server\cli.py", line 285, in run_file
    runpy.run_path(target_as_str, run_name=compat.force_str("__main__"))
  File "C:\Users\amogus\AppData\Local\Programs\Python\Python310\lib\runpy.py", line 269, in run_path
    return _run_module_code(code, init_globals, run_name,
  File "C:\Users\amogus\AppData\Local\Programs\Python\Python310\lib\runpy.py", line 96, in _run_module_code
    _run_code(code, mod_globals, init_globals,
  File "C:\Users\amogus\AppData\Local\Programs\Python\Python310\lib\runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "c:\Users\amogus\IdeaProjects\proj\proj\__init__.py", line 17, in <module>
    apps = app_store_connect.list_apps()
  File "c:\Users\amogus\IdeaProjects\proj\.venv\lib\site-packages\codemagic\cli\cli_app.py", line 417, in wrapper
    return func(*args, **kwargs)
  File "c:\Users\amogus\IdeaProjects\proj\.venv\lib\site-packages\codemagic\tools\_app_store_connect\action_groups\apps_action_group.py", line 71, in list_apps
    return self._list_resources(
  File "c:\Users\amogus\IdeaProjects\proj\.venv\lib\site-packages\codemagic\tools\_app_store_connect\resource_manager_mixin.py", line 45, in _list_resources
    resources = resource_manager.list(resource_filter=resource_filter)
  File "c:\Users\amogus\IdeaProjects\proj\.venv\lib\site-packages\codemagic\apple\app_store_connect\apps\apps.py", line 62, in list
    apps = self.client.paginate(f'{self.client.API_URL}/apps', params=params)
  File "c:\Users\amogus\IdeaProjects\proj\.venv\lib\site-packages\codemagic\apple\app_store_connect\api_client.py", line 105, in paginate
    return self._paginate(url, params, page_size, limit).data
  File "c:\Users\amogus\IdeaProjects\proj\.venv\lib\site-packages\codemagic\apple\app_store_connect\api_client.py", line 91, in _paginate
    response = self.session.get(url, params={'limit': page_size, **params}).json()
  File "c:\Users\amogus\IdeaProjects\proj\.venv\lib\site-packages\requests\sessions.py", line 542, in get
    return self.request('GET', url, **kwargs)
  File "c:\Users\amogus\IdeaProjects\proj\.venv\lib\site-packages\codemagic\apple\app_store_connect\api_session.py", line 48, in request
    headers.update(self._auth_headers_factory())
  File "c:\Users\amogus\IdeaProjects\proj\.venv\lib\site-packages\codemagic\apple\app_store_connect\api_client.py", line 71, in generate_auth_headers
    return {'Authorization': f'Bearer {self.jwt}'}
  File "c:\Users\amogus\IdeaProjects\proj\.venv\lib\site-packages\codemagic\apple\app_store_connect\api_client.py", line 67, in jwt
    jwt = self._jwt_manager.get_jwt()
  File "c:\Users\amogus\IdeaProjects\proj\.venv\lib\site-packages\codemagic\apple\app_store_connect\json_web_token_manager.py", line 151, in get_jwt
    self._jwt = self._generate_jwt()
  File "c:\Users\amogus\IdeaProjects\proj\.venv\lib\site-packages\codemagic\apple\app_store_connect\json_web_token_manager.py", line 136, in _generate_jwt
    token = self._encode_token(payload)
  File "c:\Users\amogus\IdeaProjects\proj\.venv\lib\site-packages\codemagic\apple\app_store_connect\json_web_token_manager.py", line 85, in _encode_token
    return jwt.encode(
  File "c:\Users\amogus\IdeaProjects\proj\.venv\lib\site-packages\jwt\api_jwt.py", line 63, in encode
    return api_jws.encode(json_payload, key, algorithm, headers, json_encoder)
  File "c:\Users\amogus\IdeaProjects\proj\.venv\lib\site-packages\jwt\api_jws.py", line 113, in encode
    key = alg_obj.prepare_key(key)
  File "c:\Users\amogus\IdeaProjects\proj\.venv\lib\site-packages\jwt\algorithms.py", line 418, in prepare_key
    key = load_pem_private_key(key, password=None)
  File "c:\Users\amogus\IdeaProjects\proj\.venv\lib\site-packages\cryptography\hazmat\primitives\serialization\base.py", line 22, in load_pem_private_key
    return ossl.load_pem_private_key(data, password)
  File "c:\Users\amogus\IdeaProjects\proj\.venv\lib\site-packages\cryptography\hazmat\backends\openssl\backend.py", line 823, in load_pem_private_key
    return self._load_key(
  File "c:\Users\amogus\IdeaProjects\proj\.venv\lib\site-packages\cryptography\hazmat\backends\openssl\backend.py", line 1070, in _load_key
    self._handle_key_loading_error()
  File "c:\Users\amogus\IdeaProjects\proj\.venv\lib\site-packages\cryptography\hazmat\backends\openssl\backend.py", line 1129, in _handle_key_loading_error
    raise ValueError(
ValueError: ('Could not deserialize key data. The data may be in an incorrect format, it may be encrypted with an unsupported algorithm, or it may be an unsupported key type (e.g. EC curves with explicit parameters).', [_OpenSSLErrorWithText(code=151584876, lib=9, reason=108, reason_text=b'error:0909006C:PEM routines:get_name:no start line')])

Allow `IOS_DISTRIBUTION` certificate type for creation with `app-store-connect fetch-signing-files`

With the current approach (version 0.27.0), when calling

app-store-connect fetch-signing-files

with --type IOS_APP_STORE, if a matching certificate was not found DISTRIBUTION certificate will be created.
Please allow specifying certificate type used for creation.
Because of Apple limitations, we cannot use more than 3 DISTRIBUTION certificates and we want to be able to generate IOS_DISTRIBUTION ones automatically.

export_options.plist not found when using codemagic.yaml

I was using the Workflow Editor to build and deploy a Flutter project. Everything was fine and the workflow was publishing successfully both for Android and iOS. Then we decided to use codemagic.yaml instead of the Workflow Editor.

When switching, I checked the Export current configuration as codemagic.yaml option and committed the generated codemagic.yaml to the repo. I expected that this file should be equivalent to the configurations I set in the UI. However I got the following error:

"~/export_options.plist" property list does not exist.


Build failed :|
Step 12 script `cd app && flutter build ipa --export-options-plist=~/export_options.plist --buil ...` exited with status code 1

as a result of executing this command:

#!/usr/bin/env bash

cd app && flutter build ipa --export-options-plist=~/export_options.plist --build-name=2.0.$PROJECT_BUILD_NUMBER --build-number=$PROJECT_BUILD_NUMBER --flavor stage -t lib/main/main_stage.dart

The command above was in the generated codemagic.yaml .
The build-name, build_number, flavor and target params were as per my Workflow Editor, but not the export-options-list, I didn't add that one.

I understand that export_options.plist is the output of xcode-project use-profiles. By default, the previous command should generate the desired file in $HOME/export_options.plist. Which is the same as the path passed to cd app && flutter build ipa --export-options-plist above.

I echoed $HOME and got /Users/builder as expected.

Finally, I changed --export-options-plist=~/export_options.plist in the flutter build ipa command to --export-options-plist=/Users/builder/export_options.plist and it worked.

Here's the relevant part of generated codemagic,yaml file that I get from exporting the configurations of my Workflow Editor without any change. The same workflow was working fine.

workflows:
############################################################
########################## Stage ###########################
############################################################
  stage:
    name: stage
    max_build_duration: 60
    environment:
      vars:
        dummy_env: Encrypted( encryptedValue123....end)
        env: Encrypted( encryptedValue123....end)
        FCI_KEYSTORE_PATH: /tmp/keystore.keystore
        FCI_KEYSTORE: Encrypted( encryptedValue123....end)
        FCI_KEYSTORE_PASSWORD: Encrypted( encryptedValue123....end)
        FCI_KEY_PASSWORD: Encrypted( encryptedValue123....end)
        FCI_KEY_ALIAS: Encrypted( encryptedValue123....end)
        APP_STORE_CONNECT_ISSUER_ID: cXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXX7
        APP_STORE_CONNECT_KEY_IDENTIFIER: BXXXXXXXX9
        APP_STORE_CONNECT_PRIVATE_KEY: Encrypted( encryptedValue123....end)
        CERTIFICATE_PRIVATE_KEY: Encrypted( encryptedValue123....end)
      flutter: default
      xcode: latest
      cocoapods: default
    cache:
      cache_paths: []
    scripts:
      - |
        # set up key.properties
        echo $FCI_KEYSTORE | base64 --decode > $FCI_KEYSTORE_PATH
        cat >> "$FCI_BUILD_DIR/app/android/key.properties" <<EOF
        storePassword=$FCI_KEYSTORE_PASSWORD
        keyPassword=$FCI_KEY_PASSWORD
        keyAlias=$FCI_KEY_ALIAS
        storeFile=/tmp/keystore.keystore
        EOF
      - |
        # set up local properties
        echo "flutter.sdk=$HOME/programs/flutter" > "$FCI_BUILD_DIR/app/android/local.properties"
      - cd app && flutter packages pub get
      - | 
        #the next two lines are to set our dummy_env values as codemagic env variables
        echo $dummy_env | base64 --decode > app/.env
        grep -v -e '^[[:space:]]*$' app/.env | grep -v '^#' >> $CM_ENV
      - cd app && flutter build appbundle --release --build-name=2.0.$PROJECT_BUILD_NUMBER
        --build-number=$PROJECT_BUILD_NUMBER --flavor stage -t lib/main/main_stage.dart
      - find . -name "Podfile" -execdir pod install \;
      - keychain initialize
      - app-store-connect fetch-signing-files "com.company.appname.stage" --type
        IOS_APP_STORE --create
      - keychain add-certificates
      # I think the next two commands are what's relavent to this issue
      - xcode-project use-profiles
      - cd app && flutter build ipa --export-options-plist=~/export_options.plist
        --build-name=2.0.$PROJECT_BUILD_NUMBER --build-number=$PROJECT_BUILD_NUMBER
        --flavor stage -t lib/main/main_stage.dart
      - echo $PROJECT_BUILD_NUMBER
    artifacts:
      - app/build/**/outputs/apk/**/*.apk
      - app/build/**/outputs/bundle/**/*.aab
      - app/build/**/outputs/**/mapping.txt
      - app/build/ios/ipa/*.ipa
      - /tmp/xcodebuild_logs/*.log
      - app/*.snap
      - app/build/windows/**/*.msix
      - app/flutter_drive.log
    publishing:
      email:
        recipients:
          - [email protected]
      google_play:
        credentials: Encrypted( encryptedValue123....end)
        track: internal
        in_app_update_priority: 0
      app_store_connect:
        api_key: Encrypted( encryptedValue123....end)
        key_id: BXXXXXXXX9
        issuer_id: cXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXX7
        submit_to_testflight: true
        submit_to_app_store: false
        
############################################################
########################## Prod ############################
############################################################

  prod:
    name: prod
    max_build_duration: 60
    environment:
    ## ...........
    ## ...........
    ## ...........

Am I doing something wrong? or the default output path of xcode-project use-profiles needs to be fixed?

Uploading with app-store-connect and `--max-build-processing-wait 0` results in non-zero exit code

I've noticed that passing in --max-build-processing-wait 0 to app-store-connect results in a non-zero completion code. I would think that if I deliberately request to skip waiting on processing, the command should succeed after uploading. Is this line of thinking correct?

app-store-connect publish \
  --path build/ios/ipa/*.ipa \
  --max-build-processing-wait 0 \
  --testflight

Signing Certificate type `IOS_DISTRIBUTION` doesn't fall back to `DISTRIBUTION`

I created DISTRIBUTION Signing Certificate and a corresponding provisioning profile. When I try to fetch signing files I get the following error

app-store-connect fetch-signing-files --platform IOS --type IOS_APP_STORE --strict-match-identifier io.codemagic.capybara

Found 1 Bundle ID matching specified filters: identifier=io.codemagic.capybara, platform=IOS.
Found 3 Signing Certificates matching specified filters: certificateType=IOS_DISTRIBUTION.
Did not find any Signing Certificates for given private key
Did not find IOS_DISTRIBUTION Signing Certificates

I would expect that DISTRIBUTION Signing Certificates will be fetched if IOS_DISTRIBUTION Signing Certificate were not found

documentation link for `Profiles.get_bundle_id_resource_id` is dead

cli-tools/src/codemagic/apple/app_store_connect/provisioning/profiles.py

Line 122 in 5058be6

    
                   https://developer.apple.com/documentation/appstoreconnectapi/get_the_bundle_resource_id_in_a_profile

Update docs about CLI options that can be defined using files or environment variables

In the description for --beta-build-localizations=BETA_BUILD_LOCALIZATIONS indocs/app-store-connect/publish.md, minor text edits are required. Here's the edited version:

Localized beta test info for what's new in the uploaded build as a JSON encoded list. For example, [{"locale": "en-US", "whats_new": "What's new in English"}]. See --locale for possible locale options. If not given, the value will be checked from the environment variable APP_STORE_CONNECT_BETA_BUILD_LOCALIZATIONS. Alternatively to entering BETA_BUILD_LOCALIZATIONS in plaintext, it may also be specified using the @env: prefix followed by an environment variable name, or the @file: prefix followed by a path to the file containing the value. Example: @env:<variable> uses the value in the environment variable named <variable>, and @file:<file_path> uses the value from file at <file_path>.

Link to diff: https://github.com/codemagic-ci-cd/cli-tools/pull/124/files#r660574082

Add option to specify custom xcodebuild flags when building ipa

When building ipa, it is sometimes necessary to define some custom flags or build options that would be sent directly to xcodebuild invokation. Much like it is currently possible with xcpretty options.

keychain initialize --path=$(mktmp) fails keychain initialize --path=$(mktmp).keychain-db works

keychain initialize --path=$(mktmp) fails keychain initialize --path=$(mktmp).keychain-db works. It is a bit irritating for me that i explicitly need to end the path on .keychain-db and that i cant just provide a path

handle App Store Connect API key is missing header and footer in app-store-connect commands

Sometimes users pass in a private key missing the -----BEGIN PRIVATE KEY----- or -----END PRIVATE KEY----- header or footer. In app-store-connect commands, it would be nice to handle this case: either return a helpful error message or just attempt processing the private key if these are missing.

Feature request: Add option to `app-store-connect fetch-signing-files` to exactly match specified bundle identifier

Please consider adding a toggle to make the BUNDLE_ID matching strict, instead of wildcard, for the following card

app-store-connect fetch-signing-files "$BUNDLE_ID" --type IOS_APP_STORE --create

In our case we have apps with bundle_ids "com.company.project" and "com.company.project.test".
When fetching provisioning profiles for the first it would also return for the second.

Suggeste solution could be a toggle for making the matching strict

app-store-connect fetch-signing-files "$BUNDLE_ID" --match-strict --type IOS_APP_STORE --create

When Cocoapods is not installed, xcode-project shows "Failed to set code signing settings for x" with no further information, even with verbose output

I am trying to build a Flutter project. I have successfully added my profiles to my keychain,
but when I run xcode-project use-profiles I get the error:

Failed to set code signing settings for _[my_project]_.xcodeproj

Sadly the --verbose option doesn't give any extra output. Does anyone know how I can debug this issue?

MacOS 12.0.1
XCode 13.4.1
xcode-project version 0.34.1
Python 3.8

EDIT: I think I resolved this. The problem was that I was using a fresh machine and the xcodeproj gem was not installed, which in turn was because I had yet to install CocoaPods. It would be nice at the very least if the verbose option actually gave me the ruby script output, since this helped me diagnose it. Even better would be a message to tell me that xcodeproj or Cocoapods needs installed. I'm leaving this ticket open and renaming it to reflect this.

xcode-project build-ipa CODE_SIGN_IDENTITY Xcode 14 Issue

xcode-project build-ipa automatically adds a flag, 'CODE_SIGN_IDENTITY=iPhone Distribution', and this causes issues with SPM in Xcode 14. Please find more info in this Swift forums thread.

In our case, xcode-project build-ipa created a command like this:

xcodebuild -workspace <workspace> -scheme <scheme> -config Release -archivePath <path> archive COMPILER_INDEX_STORE_ENABLE=NO DEVELOPMENT_TEAM=<dev team> 'CODE_SIGN_IDENTITY=iPhone Distribution'

and there is a resulting error:

/Users/builder/Library/Developer/Xcode/DerivedData/.../SourcePackages/checkouts/.../Package.swift: error: <package>_<library> has conflicting provisioning settings. <package>_<library> is automatically signed for development, but a conflicting code signing identity iPhone Distribution has been manually specified. Set the code signing identity value to "Apple Development" in the build settings editor, or switch to manual signing in the Signing & Capabilities editor. (in target '<package>_<library>' from project '<project>')

add documentation which commands are macOS specific

For running CLI tools on asynchronous services, we should mark in CLI tools which commands require macOS/XCode or can be ran on Linux (or Windows?)

My idea, add a tag or line in each of the command documentation that it is macOS specific and/or can be ran on Linux. We need to test the commands and make sure they work on different os. I.e. app-store-connect should be platform independent while xcode-project will surely require macOS.

Remove existing builds from review with `publish`

Is it possible to remove any pending builds from review with the publish command? We end up having a decent number of builds get rejected because there's already a build pending for review which makes our testflight not match the latest version.

Error when executing google-play get-latest-build-number --> TypeError: init() missing 1 required positional argument: 'includeRestOfWorld'

The other day I run google-play get-latest-build-number --package-name com.company.myapp and received error and version code for Production track wasn't returned. Version code for Alpha and Beta tracks were returned.

It turns out that last version of AAB on Production track has been rolled out only to two countries. When I rolled it out to all countries google-play get-latest-build-number --package-name com.company.myapp started work as expected.

Is there a way to overcome this behavior/ situation when AAB is rolled out to just a few countries and still get version code back?

Output of command google-play get-latest-build-number --package-name com.company.myapp when AAB was rolled out to only few countryes

Get information about the track "internal" for the package "com.company.myapp"
Failed to get version code from Google Play from internal track. No releases with uploaded App bundles or APKs
Get information about the track "alpha" for the package "com.company.myapp"
Latest version code for alpha track: 300012760
Get information about the track "beta" for the package "com.company.myapp"
Latest version code for beta track: 300012804
Get information about the track "production" for the package "com.company.myapp"
Executing GooglePlay action get-latest-build-number failed unexpectedly. Detailed logs are available at "/tmp/codemagic-14-03-22.log". To see more details about the error, add `--verbose` command line option.

the /tmp/codemagic-14-03-22.log

Traceback (most recent call last):
  File "/home/docker/.local/lib/python3.8/site-packages/codemagic/cli/cli_app.py", line 200, in invoke_cli
    CliApp._running_app._invoke_action(args)
  File "/home/docker/.local/lib/python3.8/site-packages/codemagic/cli/cli_app.py", line 156, in _invoke_action
    return cli_action(**action_args)
  File "/home/docker/.local/lib/python3.8/site-packages/codemagic/cli/cli_app.py", line 412, in wrapper
    return func(*args, **kwargs)
  File "/home/docker/.local/lib/python3.8/site-packages/codemagic/tools/google_play.py", line 143, in get_latest_build_number
    track = self.api_client.get_track_information(edit.id, track_name)
  File "/home/docker/.local/lib/python3.8/site-packages/codemagic/google_play/api_client.py", line 92, in get_track_information
    resource = Track(**track_response)
  File "<string>", line 5, in __init__
  File "/home/docker/.local/lib/python3.8/site-packages/codemagic/google_play/resources/track.py", line 74, in __post_init__
    self.releases = [Release(**release) for release in self.releases]
  File "/home/docker/.local/lib/python3.8/site-packages/codemagic/google_play/resources/track.py", line 74, in <listcomp>
    self.releases = [Release(**release) for release in self.releases]
  File "<string>", line 10, in __init__
  File "/home/docker/.local/lib/python3.8/site-packages/codemagic/google_play/resources/track.py", line 59, in __post_init__
    self.countryTargeting = CountryTargeting(**self.countryTargeting)
TypeError: __init__() missing 1 required positional argument: 'includeRestOfWorld'```

codemagic-ci-cd / cli-tools Goto Github PK

cli-tools's People

Contributors

Stargazers

Watchers

Forkers

cli-tools's Issues

Description

Description

Relevant resources

code

output

Recommend Projects

Recommend Topics

Recommend Org