• How feasible it is to use simple class names for span names in order to avoid very long span names. Would it cause any issues? Is there a problem with using long span names?
• Is it needed to stretch the root span in order to cover all of its children? Is there a problem with having a root span shorter than the sum of its children?

Use:

• event.name = crash
• event.domain = device

to denote crashes

Related to https://github.com/elastic/sdh-apm/issues/817

Currently, the Android agent yields the following host.os.* fields:

• host.os.full, host.os.platform and host.os.version are fine ✅
• host.os.name needs to be Android. Therefore, the agent needs to set the os.name resource attribute in OpenTelemetry ⚠️
• host.os.type will be properly derived from host.os.name (once this is set properly) by the APM server ✅

When developing the Android APM Agent we want to test new features and approaches by means of a real Android app that uses popular, representative technologies. Moreover, we need an application that can be used for demo purposes of the Android APM agent.

Therefore, we need a simple demo Android application based on our opbeans demo-system.
The application should have comparable functionality as the swift-opbeans app:

• list products
• view for product details
• cart feature
• checkout

• Service name.
• Service version.
• Http requests filtering.
• Providing cell network subtype or not, if yes, need to add READ_PHONE_STATE permission, needed to call this method.

Product Checklist

This checklist should be used by existing and newly-acquired products to ensure compliance with Elastic's open-source policy.

• Product build produces a list of dependencies included both in source code and in the product distribution
• Manual investigation has ensured that all dependencies are represented in the list
• Each dependency reviewed and either in the Green List or has an exception (Note that the Yellow List contains a few automatic exceptions)
• Product repositories contain required notices
• Product source code files all have appropriate license headers
• Third-party code contained in product repositories (including vendored code and copy/pasted code) is appropriately marked and exports NOTICE.txt and license information like other dependencies
• Product distribution contains NOTICE.txt file consists of the following items concatenated together:
  • Contents of product's NOTICE.txt file
  • For each dependency, contents of dependency's NOTICE.txt (if existing)
  • For each dependency, the full and exact license text copied from dependency's distribution/repository
• Manual investigation has ensured that the build system does not use any components with licenses on the Red list

For products not using release manager

• - Each build creates a file describing all dependencies

Optional but encouraged

• - The product build does not permit adding a dependency without automatic export in the list

The way we instrument OkHttp calls is through the registration of an EventListener factory into the OkHttpClient instance. Each OkHttpClient instance can only have 1 EventListener factory, so we must wrap and existing one (if any) in order to delegate calls to it through our own factory which will override any previously set factory.

We need to send the network type/subtype and the mobile carrier info on every Span

Now:
co.elastic.apm.opbeans.modules.customers.CustomersFragment - View appearing
Proposing:
CustomersFragment (co.elastic.apm.opbeans.modules.customers) - View appearing

As described in the spec, the http.client_ip is mapped to client.ip field. For the client.ip field the APM ingest pipeline derives the geo location for the IP. This is a valuable fallback for the geo location information in case the mobile agent cannot provide the geo information directly (e.g. due to missing permission to retrieve the geo location).

The Android agent should add the http.client_ip field to the spans to leverage the above.

We need to come up with a way to debug our Agent so that our customers can provide us with our Agent's logs easily. It should be possible to get those manually if needed, but ideally we should be able to get those automatically, probably by sending our own logs straight to their backends if they enable "debug mode" on our agent.

Onboarding the Android APM Agent should be as easy as possible. While runtime bytecode instrumentation (BCI) is not possible on Android, we can leverage compile time BCI using Android Buddy.

Steps

• Initialize Android APM Agent as an Android Buddy-based Gradle plugin
• Provide instrumentation plugins for popular (with Android) HTTP clients
• Collect Resource Attributes
• Make instrumentation and BCI work with both Java-based and Kotlin-based Android apps.
• Provide instrumentation plugins for Views and their lifecycle

Each instrumented library should have its own tracer. And, tracer instances should be named by the library name they instrument (e.g. OkHttp, Jetpack Compose, Android Views, etc.). Also, the version of the framework should mirror the version of the instrumented library (if possible).

The current name of the tracer is: ElasticApmAgent-tracer

This name is mapped by the APM server to the service.framework.name field.

We need to provide ways to configure the Agent through environment variables the same way is done for the Java Agent - So we'd need to add support for the following params that would override the gradle config params (if any):

ELASTIC_APM_SERVICE_NAME
ELASTIC_APM_SERVICE_VERSION
ELASTIC_APM_SERVER_URL
ELASTIC_APM_SECRET_TOKEN

Also, as mentioned here we also need to rename the current gradle's config param serverToken by secretToken.

We must add logs that show the origin of the config.

Closes #71

The Android gradle plugin API is currently going through big changes, one of them is related to the generation of assets files, needed to provide compile time information at runtime, such as service name, version and server parameters. The API for assets generation has changed in the latest version of the AGP (7.3.0), which makes our plugin to fail at compile time in projects that are using said version.

Most likely these kinds of issues will keep coming in future AGP releases, so ideally we should provide a flexible mechanism that would allow us to gracefully deal with them without having to discard host projects using older versions of the AGP.

For reference, this is the way it was done on version 7.2: https://github.com/android/gradle-recipes/blob/agp-7.2/Kotlin/addCustomAsset/app/build.gradle.kts

And this is the new API for version 7.3: https://github.com/android/gradle-recipes/blob/agp-7.3/Kotlin/addCustomAsset/app/build.gradle.kts

This issue explains the steps required to publish documentation to the Elastic documentation build powered by elastic/docs. Let me know when you're ready and I can get started on these tasks.

Phase one

Create basic docs. These docs will disconnected from the doc build for this phase. Docs should (mostly) follow the slightly outdated Agent documentation template. Here's a recent example: elastic/apm-agent-php#48.

• #25

Phase two

We have content! Now we can add elastic/apm-agent-android to Elastic's doc build and publish.

• Open a PR in elastic/docs to add to the build
• Open infra ticket to add the elasticsearch/ci docs check to this repo
• Add the elasticsearch/ci docs webhook to the repo
• Update elastic/apm-contrib elastic/apm-contrib#61
• Update elastic/apm elastic/apm#721
• Update elastic/apm-dev https://github.com/elastic/apm-dev/pull/898
• Add page header #52
• Remove any remaining documentation content from readme and link to published docs

Phase three

Hook the Android Reference into the rest of the Stack documentation. No list as docs change frequently, but inspiration can be taken from elastic/observability-docs#383 and elastic/observability-docs#792.

• Link from stack documentation to the Android Reference.

To continuously test the agent, we need to have a load driver for the opbenas-android app that would generate real load (and though that agent data) that we can use for end-to-end testing and demoing.

Allow session ID idle time to be configurable through central config

To 4 hours

OTLP allows for data transfer to be throttled in case of server overload.
https://opentelemetry.io/docs/reference/specification/protocol/otlp/#otlphttp-throttling

If this happens for an extended period of time the Java sdk may start dropping data after a number of retries. A metric should record the count of traces dropped when this happens.

Fetching config remotely and applying it on app start/at runtime.

Q. Do we need a second TCP connection?

Notes: Backend do fetch every 30 secs

Currently in order to connect the Android agent to an Elastic cloud instance, users need to pass their instance's endpoint + its auth token. Currently their auth token is passed in plain text, we need to figure ways to make it more secure.

• Use single Byte buddy version across all the subprojects.
• Use release variant as maven publish module of the android-sdk subproject.
• Check otel desugaring.
• Check App Bundle files instrumentation.
• Android assets generation compatible with all 7.2+ AGP versions
• Make source headers generate on assembling
• Update loggers default name
• Update android logger level debug config
• Running android-test from terminal
• Adding tests for api breaking changes
• Setting span type to "mobile"
• Adding process.runtime attrs to resources
• Adding okhttp spans tests
• Adding "postDeploy" task

We need a way to tell that the current NOTICE file is aligned with the current project's dependencies. So there should be a Gradle task that verifies whether the current NOTICE file outdated or if it's fine as it is. If it's outdated, the task should yield an exception that suggests to re-run the createNoticeFile task in order to update the project's NOTICE files.

This a first step towards an end-to-end POC of instrumenting an Android application.
Use the OpenTelemetry API and SDK to enrich the opbeans-android App with instrumentation code.

The goal is to see end-to-end tracing data in Kibana from view loads / user interactions in the app through http requests into the backend and correlation with the backend traces.

Tasks

• create a separate git branch on opbeans-android
• setup OTel SDK within opbeans-android
• instrument views
• instrument http requests
• validate tracing data in Kibana

R8 removes unused method params, so it essentially changes methods' signatures, which breaks Byte Buddy's @Advice.Origin Method reflection on the method androidx.fragment.app.Fragment#onViewCreated (its second parameter of type Bundle is removed since it's unused). So we need to find an alternative to using reflection on this Advice, or suggesting our customers to use ProGuard rules to avoid method signature's changes.

• Provide utilities for good onboarding experience (i.e. configuration and setup of the agent on a "happy path")
• Write getting started docs - #25

https://www.elastic.co/guide/en/apm/agent/java/current/configuration.html

The Open Telemetry Java SDK mentions that for Android projects, the desugaring config must be enabled.

Java language APIs/features apis are embedded in the Android OS, so when new ones are added into newer JDK versions (like what happened with version 8 where lambdas were added), a new Android OS version was needed to add support for them at runtime. However, some libraries (like OTel's), make use of "newer" JDK APIs, such as the java.time.Duration in this case, which is available from the Android OS 8 (API level 26), so in devices with OS version < 8, there will be a crash of type "NoClassDefFoundError" when running them.

The way Google came up with a solution to avoid these issues, was to add an optional compilation process where new APIs would get "desugared" in order to make them compatible for older OS versions. They have a list of supported APIs for desugaring here.

So if our customers are targeting Android OS versions < 26, they must opt in for desugaring in their app's build.gradle file.

According to Google's guide on how to add support for desugaring, it seems like it's a 2-step process, the first one is to enable a flag, and the second one is to add a dependency. However, the dependency's version to set cannot be the latest one all the time, since it depends on the version of the Android Gradle plugin that the project is on. So we must make sure to tell our customers to check on their project's AGP version before adding said library, or we could add all the "desugaring" configuration for them through our Gradle plugin, where we'd check our customer's AGP version and apply the best config for them automatically.

Use NTP for the clock to retrieve accurate timestamps.
When an NTP server is not available fall back to the device clock instead.

Through https://github.com/instacart/truetime-android

With the most recent change on Byte Buddy for Android, the previous way to delegate from the CompositeEventListener class over to the array of listeners passed to it, has gotten broken since now the Byte Buddy gradle plugin for Android cannot generate new methods inside a type, it can only decorate existing ones instead, so the option to generate the delegating methods is no longer viable.

We need to find and apply an alternative way of delegating from CompositeEventListener to all of its listeners.

When using the OpenTelemetry API / SDK for Android, we need to have some logic that initializes OpenTelemetry with all the providers (Tracing, Resource, etc.). This initialization should be encapsulated into some agent initialization logic that would take care of also setting all the resource attributes as described in the agent spec.

Note: This issue does not cover any automatic instrumentation yet.

• Collect versionCode to map file to spans.

We're using the following as screen lifecycle's root span name:

• The screen's title.
• Or The screen's class name (if there's no title available).

We should also provide a way for users to define their own name for said span, which would override anything else.

Phase 1:

• Write a test script that can be executed in SauceLabs -#10
  • on a simulator
  • on the device farm
• Automate test execution to make it run periodically (e.g. once every X minutes, X to be defined)
  • add saucelabs credentials to vault
  • automated script that pulls edge + saucelabs creds from vault to automate saucelabs run.
  • add build process that uploads app file to saucelabs for testing

Phase 2:

• Parameterize test scripts OR randomize test execution, i.e. run test on/with different ...
  • ... device models
  • ... OS versions
  • ... geo locations
  • ... opbeans-android app versions

Adding android support to the official byte buddy gradle plugin.

Provide central config way of discarding HTTP spans

Initialize the Android Agent project (basic project structure, CI, etc.)

Provide central config option to rename HTTP requests based on their URL format to group them with a single span name.

Ref. https://www.elastic.co/guide/en/apm/agent/java/current/config-core.html#config-transaction-name-groups

As defined in here: elastic/apm#771

• Instrument Activity lifecycle.
• Instrument Fragment lifecycle.
• Propagate Context across coroutines.
• Try to use screen title when available.