google-deepmind / android_env Goto Github PK

View Code? Open in Web Editor NEW

992.0 992.0 70.0 190.71 MB

RL research on Android devices.

License: Apache License 2.0

Python 100.00%

android reinforcement-learning

android_env's People

Contributors

Stargazers

Watchers

Forkers

inarikami cakiki cyborn-in edbeeching ricklentz beuaaa hephaex khanzo monskim999 ifeanyi-omeck sinaqahremani hmidani-abdelilah majacq longjohncoder cnheider shahip2016 gwanwoochoi sts-sadr simenglv tantara honeypotz-ai msoftware smbale murilo crackercat sumo43 eru-test daisuke-nomura stjordanis ivanpuhachov mmervecerit cloudenginehub suhani247 bothdev yjaeseok 999666bryce codeslord silencewood pedroysb bassel-kh neverparadise afirez cybersecsi nikehop dude-with-the-long-username gg-big-org pinkdiamond1 andresmcio shubhamojha1000 ufarooq tarpol tejasghinmine009 alsdyd98 gimme1dollar tongtongx hanshan12 ilyamk apollohuang1 ghtali mszczodrak qtanweer symoon11 tuhdo reeshogue22 tmmichi nitheshkv96 epinnock krishmatta

android_env's Issues

Emulator crashes on load()

After following the instructions for set up, I run the android emulator along with the following code:

env = android_env.load(
    avd_name='pixel2_30',
    android_avd_home='/path/.android/avd',
    android_sdk_root='/path/Android/Sdk',
    emulator_path='/path/Android/Sdk/emulator/emulator',
    adb_path='/path/Android/Sdk/platform-tools/adb',
    task_path='/path/android_env/tasks/accessibility_forwarder_clock_set_timer.textproto',
)

And the emulator crashes:

Traceback (most recent call last):
  File "/path/android_env/android_env/components/emulator_launcher.py", line 137, in launch
    self._emulator.expect('emulator: INFO: boot completed', timeout=wait_time)
  File "/path/anaconda3/lib/python3.6/site-packages/pexpect/spawnbase.py", line 344, in expect
    timeout, searchwindowsize, async_)
  File "/path/anaconda3/lib/python3.6/site-packages/pexpect/spawnbase.py", line 372, in expect_list
    return exp.expect_loop(timeout)
  File "/path/anaconda3/lib/python3.6/site-packages/pexpect/expect.py", line 179, in expect_loop
    return self.eof(e)
  File "/path/anaconda3/lib/python3.6/site-packages/pexpect/expect.py", line 122, in eof
    raise exc
pexpect.exceptions.EOF: End Of File (EOF).
<pexpect.popen_spawn.PopenSpawn object at 0x7f5a4e4416a0>
searcher: searcher_re:
    0: re.compile(b'emulator: INFO: boot completed')

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "test.py", line 11, in <module>
    env = android_env.load(
  File "/path/android_env/android_env/loader.py", line 73, in load
    coordinator = coordinator_lib.Coordinator(simulator, task_manager)
  File "/path/android_env/android_env/components/coordinator.py", line 88, in __init__
    self._restart_simulator()
  File "/path/android_env/android_env/components/coordinator.py", line 181, in _restart_simulator
    self._simulator.launch()
  File "/path/android_env/android_env/components/base_simulator.py", line 135, in launch
    self._launch_impl()
  File "/path/android_env/android_env/components/emulator_simulator.py", line 114, in _launch_impl
    self.restart()
  File "/path/android_env/android_env/components/base_simulator.py", line 144, in restart
    self._restart_impl()
  File "/path/android_env/android_env/components/emulator_simulator.py", line 102, in _restart_impl
    self._launcher.restart()
  File "/path/android_env/android_env/components/emulator_launcher.py", line 151, in restart
    self.launch()
  File "/path/android_env/android_env/components/emulator_launcher.py", line 143, in launch
    raise errors.SimulatorCrashError('The emulator has crashed: %r' % e)
android_env.components.errors.SimulatorCrashError: The emulator has crashed: EOF("End Of File (EOF).\n<pexpect.popen_spawn.PopenSpawn object at 0x7f5a4e4416a0>\nsearcher: searcher_re:\n    0: re.compile(b'emulator: INFO: boot completed')",)

I'm not sure why this happens, any idea? Thanks!

I've modified some code to make it can run on the windows.

android_env_for_windows
Enjoy it.

.textproto log_regexp syntax

Hello. Thanks for your work.

I want to clarify the usage of log_regexp in classic_2048.textproto file.

score: "^[Ss]core: ([-+]?[0-9]*\\.?[0-9]*)$"

However, in the console I see logs in a format: score: 4.0, which gives no match (but a string of score 4.\0 matches with regexp). Why do we need \\ in this regexp? Am I misunderstanding something?

I want to parse logs from my own apk.

Crash on "run_acme_agent.py"

Hello,

I did install all dependencies for acme and i am running it on python3.8 (linux) using the pong textproto and pong apk

python3 run_acme_agent.py --avd_name='Pixel_XL_API_30' --android_avd_home='/home/username/.android/avd' --android_sdk_root=/home/username/Android/Sdk/ --emulator_path=/home/username/Android/Sdk/emulator/emulator --adb_path=/home/username/Android/Sdk/platform-tools/adb --num_episodes=100 --task_path=/home/username/Dev/android_env/examples/pong_default.textproto

I am getting the following error

I0622 02:01:27.057457 139624755865408 environment.py:37] Action spec: {'action_type': DiscreteArray(shape=(), dtype=int32, name=action_type, minimum=0, maximum=2, num_values=3), 'touch_position': BoundedArray(shape=(2,), dtype=dtype('float32'), name='touch_position', minimum=[0. 0.], maximum=[1. 1.])} I0622 02:01:27.058849 139624755865408 environment.py:38] Observation spec: {'pixels': Array(shape=(2560, 1440, 3), dtype=dtype('uint8'), name='pixels'), 'timedelta': Array(shape=(), dtype=dtype('int64'), name='timedelta'), 'orientation': Array(shape=(4,), dtype=dtype('uint8'), name='orientation')} I0622 02:01:27.059457 139624755865408 environment.py:39] Task extras spec: {'ball': Array(shape=(4,), dtype=dtype('int32'), name='ball'), 'computer': Array(shape=(4,), dtype=dtype('int32'), name='computer'), 'human': Array(shape=(4,), dtype=dtype('int32'), name='human'), 'collision': Array(shape=(1,), dtype=dtype('int32'), name='collision'), 'state': Array(shape=(1,), dtype=dtype('int32'), name='state')} Traceback (most recent call last): File "run_acme_agent.py", line 88, in <module> app.run(main) File "/home/username/.local/lib/python3.8/site-packages/absl/app.py", line 312, in run _run_main(main, args) File "/home/username/.local/lib/python3.8/site-packages/absl/app.py", line 258, in _run_main sys.exit(main(argv)) File "run_acme_agent.py", line 73, in main num_actions=env_spec.actions.num_values), AttributeError: 'dict' object has no attribute 'num_values' I0622 02:01:27.711886 139624755865408 environment.py:127] Cleaning up AndroidEnv...

Thank you for any insights

Unable to run random agent

Hi I'm trying to run the random agent with one of the existing test protos. But my execution just stays at the line "creating grpc channel" and doesn't move on from there.
OS: Ubuntu 22.04
Python- python 3.9.16

Guide to deploy it on Android devices

Hi ,

I want to use it on real devices not on emulator.
Can you help me guide how to do it?

Regards,
Abhijeet

Migrate example Acme agents from TF to JAX

We currently use TF agents in our Acme example. JAX has received more attention lately so should probably encourage researchers to use that instead.

Error connecting to simulator: "Missing the \'authorization\' header with security credentials"

Hi I am trying to create a simple environment with:

from android_env import loader

env = loader.load(
    avd_name='Pixel_3a_API_34_extension_level_7_arm64-v8a',
    # avd_name='my_avd',
    android_avd_home='/Users/anujm/.android/avd',
    android_sdk_root='/Users/anujm/Library/Android/sdk',
    emulator_path='/Users/anujm/Library/Android/sdk/emulator/emulator',
    adb_path='/Users/anujm/Library/Android/sdk/platform-tools/adb',
    task_path='/Users/anujm/code/android_env/tasks/mdp/mdp_0000.textproto',
)

but I run into "Missing the 'authorization' header with security credentials"

ERROR:absl:RpcError caught. Reconnecting to emulator...
Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/android_env/components/simulators/emulator/emulator_simulator.py", line 87, in wrapper
    return func(*args, **kwargs)  # pytype: disable=missing-parameter  # always-use-return-annotations
           ^^^^^^^^^^^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/android_env/components/simulators/emulator/emulator_simulator.py", line 373, in _confirm_booted
    emu_status = self._emulator_stub.getStatus(empty_pb2.Empty())
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/grpc/_channel.py", line 1176, in __call__
    return _end_unary_response_blocking(state, call, False, None)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/grpc/_channel.py", line 1005, in _end_unary_response_blocking
    raise _InactiveRpcError(state)  # pytype: disable=not-instantiable
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
grpc._channel._InactiveRpcError: <_InactiveRpcError of RPC that terminated with:
	status = StatusCode.UNAUTHENTICATED
	details = "Missing the 'authorization' header with security credentials."
	debug_error_string = "UNKNOWN:Error received from peer  {created_time:"2024-03-27T18:30:09.412245-07:00", grpc_status:16, grpc_message:"Missing the \'authorization\' header with security credentials."}"
>

Below is the caller stack trace:

{
	"name": "SimulatorError",
	"message": "Exception caught in simulator. Please see the simulator logs above for more details.",
	"stack": "---------------------------------------------------------------------------
_InactiveRpcError                         Traceback (most recent call last)
File /Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/android_env/components/simulators/emulator/emulator_simulator.py:87, in _reconnect_on_grpc_error.<locals>.wrapper(*args, **kwargs)
     86 try:
---> 87   return func(*args, **kwargs)  # pytype: disable=missing-parameter  # always-use-return-annotations
     88 except grpc.RpcError:

File /Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/android_env/components/simulators/emulator/emulator_simulator.py:373, in EmulatorSimulator._confirm_booted(self, startup_wait_time_sec)
    372 while time.time() < deadline:
--> 373   emu_status = self._emulator_stub.getStatus(empty_pb2.Empty())
    374   logging.info('Waiting for emulator (%r) to start... (%rms)',
    375                self.adb_device_name(), emu_status.uptime)

File /Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/grpc/_channel.py:1176, in _UnaryUnaryMultiCallable.__call__(self, request, timeout, metadata, credentials, wait_for_ready, compression)
   1170 (
   1171     state,
   1172     call,
   1173 ) = self._blocking(
   1174     request, timeout, metadata, credentials, wait_for_ready, compression
   1175 )
-> 1176 return _end_unary_response_blocking(state, call, False, None)

File /Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/grpc/_channel.py:1005, in _end_unary_response_blocking(state, call, with_call, deadline)
   1004 else:
-> 1005     raise _InactiveRpcError(state)

_InactiveRpcError: <_InactiveRpcError of RPC that terminated with:
\tstatus = StatusCode.UNAUTHENTICATED
\tdetails = \"Missing the 'authorization' header with security credentials.\"
\tdebug_error_string = \"UNKNOWN:Error received from peer  {created_time:\"2024-03-27T18:30:09.412245-07:00\", grpc_status:16, grpc_message:\"Missing the \\'authorization\\' header with security credentials.\"}\"
>

During handling of the above exception, another exception occurred:

_InactiveRpcError                         Traceback (most recent call last)
File /Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/android_env/components/simulators/base_simulator.py:32, in _print_logs_on_exception.<locals>.wrapper(*args, **kwargs)
     31 try:
---> 32   return func(*args, **kwargs)
     33 except Exception as error:
     34   # Calls self.get_logs since self is the first arg.

File /Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/android_env/components/simulators/base_simulator.py:83, in BaseSimulator.launch(self)
     82 self._num_launch_attempts += 1
---> 83 self._launch_impl()

File /Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/android_env/components/simulators/emulator/emulator_simulator.py:250, in EmulatorSimulator._launch_impl(self)
    249 try:
--> 250   self._confirm_booted()
    251 except EmulatorCrashError:

File /Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/android_env/components/simulators/emulator/emulator_simulator.py:94, in _reconnect_on_grpc_error.<locals>.wrapper(*args, **kwargs)
     91 emu._emulator_stub, emu._snapshot_stub = emu._connect_to_emulator(  # pylint: disable=protected-access
     92     emu._grpc_port  # pylint: disable=protected-access
     93 )
---> 94 return func(*args, **kwargs)

File /Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/android_env/components/simulators/emulator/emulator_simulator.py:373, in EmulatorSimulator._confirm_booted(self, startup_wait_time_sec)
    372 while time.time() < deadline:
--> 373   emu_status = self._emulator_stub.getStatus(empty_pb2.Empty())
    374   logging.info('Waiting for emulator (%r) to start... (%rms)',
    375                self.adb_device_name(), emu_status.uptime)

File /Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/grpc/_channel.py:1176, in _UnaryUnaryMultiCallable.__call__(self, request, timeout, metadata, credentials, wait_for_ready, compression)
   1170 (
   1171     state,
   1172     call,
   1173 ) = self._blocking(
   1174     request, timeout, metadata, credentials, wait_for_ready, compression
   1175 )
-> 1176 return _end_unary_response_blocking(state, call, False, None)

File /Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/grpc/_channel.py:1005, in _end_unary_response_blocking(state, call, with_call, deadline)
   1004 else:
-> 1005     raise _InactiveRpcError(state)

_InactiveRpcError: <_InactiveRpcError of RPC that terminated with:
\tstatus = StatusCode.UNAUTHENTICATED
\tdetails = \"Missing the 'authorization' header with security credentials.\"
\tdebug_error_string = \"UNKNOWN:Error received from peer  {created_time:\"2024-03-27T18:30:09.418329-07:00\", grpc_status:16, grpc_message:\"Missing the \\'authorization\\' header with security credentials.\"}\"
>

The above exception was the direct cause of the following exception:

SimulatorError                            Traceback (most recent call last)
Cell In[1], line 3
      1 from android_env import loader
----> 3 env = loader.load(
      4     avd_name='Pixel_3a_API_34_extension_level_7_arm64-v8a',
      5     # avd_name='my_avd',
      6     android_avd_home='/Users/anujm/.android/avd',
      7     android_sdk_root='/Users/anujm/Library/Android/sdk',
      8     emulator_path='/Users/anujm/Library/Android/sdk/emulator/emulator',
      9     adb_path='/Users/anujm/Library/Android/sdk/platform-tools/adb',
     10     task_path='/Users/anujm/code/android_env/tasks/mdp/mdp_0000.textproto',
     11 )

File /Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/android_env/loader.py:72, in load(task_path, avd_name, android_avd_home, android_sdk_root, emulator_path, adb_path, run_headless)
     69   text_format.Parse(proto_file.read(), task)
     71 task_manager = task_manager_lib.TaskManager(task)
---> 72 coordinator = coordinator_lib.Coordinator(simulator, task_manager)
     74 # Load environment.
     75 return environment.AndroidEnv(coordinator=coordinator)

File /Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/android_env/components/coordinator.py:120, in Coordinator.__init__(self, simulator, task_manager, num_fingers, interaction_rate_sec, enable_key_events, show_touches, show_pointer_location, show_status_bar, show_navigation_bar, periodic_restart_time_min, tmp_dir)
    117 self._simulator_start_time = None
    119 logging.info('Starting the simulator...')
--> 120 self._launch_simulator()

File /Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/android_env/components/coordinator.py:223, in Coordinator._launch_simulator(self, max_retries)
    220 self._task_manager.stop()
    222 # Launch the simulator.
--> 223 self._simulator.launch()
    224 self._simulator_start_time = time.time()
    226 # From here on, the simulator is assumed to be up and running.

File /Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/android_env/components/simulators/base_simulator.py:37, in _print_logs_on_exception.<locals>.wrapper(*args, **kwargs)
     35 for line in args[0].get_logs().splitlines():
     36   logging.error(line)
---> 37 raise errors.SimulatorError(
     38     'Exception caught in simulator. Please see the simulator logs '
     39     'above for more details.') from error

SimulatorError: Exception caught in simulator. Please see the simulator logs above for more details."
}

Any help would be great,
Thanks

Cannot run "run_random_agent.py"

Hi,

Recently I tried to set up the android_env on my ubuntu22.04, however, I cannot run the environment. Currently, I am on your latest version (commit 3ebb08d), here is my command:

python run_random_agent.py \
--avd_name='my_avd_keyboard_3' \
--android_avd_home=/home/quintus/.android/avd \
--android_sdk_root=/home/quintus/Android/Sdk \
--emulator_path=/home/quintus/Android/Sdk/emulator/emulator \
--adb_path=/home/quintus/Android/Sdk/platform-tools/adb \
--num_steps=1000 \
--task_path=/home/quintus/Aalto/android_env/tasks/apple_flinger/apple_flinger_M_1_1.textproto

I get this error:

Traceback (most recent call last):
  File "run_random_agent.py", line 23, in <module>
    from android_env import loader
  File "<frozen importlib._bootstrap>", line 991, in _find_and_load
  File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 655, in _load_unlocked
  File "<frozen importlib._bootstrap>", line 618, in _load_backward_compatible
  File "<frozen zipimport>", line 259, in load_module
  File "/home/quintus/anaconda3/envs/android_py/lib/python3.8/site-packages/android_env-1.1.0-py3.8.egg/android_env/loader.py", line 20, in <module>
  File "<frozen importlib._bootstrap>", line 991, in _find_and_load
  File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 655, in _load_unlocked
  File "<frozen importlib._bootstrap>", line 618, in _load_backward_compatible
  File "<frozen zipimport>", line 259, in load_module
  File "/home/quintus/anaconda3/envs/android_py/lib/python3.8/site-packages/android_env-1.1.0-py3.8.egg/android_env/environment.py", line 21, in <module>
  File "<frozen importlib._bootstrap>", line 991, in _find_and_load
  File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 655, in _load_unlocked
  File "<frozen importlib._bootstrap>", line 618, in _load_backward_compatible
  File "<frozen zipimport>", line 259, in load_module
  File "/home/quintus/anaconda3/envs/android_py/lib/python3.8/site-packages/android_env-1.1.0-py3.8.egg/android_env/env_interface.py", line 32, in <module>
  File "/home/quintus/anaconda3/envs/android_py/lib/python3.8/site-packages/android_env-1.1.0-py3.8.egg/android_env/env_interface.py", line 38, in AndroidEnvInterface
TypeError: 'type' object is not subscriptable

I also tried your previous version (commit d9e0525 on 4 Apr), I ran the same command, and it was blocked when connecting to the emulator:

...
I0418 11:05:11.895140 140320000093056 coordinator.py:119] Starting the simulator...
I0418 11:05:11.895172 140320000093056 coordinator.py:218] Simulator launch attempt 1 of 3
I0418 11:05:11.895202 140320000093056 emulator_simulator.py:226] Attempt 1 at launching the Android Emulator ('emulator-5554')
I0418 11:05:11.895221 140320000093056 emulator_launcher.py:104] Booting new emulator [/home/quintus/Android/Sdk/emulator/emulator]
I0418 11:05:11.895246 140320000093056 emulator_launcher.py:122] extra_env_vars: ANDROID_HOME= ANDROID_SDK_ROOT=/home/quintus/Android/Sdk ANDROID_AVD_HOME=/home/quintus/.android/avd ANDROID_EMULATOR_KVM_DEVICE=/dev/kvm ANDROID_ADB_SERVER_PORT=5037 LD_LIBRARY_PATH=/home/quintus/Android/Sdk/emulator/lib64/x11/:/home/quintus/Android/Sdk/emulator/lib64/qt/lib/:/home/quintus/Android/Sdk/emulator/lib64/gles_swiftshader/:/home/quintus/Android/Sdk/emulator/lib64/ QT_XKB_CONFIG_ROOT=/home/quintus/Android/Sdk/emulator/qt_config/ ANDROID_EMU_ENABLE_CRASH_REPORTING=1
I0418 11:05:11.895323 140320000093056 emulator_launcher.py:153] Emulator launch command: /home/quintus/Android/Sdk/emulator/emulator -adb-path /home/quintus/Android/Sdk/platform-tools/adb -gpu swiftshader_indirect -no-audio -show-kernel -verbose -avd my_avd_keyboard_3 -grpc 8554 -ports 37575,5555 -no-snapshot
I0418 11:05:11.896853 140320000093056 emulator_simulator.py:342] Creating gRPC channel to the emulator on port 8554

It seems that the problem is related to "-adb-path", When I run the emulator launch command without -adb-path or run the run_random_agent.py without the adb_path in emulator_launcher.py (https://github.com/deepmind/android_env/blob/main/android_env/components/simulators/emulator/emulator_launcher.py#L143), it can successfully connect the emulator.

Could you please tell me how to solve this problem? Thanks!

Issue with running the enviroment on new ubuntu devie

There seems to be an issue with the protobuf package when trying to run the enviroment under ubuntu:

google.protobuf.text_format.ParseError: 24:5 : Message type "android_env.SetupStep" has no field named "adb_request".

steps to repproduce:

clean install android_env in conda env
-download a task (i used droidfish packages)
run from the forlder where .apk and .textproto file are located

please help

Crash on "run_random_agent.py"

Hi 👋

I tired running run_random_agent.py using the appleflinger textproto and apk, and got this error before the virtual device had booted. I tried running the following using Python 3.7 and 3.8 (Windows) but faced the same issue in both -

python run_random_agent.py \
 --avd_name='Pixel 5 API 33 - RL AVD' \
 --android_avd_home='C:/Users/abhin/.android/avd' \
 --android_sdk_root=C:/Users/abhin/AppData/Local/Android/Sdk \
 --emulator_path=C:/Users/abhin/AppData/Local/Android/Sdk/emulator/emulator \
 --adb_path=C:/Users/abhin/AppData/Local/Android/Sdk/platform-tools/adb \
 --num_episodes=1000 \
 --task_path=C:/Users/abhin/Android RL/appleflinger/apple_flinger_M_1_1.textproto

I am getting the following error -

I0308 05:20:21.676290  6432 emulator_simulator.py:74] Simulator local_tmp_dir: C:\Users\abhin\AppData\Local\Temp\tmpfsfy3tvr
I0308 05:20:21.677109  6432 adb_controller.py:144] Executing ADB command: ['C:/Users/abhin/AppData/Local/Android/Sdk/platform-tools/adb', '-P', '5037', 'devices']
I0308 05:20:21.858727  6432 adb_controller.py:149] Done executing ADB command: ['C:/Users/abhin/AppData/Local/Android/Sdk/platform-tools/adb', '-P', '5037', 'devices']
I0308 05:20:22.093404  6432 emulator_simulator.py:81] Initialized simulator with ADB server port 5037.
I0308 05:20:22.097983  6432 emulator_simulator.py:91] emulator_launcher_args: {'avd_name': 'Pixel 5 API 33 - RL AVD', 'android_avd_home': 'C:/Users/abhin/.android/avd', 'android_sdk_root': 'C:/Users/abhin/AppData/Local/Android/Sdk', 'emulator_path': 'C:/Users/abhin/AppData/Local/Android/Sdk/emulator/emulator', 'run_headless': False, 'gpu_mode': 'swiftshader_indirect', 'adb_port': 52187, 'adb_server_port': 5037, 'emulator_console_port': 52188, 'local_tmp_dir': 'C:\\Users\\abhin\\AppData\\Local\\Temp\\tmpfsfy3tvr', 'grpc_port': 52189}
Traceback (most recent call last):
  File ".\run_random_agent.py", line 95, in <module>
    app.run(main)
  File "C:\Users\abhin\OneDrive\Desktop\Android RL\env\lib\site-packages\absl\app.py", line 308, in run
    _run_main(main, args)
  File "C:\Users\abhin\OneDrive\Desktop\Android RL\env\lib\site-packages\absl\app.py", line 254, in _run_main
    sys.exit(main(argv))
  File ".\run_random_agent.py", line 66, in main
    run_headless=FLAGS.run_headless,
  File "C:\Users\abhin\OneDrive\Desktop\Android RL\env\lib\site-packages\android_env\loader.py", line 70, in load
    text_format.Parse(proto_file.read(), task)
  File "C:\Users\abhin\OneDrive\Desktop\Android RL\env\lib\site-packages\google\protobuf\text_format.py", line 687, in Parse
    allow_unknown_field=allow_unknown_field)
  File "C:\Users\abhin\OneDrive\Desktop\Android RL\env\lib\site-packages\google\protobuf\text_format.py", line 759, in ParseLines
    return parser.ParseLines(lines, message)
  File "C:\Users\abhin\OneDrive\Desktop\Android RL\env\lib\site-packages\google\protobuf\text_format.py", line 812, in ParseLines
    self._ParseOrMerge(lines, message)
  File "C:\Users\abhin\OneDrive\Desktop\Android RL\env\lib\site-packages\google\protobuf\text_format.py", line 837, in _ParseOrMerge
    self._MergeField(tokenizer, message)
  File "C:\Users\abhin\OneDrive\Desktop\Android RL\env\lib\site-packages\google\protobuf\text_format.py", line 961, in _MergeField
    merger(tokenizer, message, field)
  File "C:\Users\abhin\OneDrive\Desktop\Android RL\env\lib\site-packages\google\protobuf\text_format.py", line 1042, in _MergeMessageField
    self._MergeField(tokenizer, sub_message)
  File "C:\Users\abhin\OneDrive\Desktop\Android RL\env\lib\site-packages\google\protobuf\text_format.py", line 934, in _MergeField
    (message_descriptor.full_name, name))
google.protobuf.text_format.ParseError: 24:5 : Message type "android_env.SetupStep" has no field named "adb_request".

I haven't modified the applefliger textproto file and am able to see adb_request in the setup_steps. I looked at the other issues and resolutions in the repo but didn't find the same issue. Would you know what might be the problem?

About launching android env with home screen

Dear colleagues:

I'm now trying to use android_env for one research topic. Instead of monitoring one specific application, I need to check how users operate on the emulator (esp. click, navigation, input) and leverage adb logging or interceptor to understand what happened behind for API calling.

Is it possible to do this via textproto setting? launching to the home and accessing multiple apps to simulate a more flexible user scenario?

I saw @kenjitoyama mentioned in #19, you're mainly relying on accessibility events to access the third party apps or default apps lacking source code.

"We haven't trained explicitly for YouTube, it was just an example. However, we did train agents on apps for which we don't have the source code such as the Clock or System Settings by reusing accessibility events. Please see https://github.com/deepmind/android_env/blob/main/docs/example_tasks.md#accessibility-forwarder. "

I don't know whether there are more possibilities to do android apps monitoring, beside accessibility events?

launching multiple ends

Hi,
I'm trying to use multiple ends to collect more datas. For this, I declared multiple envs but failed with error.
How can I solve this problem?
Thank you

I0313 01:13:47.496413 139673499981632 emulator_simulator.py:183] Initialized simulator with ADB server port 5037.
I0313 01:13:47.497501 139673499981632 emulator_launcher.py:99] Simulator local_tmp_dir: /tmp/android_env/simulator/simulator_instance_4w6cz6cc
I0313 01:13:47.499626 139673499981632 coordinator.py:89] simulator launching...
I0313 01:13:47.499822 139673499981632 coordinator.py:113] Simulator launch attempt 1 of 3
I0313 01:13:47.499963 139673499981632 emulator_simulator.py:239] Attempt 1 at launching the Android Emulator ('emulator-5554')
['/home/dongyoon/.local/share/android/sdk/emulator/emulator', '-adb-path', '/home/dongyoon/.local/share/android/sdk/platform-tools/adb', '-gpu', 'swiftshader_indirect', '-no-audio', '-show-kernel', '-verbose', '-avd', 'pixel_3_train_00', '-grpc', '8554', '-no-skin', '-no-window', '-ports', '35597,5555', '-no-snapshot']
I0313 01:13:47.500376 139673499981632 emulator_launcher.py:159] Emulator launch command: /home/dongyoon/.local/share/android/sdk/emulator/emulator -adb-path /home/dongyoon/.local/share/android/sdk/platform-tools/adb -gpu swiftshader_indirect -no-audio -show-kernel -verbose -avd pixel_3_train_00 -grpc 8554 -no-skin -no-window -ports 35597,5555 -no-snapshot
I0313 01:14:13.565765 139673499981632 emulator_simulator.py:405] See the full emulator logs at '/tmp/android_env/simulator/simulator_instance_4w6cz6cc/emulator_output'
I0313 01:14:14.148301 139673499981632 coordinator.py:92] simulator launched!!!
I0313 01:14:14.165779 139673499981632 environment.py:125] Resetting AndroidEnv...
E0313 01:14:14.166253 139673499981632 adb_log_stream.py:53] stop_stream() called before get_stream_output(). This violates the LogStream API.
I0313 01:14:28.249709 139673499981632 emulator_simulator.py:183] Initialized simulator with ADB server port 5039.
I0313 01:14:28.250501 139673499981632 emulator_launcher.py:99] Simulator local_tmp_dir: /tmp/android_env/simulator/simulator_instance_8qt_m8tt
I0313 01:14:28.251915 139673499981632 coordinator.py:89] simulator launching...
I0313 01:14:28.252094 139673499981632 coordinator.py:113] Simulator launch attempt 1 of 3
I0313 01:14:28.252236 139673499981632 emulator_simulator.py:239] Attempt 1 at launching the Android Emulator ('emulator-5556')
['/home/dongyoon/.local/share/android/sdk/emulator/emulator', '-adb-path', '/home/dongyoon/.local/share/android/sdk/platform-tools/adb', '-gpu', 'swiftshader_indirect', '-no-audio', '-show-kernel', '-verbose', '-avd', 'pixel_3_train_00', '-grpc', '34569', '-no-skin', '-no-window', '-ports', '37961,5557', '-no-snapshot']
I0313 01:14:28.252711 139673499981632 emulator_launcher.py:159] Emulator launch command: /home/dongyoon/.local/share/android/sdk/emulator/emulator -adb-path /home/dongyoon/.local/share/android/sdk/platform-tools/adb -gpu swiftshader_indirect -no-audio -show-kernel -verbose -avd pixel_3_train_00 -grpc 34569 -no-skin -no-window -ports 37961,5557 -no-snapshot
E0313 01:15:28.255342 139673499981632 emulator_simulator.py:368] Failed to connect to the emulator.
Traceback (most recent call last):
File "/home/dongyoon/android_env/android_env/components/simulators/emulator/emulator_simulator.py", line 366, in _connect_to_emulator
grpc.channel_ready_future(self._channel).result(timeout=60)
File "/home/dongyoon/android_env/.eggs/grpcio-1.62.1-py3.10-linux-x86_64.egg/grpc/_utilities.py", line 162, in result
self._block(timeout)
File "/home/dongyoon/android_env/.eggs/grpcio-1.62.1-py3.10-linux-x86_64.egg/grpc/_utilities.py", line 106, in _block
raise grpc.FutureTimeoutError()
grpc.FutureTimeoutError
E0313 01:15:28.256708 139673499981632 base_simulator.py:71] INFO | Using crash reporting configuration from environment variable: ANDROID_EMU_ENABLE_CRASH_REPORTING=1
E0313 01:15:28.256886 139673499981632 base_simulator.py:71] INFO | Storing crashdata in: /tmp/android-dongyoon/emu-crash-34.1.19.db, detection is enabled for process: 869711
E0313 01:15:28.257060 139673499981632 base_simulator.py:71] INFO | Android emulator version 34.1.19.0 (build_id 11525734) (CL:N/A)
E0313 01:15:28.257220 139673499981632 base_simulator.py:71] DEBUG | Current emulator version 34.1.19 is the same as the required version 34.1.19.
E0313 01:15:28.257323 139673499981632 base_simulator.py:71] INFO | Deleting /home/dongyoon/.android/avd/../avd/pixel_3_train_00.avd/bootcompleted.ini done
E0313 01:15:28.257415 139673499981632 base_simulator.py:71] INFO | Found AVD name 'pixel_3_train_00'
E0313 01:15:28.257505 139673499981632 base_simulator.py:71] INFO | Found AVD target architecture: x86_64

Compatibility Error when using rich logging handler

Hi,
I found that if I use rich logging handler and starting the andriod env. An error will occur at this code line:
https://github.com/google-deepmind/android_env/blob/main/android_env/components/simulators/emulator/emulator_launcher.py#L60

I think the rendering rule of rich logging handler is not compatibility with strings such as "[/home/Android/Sdk/emulator/emulator]".

I sugguest changing the logging info to be "'/home/Android/Sdk/emulator/emulator'".

1V1 environmnet

Hi ,

Thank you very much for giving such a nice tool. I just have one problem, will you offer some compatible 1v1 environments in the future? For example, in current Pong and droidfish chess, we can train an rl-agent against built-in bots. But you know, we need the help of selfplay to get a stronger rl-agent.
Besides, will you also consider environments that need two ro more players to coorparate?
Thanks.

Regards,
Shenghan

Extras not always returned . ( in case of floodit_basic).

I tried to print The extras using logging.info('**Extras: %r ', env.task_extras()) in thr run_human_agent.py and found out that sometimes , it does print the corrent extras whereas quite often it just prints an empty dictionary!.
Could you please point out any possible reasons for it.
Any help would be greatly appreciated.
Regards

Java Bindings

Can this library please exposed using Java APIs so using within Android context would be seamless?

AndroidEnv quits because of an unexpected keyword argument 'use_poll'

I tried running run_random_agent.py and got a crash right after the virtual device booted. What might be the problem? Does AndroidEnv need a newer version of Python than 3.6?

I0908 16:13:04.342051 140434626111232 coordinator.py:175] Simulator launch attempt 1 of 3
I0908 16:13:04.342184 140434626111232 emulator_launcher.py:87] Booting the emulator [/home/sagar/Android/Sdk/emulator/emulator]
I0908 16:13:04.342311 140434626111232 emulator_launcher.py:107] extra_env_vars: {'ANDROID_HOME': '', 'ANDROID_SDK_ROOT': '/home/sagar/Android/Sdk', 'ANDROID_AVD_HOME': '/home/sagar/.android/avd', 'ANDROID_EMULATOR_KVM_DEVICE': '/dev/kvm', 'ANDROID_ADB_SERVER_PORT': '5037', 'LD_LIBRARY_PATH': '/home/sagar/Android/Sdk/emulator/lib64/x11/:/home/sagar/Android/Sdk/emulator/lib64/qt/lib/:/home/sagar/Android/Sdk/emulator/lib64/gles_swiftshader/:/home/sagar/Android/Sdk/emulator/lib64/', 'QT_DEBUG_PLUGINS': '1', 'QT_XKB_CONFIG_ROOT': '/home/sagar/Android/Sdk/emulator/qt_config/'}
I0908 16:13:04.342589 140434626111232 emulator_launcher.py:123] Emulator launch command: /home/sagar/Android/Sdk/emulator/emulator -no-snapshot -gpu swiftshader_indirect -no-audio -verbose -avd pixel4 -ports 19811,15342
I0908 16:13:04.345738 140434626111232 emulator_launcher.py:136] Waiting for boot for 300.0 seconds...
I0908 16:13:33.728062 140434626111232 emulator_launcher.py:138] Emulator log matched: b'emulator: INFO: boot completed'
I0908 16:13:33.728239 140434626111232 emulator_launcher.py:146] Done booting the emulator (in 29.385461 seconds).
I0908 16:13:33.728819 140434626111232 emulator_console.py:264] Connecting to Emulator console on port 19811...
I0908 16:13:33.730707 140434626111232 emulator_console.py:279] Done connecting to Emulator console on port 19811.
I0908 16:13:33.730814 140434626111232 emulator_console.py:282] Authenticating to console.
I0908 16:13:33.784988 140434626111232 adb_controller.py:458] Fetching screen dimensions...
I0908 16:13:33.785164 140434626111232 adb_controller.py:184] Initialising ADB shell with command: /home/sagar/Android/Sdk/platform-tools/adb -P 5037 -s emulator-15341 shell
I0908 16:13:33.785252 140434626111232 adb_controller.py:190] Spawning ADB shell...
Traceback (most recent call last):
  File "run_random_agent.py", line 82, in <module>
    app.run(main)
  File "/home/sagar/.local/lib/python3.6/site-packages/absl/app.py", line 303, in run
    _run_main(main, args)
  File "/home/sagar/.local/lib/python3.6/site-packages/absl/app.py", line 251, in _run_main
    sys.exit(main(argv))
  File "run_random_agent.py", line 54, in main
    run_headless=False) as env:
  File "/home/sagar/.local/lib/python3.6/site-packages/android_env/loader.py", line 73, in load
    coordinator = coordinator_lib.Coordinator(simulator, task_manager)
  File "/home/sagar/.local/lib/python3.6/site-packages/android_env/components/coordinator.py", line 88, in __init__
    self._restart_simulator()
  File "/home/sagar/.local/lib/python3.6/site-packages/android_env/components/coordinator.py", line 181, in _restart_simulator
    self._simulator.launch()
  File "/home/sagar/.local/lib/python3.6/site-packages/android_env/components/base_simulator.py", line 136, in launch
    self._post_launch_setup()
  File "/home/sagar/.local/lib/python3.6/site-packages/android_env/components/emulator_simulator.py", line 117, in _post_launch_setup
    super()._post_launch_setup()
  File "/home/sagar/.local/lib/python3.6/site-packages/android_env/components/base_simulator.py", line 151, in _post_launch_setup
    self._adb_controller.get_screen_dimensions())
  File "/home/sagar/.local/lib/python3.6/site-packages/android_env/components/adb_controller.py", line 459, in get_screen_dimensions
    self._wait_for_device(timeout=timeout)
  File "/home/sagar/.local/lib/python3.6/site-packages/android_env/components/adb_controller.py", line 234, in _wait_for_device
    ready = self._check_device_is_ready(timeout=timeout)
  File "/home/sagar/.local/lib/python3.6/site-packages/android_env/components/adb_controller.py", line 247, in _check_device_is_ready
    ['shell', 'service', 'check', service], timeout=timeout)
  File "/home/sagar/.local/lib/python3.6/site-packages/android_env/components/adb_controller.py", line 116, in _execute_command
    adb_output = self._execute_shell_command(args[1:], timeout=timeout)
  File "/home/sagar/.local/lib/python3.6/site-packages/android_env/components/adb_controller.py", line 149, in _execute_shell_command
    self._init_shell(timeout=timeout)
  File "/home/sagar/.local/lib/python3.6/site-packages/android_env/components/adb_controller.py", line 191, in _init_shell
    self._adb_shell = pexpect.spawn(command, use_poll=True, timeout=timeout)
TypeError: __init__() got an unexpected keyword argument 'use_poll'
I0908 16:13:33.866050 140434626111232 emulator_launcher.py:156] Killing the emulator process...
I0908 16:13:33.972082 140434626111232 emulator_launcher.py:161] Done killing the emulator process.

Acme example for AndroidEnv requires `reverb` (Linux only)

The Acme run_acme_agent.py example appears to require /deepmind/reverb but that package - can only run on Linux systems at the moment:

⚠️ Reverb currently only supports Linux based OSes.

(from https://github.com/deepmind/reverb)

Core Acme installation:

pip install dm-acme
pip install dm-acme[reverb]

(https://github.com/deepmind/acme)

Incompatible with Python3.10

I tried to install android_env both from pip and source code but I get an error on install. Digging a little more I found out that the installed version of numpy (which is 1.19.5) is incompatible with Python3.10.

For now I removed the version requirement from setup.py and the installation completed successfully.

How many hours did you take to train the baselines?

Hi, in the paper, https://arxiv.org/pdf/2105.13231.pdf,

In Figure 6, how many hours did you take to train the baselines?

Cannot run "run_random_agent.py"

Hello.

I'm using m1 Mac and Android emulator version 32.1.12.0 and got error
E0305 21:37:52.800976 7953584128 base_simulator.py:71] unknown option: -adb-path
Maybe it's same with issue134 but emulator at https://developer.android.com/studio/emulator_archive was different with mine and couldn't use it. How can I update update emulator and resolve this issue?

Thank you.

Running on Docker

Hi,

Thanks for the great work. I want to ask if it is possible to integrate it with Android docker image like this one. It would allow a much more cleaner setup.

Thanks

Blocked after running loader.load

I've come across an odd issue which wasn't occurring to me earlier. Whenever I run loader.load with proper configs, the emulator will start up. Once the emulator has started up, my entire Python process freezes and everything becomes blocked. The emulator doesn't start up the proper apk, it's just stuck on the home screen. Pressing Ctrl-C in the Python process exits the emulator, but does not exit the Python process. This is preventing me from even running run_random_agent.py I'm on an M2 Mac.

EDIT: I've tried reinstalling the package, reinstalling Android studio, and reinstalling everything in a fresh virtual environment. Nothing seems to work.

Full logs

Old problems in master

Hello,
There was an issue Crash on "run_acme_agent.py" #17, that is currently closed and marked as fixed, but the version cf5f631 which comments are referring to doesn't belong to the project. As far as I understand the problem is due to the force initial commit, which didn't have the oldest fixes. Can you please merge these changes in the master branch?
Thank you in advance.

AndroidEnv does not work in WSL

I've setup android studio on windows and am trying to load the environment from WSL. I have got an unexpected keyword argument 'use_poll' error while trying to load the environment. Any ideas on how I can fix this?

Full error:

>>> import android_env
>>> env = android_env.load(avd_name='Pixel_2_API_30', android_avd_home='/mnt/c/Users/Surya Penmetsa/.android/avd/Pixel_2_API_30.avd', android_sdk_root='/mnt/c/Users/Surya Penmetsa/AppData/Local/Android/Sdk', emulator_path='/mnt/c/Users/Surya Penmetsa/AppData/Local/Android/Sdk/emulator/emulator.exe', adb_path='/mnt/c/Users/Surya Penmetsa/AppData/Local/Android/Sdk/platform-tools/adb.exe', task_path='downloads/classic_2048.textproto')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/surya/anaconda3/lib/python3.6/site-packages/android_env/loader.py", line 73, in load
    coordinator = coordinator_lib.Coordinator(simulator, task_manager)
  File "/home/surya/anaconda3/lib/python3.6/site-packages/android_env/components/coordinator.py", line 88, in __init__
    self._restart_simulator()
  File "/home/surya/anaconda3/lib/python3.6/site-packages/android_env/components/coordinator.py", line 181, in _restart_simulator
    self._simulator.launch()
  File "/home/surya/anaconda3/lib/python3.6/site-packages/android_env/components/base_simulator.py", line 136, in launch
    self._post_launch_setup()
  File "/home/surya/anaconda3/lib/python3.6/site-packages/android_env/components/emulator_simulator.py", line 117, in _post_launch_setup
    super()._post_launch_setup()
  File "/home/surya/anaconda3/lib/python3.6/site-packages/android_env/components/base_simulator.py", line 151, in _post_launch_setup
    self._adb_controller.get_screen_dimensions())
  File "/home/surya/anaconda3/lib/python3.6/site-packages/android_env/components/adb_controller.py", line 459, in get_screen_dimensions
    self._wait_for_device(timeout=timeout)
  File "/home/surya/anaconda3/lib/python3.6/site-packages/android_env/components/adb_controller.py", line 234, in _wait_for_device
    ready = self._check_device_is_ready(timeout=timeout)
  File "/home/surya/anaconda3/lib/python3.6/site-packages/android_env/components/adb_controller.py", line 247, in _check_device_is_ready
    ['shell', 'service', 'check', service], timeout=timeout)
  File "/home/surya/anaconda3/lib/python3.6/site-packages/android_env/components/adb_controller.py", line 116, in _execute_command
    adb_output = self._execute_shell_command(args[1:], timeout=timeout)
  File "/home/surya/anaconda3/lib/python3.6/site-packages/android_env/components/adb_controller.py", line 149, in _execute_shell_command
    self._init_shell(timeout=timeout)
  File "/home/surya/anaconda3/lib/python3.6/site-packages/android_env/components/adb_controller.py", line 191, in _init_shell
    self._adb_shell = pexpect.spawn(command, use_poll=True, timeout=timeout)
TypeError: __init__() got an unexpected keyword argument 'use_poll'