artemkushnerov / az Goto Github PK

Hello Artem,
Sometimes when I'm downloading, I get this connectivity error. How can it be fixed?
The terminal command I used was:
az -n 1100 -d 2014-01-01: -s :3000000 -m PlayDrone -vt 0:0 -md sha256, sha1, md5, dex_date, apk_size, pkg_name, vt_detection, vt_scan_date, dex_size, markets -o /home/lechu/Documents/gw/playdrone/ -sd 15 -t 6

Dependabot couldn't authenticate with https://pypi.python.org/simple/.

You can provide authentication details in your Dependabot dashboard by clicking into the account menu (in the top right) and selecting 'Config variables'.

View the update logs.

Hello,
these two commands were run:

• az -n 10000 -d 2018-06-01: -s :3000000 -vt 3:500000 -t 128
• az -n 10000 -d 2018-06-01: -s :3000000 -vt 3:500000 -t 2
  With ls | wc-l, the first command produced 1138 files, while the second command produced 1136.

No environmental changes were made. Only difference is the amount of threads. Many warnings apk with pkg <pkg_name> already exists were given in both cases.

Is your code thread safe? Can race conditions occur?

It would seem that multiple threads get assigned to the same download entry in the dataset.

Please fix this issue

Hello. Is it possible to download the apk files by saving them with the sha256 instead of with the pkg_name?

Now, if I run the script the output file is: pkg_name.apk; I'd like to have, instead, sha256.apk

Please, can someone help me? Thanks!

@ArtemKushnerov

Environment: Windows 10
Python: 3.7.9
pip install git+https://github.com/ArtemKushnerov/[email protected]#egg=az successfully executed but when i run
'az -n 10 -d 2015-12-11: -s :3000000 -m play.google.com,appchina' command on cmd it shows an error "az is not recognized as an internal or external command".
your help will be highly appreciated.
content in .az File

Error message cmd

Hi,

I have already applied for an valid API key also use the script and lastest csv AndroZoo provided, however, it always returns invalid API key or invalid hashes. Could I know why it happened?

Thank you!

Such as this: https://androzoo.uni.lu/api/download?apikey=${API KEY}&sha256=${00008E0B8F27CFCDFA2A73A45A4450B0D5255F63C8B5AE407AEEF0DEA9842174}

I have download some apk and az save them with ..apk. Next I use the same seed and run the cmd, az will download this file again. If there some convenient way to resume from break point?

Hello,

I'm trying to run the sample from the README file and got this exception:

Traceback (most recent call last):
  File "/home/user/.local/bin/az", line 7, in <module>
    from modules.main import run
  File "/home/user/.local/lib/python2.7/site-packages/modules/main.py", line 50
    number, *criteria_args, metadata, key, input_file = Parser(args).parse()
            ^
SyntaxError: invalid syntax

I'm working on Ubuntu 18.04.

remote: The project you were looking for could not be found.
fatal: repository 'https://gitlab.com/comma-project/adownloader.git/' not found
while running-> pip install -e git+https://gitlab.com/comma-project/[email protected]#egg=az

File "d:\anaconda3\lib\site-packages\azoo-1.3.0-py3.6.e.py", line 16, in init
ValueError: invalid literal for int() with base 10: '2.0'

Though not a big problem, it is desired to be reported. I change the code "int()" in the line 16, 21,23 of apk.py to '' int(float()) ''.

env> windows 11/ conda / python 3.6 / az 1.3.1 /
used > az -n 10 -d 2015-12-11: -s :3000000 -m play.google.com,appchina
& az -n 1 -d 2020-01-01: -s :5000000 -t 6 -m play.google.com,appchina,PlayDrone
with latest.csv got this error >>

Traceback (most recent call last):
File "C:\Users\anaconda3\envs\az1\lib\runpy.py", line 193, in _run_module_as_main
"main", mod_spec)
File "C:\Users\anaconda3\envs\az1\lib\runpy.py", line 85, in run_code
exec(code, run_globals)
File "C:\Users\anaconda3\envs\az1\Scripts\az.exe_main.py", line 7, in
File "C:\Users\anaconda3\envs\az1\lib\site-packages\click\core.py", line 722, in call
return self.main(*args, **kwargs)
File "C:\Users\anaconda3\envs\az1\lib\site-packages\click\core.py", line 697, in main
rv = self.invoke(ctx)
File "C:\Users\anaconda3\envs\az1\lib\site-packages\click\core.py", line 895, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "C:\Users\anaconda3\envs\az1\lib\site-packages\click\core.py", line 535, in invoke
return callback(*args, **kwargs)
File "C:\Users\anaconda3\envs\az1\lib\site-packages\modules\main.py", line 52, in run
az.run(input_file, key, number, criteria, out_dir=out if out else os.getcwd(), metadata=metadata, seed=seed, threads=threads)
File "C:\Users\anaconda3\envs\az1\lib\site-packages\modules\az.py", line 18, in run
random_subset = RandomPicker(seed=seed).get_random_subset(dataset, number)
File "C:\Users\anaconda3\envs\az1\lib\site-packages\modules\services\random_picker.py", line 18, in get_random_subset
for num, apk in enumerate(input_dataset, 1):
File "C:\Users\anaconda3\envs\az1\lib\site-packages\modules\entities\dataset\filtered_dataset.py", line 27, in next
apk = next(self.source)
File "C:\Users\anaconda3\envs\az1\lib\site-packages\modules\entities\source.py", line 20, in next
line = next(self.reader)
File "C:\Users\anaconda3\envs\az1\lib\encodings\cp1252.py", line 23, in decode
return codecs.charmap_decode(input,self.errors,decoding_table)[0]
UnicodeDecodeError: 'charmap' codec can't decode byte 0x9d in position 445: character maps to

used > az -n 10 -d 2015-12-11: -s :3000000 -m play.google.com,appchina
& az -n 1 -d 2020-01-01: -s :5000000 -t 6 -m play.google.com,appchina,PlayDrone
with latest_with-added-date.csv got this error >>
Traceback (most recent call last):
File "C:\Users\anaconda3\envs\az1\lib\runpy.py", line 193, in _run_module_as_main
"main", mod_spec)
File "C:\Users\anaconda3\envs\az1\lib\runpy.py", line 85, in run_code
exec(code, run_globals)
File "C:\Users\anaconda3\envs\az1\Scripts\az.exe_main.py", line 7, in
File "C:\Users\anaconda3\envs\az1\lib\site-packages\click\core.py", line 722, in call
return self.main(*args, **kwargs)
File "C:\Users\anaconda3\envs\az1\lib\site-packages\click\core.py", line 697, in main
rv = self.invoke(ctx)
File "C:\Users\anaconda3\envs\az1\lib\site-packages\click\core.py", line 895, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "C:\Users\anaconda3\envs\az1\lib\site-packages\click\core.py", line 535, in invoke
return callback(*args, **kwargs)
File "C:\Users\anaconda3\envs\az1\lib\site-packages\modules\main.py", line 52, in run
az.run(input_file, key, number, criteria, out_dir=out if out else os.getcwd(), metadata=metadata, seed=seed, threads=threads)
File "C:\Users\anaconda3\envs\az1\lib\site-packages\modules\az.py", line 18, in run
random_subset = RandomPicker(seed=seed).get_random_subset(dataset, number)
File "C:\Users\anaconda3\envs\az1\lib\site-packages\modules\services\random_picker.py", line 18, in get_random_subset
for num, apk in enumerate(input_dataset, 1):
File "C:\Users\anaconda3\envs\az1\lib\site-packages\modules\entities\dataset\filtered_dataset.py", line 23, in next
apk = next(self.source)
File "C:\Users\anaconda3\envs\az1\lib\site-packages\modules\entities\source.py", line 24, in next
return Apk(*line)
TypeError: init() takes from 1 to 12 positional arguments but 13 were given

thanks in advance

I got an error when trying to process "az" in command line. How could I sovle it?

Traceback (most recent call last):
File "C:\Python37\Scripts\az-script.py", line 11, in
load_entry_point('azoo==1.3.0', 'console_scripts', 'az')()
File "C:\Python37\lib\site-packages\click\core.py", line 722, in call
return self.main(*args, **kwargs)
File "C:\Python37\lib\site-packages\click\core.py", line 697, in main
rv = self.invoke(ctx)
File "C:\Python37\lib\site-packages\click\core.py", line 895, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "C:\Python37\lib\site-packages\click\core.py", line 535, in invoke
return callback(*args, **kwargs)
File "C:\Python37\lib\site-packages\modules\main.py", line 52, in run
az.run(input_file, key, number, criteria, out_dir=out if out else os.getcwd(), metadata=metadata, seed=seed, threads=threads)
File "C:\Python37\lib\site-packages\modules\az.py", line 18, in run
random_subset = RandomPicker(seed=seed).get_random_subset(dataset, number)
File "C:\Python37\lib\site-packages\modules\services\random_picker.py", line 14, in get_random_subset
for num, apk in enumerate(input_dataset, 1):
File "C:\Python37\lib\site-packages\modules\entities\dataset\filtered_dataset.py", line 23, in next
apk = next(self.source)
File "C:\Python37\lib\site-packages\modules\entities\source.py", line 20, in next
return Apk(*columns)
TypeError: init() takes from 1 to 12 positional arguments but 13 were given

Good day to you!
Today, I was playing with this nice tool. However, it did not understand my command.
My command was
az -n 15 -d 2019-01-01: -s :3000000 -vt 5
I believed this meant: Get me 15 apks, from 2019-01-01 or newer, with size 3 mb or less, and flagged as malicious by 5 or more analysis tools.

Documentation stated that -vt takes an integer argument.

By looking into the source code, I was able to figure out that a range X:Y is the actual argument, making the command look like az -n 15 -d 2019-01-01: -s :3000000 -vt 5:50000.

Could you update your documentation (and expand the example to use -vt maybe?)

Have a nice day,
Sebastiaan

Hi,

Thanks for the very useful software,

I'm wondering whether we can download from a custom set of APKs with defined sha256 stored in a separate .csv file.
Do we still need to include the big AndroZoo CSV file as input_file in this case?

I see an option for sha256 however there is not much available instruction to exploit further with that option.

Hello there,
I have a permission error on any where on my windows machine but how is it?! It should take at least my users' permissions. I have changed the input file sever times and get the following error:

Traceback (most recent call last):
File "C:\Users\a.ashraf\AppData\Local\Programs\Python\Python37\Scripts\az-script.py", line 11, in <module>
   load_entry_point('azoo==1.3.1', 'console_scripts', 'az')()
 File "C:\Users\a.ashraf\AppData\Local\Programs\Python\Python37\lib\site-packages\click\core.py", line 722, in __call__
   return self.main(*args, **kwargs)
 File "C:\Users\a.ashraf\AppData\Local\Programs\Python\Python37\lib\site-packages\click\core.py", line 697, in main
   rv = self.invoke(ctx)
 File "C:\Users\a.ashraf\AppData\Local\Programs\Python\Python37\lib\site-packages\click\core.py", line 895, in invoke
   return ctx.invoke(self.callback, **ctx.params)
 File "C:\Users\a.ashraf\AppData\Local\Programs\Python\Python37\lib\site-packages\click\core.py", line 535, in invoke
   return callback(*args, **kwargs)
 File "C:\Users\a.ashraf\AppData\Local\Programs\Python\Python37\lib\site-packages\modules\main.py", line 52, in run
   az.run(input_file, key, number, criteria, out_dir=out if out else os.getcwd(), metadata=metadata, seed=seed, threads=threads)
 File "C:\Users\a.ashraf\AppData\Local\Programs\Python\Python37\lib\site-packages\modules\az.py", line 15, in run
   source = Source(input_file=input_file)
 File "C:\Users\a.ashraf\AppData\Local\Programs\Python\Python37\lib\site-packages\modules\entities\source.py", line 9, in __init__
   self.input_file = open(input_file)
PermissionError: [Errno 13] Permission denied: 'C:\\Users\\a.ashraf\\Desktop'

Hello,

This is my command: az -n 10 -d 2015-12-11: -s :3000000 -m play.google.com,appchina

The csv file is download by: https://androzoo.uni.lu/lists

And the resp error is:
ValueError: time data '2022-11-19 6:29' does not match format '%Y-%m-%d %H:%M:%S'

Thanks.

Hi!
There's an error when I try to download the app. I use the input file from Androzoo and the file format already CSV.

This is my code when I try to download

What should I do to fix it?

I'm trying to use the tool to download some apks.
I have done various attempts, last with this command taken from another issue:

az -n 15 -d 2019-01-01: -s :3000000 -vt 5:50000 -k XXX

And as a result I get:

Input file is not defined.

Why would I need to specify an input file if I am downloading random apks? Moreover, what columns should I include in this input file? This information is not present in the docs. I made several attempts and I always get a 0 apks satisfy the criteria with any kind of input file.

Hi,

I execute the following command on Ubuntu's 19.10 terminal and got an error. So I wanted to know if I am executing right the command to download the apks or if I am missing something.

Command:
az -n 10 -d 2015-12-11: -s :3000000 -m play.google.com,appchina

Error:

Traceback (most recent call last):
  File "/usr/local/bin/az", line 11, in <module>
    load_entry_point('azoo==1.3.0', 'console_scripts', 'az')()
  File "/home/claudiomori/.local/lib/python2.7/site-packages/pkg_resources/__init__.py", line 489, in load_entry_point
    return get_distribution(dist).load_entry_point(group, name)
  File "/home/claudiomori/.local/lib/python2.7/site-packages/pkg_resources/__init__.py", line 2843, in load_entry_point
    return ep.load()
  File "/home/claudiomori/.local/lib/python2.7/site-packages/pkg_resources/__init__.py", line 2434, in load
    return self.resolve()
  File "/home/claudiomori/.local/lib/python2.7/site-packages/pkg_resources/__init__.py", line 2440, in resolve
    module = __import__(self.module_name, fromlist=['__name__'], level=0)
  File "/home/claudiomori/.local/lib/python2.7/site-packages/modules/main.py", line 50
    number, *criteria_args, metadata, key, input_file = Parser(args).parse()
            ^
SyntaxError: invalid syntax

Before executing the command I installed the requirements indicated on the requirements.txt file and also followed the instructions on the Readme file.

Thank you in advance

hell o please help me,
how i can create .az file?

Dear AndroZoo maintainer,

I recently notice that the AndroZoo seems to have difficulty handling the latest APK bundle format. (https://developer.android.com/guide/app-bundle)

Quite a lot of APKs from AndroZoo just contain the base module for DEX code. In other words, their native library binaries are missing.
Could you kindly look into this issue?

Cheers

Hi @ArtemKushnerov ,
I am getting
Key is not defined. Please, define configuration parameter 'key' in local or global configInput file is not defined.

I have defined it in the configInput.az file. The file is placed locally in my target directory.
key=cb428d39eff8f9977f95a3c..... input_file=C:\Users\Akshay Mathur\Desktop\Androzoo
Kindly help me out on how to resolve this issue

Environment: Ubuntu16.04, Python3.5

I followed the instructions to install az on my ubuntu virtual machine

pip install -e git+https://github.com/ArtemKushnerov/[email protected]#egg=az

then I got error says that the branch 1.2.0 doesn't exist, I checked the github release page and replace "1.2.0" with "1.0.1", and it worked. I got "Successfully installed" tips.
then I created .az file and fill my api key and the csv file path.
When I typed az into commandline, I got the errors as following:

Traceback (most recent call last):
File "/usr/local/bin/az", line 11, in
load_entry_point('az', 'console_scripts', 'az')()
File "/usr/local/lib/python3.5/dist-packages/pkg_resources/init.py", line 572, in load_entry_point
return get_distribution(dist).load_entry_point(group, name)
File "/usr/local/lib/python3.5/dist-packages/pkg_resources/init.py", line 2755, in load_entry_point
return ep.load()
File "/usr/local/lib/python3.5/dist-packages/pkg_resources/init.py", line 2408, in load
return self.resolve()
File "/usr/local/lib/python3.5/dist-packages/pkg_resources/init.py", line 2414, in resolve
module = import(self.module_name, fromlist=['name'], level=0)
File "/home/yuan/src/az/cli.py", line 6, in
from modules import adownloader, logging_util
File "/home/yuan/src/az/modules/adownloader.py", line 12
logging.info(f'START. APKS TO DOWNLOAD: {number}')
^
SyntaxError: invalid syntax

So how can I fix this ? Or is there something wrong in my installation steps?

Tried this command : az -n 1 -d 2020-01-01: -s :5000000 -m play.google.com
but i keep getting the error "AttributeError: module 'collections' has no attribute 'Hashable'".
Full log of the error:

Traceback (most recent call last):
File "/home/yahia/.local/bin/az", line 8, in
sys.exit(run())
File "/home/yahia/.local/lib/python3.10/site-packages/click/core.py", line 722, in call
return self.main(*args, **kwargs)
File "/home/yahia/.local/lib/python3.10/site-packages/click/core.py", line 697, in main
rv = self.invoke(ctx)
File "/home/yahia/.local/lib/python3.10/site-packages/click/core.py", line 895, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/home/yahia/.local/lib/python3.10/site-packages/click/core.py", line 535, in invoke
return callback(*args, **kwargs)
File "/home/yahia/.local/lib/python3.10/site-packages/modules/main.py", line 49, in run
logging_util.setup_logging()
File "/home/yahia/.local/lib/python3.10/site-packages/modules/logging_util.py", line 9, in setup_logging
logging.config.dictConfig(yaml.safe_load(f.read()))
File "/home/yahia/.local/lib/python3.10/site-packages/yaml/init.py", line 94, in safe_load
return load(stream, Loader=SafeLoader)
File "/home/yahia/.local/lib/python3.10/site-packages/yaml/init.py", line 73, in load
return loader.get_single_data()
File "/home/yahia/.local/lib/python3.10/site-packages/yaml/constructor.py", line 37, in get_single_data
return self.construct_document(node)
File "/home/yahia/.local/lib/python3.10/site-packages/yaml/constructor.py", line 46, in construct_document
for dummy in generator:
File "/home/yahia/.local/lib/python3.10/site-packages/yaml/constructor.py", line 398, in construct_yaml_map
value = self.construct_mapping(node)
File "/home/yahia/.local/lib/python3.10/site-packages/yaml/constructor.py", line 204, in construct_mapping
return super().construct_mapping(node, deep=deep)
File "/home/yahia/.local/lib/python3.10/site-packages/yaml/constructor.py", line 126, in construct_mapping
if not isinstance(key, collections.Hashable):
AttributeError: module 'collections' has no attribute 'Hashable'

When I run the program, no matter the configuration, I get some configuration messages ending with:
2022-04-13 12:53:52,659 - INFO - RANDOM SEED: 108529996446726935
and then nothing happens. No apks get downloaded and the program just sits here indefinitely.
Please can someone assist