when i typed with 'sudo python setup.py install --dynamic-linking'.
it return 'error: option --dynamic-linking not recognized'

Two questions.

1. The binary link for installing yara-python on Windows is broken in the documentation.
2. Provided that binaries for Windows have been built in the past, would it be possible to ship those by default (when installing on Windows)? On a plain Windows 7 machine yara-python will not install right now as visual studio is missing (which is not necessarily an error, but not everyone will have installed that).

OSX
Yara Version 3.5
yara-python==3.5.0

`

import yara
Traceback (most recent call last):
File "", line 1, in
ImportError: dlopen(/Library/Python/2.7/site-packages/yara_python-3.5.0-py2.7-macosx-10.11-intel.egg/yara.so, 2): Symbol not found: _yr_compiler_add_file
Referenced from: /Library/Python/2.7/site-packages/yara_python-3.5.0-py2.7-macosx-10.11-intel.egg/yara.so
Expected in: flat namespace
in /Library/Python/2.7/site-packages/yara_python-3.5.0-py2.7-macosx-10.11-intel.egg/yara.so
`

I am unable to compile under Windows 10 . I have installed VC++ for Python.

Can you help?

Here is the output:

C:\Python27\Scripts>pip.exe install yara-python
Collecting yara-python
Using cached yara-python-3.4.0.00.tar.gz
Installing collected packages: yara-python
Running setup.py install for yara-python ... error
Complete output from command c:\python27\python.exe -u -c "import setuptools, tokenize;file='c:\users\mlabelle\appdata\local\temp\pip-build-ygl0dc\yara-python\setup.py';exec(compile(getattr(tokenize, 'open', open)(file).read().replace('\r\n', '\n'), file, 'exec'))" install --record c:\users\mlabelle\appdata\local\temp\pip-1ojqtr-record\install-record.txt --single-version-externally-managed --compile:
c:\python27\lib\site-packages\setuptools\dist.py:285: UserWarning: Normalizing '3.4.0.00' to '3.4.0.0'
normalized_version,
running install
running build
running build_ext
building 'yara' extension
creating build
creating build\temp.win32-2.7
creating build\temp.win32-2.7\Release
creating build\temp.win32-2.7\Release\yara
creating build\temp.win32-2.7\Release\yara\libyara
creating build\temp.win32-2.7\Release\yara\libyara\modules
C:\Users\mlabelle\AppData\Local\Programs\Common\Microsoft\Visual C++ for Python\9.0\VC\Bin\cl.exe /c /nologo /Ox /MD /W3 /GS- /DNDEBUG -D_CRT_SECURE_NO_WARNINGS=1 -DHASH=1 -Iyara/libyara/include -Iyara/libyara/ -I. -Iyara/windows/include -Ic:\python27\include -Ic:\python27\PC /Tcyara-python.c /Fobuild\temp.win32-2.7\Release\yara-python.obj -O3
cl : Command line warning D9002 : ignoring unknown option '-O3'
yara-python.c
c:\users\mlabelle\appdata\local\temp\pip-build-ygl0dc\yara-python\yara\libyara\include\yara/filemap.h(34) : fatal error C1083: Cannot open include file: 'stdint.h': No such file or directory
error: command 'C:\Users\mlabelle\AppData\Local\Programs\Common\Microsoft\Visual C++ for Python\9.0\VC\Bin\cl.exe' failed with exit status 2

----------------------------------------

Command "c:\python27\python.exe -u -c "import setuptools, tokenize;file='c:\users\mlabelle\appdata\local\temp\pip-build-ygl0dc\yara-python\setup.py';exec(compile(getattr(tokenize, 'open', open)(file).read().replace('\r\n', '\n'), file, 'exec'))" install --record c:\users\mlabelle\appdata\local\temp\pip-1ojqtr-record\install-record.txt --single-version-externally-managed --compile" failed with error code 1 in c:\users\mlabelle\appdata\local\temp\pip-build-ygl0dc\yara-python\

On linux, file names are actually bytes, not unicode. Yara can not scan file containing non-unicode bytes:

import pathlib
import os
import yara
p = pathlib.Path(os.fsdecode(b'/tmp/\x44\xf9'))
p.write_text('malware')
rules = yara.compile('main.yara')
rules.match(str(p)) # UnicodeEncodeError: 'utf-8' codec can't encode character '\udcf9' in position 6: surrogates not allowed

How should I decode bytes filename to pass it to match() ?

I'm running into a similar issue as #3, but it occurs when I am matching against compiled rules from a file object:

>>> import yara
>>> # rules.yar is a 3.1Mb file with approximately 12k rules
>>> with open("rules.yar") as rules:
...       rules = yara.compile(file=rules)
>>> r.match(data="BADNESS")
[1] segmentation fault (core dumped)  python

I'm not sure if this problem is for Yara or Yara-python.

When you compile several rules from a dictionary, it produce a seg.fault. Bellow you can follow a python code to reproduce it.

import yara
rules_dict = dict()
content = 'import "androguard"\r\nimport "file"\r\nimport "cuckoo"\r\n\r\n\r\nrule koodous : official\r\n{\r\n\tmeta:\r\n\t\tdescription = "This rule detects the koodous application, used to show all Yara rules potential"\r\n\t\tsample = "e6ef34577a75fc0dc0a1f473304de1fc3a0d7d330bf58448db5f3108ed92741b"\r\n\r\n\tstrings:\r\n\t\t$a = {63 6F 6D 24 6B 6F 6F 64 6F 75 73 24 61 6E 64 72 6F 69 64}\r\n\r\n\tcondition:\r\n\t\tandroguard.package_name("com.koodous.android") and\r\n\t\tandroguard.app_name("Koodous") and\r\n\t\tandroguard.activity(/Details_Activity/i) and\r\n\t\tandroguard.permission(/android.permission.INTERNET/) and\r\n\t\tandroguard.certificate.sha1("8399A145C14393A55AC4FCEEFB7AB4522A905139") and\r\n\t\tandroguard.url(/koodous\\.com/) and\r\n\t\tnot file.md5("d367fd26b52353c2cce72af2435bd0d5") and \r\n\t\t$a and\r\n\t\tcuckoo.network.dns_lookup(/settings.crashlytics.com/) //Yes, we use crashlytics to debug our app!\r\n\t\t\r\n}\r\n'
for i in range(1000):
    rules_dict['rule_%d'%i] = content

yara.compile(sources=rules_dict)

It would be great to have access to the ahorasick automaton in https://github.com/VirusTotal/yara/blob/b11898bf78336a1ee770e8162a1ac21114ffab88/libyara/ahocorasick.c through this binding

Issue:
yara.load method is throwing an unknown error when yara compiled file is passed on as input.
I have tried using both filepath and file args in method but it shows the same error

Hello,
when building the yara python3 module I run the tests.py and some of the tests are failing. Same tests do not fail for python2 (same environment, same directory just python3 instead of python2 resp. nosetests-3.4 instead of nosetests-2.7)

The testcase testModuleData seems to be always failing with python3.
The testcase testCompare seems to have some race condition and is sometimes failing with python3.

I am testing with version 9fd9fd2.

PYTHONPATH=/rpmbuild/BUILDROOT/python-yara-3.5.0-0.rc1.fc23.x86_64//usr/lib64/python3.4/site-packages/ nosetests-3.4 -v
testAnonymousStrings (tests.TestYara) ... ok
testArithmeticOperators (tests.TestYara) ... ok
testAt (tests.TestYara) ... ok
testBitwiseOperators (tests.TestYara) ... ok
testBooleanOperators (tests.TestYara) ... ok
testCallback (tests.TestYara) ... ok
testComments (tests.TestYara) ... ok
testCompare (tests.TestYara) ... FAIL
testComparisonOperators (tests.TestYara) ... ok
testCompileFile (tests.TestYara) ... ok
testCompileFiles (tests.TestYara) ... ok
testCount (tests.TestYara) ... ok
testEntrypoint (tests.TestYara) ... ok
testExternals (tests.TestYara) ... ok
testFilesize (tests.TestYara) ... ok
testFor (tests.TestYara) ... ok
testHexStrings (tests.TestYara) ... ok
testIn (tests.TestYara) ... ok
testIncludeFiles (tests.TestYara) ... ok
testIntegerFunctions (tests.TestYara) ... ok
testLength (tests.TestYara) ... ok
testModuleData (tests.TestYara) ... FAIL
testModules (tests.TestYara) ... ok
testOf (tests.TestYara) ... ok
testOffset (tests.TestYara) ... ok
testRE (tests.TestYara) ... ok
testStringIO (tests.TestYara) ... ok
testStrings (tests.TestYara) ... ok
testSyntax (tests.TestYara) ... ok
testWildcardStrings (tests.TestYara) ... ok

FAIL: testCompare (tests.TestYara)

Traceback (most recent call last):
File "/rpmbuild/BUILD/yara-python-9fd9fd290872e36360e5e3839c49e21a908bf128/tests.py", line 840, in testCompare
self.assertTrue(m[0] < m[1])
AssertionError: False is not true

FAIL: testModuleData (tests.TestYara)

Traceback (most recent call last):
File "/rpmbuild/BUILD/yara-python-9fd9fd290872e36360e5e3839c49e21a908bf128/tests.py", line 930, in testModuleData
self.assertTrue(data['constants']['foo'] == 'foo')
AssertionError: False is not true

Best regards
Michal Ambroz

I installed in a clean virtual machine yara from sources:

# ./bootstrap.sh
# ./configure
# make
# sudo make install
# yara -v
yara 3.4.0

Then, I installed yara-python from github:

# git clone ...
# python setup.py build
# sudo python setup.py install

And when I received the next error when I import yara in python:

# python
Python 2.6.6 (r266:84292, Jul 23 2015, 15:22:56) 
[GCC 4.4.7 20120313 (Red Hat 4.4.7-11)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import yara
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: dynamic module does not define init function (inityara)

I am attempting to compile python-yara with dotnet support. The source code seems to be here but I cannot seem to enable the dotnet option.

[amorris] /tmp/yara-python $ git pull
Already up-to-date.
[amorris] /tmp/yara-python $ date
Mon May 22 10:23:55 EDT 2017
[amorris] /tmp/yara-python $ git status
On branch master
Your branch is up-to-date with 'origin/master'.
nothing to commit, working tree clean
[amorris] /tmp/yara-python $ python setup.py install --dynamic-linking
usage: setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
   or: setup.py --help [cmd1 cmd2 ...]
   or: setup.py --help-commands
   or: setup.py cmd --help

error: option --dynamic-linking not recognized
[amorris] /tmp/yara-python $ python setup.py install --enable-dotnet
usage: setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
   or: setup.py --help [cmd1 cmd2 ...]
   or: setup.py --help-commands
   or: setup.py cmd --help

error: option --enable-dotnet not recognized
[amorris] /tmp/yara-python $

I feel like I might be doing something stupid. Please let me know what other information I can provide. Thanks so much!

@mrphilroth

I am having an issue using the PE module within yara-python, although it works fine from command line yara.

Using yara 3.5.0 and yara-python 3.5.0. It looks somewhat related to #8 but in this case I don't have an issue on the import, but I can't use the functions.

(env) vagrant@web:/opt$ yara --version
yara 3.5.0
(env) vagrant@web:/opt$ python
Python 2.7.6 (default, Jun 22 2015, 17:58:13)
[GCC 4.8.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.

import yara
yara.version
'3.5.0'
yara.compile(source='import "pe" rule test_rule { condition: pe.imphash() == "A" * 32}')
Traceback (most recent call last):
File "", line 1, in
yara.SyntaxError: invalid field name "imphash"
yara.compile(source='import "pe"')
<yara.Rules object at 0x7f06dea91d00>

import yara
import sys

rule = yara.compile(filepath=sys.argv[1])
m = rule.match(pid=int(sys.argv[2]))
print(m)

rule ExampleRule
{
strings:
$my_text_string = "AAAAAA"

condition:
    $my_text_string 

}

C:\Users\User\Desktop>python y.py test.yar 7560
Traceback (most recent call last):
File "y.py", line 6, in
m = rule.match(pid=int(sys.argv[2]))
File "C:\Users\User\AppData\Roaming\Python\Python36\site-packages\yara\rules.py", line 354, in match
File "C:\Users\User\AppData\Roaming\Python\Python36\site-packages\yara\rules.py", line 326, in match_proc
File "C:\Users\User\AppData\Roaming\Python\Python36\site-packages\yara\rules.py", line 179, in match
fnc(*args)
File "C:\Users\User\AppData\Roaming\Python\Python36\site-packages\yara\libyara_wrapper.py", line 570, in yr_scan_proc
yara.libyara_wrapper.YaraMatchError: Not enough memory

C:\Users\User\Desktop>pip show yara-python
Name: yara-python
Version: 3.7.0
Summary: Python interface for YARA
Home-page: https://github.com/VirusTotal/yara-python
Author: Victor M. Alvarez
Author-email: [email protected];[email protected]
License: Apache 2.0
Location: c:\users\user\appdata\local\programs\python\python36\lib\site-packages
Requires:

C:\Users\User\Desktop>python --version
Python 3.6.3

Instead of formating the data, is it possible to pass a parameter (example: no_format=True) to get the compiler line, file and message into a python dictionary inside the Exception object?

platform Windows 7 VS 2010
setuptools 18.2
distutils 2.7.11

Error:

AssertionError: Should have already checked this

Where:

distutils\command\bdist_msi.py

I think this is related to how setup.py delays defining the Extension()

work around

setup.py build bdist_msi

Hi,
When I try to run yara on commandline, everything is ok but when I use yara-python, I recieved errors:

>>> import yara
>>> rules = yara.compile('Android_ASSDdeveloper.yar')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
yara.SyntaxError: Android_ASSDdeveloper.yar(22): invalid field name "certificate"

Android_ASSDdeveloper.yar use androguard-yara. And I had copy androguard.c into libyara/modules then do follow guide.

Another yara rule use cuckoo module throw error when I run by yara-python

>>> import yara
>>> rules = yara.compile('MALW_AZORULT.yar')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
yara.SyntaxError: MALW_AZORULT.yar(23): invalid field name "sync"

I had run ./configure --enable-cuckoo --enable-magic --enable-dotnet when compile yara.

Those rules are taken from https://github.com/Yara-Rules/rules

I don't know where was I wrong? :(

Greetings,

I have the following proof of concept code showing yara-python pid scanning not working, I've tried all I can to narrow down the problem to no avail. Can you tell me if I'm interacting with the api wrong or if this feature is not supported on Linux?

import yara,re,sys,syslog,os
from os.path import join, getsize

def findrules(path):
        rules_dict = {}
        rgx=re.compile('^.*\.yar[a]?$')

        for root, dirs, files in os.walk(path):
           for entry in files:
              filepath=root+"/"+entry
              if rgx.match(filepath):
                rules_dict.update({filepath:filepath})

        return rules_dict

def main():

     if len(sys.argv) < 2:
        print ("Please specify path to YARA rules.")
        sys.exit(1)

     path=sys.argv[1]
     rules = yara.compile(filepaths=findrules(path))
     while 1==1:
            pids = [pid for pid in os.listdir('/proc') if pid.isdigit()]
            for procid in pids:
               print ("Scanning process ", procid)
               try:
                 matches=rules.match(pid=int(procid),timeout=30)
                 if (matches):
                   msg = ("Process ", open(os.path.join('/proc', pid, 'cmdline'), 'rb').read(),
                          "Matched against YARA rule - ", str(matches[0]))
                   print (msg)
                   syslog.syslog(join("Malicious-Process:",str(msg)))
               except Exception as e:
                    print ("Process scanner exception: ",e)
                    traceback = sys.exc_info()
                    print (traceback)

if __name__ == "__main__":
    main()

To run just do 'python script.py /path/rulesdir/' , I've used similar steps to do packet scanning successfully.

➜  yara-python git:(master) ✗ python                                          
Python 2.7.6 (default, Jun 22 2015, 17:58:13) 
[GCC 4.8.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import yara
>>> rule = yara.compile(source='import "hash"')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
yara.SyntaxError: line 1: syntax error, unexpected _IDENTIFIER_, expecting $end or _RULE_ or _PRIVATE_ or _GLOBAL_

I try to write a python script to load .yara file, which contain import "hash" but it doesn't work

I installed in a clean virtual machine yara from sources:

./bootstrap.sh
./configure
make
sudo make install

and yara works fine.

Then, I installed yara-python from github:

git clone ...
python setup.py build
sudo python setup.py install

And when I received the next error when I import yara in python:

Python 2.7.6 (default, Jun 22 2015, 17:58:13) 
[GCC 4.8.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import yara
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: /usr/local/lib/python2.7/dist-packages/yara_python-3.4.0.0-py2.7.egg/yara.so: undefined symbol: yr_finalize

Any idea?

I'd like to request the ability for yara.compile() as well as rules.match() to support Unicode filepaths. For now a dirty workaround would seem to be creating a temporary symbolic link (containing only ASCII characters) that points to the Unicode filepath. But that naturally doesn't work under Windows, so no luck there unfortunately.
AFAIK changing s to u in the Python argument parsing and fopen to wfopen (or something along those lines) will go a long way. Thanks!

Example situation where support for this would help @ cuckoosandbox/cuckoo#1573

FYI you likely want to have sdist handle updating the .y and .l generated code files

The yara-python 3.5.0 version provides via https://pypi.python.org/pypi/yara-python/3.5.0 contains stale .y and .l generated code files e.g. yara/libyara/grammar.c still includes stdint.h while gammar.y references yara/integers.h. Currently this causes yara-python unable to build on Windows.

import yara
r = yara.compile('/tmp/DC.yar')
dir(r)
['doc', 'init', 'module', 'str', '_context_args', '_contexts', '_strings', 'context', 'free', 'match', 'match_data', 'match_path', 'match_proc', 'namespaces', 'weight']
m = r.match_data(data)
print m
{'main': [{'meta': {'date': '2014/04', 'maltype': 'Remote Access Trojan', 'filetype': 'exe', 'ref': 'http://malwareconfig.com/stats/DarkComet', 'author': ' Kevin Breen [email protected]'}, 'tags': [], 'matches': True, 'strings': [{'flags': 19, 'identifier': '$b5', 'data': '#KCMDDC', 'offset': 443840L}, {'flags': 19, 'identifier': '$b4', 'data': '#BOT#VisitUrl', 'offset': 513712L}, {'flags': 19, 'identifier': '$b3', 'data': "I wasn't able to open the hosts file", 'offset': 467496L}, {'flags': 19, 'identifier': '$b2', 'data': '%s, ClassID: %s', 'offset': 176972L}, {'flags': 19, 'identifier': '$b1', 'data': 'FastMM Borland Edition', 'offset': 2056L}, {'flags': 19, 'identifier': '$a2', 'data': 'Command successfully executed!', 'offset': 513989L}, {'flags': 19, 'identifier': '$a1', 'data': '#BOT#URLUpdate', 'offset': 514220L}], 'rule': 'DarkComet'}]}

m[0].rule
Traceback (most recent call last):
File "", line 1, in
KeyError: 0

m['main'][0]['rule']
'DarkComet'

Yara-python build should also accept --with-crypto option in addition to other yara build configuration options like --enable-cuckoo.

Hello,
with the version 3.6.x (3.6.3 to be precise) the test testCompileFile throws segmentation fault on the s390x architecture.
This might or might not be related to issues with tests failing randomly on ppc and s390x architecture as reported in the issue #25

https://kojipkgs.fedoraproject.org//work/tasks/9619/20589619/build.log
This test testCompileFile was working fine with the version 3.5.0 on s390x platform.

Executing(%check): /bin/sh -e /var/tmp/rpm-tmp.VEYfW8
+ umask 022
+ cd /builddir/build/BUILD
+ cd yara-python-3.6.3
+ EXCLUDE=
+ EXCLUDE='--exclude=^testModuleData$|^testEntrypoint$|^testIn$|^testIntegerFunctions$'
++ ls /usr/bin/nosetests-2.7
+ NOSETESTS2=/usr/bin/nosetests-2.7
+ PYTHONPATH=/builddir/build/BUILDROOT/python-yara-3.6.3-2.fc27.s390x//usr/lib64/python2.7/site-packages/
+ /usr/bin/nosetests-2.7 -v '--exclude=^testModuleData$|^testEntrypoint$|^testIn$|^testIntegerFunctions$'
testAnonymousStrings (tests.TestYara) ... ok
testArithmeticOperators (tests.TestYara) ... ok
testAt (tests.TestYara) ... ok
testBitwiseOperators (tests.TestYara) ... ok
testBooleanOperators (tests.TestYara) ... ok
testCallback (tests.TestYara) ... ok
testComments (tests.TestYara) ... ok
testCompare (tests.TestYara) ... ok
testComparisonOperators (tests.TestYara) ... ok
testCompileFile (tests.TestYara) ... /var/tmp/rpm-tmp.VEYfW8: line 43: 21200 Segmentation fault      (core dumped) PYTHONPATH=/builddir/build/BUILDROOT/python-yara-3.6.3-2.fc27.s390x//usr/lib64/python2.7/site-packages/ "$NOSETESTS2" -v "$EXCLUDE"
error: Bad exit status from /var/tmp/rpm-tmp.VEYfW8 (%check)

For example:

compiled = yara.compile(filepaths = {
'from_file1': '/path/rules.1',
'from_file2': '/path/rules.2',
},
sources = {
'from_text': text,
})

Latest git as of a few minutes ago gives me:

x86_64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fno-strict-aliasing -Wdate-time -D_FORTIFY_SOURCE=2 -g -fstack-protector-strong -Wformat -Werror=format-security -fPIC -DHAVE_MEMMEM=1 -DHASH_MODULE=1 -DHAVE_LIBCRYPTO=1 -Iyara/libyara/include -Iyara/libyara/ -I. -I/usr/include/python2.7 -c yara-python.c -o build/temp.linux-x86_64-2.7/yara-python.o
yara-python.c: In function ‘convert_object_to_python’:
yara-python.c:420:17: error: ‘YR_OBJECT {aka struct _YR_OBJECT}’ has no member named ‘value’
       if (object->value.i != UNDEFINED)
                 ^
yara-python.c:421:43: error: ‘YR_OBJECT {aka struct _YR_OBJECT}’ has no member named ‘value’
         result = Py_BuildValue("i", object->value.i);
                                           ^
yara-python.c:425:17: error: ‘YR_OBJECT {aka struct _YR_OBJECT}’ has no member named ‘value’
       if (object->value.ss != NULL)
                 ^
yara-python.c:427:19: error: ‘YR_OBJECT {aka struct _YR_OBJECT}’ has no member named ‘value’
             object->value.ss->c_string,
                   ^
yara-python.c:428:19: error: ‘YR_OBJECT {aka struct _YR_OBJECT}’ has no member named ‘value’
             object->value.ss->length);
                   ^
yara-python.c:432:44: warning: implicit declaration of function ‘object_as_structure’ [-Wimplicit-function-declaration]
       result = convert_structure_to_python(object_as_structure(object));
                                            ^
yara-python.c:432:44: warning: passing argument 1 of ‘convert_structure_to_python’ makes pointer from integer without a cast [-Wint-conversion]
yara-python.c:397:11: note: expected ‘YR_OBJECT_STRUCTURE * {aka struct _YR_OBJECT_STRUCTURE *}’ but argument is of type ‘int’
 PyObject* convert_structure_to_python(
           ^
yara-python.c:436:40: warning: implicit declaration of function ‘object_as_array’ [-Wimplicit-function-declaration]
       result = convert_array_to_python(object_as_array(object));
                                        ^
yara-python.c:436:40: warning: passing argument 1 of ‘convert_array_to_python’ makes pointer from integer without a cast [-Wint-conversion]
yara-python.c:401:11: note: expected ‘YR_OBJECT_ARRAY * {aka struct _YR_OBJECT_ARRAY *}’ but argument is of type ‘int’
 PyObject* convert_array_to_python(
           ^
yara-python.c:444:45: warning: implicit declaration of function ‘object_as_dictionary’ [-Wimplicit-function-declaration]
       result = convert_dictionary_to_python(object_as_dictionary(object));
                                             ^
yara-python.c:444:45: warning: passing argument 1 of ‘convert_dictionary_to_python’ makes pointer from integer without a cast [-Wint-conversion]
yara-python.c:405:11: note: expected ‘YR_OBJECT_DICTIONARY * {aka struct _YR_OBJECT_DICTIONARY *}’ but argument is of type ‘int’
 PyObject* convert_dictionary_to_python(
           ^
In file included from /usr/include/python2.7/pyport.h:325:0,
                 from /usr/include/python2.7/Python.h:58,
                 from yara-python.c:18:
yara-python.c:448:24: error: ‘YR_OBJECT {aka struct _YR_OBJECT}’ has no member named ‘value’
       if (!isnan(object->value.d))
                        ^
yara-python.c:449:43: error: ‘YR_OBJECT {aka struct _YR_OBJECT}’ has no member named ‘value’
         result = Py_BuildValue("d", object->value.d);
                                           ^
yara-python.c: In function ‘yara_callback’:
yara-python.c:638:9: warning: passing argument 1 of ‘convert_structure_to_python’ makes pointer from integer without a cast [-Wint-conversion]
         object_as_structure(message_data));
         ^
yara-python.c:460:11: note: expected ‘YR_OBJECT_STRUCTURE * {aka struct _YR_OBJECT_STRUCTURE *}’ but argument is of type ‘int’
 PyObject* convert_structure_to_python(
           ^
yara-python.c:643:57: error: invalid type argument of ‘->’ (have ‘int’)
     object = PY_STRING(object_as_structure(message_data)->identifier);
                                                         ^
yara-python.c:52:42: note: in definition of macro ‘PY_STRING’
 #define PY_STRING(x) PyString_FromString(x)
                                          ^
yara-python.c:721:60: error: ‘YR_MATCH {aka struct _YR_MATCH}’ has no member named ‘data_length’
       object = PyBytes_FromStringAndSize((char*) m->data, m->data_length);
                                                            ^
yara-python.c: In function ‘handle_error’:
yara-python.c:908:10: error: ‘ERROR_INSUFFICIENT_MEMORY’ undeclared (first use in this function)
     case ERROR_INSUFFICIENT_MEMORY:
          ^
yara-python.c:908:10: note: each undeclared identifier is reported only once for each function it appears in
yara-python.c:934:10: error: ‘ERROR_INVALID_EXTERNAL_VARIABLE_TYPE’ undeclared (first use in this function)
     case ERROR_INVALID_EXTERNAL_VARIABLE_TYPE:
          ^
yara-python.c: In function ‘yara_compile’:
yara-python.c:1957:31: error: ‘ERROR_INSUFFICIENT_MEMORY’ undeclared (first use in this function)
         result = handle_error(ERROR_INSUFFICIENT_MEMORY, NULL);
                               ^
error: command 'x86_64-linux-gnu-gcc' failed with exit status 1

I saw the previous closed ERROR_INSUFFICIENT_MEMORY, but I thought that was fixed. Thank you.

Traceback (most recent call last):
  File "yara.py", line 5, in <module>
    import yara
  File "/home/mour/working/script/yara.py", line 8, in <module>
    rules = yara.compile("./php.yar")
AttributeError: 'module' object has no attribute 'compile'

I don't know what's wrong with it, No matter what, I use pip install yara-python also compile from source code, always was wrong.

conda:  python version: 3.6
yara lastst version compile from source code

We recently created a module for dalvik executables: https://github.com/rednaga/yara/blob/master/libyara/modules/dex.c

We're using the module for a tool we wrote called apkid, so we have a vested interest in maintaining the module: https://github.com/rednaga/APKiD

Would this be something you're interested in merging into this project? We're not C wizards but we did fuzz the code to make sure there weren't any obvious crashing vulns.

I could make a pull request with this module, but I'm not sure how you'd want it integrated. Either as an optional --with-dex compile flag or enabled by default. Please advise.

cc @strazzere

It looks like you have pushed this version up to https://testpypi.python.org, but someone else ( mjdorma/ Michael Dorman) is the owner/maintainer on pypi. Is there any push to get this consolidated/figured out so YARA 3.4 can be for the masses? Thanks!

https://pypi.python.org/pypi/yara
https://testpypi.python.org/pypi/yara-python/3.4.4

At first the error suggests to use option -std=c99 in order to compile.

~/Downloads/yara-python$ python setup.py build
running build
running build_ext
building 'yara' extension
creating build
creating build/temp.linux-x86_64-2.7
creating build/temp.linux-x86_64-2.7/yara
creating build/temp.linux-x86_64-2.7/yara/libyara
creating build/temp.linux-x86_64-2.7/yara/libyara/modules
x86_64-linux-gnu-gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -DHAVE_MEMMEM=1 -DHASH_MODULE=1 -Iyara/libyara/include -Iyara/libyara/ -I. -I/usr/include/python2.7 -c yara-python.c -o build/temp.linux-x86_64-2.7/yara-python.o
x86_64-linux-gnu-gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -DHAVE_MEMMEM=1 -DHASH_MODULE=1 -Iyara/libyara/include -Iyara/libyara/ -I. -I/usr/include/python2.7 -c yara/libyara/object.c -o build/temp.linux-x86_64-2.7/yara/libyara/object.o
x86_64-linux-gnu-gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -DHAVE_MEMMEM=1 -DHASH_MODULE=1 -Iyara/libyara/include -Iyara/libyara/ -I. -I/usr/include/python2.7 -c yara/libyara/lexer.c -o build/temp.linux-x86_64-2.7/yara/libyara/lexer.o
x86_64-linux-gnu-gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -DHAVE_MEMMEM=1 -DHASH_MODULE=1 -Iyara/libyara/include -Iyara/libyara/ -I. -I/usr/include/python2.7 -c yara/libyara/grammar.c -o build/temp.linux-x86_64-2.7/yara/libyara/grammar.o
x86_64-linux-gnu-gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -DHAVE_MEMMEM=1 -DHASH_MODULE=1 -Iyara/libyara/include -Iyara/libyara/ -I. -I/usr/include/python2.7 -c yara/libyara/sizedstr.c -o build/temp.linux-x86_64-2.7/yara/libyara/sizedstr.o
x86_64-linux-gnu-gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -DHAVE_MEMMEM=1 -DHASH_MODULE=1 -Iyara/libyara/include -Iyara/libyara/ -I. -I/usr/include/python2.7 -c yara/libyara/rules.c -o build/temp.linux-x86_64-2.7/yara/libyara/rules.o
yara/libyara/rules.c: In function ‘yr_rules_save_stream’:
yara/libyara/rules.c:743:3: error: ‘for’ loop initial declarations are only allowed in C99 mode
   for (int i = 0; i < YR_BITARRAY_NCHARS(MAX_THREADS); ++i) {
   ^
yara/libyara/rules.c:743:3: note: use option -std=c99 or -std=gnu99 to compile your code
error: command 'x86_64-linux-gnu-gcc' failed with exit status 1

++++++++++++++++++++++++++++++++++++++++++++++++++++++++
After adding the required flag build still fails at sigjmp_buf function.
(unknown type name ‘sigjmp_buf’)
Please help how to proceed with the compilation process.
Thanks ahead!
++++++++++++++++++++++++++++++++++++++++++++++++++++++++

~/Downloads/yara-python$ python setup.py build
running build
running build_ext
building 'yara' extension
x86_64-linux-gnu-gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -DHAVE_MEMMEM=1 -DHASH_MODULE=1 -Iyara/libyara/include -Iyara/libyara/ -I. -I/usr/include/python2.7 -c yara-python.c -o build/temp.linux-x86_64-2.7/yara-python.o --std=c99
x86_64-linux-gnu-gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -DHAVE_MEMMEM=1 -DHASH_MODULE=1 -Iyara/libyara/include -Iyara/libyara/ -I. -I/usr/include/python2.7 -c yara/libyara/object.c -o build/temp.linux-x86_64-2.7/yara/libyara/object.o --std=c99
x86_64-linux-gnu-gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -DHAVE_MEMMEM=1 -DHASH_MODULE=1 -Iyara/libyara/include -Iyara/libyara/ -I. -I/usr/include/python2.7 -c yara/libyara/lexer.c -o build/temp.linux-x86_64-2.7/yara/libyara/lexer.o --std=c99
x86_64-linux-gnu-gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -DHAVE_MEMMEM=1 -DHASH_MODULE=1 -Iyara/libyara/include -Iyara/libyara/ -I. -I/usr/include/python2.7 -c yara/libyara/grammar.c -o build/temp.linux-x86_64-2.7/yara/libyara/grammar.o --std=c99
x86_64-linux-gnu-gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -DHAVE_MEMMEM=1 -DHASH_MODULE=1 -Iyara/libyara/include -Iyara/libyara/ -I. -I/usr/include/python2.7 -c yara/libyara/sizedstr.c -o build/temp.linux-x86_64-2.7/yara/libyara/sizedstr.o --std=c99
x86_64-linux-gnu-gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -DHAVE_MEMMEM=1 -DHASH_MODULE=1 -Iyara/libyara/include -Iyara/libyara/ -I. -I/usr/include/python2.7 -c yara/libyara/rules.c -o build/temp.linux-x86_64-2.7/yara/libyara/rules.o --std=c99
In file included from yara/libyara/rules.c:51:0:
yara/libyara/exception.h:81:1: error: unknown type name ‘sigjmp_buf’
 sigjmp_buf *exc_jmp_buf[MAX_THREADS];
 ^
yara/libyara/exception.h: In function ‘exception_handler’:
yara/libyara/exception.h:89:7: warning: passing argument 1 of ‘siglongjmp’ makes pointer from integer without a cast [enabled by default]
       siglongjmp(*exc_jmp_buf[tidx], 1);
       ^
In file included from /usr/include/features.h:374:0,
                 from /usr/include/assert.h:35,
                 from yara/libyara/rules.c:30:
/usr/include/x86_64-linux-gnu/bits/setjmp2.h:31:13: note: expected ‘struct __jmp_buf_tag *’ but argument is of type ‘int’
 extern void __REDIRECT_NTHNL (siglongjmp,
             ^
In file included from yara/libyara/rules.c:51:0:
yara/libyara/rules.c: In function ‘yr_rules_scan_mem_blocks’:
yara/libyara/exception.h:100:22: error: storage size of ‘oldact’ isn’t known
     struct sigaction oldact;                                    \
                      ^
yara/libyara/rules.c:437:7: note: in expansion of macro ‘YR_TRYCATCH’
       YR_TRYCATCH({
       ^
yara/libyara/exception.h:101:22: error: storage size of ‘act’ isn’t known
     struct sigaction act;                                       \
                      ^
yara/libyara/rules.c:437:7: note: in expansion of macro ‘YR_TRYCATCH’
       YR_TRYCATCH({
       ^
yara/libyara/exception.h:102:5: error: unknown type name ‘sigset_t’
     sigset_t oldmask;                                           \
     ^
yara/libyara/rules.c:437:7: note: in expansion of macro ‘YR_TRYCATCH’
       YR_TRYCATCH({
       ^
yara/libyara/exception.h:101:12: warning: implicit declaration of function ‘sigemptyset’ [-Wimplicit-function-declaration]
     struct sigaction act;                                       \
            ^
yara/libyara/rules.c:437:7: note: in expansion of macro ‘YR_TRYCATCH’
       YR_TRYCATCH({
       ^
yara/libyara/exception.h:101:12: warning: implicit declaration of function ‘pthread_sigmask’ [-Wimplicit-function-declaration]
     struct sigaction act;                                       \
            ^
yara/libyara/rules.c:437:7: note: in expansion of macro ‘YR_TRYCATCH’
       YR_TRYCATCH({
       ^
yara/libyara/exception.h:108:21: error: ‘SIG_SETMASK’ undeclared (first use in this function)
     pthread_sigmask(SIG_SETMASK, &act.sa_mask, &oldmask);       \
                     ^
yara/libyara/rules.c:437:7: note: in expansion of macro ‘YR_TRYCATCH’
       YR_TRYCATCH({
       ^
yara/libyara/exception.h:108:21: note: each undeclared identifier is reported only once for each function it appears in
     pthread_sigmask(SIG_SETMASK, &act.sa_mask, &oldmask);       \
                     ^
yara/libyara/rules.c:437:7: note: in expansion of macro ‘YR_TRYCATCH’
       YR_TRYCATCH({
       ^
yara/libyara/exception.h:101:12: warning: implicit declaration of function ‘sigaction’ [-Wimplicit-function-declaration]
     struct sigaction act;                                       \
            ^
yara/libyara/rules.c:437:7: note: in expansion of macro ‘YR_TRYCATCH’
       YR_TRYCATCH({
       ^
yara/libyara/exception.h:112:5: error: unknown type name ‘sigjmp_buf’
     sigjmp_buf jb;                                              \
     ^
yara/libyara/rules.c:437:7: note: in expansion of macro ‘YR_TRYCATCH’
       YR_TRYCATCH({
       ^
yara/libyara/exception.h:101:12: warning: implicit declaration of function ‘sigsetjmp’ [-Wimplicit-function-declaration]
     struct sigaction act;                                       \
            ^
yara/libyara/rules.c:437:7: note: in expansion of macro ‘YR_TRYCATCH’
       YR_TRYCATCH({
       ^
yara/libyara/exception.h:101:22: warning: unused variable ‘act’ [-Wunused-variable]
     struct sigaction act;                                       \
                      ^
yara/libyara/rules.c:437:7: note: in expansion of macro ‘YR_TRYCATCH’
       YR_TRYCATCH({
       ^
yara/libyara/exception.h:100:22: warning: unused variable ‘oldact’ [-Wunused-variable]
     struct sigaction oldact;                                    \
                      ^
yara/libyara/rules.c:437:7: note: in expansion of macro ‘YR_TRYCATCH’
       YR_TRYCATCH({
       ^
yara/libyara/exception.h:100:22: error: storage size of ‘oldact’ isn’t known
     struct sigaction oldact;                                    \
                      ^
yara/libyara/rules.c:450:5: note: in expansion of macro ‘YR_TRYCATCH’
     YR_TRYCATCH({
     ^
yara/libyara/exception.h:101:22: error: storage size of ‘act’ isn’t known
     struct sigaction act;                                       \
                      ^
yara/libyara/rules.c:450:5: note: in expansion of macro ‘YR_TRYCATCH’
     YR_TRYCATCH({
     ^
yara/libyara/exception.h:102:5: error: unknown type name ‘sigset_t’
     sigset_t oldmask;                                           \
     ^
yara/libyara/rules.c:450:5: note: in expansion of macro ‘YR_TRYCATCH’
     YR_TRYCATCH({
     ^
yara/libyara/exception.h:112:5: error: unknown type name ‘sigjmp_buf’
     sigjmp_buf jb;                                              \
     ^
yara/libyara/rules.c:450:5: note: in expansion of macro ‘YR_TRYCATCH’
     YR_TRYCATCH({
     ^
yara/libyara/exception.h:101:22: warning: unused variable ‘act’ [-Wunused-variable]
     struct sigaction act;                                       \
                      ^
yara/libyara/rules.c:450:5: note: in expansion of macro ‘YR_TRYCATCH’
     YR_TRYCATCH({
     ^
yara/libyara/exception.h:100:22: warning: unused variable ‘oldact’ [-Wunused-variable]
     struct sigaction oldact;                                    \
                      ^
yara/libyara/rules.c:450:5: note: in expansion of macro ‘YR_TRYCATCH’
     YR_TRYCATCH({
     ^
yara/libyara/exception.h:100:22: error: storage size of ‘oldact’ isn’t known
     struct sigaction oldact;                                    \
                      ^
yara/libyara/rules.c:468:3: note: in expansion of macro ‘YR_TRYCATCH’
   YR_TRYCATCH({
   ^
yara/libyara/exception.h:101:22: error: storage size of ‘act’ isn’t known
     struct sigaction act;                                       \
                      ^
yara/libyara/rules.c:468:3: note: in expansion of macro ‘YR_TRYCATCH’
   YR_TRYCATCH({
   ^
yara/libyara/exception.h:102:5: error: unknown type name ‘sigset_t’
     sigset_t oldmask;                                           \
     ^
yara/libyara/rules.c:468:3: note: in expansion of macro ‘YR_TRYCATCH’
   YR_TRYCATCH({
   ^
yara/libyara/exception.h:112:5: error: unknown type name ‘sigjmp_buf’
     sigjmp_buf jb;                                              \
     ^
yara/libyara/rules.c:468:3: note: in expansion of macro ‘YR_TRYCATCH’
   YR_TRYCATCH({
   ^
yara/libyara/exception.h:101:22: warning: unused variable ‘act’ [-Wunused-variable]
     struct sigaction act;                                       \
                      ^
yara/libyara/rules.c:468:3: note: in expansion of macro ‘YR_TRYCATCH’
   YR_TRYCATCH({
   ^
yara/libyara/exception.h:100:22: warning: unused variable ‘oldact’ [-Wunused-variable]
     struct sigaction oldact;                                    \
                      ^
yara/libyara/rules.c:468:3: note: in expansion of macro ‘YR_TRYCATCH’
   YR_TRYCATCH({
   ^
error: command 'x86_64-linux-gnu-gcc' failed with exit status 1

Trying to install yara-python on deb. No install issues using
python setup.py
python setup.py build
sudo python setup.py install

However when I run my .py to import yara I get:
python3 1129.py

Traceback (most recent call last):
File "/home/scripts/script1/1129.py", line 3, in
import yara
ImportError: No module named 'yara'

help('modules')

Please wait a moment while I gather a list of all available modules...

Failed to connect to Mir: Failed to connect to server socket: No such file or directory
Unable to init server: Could not connect: Connection refused
Failed to connect to Mir: Failed to connect to server socket: No such file or directory
Unable to init server: Could not connect: Connection refused
/usr/lib/python3/dist-packages/UpdateManager/Dialogs.py:25: PyGIWarning: Gtk was imported without specifying a version first. Use gi.require_version('Gtk', '3.0') before import to ensure that the right version gets loaded.
from gi.repository import Gtk
/usr/lib/python3/dist-packages/pyatspi/init.py:17: PyGIWarning: Atspi was imported without specifying a version first. Use gi.require_version('Atspi', '2.0') before import to ensure that the right version gets loaded.
from gi.repository import Atspi
/usr/lib/python3/dist-packages/usbcreator/backends/udisks/backend.py:4: PyGIWarning: UDisks was imported without specifying a version first. Use gi.require_version('UDisks', '2.0') before import to ensure that the right version gets loaded.
from gi.repository import Gio, GLib, UDisks
/usr/lib/python3/dist-packages/usbcreator/frontends/gtk/unitysupport.py:27: PyGIWarning: Unity was imported without specifying a version first. Use gi.require_version('Unity', '7.0') before import to ensure that the right version gets loaded.
from gi.repository import Unity
AptUrl apport_python_hook hmac resource
CDROM apt hpmudext rlcompleter
CommandNotFound apt_inst html runpy
DLFCN apt_pkg html5lib scanext
DistUpgrade aptdaemon http sched
IN aptsources httplib2 select
LanguageSelector argparse idlelib selectors
NvidiaDetector array idna sessioninstaller
Onboard ast imaplib setup
PIL asynchat imghdr setuptools
Quirks asyncio imp shelve
TYPES asyncore importlib shlex
UbuntuDrivers atexit inspect shutil
UbuntuSystemService audioop io signal
UpdateManager base64 ipaddress site
future bdb itertools sitecustomize
_ast binascii janitor six
_bisect binhex jinja2 smtpd
_bootlocale bisect json smtplib
_bz2 blinker jwt sndhdr
_cffi_backend brlapi keyword socket
_codecs bs4 language_support_pkgs socketserver
_codecs_cn builtins lib2to3 softwareproperties
_codecs_hk bz2 linecache speechd
_codecs_iso2022 cProfile locale speechd_config
_codecs_jp cairo logging spwd
_codecs_kr calendar louis sqlite3
_codecs_tw cgi lsb_release sre_compile
_collections cgitb lxml sre_constants
_collections_abc chardet lzma sre_parse
_compat_pickle checkbox_support macpath ssh_import_id
_compression chunk macurl2path ssl
_crypt cmath mailbox stat
_csv cmd mailcap statistics
_ctypes code mako string
_ctypes_test codecs markupsafe stringprep
_curses codeop marshal struct
_curses_panel collections math subprocess
_datetime colorsys mimetypes sunau
_dbm compileall mmap symbol
_dbus_bindings concurrent modulefinder symtable
_dbus_glib_bindings configparser multiprocessing sys
_decimal contextlib netrc sysconfig
_dummy_thread copy nis syslog
_elementtree copyreg nntplib systemd
_functools crypt ntpath tabnanny
_gdbm cryptography nturl2path tarfile
_hashlib csv numbers telnetlib
_heapq ctypes oauthlib tempfile
_imp cups opcode termios
_io cupsext operator test
_json cupshelpers optparse tests
_locale curl orca textwrap
_lsprof curses os this
_lzma datetime ossaudiodev threading
_markupbase dbm padme time
_md5 dbus parser timeit
_multibytecodec deb822 pathlib tkinter
_multiprocessing debconf pcardext token
_opcode debian pdb tokenize
_operator debian_bundle pexpect trace
_osx_support decimal pickle traceback
_pickle defer pickletools tracemalloc
_posixsubprocess difflib pipes tty
_pydecimal dis pkg_resources turtle
_pyio distutils pkgutil types
_random doctest plainbox typing
_sha1 dummy_threading platform ufw
_sha256 easy_install plistlib unicodedata
_sha512 email poplib unittest
_signal encodings posix uno
_sitebuiltins enum posixpath unohelper
_socket errno pprint urllib
_sqlite3 faulthandler problem_report urllib3
_sre fcntl profile usbcreator
_ssl feedparser pstats uu
_stat feedparser_sgmllib3 pty uuid
_string filecmp ptyprocess venv
_strptime fileinput pwd warnings
_struct fnmatch py_compile wave
_symtable formatter pyasn1 weakref
_sysconfigdata fpectl pyatspi webbrowser
_sysconfigdata_m fractions pyclbr wsgiref
_testbuffer ftplib pycurl xdg
_testcapi functools pydoc xdiagnose
_testimportmultiple gc pydoc_data xdrlib
_testmultiphase genericpath pyexpat xkit
_thread getopt pygtkcompat xlsxwriter
_threading_local getpass pyparsing xml
_tracemalloc gettext queue xmlrpc
_warnings gi quopri xxlimited
_weakref glob random xxsubtype
_weakrefset grp re zipapp
abc guacamole readline zipfile
aifc gzip reportlab zipimport
antigravity hashlib reprlib zlib
apport heapq requests

Enter any module name to get more help. Or, type "modules spam" to search
for modules whose name or summary contain the string "spam".

Is this supposed to be working yet?

$ pip install -I yara-python
Downloading/unpacking yara-python
  Could not find any downloads that satisfy the requirement yara-python
Cleaning up...
No distributions at all found for yara-python
Storing debug log for failure in /home/scudette/.pip/pip.log

Also when installing by hand it seems to make an empty package. Even after git submodule init/update.

I want to use Androguard and cuckoo in yara, so i recompile yara. And in command-line, it can work well, But when i use yara-python, it raise some error, just like.

rule = yara.compile(source='import "cuckoo"')
yara.SyntaxError: line 1: unknown module "cuckoo"

so how to fix it?

When I did use yara-python released 3.5.0, It have not setup problems and so well.
but, I got memory insufficient error from committed eaba8c1 when build setup
although have enough swap space.

free -m
total used free shared buffers cached
Mem: 3009 1036 1973 12 90 671
-/+ buffers/cache: 274 2735
Swap: 13311 0 13311

I know so lack physical memory space of my workspace,
but It is not lack because of all memory space with swap space.

python setup.py build
running build
running build_ext
building 'yara' extension
x86_64-linux-gnu-gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -DHAVE_MEMMEM=1 -DHASH_MODULE=1 -Iyara/libyara/include -Iyara/libyara/ -I. -I/usr/include/python2.7 -c yara-python.c -o build/temp.linux-x86_64-2.7/yara-python.o
yara-python.c: In function ‘yara_callback’:
yara-python.c:727:60: error: ‘YR_MATCH’ has no member named ‘data_length’
object = PyBytes_FromStringAndSize((char*) m->data, m->data_length);
^
yara-python.c: In function ‘handle_error’:
yara-python.c:914:10: error: ‘ERROR_INSUFFICIENT_MEMORY’ undeclared (first use in this function)
case ERROR_INSUFFICIENT_MEMORY:
^
yara-python.c:914:10: note: each undeclared identifier is reported only once for each function it appears in
yara-python.c:940:10: error: ‘ERROR_INVALID_EXTERNAL_VARIABLE_TYPE’ undeclared (first use in this function)
case ERROR_INVALID_EXTERNAL_VARIABLE_TYPE:
^
yara-python.c: In function ‘yara_compile’:
yara-python.c:1958:31: error: ‘ERROR_INSUFFICIENT_MEMORY’ undeclared (first use in this function)
result = handle_error(ERROR_INSUFFICIENT_MEMORY, NULL);
^
error: command 'x86_64-linux-gnu-gcc' failed with exit status 1
root@ubuntu:~/yara-python# python setup.py build
running build
running build_ext
building 'yara' extension
x86_64-linux-gnu-gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -DHAVE_MEMMEM=1 -DHASH_MODULE=1 -Iyara/libyara/include -Iyara/libyara/ -I. -I/usr/include/python2.7 -c yara-python.c -o build/temp.linux-x86_64-2.7/yara-python.o
yara-python.c: In function ‘yara_callback’:
yara-python.c:727:60: error: ‘YR_MATCH’ has no member named ‘data_length’
object = PyBytes_FromStringAndSize((char*) m->data, m->data_length);
^
yara-python.c: In function ‘handle_error’:
yara-python.c:914:10: error: ‘ERROR_INSUFFICIENT_MEMORY’ undeclared (first use in this function)
case ERROR_INSUFFICIENT_MEMORY:
^
yara-python.c:914:10: note: each undeclared identifier is reported only once for each function it appears in
yara-python.c:940:10: error: ‘ERROR_INVALID_EXTERNAL_VARIABLE_TYPE’ undeclared (first use in this function)
case ERROR_INVALID_EXTERNAL_VARIABLE_TYPE:
^
yara-python.c: In function ‘yara_compile’:
yara-python.c:1958:31: error: ‘ERROR_INSUFFICIENT_MEMORY’ undeclared (first use in this function)
result = handle_error(ERROR_INSUFFICIENT_MEMORY, NULL);
^
error: command 'x86_64-linux-gnu-gcc' failed with exit status 1

It is occurred from return null(no rules) of pyobject_new.
I can not know reason why not allocate memory by pyobject_new in master(eaba8c1) source code.

This apples to building yara-python-3.5.0 on Ubuntu Trusty 14.04.

sudo apt-get install libmagic-dev

(In a virtualenv)
python setup.py build --enable-magic
python setup.py install

Then in a Python terminal, import yara returns the following error:

>>> import yara
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: /home/vagrant/yara/local/lib/python2.7/site-packages/yara_python-3.5.0-py2.7-linux-x86_64.egg/yara.so: undefined symbol: magic_load

Hello,
the dynamic linking of the yara python module is not allowed if there is also CUCKOO and MAGIC modules enabled. Is this really necessary?

The definitions -DMAGIC_MODULE=1 -DCUCKOO_MODULE=1 doesn't seem to be used for anything within yara-python.c and when linking dynamically to libyara still all the testcases from tests.py are working.

Am I missing something or can we get rid of this check?

Thank you
Michal Ambroz

When matching a lot of files against a lot of files with yara python , the callback function gets called every time , match or no match , which creates a lot of overhead if you're only interested in the matches.
Would it be possible to provide an option where you can specify whether you want your callback called for matches, non-matches or both? I think performance could increase a lot ...

After installing the latest yara-python on Ubuntu 12.04.4 LTS, there is an error when import yara. I have compiled and installed the latest yara before installing yara-python.

import yara
Traceback (most recent call last):
File "", line 1, in
ImportError: /usr/local/lib/python2.7/dist-packages/yara_python-3.4.1-py2.7.egg/yara.so: undefined symbol: yr_finalize

Seifreed@machine:~$ sudo pip install yara

The directory '/Users/Seifreed/Library/Caches/pip/http' or its parent directory is not owned by the current user and the cache has been disabled. Please check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
The directory '/Users/Seifreed/Library/Caches/pip' or its parent directory is not owned by the current user and caching wheels has been disabled. check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
Collecting yara
Downloading yara-1.7.7.tar.gz (387kB)
100% |████████████████████████████████| 389kB 1.2MB/s
Installing collected packages: yara
Running setup.py install for yara ... error
Complete output from command /usr/bin/python -u -c "import setuptools, tokenize;file='/private/tmp/pip-build-tmIEax/yara/setup.py';exec(compile(getattr(tokenize, 'open', open)(file).read().replace('\r\n', '\n'), file, 'exec'))" install --record /tmp/pip-GdhYyw-record/install-record.txt --single-version-externally-managed --compile:
running install
running build
running build_py
creating build
creating build/lib
creating build/lib/yara
copying yara/init.py -> build/lib/yara
copying yara/cli.py -> build/lib/yara
copying yara/libyara_wrapper.py -> build/lib/yara
copying yara/preprocessor.py -> build/lib/yara
copying yara/rules.py -> build/lib/yara
copying yara/scan.py -> build/lib/yara
copying yara/version.py -> build/lib/yara
creating build/lib/yara/rules
creating build/lib/yara/rules/example
copying yara/rules/example/foobar.yar -> build/lib/yara/rules/example
copying yara/rules/example/imports.yar -> build/lib/yara/rules/example
copying yara/rules/example/pe.yar -> build/lib/yara/rules/example
running install_lib
running install_data
copying ./libs/darwin/x86_64/libyara.so -> /System/Library/Frameworks/Python.framework/Versions/2.7/lib
error: [Errno 1] Operation not permitted: '/System/Library/Frameworks/Python.framework/Versions/2.7/lib/libyara.so'

----------------------------------------

Command "/usr/bin/python -u -c "import setuptools, tokenize;file='/private/tmp/pip-build-tmIEax/yara/setup.py';exec(compile(getattr(tokenize, 'open', open)(file).read().replace('\r\n', '\n'), file, 'exec'))" install --record /tmp/pip-GdhYyw-record/install-record.txt --single-version-externally-managed --compile" failed with error code 1 in /private/tmp/pip-build-tmIEax/yara/

I've seen you've edited the typo in ERROR_INSUFFICIENT_MEMORY, but the build is still not working properly.

$ python setup.py build --enable-cuckoo
running build
running build_ext
building 'yara' extension
creating build
creating build/temp.linux-x86_64-2.7
creating build/temp.linux-x86_64-2.7/yara
creating build/temp.linux-x86_64-2.7/yara/libyara
creating build/temp.linux-x86_64-2.7/yara/libyara/modules
x86_64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fno-strict-aliasing -Wdate-time -D_FORTIFY_SOURCE=2 -g -fstack-protector-strong -Wformat -Werror=format-security -fPIC -DHAVE_MEMMEM=1 -DHASH_MODULE=1 -DCUCKOO_MODULE=1 -Iyara/libyara/include -Iyara/libyara/ -I. -I/usr/include/python2.7 -c yara-python.c -o build/temp.linux-x86_64-2.7/yara-python.o
yara-python.c: In function ‘handle_error’:
yara-python.c:914:10: error: ‘ERROR_INSUFFICIENT_MEMORY’ undeclared (first use in this function)
     case ERROR_INSUFFICIENT_MEMORY:
          ^
yara-python.c:914:10: note: each undeclared identifier is reported only once for each function it appears in
yara-python.c: In function ‘yara_compile’:
yara-python.c:1958:31: error: ‘ERROR_INSUFFICIENT_MEMORY’ undeclared (first use in this function)
         result = handle_error(ERROR_INSUFFICIENT_MEMORY, NULL);
                               ^
error: command 'x86_64-linux-gnu-gcc' failed with exit status 1

Although it works after:
$ grep -rl 'ERROR_INSUFICIENT_MEMORY' ./ | xargs sed -i 's/ERROR_INSUFICIENT_MEMORY/ERROR_INSUFFICIENT_MEMORY/g'

I think bumping subrepo (where it seems to be already fixed) will work.

The test-case testEntrypoint is failing on ppc64 architecture. Possibly some endianness issue with yara on ppc64 architecture. Tested with yara 3.5.0.

https://kojipkgs.fedoraproject.org//work/tasks/591/15270591/build.log

+ PYTHONPATH=/builddir/build/BUILDROOT/python-yara-3.5.0-5.el7.ppc64//usr/lib64/python2.7/site-packages/
+ /usr/bin/nosetests-2.7 -v '--exclude=^testModuleData$'
testAnonymousStrings (tests.TestYara) ... ok
testArithmeticOperators (tests.TestYara) ... ok
testAt (tests.TestYara) ... ok
testBitwiseOperators (tests.TestYara) ... ok
testBooleanOperators (tests.TestYara) ... ok
testCallback (tests.TestYara) ... ok
testComments (tests.TestYara) ... ok
testCompare (tests.TestYara) ... ok
testComparisonOperators (tests.TestYara) ... ok
testCompileFile (tests.TestYara) ... ok
testCompileFiles (tests.TestYara) ... ok
testCount (tests.TestYara) ... ok
testEntrypoint (tests.TestYara) ... FAIL
testExternals (tests.TestYara) ... ok
testFilesize (tests.TestYara) ... ok
testFor (tests.TestYara) ... ok
testHexStrings (tests.TestYara) ... ok
testIn (tests.TestYara) ... FAIL
testIncludeFiles (tests.TestYara) ... ok
testIntegerFunctions (tests.TestYara) ... FAIL
testLength (tests.TestYara) ... ok
testModules (tests.TestYara) ... ok
testOf (tests.TestYara) ... ok
testOffset (tests.TestYara) ... ok
testRE (tests.TestYara) ... ok
testStringIO (tests.TestYara) ... ok
testStrings (tests.TestYara) ... ok
testSyntax (tests.TestYara) ... ok
testWildcardStrings (tests.TestYara) ... ok
======================================================================
FAIL: testEntrypoint (tests.TestYara)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/builddir/build/BUILD/yara-python-9fd9fd290872e36360e5e3839c49e21a908bf128/tests.py", line 681, in testEntrypoint
    ], PE32_FILE)
  File "/builddir/build/BUILD/yara-python-9fd9fd290872e36360e5e3839c49e21a908bf128/tests.py", line 274, in assertTrueRules
    self.assertTrue(r.match(data=data))
AssertionError: [] is not true
======================================================================
FAIL: testIn (tests.TestYara)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/builddir/build/BUILD/yara-python-9fd9fd290872e36360e5e3839c49e21a908bf128/tests.py", line 580, in testIn
    ], PE32_FILE)
  File "/builddir/build/BUILD/yara-python-9fd9fd290872e36360e5e3839c49e21a908bf128/tests.py", line 274, in assertTrueRules
    self.assertTrue(r.match(data=data))
AssertionError: [] is not true
======================================================================
FAIL: testIntegerFunctions (tests.TestYara)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/builddir/build/BUILD/yara-python-9fd9fd290872e36360e5e3839c49e21a908bf128/tests.py", line 898, in testIntegerFunctions
    ], b'\xAA\xBB\xCC\xDD')
  File "/builddir/build/BUILD/yara-python-9fd9fd290872e36360e5e3839c49e21a908bf128/tests.py", line 274, in assertTrueRules
    self.assertTrue(r.match(data=data))
AssertionError: [] is not true
----------------------------------------------------------------------
Ran 29 tests in 0.162s
FAILED (failures=3)

import yara
rules = yara.compile(sources={
    'namespace1':'rule dummy { condition: true }',
    'namespace2':'rule dummy { condition: true }',
    'namespace3':'rule dummy2 { condition: true }'}
)
rules.match(data='hello world')  # Returns [dummy, dummy2, dummy]

The whole point of the namespaces is to distinguish between the different rules when they may have the same name. Please include the namespaces in the match result, e.g.

[namespace1:dummy, namespace2:dummy, namespace3:dummy2]

or similar. As it stands, I may have to implement a callback to keep track of which namespaces and rules were matched.

Environment

I'm running yara-python v3.5.0, installed via pip on Mac OS X.

Copied the install example command from README:

python setup.py install --dynamic-linking                                                                                    1 ↵
usage: setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
   or: setup.py --help [cmd1 cmd2 ...]
   or: setup.py --help-commands
   or: setup.py cmd --help

error: option --dynamic-linking not recognized

Here's the log while executing "pip install yara-python":

C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\BIN\x86_amd64\link.exe /nologo /INCREMENTAL:NO /LTCG /DLL /MANIFEST:EMBED,ID=2 /MANIFESTUAC:NO /LIBPATH:yara/windows/lib /LIBPATH:C:\Python3\libs /LIBPATH:C:\Python3\PCbuild\amd64 "/LIBPATH:C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\LIB\amd64" "/LIBPATH:C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\ATLMFC\LIB\amd64" "/LIBPATH:C:\Program Files (x86)\Windows Kits\10\lib\10.0.15063.0\ucrt\x64" "/LIBPATH:C:\Program Files (x86)\Windows Kits\NETFXSDK\4.6.1\lib\um\x64" "/LIBPATH:C:\Program Files (x86)\Windows Kits\10\lib\10.0.15063.0\um\x64" advapi32.lib user32.lib libeay64.lib /EXPORT:PyInit_yara build\temp.win-amd64-3.6\Release\yara-python.obj build\temp.win-amd64-3.6\Release\yara\libyara\ahocorasick.obj build\temp.win-amd64-3.6\Release\yara\libyara\arena.obj build\temp.win-amd64-3.6\Release\yara\libyara\atoms.obj build\temp.win-amd64-3.6\Release\yara\libyara\compiler.obj build\temp.win-amd64-3.6\Release\yara\libyara\exec.obj build\temp.win-amd64-3.6\Release\yara\libyara\exefiles.obj build\temp.win-amd64-3.6\Release\yara\libyara\filemap.obj build\temp.win-amd64-3.6\Release\yara\libyara\grammar.obj build\temp.win-amd64-3.6\Release\yara\libyara\hash.obj build\temp.win-amd64-3.6\Release\yara\libyara\hex_grammar.obj build\temp.win-amd64-3.6\Release\yara\libyara\hex_lexer.obj build\temp.win-amd64-3.6\Release\yara\libyara\lexer.obj build\temp.win-amd64-3.6\Release\yara\libyara\libyara.obj build\temp.win-amd64-3.6\Release\yara\libyara\mem.obj build\temp.win-amd64-3.6\Release\yara\libyara\modules.obj build\temp.win-amd64-3.6\Release\yara\libyara\object.obj build\temp.win-amd64-3.6\Release\yara\libyara\parser.obj build\temp.win-amd64-3.6\Release\yara\libyara\proc.obj build\temp.win-amd64-3.6\Release\yara\libyara\re.obj build\temp.win-amd64-3.6\Release\yara\libyara\re_grammar.obj build\temp.win-amd64-3.6\Release\yara\libyara\re_lexer.obj build\temp.win-amd64-3.6\Release\yara\libyara\rules.obj build\temp.win-amd64-3.6\Release\yara\libyara\scan.obj build\temp.win-amd64-3.6\Release\yara\libyara\sizedstr.obj build\temp.win-amd64-3.6\Release\yara\libyara\stream.obj build\temp.win-amd64-3.6\Release\yara\libyara\strutils.obj build\temp.win-amd64-3.6\Release\yara\libyara\threading.obj build\temp.win-amd64-3.6\Release\yara\libyara\modules\demo.obj build\temp.win-amd64-3.6\Release\yara\libyara\modules\elf.obj build\temp.win-amd64-3.6\Release\yara\libyara\modules\hash.obj build\temp.win-amd64-3.6\Release\yara\libyara\modules\math.obj build\temp.win-amd64-3.6\Release\yara\libyara\modules\pe.obj build\temp.win-amd64-3.6\Release\yara\libyara\modules\tests.obj /OUT:build\lib.win-amd64-3.6\yara.cp36-win_amd64.pyd /IMPLIB:build\temp.win-amd64-3.6\Release\yara.cp36-win_amd64.lib

LINK : fatal error LNK1181: cannot open input file 'libeay64.lib'
error: command 'C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\BIN\x86_amd64\link.exe' failed with exit status 1181

I had openssl installed and specified it in the PATH. The error code 1181 seems like there exist spaces in the path of the linker. Is there a way to fix this problem?

If you compile a rule set in yara-python with defined external variables, then send it different types when matching (ie. you give it an int instead of a string) yara can segfault. If you give it an int instead of a string, it will pass that int to strlen as a pointer to the string to read:

#!/usr/bin/python                                          

import yara                                                

external_vars = {                                          
    "test1": "",                                           
    "test2": 0                                             
}                                                          

buggy_exts = {                                             
    "test1": 0xDEADBEEF,                                   
    "test2": 1                                             
}                                                          

rules = yara.compile("test.rules", externals=external_vars)
m = rules.match("test.rules", externals=buggy_exts)

Will result in something like this(from gdb):

(gdb) run test.py
Starting program: /usr/bin/python test.py
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".

Program received signal SIGSEGV, Segmentation fault.
strlen () at ../sysdeps/x86_64/strlen.S:106
106 ../sysdeps/x86_64/strlen.S: No such file or directory.
(gdb) i r
rax            0xdeadbeef   3735928559
rbx            0xdeadbeef   3735928559
...

In most (all?) cases you should be able to sanitize this input before it gets sent to yara, but that should not (IMO) be necessary - an error message or warning would be fine, but segfaulting is a bit bad.

Edit: This is with latest commit in git. 3.4 release also has this behaviour.
Edit 2: And the contents of test.rules doesn't matter.

Hi,

I'm seeing this error more often in the issue list of yara-python, however, it would appear those are based on older versions. See, e.g., the following Travis CI report, https://travis-ci.org/jbremer/cuckoo/builds/195577079. On line 2475 you will see a simple test that imports yara - and note that yara-python==3.5.0 was installed through pip, while still having the yr_finalize import error. I'd very much like to ship Yara with Cuckoo by default, but this regression test failure is preventing me from doing so. Any tips?

Jurriaan

Darwin X 15.5.0 Darwin Kernel Version 15.5.0: Tue Apr 19 18:36:36 PDT 2016; root:xnu-3248.50.21~8/RELEASE_X86_64 x86_64

setup.py build

yara/libyara/modules/hash.c:17:10: fatal error: 'openssl/md5.h' file not found
#include <openssl/md5.h>
         ^
1 error generated.

I don't know off-hand why this is broken, but when yara is built as a wheel, the library gets placed at an incorrect path, which results in a broken installation.

I suspect that this ansible issue is describing the same problem, so the notes there about absolute vs. relative paths may be relevant.

With wheel (broken):

$ virtualenv with-wheel && with-wheel/bin/pip install --upgrade pip && with-wheel/bin/pip install wheel && with-wheel/bin/pip install yara && find with-wheel -name libyara.so
Running virtualenv with interpreter /usr/bin/python2
New python executable in with-wheel/bin/python2
Also creating executable in with-wheel/bin/python
Installing setuptools, pip...done.
Downloading/unpacking pip from https://pypi.python.org/packages/9c/32/004ce0852e0a127f07f358b715015763273799bd798956fa930814b60f39/pip-8.1.2-py2.py3-none-any.whl#md5=0570520434c5b600d89ec95393b2650b
  Downloading pip-8.1.2-py2.py3-none-any.whl (1.2MB): 1.2MB downloaded
Installing collected packages: pip
  Found existing installation: pip 1.5.6
    Uninstalling pip:
      Successfully uninstalled pip
Successfully installed pip
Cleaning up...
Collecting wheel
  Downloading wheel-0.29.0-py2.py3-none-any.whl (66kB)
    100% |████████████████████████████████| 71kB 859kB/s 
Installing collected packages: wheel
Successfully installed wheel-0.29.0
Collecting yara
  Using cached yara-1.7.7.tar.gz
Building wheels for collected packages: yara
  Running setup.py bdist_wheel for yara ... done
  Stored in directory: /home/myuser/.cache/pip/wheels/75/07/12/b7042cb4c8ec9b99f5e6a320fbabd92bbdf349dbe8d54f816a
Successfully built yara
Installing collected packages: yara
Successfully installed yara-1.7.7
with-wheel/lib/python2.7/site-packages/tmp/test/with-wheel/lib/libyara.so

The normal installation process works fine:

virtualenv without-wheel && without-wheel/bin/pip install --upgrade pip && without-wheel/bin/pip install yara && find without-wheel -name libyara.so
Running virtualenv with interpreter /usr/bin/python2
New python executable in without-wheel/bin/python2
Also creating executable in without-wheel/bin/python
Installing setuptools, pip...done.
Downloading/unpacking pip from https://pypi.python.org/packages/9c/32/004ce0852e0a127f07f358b715015763273799bd798956fa930814b60f39/pip-8.1.2-py2.py3-none-any.whl#md5=0570520434c5b600d89ec95393b2650b
  Downloading pip-8.1.2-py2.py3-none-any.whl (1.2MB): 1.2MB downloaded
Installing collected packages: pip
  Found existing installation: pip 1.5.6
    Uninstalling pip:
      Successfully uninstalled pip
Successfully installed pip
Cleaning up...
Collecting yara
  Using cached yara-1.7.7.tar.gz
Installing collected packages: yara
  Running setup.py install for yara ... done
Successfully installed yara-1.7.7
without-wheel/lib/libyara.so