log2timeline / l2tdevtools Goto Github PK
View Code? Open in Web Editor NEWDevelopment tools
License: Apache License 2.0
Development tools
License: Apache License 2.0
Various development tools for the log2timeline projects and dependencies. For more information see: * Project documentation: https://github.com/log2timeline/devtools/wiki/Home
The MSIs get installed in C:\Miniconda-x64\Lib\site-packages instead of C:\Python27\Lib\site-packages
Have a look at Microsoft Visual C++ Compiler for Python 2.7 to replace full fledged VS 2008 express
http://www.microsoft.com/en-us/download/details.aspx?id=44266
It looks like download urls have changed, and include some unexpected elements:
https://pypi.python.org/packages/d3/8b/a82129243571dba8f37e8b488265984d86ddd881faf82745ede600627871/pylint-1.5.5.tar.gz#md5=208a6365f81ba072293539ae51614d09
The page_content of http://www.zlib.net currently has no line matching
u'<A HREF="http://zlib.net/{0:s}-([0-9].[0-9].[0-9]).tar.gz"
the link is relative:
<A HREF="zlib-1.2.11.tar.gz"
Note: Issues are disabled in the l2tbinaries repo. So I am adding it here.
The msi installer files created in the l2tbinaries repo does not allow you to change the install location from command line. The TARGETDIRX property allows you to change the path, but there is no way to set it to use that path instead of the path found in the Window's registry.
I am trying to create a script that automatically installs all the windows binary files found in the l2tbinaries repository into a virtualenv.
Generate the "ontology data types" based on the source
I would rather have just space delimited package names, so I can copy paste the output for apt-get install
Have:
$ ./tools/build.py --projects=libscca dpkg
[INFO] Processing: libscca
[WARNING] Missing build dependencies: quilt, python3-setuptools, python3-all, byacc, python-all, python-all-dev, python3-all-dev.
Failed building: libscca
Failed buiding:
libscca
Would prefer:
$ ./tools/build.py --projects=libscca dpkg
[INFO] Processing: libscca
[WARNING] Missing build dependencies: quilt python3-setuptools python3-all byacc python-all python-all-dev python3-all-dev
Failed building: libscca
Failed buiding:
libscca
in review.py :
self._git_repo_url = b'https://github.com/log2timeline/{0:s}.git'.format(
self._project_name)
If the repo is cloned using:
git clone https://github.com/log2timeline/plaso instead of git clone https://github.com/log2timeline/plaso.git
then the origin url will mismatch in
origin_git_repo_url = self.GetRemoteOrigin()
return origin_git_repo_url == self._git_repo_url
because of the trailing ".git" in self._git_repo_url.
To do:
To do:
Close dependency script issues reported under plaso
Improvements to msi build target:
./setup.py --user-access-control=auto bdist_wininst
, also see: https://docs.python.org/2/distutils/builtdist.htmlNotes on pywin32 support:
error: command 'mc.exe' failed: No such file or directory
cl : Command line warning D9024 : unrecognized source file type '3.0',
fatal error C1083: Cannot open source file: '3.0': No such file or directory
Some ideas for enhancements
TODO:
improve stats.py to:
Reported in: log2timeline/plaso#705
Running review.py test
and review.py lint
requires connection to github. Would be nice run these tests while not connected online and with no upstream remote available. (ie. run tests and linting based off of a diff of the master to feature branch)
look into review nobrowser not working on update
create a script to check if gift and l2tbinaries are up to date
python-
Currenly there are issues with the following packages in dev:
Error: package pyparsing-2.1.10-1.fc25.noarch requires python-pyparsing = 2.1.10-1.fc25, but none of the providers can be installed
Error: package python-ipython-console-3.2.1-8.fc25.noarch requires python-zmq, but none of the providers can be installed
python3-requests
dtfabric
Add script to maintain l2tbinaries:
Step1: https://codereview.appspot.com/240130043
requests Makefile triggers:
pip: Command not found
force setup.py as build system
Fix jinja2 no package content - requires explicit install definition in rules
.PHONY: override_dh_auto_install
override_dh_auto_install:
mkdir -p debian/python-jinja2/usr/lib/python2.7/site-packages
PYTHONPATH=debian/python-jinja2/usr/lib/python2.7/site-packages python ./setup.py install --prefix debian/python-jinja2/usr
mv debian/python-jinja2/usr/lib/python2.7/site-packages debian/python-jinja2/usr/lib/python2.7/dist-packages
rm -f debian/python-jinja2/usr/lib/python2.7/dist-packages/easy-install.*
rm -f debian/python-jinja2/usr/lib/python2.7/dist-packages/site.*
mv debian/python-jinja2/usr/lib/python2.7/dist-packages/Jinja2*.egg/jinja2 debian/python-jinja2/usr/lib/python2.7/dist-packages
To do
Currently the google code package names are ignored on on Mac OS X, this causes multiple versions of the same package to be installed.
Add srpm and copr build targets to support
https://copr.fedorainfracloud.org/groups/g/gift/coprs/
Add a packaged version of requests
Update documentation:
Improvements to dpkg build target:
BROWSER
environment variableCreate aborted - unable to create codereview issue.
Remaining items moved to: https://github.com/log2timeline/l2treviewtools/issues/22
Build target: rpm
Build directory: MyBuild/
Configuration file: MyPath/projects.ini
Reported by @Onager
Setting up python-astroid (1.4.4-2ppa1~wily) ...
File "/usr/lib/python2.7/dist-packages/astroid/tests/testdata/python3/data/module2.py", line 100
print('bonjour', file=stream)
^
SyntaxError: invalid syntax
dpkg: error processing package python-astroid (--configure):
subprocess installed post-installation script returned error exit status 101
dpkg: dependency problems prevent configuration of python-pylint:
python-pylint depends on python-astroid; however:
Package python-astroid is not configured yet.
Work-around
.PHONY: override_dh_python2
override_dh_python2:
dh_python2 -V 2.7 setup.py
Change the merge_submit to double check name and email address and determine why it does not find the name in certain situations.
dpkg_dependencies: python-dateutil,python-six,python-tz
/usr/lib/python2.7/site-packages/pkg_resources/__init__.py:203: RuntimeWarning: You have iterated
over the result of pkg_resources.parse_version. This is a legacy behavior which is inconsistent with
the new version class introduced in setuptools 8.0. In most cases, conversion to a tuple is unnecessary.
For comparison of versions, sort the Version instances directly. If you have another use case requiring
the tuple, please file a bug with the setuptools project describing that need.
python setup.py egg_info
to override_dh_auto_install
__init__.py
to contain hardcoded versionNoticed with issue https://codereview.appspot.com/281870043/
Traceback (most recent call last):
File "../l2tdevtools/tools/update.py", line 1000, in <module>
if not Main():
File "../l2tdevtools/tools/update.py", line 996, in Main
return dependency_updater.UpdatePackages(package_names)
File "../l2tdevtools/tools/update.py", line 841, in UpdatePackages
package_names)
File "../l2tdevtools/tools/update.py", line 399, in _GetPackageFilenamesAndVersions
_ = self._download_helper.DownloadFile(package_url)
File "/Users/travis/build/log2timeline/l2tdevtools/l2tdevtools/download_helper.py", line 52, in DownloadFile
url_object = urllib2.urlopen(download_url)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 154, in urlopen
return opener.open(url, data, timeout)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 437, in open
response = meth(req, response)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 550, in http_response
'http', request, response, code, msg, hdrs)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 469, in error
result = self._call_chain(*args)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 409, in _call_chain
result = func(*args)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 656, in http_error_302
return self.parent.open(new, timeout=req.timeout)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 431, in open
response = self._open(req, data)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 449, in _open
'_open', req)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 409, in _call_chain
result = func(*args)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 1240, in https_open
context=self._context)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 1200, in do_open
r = h.getresponse(buffering=True)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/httplib.py", line 1132, in getresponse
response.begin()
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/httplib.py", line 453, in begin
version, status, reason = self._read_status()
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/httplib.py", line 417, in _read_status
raise BadStatusLine(line)
httplib.BadStatusLine: ''
add l2tbinaries tests
It's been moved now: https://github.com/erocarrera/pefile
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.