Coder Social home page Coder Social logo

pyuwsgi_pex's People

Contributors

kwlzn avatar lorencarvalho avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar

pyuwsgi_pex's Issues

System level libraries taking precedence over pex

The pex import paths are appended to sys.path after the system paths. If a library is installed on both the system and in the pex, the system version will be imported which can cause problems if they are different versions.

UWSGI_AS_LIB

hey @kwlzn !

Firstly, I'm a big fan of this project -- but I recently discovered (accidentally) that when building uwsgi you can set an environment variable UWSGI_AS_LIB which the build system checks for. If supplied, it will compile uwsgi as a shared library!

This can be used with python's ctypes to invoke the uwsgi executable from within the python runtime. Here's an example (from this issue on unbit's page unbit/uwsgi#564):

import sys
import ctypes

def application(e, sr):
    sr('200 OK',[('Content-Type','text/html')])
    return ["Hello World"]

def uwsgi_run(uwsgi_args):
    # load the uwsgi library in the global namespace
    uwsgi = ctypes.CDLL('./libuwsgi.so',mode=ctypes.RTLD_GLOBAL)

    uwsgi_args.insert(0, sys.argv[0])
    uwsgi_args.insert(1, '--binary-path')
    uwsgi_args.insert(2, sys.argv[0])

    # build command line args
    argv = (ctypes.c_char_p * (len(uwsgi_args)+1))()
    for pos,arg in enumerate(uwsgi_args):
        argv[pos] = arg
    # inform the uwsgi engine, the passed environ is not safe to overwrite
    envs = (ctypes.c_char_p * 1)()
    # enter into uWSGI !!!
    uwsgi.uwsgi_init(len(uwsgi_args), argv, envs)

if __name__ == "__main__":
    uwsgi_run(['--http-socket', ':9090', '--master', '--processes', '8', '--wsgi-file', sys.argv[0]])

Since this is happening after the python runtime has initialized, I presume it would work with pex!

Lo and behold:

linux ~ ❯❯❯ pex requests -o poop.pex
linux ~ ❯❯❯ ./poop.pex u.py
*** Starting uWSGI 2.0.12 (64bit) on [Sun May 15 04:03:13 2016] ***
compiled with version: 4.4.7 20120313 (Red Hat 4.4.7-3) on 15 May 2016 04:00:01
os: Linux-2.6.32-431.17.1.el6.x86_64 #1 SMP Fri Apr 11 17:27:00 EDT 2014
nodename: lcarvalh-ld
machine: x86_64
clock source: unix
pcre jit disabled
detected number of CPU cores: 8
current working directory: /home/lcarvalh
detected binary path: u.py
your processes number limit is 81920
your memory page size is 4096 bytes
detected max file descriptor number: 280000
lock engine: pthread robust mutexes
thunder lock: disabled (you can enable it with --thunder-lock)
uwsgi socket 0 bound to TCP address :9090 fd 3
Python version: 2.6.6 (r266:84292, Oct 12 2012, 14:23:48)  [GCC 4.4.6 20120305 (Red Hat 4.4.6-4)]
--- Python VM already initialized ---
*** Python threads support is disabled. You can enable it with --enable-threads ***
Python main interpreter initialized at 0x13290a0
your server socket listen backlog is limited to 100 connections
your mercy for graceful operations on workers is 60 seconds
mapped 654912 bytes (639 KB) for 8 cores
*** Operational MODE: preforking ***
WSGI app 0 (mountpoint='') ready in 0 seconds on interpreter 0x13290a0 pid: 3749 (default app)
*** uWSGI is running in multiple interpreter mode ***
spawned uWSGI master process (pid: 3749)
spawned uWSGI worker 1 (pid: 3756, cores: 1)
spawned uWSGI worker 2 (pid: 3757, cores: 1)
spawned uWSGI worker 3 (pid: 3758, cores: 1)
spawned uWSGI worker 4 (pid: 3759, cores: 1)
spawned uWSGI worker 5 (pid: 3760, cores: 1)
spawned uWSGI worker 6 (pid: 3761, cores: 1)
spawned uWSGI worker 7 (pid: 3762, cores: 1)
spawned uWSGI worker 8 (pid: 3763, cores: 1)
[pid: 3756|app: 0|req: 1/1] 127.0.0.1 () {24 vars in 344 bytes} [Sun May 15 04:03:15 2016] GET / => generated 188 bytes in 37 msecs (HTTP/1.1 200) 1 headers in 44 bytes (1 switches on core 0)
linux ~ ❯❯❯ http localhost:9090
HTTP/1.1 200 OK
Content-Type: text/html

requests module found: /home/lcarvalh/.pex/install/requests-2.10.0-py2.py3-none-any.whl.671f167db5ff4b6702ea8ff9465a4059d31d99fd/requests-2.10.0-py2.py3-none-any.whl/requests/__init__.pyc

I thought this was pretty cool and I wanted to take advantage of it, but right now I'm stuck on how to reliably pip install uwsgi with this environment variable exported. Check it out:

linux ~ ❯❯❯ virtualenv foobar
New python executable in foobar/bin/python
Installing setuptools............done.
Installing pip...............done.
linux ~ ❯❯❯ . foobar/bin/activate
(foobar)linux ~ ❯❯❯ export UWSGI_AS_LIB=this_is_just_a_test.so
(foobar)linux ~ ❯❯❯ pip install uwsgi
Downloading/unpacking uwsgi
  Downloading uwsgi-2.0.13.1.tar.gz (784kB): 784kB downloaded
  Running setup.py egg_info for package uwsgi
Installing collected packages: uwsgi
  Running setup.py install for uwsgi
    using profile: buildconf/default.ini
    detected include path: ['/usr/local/include', '/usr/lib/gcc/x86_64-redhat-linux/4.4.7/include', '/usr/include']
    Patching "bin_name" to properly install_scripts dir
    detected CPU cores: 8
...
... 
<build output truncated >
    ################# uWSGI configuration #################

    pcre = True
    kernel = Linux
    malloc = libc
    execinfo = False
    ifaddrs = True
    ssl = True
    zlib = True
    locking = pthread_mutex
    plugin_dir = .
    timer = timerfd
    yaml = embedded
    json = False
    filemonitor = inotify
    routing = True
    debug = False
    ucontext = True
    capabilities = True
    xml = libxml2
    event = epoll

    ############## end of uWSGI configuration #############
    total build time: 5 seconds
    *** uWSGI shared library (./this_is_just_a_test.so) is ready, move it to a library directory ***
Successfully installed uwsgi
Cleaning up...

Clearly it says it compiled a shared library:

uWSGI shared library (./this_is_just_a_test.so) is ready, move it to a library directory

However, it doesn't appear to have 😞

(foobar)linux ~ ❯❯❯ ls|grep this_
(foobar)linux ~ ❯❯❯ find foobar/ -name 'this_is_just_a_test.so'

You may be wondering what's the point? idk, just wanted to share some findings. Thanks for reading 📖

-- loren

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.