kernelsauce / turbo

Turbo is a framework built for LuaJIT 2 to simplify the task of building fast and scalable network applications. It uses a event-driven, non-blocking, no thread design to deliver excellent performance and minimal footprint to high-load applications while also providing excellent support for embedded uses.

Turbo is a framework built for LuaJIT 2 to simplify the task of building fast and scalable network applications. It uses a event-driven, non-blocking, no thread design to deliver excellent performance and minimal footprint to high-load applications while also providing excellent support for embedded uses. The toolkit can be used for HTTP REST API's, traditional dynamic web pages through templating, open connections like WebSockets, or just as high level building blocks for native speed network applications.

First and foremost the framework is aimed at the HTTP(S) protocol. This means web developers and HTTP API developers are the first class citizens. But the framework contains generic nuts and bolts such as; a I/O loop, IO Stream classes, customizeable TCP (with SSL) server classes giving it value for everyone doing any kind of high performance network application. It will also speak directly to your exising C libraries, and happily also create native C struct's for the ultimate memory and CPU performance.

Keep in mind that running this with LuaJIT provides you with roughly the speed of compiled C code with only a fraction of the development time. Perfect for small devices running cheap CPU's on battery power as well as your pay per use Amazon cluster.

LuaJIT 2 is REQUIRED, PUC-RIO Lua is unsupported.

API Documentation is available at https://turbo.readthedocs.io, or it can be built from source with make html from /doc in the source tree.

It's main features and design principles are:

• Simple and intuitive API (much like Tornado).
• Low-level operations is possible if the users wishes that.
• Implemented in straight Lua and LuaJIT FFI on Linux, so the user can study and modify inner workings without too much effort. The Windows implementation uses some Lua modules to make compability possible.
• Good documentation
• Event driven, asynchronous and threadless design
• Small footprint
• SSL support (requires OpenSSL or LuaSec module for Windows)

Supported Architectures

x86, x64, ARM, PPC, MIPSEL

Supported Operating Systems

Linux distros (x86, x64), OSX (Intel-based) and Windows x64. Possibly others using LuaSocket, but not tested or supported.

Installation

You can use LuaRocks to install Turbo on Linux.

luarocks install turbo

If installation fails make sure that you have these required pacakages:

apt-get install luajit luarocks git build-essential libssl-dev

For Windows run the included install.bat file. This will install all dependencies: Visual Studio, git, mingw, gnuwin, openssl using Chocolatey. LuaJIT, the LuaRocks package manager and Turbo will be installed at C:\turbo.lua. It will also install LuaSocket and LuaFileSystem with LuaRocks. The Windows environment will be ready to use upon success.

Try: luajit C:\turbo.lua\src\turbo\examples\helloworld.lua

If any of the .dll or. so's are placed at non-default location then use environment variables to point to the correct place:

E.g: SET TURBO_LIBTFFI=C:\turbo.lua\src\turbo\libtffi_wrap.dll and SET TURBO_LIBSSL=C:\Program Files\OpenSSL\libeay32.dll

Applies for Linux based OS and OSX only:

Turbo.lua can also be installed by the included Makefile. Simply download and run make install (requires root priv). It is installed in the default Lua 5.1 and LuaJIT 2.0 module directory.

You can specify your own prefix by using make install PREFIX=<prefix>, and you can specify LuaJIT version with a LUAJIT_VERSION=2.0.0 style parameter.

To compile without support for OpenSSL (and SSL connections) use the make option SSL=none. To compile with axTLS support instead of OpenSSL use the make option SSL=axTLS. If you want to install the framework in a bytecode form use: make bcodeinstall

In essence the toolkit can run from anywere, but is must be able to load the libtffi_wrap.so at run time. To verify a installation you can try running the applications in the examples folder.

Packaging

The included Makefile supports packaging the current tree as a versioned tar.gz file. This file will include only the neccessary bits and pieces for Turbo to run. The files will be built as bytecode (luajit -b -g) with debug info. This reduces size drastically. Suitable for embedded devices with limited storage... It also reduces the startup time.

Use make package.

this results in a turbo.x.x.x.tar.gz file and a package/ directory being created.

Dependencies

All of the modules of Turbo.lua are made with the class implementation that Middleclass provides. https://github.com/kikito/middleclass.

The HTTP parser by Ryan Dahl is used for HTTP parsing. This is built and installed as part of the package.

OpenSSL is required for SSL support. It is possible to run without this feature, and thus not need an SSL library.

License

Copyright 2011 - 2016 John Abrahamsen

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

Many of the modules in the software package are derivatives of the Tornado web server. Tornado is also licensed under Apache 2.0 license. For more details on Tornado please see:

http://www.tornadoweb.org/

turbo's Issues

Implement GZIP in HTTPServer

IOStream Deque buffer replacement.

Add possibility to IOStream:write() and use offsets, so that static content does not have to be copied.

Log module documentation

Escape namespace documentation

StaticFileHandler: Dir traversal possible when using %2e%2e in url instead of ..

curl localhost:8888/static/%2e%2e/%2e%2e/%2e%2e/%2e%2e/etc/hosts #Prints my hostsfile 👎

Websocket server

TCPServer documentation

Implement HTTPServer class.

As title.

httputil namespace documentation.

Unittest for Application class

Async websocket client

Getting started tutorial

No support for Kqueue.

Should be present :).

Unittest for log module

Unittest for escape module

Unittest for util module

Implement GZip of old logs

Proper logging to file.

No support for Windows OS.

Nonsence should have a fallback for Windows development.
Production not going to happen!

Implement Websockets.

As title.

How to create async modules tutorial

Penetration attempt on HTTPServer

Unittest for HTTPServer class

Implement a async HTTP client.

The user must be able to request data from other web services.

turn off log

how to trun off the log output when ab test,

luajit helloworld.lua
ab -n 100000 -c 500 -k 127.0.0.1:8888/

[D 2013/07/07 12:39:58] [ioloop.lua] Uncaught error in handler. /usr/local/share/lua/5.1/turbo/iostream.lua:455: Error when reading from socket 7. Errno: 104. Connection reset by peer
stack traceback:
    /usr/local/share/lua/5.1/turbo/ioloop.lua:120: in function </usr/local/share/lua/5.1/turbo/ioloop.lua:118>
    [C]: in function 'error'
    /usr/local/share/lua/5.1/turbo/iostream.lua:455: in function '_read_from_socket'
    /usr/local/share/lua/5.1/turbo/iostream.lua:476: in function '_read_to_buffer'
    /usr/local/share/lua/5.1/turbo/iostream.lua:397: in function '_handle_read'
    /usr/local/share/lua/5.1/turbo/iostream.lua:313: in function '_handle_events'
    /usr/local/share/lua/5.1/turbo/iostream.lua:601: in function 'ioloop_fd_callback'
    /usr/local/share/lua/5.1/turbo/ioloop.lua:124: in function </usr/local/share/lua/5.1/turbo/ioloop.lua:123>
    [C]: in function 'xpcall'
    /usr/local/share/lua/5.1/turbo/ioloop.lua:133: in function '_run_handler'
    /usr/local/share/lua/5.1/turbo/ioloop.lua:352: in function 'start'
    httpclient.lua:58: in main chunk
    [C]: at 0x00404180

[root@localhost libuv-lua-http-server]# grep model /proc/cpuinfo 
model       : 58
model name  : Intel(R) Core(TM) i5-3470 CPU @ 3.20GHz
model       : 58
model name  : Intel(R) Core(TM) i5-3470 CPU @ 3.20GHz
model       : 58
model name  : Intel(R) Core(TM) i5-3470 CPU @ 3.20GHz
model       : 58
model name  : Intel(R) Core(TM) i5-3470 CPU @ 3.20GHz
[root@localhost libuv-lua-http-server]# grep Total /proc/meminfo 
MemTotal:        7941300 kB
SwapTotal:       8216568 kB
VmallocTotal:   34359738367 kB
HugePages_Total:       0

[root@localhost libuv-lua-http-server]# ab -n 100000 -c 500 -k 127.0.0.1:8888/
This is ApacheBench, Version 2.3 <$Revision: 655654 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking 127.0.0.1 (be patient)
Completed 10000 requests
Completed 20000 requests
Completed 30000 requests
Completed 40000 requests
Completed 50000 requests
Completed 60000 requests
Completed 70000 requests
Completed 80000 requests
Completed 90000 requests
Completed 100000 requests
Finished 100000 requests


Server Software:        Turbo
Server Hostname:        127.0.0.1
Server Port:            8888

Document Path:          /
Document Length:        9 bytes

Concurrency Level:      500
Time taken for tests:   15.975 seconds
Complete requests:      100000
Failed requests:        0
Write errors:           0
Non-2xx responses:      100000
Keep-Alive requests:    100000
Total transferred:      17500000 bytes
HTML transferred:       900000 bytes
Requests per second:    6259.61 [#/sec] (mean)
Time per request:       79.877 [ms] (mean)
Time per request:       0.160 [ms] (mean, across all concurrent requests)
Transfer rate:          1069.76 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.4      0       7
Processing:     1   80  38.2     90     209
Waiting:        1   80  38.2     90     209
Total:          1   80  38.2     90     209

Percentage of the requests served within a certain time (ms)
  50%     90
  66%     99
  75%    111
  80%    114
  90%    119
  95%    125
  98%    141
  99%    148
 100%    209 (longest request)

Request handler methods does not recieve multiple arguments (routes).

When defining a route like "/(%d)/(%d)$" request handler methods only get the first argument.

Test Case:

local turbo = require("turbo")

local TestHandler = class("TestHandler", turbo.web.RequestHandler)
function TestHandler:get(a1, a2)
    print("a1 == ", a1)
    print("a2 == ", a2)
    assert(a1 ~= nil)
    assert(a2 ~= nil)
    self:write("..")
end

local application = turbo.web.Application:new({
    {"/(%d)/(%d)$", TestHandler}
})

application:listen(8888) 
turbo.ioloop.instance():start()

Spec the workflow of log writer.

Better file upload convinence

Unittest for RequestHandler class

Async HTTP client

Making a LuaJIT FFI module for Epoll/Kqueue

Short be easy peasy.

Deque class documentation

Unittest for TCPServer class

Full test run script.

AsyncHTTPClient SSL support.

Unittest for HTTPUtil class

Unittest for IOStream class

HTTPServer documentation

Unittest for deque class.

turbolua.org

Domain is registered. Started working on the official site. Repo at:

https://github.com/kernelsauce/turbo-www

Unittest for epoll_ffi module

Websocket server / client support SSL

Implement Application class.

As title.

Async DNS

getaddrinfo blocks, we need an alternative.

timeout is not available,when site was blocked(china gfw).

     local res = coroutine.yield(
        turbo.async.HTTPClient:new():fetch("http://search.twitter.com/search.json",
        {
            params = {
                q=search,
                result_type="mixed"
            },
            request_timeout = 3,
            connect_timeout = 3,
        }))

    if (res.error) then
        error(turbo.web.HTTPError:new(500, res.error.message))
    end

TCPServer supports SSL

Removed dependency of nixio.

Implement Websocket protocol

Create globals counters.

Create rst documentation.

Create rich and nice docs are a requirement for people to be able to use the package.