Coder Social home page Coder Social logo

facebookarchive / libphenom Goto Github PK

View Code? Open in Web Editor NEW
1.7K 215.0 364.0 1.4 MB

An eventing framework for building high performance and high scalability systems in C.

Home Page: http://facebook.github.io/libphenom

License: Apache License 2.0

PHP 3.59% Shell 0.13% C 94.43% C++ 0.16% R 0.07% Makefile 0.60% M4 1.02%

libphenom's Introduction

NOTE: THIS PROJECT HAS BEEN DEPRECATED AND IS NO LONGER ACTIVELY MAINTAINED

As of 2019-03-08, this project will no longer be maintained and will be archived. Thank you to all the contributors and users of libphenom for their work over the years.

If you are looking for a comparable library (albeit in C++) that has many of the same features and more, you should check out the actively-maintained folly library.

The rest of the README is preserved as-is for historical purposes:

libPhenom

libPhenom is an eventing framework for building high performance and high scalability systems in C

System Requirements

libPhenom is known to compile and pass its test suite on:

  • Linux systems with epoll
  • OS X
  • Illumos and Solaris style systems that have port_create(3C).

libPhenom has been known to compile and pass its test suite on these systems, but they have not been tried in a little while, so may require a little bit of TLC:

  • BSDish systems that have the kqueue(2) facility, including FreeBSD 9.1 and OpenBSD 5.2

libPhenom depends on:

  • pkg-config to find dependencies.
  • Concurrency Kit for its excellent concurrency primitives and key data structures. If you are on OS/X, you can brew install concurrencykit. On other platforms, you will You will need to install CK yourself as it is not widely distributed at this time. We expect to be able to find CK using pkg-config.
  • OpenSSL is required; this should be provided by your OS. You will need to install openssl-devel or libssl-dev or a similarly named package.
  • autoconf, automake and libtool are required to build libPhenom

libPhenom works best if built with GCC version 4.3 or later, but should be able to build with any C99 compiler.

Build Status

Facilities

  • Memory management with counters - record how much of which kinds of memory your application is using.
  • Jobs - decompose your application into portions of work and let the phenom scheduler manage getting them done
  • streaming I/O with buffers
  • Handy data structures (hash tables, lists, queues)
  • Variant data type to enable serialization and deserialization of JSON
  • A printf implementation with registerable object formatting

Goals

  • Balance ease of use with performance
  • Aim to be neutral wrt. your choice of threaded or event-based dispatch and work well with both.
  • Where possible, avoid contention points in our implementation so as to avoid limiting scalability with the number of cores in the system.

How to use these docs

If you're reading these on http://facebook.github.io/libphenom, simply start typing and the search box will suggest topics. You may select topics from the Topics menu or browse the header files via the Headers menu.

Getting it

You can obtain the sources from https://github.com/facebook/libphenom:

$ git clone https://github.com/facebook/libphenom.git

or grab a snapshot of master

Build

$ ./autogen.sh
$ ./configure
$ make
$ make check
$ sudo make install

Quick Start for using the library

You'll want to set up the main loop using something like this:

// Always include phenom/defs.h first to correctly set up the compilation env
#include "phenom/defs.h"
#include "phenom/configuration.h"
#include "phenom/job.h"
#include "phenom/log.h"
#include "phenom/sysutil.h"

int main(int argc, char **argv)
{
  // Must be called prior to calling any other phenom functions
  ph_library_init();
  // Optional config file for tuning internals
  ph_config_load_config_file("/path/to/my/config.json");
  // Enable the non-blocking IO manager
  ph_nbio_init(0);

  // Do stuff here to register client/server stuff.
  // This enables a very simple request/response console
  // that allows you to run diagnostic commands:
  // `echo memory | nc -UC /tmp/phenom-debug-console`
  // (on BSD systems, use `nc -Uc`!)
  // The code behind this is in
  // https://github.com/facebook/libphenom/blob/master/corelib/debug_console.c
  ph_debug_console_start("/tmp/phenom-debug-console");

  // Run
  ph_sched_run();

  return 0;
}

And compile it using something like this: (you need optimization enabled)

$ gcc -O2 main.c `pkg-config libphenom --cflags --libs`

(this will link against libphenom).

A more detailed example can be found at https://github.com/facebook/libphenom/blob/master/examples/echo.c

Want more inspiration? Take a look at the code in the test suite.

Status

We're still hacking and evolving this library, so there may be some rough edges. We're very open to feedback; check out the Contributing section below.

Getting Help

We've started an IRC channel for realtime conversations; join us in #phenom @ freenode

Contributing

If you're thinking of hacking on libPhenom we'd love to hear from you! Feel free to use the Github issue tracker and pull requests to discuss and submit code changes.

We (Facebook) have to ask for a "Contributor License Agreement" from someone who sends in a patch or code that we want to include in the codebase. This is a legal requirement; a similar situation applies to Apache and other ASF projects.

If we ask you to fill out a CLA we'll direct you to our online CLA page where you can complete it easily. We use the same form as the Apache CLA so that friction is minimal.

License

libPhenom is made available under the terms of the Apache License 2.0. See the LICENSE file that accompanies this distribution for the full text of the license.

libphenom's People

Contributors

abelmathew avatar brucespang avatar deepak1556 avatar dhobsd avatar flowly avatar imxset21 avatar jsavolainen avatar kev009 avatar mattus avatar richardjrossiii avatar saghul avatar sbahra avatar tyrellj avatar wez avatar wilsonwen avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

libphenom's Issues

Compilation error in the README example code

I tried compiling the example code in the README with the following steps:

OS X v10.8.4

Directory structure:
/hello
- main.c
- /libphenom

  1. Installed Autotools (autoconf, automake, libtool)
  2. Installed c-ares
  3. git clone https://github.com/facebook/libphenom
  4. $ ./autogen.sh
  5. $ ./configure
  6. $ sudo make
  7. $ sudo make check
  8. $ sudo make install
  9. $ gcc -o main main.c pkg-config libphenom --cflags --libs

And I got this error:
cc1: error in backend: Couldn't allocate output reg for constraint 'q'!

I am new to C programming though, that's why googling this error doesn't make sense to me. It just leads to some llvm bug related pages.

And also, I think step 8 above should be included in the build instruction in the README. Can I submit a pull request for this?

Proper way to close a connection from the server side?

Hey y'all!

I've been working through trying to build a grossly oversimplified static web server using libphenom. I basically took a look at the echo server example, and started modifying. My code is here: https://gist.github.com/anonymous/5f51f56b87a2f6844d03

I'm hitting a problem where, when I'm done copying the file to the socket stream, I want to immediately disconnect the connection. Right now, my code for doing that is borrowed from roughly what the echo server does when it hits an actual error:

ph_sock_enable(sock, false);
ph_sock_shutdown(sock, PH_SOCK_SHUT_RDWR);
ph_mem_free(mt_parser, state->parser);
ph_mem_free(mt_state, state);
ph_sock_free(sock);

What I'm seeing is that, after a few requests, or if I do a big burst of them all at once, there's a failed assertion about a pending free on the socket:

1407245254.540 panic: sched/1 assertion job->epoch_entry.function == NULL failed: job is pending free at corelib/nbio/common.c:486
1407245254.540 panic: sched/1 Fatal error detected at:
1407245254.540 panic: sched/1 /usr/lib64/libphenom.so.0(ph_log_stacktrace+0x29) [0x7ffff7db45d9]
1407245254.544 debug: phenom:sched/0 fd=17 setting mask=1 timeout={60,0}
1407245254.540 panic: sched/1 /usr/lib64/libphenom.so.0(ph_panic+0xbe) [0x7ffff7db46ee]
1407245254.540 panic: sched/1 /usr/lib64/libphenom.so.0(+0x1482f) [0x7ffff7db682f]
1407245254.540 panic: sched/1 /usr/lib64/libphenom.so.0(ph_job_set_nbio_timeout_in+0x73) [0x7ffff7db6993]
1407245254.540 panic: sched/1 /usr/lib64/libphenom.so.0(ph_nbio_emitter_run+0xe0) [0x7ffff7db7730]
1407245254.540 panic: sched/1 /usr/lib64/libphenom.so.0(+0x14c51) [0x7ffff7db6c51]
1407245254.540 panic: sched/1 /usr/lib64/libphenom.so.0(+0x1a070) [0x7ffff7dbc070]
1407245254.540 panic: sched/1 /lib64/libpthread.so.0() [0x334da079d1]
1407245254.540 panic: sched/1 /lib64/libc.so.6(clone+0x6d) [0x334d2e8b6d]

The code to disconnect the socket is queued up in a separate job call to try and let the NBIO stack do it's thing, but I feel like I'm missing something here.

Is there something I'm missing to have to shut down cleanly without this race condition? Is my approach entirely wrong?

make check error in cygwin.

$ make
gcc -DHAVE_CONFIG_H -Ithirdparty/ck/include -Iinclude -DPHENOM_IMPL=1 -Werror -m32 -march=i686 -msse -msse2 -D_LARGEF
ILE_SOURCE -D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64 -I/usr/local/include -D__x86__=1 -std=gnu99 -pedantic -
Wall -Wextra -Wno-declaration-after-statement -W -Wundef -Wendif-labels -Wshadow -Wpointer-arith -Wcast-align -W
write-strings -Wstrict-prototypes -Wmissing-prototypes -Wno-nested-externs -Winline -Wdisabled-optimization -fst
rict-aliasing -pipe -Wno-parentheses -O2 -g -gdwarf-2 -MT corelib/libphenom_a-init.o -MD -MP -MF corelib/.deps/libphenom
_a-init.Tpo -c -o corelib/libphenom_a-init.o test -f 'corelib/init.c' || echo './'corelib/init.c
In file included from include/phenom/socket.h:34,
from include/phenom/sysutil.h:21,
from corelib/init.c:17:
include/phenom/openssl.h:25: warning: ignoring #pragma GCC diagnostic
In file included from include/phenom/socket.h:34,
from include/phenom/sysutil.h:21,
from corelib/init.c:17:
include/phenom/openssl.h:32: warning: ignoring #pragma GCC diagnostic
make: *** [corelib/libphenom_a-init.o] Error 1

profile and optimize find_record

Throwing this one out there for folks that want to dive in.

In the bufq API we have ph_bufq_consume_record which searches the bufq for a matching record delimiter.

For many internet protocols this delimiter is CRLF. For many other applications, we're likely to be looking for LF.

There are a couple of optimizations that could be investigated.

  • Can we accelerate the memmem call using the sse3_memchr function from here? http://repnop.org/carte/snippets/simd/. The performance.data file indicates that this performs consistently better than the darwin libc. To adopt this, we'd need to detect sse3 either at runtime or compile time
  • are there specializations of sse3_memchr that can be made for detecting CRLF?
  • are there specializations of string matching algorithms with a constant, known needle that we could use?
  • is there a more efficient way to match needles across the "straddle" point in cases where the delimiter straddles discontiguous memory regions?
  • for long records, we make repeated calls and search across the same memory regions repeatedly. We can surely cache the last searched offset and improve efficiency. We'd need to make sure that we invalidate this offset in the appropriate circumstances (mostly when we've consumed past that point)

compile error with the latest libphenom and concurrencyKit

I got the following compile error when doing ./autogen.sh; ./configure; make.
I did a git clone on this repo and the github repo on concurrencyKit.

Any ideas? Thanks.

libtool: compile:  gcc -DHAVE_CONFIG_H -Ithirdparty/ck/include -Iinclude -I. -DPHENOM_IMPL=1 -Werror -m64 -fno-omit-frame-pointer -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64 -D__x86_64__ -I/usr/local/include -std=gnu1x -pedantic -Wall -Wextra -Wno-declaration-after-statement -W -Wundef -Wendif-labels -Wshadow -Wpointer-arith -Wcast-align -Wwrite-strings -Wstrict-prototypes -Wmissing-prototypes -Wno-nested-externs -Winline -Wdisabled-optimization -fstrict-aliasing -pipe -Wno-parentheses -O2 -g -gdwarf-2 -MT corelib/libphenom_la-counter.lo -MD -MP -MF corelib/.deps/libphenom_la-counter.Tpo -c corelib/counter.c  -fPIC -DPIC -o corelib/.libs/libphenom_la-counter.o
In file included from include/phenom/counter.h:81:0,
                 from corelib/counter.c:17:
include/phenom/defs.h:368:41: error: static assertion failed: "counter_iterator_definition_bad"
 #  define ph_static_assert(expr, msg)   _Static_assert(expr, #msg)
                                         ^
corelib/counter.c:170:1: note: in expansion of macro 'ph_static_assert'
 ph_static_assert(sizeof(struct ph_counter_scope_iterator)

RFC: Remove libcares DNS support in favor of getdnsapi

I want to remove libcares support because I'm not especially happy with either its API or how it is exposed up through libphenom.

The getdnsapi project proposes a promising modern API that is easier to use and supports features like DNSSEC.

This issue is tracking booting out libcares support and instead making it very easy to enable support for getdnsapi implementations. There is currently only one implementation, built on top of libunbound.

libunbound uses libevent under the covers, it feels a bit weird to have two eventing libraries running but it doesn't harm anything from a practical perspective.

I'm tracking this as a low-pri issue as it isn't a burning problem. I am sensitive to breaking existing integrations, so putting this out for feedback before I just go ahead and remove support for it.

Use custom memory allocator?

Hi, @wez.

First of all, I just want to say thank you for this amazing library. I've been using libPhenom for a while now (almost close to a year) and we've building software with it at my startup for several months now. So yeah, thank you very much for your awesome work.

Right now, we have one little issue. We use jemalloc for almost all of our C code when it comes to memory management. Since we've been using libPhenom, we've avoided the memory utilities in order to use jemalloc. However, we really like the memory utilities and we feel we're missing out. I'm wondering if you'd consider adding a facility to the memory utilities to allow hooking a custom memory allocator?

In fact, if you'd be interested in that, I can work on a PR.

Please let me know!
Much thanks!

- Jonathan

Makefile does not work on CentOS 6.6

CentOS 6.6

steps:
download ck, build and install
download libphenom, run "configure", and then run "make"

result:
the make process run into a infinite loop. It does not stop until I press ctrl+c

RFC: fix SSL_CTX lifetime management for ph_sock_t

This is a heads-up; I'm going to change corelib/net/socket.c so that it doesn't SSL_CTX_free() in sock_dtor.

The TL;DR is that SSL_CTX's are relatively expensive to create (reading and loading keys, certs) and can be shared between SSL instances, so having the sock unilaterally tear everything down imposes a higher cost on the use of SSL than is needed.

The plan is to simply make the socket.c code agnostic when it comes to SSL_CTX. This means that any code that uses SSL enabled ph_sock_t's will need to be changed to either create a long lived SSL_CTX, or to handle the tear down for each session as part of destroying the sock object.

New Function ph_string_concatenate

Why ph_string_concatenate function is not included in the libPhenom??

ph_string_t *ph_string_concatenate(ph_string_t *a,ph_string_t *b)
{
uint32_t len1=strlen(a);
uint32_t len2=strlen(b);
uint32_t i;

  for(i=len1;i<len1+len2;i++)
 {
          a->buf[i]=b->buf[i];
 }

 return(a);

}

Add IO emitter affinity

Goals:

  • Add configuration option to create N epoll emitters rather than just the one that services the M scheduler threads. For sanity, should probably make N == M by default.
  • Add the concept of IO emitter affinity, which is essentially a field in the ph_job_t that identifies which of the emitters the job is associated with.
  • Any jobs with the same emitter affinity will be dispatched by the same scheduler thread and thus will run serially wrt. each other. This makes it simpler to connect two ends of a pipe together for proxy sessions that feed in to the same state machine
  • Should have one timer wheel per emitter so that timers are dispatched serially wrt. their associated event
  • listeners should default to a simple round robin distribution among the emitters when they accept new sessions

compile error

corelib/job.c: In function ‘init_ring’:
corelib/job.c:372:3: error: implicit declaration of function ‘ck_pr_or_64’ [-Werror=implicit-function-declaration]
cc1: all warnings being treated as errors

we must set limit for wbuf and rbuf in struct ph_sock.

struct ph_bufq {
PH_STAILQ_HEAD(bufqhead, ph_bufq_ent) fifo;
// Maximum amount of storage to allow
uint64_t max_size;
};

we may use max_size to set limit for rbuf and wbuf in ph_sock to prevent too many memory cost.
But i can't find detail describe for this.

we may disable socket read in nbio when rbuf reached its limit,
and disable wbuf write when wbuf reach its limit.

why not implement this feature ?

Add socket utilities

Add helpers and structs to make working with the various types of sockets nicer.

For example:

ph_sockaddr_t - a type to portably and easily represent sockaddrs

Functions that express socket operations in terms of our sockaddr type:

  • getting/setting sockaddr from a textual rendition for ipv4, ipv6 or unix addresses
  • getting the local or peer "name" of a socket
  • binding a socket to an address

Can't call ph_sched_run() after ph_sched_stop()

After being stopped with ph_sched_stop(), ph_sched_run() tears down the emitters structure, which it didn't create (https://github.com/facebook/libphenom/blob/e486ef426f1af668869900e982d13d314b45cccb/corelib/nbio/common.c#L399). If we try to run ph_sched_run() again, emitters is null so it segfaults.

We can't run ph_nbio_init() to fix this, because counter_scope is still around from the first call (https://github.com/facebook/libphenom/blob/e486ef426f1af668869900e982d13d314b45cccb/corelib/nbio/common.c#L249).

This makes it hard to use libphenom for e.g. DNS resolution in tests.

Higher level client/server API

Convenient functions that allow configuring a client or server socket to:

  • Buffer send chunks by block and/or line (queue up the writes, they happen async, reads can continue independently unless you request that the write buffer be sync'd up to a certain point)
  • Initiate a read of a fixed buffer size. Trigger a callback when the complete read is satisfied or if an error occurred. Set a time constraint on the read.
  • Initiate a read of a variable size record and specified terminator. For example, CRLF and LF are two common terminators. Trigger a callback when a complete record is read or if an error occurred. Set a time constraint on the read.

It is desirable for these read functions to return slices over the underlying buffers to reduce copying.

For clients, add an API that allows a new client to be created:

  • Specify the target sockaddr to which we want to connect
  • If the target is specified textually, handle name resolution using the job API to run eg: getaddrinfo in a thread pool
  • client connection happens async; trigger a callback when it completes or fails, or fails to complete with a desired timeout. Provide simple access to the underlying failure reason.

The callbacks mentioned here should be able to be dispatched in a thread pool (or the active scheduler thread) as configured by code for that particular client/server/callback.

hashtable crash

when i set 67108864 or more elements, the hashtalbe crash.

2013-11-27 4 45 30 pm

(gdb) p *ht
$5 = {nelems = 67108864, table_size = 134217728, elem_size = 16, mask = 134217727, kdef = 0x7ffff7ff6ea0, vdef = 0x7ffff7ff68e0,
table = 0x7ffe68e92020 ""}

my init:
reph = ph_ht_init(&(mth->ht), 10000,
&ph_ht_string_key_def, &ph_ht_uint32_val_def);

Installation Problem

Following are the errors,while installing libphenom on my ubuntu 10.04
sakky@ubuntu:~/libphenom-master$ make check

gcc -m32 -march=i686 -msse -msse2 -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64 -I/home/sakky/include -D__x86__=1 -std=gnu99 -pedantic -Wall -Wextra -Wno-declaration-after-statement -W -Wundef -Wendif-labels -Wshadow -Wpointer-arith -Wcast-align -Wwrite-strings -Wstrict-prototypes -Wmissing-prototypes -Wno-nested-externs -Winline -Wdisabled-optimization -fstrict-aliasing -pipe -Wno-parentheses -O2 -g -gdwarf-2 -o examples/echo examples/examples_echo-echo.o libphenom.a libtap.a -lrt -lpthread -L/home/sakky/lib -lcares -Wl,-rpath=/home/sakky/lib -lssl -lcrypto
/home/sakky/lib/libssl.a(ssl_algs.o): In function SSL_library_init': ssl_algs.c:(.text+0x1e): undefined reference toEVP_idea_cbc'
collect2: ld returned 1 exit status
make: *** [examples/echo] Error 1

sakky@ubuntu:~/libphenom-master$ sudo make install

gcc -m32 -march=i686 -msse -msse2 -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64 -I/home/sakky/include -D__x86__=1 -std=gnu99 -pedantic -Wall -Wextra -Wno-declaration-after-statement -W -Wundef -Wendif-labels -Wshadow -Wpointer-arith -Wcast-align -Wwrite-strings -Wstrict-prototypes -Wmissing-prototypes -Wno-nested-externs -Winline -Wdisabled-optimization -fstrict-aliasing -pipe -Wno-parentheses -O2 -g -gdwarf-2 -o examples/echo examples/examples_echo-echo.o libphenom.a libtap.a -lrt -lpthread -L/home/sakky/lib -lcares -Wl,-rpath=/home/sakky/lib -lssl -lcrypto
/home/sakky/lib/libssl.a(ssl_algs.o): In function SSL_library_init': ssl_algs.c:(.text+0x1e): undefined reference toEVP_idea_cbc'
collect2: ld returned 1 exit status
make: *** [examples/echo] Error 1
sakky@ubuntu:~/libphenom-master$

Installation Problem

I am not getting how to install it on my ubuntu 10.04,following are the errors
./autogen.sh: 6: aclocal: not found
./autogen.sh: 7: autoheader: not found
./autogen.sh: 8: automake: not found
./autogen.sh: 9: autoconf: not found

Tag a (stable) release?

Hello,

It’d be very helpful if you could tag a release so we know the code there is stable and can be used. I work on Homebrew and we provide a way to install libphenom but we rely on the git HEAD so can’t provide checksums and can’t ensure users will always get the same code depending on the time they install it, making it harder for us to debug issues.

Thanks!

Compilation error, make check, dns.t

Hi, there is a compilation error:

OS X 10.8.5
Installed Autotools (autoconf, automake, libtool)
Installed c-ares (with macport)

$ pkg-config --libs libcares
-L/opt/local/lib -lcares

$ ./autogen.sh
$ ./configure
$ make

$ make check
make  check-TESTS
PASS: tests/counter.t
PASS: tests/memory.t
PASS: tests/timer.t
PASS: tests/printf.t
PASS: tests/iobasic.t
PASS: tests/stream.t
PASS: tests/tpool.t
PASS: tests/string.t
PASS: tests/hashtable.t
PASS: tests/sockaddr.t
./test-driver: line 95: 16030 Segmentation fault: 11  "$@" > $log_file 2>&1
FAIL: tests/dns.t
PASS: tests/variant.t
PASS: tests/bench/iopipes.t
============================================================================
Testsuite summary for phenom 0.1
============================================================================
# TOTAL: 13
# PASS:  12
# SKIP:  0
# XFAIL: 0
# FAIL:  1
# XPASS: 0
# ERROR: 0
============================================================================
See ./test-suite.log
============================================================================
make[2]: *** [test-suite.log] Error 1
make[1]: *** [check-TESTS] Error 2
make: *** [check-am] Error 2

The content of the test-suite.log is:

==================================
   phenom 0.1: ./test-suite.log
==================================

# TOTAL: 13
# PASS:  12
# SKIP:  0
# XFAIL: 0
# FAIL:  1
# XPASS: 0
# ERROR: 0

.. contents:: :depth: 2

FAIL: tests/dns.t
=================

#     Failed test (tests/dns.c:check_addrinfo_result() at line 46)

Please help~

Compile Failure

Hey guys,

Libphenom is currently refusing to compile on OS X 10.10, both with Clang and (GNU) gcc.

Here's a gist of the build log.

Using:

OpenSSL 1.0.1j
Concurrencykit 0.4.4
Automake 1.14.1
Autoconf 2.69
GNU Libtool 2.4.2
Pkg-config 0.28

Installation

While installing libphenom.I am getting followinf errors.
Please help.

gcc -m32 -march=i686 -msse -msse2 -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64 -I/home/sakky/include -D__x86__=1 -std=gnu99 -pedantic -Wall -Wextra -Wno-declaration-after-statement -W -Wundef -Wendif-labels -Wshadow -Wpointer-arith -Wcast-align -Wwrite-strings -Wstrict-prototypes -Wmissing-prototypes -Wno-nested-externs -Winline -Wdisabled-optimization -fstrict-aliasing -pipe -Wno-parentheses -O2 -g -gdwarf-2 -o examples/echo examples/examples_echo-echo.o libphenom.a libtap.a -lrt -lpthread -L/home/sakky/lib -lcares -Wl,-rpath=/home/sakky/lib -lssl -lcrypto

/home/sakky/lib/libssl.a(ssl_algs.o): In function SSL_library_init':
ssl_algs.c:(.text+0x1e): undefined reference toEVP_idea_cbc'
collect2: ld returned 1 exit status
make: *** [examples/echo] Error 1

DNS resolution requires running the scheduler.

There's no way to use libphenom's DNS resolution without running ph_sched_run(). This could be undesirable if we want to resolve something without invoking ph_sched_run() (especially in a test, since #51 prevents us from restarting the scheduler).

If this is desirable, it would be nice to have some documentation around it.

Installation

I am still facing problems with libphenom installation.

I have installed openssl,but openssl.pc is not getting created,thats why whenever i run it gives me error openssl.pc not found.(I have set PKG_CONFIG_PATH).

make check error in cygwin.

$ make
gcc -DHAVE_CONFIG_H -Ithirdparty/ck/include -Iinclude -DPHENOM_IMPL=1 -Werror -m32 -march=i686 -msse -msse2 -D_LARGEF
ILE_SOURCE -D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64 -I/usr/local/include -D__x86__=1 -std=gnu99 -pedantic -
Wall -Wextra -Wno-declaration-after-statement -W -Wundef -Wendif-labels -Wshadow -Wpointer-arith -Wcast-align -W
write-strings -Wstrict-prototypes -Wmissing-prototypes -Wno-nested-externs -Winline -Wdisabled-optimization -fst
rict-aliasing -pipe -Wno-parentheses -O2 -g -gdwarf-2 -MT corelib/libphenom_a-init.o -MD -MP -MF corelib/.deps/libphenom
_a-init.Tpo -c -o corelib/libphenom_a-init.o test -f 'corelib/init.c' || echo './'corelib/init.c
In file included from include/phenom/socket.h:34,
from include/phenom/sysutil.h:21,
from corelib/init.c:17:
include/phenom/openssl.h:25: warning: ignoring #pragma GCC diagnostic
In file included from include/phenom/socket.h:34,
from include/phenom/sysutil.h:21,
from corelib/init.c:17:
include/phenom/openssl.h:32: warning: ignoring #pragma GCC diagnostic
make: *** [corelib/libphenom_a-init.o] Error 1

potential leak with SMR

We can't safely issue a barrier to reclaim resources in the TLS dtor we have for ph_thread_t because those dtors may trigger code that attempts to re-create the TLS segment(s).

When we're on a __thread capable platform (specifically clang) this attempt sometimes corrupts the ph_thread_t itself.

We've avoided the corruption in the work associated with issue #6 but now have a potential leak for threads that were not spawned by ph_thread_spawn(), call into libphenom, and that don't explicitly barrier before they complete.

One way to avoid this is to tie the counter and thread data together, but this may only be a partial solution.

getaddrinfo problems

AI_V4MAPPED in corelib/net/sockaddr.c:238 breaks some of the sockaddr tests on FreeBSD and removing it allows clean pass. I'd like to flesh out a fix that you'll agree with.

The easiest thing would be to conditionally exclude this on FreeBSD. However it seems libevent implemented a portable getaddrinfo in ev_utils to work around rampant portability problems:
http://www.wangafu.net/~nickm/libevent-book/Ref9_dns.html

Cursory look seems that c-ares offers no help here.

build failed with redhat 2.6.32 gcc 4.4.6, error msg: inlining failed in call to ‘dump’

i cloned libphenom in redhat 2.6.32 and builded it with gcc 4.4.6, but there was some error msg:

corelib/variant/json-dump.c: In function ‘dump_string’:
corelib/variant/json-dump.c:34: error: inlining failed in call to ‘dump’: call is unlikely and code size would grow
corelib/variant/json-dump.c:151: error: called from here
make: *** [corelib/variant/libphenom_a-json-dump.o] Error 1

i fixed it with remove the inline.

Resolve library initialization

While bootstrapping the library guts, I made a m[ea]ss of various initialization routines.

Make a pass over them all and try to resolve them into a sane initialization API.

Goals:

  • Where we can cheaply initialize JIT without adding overhead in the hot path, do so
  • Where the subsystem has minimal extraneous memory footprint, but can't be set up JIT, make it initialized by default in some kind of ph_init_library API
  • Where we do things like spin up threads or establish signal handlers, the application should probably be able to enable what they need.

Avoid:

  • pitfalls where multiple libraries that use libphenom make conflicting requests
  • Blowing up if the initialization function(s) are called multiple times from eg: dlopen'd libraries

tests/timer failures on FreeBSD

I haven't dug into this yet. I'm not getting these consistently, sometime the run will be successful. The affinity failed message is always present ("nbio" and "job" in the message were added by me).

1387793688.893 err: /1 nbio failed to set thread 0x80240e200 affinity
1387793688.893 err: /2 nbio failed to set thread 0x80280e200 affinity
1387793688.893 err: /3 nbio failed to set thread 0x802c0e200 affinity
1387793688.894 err: /4 nbio failed to set thread 0x80300e200 affinity
1387793688.894 err: /5 nbio failed to set thread 0x80340e200 affinity
1387793688.894 err: /6 nbio failed to set thread 0x80380e200 affinity
1387793688.894 err: /7 nbio failed to set thread 0x803c0e200 affinity
1387793688.894 err: /8 nbio failed to set thread 0x80400e200 affinity
1387793688.894 err: /9 nbio failed to set thread 0x80440e200 affinity
1387793688.894 err: /10 nbio failed to set thread 0x80480e200 affinity
1387793688.894 err: /11 nbio failed to set thread 0x804c0e200 affinity
1387793688.895 err: dns/13 job failed to set thread 0x80540e200 affinity
1387793688.895 err: dns/12 job failed to set thread 0x80500e200 affinity
1387793688.893 err: phenom:sched/0 nbio failed to set thread 0x80209a200 affinity
#     Failed test (tests/timer.c:record_tick() at line 38)
#     Failed test (tests/timer.c:record_tick() at line 38)
# Looks like you failed 2 tests of 8.
# ELAPSED: 749ms

1..8
ok 1 - PH_OK == ph_nbio_init(0)
ok 2 - PH_OK == ph_job_init(&timer)
ok 3 - PH_OK == ph_job_set_timer_at(&timer, last_tick)
ok 4 - 100ms resolution: diff=106
ok 5 - 100ms resolution: diff=106
not ok 6 - 100ms resolution: diff=213
not ok 7 - 100ms resolution: diff=214
ok 8 - PH_OK == ph_sched_run()

ck usage in libphenom

I saw a lot of usages of ck api, such as 'ck_pr_fence_load', in libphenom source code.
Take 'ph_thread_spawn' function as an example. Why we need to call 'ck_pr_fence_load' after create the thread and call 'ck_pr_load_ptr' to reference the 'thr' pointer?
Is there any document to demonstate the usage of concurrency kit in libphenom?

configure issue

to the PKG_CONFIG_PATH environment variable
No Package 'libcares' found
configure: error ; C-ares is required

Add socket listener API

Provide a convenient but still lowish level API for defining socket based servers and listening for connections.

Need to be able to specify:

  • Local sockaddr to bind to
  • listen(2) backlog
  • Common socket options (eg: SO_REUSEADDR)
  • Callback function to receive the newly connected client socket and phenom sockaddr

It is desirable to enumerate all the active listeners so that they can be shutdown safely in response to the software closing down, or an operator performing an explicit shutdown of a specific listener.

potential race condition

I am only reading the code, and maybe someone can point why I'm wrong, but it seems like there's a potential race condition between ph_counter_block_add() and ph_counter_scope_get()/ph_counter_scope_get_view().

If the thread calling ph_counter_block_add() is switched out before the call to ph_counter_block_record_write(), won't the version be wrong/old and allow inconsistent information to be read? I would think seqno would need incremented before and after the block value update in ph_counter_block_add().

I'm sure I'm probably missing something important here though, I apologize if that is the case.

remove ph_stm_init prototype

Looks like the ph_stm_init() prototype needs to be removed.

The function was replaced with stm_init/PH_LIBRARY_INIT_PRI, but the prototype was left behind.

iobasic.t segfault: os x 10.8.5 w/ clang 3.3

Not able to reproduce this on every run, but here's a backtrace:

Program exited normally.
(gdb) run
Starting program: /Users/jbergstroem/Work/libphenom/tests/iobasic.t
1..13
ok 1 - PH_OK == ph_nbio_init(0)
ok 2 - PH_OK == ph_job_init(&pipe_job)
ok 3 - 0 == ph_pipe(pipe_fd, PH_PIPE_NONBLOCK)
ok 4 - set up ping
ok 5 - 100ms resolution: diff=100
ok 6 - 1 == read(pipe_fd[0], &buf, sizeof(buf))
ok 7 - 100ms resolution: diff=101

Program received signal EXC_BAD_ACCESS, Could not access memory.
Reason: KERN_INVALID_ADDRESS at address: 0x0000000000000004
[Switching to process 21613 thread 0x10b]
0x00000001000165f5 in ck_epoch_synchronize (global=<value temporarily unavailable, due to optimizations>, record=0x0) at thirdparty/ck/src/ck_epoch.c:395
395     record->epoch = delta;
(gdb) bt
#0  0x00000001000165f5 in ck_epoch_synchronize (global=<value temporarily unavailable, due to optimizations>, record=0x0) at thirdparty/ck/src/ck_epoch.c:395
#1  0x000000010001660e in ck_epoch_barrier (global=0x100020600, record=0x0) at thirdparty/ck/src/ck_epoch.c:403
#2  0x000000010000d149 in destroy_thread (ptr=0x101200008) at corelib/thread.c:108
#3  0x00000001001104b0 in _pthread_tsd_cleanup ()
#4  0x00000001001099d2 in _pthread_exit ()
#5  0x000000010010977d in _pthread_start ()
#6  0x00000001000f61a1 in thread_start ()
Current language:  auto; currently minimal

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.