erlyaws / yaws Goto Github PK

Tasks t3, t4, t5 in test/t1/Makefile are the same.

I can't find how to make Yaws listen to a IPv6 address and port.

I can have Yaws work with my IPv4 addresses (behind an NAT etc...), but not with a IPv6 address/port. I have asked on Debian mailing list for Erlang, and they say that ejabberd has support for IPv6, but not Yaws.

I run Debian/Lenny (stable) on a NSLU/2 machine.

The following line in yaws_session_server:start_link/0 is really boring since it breaks the embedded start functionality. The yaws_server isn't started at this point:

{ok, #gconf{ysession_mod = Backend}, _} = yaws_server:getconf(),

The line was apparently added:

c6f26aa (Nicolas Thauvin 2011-01-06 01:23:08 +0100 58)

grrr...
--Tobbe

Currently yaws is not buildable with sinan or installable using faxien. Update the package to build and publish it to the recent erlware.org repository.

Hi,

I am seeing the following error after upgrading to R13B04 that I wasn't seeing on R13B03.

http://pastie.org/853105

I have an OTP app which is starting embedded yaws in the following way (alternative better suggestions welcome!).

http://pastie.org/853110

I was not seeing this error on my OS X dev erlang install until I upgraded to R13B04. I was able to resolve it on my dev box by adding the line:

    application:start(ssl),

Which was just based on a hunch from looking at:

http://github.com/klacke/yaws/blob/4683f85d8872201d706a6b4850b6a44a565ad1ef/src/yaws_api.erl#L1761

However, on my Ubuntu 9.10 box, also with a freshly upgraded R13B04 I always get the error above even if I start the ssl app. I saw only one other mention of this issue on Google in the following similar unanswered thread:

http://groups.google.com/group/erlang-programming/browse_thread/thread/16c471a7263792b6

Here is the erl info on the Ubuntu box:

[glenn@www1 txchat-otp (master)]$ erl
Erlang R13B04 (erts-5.7.5) [source] [smp:2:2] [rq:2] [async-threads:0] [hipe] [kernel-poll:false]

And on OS X:

glenn@macbook-pro ~/src/git/txchat-otp [1.9.1][master]$ erl
Erlang R13B04 (erts-5.7.5) [source] [64-bit] [smp:2:2] [rq:2] [async-threads:0] [hipe] [kernel-poll:false]

Hope you can provide some insight. Thx

Problem 1: The 1st chunk is not sent (flushed) right away to the requesting browser

When a timeout, for example 1 minute, is specified (e.g. by {streamcontent_with_timeout, MimeType, FirstChunk, Timeout} in out/1) and the 2nd chunk is not sent within this time, then the 1st chunk will not sent right away to the requesting browser. The 1st chunk will be sent after 1 minute.

Problem 2: Timeout is reset to 30s

Even when infinity is specified, after sending the 2nd chunk, the timeout will be reset to 30s.

About the design of stream_loop_send:

The FlushStatus argument of stream_loop_send/3 has 3 states: flushed, unflushed, and a timeout value. This reminds me of the "authentication system that returns THREE STATES" problem here:
http://weblog.rubyonrails.org/2009/6/3/security-problem-with-authenticate_with_http_digest

After old http worker processes are wiped the counter is not properly reset, leading to a situation where no worker processes are cached anymore. I inline the patch as it is very short.

Find the patch here: http://gist.github.com/478687

In http://yaws.hyber.org/yman.yaws?page=yaws_api there's no document for streamcontent_with_size.

See:
http://github.com/klacke/yaws/blob/c3a8eb228d989fb8a9e7694817af80b2aa5f621b/src/yaws_server.erl

Hello, WebSockets handshake (and other things) is now changed, and I am not smart enough with Erlang yet to change any code for yaws myself =) Right now this means that yaws does not work with the new browsers (Chromium and WebKit nightly) do not work with our current implementation.

here is some information:
http://axod.blogspot.com/2010/06/websocket-gets-update-and-it-breaks.html
and here: http://webreflection.blogspot.com/2010/06/websocket-handshake-76-simplified.html

Hi.

I recently discovered YAWS - this is a great server!
but would like more control over the configuration settings, without changing the source code
I talk about hard coded definitions, such as :

-define(FCGI_CONNECT_TIMEOUT_MSECS, 10000).

and

-define(FCGI_READ_TIMEOUT_MSECS, 10000).

and other parameters that could be in the configuration file.
( IndexFile of root directory... etc..)

Thank you for the wonderful software.

~/Documents/workspace/yaws> make docs
( cd doc && make docs )
yaws.tex
make[1]: yaws.tex: Command not found
make[1]: *** [yaws.dvi] Error 127
make: *** [docs] Error 2

The website lists a few ways under BSD/Solaris to bind to priv ports as a normal user.

There is a similar solution under linux that doesn't require something like authbind. If you are running 2.6.24 or later (or a vendor kernel with capabilities).

setcap 'cap_net_bind_service=+ep' /usr/bin/erl

[Note: that is probably a symlink, you need to run setcap on the dest.]

After this erl will be allowed to bind to low ports.

Or something like:

setcap 'cap_net_bind_service=+ep' /usr/lib/erlang/erts-5.7.4/bin/beam for running apps.

Note also that capabilities will be cleared when you install a new version of erlang.

In yaws_api, yaws_fcgi:call_fcgi_responder should be written yaws_cgi:call_fcgi_responder. There is no yaws_fcgi.erl.

php fastcgi needs SCRIPT_FILENAME fastcgi param to work properly? otherwise it'll return "no input files"

I tried to write appmod to handle upload file,
When uploaded file size > option max_file_size( about 50MB for exp), in out function, we have to return {ehtml, {p,[], "error message"}}, but the out function is called twice and in browser, i see "Connection Interrupted".

This is my code:
out(Arg) ->
Options = [{max_file_size, 10000000}],
Ret = yaws_multipart:read_multipart_form(Arg, Options),
case Ret of
{done, UploadParams} ->
{ehtml, {p,[], "File upload done"}};
{error, Reason} ->
ErrDesc = wf:f("Error '~s'", [Reason]),
{ehtml, {p,[], ErrDesc}};
{get_more, Cont, NewState} ->
{get_more, Cont, NewState};
Other -> {ehtml, {p,[], "other reason"}}
end.

I use debugger to investigated the Arg variables:
with 1 upload request, there are 2 different Arg :
< Arg = {arg,#Port<0.14942>,
{{10,30,255,12},33023},
{headers,"keep-alive",
"text/html,applition....

< Arg = {arg,#Port<0.14947>,
{{10,30,255,12},33076},
{headers,"keep-alive",
"text/html,application

As you see there are 2 CliSock and 2 different Port number are created with only 1 upload request.

I try with normal get request or upload request with file size < max_file_size there are only 1 CliSock and the Arg is only 1, only 1 Port number.

With the uploaded file size < max_file_size, i uploaded ok without any problem, no matter how large the file is.

Can you tell me how to deal with the case, when size > max_file_size.

Thank you very much.

The README says:
./configure --prefix=/usr/local

But there's no configure file (it has been removed some commits ago).

it is not a good idea to ship the ssl key file in the tarball, because
everyone can download it. I recommend shipping a script that can generate
the *.pem files during or after installation.

If the ebin directory that is specified in the yaws.conf file has not had a
beam file (untested with other file types) placed in it, the server will
fail to launch at the command line stating that it expects a directory at
the ebin_dir line. Once a beam has been placed in this directory and the
server started, it does not seem to mind if the file is removed.

This was noticed on Ubuntu Gutsy, package install of erlang, source install
of yaws 1.76.

OS: Arch Linux x86-64
Erlang: R13B04
Yaws: 1.87

Appmod that does that: http://pastebin.com/1K61xNAs
Configuration:

<server localhost>
    port = 8001
    listen = 127.0.0.1
    docroot = /home/foxbunny/code/gallery
    appmods = <gallery, gallery>
    dir_listings = true
</server>

I've tried uploading a 12MB file and Yaws crashed with:

eheap_alloc: Cannot allocate 934157120 bytes of memory (of type "heap").
Aborted

I still haven't gotten around to testing the smallest file size that causes this. Tested on both my desktop (2G RAM + 1G swap) and laptop (1.5G RAM + 1G swap).

I've to admit that iam relatively new to erlang.
Yesterday i did some yaws experiments:

I configured a module to be a root appmod and expected this appmod to be invoked for all requests.
This seemed to work fine at first glance but then i explored the following (Yaws-1.85)
The case

1. user submits a form (method post)
2. yaws invokes the appmod
3. the appmod's out function returns {page, "/something.yaws"}
4. yaws processes "/something.yaws" and cashes the outcome (the cashed outcome includes url type yaws not appmod)
5. browser displays the outcome to user
6. user steps back to the form using the browser back button and re-submits the form (and enters some different values prior to submitting)
7. yaws handles the 'POST' and looks up the cashed outcome with the url-type yaws and not appmod
8. yaws does not invoke the appmod. So the appmod can't process the input parameters.

Hello,

Trying to change database backend for a old yaws project to couchdb , I've found that json in couch it's not compatible with json:encode, at least in binaries related stuff. I've patched json.erl to support only encoding of binaries in keys and value,

The patch it's the following, applied to 1.85 version, feel free to apply it if it's useful for another people.

Index: src/json.erl                                   
===================================================================
--- src/json.erl    (revisión: 9410)                     
+++ src/json.erl    (copia de trabajo)                   
@@ -109,6 +109,8 @@                                   
    unicode -> encode_string(xmerl_ucs:to_utf8(L));     
    no -> exit({json_encode, {not_string, L}})          
     end;                                             
+encode(B) when is_binary(B) ->                       
+    encode(erlang:binary_to_list(B));                
 encode({array, Props}) when is_list(Props) -> encode_array(Props);
 encode({struct, Props} = T) when is_list(Props) -> encode_object(T);
 encode(Bad) -> exit({json_encode, {bad_term, Bad}}). 
@@ -116,6 +118,8 @@                                   
 %% Encode an Erlang string to JSON.                  
 %% Accumulate strings in reverse.                    

+encode_string(B) when is_binary(B) ->                
+    encode_string(erlang:binary_to_list(B));         
 encode_string(S) -> encode_string(S, [$"]).             % " fix highlight for vim :)

 encode_string([], Acc) -> lists:reverse([$" | Acc]); % " fix highlight for vim :)
@@ -145,6 +149,7 @@                                   
 encode_object({struct, _Props} = Obj) ->             
     M = obj_fold(fun({Key, Value}, Acc) ->           
    S = case Key of                                     
+        B when is_binary(B) -> encode_string(B);     
        L when is_list(L) -> encode_string(L);          
        A when is_atom(A) -> encode_string(atom_to_list(A));
             _ -> exit({json_encode, {bad_key, Key}}) 

I get this after following the instructions in the wiki:

Debugger entered--Lisp error: (void-variable c)
  (car c)
  (assoc (car c) auto-mode-alist)
  (or (assoc (car c) auto-mode-alist) (setq auto-mode-alist (cons c auto-mode-alist)))
  (let ((a ...) (b ...)) (or (assoc ... auto-mode-alist) (setq auto-mode-alist ...)) (or (assoc ... auto-mode-alist) (setq auto-mode-alist ...)) (Or (assoc ... auto-mode-alist) (setq auto-mode-alist ...)))
  eval-buffer(#<buffer  *load*> nil "/home/foxbunny/.emacs" nil t)  ; Reading at buffer position 2492
  load-with-code-conversion("/home/foxbunny/.emacs" "/home/foxbunny/.emacs" t t)
  load("~/.emacs" t t)
  #[nil "�\205\264

I'm running Emacs 23.2 on Linux, w/ erlang mode already installed. It works ok in Erlang mode, but when I open .yaws file, it shows I'm in SGML major mode & two-mode minor mode (but I'm not sure since I'm rather new to Emacs, I'm guessing based on SMGL two-mode text in the editor), and it doesn't recognize the erl block, nor it handles any Erlang code correctly.

In test/t2, I ran TEST=server_options_test make test and the test did not pass.

Checking the headers returned from Yaws, I see that there is a 'Date' header which is not matched by any app_test/server_options_recv/2.

An example list of headers returned from Yaws:

{http_response,{1,1},200,"OK"}
{http_header,30,'Server',undefined,"Yaws/1.82 Yet Another Web Server"}
{http_header,3,'Date',undefined,"Mon, 08 Jun 2009 06:21:34 GMT"}
{http_header,34,'Allow',undefined,"GET, HEAD, OPTIONS, PUT, POST, DELETE"}
{http_header,38,'Content-Length',undefined,"0"}
http_eoh

Should the following 2 lines be added to app_test.erl?
server_options_recv(S, [{http_header,,'Date',,}|]=Hdrs) ->
do_server_options_recv(S, Hdrs);

Hello,

I'm trying to generate a soap fault from a yaws soap endpoint, what's the correct return from handle function ?

I'm trying with:
{error,undefined,
#'soap:Fault'{
anyAttribs = [],
faultcode = "error",
faultstring = Error
}
}
But yaws_soap_srv prints :
yaws_soap_srv(239): Srv Error: "Error processing message: {error,undefined,\n {'soap:Fault',[],"error","econnrefused",\n undefined,undefined}}"

Any idea ?

Thanks. Regards.

If there are:

• A static file at path /xxx
• An appmod also mounted at path /xxx
  Currently Yaws always forward requests to /xxx to the appmod. This is not desirable for application programmers.

More information:
http://wiki.github.com/klacke/yaws/root-appmod

yaws_ctl:400: match {ok, Port} instead of Port?

yaws_content_negotiation:30: Headers is a record (#headers{}) but is passed to lists:keysearch/3 which expects a list

yaws_server.erl:2511: ut_open/1 returns Fd={bin, B} and is used in skip_data/3 which expects Fd to be a #file_descriptor{}/port()

haxe.erl:370: can never match

yaws_rpc.erl:264 can never match because yaws_rpc.erl:313 returns {ok, StructStr}

Thanks to Kostis for Dialyzer help.

This is a real issue.
We know that atoms are not garbage collected. Yet someone designed the jaws library the way, that it always create atoms when user decodes json objects. This is dangerous. We should rather change the design so that either we'll use list_to_atom( X, [safe]), or even better - we shan't use list_to_atom at all.
It's important, because json decoding is intended to be heavily used in servers - in deployment environment, not just e.g. while the compilation. This makes yaws (and in fact any application using this version of library) vulnerable to attack on server's memory.

Markup without content do not generate a well formed HTML. For example {img, [{src, "img/foo.png"}]} will be translated to <img src="img/foo.png"></img> instead of <img src="img/foo.png" />.

W3C example: http://validator.w3.org/check?uri=http%3A%2F%2Fwww.d-lan.net%2Fdonate.html&charset=%28detect+automatically%29&doctype=Inline&group=0&user-agent=W3C_Validator%2F1.2
source: https://github.com/Ummon/D-LAN/blob/2a114f46a09f063c22aee5f20765049789c670ef/yssi/donate.yaws

I don't think it costs to much to change this behavior.

Yaws version I use: 1.88

Hi folks, Let's say I want to always return a 'Origin:' header (see: http://www.w3.org/TR/cors/). How can I do that? For an appmod, or yaws-page that is easy, but how about index.html or Javascript files being served? Perhaps a server-config <header>Origin: http://example.com </header> could work?

Cheers, Tobbe

testing the issue system

Hello respected public!

I use embedded Yaws with SSL autorisation by clients certifikate.
Version 1.84 Yaws work fine with both Erlang R12B so and R13B.
A Version 1.88 of Yavs under Erlang R13B in this case has a problems (Web browser hangs in data wait state after SSL connection).

Use of use_old_ssl resolve this problem, but I can not set up GC_USE_OLD_SSL flag in embedded mode (error in set_gc_flags ... ).

With help of next little patch can be fix an this error.

diff --git a/src/yaws.erl b/src/yaws.erl
index 323803f..d20d14a 100644
--- a/src/yaws.erl
+++ b/src/yaws.erl
@@ -243,6 +243,8 @@ set_gc_flags([{pick_first_virthost_on_nomatch, Bool}|T], Flags) ->
set_gc_flags(T, flag(Flags, ?GC_PICK_FIRST_VIRTHOST_ON_NOMATCH,Bool));
set_gc_flags([{use_fdsrv, Bool}|T], Flags) ->
set_gc_flags(T, flag(Flags,?GC_USE_FDSRV,Bool));
+set_gc_flags([{use_old_ssl, Bool}|T], Flags) ->

• set_gc_flags(T, flag(Flags,?GC_USE_OLD_SSL,Bool));
  set_gc_flags([_|T], Flags) ->
  set_gc_flags(T, Flags);
  set_gc_flags([], Flags) ->

With best
Sergey Shilov.
The Yaws maintainer in http://sisyphus.ru/en/ packages repository.

(

logdir = /var/log/yaws

file /var/log/yaws/pt.foothold.ru:8080.access is empty (((

virt server pt.foothold.ru

Just wondering why this module isn't included in the Makefile. :)
(I thought of forking+changing+doing a pull request but there might be a valid reason for the module being left out hence opening an issue instead)
:Davide

It doesn't seem to work with docroot set to a relative path in yaws.conf
The server is just ignored it seems

you can apply small patch for fix

-AS_HELP_STRING([--with-defaultcharset=String], [specify default charset, i.e UTF-8]))
+AS_HELP_STRING([--with-default-charset=String], [specify default charset, i.e UTF-8]))

When spawning a new process from within a query to parallelize the few tasks that can be, the default first parameter (#arg) can be sent to the new process, but the yaws libraries won't be able to fetch all the information in it as it appears to be stored in the parent's process dictionary.

Data thus appears missing for no reason, unless when going in the source­ to find out you have to copy the process dictionary to any spawned process in order to keep full functionality.

As far as I know, this is undocumented behavior, so either documenting it or correcting it could possibly help the few other people who will trip on that bug.

I'm on Leopard with Yaws 1.84.

1. Mount myappmod at / (in yaws.conf: appmods = </, myappmod>) so that it catches all request

2. In myappmod:out/1, put({yaws, page}, Arg#arg.server_path) then return {page, Arg#arg.server_path}

3. Request a non-existing file, Yaws will freeze for about one minute and crash with something like this on the screen:

   beam.smp(513,0xb01aa000) malloc: *** mmap(size=1140850688) failed (error code=12)
                                                                                                *** error: can't allocate region
                                                                                                                                *** set a breakpoint in malloc_error_break to debug
                                                                                                                                                                                   beam.smp(513,0xb01aa000) malloc: *** mmap(size=1140850688) failed (error code=12)
                                    *** error: can't allocate region
                                                                    *** set a breakpoint in malloc_error_break to debug
                                                                                                                       beam.smp(513,0xb01aa000) malloc: *** mmap(size=1140330496) failed (error code=12)
                                                                                                                                                                                                        *** error: can't allocate region
        *** set a breakpoint in malloc_error_break to debug
                                                           beam.smp(513,0xb01aa000) malloc: *** mmap(size=1140330496) failed (error code=12)
                                                                                                                                            *** error: can't allocate region
                                                                                                                                                                            *** set a breakpoint in malloc_error_break to debug
   
   Crash dump was written to: erl_crash.dump
   eheap_alloc: Cannot allocate 1140328500 bytes of memory (of type "old_heap").
   

Is there any plans to support yaws as a rebar dependency for other projects?

The text/_ content is generally minuscule contribution to the loading times.
files like image/png, application/javascript amongst other file types are also in need to be compressed.

Having the compressible_mime_type definitions in the hardcoded like it is in yaws_server.erl is really limiting. I suggest loading configuration from config.

I attempted to turn caching off by specifying "max_num_cached_files = 0" in
yaws.conf, but after the first page access, beam uses 100% CPU, and there
is hundreds of messages like below added to the report.log:

=INFO REPORT==== 3-Aug-2008::21:55:01 ===
Max size cached bytes reached for server "ltop"

perhaps the full original uri could be sent along arg? (or retrieved if wanted)

just checked out a clean version and ran ./configure; make
build output, using R13B

gcc -c -fPIC -g -O2 -I/usr/include/security -I/usr/include/pam -I"/usr/local/erlang/R13B/lib/erlang/usr/include" -I/usr/include/pam/ -DDYNAMIC_DRIVER setuid_drv.c
gcc -bundle -fPIC -flat_namespace -undefined suppress -o ../priv/lib/setuid_drv.so setuid_drv.o
gcc -c -g -O2 -I/usr/include/security -I/usr/include/pam -I"/usr/local/erlang/R13B/lib/erlang/usr/include" -I/usr/include/pam/ epam.c
gcc -o ../priv/epam epam.o -lpam
gcc -c -fPIC -g -O2 -I/usr/include/security -I/usr/include/pam -I"/usr/local/erlang/R13B/lib/erlang/usr/include" -I/usr/include/pam/ -DDYNAMIC_DRIVER yaws_sendfile_drv.c
gcc -c -fPIC -g -O2 -I/usr/include/security -I/usr/include/pam -I"/usr/local/erlang/R13B/lib/erlang/usr/include" -I/usr/include/pam/ hashtable.c
gcc -bundle -fPIC -flat_namespace -undefined suppress -o ../priv/lib/yaws_sendfile_drv.so yaws_sendfile_drv.o hashtable.o
. ../vsn.mk;
cat yaws_generated.template |
../scripts/Subst %VSN% 1.81 |
../scripts/Subst %VARDIR% /Users/dbudworth/yaws/var |
../scripts/Subst %localinstall% true |
../scripts/Subst %ETCDIR% /Users/dbudworth/yaws/etc > yaws_generated.erl
set -x;
if [ true = "true" ]; then
echo "-define(HAVE_SENDFILE, true)." > yaws_configure.hrl;
else touch yaws_configure.hrl; fi

• '[' true = true ']'
• echo '-define(HAVE_SENDFILE, true).'
  "/usr/local/erlang/current/bin/erlc" -W -pa ../../yaws -I ../include -o ../ebin yaws.erl
  "/usr/local/erlang/current/bin/erlc" -W -pa ../../yaws -I ../include -o ../ebin yaws_app.erl
  "/usr/local/erlang/current/bin/erlc" -W -pa ../../yaws -I ../include -o ../ebin yaws_ticker.erl
  "/usr/local/erlang/current/bin/erlc" -W -pa ../../yaws -I ../include -o ../ebin yaws_config.erl
  ./yaws_config.erl:1145: field extra_cgi_vars undefined in record sconf
  make[1]: *** [../ebin/yaws_config.beam] Error 1
  make: *** [all] Error 1

Using the server configuration, appmods = </, myappmod> prevents yssi to execute. Any {yssi, Filename} will simply result in nothing for the client.

Using a normal word instead of / for the appmod trigger do work.

Bad parsing of the allowed_scripts = line in yaws.conf

allowd_scripts = [yaws]

fails

Looking at the JSON-RPC specification at http://json-rpc.org/wiki/specification the returned result of a call needs to contain an error field, with value null for the success case. The example json-rpc python client code on the json-rpc.org site expects this field to be present.

It should be as simple as adding it to encode_handler_payload's json:encode call at http://github.com/klacke/yaws/blob/master/src/yaws_rpc.erl#L314 (as of http://github.com/klacke/yaws/commit/6144b4fe2d6f66a0f7be0ba7931f7ff5eda554be)

the instructions for cloning and building from github should indicate the necessity to first pull in the needed prerequisites before building by running apt-get build-dep yaws (if ubuntu) - or otherwise list what they are - the errors on "make" really don't indicate much of utility to the user ...

It is better if Yaws allows specifying both size and timeout when streaming contents. See:
http://osdir.com/ml/web.server.yaws.general/2007-02/msg00028.html

in apache i can set option
DirectoryIndex index.html index.cgi index.pl index.php index.xhtml index.htm

and when i write url http://whoz.ru, automate go to http://whoz.ru/index.php

can you make this option in you web server

I'm getting strange characters when yaws displays the man pages. Had a quick look and I'm not sure what's going on. Using Firefox 3.0.10 on Ubuntu.

Picture:
http://ovik.net/~faah/man-error.png

Hello,

I've been using yaws embedded with revproxy from a long time, after upgrading to 1.90 (HEAD too) from 1.85 revproxy blocks yaws after a few requests.

You can clone the test case from git://github.com/mdaguete/Yaws_test.git

compile with make, run with make console

Eshell V5.8.3 (abort with ^G)
1> application:start(yt).
ok

3> yt_test:test_embedded().
test [1]: {ok,"404"}
test [2]: {ok,"404"}
test [3]: {error,req_timedout}

Thank You.