Running web-serving problem about cloudsuite

[Hnefi]

Hi! Apologies for the delay in replying to you.

From looking at your log output, it looks like you made a typo in specifying the web server's IP, here:
WEB_SERVER_IP="web_server"
That needs to contain an actual IP address where Faban will direct its load, such as localhost or 127.0.0.1.
Let me know if you have any more questions!

[Hnefi]

Hi @yunjeongmun I have done more looking into this problem, and have managed to reproduce your results (with some different values). Here is what I got when running your commands (docker networks) on an OS X system.

     [java] 	at java.io.FileInputStream.open(Native Method)
     [java] 	at java.io.FileInputStream.<init>(FileInputStream.java:146)
     [java] 	at java.io.FileReader.<init>(FileReader.java:72)
     [java] 	at workload.driver.Web20Driver.<init>(Web20Driver.java:229)
     [java] 	at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
     [java] 	at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
     [java] 	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
     [java] 	at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
     [java] 	at java.lang.Class.newInstance(Class.java:383)
     [java] 	at com.sun.faban.driver.engine.TimeThreadWithBackground.doRun(TimeThreadWithBackground.java:81)
     [java] 	at com.sun.faban.driver.engine.AgentThread.run(AgentThread.java:202)
     [java]
     [java] Nov 05, 2017 10:21:47 AM com.sun.faban.driver.engine.AgentImpl run
     [java] WARNING: ElggDriverAgent[0]: Run aborted before starting 7 driver threads.
     [java] 1 threads were started.
     [java] Nov 05, 2017 10:21:48 AM com.sun.faban.driver.engine.AgentImpl kill
     [java] WARNING: ElggDriverAgent[0]: Killing benchmark run
     [java] Nov 05, 2017 10:21:57 AM com.sun.faban.driver.engine.MasterImpl main
     [java] SEVERE: Master terminated with errors.
     [java] com.sun.faban.driver.FatalException: Run Aborted.
     [java] 	at com.sun.faban.driver.engine.MasterImpl.executeRun(MasterImpl.java:677)
     [java] 	at com.sun.faban.driver.engine.MasterImpl.runBenchmark(MasterImpl.java:276)
     [java] 	at com.sun.faban.driver.engine.MasterImpl.main(MasterImpl.java:1566)
     [java]

BUILD FAILED
/web20_benchmark/build.xml:81: Java returned: 1

I will take a look into this. Out of curiosity are you running on OS X or Linux? Docker bridge networking operates slightly differently on OS X.

[yunjeongmun]

I'm running on Linux system.
And I'll try to change the web server ip like WEB_SERVER_IP=localhost or 127.0.0.1 , not WEB_SERVER_IP="web_server".

[yunjeongmun]

As you siad, I modified WEB_SERVER_IP=actual host ip address, and I used 80 port because of the error with port 8080:

curl: (7) Failed to connect to 10.95.111.200 port 8080: Connection refused

So I added 'EXPOSE 80' in Dockerfile of db_server, faban_client, memcached server, and webserver to open port 80.
Also I used a host network. I ran below commands.

WEB_SERVER_IP=10.95.111.200
docker run -dt --net=host --name=mysql_server cloudsuite/web-serving:db_server_02 $WEB_SERVER_IP &
docker run -dt --net=host --name=memcache_server cloudsuite/web-serving:memcached_server_02 &
docker run -dt --net=host --name=web_server cloudsuite/web-serving:web_server_02 /etc/bootstrap.sh &
docker run --net=host --name=faban_client cloudsuite/web-serving:faban_client_02 ${WEB_SERVER_IP}

And I still got error messages.

Please point your browser to http://dl360g9s39:9980/
Exception in thread "main" java.io.FileNotFoundException: http://10.95.111.200:80/action/login
        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
        at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
        at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
        at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
        at sun.net.www.protocol.http.HttpURLConnection$7.run(HttpURLConnection.java:1725)
        at sun.net.www.protocol.http.HttpURLConnection$7.run(HttpURLConnection.java:1723)
        at java.security.AccessController.doPrivileged(Native Method)
        at sun.net.www.protocol.http.HttpURLConnection.getChainedException(HttpURLConnection.java:1721)
        at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1292)
        at com.sun.faban.driver.transport.sunhttp.SunHttpTransport.fetchResponse(SunHttpTransport.java:673)
        at com.sun.faban.driver.transport.sunhttp.SunHttpTransport.fetchURL(SunHttpTransport.java:557)
        at com.sun.faban.driver.transport.sunhttp.SunHttpTransport.fetchURL(SunHttpTransport.java:519)
        at setup.UserGenerator.createUsers(UserGenerator.java:114)
        at setup.UserGenerator.main(UserGenerator.java:147)
Caused by: java.io.FileNotFoundException: http://10.95.111.200:80/action/login
        at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1674)
        at java.net.HttpURLConnection.getResponseCode(HttpURLConnection.java:468)
        at com.sun.faban.driver.transport.sunhttp.SunHttpTransport.fetchResponse(SunHttpTransport.java:654)
        ... 4 more

And then, some running log like below..

     [java] FINE: Obtaining registry at localhost:9998
     [java] Nov 07, 2017 4:24:04 AM com.sun.faban.common.FabanNamespaceContext addNamespace
     [java] FINER: Adding http://www.w3.org/XML/1998/namespace, xml
     [java] Nov 07, 2017 4:24:04 AM com.sun.faban.common.FabanNamespaceContext addNamespace
     [java] FINER: Adding http://www.w3.org/2000/xmlns/, xmlns
     [java] Nov 07, 2017 4:24:04 AM com.sun.faban.common.FabanNamespaceContext addNamespace
     [java] FINER: Adding http://faban.sunsource.net/ns/faban, fa`

And finally, I got error message as below

     [java] Nov 07, 2017 4:24:07 AM com.sun.faban.driver.engine.TimeThreadWithBackground doRun
     [java] SEVERE: ElggDriverAgent[0].0: Error initializing driver object.
     [java] java.io.FileNotFoundException: /faban/users.list (No such file or directory)
     [java]     at java.io.FileInputStream.open(Native Method)
     [java]     at java.io.FileInputStream.<init>(FileInputStream.java:146)
     [java]     at java.io.FileReader.<init>(FileReader.java:72)
     [java]     at workload.driver.Web20Driver.<init>(Web20Driver.java:229)
     [java]     at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
     [java]     at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
     [java]     at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
     [java]     at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
     [java]     at java.lang.Class.newInstance(Class.java:383)
     [java]     at com.sun.faban.driver.engine.TimeThreadWithBackground.doRun(TimeThreadWithBackground.java:81)
     [java]     at com.sun.faban.driver.engine.AgentThread.run(AgentThread.java:202)
     [java]
     [java] Nov 07, 2017 4:24:08 AM com.sun.faban.driver.engine.AgentImpl run
     [java] WARNING: ElggDriverAgent[0]: Run aborted before starting 7 driver threads.
     [java] 1 threads were started.
     [java] Nov 07, 2017 4:24:08 AM com.sun.faban.driver.engine.AgentImpl kill
     [java] WARNING: ElggDriverAgent[0]: Killing benchmark run
     [java] Nov 07, 2017 4:24:18 AM com.sun.faban.driver.engine.MasterImpl main
     [java] SEVERE: Master terminated with errors.
     [java] com.sun.faban.driver.FatalException: Run Aborted.
     [java]     at com.sun.faban.driver.engine.MasterImpl.executeRun(MasterImpl.java:677)
     [java]     at com.sun.faban.driver.engine.MasterImpl.runBenchmark(MasterImpl.java:276)
     [java]     at com.sun.faban.driver.engine.MasterImpl.main(MasterImpl.java:1566)
     [java]

BUILD FAILED
/web20_benchmark/build.xml:81: Java returned: 1

Total time: 14 seconds
cat: /faban/output/*/summary.xml: No such file or directory

Thank you

[Hnefi]

In the following command, the script bootstrap.sh has two optional arguments specifying the SQL container IP as well as the caching server:

docker run -dt --net=host --name=web_server cloudsuite/web-serving:web_server_02 /etc/bootstrap.sh &

You can find those instructions on the Web Serving section of the Cloudsuite website. So, in your case (using the host network) it would be something like:

docker run -dt --net=host --name=web_server cloudsuite/web-serving:web_server_02 /etc/bootstrap.sh localhost localhost 

However, in my tests the user session creation will still fail as before. I have managed to trace this problem to the generation of the file users.list, in the file UserGenerator.java. Currently I am testing a fix that I hope to finish soon.

[yunjeongmun]

Thanks for your reply. Now, I successfully run web-serving benchmark.
I added "-p 8080:8080" in faban client running command line.. and it worked!
And I used cloudsuite/web-serving:faban_client image offered by CloudSuite.
Below is my command line

WEB_SERVER_IP=10.95.111.198
DATABASE_SERVER_IP=$WEB_SERVER_IP
MEMCACHED_SERVER_IP=$WEB_$SERVER_IP
MAX_PM_CHILDREN=20000
LOAD_SCALE=7
#WEB_SERVER_IP=localhost
docker run -dt --net=host --name=mysql_server cloudsuite/web-serving:db_server $WEB_SERVER_IP &
docker run -dt --net=host --name=memcache_server cloudsuite/web-serving:memcached_server &
docker run -dt --net=host --name=web_server cloudsuite/web-serving:web_server /etc/bootstrap.sh $DATABASE_SERVER_IP $MEMCACHED_SERVER_IP $MAX_PM_CHILDREN&
docker run -p 8080:8080 --net=host --name=faban_client cloudsuite/web-serving:faban_client ${WEB_SERVER_IP} $LOAD_SCALE 

Actually, I don't know why it works well..
In my understanding.. "-p 8080:8080" open port 8080 of container faban_client.. Why do I have to include '-p 8080:8080' explicitly?

And I have one more question, is it correct to increase 'LOAD_SCALE' value until CPU resources are full?
Do you have any recommendation regarding the LOAD_SCALE value setting? 7 users didn't spend many CPU resources.

[Hnefi]

I am not sure why adding a port publish has made this work for you. Since you are using the host network, there is no requirement to expose or publish ports explicitly. While the containers are running, can you post the output of running docker network inspect host, and docker ps?

I am still testing the fix for running the client driver with docker networks, and will let you know when we have finished it.

Regarding LOAD_SCALE, that controls the number of users that are simultaneously logging in to the web servers and requesting social networking pages. You can scale it up and down as much as you would like - keeping in mind that right now, you are running a single container as a client driver, and scaling the number of threads may start to stress the system. To tune the benchmark, we recommend testing your machines for the maximum possible request throughput, while maintaining your target QoS metric (we use 99th percentile latency). CPU utilization is less important than the latency and responsiveness for these benchmarks.

[yunjeongmun]

Thanks for your reply!

docker network inspect host shows

[
    {
        "Name": "host",
        "Id": "0e9cab7b529302d82858c3066c63886e5afb166758f49bef5d5d44171cc143ed",
        "Scope": "local",
        "Driver": "host",
        "EnableIPv6": false,
        "IPAM": {
            "Driver": "default",
            "Options": null,
            "Config": []
        },
        "Internal": false,
        "Containers": {
            "0fdc2255a0e98e1dab2ffb112016e1121c629a80dd8ba47dc3cd20ad70c51b2b": {
                "Name": "memcache_server",
                "EndpointID": "27e96eb72a81aedd2c1e7773b6c77a63cec3b85b2d1525c23afba88b7963d4cc",
                "MacAddress": "",
                "IPv4Address": "",
                "IPv6Address": ""
            },
            "35db3f12dcab527b777fd4f7e9b1a558a731ee1349f6d046ba761bffad0df101": {
                "Name": "mysql_server",
                "EndpointID": "64bf8ea4530a33d331b9cf4ce5c94df640187951dbf8cc3ca07c2e15b252af8d",
                "MacAddress": "",
                "IPv4Address": "",
                "IPv6Address": ""
            },
            "6533212505a0f87c748eff229d2cd65cecfae92d9506063604f9d84a95aa5c30": {
                "Name": "faban_client",
                "EndpointID": "6b52016e142a8d05d4e4f91199e7d5788c2dc8d816d47fe50af7956516afd9c7",
                "MacAddress": "",
                "IPv4Address": "",
                "IPv6Address": ""
            },
            "d47a6857d05d2766431b894c45581073b18a0d4dde35af4e2df8e6ac0959a14d": {
                "Name": "web_server",
                "EndpointID": "53258e0ce97daf87f950d43464a6530ed0255717b55ef8bc25803b4c4cc205c9",
                "MacAddress": "",
                "IPv4Address": "",
                "IPv6Address": ""
            }
        },
        "Options": {},
        "Labels": {}
    }
]

docker ps shows

CONTAINER ID        IMAGE                                     COMMAND                  CREATED             STATUS              PORTS               NAMES
6533212505a0        cloudsuite/web-serving:faban_client       "/etc/bootstrap.sh 10"   31 seconds ago      Up 30 seconds                           faban_client
0fdc2255a0e9        cloudsuite/web-serving:memcached_server   "/bin/sh -c 'bash -c "   31 seconds ago      Up 30 seconds                           memcache_server
d47a6857d05d        cloudsuite/web-serving:web_server         "/etc/bootstrap.sh 10"   31 seconds ago      Up 30 seconds                           web_server
35db3f12dcab        cloudsuite/web-serving:db_server          "/etc/bootstrap.sh 10"   31 seconds ago      Up 30 seconds                           mysql_server

I increase the LOAD_SCALE value 10 to 1000, and I got the summary.xml output file for each running.
In the summary.xml , some are 'true' and others are 'false', so I confuse that each test run successfully.

Does <passed>false</passed> means that the test is failed?
Is <responseTimes> metric you said before? (we use 99th percentile latency)

Thank you

[Hnefi]

Hello:
Apologize for the delay in responding to your question. A failed test can happen for many reasons, such as: the frequency of queries not meeting the test bench requirement (defined in the test harness), or latency violations.

For example, you have requested one machine to run 1000 user load generating threads (with that parameter of LOAD_SCALE), which likely will cause QoS violations. In order to "tune" the workload to properly model the behaviour of a frontend web server, you need to play with the system's parameters, such as client load mix, number of incoming requests, and latency constraints.

Regarding the issue with Docker networks - I have figured out the reason why this happens, and will write a quick fix as soon as I can.

[neo-apz]

I'm closing this for now.
Feel free to reopen it if you have any further questions.

[pestep-micron]

Hello:
Apologize for the delay in responding to your question. A failed test can happen for many reasons, such as: the frequency of queries not meeting the test bench requirement (defined in the test harness), or latency violations.

For example, you have requested one machine to run 1000 user load generating threads (with that parameter of LOAD_SCALE), which likely will cause QoS violations. In order to "tune" the workload to properly model the behaviour of a frontend web server, you need to play with the system's parameters, such as client load mix, number of incoming requests, and latency constraints.

Regarding the issue with Docker networks - I have figured out the reason why this happens, and will write a quick fix as soon as I can.

Hello:
Apologize for the delay in responding to your question. A failed test can happen for many reasons, such as: the frequency of queries not meeting the test bench requirement (defined in the test harness), or latency violations.

For example, you have requested one machine to run 1000 user load generating threads (with that parameter of LOAD_SCALE), which likely will cause QoS violations. In order to "tune" the workload to properly model the behaviour of a frontend web server, you need to play with the system's parameters, such as client load mix, number of incoming requests, and latency constraints.

Regarding the issue with Docker networks - I have figured out the reason why this happens, and will write a quick fix as soon as I can.

@Hnefi did you

Hello:
Apologize for the delay in responding to your question. A failed test can happen for many reasons, such as: the frequency of queries not meeting the test bench requirement (defined in the test harness), or latency violations.

For example, you have requested one machine to run 1000 user load generating threads (with that parameter of LOAD_SCALE), which likely will cause QoS violations. In order to "tune" the workload to properly model the behaviour of a frontend web server, you need to play with the system's parameters, such as client load mix, number of incoming requests, and latency constraints.

Regarding the issue with Docker networks - I have figured out the reason why this happens, and will write a quick fix as soon as I can.

@Hnefi did you ever create a fix for this? Do you have a patch I could try? Thanks.

[pradeepk5]

Hello:
Apologize for the delay in responding to your question. A failed test can happen for many reasons, such as: the frequency of queries not meeting the test bench requirement (defined in the test harness), or latency violations.
For example, you have requested one machine to run 1000 user load generating threads (with that parameter of LOAD_SCALE), which likely will cause QoS violations. In order to "tune" the workload to properly model the behaviour of a frontend web server, you need to play with the system's parameters, such as client load mix, number of incoming requests, and latency constraints.
Regarding the issue with Docker networks - I have figured out the reason why this happens, and will write a quick fix as soon as I can.

Hello:
Apologize for the delay in responding to your question. A failed test can happen for many reasons, such as: the frequency of queries not meeting the test bench requirement (defined in the test harness), or latency violations.
For example, you have requested one machine to run 1000 user load generating threads (with that parameter of LOAD_SCALE), which likely will cause QoS violations. In order to "tune" the workload to properly model the behaviour of a frontend web server, you need to play with the system's parameters, such as client load mix, number of incoming requests, and latency constraints.
Regarding the issue with Docker networks - I have figured out the reason why this happens, and will write a quick fix as soon as I can.

@Hnefi did you

Hello:
Apologize for the delay in responding to your question. A failed test can happen for many reasons, such as: the frequency of queries not meeting the test bench requirement (defined in the test harness), or latency violations.
For example, you have requested one machine to run 1000 user load generating threads (with that parameter of LOAD_SCALE), which likely will cause QoS violations. In order to "tune" the workload to properly model the behaviour of a frontend web server, you need to play with the system's parameters, such as client load mix, number of incoming requests, and latency constraints.
Regarding the issue with Docker networks - I have figured out the reason why this happens, and will write a quick fix as soon as I can.

@Hnefi did you ever create a fix for this? Do you have a patch I could try? Thanks.

The docker network issue no longer exist. Please follow the latest document to run the workloads: https://github.com/parsa-epfl/cloudsuite/blob/master/docs/benchmarks/web-serving.md