fudanselab / train-ticket

Train Ticket - A Benchmark Microservice System

Train Ticket：A Benchmark Microservice System

The project is a train ticket booking system based on microservice architecture which contains 41 microservices. The programming languages and frameworks it used are as below.

• Java - Spring Boot, Spring Cloud
• Node.js - Express
• Python - Django
• Go - Webgo
• DB - Mongo、MySQL

You can get more details at Wiki Pages.

Service Architecture Graph

Quick Start

We provide k8s deployment to quickly deploy our application: Using Kubernetes.

Using Kubernetes

Here is the steps to deploy the Train Ticket onto any existing Kubernetes cluster.

Presequisite

• An existing Kubernetes cluster
• Helm supported, you can see https://helm.sh/docs/helm/helm_install/ for helm install
• PVC supported, you can see https://openebs.io/docs/2.12.x/user-guides/installation for localPV support.

1. Clone the Repository

git clone --depth=1 https://github.com/FudanSELab/train-ticket.git 
cd train-ticket/

2. Deploy the application

For Quick Start

make deploy

Note: if you want specify namespace, set Namespace paramter:

make deploy Namespace=yournamespace

Deploy Mysql Clusters For Each Services

make deploy DeployArgs="--independent-db"

With Moinitorig

make deploy DeployArgs="--with-monitoring"

With Distributed Tracing

make deploy DeployArgs="--with-tracing"

Deploy All

make deploy DeployArgs="--all"

Customise Deployment

You can freely combine parameters for custom deployment， for example, deploy with monitoring and tracing:

make deploy DeployArgs="--with-tracing --with-monitoring"

Reset Deployment

make reset-deploy
# if you specify namespace when deploy, set namespace as well when reset
# make reset-deploy Namespace=yournamespace

3. Run kubectl get pods to see pods are in a ready state

4. Visit the Train Ticket web page at http://[Node-IP]:32677.

Build From Source

In the above, We use pre-built images to quickly deploy the application.

If you want to build the application from source, you can refer to the Installation Guide.

Test scripts

Use scripts to test train-ticket: https://github.com/FudanSELab/train-ticket-auto-query

Screenshot

In order to know how to use the application, you can refer to the User Guide.

Communication

• FAQ
• Submit an issue
• Open a pull request

Information

• Release Note

Serverless Train Ticket

We have released a serverless version of Train Ticket.

• serverless-trainticket

Paper Reference

Bowen Li, Xin Peng, Qilin Xiang, Hanzhang Wang, Tao Xie, Jun Sun, Xuanzhe Liu.
Enjoy your observability: an industrial survey of microservice tracing and analysis
Empirical Software Engineering, Volume 27, 25, 2022.
Download:[PDF]

Chenxi Zhang, Xin Peng, Chaofeng Sha, Ke Zhang, Zhenqing Fu, Xiya Wu, Qingwei Lin, Dongmei Zhang
DeepTraLog: Trace-Log Combined Microservice Anomaly Detection through Graph-based Deep Learning
In Proceedings of the 44th International Conference on Software Engineering (ICSE 2022) , Pittsburgh, USA, May, 2022.
Download:[PDF]

Dewei Liu, Chuan He, Xin Peng, Fan Lin, Chenxi Zhang, Shengfang Gong, Ziang Li, Jiayu Ou, Zheshun Wu
MicroHECL: High-Efficient Root Cause Localization in Large-Scale Microservice Systems
In Proceedings of the 43rd IEEE/ACM International Conference on Software Engineering: Software Engineering in Practice (ICSE-SEIP 2021) , Madrid, Spain, May, 2021.
Download:[PDF]

Qilin Xiang, Xin Peng, Chuan He, Hanzhang Wang, Tao Xie, Dewei Liu, Gang Zhang, Yuanfang Cai
No Free Lunch: Microservice Practices Reconsidered in Industry
arXiv preprint arXiv:2106.07321, 2021.
Download:[PDF]

Xiaofeng Guo, Xin Peng, Hanzhang Wang, Wanxue Li, Huai Jiang, Dan Ding, Tao Xie, Liangfei Su
Graph-based trace analysis for microservice architecture understanding and problem diagnosis
In Proceedings of the 28th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE 2020) , Virtual Event, USA, November, 2020.
Download:[PDF]

Xiang Zhou, Xin Peng, Tao Xie, Jun Sun, Chao Ji, Dewei Liu, Qilin Xiang, and Chuan He.
Latent Error Prediction and Fault Localization for Microservice Applications by Learning from System Trace Logs.
In Proceedings of the 27th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE 2019) , Tallinn, Estonia, August 2019.
Download: [PDF] [BibTeX]

Xiang Zhou, Xin Peng, Tao Xie, Jun Sun, Chao Ji, Wenhai Li, and Dan Ding.
Fault Analysis and Debugging of Microservice Systems: Industrial Survey, Benchmark System, and Empirical Study.
IEEE Transactions on Software Engineering , To appear.
Download: [PDF]

Xiang Zhou, Xin Peng, Tao Xie, Jun Sun, Wenhai Li, Chao Ji, and Dan Ding.
Delta Debugging Microservice Systems.
In Proceedings of 33rd IEEE/ACM International Conference on Automated Software Engineering (ASE 2018) , Short Paper, Montpellier, France, September 2018.
Download: [PDF] [BibTeX]
An extended version to appear in IEEE Transactions on Services Computing.

Xiang Zhou, Xin Peng, Tao Xie, Jun Sun, Chenjie Xu, Chao Ji, and Wenyun Zhao.
Poster: Benchmarking Microservice Systems for Software Engineering Research.
In Proceedings of the 40th International Conference on Software Engineering (ICSE 2018) , Posters, Gothenburg, Sweden, May 2018.
Download: [PDF] [BibTeX]

train-ticket's Issues

你好，为什么购买的票的价格和订单里显示的价格不一样？

购买的是一等座的票：

订单列表显示的是二等座的价格：

购买的是二等座的票：

订单列表显示的是一等座的价格：

异常注入

您好，我们现在希望在trainticket系统中注入异常，在调用链中做根因定位。看到了您这边有一个fault的项目，每个异常都有单独的docker compose，而且每个异常的容器和trainticket的容器改动比较大（比如F1异常在ts-auth-service;ts-click-twice;ts-common;ts-launcher;ts-login;ts-payment,tsp-core,tsp-dashboard;ts-register;ts-sso;ts-ui-test;ts-user-service这十二个容器有差异）。我们现在希望并不是重新搭建一个新的系统，而是在原先的系统中做一些更改（比如通过istio重定向到异常容器，正常容器不变），即要在目前版本上复现应该改哪些地方?

Several MongoDB Connection Failures

Hello,
I deployed all train-ticket services with the default docker-compose file on a Ubuntu 18.04 LTS server. The application in general starts up correctly and I'm able to access the UI and perform actions. However, when I access the container logs via
docker container logs <CONTAINER_NAME>
I see two kinds of exception in several services.

1. MongoSocketOpenException (Example: admin-user-service)
   Relevant Log output:

2019-09-28 14:56:21.439  INFO [ts-admin-user-service,,,] 1 --- [localhost:27017] org.mongodb.driver.cluster               : Exception in monitor thread while connecting to server localhost:27017

com.mongodb.MongoSocketOpenException: Exception opening socket
	at com.mongodb.connection.SocketStream.open(SocketStream.java:63) ~[mongodb-driver-core-3.4.2.jar!/:na]
	at com.mongodb.connection.InternalStreamConnection.open(InternalStreamConnection.java:115) ~[mongodb-driver-core-3.4.2.jar!/:na]
	at com.mongodb.connection.DefaultServerMonitor$ServerMonitorRunnable.run(DefaultServerMonitor.java:113) ~[mongodb-driver-core-3.4.2.jar!/:na]
	at java.lang.Thread.run(Thread.java:745) [na:1.8.0_111]
Caused by: java.net.ConnectException: Connection refused (Connection refused)
	at java.net.PlainSocketImpl.socketConnect(Native Method) ~[na:1.8.0_111]
	at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350) ~[na:1.8.0_111]
	at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206) ~[na:1.8.0_111]
	at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188) ~[na:1.8.0_111]
	at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) ~[na:1.8.0_111]
	at java.net.Socket.connect(Socket.java:589) ~[na:1.8.0_111]
	at com.mongodb.connection.SocketStreamHelper.initialize(SocketStreamHelper.java:57) ~[mongodb-driver-core-3.4.2.jar!/:na]
	at com.mongodb.connection.SocketStream.open(SocketStream.java:58) ~[mongodb-driver-core-3.4.2.jar!/:na]
	... 3 common frames omitted

This might be related to the issue that docker containers have problems with connecting to localhost. However, any idea how to overcome this? Or a possibility to set address for MongoDB? The error repeats some times and then I get a notification for successful Tomcat startup:

2019-09-28 14:57:29.794 INFO [ts-admin-user-service,,,] 1 --- [ main] adminuser.AdminUserApplication : Started AdminUserApplication in 172.133 seconds (JVM running for 183.587)

2. MongoSocketException (Example: consign-service)
   Relevant Log Output:

2019-09-28 14:54:40.335  INFO [ts-consign-service,,,] 1 --- [ign-mongo:27017] org.mongodb.driver.cluster               : Exception in monitor thread while connecting to server ts-consign-mongo:27017

com.mongodb.MongoSocketException: ts-consign-mongo: Name or service not known
	at com.mongodb.ServerAddress.getSocketAddress(ServerAddress.java:188) ~[mongodb-driver-core-3.4.2.jar!/:na]
	at com.mongodb.connection.SocketStreamHelper.initialize(SocketStreamHelper.java:57) ~[mongodb-driver-core-3.4.2.jar!/:na]
	at com.mongodb.connection.SocketStream.open(SocketStream.java:58) ~[mongodb-driver-core-3.4.2.jar!/:na]
	at com.mongodb.connection.InternalStreamConnection.open(InternalStreamConnection.java:115) ~[mongodb-driver-core-3.4.2.jar!/:na]
	at com.mongodb.connection.DefaultServerMonitor$ServerMonitorRunnable.run(DefaultServerMonitor.java:113) ~[mongodb-driver-core-3.4.2.jar!/:na]
	at java.lang.Thread.run(Thread.java:745) [na:1.8.0_111]
Caused by: java.net.UnknownHostException: ts-consign-mongo: Name or service not known
	at java.net.Inet4AddressImpl.lookupAllHostAddr(Native Method) ~[na:1.8.0_111]
	at java.net.InetAddress$2.lookupAllHostAddr(InetAddress.java:928) ~[na:1.8.0_111]
	at java.net.InetAddress.getAddressesFromNameService(InetAddress.java:1323) ~[na:1.8.0_111]
	at java.net.InetAddress.getAllByName0(InetAddress.java:1276) ~[na:1.8.0_111]
	at java.net.InetAddress.getAllByName(InetAddress.java:1192) ~[na:1.8.0_111]
	at java.net.InetAddress.getAllByName(InetAddress.java:1126) ~[na:1.8.0_111]
	at java.net.InetAddress.getByName(InetAddress.java:1076) ~[na:1.8.0_111]
	at com.mongodb.ServerAddress.getSocketAddress(ServerAddress.java:186) ~[mongodb-driver-core-3.4.2.jar!/:na]
	... 5 common frames omitted

2019-09-28 14:54:40.882  INFO [ts-consign-service,,,] 1 --- [ign-mongo:27017] org.mongodb.driver.cluster               : Exception in monitor thread while connecting to server ts-consign-mongo:27017

com.mongodb.MongoSocketException: ts-consign-mongo
	at com.mongodb.ServerAddress.getSocketAddress(ServerAddress.java:188) ~[mongodb-driver-core-3.4.2.jar!/:na]
	at com.mongodb.connection.SocketStreamHelper.initialize(SocketStreamHelper.java:57) ~[mongodb-driver-core-3.4.2.jar!/:na]
	at com.mongodb.connection.SocketStream.open(SocketStream.java:58) ~[mongodb-driver-core-3.4.2.jar!/:na]
	at com.mongodb.connection.InternalStreamConnection.open(InternalStreamConnection.java:115) ~[mongodb-driver-core-3.4.2.jar!/:na]
	at com.mongodb.connection.DefaultServerMonitor$ServerMonitorRunnable.run(DefaultServerMonitor.java:113) ~[mongodb-driver-core-3.4.2.jar!/:na]
	at java.lang.Thread.run(Thread.java:745) [na:1.8.0_111]
Caused by: java.net.UnknownHostException: ts-consign-mongo
	at java.net.InetAddress.getAllByName0(InetAddress.java:1280) ~[na:1.8.0_111]
	at java.net.InetAddress.getAllByName(InetAddress.java:1192) ~[na:1.8.0_111]
	at java.net.InetAddress.getAllByName(InetAddress.java:1126) ~[na:1.8.0_111]
	at java.net.InetAddress.getByName(InetAddress.java:1076) ~[na:1.8.0_111]
	at com.mongodb.ServerAddress.getSocketAddress(ServerAddress.java:186) ~[mongodb-driver-core-3.4.2.jar!/:na]
	... 5 common frames omitted

In contrast to the first error, this error repeats many times and no correct startup is reported.

I am thankful for any help or advices!

Jaeger tracing enabled?

Hi,

Are the services instrumented with Jaeger tracing framework?
Thanks!

ERROR: Service 'ts-voucher-service' failed to build: OCI runtime create failed: container_linux.go

I have build the k8s enviroment,and mvn clean pacakge is ok, but when i run docker-compose build,it report:
ERROR: Service 'ts-voucher-service' failed to build: OCI runtime create failed: container_linux.go:348: starting container process caused "process_linux.go:402: container init caused \"process_linux.go:385: running prestart hook 0 caused \\\"fork/exec /usr/bin/dockerd;5ba4d87e (deleted): no such file or directory\\\"\"": unknown
And this is the information of my docker-compose

docker-compose version 1.18.0, build 8dd22a9
docker-py version: 2.6.1
CPython version: 3.4.9
OpenSSL version: OpenSSL 1.0.2k-fips  26 Jan 2017

Lack of validation for register email format

In the single micro-service test page, there is no validation for register email format, users may input the wrong email to register, such as "aaa@", expected "[email protected]".

cannot pass compile

when I compile via mvn -Dmaven.test.skip=true clean package,
the following errors were reported:
[ERROR] Failed to execute goal on project ts-login-service: Could not resolve dependencies for project fdse.microservice:ts-login-service:jar:1.0: The following artifacts could not be resolved: myproject.microservices:micro-service-monitoring-core:jar:1.2.2.BUILD-SNAPSHOT, myproject.microservices:micro-service-monitoring-zipkin:jar:1.2.2.BUILD-SNAPSHOT: Could not find artifact myproject.microservices:micro-service-monitoring-core:jar:1.2.2.BUILD-SNAPSHOT -> [Help 1]

It shows that the artifact micro-service-monitoring-core is not exists.

check the pom.xml

the micro-service-monitoring-core appears in dependency, but not in modules.

How to run test case?

I hope to run test case after deploy.
Can you describe more about run test case.

OOM ERROR in ts-inside-payment-service

K8S+istio部署环境下，一段时间运行后（14小时以内）inside_payment_service出现了OutOfMemery错误。

此时容器内存使用达到500M。

相关日志如下：

Exception in thread "http-nio-18673-exec-31" Exception in thread "http-nio-18673-exec-41" java.lang.OutOfMemoryError: Java heap space
	at java.nio.HeapByteBuffer.<init>(HeapByteBuffer.java:57)
	at java.nio.ByteBuffer.allocate(ByteBuffer.java:335)
	at org.apache.coyote.http11.Http11InputBuffer.init(Http11InputBuffer.java:683)
	at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:668)
	at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:66)
	at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:861)
	at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1455)
Exception: java.lang.OutOfMemoryError thrown from the UncaughtExceptionHandler in thread "http-nio-18673-exec-52"

	at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
	at java.lang.Thread.run(Thread.java:745)
Exception in thread "http-nio-18673-exec-37" java.lang.OutOfMemoryError: Java heap space

jmap -histo 部分结果如下：

 num     #instances         #bytes  class name
----------------------------------------------
   1:       1156307       63289032  [C
   2:       1154379       27705096  java.lang.String
   3:        621267       24850680  java.util.LinkedHashMap$Entry
   4:          7328       16704240  [B
   5:        128122       10323936  [Ljava.util.HashMap$Node;
   6:        119617        7655488  com.mongodb.BasicDBObject
   7:         18863        1659944  java.lang.reflect.Method
   8:         12611        1340000  [Ljava.lang.Object;
   9:         38358        1227456  java.util.concurrent.ConcurrentHashMap$Node
  10:         36276        1160832  inside_payment.entity.Payment
  11:         10342        1143520  java.lang.Class
  12:         25793         825376  java.util.HashMap$Node
  13:          5246         789208  [I
  14:         11537         646072  java.util.LinkedHashMap
  15:         17771         568672  java.lang.ref.WeakReference
  16:         28535         456560  java.lang.Object
  17:           339         437152  [Ljava.util.concurrent.ConcurrentHashMap$Node;
  18:          8304         332160  java.lang.ref.SoftReference
  19:         12434         298416  org.springframework.security.access.method.DelegatingMethodSecurityMetadataSource$DefaultCacheKey
  20:         11473         247936  [Ljava.lang.Class;
  21:         10298         247152  java.beans.MethodRef
  22:          4774         229152  org.apache.tomcat.util.buf.ByteChunk
  23:          7146         226752  [Ljava.lang.String;
  24:          4512         216576  org.apache.tomcat.util.buf.CharChunk
  25:          4460         214080  org.apache.tomcat.util.buf.MessageBytes
  26:          8253         198072  java.util.ArrayList
  27:          3414         191184  java.beans.MethodDescriptor

consider setting COMPOSE_HTTP_TIMEOUT to a higher value

Hi,chao,
there is a problem when i try the Clustering runtime environment(docker swarm) step:
when i try:
docker-compose up
it reported:
ERROR: An HTTP request took too long to complete. Retry with --verbose to obtain debug information. If you encounter this issue regularly because of slow network conditions, consider setting COMPOSE_HTTP_TIMEOUT to a higher value (current value: 60).
And i try to:
COMPOSE_HTTP_TIMEOUT=200 docker-compose up
But it doesn't work and report:
If you encounter this issue regularly because of slow network conditions, consider setting COMPOSE_HTTP_TIMEOUT to a higher value (current value: 200).

为什么访问特别慢？

Which version of Kubernetes do you support? Seems like 1.16 has some problem with deploying the services.

I got this error message when executing kubectl apply -f ts-deployment-part1.yml:

unable to recognize "ts-deployment-part1.yml": no matches for kind "Deployment" in version "extensions/v1beta1"

It seems like deprecated API issues. See https://github.com/kubernetes/minikube/issues/5420.

By the way, what do you mean by "Move "ts-deployment-part1.yml", "ts-deployment-part2.yml", "ts-deployment-part3.yml"."

Thank you!

An error occurred in ts-travel2-service with mongodb connection

Are there public image for the project in docker hub

I trid to deploy the project and found that you build a private image repositories in your k8s, I can not find any related images in docker hub either using the image in "docker-compose.yml". And I want to ask if there are public images of the project. Or I need to build all images by the dockerfile?

Register service post request missing cookie.

In register service, when we have inputted the user's information, it will send a post request. However, the request has missed the cookie. Error screenshot as the below picture:

Route service without validation for distance when update the route.

In route service, when we update the route without the error input for distance such as "0,aaa", there will be an exception: "java.lang.NumberFormatException: For input string: "aaa"".

Lack of status message when log in and finish in flow four.

In flow four, there is no status message when we log in and finish this flow, so we can't get the status of the operations.

An error occurred in ts-verification-code-service.

When using the system, it can't load the verify code. The tracing data get the error message as: Request processing failed; nested exception is org.springframework.dao.DataAccessResourceFailureException: Timed out after 30000 ms while waiting for a server that matches WritableServerSelector. Client view of cluster state is {type=UNKNOWN, servers=[{address=ts-verification-code-mongo:27017, type=UNKNOWN, state=CONNECTING, exception={com.mongodb.MongoSocketOpenException: Exception opening socket}, caused by {java.net.ConnectException: Connection refused (Connection refused)}}]; nested exception is com.mongodb.MongoTimeoutException: Timed out after 30000 ms while waiting for a server that matches WritableServerSelector. Client view of cluster state is {type=UNKNOWN, servers=[{address=ts-verification-code-mongo:27017, type=UNKNOWN, state=CONNECTING, exception={com.mongodb.MongoSocketOpenException: Exception opening socket}, caused by {java.net.ConnectException: Connection refused (Connection refused)}}]

ts-seat-service exception: 404 not found

When request the seat distribution of normal train(not started with D or G), the system encountered an exception: 404 not found.

fault inject

您好，我是清华裴丹老师的同学，我们现在的一些实验数据是基于tratinticket系统的，前几天裴老师和您这边彭老师有过交流，可能有些问题需要您这边的一些支持。
我们在集群上搭建了tratinticket系统，并且搭建了监控系统收集了日志/调用链等数据，之后我们尝试了故障注入：包括istio中的abort/delay故障，还有stress-ng注入了cpu故障。现在有几个问题想请教一下：
1.我们也尝试了stress进行内存故障的注入，但是发现如果内存指标不超过一个阈值则无效果，超过阈值则pod会杀死，所以没有效果。内存故障注入方面您这边有什么方法建议么？
2.我们还希望能够收集到更多的故障种类。看到您这边基于之前的系统实现了很多故障注入，请问一下有计划在现在的系统上复现几种吗？

设计文档

请问有关于每个微服务的描述和设计文档吗

调用链分析

您好，我搭建了这个trainticket项目，并且收集了调用链的数据，发现存在非常多的服务存在自调用的情况，请问从程序设计角度来看这种原因是什么呢？

Easy way to disable security

I am trying to see if I can write a gatling script to stress the application by sending sequence of requests to targeted URL's

For example, if I want to create a new contact, I want to be able to send a POST request to the below URL:
http://<hostname>:12347/api/v1/contactservice/contacts

As of now, almost all of the endpoints in the application are secured which does not allow its access directly from external source.

I am currently working around that problem by individually opening SecurityConfig.java files in all of the services and commenting out the security feature in the below method:

protected void configure(HttpSecurity httpSecurity) throws Exception {
.
.
.
}

I want to know if there is an easier way to do the same ?
Is there anything that I can add in the application.yml file to achieve the same ?

Is there an alternate (recommended/best practice) method that I can use to stress the application from other tools/libraries such as gatling/locust/wrk ?

关于搜索停靠车站数最少的车次的问题

你好：
RoutePlanServiceImpl.java的searchMinStopStations方法的：

    for(int i = 0; i < Math.min(info.getNum(),routeList.size()); i++){
        int minIndex = 0;
        int tempMinGap = Integer.MAX_VALUE;
        for(int j = 0; j < gapList.size(); j++){
            if(gapList.get(j) < tempMinGap){
                tempMinGap = gapList.get(j);
                minIndex = j;
            }
        }
        resultRoutes.add(routeList.get(minIndex).getId());
        routeList.remove(minIndex);
    }

routeList.remove(minIndex);这一行是不是有点问题，例如：从南京到上海有4条路线，一开始的时候i是0，info.getNum()固定是5，routeList.size()是4，第一次循环以后，i是1，routeList.size()是3，第二次循环以后，i是2，routeList.size()是2，退出循环，只查到2条路线。搜索结果如图：

ts-preserve-service访问ts-notification-service报403错误

你好，我在0.01版本的tracin-ticket中使用了istio-1.0.0方式部署，使用TrainTicketTestCases的bookingflow时，ts-preserve-service访问ts-notification-service报403错误，日志如下：
P.S.我在几个月之前使用同样版本的Train-Ticket和Istio部署时没有出现过这个问题。

[Preserve Service][Preserve] Account 4d2a46c7-71cb-4cf1-b5bb-b68406d9da6f order from Shang Hai -----> Su Zhou at Fri Sep 20 16:00:00 UTC 2019
[Preserve Service][Preserve] Account 4d2a46c7-71cb-4cf1-b5bb-b68406d9da6f order from Shang Hai -----> Su Zhou at Fri Sep 20 16:00:00 UTC 2019
[Preserve Other Service][Verify Login] Verifying....
[Preserve Other Service][Verify Login] Verifying....
[Preserve Service][Verify Login] Success
[Preserve Service] [Step 1] Check Security
[Preserve Other Service][Check Security] Checking....
[Preserve Service][Verify Login] Fail
[Preserve Service] [Step 1] Check Security Complete
[Preserve Service] [Step 2] Find contacts
[Preserve Service] [Step 2] Contacts Id:4d2a46c7-71cb-4cf1-a5bb-b68406d9da6f
[Preserve Other Service][Get Contacts By Id] Getting....
[Preserve Service][Step 2] Complete
[Preserve Service] [Step 3] Check tickets num
[Preserve Service] [Step 3] TripId:D1345
[Preserve Other Service][Get Trip All Detail Information] Getting....
[Preserve Service] [Step 3] Tickets Enough
[Preserve Service] [Step 4] Do Order
[Preserve Other Service][Get Station Name]
[Preserve Other Service][Get Station Name]
[Preserve Service][Order] Order Travel Date:Fri Sep 20 16:00:00 UTC 2019
[Preserve Service][Order Price] Price is: 50.0
[Preserve Other Service][Get Contacts By Id] Creating....
[Preserve Service] [Step 4] Do Order Complete
[Preserve Service][Add Assurance For Order]
[Preserve Service][Step 5] Buy Assurance Success
[Preserve Service][Add food Order] Creating....
[Preserve Service][Step 6] Buy Food Success
[Preserve Service][Add Condign] Creating....
[Preserve Service][Step 7] Consign Success
[Preserve Service]
[Cancel Order Service][Get By Id]
[Preserve Service][Send Email]
2019-09-20 08:27:12.696 ERROR 1 --- [io-14568-exec-2] o.a.c.c.C.[.[.[/].[dispatcherServlet]    : Servlet.service() for servlet [dispatcherServlet] in context with path [] threw exception [Request processing failed; nested exception is org.springframework.web.client.HttpClientErrorException: 403 Forbidden] with root cause

org.springframework.web.client.HttpClientErrorException: 403 Forbidden
	at org.springframework.web.client.DefaultResponseErrorHandler.handleError(DefaultResponseErrorHandler.java:63) ~[spring-web-4.3.8.RELEASE.jar!/:4.3.8.RELEASE]
	at org.springframework.web.client.RestTemplate.handleResponse(RestTemplate.java:700) ~[spring-web-4.3.8.RELEASE.jar!/:4.3.8.RELEASE]
	at org.springframework.web.client.RestTemplate.doExecute(RestTemplate.java:653) ~[spring-web-4.3.8.RELEASE.jar!/:4.3.8.RELEASE]
	at org.springframework.web.client.RestTemplate.execute(RestTemplate.java:613) ~[spring-web-4.3.8.RELEASE.jar!/:4.3.8.RELEASE]
	at org.springframework.web.client.RestTemplate.exchange(RestTemplate.java:531) ~[spring-web-4.3.8.RELEASE.jar!/:4.3.8.RELEASE]
	at preserve.service.PreserveServiceImpl.sendEmail(PreserveServiceImpl.java:292) ~[classes!/:1.0]
	at preserve.service.PreserveServiceImpl.preserve(PreserveServiceImpl.java:256) ~[classes!/:1.0]
	at preserve.controller.PreserveController.preserve(PreserveController.java:26) ~[classes!/:1.0]
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_111]
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_111]
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_111]
	at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_111]
	at org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:205) ~[spring-web-4.3.8.RELEASE.jar!/:4.3.8.RELEASE]
	at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:133) ~[spring-web-4.3.8.RELEASE.jar!/:4.3.8.RELEASE]
	at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:97) ~[spring-webmvc-4.3.8.RELEASE.jar!/:4.3.8.RELEASE]
	at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:827) ~[spring-webmvc-4.3.8.RELEASE.jar!/:4.3.8.RELEASE]
	at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:738) ~[spring-webmvc-4.3.8.RELEASE.jar!/:4.3.8.RELEASE]
	at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:85) ~[spring-webmvc-4.3.8.RELEASE.jar!/:4.3.8.RELEASE]
	at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:963) ~[spring-webmvc-4.3.8.RELEASE.jar!/:4.3.8.RELEASE]
	at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:897) ~[spring-webmvc-4.3.8.RELEASE.jar!/:4.3.8.RELEASE]
	at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:970) ~[spring-webmvc-4.3.8.RELEASE.jar!/:4.3.8.RELEASE]
	at org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.java:872) ~[spring-webmvc-4.3.8.RELEASE.jar!/:4.3.8.RELEASE]
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:661) ~[tomcat-embed-core-8.5.14.jar!/:8.5.14]
	at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:846) ~[spring-webmvc-4.3.8.RELEASE.jar!/:4.3.8.RELEASE]
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:742) ~[tomcat-embed-core-8.5.14.jar!/:8.5.14]
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:231) ~[tomcat-embed-core-8.5.14.jar!/:8.5.14]
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) ~[tomcat-embed-core-8.5.14.jar!/:8.5.14]
	at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52) ~[tomcat-embed-websocket-8.5.14.jar!/:8.5.14]
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) ~[tomcat-embed-core-8.5.14.jar!/:8.5.14]
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) ~[tomcat-embed-core-8.5.14.jar!/:8.5.14]
	at org.springframework.boot.web.filter.ApplicationContextHeaderFilter.doFilterInternal(ApplicationContextHeaderFilter.java:55) ~[spring-boot-1.5.3.RELEASE.jar!/:1.5.3.RELEASE]
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) ~[spring-web-4.3.8.RELEASE.jar!/:4.3.8.RELEASE]
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) ~[tomcat-embed-core-8.5.14.jar!/:8.5.14]
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) ~[tomcat-embed-core-8.5.14.jar!/:8.5.14]
	at org.springframework.boot.actuate.trace.WebRequestTraceFilter.doFilterInternal(WebRequestTraceFilter.java:110) ~[spring-boot-actuator-1.5.3.RELEASE.jar!/:1.5.3.RELEASE]
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) ~[spring-web-4.3.8.RELEASE.jar!/:4.3.8.RELEASE]
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) ~[tomcat-embed-core-8.5.14.jar!/:8.5.14]
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) ~[tomcat-embed-core-8.5.14.jar!/:8.5.14]
	at org.springframework.web.filter.RequestContextFilter.doFilterInternal(RequestContextFilter.java:99) ~[spring-web-4.3.8.RELEASE.jar!/:4.3.8.RELEASE]
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) ~[spring-web-4.3.8.RELEASE.jar!/:4.3.8.RELEASE]
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) ~[tomcat-embed-core-8.5.14.jar!/:8.5.14]
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) ~[tomcat-embed-core-8.5.14.jar!/:8.5.14]
	at org.springframework.web.filter.HttpPutFormContentFilter.doFilterInternal(HttpPutFormContentFilter.java:105) ~[spring-web-4.3.8.RELEASE.jar!/:4.3.8.RELEASE]
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) ~[spring-web-4.3.8.RELEASE.jar!/:4.3.8.RELEASE]
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) ~[tomcat-embed-core-8.5.14.jar!/:8.5.14]
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) ~[tomcat-embed-core-8.5.14.jar!/:8.5.14]
	at org.springframework.web.filter.HiddenHttpMethodFilter.doFilterInternal(HiddenHttpMethodFilter.java:81) ~[spring-web-4.3.8.RELEASE.jar!/:4.3.8.RELEASE]
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) ~[spring-web-4.3.8.RELEASE.jar!/:4.3.8.RELEASE]
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) ~[tomcat-embed-core-8.5.14.jar!/:8.5.14]
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) ~[tomcat-embed-core-8.5.14.jar!/:8.5.14]
	at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:197) ~[spring-web-4.3.8.RELEASE.jar!/:4.3.8.RELEASE]
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) ~[spring-web-4.3.8.RELEASE.jar!/:4.3.8.RELEASE]
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) ~[tomcat-embed-core-8.5.14.jar!/:8.5.14]
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) ~[tomcat-embed-core-8.5.14.jar!/:8.5.14]
	at org.springframework.boot.actuate.autoconfigure.MetricsFilter.doFilterInternal(MetricsFilter.java:106) ~[spring-boot-actuator-1.5.3.RELEASE.jar!/:1.5.3.RELEASE]
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) ~[spring-web-4.3.8.RELEASE.jar!/:4.3.8.RELEASE]
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) ~[tomcat-embed-core-8.5.14.jar!/:8.5.14]
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) ~[tomcat-embed-core-8.5.14.jar!/:8.5.14]
	at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:198) ~[tomcat-embed-core-8.5.14.jar!/:8.5.14]
	at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:96) [tomcat-embed-core-8.5.14.jar!/:8.5.14]
	at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:478) [tomcat-embed-core-8.5.14.jar!/:8.5.14]
	at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:140) [tomcat-embed-core-8.5.14.jar!/:8.5.14]
	at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:80) [tomcat-embed-core-8.5.14.jar!/:8.5.14]
	at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:87) [tomcat-embed-core-8.5.14.jar!/:8.5.14]
	at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:342) [tomcat-embed-core-8.5.14.jar!/:8.5.14]
	at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:799) [tomcat-embed-core-8.5.14.jar!/:8.5.14]
	at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:66) [tomcat-embed-core-8.5.14.jar!/:8.5.14]
	at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:861) [tomcat-embed-core-8.5.14.jar!/:8.5.14]
	at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1455) [tomcat-embed-core-8.5.14.jar!/:8.5.14]
	at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49) [tomcat-embed-core-8.5.14.jar!/:8.5.14]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [na:1.8.0_111]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [na:1.8.0_111]
	at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) [tomcat-embed-core-8.5.14.jar!/:8.5.14]
	at java.lang.Thread.run(Thread.java:745) [na:1.8.0_111]

Service 'ts-auth-service' failed to build: ADD failed

trainticket-gateway配置错误

deployment/k8s-deployment-v0.0.2/trainticket-gateway.yaml中一些service的uri prefix与ts-ui-dashboard中配置的不符。
比如dashboard里travel service的uri prefix是/travel，而gateway中配置的却是/api/v1/travelservice。

我猜你们是想统一后端服务的接口，但ts-ui-dashboard的配置还没改过来？

构建python3镜像问题

Lost connection to MySQL Server during query

Describe the bug
When running Train-Ticket system on K8S with istio, ts-voucher-service attempt a connection to ts-voucher-mysql, but the connection fails and the ts-voucher-service pod keeps restarting. Following is the error log of the ts-voucher-service pod.

Expected behavior
The connection should succeed and all pods are running.

Steps to reproduce the bug
Follow the install instructions in Wiki:

1. Build docker image
2. Upload docker image to my own registry
3. Deploy the system on K8S with istio in an auto-injection namespace

Then the pod crash.

Version

$ kubectl version
Client Version: version.Info{Major:"1", Minor:"12", GitVersion:"v1.12.2", GitCommit:"17c77c7898218073f14c8d573582e8d2313dc740", GitTreeState:"clean", BuildDate:"2018-10-30T21:39:16Z", GoVersion:"go1.11.1", Compiler:"gc", Platform:"darwin/amd64"}
Server Version: version.Info{Major:"1", Minor:"12", GitVersion:"v1.12.1", GitCommit:"4ed3216f3ec431b140b1d899130a69fc671678f4", GitTreeState:"clean", BuildDate:"2018-10-05T16:36:14Z", GoVersion:"go1.10.4", Compiler:"gc", Platform:"linux/amd64"}

$ istioctl version
client version: 1.2.0
citadel version: 1.2.0
galley version: 1.2.0
ingressgateway version: 1.2.0
pilot version: 1.2.0
policy version: 1.2.0
sidecar-injector version: 1.2.0
telemetry version: 1.2.0

Installation
Customizable Install with Helm

k8s部署方案下添加用户时出错

使用admin用户登录后尝试创建新的用户，然而提交后报了error code 500

trainticket发现的几个bug

您好，我们在自己集群上搭建了trainticket并且部署了zipkin+prometheus监控调用链数据，并通过istio向服务注入delay和abort的故障。在向29个服务分别注入了故障并分析获取的监控数据后我们发现了三个问题，跟您反馈一下：
1.我们发现系统中有大量的服务自调用，这些自环数量很多，且很严重的问题是，我们分析zipkin收集到的span数据，发现部分自环是伴随着其他调用一起形成trace，而有的自环单独就是一个trace。比如：ts-food-service->ts-food-service就是一个trace，而这个traceid上无其他任何调用了，这肯定是有问题的，不知道是代码设计的问题还是zipkin的问题。
2.我们通过istio向多个服务分别注入了delay故障（服务是被调用方），绝大多数svc都是会有延迟的，唯独istio-ingressgateway作为调用方的时候会出现奇怪的现象：即会有接近一半的span出现了delay，一半的span没有被delay。比如：我们向food服务通过istio注入delay故障，istio-ingressgateway—>food的所有span中latency一半被delay一半没有delay，这个原因是什么呢？
3.我们向指定的服务注入故障后，发现有些trace中完全没有出现故障的服务，但是也会发生错误，而正常时段是没有错误的。比如：我们向travel2注入了abort的故障（返回503），确实所有经过travel2的trace都有故障，可是也有部分trace根本就没有出现travel2但是返回码也出现了503。我们也不清楚这个会是代码设计的问题？还是zipkin没有收集全？
以上是我们在分析数据时发现的问题，您这边对trainticket更加熟悉，对这三个问题的看法会大大帮助我们分析，还希望能和您们一起探讨。

ts-voucher-service error: 1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax

When try to up the system, voucher service encounter the problem: 1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax. However, if I use the exec command to enter the voucher sql container and then execute the sql, it created the table successfully.

数据可持久化

现在的trainticket似乎没有做数据持久化的方案，如果一些容器挂掉重启后就会导致整个系统无法继续正常运行，希望能针对这个问题做一些相关优化

ts-preserve-service用sonarqube检测出的问题

在该微服务下的FoodOrder.java和OrderTicketInfo.java所涉及的两个类中存在类方法的重复
具体来说是ts-preserve-service/src/main/java/preserve/entity/FoodOrder.java的43-73行与ts-preserve-service/src/main/java/preserve/entity/OrderTicketInfo.java的57-87行所涉及的方法定义有重复

Error response from daemon: cgroup-parent for systemd cgroup should be a valid slice named as "xxx.slice?

Hi,chao,thanks for your help,some sevices in the k8s has run.But other are not .such like this:

I thought "ts-ui-dashboard-85f4f7dcbc-nrhz7" is the notePort type,but it havn't run and i try this:
kubectl describe pod ts-ui-dashboard-85f4f7dcbc-nrhz7
I find the error information:
Error response from daemon: cgroup-parent for systemd cgroup should be a valid slice named as "xxx.slice"
The other pods failed to start has the same problem.
Thanks!

请问对机器资源的要求

请问k8s+istio的部署方案需要多少机器资源？CPU，MEM和DISK需要多少才能稳定运行？

Inside Payement Get Cookie Error.

When I Click Inside Payment Service "Pay" button, there is no response and the backend console reports a java.lang.NullPointerException.
After analyzing the source code, I found that in ts-inside-payment-service, InsidePaymentServiceImpl try to get cookie named "loginId" at line 39, but it failed to get that cookie. Then invoke getValue() method, I think this is why cause the java.lang.NullPointerException.

关于zipkin的小问题

我注意到这个项目里似乎有许多使用zipkin的痕迹[捂脸]，请问这个项目是支持k8s+istio环境下使用zipkin收集trace数据的吗？如果支持，可以告知操作方法吗？（我自己使用istio自带的zipkin似乎并不起作用，是还要修改什么配置吗？）

KPIs' meanings

Hello All,

Sorry, this is not an 'issue' but a question for discussion :)
In Prometheus, what do these KPIs mean? Is there a general way to figure out their functionalities to some extent?

开源故障数据集

请问你们团队会考虑开源一些关于微服务故障中的数据集吗？

Docker-compose 部署错误

请问我按wiki教程部署的时候报了如下错误，是方式不对吗？
ERROR: Service 'ts-auth-service' failed to build: ADD failed: stat /var/lib/docker/tmp/docker-builder284038967/target/ts-auth-service-1.0.jar: no such file or directory

How to deploy this project in Kubernetes?

Is there any easy way to deploy the whole project on Kubernetes?
Such as the Sock-shop: https://github.com/microservices-demo/microservices-demo/blob/master/deploy/kubernetes/complete-demo.yaml

请问一下程序的日志文件在哪个文件夹下

ts-order-service用sonarqube检测出的问题

在ts-order-service/src/main/java/order/service/OrderServicelmpl.java中存在代码重复的情况，具体有60-73与375-387重复，197-216与398-415重复

ts-travel-service用sonarqube检测出的问题

在该微服务下也存在多处明显的代码重复情况

1. ts-travel-service/src/main/java/travel/init/InitData.java中自21行起存在大片重复
2. ts-travel-service/src/main/java/travel/entity/TravelInfo.java 的35-89行与ts-travel-service/src/main/java/travel/entity/TravelInfo.java的68-122行存在方法上的重复
3. ts-travel-service/src/main/java/travel/entity/TripInfo.java的25-48行与ts-travel-service/src/main/java/travel/entity/Trip.java 有方法上的重复

测试系统性能

请问程序内部有提供程序的性能指标API么

Invoke Query-Basic-Information Error

When I click Query-Basic-Information Button, there is no response and the backend console reports a java.lang.NullPointerException at InsidePaymentServiceImpl line 87.
After analyzing the source code, in ts-basic-service, I found that there is an remote method invoke at line 82 to get specific "Route" in InsidePaymentServiceImpl, but there no validation for "Route". If the RMI is failed, it will cause java.lang.NullPointerException at line 87. I also suggest that add some fallback mechanism (such as hystrix fallback method) once RMI is failed for some unknown reason.

docker服务访问

请问我在本地虚拟机中启动了所有的服务，我在外部的宿主主机里如何访问docker服务？

client_order_list.html刷新时数据有时会重复

刷新client_order_list.html时，数据有时会重复出现：

有时又不会：

Is this project containts CircuitBreaker implimentation ?

Is this project containts CircuitBreaker implimentation ?
Described in https://martinfowler.com/bliki/CircuitBreaker.html