The yuan's discuss from no-trade-no-life

feat: add tick composer to compose ticks into klines

能否提供一个盘中交易策略模型案例

案例中提供的模型都是当前K线结束后交易（close.length-1）,如下，

// NOTE: 使用当前 K 线的上一根 K 线的收盘价，保证策略在 K 线结束时才会执行
  const idx = close.length - 2;
  // 设置仓位管理器
  const pL = useSinglePosition(product_id, PositionVariant.LONG);
  const pS = useSinglePosition(product_id, PositionVariant.SHORT);

  useEffect(() => {
    if (idx < N) return; // 略过一开始不成熟的均线数据

    if (close[idx] > Upper[idx]) {
      pL.setTargetVolume(1);
      pS.setTargetVolume(0);
    }
    if (close[idx] < Lower[idx]) {
      pL.setTargetVolume(0);
      pS.setTargetVolume(1);
    }
  }, [idx]);

能否提供一个盘中交易的模型案例

chore: manifest deployment fully embrace operator

feat: make sure trade copier send order with position id filled

for those exchanges that distinguish position IDs, orders should be sent with the position ID filled to reduce the complexity of vendor

feat: Consistent Data Record UI

Having a base UI solution for data view & manipulation.

product
pull_source_relation
general_specific_relation
and so on

This UI has the following features:

basic CRUD UI: create, list, reload, filter, search, edit, delete, batch delete, import, export.
Show table in big screen and list in small screen responsively.
Link to other data, e.g. by foreign key.
Custom actions depend on business.

We can rapidly build a complete and consistent UI for new data record types.

build: use package's semver as image tag when releasing npm package

Previously, we used to be a closed-source software with a mono-repo structure and did not release npm packages. As for the docker images, we needed stable image tags, meaning the tag of an image should not change if the content of the corresponding package did not change. Therefore, we used to calculate the git tracked files' hash as the docker image tag.

However, now that Yuan is an open source software and has a good npm release routine, it is time to move forward and use the package's semver as the docker image tag, if available.

feat: Terminal Load Balance

The server-side terminal should provide a JSON schema for its service.
The client-side terminal should choose one target terminal to process the request.

Why:

we can easily horizontally extend the service.
We can deploy in different regions and keep working if anyone is down.
We can remove the hard-code MongoDB terminal_id

build: enable build cache

Describe the feature

Enable rush build cache to speed up CI build

Why do you need this feature?

No response

Additional context

No response

feat: storage terminal sub/pub support

chore: add renovatebot or dependabot and enable CI

fix: editor view import statemens errors

feat: support User settings in FS

It is crucial that we store the user's settings in the FileSystem to ensure the safety and accessibility of their data.

By doing so, we can easily sync the settings using OneDrive or any other similar tools.

Currently, we are storing some settings in IndexedDB, which unfortunately can be cleaned for various reasons, resulting in the loss of important data.

That's why it's essential to store certain settings, such as the "host config," in the FileSystem, as it provides a more reliable and secure option for storing and accessing user data.

build: group CI steps into jobs

Currently, we have only one CI workflow with one job for rush build docker image build & publish.

Soon, we will add npm publish into the CI workflow, which is not relevant to the docker build, thus, we should separate these steps into different jobs so that the CI workflow would be running in parallel.

Foreseeably, we shall have three jobs:

rush build & deploy
Dockerbuild and push
npm publish

Job 2 and job 3 can be running in parallel.

refactor: market data collector for higher successful rate

Describe the feature

Currently, the App market-data-collector is using Cronjobs with retries to schedule its sync tasks. It is working fine when dealing with no more than 200 different jobs.

However, the failure rate tends to increase as more new jobs are added from this point onwards.

The scheduling mechanism is not robust enough to support more jobs, thus a refactor is required to resolve the issue.

Why do you need this feature?

No response

Additional context

No response

feat: make Observable class internal

Describe the feature

Check all libraries's exporting API:

Remove all Observable<T> and replace them with AsyncIterable<T>
Remove all Subject<T> and replace them with NativeSubject<T> = AsyncIterable<T> | AsyncIterator<T, void, T>.
Remove all ObservableInput<T> and replace them with AsyncIterable<T> | PromiseLike<T> | ArrayLike<T> | Iterable<T>

Hint: All Observable<T> can be transformed to AsyncIterable<T>, and back to Observable<T>.

Why do you need this feature?

It might cause compatibility issues.

Confusing when someone is new to development.

Additional context

Observable of rxjs is useful for defining a stream interface.
But, the issue is that when we export an interface with Observable, it will bind the version of rxjs. (because Observable is a class rather than an interface)
We met a compiling error: Observable cannot be assigned to another Observable from a different version of rxjs.

The solutions are:

Keep all related rxjs in the same version. (Hard to ensure when code dispersed in multiple repos)
Make rxjs as a peer-dependency. (Peer dependency has many unresolved issues)
Ensure all Observables are used inside the package, and never export them as Param and Return types.

To introduce the short rule: Never export a class type

In JavaScript & node modules cache & npm:

Class is a named type, not a structured type (duck type), with a more strict compatible issue.
Node.js distinct JS files by its path.
You can install different versions of one node package, but they will be installed in different paths.

So, exporting a class type might cause compatibility issues.

feat: Globalize Product ID

Describe the feature

Make all product ID unique for global.

Embed datasource_id into product_id field.

#707
Remove datasource_id in Product
Remove datasource_id in Period
Remove datasource_id in PSR
Remove datasource_id in GSR
Remove datasource_id in AccountInfo's positions
Remove datasource_id in FundingRate
Remove datasource_id in Tick

Why do you need this feature?

We can view all products in a tree view
Easily compose accounts
Remove the 'datasource' concept (make it simpler)
Position can link to a certain product

Additional context

No response

build: create a github actions workflow to publish packages to npm

Since the rush publish process changes git repo with the -a option (for version bump), to publish npm packages in CI, we instead always execute rush publish without the -a option. Thus, a manually triggered workflow for just npm publish is not needed,

refactor: extract a new lib for list config pattern

主图指标显示在k线的最左边

Describe the bug

To Reproduce

运行有主图指标的策略即可。

Expected behavior

No response

Additional context

No response

feat: support Notebook UI

If you're familiar with Jupyter Notebook, you know how useful it is for managing research. But what if I told you there's an even better way? Imagine being able to effortlessly create a notebook that only highlights the key points you care about. That's exactly what we're offering. With our web GUI, you can easily create code boxes and render various charts, tables, and more, making it a breeze to create your own version of Jupyter Notebook. Don't waste time clicking around to find what you need. Simplify your research management with our innovative solution today.

doc: wiki for each jargon in UI

and make links for these jargons

bug: 订单列表中时间显示不准确。

Describe the bug

To Reproduce

运行策略触发订单即可出现。

Expected behavior

No response

Additional context

No response

feat: provide package templates

feat: support redis as a cache to mongodb

feat: migrate docker bake tool to empower apps build

feat: Kernel State Dump & Restore

Many features benefit from this feature:

A kernel's state can be dumped and restored.

We can inspect a remote kernel by dumping the state and restoring it locally.
We can make the data-checking logic more efficient.
We can make the indicator-monitor scene more efficient.

TODO:

Kernel supports methods dump and restore.
Every unit in the kernel supports methods dump and restore.
Kernel terminal exposes an interface method dump for other terminals.
Kernel support re-starting and incremental mode.

参数优化这块，如果能展示参数-盈利的多维分布图，可以更方便识别参数的峰值和平原区，便于挑选合适的参数。

Describe the feature

参数优化这块，如果能展示参数-盈利的多维分布图，可以更方便识别参数的峰值和平原区，便于挑选合适的参数。

Why do you need this feature?

No response

Additional context

No response

feat: Terminal support dispose

Terminals now hold resources like IConnection, which is implemented using rxjs Subject and Observables, when not dispose properly, it may cause memory leak.

Benefits:

Shared Terminal can dispose terminal when host deleted.
GUI can change hosts without refreshing the pages. (no need to re-grant the workspace permission)

Work Items:

Add a dispose method to Terminal. It disposes connection and unsubscribes all backend subscriptions.

如何计算连个序列的差值序列

已通过如下方式获取到high和low序列，如何计算high-low的差值序列
const { product_id, open, high, low, close } = useParamOHLC("SomeKey");

feat: using length -2 as the current K-line index can be very confusing for the user

// 双均线策略 (Double Moving Average)
// 当短期均线由下向上穿越长期均线时做多 (金叉)
// 当短期均线由上向下穿越长期均线时做空 (死叉)
import { useSMA, useSimplePositionManager } from "@libs";

export default () => {
  // 使用收盘价序列
  const { product_id, close } = useParamOHLC("SomeKey");
  // NOTE: 使用当前 K 线的上一根 K 线的收盘价，保证策略在 K 线结束时才会执行
  const idx = close.length - 2;

  // 使用 20，60 均线
  const sma20 = useSMA(close, 20);
  const sma60 = useSMA(close, 60);

  const accountInfo = useAccountInfo();

  // 设置仓位管理器
  const [targetVolume, setTargetVolume] = useSimplePositionManager(
    accountInfo.account_id,
    product_id
  );

  useEffect(() => {
    if (idx < 60) return; // 略过一开始不成熟的均线数据
    // 金叉开多平空
    if (sma20[idx] > sma60[idx]) {
      setTargetVolume(1);
    }
    // 死叉开空平多
    if (sma20[idx] < sma60[idx]) {
      setTargetVolume(-1);
    }
  }, [idx]);
};