[Feature]How to load multiple models at the same time on startup? Does the model now support persistence? about db-gpt

[Aries-ckt]

1. you can update local manager mode in manager.py to support multi llm start.

    if run_locally:
        # TODO start ModelController
        worker_params.standalone = True
        worker_params.register = True
        worker_params.port = local_port
        logger.info(f"Worker params: {worker_params}")
        _setup_fastapi(worker_params, app, ignore_exception=True)
        _start_local_worker(worker_manager, worker_params)
        worker_manager.after_start(start_listener)
        _start_local_embedding_worker(
            worker_manager, embedding_model_name, embedding_model_path
        )

2. llm persistence will support soon.

[LZC6244]

1. you can update local manager mode in manager.py to support multi llm start.

    if run_locally:
        # TODO start ModelController
        worker_params.standalone = True
        worker_params.register = True
        worker_params.port = local_port
        logger.info(f"Worker params: {worker_params}")
        _setup_fastapi(worker_params, app, ignore_exception=True)
        _start_local_worker(worker_manager, worker_params)
        worker_manager.after_start(start_listener)
        _start_local_embedding_worker(
            worker_manager, embedding_model_name, embedding_model_path
        )

2. llm persistence will support soon.

How should I configure different proxyllm? Different proxyllm api-keys and urls are different. It seems that the PROXY_API_KEY and PROXY_SERVER_URL in the environment variables are used.

    xx_model_worker_params = ModelWorkerParameters(
        model_name='test_llm',
        model_path='chatgpt_proxyllm'
    )
    _start_local_worker(worker_manager, xx_model_worker_params)

[paul-yangmy]

Could you explain this implementation process in detail :) I add "_start_local_worker" after worker_manager.after_start(start_listener)

However, it setups 2 same model path LLM and named is chatglm3-6b-128k ;<

[LZC6244]

Is there any date for model persistence?