Coder Social home page Coder Social logo

cloudapi's Introduction

Yandex.Cloud API

This repository contains the interface definitions of Yandex.Cloud API.

For more details see documentation.

Overview

You can access Yandex.Cloud by using REST or GRPC API.

Yandex.Cloud API uses Protocol Buffers to define API interfaces and data structures. These definitions are used for both REST and GRPC versions of the Yandex.Cloud API.

Repository structure

Repository structure reflects Yandex.Cloud service hierarchy. Folders represent different services of the platform and contain protobuf files. Each service folder contains resource data structure and interface definitions for API calls.

cloudapi's People

Contributors

baranov1ch avatar kudep avatar manykey avatar yandex-cloud-bot avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

cloudapi's Issues

Problem with streaming recognition

Добрый день!
Прошу мне помочь. Я делаю приложение под андроид, и хотел воспользоваться методом для потокового распознавания, но при генерации файла Stt.java я не нашел класс StreamingRecognize, а также я не совсем понимаю куда мне передать созданный экземпляр ManagedChannel
ManagedChannelBuilder.forAddress("https://stt.api.cloud.yandex.net", 433) .usePlaintext() .build()

Stt.zip

DNS resolution failed for service: stt.api.cloud.yandex-cloud.net:443

I'm following this example and it was fine till today. Now I get DNS error. Any ideas how it can be fixed?

<_MultiThreadedRendezvous of RPC that terminated with: status = StatusCode.UNAVAILABLE details = "DNS resolution failed for service: stt.api.cloud.yandex-cloud.net:443" debug_error_string = " { "created": "@1642795937.739745884", "description": "Resolver transient failure", "file": "src/core/ext/filters/client_channel/client_channel.cc", "file_line": 1324, "referenced_errors": [ { "created": "@1642795937.739740447", "description": "DNS resolution failed for service: stt.api.cloud.yandex-cloud.net:443", "file": "src/core/ext/filters/client_channel/resolver/dns/c_ares/dns_resolver_ares.cc", "file_line": 359, "grpc_status": 14, "referenced_errors": [ { "created": "@1642795937.739600798", "description": "C-ares status is not ARES_SUCCESS qtype=A name=stt.api.cloud.yandex-cloud.net is_balancer=0: Domain name not found", "file": "src/core/ext/filters/client_channel/resolver/dns/c_ares/grpc_ares_wrapper.cc", "file_line": 698, "referenced_errors": [ { "created": "@1642795937.739131933", "description": "C-ares status is not ARES_SUCCESS qtype=AAAA name=stt.api.cloud.yandex-cloud.net is_balancer=0: Domain name not found", "file": "src/core/ext/filters/client_channel/resolver/dns/c_ares/grpc_ares_wrapper.cc", "file_line": 698 } ] } ] } ] }"

pcm -> ogg

Есть ли у вас пример на питоне для потокового распознования (https://github.com/yandex-cloud/docs/blob/master/ru/speechkit/stt/streaming.md), который включает перекодирование из pcm в opus и оборачивание в контейнер ogg, но без использования внешних файлов/файловой системы. В вашем примере, показано с использованием внешнего файла https://github.com/yandex-cloud/docs/blob/master/ru/speechkit/stt/streaming.md#%D0%BF%D1%80%D0%B8%D0%BC%D0%B5%D1%80%D1%8B-examples .

Проблема с yandex/cloud/validation.proto

Я следовал документации на странице https://cloud.yandex.ru/docs/speechkit/stt/api/microphone-streaming и при запуске test.py получаю (я даже пробовал скопировать validation.proto из yandex/cloud в output/yandex/cloud):
$ python test.py
Traceback (most recent call last):
File "/Users/macbook/code/speechkit/cloudapi/output/test.py", line 5, in
import yandex.cloud.ai.stt.v3.stt_service_pb2_grpc as stt_service_pb2_grpc
File "/Users/macbook/code/speechkit/cloudapi/output/yandex/cloud/ai/stt/v3/stt_service_pb2_grpc.py", line 6, in
from yandex.cloud.ai.stt.v3 import stt_service_pb2 as yandex_dot_cloud_dot_ai_dot_stt_dot_v3_dot_stt__service__pb2
File "/Users/macbook/code/speechkit/cloudapi/output/yandex/cloud/ai/stt/v3/stt_service_pb2.py", line 21, in
DESCRIPTOR = _descriptor_pool.Default().AddSerializedFile(b'\n(yandex/cloud/ai/stt/v3/stt_service.proto\x12\x10speechkit.stt.v3\x1a yandex/cloud/ai/stt/v3/stt.proto\x1a\x1cgoogle/api/annotations.proto\x1a\x1dyandex/cloud/validation.proto\x1a yandex/cloud/api/operation.proto\x1a&yandex/cloud/operation/operation.proto";\n\x15GetRecognitionRequest\x12"\n\x0coperation_id\x18\x01 \x01(\tB\x0c\xe8\xc7\x31\x01\x8a\xc8\x31\x04<=502q\n\nRecognizer\x12\x63\n\x12RecognizeStreaming\x12".speechkit.stt.v3.StreamingRequest\x1a#.speechkit.stt.v3.StreamingResponse"\x00(\x01\x30\x01\x32\xb3\x02\n\x0f\x41syncRecognizer\x12\x9c\x01\n\rRecognizeFile\x12&.speechkit.stt.v3.RecognizeFileRequest\x1a!.yandex.cloud.operation.Operation"@\xb2\xd2*\x17\x12\x15google.protobuf.Empty\x82\xd3\xe4\x93\x02\x1f"\x1a/stt/v3/recognizeFileAsync:\x01*\x12\x80\x01\n\x0eGetRecognition\x12'.speechkit.stt.v3.GetRecognitionRequest\x1a#.speechkit.stt.v3.StreamingResponse"\x1e\x82\xd3\xe4\x93\x02\x18\x12\x16/stt/v3/getRecognition0\x01\x42\\n\x1ayandex.cloud.api.ai.stt.v3Z>github.com/yandex-cloud/go-genproto/yandex/cloud/ai/stt/v3;sttb\x06proto3')
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: Couldn't build proto file into descriptor pool: Depends on file 'yandex/cloud/validation.proto', but it has not been loaded

API Versioning?

Hi!

Yersteday I read about YC CLI 0.46.0 release in Y.Cloud telegram channel. This is cool but how may I know which commit corresponds to current production version of YC?

We have here only master branch with "sync upstream" commits. No tags. No VERSION.txt. It would be good to add versioning in some form.

Проксирование запросов с Фронтенда в Yandex Speechkit в Django при помощи Websocket

Здравствуйте, я плохо знаком с данным пакетом, поэтому прошу помощи у вас. Мне нужно написать бэкенд приложение на Джанго, который будет использоваться React'ом как прокси запросов к Yandex SpeechKit, используя WebSocket протокол. В документации приведен пример кода, демонстрирующий потоковое распазнование речи с микрофона, но я слишком глуп, чтобы реконструировать его под мой случай. В Read.me указана документация ко всему YandexCloud, но я не нашел там документацию именно к этому репо. Для каждого вебсокет соединения создается объект питон WebsocketConsumer, у него есть методы connect, receive, disconnect. В методе receive принимается порция аудиоданных в виде bytes, конвертируется при помощи Speechkit и возвращается обратно клиенту. Можете ли привести пример кода, который будет конвертировать аудио в текст при помощи cloudapi при проксировании? Пример WebsocketConsumer, который я успел написать(он неправильный):

from channels.generic.websocket import WebsocketConsumer
import grpc
import cloudapi.output.yandex.cloud.ai.stt.v3.stt_pb2 as stt_pb2
import cloudapi.output.yandex.cloud.ai.stt.v3.stt_service_pb2_grpc as stt_service_pb2_grpc

class MyConsumer(WebsocketConsumer):
    secret = 'some-token'
    rate = 8000
    chunk = 4000
    def get_settings_recognition(self):
        '''Генератор возвращяющий настройки распознавания.'''
        # Задать настройки распознавания.
        recognize_options = stt_pb2.StreamingOptions(
          recognition_model=stt_pb2.RecognitionModelOptions(
             audio_format=stt_pb2.AudioFormatOptions(
                raw_audio=stt_pb2.RawAudio(
                   audio_encoding=stt_pb2.RawAudio.LINEAR16_PCM,
                   sample_rate_hertz=8000,
                   audio_channel_count=1
                )
             ),
             text_normalization=stt_pb2.TextNormalizationOptions(
                text_normalization=stt_pb2.TextNormalizationOptions.TEXT_NORMALIZATION_ENABLED,
                profanity_filter=True,
                literature_text=False
             ),
             language_restriction=stt_pb2.LanguageRestrictionOptions(
                restriction_type=stt_pb2.LanguageRestrictionOptions.WHITELIST,
                language_code=['ru-RU']
             ),
             audio_processing_type=stt_pb2.RecognitionModelOptions.REAL_TIME
          )
        )


        yield stt_pb2.StreamingRequest(session_options=recognize_options)

    def connect(self):

        # Установить соединение с сервером.
        cred = grpc.ssl_channel_credentials()
        channel = grpc.secure_channel('stt.api.cloud.yandex.net:443', cred)
        self.stub = stt_service_pb2_grpc.RecognizerStub(channel)

        # Отправить данные для распознавания.
        self.stub.RecognizeStreaming(self.get_settings_recognition(), metadata=(
          ('authorization', f'Api-Key {self.secret}'),
        ))
        self.accept()

    def disconnect(self, close_code):
        pass

    def gen_chunk(self, audio_data: bytes):
        yield stt_pb2.StreamingRequest(chunk=stt_pb2.AudioChunk(data=audio_data))

    def receive(self, bytes_data):
        audio_data = bytes_data
        audio_data = stt_pb2.StreamingRequest(chunk=stt_pb2.AudioChunk(data=audio_data))
        resp = self.stub.RecognizeStreaming(self.gen_chunk(audio_data), metadata=(
            ('authorization', f'Api-Key {self.secret}'),
        ))
        try:
            event_type, alternatives = resp.WhichOneof('Event'), None
            if event_type == 'partial' and len(resp.partial.alternatives) > 0:
                alternatives = [a.text for a in resp.partial.alternatives]
            if event_type == 'final':
                alternatives = [a.text for a in resp.final.alternatives]
            if event_type == 'final_refinement':
                alternatives = [a.text for a in resp.final_refinement.normalized_text.alternative]
        except grpc._channel._Rendezvous as err:
            print(f'Error code {err._state.code}, message: {err._state.details}')
            raise err
        if not alternatives:
            alternatives = ''
        self.send(text_data=alternatives)

Authentication SSL/TLS; Custom CA root certificates for Android

Я реализую простое приложение на android и пытаюсь использовать grpc потоковое распознавания. Но возникла проблема при использовании SSL/TLS.
На примерах из официальной документации (https://grpc.io/docs/guides/auth/) я попытался настроит SSLSocketFactory для OkHttpChannelBuilder

private fun getManagedChannel(): ManagedChannel {
    val metadata = Metadata()
    metadata.put(AUTHORIZATION_HEADER, "Bearer $IAM_TOKEN")
    val attachHeadersInterceptor = MetadataUtils.newAttachHeadersInterceptor(metadata)

    val builder = OkHttpChannelBuilder.forAddress(BASE_URL, PORT)
        .intercept(attachHeadersInterceptor)
    try {
        builder.sslSocketFactory(
            newSslSocketFactoryForCa(
                Platform.get().provider,
                File("roots.pem")
            )
        )
    } catch (exc: Exception) {
        exc.printStackTrace()
    }
    return builder.build()
}
@Throws(Exception::class)
fun newSslSocketFactoryForCa(provider: Provider, certChainFile: File): SSLSocketFactory {
    val ks = KeyStore.getInstance(KeyStore.getDefaultType())
    ks.load(null, null)
    val cf = CertificateFactory.getInstance("X.509")
    val bufferedInputStream = BufferedInputStream(FileInputStream(certChainFile))
    try {
        val cert = cf.generateCertificate(bufferedInputStream) as X509Certificate
        val principal = cert.subjectX500Principal
        ks.setCertificateEntry(principal.getName("RFC2253"), cert)
    } finally {
        bufferedInputStream.close()
    }

    // Set up trust manager factory to use our key store.
    val trustManagerFactory = TrustManagerFactory.getInstance(TrustManagerFactory.getDefaultAlgorithm())
    trustManagerFactory.init(ks)
    val context = SSLContext.getInstance("TLS", provider)
    context.init(null, trustManagerFactory.trustManagers, null)
    return context.socketFactory
}

Но падает ошибка

2020-11-11 14:36:06.328 17495-17495/ru.android.thread_asr_tinkoff W/System.err: java.security.cert.CertificateException: com.android.org.conscrypt.OpenSSLX509CertificateFactory$ParsingException: com.android.org.conscrypt.OpenSSLX509CertificateFactory$ParsingException: java.lang.RuntimeException: error:0c0000be:ASN.1 encoding routines:OPENSSL_internal:WRONG_TAG
2020-11-11 14:36:06.329 17495-17495/ru.android.thread_asr_tinkoff W/System.err:     at com.android.org.conscrypt.OpenSSLX509CertificateFactory.engineGenerateCertificate(OpenSSLX509CertificateFactory.java:280)
2020-11-11 14:36:06.329 17495-17495/ru.android.thread_asr_tinkoff W/System.err:     at java.security.cert.CertificateFactory.generateCertificate(CertificateFactory.java:366)
2020-11-11 14:36:06.330 17495-17495/ru.android.thread_asr_tinkoff W/System.err:     at com.example.asryandex.AsrYandex.newSslSocketFactoryForCa(AsrYandex.kt:84)
2020-11-11 14:36:06.330 17495-17495/ru.android.thread_asr_tinkoff W/System.err:     at com.example.asryandex.AsrYandex.getManagedChannel(AsrYandex.kt:66)
2020-11-11 14:36:06.330 17495-17495/ru.android.thread_asr_tinkoff W/System.err:     at com.example.asryandex.AsrYandex.<init>(AsrYandex.kt:54)
2020-11-11 14:36:06.330 17495-17495/ru.android.thread_asr_tinkoff W/System.err:     at ru.android.thread_asr_tinkoff.VoiceRecognition.<init>(VoiceRecognition.kt:38)
2020-11-11 14:36:06.331 17495-17495/ru.android.thread_asr_tinkoff W/System.err:     at ru.android.thread_asr_tinkoff.MainActivity.initPhraseDetector(MainActivity.kt:33)
2020-11-11 14:36:06.331 17495-17495/ru.android.thread_asr_tinkoff W/System.err:     at ru.android.thread_asr_tinkoff.MainActivity.onRequestPermissionsResult(MainActivity.kt:40)
2020-11-11 14:36:06.331 17495-17495/ru.android.thread_asr_tinkoff W/System.err:     at android.app.Activity.dispatchRequestPermissionsResult(Activity.java:7616)
2020-11-11 14:36:06.331 17495-17495/ru.android.thread_asr_tinkoff W/System.err:     at android.app.Activity.dispatchActivityResult(Activity.java:7466)
2020-11-11 14:36:06.332 17495-17495/ru.android.thread_asr_tinkoff W/System.err:     at android.app.ActivityThread.deliverResults(ActivityThread.java:4354)
2020-11-11 14:36:06.332 17495-17495/ru.android.thread_asr_tinkoff W/System.err:     at android.app.ActivityThread.handleSendResult(ActivityThread.java:4403)
2020-11-11 14:36:06.332 17495-17495/ru.android.thread_asr_tinkoff W/System.err:     at android.app.servertransaction.ActivityResultItem.execute(ActivityResultItem.java:49)
2020-11-11 14:36:06.332 17495-17495/ru.android.thread_asr_tinkoff W/System.err:     at android.app.servertransaction.TransactionExecutor.executeCallbacks(TransactionExecutor.java:108)
2020-11-11 14:36:06.333 17495-17495/ru.android.thread_asr_tinkoff W/System.err:     at android.app.servertransaction.TransactionExecutor.execute(TransactionExecutor.java:68)
2020-11-11 14:36:06.333 17495-17495/ru.android.thread_asr_tinkoff W/System.err:     at android.app.ActivityThread$H.handleMessage(ActivityThread.java:1809)
2020-11-11 14:36:06.333 17495-17495/ru.android.thread_asr_tinkoff W/System.err:     at android.os.Handler.dispatchMessage(Handler.java:106)
2020-11-11 14:36:06.333 17495-17495/ru.android.thread_asr_tinkoff W/System.err:     at android.os.Looper.loop(Looper.java:193)
2020-11-11 14:36:06.333 17495-17495/ru.android.thread_asr_tinkoff W/System.err:     at android.app.ActivityThread.main(ActivityThread.java:6680)
2020-11-11 14:36:06.334 17495-17495/ru.android.thread_asr_tinkoff W/System.err:     at java.lang.reflect.Method.invoke(Native Method)
2020-11-11 14:36:06.334 17495-17495/ru.android.thread_asr_tinkoff W/System.err:     at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run(RuntimeInit.java:493)
2020-11-11 14:36:06.334 17495-17495/ru.android.thread_asr_tinkoff W/System.err:     at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:858)
2020-11-11 14:36:06.335 17495-17495/ru.android.thread_asr_tinkoff W/System.err: Caused by: com.android.org.conscrypt.OpenSSLX509CertificateFactory$ParsingException: com.android.org.conscrypt.OpenSSLX509CertificateFactory$ParsingException: java.lang.RuntimeException: error:0c0000be:ASN.1 encoding routines:OPENSSL_internal:WRONG_TAG
2020-11-11 14:36:06.335 17495-17495/ru.android.thread_asr_tinkoff W/System.err:     at com.android.org.conscrypt.OpenSSLX509CertificateFactory$Parser.generateItem(OpenSSLX509CertificateFactory.java:121)
2020-11-11 14:36:06.335 17495-17495/ru.android.thread_asr_tinkoff W/System.err:     at com.android.org.conscrypt.OpenSSLX509CertificateFactory.engineGenerateCertificate(OpenSSLX509CertificateFactory.java:278)
2020-11-11 14:36:06.336 17495-17495/ru.android.thread_asr_tinkoff W/System.err: 	... 21 more
2020-11-11 14:36:06.337 17495-17495/ru.android.thread_asr_tinkoff W/System.err: Caused by: com.android.org.conscrypt.OpenSSLX509CertificateFactory$ParsingException: java.lang.RuntimeException: error:0c0000be:ASN.1 encoding routines:OPENSSL_internal:WRONG_TAG
2020-11-11 14:36:06.337 17495-17495/ru.android.thread_asr_tinkoff W/System.err:     at com.android.org.conscrypt.OpenSSLX509Certificate.fromX509DerInputStream(OpenSSLX509Certificate.java:103)
2020-11-11 14:36:06.337 17495-17495/ru.android.thread_asr_tinkoff W/System.err:     at com.android.org.conscrypt.OpenSSLX509CertificateFactory$1.fromX509DerInputStream(OpenSSLX509CertificateFactory.java:232)
2020-11-11 14:36:06.338 17495-17495/ru.android.thread_asr_tinkoff W/System.err:     at com.android.org.conscrypt.OpenSSLX509CertificateFactory$1.fromX509DerInputStream(OpenSSLX509CertificateFactory.java:222)
2020-11-11 14:36:06.338 17495-17495/ru.android.thread_asr_tinkoff W/System.err:     at com.android.org.conscrypt.OpenSSLX509CertificateFactory$Parser.generateItem(OpenSSLX509CertificateFactory.java:112)
2020-11-11 14:36:06.338 17495-17495/ru.android.thread_asr_tinkoff W/System.err: 	... 22 more
2020-11-11 14:36:06.339 17495-17495/ru.android.thread_asr_tinkoff W/System.err: Caused by: java.lang.RuntimeException: error:0c0000be:ASN.1 encoding routines:OPENSSL_internal:WRONG_TAG
2020-11-11 14:36:06.339 17495-17495/ru.android.thread_asr_tinkoff W/System.err:     at com.android.org.conscrypt.NativeCrypto.d2i_X509_bio(Native Method)
2020-11-11 14:36:06.339 17495-17495/ru.android.thread_asr_tinkoff W/System.err:     at com.android.org.conscrypt.OpenSSLX509Certificate.fromX509DerInputStream(OpenSSLX509Certificate.java:97)
2020-11-11 14:36:06.339 17495-17495/ru.android.thread_asr_tinkoff W/System.err: 	... 25 more

Может вы сможете мне помочь или направить в какую сторону мне копать по использованию grpc потокового распознавания в Android?
Примеры в вашей документации реализованы на node.js и python, причем в примере на python файл roots.pem не используется. Пример на node.js у меня завелся.

Missing "google/protobuf/descriptor.proto".

Hi!
I've faced with the problem of compiling 'annotations.proto'-file. It requires an import of "google/protobuf/descriptor.proto" that was not in the repository.

On official repository of 'Protobuf', I've found missing import-file, by the link:
https://github.com/protocolbuffers/protobuf/releases/download/v3.11.1/protobuf-csharp-3.11.1.tar.gz
And, after unzipping, go to 'protobuf-3.11.1\src\google\protobuf\descriptor.proto' and faced another problem - it uses syntax of proto2.

Here is the question:
How did you solve this issue? Please, tell me, I've stuck with my project for 2 days...
By my little experience, gRPC in C# only works with proto3:(

yandex/cloud/speechsense/v1/talk_service.proto:1:1: Expected top-level statement (e.g. "message").

Пытаюсь собрать апи по этой инструкции

(venv) ➜  cloudapi git:(master) ✗ python3 -m grpc_tools.protoc -I . \
    --python_out=./upload_data/ \
    --grpc_python_out=./upload_data/ \
    yandex/cloud/speechsense/v1/talk_service.proto \
    yandex/cloud/speechsense/v1/audio.proto

WARNING: All log messages before absl::InitializeLog() is called are written to STDERR
W0000 00:00:1710514993.503769 26819011 parser.cc:678] No syntax specified for the proto file: yandex/cloud/speechsense/v1/talk_service.proto. Please use 'syntax = "proto2";' or 'syntax = "proto3";' to specify a syntax version. (Defaulted to proto2 syntax.)
yandex/cloud/speechsense/v1/talk_service.proto:1:1: Expected top-level statement (e.g. "message").
yandex/cloud/speechsense/v1/talk_service.proto:35:3: Expected "required", "optional", or "repeated".
yandex/cloud/speechsense/v1/talk_service.proto:48:3: Expected "required", "optional", or "repeated".
yandex/cloud/speechsense/v1/talk_service.proto:57:3: Expected "required", "optional", or "repeated".

Implement support for internal loadbalancer type

This is just a feature request (enhancement) for yandex cloudapi that we would like to see.

It would be great to have possibility to create Internal network load balancers. As far as I see in yandex/cloud/loadbalancer/v1/network_load_balancer.proto, someone suggested that internal balancer type will be introduced in the future:

  // Type of the load balancer. Only external load balancers are currently available.
  enum Type {
    TYPE_UNSPECIFIED = 0;

    // External network load balancer.
    EXTERNAL = 1;

    // Internal network load balancer.
    INTERNAL = 2;
  }

If you wonder if anyone is looking for this feature, we are here and glad to discuss implementation

Strange result on streaming recognition of english speech

I am testing streaming recognition with python grpc API.
I set language_code='en-US' in stt_service_pb2.RecognitionSpec() and send audio of english speech to the recognizer.
Expected result is english recognized text ("i mean sometimes they have methods that don't have any explanation to them at all") but instead I obtain something that looks like swedish text ("den svarta färgen beror på interaktionen mellan hemoglobin och saltsyra av magsaft vilket resulterar i svart hemin") and this text doesn't seem to be similar to the reference one at all.

It looks like automatic lagnuage recognition is turned on although language code is set explicitly

Zipped audio file is attached test_eng.zip

Code I use is below:
`import grpc

import yandex.cloud.ai.stt.v2.stt_service_pb2 as stt_service_pb2
import yandex.cloud.ai.stt.v2.stt_service_pb2_grpc as stt_service_pb2_grpc

def gen(folder_id, audio_file_name, lang='ru-RU', model='general', profanity_filter=False, audio_encoding='LINEAR16_PCM', sample_rate=16000, chunk_size=8000):
# Задать настройки распознавания.
specification = stt_service_pb2.RecognitionSpec(
language_code=lang,
profanity_filter=profanity_filter,
model=model,
partial_results=True,
audio_encoding=audio_encoding,
sample_rate_hertz=sample_rate,
raw_results=True
)
streaming_config = stt_service_pb2.RecognitionConfig(specification=specification, folder_id=folder_id)

# Отправить сообщение с настройками распознавания.
yield stt_service_pb2.StreamingRecognitionRequest(config=streaming_config)

# Прочитать аудиофайл и отправить его содержимое порциями.
with open(audio_file_name, 'rb') as f:
    data = f.read(chunk_size)
    while data != b'':
        yield stt_service_pb2.StreamingRecognitionRequest(audio_content=data)
        data = f.read(chunk_size)

def transcribe_file(folder_id, iam_token, audio_file_name,
lang='ru-RU',
model='general',
profanity_filter=False,
audio_encoding='LINEAR16_PCM',
sample_rate=16000,
chunk_size=8000):
# Установить соединение с сервером.
cred = grpc.ssl_channel_credentials()
channel = grpc.secure_channel('stt.api.cloud.yandex.net:443', cred)
stub = stt_service_pb2_grpc.SttServiceStub(channel)

# Отправить данные для распознавания.
it = stub.StreamingRecognize(gen(folder_id, audio_file_name, lang, model, profanity_filter, audio_encoding, sample_rate, chunk_size),
                             metadata=(('authorization', 'Bearer %s' % iam_token),))

result=""
# Обработать ответы сервера и вывести результат в консоль.
try:
    for r in it:
        try:
            if r.chunks[0].final:
                result += " " + r.chunks[0].alternatives[0].text
        except LookupError:
            result = "<no_chunks>"
except grpc._channel._Rendezvous as err:
    print(err.code())
    result="<error>"
return result

`

Incompatible http streaming protocol with python requests

How to reproduce the issue:

import requests

token = "token"
cluster_id = "cluster_id"
url = f"https://mdb.api.cloud.yandex.net/managed-postgresql/v1/clusters/{cluster_id}:stream_logs"

with requests.get(url,
                  params={"serviceType":"POSTGRESQL"},
                  headers={"Authorization":f"Bearer {token}"},
                  stream=True
                  ) as response:
    for chunk in response.iter_content(chunk_size=None, decode_unicode=True):
        print(chunk)

The above code outputs the logs as they come in. However, an exception like the one below occurs when a timeout occurs.

Traceback (most recent call last):
File "/usr/lib/python3.10/site-packages/urllib3/response.py", line 748, in _update_chunk_length
self.chunk_left = int(line, 16)
ValueError: invalid literal for int() with base 16: b''

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/usr/lib/python3.10/site-packages/urllib3/response.py", line 443, in _error_catcher
yield
File "/usr/lib/python3.10/site-packages/urllib3/response.py", line 815, in read_chunked
self._update_chunk_length()
File "/usr/lib/python3.10/site-packages/urllib3/response.py", line 752, in _update_chunk_length
raise InvalidChunkLength(self, line)
urllib3.exceptions.InvalidChunkLength: InvalidChunkLength(got length b'', 0 bytes read)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/usr/lib/python3.10/site-packages/requests/models.py", line 760, in generate
for chunk in self.raw.stream(chunk_size, decode_content=True):
File "/usr/lib/python3.10/site-packages/urllib3/response.py", line 623, in stream
for line in self.read_chunked(amt, decode_content=decode_content):
File "/usr/lib/python3.10/site-packages/urllib3/response.py", line 803, in read_chunked
with self._error_catcher():
File "/usr/lib64/python3.10/contextlib.py", line 153, in exit
self.gen.throw(typ, value, traceback)
File "/usr/lib/python3.10/site-packages/urllib3/response.py", line 460, in _error_catcher
raise ProtocolError("Connection broken: %r" % e, e)
urllib3.exceptions.ProtocolError: ("Connection broken: InvalidChunkLength(got length b'', 0 bytes read)", InvalidChunkLength(got length b'', 0 bytes read))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/home/user/scripts/test.py", line 12, in <module>
for ind, chunk in enumerate(response.iter_content(chunk_size=None, decode_unicode=True)):
File "/usr/lib/python3.10/site-packages/requests/utils.py", line 549, in stream_decode_response_unicode
for chunk in iterator:
File "/usr/lib/python3.10/site-packages/requests/models.py", line 763, in generate
raise ChunkedEncodingError(e)
requests.exceptions.ChunkedEncodingError: ("Connection broken: InvalidChunkLength(got length b'', 0 bytes read)", InvalidChunkLength(got length b'', 0 bytes read))

Apparently this only happens if any logs were output. If there are no logs, one message about stream timeout will be output and the program will end properly.
I found a discussion of a similar error. Maintainer sad it's not a bug. This is a faulty implementation of the protocol on the server side.

Error in streaming recognition for Turkish

I am testing a streaming recognition for Turkish.
I set model as "regular:rc" and language_code as "tr-TR"
When sending recognition request I receive an exception grpc._channel._Rendezvous and no recognition results is returned.
If I replace language_code with en-US the recognition works somehow and no exception is thrown.
Is tr-TR indeed supported in regular:rc model?

"You have to specify folder ID for user account"

Можно ли в SDK указать folder_id? Пробую использовать YandexGPT https://cloud.yandex.ru/docs/yandexgpt/api-ref/authentication, получаю ошибку

_InactiveRpcError: <_InactiveRpcError of RPC that terminated with:
	status = StatusCode.UNAUTHENTICATED
	details = "You have to specify folder ID for user account"
	debug_error_string = "UNKNOWN:Error received from peer ipv6:%5B2a0d:d6c1:0:1c::3e3%5D:443 {grpc_message:"You have to specify folder ID for user account", grpc_status:16, created_time:"2023-12-02T13:03:07.648715+03:00"}"

Примерный код

import yandexcloud
from yandex.cloud.ai.llm.v1alpha import (
    llm_pb2,
    llm_service_pb2,
    llm_service_pb2_grpc
)


def yagpt_tokenize(sdk, text, model='general'):
    service = sdk.client(llm_service_pb2_grpc.TokenizerServiceStub)
    request = llm_service_pb2.TokenizeRequest(
        model=model,
        text=text
    )
    response = service.Tokenize(request)
    return [_.text for _ in response.tokens]


sdk = yandexcloud.SDK(iam_token=iam_token)
yagpt_tokenize(sdk, 'yagpt')

iam_token из yc iam create-token

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.