The audio-processing-suite from vitorgithub

audio-processing-suite's Introduction

🌎 Engenharia de Software & Engenharia de Dados;
🔎 Autodidata, analítico, multidisciplinar e detalhista;
🧐 Crio e desenvolvo documentos autorais teóricos e práticos de estudo;
📖 Minhas referências: (Documentações oficiais, livros e cursos qualificados);

DATA SOURCES	Data Ingestion	Data Processing	Data Storage	Workflow scheduling	Monitoring data pipelines
SGBD SQL - MySQL - Postgres - MS SQL SERVER - AWS Aurora SGBD NoSQL (document) - MongoDB - Azure Cosmos DB - ElasticSearch (data retrieval) SGBD NoSQL (key-value) - AWS DynamoDB - Redis SGBD NoSQL (wide-column) - GCP Bigtable - Apache HBase - Apache Cassandra SGBD NoSQL (graph) - AWS Neptune - Neo4j TELEMETRIA - Application logs - Device sensors FORNECEDORES DE DADOS EXTERNOS - APIs (ex: RESTful, SOAP) - Web scraping (ex: BeautifulSoup, Scrapy)	Event Streaming - Apache Kafka (c/ RabbitMQ) - AWS Kinesis (c/ AWS SNS & SQS) - Azure Event Hubs Data Movement - Apache NiFi - StreamSets - Talend - AWS Glue Messaging - RabbitMQ - AWS SNS & SQS - Azure Service Bus	Batch Processing: - Apache Spark - Apache Flink - Apache Hive - Apache Pig Stream Processing: - Apache Kafka - Apache Storm - AWS Kinesis	"batch" Distributed File Systems - Hadoop HDFS - Azure Data Lake Storage "hybrid" Data Lake (Storage for Pre-processing) - AWS S3 - Azure Blob Storage Gen2 - GCP Cloud Storage "hybrid" Data Lake (Storage for Post-processing) - Delta Lake - Apache Hudi - Apache Iceberg Data Warehouses - Azure Synapse Analytics - AWS Redshift - Snowflake Data Marts - Apache Druid - Apache Pinot	Workflow scheduling - Apache Airflow - Azure Data Factory	Monitoring data pipelines - Datadog - Prometheus - Sentry - Grafana

Recommend Projects