fatimashehab99 / data-platform

Data Platform Analytics System is a comprehensive data anlytics tool that helps users collect, analyze, and display data in a straightforward and interactive manner, aiding them in making informed decisions.

I am a positive, hardworking, and ambitious engineer with a strong desire to gain practical experience and advance my career. I prioritize continual learning and improvement to stay ahead in my field. Working well under pressure, I excel at collaborating effectively and demonstrating excellent judgment. My dedication lies in creating a positive and dynamic work environment. Currently, I am working as a backend developer and I am eager to expand my knowledge and experience in the field of data engineer and analytics

🔭 I’m currently working on developing APIs, backend systems for database management,big data and analytics systems. 🌱 I’m currently learning advanced techniques in data analytics , big data and engineering. 👯 I’m looking to collaborate on projects related to data engineering, analytics, and big data. 🤔 I’m looking for help with developing more analytics and data systems, and working with big data.

• Email: [email protected]
• LinkedIn: LinkedIn Profile

Skills & Abilities:

• Experienced with PHP, Python, C#, Java, NumPy, Pandas, Laravel, Flask, ASP.NET, SQL, MySQL, PostgreSQL, MongoDB, SQL Alchemy, data modeling, ETL, data pipeline, data engineering, data visualization, data warehousing, and data cleaning, Microsoft Power BI, Git, GitHub.

• I am familiar with Google Cloud Platform, Big Query,Looker Studio, Cloud Storage, Cloud Function, Apache Beam, Dataflow. Possess strong teamwork, communication skills, and problem-solving abilities.

data-platform's Issues

Re Write the readme file

https://github.com/fatimashehab99/data-platform/blob/main/README.md

Create Recommendations API api/recommend

• Trending Logic: Get Top posts sorted by pageviews in the last 3 Days (Constant) , for a specific domain.
• Content Logic: Get posts relevant to a user (param userId), posts should be similar to the old posts the user has read, similarity is based on the intersection of category and keywords

Example:

https://servingx.tawsiyat.net/recommend?mwv=true&widgetids=5714101758066688,5111534002896896&userid=3f8ea6de-9430-42e4-821d-c1640d167b9c&post_id=1830311&appdomain=almayadeen.net&history_postids=1788836,1372110,1439952,966609,1352231,1393550,1372671,1830311,1830483&exc_ads=

Add ASP.NET static file caching

Save user profile in memory cache

user profile (user id, location ,categories ,authors ,tags)

Create the Visualization API and the Demo Dashboard

This is the design that I did
I add a filter to get data based on date (today , 7 days ago ..)
The pageviews , authors , articles and users are the total pageviews,total number of authors , total number of articles and total number of users (within specific range of time)
-The first dashboard is to get the top articles' title (within specific range of time)
-The second dashboard is to get pageviews with respect to date
-The third dashboard is to get pageviews for each author (with in specific range of time )
-The fourth dashboard is to the map that get the pageviews in specific location (I will do the design later )

I added the code on github on Friday but I forget to push the code
so yesterday I designed the dashboard and I fill it with dumpy data and I practice how to make mongo db query in C# to save time today
Kindly give me feed back about the dashboards before writing the APIs and if I can change anything

Refactor the Project structure to be similar to Data Engineering projects

Data Engineering Bootcamp Syllabus.pdf

Create the Collector Api /api/collect

• Get data from request
• Parse useragent and get the platform and browser using a library
• get the country name/Code from the IP
• Save data in mongodb
• Write many tests failure and happy senario

Edit mondo DB fields names

Deploy the project on mango pulse

api/collect request fixes

if no postimage do call collect api, also posttags if no data dont send empty array ["] it should be null, also if no category in classes use posttype for postcategory

Collect post type

add post type to the collection