frddl / ads-parser

How to run via docker

If running first time, follow these steps:

cd src

cp .env.example .env

Open .env file in src directory. Change DB_USERNAME to anything else than root. Put password. Save it.

cd ../docker

docker-compose --env-file ../src/.env build app

docker-compose --env-file ../src/.env up -d

Either ssh to ads-parser-app container or run it through docker-compose:

docker-compose exec app rm -rf vendor composer.lock

docker-compose exec app composer install

docker-compose exec app php artisan key:generate

docker-compose exec app php artisan migrate

docker-compose exec app php artisan db:seed

docker-compose exec app php artisan schedule:work

Here you go!

ads-parser's Issues

Not found if $keyword is not specified.

str_contains() needs a non-empty needle, Str::contains() returns false due to emptiness.

Check for empty($keyword)

proxy list

proxy feature should be built in order not to get the ip banned.
provider should use list of available proxies from config and attach them to Guzzle

to check: proxy6.net

translations

Nural should provide today

user-agent modifier

currently, guzzle sends a user agent such as Guzzle/4.0 curl/7.21.4 PHP/$version
provider should randomly pick a user agent from list in config and use it for guzzle headers

the list to use

notification providers

add email sender information and style the email blade and test:
(to be used: yandex - is free)

1. telegram
2. email

currency converter

config/parsers.php, for each site:

convert_currency = true
currency_properties = [
   'usd' => [
      'selector' => 'div.price_cur'
      'parse_regex' => 'r"\$[^ ]+"',
      'matches' => ['usd', 'USD', '$']
   ],
]