allegro / ralph_pricing

A pricing module for Ralph

Ralph Scrooge

Scrooge module brings billing functionality to Ralph. Using Ralph Scrooge, you can retrieve information on the use of resources from many other systems and charge other services. Scrooge generates flexible and accurate cost reports and let you refer to historical usages and costs.

Ralph Scrooge is a combination of IT management and accounting software. By using Scrooge your company can optimize the costs of internal services and departments, by reviewing their structure and dependencies.

Features:

• costs module, where you can provide invoice costs, staff costs and other costs (licences, supports etc.)
• many plugins to retrieve information on the use of resources from other systems
• charging other services proportionally to the use of resources by their devices
• many full-featured reports (costs per service, costs and usages per single device, dependency structure)
• OpenStack support (tenants, simple usages, ceilometer)
• pluginable architecture - write your own plugins to collect data and calculate costs of your services!
• API to upload information from other systems

Visit Ralph documentation to learn more about Ralph and Scrooge features!

ralph_pricing's Issues

Monthly costs: Calculation fails for some date(s)

This is just a kind request to add a tip (hint) when moving a mouse on X (next to some date),
when VerifiedDailyCostsExistsError raises, explaining the reason (i.e. "Cost for given date has been already accepted")

Calculation preconditions

There should be extra step before calculating costs: checking plugins and overall preconditions. If plugin raise precondition error, calculation for single day should be interrupted.

Possible plugins preconditions:

• cost is defined for every warehouse/data center
• team cost is defined
• dynamic extra cost is defined
• price is defined for every usage type in pricing service with fixed prices

Posible overall preconditions:

• check if there is any cycle between pricing services
• check if costs for single day (and type - forecast and real) was already accepted

Notice that preconditions could be checked for single day or for interval.

High memory usage during costs calculation

"Master" job collect results from secondary level jobs, converts them to DailyCost instances and at the end saves everything into database (using bulk save). Creating DailyCost instances is unnecessary step which not only affect (time) performance but memory usage also - collected data could be transformed directly into SQL query (which also happen when using DailyCost) - maybe DailyCostcould be used to automate it.

Besides that master job should be executed on separated queue (ex. scrooge_costs_master - actually it's executed on scrooge_costs with second-level jobs) to control it's destination in better way.

Rack "parent" is Unknown

My rack "parent" is unknown. It is an error or some information is missing. What should be the parent of the rack? (venture, data center, etc?

Authors

Please update authors list according to the reality of this project ;-)

https://github.com/allegro/ralph_pricing/blob/develop/AUTHORS

Negative costs

Scrooge should allow to save negative costs in DailyCost instances (see

ralph_pricing/src/ralph_scrooge/models/cost.py

Line 91 in 3a516f4

def _are_params_valid(self, params):

)

Excluded services vs PricingService with multiple services

Assume that PricingService PS_X has Service S_Y in excluded services, and Service S_Y is assigned to PricingService PS_Y, which besides S_Y has also Services S_W, S_Z and S_A.

Services S_W, S_Z and S_A should be charged by PricingService PS_X, but this line:

ralph_pricing/src/ralph_scrooge/models/service.py

Line 270 in c3fc864

ps = ps.exclude(excluded_services__in=self.services.all())

cause that whole PricingService PS_X (and Services S_W, S_Z, S_A which are assigned to it) won't be charged by PricingService PS_X.

UsageType price vs Warehouse: Problematic need to select unused warehouse for given usage type

The scenario is as follows:
1/ For given usage type (UT) prices (or costs) are defined and matched only with warehouses (WH) using this UT (not all of existing WHs)
2/ Generate Monthly costs and then go to Cost report
3/ Cost for given UT equals 0.

To make it work (to show the real costs of this UT) the unused (fake) WH has to be selected in Usage Prices form for given UT and price/cost has to be set to 0).
Of course this (unused) WH uses other metrics (UTs) so the checkbox "Show warehouse in report" has to be selected.
The problem is how to make it work without selecting WH which does not use this UT

CSV import

Add CSV import to:

• team allocation
• service allocation
• extra costs (for single type)

Cannot edit Device because of empty device name

Cannot edit (and click) on a device due to empty device name.
Cannot find a form (i.e. in Ralph_assets) where the name should be obligatory

Input fields i18n

Input fields should allow numbers with spaces (thousands separator) and commas (decimal separator)

ralph makeconf failure

It doesn't add reports_pricing to the list. What are the required queues to run ralph in minimal way?

Double page refresh when generating the report

The scenario is as follows (will be difficult to reproduce at your site):

• I go to /scrooge/pricing-objects-usages-report and select my custom usage type (Active Energy)
• Press Update button
• It looks like the message "Calculating the report. Please wait..." which is displayed after button is presse appears for the second time one after another. In other words it look like the report is generated twice but is displayed after the message appears for the second time.

The worker section in supervisord.conf looks like this:

[program:rqworker]
command=/home/ralph/bin/ralph rqworker default scrooge_report scrooge_costs scrooge_collect
priority=12
numprocs=4
process_name=rq%(process_num)s
user=ralph
group=ralph

and the piece of log related to described behaviour is as follows:

[02.04.2015 11:50:51,488] INFO    [MainProcess 13585] report_pricing_objects_usages - Generating service environments dailyusages report from 2015-03-26 to 2015-04-02
[02.04.2015 11:50:51,522] INFO    [MainProcess 13585] base_report - Report generated
[02.04.2015 11:51:02,602] INFO    [MainProcess 13586] report_pricing_objects_usages - Generating service environments dailyusages report from 2015-03-26 to 2015-04-02
[02.04.2015 11:51:02,637] INFO    [MainProcess 13586] base_report - Report generated
[02.04.2015 11:51:14,021] ERROR   [MainProcess 21756] base - Internal Server Error: /scrooge/pricing-objects-usages-report/
Traceback (most recent call last):
  File "/home/ralph/local/lib/python2.7/site-packages/django/core/handlers/base.py", line 109, in get_response
    response = callback(request, *callback_args, **callback_kwargs)
  File "/home/ralph/local/lib/python2.7/site-packages/django/contrib/auth/decorators.py", line 20, in _wrapped_view
    return view_func(request, *args, **kwargs)
  File "/home/ralph/ralph_scrooge/src/ralph_scrooge/utils/security.py", line 24, in _wrapped_view
    return view_func(request, *args, **kwargs)
  File "/home/ralph/local/lib/python2.7/site-packages/django/views/generic/base.py", line 48, in view
    return self.dispatch(request, *args, **kwargs)
  File "/home/ralph/ralph_develop/src/ralph/account/models.py", line 297, in inner_decorator
    return func(self, *args, **kwargs)
  File "/home/ralph/ralph_scrooge/src/ralph_scrooge/views/base.py", line 28, in dispatch
    return super(Base, self).dispatch(*args, **kwargs)
  File "/home/ralph/ralph_develop/src/ralph/ui/views/common.py", line 250, in dispatch
    return super(MenuMixin, self).dispatch(request, *args, **kwargs)
  File "/home/ralph/local/lib/python2.7/site-packages/django/views/generic/base.py", line 69, in dispatch
    return handler(request, *args, **kwargs)
  File "/home/ralph/ralph_scrooge/src/ralph_scrooge/views/base_report.py", line 93, in get
    **self.form.cleaned_data
  File "/home/ralph/ralph_scrooge/src/ralph_scrooge/views/base_report.py", line 146, in run_on_worker
    return self.report.run_on_worker(**kwargs)
  File "/home/ralph/ralph_scrooge/src/ralph_scrooge/utils/worker_job.py", line 54, in run_on_worker
    job = Job.fetch(job_id, connection)
  File "/home/ralph/local/lib/python2.7/site-packages/rq/job.py", line 278, in fetch
    job.refresh()
  File "/home/ralph/local/lib/python2.7/site-packages/rq/job.py", line 377, in refresh
    raise NoSuchJobError('No such job: %s' % (key,))
NoSuchJobError: No such job: rq:job:7ba2c4c9-01df-4b6f-8159-cbfc586c663b

scrooge scrooge_ralph3_migrate_model , how to use this command

I have install scrooge !
Is the data must sync from ralph ?
if yes , how to user "scrooge scrooge_ralph3_migrate_model -m -f "

-f FILE, --file FILE CSV file with mapping Ralph2 (U)ID;Ralph3 I
the CSV file is download from the ralph ?

thanks !!!!

Usage types: Value rounding: Not rounding

Despite setting Value rounding: 'Decimal places = 2', when they are ending with '0' i.e. 10, 20, 50, they are shown as 1, 2, 5.
Examples:
47.50 is shown as 47.5
34.80 is shown as 34.8
etc

Report generating when environment has change for the service

The scenario is for my Devices usages report vel. PDUs Usaged report. And it is as follows:

• Some assets have beed moved to other location (service stayed the same)
• New environment has been created for this case
• old service has been linked (in CMDB) with new environment

PDUS usages report can be now generated for both service<->environments pairs (service->old env and service->new env).
If I choose new environment the usages should start from the moment I moved the assets to a new location. But they don't. Report for a new service-environment pair shows also historical data (from the times when assets were associated with old environment)