internetee / eis_billing_system Goto Github PK

Current situation
We receive similar messages to this:

From: <[email protected]>
Date: Tue, Dec 6, 2022 at 9:22 AM
Subject: 136288 doesn't find
To: <[email protected]>, <[email protected]>, <[email protected]>

Reference number 136288 doesn't find from LHV transaction

Change email subject to "Billing: Ref no 136288 not found" (replace the number in the subject with the real number found in LHV-connect message)

In the email message/body please put the data/message that was received from the LHV-connect

1st of January 2024 new VAT rate of 22% comes into force.

In regards to that we need to set new vat rates to all services we provide to Estonian private and business registrants and registrars

This also affects data forwarded to the accounting system - there will be a new VAT code for that. Will declare the new value as soon as the accountant sets everything on their side.

Received this error from production on email:

undefined method `update' for nil:NilClass

email subject: Status received standard error

Received internal server error when tried to pay for an overdue auction invoice.
The invoice is missing from billing as it predates billing system so this might be the cause for the error.

Dec 13 11:40:59 testbilling billing system[2666363]: [a672ca42-4db0-4ff4-98e5-abb28c424e71] Started POST "/api/v1/invoice_generator/oneoff" for 192.168.100.122 at 2022-12-13 11:40:59 +0200
Dec 13 11:40:59 testbilling billing system[2666363]: [a672ca42-4db0-4ff4-98e5-abb28c424e71] Processing by Api::V1::InvoiceGenerator::OneoffController#create as JSON
Dec 13 11:40:59 testbilling billing system[2666363]: [a672ca42-4db0-4ff4-98e5-abb28c424e71]   Parameters: {"invoice_number"=>"250015", "customer_url"=>"https://testauction.internet.ee/linkpay_callback", "oneoff"=>{"invoice_number"=>"250015", "customer_url"=>"https://testauction.internet.ee/linkpay_callback"}}
Dec 13 11:40:59 testbilling billing system[2666363]: [a672ca42-4db0-4ff4-98e5-abb28c424e71] Completed 500 Internal Server Error in 64ms (ActiveRecord: 16.7ms | Allocations: 5879)
Dec 13 11:40:59 testbilling billing system[2666363]: [a672ca42-4db0-4ff4-98e5-abb28c424e71]   #012[a672ca42-4db0-4ff4-98e5-abb28c424e71] NoMethodError (undefined method `invoice_number' for nil:NilClass):#012[a672ca42-4db0-4ff4-98e5-abb28c424e71]   #012[a672ca42-4db0-4ff4-98e5-abb28c424e71] app/services/oneoff.rb:48:in `body'#012[a672ca42-4db0-4ff4-98e5-abb28c424e71] app/services/oneoff.rb:44:in `base_request'#012[a672ca42-4db0-4ff4-8e5-abb28c424e71] app/services/oneoff.rb:33:in `call'#012[a672ca42-4db0-4ff4-98e5-abb28c424e71] app/services/oneoff.rb:24:in `call'#012[a672ca42-4db0-4ff4-98e5-abb28c424e71] app/controllers/api/v1/invoice_generator/oneoff_controller.rb:6:in `create'

the lhv-connect checks only registry invoices at the moment, also the invoice status updating and syncing is available only for registry invoices.

As we have now examples of auction payments via bank transfer we need to make these features available for auction invoices as well

Subject: Invoice not found
To:

Invoice with 254401 number not found

the subject should include info where the message is coming from ie .ee billing

the email body could also include more information about the received payment

• date and time of the payment
• sum
• payment description
• reference number (viitenumber)

When importing invoice data, the following type of data comes into the amount field, although this column implies something else

Not right:

 transaction_amount:
  "{\"fractional\"=>\"4800.0\", \"currency\"=>{\"id\"=>\"eur\", \"alternate_symbols\"=>[], \"decimal_mark\"=>\",\", \"disambiguate_symbol\"=>nil, \"html_entity\"=>\"€\", \"iso_code\"=>\"EUR\", \"iso_numeric\"=>\"978\", \"name\"=>\"Euro\", \"priority\"=>2, \"smallest_denomination\"=>1, \"subunit\"=>\"Cent\", \"subunit_to_unit\"=>100, \"symbol\"=>\"€\", \"symbol_first\"=>true, \"thousands_separator\"=>\".\", \"format\"=>nil}, \"bank\"=>{\"store\"=>{\"rates\"=>{}, \"options\"=>{}, \"guard\"=>{}}, \"rounding_method\"=>nil}}",

Right

transaction_amount: "10.00",

1. registrar paid the topup invoice using openbanking bank link.
2. invoice was successfully marked as paid in billing and in registry and funds were added to the credit balance
3. lhv-connect picked up the payment (I, [2022-09-13T10:22:04.044336 #2071108] INFO -- : #<struct Lhv::ConnectApi::Messages::CreditDebitNotification::Transaction amount=240.0, currency="EUR", date=Tue, 13 Sep 2022, payment_reference_number="5337943", payment_description="billing.internet.ee/EE, st961268, 135845">)
4. new invoice was automatically created (135846) and additional funds added to the prepayment account

• we need to add some special label to the bank-link payments - no need to process these payments (should be already marked as paid)
• if an invoice for referred registrar for the same sum was paid within last 15 minutes do not create new invoice and notify admin about the received payment.

The billing system works in such a way that when it receives parsed data from an lhv connection, it first of all looks at the reference number to determine the initiator of this transaction. The Billing System is designed in such a way that it looks for this information in its database. But it may be that the reference number does not come or a completely different one comes, what then to do in this situation? Make an additional request to the registry or send information to the admin?

At the moment, authentication occurs through a token. There are some secret words with which a token is generated and added to the header. The jwt gem is used for this. But, the peculiarity of this gem is that it can encode json data specifically. And since communication occurs through json between services, it probably makes sense to make this data encoded using jwt. Thus, avoid adding tokens to the header (or leave it for better "protection")

https://gist.github.com/ratM1n/529d5f0009f3387a666191bb34c0e30b

not clear yet what happens there, what caused the error and if something failed because of this,

Auto-topup scenario:

1. create invoice
2. make bank transfer payment to EIS's account using correct sum and reference nr
3. run lhv-connect task

result: invoice is marked as paid in registry, credits are added to the registrar account, invoice is even sent to directo once the directo job is run, but in billing invoice is still marked as unpaid

random payment scenario:

1. make bank transfer payment to EIS's account using reference nr of an existing registrar
2. run lhv-connect task

result: new invoice was created for the correct registrar, the invoice was marked as paid in registry and the credits were added to the account. Invoice is also present in billing but still marked as unpaid

We have now multiple examples of billing system unable to match receipts with reference number stated in the payment description to the corresponding invoice

example1: description field contains only reference nr ie "5368815"
example2: description field contains text before ref nr ie ".EE PREPAYMENT REF 4783488"

both cases are registrars toping up their credit accounts. In neither case existing invoice was marked as paid nor was new invoice created indicating that the reference nr was not parsed/processed correctly from the description field

This issue lists Renovate updates and detected dependencies. Read the Dependency Dashboard docs to learn more.

Pending Status Checks

These updates await pending status checks. To force their creation now, click the checkbox below.

• Update dependency apipie-rails to "~> 1.3.0"

Open

These updates have all been created already. Click a checkbox below to force a retry/rebase of any.

• Update dependency rails to '~> 7.1.1'

Ignored or Blocked

These are blocked by an existing closed PR and will not be recreated unless you click a checkbox below.

• Update dependency ruby to v3.2.2
• Update dependency simplecov to v0.22.0

Detected dependencies

• Check this box to trigger a request for Renovate to run again on this repository

In theory, there may be such a situation that a data conflict has occurred, for example, an invoice with the number 10001 has been created in the registry, and it will receive the same number 10001 from the billing system. This is solved by running the rake eis_billing:import_invoices task. But, manually launching it every time is not the best, so we need to think about a mechanism for automating collision avoidance

Billing logs currently only into local log files.

For central management and monitoring the logs should be sent to syslog instead

https://gist.github.com/ratM1n/06969235487ba6e2dee4d2452162b8a8

it seems that the error is returned every time an invoice is sent to omniva

last two cases have errors also on omniva side. The one referenced in gist is not forwarded as the registrar does not accept e-invoices. The last two do accept e-invoices and omniva gets error while trying to forward.

error message: faili struktuur vigane või vale

got and email message from billing about and error: wrong number of arguments (given 1, expected 2)

Not entirely sure what was the request that failed and why. The log from this timerange is sent as private message - hopefully there is an explanation somewhere in there

We have multiple auto-topup payments and non are processed by the billing system - all are still marked as unpaid even though the payment is received, but strangely the in directo is marked as true

PS: there are double records in logs about every payment

PPS: we received an uninformative message from lhv-connect job

From: [email protected]
Date: Tue, Aug 30, 2022 at 10:14 AM
Subject: doesn't find

Reference number doesn't find

we have four instances of this error only from today. They look like to e related to the bulk payment functionality in auction, but probably not all as there are only three auction payments from today

sample log from one of the errors. https://gist.github.com/ratM1n/58c1558922f35ae468ede5797bae764f

But important thing to fix here is the message itself - it needs more details to enable to understand what is the issue - we are lacking source of the error (billing), function that triggered the error, function parameters etc

The reference number list in billing dashboard includes only ref numbers and create dates - would it be possible to add info about to whom is it assigned to?

System generates new invoices to handle bulk invoice payments in auction system for example. But system does not actually issue this invoice - its not in auction system, registry, its not sent to Directo not to the auction winner. But the invoice number is on the payment description. That makes difficult for the accountant to match payments to the invoices in Directo and now there is also one auction participant that requests the invoices for their own accounting.

Quick fix here would be to replace the bulk invoice nr in the payment description with the numbers of actual invoices being paid and not reference the bulk invoice nr in the payment description at all