tarioch / beancounttools

Beancount Tools

tariochbctools

Some importers, plugins and price fetchers for the double-entry bookkeeping software Beancount.

See the detailed Documentation.

beancounttools's Issues

Nordigen EUA expiration

Hello, I'm using the Nordigen API from this package to download all transactions from my 'long tail' or non-main spending accounts (Revolut, etc.). Thanks so much for open sourcing this. However, occasionally I get the following message:

EUA was valid for 90 days and it expired at 2022-02-11 18:12:34.959764. The end user must connect the account once more with new EUA and Requisition","status_code":400}

What is the best way to generate a new EUA without having to link the accounts again which creates also a new UUID..

Revolut importer ignoring fees

@tarioch Is the Revolut importer on purpose ignoring the fees provided in the CSV file?
As the balances dont balance out when the fees are missing.

Cleanup IB download/parsing

As csingley/ibflex#8 got fixed, the code can be simplified a bit.

Revolut importer is missing meta field when returning a balance

I have problems using the Revolut importer with https://github.com/jbms/beancount-import
If the Revolut importer returns a balance, it is not providing the meta field and so beancount-import fails due to the missing filename.

beancounttools/src/tariochbctools/importers/revolut/importer.py

Line 80 in 5ae47a5

entry = data.Balance(meta, book_date, self.account, balance, None, None)

The importer should have the meta variable set too, like it does for transactions:

beancounttools/src/tariochbctools/importers/revolut/importer.py

Line 62 in 5ae47a5

meta = data.new_metadata(file.name, 0, metakv)

At least the sample importers of beancount are providing the meta with balance entries too:
https://github.com/beancount/beancount/tree/v2/beancount/ingest/importers/

I'll try to test and submit a PR later on.

Nordigen importer: include debtorName/creditorName

In the Nordigen API, there's a debtorName/creditorName field (depending on if it's an incoming or outgoing transaction). This data is very useful for labeling transactiosn/setting the incoming and outgoing account when importing transactions; it would be nice if this were included in the import.

generate_base_ccy_prices produces wrong prices

Thank you for this plugin, I was looking for something like this for quite some time (didn't think it was this easy to implement)

After switching the plugin on, my balance sheet didn't look right to me:
Input:

plugin "tariochbctools.plugins.generate_base_ccy_prices" "CCC"

2020-01-01 price X 100 USD
2020-01-01 price USD 2 CCC

it should produce

2020-01-01 price X 200 CCC <-- (not 50 CCC)

It's working after replacing division with multiplication:
https://github.com/tarioch/beancounttools/blob/master/src/tariochbctools/plugins/generate_base_ccy_prices.py#L14

Postfinance date format

Hi,

the csv import for postfinance expects a booking date with in a sensible ISO format:

book_date = datetime.strptime(book_date, "%Y-%m-%d").date()

However in my csv export from postfinance I get the following format:

Booking date;Notification text;Credit in CHF;Debit in CHF;Value;Balance in CHF
31.12.2022;"Preis für Bankpaket Smart";;-5;31.12.2022;7881.98

Am I using the wrong export or did they change the format? In case of the latter I can gladly provide a fix :-)

Quick help on IBKR importer

Hi,

Thanks for this very useful framework. I'm unfortunately quite new to python and am trying to get the ibkr importer customized for my setup. I had already tried to refer to the documentation but do not see this answered there (maybe because it is too basic). Essentially, I have 4 quick questions below:

1. I see this account (sample) in the code. How can I change it to the specific ones per my setup. Is it via the configuration file and if yes, is the below the right way for that?

    def getFeeAccount(self, account: str):
        return f"Expenses:{account}:Fees"

CONFIG = [
    apply_hooks(ibkrimp.Importer({
        'getFeeAccount'   : 'Expenses:Financial:Commissions:IBKR',
        }), [PredictPostings()]),
]

2. How do I customize the location of the ibkr.yaml file instead of the import directory? E.g. "~.config\beancount"

3. Does this importer also account for cash withdrawals and transfers in various currencies?

4. What are the specific fields needed for the Flex Web queries when putting together a customized report for this importer? Does the order of the fields matter?

Thank you so much for the kind guidance!

Nordigen importer crashes

(redacted) part of the crash log:

File "/home/user/venv/lib/python3.10/site-packages/tariochbctools/importers/nordigen/importer.py", line 75, in extract
    " ".join(trx["remittanceInformationUnstructuredArray"]),
KeyError: 'remittanceInformationUnstructuredArray'

When I go to the developer view over at the Nordigen website and look at my transactions, I see that the JSON dicts contain a remittanceInformationUnstructured key, but not the expected remittanceInformationUnstructuredArray key. This matches the API docs (https://ob.nordigen.com/api/docs#/accounts/retrieve%20account%20transactions%20v2_2). Perhaps the API got updated?

Integrate into jbms/beancount-import

It would be great to see these importers (and even some that were added in the forks like master...c-vigo:beancounttools:master) integrated/merged into https://github.com/jbms/beancount-import

Nordigen importer + smart importer give duplicate asset postings

Hi, first of all, thanks for providing this great package. I've been moving all the families' small accounts to the Nordigen API and was able to build some additional scripts and functionality to automatically generate new links when the account link expires and extend the max_historical_days to >90 days so I'm not in trouble when I forget to import data every 90 days. This proves to be great.

However, there's a problem that I'm struggling to debug. Specifically, about 30% of the Nordigen transactions (this issue doesn't occur with csvs, xls, etc.) end up with three postings when using smart_importer. Here's an example:

2023-08-19 * "amazon.co.uk"
  creditorName: "Amazon.co.uk*1f37b5qz4"
  nordref: "64e135f0-75fa-XXXX-XXXXXX-XXXXXX"
  Expenses:Shopping
  Assets:Person1:Bank:Revolut:GBP <--- Randomly added
  Assets:Person2:Bank:Revolut:GBP   -5.99 GBP

It always seems to add an extra random Asset: posting. After researching a while ago I stumbled upon an smart_import caching issue but that issue was fixed.

My importer looks like this:

# Nordigen API
apply_hooks(nordigen.Importer(), [categories, PredictPostings(), DuplicateDetector(comparator=ReferenceDuplicatesComparator('nordref'), window_days=10)])

Removing PredictPostings() from here gives me the right results.

I call bean-extract like this:

# filter only .yaml files to debug the Nordigen issue
bean-extract config.py ./import-files/*.yaml -e main.beancount > tmp.beancount && code tmp.beancount 

For the last months, I've been removing the extra postings with a regex find&replace but recently I found out it also impacts deduplication so it doesn't duplicate those transactions. Not sure if it's because of how the API calls are made or if it's a smart_imported issue. I'm importing 5 accounts/references through the Nordigen importer.

Any ideas that can point me in the right direction to a solution? Much appreciated!

PostFinance camt.053 xml support

As I saw that you have an importer for MT940 files from PostFinance and I am looking for an importer for the camt.053 xml files from PostFinance, I thought asking if you have the need for importing camt.053 files too?

Document how to install

Plugin fails to load

Hi!

I'm just starting out with beancount and have found your useful collection of tools. When adding your plugin to my beancount file (plugin "tariochbctools.plugins.generate_base_ccy_prices" "CHF") beancount spits out the following error:

<load>:0:       Error importing "tariochbctools.plugins.generate_base_ccy_prices": unsupported operand type(s) for *: 'decimal.Decimal' and 'NoneType'`

I'm using tariochbctools v0.6.0, beancount v2.2.3 and Python v3.7.4 on Windows.

Move price lookup over to official bean-price repo

• exchangeratesapi -> done, it's called ratesapi
• alphavantage/alphavantagefx -> beancount/beanprice#52
• bitstamp -> use coinbase or coinmarketcap instead
• coinmarketcap -> beancount/beanprice#50
• ibkr -> not going to move it, it's very special and has additional dependencies

Make price fetcher ccy flexible