michaelchiew08 / tng_statement_in_csv

Build a CSV table that records all transactions within TNG e-Wallet.

TNG Statement in CSV

This code builds a CSV table that records all transactions within TNG e-Wallet from its statement.

Usage

1. Clone this repo.

   git clone https://github.com/Rexpert/TNG_Statement_in_CSV.git
   

2. You need to install Python 3 and its relevant dependencies:

   **For Mac / Linux / Windows WSL user

   It is suggested to setup Python virtual environment within the project folder. Assuming that you have successfully installed Python 3 in your machine, and your starting command can either be python or python3 (Use the one that works for you).

   Below command is to create a virtual environment with folder name .venv inside your project.

   python -m venv .venv

   Below command is to activate the virtual environment. After activating it, the rest of the dependencies installation can continue as follow.

   source .venv/bin/activate

   Command to deactivate the virtual environment.

   deactivate

   camelot-py: to read PDF statement

   • Installation via pip

     pip install camelot-py
     

   • or if you're using conda environment

     conda install -c conda-forge camelot-py
     

   • Detail installation please refer to camelot-py Documentation
   pandas: data manipulation

   • Installation via pip

     pip install pandas
     

   • or if you're using conda environment

     conda install -c conda-forge pandas
     

   • Detail installation please refer to pandas Documentation
   opencv-python: Handle missing / no module named cv2

   • Installation via pip

     pip install opencv-python
     

   • or if you're using conda environment

     conda install -c conda-forge opencv
     

   matplotlib: page visualization (Optional)

   • Installation via pip

     pip install matplotlib
     

   • or if you're using conda environment

     conda install -c conda-forge matplotlib
     

   • Visual Debug on table generation
   • Detail installation please refer to matplotlib Documentation
   Recommended setup:

   Installation	Version
   python	3.9.12
   camelot-py	0.10.1
   pandas	1.4.3
   opencv-python	4.9.0
   matplotlib	3.5.2

3. Download your TNG statement (which is named tng_ewallet_transactions.pdf) and locating it into a data folder

4. Run the main.py

   python main.py
   

5. Get your transaction table named tng_ewallet_transactions.csv in output folder.

Troubleshoot

Some known bugs happen during the generation of the pdf transaction report by TNG, but the only thing we can do is to manually make correction on the data:

1. Reverse Entry (Found on c5156d7)

   • The latest transaction recorded before an older transaction.
   • This usually happens during the Quick Reload Payment via Go+. In this scenario, the payment is recorded first, then the reload occurs after.
   • I have implemented an autofix in the code to address this.
   • Example:

2. Money Packet Balance (Found on 8fe26a5)

   • The wallet balance of the money packet entries unexpectedly equals the amount of money packet received.
   • This is happened in the CNY 2023 when the Money Packet Campaign took place.
   • I have implemented an autofix in the code to address this.
   • Example:

3. Missing Direct Credit entry (Found on 362cc8a)

   • Direct Credit entries are not recorded in the transaction history
   • Some of us might be involved in the Weekly Check-in on the A+ reward. The check-in for 9 Sept 2023, 10 Sept 2023 and 12 Sept 2023 rewards free credits, but the transactions are not recorded in the pdf. However, they can be viewed in the TNG e-Wallet app's history.
   • You need to input those transactions manually if you were involved in those rewards, otherwise the ValueError: Some Entry Not Recorded Properly will be raised.
   • Example:

4. Other Unknown Bugs

   • Any uncaught bug will raise ValueError: Some Entry Not Recorded Properly and exit the code unexpectedly. Please open an issue and attach/screenshot the relevant transaction history pdf if found such case.

Disclaimer

1. I don't work in Touch 'n Go company, and hence do not represent Touch 'n Go.
2. This repository is my work to ease myself in analyzing my own expenses in Touch 'n Go e-Wallet. But you can freely use it and welcome to contribute, you are helping me to make this code more meaningful.
3. Please consider use this code in your own responsibility, although it is not harmful at all.

MIT @Rexpert 2022