How to deduplicate 2 transactions that are really one transfer from one bank account to another

[lestephane]

in bank1.journal, imported from bank1.csv

2018/11/09 (6138507615589995706)
    Personal:Assets:BANK1:Checking:EUR      EUR -11,22 = EUR 33,44
    Personal:Assets:BANK2:Girokonto:EUR

in bank2.journal, imported from bank2.csv

2018/11/09 (6138507615589995706)
    Personal:Assets:BANK2:Girokonto:EUR        EUR11,22
    Personal:Assets:BANK1:Checking:EUR       EUR-11,22

When I then run hledger-ui, I get an balance assertion error, because it sees two debits of EUR-11,22, not a single one, within the same day. I added the transaction code to try to make hledger-ui see one unique transaction, to no avail. It seems to work that way by design, which makes it difficult to handle transfers between my accounts in the hledger-flow world.

I tried to use virtual postings for bank2, to no avail.

What is the correct way to do this in the hledger world, and most importantly in the hledger-flow world?

Approach 1: Remove problematic line in the bank2 preprocess script

Probably the simplest approach, the bank2 preprocess script could have a grep -v <txid we want to ignore>, so that the transaction ends up in only one journal.

Problem: In the bank subdirectory, there is a CSV line for the transaction in question in 1-in, but no corresponding transaction in 3-journal. It feels like we're rewriting history.

Additionally, we now lose some information, such as when the day on which the money leaves bank1 is different from the day on which the money arrives in bank2.

Approach 2: Comment out the problematic journal entry using a bank2 construct script

Problem: You need to switch to using a construct file, you lose the ability to use a .rules file.

---

My current approach is Approach 1, since in my case bank2 uses a .rules file.

Have you encountered this situation, and how did you resolve it?

[apauley]

Thanks for raising this @lestephane, I think this is mostly a gap in our documentation at the moment.

The solution is to change your rules files to introduce an intermediate account as soon as you start importing statements for both accounts.

I use Assets:Transfers:*.

In your example bank1 would move money into something like Assets:Transfers:Bank1Bank2, and bank2 would use the same transfer account to fetch the money from.

Any posting to Assets:Transfers:* indicates in "in-flight" amount, and you would expect the balance of Assets:Transfers to be zero most of the time.
Whenever it isn't zero it means that you either don't yet have the other side of the transfer, or that something is wrong in your rules.

You can just use Assets:Transfers without any subaccounts, but I found it useful to use subaccounts, because I monitor Assets:Transfers in my reports, and then the subaccounts shows me where I should look for the missing transfer transaction.

I typically use the sorted account names as the subaccount (Python code sample):

"Assets:Transfers:" + "".join(sorted(["Bank2", "Bank1"]))

This approach is based on what is described in Full-fledged hledger:
https://github.com/adept/full-fledged-hledger/wiki/Adding-more-accounts#lets-make-sure-that-transfers-are-not-double-counted

[lestephane]

Thanks for the explanation and the full-fledged hledger page (which somehow passed me by),
it's a saner way of handling it.

I want the directionality of the transfer to be apparent without digging in the journal,
so I settled for a ":TopUp" prefix to my "Assets:" accounts. So if you have regular transfers
between two accounts in both directions, you'll have a pair of transfer accounts.

Example:
You want to buy and sell stocks regularly

• Bank -> Depot (for a buy order) uses Assets:Broker:Depot:TopUp
• Depot -> Bank (for a sell order) uses Assets:Bank:Girokonto:TopUp

I think this issue can be closed for now, as the problem seems well known, has a solution,
and the solution is well documented in the full-fledged hledger page.

[apauley]

I've updated the FAQ with this solution:
https://github.com/apauley/hledger-flow#faq