Configuration

The configuration file is a YAML file that contains the configuration of the importer. The configuration file is used to define the rules that the importer will use to process the data. The configuration file is divided into several sections, each of which defines a different aspect of the importer's behavior.

---

ImportDoc

The import configuration file for beancount-importer-rules.

Examples

import-config.yml

# yaml-language-server: $schema=https://raw.githubusercontent.com/zenobi-us/beancount-importer-rules/master/schema.json

inputs:
- match: "sources/*.csv" # (1)
  config:
    extractor:
        import_path: "extractors.my_extractor:YourExtractorClass" # (2)
        as_name: "custom name for this extractor instance"
        date_format: "%Y-%m-%d"
        datetime_format: "%Y-%m-%d %H:%M:%S"
    default_file: "books/{{ date.year }}.bean" # (3)
    prepend_postings:
        - account: "Assets:Bank"

imports:
- name: "simple"
    match:
      desc: "Simple Transaction"
    actions:
    - type: "add_txn"
      txn:
        date: "2021-01-01"
        flag: "*"
        narration: "Simple Transaction"
        postings:
            - account: "Expenses:Simple"
              amount:
                number: "{{ amount }}"
                currency: "USD"

1. pathname is relative to the workspace root
2. import path is relative to the workspace root
3. pathname is relative to the workspace root

You can view the schema for more details or refer to the ImportDoc api

context: dict | None = None class-attribute instance-attribute

Context comes in handy when you need to define variables to be referenced in the template. As you can see in the example, we define a routine_expenses dictionary variable in the context.

context:
  routine_expenses:
    "Amazon Web Services":
      account: Expenses:Engineering:Servers:AWS
    Netlify:
      account: Expenses:Engineering:ServiceSubscription
    Mailchimp:
      account: Expenses:Marketing:ServiceSubscription
    Circleci:
      account: Expenses:Engineering:ServiceSubscription
    Adobe:
      account: Expenses:Design:ServiceSubscription
    "Digital Ocean":
      account: Expenses:Engineering:ServiceSubscription
    Microsoft:
      account: Expenses:Office:Supplies:SoftwareAsService
      narration: "Microsoft 365 Apps for Business Subscription"
    "Mercury IO Cashback":
      account: Expenses:CreditCardCashback
      narration: "Mercury IO Cashback"
    WeWork:
      account: Expenses:Office
      narration: "Virtual mailing address service fee from WeWork"

Then, in the transaction template, we look up the dictionary to find out what narration value to use:

"{{ routine_expenses[desc].narration | default(desc, true) | default(bank_desc, true) }}"

imports: ImportList instance-attribute

The import rules

inputs: list[InputConfig] instance-attribute

The input rules

outputs: list[OutputConfig] | None = None class-attribute instance-attribute

The output configuration

ImportList

The list of import rules.

Can be a list of ImportRule or IncludeRule

IncludeRule

Include other yaml files that contain lists of ImportRule

include: str | list[str] instance-attribute

The file path(s) to include

ImportRule

An import rule to match and process transactions.

The following keys are available for the import configuration:

• name: An optional name for the user to comment on this matching rule. Currently, it has no functional purpose.
• match: The rule for matching raw transactions extracted from the input CSV files. As described in the Import Match Rule Definition
• actions: Decide what to do with the matched raw transactions, as the Import Action Definition describes.

actions: list[Action] instance-attribute

The actions to perform

common_cond: TxnMatchRule | None = None class-attribute instance-attribute

common condition to meet on top of the match rules

match: TxnMatchRule | list[TxnMatchVars] instance-attribute

The match rule

name: str | None = None class-attribute instance-attribute

Name of import rule, Not used, just for reference

ActionAddTxn

Add a transaction to the beancount file.

This is the default action type. If your action does not specify a type, it will be assumed to be an add transaction action.

The following keys are available for the add transaction action:

• file: output beancount file name to write the transaction to
• txn: the template of the transaction to insert

A transaction template is an object that contains the following keys:

• id: the optional import-id to overwrite the default one. By default, {{ file | as_posix_path }}:{{ lineno }} will be used unless the extractor provides a default value.
• date: the optional date value to overwrite the default one. By default, {{ date }} will be used.
• flag: the optional flag value to overwrite the default one. By default, * will be used.
• narration: the optional narration value to overwrite the default one. By default {{ desc | default(bank_desc, true) }} will be used.
• payee: the optional payee value of the transaction.
• tags: an optional list of tags for the transaction
• links: an optional list of links for the transaction
• metadata: an optional list of name and value objects as the metadata items for the transaction.
• postings: a list of templates for postings in the transaction.

The structure of the posting template object looks like this.

• account: the account of posting
• amount: the optional amount object with number and currency keys
• price: the optional amount object with number and currency keys
• cost: the optional template of cost spec

file: str | None = None class-attribute instance-attribute

Which file to add the transaction to. If not provided, the default file will be used.

txn: TransactionTemplate instance-attribute

The transaction transform template

type: typing.Literal[ActionType.add_txn] = pydantic.Field(ActionType.add_txn) class-attribute instance-attribute

indicates that this action is to add a transaction

ActionDelTxn

Delete a transaction from the beancount file.

The following keys are available for the delete transaction action:

• txn: the template of the transaction to insert

A deleting transaction template is an object that contains the following keys:

• id: the import-id value for ensuring transactions to be deleted. By default, {{ file | as_posix_path }}:{{ lineno }} will be used unless the extractor provides a default value.

txn: DeleteTransactionTemplate instance-attribute

The transaction to delete

type: typing.Literal[ActionType.del_txn] = pydantic.Field(ActionType.del_txn) class-attribute instance-attribute

indicates that this action is to delete a transaction

ActionIgnore

Ignore the transaction.

This prevents the transaction from being added to the beancount file.

Sometimes, we are not interested in some transactions, but if we don't process them, you will still see them appear in the "unprocessed transactions" section of the report provided by our command line tool. To mark one transaction as processed, you can simply use the ignore action like this:

- name: Ignore unused entries
  match:
    extractor:
      equals: "mercury"
    desc:
      one_of:
        - Mercury Credit
        - Mercury Checking xx1234
  actions:
    - type: ignore

type: typing.Literal[ActionType.ignore] = pydantic.Field(ActionType.ignore) class-attribute instance-attribute

indicates that this action is to ignore the transaction

SimpleTxnMatchRule

The raw transactions extracted by the extractor come with many attributes. Here we list only a few from it:

The match object should be a dictionary.

The key is the transaction attribute to match, and the value is the regular expression of the target pattern to match.

All listed attributes need to match so that a transaction will considered matched.

Only simple matching logic is possible with the current approach.

We will extend the matching rule to support more complex matching logic in the future, such as NOT, AND, OR operators.

bank_desc: StrMatch | None = None class-attribute instance-attribute

The bank description of the transaction to match.

category: StrMatch | None = None class-attribute instance-attribute

The category of the transaction to match.

currency: StrMatch | None = None class-attribute instance-attribute

The currency of the transaction to match.

date: StrMatch | None = None class-attribute instance-attribute

The date of the transaction to match.

Examples

imports:
- match:
    date: "2021-01-01"

imports:
- match:
    date:
        before: "2021-01-01"
        format: "%Y-%m-%d"

imports:
- match:
    date:
        after: "2021-01-01"
        format: "%Y-%m-%d"

desc: StrMatch | None = None class-attribute instance-attribute

The description of the transaction to match.

Probably the most common field to match.

Examples

imports:
- match:
    desc: "DoorDash"

imports:
- match:
    desc:
        prefix: "DoorDash"

imports:
- match:
    desc:
        suffix: "DoorDash"

imports:
- match:
    desc:
        contains: "DoorDash"

imports:
- match:
    desc:
        one_of:
            - DoorDash
            - UberEats
            - Postmate

dest_account: StrMatch | None = None class-attribute instance-attribute

The destination account of the transaction to match.

extractor: StrMatch | None = None class-attribute instance-attribute

The extractor to match. This will be produced by the Extractor.get_name() method.

file: StrMatch | None = None class-attribute instance-attribute

The file to match. This will be the file path of the source file.

Examples

imports:
- match:
    file: "data/transactions.csv"

imports:
- match:
    file:
        prefix: "data/"

imports:
- match:
    file:
        suffix: ".csv"

gl_code: StrMatch | None = None class-attribute instance-attribute

The general ledger code of the transaction to match.

last_four_digits: StrMatch | None = None class-attribute instance-attribute

The last four digits of the card of the transaction to match.

name_on_card: StrMatch | None = None class-attribute instance-attribute

The name on the card of the transaction to match.

note: StrMatch | None = None class-attribute instance-attribute

The note of the transaction to match.

payee: StrMatch | None = None class-attribute instance-attribute

The payee of the transaction to match.

post_date: StrMatch | None = None class-attribute instance-attribute

The post date of the transaction to match.

Examples

imports:
- match:
    post_date: "2021-01-01"

imports:
- match:
    post_date:
        before: "2021-01-01"
        format: "%Y-%m-%d"

reference: StrMatch | None = None class-attribute instance-attribute

The reference of the transaction to match.

source_account: StrMatch | None = None class-attribute instance-attribute

The source account of the transaction to match.

status: StrMatch | None = None class-attribute instance-attribute

The status of the transaction to match.

subcategory: StrMatch | None = None class-attribute instance-attribute

The subcategory of the transaction to match.

timezone: StrMatch | None = None class-attribute instance-attribute

The timezone of the transaction to match.

Examples

imports:
- match:
    timezone: "America/Los_Angeles"

transaction_id: StrMatch | None = None class-attribute instance-attribute

The transaction id of the transaction to match.

type: StrMatch | None = None class-attribute instance-attribute

The type of the transaction to match.

TransactionTemplate

A transaction transform template.

Used to transform the raw transaction into a beancount transaction.

txn:
    date: "2021-01-01"
    flag: "*"
    narration: "Simple Transaction"
    metadata:
        - name: "icon"
          value: "🍔"
    postings:
        - account: "Expenses:Simple"
          amount:
              number: "{{ amount }}"
              currency: "USD"

results in the following beancount transaction:

2021-01-01 * "Simple Transaction"
    icon: 🍔
    Expenses:Simple                            100 USD

date: str | None = None class-attribute instance-attribute

the date of the transaction

flag: str | None = None class-attribute instance-attribute

the flag of the transaction

id: str | None = None class-attribute instance-attribute

the import-id for de-duplication

links: list[str] | None = None class-attribute instance-attribute

the links of the transaction

metadata: list[MetadataItemTemplate] | None = None class-attribute instance-attribute

the metadata of the transaction

narration: str | None = None class-attribute instance-attribute

the narration of the transaction

payee: str | None = None class-attribute instance-attribute

the payee of the transaction

postings: list[PostingTemplate] | None = None class-attribute instance-attribute

the postings of the transaction

tags: list[str] | None = None class-attribute instance-attribute

the tags of the transaction

DeleteTransactionTemplate

A transaction delete template.

id: str | None = None class-attribute instance-attribute

the import-id for deleting

MetadataItemTemplate

A metadata list item template.

txn:
    metadata:
        - name: "import-id"
          value: "123456"
        - name: "import-src"
          value: "plaid"

name: str instance-attribute

the name of the metadata

value: str | bool | int | None instance-attribute

the value of the metadata

PostingTemplate

A posting transform template.

Used to transform the raw transaction into a beancount posting.

txn:
    postings:
        - account: "Expenses:Simple"
          amount:
              number: "{{ amount }}"
              currency: "USD"

account: str | None = None class-attribute instance-attribute

The account of the posting.

amount: AmountTemplate | None = None class-attribute instance-attribute

The amount of the posting.

cost: str | None = None class-attribute instance-attribute

The cost of the posting.

price: AmountTemplate | None = None class-attribute instance-attribute

The price of the posting.

AmountTemplate

StrPrefixMatch

To match values with a prefix, one can do this:

imports:
- match:
    desc:
      prefix: "DoorDash"

prefix: str instance-attribute

Does the transaction field start with the string

StrSuffixMatch

To match values with a suffix, one can do this:

imports:
- match:
    desc:
      suffix: "DoorDash"

suffix: str instance-attribute

Does the transaction field end with the string

StrExactMatch

To match an exact value, one can do this:

imports:
- match:
    desc:
      equals: "DoorDash"

equals: str instance-attribute

Does the transaction field equal the value

StrContainsMatch

To match values containing a string, one can do this:

imports:
- match:
    desc:
      contains: "DoorDash"

contains: str instance-attribute

Does the transaction field contain the string

StrOneOfMatch

To match values belonging to a list of values, one can do this:

imports:
- match:
    desc:
      one_of:
        - DoorDash
        - UberEats
        - Postmate

one_of: list[str] instance-attribute

Does the transaction field match one of the values

DateAfterMatch

To match values after a date, one can do this:

imports:
- match:
    date:
      date_after: "2021-01-01"
      format: "%Y-%m-%d"

date_after: str instance-attribute

The date to match after

format: str instance-attribute

The format of the date. used to parse the value date and the date to match

DateBeforeMatch

To match values before a date, one can do this:

imports:
- match:
    date:
      date_before: "2021-01-01"
      format: "%Y-%m-%d"

date_before: str instance-attribute

The date to match before

format: str instance-attribute

The format of the date. used to parse the value date and the date to match

DateSameDayMatch

To match values with the same day, one can do this:

imports:
- match:
    date:
      date_same_day: "2021-01-01"
      format: "%Y-%m-%d"

date_same_day: str instance-attribute

The date to match on the day

format: str instance-attribute

The format of the date. used to parse the value date and the date to match

DateSameMonthMatch

To match values with the same month, one can do this:

imports:
- match:
    date:
      date_same_month: "2021-01-01"
      format: "%Y-%m-%d"

date_same_month: str instance-attribute

The date to match on the month

format: str instance-attribute

The format of the date. used to parse the value date and the date to match

DateSameYearMatch

To match values with the same year, one can do this:

imports:
- match:
    date:
      date_same_year: "2021-01-01"
      format: "%Y-%m-%d"

date_same_year: str instance-attribute

The date to match on the year

format: str instance-attribute

The format of the date. used to parse the value date and the date to match

TxnMatchVars

From time to time, you may find yourself writing similar import-matching rules with similar transaction templates. To avoid repeating yourself, you can also write multiple match conditions with their corresponding variables to be used by the template in the same import statement. For example, you can simply do the following two import statements:

imports:
- name: PG&E Gas
    match:
    extractor:
      equals: "plaid"
    desc:
      prefix: "PGANDE WEB ONLINE "
    actions:
    - txn:
        payee: "{{ payee }}"
        narration: "Paid American Express Blue Cash Everyday"
        postings:
        - account: "Expenses:Util:Gas:PGE"
          amount:
            number: "{{ -amount }}"
            currency: "{{ currency | default('USD', true) }}"

- name: Comcast
  match:
    extractor:
      equals: "plaid"
    desc: "Comcast"
  actions:
  - txn:
      payee: "{{ payee }}"
      narration: "Comcast"
      postings:
      - account: "Expenses:Util:Internet:Comcast"
        amount:
          number: "{{ -amount }}"
          currency: "{{ currency | default('USD', true) }}"

With match and variables, you can write:

imports:
- name: Household expenses
  common_cond:
    extractor:
      equals: "plaid"
  match:
  - cond:
      desc:
          prefix: "PGANDE WEB ONLINE "
      vars:
      account: "Expenses:Util:Gas:PGE"
      narration: "Paid American Express Blue Cash Everyday"
  - cond:
      desc: "Comcast"
      vars:
      account: "Expenses:Housing:Util:Internet:Comcast"
      narration: "Comcast"
  actions:
  - txn:
      payee: "{{ payee }}"
      narration: "{{ narration }}"
      postings:
      - account: "{{ account } "
          amount:
            number: "{{ -amount }}"
            currency: "{{ currency | default('USD', true) }}"

The common_cond is the condition to meet for all the matches. Instead of a map, you define the match with the cond field and the corresponding variables with the vars field. Please note that the vars can also be the Jinja2 template and will rendered before feeding into the transaction template. If there are any original variables from the transaction with the same name defined in the vars field, the variables from the vars field always override.

InputConfig

The input configuration for the import rule.

OutputConfig