linuxfabrik.lfops.combine_lod (filter)

Merge lists of dictionaries by a unique key

Synopsis

• Merges one or more lists of dictionaries into a single list. Dictionaries that share the same value under unique_key are folded into one entry; later entries overwrite earlier ones (non-recursive, like Python dict.update).
• The folding also collapses duplicates inside a single input list, so passing one list with duplicates is a valid way to deduplicate it.
• Useful for layering inventory. A role can ship sensible defaults, and the user can selectively override individual keys per item from their inventory without having to redeclare the whole list.
• Sub-dictionaries and sub-lists are replaced wholesale, not merged recursively. To merge nested structures, build the nested values up using this filter at each level.
• The result keeps the order in which each unique key is first seen across all input lists; later updates to an existing key do not move it.
• Every list item must be a dictionary; a non-dictionary item raises an error.
• Every input dictionary must contain the unique key (or all of them, when a list of keys is passed). A missing or falsy key value (None, 0, empty string) raises an error.

Available since LFOps 3.0.0.

Mandatory Parameters

_input

• First list of dictionaries to combine.
• Type: List.

Optional Parameters

_dicts

• Zero or more additional lists of dictionaries to combine. Items are folded in the order the lists are passed; the last occurrence of a unique key wins.
• Type: List.

unique_key

• Dictionary key (or list of keys, for composite keys) used to decide which entries belong together. Pass a list when no single key is unique on its own (e.g. ["server_name", "server_port"] for vHosts where the same hostname can appear on multiple ports).
• Type: Raw.
• Default: name

Examples

# create two lists of dictionaries
- set_fact:
    # this list could be in the role defaults
    role_defaults:
      - name: 'setting 1'
        first_value: 'sensible default for first value'
        second_value: 'sensible default for second value'
      - name: 'setting 2'
        allow_all: false
        allowed_users:
          - 'root'

    # this list could be in the users inventory
    my_user_adjustments:
      - name: 'setting 1'
        first_value: 'better setting for me'
      - name: 'setting 2'
        # only allow linuxfabrik, nobody else
        allowed_users:
          - 'linuxfabrik'

- name: 'combine for the actual use in the role'
  debug:
    msg: '{{ role_defaults | linuxfabrik.lfops.combine_lod(my_user_adjustments) }}'

# => (keys keep their first-seen order)
# - name: setting 1
#   first_value: better setting for me
#   second_value: sensible default for second value
# - name: setting 2
#   allow_all: false
#   allowed_users:
#   - linuxfabrik


# composite unique_key: the same server_name on different ports stays separate
- name: 'merge vhosts by name + port'
  debug:
    msg: >-
      {{ [{'server_name': 'example.com', 'server_port': 80, 'root': '/var/www'},
          {'server_name': 'example.com', 'server_port': 443, 'root': '/var/www'}]
         | linuxfabrik.lfops.combine_lod([{'server_name': 'example.com', 'server_port': 443, 'root': '/srv/tls'}],
                                         unique_key=['server_name', 'server_port']) }}

# =>
# - server_name: example.com
#   server_port: 80
#   root: /var/www
# - server_name: example.com
#   server_port: 443
#   root: /srv/tls


# more complicated example
# defaults:
mariadb_server__kernel_settings__sysctl__dependent_var:
  - name: 'fs.aio-max-nr'
    value: 1048576
  - name: 'sunrpc.tcp_slot_table_entries'
    value: 128
  - name: 'vm.swappiness'
    value: 10

redis__kernel_settings__sysctl__dependent_var:
  - name: 'vm.overcommit_memory'
    value: 1
  - name: 'net.core.somaxconn'
    value: 1024

kernel_settings__sysctl__dependent_var: '{{
    mariadb_server__kernel_settings__sysctl__dependent_var | d([]) +
    redis__kernel_settings__sysctl__dependent_var | d([])
  }}'

kernel_settings__sysctl__host_var:
  - name: 'net.core.somaxconn'
    value: 2048

kernel_settings__sysctl__group_var: []
kernel_settings__sysctl__role_var: []
kernel_settings__sysctl__combined_var: '{{ (
  kernel_settings__sysctl__role_var +
  kernel_settings__sysctl__dependent_var +
  kernel_settings__sysctl__group_var +
  kernel_settings__sysctl__host_var
  ) | linuxfabrik.lfops.combine_lod
 }}'

# tasks:
  - name: 'display combined var'
    debug:
      msg: '{{ kernel_settings__sysctl__combined_var }}'

# =>
# - name: fs.aio-max-nr
#   value: 1048576
# - name: sunrpc.tcp_slot_table_entries
#   value: 128
# - name: vm.swappiness
#   value: 10
# - name: vm.overcommit_memory
#   value: 1
# - name: net.core.somaxconn
#   value: 2048

Return Values

_value

• Resulting merged list of dictionaries.
• Type: List.