Skip to content

dew.data.preferences

Preference pairs for DPO: chosen and rejected rows with their masks.

A PreferencePair step reads records of chosen and rejected token ids, lists or one-dimensional arrays, with chosen_mask and rejected_mask marking the completion tokens. Each batch stacks them in TRL’s layout: chosen rows over rejected rows, padded to the longest row, with completion_mask alongside the ids and real_mask marking each row’s ids apart from its padding. A mask must run as long as its ids; anything else fails naming the row.

NameSummary
IDS_KEYBatch key holding the [B, 2, S] pair ids, chosen at index 0.
MASK_KEYBatch key holding the [B, 2, S] completion marks, 1 on completion tokens.
REAL_KEYBatch key holding the [B, 2, S] marks of each half’s ids, 1 on its tokens and 0 on its padding.
FIELDS
PreferencePairA chosen and a rejected completion’s ids and loss masks laid out as two seq_len rows.

attribute source

IDS_KEY = 'input_ids'

Batch key holding the [B, 2, S] pair ids, chosen at index 0.

attribute source

MASK_KEY = 'completion_mask'

Batch key holding the [B, 2, S] completion marks, 1 on completion tokens.

attribute source

REAL_KEY = 'real_mask'

Batch key holding the [B, 2, S] marks of each half’s ids, 1 on its tokens and 0 on its padding.

attribute source

FIELDS = ('chosen', 'rejected', 'chosen_mask', 'rejected_mask')

dataclass source

class PreferencePair(seq_len: int = 256, *, pad_id: int = 0)

A chosen and a rejected completion’s ids and loss masks laid out as two seq_len rows.

def declared(facts: Mapping[str, object]) -> Mapping[str, object]

A chosen and a rejected row of seq_len ids and their completion mask, in place of the completions.

def map(element: Batch) -> Batch