Skip to content

Memory issues for large inputs in loss  #3

@TimRoith

Description

@TimRoith

In bc69cd0 the multirun strategy via $M\times N\times d$ particles was introduced. In the typical use case we assume a function

$$ \begin{align*} \tilde{f} :\mathbb{R}^{k\times d} &\to \mathbb{R}^{ k \times d}\\ x&\mapsto (f(x_{1,:}) ,\ldots, f(x_{k, :})) \end{align*} $$

where $f$ denotes the given objective. In order to apply this function on $x\in\mathbb{R}^{M\times N\times d}$ we would transform $x$ via reshaping to $\tilde{x}\in\mathbb{R}^{(M\cdot N) \times d}$ and use this as the input. However, this amounts to applying $f$ to $M\cdot N$ vectors simultaneously. This is handy and quick for loss functions that are easy to evaluate, but problematic for memory-heavy loss functions. There we would need a batching strategy so that the loss is successively applied to sub-batches of $\tilde{x}$.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions