Skip to content

Idea: serial transformation representation #18

Description

@vreuter

Maybe sort of pie-in-the-sky, but I think It'd be cool to be able to provide a function that takes a list of instructions and interprets it as a sequential series of transformations from one cache to another, like Spark does with RDDs. The function could work backward from the end of the list, determining the filepath for the cache to be created or loaded and seeing if it exists. recreate would be used as-is but would only apply to the "leaf" cache, list elements would just be instructions, not entire bundles of simpleCache arguments. The function would backtrack until it hit an existing cache, then sequentially execute the instructions from there, generating the intermediate cache(s). I've found myself wanting to reuse caches between scripts, which forces a tradeoff. Either duplicate the code used to create it, or invoke loadCaches and lose the create-if-needed benefit of simpleCache.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions