zendag.core
- zendag.core.default_stage_dir_fn(stage, name)[source]
Generates the default path for a stage’s output directory.
- zendag.core.default_configs_dir_fn(stage)[source]
Generates the default path for storing composed stage configs.
- zendag.core.configure_pipeline(store, stage_groups, stage_dir_fn=<function default_stage_dir_fn>, configs_dir_fn=<function default_configs_dir_fn>, dvc_filename='dvc.yaml', run_script='zendag.run', config_root=None, manual_dvc=None)[source]
Configures the DVC pipeline based on Hydra-Zen stored configurations.
Generates composed Hydra configs for each stage instance and creates a dvc.yaml file defining the pipeline stages, dependencies, and outputs.
Dependencies and outputs are automatically discovered during Hydra config resolution via specially registered resolvers for ‘deps’ and ‘outs’.
- Parameters:
store (
ZenStore) – The Hydra-Zen store containing the configured stage components.stage_groups (
List[str]) – A list of stage group names (e.g., ‘training’, ‘data_prep’) present in the store. Stages within these groups will be processed.stage_dir_fn (
Callable[[str,str],str]) – A function fn(stage_name, config_name) -> str that returns the base output directory path for a given stage instance. Defaults to artifacts/<stage_name>/<config_name>.configs_dir_fn (
Callable[[str],str]) – A function fn(stage_name) -> str that returns the directory path where composed Hydra configs for a stage group will be stored. Defaults to artifacts/<stage_name>.dvc_filename (
str) – The name of the DVC pipeline file to generate. Defaults to ‘dvc.yaml’.run_script (
str) – The Python module path to execute for running a stage (e.g., ‘my_project.run’). Defaults to ‘zendag.run’.config_root (
Optional[str]) – The path relative to which Hydra should initialize (defaults to cwd). Needed if configs are stored outside the cwd.
- Return type: