clowder.data package

Submodules

clowder.data.resolved_git_settings module

Representation of clowder yaml git settings

class clowder.data.resolved_git_settings.ResolvedGitSettings

Bases: object

clowder yaml GitSettings model class

Variables:
  • submodules (bool) – Whether to fetch submodules
  • recursive (bool) – Whether to fetch submodules recursively
  • lfs (bool) – Whether to set up lfs hooks and pull files
  • depth (bool) – Depth to clone git repositories
  • config (Optional[GitConfig]) – Custom git config values to set
get_processed_config() → Optional[Dict[str, str]]

Return version of config converted to strings

Returns:Config processed to create strings
Return type:Optional[Dict[str, str]]
update(git_settings: clowder.data.model.git_settings.GitSettings) → None

Update with overrides from given GitSettings

Returns:Config processed to create strings
Return type:dict

clowder.data.resolved_project module

Representation of clowder yaml project

class clowder.data.resolved_project.ResolvedProject(project: clowder.data.model.project.Project, defaults: Optional[clowder.data.model.defaults.Defaults] = None, group: Optional[clowder.data.model.group.Group] = None, protocol: Optional[str] = None)

Bases: object

clowder yaml Project model class

Variables:
  • name (str) – Project name
  • path (Path) – Project relative path
  • groups (Set[str]) – Groups project belongs to
  • remote (str) – Project remote name
  • source (Source) – Project source
  • git_settings (ResolvedGitSettings) – Custom git settings
  • upstream (Optional[ResolvedUpstream]) – Project’s associated upstream
  • default_protocol (Optional[str]) – Protocol defined in defaults
  • ref (str) – Project git ref
  • default_protocol – Default git protocol to use
branch(local: bool = False, remote: bool = False) → None

Print branches for project

Parameters:
  • local (bool) – Print local branches
  • remote (bool) – Print remote branches
checkout(branch: str) → None

Checkout branch

Parameters:branch (str) – Branch to check out
clean(args: str = '', submodules: bool = False) → None

Discard changes for project

Parameters:
  • args (str) – Git clean options - d Remove untracked directories in addition to untracked files - f Delete directories with .git sub directory or file - X Remove only files ignored by git - x Remove all untracked files
  • submodules (bool) – Clean submodules recursively
clean_all() → None

Discard all changes for project

Equivalent to: git clean -ffdx; git reset --hard; git rebase --abort git submodule foreach --recursive git clean -ffdx git submodule foreach --recursive git reset --hard git submodule update --checkout --recursive --force

diff() → None

Show git diff for project

Equivalent to: git status -vv

existing_branch(branch: str, is_remote: bool) → bool

Check if branch exists

Parameters:
  • branch (str) – Branch to check for
  • is_remote (bool) – Check for remote branch
Returns:

True, if branch exists
Return type:

bool
exists() → bool

Check if branch exists

Returns:True, if repo exists
Return type:bool
fetch_all() → None

Fetch upstream changes if project exists on disk

formatted_project_output() → str

Return formatted project path/name

Returns:Formatted string of full file path if cloned, otherwise project name
Return type:str
full_path() → pathlib.Path

Return full path to project

Returns:Project’s full file path
Return type:str
get_current_timestamp() → str

Return timestamp of current HEAD commit

Returns:HEAD commit timestamp
Return type:str
herd(branch: Optional[str] = None, tag: Optional[str] = None, depth: Optional[int] = None, rebase: bool = False, parallel: bool = False) → None

Clone project or update latest from upstream

Parameters:
  • branch (Optional[str]) – Branch to attempt to herd
  • tag (Optional[str]) – Tag to attempt to herd
  • depth (Optional[int]) – Git clone depth. 0 indicates full clone, otherwise must be a positive integer
  • rebase (bool) – Whether to use rebase instead of pulling latest changes
  • parallel (bool) – Whether command is being run in parallel, affects output
is_dirty() → bool

Check if project is dirty

Returns:True, if dirty
Return type:bool
is_valid(allow_missing_repo: bool = True) → bool

Validate status of project

Parameters:allow_missing_repo (bool) – Whether to allow validation to succeed with missing repo
Returns:True, if not dirty or if the project doesn’t exist on disk
Return type:bool
print_existence_message() → None

Print existence validation message for project

print_validation(allow_missing_repo: bool = True) → None

Print validation message for project

Parameters:allow_missing_repo (bool) – Whether to allow validation to succeed with missing repo
prune(branch: str, force: bool = False, local: bool = False, remote: bool = False) → None

Prune branch

Parameters:
  • branch (str) – Branch to prune
  • force (bool) – Force delete branch
  • local (bool) – Delete local branch
  • remote (bool) – Delete remote branch
reset(timestamp: Optional[str] = None, parallel: bool = False) → None

Reset project branch to upstream or checkout tag/sha as detached HEAD

Parameters:
  • timestamp (Optional[str]) – Reset to commit at timestamp, or closest previous commit
  • parallel (bool) – Whether command is being run in parallel, affects output
run(commands: List[str], ignore_errors: bool, parallel: bool = False) → None

Run commands or script in project directory

Parameters:
  • commands (list[str]) – Commands to run
  • ignore_errors (bool) – Whether to exit if command returns a non-zero exit code
  • parallel (bool) – Whether commands are being run in parallel, affects output
sha(short: bool = False) → str

Return sha for currently checked out commit

Parameters:short (bool) – Whether to return short or long commit sha
Returns:Commit sha
Return type:str
start(branch: str, tracking: bool) → None

Start a new feature branch

Parameters:
  • branch (str) – Local branch name to create
  • tracking (bool) – Whether to create a remote branch with tracking relationship
stash() → None

Stash changes for project if dirty

status(padding: Optional[int] = None) → str

Return formatted status for project

Parameters:padding (Optional[int]) – Amount of padding to use for printing project on left and current ref on right
Returns:Formatting project name and status
Return type:str
clowder.data.resolved_project.project_repo_exists(func)

If no git repo exists, print message and return

clowder.data.resolved_upstream module

Representation of clowder yaml upstream

class clowder.data.resolved_upstream.ResolvedUpstream(path: pathlib.Path, upstream: clowder.data.model.upstream.Upstream, defaults: Optional[clowder.data.model.defaults.Defaults], group: Optional[clowder.data.model.group.Group], protocol: Optional[str])

Bases: object

clowder yaml Upstream model class

Variables:
  • name (str) – Upstream name
  • path (str) – Project relative path
  • source (Source) – Upstream source
  • remote (str) – Upstream remote name
  • ref (str) – Upstream git ref
  • default_protocol (Optional[str]) – Default git protocol to use
full_path() → pathlib.Path

Return full path to project

Returns:Project’s full file path
Return type:Path
status() → str

Return formatted upstream status

Returns:Formatted upstream status
Return type:str
url() → str

Return project url

clowder.data.source_controller module

Source controller class

class clowder.data.source_controller.SourceController

Bases: object

Class encapsulating project information from clowder yaml for controlling clowder

Variables:protocol_override (Optional[str]) – The protocol to override sources without an explicitly specified protcol
add_source(source: Union[clowder.data.model.source.Source, clowder.data.model.source_name.SourceName, None])

Register source with controller

Parameters:SourceName]] source (Optional[Union[Source,) – Source to add
get_default_protocol() → str

Returns Source by name

Returns:Default git protocol
Return type:str
get_source(source: Union[clowder.data.model.source_name.SourceName, clowder.data.model.source.Source]) → clowder.data.model.source.Source

Returns Source by name

Parameters:Source] source (Union[SourceName,) – Source to return
Returns:Source with supplied name
Return type:Source
validate_sources() → None

Validate sources: check for unknown names

:raises

clowder.data.util module

Clowder model utilities

clowder.data.util.existing_branch_projects(projects: Tuple[clowder.data.resolved_project.ResolvedProject, ...], branch: str, is_remote: bool) → bool

Checks if given branch exists in any project

Parameters:
  • ..] projects (Tuple[ResolvedProject,) – Projects to check
  • branch (str) – Branch to check for
  • is_remote (bool) – Check for remote branch
Returns:

True, if at least one branch exists
Return type:

bool
clowder.data.util.filter_projects(projects: Tuple[clowder.data.resolved_project.ResolvedProject, ...], project_names: Tuple[str, ...]) → Tuple[clowder.data.resolved_project.ResolvedProject, ...]

Filter projects based on given project or group names

Parameters:
  • ..] projects (Tuple[ResolvedProject,) – Projects to filter
  • ..] project_names (Tuple[str,) – Project names to match against
Returns:

Projects in groups matching given names
Return type:

Tuple[ResolvedProject, ..]
clowder.data.util.print_parallel_projects_output(projects: Tuple[clowder.data.resolved_project.ResolvedProject, ...]) → None

Print output for parallel project command

Parameters:..] projects (Tuple[ResolvedProject,) – Projects to print output for
clowder.data.util.validate_project_statuses(projects: Tuple[clowder.data.resolved_project.ResolvedProject, ...], allow_missing_repo: bool = True) → None

Validate status of all projects

Parameters:
  • ..] projects (Tuple[ResolvedProject,) – Projects to validate
  • allow_missing_repo (bool) – Whether to allow validation to succeed with missing repo
Raises:

ClowderError –

Module contents

clowder.data module __init__