Project Data

Functions

wrapica.project_data.add_tag_to_data_object(project_id, data_id, tag, tag_type)

Add tags to a data object

Parameters:

  • project_id (Union[UUID, str])

  • data_id (Union[UUID, str])

  • tag (str)

  • tag_type (Literal['technical_tag', 'user_tag', 'connector_tag', 'run_in_tag', 'run_out_tag', 'reference_tag'])

Return type:

ProjectData

Returns:

Returns:

Raises:

ApiException

Examples:

add_tag_to_data_obj(
  project_id,
  data_id,
  "to_be_archived"
)
wrapica.project_data.check_file_exists(project_id, file_path)

Check if a file exists in a project

Parameters:

  • project_id (Union[UUID, str]) – The owning project id of the file

  • file_path (Path) – The file path

Returns:

True if file exists, otherwise false

Return type:

bool

Examples:

# Imports
from wrapica.project_data import check_file_exists

# Check if a file exists
print(check_file_exists("abcdef1234567890", Path("/path/to/file.txt")))
wrapica.project_data.check_folder_exists(project_id, folder_path)

Check folder path is a folder on icav2

Parameters:

  • project_id (Union[UUID, str]) – The owning project id of the folder

  • folder_path (Path) – The folder path

Returns:

True if folder exists, otherwise false

Return type:

bool

Examples:

1# Imports
2from wrapica.project_data import check_folder_exists
3
4# Check if a folder exists
5print(check_folder_exists("abcdef1234567890", Path("/path/to/folder/")))
wrapica.project_data.check_uri_exists(data_uri)

Given a uri, check if the uri exists

Parameters:

data_uri (str)

Returns:

True if the uri exists, otherwise false

Return type:

bool

Examples:

1from wrapica.project_data import check_uri_exists
2
3print(check_uri_exists("icav2://project-name/path/to/file.txt"))
wrapica.project_data.coerce_data_id_icav2_uri_or_path_to_project_data_obj(data_id_path_or_uri, create_data_if_not_found=False)
Return type:

Optional[ProjectData]

wrapica.project_data.coerce_data_id_or_icav2_uri_to_project_data_obj(data_id_or_uri, create_data_if_not_found=False)
Return type:

ProjectData

wrapica.project_data.coerce_data_id_or_uri_to_project_data_obj(data_id_or_uri, create_data_if_not_found=False)

Given a data id or a uri, return a tuple of project id and data id

Parameters:

  • data_id_or_uri (str) – The data id or uri

  • create_data_if_not_found (bool) – If uri_or_id is in uri format, and the data is not found, and create is True, create the data

Return type:

ProjectData

Returns:

wrapica.project_data.coerce_data_id_uri_or_path_to_project_data_obj(data_id_path_or_uri, create_data_if_not_found=False)

Coerce a data id, uri, path to a project data object

Parameters:

  • data_id_path_or_uri (str)

  • create_data_if_not_found (bool)

Returns:

A ProjectData object

Return type:

ProjectData

Raises:

ValueError, ApiException

Examples:

1from wrapica.project_data import coerce_data_id_uri_or_path_to_project_data_obj
2
3project_data_obj = coerce_data_id_uri_or_path_to_project_data_obj(
4    data_id_path_or_uri="icav2://project-name/path/to/data/"
5)
wrapica.project_data.convert_icav2_uri_to_data_obj(data_uri, create_data_if_not_found=False)
Return type:

ProjectData

wrapica.project_data.convert_icav2_uri_to_project_data_obj(data_uri, create_data_if_not_found=False)
Return type:

ProjectData

wrapica.project_data.convert_project_data_obj_to_icav2_uri(project_data)

Return the file object as an icav2:// uri

Parameters:

project_data (ProjectData) – The ProjectData object

Return type:

str

Returns:

The icav2:// uri string

wrapica.project_data.convert_project_data_obj_to_uri(project_data, uri_type='icav2')

Return the file object as an icav2:// uri

Parameters:

  • project_data (ProjectData) – The ProjectData object

  • uri_type (Literal['s3', 'icav2']) – One of UriType.ICAV2, UriType.S3

Return type:

str

Returns:

The icav2:// uri string

wrapica.project_data.convert_project_id_and_data_path_to_icav2_uri(project_id, data_path, data_type)
Return type:

str

wrapica.project_data.convert_project_id_and_data_path_to_uri(project_id, data_path, data_type, uri_type='icav2')

Given a project_id and a data_id, return the icav2:// uri

Unlike convert_project_data_obj_to_icav2_uri, this does not require the path to exist.

If the data_type is “FOLDER”, the path should end with a forward slash.

Parameters:

  • project_id (Union[UUID, str]) – The project id to search in

  • data_path (Path) – The path to the data in the project

  • data_type (Literal['FOLDER', 'FILE']) – The data_type, one of “FILE”, “FOLDER”

  • uri_type (Literal['s3', 'icav2']) – One of UriType.ICAV2, UriType.S3

Returns:

The icav2:// uri string

Return type:

str

Examples:

:linenos:
from wrapica.project_data import convert_project_id_and_data_path_to_uri
from wrapica.enums import DataType

icav2_uri: str = convert_project_id_and_data_path_to_icav2_uri(
    project_id="abcd-1234-efab-5678",
    data_path=Path("/path/to/folder/"),
    data_type="FOLDER"
)
wrapica.project_data.convert_uri_to_project_data_obj(data_uri, create_data_if_not_found=False)

Given an ICAv2 URI, return a project data object

Parameters:

  • data_uri (str) – The icav2 uri to convert to a data object

  • create_data_if_not_found (bool) – If the data is not found, and create_data_if_not_found is True, create the data

Returns:

libica v2 Project Data Object

Return type:

Project Data

Raises:

ValueError, ApiException

Examples:

1from wrapica.project_data import convert_uri_to_project_data_obj, ProjectData
2
3project_data_object: ProjectData = convert_uri_to_project_data_obj(
4    "icav2://project-name/path/to/data/"
5)
6
7print(project_data_object.data.id)
8# file.abcdef1234567890
wrapica.project_data.create_data_in_project(project_id, parent_folder_path, data_name, data_type)

Create a data object in a project context

Parameters:

  • project_id (Union[UUID, str]) – The project ID

  • parent_folder_path (Path) – The parent folder path of where the data object needs to be created

  • data_name (str) – The name of the file or folder

  • data_type (Literal['FOLDER', 'FILE']) – One of “FILE” or “FOLDER”

Returns:

The newly created project data object

Return type:

List[ProjectData]

Raises:

ApiException

Examples:

 1from pathlib import Path
 2from wrapica.project_data import (
 3    create_data_in_project, create_data_in_project,
 4)
 5from wrapica.enums import DataType
 6from wrapica.libica_models import ProjectData
 7
 8# Use wrapica.project.get_project_id_from_project_name
 9# If you need to convert a project_name to a project_id
10
11project_data_obj: ProjectData = create_data_in_project(
12    project_id="abcd-1234-efab-5678",
13    parent_folder_path=Path("/path/to/folder/"),
14    data_name="file.txt",
15    data_type="FILE"
16)
wrapica.project_data.create_download_url(project_id, file_id)

Given a project_id and a data_id, create a presigned url for a file in project-pipeline, or view a file in project-data Create download URL

Parameters:

  • project_id (Union[UUID, str]) – The owning project id

  • file_id (Union[UUID, str]) – The id of the file

Returns:

The download URL string

Return type:

str

Raises:

ApiException

Examples:

 1from wrapica.project_data import create_download_url
 2
 3# Use wrapica.project.get_project_id_from_project_name
 4# If you need to convert a project_name to a project_id
 5
 6download_url: str = create_download_url(
 7    project_id="proj.abcdef1234567890",
 8    file_id="fil.abcdef1234567890"
 9)
10
11print(download_url)
12# https://s3.amazonaws.com/umccr-illumina-prod/abcd-1234-efab-5678/abcdef1234567890
wrapica.project_data.create_download_urls(project_id, folder_id, recursive=False)

Given a project data folder return a list where each item is an object with the following attributes

Parameters:

  • project_id (Union[UUID, str]) – The owning project id

  • folder_id (Union[UUID, str]) – The id of the folder

  • recursive (bool) – Whether to provide download urls recursively

Returns:

List of download urls

Return type:

List[DataUrlWithPath]

Examples:

 1from wrapica.project_data import (
 2    create_download_urls, get_project_folder_id_from_project_id_and_path
 3)
 4from wrapica.libica_models import ProjectData, DataUrlWithPath
 5
 6# Use wrapica.project.get_project_id_from_project_name
 7# If you need to convert a project_name to a project_id
 8
 9project_folder_obj: ProjectData = get_project_folder_id_from_project_id_and_path(
10    project_id="abcd-1234-efab-5678",
11    folder_path=Path("/path/to/folder/")
12)
13
14download_urls: List[DataUrlWithPath] = create_download_urls(
15    project_id="proj.abcdef1234567890",
16    folder_id=project_folder_obj.data.id,
17    recursive=True
18)
19
20for download_url in download_urls:
21    print(download_url.url)
wrapica.project_data.create_file_in_project(project_id, file_path)

Create a file in a project

Parameters:

  • project_id (Union[UUID, str]) – The project id to create the file in

  • file_path (Path) – The path to the file

Returns:

The newly created file

Return type:

ProjectData

Raises:

ApiException

Examples:

 1from pathlib import Path
 2from wrapica.project_data import create_file_in_project
 3from wrapica.libica_models import ProjectData
 4
 5# Use wrapica.project.get_project_id_from_project_name
 6# If you need to convert a project_name to a project_id
 7
 8project_data_obj: ProjectData = create_file_in_project(
 9    project_id="abcd-1234-efab-5678",
10    file_path=Path("/path/to/file.txt")
11)
wrapica.project_data.create_file_with_upload_url(project_id, folder_id, file_name)

Create a new file in a project and return the upload URL

Parameters:

  • project_id (Union[UUID, str]) – The owning project id of the file

  • folder_id (Union[UUID, str]) – The folder id to create the file in

  • file_name (str) – The name of the file to create

Return type:

str

Returns:

wrapica.project_data.create_folder_in_project(project_id, folder_path)

Create a folder in a project

Parameters:

  • project_id (Union[UUID, str]) – The project ID

  • folder_path (Path) – The path to the folder to create

Returns:

The newly created folder project data object

Return type:

ProjectData

Raises:

ApiException

Examples:

 1from pathlib import Path
 2from wrapica.project_data import create_folder_in_project
 3from wrapica.libica_models import ProjectData
 4
 5# Use wrapica.project.get_project_id_from_project_name
 6# If you need to convert a project_name to a project_id
 7
 8project_data_obj: ProjectData = create_folder_in_project(
 9    project_id="abcd-1234-efab-5678",
10    folder_path=Path("/path/to/folder/new/")
11)
wrapica.project_data.delete_project_data(project_id, data_id)

Delete a project data item using the projectData:delete endpoint

Parameters:

  • project_id (Union[UUID, str]) – The project id the data belongs to

  • data_id (Union[UUID, str]) – The data id we want to delete

Returns:

None

Return type:

None

Raises:

ValueError, ApiException

Examples:

from wrapica.project_data import delete_project_data

# Use wrapica.project.get_project_id_from_project_name
# If you need to convert a project_name to a project_id

delete_project_data(
    project_id="abcd-1234-efab-5678",
    data_id="fol.abcdef1234567890"
)
wrapica.project_data.find_project_data_bulk(project_id, parent_folder_id=None, parent_folder_path=None, data_type=None)

Given a project_id and a parent_folder_id, return a list of all data objects in the folder (recursively)

Parameters:

  • project_id (Union[UUID, str]) – The project id to search in

  • parent_folder_id (Union[UUID, str, None]) – The parent folder id (alternative to parent_folder_path)

  • parent_folder_path (Optional[Path]) – The path to the parent folder (alternative to parent_folder_id)

  • data_type (Optional[Literal['FOLDER', 'FILE']]) – The type of the data, one of “FILE”, “FOLDER”

Returns:

List of data objects

Return type:

List[ProjectData]

Raises:

ApiException, AssertionError

Examples:

 1from wrapica.project_data import find_project_data_bulk
 2from wrapica.enums import DataType
 3from wrapica.libica_models import ProjectData
 4
 5# Use wrapica.project.get_project_id_from_project_name
 6# If you need to convert a project_name to a project_id
 7
 8project_data_list: List[ProjectData] = find_project_data_bulk(
 9    project_id="abcd-1234-efab-5678",
10    parent_folder_id="fol.abcdef1234567890",
11    data_type="FILE"
12)
13
14for project_data in project_data_list:
15    print(project_data.data.details.name)
wrapica.project_data.find_project_data_recursively(project_id, parent_folder_id=None, parent_folder_path=None, name=None, data_type=None, min_depth=None, max_depth=None)

Given a project_id, a parent_folder_id, a data_name and a data_type, return a list of data objects This is a slow exercise and should only be used if the max_depth is low and the total number of items in the directory is very high

Parameters:

  • project_id (Union[UUID, str]) – The project id to search in

  • parent_folder_id (Union[UUID, str, None]) – The parent folder id (alternative to parent_folder_path)

  • parent_folder_path (Optional[Path]) – The path to the parent folder (alternative to parent_folder_id)

  • name (Optional[str]) – The name of the file or directory to look for, may also use a regex pattern

  • data_type (Optional[Literal['FOLDER', 'FILE']]) – The type of the data, one of “FILE”, “FOLDER”

  • min_depth (Optional[int]) – The minimum depth to search

  • max_depth (Optional[int]) – The maximum depth to search

Returns:

List of data objects

Return type:

List[ProjectData]

Raises:

ApiException, AssertionError, ValueError

Examples:

 1from wrapica.project_data import find_project_data_recursively
 2from wrapica.enums import DataType
 3from wrapica.libica_models import ProjectData
 4
 5# Use wrapica.project.get_project_id_from_project_name
 6# If you need to convert a project_name to a project_id
 7
 8project_data_list: List[ProjectData] = find_project_data_recursively(
 9    project_id="abcd-1234-efab-5678",
10    parent_folder_id="fol.abcdef1234567890",
11    name="file.txt",
12    data_type="FILE",
13    min_depth=1,
14    max_depth=3
15)
16
17for project_data in project_data_list:
18    print(project_data.data.details.name)
wrapica.project_data.get_aws_credentials_access_for_project_folder(project_id, folder_id=None, folder_path=None, read_only=None)

Given a project_id and a folder_id or folder_path, collect the AWS Access Credentials for downloading this data.

Parameters:

  • project_id (Union[UUID, str]) – The project id of the data

  • folder_id (Union[UUID, str, None]) – The folder id (alternative to folder_path)

  • folder_path (Optional[Path]) – The folder path (alternative to folder_id)

  • read_only (Optional[bool]) – True to create read only credentials, otherwise defaults to read+write credentials

Returns:

An object with the following attributes: * access_key * secret_key * session_token * region * bucket * object_prefix * server_side_encryption_algorithm * server_side_encryption_key

Return type:

AwsTempCredentials

Raises:

AssertionError, ApiException, ValueError

Examples:

 1import subprocess
 2from wrapica.project_data import get_aws_credentials_access_for_project_folder
 3from wrapica.libica_models import AwsTempCredentials
 4
 5# Use wrapica.project.get_project_id_from_project_name
 6# If you need to convert a project_name to a project_id
 7
 8aws_temp_credentials: AwsTempCredentials = get_aws_credentials_access_for_project_folder(
 9    project_id="proj.abcdef1234567890",
10    folder_path=Path("/path/to/folder/")
11)
12
13local_path = Path("/path/to/local/folder/")
14
15subprocess.run(
16  [
17    "aws", "s3", "sync",
18    # Can add sync parameters here like --dryrun or --exclude / --include
19    "s3://{}/{}".format(aws_temp_credentials.bucket, aws_temp_credentials.object_prefix),
20    str(local_path)
21  ],
22  env={
23    "AWS_ACCESS_KEY_ID": aws_temp_credentials.access_key,
24    "AWS_SECRET_ACCESS_KEY": aws_temp_credentials.secret_key,
25    "AWS_SESSION_TOKEN": aws_temp_credentials.session_token,
26    "AWS_REGION": aws_temp_credentials.region
27  }
28)
wrapica.project_data.get_credentials_access_for_project_folder(project_id, folder_id=None, folder_path=None, read_only=None, credentials_format=None)

Retrieve temporary access credentials for a folder within a project.

This function requests temporary credentials for a specific project folder using the ProjectDataApi.create_temporary_credentials_for_data endpoint. A folder can be specified either by its folder_id or by its folder_path within the given project, but not both.

Parameters:

  • project_id (Union[UUID, str]) – The ID of the project containing the folder.

  • folder_id (Union[UUID, str, None]) – The ID of the folder to retrieve credentials for (optional if folder_path is provided).

  • folder_path (Optional[Path]) – The path to the folder within the project (optional if folder_id is provided).

  • read_only (Optional[bool]) – If True, request read-only credentials;

Returns:

An object containing temporary credentials, either in AWS format or RCLONE format depending on the credentials_format parameter.

Return type:

Union[AwsTempCredentials, RcloneTempCredentials]

Raises:

AssertionError if both or neither of folder_id and folder_path are provided, ValueError if credentials cannot be retrieved.

Examples:

wrapica.project_data.get_file_by_file_name_from_project_data_list(file_name, project_data_list)

Useful for collecting a file object from an analysis output object

Parameters:

  • file_name (str) – The name of the file to get

  • project_data_list (List[ProjectData]) – The list of project data objects to search through

Returns:

The file object

Return type:

ProjectData

Raises:

ValueError

Examples:

# Imports
from wrapica.project_data import get_file_by_file_name_from_project_data_list

# Use wrapica.project.get_project_id_from_project_name
# If you need to convert a project_name to a project_id

project_data_list: List[ProjectData] = find_project_data_bulk(
    project_id="abcd-1234-efab-5678",
    parent_folder_id="fol.abcdef1234567890",
    data_type="FILE"
)

file_obj: ProjectData = get_file_by_file_name_from_project_data_list(
    file_name="file.txt",
    project_data_list=project_data_list
)
wrapica.project_data.get_project_data_file_id_from_project_id_and_path(project_id, file_path, create_file_if_not_found=False)

Given a project id, parent folder path and file_name, return the file id If the file is not found, and create_file_if_not_found is True, create the file

Parameters:

  • project_id (Union[UUID, str]) – The project id

  • file_path (Path) – The path to the file

  • create_file_if_not_found (bool) – Create the file object if it does not exist

Return type:

str

Returns:

The file id

Raises:

FileNotFoundError, ApiException

Examples:

 1from pathlib import Path
 2from wrapica.project_data import (
 3    create_download_urls, get_project_folder_id_from_project_id_and_path
 4)
 5from wrapica.libica_models import DataUrlWithPath
 6
 7# Use wrapica.project.get_project_id_from_project_name
 8# If you need to convert a project_name to a project_id
 9
10file_id: str = get_project_file_id_from_project_id_and_path(
11    project_id="abcd-1234-efab-5678",
12    file_path=Path("/path/to/file.txt")
13)
14
15download_urls: List[DataUrlWithPath] = create_download_urls(
16    project_id="proj.abcdef1234567890",
17    folder_id=project_folder_obj.data.id,
18    recursive=True
19)
20
21for download_url in download_urls:
22    print(download_url.url)
wrapica.project_data.get_project_data_folder_id_from_project_id_and_path(project_id, folder_path, create_folder_if_not_found=False)

Given a project_id and a path, return the folder_id.

Note that given the folder_path is a Path object (which do not end in /), we need to append a ‘/’ to the end of the path before calling the API In the case that the folder_path is the root folder, we ensure that only a single ‘/’ is provided.

Parameters:

  • project_id (Union[UUID, str]) – The project id to search in

  • folder_path (Path) – The path to the folder

  • create_folder_if_not_found (bool) – If folder does not exist in project, do we want to create it?

Return type:

str

Returns:

The folder id

Raises:

NotADirectoryError, ApiException

Examples:

 1from pathlib import Path
 2from wrapica.project_data import get_project_folder_id_from_project_id_and_path
 3
 4# Use wrapica.project.get_project_id_from_project_name
 5# If you need to convert a project_name to a project_id
 6
 7folder_id: str = get_project_folder_id_from_project_id_and_path(
 8    project_id="abcd-1234-efab-5678",
 9    folder_path=Path("/path/to/folder/")
10)
wrapica.project_data.get_project_data_id_from_project_id_and_path(project_id, data_path, data_type, create_data_if_not_found=False)

Given a project_id and a path, return the data_id, where DATA_TYPE is one of FILE or FOLDER Should call the underlying get_data_id_from_project_id_and_path function split by data type

Parameters:

  • project_id (Union[UUID, str]) – The project context the data exists in

  • data_path (Path) – The path to the data in the project

  • data_type (Literal['FOLDER', 'FILE']) – The data_type, one of “FILE”, “FOLDER”

  • create_data_if_not_found (bool)

Raises:

FileNotFoundError, NotADirectoryError, ApiException

Return type:

str

Returns:

The data id

Note:

Use get_file_id_from_project_id_and_path or get_folder_id_from_project_id_and_path instead if data_type is known

Examples:

 1from pathlib import Path
 2from wrapica.project_data import get_project_data_id_from_project_id_and_path
 3from wrapica.enums import DataType
 4
 5# Use wrapica.project.get_project_id_from_project_name
 6# If you need to convert a project_name to a project_id
 7
 8# Get a folder
 9data_id: str = get_project_data_id_from_project_id_and_path(
10    project_id="abcd-1234-efab-5678",
11    data_path=Path("/path/to/folder/"),
12    data_type="FOLDER"
13)
14
15# Get a file
16data_id: str = get_project_data_id_from_project_id_and_path(
17    project_id="abcd-1234-efab-5678",
18    data_path=Path("/path/to/file.txt"),
19    data_type="FILE"
20)
wrapica.project_data.get_project_data_obj_by_id(project_id, data_id)

Given a project_id and a data_id, return the data object

Parameters:

  • project_id (Union[UUID, str]) – The project id to search in

  • data_id (Union[UUID, str]) – The data id

Returns:

The project data object

Return type:

ProjectData

Raises:

ApiException

Examples:

 1from wrapica.project_data import get_project_data_obj_by_id
 2from wrapica.libica_models import ProjectData
 3
 4# Use wrapica.project.get_project_id_from_project_name
 5# If you need to convert a project_name to a project_id
 6
 7project_data_obj: ProjectData = get_project_data_obj_by_id(
 8    project_id="abcd-1234-efab-5678",
 9    data_id="fil.abcdef1234567890"
10)
wrapica.project_data.get_project_data_obj_from_project_id_and_path(project_id, data_path, data_type, create_data_if_not_found=False)

Given a project_id and a path, return the data object, where DATA_TYPE is one of FILE or FOLDER Will call the get_project_data_id and then call get_project_data_obj_by_id

Parameters:

  • project_id (Union[UUID, str]) – The project id to search in

  • data_path (Path) – The path to the data in the project

  • data_type (Literal['FOLDER', 'FILE']) – The data_type, one of “FILE”, “FOLDER”

  • create_data_if_not_found (bool) – If the data is not found, and create_data_if_not_found is True, create the data

Returns:

The project data object

Return type:

ProjectData

Raises:

FileNotFoundError, NotADirectoryError, ApiException

Examples:

 1from pathlib import Path
 2from wrapica.project_data import get_project_data_obj_from_project_id_and_path
 3from wrapica.enums import DataType
 4from wrapica.libica_models import ProjectData
 5
 6# Use wrapica.project.get_project_id_from_project_name
 7# If you need to convert a project_name to a project_id
 8
 9# Get a folder project data object
10project_folder_data_obj: ProjectData = get_project_data_obj_from_project_id_and_path(
11    project_id="abcd-1234-efab-5678",
12    data_path=Path("/path/to/folder/"),
13    data_type="FOLDER"
14)
15
16# Get a file project data object
17project_file_data_obj: ProjectData = get_project_data_obj_from_project_id_and_path(
18    project_id="abcd-1234-efab-5678",
19    data_path=Path("/path/to/file.txt"),
20    data_type="FILE"
21)
22
23print(project_folder_data_obj.data.id)
24# fol.abcdef1234567890
25
26print(project_file_data_obj.data.id)
27# fil.abcdef1234567890
wrapica.project_data.get_project_data_path_by_id(project_id, data_id)

Given a project id and data id, return the path of the data

Parameters:

  • project_id (Union[UUID, str]) – The project id to search in

  • data_id (Union[UUID, str]) – The data id

Returns:

The path of the data

Return type:

Path

Raises:

ApiException

Examples:

 1from pathlib import Path
 2from wrapica.project_data import get_project_data_path_by_id
 3
 4# Use wrapica.project.get_project_id_from_project_name
 5# If you need to convert a project_name to a project_id
 6
 7project_data_path: Path = get_project_data_path_by_id(
 8    project_id="abcd-1234-efab-5678",
 9    data_id="fil.abcdef1234567890"
10)
11
12print(project_data_path)
13# /path/to/file.txt
wrapica.project_data.get_project_data_upload_url(project_id, data_id)

Get upload url for project data object

This can only be used for a file that has been created but not yet written to.

Parameters:

  • project_id (Union[UUID, str]) – The owning project id of the data

  • data_id (Union[UUID, str]) – The data id

Returns:

The upload url

Return type:

str

Raises:

ApiException

Examples:

 1# Imports
 2from wrapica.project_data import (
 3    get_project_data_upload_url,
 4    create_file_in_project
 5)
 6
 7# Create a file in a project
 8new_file_obj = create_file_in_project(
 9    project_id="abcd-1234-efab-5678",
10    file_path=Path("/path/to/file.txt")
11)
12
13# Get the upload url for the new file
14upload_url = get_project_data_upload_url(
15    project_id="abcd-1234-efab-5678",
16    data_id=new_file_obj.data.id
17)
wrapica.project_data.get_rclone_credentials_access_for_project_folder(project_id, folder_id=None, folder_path=None, read_only=None)

Given a project_id and a folder_id or folder_path, collect the Rclone Temp Credentials for downloading this data.

Parameters:

  • project_id (Union[UUID, str]) – The project id of the data

  • folder_id (Union[UUID, str, None]) – The folder id (alternative to folder_path)

  • folder_path (Optional[Path]) – The folder path (alternative to folder_id)

  • read_only (Optional[bool]) – True to create read only credentials, otherwise defaults to read+write credentials

Returns:

An object with the following attributes: * type * provider * … other attributes depending on the provider, for example for AWS provider, the following attributes are included:

  • access_key_id

  • secret_access_key

  • session_token

  • region

  • bucket

  • object_prefix

  • server_side_encryption

  • expiration_time

Return type:

RcloneTempCredentials

Raises:

AssertionError, ApiException, ValueError

Examples:

wrapica.project_data.is_data_id_format(data_id)

Check if data id is a data id

Parameters:

data_id (Union[UUID, str]) – The data id to check

Returns:

True if the data id is a data id

Return type:

bool

Examples:

1from wrapica.project_data import is_data_id
2
3print(is_data_id("fil.abcdef1234567890"))
4# True
wrapica.project_data.is_file_id_format(file_id_str)

Does this string look like a folder id?

Parameters:

file_id_str (str) – The string to check

Returns:

True if the string looks like a file id

Return type:

bool

Examples:

1from wrapica.project_data import is_file_id_format
2
3print(is_file_id_format("fil.abcdef1234567890"))
4# True
wrapica.project_data.is_folder_id_format(folder_id_str)

Does this string look like a folder id?

Parameters:

folder_id_str (str) – The string to check

Returns:

True if the string looks like a folder id

Return type:

bool

Examples:

1from wrapica.project_data import is_folder_id_format
2
3print(is_folder_id_format("fol.abcdef1234567890"))
4# True
wrapica.project_data.list_project_data_non_recursively(project_id, parent_folder_id=None, parent_folder_path=None, file_name=None, status=None, data_type=None, creation_date_after=None, creation_date_before=None, status_date_after=None, status_date_before=None, sort='')

Given a project id and parent folder id or path, return a list of data objects that are directly under that folder

Parameters:

  • project_id (Union[UUID, str]) – The project id to search in

  • parent_folder_path (Optional[Path]) – The path to the parent folder (can use parent_folder_id instead)

  • parent_folder_id (Union[UUID, str, None]) – The parent folder id (can use parent_folder_path instead)

  • file_name (Union[str, List[str], None]) – The name of the file or directory to look for, can also be a list of names, may also use * as a wildcard

  • status (Union[Literal['PARTIAL', 'AVAILABLE', 'ARCHIVING', 'ARCHIVED', 'UNARCHIVING', 'DELETING'], List[Literal['PARTIAL', 'AVAILABLE', 'ARCHIVING', 'ARCHIVED', 'UNARCHIVING', 'DELETING']], None]) – The status of the data, one of ProjectDataStatusValues

  • data_type (Optional[Literal['FOLDER', 'FILE']]) – The type of the data, one of “FILE”, “FOLDER”

  • creation_date_after (Optional[datetime]) – Return only data created after this date

  • creation_date_before (Optional[datetime]) – Return only data created before this date

  • status_date_after (Optional[datetime]) – Return only data with status date after this date

  • status_date_before (Optional[datetime]) – Return only data with status date before this date

  • sort (Union[Literal['timeCreated', '-timeCreated', 'timeModified', '-timeModified', 'name', '-name', 'path', '-path', 'fileSizeInBytes', '-fileSizeInBytes', 'status', '-status', 'format', '-format', 'dataType', '-dataType', 'willBeArchivedAt', '-willBeArchivedAt', 'willBeDeletedAt', '-willBeDeletedAt'], List[Literal['timeCreated', '-timeCreated', 'timeModified', '-timeModified', 'name', '-name', 'path', '-path', 'fileSizeInBytes', '-fileSizeInBytes', 'status', '-status', 'format', '-format', 'dataType', '-dataType', 'willBeArchivedAt', '-willBeArchivedAt', 'willBeDeletedAt', '-willBeDeletedAt']], None]) – The sort order, one or more of ProjectDataSortParameters (Use ‘-’ prefix to sort in descending order) * timeCreated - Sort by time created * timeModified - Sort by time modified * name - Sort by name * path - Sort by path * fileSizeInBytes - Sort by file size in bytes * status - Sort by status * format - Sort by format * dataType - Sort by data type * willBeArchivedAt - Sort by when the data will be archived * willBeDeletedAt - Sort by when the data will be deleted

Returns:

List of data objects

Return type:

List[ProjectData]

Raises:

AssertionError, ApiException, ValueError

Examples:

 1from pathlib import Path
 2from wrapica.project_data import list_project_data_non_recursively
 3from wrapica.libica_models import ProjectData, ProjectDataSortParameters
 4from wrapica.enums import ProjectDataStatusValues, DataType
 5
 6# Use wrapica.project.get_project_id_from_project_name
 7# If you need to convert a project_name to a project_id
 8
 9project_data_list: List[ProjectData] = list_project_data_non_recursively(
10    project_id="abcd-1234-efab-5678",
11    parent_folder_path=Path("/path/to/folder/"),
12    file_name="file.txt",
13    status=ProjectDataStatusValues.COMPLETED,
14    data_type="FILE",
15    creation_date_after=datetime(2021, 1, 1),
16    creation_date_before=datetime(2021, 12, 31),
17    status_date_after=datetime(2021, 1, 1),
18    status_date_before=datetime(2021, 12, 31),
19    sort=ProjectDataSortParameters.TIME_CREATED
20)
21
22for project_data in project_data_list:
23    print(project_data.data.details.name)
wrapica.project_data.move_project_data(dest_project_id, dest_folder_id, src_data_list)

Move a list of data ids to a destination project :type dest_project_id: Union[UUID, str] :param dest_project_id: :type dest_folder_id: Union[UUID, str] :param dest_folder_id: :type src_data_list: List[Union[UUID, str]] :param src_data_list:

Returns:

Return type:

Job

Raises:

ApiException

Examples:

from wrapica.project_data import move_data

job = move_data(
    dest_project_id="abcd-1234-efab-5678",
    dest_folder_id="fol.abcdef1234567890",
    src_data_list=[
        "fil.abcdef1234567890",
        "fil.abcdef1234567891"
    ]
)
wrapica.project_data.presign_cwl_directory(project_id, data_id)

Given a CWL directory object, presign all files in the directory recursively, and return the list of presigned url

Parameters:

  • project_id (Union[UUID, str]) – The project id to search in

  • data_id (Union[UUID, str]) – The data id

Returns:

The CWL input json Directory object where each file in the listing has a presigned url for a location attributes

Return type:

List[Dict[str, Union[Union[dict, str], Any]]]

Raises:

ApiException

Examples:

 1from wrapica.project_data import presign_cwl_directory
 2
 3# Use wrapica.project.get_project_id_from_project_name
 4# If you need to convert a project_name to a project_id
 5# Use wrapica.project_data.get_folder_id_from_project_id_and_path
 6# If you need to convert a folder path to a folder id
 7
 8cwl_directory: List[Dict[str, Union[Union[dict, str], Any]]] = presign_cwl_directory(
 9    project_id="abcd-1234-efab-5678",
10    data_id="fol.abcdef1234567890"
11)
12
13print(cwl_directory)
14# [
15#   {
16#     "class": "Directory",
17#     "basename": "folder",
18#     "listing": [
19#       {
20#         "class": "File",
21#         "basename": "file.txt",
22#         "location": "https://s3.amazonaws.com/umccr-illumina-prod/abcd-1234-efab-5678/abcdef1234567890"
23#       }
24#     ]
25#   }
26# ]
wrapica.project_data.presign_cwl_directory_with_external_data_mounts(project_id, data_id)

Given a cwl directory with a listing attribute, presign all files in the directory recursively, and return the list of presigned url mount objects and the cwl directory listing object

Parameters:

  • project_id (Union[UUID, str]) – The project id to search in

  • data_id (Union[UUID, str]) – The data id

Returns:

external_data_mounts, cwl_item_objs

Return type:

Tuple[List[AnalysisInputExternalData], List[Dict]]

Raises:

ApiException

Examples:

 1from wrapica.project_data import presign_cwl_directory_with_external_data_mounts
 2
 3# Use wrapica.project.get_project_id_from_project_name
 4# If you need to convert a project_name to a project_id
 5# Use wrapica.project_data.get_folder_id_from_project_id_and_path
 6# If you need to convert a folder path to a folder id
 7
 8external_data_mounts, cwl_item_objs = presign_cwl_directory_with_external_data_mounts(
 9    project_id="abcd-1234-efab-5678",
10    data_id="fol.abcdef1234567890"
11)
12
13print(external_data_mounts)
14# [
15#   {
16#     "url": "https://s3.amazonaws.com/umccr-illumina-prod/abcd-1234-efab-5678/abcdef1234567890",
17#     "type": "FILE",
18#     "mount_path": "abcd-1234-efab-5678/abcdef1234567890/file.txt"
19#   }
20# ]
21
22print(cwl_item_objs)
23# [
24#   {
25#     "class": "Directory",
26#     "basename": "folder",
27#     "listing": [
28#       {
29#         "class": "File",
30#         "basename": "file.txt",
31#         "location": "abcd-1234-efab-5678/abcdef1234567890/file.txt"
32#       }
33#     ]
34#   }
35# ]
wrapica.project_data.presign_folder(project_id, folder_path=None, folder_id=None)

Presign a folder recursively

Given a project_id and a folder_path or folder_id, return a list of presigned urls for the folder

Parameters:

  • project_id (Union[UUID, str]) – The owning project id of the folder

  • folder_path (Optional[Path]) – The folder path

  • folder_id (Union[UUID, str, None]) – The folder id

Returns:

List of presigned urls

Return type:

List[DataUrlWithPath]

Examples:

 1# Imports
 2from wrapica.project_data import presign_folder
 3
 4# Presign a folder
 5presigned_urls_list = list(
 6    map(
 7        lambda data_uri_iter: data_uri_iter.url,
 8        presign_folder(
 9            project_id="abcdef1234567890",
10            folder_path="/path/to/folder/"
11        )
12    )
13)
wrapica.project_data.project_data_copy_batch_handler(source_data_ids, destination_project_id, destination_folder_path)

Copy a batch of files from one project to another

Parameters:

  • source_data_ids (List[Union[UUID, str]]) – The list of source data ids

  • destination_project_id (Union[UUID, str]) – The destination project id

  • destination_folder_path (Path) – The destination folder path

Returns:

The job id for the project data copy batch

Return type:

Job

Raises:

ApiException

Examples:

 1from wrapica.project_data import project_data_copy_batch_handler
 2
 3# Use wrapica.project.get_project_id_from_project_name
 4# If you need to convert a project_name to a project_id
 5
 6job_id: str = project_data_copy_batch_handler(
 7    source_data_ids=[
 8        "fil.abcdef1234567890",
 9        "fil.abcdef1234567891"
10    ],
11    destination_project_id="abcd-1234-efab-5678",
12    destination_folder_path=Path("/path/to/folder/")
13)
wrapica.project_data.read_icav2_file_contents(project_id, data_id, output_path=None)

Write icav2 file contents to a path

Parameters:

  • project_id (Union[UUID, str]) – The project id

  • data_id (Union[UUID, str]) – The data id

  • output_path (Union[Path, TextIOWrapper, None]) – The output path to write the file contents to

Returns:

The file contents as a string if output_path is None

Return type:

Optional[str]

Raises:

NotADirectoryError, ApiException

Examples:

:linenos:

# Imports
from wrapica.project_data import read_icav2_file_contents

# Read icav2 file contents to a file
with open("foo.txt", "w") as f:
    read_icav2_file_contents(
        project_id="abcd-1234-efab-5678",
        data_id="fil.abcdef1234567890",
        output_path=f
    )
wrapica.project_data.read_icav2_file_contents_to_string(project_id, data_id)

Stream down the icav2 file contents and return as a string

Parameters:

  • project_id (Union[UUID, str]) – The project id

  • data_id (Union[UUID, str]) – The data id

Returns:

The file contents as a string

Return type:

str

Raises:

ApiException

Examples:

:linenos:
from wrapica.project_data import read_icav2_file_contents_to_string

# Use wrapica.project.get_project_id_from_project_name
# If you need to convert a project_name to a project_id

file_contents: str = read_icav2_file_contents_to_string(
    project_id="abcd-1234-efab-5678",
    data_id="fil.abcdef1234567890"
)

print(file_contents)
# this is the file contents
wrapica.project_data.unpack_icav2_uri(uri)
Return type:

Tuple[str, str]

wrapica.project_data.unpack_uri(uri)

Unpack an icav2 or s3 uri

Parameters:

uri (str) – The icav2 uri in the format of ‘icav2://project_id/path/to/file.txt’ or ‘icav2://project_id/path/to/dir/’

Returns:

Tuple with project_id and data_path

Return type:

Tuple[str, str]

Examples:

1from wrapica.project_data import unpack_icav2_uri
2
3project_id, data_path = unpack_icav2_uri("icav2://project_id/path/to/dir")
wrapica.project_data.write_icav2_file_contents(project_id, data_path, file_stream_or_path)

Write data to an icav2 file

Parameters:

  • project_id (Union[UUID, str]) – The owning project id of the file

  • data_path (Path) – The file path

  • file_stream_or_path (Union[Path, TextIOWrapper]) – The file stream or path to write to

Returns:

The new file id

Return type:

str

Raises:

ValueError, ApiException

Examples:

1# Imports
2from wrapica.project_data import write_icav2_file_contents
3
4write_icav2_file_contents(
5    project_id="abcd-1234-efab-5678",
6    data_path=Path("/path/to/file.txt"),
7    file_stream_or_path=Path("/path/to/local/file.txt")
8)