Python Client and Girder CLI

In addition to the web clients, Girder comes with a python client library and a CLI to allow for programmatic interaction with a Girder server, and also to workaround limitations of the web client. For example, the python CLI makes it much easier to upload a large, nested hierarchy of data from a local directory to Girder, and also makes it much easier to download a large, nested hierarchy of data from Girder to a local directory.

Installation

If you have the source directory of Girder, you can find the girder_client package within the clients/python directory. If you do not have the source directory of Girder, you can install the client via pip:

pip install girder-client

The Command Line Interface

The girder_client package ships with a command-line utility that wraps some of its common functionality to make it easy to invoke operations without having to write any custom python scripts. If you have installed girder_client via pip, you can use the special girder-cli executable:

girder-cli <arguments>

Otherwise you can equivalently just invoke the module directly:

python -m girder_client <arguments>

Specifying the Girder Instance

The default host for the girder_client is localhost. To specify the host with cli usage

python cli.py --host girder.example.com

To specify a host using SSL (https)

python cli.py --host girder.example.com --scheme https --port 443

Upload a local file hierarchy

The girder_client can upload to an S3 Assetstore when uploading to a Girder server that is version 1.3.0 or later.

The upload command, -c upload, is the default, so the following two forms are equivalent

python cli.py
python cli.py -c upload

To upload a folder hierarchy rooted at test_folder to the Girder Folder with id 54b6d41a8926486c0cbca367

python cli.py 54b6d41a8926486c0cbca367 test_folder

When using the upload command, the default --parent-type, meaning the type of resource the local folder will be created under in Girder, is Folder, so the following are equivalent

python cli.py 54b6d41a8926486c0cbca367 test_folder
python cli.py 54b6d41a8926486c0cbca367 test_folder --parent-type folder

To upload that same local folder to a Collection or User, specify the parent type as follows

python cli.py 54b6d41a8926486c0cbca459 test_folder --parent-type user

To see what local folders and files on disk would be uploaded without actually uploading anything, add the --dryrun flag

python cli.py 54b6d41a8926486c0cbca367 test_folder --dryrun

To have leaf folders (those folders with no subfolders, only containing files) be uploaded to Girder as single Items with multiple Files, i.e. those leaf folders will be created as Items and all files within the leaf folders will be Files within those Items, add the --leaf-folders-as-items flag

python cli.py 54b6d41a8926486c0cbca367 test_folder --leaf-folders-as-items

If you already have an existing Folder hierarchy in Girder which you have a superset of on your local disk (e.g. you previously uploaded a hierarchy to Girder and then added more folders and files to the hierarchy on disk), you can reuse the existing hierarchy in Girder, which will not create new Folders and Items for those that match folders and files on disk, by using the --reuse flag.

python cli.py 54b6d41a8926486c0cbca367 test_folder --reuse

To include a blacklist of file patterns that will not be uploaded, pass a comma separated list to the --blacklist arg

python cli.py 54b6d41a8926486c0cbca367 test_folder --blacklist .DS_Store

Download a Folder hierarchy into a local folder

To download a Girder Folder hierarchy rooted at Folder id 54b6d40b8926486c0cbca364 under the local folder download_folder

python cli.py -c download 54b6d40b8926486c0cbca364 download_folder

Downloading is only supported from a parent type of Folder.

The Python Client Library

For those wishing to write their own python scripts that interact with Girder, we recommend using the Girder python client library, documented below.

Recursively inherit access control to a Folder’s descendants

This will take the access control and public value in the Girder Folder with id 54b43e9b8926486c0c06cb4f and copy those to all of the descendant Folders

import girder_client
gc = girder_client.GirderClient()
gc.authenticate('username', 'password')
gc.inheritAccessControlRecursive('54b43e9b8926486c0c06cb4f')

Set callbacks for Folder and Item uploads

If you have a function you would like called upon the completion of an Item or Folder upload, you would do the following.

N.B. The Item callbacks are called after the Item is created and all Files are uploaded to the Item. The Folder callbacks are called after the Folder is created and all child Folders and Items are uploaded to the Folder.

import girder_client
gc = girder_client.GirderClient()

def folder_callback(folder, filepath):
    # assume we have a folder_metadata dict that has
    # filepath: metadata_dict_for_folder
    gc.addMetadataToFolder(folder['_id'], folder_metadata[filepath])

def item_callback(item, filepath):
    # assume we have an item_metadata dict that has
    # filepath: metadata_dict_for_item
    gc.addMetadataToItem(item['_id'], item_metadata[filepath])

gc.authenticate('username', 'password')
gc.add_folder_upload_callback(folder_callback)
gc.add_item_upload_callback(item_callback)
gc.upload(local_folder, parent_id)

Further Examples and Function Level Documentation

class girder_client.GirderClient(host=None, port=None, apiRoot=None, scheme=None, dryrun=False, blacklist=None)[source]

A class for interacting with the Girder RESTful API. Some simple examples of how to use this class follow:

client = GirderClient('myhost', 8080)
client.authenticate('myname', 'mypass')

folder_id = '53b714308926486402ac5aba'
item = client.createItem(folder_id, 'an item name', 'a description')
client.addMetadataToItem(item['_id'], {'metadatakey': 'metadatavalue'})
client.uploadFileToItem(item['_id'], 'path/to/your/file.txt')

r1 = client.getItem(item['_id'])
r2 = client.sendRestRequest('GET', 'item',
    {'folderId': folder_id, 'sortdir': '-1' })
r3 = client.sendRestRequest('GET', 'resource/search',
    {'q': 'aggregated','types': '["folder", "item"]'})
addMetadataToFolder(folderId, metadata)[source]

Takes an folder ID and a dictionary containing the metadata

Parameters:
  • folderId – ID of the folder to set metadata on.
  • metadata – dictionary of metadata to set on folder.
addMetadataToItem(itemId, metadata)[source]

Takes an item ID and a dictionary containing the metadata

Parameters:
  • itemId – ID of the item to set metadata on.
  • metadata – dictionary of metadata to set on item.
add_folder_upload_callback(callback)[source]

Saves a passed in callback function that will be called after each folder has completed. Multiple callback functions can be added, they will be called in the order they were added by calling this function. Callback functions will be called after a folder in Girder is created and all subfolders and items for that folder have completed uploading. Callback functions should take two parameters: - the folder in Girder - the full path to the local folder

Parameters:callback – callback function to be called.
add_item_upload_callback(callback)[source]

Saves a passed in callback function that will be called after each item has completed. Multiple callback functions can be added, they will be called in the order they were added by calling this function. Callback functions will be called after an item in Girder is created and all files for that item have been uploaded. Callback functions should take two parameters: - the item in Girder - the full path to the local folder or file comprising the item

Parameters:callback – callback function to be called.
authenticate(username=None, password=None, interactive=False)[source]

Authenticate to Girder, storing the token that comes back to be used in future requests.

Parameters:
  • username – A string containing the username to use in basic authentication.
  • password – A string containing the password to use in basic authentication.
  • interactive – If you want the user to type their username or password in the shell rather than passing it in as an argument, set this to True. If you pass a username in interactive mode, the user will only be prompted for a password.
createFolder(parentId, name, description='', parentType='folder')[source]

Creates and returns a folder

Parameters:parentType – One of (‘folder’, ‘user’, ‘collection’)
createItem(parentFolderId, name, description)[source]

Creates and returns an item.

createResource(path, params)[source]

Creates and returns a resource.

downloadFile(fileId, path)[source]

Download a file to the given local path.

Parameters:
  • fileId – The ID of the Girder file to download.
  • path – The local path to write the file to.
downloadFolderRecursive(folderId, dest)[source]

Download a folder recursively from Girder into a local directory.

Parameters:
  • folderId – Id of the Girder folder to download.
  • dest – The local download destination.
downloadItem(itemId, dest, name=None)[source]

Download an item from Girder into a local folder. Each file in the item will be placed into the directory specified by the dest parameter. If the item contains multiple files or a single file with a different name than the item, the item will be created as a directory under dest and the files will become files within that directory.

Parameters:
  • itemId – The Id of the Girder item to download.
  • dest – The destination directory to write the item into.
  • name – If the item name is known in advance, you may pass it here which will save a lookup to the server.
getFolder(folderId)[source]

Retrieves a folder by its ID.

Parameters:folderId – A string containing the ID of the folder to retrieve from Girder.
getFolderAccess(folderId)[source]

Retrieves a folder’s access by its ID.

Parameters:folderId – A string containing the ID of the folder to retrieve access for from Girder.
getItem(itemId)[source]

Retrieves a item by its ID.

Parameters:itemId – A string containing the ID of the item to retrieve from Girder.
getResource(path, id=None, property=None)[source]

Loads a resource or resource property of property is not None by id or None if no resource is returned.

inheritAccessControlRecursive(ancestorFolderId, access=None, public=None)[source]

Take the access control and public value of a folder and recursively copy that access control and public value to all folder descendants, replacing any existing access control on the descendant folders with that of the ancestor folder.

Parameters:
  • ancestorFolderId – Id of the Girder folder to copy access control from, to all of its descendant folders.
  • access – Dictionary Access control target, if None, will take existing access control of ancestor folder
  • public – Boolean public value target, if None, will take existing public value of ancestor folder
isFileCurrent(itemId, filename, filepath)[source]

Tests whether the passed in filepath exists in the item with itemId, with a name of filename, and with the same contents as the file at filepath. Returns a tuple (file_id, current) where file_id = id of the file with that filename under the item, or None if no such file exists under the item. current = boolean if the file with that filename under the item has the same contents as the file at filepath.

Parameters:
  • itemId – ID of parent item for file.
  • filename – name of file to look for under the parent item.
  • filepath – path to file on disk.
listFolder(parentId, parentFolderType='folder', name=None)[source]

Retrieves a folder set from this parent ID.

Parameters:
  • parentId – The parent’s ID.
  • parentFolderType – One of (‘folder’, ‘user’, ‘collection’).
listItem(folderId, text=None, name=None)[source]

Retrieves a item set from this folder ID.

Parameters:
  • folderId – the parent folder’s ID.
  • text – query for full text search of items.
  • name – query for exact name match of items.
listResource(path, params)[source]

search for a list of resources based on params.

load_or_create_folder(folder_name, parent_id, parent_type)[source]

Returns a folder in Girder with the given name under the given parent. If none exists yet, it will create it and return it.

Parameters:
  • folder_name – the name of the folder to look up.
  • parent_id – id of parent in Girder
  • parent_type – one of (collection, folder, user)
Returns:

The folder that was found or created.

load_or_create_item(name, parent_folder_id, reuse_existing=True)[source]

Create an item with the given name in the given parent folder.

Parameters:
  • name – The name of the item to load or create.
  • parent_folder_id – id of parent folder in Girder
  • reuse_existing – boolean indicating whether to load an existing item of the same name in the same location, or create a new one.
sendRestRequest(method, path, parameters=None, data=None, files=None)[source]

This method looks up the appropriate method, constructs a request URL from the base URL, path, and parameters, and then sends the request. If the method is unknown or if the path is not found, an exception is raised, otherwise a JSON object is returned with the Girder response.

This is a convenience method to use when making basic requests that do not involve multipart file data that might need to be specially encoded or handled differently.

Parameters:
  • method – One of ‘GET’, ‘POST’, ‘PUT’, or ‘DELETE’
  • path – A string containing the path elements for this request. Note that the path string should not begin or end with the path separator, ‘/’.
  • parameters – A dictionary mapping strings to strings, to be used as the key/value pairs in the request parameters
setFolderAccess(folderId, access, public)[source]

Sets the passed in access control document along with the public value to the target folder.

Parameters:
  • folderId – Id of the target folder.
  • access – JSON document specifying access control.
  • public – Boolean specificying the public value.
upload(file_pattern, parent_id, parent_type='folder', leaf_folders_as_items=False, reuse_existing=False)[source]

Upload a pattern of files.

This will recursively walk down every tree in the file pattern to create a hierarchy on the server under the parent_id.

Parameters:
  • file_pattern – a glob pattern for files that will be uploaded, recursively copying any file folder structures.
  • parent_id – id of the parent in Girder.
  • parent_type – one of (collection,folder,user), default of folder.
  • leaf_folders_as_items – bool whether leaf folders should have all files uploaded as single items.
  • reuse_existing – bool whether to accept an existing item of the same name in the same location, or create a new one instead.
uploadFile(parentId, stream, name, size, parentType='item', progressCallback=None)[source]

Uploads a file into an item or folder.

Parameters:
  • parentId (str) – The ID of the folder or item to upload into.
  • stream (file-like) – Readable stream object.
  • name (str) – The name of the file to create.
  • size (str) – The length of the file. This must be exactly equal to the total number of bytes that will be read from stream, otherwise the upload will fail.
  • parentType (str) – ‘item’ or ‘folder’.
  • progressCallback (callable) – If passed, will be called after each chunk with progress information. It passes a single positional argument to the callable which is a dict of information about progress.
Returns:

The file that was created on the server.

uploadFileToItem(itemId, filepath)[source]

Uploads a file to an item, in chunks. If ((the file already exists in the item with the same name and sha512) or (if the file has 0 bytes), no uploading will be performed.

Parameters:
  • itemId – ID of parent item for file.
  • filepath – path to file on disk.
exception girder_client.HttpError(status, text, url, method)[source]

Raised if the server returns an error status code from a request.