Girder: a data management platform

What is Girder?

Girder is a free and open source web-based data management platform developed by Kitware as part of the Resonant data and analytics ecosystem. What does that mean? Girder is both a standalone application and a platform for building new web services. It’s meant to enable quick and easy construction of web applications that have some or all of the following requirements:

• Data organization and dissemination

  Many web applications need to manage data that are dynamically provided by users of the system, or exposed through external data services. Girder makes construction and organization of dynamic data hierarchies simple. One of the most powerful aspects of Girder is that it can transparently store, serve, and proxy data from heterogeneous backend storage engines through a single RESTful web API, including local filesystems, MongoDB databases, Amazon S3-compliant key-value stores, and Hadoop Distributed Filesystems (HDFS).

• User management & authentication

  Girder also includes everything needed for pluggable user management and authentication out of the box and adheres to best practices in web security. The system can be configured to securely store credentials itself, or defer to third-party authentication services such as OAuth or LDAP.

• Authorization management

  Girder supports a simple access control scheme that allows both user-based and role-based access control on resources managed in the system. The project has undergone rigorous security audits and has extensive automated testing to exercise authorization behavior and ensure correctness.

For an overview of the concepts present in Girder, we recommend checking out the User Guide.

Girder is published under the Apache 2.0 License. Its source code can be found at https://github.com/girder/girder.

The architecture

Girder’s server-side architecture is focused around the construction of RESTful web APIs to afford minimal coupling between the backend services and the frontend clients. This decoupling allows multiple clients all to use the same server-side interface. While Girder does contain its own single-page javascript web application, the system can be used by any HTTP-capable client, either inside or outside of the web browser environment. Girder can even be run without its front-end application present at all, only serving the web API routes.

The web API is mostly used to interact with resources that are represented by models in the system. Models internally interact with a Mongo database to store and retrieve persistent records. The models contain methods for creating, changing, retrieving, and deleting those records. The core Girder model types are described in the Concepts section of the user guide.

The primary method of customizing and extending Girder is via the development of plugins, the process of which is described in the Plugin Development section of this documentation. Plugins can, for example, add new REST routes, modify or remove existing ones, serve up a different web application from the server root, hook into model lifecycle events or specific API calls, override authentication behavior to support new authentication services or protocols, add a new backend storage engine for file storage, or even interact with a completely different DBMS to persist system records – the extent to which plugins are allowed to modify and extend the core system behavior is nearly limitless.

Plugins are self-contained in their own directory within the Girder source tree. Therefore they can reside in their own separate source repository, and are installed by simply copying the plugin source tree under an existing Girder installation’s plugins directory. The Girder repository contains several generally useful plugins out of the box, which are described in the Plugins section.

Table of contents

Administrator Documentation

Deployment

This tutorial will deploy a production-ready Girder on Ubuntu 18.04, using the popular IT automation tool Ansible.

Prerequisites

Before running this, you must provide:

• A “server”: an (ideally fresh) Ubuntu 18.04 system, with:
  • A sudo-capable user.
  • Inbound access from the internet on TCP port 80 and 443.
  • Outbound access to the internet on UDP port 53. Many firewalls (e.g. the AWS EC2 default security group) do not allow this by default.
  • A DNS entry, so its public IP address is resolvable from the internet.
• A “controller”: a machine with Ansible installed and SSH access to the server.
• An “assetstore” for storing uploaded files on Girder. This may be either:
  • A location on the server’s filesystem (which may be mounted external storage).
  • An AWS S3 bucket.
• Credentials for an outbound SMTP server, ideally with STARTTLS or TLS.
• An email address for the administrator of the system.

Install with Ansible

Download Template Files

Download the following files to a fresh directory on the controller machine:

• requirements.yml
• hosts.yml
• playbook.yml
• provision.sh

Make provision.sh executable:

chmod +x ./provision.sh

Setup Inventory

Edit hosts.yml, in accordance with the inline comments. This will configure Ansible to find and login to the server.

Setup the Playbook

Edit playbook.yml, in accordance with the inline comments. This will configure necessary details of the provisioning process.

Note

Spoof nginx_registration_email at your own risk. The email address is only provided to Let’s Encrypt, to provide warnings in case HTTPS auto-renewal is failing. Under normal circumstances, no emails should ever be sent.

Note

When specifying Girder plugins, PyPI package names of published Girder plugin packages should be used whenever possible. See Plugins for a list of official Girder plugins and associated PyPI package names.

Unpublished plugin packages may be specified in accordance with pip’s VCS support.

Run the Playbook

Run the provision.sh script, which will download external role files and run the playbook:

./provision.sh

If the server user requires a password to use sudo, you may be prompted for a “become” password. Enter the password of the server user at this point.

When the script completes, Girder should be fully installed! There is no need for additional setup via SSH.

Configure the Server

Create a Site Admin User

Visit the URL of the new Girder server in a web browser, click the Register link, and create a new user.

The first user to be created on a Girder instance is automatically given site admin permissions. As a site admin, you should see an Admin console link in the left-side navigation bar. If you do not see this link while logged in, then another user has already have created an account first.

Create an Assetstore

While logged in as a site admin, visit Admin console -> Assetstores. Here, create at least one new assetstore. See the Assetstores section for further details on assetstore types.

No users can upload files until an assetstore is created, since all files in Girder must reside within an assetstore.

Configure Email Sending

While logged in as a site admin, visit Admin console -> Server Configuration, then scroll down to the Email Delivery section. Here, enter the credentials for an outgoing SMTP server, then click Save at the bottom of the page.

Outgoing email support is essential to allowing reset of forgotten passwords and, if enabled, email address validation for new users.

Plugins

To change settings for plugins, click the Admin console navigation link, then click Plugins. Here, you will see a list of installed plugins. If the plugin has settings, click on the associated gear icon to modify them.

Configuration

In many cases, Girder will work with default configuration whether installed via pip or from a source checkout or tarball. That said, the Girder config file can be set at the following locations (ordered by precedent):

1. The path specified by the environment variable GIRDER_CONFIG.
2. ~/.girder/girder.cfg
3. /etc/girder.cfg

Logging

Much of Girder’s output is placed into the error or info log file. By default, these logs are stored in ~/.girder/logs. To set the Girder log root or error and info logs specifically, set the log_root, error_log_file, and/or info_log_file variables in the logging config group. If log_root is set, error and info will be set to error.log and info.log within log_root respectively. The _log_file variables will override that setting and are absolute paths.

Log files are written to until they reach a specified size. They are then rotated, keeping a number of files before the oldest file is deleted. The size of the files can be set via the log_max_size setting. This is either an integer value in bytes or a string with a integer value followed by a ‘kb’, ‘Mb’, or ‘Gb’ suffix. The number of files can be adjusted using the log_backup_count setting.

By default, http accesses are logged just to stdout The log_access setting is a list of where http accesses are logged that can include ‘screen’ and ‘info’. An empty list will stop logging http accesses.

Girder logs various errors and information a different log levels. The default log level is ‘INFO’, and can be adjusted by the log_level setting. Valid choices in increasing order of verbosity include ‘CRITICAL’, ‘ERROR’, ‘WARNING’, ‘INFO’, and ‘DEBUG’. By default, when logs that are WARNING, ERROR, or CRITICAL are sent to the error log file, and logs that are INFO or DEBUG are sent to the info log file. The log_max_info_level setting can be adjusted to send more log messages to the info log (in addition to the error log). For instance, if log_max_info_level is set to ‘CRITICAL’, all messages are sent to the info log, while the error log will just contain warnings and errors.

Setting log_quiet to True will suppress all output to stdout (even http access logs if specified in log_access). Information will still be sent to the log files.

Server thread pool

Girder can handle multiple requests at one time. The maximum number of simultaneous requests is set with the server.thread_pool value in the global config group. Once this many connections have been made to Girder, additional connections will block until existing connections finish.

Most operations on Girder are quick, and therefore do not use up a connection for a long duration. Some connections, notably calls to the notification/stream endpoint, can block for long periods. If you expect to have many clients, either increase the size of the thread pool or switch to using intermittent polling rather than long-duration connections.

Each available thread uses up some additional memory and requires internal socket or handle resources. The exact amount of memory and resources is dependent on the host operating system and the types of queries made to Girder. As one benchmark from an Ubuntu server, each additional available but unused connection requires roughly 25 kb of memory. If all connections are serving notification streams, each uses around 50 kb of memory.

Changing file limits

If all server threads are in use, additional attempts to connect will use a file handle while waiting to be processed. The number of open files is limited by the operating system, and may need to be increased. This limit affects actual connections, pending connections, and file use.

The method of changing file limits varies depending on your operating system. If your operating system is not listed here, try a web search for “Open Files Limit” along with your OS’s name.

Linux

You can query the current maximum number of files with the command:

ulimit -Sn

To increase this number for all users, as root or with sudo privileges, edit /etc/security/limits.conf and append the following lines to the end of the file:

*    soft    nofile    32768
*    hard    nofile    32768

Save and close the file. The user running the Girder server will need to logout and log back in and restart the Girder server for the new limits to take effect.

This raises the limits for all users on the system. You can limit this change to just the user that runs the Girder server. See the documentation for /etc/security/limits.conf for details.

Managing Routes

When plugins which have their own custom webroot are enabled, they are mounted at /pluginName. In certain cases it may be desirable for the site administrator to mount such plugins at their own specified paths.

These paths can be modified by navigating to Admin Console -> Server Configuration and visiting the Routing section.

Girder FUSE Mount

The files in Girder can be mounted in a user file system (FUSE). The allows Girder plugins and C extensions that cannot directly access Girder files to read them as if they were physical files. This allows external libraries that require file access to read Girder files, regardless of which assetstore they are stored on. It also uses the underlying operating system’s caching to improve reading these files.

Additionally, some C extensions expect multiple files in the same Girder item to be stored in the same directory and with specific file extensions. When a Girder FUSE mount is available, these will work – instead of passing the Girder file object, call File().getGirderMountFilePath() to get a path to the file.

To enable a Girder FUSE mount, run girder mount <mount path>, where <mount path> is an empty directory. This can be done before or after a Girder server is started – the mount process sets a value in the Girder Mongo database to inform Girder servers where the mount is located.

The mount can be unmounted used girder mount <mount path> -u or with standard system unmount commands (e.g., fusermount -u <mount path> or sudo umount <mount path>.

Note

If the Girder mount process is sent SIGKILL with open file handles, it may not be possible to fully clean up the open file system, and defunct processes may linger. This is a limitation of libfuse, and may require a reboot to clear the lingering mount. Use an unmount command or SIGTERM.

Installation

There are some additional python dependencies to use the Girder mount commnd. These can be installed via:

$ pip install -e .[mount]

SFTP Service

In addition to its normal HTTP server, the Girder package also includes a lightweight SFTP server that can be used to serve the same underlying data using the SSH File Transfer Protocol. This server provides a read-only view of the data in a Girder instance, and supports either anonymous access (users must pass the username “anonymous” in their SFTP client), or authenticated access with their Girder login name and password. This protocol can make it much easier to download large nested datasets with many individual files, and is tolerant to network failure since it supports resuming interrupted downloads of entire data hierarchies.

After installing the Girder package via pip, you should now see the girder executable in your PATH. Running girder sftpd will start the SFTP server using the same database configuration as the main Girder HTTP server.

The SFTP server requires a private key file for secure communication with clients. If you do not pass an explicit path to an RSA private key file, the service will look for one at ~/.ssh/id_rsa. It’s recommended to make a special key just for the SFTP service, e.g.:

ssh-keygen -t rsa -b 2048 -f my_key.rsa -N ''
girder sftpd -i my_key.rsa

You can control the port on which the server binds by passing a -p <port> argument to the server CLI. The default port is 8022.

Note

If SFTP clients are logging in as a user with two-factor authentication (one-time passwords) enabled, they must append the one-time authentication code to the user’s basic password.

Deployment Alternatives

There are many ways to deploy Girder into production. Here is a set of guides on how to deploy Girder to several different platforms.

Reverse Proxy

In many cases, it is useful to route multiple web services from a single server. For example, if you have a server accepting requests at www.example.com, you may want to forward requests to www.example.com/girder to a Girder instance listening on port 9000.

Anytime you deploy behind a proxy, Girder must be configured properly in order to serve content correctly. This can be accomplished by setting a few parameters in your local configuration file (see Configuration). In this example, we have the following:

[global]
server.socket_host = "127.0.0.1"
server.socket_port = 9000
tools.proxy.on = True

[server]
api_root = "/girder/api/v1"
static_public_path = "/girder/static"

Note

If your chosen proxy server does not add the appropriate X-Forwarded-Host header (containing the host used in http requests, including any non-default port to proxied requests), the tools.proxy.base and tools.proxy.local configuration options must also be set in the [global] section as:

tools.proxy.base = "http://www.example.com/girder"
tools.proxy.local = ""

Note that after changing these parameters, it is necessary to rebuild the web client.

Nginx

Nginx can be used by adding a block such as:

location /girder/ {
    proxy_set_header Host $proxy_host;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header X-Forwarded-Host $host;
    proxy_set_header X-Forwarded-Proto $scheme;
    proxy_pass http://localhost:9000/;
    # Must set the following for SSE notifications to work
    proxy_buffering off;
    proxy_cache off;
    proxy_set_header Connection '';
    proxy_http_version 1.1;
    chunked_transfer_encoding off;
    proxy_read_timeout 600s;
    proxy_send_timeout 600s;
    # proxy_request_buffering option only works on nginx >= 1.7.11
    # but is necessary to support streaming requests
    proxy_request_buffering off;
}

And under the containing server block, make sure to add the following rule:

server {
    client_max_body_size 500M;
    # ... elided configuration
}

Ansible

Installing / Upgrading provides a complete installation workflow, which depends primarily on the Ansible roles:

• girder.girder
• girder.mongodb
• girder.nginx

These roles are also independently usable and may be composed as part of larger, custom Ansible playbooks.

Docker Container

Every time a new commit is pushed to master, Docker Hub is updated with new images for running Girder. These containers expose Girder at port 8080 and require the database URL to be passed in as an option. For more information, see the Docker Hub Page. Since the container does not run a database, you’ll need to run a command in the form:

$ docker run -p 8080:8080 girder/girder --database mongodb://db-server-external-ip:27017/girder --host 0.0.0.0

The girder/girder:latest image is quite large and intended for use as a base image for plugin developers. If you just need to run Girder, a slim image based on alpine linux is provided without many of the build tools needed to compile wheels or run girder client builds:

$ docker run -p 8080:8080 girder/girder:slim --database mongodb://db-server-external-ip:27017/girder

User Documentation

User Guide

Girder is a Data Management Toolkit. It is a complete back-end (server side) technology that can be used with other applications via its RESTful API, or it can be used via its own front-end (client side web pages and JavaScript).

Girder is designed to be robust, fast, scalable, extensible, and easy to understand.

Girder is built in Python.

Girder is open source, licensed under the Apache License, Version 2.0.

Document Conventions

This User Guide is written for end-users of Girder, rather than developers. If you have suggestions or questions about this documentation, feel free to contact us through our Discourse discussion forum, on GitHub, or through Kitware support.

Girder specific entities will be formatted like this.

Concepts

Users

Like in many systems, Users in Girder correspond to the identity of a user of the system. It is possible to use many features of Girder anonymously (that is, without being logged in as a registered user), but typically in order to make changes to the system, a user must be logged in as their corresponding User account. Users can be granted permissions on resources in the system directly, and can belong to Groups.

Groups

Groups group together Users. Users can belong to any number of Groups, and usually join by being invited and accepting the invitation. One of the main purposes of Groups is to allow role-based access control; resources can grant access to Groups rather than just individual users, such that changing access to sets of resources can be managed simply by changing Group membership. See the Permissions section for more information about group-based access control.

Collections

Collections are the top level objects in the data organization hierarchy. Within each Collection, there can be many Folders, and the Collection itself is also an access controlled resource. Typically Collections are used to group data that share something in common, such as what project the data are used for, or what institution they belong to.

Folders

A Girder Folder is the common software concept of a folder, namely a hierarchically nested organizational structure. Girder Folders can contain nothing (although this may not be particularly useful), other Folders, Items, or a combination of Folders and Items. Folders in Girder have permissions set on them, and the Items within them inherit permissions from their containing Folders.

Items

A Girder Item is the basic unit of data in the system. Items live beneath Folders and contain 0 or more Files. Items in Girder do not have permissions set on them, they inherit permissions by virtue of living in a Folder (which has permissions set on it). Most Items contain a single File, except in special cases where multiple files make up a single piece of data.

Each Item may contain any number of arbitrary key/value pairs, termed metadata. Metadata keys must be non-empty strings and must not contain a period (‘.’) or begin with a dollar sign (‘$’). Metadata values can be anything, including strings, numeric values, and even arbitrary JSON objects.

Files

Files represent raw data objects, just like the typical concept of files in a filesystem. Files exist within Items, typically with a one-to-one relationship between the File and its containing Item. Files in Girder are much like files on a filesystem, but they are actually more abstract. For instance, some Files are simply links to external URLs. All Files that are not external links must be contained within an Assetstore.

Assetstores

Assetstores are an abstraction representing a repository where the raw bytes of Files are actually stored. The Assetstores known to a Girder instance may only be set up and managed by administrator Users.

In the core of Girder, there are three supported Assetstore types:

• Filesystem

Files uploaded into this type of Assetstore will be stored on the local system filesystem of the server using content-addressed storage. Simply specify the root directory under which files should be stored.

Note

If your Girder environment has multiple different application servers and you plan to use the Filesystem assetstore type, you must set the assetstore root to a location on the filesystem that is shared between all of the application servers.

• GridFS

This Assetstore type stores files directly within your Mongo database using the GridFS model. You must specify the database name where files will be stored; for now, the same credentials will be used for this database as for the main application database.

This database type has the advantage of automatically scaling horizontally with your DBMS. However, it is marginally slower than the Filesystem assetstore type in a typical single-server use case.

• S3

This Assetstore type stores files in an Amazon S3 bucket. You must provide the bucket name, an optional path prefix within the bucket, and authentication credentials for the bucket. When using this assetstore type, Girder acts as the broker for the data within S3 by authorizing the user agent via signed HTTP requests. The primary advantage of this type of assetstore is that none of the actual bytes of data being uploaded and downloaded ever go through the Girder system, but instead are sent directly between the client and S3.

If you want to use an S3 assetstore, the bucket used must support CORS requests. This can be edited by navigating to the bucket in the AWS S3 console, selecting Properties, then Permissions, and then clicking Edit CORS Configuration. The below CORS configuration is sufficient for Girder’s needs:

<?xml version="1.0" encoding="UTF-8"?>
<CORSConfiguration xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
    <CORSRule>
        <AllowedOrigin>*</AllowedOrigin>
        <AllowedMethod>GET</AllowedMethod>
        <AllowedMethod>PUT</AllowedMethod>
        <AllowedMethod>POST</AllowedMethod>
        <MaxAgeSeconds>3000</MaxAgeSeconds>
        <ExposeHeader>ETag</ExposeHeader>
        <AllowedHeader>*</AllowedHeader>
    </CORSRule>
</CORSConfiguration>

Note

Google Storage is supported through an S3 assetstore. Specify the service as storage.googleapis.com. Public storage buckets can be accessed in read-only mode without an access key ID or secret access key.

Support for Google Storage may not be complete, as it is accessed via an S3 access library and is not automatically tested.

Permissions

Permission Levels

There are four levels of permission a User can have on a resource. These levels are in a strict hierarchy with a higher permission level including all of the permissions below it. The levels are:

1. No permission (cannot view, edit, or delete a resource)
2. READ permission (can view and download resources)
3. WRITE permission (includes READ permission, can edit the properties of a resource)
4. ADMIN also known as own permission, (includes READ and WRITE permission, can delete the resource and also control access on it)

A site administrator always has permission to take any action.

Permission Model

Permissions are resolved at the level of a User, i.e., for any User, an attempt to take a certain action will be allowed or disallowed based on the permissions for that User, as a function of the resource, the operation, the permissions set on that resource for that User, and the permissions set on that resource by any Groups the User is a member of.

Permissions are always additive. That is, given a User with a certain permission on a resource, that permission can not be taken away from the User by addition of other permissions to the system, but only through removing existing permissions to that User or removing that User from a Group. Once again, a site admin always has permission to take any action.

Collections

Collections can be Public (meaning viewable even by anonymous users) or Private (meaning viewable only by those with READ access). Collections can have permissions set on them at the individual User level and Group level, meaning that a given User or Group can have READ, WRITE, or ADMIN permissions set on the Collection.

Folders

Folders can be Public (meaning viewable even by anonymous users) or Private (meaning viewable only by those with READ access). Folders can have permissions set on them at the individual User level and Group level, meaning that a given User or Group can have READ, WRITE, or ADMIN permissions set on the Folder. Folders inherit permissions from their parent Folder.

Items

Items always inherit their permissions from their parent Folder. Each access-controlled resource (e.g., Folder, Collection) has a list of permissions granted on it, and each item in that list is a mapping of either Users to permission level or Groups to permission level. This is best visualized by opening the “Access control” dialog on a Folder in the hierarchy. The actual permission level that a User has on that resource is defined as: the maximum permission level available based on the permissions granted to any Groups that the User is member of, or permissions granted to that User specifically.

Groups

For access control, Groups can be given any level of access to a resource that an individual User can, and this is managed at the level of the resource in question.

For permissions on Groups themselves, Public Groups are viewable (READ permission) to anyone, even anonymous users. Private Groups are not viewable or even listable to any Users except those that are members of the Group, or those that have been invited to the Group.

Groups have three levels of roles that Users can have within the Group. They can be Members, Moderators (also indicates that they are Members), and Administrators (also indicates that they are Members).

Users that are not Members of a group can request to become Members of a Group if that Group is Public.

Members of a Group can see the membership list of the Group, including roles, and can see pending requests and invitations for the group. If a User has been invited to a Group, they have Member access to the Group even before they have accepted the invitation. A Member of a Group can leave the group, at which point they are no longer Members of the Group.

Moderators of a Group have all of the abilities of Group Members. Moderators can also invite Users to become Members, can accept or reject a request by a User to become a Member, can remove Members or Moderators from the Group, and can edit the Group which includes changing the name and description and changing the Public/Private status of the Group.

Administrators of a Group have all of the abilities of Group Moderators. Administrators can also delete the Group, promote a Member to Moderator or Administrator, demote an Administrator or Moderator to Member, and remove any Member, Moderator, or Administrator from the Group.

The creator of a Group is an Administrator of a group. Any logged in User can create a Group.

User

Users have ADMIN access on themselves, and have READ access on other Users.

API keys

Like many web services, Girder’s API is designed for programmatic interaction. API keys can facilitate these sorts of interactions – they enable client applications to interact with the server on behalf of your user without actually authenticating with your password. They can also be granted restricted access to only a limited set of functionality of the API.

Under the My account page, there is a tab called API keys where these keys can be created and managed. You can have many API keys; in fact, it’s recommended to use a different key for each different client application that needs authenticated access to the Girder server. By convention, the Name field of API keys can be used to specify what application is making use of the key in a human-readable way, although you may name your keys however you want.

Each API key can be used to gain authentication tokens just like when you log in with a username and password. If you want to limit the maximum amount of time that these tokens last, you can do so on a per-key basis, or leave the token duration field empty to use the server default.

When creating and updating API keys, you can also select among two modes: you can either grant full access to the API key, which gives unrestricted API access as though you are logged in as your user, or you can choose limited functionality scopes from a list of checkboxes to restrict the sorts of actions that the key will allow.

It is also possible to deactivate a key temporarily. If you deactivate an existing key, it will immediately delete all active tokens created with that key, and also stop that key from being able to create new tokens until you activate it once again. Alternatively, you can delete the key altogether, which will make the key and any tokens created with it never work again.

Using Girder CLI to Upload and Download data

See Python Client and Girder CLI

Developer Documentation

Developer Installation

You can either install Girder natively on your machine or inside a virtual machine (VM) with Vagrant.

Virtual Machine

The easiest way to develop for Girder is within a VM using Vagrant. For this, you need Vagrant and VirtualBox. Girder is tested to work seamlessly with Vagrant 2.0 and VirtualBox 5.1.

Once you have those installed, obtain the Girder source code by cloning the Git repository on GitHub:

git clone https://github.com/girder/girder.git
cd girder

Inside of the Girder directory, simply run:

vagrant up
vagrant ssh
girder serve

This creates a VM running Ubuntu 18.04, then automatically installs Girder within it. After it completes, Girder will be up and running at http://localhost:9080/ on the host machine.

The VM is linked to your local Girder directory, so changes made locally will impact the Girder instance running in the VM.

To access the VM, run from the Girder directory:

vagrant ssh

To rebuild the web client:

girder build --dev

For more development documentation, see During Development

System Dependency Reference

This provides an authoritative listing of Girder’s external system dependencies, for development reference.

Server Install

Installation of Girder’s server has the following system dependencies:

• Python v3.6+

  • This is necessary to run most of the server software.

  Note

  Girder performs continuous integration testing using Python v3.6. Girder should work on newer Python v3.7+ versions as well, but that support is not verified by Girder’s automated testing at this time, so use at your own risk.

• pip v8.1+

  • This is necessary to install Python packages.
  • v8.1 is required to install packages via Python wheels, which significantly reduce system dependencies on manylinux-compatible distributions (which includes almost everything except Alpine).

• setuptools v21+

  • This is necessary to install Python packages.
  • v21+ is required to parse Girder’s setup.py file, but the latest version can and should be installed via pip.

• A Python virtual environment (optional)

  • This is necessary to prevent conflicts with Girder’s Python dependencies and allow installation as a non-root user.

  • Virtual environments are managed by the virtualenv package in Python v2.x and by the built-in venv module in Python v3.5+.

    Note

    Developers needing to work on several virtual environments should consider using other packages that help manage them such as:

    • virtualenvwrapper
    • autoenv
    • pyenv-virtualenv
    • pyenv-virtualenvwrapper

• A C build environment for Python

  • This is necessary to install the psutil Python package (a requirement of Girder).
  • At a minimum, the environment must include GCC and the Python development headers.

Server Plugins Install

Some of Girder’s plugins require additional system dependencies, if they are installed:

• OpenLDAP v2.4.11+ development headers (for the ldap plugin)
  • This is necessary to install the python-ldap Python package (a requirement of the ldap plugin).
• Cyrus SASL development headers (for the ldap plugin)
  • This is an optional dependency for installing the python-ldap Python package (a requirement of the ldap plugin).

Server Runtime

Running Girder’s server has the following additional system dependencies:

• MongoDB v3.6+
  • This is necessary for Girder’s primary application database.
  • MongoDB does not need to be installed on the same machine as Girder’s server, so long as a network connection to a remote MongoDB is possible.
• An SMTP relay (optional)
  • This is necessary to send emails to users and administrators.
  • There are multiple web-based services that provide SMTP relays at no cost for smaller volumes of mail.
    • Mailgun is known to work well for typical deployments.
    • Amazon SES may be simpler to manage for deployments already in AWS.
    • Any other SMTP service should be compatible, though some may only accept mail from a limited set of “From” addresses (which may be configured via a setting within Girder).
  • Advanced system administrators may wish to run their own SMTP server, like Postfix, as a relay.
    • Beware that self-hosted SMTP relays may need additional configuration with IP whitelisting, SPF, DKIM, and other measures before their mail can be reliably delivered to some email providers.

Web Client Build

Building Girder’s web client has the following system dependencies:

• Node.js v10+
  • This is necessary to execute many aspects of the web client build system.
• npm v5.2+
  • This is necessary to install web client packages.
• Git
  • This is necessary to introspect Girder’s install environment and generate version information as part of the web client build process.

API Documentation

RESTful API

Clients access Girder servers uniformly via its RESTful web API. By providing a single, stable, consistent web API, it is possible to write multiple interchangeable clients using different technologies.

When a Girder instance is deployed, it typically also serves a page that uses Swagger to document all available RESTful endpoints in the web API and also provide an easy way for users to execute those endpoints with parameters of their choosing. In this way, the Swagger page is just the simplest and lightest client application for Girder. This page is served out of the path /api.

Internal Python API

Models

In Girder, the model layer is responsible for actually interacting with the underlying database. Model classes are where the documents representing resources are actually saved, retrieved, and deleted from the DBMS. Validation of the resource documents is also done in the model layer, and is invoked each time a document is about to be saved.

Typically, there is a model class for each resource type in the system. These models are loaded as singletons for efficiency, but you should use them like normal objects. For example, to use the list method of the Group model:

from girder.models.group import Group
groups = Group().list(user=self.getCurrentUser())

All models that require the standard access control semantics should extend the AccessControlledModel class. Otherwise, they should extend the Model class.

All model classes must have an initialize method in which they declare the name of their corresponding Mongo collection, as well as any collection indexes they require. For example, to make a model whose documents live in a collection called cat_collection and ensure that the name key is indexed on that collection, you would use the following initialize method:

from girder.models.model_base import Model

class Cat(Model):
    def initialize(self):
        self.name = 'cat_collection'
        self.ensureIndex('name')

Model Helper Functions

girder.models.getDbConfig()

Get the database configuration values from the cherrypy config.

girder.models.getDbConnection(uri=None, replicaSet=None, quiet=False, **kwargs)

Get a MongoClient object that is connected to the configured database. We lazy-instantiate a module-level singleton, the MongoClient objects manage their own connection pools internally. Any extra kwargs you pass to this method will be passed through to the MongoClient.

Parameters:

• uri – if specified, connect to this mongo db rather than the one in the config.
• replicaSet – if uri is specified, use this replica set.
• quiet (bool) – if true, don’t logprint warnings and success.

Model Base

class girder.models.model_base.AccessControlledModel

Any model that has access control requirements should inherit from this class. It enforces permission checking in the load() method and provides convenient methods for testing and requiring user permissions. It also provides methods for setting access control policies on the resource.

copyAccessPolicies(src, dest, save=False)

Copies the set of access control policies from one document to another.

Parameters:

• src (dict) – The source document to copy policies from.
• dest (dict) – The destination document to copy policies onto.
• save (bool) – Whether to save the destination document after copying.

Returns:

The modified destination document.

filter(doc, user, additionalKeys=None)

Filter this model for the given user according to the user’s access level. Also adds the special _accessLevel field to the document to indicate the user’s highest access level. This filters a single document that the user has at least read access to. For filtering a set of documents, see filterResultsByPermission().

Parameters:

• doc (dict or None) – The document of this model type to be filtered.
• user (dict or None) – The current user for whom we are filtering.
• additionalKeys (list, tuple, or None) – Any additional keys that should be included in the document for this call only.

Returns:

The filtered document (dict).

filterResultsByPermission(cursor, user, level, limit=0, offset=0, removeKeys=(), flags=None)

Given a database result cursor, this generator will yield only the results that the user has the given level of access and specified access flags on, respecting the limit and offset specified.

Parameters:

• cursor – The database cursor object from “find()”.
• user (dict or None) – The user to check policies against.
• level (AccessType) – The access level.
• limit (int) – Maximum number of documents to return
• offset (int) – The offset into the results
• removeKeys (list) – List of keys that should be removed from each matching document.
• flags (flag identifier, or a list/set/tuple of them) – A flag or set of flags to test.

findWithPermissions(query=None, offset=0, limit=0, timeout=None, fields=None, sort=None, user=None, level=0, **kwargs)

Search the collection by a set of parameters, only returning results that the combined user and level have permission to access. Passes any extra kwargs through to the underlying pymongo.collection.find function.

Parameters:

• query (dict) – The search query (see general MongoDB docs for “find()”)
• offset (int) – The offset into the results
• limit (int) – Maximum number of documents to return
• timeout (int) – Cursor timeout in ms. Default is no timeout.
• fields (str, list, set, or tuple) – A mask for filtering result documents by key, or None to return the full document, passed to MongoDB find() as the projection param. This is a string or iterable of strings to be included from the document, or dict for an inclusion or exclusion projection`.
• sort (List of (key, order) tuples.) – The sort order.
• user (dict or None) – The user to check policies against.
• level (AccessType) – The access level. Explicitly passing None skips doing permissions checks.

Returns:

A pymongo Cursor or CommandCursor. If a CommandCursor, it has been augmented with a count function.

getAccessLevel(doc, user)

Return the maximum access level for a given user on a given object. This can be useful for alerting the user which set of actions they are able to perform on the object in advance of trying to call them.

Parameters:

• doc – The object to check access on.
• user – The user to get the access level for.

Returns:

The max AccessType available for the user on the object.

getFullAccessList(doc)

Return an object representing the full access list on this document. This simply includes the names of the users and groups with the ACL.

If the document contains references to users or groups that no longer exist, they are simply removed from the ACL, and the modified ACL is persisted at the end of this method if any removals occurred.

Parameters:doc (dict) – The document whose ACL to return. Returns:A dict containing users and groups keys.

hasAccess(doc, user=None, level=0)

This method looks through the object’s permission set and determines whether the user has the given permission level on the object.

Parameters:

• doc (dict) – The document to check permission on.
• user (dict) – The user to check against.
• level (AccessType) – The access level.

Returns:

Whether the access is granted.

hasAccessFlags(doc, user=None, flags=None)

Test whether a specific user has a given set of access flags on the given resource. Returns True only if the user has all of the flags by virtue of either group membership, public flags, or explicit access granted to the user.

Parameters:

• doc (dict) – The resource to test access on.
• user (dict or None) – The user to check against.
• flags – A flag or set of flags to test.

list(user=None, limit=0, offset=0, sort=None)

Return a list of documents that are visible to a user.

Parameters:

• user (dict or None) – The user to filter for
• limit (int) – Maximum number of documents to return
• offset (int) – The offset into the results
• sort (List of (key, order) tuples) – The sort order

load(id, level=2, user=None, objectId=True, force=False, fields=None, exc=False)

Override of Model.load to also do permission checking.

Parameters:

• id (str or ObjectId) – The id of the resource.
• level (AccessType) – The required access type for the object.
• user (dict or None) – The user to check access against.
• objectId (bool) – Whether the id should be coerced to ObjectId type.
• force (bool) – If you explicitly want to circumvent access checking on this resource, set this to True.
• fields (list or dict) – A mask for filtering result documents by key, or None to return the full document, passed to MongoDB find() as the projection param.
• exc (bool) – If not found, throw a ValidationException instead of returning None.

Raises:

ValidationException – If an invalid ObjectId is passed.

Returns:

The matching document, or None if no match exists.

prefixSearch(query, user=None, filters=None, limit=0, offset=0, sort=None, fields=None, level=0, prefixSearchFields=None)

Custom override of Model.prefixSearch to also force permission-based filtering. The parameters are the same as Model.prefixSearch.

Parameters:

• query (str) – The prefix string to look for
• user (dict or None) – The user to apply permission filtering for.
• filters (dict) – Any additional query operators to apply.
• limit (int) – Maximum number of documents to return
• offset (int) – The offset into the results
• sort (List of (key, order) tuples.) – The sort order.
• fields (str, list, set, or tuple) – A mask for filtering result documents by key, or None to return the full document, passed to MongoDB find() as the projection param. This is a string or iterable of strings to be included from the document, or dict for an inclusion or exclusion projection`.
• level (girder.constants.AccessType) – The access level to require.
• prefixSearchFields – To override the model’s prefixSearchFields attribute for this invocation, pass an alternate iterable.

Returns:

A pymongo cursor. It is left to the caller to build the results from the cursor.

requireAccess(doc, user=None, level=0)

This wrapper just provides a standard way of throwing an access denied exception if the access check fails.

requireAccessFlags(doc, user=None, flags=None)

Provides a standard way of throwing an access exception if a flag access check fails.

setAccessList(doc, access, save=False, user=None, force=False)

Set the entire access control list to the given value. This also saves the resource in its new state to the database.

Parameters:

• doc (dict) – The resource to update.
• access (dict) – The new access control list to set on the object.
• save (boolean) – Whether to save after updating.
• user (dict) – The user performing the update. This is used to control updating of access flags that require admin permission to enable.
• force (bool) – Set this to True to set the flags regardless of the passed in user’s permissions.

Returns:

The updated resource.

setGroupAccess(doc, group, level, save=False, flags=None, currentUser=None, force=False)

Set group-level access on the resource.

Parameters:

• doc (dict) – The resource document to set access on.
• group (dict) – The group to grant or remove access to.
• level (AccessType or None) – What level of access the group should have. Set to None to remove all access for this group.
• save (bool) – Whether to save the object to the database afterward. Set this to False if you want to wait to save the document for performance reasons.
• flags (specific flag identifier, or a list/tuple/set of them) – List of access flags to grant to the group.
• currentUser (dict or None) – The user performing this action. Only required if attempting to set admin-only flags on the resource.
• force (bool) – Set this to True to set the flags regardless of the passed in currentUser’s permissions (only matters if flags are passed).

Returns:

The updated resource document.

setPublic(doc, public, save=False)

Set the flag for public read access on the object.

Parameters:

• doc (dict) – The document to update permissions on.
• public (bool) – Flag for public read access.
• save (bool) – Whether to save the object to the database afterward. Set this to False if you want to wait to save the document for performance reasons.

Returns:

The updated resource document.

setPublicFlags(doc, flags, user=None, append=False, save=False, force=False)

Set access flags that are granted on this resource to anonymous users. This means any user, whether anonymous or logged in, will receive all of the specified permissions. This also validates that the user attempting to set the flags has permission to do so. Any flags that are invalid or that the user is not authorized to enable will be discarded from the list.

Parameters:

• doc (dict) – The document to update access flags on.
• flags (flag identifier, or a list/set/tuple of them) – Flags or set of flags to add.
• user (dict) – The user performing this action.
• append (bool) – Whether to append to the list or replace it.
• save (bool) – Whether to save the document to the database afterward.
• force (bool) – Set this to True to set the flags regardless of the passed in user’s permissions.

setUserAccess(doc, user, level, save=False, flags=None, currentUser=None, force=False)

Set user-level access on the resource.

Parameters:

• doc (dict) – The resource document to set access on.
• user (dict) – The user to grant or remove access to.
• level (AccessType or None) – What level of access the user should have. Set to None to remove all access for this user.
• save (bool) – Whether to save the object to the database afterward. Set this to False if you want to wait to save the document for performance reasons.
• flags (specific flag identifier, or a list/tuple/set of them) – List of access flags to grant to the group.
• currentUser – The user performing this action. Only required if attempting to set admin-only flags on the resource.
• force (bool) – Set this to True to set the flags regardless of the passed in currentUser’s permissions (only matters if flags are passed).

Returns:

The modified resource document.

textSearch(query, user=None, filters=None, limit=0, offset=0, sort=None, fields=None, level=0)

Custom override of Model.textSearch to also force permission-based filtering. The parameters are the same as Model.textSearch.

Parameters:

• query (str) – The text query. Will be stemmed internally.
• user (dict or None) – The user to apply permission filtering for.
• filters (dict) – Any additional query operators to apply.
• limit (int) – Maximum number of documents to return
• offset (int) – The offset into the results
• sort (List of (key, order) tuples) – The sort order
• fields (str, list, set, or tuple) – A mask for filtering result documents by key, or None to return the full document, passed to MongoDB find() as the projection param. This is a string or iterable of strings to be included from the document, or dict for an inclusion or exclusion projection`.
• level (girder.constants.AccessType) – The access level to require.

class girder.models.model_base.Model

Model base class. Models are responsible for abstracting away the persistence layer. Each collection in the database should have its own model. Methods that deal with database interaction belong in the model layer.

ensureIndex(index)

Like ensureIndices, but declares just a single index rather than a list of them.

ensureIndices(indices)

Subclasses should call this with a list of strings representing fields that should be indexed in the database if there are any. Otherwise, it is not necessary to call this method. Elements of the list may also be a list or tuple, where the second element is a dictionary that will be passed as kwargs to the pymongo create_index call.

ensureTextIndex(index, language='english')

Call this during initialize() of the subclass if you want your model to have a full-text searchable index. Each collection may have zero or one full-text index.

Parameters:language (str) – The default_language value for the text index, which is used for stemming and stop words. If the text index should not use stemming and stop words, set this param to ‘none’.

exposeFields(level, fields)

Expose model fields to users with the given access level. Subclasses should call this in their initialize method to declare what fields should be exposed to what access levels if they are using the default filter implementation in this class. Since filtered fields are sets, this method is idempotent.

Parameters:

• level (AccessType) – The required access level for the field.
• fields (str, list, set, or tuple) – A field or list of fields to expose for that level.

filter(doc, user=None, additionalKeys=None)

Filter this model for the given user. This is a default implementation that assumes this model has no notion of access control, and simply allows all keys under READ access level, and conditionally allows any keys assigned to SITE_ADMIN level.

Parameters:

• doc (dict or None) – The document of this model type to be filtered.
• user (dict or None) – The current user for whom we are filtering.
• additionalKeys (list, tuple, set, or None) – Any additional keys that should be included in the document for this call only.

Returns:

The filtered document (dict).

filterDocument(doc, allow=None)

This method will filter the given document to make it suitable to output to the user.

Parameters:

• doc (dict) – The document to filter.
• allow (List of strings) – The whitelist of fields to allow in the output document.

find(query=None, offset=0, limit=0, timeout=None, fields=None, sort=None, **kwargs)

Search the collection by a set of parameters. Passes any extra kwargs through to the underlying pymongo.collection.find function.

Parameters:

• query (dict) – The search query (see general MongoDB docs for “find()”)
• offset (int) – The offset into the results
• limit (int) – Maximum number of documents to return
• timeout (int) – Cursor timeout in ms. Default is no timeout.
• fields (str, list, set, or tuple) – A mask for filtering result documents by key, or None to return the full document, passed to MongoDB find() as the projection param. This is a string or iterable of strings to be included from the document, or dict for an inclusion or exclusion projection`.
• sort (List of (key, order) tuples.) – The sort order.

Returns:

A pymongo database cursor.

findOne(query=None, fields=None, **kwargs)

Search the collection by a set of parameters. Passes any kwargs through to the underlying pymongo.collection.find_one function.

Parameters:

• query (dict) – The search query (see general MongoDB docs for “find()”)
• fields (str, list, set, or tuple) – A mask for filtering result documents by key, or None to return the full document, passed to MongoDB find() as the projection param. This is a string or iterable of strings to be included from the document, or dict for an inclusion or exclusion projection`.
• sort (List of (key, order) tuples.) – The sort order.

Returns:

the first object that was found, or None if none found.

hideFields(level, fields)

Hide a field, i.e. make sure it is not exposed via the default filtering method. Since the filter uses a white list, it is only ever necessary to call this for fields that were added previously with exposeFields().

Parameters:

• level (AccessType) – The access level to remove the fields from.
• fields (str, list, set, or tuple) – The field or fields to remove from the white list.

increment(query, field, amount, **kwargs)

This is a specialization of the update method that atomically increments a field by a given amount. Additional kwargs are passed directly through to update.

Parameters:

• query (dict) – The search query for documents to update, see general MongoDB docs for “find()”
• field (str) – The name of the field in the document to increment.
• amount (int or float) – The amount to increment the field by.

initialize()

Subclasses should override this and set the name of the collection as self.name. Also, they should set any indexed fields that they require.

load(id, objectId=True, fields=None, exc=False)

Fetch a single object from the database using its _id field.

Parameters:

• id (string or ObjectId) – The value for searching the _id field.
• objectId (bool) – Whether the id should be coerced to ObjectId type.
• fields (str, list, set, or tuple) – A mask for filtering result documents by key, or None to return the full document, passed to MongoDB find() as the projection param. This is a string or iterable of strings to be included from the document, or dict for an inclusion or exclusion projection`.
• exc (bool) – Whether to raise a ValidationException if there is no document with the given id.

Returns:

The matching document, or None.

prefixSearch(query, offset=0, limit=0, sort=None, fields=None, filters=None, prefixSearchFields=None, **kwargs)

Search for documents in this model’s collection by a prefix string. The fields that will be searched based on this prefix must be set as the prefixSearchFields attribute of this model, which must be an iterable. Elements of this iterable must be either a string representing the field name, or a 2-tuple in which the first element is the field name, and the second element is a string representing the regex search options.

Parameters:

• query (str) – The prefix string to look for
• offset (int) – The offset into the results
• limit (int) – Maximum number of documents to return
• sort (List of (key, order) tuples.) – The sort order.
• fields (str, list, set, or tuple) – A mask for filtering result documents by key, or None to return the full document, passed to MongoDB find() as the projection param. This is a string or iterable of strings to be included from the document, or dict for an inclusion or exclusion projection`.
• filters (dict) – Any additional query operators to apply.
• prefixSearchFields – To override the model’s prefixSearchFields attribute for this invocation, pass an alternate iterable.

Returns:

A pymongo cursor. It is left to the caller to build the results from the cursor.

reconnect()

Reconnect to the database and rebuild indices if necessary. Users should typically not have to call this method.

remove(document, **kwargs)

Delete an object from the collection; must have its _id set.

Parameters:document – the item to remove.

removeWithQuery(query)

Remove all documents matching a given query from the collection. For safety reasons, you may not pass an empty query.

Parameters:query (dict) – The search query for documents to delete, see general MongoDB docs for “find()”

save(document, validate=True, triggerEvents=True)

Create or update a document in the collection. This triggers two events; one prior to validation, and one prior to saving. Either of these events may have their default action prevented.

Parameters:

• document (dict) – The document to save.
• validate (bool) – Whether to call the model’s validate() before saving.
• triggerEvents – Whether to trigger events for validate and pre- and post-save hooks.

subtreeCount(doc)

Return the size of the subtree rooted at the given document. In general, if this contains items or folders, it will be the count of the items and folders in all containers. If it does not, it will be 1. This returns the absolute size of the subtree, it does not filter by permissions.

Parameters:doc (dict) – The root of the subtree.

textSearch(query, offset=0, limit=0, sort=None, fields=None, filters=None, **kwargs)

Perform a full-text search against the text index for this collection.

Parameters:

• query (str) – The text query. Will be stemmed internally.
• offset (int) – The offset into the results
• limit (int) – Maximum number of documents to return
• sort (List of (key, order) tuples.) – The sort order.
• fields (str, list, set, or tuple) – A mask for filtering result documents by key, or None to return the full document, passed to MongoDB find() as the projection param. This is a string or iterable of strings to be included from the document, or dict for an inclusion or exclusion projection`.
• filters (dict) – Any additional query operators to apply.

Returns:

A pymongo cursor. It is left to the caller to build the results from the cursor.

update(query, update, multi=True)

This method should be used for updating multiple documents in the collection. This is useful for things like removing all references in this collection to a document that is being deleted from another collection.

For updating a single document, use the save() model method instead.

Parameters:

• query (dict) – The search query for documents to update, see general MongoDB docs for “find()”
• update (dict) – The update specifier.
• multi (bool) – Whether to update a single document, or all matching documents.

Returns:

A pymongo UpdateResult object.

validate(doc)

Models should implement this to validate the document before it enters the database. It must return the document with any necessary filters applied, or throw a ValidationException if validation of the document fails.

Parameters:doc (dict) – The document to validate before saving to the collection.

validateKeys(keys)

Validate a set of keys to make sure they are able to be used in the database. This enforces MongoDB rules about key names. @TODO Add recurse=True argument if keys is a dict.

Parameters:keys (iterable) – An iterable of keys to validate. Raises:ValidationException

API Key

class girder.models.api_key.ApiKey

This model represents API keys corresponding to users.

createApiKey(user, name, scope=None, days=None, active=True)

Create a new API key for a user.

Parameters:

• user (dict) – The user who owns the API key.
• name – A human readable name for the API key
• days (float or int) – The lifespan of the session in days. If not passed, uses the database setting for cookie lifetime. Note that this is a maximum duration; clients may request tokens with a shorter lifetime than this value.
• scope (str, list of str, or set of str) – Scope or list of scopes this API key grants. By default, will grant tokens provided full access on behalf of the user.
• active – Whether this key is active.

Returns:

The API key document that was created.

createToken(key, days=None)

Create a token using an API key.

Parameters:

• key (str) – The API key (the key itself, not the full document).
• days (float or None) – You may request a token duration up to the token duration of the API key itself, or pass None to use the API key duration.

initialize()

Subclasses should override this and set the name of the collection as self.name. Also, they should set any indexed fields that they require.

list(user, limit=0, offset=0, sort=None)

List API keys for a given user.

Parameters:

• user (dict) – The user whose keys to list.
• limit – Result limit.
• offset – Result offset.
• sort – The sort structure to pass to pymongo.

Return type:

iterable of API keys for the user.

remove(doc)

Delete an object from the collection; must have its _id set.

Parameters:document – the item to remove.

validate(doc)

Models should implement this to validate the document before it enters the database. It must return the document with any necessary filters applied, or throw a ValidationException if validation of the document fails.

Parameters:doc (dict) – The document to validate before saving to the collection.

User

class girder.models.user.User

This model represents the users of the system.

adminApprovalRequired(user)

Returns True if the registration policy requires admin approval and this user is pending approval.

authenticate(login, password, otpToken=None)

Validate a user login via username and password. If authentication fails, a AccessException is raised.

Parameters:

• login (str) – The user’s login or email.
• password (str) – The user’s password.
• otpToken (str or bool or None) – A one-time password for the user. If “True”, then the one-time password (if required) is assumed to be concatenated to the password.

Returns:

The corresponding user if the login was successful.

Return type:

dict

canLogin(user)

Returns True if the user is allowed to login, e.g. email verification is not needed and admin approval is not needed.

countFolders(user, filterUser=None, level=None)

Returns the number of top level folders under this user. Access checking is optional; to circumvent access checks, pass level=None.

Parameters:

• user – The user whose top level folders to count.
• filterUser (dict or None) – If performing access checks, the user to check against.
• level – The required access level, or None to return the raw top-level folder count.

createUser(login, password, firstName, lastName, email, admin=False, public=True)

Create a new user with the given information. The user will be created with the default “Public” and “Private” folders.

Parameters:

• admin (bool) – Whether user is global administrator.
• public (bool) – Whether user is publicly visible.

Returns:

The user document that was created.

emailVerificationRequired(user)

Returns True if email verification is required and this user has not yet verified their email address.

fileList(doc, user=None, path='', includeMetadata=False, subpath=True, data=True)

This function generates a list of 2-tuples whose first element is the relative path to the file from the user’s folders root and whose second element depends on the value of the data flag. If data=True, the second element will be a generator that will generate the bytes of the file data as stored in the assetstore. If data=False, the second element is the file document itself.

Parameters:

• doc – the user to list.
• user – a user used to validate data that is returned.
• path – a path prefix to add to the results.
• includeMetadata – if True and there is any metadata, include a result which is the JSON string of the metadata. This is given a name of metadata[-(number).json that is distinct from any file within the item.
• subpath – if True, add the user’s name to the path.
• data (bool) – If True return raw content of each file as stored in the assetstore, otherwise return file document.

filter(doc, user, additionalKeys=None)

Filter this model for the given user according to the user’s access level. Also adds the special _accessLevel field to the document to indicate the user’s highest access level. This filters a single document that the user has at least read access to. For filtering a set of documents, see filterResultsByPermission().

Parameters:

• doc (dict or None) – The document of this model type to be filtered.
• user (dict or None) – The current user for whom we are filtering.
• additionalKeys (list, tuple, or None) – Any additional keys that should be included in the document for this call only.

Returns:

The filtered document (dict).

getAdmins()

Helper to return a cursor of all site-admin users. The number of site admins is assumed to be small enough that we will not need to page the results for now.

hasPassword(user)

Returns whether or not the given user has a password stored in the database. If not, it is expected that the user will be authenticated by an external service.

Parameters:user (dict) – The user to test. Returns:bool

initialize()

Subclasses should override this and set the name of the collection as self.name. Also, they should set any indexed fields that they require.

initializeOtp(user)

Initialize the use of one-time passwords with this user.

This does not save the modified user model.

Parameters:user – The user to modify. Returns:The new OTP keys, each in KeyUriFormat. Return type:dict

remove(user, progress=None, **kwargs)

Delete a user, and all references to it in the database.

Parameters:

• user (dict) – The user document to delete.
• progress (girder.utility.progress.ProgressContext or None.) – A progress context to record progress on.

search(text=None, user=None, limit=0, offset=0, sort=None)

List all users. Since users are access-controlled, this will filter them by access policy.

Parameters:

• text – Pass this to perform a full-text search for users.
• user – The user running the query. Only returns users that this user can see.
• limit – Result limit.
• offset – Result offset.
• sort – The sort structure to pass to pymongo.

Returns:

Iterable of users.

setPassword(user, password, save=True)

Change a user’s password.

Parameters:

• user – The user whose password to change.
• password – The new password. If set to None, no password will be stored for this user. This should be done in cases where an external system is responsible for authenticating the user.

subtreeCount(doc, includeItems=True, user=None, level=None)

Return the size of the user’s folders. The user is counted as well.

Parameters:

• doc – The user.
• includeItems (bool) – Whether to include items in the subtree count, or just folders.
• user – If filtering by permission, the user to filter against.
• level (AccessLevel) – If filtering by permission, the required permission level.

updateSize(doc)

Recursively recomputes the size of this user and its underlying folders and fixes the sizes as needed.

Parameters:doc (dict) – The user.

validate(doc)

Validate the user every time it is stored in the database.

Token

class girder.models.token.Token

This model stores session tokens for user authentication.

addScope(token, scope)

Add a scope to this token. If the token already has the scope, this is a no-op.

clearForApiKey(apiKey)

Delete all tokens corresponding to an API key.

createToken(user=None, days=None, scope=None, apiKey=None)

Creates a new token. You can create an anonymous token (such as for CSRF mitigation) by passing “None” for the user argument.

Parameters:

• user (dict) – The user to create the session for.
• days (float or int) – The lifespan of the session in days. If not passed, uses the database setting for cookie lifetime.
• scope (str or list of str) – Scope or list of scopes this token applies to. By default, will create a user authentication token.
• apiKey (dict) – If this token is being created via an API key, pass it so that we can record the provenance for cleanup and auditing.

Returns:

The token document that was created.

getAllowedScopes(token)

Return the list of allowed scopes for a given token.

hasScope(token, scope)

Test whether the given token has the given set of scopes. Use this rather than comparing manually, since this method is backward compatible with tokens that do not contain a scope field.

Parameters:

• token (dict) – The token object.
• scope (str or list of str) – A scope or set of scopes that will be tested as a subset of the given token’s allowed scopes.

initialize()

Subclasses should override this and set the name of the collection as self.name. Also, they should set any indexed fields that they require.

requireScope(token, scope)

Raise an error if given set of scopes are not included.

Parameters:

• token (dict) – The token object.
• scope (str or list of str) – A scope or set of scopes that will be tested as a subset of the given token’s allowed scopes.

validate(doc)

Models should implement this to validate the document before it enters the database. It must return the document with any necessary filters applied, or throw a ValidationException if validation of the document fails.

Parameters:doc (dict) – The document to validate before saving to the collection.

Group

class girder.models.group.Group

Groups are simply groups of users. The primary use of grouping users is to simplify access control for resources in the system, but they can be used for other purposes that require groupings of users as well.

Group membership is stored in the database on the user document only; there is no “users” field in this model. This is to optimize for the most common use case for querying membership, which involves checking access control policies, which is always done relative to a specific user. The task of querying all members within a group is much less common and typically only performed on a single group at a time, so doing a find on the indexed group list in the user collection is sufficiently fast.

Users with READ access on the group can see the group and its members. Users with WRITE access on the group can add and remove members and change the name or description. Users with ADMIN access can promote group members to grant them WRITE or ADMIN access, and can also delete the entire group.

This model uses a custom implementation of the access control methods, because it uses only a subset of its capabilities and provides a more optimized implementation for that subset. Specifically: read access is implied by membership in the group or having an invitation to join the group, so we don’t store read access in the access document as normal. Another constraint is that write and admin access on the group can only be granted to members of the group. Also, group permissions are not allowed on groups for the sake of simplicity.

addUser(group, user, level=0)

Add the user to the group. Records membership in the group in the user document, and also grants the specified access level on the group itself to the user. Any group member has at least read access on the group. If the user already belongs to the group, this method can be used to change their access level within it.

createGroup(name, creator, description='', public=True)

Create a new group. The creator will be given admin access to it.

Parameters:

• name (str) – The name of the folder.
• description (str) – Description for the folder.
• public (bool) – Whether the group is publicly visible.
• creator (dict) – User document representing the creator of the group.

Returns:

The group document that was created.

getAccessLevel(doc, user)

Return the maximum access level for a given user on the group.

Parameters:

• doc – The group to check access on.
• user – The user to get the access level for.

Returns:

The max AccessType available for the user on the object.

getFullRequestList(group)

Return the set of all outstanding requests, filled in with the login and full names of the corresponding users.

Parameters:group (dict) – The group to get requests for.

getInvites(group, limit=0, offset=0, sort=None)

Return a page of outstanding invitations to a group. This is simply a list of users invited to the group currently.

Parameters:

• group – The group to find invitations for.
• limit – Result set size limit.
• offset – Offset into the results.
• sort – The sort field.

getMembers(group, offset=0, limit=0, sort=None)

Return the list of all users who belong to this group.

Parameters:

• group – The group to list members on.
• offset – Offset into the result set of users.
• limit – Result set size limit.
• sort – Sort parameter for the find query.

Returns:

List of user documents.

hasAccess(doc, user=None, level=0)

This overrides the default AccessControlledModel behavior for checking access to perform an optimized subset of the access control behavior.

Parameters:

• doc (dict) – The group to check permission on.
• user (dict) – The user to check against.
• level (AccessType) – The access level.

Returns:

Whether the access is granted.

initialize()

Subclasses should override this and set the name of the collection as self.name. Also, they should set any indexed fields that they require.

inviteUser(group, user, level=0)

Invite a user to join the group. Inviting them automatically grants the user read access to the group so that they can see it. Once they accept the invitation, they will be given the specified level of access.

If the user has requested an invitation to this group, calling this will accept their request and add them to the group at the access level specified.

joinGroup(group, user)

This method either accepts an invitation to join a group, or if the given user has not been invited to the group, this will create an invitation request that moderators and admins may grant or deny later.

listMembers(group, offset=0, limit=0, sort=None)

List members of the group.

remove(group, **kwargs)

Delete a group, and all references to it in the database.

Parameters:group (dict) – The group document to delete.

removeUser(group, user)

Remove the user from the group. If the user is not in the group but has an outstanding invitation to the group, the invitation will be revoked. If the user has requested an invitation, calling this will deny that request, thereby deleting it.

setGroupAccess(doc, group, level, save=False)

Set group-level access on the resource.

Parameters:

• doc (dict) – The resource document to set access on.
• group (dict) – The group to grant or remove access to.
• level (AccessType or None) – What level of access the group should have. Set to None to remove all access for this group.
• save (bool) – Whether to save the object to the database afterward. Set this to False if you want to wait to save the document for performance reasons.
• flags (specific flag identifier, or a list/tuple/set of them) – List of access flags to grant to the group.
• currentUser (dict or None) – The user performing this action. Only required if attempting to set admin-only flags on the resource.
• force (bool) – Set this to True to set the flags regardless of the passed in currentUser’s permissions (only matters if flags are passed).

Returns:

The updated resource document.

setUserAccess(doc, user, level, save=False)

This override is used because we only need to augment the access field in the case of WRITE access and above since READ access is implied by membership or invitation.

updateGroup(group)

Updates a group.

Parameters:group (dict) – The group document to update Returns:The group document that was edited.

validate(doc)

Models should implement this to validate the document before it enters the database. It must return the document with any necessary filters applied, or throw a ValidationException if validation of the document fails.

Parameters:doc (dict) – The document to validate before saving to the collection.

Collection

class girder.models.collection.Collection

Collections are the top level roots of the data hierarchy. They are used to group and organize data that is meant to be shared amongst users.

countFolders(collection, user=None, level=None)

Returns the number of top level folders under this collection. Access checking is optional; to circumvent access checks, pass level=None.

Parameters:

• collection (dict) – The collection.
• user (dict or None) – If performing access checks, the user to check against.
• level – The required access level, or None to return the raw top-level folder count.

createCollection(name, creator=None, description='', public=True, reuseExisting=False)

Create a new collection.

Parameters:

• name (str) – The name of the collection. Must be unique.
• description (str) – Description for the collection.
• public (bool) – Public read access flag.
• creator (dict) – The user who is creating this collection.
• reuseExisting (bool) – If a collection with the given name already exists return that collection rather than creating a new one.

Returns:

The collection document that was created.

deleteMetadata(collection, fields)

Delete metadata on an collection. A ValidationException is thrown if the metadata field names contain a period (‘.’) or begin with a dollar sign (‘$’).

Parameters:

• collection (dict) – The collection to delete metadata from.
• fields – An array containing the field names to delete from the collection’s meta field

Returns:

the collection document

fileList(doc, user=None, path='', includeMetadata=False, subpath=True, mimeFilter=None, data=True)

This function generates a list of 2-tuples whose first element is the relative path to the file from the collection’s root and whose second element depends on the value of the data flag. If data=True, the second element will be a generator that will generate the bytes of the file data as stored in the assetstore. If data=False, the second element is the file document itself.

Parameters:

• doc – the collection to list.
• user – a user used to validate data that is returned.
• path – a path prefix to add to the results.
• includeMetadata – if True and there is any metadata, include a result which is the JSON string of the metadata. This is given a name of metadata[-(number).json that is distinct from any file within the item.
• subpath – if True, add the collection’s name to the path.
• mimeFilter (list or tuple) – Optional list of MIME types to filter by. Set to None to include all files.
• data (bool) – If True return raw content of each file as stored in the assetstore, otherwise return file document.

filter(doc, user=None, additionalKeys=None)

Overrides the parent filter method to add an empty meta field (if it doesn’t exist) to the returned collection.

hasCreatePrivilege(user)

Tests whether a given user has the authority to create collections on this instance. This is based on the collection creation policy settings. By default, only admins are allowed to create collections.

Parameters:user – The user to test. Returns:bool

initialize()

Subclasses should override this and set the name of the collection as self.name. Also, they should set any indexed fields that they require.

load(id, level=2, user=None, objectId=True, force=False, fields=None, exc=False)

Calls AccessControlMixin.load, and if no meta field is present, adds an empty meta field and saves.

Takes the same parameters as girder.models.model_base.AccessControlMixin.load().

remove(collection, progress=None, **kwargs)

Delete a collection recursively.

Parameters:

• collection (dict) – The collection document to delete.
• progress (girder.utility.progress.ProgressContext or None.) – A progress context to record progress on.

setAccessList(doc, access, save=False, recurse=False, user=None, progress=<girder.utility.progress.ProgressContext object>, setPublic=None, publicFlags=None, force=False)

Overrides AccessControlledModel.setAccessList to add a recursive option. When recurse=True, this will set the access list on all subfolders to which the given user has ADMIN access level. Any subfolders that the given user does not have ADMIN access on will be skipped.

Parameters:

• doc (collection) – The collection to set access settings on.
• access (dict) – The access control list.
• save (bool) – Whether the changes should be saved to the database.
• recurse (bool) – Whether this access list should be propagated to all folders underneath this collection.
• user – The current user (for recursive mode filtering).
• progress (girder.utility.progress.ProgressContext) – Progress context to update.
• setPublic (bool or None) – Pass this if you wish to set the public flag on the resources being updated.
• publicFlags (flag identifier str, or list/set/tuple of them, or None) – Pass this if you wish to set the public flag list on resources being updated.
• force (bool) – Set this to True to set the flags regardless of the passed in user’s permissions.

setMetadata(collection, metadata, allowNull=False)

Set metadata on an collection. A ValidationException is thrown in the cases where the metadata JSON object is badly formed, or if any of the metadata keys contains a period (‘.’).

Parameters:

• collection (dict) – The collection to set the metadata on.
• metadata (dict) – A dictionary containing key-value pairs to add to the collection’s meta field
• allowNull – Whether to allow null values to be set in the collection’s metadata. If set to False or omitted, a null value will cause that metadata field to be deleted.

Returns:

the collection document

subtreeCount(doc, includeItems=True, user=None, level=None)

Return the size of the folders within the collection. The collection is counted as well.

Parameters:

• doc – The collection.
• includeItems (bool) – Whether items should be included in the count.
• user – If filtering by permission, the user to filter against.
• level (AccessLevel) – If filtering by permission, the required permission level.

updateCollection(collection)

Updates a collection.

Parameters:collection (dict) – The collection document to update Returns:The collection document that was edited.

updateSize(doc)

Recursively recomputes the size of this collection and its underlying folders and fixes the sizes as needed.

Parameters:doc (dict) – The collection.

validate(doc)

Models should implement this to validate the document before it enters the database. It must return the document with any necessary filters applied, or throw a ValidationException if validation of the document fails.

Parameters:doc (dict) – The document to validate before saving to the collection.

Folder

class girder.models.folder.Folder

Folders are used to store items and can also store other folders in a hierarchical way, like a directory on a filesystem. Every folder has its own set of access control policies, but by default the access control list is inherited from the folder’s parent folder, if it has one. Top-level folders are ones whose parent is a user or a collection.

childFolders(parent, parentType, user=None, limit=0, offset=0, sort=None, filters=None, **kwargs)

This generator will yield child folders of a user, collection, or folder, with access policy filtering. Passes any kwargs to the find function.

Parameters:

• parent – The parent object.
• parentType ('user', 'folder', or 'collection') – The parent type.
• user – The user running the query. Only returns folders that this user can see.
• limit – Result limit.
• offset – Result offset.
• sort – The sort structure to pass to pymongo.
• filters – Additional query operators.

childItems(folder, limit=0, offset=0, sort=None, filters=None, **kwargs)

Generator function that yields child items in a folder. Passes any kwargs to the find function.

Parameters:

• folder – The parent folder.
• limit – Result limit.
• offset – Result offset.
• sort – The sort structure to pass to pymongo.
• filters – Additional query operators.

clean(folder, progress=None, **kwargs)

Delete all contents underneath a folder recursively, but leave the folder itself.

Parameters:

• folder (dict) – The folder document to delete.
• progress (girder.utility.progress.ProgressContext or None.) – A progress context to record progress on.

copyFolder(srcFolder, parent=None, name=None, description=None, parentType=None, public=None, creator=None, progress=None, firstFolder=None)

Copy a folder, including all child items and child folders.

Parameters:

• srcFolder (dict) – the folder to copy.
• parent (dict) – The parent document. Must be a folder, user, or collection.
• name (str) – The name of the new folder. None to copy the original name.
• description (str) – Description for the new folder. None to copy the original description.
• parentType (str) – What type the parent is: (‘folder’ | ‘user’ | ‘collection’)
• public (bool, None, or 'original'.) – Public read access flag. None to inherit from parent, ‘original’ to inherit from original folder.
• creator (dict) – user representing the creator of the new folder.
• progress (girder.utility.progress.ProgressContext or None.) – a progress context to record process on.
• firstFolder – if not None, the first folder copied in a tree of folders.

Returns:

the new folder document.

copyFolderComponents(srcFolder, newFolder, creator, progress, firstFolder=None)

Copy the items, subfolders, and extended data of a folder that was just copied.

Parameters:

• srcFolder (dict) – the original folder.
• newFolder (dict) – the new folder.
• creator (dict) – user representing the creator of the new folder.
• progress (girder.utility.progress.ProgressContext or None.) – a progress context to record process on.
• firstFolder – if not None, the first folder copied in a tree of folders.

Returns:

the new folder document.

countFolders(folder, user=None, level=None)

Returns the number of subfolders within the given folder. Access checking is optional; to circumvent access checks, pass level=None.

Parameters:

• folder (dict) – The parent folder.
• user (dict or None) – If performing access checks, the user to check against.
• level – The required access level, or None to return the raw subfolder count.

countItems(folder)

Returns the number of items within the given folder.

createFolder(parent, name, description='', parentType='folder', public=None, creator=None, allowRename=False, reuseExisting=False)

Create a new folder under the given parent.

Parameters:

• parent (dict) – The parent document. Should be a folder, user, or collection.
• name (str) – The name of the folder.
• description (str) – Description for the folder.
• parentType (str) – What type the parent is: (‘folder’ | ‘user’ | ‘collection’)
• public (bool or None to inherit from parent) – Public read access flag.
• creator (dict) – User document representing the creator of this folder.
• allowRename (bool) – if True and a folder or item of this name exists, automatically rename the folder.
• reuseExisting (bool) – If a folder with the given name already exists under the given parent, return that folder rather than creating a new one.

Returns:

The folder document that was created.

deleteMetadata(folder, fields)

Delete metadata on a folder. A ValidationException is thrown if the metadata field names contain a period (‘.’) or begin with a dollar sign (‘$’).

Parameters:

• folder (dict) – The folder to delete metadata from.
• fields – An array containing the field names to delete from the folder’s meta field

Returns:

the folder document

fileList(doc, user=None, path='', includeMetadata=False, subpath=True, mimeFilter=None, data=True)

This function generates a list of 2-tuples whose first element is the relative path to the file from the folder’s root and whose second element depends on the value of the data flag. If data=True, the second element will be a generator that will generate the bytes of the file data as stored in the assetstore. If data=False, the second element is the file document itself.

Parameters:

• doc – The folder to list.
• user – The user used for access.
• path (str) – A path prefix to add to the results.
• includeMetadata (bool) – if True and there is any metadata, include a result which is the JSON string of the metadata. This is given a name of metadata[-(number).json that is distinct from any file within the folder.
• subpath (bool) – if True, add the folder’s name to the path.
• mimeFilter (list or tuple) – Optional list of MIME types to filter by. Set to None to include all files.
• data (bool) – If True return raw content of each file as stored in the assetstore, otherwise return file document.

Returns:

Iterable over files in this folder, where each element is a tuple of (path name of the file, stream function with file data or file object).

Return type:

generator(str, func)

filter(doc, user=None, additionalKeys=None)

Overrides the parent filter method to add an empty meta field (if it doesn’t exist) to the returned folder.

getSizeRecursive(folder)

Calculate the total size of the folder by recursing into all of its descendant folders.

initialize()

Subclasses should override this and set the name of the collection as self.name. Also, they should set any indexed fields that they require.

isOrphan(folder)

Returns True if this folder is orphaned (its parent is missing).

Parameters:folder (dict) – The folder to check.

load(id, level=2, user=None, objectId=True, force=False, fields=None, exc=False)

We override load in order to ensure the folder has certain fields within it, and if not, we add them lazily at read time.

Parameters:

• id (string or ObjectId) – The id of the resource.
• user (dict or None) – The user to check access against.
• level (AccessType) – The required access type for the object.
• force (bool) – If you explicitly want to circumvent access checking on this resource, set this to True.

move(folder, parent, parentType)

Move the given folder from its current parent to another parent object. Raises an exception if folder is an ancestor of parent.

Parameters:

• folder (dict) – The folder to move.
• parent – The new parent object.
• parentType (str) – The type of the new parent object (user, collection, or folder).

parentsToRoot(folder, curPath=None, user=None, force=False, level=0)

Get the path to traverse to a root of the hierarchy.

Parameters:folder (dict) – The folder whose root to find Returns:an ordered list of dictionaries from root to the current folder

remove(folder, progress=None, **kwargs)

Delete a folder recursively.

Parameters:

• folder (dict) – The folder document to delete.
• progress (girder.utility.progress.ProgressContext or None.) – A progress context to record progress on.

setAccessList(doc, access, save=False, recurse=False, user=None, progress=<girder.utility.progress.ProgressContext object>, setPublic=None, publicFlags=None, force=False)

Overrides AccessControlledModel.setAccessList to add a recursive option. When recurse=True, this will set the access list on all subfolders to which the given user has ADMIN access level. Any subfolders that the given user does not have ADMIN access on will be skipped.

Parameters:

• doc (girder.models.folder) – The folder to set access settings on.
• access (dict) – The access control list.
• save (bool) – Whether the changes should be saved to the database.
• recurse (bool) – Whether this access list should be propagated to all subfolders underneath this folder.
• user – The current user (for recursive mode filtering).
• progress (girder.utility.progress.ProgressContext) – Progress context to update.
• setPublic (bool or None) – Pass this if you wish to set the public flag on the resources being updated.
• publicFlags (flag identifier str, or list/set/tuple of them, or None) – Pass this if you wish to set the public flag list on resources being updated.
• force (bool) – Set this to True to set the flags regardless of the passed in user’s permissions.

setMetadata(folder, metadata, allowNull=False)

Set metadata on a folder. A ValidationException is thrown in the cases where the metadata JSON object is badly formed, or if any of the metadata keys contains a period (‘.’).

Parameters:

• folder (dict) – The folder to set the metadata on.
• metadata (dict) – A dictionary containing key-value pairs to add to the folder’s meta field
• allowNull – Whether to allow null values to be set in the item’s metadata. If set to False or omitted, a null value will cause that metadata field to be deleted.

Returns:

the folder document

subtreeCount(folder, includeItems=True, user=None, level=None)

Return the size of the subtree rooted at the given folder. Includes the root folder in the count.

Parameters:

• folder (dict) – The root of the subtree.
• includeItems (bool) – Whether to include items in the subtree count, or just folders.
• user – If filtering by permission, the user to filter against.
• level (AccessLevel) – If filtering by permission, the required permission level.

updateFolder(folder)

Updates a folder.

Parameters:folder (dict) – The folder document to update Returns:The folder document that was edited.

updateSize(doc)

Recursively recomputes the size of this folder and its underlying folders and fixes the sizes as needed.

Parameters:doc (dict) – The folder.

validate(doc, allowRename=False)

Validate the name and description of the folder, ensure that it is associated with a valid parent and that it has a unique name.

Parameters:

• doc – the folder document to validate.
• allowRename – if True and a folder or item exists with the same name, rename the folder so that it is unique.

Returns:

the validated folder document

Item

class girder.models.item.Item

Items are leaves in the data hierarchy. They can contain 0 or more files within them, and can also contain arbitrary metadata.

childFiles(item, limit=0, offset=0, sort=None, **kwargs)

Returns child files of the item. Passes any kwargs to the find function.

Parameters:

• item – The parent item.
• limit – Result limit.
• offset – Result offset.
• sort – The sort structure to pass to pymongo.

copyItem(srcItem, creator, name=None, folder=None, description=None)

Copy an item, including duplicating files and metadata.

Parameters:

• srcItem (dict) – the item to copy.
• creator – the user who will own the copied item.
• name (str) – The name of the new item. None to copy the original name.
• folder – The parent folder of the new item. None to store in the same folder as the original item.
• description (str) – Description for the new item. None to copy the original description.

Returns:

the new item.

createItem(name, creator, folder, description='', reuseExisting=False)

Create a new item. The creator will be given admin access to it.

Parameters:

• name (str) – The name of the item.
• description (str) – Description for the item.
• folder – The parent folder of the item.
• creator (dict) – User document representing the creator of the item.
• reuseExisting (bool) – If an item with the given name already exists under the given folder, return that item rather than creating a new one.

Returns:

The item document that was created.

deleteMetadata(item, fields)

Delete metadata on an item. A ValidationException is thrown if the metadata field names contain a period (‘.’) or begin with a dollar sign (‘$’).

Parameters:

• item (dict) – The item to delete metadata from.
• fields – An array containing the field names to delete from the item’s meta field

Returns:

the item document

fileList(doc, user=None, path='', includeMetadata=False, subpath=True, mimeFilter=None, data=True)

This function generates a list of 2-tuples whose first element is the relative path to the file from the item’s root and whose second element depends on the value of the data flag. If data=True, the second element will be a generator that will generate the bytes of the file data as stored in the assetstore. If data=False, the second element will be the file document itself.

Parameters:

• doc – The item to list.
• user – A user used to validate data that is returned. This isn’t used, but is present to be consistent across all model implementations of fileList.
• path (str) – A path prefix to add to the results.
• includeMetadata (bool) – If True and there is any metadata, include a result which is the JSON string of the metadata. This is given a name of metadata[-(number).json that is distinct from any file within the item.
• subpath (bool) – If True and the item has more than one file, any metadata, or the sole file is not named the same as the item, then the returned paths include the item name.
• mimeFilter (list or tuple) – Optional list of MIME types to filter by. Set to None to include all files.
• data (bool) – If True return raw content of each file as stored in the assetstore, otherwise return file document.

Returns:

Iterable over files in this item, where each element is a tuple of (path name of the file, stream function with file data or file object).

Return type:

generator(str, func)

filter(doc, user=None, additionalKeys=None)

Overrides the parent filter method to add an empty meta field (if it doesn’t exist) to the returned folder.

initialize()

Subclasses should override this and set the name of the collection as self.name. Also, they should set any indexed fields that they require.

isOrphan(item)

Returns True if this item is orphaned (its folder is missing).

Parameters:item (dict) – The item to check.

load(id, level=2, user=None, objectId=True, force=False, fields=None, exc=False)

Calls AccessControlMixin.load while doing some auto-correction.

Takes the same parameters as girder.models.model_base.AccessControlMixin.load().

move(item, folder)

Move the given item from its current folder into another folder.

Parameters:

• item (dict) – The item to move.
• folder (dict.) – The folder to move the item into.

parentsToRoot(item, user=None, force=False)

Get the path to traverse to a root of the hierarchy.

Parameters:

• item (dict) – The item whose root to find
• user (dict or None) – The user making the request (not required if force=True).
• force (bool) – Set to True to skip permission checking. If False, the returned models will be filtered.

Returns:

an ordered list of dictionaries from root to the current item

recalculateSize(item)

Recalculate the item size based on the files that are in it. If this is different than the recorded size, propagate the changes. :param item: The item to recalculate the size of. :returns: the recalculated size in bytes

remove(item, **kwargs)

Delete an item, and all references to it in the database.

Parameters:item (dict) – The item document to delete.

setMetadata(item, metadata, allowNull=False)

Set metadata on an item. A ValidationException is thrown in the cases where the metadata JSON object is badly formed, or if any of the metadata keys contains a period (‘.’).

Parameters:

• item (dict) – The item to set the metadata on.
• metadata (dict) – A dictionary containing key-value pairs to add to the items meta field
• allowNull – Whether to allow null values to be set in the item’s metadata. If set to False or omitted, a null value will cause that metadata field to be deleted.

Returns:

the item document

updateItem(item)

Updates an item.

Parameters:item (dict) – The item document to update Returns:The item document that was edited.

updateSize(doc)

Recomputes the size of this item and its underlying files and fixes the sizes as needed.

Parameters:doc (dict) – The item.

validate(doc)

Models should implement this to validate the document before it enters the database. It must return the document with any necessary filters applied, or throw a ValidationException if validation of the document fails.

Parameters:doc (dict) – The document to validate before saving to the collection.

Setting

class girder.models.setting.Setting

This model represents server-wide configuration settings as key/value pairs.

get(key)

Retrieve a setting by its key.

Parameters:key (str) – The key identifying the setting.

getDefault(key)

Retrieve the system default for a value.

Parameters:key (str) – The key identifying the setting. Returns:The default value if the key is present in both SettingKey and referenced in SettingDefault; otherwise None.

initialize()

Subclasses should override this and set the name of the collection as self.name. Also, they should set any indexed fields that they require.

reconnect()

Reconnect to the database and rebuild indices if necessary. If a unique index on key does not exist, make one, first discarding any extant index on key and removing duplicate keys if necessary.

set(key, value)

Save a setting. If a setting for this key already exists, this will replace the existing value.

Parameters:

• key (str) – The key identifying the setting.
• value – The object to store for this setting.

Returns:

The document representing the saved Setting.

unset(key)

Remove the setting for this key. If no such setting exists, this is a no-op.

Parameters:key (str) – The key identifying the setting to be removed.

validate(doc)

This method is in charge of validating that the setting key is a valid key, and that for that key, the provided value is valid. It first allows plugins to validate the setting, but if none of them can, it assumes it is a core setting and does the validation here.

Assetstore

class girder.models.assetstore.Assetstore

This model represents an assetstore, an abstract repository of Files.

addComputedInfo(assetstore)

Add all runtime-computed properties about an assetstore to its document.

Parameters:assetstore (dict) – The assetstore object.

getCurrent()

Returns the current assetstore. If none exists, this will raise a 500 exception.

importData(assetstore, parent, parentType, params, progress, user, **kwargs)

Calls the importData method of the underlying assetstore adapter.

initialize()

Subclasses should override this and set the name of the collection as self.name. Also, they should set any indexed fields that they require.

list(limit=0, offset=0, sort=None)

List all assetstores.

Parameters:

• limit – Result limit.
• offset – Result offset.
• sort – The sort structure to pass to pymongo.

Returns:

List of users.

remove(assetstore, **kwargs)

Delete an assetstore. If there are any files within this assetstore, a validation exception is raised.

Parameters:assetstore (dict) – The assetstore document to delete.

validate(doc)

Models should implement this to validate the document before it enters the database. It must return the document with any necessary filters applied, or throw a ValidationException if validation of the document fails.

Parameters:doc (dict) – The document to validate before saving to the collection.

File

class girder.models.file.File

This model represents a File, which is stored in an assetstore.

copyFile(srcFile, creator, item=None)

Copy a file so that we don’t need to duplicate stored data.

Parameters:

• srcFile (dict) – The file to copy.
• creator – The user copying the file.
• item – a new item to assign this file to (optional)

Returns:

a dict with the new file.

createFile(creator, item, name, size, assetstore, mimeType=None, saveFile=True, reuseExisting=False, assetstoreType=None)

Create a new file record in the database.

Parameters:

• item – The parent item.
• creator – The user creating the file.
• assetstore – The assetstore this file is stored in.
• name (str) – The filename.
• size (int) – The size of the file in bytes.
• mimeType (str) – The mimeType of the file.
• saveFile (bool) – if False, don’t save the file, just return it.
• reuseExisting (bool) – If a file with the same name already exists in this location, return it rather than creating a new file.
• assetstoreType (str or tuple) – If a model other than assetstore will be used to initialize the assetstore adapter for this file, use this parameter to specify it. If it’s a core model, pass its string name. If it’s a plugin model, use a 2-tuple of the form (modelName, pluginName).

createLinkFile(name, parent, parentType, url, creator, size=None, mimeType=None, reuseExisting=False)

Create a file that is a link to a URL, rather than something we maintain in an assetstore.

Parameters:

• name (str) – The local name for the file.
• parent (girder.models.folder or girder.models.item) – The parent object for this file.
• parentType (str) – The parent type (folder or item)
• url – The URL that this file points to
• creator (dict) – The user creating the file.
• size (int) – The size of the file in bytes. (optional)
• mimeType (str) – The mimeType of the file. (optional)
• reuseExisting (bool) – If a file with the same name already exists in this location, return it rather than creating a new file.

download(file, offset=0, headers=True, endByte=None, contentDisposition=None, extraParameters=None)

Use the appropriate assetstore adapter for whatever assetstore the file is stored in, and call downloadFile on it. If the file is a link file rather than a file in an assetstore, we redirect to it.

Parameters:

• file – The file to download.
• offset (int) – The start byte within the file.
• headers (bool) – Whether to set headers (i.e. is this an HTTP request for a single file, or something else).
• endByte (int or None) – Final byte to download. If None, downloads to the end of the file.
• contentDisposition (str or None) – Content-Disposition response header disposition-type value.

getAssetstoreAdapter(file)

Return the assetstore adapter for the given file. Return None if the file has no assetstore.

getGirderMountFilePath(file, validate=True)

If possible, get the path of the file on a local girder mount.

Parameters:

• file – The file document.
• validate – if True, check if the path exists and raise an exception if it does not.

Returns:

a girder mount path to the file or None if no such path is available.

getLocalFilePath(file)

If an assetstore adapter supports it, return a path to the file on the local file system.

Parameters:file – The file document. Returns:a local path to the file or None if no such path is known.

initialize()

Subclasses should override this and set the name of the collection as self.name. Also, they should set any indexed fields that they require.

isOrphan(file)

Returns True if this file is orphaned (its item or attached entity is missing).

Parameters:file (dict) – The file to check.

open(file)

Use this to expose a Girder file as a python file-like object. At the moment, this is a read-only interface, the equivalent of opening a system file with 'rb' mode. This can also be used as a context manager, e.g.:

>>> with File().open(file) as fh:
>>>    while True:
>>>        chunk = fh.read(CHUNK_LEN)
>>>        if not chunk:
>>>            break

Using it this way will automatically close the file handle for you when the with block is left.

Parameters:file (dict) – A Girder file document. Returns:A file-like object containing the bytes of the file. Return type:girder.utility.abstract_assetstore_adapter.FileHandle

propagateSizeChange(item, sizeIncrement, updateItemSize=True)

Propagates a file size change (or file creation) to the necessary parents in the hierarchy. Internally, this records subtree size in the item, the parent folder, and the root node under which the item lives. Should be called anytime a new file is added, a file is deleted, or a file size changes.

Parameters:

• item (dict) – The parent item of the file.
• sizeIncrement (int) – The change in size to propagate.
• updateItemSize – Whether the item size should be updated. Set to False if you plan to delete the item immediately and don’t care to update its size.

remove(file, updateItemSize=True, **kwargs)

Use the appropriate assetstore adapter for whatever assetstore the file is stored in, and call deleteFile on it, then delete the file record from the database.

Parameters:

• file – The file document to remove.
• updateItemSize – Whether to update the item size. Only set this to False if you plan to delete the item and do not care about updating its size.

updateFile(file)

Call this when changing properties of an existing file, such as name or MIME type. This causes the updated stamp to change, and also alerts the underlying assetstore adapter that file information has changed.

updateSize(file)

Returns the size of this file. Does not currently check the underlying assetstore to verify the size.

Parameters:file (dict) – The file.

validate(doc)

Models should implement this to validate the document before it enters the database. It must return the document with any necessary filters applied, or throw a ValidationException if validation of the document fails.

Parameters:doc (dict) – The document to validate before saving to the collection.

Upload

class girder.models.upload.Upload

This model stores temporary records for uploads that have been approved but are not yet complete, so that they can be uploaded in chunks of arbitrary size. The chunks must be uploaded in order.

cancelUpload(upload)

Discard an upload that is in progress. This asks the assetstore to discard the data, then removes the item from the upload database.

Parameters:upload (dict) – The upload document to remove.

createUpload(user, name, parentType, parent, size, mimeType=None, reference=None, assetstore=None, attachParent=False, save=True)

Creates a new upload record, and creates its temporary file that the chunks will be written into. Chunks should then be sent in order using the _id of the upload document generated by this method.

Parameters:

• user (dict) – The user performing the upload.
• name (str) – The name of the file being uploaded.
• parentType (str ('folder' or 'item')) – The type of the parent being uploaded into.
• parent (dict.) – The document representing the parent.
• size (int) – Total size in bytes of the whole file.
• mimeType (str) – The mimeType of the file.
• reference (str) – An optional reference string that will be sent to the data.process event.
• assetstore – An optional assetstore to use to store the file. If unspecified, the current assetstore is used.
• attachParent (boolean) – if True, instead of creating an item within the parent or giving the file an itemId, set itemId to None and set attachedToType and attachedToId instead (using the values passed in parentType and parent). This is intended for files that shouldn’t appear as direct children of the parent, but are still associated with it.
• save (boolean) – if True, save the document after it is created.

Returns:

The upload document that was created.

createUploadToFile(file, user, size, reference=None, assetstore=None)

Creates a new upload record into a file that already exists. This should be used when updating the contents of a file. Deletes any previous file content from the assetstore it was in. This will upload into the current assetstore rather than assetstore the file was previously contained in.

Parameters:

• file – The file record to update.
• user – The user performing this upload.
• size – The size of the new file contents.
• reference (str) – An optional reference string that will be sent to the data.process event.
• assetstore – An optional assetstore to use to store the file. If unspecified, the current assetstore is used.

finalizeUpload(upload, assetstore=None)

This should only be called manually in the case of creating an empty file, i.e. one that has no chunks.

Parameters:

• upload (dict) – The upload document.
• assetstore (dict) – If known, the containing assetstore for the upload.

Returns:

The file object that was created.

getTargetAssetstore(modelType, resource, assetstore=None)

Get the assetstore for a particular target resource, i.e. where new data within the resource should be stored. In Girder core, this is always just the current assetstore, but plugins may override this behavior to allow for more granular assetstore selection.

Parameters:

• modelType – the type of the resource that will be stored.
• resource – the resource to be stored.
• assetstore – if specified, the preferred assetstore where the resource should be located. This may be overridden.

Returns:

the selected assetstore.

handleChunk(upload, chunk, filter=False, user=None)

When a chunk is uploaded, this should be called to process the chunk. If this is the final chunk of the upload, this method will finalize the upload automatically.

This method will return EITHER an upload or a file document. If this is the final chunk of the upload, the upload is finalized and the created file document is returned. Otherwise, it returns the upload document with the relevant fields modified.

Parameters:

• upload (dict) – The upload document to update.
• chunk (file) – The file object representing the chunk that was uploaded.
• filter (bool) – Whether the model should be filtered. Only affects behavior when returning a file model, not the upload model.
• user (dict or None) – The current user. Only affects behavior if filter=True.

initialize()

Subclasses should override this and set the name of the collection as self.name. Also, they should set any indexed fields that they require.

list(limit=0, offset=0, sort=None, filters=None)

Search for uploads or simply list all visible uploads.

Parameters:

• limit – Result set size limit.
• offset – Offset into the results.
• sort – The sort direction.
• filters – if not None, a dictionary that can contain ids that must match the uploads, plus an minimumAge value.

moveFileToAssetstore(file, user, assetstore, progress=<girder.utility.progress.ProgressContext object>)

Move a file from whatever assetstore it is located in to a different assetstore. This is done by downloading and re-uploading the file.

Parameters:

• file – the file to move.
• user – the user that is authorizing the move.
• assetstore – the destination assetstore.
• progress – optional progress context.

Returns:

the original file if it is not moved, or the newly ‘uploaded’ file if it is.

requestOffset(upload)

Requests the offset that should be used to resume uploading. This makes the request from the assetstore adapter.

untrackedUploads(action='list', assetstoreId=None)

List or discard any uploads that an assetstore knows about but that our database doesn’t have in it.

Parameters:

• action (str) – ‘delete’ to discard the untracked uploads, anything else to just return with a list of them.
• assetstoreId (str) – if present, only include untracked items from the specified assetstore.

Returns:

a list of items that were removed or could be removed.

uploadFromFile(obj, size, name, parentType=None, parent=None, user=None, mimeType=None, reference=None, assetstore=None, attachParent=False)

This method wraps the entire upload process into a single function to facilitate “internal” uploads from a file-like object. Example:

size = os.path.getsize(filename)

with open(filename, 'rb') as f:
    Upload().uploadFromFile(f, size, filename, 'item', parentItem, user)

Parameters:

• obj (file-like) – The object representing the content to upload.
• size (int) – The total size of the file.
• name (str) – The name of the file to create.
• parentType (str) – The type of the parent: “folder” or “item”.
• parent (dict) – The parent (item or folder) to upload into.
• user (dict) – The user who is creating the file.
• mimeType (str) – MIME type of the file.
• reference (str) – An optional reference string that will be sent to the data.process event.
• assetstore – An optional assetstore to use to store the file. If unspecified, the current assetstore is used.
• attachParent (boolean) – if True, instead of creating an item within the parent or giving the file an itemId, set itemId to None and set attachedToType and attachedToId instead (using the values passed in parentType and parent). This is intended for files that shouldn’t appear as direct children of the parent, but are still associated with it.

validate(doc)

Models should implement this to validate the document before it enters the database. It must return the document with any necessary filters applied, or throw a ValidationException if validation of the document fails.

Parameters:doc (dict) – The document to validate before saving to the collection.

Notification

class girder.models.notification.Notification

This model is used to represent a notification that should be streamed to a specific user in some way. Each notification contains a type field indicating what kind of notification it is, a userId field indicating which user the notification should be sent to, a data field representing the payload of the notification, a time field indicating the time at which the event happened, and an optional expires field indicating at what time the notification should be deleted from the database.

createNotification(type, data, user, expires=None, token=None)

Create a generic notification.

Parameters:

• type (str) – The notification type.
• data – The notification payload.
• user (dict) – User to send the notification to.
• expires (datetime.datetime) – Expiration date (for transient notifications).
• token (dict) – Set this if the notification should correspond to a token instead of a user.

get(user, since=None, token=None, sort=None)

Get outstanding notifications for the given user.

Parameters:

• user – The user requesting updates. None to use the token instead.
• since (datetime) – Limit results to entities that have been updated since a certain timestamp.
• token – if the user is None, the token requesting updated.
• sort – Sort field for the database query.

initProgress(user, title, total=0, state='active', current=0, message='', token=None, estimateTime=True, resource=None, resourceName=None)

Create a “progress” type notification that can be updated anytime there is progress on some task. Progress records that are not updated for more than one hour will be deleted. The “time” field of a progress record indicates the time the task was started.

Parameters:

• user – the user associated with this notification. If this is None, a session token must be specified.
• title (str) – The title of the task. This should not change over the course of the task. (e.g. ‘Deleting folder “foo”’)
• total (int, long, or float) – Some numeric value representing the total task length. By convention, setting this <= 0 means progress on this task is indeterminate.
• state (ProgressState enum value.) – Represents the state of the underlying task execution.
• current (int, long, or float) – Some numeric value representing the current progress of the task (relative to total).
• message (str) – Message corresponding to the current state of the task.
• token – if the user is None, associate this notification with the specified session token.
• estimateTime – if True, generate an estimate of the total time the task will take, if possible. If False, never generate a time estimate.
• resource – a partial or complete resource that the notification is associated with. This must at a minimum include the id of the resource.
• resourceName – the type of resource the notification is associated with.

initialize()

Subclasses should override this and set the name of the collection as self.name. Also, they should set any indexed fields that they require.

updateProgress(record, save=True, **kwargs)

Update an existing progress record.

Parameters:

• record (dict) – The existing progress record to update.
• total (int, long, or float) – Some numeric value representing the total task length. By convention, setting this <= 0 means progress on this task is indeterminate. Generally this shouldn’t change except in cases where progress on a task switches between indeterminate and determinate state.
• state (ProgressState enum value.) – Represents the state of the underlying task execution.
• current (int, long, or float) – Some numeric value representing the current progress of the task (relative to total).
• increment (int, long, or float) – Amount to increment the progress by. Don’t pass both current and increment together, as that behavior is undefined.
• message (str) – Message corresponding to the current state of the task.
• expires (datetime) – Set a custom (UTC) expiration time on the record. Default is one hour from the current time.
• save (bool) – Whether to save the record to the database.

validate(doc)

Models should implement this to validate the document before it enters the database. It must return the document with any necessary filters applied, or throw a ValidationException if validation of the document fails.

Parameters:doc (dict) – The document to validate before saving to the collection.

class girder.models.notification.ProgressState

Enum of possible progress states for progress records.

Web API Endpoints

Base Classes and Helpers

girder.api.access.admin(fun, scope=None, cookie=False)

REST endpoints that require administrator access should be wrapped in this decorator.

Parameters:

• fun (callable) – A REST endpoint.
• scope (str or list of str or None) – To also expose this endpoint for certain token scopes, pass those scopes here. If multiple are passed, all will be required.
• cookie (bool) – if True, this rest endpoint allows the use of a cookie for authentication. If this is specified on routes that can alter the system (those other than HEAD and GET), it can expose an application to Cross-Site Request Forgery (CSRF) attacks.

girder.api.access.public(fun, scope=None, cookie=False)

Functions that allow any client access, including those that haven’t logged in should be wrapped in this decorator.

Parameters:

• fun (callable) – A REST endpoint.
• scope (str or list of str or None) – The scope or list of scopes required for this token.
• cookie (bool) – if True, this rest endpoint allows the use of a cookie for authentication. If this is specified on routes that can alter the system (those other than HEAD and GET), it can expose an application to Cross-Site Request Forgery (CSRF) attacks.

girder.api.access.token(fun, scope=None, required=False, cookie=False)

REST endpoints that require a token, but not necessarily a user authentication token, should use this access decorator.

Parameters:

• fun (callable) – A REST endpoint.
• scope (str or list of str or None) – The scope or list of scopes required for this token.
• required (bool) – Whether all of the passed scope are required to access the endpoint at all.
• cookie (bool) – if True, this rest endpoint allows the use of a cookie for authentication. If this is specified on routes that can alter the system (those other than HEAD and GET), it can expose an application to Cross-Site Request Forgery (CSRF) attacks.

girder.api.access.user(fun, scope=None, cookie=False)

REST endpoints that require a logged-in user should be wrapped with this access decorator.

Parameters:

• fun (callable) – A REST endpoint.
• scope (str or list of str or None) – To also expose this endpoint for certain token scopes, pass those scopes here. If multiple are passed, all will be required.
• cookie (bool) – if True, this rest endpoint allows the use of a cookie for authentication. If this is specified on routes that can alter the system (those other than HEAD and GET), it can expose an application to Cross-Site Request Forgery (CSRF) attacks.

class girder.api.describe.ApiDocs(templatePath=None)

This serves up the Swagger page.

class girder.api.describe.Describe

class girder.api.describe.Description(summary)

This class provides convenient chainable semantics to allow api route handlers to describe themselves to the documentation. A route handler function can apply the girder.api.describe.describeRoute decorator to itself (called with an instance of this class) in order to describe itself.

asDict()

Returns this description object as an appropriately formatted dict

deprecated()

Mark the route as deprecated.

errorResponse(reason='A parameter was invalid.', code=400)

This helper will build an errorResponse declaration for you. Many endpoints will be able to use the default parameter values for one of their responses.

Parameters:

• reason (str, list, or tuple) – The reason or list of reasons why the error occurred.
• code (int) – HTTP status code.

jsonParam(name, description, paramType='query', dataType='string', required=True, default=None, requireObject=False, requireArray=False, schema=None)

Specifies a parameter that should be processed as JSON.

Parameters:

• requireObject (bool) – Whether the value must be a JSON object / Python dict.
• requireArray (bool) – Whether the value must be a JSON array / Python list.
• schema (dict) – A JSON schema that will be used to validate the parameter value. If this is passed, it overrides any requireObject or requireArray values that were passed.

modelParam(name, description=None, model=None, destName=None, paramType='path', plugin='_core', level=None, required=True, force=False, exc=True, requiredFlags=None, **kwargs)

This should be used in lieu of param if the parameter is a model ID and the model should be loaded and passed into the route handler. For example, if you have a route like GET /item/:id, you could do:

>>> from girder.models.item import Item
>>> modelParam('id', model=Item, level=AccessType.READ)

Which would cause the id parameter in the path to be mapped to an item model parameter named item, and ensure that the calling user has at least READ access on that item. For parameters passed in the query string or form data, for example a request like POST /item?folderId=..., you must specify the paramType.

>>> modelParam('folderId', 'The ID of the parent folder.', model=Folder,
...            level=AccessType.WRITE, paramType='query')

Note that in the above example, model is omitted; in this case, the model is inferred to be 'folder' from the parameter name 'folderId'.

Parameters:

• name (str) – The name passed in via the request, e.g. ‘id’.
• description (str) – The description of the parameter. If not passed, defaults to “The ID of the <model>.”
• destName (str) – The kwarg name after model loading, e.g. ‘folder’. Defaults to the value of the model parameter.
• paramType – how is the parameter sent. One of ‘query’, ‘path’, ‘body’, ‘header’, or ‘formData’.
• model (class or str) – The model class to use for loading, or a name, e.g. ‘folder’. If not passed, defaults to stripping the last two characters from the name, such that e.g. ‘folderId’ would make the model become ‘folder’.
• plugin (str) – Plugin name, if loading a plugin model. Only used when the model param is a string rather than a class.
• level (AccessType) – Access level, if this is an access controlled model.
• required (bool) – Whether this parameter is required.
• force (bool) – Force loading of the model (skip access check).
• exc (bool) – Whether an exception should be raised for a nonexistent resource.
• requiredFlags (str or list/set/tuple of str or None) – Access flags that are required on the object being loaded.

pagingParams(defaultSort, defaultSortDir=1, defaultLimit=50)

Adds the limit, offset, sort, and sortdir parameter documentation to this route handler.

Parameters:

• defaultSort (str) – The default field used to sort the result set.
• defaultSortDir (int) – Sort order: -1 or 1 (desc or asc)
• defaultLimit (int) – The default page size.

param(name, description, paramType='query', dataType='string', required=True, enum=None, default=None, strip=False, lower=False, upper=False)

This helper will build a parameter declaration for you. It has the most common options as defaults, so you won’t have to repeat yourself as much when declaring the APIs.

Note that we could expose more parameters from the Parameter Object spec, for example: format, allowEmptyValue, minimum, maximum, pattern, uniqueItems.

Parameters:

• name – name of the parameter used in the REST query.
• description – explanation of the parameter.
• paramType – how is the parameter sent. One of ‘query’, ‘path’, ‘body’, ‘header’, or ‘formData’.
• dataType – the data type expected in the parameter. This is one of ‘integer’, ‘long’, ‘float’, ‘double’, ‘string’, ‘byte’, ‘binary’, ‘boolean’, ‘date’, ‘dateTime’, ‘password’, or ‘file’.
• required – True if the request will fail if this parameter is not present, False if the parameter is optional.
• enum (list) – a fixed list of possible values for the field.
• strip (bool) – For string types, set this to True if the string should be stripped of white space.
• lower (bool) – For string types, set this to True if the string should be converted to lowercase.
• upper (bool) – For string types, set this to True if the string should be converted to uppercase.

girder.api.docs.addModel(name, model, resources=None, silent=False)

Add a model to the Swagger documentation.

Parameters:

• resources – The type(s) of resource(s) to add the model to. New resource types may be implicitly defined, with the expectation that routes will be added for them at some point. If no resources are passed, the model will be exposed for every resource type
• resources – str or tuple/list[str]
• name (str) – The name of the model.
• model (dict) – The model to add.
• silent (bool) – Set this to True to suppress warnings.

Warning

This is a low-level API which does not validate the format of model. See the Swagger Model documentation for a complete specification of the correct format for model.

Changed in version The: syntax and behavior of this function was modified after v1.3.2. The previous implementation did not include a resources parameter.

girder.api.docs.addRouteDocs(resource, route, method, info, handler)

This is called for route handlers that have a description attr on them. It gathers the necessary information to build the swagger documentation, which is consumed by the docs.Describe endpoint.

Parameters:

• resource (str) – The name of the resource, e.g. “item”
• route (tuple[str]) – The route to describe.
• method (str) – The HTTP method for this route, e.g. “POST”
• info (dict) – The information representing the API documentation, typically from girder.api.describe.Description.asDict.
• handler (function) – The actual handler method for this route.

girder.api.docs.removeRouteDocs(resource, route, method, info, handler)

Remove documentation for a route handler.

Parameters:

• resource (str) – The name of the resource, e.g. “item”
• route (tuple[str]) – The route to describe.
• method (str) – The HTTP method for this route, e.g. “POST”
• info (dict) – The information representing the API documentation.
• handler (function) – The actual handler method for this route.

class girder.api.filter_logging.RegexLoggingFilter(name='')

Check log messages against a list of compiled regular expressions. If any of them match, throttle logs.

filter(record)

Determine if the specified record is to be logged.

Is the specified record to be logged? Returns 0 for no, nonzero for yes. If deemed appropriate, the record may be modified in-place.

girder.api.filter_logging.addLoggingFilter(regex, frequency=None, duration=None)

Add a regular expression to the logging filter. If the regular expression matches a registered regex exactly, just update the frequency value.

Parameters:

• regex – a regular expression to match against log messages. For matching cherrypy endpoint logging, this should probably be something like ‘GET /api/v1/item/[0-9a-fA-F]+/download[/ ?#]’. More generally, a value like GET (/[^/ ?#]+)*/item/[^/ ?#]+/download[/ ?#] would be agnostic to the api_root.
• frequency – either None to never log matching log messages, or an integer, where one log message is emitted out of the specified number.
• duration – either None to not log based on elapsed time, or a float value of seconds between logging.

girder.api.filter_logging.removeLoggingFilter(regex)

Remove a regular expression from the logging filter.

Parameters:regex – the regular expression to remove. Returns:True if a filter was removed.

class girder.api.rest.Prefix

Utility class used to provide api prefixes.

class girder.api.rest.Resource

All REST resources should inherit from this class, which provides utilities for adding resources/routes to the REST API.

boolParam(key, params, default=None)

Coerce a parameter value from a str to a bool.

Parameters:

• key (str) – The parameter key to test.
• params (dict) – The request parameters.
• default (bool or None) – The default value if no key is passed.

deleteAuthTokenCookie()

Helper method to kill the authentication cookie

ensureTokenScopes(scope)

Ensure that the token passed to this request is authorized for the designated scope or set of scopes. Raises an AccessException if not.

Parameters:scope (str or list of str) – A scope or set of scopes that is required.

getBodyJson()

Bound wrapper for girder.api.rest.getBodyJson().

getCurrentToken()

Returns the current valid token object that was passed via the token header or parameter, or None if no valid token was passed.

getCurrentUser(returnToken=False)

Returns the currently authenticated user based on the token header or parameter.

Parameters:returnToken (bool) – Whether we should return a tuple that also contains the token. Returns:The user document from the database, or None if the user is not logged in or the token is invalid or expired. If returnToken=True, returns a tuple of (user, token).

getPagingParameters(params, defaultSortField=None, defaultSortDir=1)

Pass the URL parameters into this function if the request is for a list of resources that should be paginated. It will return a tuple of the form (limit, offset, sort) whose values should be passed directly into the model methods that are finding the resources. If the client did not pass the parameters, this always uses the same defaults of limit=50, offset=0, sort=’name’, sortdir=SortDir.ASCENDING=1.

Parameters:

• params (dict) – The URL query parameters.
• defaultSortField (str or None) – If the client did not pass a ‘sort’ parameter, set this to choose a default sort field. If None, the results will be returned unsorted.
• defaultSortDir (girder.constants.SortDir) – Sort direction.

getParamJson(name, params, default=None)

Bound wrapper for girder.api.rest.getParamJson().

getRouteHandler(method, route)

Get the handler method for a given method and route.

Parameters:

• method (str) – The HTTP method, e.g. ‘GET’, ‘POST’, ‘PUT’
• route (tuple[str]) – The route, as a list of path params relative to the resource root, exactly as it was passed to the route method.

Returns:

The handler method for the route.

Return type:

Function

Raises:

Exception, when no route can be found.

handleRoute(method, path, params)

Match the requested path to its corresponding route, and calls the handler for that route with the appropriate kwargs. If no route matches the path requested, throws a RestException.

This method fires two events for each request if a matching route is found. The names of these events are derived from the route matched by the request. As an example, if the user calls GET /api/v1/item/123, the following two events would be fired:

rest.get.item/:id.before

would be fired prior to calling the default API function, and

rest.get.item/:id.after

would be fired after the route handler returns. The query params are passed in the info of the before and after event handlers as event.info[‘params’], and the matched route tokens are passed in as dict items of event.info, so in the previous example event.info would also contain an ‘id’ key with the value of 123. For endpoints with empty sub-routes, the trailing slash is omitted from the event name, e.g.:

rest.post.group.before

Note

You will normally not need to call this method directly, as it is called by the internals of this class during the routing process.

Parameters:

• method (str) – The HTTP method of the current request.
• path (tuple[str]) – The path params of the request.

removeRoute(method, route, resource=None)

Remove a route from the handler and documentation.

Parameters:

• method (str) – The HTTP method, e.g. ‘GET’, ‘POST’, ‘PUT’
• route (tuple[str]) – The route, as a list of path params relative to the resource root. Elements of this list starting with ‘:’ are assumed to be wildcards.
• resource – the name of the resource at the root of this route.

requireAdmin(user, message=None)

Calling this on a user will ensure that they have admin rights. If not, raises an AccessException.

Parameters:

• user (dict.) – The user to check admin flag on.
• message (str or None) – The exception message.

Raises:

AccessException – If the user is not an administrator.

requireParams(required, provided=None)

This method has two modes. In the first mode, this takes two parameters, the first being a required parameter or list of them, and the second the dictionary of parameters that were passed. If the required parameter does not appear in the passed parameters, a ValidationException is raised.

The second mode of operation takes only a single parameter, which is a dict mapping required parameter names to passed in values for those params. If the value is None, a ValidationException is raised. This mode works well in conjunction with the autoDescribeRoute decorator, where the parameters are not all contained in a single dictionary.

Parameters:

• required (list, tuple, or str) – An iterable of required params, or if just one is required, you can simply pass it as a string.
• provided (dict) – The list of provided parameters.

route(method, route, handler, nodoc=False, resource=None)

Define a route for your REST resource.

Parameters:

• method (str) – The HTTP method, e.g. ‘GET’, ‘POST’, ‘PUT’, ‘PATCH’
• route (tuple[str]) – The route, as a list of path params relative to the resource root. Elements of this list starting with ‘:’ are assumed to be wildcards.
• handler (function) – The method to be called if the route and method are matched by a request. Wildcards in the route will be expanded and passed as kwargs with the same name as the wildcard identifier.
• nodoc (bool) – If your route intentionally provides no documentation, set this to True to disable the warning on startup.
• resource – The name of the resource at the root of this route. The resource instance (self) can also be passed. This allows the mount path to be looked up. This allows a resource to be mounted at a prefix.

sendAuthTokenCookie(user=None, scope=None, token=None, days=None)

Helper method to send the authentication cookie

setRawResponse(*args, **kwargs)

Bound alias for girder.api.rest.setRawResponse.

girder.api.rest.boundHandler(fun, ctx=None)

This decorator allows unbound functions to be conveniently added as route handlers to existing girder.api.rest.Resource instances. With no arguments, this uses a shared, generic Resource instance as the context. If you need a specific instance, pass that as the ctx arg, for instance if you need to reference the resource name or any other properties specific to a Resource subclass.

Plugins that add new routes to existing API resources are encouraged to use this to gain access to bound convenience methods like self.getCurrentUser, etc.

Parameters:

• fun (callable) – A REST endpoint.
• ctx (Resource or None) – A Resource instance, to be bound to fun.

girder.api.rest.disableAuditLog(fun)

If calls to a REST route should not be logged in the audit log, decorate it with this function.

girder.api.rest.endpoint(fun)

REST HTTP method endpoints should use this decorator. It converts the return value of the underlying method to the appropriate output format and sets the relevant response headers. It also handles RestExceptions, which are 400-level exceptions in the REST endpoints, AccessExceptions resulting from access denial, and also handles any unexpected errors using 500 status and including a useful traceback in those cases.

If you want a streamed response, simply return a generator function from the inner method.

girder.api.rest.ensureTokenScopes(token, scope)

Call this to validate a token scope for endpoints that require tokens other than a user authentication token. Raises an AccessException if the required scopes are not allowed by the given token.

Parameters:

• token (dict) – The token object used in the request.
• scope (str or list of str) – The required scope or set of scopes.

girder.api.rest.getApiUrl(url=None, preferReferer=False)

In a request thread, call this to get the path to the root of the REST API. The returned path does not end in a forward slash.

Parameters:

• url – URL from which to extract the base URL. If not specified, uses the server root system setting. If that is not specified, uses cherrypy.url()
• preferReferer – if no url is specified, this is true, and this is in a cherrypy request that has a referer header that contains the api string, use that referer as the url.

girder.api.rest.getBodyJson(allowConstants=False)

For requests that are expected to contain a JSON body, this returns the parsed value, or raises a girder.api.rest.RestException for invalid JSON.

Parameters:allowConstants (bool) – Whether the keywords Infinity, -Infinity, and NaN should be allowed. These keywords are valid JavaScript and will parse to the correct float values, but are not valid in strict JSON.

girder.api.rest.getCurrentToken(allowCookie=None)

Returns the current valid token object that was passed via the token header or parameter, or None if no valid token was passed.

Parameters:allowCookie (bool) – Normally, authentication via cookie is disallowed to protect against CSRF attacks. If you want to expose an endpoint that can be authenticated with a token passed in the Cookie, set this to True. This should only be used on read-only operations that will not make any changes to data on the server, and only in cases where the user agent behavior makes passing custom headers infeasible, such as downloading data to disk in the browser. In the event that allowCookie is not explicitly passed, it will default to False unless the access.cookie decorator is used.

girder.api.rest.getCurrentUser(returnToken=False)

Returns the currently authenticated user based on the token header or parameter.

Parameters:returnToken (bool) – Whether we should return a tuple that also contains the token. Returns:the user document from the database, or None if the user is not logged in or the token is invalid or expired. If returnToken=True, returns a tuple of (user, token).

girder.api.rest.getParamJson(name, params, default=None)

For parameters that are expected to be specified as JSON, use this to parse them, or raises a RestException if parsing fails.

Parameters:

• name (str) – The param name.
• params (dict) – The dictionary of parameters.
• default – The default value if no such param was passed.

girder.api.rest.getUrlParts(url=None)

Calls urllib.parse.urlparse on a URL.

Parameters:url (str or None) – A URL, or None to use the current request’s URL. Returns:The URL’s separate components. Return type:urllib.parse.ParseResult

girder.api.rest.iterBody(length=65536, strictLength=False)

This is a generator that will read the request body a chunk at a time and yield each chunk, abstracting details of the underlying HTTP server. This function works regardless of whether the body was sent with a Content-Length or using Transfer-Encoding: chunked, but the behavior is slightly different in each case.

If Content-Length is provided, the length parameter is used to read the body in chunks up to size length. This will block until end of stream or the specified number of bytes is ready.

If Transfer-Encoding: chunked is used, the length parameter is ignored by default, and the generator yields each chunk that is sent in the request regardless of its length. However, if strictLength is set to True, it will block until length bytes have been read or the end of the request.

Parameters:

• length (int) – Max buffer size to read per iteration if the request has a known Content-Length.
• strictLength (bool) – If the request is chunked, set this to True to block until length bytes have been read or end-of-stream.

class girder.api.rest.loadmodel(map=None, model=None, plugin='_core', level=None, force=False, exc=True, requiredFlags=None, **kwargs)

This is a decorator that can be used to load a model based on an ID param. For access controlled models, it will check authorization for the current user. The underlying function is called with a modified set of keyword arguments that is transformed by the “map” parameter of this decorator. Any additional kwargs will be passed to the underlying model’s load.

Parameters:

• map (dict or None) – Map of incoming parameter name to corresponding model arg name. If None is passed, this will map the parameter named “id” to a kwarg named the same as the “model” parameter.
• model (str) – The model name, e.g. ‘folder’
• plugin (str) – Plugin name, if loading a plugin model.
• level (AccessType) – Access level, if this is an access controlled model.
• force (bool) – Force loading of the model (skip access check).
• exc (bool) – Whether an exception should be raised for a nonexistent resource.
• requiredFlags (str or list/set/tuple of str or None) – Access flags that are required on the object being loaded.

girder.api.rest.rawResponse(fun)

This is a decorator that can be placed on REST route handlers, and is equivalent to calling setRawResponse() in the handler body.

girder.api.rest.requireAdmin(user, message=None)

Calling this on a user will ensure that they have admin rights. If not, raises an AccessException.

Parameters:

• user (dict.) – The user to check admin flag on.
• message (str or None) – The exception message.

Raises:

AccessException – If the user is not an administrator.

girder.api.rest.setContentDisposition(filename, disposition='attachment', setHeader=True)

Set the content disposition header to either inline or attachment, and specify a filename that is properly escaped. See developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Content-Disposition, tools.ietf.org/html/rfc2183, tools.ietf.org/html/rfc6266, and tools.ietf.org/html/rfc5987 for specifications and details.

Parameters:

• filename – the filename to add to the content disposition header.
• disposition – either ‘inline’ or ‘attachment’. None is the same as ‘attachment’. Any other value skips setting the content disposition header.
• setHeader – if False, return the value that would be set to the Content-Disposition header, but do not set it.

Returns:

the content-disposition header value.

girder.api.rest.setCurrentUser(user)

Explicitly set the user for the current request thread. This can be used to enable specialized auth behavior on a per-request basis.

Parameters:user (dict or None) – The user to set as the current user of this request.

girder.api.rest.setRawResponse(val=True)

Normally, non-streaming responses go through a serialization process in accordance with the “Accept” request header. Endpoints that wish to return a raw response without using a streaming response should call this, or use its bound version on the Resource class, or add the rawResponse decorator on the REST route handler function.

Parameters:val (bool) – Whether the return value should be sent raw.

girder.api.rest.setResponseHeader(header, value)

Set a response header to the given value.

Parameters:

• header (str) – The header name.
• value (str) – The value for the header.

Utility

class girder.utility.JsonEncoder(*, skipkeys=False, ensure_ascii=True, check_circular=True, allow_nan=True, sort_keys=False, indent=None, separators=None, default=None)

This extends the standard json.JSONEncoder to allow for more types to be sensibly serialized. This is used in Girder’s REST layer to serialize route return values when JSON is requested.

default(obj)

Implement this method in a subclass such that it returns a serializable object for o, or calls the base implementation (to raise a TypeError).

For example, to support arbitrary iterators, you could implement default like this:

def default(self, o):
    try:
        iterable = iter(o)
    except TypeError:
        pass
    else:
        return list(iterable)
    # Let the base class default method raise the TypeError
    return JSONEncoder.default(self, o)

class girder.utility.RequestBodyStream(stream, size=None)

Wraps a cherrypy request body into a more abstract file-like object.

getSize()

Returns the size of the body data wrapped by this class. For multipart encoding, this is the size of the part. For sending as the body, this is the Content-Length header.

girder.utility.camelcase(value)

Convert a module name or string with underscores and periods to camel case.

Parameters:value (str) – the string to convert Returns:the value converted to camel case.

girder.utility.genToken(length=64)

Use this utility function to generate a random string of a desired length.

girder.utility.mkdir(path, mode=511, recurse=True, existOk=True)

Create a new directory or ensure a directory already exists.

Parameters:

• path (str) – The directory to create.
• mode (int) – The mode (permission mask) prior to applying umask.
• recurse (bool) – Whether intermediate missing dirs should be created.
• existOk (bool) – Set to True to suppress the error if the dir exists.

girder.utility.optionalArgumentDecorator(baseDecorator)

This decorator can be applied to other decorators, allowing the target decorator to be used either with or without arguments.

The target decorator is expected to take at least 1 argument: the function to be wrapped. If additional arguments are provided by the final implementer of the target decorator, they will be passed to the target decorator as additional arguments.

For example, this may be used as:

@optionalArgumentDecorator
def myDec(fun, someArg=None):
    ...

@myDec
def a(...):
    ...

@myDec()
def a(...):
    ...

@myDec(5)
def a(...):
    ...

@myDec(someArg=5)
def a(...):
    ...

Parameters:baseDecorator (callable) – The target decorator.

girder.utility.parseTimestamp(x, naive=True)

Parse a datetime string using the python-dateutil package.

If no timezone information is included, assume UTC. If timezone information is included, convert to UTC.

If naive is True (the default), drop the timezone information such that a naive datetime is returned.

girder.utility.toBool(val)

Coerce a string value to a bool. Meant to be used to parse HTTP parameters, which are always sent as strings. The following string values will be interpreted as True:

• 'true'
• 'on'
• '1'
• 'yes'

All other strings will be interpreted as False. If the given param is not passed at all, returns the value specified by the default arg. This function is case-insensitive.

Parameters:val (str) – The value to coerce to a bool.

class girder.utility.abstract_assetstore_adapter.AbstractAssetstoreAdapter(assetstore)

This defines the interface to be used by all assetstore adapters.

cancelUpload(upload)

This is called when an upload has been begun and it should be abandoned. It must clean up temporary files, chunks, or whatever other information the assetstore contains.

capacityInfo()

Assetstore types that are able to report how much free and/or total capacity they have should override this method. Default behavior is to report both quantities as unknown.

Returns:A dict with ‘free’ and ‘total’ keys whose values are either bytes (ints) or None for an unknown quantity.

checkUploadSize(upload, chunkSize)

Check if the upload is valid based on the chunk size. If this raises an exception, then the caller should clean up and reraise the exception.

Parameters:

• upload – the dictionary of upload information. The received and size values are used.
• chunkSize (a non-negative integer or None if unknown.) – the chunk size that needs to be validated.

copyFile(srcFile, destFile)

This method copies the necessary fields and data so that the destination file contains the same data as the source file.

Parameters:

• srcFile (dict) – The original File document.
• destFile (dict) – The File which should have the data copied to it.

Returns:

A dict with the destination file.

deleteFile(file)

This is called when a File is deleted to allow the adapter to remove the data from within the assetstore. This method should not modify or delete the file object, as the caller will delete it afterward.

Parameters:file (dict) – The File document about to be deleted.

downloadFile(file, offset=0, headers=True, endByte=None, contentDisposition=None, extraParameters=None, **kwargs)

This method is in charge of returning a value to the RESTful endpoint that can be used to download the file. This should either return a generator function that yields the bytes of the file (which will stream the file directly), or modify the response headers and raise a cherrypy.HTTPRedirect.

Parameters:

• file (dict) – The file document being downloaded.
• offset (int) – Offset in bytes to start the download at.
• headers (bool) – Flag for whether headers should be sent on the response.
• endByte (int or None) – Final byte to download. If None, downloads to the end of the file.
• contentDisposition (str or None) – Value for Content-Disposition response header disposition-type value.

static fileIndexFields()

Default behavior is that no additional file fields need to be indexed within the database.

fileUpdated(file)

This is called when the file document has been changed. Any assetstore implementation that needs to do anything when the file document changes should override this method.

Parameters:file (dict) – The updated file document.

finalizeUpload(upload, file)

Call this once the last chunk has been processed. This method does not need to delete the upload document as that will be deleted by the caller afterward. This method may augment the File document, and must return the File document.

Parameters:

• upload (dict) – The upload document.
• file (dict) – The file document that was created.

Returns:

The file document with optional modifications.

findInvalidFiles(progress=<girder.utility.progress.ProgressContext object>, filters=None, checkSize=True, **kwargs)

Finds and yields any invalid files in the assetstore. It is left to the caller to decide what to do with them.

Parameters:

• progress (girder.utility.progress.ProgressContext) – Pass a progress context to record progress.
• filters (dict or None) – Additional query dictionary to restrict the search for files. There is no need to set the assetstoreId in the filters, since that is done automatically.
• checkSize (bool) – Whether to make sure the size of the underlying data matches the size of the file.

getChunkSize(chunk)

Given a chunk that is either a file-like object or a string, attempt to determine its length. If it is a file-like object, then this relies on being able to use fstat.

Parameters:chunk (a file-like object or a string) – the chunk to get the size of Returns:the length of the chunk if known, or None.

getLocalFilePath(file)

If an assetstore adapter supports it, return a path to the file on the local file system. Otherwise, raise an exception.

Parameters:file (dict) – The file document. Returns:a local path to the file. Return type:str

importData(parent, parentType, params, progress, user, **kwargs)

Assetstores that are capable of importing pre-existing data from the underlying storage medium can implement this method.

Parameters:

• parent – The parent object to import into.
• parentType (str) – The model type of the parent object (folder, user, or collection).
• params (dict) – Additional parameters required for the import process. Typically includes an importPath field representing a root path on the underlying storage medium.
• progress (girder.utility.progress.ProgressContext) – Object on which to record progress if possible.
• user (dict or None) – The Girder user performing the import.

initUpload(upload)

This must be called before any chunks are uploaded to do any additional behavior and optionally augment the upload document. The method must return the upload document. Default behavior is to simply return the upload document unmodified.

Parameters:upload (dict) – The upload document to optionally augment.

open(file)

Exposes a Girder file as a python file-like object. At the moment, this is a read-only interface, the equivalent of opening a system file with ‘rb’ mode.

Parameters:file (dict) – A Girder file document. Returns:A file-like object containing the bytes of the file. Return type:FileHandle

requestOffset(upload)

Request the offset for resuming an interrupted upload. Default behavior simply returns the ‘received’ field of the upload document. This method exists because in some cases, such as when the server crashes, it’s possible that the received field is not accurate, so adapters may implement this to provide the actual next byte required.

setContentHeaders(file, offset, endByte, contentDisposition=None)

Sets the Content-Length, Content-Disposition, Content-Type, and also the Content-Range header if this is a partial download.

Parameters:

• file – The file being downloaded.
• offset (int) – The start byte of the download.
• endByte (int) – The end byte of the download (non-inclusive).
• contentDisposition (str or None) – Content-Disposition response header disposition-type value, if None, Content-Disposition will be set to ‘attachment; filename=$filename’.

shouldImportFile(path, params)

This is a helper used during the import process to determine if a file located at the specified path should be imported, based on the request parameters. Exclusion takes precedence over inclusion.

Parameters:

• path (str) – The path of the file.
• params (dict) – The request parameters.

Return type:

bool

untrackedUploads(knownUploads=(), delete=False)

List and optionally discard uploads that are in the assetstore but not in the known list.

Parameters:

• knownUploads (list) – a list of upload dictionaries of all known incomplete uploads.
• delete (bool) – if True, delete any unknown uploads.

Returns:

a list of unknown uploads.

uploadChunk(upload, chunk)

Call this method to process each chunk of an upload.

Parameters:

• upload (dict) – The upload document to update.
• chunk (file) – The file object representing the chunk that was uploaded.

Returns:

Must return the upload document with any optional changes.

static validateInfo(doc)

Adapters may implement this if they need to perform any validation steps whenever the assetstore info is saved to the database. It should return the document with any necessary alterations in the success case, or throw an exception if validation fails.

class girder.utility.abstract_assetstore_adapter.FileHandle(file, adapter)

This is the base class that is returned for the file-like API into Girder file objects. The open method of assetstore implementations is responsible for returning an instance of this class or one of its subclasses. This base class implementation is returned by the abstract assetstore adapter, and does not leverage any details of the assetstore implementations.

These file handles are stateful, and therefore not safe for concurrent access. If used by multiple threads, mutexes should be used.

Parameters:

• file (dict) – The file object to which this file-like object corresponds.
• adapter (girder.utility.abstract_assetstore_adapter.AbstractAssetstoreAdapter) – The assetstore adapter corresponding to this file.

read(size=None)

Read size bytes from the file data.

Parameters:size (int) – The number of bytes to read from the current position. The actual number returned could be less than this if the end of the file is reached. An empty response indicates that the file has been completely consumed. If None or negative, read to the end of the file. Return type:bytes

class girder.utility.acl_mixin.AccessControlMixin

This mixin is intended to be used for resources which aren’t access controlled by default, but resolve their access controls through other resources. As such, the overridden methods retain the same parameters and only alter functionality related to access control resolution.

resourceColl corresponds to the resource collection that needs to be used for resolution, for example the Item model has a resourceColl of folder.

resourceParent corresponds to the field in which the parent resource belongs, so for an item it would be the folderId.

filterResultsByPermission(cursor, user, level, limit=0, offset=0, removeKeys=(), flags=None)

Yields filtered results from the cursor based on the access control existing for the resourceParent.

Takes the same parameters as girder.models.model_base.AccessControlledModel.filterResultsByPermission().

findWithPermissions(query=None, offset=0, limit=0, timeout=None, fields=None, sort=None, user=None, level=0, aggregateSort=None, **kwargs)

Search the collection by a set of parameters, only returning results that the combined user and level have permission to access. Passes any extra kwargs through to the underlying pymongo.collection.find function.

Parameters:

• query (dict) – The search query (see general MongoDB docs for “find()”)
• offset (int) – The offset into the results
• limit (int) – Maximum number of documents to return
• timeout (int) – Cursor timeout in ms. Default is no timeout.
• fields (str, list of strings or tuple of strings for fields to be included from the document, or dict for an inclusion or exclusion projection.) – A mask for filtering result documents by key, or None to return the full document, passed to MongoDB find() as the projection param.
• sort (List of (key, order) tuples.) – The sort order.
• user (dict or None) – The user to check policies against.
• level (AccessType) – The access level. Explicitly passing None skips doing permissions checks.
• aggregateSort (List of (key, order) tuples.) – A sort order to use if sort is None and an aggregation is used.

Returns:

A pymongo Cursor, CommandCursor, or an iterable. If a CommandCursor, it has been augmented with a count function.

hasAccess(resource, user=None, level=0)

Determines if a user has access to a resource based on their access to the resourceParent.

Takes the same parameters as girder.models.model_base.AccessControlledModel.hasAccess().

hasAccessFlags(doc, user=None, flags=None)

See the documentation of AccessControlledModel.hasAccessFlags, which this wraps.

load(id, level=2, user=None, objectId=True, force=False, fields=None, exc=False)

Calls Model.load on the current item, and then attempts to load the resourceParent which the user must have access to in order to load this model.

Takes the same parameters as girder.models.model_base.AccessControlledModel.load().

prefixSearch(query, user=None, filters=None, limit=0, offset=0, sort=None, fields=None, level=0, prefixSearchFields=None)

Custom override of Model.prefixSearch to also force permission-based filtering. The parameters are the same as Model.prefixSearch.

Parameters:

• user (dict or None) – The user to apply permission filtering for.
• level (girder.constants.AccessType) – The access level to require.

requireAccess(doc, user=None, level=0)

This wrapper just provides a standard way of throwing an access denied exception if the access check fails.

requireAccessFlags(doc, user=None, flags=None)

See the documentation of AccessControlledModel.requireAccessFlags, which this wraps.

girder.utility.assetstore_utilities.fileIndexFields()

This will return a set of all required index fields from all of the different assetstore types.

girder.utility.assetstore_utilities.getAssetstoreAdapter(assetstore, instance=True)

This is a factory method that will return the appropriate assetstore adapter for the specified assetstore. The returned object will conform to the interface of the AbstractAssetstoreAdapter.

Parameters:

• assetstore (dict) – The assetstore document used to instantiate the adapter.
• instance (bool) – Whether to return an instance of the adapter or the class. If you are performing validation, set this to False to avoid throwing unwanted exceptions during instantiation.

Returns:

An adapter descending from AbstractAssetstoreAdapter

girder.utility.assetstore_utilities.setAssetstoreAdapter(storeType, cls)

This updates the internal assetstore adapter table with either a new entry, or a modification to an existing entry. Subsequent calls to getAssetstoreAdapter() will return the modified class (or instance thereof), allowing for dynamic updating of assetstore behavior at runtime.

Parameters:

• storeType (enum | any) – The assetstore type to create/modify.
• cls (AbstractAssetstoreAdapter) – The new assetstore adapter class to install in the table. This should be an adapter descending from AbstractAssetstoreAdapter.

class girder.utility.filesystem_assetstore_adapter.FilesystemAssetstoreAdapter(assetstore)

This assetstore type stores files on the filesystem underneath a root directory. Files are named by their SHA-512 hash, which avoids duplication of file content.

Parameters:assetstore (dict) – The assetstore to act on.

cancelUpload(upload)

Delete the temporary files associated with a given upload.

capacityInfo()

For filesystem assetstores, we just need to report the free and total space on the filesystem where the assetstore lives.

deleteFile(file)

Deletes the file from disk if it is the only File in this assetstore with the given sha512. Imported files are not actually deleted.

downloadFile(file, offset=0, headers=True, endByte=None, contentDisposition=None, extraParameters=None, **kwargs)

Returns a generator function that will be used to stream the file from disk to the response.

static fileIndexFields()

File documents should have an index on their sha512 field, as well as whether or not they are imported.

finalizeUpload(upload, file)

Moves the file into its permanent content-addressed location within the assetstore. Directory hierarchy yields 256^2 buckets.

findInvalidFiles(progress=<girder.utility.progress.ProgressContext object>, filters=None, checkSize=True, **kwargs)

Goes through every file in this assetstore and finds those whose underlying data is missing or invalid. This is a generator function – for each invalid file found, a dictionary is yielded to the caller that contains the file, its absolute path on disk, and a reason for invalid, e.g. “missing” or “size”.

Parameters:

• progress (girder.utility.progress.ProgressContext) – Pass a progress context to record progress.
• filters (dict or None) – Additional query dictionary to restrict the search for files. There is no need to set the assetstoreId in the filters, since that is done automatically.
• checkSize (bool) – Whether to make sure the size of the underlying data matches the size of the file.

fullPath(file)

Utility method for constructing the full (absolute) path to the given file.

getLocalFilePath(file)

Return a path to the file on the local file system.

Parameters:file – The file document. Returns:a local path to the file or None if no such path is known.

importData(parent, parentType, params, progress, user, leafFoldersAsItems)

Assetstores that are capable of importing pre-existing data from the underlying storage medium can implement this method.

Parameters:

• parent – The parent object to import into.
• parentType (str) – The model type of the parent object (folder, user, or collection).
• params (dict) – Additional parameters required for the import process. Typically includes an importPath field representing a root path on the underlying storage medium.
• progress (girder.utility.progress.ProgressContext) – Object on which to record progress if possible.
• user (dict or None) – The Girder user performing the import.

importFile(item, path, user, name=None, mimeType=None, **kwargs)

Import a single file from the filesystem into the assetstore.

Parameters:

• item (dict) – The parent item for the file.
• path (str) – The path on the local filesystem.
• user (dict) – The user to list as the creator of the file.
• name (str) – Name for the file. Defaults to the basename of path.
• mimeType (str) – MIME type of the file if known.

Returns:

The file document that was created.

initUpload(upload)

Generates a temporary file and sets its location in the upload document as tempFile. This is the file that the chunks will be appended to.

requestOffset(upload)

Returns the size of the temp file.

uploadChunk(upload, chunk)

Appends the chunk into the temporary file.

static validateInfo(doc)

Makes sure the root field is a valid absolute path and is writeable. It also conveniently update the root field replacing the initial component by the user home directory running the server if it matches ~ or ~user.

class girder.utility.gridfs_assetstore_adapter.GridFsAssetstoreAdapter(assetstore)

This assetstore type stores files within MongoDB using the GridFS data model.

cancelUpload(upload)

Delete all of the chunks associated with a given upload.

deleteFile(file)

Delete all of the chunks in the collection that correspond to the given file.

downloadFile(file, offset=0, headers=True, endByte=None, contentDisposition=None, extraParameters=None, **kwargs)

Returns a generator function that will be used to stream the file from the database to the response.

static fileIndexFields()

Default behavior is that no additional file fields need to be indexed within the database.

finalizeUpload(upload, file)

Grab the final state of the checksum and set it on the file object, and write the generated UUID into the file itself.

initUpload(upload)

Creates a UUID that will be used to uniquely link each chunk to

requestOffset(upload)

The offset will be the CHUNK_SIZE * total number of chunks in the database for this file. We return the max of that and the received count because in testing mode we are uploading chunks that are smaller than the CHUNK_SIZE, which in practice will not work.

uploadChunk(upload, chunk)

Stores the uploaded chunk in fixed-sized pieces in the chunks collection of this assetstore’s database.

static validateInfo(doc)

Validate the assetstore – make sure we can connect to it and that the necessary indexes are set up.

girder.utility.mail_utils.addTemplateDirectory(dir, prepend=False)

Adds a directory to the search path for mail templates. This is useful for plugins that have their own set of mail templates.

Parameters:

• dir (str) – The directory to add to the template lookup path.
• prepend (bool) – If True, adds this directory at the beginning of the path so that it will override any existing templates with the same name. Otherwise appends to the end of the lookup path.

girder.utility.mail_utils.getEmailUrlPrefix()

Return the URL prefix for links back to the server. This is the link to the server root, so Girder-level path information and any query parameters or fragment value should be appended to this value.

girder.utility.mail_utils.renderTemplate(name, params=None)

Renders one of the HTML mail templates located in girder/mail_templates.

Parameters:

• name – The name of the file inside girder/mail_templates to render.
• params (dict) – The parameters to pass when rendering the template.

Returns:

The rendered template as a string of HTML.

girder.utility.mail_utils.sendMail(subject, text, to, bcc=None)

Send an email asynchronously.

Parameters:

• subject (str) – The subject line of the email.
• text (str) – The body of the email.
• to (list) – The list of recipient email addresses.
• bcc (list or None) – Recipient email addresses that should be specified using the Bcc header.

girder.utility.mail_utils.sendMailIndividually(subject, text, to)

Send emails asynchronously to all recipients individually.

girder.utility.mail_utils.sendMailSync(subject, text, to, bcc=None)

Send an email synchronously.

girder.utility.mail_utils.sendMailToAdmins(subject, text)

Send an email asynchronously to site admins.

girder.utility.mail_utils.validateEmailAddress(address)

Determines whether a string is a valid email address.

This implements the grammar from 4.10.5.1.5 of the HTML Standard.

Parameters:address (str) – The string to test. Return type:bool

class girder.utility.model_importer.ModelImporter

Any class that wants to have convenient model importing semantics should extend/mixin this class.

static model(model, plugin='_core')

Call this to get the instance of the specified model. It will be lazy-instantiated.

Parameters:

• model (string) – The name of the model to get. This must have been registered using the registerModel() method.
• plugin (str) – Plugin identifier (if this is a plugin model).

Returns:

The instantiated model, which is a singleton.

static registerModel(model, cls, plugin='_core')

Use this method to register a model class to a name. Using this, it can be referenced via the model method of this class.

Parameters:

• model (str) – The model name.
• plugin (str) – Plugin identifier (if this is a plugin model).
• cls (type) – The model class, should be a subclass of girder.models.model_base.Model.

This module contains utility methods for parsing girder path strings.

girder.utility.path.decode(token)

Un-escape special characters in a token from a path representation.

Parameters:token (str) – The token to decode Returns:The decoded string Return type:str

girder.utility.path.encode(token)

Escape special characters in a token for path representation.

Parameters:token (str) – The token to encode Returns:The encoded string Return type:str

girder.utility.path.getResourceName(type, doc)

Get the name of a resource that can be put in a path,

Parameters:

• type (str) – the resource model type.
• doc (dict) – the resource document.

Returns:

the name of the resource.

Return type:

str

girder.utility.path.getResourcePath(type, doc, user=None, force=False)

Get the path for a resource.

Parameters:

• type (str) – the resource model type.
• doc (dict) – the resource document.
• user (dict or None) – user with correct privileges to access path
• force (bool) – if True, don’t validate the access.

Returns:

the path to the resource.

Return type:

str

girder.utility.path.join(tokens)

Join a list of tokens into an encoded path string.

Parameters:tokens – A list of tokens Returns:The encoded path string Return type:str

girder.utility.path.lookUpPath(path, user=None, filter=True, force=False)

Look up a resource in the data hierarchy by path.

Parameters:

• path – path of the resource
• user – user with correct privileges to access path
• filter (bool) – Whether the returned model should be filtered.
• force (bool) – if True, don’t validate the access.

girder.utility.path.lookUpToken(token, parentType, parent)

Find a particular child resource by name or throw an exception.

Parameters:

• token – the name of the child resource to find
• parentType – the type of the parent to search
• parent – the parent resource

Returns:

the child resource

girder.utility.path.split(path)

Split an encoded path string into decoded tokens.

Parameters:path (str) – An encoded path string Returns:A list of decoded tokens Return type:list

class girder.utility.progress.ProgressContext(on, interval=0.5, **kwargs)

This class is a context manager that can be used to update progress in a way that rate-limits writes to the database and guarantees a flush when the context is exited. This is a no-op if “on” is set to False, which is meant as a convenience for callers. Any additional kwargs passed to this constructor are passed through to the initProgress method of the notification model.

Parameters:

• on (bool) – Whether to record progress.
• interval (int or float) – Minimum time interval at which to write updates to the database, in seconds.
• user (dict) – The user creating this progress.
• title (str) – The title for the task being tracked.

update(force=False, **kwargs)

Update the underlying progress record. This will only actually save to the database if at least self.interval seconds have passed since the last time the record was written to the database. Accepts the same kwargs as Notification.updateProgress.

Parameters:force (bool) – Whether we should force the write to the database. Use only in cases where progress may be indeterminate for a long time.

girder.utility.progress.setResponseTimeLimit(duration=600, onlyExtend=True)

If we are currently within a cherrypy response, extend the time limit. By default, cherrypy (version < 12.0) responses will timeout after 300 seconds, so any activity which can take longer should call this function.

Note that for cherrypy responses that include streaming generator functions, such as downloads, the timeout is only relevant until the first yield is reached. As such, long running generator responses do not generally need to call this function.

@deprecated - remove this function once we pin to CherryPy >= 12.0

Parameters:

• duration – additional duration in seconds to allow for the response.
• onlyExtend – if True, only ever increase the timeout. If False, the new duration always replaces the old one.

class girder.utility.s3_assetstore_adapter.S3AssetstoreAdapter(assetstore)

This assetstore type stores files on S3. It is responsible for generating HMAC-signed messages that authorize the client to communicate directly with the S3 server where the files are stored.

cancelUpload(upload)

Delete the temporary files associated with a given upload.

deleteFile(file)

We want to queue up files to be deleted asynchronously since it requires an external HTTP request per file in order to delete them, and we don’t want to wait on that.

Files that were imported as pre-existing data will not actually be deleted from S3, only their references in Girder will be deleted.

downloadFile(file, offset=0, headers=True, endByte=None, contentDisposition=None, extraParameters=None, **kwargs)

When downloading a single file with HTTP, we redirect to S3. Otherwise, e.g. when downloading as part of a zip stream, we connect to S3 and pipe the bytes from S3 through the server to the user agent.

fileUpdated(file)

On file update, if the name or the MIME type changed, we must update them accordingly on the S3 key so that the file downloads with the correct name and content type.

finalizeUpload(upload, file)

Call this once the last chunk has been processed. This method does not need to delete the upload document as that will be deleted by the caller afterward. This method may augment the File document, and must return the File document.

Parameters:

• upload (dict) – The upload document.
• file (dict) – The file document that was created.

Returns:

The file document with optional modifications.

importData(parent, parentType, params, progress, user, **kwargs)

Assetstores that are capable of importing pre-existing data from the underlying storage medium can implement this method.

Parameters:

• parent – The parent object to import into.
• parentType (str) – The model type of the parent object (folder, user, or collection).
• params (dict) – Additional parameters required for the import process. Typically includes an importPath field representing a root path on the underlying storage medium.
• progress (girder.utility.progress.ProgressContext) – Object on which to record progress if possible.
• user (dict or None) – The Girder user performing the import.

initUpload(upload)

Build the request required to initiate an authorized upload to S3.

requestOffset(upload)

Request the offset for resuming an interrupted upload. Default behavior simply returns the ‘received’ field of the upload document. This method exists because in some cases, such as when the server crashes, it’s possible that the received field is not accurate, so adapters may implement this to provide the actual next byte required.

untrackedUploads(knownUploads=None, delete=False)

List and optionally discard uploads that are in the assetstore but not in the known list.

Parameters:

• knownUploads (list) – a list of upload dictionaries of all known incomplete uploads.
• delete (bool) – if True, delete any unknown uploads.

Returns:

a list of unknown uploads.

uploadChunk(upload, chunk)

Rather than processing actual bytes of the chunk, this will generate the signature required to upload the chunk. Clients that do not support direct-to-S3 upload can pass the chunk via the request body as with other assetstores, and Girder will proxy the data through to S3.

Parameters:chunk – This should be a JSON string containing the chunk number and S3 upload ID. If a normal chunk file-like object is passed, we will send the data to S3.

static validateInfo(doc)

Makes sure the root field is a valid absolute path and is writeable.

girder.utility.s3_assetstore_adapter.makeBotoConnectParams(accessKeyId, secret, service=None, region=None, inferCredentials=False)

Create a dictionary of values to pass to the boto connect_s3 function.

Parameters:

• accessKeyId – the S3 access key ID
• secret – the S3 secret key
• service – alternate service URL
• region – the AWS region name of the bucket (if not “us-east-1”)
• inferCredentials – Whether or not Boto should infer the credentials without directly using accessKeyId and secret.

Returns:

boto connection parameter dictionary.

girder.utility.search.addSearchMode(mode, handler)

Register a search mode.

New searches made for the registered mode will call the handler function. The handler function must take parameters: query, types, user, level, limit, offset, and return the search results.

Parameters:

• mode (str) – A search mode identifier.
• handler (function) – A search mode handler function.

girder.utility.search.getSearchModeHandler(mode)

Get the handler function for a search mode

Parameters:mode (str) – A search mode identifier. Returns:A search mode handler function, or None. Return type:function or None

girder.utility.search.removeSearchMode(mode)

Remove a search mode.

This will fail gracefully (returning False) if no search mode mode was registered.

Parameters:mode (str) – A search mode identifier. Returns:Whether the search mode was actually removed. Return type:bool

girder.utility.server.configureServer(mode=None, plugins=None, curConfig=None)

Function to setup the cherrypy server. It configures it, but does not actually start it.

Parameters:

• mode (string) – The server mode to start in.
• plugins – If you wish to start the server with a custom set of plugins, pass this as a list of plugins to load. Otherwise, all installed plugins will be loaded.
• curConfig – The configuration dictionary to update.

girder.utility.server.loadRouteTable(reconcileRoutes=False)

Retrieves the route table from Girder and reconciles the state of it with the current application state.

Reconciliation ensures that every enabled plugin has a route by assigning default routes for plugins that have none, such as newly-enabled plugins.

Returns:The non empty routes (as a dict of name -> route) to be mounted by CherryPy during Girder’s setup phase.

girder.utility.server.setup(mode=None, plugins=None, curConfig=None)

Configure and mount the Girder server and plugins under the appropriate routes.

See ROUTE_TABLE setting.

Parameters:

• mode (string) – The server mode to start in.
• plugins – List of plugins to enable.
• curConfig – The config object to update.

girder.utility.server.staticFile(path, contentType=None)

Helper function to serve a static file. This should be bound as the route object, i.e. info[‘serverRoot’].route_name = staticFile(‘…’)

Parameters:

• path (str) – The path of the static file to serve from this route.
• contentType – The MIME type of the static file. If set to None, the content type wll be guessed by the file extension of the ‘path’ argument.

class girder.utility.setting_utilities.default(key)

Create a decorator indicating that the wrapped function is responsible for providing the default value for the given key or set of keys.

Parameters:key (str or iterable of str) – The key(s) that this function validates.

girder.utility.setting_utilities.getDefaultFunction(key)

Retrieve the default value function for the given key. Returns None if none is registered.

girder.utility.setting_utilities.getValidator(key)

Retrieve the validator function for the given key. Returns None if none is registered.

girder.utility.setting_utilities.registerDefaultFunction(key, fn)

Register a default value function for a given setting key.

Parameters:

• key (str) – The setting key.
• fn (callable) – The function that will return the default value for this key.

girder.utility.setting_utilities.registerValidator(key, fn, replace=False)

Register a validator for a given setting key.

Parameters:

• key (str) – The setting key.
• fn (callable) – The function that will validate this key.
• replace (bool) – If a validator already exists for this key, set this to True to replace the existing validator. The default is to add the new validator in addition to running the old validation function.

class girder.utility.setting_utilities.validator(key, replace=False)

Create a decorator indicating that the wrapped function is responsible for validating the given key or set of keys. For example,

>>> @validator('my_plugin.setting_key')
>>> def validateMySetting(doc):
>>>     if not doc['value']:
>>>         raise ValidationException('This key must not be empty.')

Parameters:

• key (str or iterable of str) – The key(s) that this function validates.
• replace (bool) – If a validator already exists for this key, set this to True to replace the existing validator. The default is to add the new validator in addition to running the old validation function.

class girder.utility.system.StatusMonitor

Register the status of each thread.

unregister()

Unregister the current thread.

girder.utility.system.formatSize(sizeBytes)

Format a size in bytes into a human-readable string with metric unit prefixes.

Parameters:sizeBytes – the size in bytes to format. Returns:the formatted size string.

girder.utility.system.getStatus(mode='basic', user=None)

Get a dictionary of status information regarding the Girder server.

Parameters:

• mode – ‘basic’ returns values available to any anonymous user. ‘quick’ returns only values that are cheap to acquire. ‘slow’ provides all of that information and adds additional
• user – a user record. Must have admin access to get anything other than basic mode.

Returns:

a status dictionary.

class girder.utility.webroot.Webroot(templatePath=None)

The webroot endpoint simply serves the main index HTML file.

class girder.utility.webroot.WebrootBase(templatePath)

Serves a template file in response to GET requests.

This will typically be the base class of any non-API endpoints.

setTemplatePath(templatePath)

Set the path to a template file to render instead of the default template.

The default template remains available so that custom templates can inherit from it. To do so, save the default template filename from the templateFilename attribute before calling this function, pass it as a variable to the custom template using updateHtmlVars(), and reference that variable in an <%inherit> directive like:

<%inherit file=”${context.get(‘defaultTemplateFilename’)}”/>

updateHtmlVars(vars)

If any of the variables in the index html need to change, call this with the updated set of variables to render the template with.

This module is essentially a subset of the python zipfile module that has been modified to allow it to read arbitrary streams (using generators) as input, instead of only accepting files. It also streams the output using generators.

Example of creating and consuming a streaming zip:

zip = ziputil.ZipGenerator(‘TopLevelFolder’)

for data in zip.addFile(lambda: ‘hello world’, ‘hello.txt’):

yield data

yield zip.footer()

class girder.utility.ziputil.ZipGenerator(rootPath='', compression=0)

This class can be used to create a streaming zip file that consumes from one generator and writes to another.

addFile(generator, path)

Generates data to add a file at the given path in the archive. :param generator: Generator function that will yield the file contents. :type generator: function :param path: The path within the archive for this entry. :type path: str

footer()

Once all zip files have been added with addFile, you must call this to get the footer of the archive.

Constants

Constants should be defined here.

class girder.constants.AccessType

Represents the level of access granted to a user or group on an AccessControlledModel. Having a higher access level on a resource also confers all of the privileges of the lower levels.

Semantically, READ access on a resource means that the user can see all the information pertaining to the resource, but cannot modify it.

WRITE access usually means the user can modify aspects of the resource.

ADMIN access confers total control; the user can delete the resource and also manage permissions for other users on it.

class girder.constants.AssetstoreType

All possible assetstore implementation types.

class girder.constants.CoreEventHandler

This enum represents handler identifier strings for core event handlers. If you wish to unbind a core event handler, use one of these as the handlerName argument. Unbinding core event handlers can be used to disable certain default functionalities.

girder.constants.STATIC_PREFIX = '/home/docs/checkouts/readthedocs.org/user_builds/girder/envs/v3.1.2/share/girder'

The local directory containing the static content.

class girder.constants.TerminalColor

Provides a set of values that can be used to color text in the terminal.

class girder.constants.TokenScope

Constants for core token scope strings. Token scopes must not contain spaces, since many services accept scope lists as a space-separated list of strings.

classmethod describeScope(scopeId, name, description, admin=False)

Register a description of a scope.

Parameters:

• scopeId (str) – The unique identifier string for the scope.
• name (str) – A short human readable name for the scope.
• description (str) – A more complete description of the scope.
• admin (bool) – If this scope only applies to admin users, set to True.

girder.constants.registerAccessFlag(key, name, description=None, admin=False)

Register a new access flag in the set of ACCESS_FLAGS available on data in the hierarchy. These are boolean switches that can be used to control access to specific functionality on specific resoruces.

Parameters:

• key (str) – The unique identifier for this access flag.
• name (str) – Human readable name for this permission (displayed in UI).
• description (str) – Human readable longer description for the flag.
• admin – Set this to True to only allow site admin users to set this flag. If True, the flag will only appear in the list for site admins. This can be useful for flags with security considerations.

Events

This module contains the Girder events framework. It maintains a global mapping of events to listeners, and contains utilities for callers to handle or trigger events identified by a name.

Listeners should bind to events by calling:

girder.events.bind('event.name', 'my.handler', handlerFunction)

And events should be fired in one of two ways; if the event should be handled synchronously, fire it with:

girder.events.trigger('event.name', info)

And if the event should be handled asynchronously, use:

girder.events.daemon.trigger('event.name', info, callback)

For obvious reasons, the asynchronous method does not return a value to the caller. Instead, the caller may optionally pass the callback argument as a function to be called when the task is finished. That callback function will receive the Event object as its only argument.

class girder.events.AsyncEventsThread

This class is used to execute the pipeline for events asynchronously. This should not be invoked directly by callers; instead, they should use girder.events.daemon.trigger().

run()

Loops over all queued events. If the queue is empty, this thread gets put to sleep until someone calls trigger() on it with a new event to dispatch.

stop()

Gracefully stops this thread. Will finish the currently processing event before stopping.

trigger(eventName=None, info=None, callback=None)

Adds a new event on the queue to trigger asynchronously.

Parameters:

• eventName – The event name to pass to the girder.events.trigger
• info – The info object to pass to girder.events.trigger
• callback – Optional callable to be called upon completion of all bound event handlers. It takes one argument, which is the event object itself.

class girder.events.Event(name, info, asynchronous=False)

An Event object is created when an event is triggered. It is passed to each of the listeners of the event, which have a chance to add information to the event, and also optionally stop the event from being further propagated to other listeners, and also optionally instruct the caller that it should not execute its default behavior.

addResponse(response)

Listeners that wish to return data back to the caller who triggered this event should call this to append their own response to the event.

Parameters:response – The response value, which can be any type.

preventDefault()

This can be used to instruct the triggerer of the event that the default behavior it would normally perform should not be performed. The semantics of this action are specific to the context of the event being handled, but a common use of this method is for a plugin to provide an alternate behavior that will replace the normal way the event is handled by the core system.

stopPropagation()

Listeners should call this on the event they were passed in order to stop any other listeners to the event from being executed.

class girder.events.ForegroundEventsDaemon

This is the implementation used for girder.events.daemon if the config file chooses to disable using the background thread for the daemon. It executes all bound handlers in the current thread, and provides no-op start() and stop() implementations to remain compatible with the API of AsyncEventsThread.

girder.events.bind(eventName, handlerName, handler)

Bind a listener (handler) to the event identified by eventName. It is convention that plugins will use their own name as the handlerName, so that the trigger() caller can see which plugin(s) responded to the event.

Parameters:

• eventName (str) – The name that identifies the event.
• handlerName (str) – The name that identifies the handler calling bind().
• handler (function) – The function that will be called when the event is fired. It must accept a single argument, which is the Event that was created by trigger(). This function should not return a value; any data that it needs to pass back to the triggerer should be passed via the addResponse() method of the Event.

girder.events.bound(eventName, handlerName, handler)

A context manager to temporarily bind an event handler within its scope.

Parameters are the same as those to girder.events.bind().

girder.events.trigger(eventName, info=None, pre=None, asynchronous=False, daemon=False)

Fire an event with the given name. All listeners bound on that name will be called until they are exhausted or one of the handlers calls the stopPropagation() method on the event.

Parameters:

• eventName (str) – The name that identifies the event.
• info – The info argument to pass to the handler function. The type of this argument is opaque, and can be anything.
• pre (function or None) – A function that will be executed prior to the handler being executed. It will receive a dict with a “handler” key, (the function), “info” key (the info arg to this function), and “eventName” and “handlerName” values.
• asynchronous (bool) – Whether this event is executing on the background thread (True) or on the request thread (False).
• daemon (bool) – Whether this was triggered via girder.events.daemon.

girder.events.unbind(eventName, handlerName)

Removes the binding between the event and the given listener.

Parameters:

• eventName (str) – The name that identifies the event.
• handlerName (str) – The name that identifies the handler calling bind().

girder.events.unbindAll()

Clears the entire event map. All bound listeners will be unbound.

Warning

This will also disable internal event listeners, which are necessary for normal Girder functionality. This function should generally never be called outside of testing.

Exceptions

exception girder.exceptions.AccessException(message, extra=None)

Represents denial of access to a resource.

exception girder.exceptions.FilePathException(message='No assetstore adapter', identifier=None)

Thrown when a file path is requested and cannot be returned.

exception girder.exceptions.GirderBaseException

A class from which all Girder exceptions are based.

exception girder.exceptions.GirderException(message, identifier=None)

Represents a general exception that might occur in regular use. From the user perspective, these are failures, but not catastrophic ones. An identifier can be passed, which allows receivers to check the exception without relying on the text of the message. It is recommended that identifiers are a dot-separated string consisting of the originating python module and a distinct error. For example, ‘girder.model.assetstore.no-current-assetstore’.

exception girder.exceptions.NoAssetstoreAdapter(message='No assetstore adapter')

Raised when no assetstore adapter is available.

exception girder.exceptions.ResourcePathNotFound(message, field=None)

A special case of ValidationException representing the case when the resource at a given path does not exist.

exception girder.exceptions.RestException(message, code=400, extra=None)

Throw a RestException in the case of any sort of incorrect request (i.e. user/client error). Login and permission failures should set a 403 code; almost all other validation errors should use status 400, which is the default.

exception girder.exceptions.ValidationException(message, field=None)

Represents validation failure in the model layer. Raise this with a message and an optional field property. If one of these is thrown in the model during a REST request, it will respond as a 400 status.

Logging

class girder.LogFormatter(fmt=None, datefmt=None, style='%')

Custom formatter that adds useful information about the request to the logs when an exception happens. Cherrypy access logs are passed through without change.

format(record, *args, **kwargs)

Format the specified record as text.

The record’s attribute dictionary is used as the operand to a string formatting operation which yields the returned string. Before formatting the dictionary, a couple of preparatory steps are carried out. The message attribute of the record is computed using LogRecord.getMessage(). If the formatting string uses the time (as determined by a call to usesTime(), formatTime() is called to format the event time. If there is exception information, it is formatted using formatException() and appended to the message.

formatException(exc)

Format and return the specified exception information as a string.

This default implementation just uses traceback.print_exception()

class girder.LogLevelFilter(min, max)

Filter log records based on whether they are between a min and max level.

class girder.StreamToLogger(stream, logger, level)

Redirect a file-like stream to a logger.

girder.getLogPaths()

Return the paths to the error and info log files. These are returned as a dict with “error” and “info” keys that point to the respective file, as well as a “root” key pointing to the log root directory.

girder.logprint(*args, **kwargs)

Send a message to both stdout and the appropriate logs. This behaves like Python3’s print statement, plus takes additional named parameters:

Parameters:

• level – the log level. This determines which log handlers will store the log message. The log is always sent to stdout.
• color – one of the constants.TerminalColor values or None.
• exc_info – None to not print exception information. True for the last exception, or a tuple of exception information.

Plugins

This module defines functions for registering, loading, and querying girder plugins.

class girder.plugin.GirderPlugin(entrypoint)

This is a base class for describing a girder plugin. A plugin is registered by adding an entrypoint under the namespace girder.plugin. This entrypoint should return a class derived from this class.

Example ::

class Cats(GirderPlugin):

def load(self, info):

# load dependent plugins girder.plugin.getPlugin(‘pets’).load(info)

import rest # register new rest endpoints, etc.

CLIENT_SOURCE_PATH = None

The path of the plugin’s web client source code. This path is given relative to the python package. This property is used to link the web client source into the staging area while building in development mode. When this value is None it indicates there is no web client component.

DISPLAY_NAME = None

This is the named displayed to users on the plugin page. Unlike the entrypoint name used internally, this name can be an arbitrary string.

description

Return the plugin description defaulting to the classes docstring.

displayName

Return a user-friendly plugin name (defaults to the entrypoint name).

loaded

Return true if this plugin has been loaded.

name

Return the plugin name defaulting to the entrypoint name.

npmPackages()

Return a dictionary of npm packages -> versions for building the plugin client.

By default, this dictionary will be assembled from the CLIENT_SOURCE_PATH property by inspecting the package.json file in the indicated directory. Plugins can override this method customize the behaivor for advanced usage.

url

Return a url reference to the plugin (usually a readthedocs page).

version

Return the version of the plugin automatically determined from setup.py.

girder.plugin.allPlugins()

Return a list of all detected plugins.

girder.plugin.getPlugin(name)

Return a plugin configuration object or None if the plugin is not found.

girder.plugin.loadedPlugins()

Return a list of successfully loaded plugins.

girder.plugin.registerPluginWebroot(webroot, name)

Adds a webroot to the global registry for plugins based on the plugin name.

Python Client

See Python Client and Girder CLI

Web client

Documentation for Girder’s web client library is not hosted online, but may be built locally by running (from the top-level directory): `bash npm install && npm run docs `

Developer Guide

Girder is a platform-centric web application whose client and server are very loosely coupled. As such, development of Girder can be divided into the server (a CherryPy-based Python module) and the primary client (a Backbone-based) web client. This section is intended to get prospective contributors to understand the tools used to develop Girder.

Configuring Your Development Environment

Next, you should install the Python development dependencies with pip, to provide helpful development tools and to allow the test suite to run:

pip install -r requirements-dev.txt

Install front-end web client development dependencies and build the web client code:

girder build --dev

For more options for building the web client, run:

girder build --help

Finally, you’ll want to set your server into development mode. Add the following entry into your local config file (see Configuration for instructions):

[server]
mode="development"

Vagrant

A shortcut to going through the development environment configuration steps is to use Vagrant to setup the environment on a VirtualBox virtual machine. For more documentation on how to set this up, see Developer Installation

During Development

Once Girder is started via girder serve, the server will reload itself whenever a Python file is modified.

If you are doing front-end development, it’s much faster to use a watch process to perform automatic fast rebuilds of your code whenever you make changes to source files.

If you are making changes to Girder’s core web client, run the following watch command:

girder build --watch

If you are developing a web client of a plugin, run:

girder build --watch-plugin your_plugin_name

With watch option, sourcemaps will be generated, which helps debugging front-end code in browser. When you want to end the watch process, press Ctrl+C (or however you would normally terminate a process in your terminal).

Girder Shell

To test various functionality in a typical REPL (Python, IPython, etc) some bootstrapping is required to configure the Girder server. This sets up an “embedded” server, meaning no TCP ports are actually bound but requests can still be performed via Python. Bootstrapping the server involves running girder.utility.server.configureServer with the plugins to be enabled.

Girder provides a utility script for entering into a shell with the server preconfigured. Once Girder is installed the script can be run using girder shell which optionally takes a comma separated list of plugins to enable.

Utilities

Girder has a set of utility modules and classes that provide handy extractions for certain functionality. Detailed API documentation can be found here.

Configuration Loading

The Girder configuration loader allows for lazy-loading of configuration values in a CherryPy-agnostic manner. The recommended idiom for getting the config object is:

from girder.utility import config
cur_config = config.getConfig()

There is a configuration file for Girder located in girder/conf. The file girder.dist.cfg is the file distributed with the repository and containing the default configuration values. This file should not be edited when deploying Girder. Rather, create a custom girder.cfg file and place it in one of the supported locations (see Configuration). You only need to edit the values in the file that you wish to change from their default values; the system loads the dist file first, then the custom file, so your local settings will override the defaults.

Client Development

If you are writing a custom client application that communicates with the Girder REST API, you should look at the Swagger page that describes all of the available API endpoints. The Swagger page can be accessed by navigating a web browser to /api/v1. If you wish to consume the Swagger-compliant API specification programmatically, the JSON listing is served out of /api/v1/describe.

If you are working on the main Girder web client, either in core or extending it via plugins, there are a few conventions that should be followed. Namely, if you write code that instantiates new View descendant objects, you should pass a parentView property when constructing it. This will allow the child view to be cleaned up recursively when the parent view is destroyed. If you forget to set the parentView property when constructing the view, the view will still work as expected, but a warning message will appear in the console to remind you. Example:

import View from '@girder/core/views/View';

MySubView = View.extend({
   ...
});

new MySubView({
    el: ...,
    otherProperty: ...,
    parentView: this
});

If you use View in custom Backbone apps and need to create a new root view object, set the parentView to null. If you are using a Girder widget in a custom app that does not use the View as the base object for its views, you should pass parentView: null and make sure to call destroy() on the view manually when it should be cleaned up.

Server Side Testing

Most of Girder’s server tests are run via tox, which provides virtual environment isolation and automatic dependency installation for test environments. The tox Python package must be installed:

pip install tox

To run the basic test suite, ensure that a MongoDB instance is ready on localhost:27017, then run:

tox

To destroy and recreate all virtual environments used for testing, pass the -r flag to tox.

Static Analysis Tests

Girder’s static analysis (linting) tests are fast to execute, run on all code, and don’t require a running MongoDB. It’s recommended to run them locally before any Python code changes are committed. To execute them, run:

tox -e lint

pytest Tests

Girder’s modern automated tests are written with pytest. To execute them, ensure MongoDB is ready, then run:

tox -e pytest

Specific arguments can be passed through tox to pytest by adding them after a --.

For example, pytest uses -k to filter tests; to run only the testLoadModelDecorator test, run:

tox -e pytest -- -k testLoadModelDecorator

Legacy unittest Tests

Girder’s legacy automated tests are written with Python’s unittest framework and executed with CMake. All new tests should be written with pytest, but many important test cases are still covered only by unitest.

Note

Unless debugging code that is already coverered by a legacy test case, it may be more convenient to allow these tests to be run by Girder’s CI envionment, instead of configuring them locally.

To initialize the legacy tests, from the root girder repo, run:

mkdir ../girder-build
cd ../girder-build
cmake ../girder
make

You only need to do this once. From then on, whenever you want to run the tests, run:

cd girder-build
ctest

There are many ways to filter tests when running CTest or run the tests in parallel. For example, this command will run tests with name matches regex server_user with verbose output:

ctest -V -R server_user

Client Side Testing

Static Analysis Tests

To run static analysis tests on client side code, run from the top-level Girder directory:

npm i
npm run lint

Running the Tests with CTest

Using the same setup as above for the Server Side Tests, your environment will be set up. The client side tests and server side tests are both harnessed with CTest, so use the following commands to run both

cd girder-build
ctest

will run all of the tests, which include the client side tests. Our client tests use the Jasmine JS testing framework.

If you encounter errors, there is a chance you missed certain steps for setting up development dependencies. You could use ccmake to change CMake configuration. Or, it might be easier to recreate the environment from the beginning.

When running client side tests, if you try to SIGINT (ctrl+c) the CTest process, CTest won’t pass that signal down to the test processes for them to handle. This can result in orphaned python unittest processes and can prevent future runs of client tests. If you run a client side test and see an error message similar to IOError: Port 30015 not free on '0.0.0.0', then look for an existing process similar to /usr/bin/python3 -m unittest -v tests.web_client_test, kill the process, and then try your tests again.

Adding a New Client Side Test

To add a new client side test, add a new spec file in /girder/web_client/test/spec/, add a line referencing your spec file to /girder/tests/CMakeLists.txt using the add_web_client_test function, and then run in your build directory

cmake ../girder

before running your tests.

An example of a very simple client side test would be as follows

add_web_client_test(some_client_test "someSpec.js" PLUGIN "my_plugin")

The PLUGIN argument indicates that “my_plugin” is the owner of some_client_test, at the time of the test my_plugin and all of its dependencies will be loaded.

If additional plugins are needed for a specific test, that can be achieved using the ENABLEDPLUGINS argument

add_web_client_test(another_client_test "anotherSpec.js" PLUGIN "my_plugin" ENABLEDPLUGINS "my_plugin" "jobs")

Here ENABLEDPLUGINS ensures that my_plugin and the jobs plugin are loaded, along with their dependencies at the time of another_client_test.

Note

Core functionality shouldn’t depend on plugins being enabled, this test definition is more suitable for a plugin. Information for testing plugins can be found under Plugin Development.

You will find many useful methods for client side testing in the girderTest object defined at /girder/web_client/test/testUtils.js.

Test Coverage Reporting

When Girder’s full test suite is run in the CI environment, a test coverage report for both server and client code is generated and uploaded to Codecov. This may be viewed online at any time.

Initializing the Database for a Test

Note

This functionality has not yet been ported to our pytest tests.

When running tests in Girder, the database will initially be empty. Often times, you want to be able to start the test with the database in a particular state. To avoid repetitive initialization code, Girder provides a way to import a folder hierarchy from the file system using a simple initialization file. This file is in YAML (or JSON) format and provides a list of objects to insert into the database before executing your test. A typical example of this format is as follows

---
users:
  - login: 'admin'
    password: 'password'
    firstName: 'First'
    lastName: 'Last'
    email: 'admin@girder.test'
    admin: true
    import: 'files/user'

collections:
  - name: 'My collection'
    public: true
    creator: 'admin'
    import: 'files/collection'

This will create one admin user and a public collection owned by that user. Both the generated user and collection objects will contain folders imported from the file system. Relative paths provided by the import key will be resolved relative to the location of the YAML file on disk. You can also describe the full hierarchy in the YAML file itself for more complicated use cases. See the test spec in tests/cases/setup_database_test.yml for a more complete example.

Note

When importing from a local path into a user or collection, files directly under that path are ignored because items can be only inserted under folders.

To use the initialization mechanism, you should add the YAML file next to your test file. For example, if your test is defined in tests/cases/my_test.py, then the initialization spec should go in tests/cases/my_test.yml. This file will be automatically detected and loaded before executing your test code. This is true for both python and javascript tests added in core or inside plugins.

The python module setup_database.py that generates the database can also be run standalone to help in development. To use it, you should point girder to an empty database

GIRDER_MONGO_URI='mongodb://127.0.0.1:27017/mytest' python tests/setup_database.py tests/test_database/spec.yml

You can browse the result in Girder by running

GIRDER_MONGO_URI='mongodb://127.0.0.1:27017/mytest' girder serve

Note

The setup_database module is meant to provision fixures for tests only. If you want to provision a Girder instance for deployment, see the Girder ansible client.

Code Review

Contributions to Girder are done via pull requests with a core developer approving the PR with GitHub review system. At this point, the topic branch can be merged to master. This is meant to be a simple, low-friction process; however, code review is very important. It should be done carefully and not taken lightly. Thorough code review is a crucial part of developing quality software. When performing a code review, ask the following:

1. Is the continuous integration server happy with this?
2. Are there tests for this feature or bug fix?
3. Is this documented (for users and/or developers)?
4. Are the commits modular with good notes?
5. Will this merge cleanly?
6. Does this break backward compatibility? Is that okay?
7. What are the security implications of this change? Does this open Girder up to any vulnerabilities (XSS, CSRF, DB Injection, etc)?

Third-Party Libraries

Girder’s standard procedure is to use a tool like piprot to check for out-of-date third-party library requirements on a quarterly basis (typically near the dates of the solstices and equinoxes). Library packages should generally be upgraded to the latest released version, except when:

• Doing so would introduce any new unfixable bugs or regressions.
• Other closely-affiliated projects (e.g. Romanesco, Minerva) use the same library and the other project cannot also feasibly be upgraded simultaneously.
• The library has undergone a major API change, and development resources do not permit updating Girder accordingly or Girder exposes parts of the library as members of Girder’s API surface (e.g. CherryPy) and upgrading would cause incompatible API changes to be exposed. In this case, the library should still be upgraded to the highest non-breaking version that is available at the time.

Note

In the event that a security vulnerability is discovered in a third-party library used by Girder, the library must be upgraded to patch the vulnerability immediately and without regard to the aforementioned exceptions. However, attempts should still be made to maintain API compatibility via monkey patching, wrapper classes, etc.

Modifying core web client libraries

Web client libraries in Girder core are managed via npm. When a new npm package is required, or an existing package is upgraded, the following should be done:

1. Ensure that you are using a development environment with version >=5.6 of npm installed:

   npm install -g 'npm@>=5.6'
   

2. Update girder/web_client/package.json.template or girder/web_client/src/package.json to add a new abstract specifier for the package:

• Packages that are bundled into the web client must be listed under the dependencies field of girder/web_client/src/package.json and should generally use the tilde range to specify versions.
• Packages that are part of the build or testing process should be listed under either the dependencies or devDependencies fields of girder/web_client/package.json.template and should generally use the caret range to specify versions.

If updating npm libraries related to linting or documentation, you should instead modify the top-level package.json file, run npm update, then commit the modified files.

Creating a new release

Girder releases are uploaded to PyPI for easy installation via pip. Each time a pull request is merged to master, an incremental “dev” release is created during CI as a pre-release package and published to PyPI, making it easy for downstreams to install bleeding edge packages without needing to clone the Girder repository.

The major, minor, and patch version are inferred automatically using setuptools-scm based on the latest git tag. Hence, creating a new release is as simple as pushing a new git tag. For example, from the target commit, you could simply run:

git tag v4.5.6
git push --tags

That will trigger CircleCI to run, and if all tests pass, the 4.5.6 release will be uploaded to PyPI.

Plugin Development

The capabilities of Girder can be extended via plugins. The plugin framework is designed to allow Girder to be as flexible as possible, on both the client and server sides.

A plugin is self-contained python package with an optional “web_client” directory containing the client side extension. In this document, we describe the minimal components necessary to create and distribute a python package containing a Girder plugin for basic use cases. For a more detailed guide, see python’s own packaging documentation and tutorial.

Quick Start

We maintain a Cookiecutter template to help developers get started with their own Girder plugin. To generate your own plugin using this template, install the cookiecutter package

pip install cookiecutter

and run this command

cookiecutter gh:girder/cookiecutter-girder-plugin

It will ask you a few questions to customize the plugin. For details on these options see the README in the template’s git repository.

Example Plugin

We’ll use a contrived example to demonstrate the capabilities and components of a plugin. Our plugin will be called cats.

mkdir cats

The first thing we should do is create a setup.py file describing the package we are going to create.

from setuptools import setup, find_packages

setup(
    name='girder_cats',
    version='1.0.0',
    description='A contrived example of a Girder plugin.',
    author='Plugin Development, Inc.',
    author_email='plugin-developer@girder.test',
    url='https://my-plugin-documentation-page.com/cats',
    license='Apache 2.0',
    classifiers=[
      'Development Status :: 5 - Production/Stable',
      'Environment :: Web Environment',
      'License :: OSI Approved :: Apache Software License'
    ],
    include_package_data=True,
    packages=find_packages(exclude=['plugin_tests']),
    zip_safe=False,
    setup_requires=['setuptools-git'],
    install_requires=['girder>=3', 'girder-jobs'],
    entry_points={
        'girder.plugin': [ 'cats = girder_cats:CatsPlugin' ]
    }
)

Many of these values are metadata associated with a standard python package. See also the list of available classifiers that can be added to aid in discoverability of your package. Several arguments in this example are specific to a Girder plugin. These are as follows:

include_package_data=True

This tells python to include data files in addition to python modules found in your repository. This is necessary to ensure the web client assets are included in the package distribution. See the setuptools documentation for details.

setup_requires=['setuptools-git']

This works with the include_package_data option to automatically include all non-python files that are checked into your git repository. Alternatively, you can generate a MANIFEST.in for more fine-grained control over which files are included.

packages=find_packages(exclude=['plugin_tests'])

This will include all python “packages” found inside the local path in the distribution with the exception of the plugin_tests directory. If any other python modules or testing files are not desired in the distributed bundle, they should be added to this list.

zip_safe=False

Unless this flag is provided, the python installer will install the package (and its non-python data files) as a python “egg”. Girder plugins including web extensions do not support this feature.

install_requires=['girder>=3', 'girder_jobs']

This tells the installer that Girder of at least version 3 is required for this package to function. When installing with pip, Girder will be automatically installed from pypi if it is not already installed. Any additional dependencies (including other Girder plugins) should be added to this list as well.

entry_points={'girder.plugin': [ 'cats = girder_cats.CatsPlugin' ]}

This is the piece that registers your plugin with Girder’s application. When Girder starts up, it queries this entrypoint (girder.plugin) for all registered plugins. Here the name cats is the internal name registered to this plugin. The value girder_cats.CatsPlugin is an import path resolving to a class that is expected to derive from girder.plugin.GirderPlugin. See below for an example of how to define this object.

Defining a Plugin Descriptor Class

Once you have created your setup.py file, you should begin to define the python package that will contain your plugin. In our case, we will name the python package girder_cats so that it is a unique name on pypi

mkdir girder_cats
touch girder_cats/__init__.py

The base class at girder.plugin.GirderPlugin defines the interface between Girder and its plugins. For advanced requirements, plugin authors can override the properties defined on this class, but for most use cases inserting the following into the top level __init__.py will suffice.

from girder.plugin import getPlugin, GirderPlugin

class CatsPlugin(GirderPlugin):
    DISPLAY_NAME = 'Cats in Girder'
    CLIENT_SOURCE_PATH = 'web_client'

    def load(self, info):
        getPlugin('jobs').load(info)
        # attach endpoints, listen to events, etc...

Girder inspects attributes on this class for several pieces of metadata. Most of this metadata is automatically determined from the package-level metadata defined in your setup.py file. The additional attributes defined on this class instance provide the following:

DISPLAY_NAME

This provides Girder with a “user facing” name, e.g. a short description of the plugin not limited by the tokenization rules inherent in the “entrypoint name”. By default, the entrypoint name will be used if none is provided here.

CLIENT_SOURCE_PATH

If your plugin contains a web client extension, you need to set this property to a path containing an npm package. The path is always interpreted relative the python package install path.

Other optional attributes are defined on this class for more advanced use cases, see the class documentation at girder.plugin.GirderPlugin for details.

Adding a new route to the web API

If you want to add a new route to an existing core resource type, just call the route() function on the existing resource type. For example, to add a route for GET /item/:id/cat to the system,

from girder.api import access
from girder.api.rest import boundHandler

@access.public
@boundHandler
def myHandler(self, id, params):
    self.requireParams('cat', params)

    return {
       'itemId': id,
       'cat': params['cat']
    }

You can then attach this route to Girder in your plugin’s load method

from girder.plugin import GirderPlugin
class CatsPlugin(GirderPlugin)
  def load(self, info):
      info['apiRoot'].item.route('GET', (':id', 'cat'), myHandler)

You should always add an access decorator to your handler function or method to indicate who can call the new route. The decorator is one of @access.admin (only administrators can call this endpoint), @access.user (any user who is logged in can call the endpoint), or @access.public (any client can call the endpoint).

In the above example, the girder.api.rest.boundHandler decorator is used to make the unbound method myHandler behave as though it is a member method of a girder.api.rest.Resource instance, which enables convenient access to methods like self.requireParams.

If you do not add an access decorator, a warning message appears: WARNING: No access level specified for route GET item/:id/cat. The access will default to being restricted to administrators.

When you start the server, you may notice a warning message appears: WARNING: No description docs present for route GET item/:id/cat. You can add self-describing API documentation to your route using the autoDescribeRoute decorator and girder.api.describe.Description class as in the following example:

from girder.api.describe import Description, autoDescribeRoute
from girder.api import access

@access.public
@autoDescribeRoute(
    Description('Retrieve the cat for a given item.')
    .param('id', 'The item ID', paramType='path')
    .param('cat', 'The cat value.', required=False)
    .errorResponse())
def myHandler(id, cat):
    return {
       'itemId': id,
       'cat': cat
    }

That will make your route automatically appear in the Swagger documentation and will allow users to interact with it via that UI. See the RESTful API docs for more information about the Swagger page. In addition, the autoDescribeRoute decorator handles a lot of the validation and type coercion for you, with the benefit of ensuring that the documentation of the endpoint inputs matches their actual behavior. Documented parameters will be sent to the method as kwargs (so the order you declare them in the header doesn’t matter). Any additional parameters that were passed but not listed in the Description object will be contained in the params kwarg as a dictionary, if that parameter is present. The validation of required parameters, coercion to the correct data type, and setting default values is all handled automatically for you based on the parameter descriptions in the Description object passed. Two special methods of the Description object can be used for additional behavior control: girder.api.describe.Description.modelParam() and girder.api.describe.Description.jsonParam().

The modelParam method is used to convert parameters passed in as IDs to the model document corresponding to those IDs, and also can perform access checks to ensure that the user calling the endpoint has the requisite access level on the resource. For example, we can convert the above handler to use it:

@access.public
@autoDescribeRoute(
    Description('Retrieve the cat for a given item.')
    .modelParam('id', 'The item ID', model='item', level=AccessType.READ)
    .param('cat', 'The cat value.', required=False)
    .errorResponse())
def myHandler(item, cat, params):
    return {
       'item': item,
       'cat': cat
    }

The jsonParam method can be used to indicate that a parameter should be parsed as a JSON string into the corresponding python value and passed as such.

If you are creating routes that you explicitly do not wish to be exposed in the Swagger documentation for whatever reason, you can pass hide=True to the autoDescribeRoute decorator, and no warning will appear.

@autoDescribeRoute(Description(...), hide=True)

Adding a new resource type to the web API

Perhaps for our use case we determine that cat should be its own resource type rather than being referenced via the item resource. If we wish to add a new resource type entirely, it will look much like one of the core resource classes, and we can add it to the API in the load() method.

from girder.api.rest import Resource

class Cat(Resource):
    def __init__(self):
        super().__init__()
        self.resourceName = 'cat'

        self.route('GET', (), self.findCat)
        self.route('GET', (':id',), self.getCat)
        self.route('POST', (), self.createCat)
        self.route('PUT', (':id',), self.updateCat)
        self.route('DELETE', (':id',), self.deleteCat)

    def getCat(self, id, params):
        ...

As done when extending an existing resource, this should be mounted into Girder’s API inside your plugin’s load method:

from girder.plugin import GirderPlugin
class CatsPlugin(GirderPlugin)
    def load(self, info):
        info['apiRoot'].cat = Cat()

Adding a prefix to an API

It is possible to provide a prefix to your API, allowing associated endpoints to be grouped together. This is done by creating a prefix when mounting the resource. Note that resourceName is not provided as the resource name is also derived from the mount location.

from girder.api.rest import Resource, Prefix
from girder.plugin import GirderPlugin

class Cat(Resource):
    def __init__(self):
        super().__init__()

        self.route('GET', (), self.findCat)
        self.route('GET', (':id',), self.getCat)
        self.route('POST', (), self.createCat)
        self.route('PUT', (':id',), self.updateCat)
        self.route('DELETE', (':id',), self.deleteCat)

    def getCat(self, id, params):
        ...

class CatsPlugin(GirderPlugin):
    def load(self, info):
        info['apiRoot'].meow = Prefix()
        info['apiRoot'].meow.cat = Cat()

The endpoints are now mounted at meow/cat/

Adding a new model type in your plugin

Most of the time, if you add a new resource type in your plugin, you’ll have a Model class backing it. These model classes work just like the core model classes as described in the Models section. If you need to use the ModelImporter class with your model type, you will need to explicitly register the model type to a string, e.g.

from girder.plugin import GirderPlugin
from girder.utility.model_importer import ModelImporter
from .models.cat import Cat

class CatsPlugin(GirderPlugin):
    def load(self, info):
        ModelImporter.registerModel('cat', Cat, plugin='cats')

Adding custom access flags

Girder core provides a way to assign a permission level (read, write, and own) to data in the hierarchy to individual users or groups. In addition to this level, users and groups can also be granted special access flags on resources in the hierarchy. If you want to expose a new access flag on data, have your plugin globally register the flag in the system:

from girder.constants import registerAccessFlag

registerAccessFlag(key='cats.feed', name='Feed cats', description='Allows users to feed cats')

When your plugin is installed, a new checkbox will automatically appear in the access control dialog allowing resource owners to specify what users and groups are allowed to feed cats (assuming cats are represented by data in the hierarchy). Additionally, if your resource is public, you will also be able to configure which access flags are available to the public. If your plugin exposes another endpoint, say POST cat/{id}/food, inside that route handler, you can call requireAccessFlags, e.g.:

from girder_cat import Cat

@access.user
@autoDescribeRoute(
    Description('Feed a cat')
    .modelParam('id', 'ID of the cat', model=Cat, level=AccessType.WRITE)
)
def feedCats(self, cat, params):
    Cat().requireAccessFlags(item, user=getCurrentUser(), flags='cats.feed')

    # Feed the cats ...

That will throw an AccessException if the user does not possess the specified access flag(s) on the given resource. You can equivalently use the Description.modelParam method using autoDescribeRoute, passing a requiredFlags parameter, e.g.:

@access.user
@autoDescribeRoute(
    Description('Feed a cat')
    .modelParam('id', 'ID of the cat', model=Cat, level=AccessType.WRITE,
                requiredFlags='cats.feed')
)
def feedCats(self, cat, params):
    # Feed the cats ...

Normally, anyone with ownership access on the resource will be allowed to enable the flag on their resources. If instead you want to make it so that only site administrators can enable your custom access flag, pass admin=True when registering the flag, e.g.

registerAccessFlag(key='cats.feed', name='Feed cats', admin=True)

We cannot prescribe exactly how access flags should be used; Girder core does not expose any on its own, and the sorts of policies that they will enforce will be entirely defined by the logic of your plugin.

The events system

In addition to being able to augment the core API as described above, the core system fires a known set of events that plugins can bind to and handle as they wish.

In the most general sense, the events framework is simply a way of binding arbitrary events with handlers. The events are identified by a unique string that can be used to bind handlers to them. For example, if the following logic is executed by your plugin at startup time,

from girder import events

def handler(event):
    print event.info

events.bind('some_event', 'my_handler', handler)

And then during runtime the following code executes:

events.trigger('some_event', info='hello')

Then hello would be printed to the console at that time. More information can be found in the API documentation for Events.

There are a specific set of known events that are fired from the core system. Plugins should bind to these events at load time. The semantics of these events are enumerated below.

• Before REST call

Whenever a REST API route is called, just before executing its default handler, plugins will have an opportunity to execute code or conditionally override the default behavior using preventDefault and addResponse. The identifiers for these events are of the form rest.get.item/:id.before. They receive the same kwargs as the default route handler in the event’s info.

Since handlers of this event run prior to the normal access level check of the underlying route handler, they are bound by the same access level rules as route handlers; they must be decorated by one of the functions in girder.api.access. If you do not decorate them with one, they will default to requiring administrator access. This is to prevent accidental reduction of security by plugin developers. You may change the access level of the route in your handler, but you will need to do so explicitly by declaring a different decorator than the underlying route handler.

• After REST call

Just like the before REST call event, but this is fired after the default handler has already executed and returned its value. That return value is also passed in the event.info for possible alteration by the receiving handler. The identifier for this event is, e.g., rest.get.item/:id.after.

You may alter the existing return value, for example adding an additional property

event.info['returnVal']['myProperty'] = 'myPropertyValue'

or override it completely using preventDefault and addResponse on the event

event.addResponse(myReplacementResponse)
event.preventDefault()

• Before model save

You can receive an event each time a document of a specific resource type is saved. For example, you can bind to model.folder.save if you wish to perform logic each time a folder is saved to the database. You can use preventDefault on the passed event if you wish for the normal saving logic not to be performed.

• After model creation

You can receive an event after a resource of a specific type is created and saved to the database. This is sent immediately before the after-save event, but only occurs upon creation of a new document. You cannot prevent any default actions with this hook. The format of the event name is, e.g. model.folder.save.created.

• After model save

You can also receive an event after a resource of a specific type is saved to the database. This is useful if your handler needs to know the _id field of the document. You cannot prevent any default actions with this hook. The format of the event name is, e.g. model.folder.save.after.

• Before model deletion

Triggered each time a model is about to be deleted. You can bind to this via e.g., model.folder.remove and optionally preventDefault on the event.

• During model copy

Some models have a custom copy method (folder uses copyFolder, item uses copyItem). When a model is copied, after the initial record is created, but before associated models are copied, a copy.prepare event is sent, e.g. model.folder.copy.prepare. The event handler is passed a tuple of ((original model document), (copied model document)). If the copied model is altered, the handler should save it without triggering events.

When the copy is fully complete, and copy.after event is sent, e.g. model.folder.copy.after.

• Override model validation

You can also override or augment the default validate methods for a core model type. Like the normal validation, you should raise a ValidationException for failure cases, and you can also preventDefault if you wish for the normal validation procedure not to be executed. The identifier for these events is, e.g., model.user.validate.

• Override user authentication

If you want to override or augment the normal user authentication process in your plugin, bind to the auth.user.get event. If your plugin can successfully authenticate the user, it should perform the logic it needs and then preventDefault on the event and addResponse containing the authenticated user document.

• Before file upload

This event is triggered as an upload is being initialized. The event model.upload.assetstore is sent before the model.upload.save event. The event information is a dictionary containing model and resource with the resource model type and resource document of the upload parent. For new uploads, the model type will be either item or folder. When the contents of a file are being replaced, this will be a file. To change from the current assetstore, add an assetstore key to the event information dictionary that contains an assetstore model document.

• Just before a file upload completes

The event model.upload.finalize after the upload is completed but before the new file is saved. This can be used if the file needs to be altered or the upload should be cancelled at the last moment.

• On file upload

This event is always triggered asynchronously and is fired after a file has been uploaded. The file document that was created is passed in the event info. You can bind to this event using the identifier data.process.

• Before file move

The event model.upload.movefile is triggered when a file is about to be moved from one assetstore to another. The event information is a dictionary containing file and assetstore with the current file document and the target assetstore document. If preventDefault is called, the move will be cancelled.

Note

If you anticipate your plugin being used as a dependency by other plugins, and want to potentially alert them of your own events, it can be worthwhile to trigger your own events from within the plugin. If you do that, the identifiers for those events should begin with the name of your plugin, e.g., events.trigger('cats.something_happened', info='foo')

• User login

The event model.user.authenticate is fired when a user is attempting to login via a username and password. This allows alternative authentication modes to be used instead of core, or prior to attempting core authentication. The event info contains two keys, “login” and “password”.

Customizing the Swagger page

To customize text on the Swagger page, create a Mako template file that inherits from the base template and overrides one or more blocks. For example, plugins/cats/server/custom_api_docs.mako:

<%inherit file="${context.get('baseTemplateFilename')}"/>

<%block name="docsHeader">
  <span>Cat programming interface</span>
</%block>

<%block name="docsBody">
  <p>Manage your cats using the resources below.</p>
</%block>

Install the custom template in the plugin’s load function:

import os
from girder.plugin import GirderPlugin

PLUGIN_PATH = os.path.dirname(__file__)
class CustomTemplatePlugin(GirderPlugin):
    def load(self, info):
        # Initially, the value of info['apiRoot'].templateFilename is
        # 'api_docs.mako'. Because custom_api_docs.mako inherits from this
        # base template, pass 'api_docs.mako' in the variable that the
        # <%inherit> directive references.
        baseTemplateFilename = info['apiRoot'].templateFilename
        info['apiRoot'].updateHtmlVars({
            'baseTemplateFilename': baseTemplateFilename
        })

        # Set the path to the custom template
        templatePath = os.path.join(PLUGIN_PATH, 'custom_api_docs.mako')
        info['apiRoot'].setTemplatePath(templatePath)

Extending the Client-Side Application

The web client may be extended independently of the server side. Plugins may import Pug templates, Stylus files, and JavaScript files into the application.

All of your plugin’s extensions to the web client must live in a directory inside of your python package. By convention, this is in a directory called web_client.

cd girder_cats ; mkdir web_client

When present, this directory must contain a valid npm package, which includes a package.json file. (See the npm documentation for details.) What follows is a typical npm package file for a Girder client side extension:

{
    "name": "@girder/cats",
    "version": "1.0.0",
    "peerDependencies": {
        "@girder/core": "*",
        "@girder/jobs": "*"
    },
    "dependencies": {
        "othermodule": "^1.2.4"
    },
    "girderPlugin": {
        "name": "cats",
        "main": "./main.js",
        "dependencies": ["jobs"],
        "webpack": "webpack.helper"
    }
}

In addition to the standard package.json properties, Girder plugins must also define a girderPlugin object to register themselves with Girder’s client build system. The important keys in the object are as follows:

name

This must be exactly the entrypoint name registered in your setup.py file.

main

This is the entrypoint into your plugin on the client. All runtime initialization should occur from here.

dependencies

This is an array of entrypoint names that your plugin depends on. Specifying this explicitly here is what allows Girder’s client build system to build the plugin assets in the correct order.

webpack

This is an optional property whose value is a node module that exports a function that can make arbitrary modification the webpack config used to build the plugin bundle.

By default, Girder includes loaders for pug, stylus, css, fonts, and images in all paths. For javascript inside the plugin, the code is transpiled through babel using babel-preset-env; however, this is not done for dependencies resolved inside node_modules. This option makes it easy to include additional transpilation rules. For an example of this in use, see the built in dicom_viewer plugin.

Core Girder code can be imported relative to the path @girder/core, for example import View from '@girder/core/views/View';. The entry point defined in your “main” file will be loaded into the browser after Girder’s core library, but before the application is initialized.

JavaScript extension capabilities

Plugins may bind to any of the normal events triggered by core via a global events object that can be imported like so:

import events from '@girder/core/events';

...

this.listenTo(events, 'g:event_name', () => { do.something(); });

This will accommodate certain events, such as before and after the application is initially loaded, and when a user logs in or out, but most of the time plugins will augment the core system using the power of JavaScript rather than the explicit events framework. One of the most common use cases for plugins is to execute some code either before or after one of the core model or view functions is executed. In an object-oriented language, this would be a simple matter of extending the core class and making a call to the parent method. The prototypal nature of JavaScript makes that pattern impossible; instead, we’ll use a slightly less straightforward but equally powerful mechanism. This is best demonstrated by example. Let’s say we want to execute some code any time the core HierarchyWidget is rendered, for instance to inject some additional elements into the view. We use Girder’s wrap utility function to wrap the method of the core prototype with our own function.

import HierarchyWidget from '@girder/core/views/widgets/HierarchyWidget';
import { wrap } from '@girder/core/utilities/PluginUtils';

// Import our template file from our plugin using a relative path
import myTemplate from './templates/hierachyWidgetExtension.pug';

// CSS files pertaining to this view should be imported as a side-effect
import './stylesheets/hierarchyWidgetExtension.styl';

wrap(HierarchyWidget, 'render', function (render) {
    // Call the underlying render function that we are wrapping
    render.call(this);

    // Add a link just below the widget using our custom template
    this.$('.g-hierarchy-widget').after(myTemplate());
});

Notice that instead of simply calling render(), we call render.call(this). That is important, as otherwise the value of this will not be set properly in the wrapped function.

Now that we have added the link to the core view, we can bind an event handler to it to make it functional:

HierarchyWidget.prototype.events['click a.cat-link'] = () => {
    alert('meow!');
};

This demonstrates one simple use case for client plugins, but using these same techniques, you should be able to do almost anything to change the core application as you need.

JavaScript events

The JavaScript client handles notifications from the server and Backbone events in client-specific code. The server notifications originate on the server and can be monitored by both the server’s Python code and the client’s JavaScript code. The client Backbone events are solely within the web client, and do not get transmitted to the server.

If the connection to the server is interrupted, the client will not receive server events. Periodically, the client will attempt to reconnect to the server to resume handling events. Similarly, if client’s browser tab is placed in the background for a long enough period of time, the connection that listens for server events will be stopped to prevent excessive resource use. When the browser’s tab regains focus, the client will once again receive server events.

When the connection to the server’s event stream is interrupted, a g:eventStream.stop Backbone event is triggered on the EventStream object. When the server is once more sending events, it first sends a g:eventStream.start event. Clients can listen to these events and refresh necessary components to ensure that data is current.

Setting an empty layout for a route

If you have a route in your plugin that you would like to have an empty layout, meaning that the Girder header, nav bar, and footer are hidden and the Girder body is evenly padded and displayed, you can specify an empty layout in the navigateTo event trigger.

As an example, say your plugin wanted a frontPage route for a Collection which would display the Collection with only the Girder body shown, you could add the following route to your plugin.

import events from '@girder/core/events';
import router from '@girder/core/router';
import { Layout } from '@girder/core/constants';
import CollectionModel from '@girder/core/models/CollectionModel';
import CollectionView from '@girder/core/views/body/CollectionView';

router.route('collection/:id/frontPage', 'collectionFrontPage', function (collectionId, params) {
    var collection = new CollectionModel();
    collection.set({
        _id: collectionId
    }).on('g:fetched', function () {
        events.trigger('g:navigateTo', CollectionView, _.extend({
            collection: collection
        }, params || {}), {layout: Layout.EMPTY});
    }, this).on('g:error', function () {
        router.navigate('/collections', {trigger: true});
    }, this).fetch();
});

Using another plugin inside a plugin

Girder plugins can use and extend other plugins as well. To do this, you need to add and load the other plugin explicitly so that it installs and initializes automatically. There are a number of places that the dependency relationship needs to be specified.

• Python package

If you directly rely on another plugin for any reason, you should always add the dependency to your plugin’s setup.py file. This is done in the same way all python dependencies are specified and will ensure that all the required packages are installed when you plugin is “pip installed”.

# setup.py depending on girder-jobs and girder-homepage
setup(
    name='girder-example-plugin',
    # ...
    install_requires=['girder-jobs', 'girder-homepage']
)

• Plugin loading

By default, Girder does not load its installed plugins in a deterministic order. If your plugin depends on other Girder being loaded prior to itself, your plugin must explicitly load the other dependant plugins during your plugin’s own loading.

Girder will guarantee that a given plugin is actually loaded only once, so multiple calls to load another plugin are safe and have no effect. Finally, it is possible to check for the existence of another plugin before loading it or performing other configuration, to support optional dependencies.

from girder.plugin import getPlugin, GirderPlugin
# An example of loading dependent plugins
class ExamplePlugin(GirderPlugin)
    def load(self, info):
        getPlugin('jobs').load(info)
        homepagePlugin = getPlugin('homepage')
        if homepagePlugin:
            # Optional dependency
            homepagePlugin.load(info)
        # ...

• Javascript client

If your plugin contains a javascript client and it imports code from another plugin, then you need to add this dependency relationship to your web client package.json file. If you depend on another plugin, but do not directly import code from the other package in you javascript code, then this is not necessary.

// package.json depending on "girder-jobs"
{
    "name": "@girder/example",
    "peerDependencies": {
        "@girder/core": "*",

        // This ensures that `import '@girder/jobs'` can be resolved.
        "@girder/jobs": "*"
    },

    "girderPlugin": {
        "name": "example",
        "main": "./main.js",

        // This ensures that "girder-jobs" is built before this plugin.
        "dependencies": ["jobs"]
    }
}

Automated testing for plugins

We recommend using pytest to create automated tests for your plugin code. The core Girder development team maintains the pytest-girder package, which contains several useful fixtures and other utilities that make testing Girder plugins easier.

Example

This example shows the use of the server fixture, which spins up the Girder server and allows requests to be made against its REST API.

from girder_cats.models import Cat
import pytest
from pytest_girder.assertions import assertStatusOk

@pytest.mark.plugin('cats')    # Makes sure the cats plugin will load for this test
def testCatCreation(server):
    resp = server.request('/cat', method='POST', params={
        'name': 'Helga',
        'age': 4
    })
    assertStatusOk(resp)
    records = Cat().find()
    assert records.count() == 1
    assert records[0]['name'] == 'Helga'

Developer Cookbook

This cookbook consists of a set of examples of common tasks that developers may encounter when developing Girder applications.

Client cookbook

The following examples are for common tasks that would be performed by a Girder client application.

Authenticating to the web API

Clients can make authenticated web API calls by passing a secure temporary token with their requests. Tokens are obtained via the login process; the standard login process requires the client to make an HTTP GET request to the /api/v1/user/authentication route, using HTTP Basic Auth to pass the user credentials. For example, for a user with login “john” and password “hello”, first base-64 encode the string "john:hello" which yields "am9objpoZWxsbw==". Then take the base-64 encoded value and pass it via the Authorization header:

Authorization: Basic am9objpoZWxsbw==

If the username and password are correct, you will receive a 200 status code and a JSON document from which you can extract the authentication token, e.g.:

{
  "authToken": {
    "token": "urXQSHO8aF6cLB5si0Ch0WCiblvW1m8YSFylMH9eqN1Mt9KvWUnghVDKQy545ZeA",
    "expires": "2015-04-11 00:06:14.598570"
  },
  "message": "Login succeeded.",
  "user": {
    ...
  }
}

The authToken.token string is the token value you should pass in subsequent API calls, which should either be passed as the token parameter in the query or form parameters, or as the value of a custom HTTP header with the key Girder-Token, e.g.

Girder-Token: urXQSHO8aF6cLB5si0Ch0WCiblvW1m8YSFylMH9eqN1Mt9KvWUnghVDKQy545ZeA

Note

When logging in, the token is also sent to the client in a Cookie header so that web-based clients can persist its value conveniently for its duration. However, for security reasons, merely passing the cookie value back is not sufficient for authentication.

Note

If you are using Girder’s JavaScript web client library in a CORS environment, be sure to set girder.corsAuth = true; in your application prior to calling girder.login. This will allow users’ login sessions to be saved on the origin site’s cookie.

Upload a file

If you are using the Girder javascript client library, you can simply call the upload method of the @girder/core/models/FileModel. The first argument is the parent model object (an ItemModel or FolderModel instance) to upload into, and the second is a browser File object that was selected via a file input element. You can bind to several events of that model, as in the example below.

import FileModel from '@girder/core/models/FileModel';

var fileModel = new FileModel();
fileModel.on('g:upload.complete', function () {
    // Called when the upload finishes
}).on('g:upload.chunkSent', function (info) {
    // Called on each chunk being sent
}).on('g:upload.progress', function (info) {
    // Called regularly with progress updates
}).on('g:upload.error', function (info) {
    // Called if an upload fails partway through sending the data
}).on('g:upload.errorStarting', function (info) {
    // Called if an upload fails to start
});
fileModel.upload(parentFolder, fileObject);

If you don’t feel like making your own upload interface, you can simply use the @girder/core/views/widgets/UploadWidget to provide a nice GUI interface for uploading. It will prompt the user to drag and drop or browse for files, and then shows a current and overall progress bar and also provides controls for resuming a failed upload.

Using the Girder upload widget in a custom app

Your custom javascript application can easily reuse the existing upload widget provided in the Girder javascript library if you don’t want to write your own upload view. This can save time spent duplicating functionality, since the upload widget provides current and overall progress bars, file displays, a drag-and-droppable file selection button, resume behavior in failure conditions, and customizable hooks for various stages of the upload process.

The default behavior of the upload widget is to display as a modal dialog, but many users will want to simply embed it underneath a normal DOM element flow. The look and behavior of the widget can be customized when the widget is instantiated by passing in options like so:

import UploadWidget from '@girder/core/views/widgets/UploadWidget';

new UploadWidget({
    option: value,
    ...
});

The following options are not required, but may be used to modify the behavior of the widget:

• [parent] - If the parent object is known when instantiating this upload widget, pass the object here.
• [parentType=folder] - If the parent type is known when instantiating this upload widget, pass the object here. Otherwise set noParent: true and set it later, prior to starting the upload.
• [noParent=false] - If the parent object being uploaded into is not known at the time of widget instantiation, pass noParent: true. Callers must ensure that the parent is set by the time uploadNextFile() actually gets called.
• [title="Upload files"] - Title for the widget. This is highly recommended when rendering as a modal dialog. To disable rendering of the title, simply pass a falsy object.
• [modal=true] - This widget normally renders as a modal dialog. Pass modal: false to disable the modal behavior and simply render underneath a parent element.
• [overrideStart=false] - Some callers will want to hook into the pressing of the start upload button and add their own logic prior to actually sending the files. To do so, set overrideStart: true and bind to the g:uploadStarted event of this widget. The caller is then responsible for calling uploadNextFile() on the widget when they have completed their actions and are ready to actually send the files.

For general documentation on embedding Girder widgets in a custom application, see the section on client development.

Server cookbook

The following examples refer to tasks that are executed by the Girder application server.

Creating a REST route

The process of creating new REST resources and routes is documented here.

The API docs of the route method can be found here.

Loading a resource by its ID

This is a fundamental element of many REST operations; they receive a parameter representing a resource’s unique ID, and want to load the corresponding resource from that ID. This behavior is known as model loading. As a brief example, if we had the ID of a folder within our REST route handler, and wanted to load its corresponding document from the database, it would look like:

from girder.models.folder import Folder
folder = Folder().load(theFolderId, user=self.getCurrentUser(), level=AccessType.READ)

The load method of each model class takes the resource’s unique ID as its first argument (this is the _id field in the documents). For access controlled models like the above example, it also requires the developer to specify which user is requesting the loading of the resource, and what access level is required on the resource. If the ID passed in does not correspond to a record in the database, None is returned.

Sometimes models need to be loaded outside the context of being requested by a specific user, and in those cases the force flag should be used:

Folder().load(theFolderId, force=True)

Send a raw or streaming HTTP response body

For consistency, the default behavior of a REST endpoint in Girder is to take the return value of the route handler and encode it in the format specified by the client in the Accepts header, usually application/json. However, in some cases you may want to force your endpoint to send a raw response body back to the client.

If you want to send a raw response, you can simply decorate your route handler with the girder.api.rest.rawResponse decorator, or call girder.api.rest.setRawResponse() within the body of the route handler. For example:

from girder.api import access, rest

@access.public
@rest.rawResponse
def rawExample(self, params):
    return 'raw string'

That will make the response body precisely the string raw string. If the data size being sent to the client is large or unbounded, you should use a streaming response.

If you want to send a streaming response, simply make your route handler return a generator function. A streaming response is automatically sent as a raw response. Developers have full control of the buffer size of the streamed response body; each time you yield data in your generator function, the buffer will be flushed to the client. As a minimal example, the following route handler would flush 10 chunks to the client, and the full response body would be 0123456789.

from girder.api import access

@access.public
def streamingExample(self, params):
    def gen():
        for i in range(10):
            yield str(i)
    return gen

Serving a static file

If you are building a plugin that needs to serve up a static file from a path on disk, you can make use of the staticFile utility, as in the following example:

import os
from girder.utility.server import staticFile

def load(info):
    path = os.path.join(PLUGIN_ROOT_DIR, 'static', 'index.html')
    info['serverRoot'].static_route = staticFile(path)

The staticFile utility should be assigned to the route corresponding to where the static file should be served from.

Note

If a relative path is passed to staticFile, it will be interpreted relative to the current working directory, which may vary. If your static file resides within your plugin, it is recommended to use the special PLUGIN_ROOT_DIR property of your server module.

Sending Emails

Girder has a utility module that make it easy to send emails from the server. For the sake of maintainability and reusability of the email content itself, emails are stored as Mako templates in the girder/mail_templates directory. By convention, email templates should include _header.mako above and _footer.mako below the content. If you wish to send an email from some point within the application, you can use the utility functions within girder.utility.mail_utils, as in the example below:

from girder.utility import mail_utils

...

def my_email_sending_code():
    html = mail_utils.renderTemplate('myContentTemplate.mako', {
        'param1': 'foo',
        'param2': 'bar'
    })
    mail_utils.sendMail(subject='My mail from Girder', text=html, to=[email])

If you wish to send email from within a plugin, simply create a mail_templates directory within your plugin and register it inside your plugin’s load method as follows

from girder.utility import mail_utils
mail_utils.addTemplateDirectory(os.path.join(PLUGIN_ROOT_DIR, 'mail_templates'))

To avoid name collisions, convention dictates that mail templates within your plugin should be prefixed by your plugin name, e.g., my_plugin.my_template.mako.

If you want to send email to all of the site administrators, there is a convenience function for that:

mail_utils.sendMailToAdmins(subject='...', text='...')

Note

All emails are sent as rich text (text/html MIME type).

Logging a Message

Girder application servers maintain an error log and an information log and expose a utility module for sending events to them. Any 500 error that occurs during execution of a request will automatically be logged in the error log with a full stack trace. Also, any 403 error (meaning a user who is logged in but requests access to a resource that they don’t have permission to access) will also be logged automatically. All log messages automatically include a timestamp, so there is no need to add your own.

If you want to log your own custom error or info messages outside of those default behaviors, use the following examples:

from girder import logger

try:
    ...
except Exception:
    # Will log the most recent exception, including a traceback, request URL,
    # and remote IP address. Should only be called from within an exception handler.
    logger.exception('A descriptive message')

# Will log a message to the info log.
logger.info('Test')

Adding Automated Tests with CTest

Note

Girder is transitioning towards using pytest for its testing. The section below describes how to add automated tests using pytest.

The server side Python tests are run using unittest. All of the actual test cases are stored under tests/cases.

Adding to an Existing Test Case

If you want to add tests to an existing test case, just create a new function in the relevant TestCase class. The function name must start with test. If the existing test case has setUp or tearDown methods, be advised that those methods will be run before and after each of the test methods in the class.

Creating a New Test Case

To create an entirely new test case, create a new file in cases that ends with _test.py. To start off, put the following code in the module (with appropriate class name of course):

from .. import base

def setUpModule():
    base.startServer()

def tearDownModule():
    base.stopServer()

class MyTestCase(base.TestCase):

Note

If your test case does not need to communicate with the server, you do not need to call base.startServer() and base.stopServer() in the setUpModule() and tearDownModule() functions. Those functions are called once per module rather than once per test method.

Then, in the MyTestCase class, just add functions that start with test, and they will automatically be run by unittest.

Finally, you’ll need to register your test in the CMakeLists.txt file in the tests directory. Just add a line like the ones already there at the bottom. For example, if the test file you created was called thing_test.py, you would add:

add_python_test(thing)

Re-run CMake in the build directory, and then run CTest, and your test will be run.

Note

By default, add_python_test allows the test to be run in parallel with other tests, which is normally fine since each python test has its own assetstore space and its own mongo database, and the server is typically mocked rather than actually binding to its port. However, some tests (such as those that actually start the cherrypy server) should not be run concurrently with other tests that use the same resource. If you have such a test, use the RESOURCE_LOCKS argument to add_python_test. If your test requires the cherrypy server to bind to its port, declare that it locks the cherrypy resource. If it also makes use of the database, declare that it locks the mongo resource. For example:

add_python_test(my_test RESOURCE_LOCKS cherrypy mongo)

Adding Automated Tests with pytest

Test cases for pytest are stored in test/.

Provided Fixtures

All helper utilities for developing tests with pytest are stored in the pytest_girder package within Girder. A list of fixtures provided can be viewed by running pytest --fixtures from the Girder directory.

Adding to an Existing Test Case

If you want to add tests to an existing test case, just create a new function in the file. The function name must start with test.

Creating a New Test Case

To create an entirely new test, create a new file in test/ that starts with test_. To start off, put the following code in the module (with appropriate function name of course):

def testCase(server):
    pass

Note

If your test case does not need to communicate with the server, you do not need include server as an argument to your test. Refer to pytest fixtures for more information on using dependency injection in this manner.

Enabling a plugin inside a test

By default, pytest tests do not enable any plugins. You can decorate your test with the plugin mark to enable a plugin that installed into the python environment. For example,

@pytest.mark.plugin('jobs')
def testWithJobsEnabled(server):
    pass

You can also define a “test plugin” that will be injected into runtime environment without actually being installed. This is done by passing a class derived from GirderPlugin into the mark. For example,

from girder.plugin import GirderPlugin

class TestPlugin(GirderPlugin):
    def load(self, info):
        pass

@pytest.mark.plugin('test_plugin', TestPlugin)
def testWithTestPlugin(server):
    pass

Downloading External Data Artifacts for Test Cases

In some cases, it is necessary to perform a test on an artifact that is too big to store inside a repository. For tests such as these, Girder provides a way to link to test artifacts served at https://data.kitware.com (or any Girder instance with the hashsum_download plugin enabled) and have them automatically downloaded and cached during the build stage, while storing only the SHA512 hashsum of the test artifact in the source repository as a “key file”.

To add a new external artifact, first make an account at https://data.kitware.com and upload a publicly accessible file (ideally to the Girder collection, which you can request access to in your feature’s GitHub pull request). Then, while viewing the newly created item page, open the file info dialog for the test artifact (by clicking the “i” icon to “Show info”, next to the file link), then click the “SHA-512” field’s key icon to “Download key file”. Save and commit this downloaded key file to the tests/data/ directory of Girder’s repository. This file can then be listed as an optional EXTERNAL_DATA argument to the add_python_test function to have the artifact downloaded as an extra build step. As an example, consider the artifact currently used for testing called tests/data/test_file.txt.md5. To use this artifact in you test, you would add the test as follows:

add_python_test(my_test EXTERNAL_DATA test_file.txt)

The EXTERNAL_DATA keyword argument can take a list of files or even directories. When a directory is provided, it will download all files that exist in the given path. Inside your test, you can access these artifact files under the path given by the environment variable GIRDER_TEST_DATA_PREFIX as follows

import os
test_artifact = os.path.join(
    os.environ['GIRDER_TEST_DATA_PREFIX'],
    'test_file.txt'
)
with open(test_artifact, 'r') as f:
    content = f.read() # The content of the downloaded test artifact

Setting python code coverage paths

By default, python tests added via the add_python_test cmake function will only report coverage for python files within certain paths. For core tests, the following paths are used:

• girder
• clients/python/girder_client

For tests created within a plugin, the plugin’s server directory is added to this list. If you wish to report coverage on files residing outside of one of these directories, you can add a COVERAGE_PATHS argument to the add_python_test call. This argument accepts one or more comma-separated paths. For example, to add coverage reporting to to python files in a plugin’s utils directory for a given test:

add_python_test(cat
    PLUGIN cats
    COVERAGE_PATHS "${PROJECT_SOURCE_DIRECTORY}/plugins/cats/utils"
)

Mounting a custom application

Normally, the root node (/) of the server will serve up the Girder web client. A plugin may contain an entire application separate from the default Girder web client. This plugin may be written in a way which enables administrators to mount the application at a configured endpoint, including the option of replacing the root node with the plugin application.

To achieve this, you simply have to register your own root and configure your routes as you wish. In your plugin’s load method, you would follow this convention:

from girder.plugin import registerPluginWebroot
registerPluginWebroot(CustomAppRoot(), info['name'])

This will register your CustomAppRoot with Girder so that it can then be mounted wherever an Administrator specifies using the Server Configuration Panel. See Managing Routes.

Supporting web browser operations where custom headers cannot be set

Some aspects of the web browser make it infeasible to pass the usual Girder-Token authentication header when making a request. For example, if using an EventSource object for SSE, or when you must redirect the user’s browser to a download endpoint that serves its content as an attachment.

In such cases, you may allow specific REST API routes to authenticate using the Cookie. To avoid vulnerabilities to Cross-Site Request Forgery attacks, you should only do this if the endpoint is “read-only” (that is, the endpoint does not make modifications to data on the server).

In order to allow cookie authentication for your route, simply add the cookie=True option to the access decorator on your function. Example:

from girder.api import access

@access.public(cookie=True)
def download(self, params):
    ...

As a last resort, if your endpoint is not read-only and you are unable to pass the Girder-Token header to it, you can pass a token query parameter containing the token , but in practice this will probably never be the case.

External Web Clients

You may want to build your own custom web applications using Girder. Since Girder cleanly separates API from UI, it is straightforward to use a mounted Girder API for app authentication and data storage. You may additionally use Girder’s JavaScript libraries and UI templates to assist in building applications.

Including the Girder REST API

Apache

See the Deployment Alternatives section for instructions on deployment of Girder under Apache. You may host your web application alongside Girder and use its REST interface.

Tangelo

Tangelo is a CherryPy based web server framework for rapid data analytics and visualization application development. Tangelo has options for directly mounting the Girder API and static application files inside a Tangelo instance. See details in Tangelo’s setup documentation.

Using Girder JavaScript Utilities and Views

Including the JavaScript

All Girder code is packaged within ES6 modules, and may be directly included in other web applications via imports. The Webpack tool is recommended as a way to resolve imports and build deployable applications.

Extending Girder’s Backbone application

Girder defines a main application class, App. This object is responsible for bootstrapping the application, setting up the overall layout, and responding to global events like g:login and g:navigateTo. Developers can choose to derive their own application from this class to use the functionality that it provides. For example, the following derivation would modify the normal application bootstrapping

import Backbone from 'backbone';

import { setCurrentUser } from '@girder/core/auth';
import UserModel from '@girder/core/models/UserModel';
import { setApiRoot } from '@girder/core/rest';
import router from '@girder/core/router';
import eventStream from '@girder/core/utilities/EventStream';
import App from '@girder/core/views/App';

// set the path where girder's API is mounted
setApiRoot('/girder/api/v1');

var MyApp = App.extend({
    start: function () {
        // disable girder's router
        router.enabled(false);

        // call the super method
        return App.prototype.start.call(this, {
            fetch: false,   // disable automatic fetching of the user model
            history: false, // disable initialization of Backbone's router
            render: false   // disable automatic rendering on start
        }).done(() => {
            // set the current user somehow
            setCurrentUser(new UserModel({...}));
            eventStream.open();

            // replace the header with a customized class
            this.headerView = new MyHeaderView({parentView: this});

            // render the main page
            this.render();

            // start up the router with the `pushState` option enabled
            Backbone.history.start({pushState: true});
        });
    }
});

// initialize the application without starting it
var app = new MyApp({start: false});

// start your application after the page loads
$(function () {
    app.start();
});

Other methods that one may need to override include the following:

bindGirderEvents

Bind handlers to the global events object.

render

Render (or re-render) the entire page.

Note

router.enabled(false) must be set to false to disable URL routing behavior specific to the full Girder web application.

Using Girder Register and Login UI

To use Girder UI components, you will need the following CSS file in your page:

<link rel="stylesheet" href="/girder/static/built/girder_lib.min.css">

To make login and logout controls, provide a dialog container and login/logout/register links, and a container where the dialogs will be rendered:

<button class="btn btn-link" id="login" href="#">Login</button>
<button class="btn btn-link" id="register" href="#">Register</button>
<label class="hidden" id="name" href="#"></label>
<button class="btn btn-link hidden" id="logout" href="#">Logout</button>
<div class="modal fade" id="dialog-container"></div>

In your JavaScript, perform callbacks such as the following:

import { getCurrentUser, setCurrentUser } from '@girder/core/auth';
import events from '@girder/core/events';
import UserModel from '@girder/core/models/UserModel';
import { restRequest } from '@girder/core/rest';
import LoginView from '@girder/core/views/layout/LoginView';
import RegisterView from '@girder/core/views/layout/RegisterView';

$('#login').on('click', function () {
    var loginView = new LoginView({
        el: $('#dialog-container')
    });
    loginView.render();
});

$('#register').on('click', function () {
    var registerView = new RegisterView({
        el: $('#dialog-container')
    });
    registerView.render();
});

$('#logout').on('click', function () {
    restRequest({
        url: 'user/authentication',
        type: 'DELETE'
    }).done(function () {
        setCurrentUser(null);
        events.trigger('g:login');
    });
});

events.on('g:login', function () {
    console.log('g:login');
    var currentUser = getCurrentUser();
    if (currentUser) {
        $('#login').addClass('hidden');
        $('#register').addClass('hidden');
        $('#name').removeClass('hidden');
        $('#logout').removeClass('hidden');
        $('#name').text(currentUser.get('firstName') + ' ' + currentUser.get('lastName'));

        // Do anything else you'd like to do on login.
    } else {
        $('#login').removeClass('hidden');
        $('#register').removeClass('hidden');
        $('#name').addClass('hidden');
        $('#logout').addClass('hidden');

        // Do anything else you'd like to do on logout.
    }
});

// Check for who is logged in initially
restRequest({
    url: 'user/authentication',
    error: null
}).done(function (resp) {
    setCurrentUser(UserModel(resp.user));
    events.trigger('g:login');
});

Python Client and Girder CLI

In addition to the web clients, Girder comes with a python client library and a CLI to allow for programmatic interaction with a Girder server, and also to workaround limitations of the web client. For example, the python CLI makes it much easier to upload a large, nested hierarchy of data from a local directory to Girder, and also makes it much easier to download a large, nested hierarchy of data from Girder to a local directory.

Installation

If you have the source directory of Girder, you can find the girder_client package within the clients/python directory. If you do not have the source directory of Girder, you can install the client via pip:

pip install girder-client

After installing the client via pip and if you are using bash, auto-completion can easily be enabled executing:

eval "$(_GIRDER_CLI_COMPLETE=source girder-client)"

For convenience, adding this line at the end of .bashrc will make sure auto-completion is always available.

For more details, see http://click.pocoo.org/6/bashcomplete/

The Command Line Interface

The girder_client package ships with a command-line utility that wraps some of its common functionality to make it easy to invoke operations without having to write any custom python scripts. If you have installed girder_client via pip, you can use the girder-client executable:

girder-client <arguments>

To see all available commands, run:

girder-client --help

For help with a specific command, run:

girder-client <command> --help

Specifying the Girder Instance

When constructing a Girder client, you must declare what instance of Girder you wish to connect to. The easiest way to do so is to pass the full URL to the REST API of the Girder instance you wish to connect to using the api-url argument to girder-client. For example:

girder-client --api-url http://localhost:8080/api/v1 <command> ...

Instead of using api-url argument, you may also specify the URL in parts, using the host argument, and optional scheme, port, and api-root args.

Specifying credentials

The recommended way is to generate an API key and specify the api-key argument.

girder-client --api-url https://girder.example.com:443/api/v1  --api-key abcdefghijklmopqrstuvwxyz012345678901234 ...

Setting the GIRDER_API_KEY environment variable is also supported:

export GIRDER_API_KEY=abcdefghijklmopqrstuvwxyz012345678901234
girder-client --api-url https://girder.example.com:443/api/v1 ...

The client also supports username and password args. If only the username is specified, the client will prompt the user to interactively input their password.

Disabling SSL verification

Specifying --no-ssl-verify allows to ignore SSL verification. This is usually required when using the client behind a proxy that is not configured to accept the certificate of the given host.

Specifying a custom SSL certificate

Specifying --certificate /path/to/custom_bundle.pem allows to use a custom “bundle” of Certificate Authority (CA) public keys (CA certs) for performing the SSL verification applied when the https scheme is associated with the API url.

By default, the carefully curated collection of Root Certificates from Mozilla is used. See https://pypi.python.org/pypi/certifi

Upload a local file hierarchy

To upload a folder hierarchy rooted at test_folder to the Girder Folder with id 54b6d41a8926486c0cbca367

girder-client upload 54b6d41a8926486c0cbca367 test_folder

When using the upload command, the default --parent-type, meaning the type of resource the local folder will be created under in Girder, is Folder, so the following are equivalent

girder-client upload 54b6d41a8926486c0cbca367 test_folder
girder-client upload 54b6d41a8926486c0cbca367 test_folder --parent-type folder

To upload that same local folder to a Collection or User, specify the parent type as follows

girder-client upload 54b6d41a8926486c0cbca459 test_folder --parent-type user

To see what local folders and files on disk would be uploaded without actually uploading anything, add the --dry-run flag

girder-client upload 54b6d41a8926486c0cbca367 test_folder --dry-run

To have leaf folders (those folders with no subfolders, only containing files) be uploaded to Girder as single Items with multiple Files, i.e. those leaf folders will be created as Items and all files within the leaf folders will be Files within those Items, add the --leaf-folders-as-items flag

girder-client upload 54b6d41a8926486c0cbca367 test_folder --leaf-folders-as-items

If you already have an existing Folder hierarchy in Girder which you have a superset of on your local disk (e.g. you previously uploaded a hierarchy to Girder and then added more folders and files to the hierarchy on disk), you can reuse the existing hierarchy in Girder, which will not create new Folders and Items for those that match folders and files on disk, by using the --reuse flag.

girder-client upload 54b6d41a8926486c0cbca367 test_folder --reuse

To include a blacklist of file patterns that will not be uploaded, pass a comma separated list to the --blacklist arg

girder-client upload 54b6d41a8926486c0cbca367 test_folder --blacklist .DS_Store

Download a hierarchy of data into a local folder

Folder

To download a Girder Folder hierarchy rooted at Folder id 54b6d40b8926486c0cbca364 under the local folder download_folder

girder-client download 54b6d40b8926486c0cbca364 download_folder

Collection

To download the Girder Folder hierarchies associated with a Girder Collection with id 57b5c9e58d777f126827f5a1 under the local folder download_folder

girder-client download --parent-type collection 57b5c9e58d777f126827f5a1 download_folder

User

To download the Girder Folder hierarchies associated with a Girder User with id 54f8ac238d777f69813604af under the local folder download_folder

girder-client download --parent-type user 54b6d40b8926486c0cbca364 download_folder

Item

To download the file(s) associated with a Girder Item with if 58b8eb798d777f0aef5d0f78 under the local folder download_folder:

girder-client download --parent-type item 8b8eb798d777f0aef5d0f78 download_folder

File

To download a specific file from girder with id 58b8eb798d777f0aef5d0f78 to the local file local_file

girder-client download --parent-type file 8b8eb798d777f0aef5d0f78  local_file

Auto-detecting parent-type

Both download and upload commands accept a –parent-type argument allowing the users to specify the type (folder, collection, user, or item) associated with the chosen object id.

If the argument is omitted, the client will conveniently try to autodetect the type by iteratively invoking the resource/%id/path?type=%type API end point and checking if a resource is found.

Note that relying on auto-detection incurs extra network requests, which will slow down the script, so it should be avoided for time-sensitive operations.

Synchronize local folder with a Folder hierarchy

If the download_folder is a local copy of a Girder Folder hierarchy rooted at Folder id 54b6d40b8926486c0cbca364, any change made to the Girder Folder remotely can be synchronized locally by

girder-client localsync 54b6d40b8926486c0cbca364 download_folder

This will only download new Items or Items that have been modified since the last download/localsync. Local files that are no longer present in the remote Girder Folder will not be removed. This command relies on a presence of metadata file .metadata-girder within download_folder, which is created upon girder-client download. If .metadata-girder is not present, localsync will fallback to download.

The Python Client Library

For those wishing to write their own python scripts that interact with Girder, we recommend using the Girder python client library, documented below.

Recursively inherit access control to a Folder’s descendants

This will take the access control and public value in the Girder Folder with id 54b43e9b8926486c0c06cb4f and copy those to all of the descendant Folders

import girder_client
gc = girder_client.GirderClient(apiUrl='https://data.kitware.com/api/v1')
gc.authenticate('username', 'password')
gc.inheritAccessControlRecursive('54b43e9b8926486c0c06cb4f')

Set callbacks for Folder and Item uploads

If you have a function you would like called upon the completion of an Item or Folder upload, you would do the following.

N.B. The Item callbacks are called after the Item is created and all Files are uploaded to the Item. The Folder callbacks are called after the Folder is created and all child Folders and Items are uploaded to the Folder.

import girder_client
gc = girder_client.GirderClient()

def folderCallback(folder, filepath):
    # assume we have a folderMetadata dict that has
    # filepath: metadata_dict_for_folder
    gc.addMetadataToFolder(folder['_id'], folderMetadata[filepath])

def itemCallback(item, filepath):
    # assume we have an itemMetadata dict that has
    # filepath: metadata_dict_for_item
    gc.addMetadataToItem(item['_id'], itemMetadata[filepath])

gc.authenticate('username', 'password')
gc.addFolderUploadCallback(folderCallback)
gc.addItemUploadCallback(itemCallback)
gc.upload(localFolder, parentId)

Further Examples and Function Level Documentation

exception girder_client.AuthenticationError

class girder_client.GirderClient(host=None, port=None, apiRoot=None, scheme=None, apiUrl=None, cacheSettings=None, progressReporterCls=None)

A class for interacting with the Girder RESTful API. Some simple examples of how to use this class follow:

client = GirderClient(apiUrl='http://myhost:8080')
client.authenticate('myname', 'mypass')

folder_id = '53b714308926486402ac5aba'
item = client.createItem(folder_id, 'an item name', 'a description')
client.addMetadataToItem(item['_id'], {'metadatakey': 'metadatavalue'})
client.uploadFileToItem(item['_id'], 'path/to/your/file.txt')

r1 = client.getItem(item['_id'])
r2 = client.sendRestRequest('GET', 'item',
    {'folderId': folder_id, 'sortdir': '-1' })
r3 = client.sendRestRequest('GET', 'resource/search',
    {'q': 'aggregated','types': '["folder", "item"]'})

__init__(host=None, port=None, apiRoot=None, scheme=None, apiUrl=None, cacheSettings=None, progressReporterCls=None)

Construct a new GirderClient object, given a host name and port number, as well as a username and password which will be used in all requests (HTTP Basic Auth). You can pass the URL in parts with the host, port, scheme, and apiRoot kwargs, or simply pass it in all as one URL with the apiUrl kwarg instead. If you pass apiUrl, the individual part kwargs will be ignored.

Parameters:

• apiUrl – The full path to the REST API of a Girder instance, e.g. http://my.girder.com/api/v1.
• host – A string containing the host name where Girder is running, the default value is ‘localhost’
• port – The port number on which to connect to Girder, the default value is 80 for http: and 443 for https:
• apiRoot – The path on the server corresponding to the root of the Girder REST API. If None is passed, assumes ‘/api/v1’.
• scheme – A string containing the scheme for the Girder host, the default value is ‘http’; if you pass ‘https’ you likely want to pass 443 for the port
• cacheSettings – Settings to use with the diskcache library, or None to disable caching.
• progressReporterCls – the progress reporter class to instantiate. This class is expected to be a context manager with a constructor accepting label and length keyword arguments, an update method accepting a chunkSize argument and a class attribute reportProgress set to True (It can conveniently be initialized using sys.stdout.isatty()). This defaults to _NoopProgressReporter.

addFolderUploadCallback(callback)

Saves a passed in callback function that will be called after each folder has completed. Multiple callback functions can be added, they will be called in the order they were added by calling this function. Callback functions will be called after a folder in Girder is created and all subfolders and items for that folder have completed uploading. Callback functions should take two parameters: - the folder in Girder - the full path to the local folder

Parameters:callback – callback function to be called.

addItemUploadCallback(callback)

Saves a passed in callback function that will be called after each item has completed. Multiple callback functions can be added, they will be called in the order they were added by calling this function. Callback functions will be called after an item in Girder is created and all files for that item have been uploaded. Callback functions should take two parameters: - the item in Girder - the full path to the local folder or file comprising the item

Parameters:callback – callback function to be called.

addMetadataToCollection(collectionId, metadata)

Takes a collection ID and a dictionary containing the metadata

Parameters:

• collectionId – ID of the collection to set metadata on.
• metadata – dictionary of metadata to set on collection.

addMetadataToFolder(folderId, metadata)

Takes a folder ID and a dictionary containing the metadata

Parameters:

• folderId – ID of the folder to set metadata on.
• metadata – dictionary of metadata to set on folder.

addMetadataToItem(itemId, metadata)

Takes an item ID and a dictionary containing the metadata

Parameters:

• itemId – ID of the item to set metadata on.
• metadata – dictionary of metadata to set on item.

authenticate(username=None, password=None, interactive=False, apiKey=None)

Authenticate to Girder, storing the token that comes back to be used in future requests. This method can be used in two modes, either username and password authentication, or using an API key. Username example:

gc.authenticate(username='myname', password='mypass')

Note that you may also pass interactive=True and omit either the username or password argument to be prompted for them in the shell. The second mode is using an API key:

API keys can be created and managed on your user account page in the Girder web client, and can be used to provide limited access to the Girder web API.

Parameters:

• username – A string containing the username to use in basic authentication.
• password – A string containing the password to use in basic authentication.
• interactive – If you want the user to type their username or password in the shell rather than passing it in as an argument, set this to True. If you pass a username in interactive mode, the user will only be prompted for a password. This option only works in username/password mode, not API key mode.
• apiKey (str) – Pass this to use an API key instead of username/password authentication.

createCollection(name, description='', public=False)

Creates and returns a collection.

createFolder(parentId, name, description='', parentType='folder', public=None, reuseExisting=False, metadata=None)

Creates and returns a folder.

Parameters:

• parentId – The id of the parent resource to create the folder in.
• name – The name of the folder.
• description – A description of the folder.
• parentType – One of (‘folder’, ‘user’, ‘collection’)
• public – Whether the folder should be marked a public.
• reuseExisting – Whether to return an existing folder if one with the same name exists.
• metadata – JSON metadata to set on the folder.

createItem(parentFolderId, name, description='', reuseExisting=False, metadata=None)

Creates and returns an item.

Parameters:

• parentFolderId – the folder this item should be created in.
• name – the item name.
• description – a description of the item.
• reuseExisting – whether to return an existing item if one with same name already exists.
• metadata – JSON metadata to set on item.

createResource(path, params)

Creates and returns a resource.

createUser(login, email, firstName, lastName, password, admin=None)

Creates and returns a user.

delete(path, parameters=None, jsonResp=True)

Convenience method to call sendRestRequest() with the ‘DELETE’ HTTP method.

downloadFile(fileId, path, created=None)

Download a file to the given local path or file-like object.

Parameters:

• fileId – The ID of the Girder file to download.
• path – The path to write the file to, or a file-like object.

downloadFileAsIterator(fileId, chunkSize=65536)

Download a file streaming the contents as an iterator.

Parameters:

• fileId – The ID of the Girder file to download.
• chunkSize – The chunk size to download the contents in.

Returns:

The request content iterator.

downloadFolderRecursive(folderId, dest, sync=False)

Download a folder recursively from Girder into a local directory.

Parameters:

• folderId (ObjectId or Unix-style path to the resource in Girder.) – Id of the Girder folder or resource path to download.
• dest (str) – The local download destination.
• sync (bool) – If True, check if item exists in local metadata cache and skip download provided that metadata is identical.

downloadItem(itemId, dest, name=None)

Download an item from Girder into a local folder. Each file in the item will be placed into the directory specified by the dest parameter. If the item contains multiple files or a single file with a different name than the item, the item will be created as a directory under dest and the files will become files within that directory.

Parameters:

• itemId – The Id of the Girder item to download.
• dest – The destination directory to write the item into.
• name – If the item name is known in advance, you may pass it here which will save a lookup to the server.

downloadResource(resourceId, dest, resourceType='folder', sync=False)

Download a collection, user, or folder recursively from Girder into a local directory.

Parameters:

• resourceId (ObjectId or Unix-style path to the resource in Girder.) – ID or path of the resource to download.
• dest (str) – The local download destination. Can be an absolute path or relative to the current working directory.
• resourceType (str) – The type of resource being downloaded: ‘collection’, ‘user’, or ‘folder’.
• sync (bool) – If True, check if items exist in local metadata cache and skip download if the metadata is identical.

get(path, parameters=None, jsonResp=True)

Convenience method to call sendRestRequest() with the ‘GET’ HTTP method.

getCollection(collectionId)

Retrieves a collection by its ID.

Parameters:collectionId – A string containing the ID of the collection to retrieve from Girder.

static getDefaultPort(hostname, scheme)

Get default port based on the hostname. Returns GirderClient.DEFAULT_HTTPS_PORT if scheme is https, otherwise returns GirderClient.DEFAULT_LOCALHOST_PORT if hostname is localhost, and finally returns GirderClient.DEFAULT_HTTP_PORT.

static getDefaultScheme(hostname)

Get default scheme based on the hostname. Returns http if hostname is localhost otherwise returns https.

getFile(fileId)

Retrieves a file by its ID.

Parameters:fileId – A string containing the ID of the file to retrieve from Girder.

getFolder(folderId)

Retrieves a folder by its ID.

Parameters:folderId – A string containing the ID of the folder to retrieve from Girder.

getFolderAccess(folderId)

Retrieves a folder’s access by its ID.

Parameters:folderId – A string containing the ID of the folder to retrieve access for from Girder.

getItem(itemId)

Retrieves a item by its ID.

Parameters:itemId – A string containing the ID of the item to retrieve from Girder.

getResource(path, id=None, property=None)

Returns a resource based on id or None if no resource is found; if property is passed, returns that property value from the found resource.

getServerAPIDescription(useCached=True)

Fetch server RESTful API description.

Parameters:useCached (bool) – Whether to return the previously fetched value. Set to False to force a re-fetch of the description from the server. Returns:The API descriptions as a dict.

For example:

{
    "basePath": "/api/v1",
    "definitions": {},
    "host": "girder.example.com",
    "info": {
        "title": "Girder REST API",
        "version": "X.Y.Z"
    },
    "paths": {
        "/api_key": {
            "get": {
                "description": "Only site administrators [...]",
                "operationId": "api_key_listKeys",
                "parameters": [
                    {
                        "description": "ID of the user whose keys to list.",
                        "in": "query",
                        "name": "userId",
                        "required": false,
                        "type": "string"
                    },
                    ...
                ]
            }.
            ...
        }
    ...
    }
}

getServerVersion(useCached=True)

Fetch server API version. By default, caches the version such that future calls to this function do not make another request to the server.

Parameters:useCached (bool) – Whether to return the previously fetched value. Set to False to force a re-fetch of the version from the server. Returns:The API version as a list (e.g. ['1', '0', '0'])

getUser(userId)

Retrieves a user by its ID.

Parameters:userId – A string containing the ID of the user to retrieve from Girder.

inheritAccessControlRecursive(ancestorFolderId, access=None, public=None)

Take the access control and public value of a folder and recursively copy that access control and public value to all folder descendants, replacing any existing access control on the descendant folders with that of the ancestor folder.

Parameters:

• ancestorFolderId – Id of the Girder folder to copy access control from, to all of its descendant folders.
• access – Dictionary Access control target, if None, will take existing access control of ancestor folder
• public – Boolean public value target, if None, will take existing public value of ancestor folder

isFileCurrent(itemId, filename, filepath)

Tests whether the passed in filepath exists in the item with itemId, with a name of filename, and with the same length. Returns a tuple (file_id, current) where file_id = id of the file with that filename under the item, or None if no such file exists under the item. current = boolean if the file with that filename under the item has the same size as the file at filepath.

Parameters:

• itemId – ID of parent item for file.
• filename – name of file to look for under the parent item.
• filepath – path to file on disk.

listCollection(limit=None, offset=None)

This is a generator that will yield all collections in the system.

Parameters:

• limit – If requesting a specific slice, the length of the slice.
• offset – Starting offset into the list.

listFile(itemId, limit=None, offset=None)

This is a generator that will yield files under the given itemId.

Parameters:

• itemId – the item’s ID
• limit – the result set size limit.
• offset – the result offset.

listFolder(parentId, parentFolderType='folder', name=None, limit=None, offset=None)

This is a generator that will yield a list of folders based on the filter parameters.

Parameters:

• parentId – The parent’s ID.
• parentFolderType – One of (‘folder’, ‘user’, ‘collection’).
• name – query for exact name match of items.
• limit – If requesting a specific slice, the length of the slice.
• offset – Starting offset into the list.

listItem(folderId, text=None, name=None, limit=None, offset=None)

This is a generator that will yield all items under a given folder.

Parameters:

• folderId – the parent folder’s ID.
• text – query for full text search of items.
• name – query for exact name match of items.
• limit – If requesting a specific slice, the length of the slice.
• offset – Starting offset into the list.

listResource(path, params=None, limit=None, offset=None)

This is a generator that will yield records using the given path and params until exhausted. Paging of the records is done internally, but can be overriden by manually passing a limit value to select only a single page. Passing an offset will work in both single-page and exhaustive modes.

listUser(limit=None, offset=None)

This is a generator that will yield all users in the system.

Parameters:

• limit – If requesting a specific slice, the length of the slice.
• offset – Starting offset into the list.

loadLocalMetadata(dest)

Reads item metadata from a local folder.

Parameters:dest – The local download destination.

loadOrCreateFolder(folderName, parentId, parentType, metadata=None)

Returns a folder in Girder with the given name under the given parent. If none exists yet, it will create it and return it.

Parameters:

• folderName – the name of the folder to look up.
• parentId – id of parent in Girder
• parentType – one of (collection, folder, user)
• metadata – JSON metadata string to set on folder.

Returns:

The folder that was found or created.

loadOrCreateItem(name, parentFolderId, reuseExisting=True, metadata=None)

Create an item with the given name in the given parent folder.

Parameters:

• name – The name of the item to load or create.
• parentFolderId – id of parent folder in Girder
• reuseExisting – boolean indicating whether to load an existing item of the same name in the same location, or create a new one.
• metadata – JSON metadata string to set on item.

patch(path, parameters=None, data=None, json=None, jsonResp=True)

Convenience method to call sendRestRequest() with the ‘PATCH’ HTTP method.

post(path, parameters=None, files=None, data=None, json=None, headers=None, jsonResp=True)

Convenience method to call sendRestRequest() with the ‘POST’ HTTP method.

put(path, parameters=None, data=None, json=None, jsonResp=True)

Convenience method to call sendRestRequest() with the ‘PUT’ HTTP method.

resourceLookup(path)

Look up and retrieve resource in the data hierarchy by path.

Parameters:path – The path of the resource. The path must be an absolute Unix path starting with either “/user/[user name]” or “/collection/[collection name]”.

saveLocalMetadata(dest)

Dumps item metadata collected during a folder download.

Parameters:dest – The local download destination.

sendRestRequest(method, path, parameters=None, data=None, files=None, json=None, headers=None, jsonResp=True, **kwargs)

This method looks up the appropriate method, constructs a request URL from the base URL, path, and parameters, and then sends the request. If the method is unknown or if the path is not found, an exception is raised, otherwise a JSON object is returned with the Girder response.

This is a convenience method to use when making basic requests that do not involve multipart file data that might need to be specially encoded or handled differently.

Parameters:

• method (str) – The HTTP method to use in the request (GET, POST, etc.)
• path (str) – A string containing the path elements for this request. Note that the path string should not begin or end with the path separator, ‘/’.
• parameters (dict) – A dictionary mapping strings to strings, to be used as the key/value pairs in the request parameters.
• data – A dictionary, bytes or file-like object to send in the body.
• files (dict) – A dictionary of ‘name’ => file-like-objects for multipart encoding upload.
• json (dict) – A JSON object to send in the request body.
• headers (dict) – If present, a dictionary of headers to encode in the request.
• jsonResp (bool) –

  Whether the response should be parsed as JSON. If False, the raw response object is returned. To get the raw binary content of the response, use the content attribute of the return value, e.g.

  resp = client.get('my/endpoint', jsonResp=False)
  print(resp.content)  # Raw binary content
  print(resp.headers)  # Dict of headers
  

session(session=None)

Use a requests.Session object for all outgoing requests from GirderClient. If session isn’t passed into the context manager then one will be created and yielded. Session objects are useful for enabling persistent HTTP connections as well as partially applying arguments to many requests, such as headers.

Note: session is closed when the context manager exits, regardless of who created it.

with gc.session() as session:
    session.headers.update({'User-Agent': 'myapp 1.0'})

    for itemId in itemIds:
        gc.downloadItem(itemId, fh)

In the above example, each request will be executed with the User-Agent header while reusing the same TCP connection.

Parameters:session – An existing requests.Session object, or None.

setFolderAccess(folderId, access, public)

Sets the passed in access control document along with the public value to the target folder.

Parameters:

• folderId – Id of the target folder.
• access – JSON document specifying access control.
• public – Boolean specificying the public value.

setResourceTimestamp(id, type, created=None, updated=None)

Set the created or updated timestamps for a resource.

setToken(token)

Set a token on the GirderClient instance. This is useful in the case where the client has already been given a valid token, such as a remote job.

Parameters:token – A string containing the existing Girder token

transformFilename(name)

Sanitize a resource name from Girder into a name that is safe to use as a filesystem path.

Parameters:name (str) – The name to transform.

upload(filePattern, parentId, parentType='folder', leafFoldersAsItems=False, reuseExisting=False, blacklist=None, dryRun=False, reference=None)

Upload a pattern of files.

This will recursively walk down every tree in the file pattern to create a hierarchy on the server under the parentId.

Parameters:

• filePattern (str) – a glob pattern for files that will be uploaded, recursively copying any file folder structures. If this is a list or tuple each item in it will be used in turn.
• parentId (ObjectId or Unix-style path to the resource in Girder.) – Id of the parent in Girder or resource path.
• parentType (str) – one of (collection,folder,user), default of folder.
• leafFoldersAsItems (bool) – bool whether leaf folders should have all files uploaded as single items.
• reuseExisting (bool) – bool whether to accept an existing item of the same name in the same location, or create a new one instead.
• dryRun (bool) – Set this to True to print out what actions would be taken, but do not actually communicate with the server.
• reference (str) – Option reference to send along with the upload.

uploadFile(parentId, stream, name, size, parentType='item', progressCallback=None, reference=None, mimeType=None)

Uploads a file into an item or folder.

Parameters:

• parentId (str) – The ID of the folder or item to upload into.
• stream (file-like) – Readable stream object.
• name (str) – The name of the file to create.
• size (str) – The length of the file. This must be exactly equal to the total number of bytes that will be read from stream, otherwise the upload will fail.
• parentType (str) – ‘item’ or ‘folder’.
• progressCallback (callable) – If passed, will be called after each chunk with progress information. It passes a single positional argument to the callable which is a dict of information about progress.
• reference (str) – optional reference to send along with the upload.
• mimeType (str or None) – MIME type to set on the file. Attempts to guess if not explicitly passed.

Returns:

The file that was created on the server.

uploadFileContents(fileId, stream, size, reference=None)

Uploads the contents of an existing file.

Parameters:

• fileId – ID of file to update
• stream (file-like) – Readable stream object.
• size (str) – The length of the file. This must be exactly equal to the total number of bytes that will be read from stream, otherwise the upload will fail.
• reference (str) – optional reference to send along with the upload.

uploadFileToFolder(folderId, filepath, reference=None, mimeType=None, filename=None, progressCallback=None)

Uploads a file to a folder, creating a new item in the process. If the file has 0 bytes, no uploading will be performed, and no item will be created.

Parameters:

• folderId – ID of parent folder for file.
• filepath – path to file on disk.
• reference (str) – optional reference to send along with the upload.
• mimeType (str or None) – MIME type for the file. Will be guessed if not passed.
• filename – path with filename used in Girder. Defaults to basename of filepath.
• progressCallback (callable) – If passed, will be called after each chunk with progress information. It passes a single positional argument to the callable which is a dict of information about progress.

Returns:

the file that was created.

uploadFileToItem(itemId, filepath, reference=None, mimeType=None, filename=None, progressCallback=None)

Uploads a file to an item, in chunks. If ((the file already exists in the item with the same name and size) or (if the file has 0 bytes), no uploading will be performed.

Parameters:

• itemId – ID of parent item for file.
• filepath – path to file on disk.
• reference (str) – optional reference to send along with the upload.
• mimeType (str or None) – MIME type for the file. Will be guessed if not passed.
• filename – path with filename used in Girder. Defaults to basename of filepath.
• progressCallback (callable) – If passed, will be called after each chunk with progress information. It passes a single positional argument to the callable which is a dict of information about progress.

Returns:

the file that was created.

uploadStreamToFolder(folderId, stream, filename, size, reference=None, mimeType=None, progressCallback=None)

Uploads a file-like object to a folder, creating a new item in the process. If the file has 0 bytes, no uploading will be performed, and no item will be created.

Parameters:

• folderId – ID of parent folder for file.
• stream – Readable stream object.
• filename – Filename used for Girder only.
• size – The length of the file. This must be exactly equal to the total number of bytes that will be read from stream, otherwise the upload will fail.
• reference – optional reference to send along with the upload.
• mimeType – MIME type for the file.
• progressCallback – If passed, will be called after each chunk with progress information. It passes a single positional argument to the callable which is a dict of information about progress.

exception girder_client.HttpError(status, text, url, method, response=None)

Raised if the server returns an error status code from a request. @deprecated This will be removed in a future release of Girder. Raisers of this exception should instead raise requests.HTTPError manually or through another mechanism such as requests.Response.raise_for_status.

__init__(status, text, url, method, response=None)

Initialize RequestException with request and response objects.

exception girder_client.IncompleteResponseError(message, expected, received, response=None)

__init__(message, expected, received, response=None)

Initialize RequestException with request and response objects.

exception girder_client.IncorrectUploadLengthError(message, upload=None)

__init__(message, upload=None)

Initialize self. See help(type(self)) for accurate signature.

Security

Girder maintains data security through a variety of mechanisms.

Default Authorization

Internally, endpoints default to requiring administrator permissions in order to use them. This means that, for example, when writing a plugin, a developer must consciously choose to allow non-administrator access. Basic administrator, user, or token access restrictions are applied before any other endpoint code is executed.

CORS (Cross-Origin Resource Sharing)

In an out-of-the-box Girder deployment, CORS is disabled for API calls. If you want your server to support API calls that are cross-origin requests from web browsers, you’ll need to modify some configuration settings.

As an administrator, go to the Admin console, then to Server configuration. Open the Advanced Settings panel and you will see several settings that allow you to specify the CORS policies for the REST API. The most important setting is the CORS Allowed Origins field, which is used to specify what origins are allowed to make cross-origin requests to the instance’s REST API. By default, this is blank, meaning no cross-origin requests are allowed. To allow requests from any origin, simply set this to *. You can also specify this as a comma-separated list of explicit origins to allow. If the Origin header occurs in this explicit list, the Access-Control-Allow-Origin header of the response will be set to the request’s Origin header value. You may also specify both * and a white list of explicit origins to allow. In this case, the response header will be set to the Origin header if it is explicitly listed, otherwise it will be set to *.

If you want more fine-grained control over the CORS policies, you can also restrict the allowed methods and allowed request headers by providing them in comma-separated lists in the CORS Allowed Methods and CORS Allowed Headers fields, though this is usually not necessary–the default values for these two fields are quite permissive and should enable complete access to the web API so long as the origin is allowed.

These settings simply control the CORS headers that are sent to the browser; actual enforcement of the CORS policies takes place on the user’s browser.

Database Injection Attacks

Girder defends against database injection attacks by using PyMongo as the only pathway between the application server and the database server. This protects against many injection vulnerabilities as described in the MongoDB Documentation. Girder also uses a model layer to mediate and validate all interaction with the database. This ensures that for all database operations, structural attributes (collection name, operation type, etc.) are hardcoded and not modifiable by the client, while data attributes (stored content) are validated for proper form before being accepted from a client.

Additionally, we strongly recommend configuring your MongoDB server with JavaScript disabled unless explicitly needed for your Girder-based application or plugin. Again, see the MongoDB Documentation for more information.

Session Management

Girder uses session management performed through the Girder-Token header or through a token passed through a GET parameter. This token is provided to the client through the cookie and expires after a configurable amount of time. In order to prevent session stealing, it is highly recommended to run Girder under HTTPS.

Cross-Site Scripting (XSS)

In order to protect against XSS attacks, all input from users is sanitized before presentation of the content on each page. This is handled by the template system Girder uses (Pug). This sanitizes user-provided content.

Cross-Site Request Forgery (CSRF)

To prevent CSRF attacks, Girder requires the Girder-Token parameter as a header for all state-changing requests. This token is taken from the user’s cookie and then passed in the request as part of the Girder one-page application and other clients such that the cookie alone is not enough to form a valid request. A sensible CORS policy (discussed above) also helps mitigate this attack vector.

Dependent Libraries

Another common attack vector is through libraries upon which Girder depends such as Cherrypy, Pug, PyMongo, etc. Girder’s library dependencies reference specific versions, ensuring that arbitrary upstream changes to libraries are not automatically accepted into Girder’s environment. Conversely, during development and before releases we work to ensure our dependencies are up to date in order to get the latest security fixes.

Notes on Secure Deployment

It is recommended that Girder be deployed using HTTPS as the only access method. Additionally, we recommend encrypting the volume where the Mongo database is stored as well as always connecting to Mongo using authenticated access. The volume containing any on-disk assetstores should also be encrypted to provide encryption of data at rest. We also recommend using a tool such as logrotate to enable the audit of Girder logs in the event of a data breach. Finally, we recommend a regular (and regularly tested) backup of the Girder database, configuration, and assetstores. Disaster recovery is an important part of any security plan.

Build the Sphinx Documentation

In order to build the Sphinx documentation, after setting up your development environment, you can run the following:

tox -e docs

Migration Guide

This document is meant to guide Girder plugin developers in transitioning between major versions of Girder. Major version bumps contain breaking changes to the Girder core library, which are enumerated in this guide along with instructions on how to update your plugin code to work in the newer version.

2.x → 3.x

Girder 3.0 changed how plugins are installed and loaded into the runtime environment. These changes require that all plugins exist as a standalone python package. Automatic loading of plugins from Girder’s plugins directory is no longer supported. This also applies to plugins that have no server (python) component.

General plugin migration steps

The following is a list of changes that are necessary for a typical Girder plugin:

• Create a setup.py in the root directory of your plugin. A minimal example is as follows:

  from setuptools import setup
  setup(
      name='example-plugin',            # The name registered on PyPI
      version='1.0.0',
      description='An example plugin.', # This text will be displayed on the plugin page
      packages=['example_plugin'],
      install_requires=['girder'],      # Add any plugin dependencies here
      entry_points={
        'girder.plugin': [              # Register the plugin with girder.  The next line registers
                                        # our plugin under the name "example".  The name here must be
                                        # unique for all installed plugins.  The right side points to
                                        # a subclass of GirderPlugin inside your plugin.
            'example = example_plugin:ExamplePlugin'
        ]
      }
  )
  

• Move your plugin’s python source code from the server directory to the package name defined in your setup.py. In this example, you would move all python files from server to a new directory named example_plugin.

• Move your web_client directory under the directory created in the previous step, which was example_plugin in the previous step.

• Create a package.json file inside your web_client directory defining an npm package. A minimal example is as follows:

  {
      "name": "@girder/my-plugin",
      "version": "1.0.0",
      "peerDepencencies": {
          "@girder/other_plugin": "*",      // Peer dependencies should be as relaxed as possible.
                                            // Add in any other girder plugins your plugin depends
                                            // on for web_client code.
                                            // Plugin dependencies should also be listed by entrypoint
                                            // name in "girderPlugin" as shown below.
          "@girder/core": "*"               // Your plugin will likely depend on girder/core.
      },
      "dependencies": {},                   // Any other dependencies of the client code
      "girderPlugin": {
          "name": "example",                // The entrypoint name defined in setup.py.
          "main": "./main.js"               // The plugin client entrypoint containing code that is executed on load.
          "webpack": "webpack.helper",      // If your plugin needs to modify the webpack config
          "dependencies": ["other_plugin"]  // If you plugin web_client requires another plugin
      }
  }
  

• Delete the plugin.json file at the root of your plugin. Move the dependencies from that file to the top level dependencies key of the package.json file created in the previous step.

• Create a subclass of girder.plugin.GirderPlugin in your plugin package. This class can be anywhere in your package, but a sensible place to put it is in the top-level __init__.py. There are hooks for custom behavior in this class, but at a minimum you should move the old load method into this class and point to an npm package name containing your web client code.

  from girder.plugin import getPlugin, GirderPlugin
  
  class ExamplePlugin(GirderPlugin):
      DISPLAY_NAME = 'My Plugin'              # a user-facing plugin name, the plugin is still
                                              # referenced internally by the entrypoint name.
      CLIENT_SOURCE_PATH = 'web_client'       # path to the web client relative to the python package
  
      def load(self, info):
          getPlugin('mydependency').load(info)  # load plugins you depend on
  
          # run the code that was in the original load method
  

Warning

The plugin name was removed from the info object. Where previously used, plugins should replace references to info['name'] with a hard-coded string.

• Migrate all imports in Python and Javascript source files. The old plugin module paths are no longer valid. Any import reference to:
  • girder.plugins in Python must be changed to the actual installed module name
    • For example, change from girder.plugins.jobs.models.job import Job to from girder_jobs.models.job import Job
  • girder_plugins in Javascript must be changed to the actual installed package name
    • For example, change import { JobListWidget } from 'girder_plugins/jobs/views'; to import { JobListWidget } from '@girder/jobs/views';
  • girder in Javascript must be changed to @girder/core
    • For example, change import { restRequest } from 'girder/rest'; to import { restRequest } from '@girder/core/rest';

Other backwards incompatible changes affecting plugins

• Automatic detection of mail templates has been removed. Instead, plugins should register them in their load method with girder.utility.mail_utils.addTemplateDirectory().

• The mockPluginDir methods have been removed from the testing infrastructure. If plugins need to generate a one-off plugin for testing, they can generate a subclass of girder.plugin.GirderPlugin in the test file and register it in a test context with the test_plugin mark. For example,

  class FailingPlugin(GirderPlugin):
      def load(self, info):
          raise Exception('This plugin fails on load')
  
  @pytest.mark.plugin('failing_plugin', FailingPlugin)
  def test_with_failing_plugin(server):
      # the test plugin will be installed in this context
  

• When running the server in testing mode (girder serve --mode=testing), the source directory is no longer served. If you need any assets for testing, they have to be installed into the static directory during the client build process.

• Automatic registration of plugin models is no longer provided. If your plugin contains any custom models that must be resolved dynamically (with ModelImporter.model(name, plugin=plugin)) then you must register the model in your load method. In the jobs plugin for example, we register the job model as follows:

  from girder.utility.model_importer import ModelImporter
  from .models.job import Job
  
  class JobsPlugin(GirderPlugin):
      def load(self, info):
          ModelImporter.registerModel('job', Job, 'jobs')
  

• In the web client, girder.rest.restRequest no longer accepts the deprecated path parameter; callers should use the url parameter instead. Callers are also encouraged to use the method parameter instead of type.

• The CMake function add_standard_plugin_tests can not detect the python package of your plugin. It now requires you pass the keyword argument PACKAGE with the package name. For example, the jobs plugin plugin.cmake file contains the following line:

  add_standard_plugin_tests(PACKAGE "girder_jobs")
  

Client build changes

The girder_install command has been removed. This command was primarily used to install plugins and run the client build. Plugins should now be installed (and uninstalled) using pip directly. For the client build, there is a new command, girder build. Without any arguments this command will execute a production build of all installed plugins. Executing girder build --dev will build a development install of Girder’s static assets as well as building targets only necessary when running testing.

The new build process works by generating a package.json file in girder/web_client from the template (girder/web_client/package.json.template). The generated package.json itself depends on the core web client and all plugin web clients. The build process is executed in place (in the Girder Python package) in both development and production installs. The built assets are installed into a virtual environment specific static path {sys.prefix}/share/girder.

Static public path is required during web client build

The static public path, indicating the base URL where web client files are served from, must now be known when the web client is built. Most deployments can simply accept the default value of /static, unless serving Girder from a CDN or mounting at a subpath using a reverse proxy.

The static public path may be changed via the config file:

[server]
static_public_path = "/someprefix/static"

If the static public path setting is changed, the web client must be immediately rebuilt.

The static public path setting replaces all previous “static root” functionality. Accordingly:

• The server now serves all static content from /static. The GIRDER_STATIC_ROUTE_ID constant has been removed.
• In the server, girder.utility.server.getStaticRoot has been removed.
• In the web client, girder.rest.staticRoot, girder.rest.getStaticRoot, and girder.rest.setStaticRoot have been removed.
• The ability to set the web client static root / public path via the special element <div id="g-global-info-staticroot"> has been removed

Server changes

The GET /user API endpoint is only open to logged-in users

This is a policy change from 2.x, in which this endpoint was publicly accessible. Since user name data may be considered sensitive, and many Girder instances have controlled registration policies, it made sense to change this access policy.

ModelImporter behavior changes

The girder.utility.model_importer.ModelImporter class allows model types to be mapped from strings, which is useful when model types must be provided by users via the REST API. In Girder 2, there was logic to infer automatically where a model class resides without having to explicitly register it, but that logic was removed. If your plugin needs to expose a Model subclass for string-based lookup, it must be explicitly registered, e.g.

class MyModel(Model):
   ...

ModelImporter.registerModel('my_plugin_model', MyModel, plugin='my_plugin')

The load method of your plugin is a good place to register your plugin’s models.

In addition to explicitly requiring registration, the API of registerModel() has also changed. Before, one would pass the model instance, but now, one passes the model class.

# Girder 2:
ModelImporter.registerModel('my_thing', MyThing())

# Girder 3:
ModelImporter.registerModel('my_thing', MyThing)

Additionally, several key base classes in Girder no longer mixin ModelImporter, and mixing it in is now generally discouraged. So instead of self.model, just use ModelImporter.model if you must convert a string to a model instance. The following base classes are affected:

• girder.api.rest.Resource
• girder.models.model_base.Model
• girder.utility.abstract_assetstore_adapter.AbstractAssetstoreAdapter

Multipart-encoded upload chunk support has been removed

Prior to version 3, Girder supported multipart/form-data content type for passing fields into the POST /file/chunk endpoint. This has been deprecated since v2.2, and has been removed. Now, the uploadId and offset fields should be passed in the query string, and the chunk data should be passed as the request body.

Event bindings are now unique by handler name

In Girder 2, it was possible to bind multiple handler callbacks to the same event with the same handler name. This has changed in Girder 3; for any given event identifier, each callback must be bound to it with a unique handler name. Example:

def cb(event):
   print('hello')

for _ in range(5):
  events.bind('an_event', 'my_handler', cb)

# Prints 'hello' five times in Girder 2, but only once in Girder 3
events.trigger('an_event')

In the new behavior, a call to bind with the same event name and handler name as an existing handler will be ignored, and will emit a warning to the log. If you wish to overwrite the existing handler, you must call girder.events.unbind() on the existing mapping first.

def a(event):
  print('a')

def b(event):
  print('b')

events.bind('an_event', 'my_handler', a)
events.bind('an_event', 'my_handler', b)

# Prints 'a' and 'b' in Girder 2, but only 'a' in Girder 3
events.trigger('an_event')

Async keyword arguments and properties changed to async_ PR #2817

In version 3.7 of python async is a reserved keyword argument. To mitigate any issues all instances of async in the codebase have changed to asynchronous. This affects:

• The event framework girder/events.py
• The built-in job plugin plugins/jobs/girder_jobs/models/job.py

The cookie access decorator has been removed

The @access.cookie decorator has been removed. To allow cookie authentication on an endpoint, include cookie=True as a parameter to one of the other access decorators (e.g., @access.user(cookie=True)).

Storing girder.local.cfg inside the package directory is no longer supported

In order to facilitate the ability to upgrade Girder using pip, the user configuration file can no longer be stored inside the package directory since it would be deleted on upgrade. Users must now store their configuration in one of the approved locations, or use GIRDER_CONFIG to specify the exact location. See the configuration documentation for more details.

Invoking Girder and Girder Client with python -m is no longer supported

Using python -m girder and python -m girder-cli was deprecated in Girder 2.5 and is no longer supported. Users are expected to have the appropriate packages installed and then use girder serve and girder-client respectively.

Removed insecure sha512 password hashing

The core.hash_alg and core.bcrypt_rounds configuration parameters were also removed. Password hashing now always occurs with 12-round bcrypt. Please reach out to us on Discourse if you have existing databases with sha512 passwords or believe you need to configure bcrypt to use additional rounds.

Core setting constants now reside in settings.py

The SettingKey and SettingDefault classes (which contain constants for core settings) must now be imported from the girder.settings module.

from girder.settings import SettingDefault, SettingKey

The API for sending email has changed

The mail_utils.sendEmail function has been replaced with several new functions: mail_utils.sendMailSync, mail_utils.sendMail, mail_utils.sendMailToAdmins, mail_utils.sendMailIndividually. Note that the argument order and expected types have changed. See function documentation for details on the new usage.

Removed or moved plugins

Many plugins were either deleted from the main repository, or moved to other repositories. Plugins are no longer installable via a [plugins] extra when installing the girder Python package; rather, all are installed by pip install girder-[plugin_name]. If you were depending on a plugin that was deleted altogether, please reach out to us on Discourse for discussion of a path forward.

The following plugins were deleted:

• celery_jobs
• item_previews
• jquery_widgets
• metadata_extractor
• mongo_search
• provenance
• treeview
• vega

The following plugins were moved to different repositories:

• candela
• curation (renamed to publication_approval)
• geospatial
• hdfs_assetstore
• item_tasks
• table_view
• worker

1.x → 2.x

Existing installations may be upgraded to the latest 2.x release by running pip install -U girder<3 and re-running girder-install web. You may need to remove node_modules directory from the installed girder package if you encounter problems while re-running girder-install web. Note that the prerequisites may have changed in the latest version: make sure to review System Dependency Reference prior to the upgrade.

Server changes

• The deprecated event 'assetstore.adapter.get' has been removed. Plugins using this event to register their own assetstore implementations should instead just call the girder.utility.assetstore_utilities.setAssetstoreAdapter method at load time.

• The 'model.upload.assetstore' event no longer supports passing back the target assetstore by adding it to the event.info dictionary. Instead, handlers of this event should use event.addResponse with the target assetstore as the response.

• The unused user parameter of the updateSize methods in the collection, user, item, and folder models has been removed.

• The unused user parameter of the isOrphan methods in the file, item, and folder models has been removed.

• Several core models supported an older, nonstandard kwarg format in their filter method. This is no longer supported; the argument representing the document to filter is now always called doc rather than using the model name for the kwarg. If you were using positional args or using the filtermodel decorator, this change will not affect your code.

• Multiple configurable plugin loading paths are no longer supported. Use girder-install plugin <your_plugin_path> to install plugins that are not already in the plugins directory. Pass -s to that command to symlink instead of copying the directory. This also means:

  • The plugins.plugin_directory and plugins.plugin_install_path config file settings are no longer supported, but their presence will not cause problems.
  • The defaultPluginDir, getPluginDirs, getPluginParentDir methods inside girder.utility.plugin_utilities were removed.
  • All of the methods in girder.utility.plugin_utilities no longer accept a curConfig argument since the configuration is no longer read.

• The girder.utility.sha512_state module has been removed.

• The girder.utility.hash_state module has been made private. It should not be used downstream.

Web client changes

• In version 1.x, running npm install would install our npm dependencies, as well as run the web client build process afterwards. That is no longer the case; npm install now only installs the dependencies, and the build is run with npm run build.

  • The old web client build process used to build all available plugins in the plugin directory. Now, running npm run build will only build the core code. You can pass a set of plugins to additionally build by passing them on the command like, e.g. npm run build -- --plugins=x,y,z.
  • The grunt watch command has been deprecated in favor of npm run watch. This also only watches the core code by default, and if you wish to also include other plugins, you should pass them in the same way, e.g. npm run watch -- --plugins=x,y,z.
  • The girder-install web command is now the recommended way to build web client code. It builds all enabled plugins in addition to the core code. The ability to rebuild the web client code for the core and all enabled plugins has been exposed via the REST API and the admin console of the core web client. The recommended process for administrators is to turn on all desired plugins via the switches, click the Rebuild web code button, and once that finishes, click the button to restart the server.

• Jade → Pug rename: Due to trademark issues, our upstream HTML templating engine was renamed from Jade to Pug. In addition, this rename coincides with a major version bump in the language which comes with notable breaking changes.

  • Template files should now end in .pug instead of .jade. This affects how they are imported as modules in webpack.

  • Jade-syntax interpolation no longer works inside string values of attributes. Use ES2015-style string templating instead. Examples:

    • a(href="#item/#{id}/foo") → a(href=`#item/${id}/foo`)
    • .g-some-element(cid="#{obj.cid}") → .g-some-element(cid=obj.cid)

  • Full list of breaking changes are listed here, though most of the others are relatively obscure.

• Testing specs no longer need to manually import all of the source JS files under test. We now have better source mapping in our testing infrastructure, so it’s only necessary to import the built target for your plugin, e.g.

  • 1.x:

    girderTest.addCoveredScripts([
        '/static/built/plugins/jobs/templates.js',
        '/plugins/jobs/web_client/js/misc.js',
        '/plugins/jobs/web_client/js/views/JobDetailsWidget.js',
        '/plugins/jobs/web_client/js/views/JobListWidget.js'
    ]);
    

  • 2.x:

    girderTest.importPlugin('jobs');
    

• Build system overhaul: Girder web client code is now built with Webpack instead of uglify, and we use the Babel loader to enable ES2015 language support. The most important result of this change is that plugins can now build their own targets based on the Girder core library in a modular way, by importing specific components. See the plugin development guide for a comprehensive guide on developing web-client plugins in the new infrastructure.

Python client changes

• Girder CLI: Subcommands are no longer specified with the -c option. Instead, the subcommand is specified just after all the general flags used for connection and authentication. For example:

  • Before: girder-cli --api-key=abcdefg --api-url=https://mygirder.org/api/v1 -c upload 1234567890abcdef ./foo
  • After: girder-cli --api-key=abcdefg --api-url=https://mygirder.org/api/v1 upload 1234567890abcdef ./foo

• The blacklist and dryrun kwargs are no longer available in the GirderClient constructor because they only apply to uploading. If you require the use of a blacklist, you should now pass it into the upload method. These options can still be passed on the CLI, though they should now come after the upload subcommand argument.

• Legacy method names in the GirderClient class API have been changed to keep naming convention consistent.

  • add_folder_upload_callback → addFolderUploadCallback
  • add_item_upload_callback → addItemUploadCallback
  • load_or_create_folder → loadOrCreateFolder
  • load_or_create_item → loadOrCreateItem

• All kwargs to GirderClient methods have been changed from snake_case to camelCase for consistency.

• Listing methods in the GirderClient class (e.g. listItem) have been altered to be generators rather than return lists. By default, they will now iterate until exhaustion, and callers won’t have to pass limit and offset parameters unless they want a specific slice of the results. As long as you are just iterating over results, this will not break your existing code, but if you were using other operations only available on lists, this could break. The recommended course of action is to modify your logic so that you only require iteration over the results, though it is possible to simply wrap the return value in a list() constructor. Use caution if you use the list() method, as it will load the entire result set into memory.

Built-in plugin changes

• Jobs: The deprecated jobs.filter event was removed. Use the standard exposeFields and hideFields methods on the job model instead.
• OAuth: For legacy backward compatibility, the Google provider was previously enabled by default. This is no longer the case.

Plugins

One of the most useful aspects of the Girder platform is its ability to be extended in almost any way by custom plugins. Developers looking for information on writing their own plugins should see the Plugin Development section. Below is a listing and brief documentation of some of Girder’s standard plugins that come pre-packaged with the application.

Audit Logging

PyPI package: girder-audit-logs

Authorized Uploads

PyPI package: girder-authorized-upload

This plugin allows registered users to grant access to others to upload data on their behalf via a secure URL. The secure URL allows a third party to upload a single file into the selected folder, even if that third party does not have a registered user in Girder.

To authorize an upload on behalf of your user:

1. Navigate into any folder to which you have write access. From the Folder actions dropdown menu on the right, choose Authorize upload here. You will be taken to a page that allows generation of a secure, single-use URL. You can optionally specify a number of days until the URL expires; if none is specified, the user session lifetime is used, which defaults to 180 days.
2. Click Generate URL, and your secure URL will appear below.
3. Copy that URL and send it to the third party, and they will be taken to a simple page allowing them to upload the file without having to see any details of the normal Girder application.

Note

When an upload is authorized, it’s authorized into a particular folder, and inherits the access control configured on that folder.

Auto Join

PyPI package: girder-autojoin

The Auto Join plugin allows you to define rules to automatically assign new users to groups based on their email address domain. Typically, this is used in conjunction with email verification.

When a new user registers, each auto join rule is checked to see if the user’s email address contains the rule pattern as a substring (case insensitive).

If there is a match, the user is added to the group with the specified access level.

DICOM Viewer

PyPI package: girder-dicom-viewer

The DICOM Viewer plugin adds support for previewing DICOM files when viewing an item in girder. If multiple DICOM files are present in a single item, they are presented as multiple slices. The DICOM image is shown as well as a table of DICOM tags. The window center and width can be changed by the user. Controls allow the user to step through slices, auto-level the window, auto-zoom, or playback the slices at different speeds.

This plugin parses the DICOM tags when files are uploaded and stores them in the MongoDB database for quick retrieval. This is mostly used to sort multiple images by series and instance.

DICOM imagery from: https://wiki.cancerimagingarchive.net/display/Public/RIDER+NEURO+MRI

Download Statistics

PyPI package: girder-download-statistics

This plugin tracks and records file download activity. The recorded information (downloads started, downloads completed, and total requests made) is stored on the file model:

file['downloadStatistics']['started']
file['downloadStatistics']['requested']
file['downloadStatistics']['completed']

Google Analytics

PyPI package: girder-google-analytics

The Google Analytics plugin enables the use of Google Analytics to track page views with the Girder one-page application. It is primarily a client-side plugin with the tracking ID stored in the database. Each routing change will trigger a page view event and the hierarchy widget has special handling (though it does not technically trigger routing events for hierarchy navigation).

To use this plugin, simply copy your tracking ID from Google Analytics into the plugin configuration page.

Gravatar Portraits

PyPI package: girder-gravatar

This lightweight plugin makes all users’ Gravatar image URLs available for use in clients. When enabled, user documents sent through the REST API will contain a new field gravatar_baseUrl if the value has been computed. If that field is not set on the user document, instead use the URL /user/:id/gravatar under the Girder API, which will compute and store the correct Gravatar URL, and then redirect to it. The next time that user document is sent over the REST API, it should contain the computed gravatar_baseUrl field.

Javascript clients

The Gravatar plugin’s javascript code extends the Girder web client’s girder.models.UserModel by adding the getGravatarUrl(size) method that adheres to the above behavior internally. You can use it on any user model with the _id field set, as in the following example:

import { getCurrentUser } from '@girder/core/auth';

const currentUser = getCurrentUser();
if (currentUser) {
    this.$('div.gravatar-portrait').css(
        'background-image', `url(${currentUser.getGravatarUrl(36)})`);
}

Note

Gravatar images are always square; the size parameter refers to the side length of the desired image in pixels.

Hashsum Download

PyPI package: girder-hashsum-download

The hashum_download plugin allows a file to be downloaded from Girder given a hash value and hash algorithm. Use this plugin when you have large data that you don’t want to keep in a software repository, but want to access that data from the repository, e.g. during a build or test of that software project. This plugin is written to satisfy the needs of CMake ExternalData. These docs describe how to use this plugin along with ExternalData, but the plugin could be used outside of that context. For more detailed documentation on how to use this in a software repository see the ITKExamples. This example project uses the Girder instance https://data.kitware.com.

Note

The use of the hashsum_download plugin with CMake ExternalData is only supported with a filesystem assetstore and SHA512 as the hash algorithm.

As every local Git repository contains a copy of the entire project history, it is important to avoid adding large binary files directly to the repository. Large binary files added and removed throughout a project’s history will cause the repository to become bloated and take up too much disk space, requiring excessive time and bandwidth to download.

A solution to this problem, when using the CMake build system, is to store binary files in a separate location outside the Git repository, then download the files at build time with CMake.

CMake uses the notion of a content link file, which contains an identifying hash calculated from the original data file. The content link file has the same name as the data file, with a “.sha512” extension appended to the file name, and should be stored in the Git repository. CMake will find these content link files at build time, download the corresponding data files from a list of server resources, and create symlinks or copies of the original files in the build tree, which is why the files are called “content links”.

What CMake calls a content link file, Girder calls a key file, as the notion of content link doesn’t apply in the context of Girder, and the hash value is a key into the original data file. When using the hashsum_download plugin, the data file is stored in Girder, with the SHA512 for the data added as metadata and provided as the key file, which can be downloaded from Girder and added to a project repository. The hashsum_plugin allows the data file to be downloaded based on the hash of the data. CMake ExternalData provides tooling to connect with a Girder instance, download the actual data file pointed to by the content link (key) file by passing the hash to Girder, and provide a local file path to access the data file contents.

Usage by a software project maintainer

Again, for more background, using the example Girder instance https://data.kitware.com, see the ITKExamples. Also see the CMake External Data documentation for CMake project configuration help.

In your project, you must set ExternalData_URL_TEMPLATES to a girder url, e.g. “https://data.kitware.com/api/v1/file/hashsum/%(algo)/%(hash)/download”.

See the ITK configuration for an example application of ExternalData_URL_TEMPLATES.

Project contributors will add data files to a Girder instance in arbitrary folders. At a project release and on a regular basis, perhaps nightly, the data should be archived in a new Girder folder to ensure its persistence. A script that provides this functionality is available, as is an example folder produced by the script for a release.

Usage by a software project contributor

Upload a file to a Girder instance, which will create a Girder Item to house the file. Navigate to the Item, then click on the i (information) icon next to the file, which will show the id, and since the hashsum_download plugin is enabled, the sha512 field will also be displayed. Click on the key icon to download a hashfile, which will be the full sha512 of the file, with the same name as the file, and an extension of .sha512, and you can use this key file as your CMake content link. E.g., upload my_datafile.txt and download the my_data.txt.sha512 file, then check the my_data.txt.sha512 file into your source repository.

You can use the Girder API to get the hash of the file given the file id, with the endpoint /api/v1/file/<file id>/hashsum_file/sha512, where the file id comes from the specific file in Girder.

You can also use the API to download the file based on the hash returned by the previous endpoint, with an endpoint /api/v1/file/hashsum/sha512/<file sha512 hash>/download, where the sha512 hash comes from the specific file in Girder.

Homepage

PyPI package: girder-homepage

The Homepage plugin allows the default Girder front page to be replaced by content written in Markdown format. After enabling this plugin, visit the plugin configuration page to edit and preview the Markdown.

Item Licenses

PyPI package: girder-item-licenses

Jobs

PyPI package: girder-jobs

The jobs plugin is useful for representing long-running (usually asynchronous) jobs in the Girder data model. Since the notion of tracking batch jobs is so common to many applications of Girder, this plugin is very generic and is meant to be an upstream dependency of more specialized plugins that actually create and execute the batch jobs.

The job resource that is the primary data type exposed by this plugin has many common and useful fields, including:

• title: The name that will be displayed in the job management console.
• type: The type identifier for the job, used by downstream plugins opaquely.
• args: Ordered arguments of the job (a list).
• kwargs: Keyword arguments of the job (a dictionary).
• created: Timestamp when the job was created
• progress: Progress information about the job’s execution.
• status: The state of the job, e.g. Inactive, Running, Success.
• log: Log output from this job’s execution.
• handler: An opaque value used by downstream plugins to identify what should handle this job.
• meta: Any additional information about the job should be stored here by downstream plugins.

Jobs should be created with the createJob method of the job model. Downstream plugins that are in charge of actually scheduling a job for execution should then call scheduleJob, which triggers the jobs.schedule event with the job document as the event info.

The jobs plugin contains several built-in status codes within the girder.plugins.jobs.constants.JobStatus namespace. These codes represent various states a job can be in, which are:

• INACTIVE (0)
• QUEUED (1)
• RUNNING (2)
• SUCCESS (3)
• ERROR (4)
• CANCELED (5)

Downstream plugins that wish to expose their own custom job statuses must hook into the jobs.status.validate event for any new valid status value, which by convention must be integer values. To validate a status code, the default must be prevented on the event, and the handler must add a True response to the event. For example, a downstream plugin with a custom job status with the value 1234 would add the following hook:

from girder import events

def validateJobStatus(event):
    if event.info == 1234:
        event.preventDefault().addResponse(True)

def load(info):
    events.bind('jobs.status.validate', 'my_plugin', validateJobStatus):

Downstream plugins that want to hook into job updates must use a different convention than normal; for the sake of optimizing data transfer, job updates do not occur using the normal save method of Girder models. Therefore, plugins that want to listen to job updates should bind to either jobs.job.update (which is triggered prior to persisting the updates and can be used to prevent the update) or jobs.job.update.after (which is triggered after the update). Users of these events should be aware that the log field of the job will not necessarily be in sync with the persisted version, so if your event handler requires access to the job log, you should manually re-fetch the full document in the handler.

LDAP Authentication

PyPI package: girder-ldap

This plugin allows administrators to configure the server so that users can log in against one or more LDAP servers. If the user fails to authenticate to any of the available LDAP servers, they will fall back to normal core authentication. Documentation of the LDAP standard in general can be found here.

Installation of this plugin requires LDAP and SASL shared libraries to be installed and available to the Girder process. These may be installed system-wide via package managers in the following way:

• On Ubnutu 18.04, install the libldap2-dev and libsasl2-dev APT packages.
• On RHEL (CentOS) 7, install the openldap-devel and cyrus-sasl-devel RPM packages.

Administrators can configure the ordered list of LDAP servers to try on the plugin configuration page. Each server in the list has several properties:

• URI: The URI of the LDAP server. Example: ldaps://my.ldap.org:636.
• Bind name: The Distinguished Name (DN) to use when connecting to the LDAP server to perform directory searches. Example: cn=me,cn=Users,dc=my,dc=ldap,dc=org.
• Password: (Optional) The password to use when connecting to the LDAP server to perform directory searches.
• Base DN: The Distinguished Name (DN) under which to search for users during login. Example: cn=Users,dc=my,dc=ldap,dc=org.
• Search field: (Optional) This specifies what field should be searched in the directory for the login field entered by the user. The default value is the uid field, though some implementations would want to use, e.g. mail. It is assumed that the search field will uniquely identify at most one user in the directory under the Base DN.

Note

This plugin is known to work against LDAP version 3. Using it with older versions of the protocol might work, but is not tested at this time.

OAuth2 Login

PyPI package: girder-oauth

This plugin allows users to log in using OAuth against a set of supported providers, rather than storing their credentials in the Girder instance. Specific instructions for each provider can be found below.

By using OAuth, Girder users can avoid registering a new user in Girder, leaving it up to the OAuth provider to store their password and provide details of their identity. The fact that a Girder user has logged in via an OAuth provider is stored in their user document instead of a password. OAuth users who need to authenticate with programmatic clients such as the girder-client python library should use API keys to do so.

Google

On the plugin configuration page, you must enter a Client ID and Client secret. Those values can be created in the Google Developer Console, in the APIS & AUTH > Credentials section. When you create a new Client ID, you must enter the AUTHORIZED_JAVASCRIPT_ORIGINS and AUTHORIZED_REDIRECT_URI fields. These must point back to your Girder instance. For example, if your Girder instance is hosted at https://my.girder.com, then you should specify the following values:

AUTHORIZED_JAVASCRIPT_ORIGINS: https://my.girder.com
AUTHORIZED_REDIRECT_URI: https://my.girder.com/api/v1/oauth/google/callback

After successfully creating the Client ID, copy and paste the client ID and client secret values into the plugin’s configuration page, and hit Save. Users should then be able to log in with their Google account when they click the log in page and select the option to log in with Google.

Extension

This plugin can also be extended to do more than just login behavior using the OAuth providers. For instance, if you wanted some sort of integration with a user’s Google+ circles, you would add a custom scope that the user would have to authorize during the OAuth login process.

from girder.plugins.oauth.providers.google import Google
Google.addScopes(['https://www.googleapis.com/auth/plus.circles.read'])

Then, you can hook into the event of a user logging in via OAuth. You can hook in either before the Girder user login has occurred, or afterward. In our case, we want to do it after the Girder user has been fetched (or created, if this is the first time logging in with these OAuth credentials).

def readCircles(event):
    # Read user's circles, do something with them
    if event.info['provider'] == 'google':
        token = event.info['token']
        user = event.info['user']
        ...

from girder import events
events.bind('oauth.auth_callback.after', 'my_plugin', readCircles)

Note

If event.preventDefault() is called in the event handler for oauth.auth_callback.before or oauth.auth_callback.after, the OAuth callback does not create a new Girder Token, nor sets a new authentication cookie.

README

PyPI package: girder-readme

This plugin will render a README item found in a folder on the folder hierarchy view.

Sentry

PyPI package: girder-sentry

The Sentry plugin enables the use of Sentry to detect and report errors in Girder.

Terms of Use

PyPI package: girder-terms

This plugin allows collection admins to define a set of textual “Terms of Use”, which other users must accept before browsing within the collection. The terms may be set with markdown-formatted text, and users will be required to re-accept the terms whenever the content changes. Logged-in users have their acceptances stored and remembered permanently, while anonymous users have their acceptances stored only on the local browser.

Thumbnails

PyPI package: girder-thumbnails

User and Collection Quotas

PyPI package: girder-user-quota

Virtual Folders

PyPI package: girder-virtual-folders

Worker

This plugin should be enabled if you want to use the Girder worker distributed processing engine to execute batch jobs initiated by the server. This is useful for deploying service architectures that involve both data management and scalable offline processing. This plugin provides utilities for sending generic tasks to worker nodes for execution. The worker itself uses celery to manage the distribution of tasks, and builds in some useful Girder integrations on top of celery. Namely,

• Data management: This plugin provides python functions for building task input and output specs that refer to data stored on the Girder server, making it easy to run processing on specific folders, items, or files. The worker itself knows how to authenticate and download data from the server, and upload results back to it.
• Job management: This plugin depends on the Jobs plugin. Tasks are specified as python dictionaries inside of a job document and then scheduled via celery. The worker automatically updates the status of jobs as they are received and executed so that they can be monitored via the jobs UI in real time. If the script prints any logging information, it is automatically collected in the job log on the server, and if the script raises an exception, the job status is automatically set to an error state.

Release Notes

This is the summary list of changes to Girder between each release. For full details, see the commit logs at https://github.com/girder/girder

Unreleased

Changes

• Move minimum node version to 8.x due to upstream packages using newer ES features. (#2707).
• Require MongoDB 3.4+ (#2900)
• Removed the @access.cookie decorator; this is now an option on the other access decorators.

Bug Fixes

Server

• Fixed a bug where removing admin status from a user via the REST API did not work. (#2720).

Web Client

• Ensure that a text input doesn’t appear after downloading checked items. (#2736)

Python Client

• Added support for uploading empty files. (#2714).

Added Features

• Google Storage is provisionally supported via S3 Assetstore (#2876).
• Added an NPM_EXE environment variable and –npm flag to girder build to configure the npm executable used. (#2826).
• Added a privacy notice link to the footer which can be set on the Server Configuration view of the web client ( #2728).
• Added a setting to disable the notification stream. This may improve Girder’s performance in runtime environments with fewer threads (#2712).
• Added a task information panel that shows remote worker status. This is accessible from the jobs list when the remote worker plugin is enabled and also from the worker plugin configuration page. (#2678)
• Allow users to login with two-factor authentication via TOTP on a mobile authenticator app. (#2655).

Python Client

• Added a --token option to the girder-client command line interface to allow users to specify a pre-created authentication token. (#2689).
• Added a --retry option to the girder-client command line interface to retry connection and certain error responses (#2697).
• Added a --verbose option to the girder-client command line interface to increase the verbosity of information dumped to stderr (#2699).

Web Client

• Filesystem and S3 assetstore imports show the selected destination resource path next to the ID when selected via the browser widget (#2775).

Changes

• The Server FUSE plugin has been refactored into the girder mount command. Instead of setting a value in the configuration file and turning on a plugin, run girder mount <mount path> before or after starting a Girder server. (#2691)

Girder 2.5.0

Added Features

• Added a new system setting that will allow admins to disable logging in via a password. If disabled, the login dialog in the web client will no longer show the password login form. (#2504)
• Added a new system setting that will allow admins to disable the API key authentication functionality. (#2438)
• API endpoint in the hashsum_download plugin that returns a list of files matching a given hash sum. (#2548)
• Integration with dogpile.cache for caching key value pairs via girder.utility._cache.cache and girder.utility._cache.requestCache. (#2274)
• Plugins can customize the header and description on the Swagger page. (#2607)
• Common Girder operations can now be executed with a top level girder command. (#2596)
• Added the server FUSE plugin that mounts Girder files in a read-only user-space filesystem. (#2521)
• Added support for the --host flag to girder serve to allow dynamically setting the host. (#2570)
• Added support for running the Python test suite via the tox command. (#2528)
• Created a new “virtual folders” plugin. This plugin allows administrators to configure special read-only folders whose list of child items comes from a custom, user-defined database query. These folders act like database “views” into the item collection. (#2620)
• Added a File().getLocalFilePath method. (#2633)
• The stream logger handles more properties. (#2522)
• Future-proof for CherryPy removal of response timeouts. (#2487)
• Only use new style Python 2 classes. (#2656)
• Allow cancellation of raw Celery tasks. (#2602)
• Allow assetstore implementations from models besides the assetstore model itself. This enables assetstore adapters in plugins to be managed by users who are not necessarily site administrators. (#2599)
• Add validation logic to rest endpoint paging parameters. (#2462)
• Add rest endpoint to send user validation email. (#2622)
• Add a search mode registry and a search mode for dicom metedata. (#2450)
• Allow creation of item_tasks tasks from girder_worker describe decorators. (#2270)
• New plugin to allow browsing of Girder data in a tree view. (#2086)

Python Client

• Add a method to stream contents of a file download. (#2476)

Web Client

• Added a new, more fully-featured view for search results. (#2347)
• For added safety, when deleting a collection a user will now be required to type the name of the collection into the confirmation dialog. (#2473)
• New table_view plugin renders .csv and .tsv files as tables on the item page. (#2480)
• Modal dialogs have a default maximum height and will have a scroll bar if needed. (#2523)
• We now use the webpack DefinePlugin to add a build-time definition of the environment. This can be used to allow for different build output in production vs. development. (#2631)
• Use href properties for navigation links in addition to JavaScript onclick events. (#2489) (#2578)
• Change instances of .g-server-config to .g-server-config a to enable adding of href properties to those links
• Add new methods: folder.removeContents, item.getFiles, user.fromTemporaryToken. (#2615)

Swagger Client

• Swagger now expects zip files to be binary data, allowing them to be downloaded through the Web API. (#2562)

Testing

• PYTHONPATH can be specified for client tests. (#2535)

• Support for writing server-side tests using pytest. (#2412)

  • Added the pytest-girder package for downstream packages.
  • Added support for the mongomock package in the new pytest suite.
  • Plugins can be enabled for Pytest. (#2634)

• Flake8 settings are now able to be automatically detected by many editors and IDEs. The flake8 tool may now be invoked directly from the command line, without necessarily using a CMake test. (#2543)
• ESLint settings for plugin tests are now able to be automatically detected by many editors and IDEs. The eslint tool (including options such as --fix) may now be invoked directly from the command line, without necessarily using a CMake test. (#2550)

Bug fixes

Server

• Support range requests of S3 non-redirected data handling. This fixes seeking on S3 assetstore files in the file context handler. (#2468)
• Pin to a specific version of CherryPy to work around upstream issues on OPTION endpoints. (#2499)
• When a plugin supplying an assetstore fails to load, other assetstores could not be listed. (#2498)
• Run pip installation of plugins using a subprocess rather than the pip module, for forward compatbility with pip. (#2669)
• Correct complex plugin dependencies parsing. (#2496)

Security Fixes

• The default Girder server now binds to localhost by default instead of 0.0.0.0. (#2565)

Changes

• Exceptions are now all accessible in the exceptions module and are descended from the GirderBaseException class. (#2498)
• Require npm 5.2+ (with npm 5.6+ strongly recommended) to build the web client
• Require MongoDB 3.2+ (#2540)
• Disable the background event thread in WSGI mode. (#2642)
• Update imports of library from “dicom” to “pydicom”. (#2617)
• A log message is now emitted whenever a file is uploaded. (#2571)

Deprecations

• Server side tests should be written using the new pytest infrastructure.
• Move CLI commands to a “cli” module and deprecate “python -m” methods for starting Girder servers. (#2616 <https://github.com/girder/girder/pull/2616>)

Removals

• The CMake options PYTHON_COVERAGE, PYTHON_BRANCH_COVERAGE, and PYTHON_COVERAGE_CONFIG are removed, and will have no effect if set. Python tests will always output coverage information, using a standardized configuration. If external test infrastructure needs to be run with different options, it should invoke pytest -cov-config ... or coverage run --rcfile=... directly. (#2517)
• The CMake options COVERAGE_MINIMUM_PASS and JS_COVERAGE_MINIMUM_PASS are removed, and will have no effect if set. If external test infrastructure needs to set a coverage threshold, it should be done with a Codecov (or similar service) configuration. (#2545)
• The CMake options ESLINT_CONFIG_FILE and ESLINT_IGNORE_FILE are removed, and will have no effect if set. If external test infrastructure needs to override ESLint configuration, it should be done using ESLint’s built-in configuration cascading mechanisms. Most typical external plugins will continue to work with their current configuration.
• The deprecated DELETE /user/password endpoint is removed. The PUT /user/password/temporary endpoint should always be used to reset passwords, as it uses a secure, token-based password mechanism. (#2621)
• Dropped support for Python3 < 3.5. (#2572)

Girder 2.4.0

Added Features

Server

• Support for S3 buckets in regions other than us-east-1. (#2153)
• Allow S3 credentials to be inferred by Boto. (#2229)
• girder-shell console script which drops the user into a python repl with a configured webroot, giving the user the ability to import from any of the plugins specified. (#2141)
• Support for configuration of pymongo client options as Girder config file options. (#2380)
• Support for idiomatic use of Girder’s model classes. Rather than using ModelImporter.model with strings for the model and plugin names, you can now use python imports of the model classes and instantiate and use them directly. (#2376)
• Support for mounting REST endpoints under a prefix. Useful for grouping related endpoints, such as those exposed by a plugin. (#2395)
• Option in worker task input specs to use local file paths in the worker when available, to avoid downloading files. (#2356)
• Core setting allowing the instance brand name to be set. (#2283)
• Core setting allowing the instance contact email address to be set. (#2279)
• Core setting allowing the GUI header color to be set. (#2334)
• “terms” plugin, which provides the option to require users to agree to a “Terms of Use” before accessing a collection. (#2138)
• Improve the “homepage” plugin’s capabilities for making in-place changes to the home page. (#2328)
• API endpoint, “/user/details”, allowing site admins to list the total number of users. (#2262)
• Job cancellation support to Girder Worker jobs. (#1983)
• Accept metadata on item and folder creation via the REST API. (#2259)
• Allow girder-install plugin to get dependencies from a setup.py file. (#2370)
• Create a registry for adding new search modes. (#2363)

Web Client

• Published the Girder client side code as an npm package (https://www.npmjs.com/package/girder). (#2242)

Python Client

• Support for turning off certificate checking with --no-ssl-verify. (#2433)
• Support for specifying a custom certificate with --certificate. (#2267)
• Support for downloading individual files. (#2429)

DevOps

• Added a Terraform module for creating an S3 bucket compliant with Girder assetstore policies. (#2267)
• Published a latest-py3 tag to Dockerhub for Girder images built with Python 3. (#2321)

Deprecations

Python Client

• requests.HTTPError should be caught in places where girder_client.HttpError used to. (#2223)

Bug fixes

Server

• Fixed an error where certain filenames could cause broken Content-Disposition header values. (#2330)
• AccessControlledModel.load fields argument now works more reliably. (#2366, #2352)
• Fixed an issue where the events daemon was failing to terminate correctly. (#2379)

Web Client

• Remove Bootstrap re-styling of tooltips. (#2406)

DevOps

• Fixed an issue which disallowed provisioning with the Girder Ansible module under Python 3. (#2449)

Girder 2.3.0

Bug fixes

• Fix uploading into HDFS assetstore using new upload mode

Security Fixes

• Ensure token scopes on API keys are valid
• Add secure cookie setting
• Upgrade swagger-ui version to fix XSS issues

Added Features

• Add REST endpoint for creating job models
• Add graphs for Jobs status history to Admin UI
• Improvements to item_tasks job execution, task import, task lists, and permission flag UIs
• Show plugin load failures on plugins page
• Add Candela plugin
• Compute missing hashes when files are uploaded, and allow for hashsum calculation for non-filesystem assetstore files
• Add support for running Girder in AWS Elastic Beanstalk
• Upgrade S3 assetstore to Boto3
• Add LDAP authentication plugin
• Send all http server errors to the error log
• Added an event when the web client connection to the server is stopped or started
• Support uploading small files in a single REST call
• Improved GridFS support, including better sharding support and faster writes
• Add a Description method to mark a route as deprecated
• Many improvements to the web client test infrastructure including
  • A new CMake macro, add_standard_plugin_tests, to enable basic tests for a typical plugin layout
  • A new girderTest.importPlugin function, to load plugin JS and CSS in web client tests
  • A static analysis test for Stylus files
  • New rules for Javascript and Pug static analysis tests
• A facility to initialize the database to a specific state for testing

Changes

• Upgrade web client to use jQuery 3
• Upgrade web client to use Backbone 1.3
• Require Node.js 6.5+ and npm 3.10+ (with npm 5.3 strongly recommended) to build the web client

Deprecations

• job Plugin: Python Job model listAll method
• hashsum_download plugin: Python HashedFile.supportedAlgorithms symbol
• item_tasks plugin: item_task_json_description and item_task_json_specs routes
• module.loaders in webpack helper files, and the use of Webpack 1.0 syntax in plugins’ webpack helper files
• restRequest.error in rest.js
• npm-install in client side build
• girderTest.addCoveredScript and girderTest.addCoveredScripts in testUtilities.js
• access to file paths outside /static/built/ in the web client test environment

Removals

• Remove the unmaintained external web client
• Remove the unmaintained jQuery “girderBrowser” client, and associated “jquery_widgets” plugin

API index

• Index
• Module Index