User Guide

Girder is a Data Management Toolkit. It is a complete back-end (server side) technology that can be used with other applications via its RESTful API, or it can be used via its own front-end (client side web pages and JavaScript).

Girder is designed to be robust, fast, scalable, extensible, and easy to understand.

Girder is built in Python.

Girder is open source, licensed under the Apache License, Version 2.0.

Document Conventions

This User Guide is written for end-users of Girder, rather than developers. If you have suggestions or questions about this documentation, feel free to contact us through our Github Discussions forum, on GitHub, or through Kitware support.

Girder specific entities will be formatted like this.

Concepts

Users

Like in many systems, Users in Girder correspond to the identity of a user of the system. It is possible to use many features of Girder anonymously (that is, without being logged in as a registered user), but typically in order to make changes to the system, a user must be logged in as their corresponding User account. Users can be granted permissions on resources in the system directly, and can belong to Groups.

Groups

Groups group together Users. Users can belong to any number of Groups, and usually join by being invited and accepting the invitation. One of the main purposes of Groups is to allow role-based access control; resources can grant access to Groups rather than just individual users, such that changing access to sets of resources can be managed simply by changing Group membership. See the Permissions section for more information about group-based access control.

Collections

Collections are the top level objects in the data organization hierarchy. Within each Collection, there can be many Folders, and the Collection itself is also an access controlled resource. Typically Collections are used to group data that share something in common, such as what project the data are used for, or what institution they belong to.

Folders

A Girder Folder is the common software concept of a folder, namely a hierarchically nested organizational structure. Girder Folders can contain nothing (although this may not be particularly useful), other Folders, Items, or a combination of Folders and Items. Folders in Girder have permissions set on them, and the Items within them inherit permissions from their containing Folders.

Items

A Girder Item is the basic unit of data in the system. Items live beneath Folders and contain 0 or more Files. Items in Girder do not have permissions set on them, they inherit permissions by virtue of living in a Folder (which has permissions set on it). Most Items contain a single File, except in special cases where multiple files make up a single piece of data.

Each Item may contain any number of arbitrary key/value pairs, termed metadata. Metadata keys must be non-empty strings and must not contain a period (‘.’) or begin with a dollar sign (‘$’). Metadata values can be anything, including strings, numeric values, and even arbitrary JSON objects.

Files

Files represent raw data objects, just like the typical concept of files in a filesystem. Files exist within Items, typically with a one-to-one relationship between the File and its containing Item. Files in Girder are much like files on a filesystem, but they are actually more abstract. For instance, some Files are simply links to external URLs. All Files that are not external links must be contained within an Assetstore.

Assetstores

Assetstores are an abstraction representing a repository where the raw bytes of Files are actually stored. The Assetstores known to a Girder instance may only be set up and managed by administrator Users.

In the core of Girder, there are three supported Assetstore types:

Filesystem

Files uploaded into this type of Assetstore will be stored on the local system filesystem of the server using content-addressed storage. Simply specify the root directory under which files should be stored.

Note

If your Girder environment has multiple different application servers and you plan to use the Filesystem assetstore type, you must set the assetstore root to a location on the filesystem that is shared between all of the application servers.

S3

This Assetstore type stores files in an Amazon S3 bucket. You must provide the bucket name, an optional path prefix within the bucket, and authentication credentials for the bucket. When using this assetstore type, Girder acts as the broker for the data within S3 by authorizing the user agent via signed HTTP requests. The primary advantage of this type of assetstore is that none of the actual bytes of data being uploaded and downloaded ever go through the Girder system, but instead are sent directly between the client and S3.

If you want to use an S3 assetstore, the bucket used must support CORS requests. This can be edited by navigating to the bucket in the AWS S3 console, selecting Properties, then Permissions, and then clicking Edit CORS Configuration. The below CORS configuration is sufficient for Girder’s needs:

[
    {
        "AllowedHeaders": [
            "*"
        ],
        "AllowedMethods": [
            "GET",
            "PUT",
            "POST"
        ],
        "AllowedOrigins": [
            "*"
        ],
        "ExposeHeaders": [
            "ETag"
        ],
        "MaxAgeSeconds": 3000
    }
]

Note

Google Storage is supported through an S3 assetstore. Specify the service as storage.googleapis.com. Public storage buckets can be accessed in read-only mode without an access key ID or secret access key.

Support for Google Storage may not be complete, as it is accessed via an S3 access library and is not automatically tested.

Permissions

Permission Levels

There are four levels of permission a User can have on a resource. These levels are in a strict hierarchy with a higher permission level including all of the permissions below it. The levels are:

No permission (cannot view, edit, or delete a resource)
READ permission (can view and download resources)
WRITE permission (includes READ permission, can edit the properties of a resource)
ADMIN also known as own permission, (includes READ and WRITE permission, can delete the resource and also control access on it)

A site administrator always has permission to take any action.

Permission Model

Permissions are resolved at the level of a User, i.e., for any User, an attempt to take a certain action will be allowed or disallowed based on the permissions for that User, as a function of the resource, the operation, the permissions set on that resource for that User, and the permissions set on that resource by any Groups the User is a member of.

Permissions are always additive. That is, given a User with a certain permission on a resource, that permission can not be taken away from the User by addition of other permissions to the system, but only through removing existing permissions to that User or removing that User from a Group. Once again, a site admin always has permission to take any action.

Collections

Collections can be Public (meaning viewable even by anonymous users) or Private (meaning viewable only by those with READ access). Collections can have permissions set on them at the individual User level and Group level, meaning that a given User or Group can have READ, WRITE, or ADMIN permissions set on the Collection.

Folders

Folders can be Public (meaning viewable even by anonymous users) or Private (meaning viewable only by those with READ access). Folders can have permissions set on them at the individual User level and Group level, meaning that a given User or Group can have READ, WRITE, or ADMIN permissions set on the Folder. Folders inherit permissions from their parent Folder.

Items

Items always inherit their permissions from their parent Folder. Each access-controlled resource (e.g., Folder, Collection) has a list of permissions granted on it, and each item in that list is a mapping of either Users to permission level or Groups to permission level. This is best visualized by opening the “Access control” dialog on a Folder in the hierarchy. The actual permission level that a User has on that resource is defined as: the maximum permission level available based on the permissions granted to any Groups that the User is member of, or permissions granted to that User specifically.

Groups

For access control, Groups can be given any level of access to a resource that an individual User can, and this is managed at the level of the resource in question.

For permissions on Groups themselves, Public Groups are viewable (READ permission) to anyone, even anonymous users. Private Groups are not viewable or even listable to any Users except those that are members of the Group, or those that have been invited to the Group.

Groups have three levels of roles that Users can have within the Group. They can be Members, Moderators (also indicates that they are Members), and Administrators (also indicates that they are Members).

Users that are not Members of a group can request to become Members of a Group if that Group is Public.

Members of a Group can see the membership list of the Group, including roles, and can see pending requests and invitations for the group. If a User has been invited to a Group, they have Member access to the Group even before they have accepted the invitation. A Member of a Group can leave the group, at which point they are no longer Members of the Group.

Moderators of a Group have all of the abilities of Group Members. Moderators can also invite Users to become Members, can accept or reject a request by a User to become a Member, can remove Members or Moderators from the Group, and can edit the Group which includes changing the name and description and changing the Public/Private status of the Group.

Administrators of a Group have all of the abilities of Group Moderators. Administrators can also delete the Group, promote a Member to Moderator or Administrator, demote an Administrator or Moderator to Member, and remove any Member, Moderator, or Administrator from the Group.

The creator of a Group is an Administrator of a group. Any logged in User can create a Group.

User

Users have ADMIN access on themselves, and have READ access on other Users.

API keys

Like many web services, Girder’s API is designed for programmatic interaction. API keys can facilitate these sorts of interactions – they enable client applications to interact with the server on behalf of your user without actually authenticating with your password. They can also be granted restricted access to only a limited set of functionality of the API.

Under the My account page, there is a tab called API keys where these keys can be created and managed. You can have many API keys; in fact, it’s recommended to use a different key for each different client application that needs authenticated access to the Girder server. By convention, the Name field of API keys can be used to specify what application is making use of the key in a human-readable way, although you may name your keys however you want.

Each API key can be used to gain authentication tokens just like when you log in with a username and password. If you want to limit the maximum amount of time that these tokens last, you can do so on a per-key basis, or leave the token duration field empty to use the server default.

When creating and updating API keys, you can also select among two modes: you can either grant full access to the API key, which gives unrestricted API access as though you are logged in as your user, or you can choose limited functionality scopes from a list of checkboxes to restrict the sorts of actions that the key will allow.

It is also possible to deactivate a key temporarily. If you deactivate an existing key, it will immediately delete all active tokens created with that key, and also stop that key from being able to create new tokens until you activate it once again. Alternatively, you can delete the key altogether, which will make the key and any tokens created with it never work again.

Using Girder CLI to Upload and Download data

See Python Client and Girder CLI