Configuration

Configuring the CCP-EM Pipeliner

The CCP-EM Pipeliner finds external software by searching for the program executables in the directories listed in the PATH environment variable. To add new software, you can do one of the following things:

• Add the directory containing the program executables (typically called bin/) to your PATH, for example in a bash terminal you can run

  export PATH="/path/to/new/program/bin:$PATH"
  

  or put that line in your .bashrc file so it is run automatically when you start a new shell.

• Add the directory containing the program executables (typically called bin/) to the "additional_program_paths" list in the CCP-EM Pipeliner settings file. When the Pipeliner is first used (or Doppio is first opened) a JSON file will be created in the system’s default user config directory. On Linux, this is usually ~/.config/ccpem/pipeliner/ccpem-pipeliner-settings.json and on Mac ~/Library/Application Support/ccpem/pipeliner/ccpem-pipeliner-settings.json.

Typically, if you intend to run Doppio from the command line, it’s best to set up the external software in your .bashrc (or equivalent) setup file, because then you can interactively check if the programs can be found in the shell environment, and if they can be found by the shell then they will also be found by Doppio.

If you intend to run Doppio by clicking on an icon (the Mac app or the Linux AppImage file) then usually your system’s shell setup scripts will not be read first, and so it is better to add external software to the Pipeliner settings file.

The CCP4 and CCP-EM v1 software packages use setup scripts that must be “sourced” to set up the environment correctly. To do this in your shell, edit your .bashrc (or equivalent) file and add the following lines, modifying the paths as appropriate for your installation:

source "/path/to/ccpem-20221108/setup_ccpem.sh"
source "/path/to/ccp4-9/bin/ccp4.setup-sh"

Or to do this in your Pipeliner settings file, add the files to the "path_to_source_files" setting, for example:

{
  "path_to_source_files": [
    "/path/to/ccpem-20221108/setup_ccpem.sh",
    "/Applications/ccp4-9.0/bin/ccp4.setup-sh"
  ]
}

Note

The JSON format is very strict, make sure all quotes and brackets are closed correctly, items in a list are separated by commas and that the last item in a list does not end with a comma.

The changes will take effect the next time the Pipeliner (or Doppio) is started.

Doppio Settings

Configuring server settings

Typically, the Doppio API server does not need any configuration. However, if required, it can be configured using command line arguments and environment variables when launching the program.

Host

For example, doppio-web --host localhost.

The host name or IP address that the server listens on. The default is 127.0.0.1, which will make Doppio available only on the computer that the server is running on.

It is possible to make Doppio available on your local network by providing address 0.0.0.0 (or another name or address that has been configured for external access). This has important security implications (see warning below). Also note that remote access to Doppio is not yet fully supported and some functions will not work when the client and server are on different computers, for example launching external GUI programs.

Warning

Doppio does not provide any user authentication or access control, and runs with the privileges of the user who starts the server. You should assume that anyone who is able to connect to Doppio will be able to run arbitrary code on the server.

On a shared computer, this means that any user of the computer would be able to connect to Doppio and could potentially run malicious code. Please DO NOT run Doppio on a shared computer unless you completely trust all users.

If you make the server available on a network, anyone who can connect to the port Doppio is listening on could potentially run malicious code. Unless you are totally sure of your network security arrangements we recommend you do not allow remote access to Doppio.

Port

For example, doppio-web --port 1430.

The port number to use for the server. If this is not provided or is given as 0, an available port will be chosen automatically and logged to the terminal when the server starts.

No browser

doppio-web will normally try to open your default web browser to view Doppio when it starts. To prevent this, use the --no-browser argument: doppio-web --no-browser.

Debug mode

Doppio debug mode is activated by setting the DOPPIO_DEBUG environment variable to a non-empty value, for example, DOPPIO_DEBUG=1 doppio-web. In debug mode, full stack traces will be printed to the server log if there is an error.

Base URL

If needed, you can set a base URL prefix for Doppio by setting the DOPPIO_BASE_URL environment variable. For example, if you run Doppio like this:

DOPPIO_BASE_URL=/abc123/ doppio-web --host localhost --port 1430

then the URL used to access Doppio would be http://localhost:1430/abc123/ rather than the normal default of http://localhost:1430/.

Note

The base URL setting is new and some Doppio features (for example uploading a file) do not yet work correctly when a base URL is set.

Authentication token

If necessary, and if you fully understand the security implications, you can control the authentication token used by the Doppio server by setting the DOPPIO_TOKEN environment variable. See Security in Doppio for more information.

Configuring application settings

Doppio application settings and themes are stored in the doppio-config/ directory.

Note

For Mac:

• on Mac, the configuration will be loaded from ~/Library/Application Support/ccpem/doppio/doppio-config

For Linux:

• on Linux, the configuration will be loaded from ~/.config/ccpem/doppio/doppio-config

Note

The pipeliner and Doppio settings files have moved. If you used previous versions, you will have old settings in ccpem-pipeliner/ and/or doppio/ directories in your user config directory (rather than the new ccpem/pipeliner/ and ccpem/doppio/ ones). The old settings files will no longer be used. If you made changes, you will need to copy them into the new settings files.

Some configuration, like the theme, will be reloaded by the application on a page refresh.

The application settings are stored as a .json file in doppio-config/app-settings.json

apiCheckTimeoutMs

Doppio periodically checks the availability of the API server. This is the time in milliseconds that it waits before displaying an error.

maxResultSizeMB

This is the maximum size of any result files that will be downloaded to view in Doppio.

projectDirectory

Directory of the project, relative to the user’s home directory or as an absolute path. For example, if projectDirectory is set to ccpem-project (the default) the API will open the project ~/ccpem-project. If a project does not already exist in that location, a new one will be created. This setting can also be modified from the Doppio GUI using the settings menu.

Warning

The current project is a global setting shared between all instances of Doppio run by the same user on the same computer. If you run multiple copies of Doppio, they will all work on the same project, and if one of them is used to change the project, they will all switch to the new one. Support for working with multiple projects simultaneously will be added in a future version of Doppio.

suggestHiddenFilesAsInput

This specifies the behaviour of hidden files (i.e., files starting with “.”) when setting job input parameters.

When set to True, hidden files will be shown. Otherwise, they will not be visible in the suggestion menu. The user can still access them by typing the full path, but they will not be aided by the suggestions.

themeFile

This specifies a theme file which contains many addition parameters defining how the Doppio looks and which jobs are displayed. This parameter must be set to the name of one of the theme files contained in doppio-config/themes/. One of the existing themes can ben used or a new them file created. The selected theme file can also be set through the settings menu in the GUI.

Creating theme files

The UI theme configuration is stored as a .json file in doppio-config/themes/theme.json. Themes define how the GUI looks, which jobs are displayed by default, and what order they will appear in.

There are currently three example theme files:

• doppio-default.json: All jobs in Doppio’s default order

• relion_SPA.json: Only jobs associated with single particle processing in RELION, in the same order as the original RELION GUI

• relion_SPA_helical: Jobs associated with helical processing in RELION, in the same order as the orignal RELION GUI

New themes can be created by writing a .json file with the following parameters and putting it in doppio-config/themes. Any omitted parameter will be replaced by its default value.

Theme file parameters

primary_color and secondary_color

These are the two main colors used by the application in the header, buttons, icons, etc.

Only the main shade needs to be provided, as all others can be calculated automatically. If you specify other shades, however, this is the value that will be used.

E.g.:

"primary_color": {
  "main": "#3f50b5",
  "light": "#757ce8",
  "dark": "#002884",
  "contrast_text": "#ffffff"
}

font_family

A list of fonts to be used for the application text. The first font available on the host system will be used.

E.g.:

"font_family": [
  "Source Code Pro",
  "Lato",
  "Segoe UI",
  "Roboto",
  "Oxygen",
  "Ubuntu",
  "Cantarell",
  "Fira Sans",
  "Droid Sans",
  "Helvetica Neue",
  "sans-serif"
]

code_font_family

A list of monospace fonts to be used when displaying code (e.g., when displaying the command used to run a job). The first font available on the host system will be used.

E.g.:

"code_font_family": [
  "Source Code Pro",
  "monospace"
],

banner_height

Height of the application banner. This can be expressed in all valid CSS length units (e.g., px, rem, %, cm, etc.)

E.g.:

"banner_height": "2.5em"

left_panel_min_width

Width of the application left panel. This can be expressed in all valid CSS length units (e.g., px, rem, %, cm, etc.) and must be above 200px.

E.g.:

"left_panel_min_width": "300px"

appName

This is a name you want to give to the app. It will be shown in the window title (or tab name for the web version) and in the application banner. This will not replace the CCP-EM Doppio name, rather it will be appended to it. Leave the field to null to keep the default name only. Reload the page for changes to take effect.

appLogo

This is the name of an image inside the doppio-config folder you want to use as a logo for the application (in addition to the CCP-EM one). The image should be in a supported format. Leave the field to null to keep the default logo only. Reload the page for changes to take effect.

appName and appLogo can be combined to customize the app header, e.g.:

visibleJobs

This is a list of job names (e.g., relion.multibody.refine) that are to be displayed by default in the New Job tab. This allows institutions to customize the list of jobs they want to provide their users with. The full job list can still be accessed from the UI via a settings menu.

If the field is omitted (or set to null), the full list of jobs will be shown.

The visibleJobs list also allows to control the order of the jobs. Categories are ordered by the order of occurrence of the first job of each. Within each category, jobs are displayed in the same order as visibleJobs.

When viewing the full list, jobs are displayed in the same order they are returned by the Pipeliner.