Docker¶

Environment Variables¶

S3DATA¶

S3DATA=multiple¶

This variable enables running CloudServer with multiple data backends, defined as regions.

For multiple data backends, a custom locationConfig.json file is required. This file enables you to set custom regions. You must provide associated rest_endpoints for each custom region in config.json.

Learn more about multiple-backend configurations

If you are using Scality RING endpoints, refer to your customer documentation.

Running CloudServer with an AWS S3-Hosted Backend¶

To run CloudServer with an S3 AWS backend, add a new section to the locationConfig.json file with the aws_s3 location type:

(...)
"awsbackend": {
    "type": "aws_s3",
    "details": {
        "awsEndpoint": "s3.amazonaws.com",
        "bucketName": "yourawss3bucket",
        "bucketMatch": true,
        "credentialsProfile": "aws_hosted_profile"
    }
}
(...)

Edit your AWS credentials file to enable your preferred command-line tool. This file must mention credentials for all backends in use. You can use several profiles if multiple profiles are configured.

[default]
aws_access_key_id=accessKey1
aws_secret_access_key=verySecretKey1
[aws_hosted_profile]
aws_access_key_id={{YOUR_ACCESS_KEY}}
aws_secret_access_key={{YOUR_SECRET_KEY}}

As with locationConfig.json, the AWS credentials file must be mounted at run time: -v ~/.aws/credentials:/root/.aws/credentials on Unix-like systems (Linux, OS X, etc.), or -v C:\Users\USERNAME\.aws\credential:/root/.aws/credentials on Windows

Note

One account cannot copy to another account with a source and destination on real AWS unless the account associated with the accessKey/secretKey pairs used for the destination bucket has source bucket access privileges. To enable this, update ACLs directly on AWS.

S3BACKEND¶

S3BACKEND=file¶

For stored file data to persist, you must mount Docker volumes for both data and metadata. See In Production with a Docker-Hosted CloudServer

S3BACKEND=mem¶

This is ideal for testing: no data remains after the container is shut down.

ENDPOINT¶

This variable specifies the endpoint. To direct CloudServer requests to new.host.com, for example, specify the endpoint with:

$ docker run -d --name cloudserver -p 8000:8000 -e ENDPOINT=new.host.com zenko/cloudserver

Note

On Unix-like systems (Linux, OS X, etc.) edit /etc/hosts to associate 127.0.0.1 with new.host.com.

REMOTE_MANAGEMENT_DISABLE¶

CloudServer is a part of Zenko. When you run CloudServer standalone it will still try to connect to Orbit by default (browser-based graphical user interface for Zenko).

Setting this variable to true(1) will default to accessKey1 and verySecretKey1 for credentials and disable the automatic Orbit management:

$ docker run -d --name cloudserver -p 8000:8000 -e REMOTE_MANAGEMENT_DISABLE=1 zenko/cloudserver

SCALITY_ACCESS_KEY_ID and SCALITY_SECRET_ACCESS_KEY¶

These variables specify authentication credentials for an account named “CustomAccount”.

Set account credentials for multiple accounts by editing conf/authdata.json (see below for further details). To specify one set for personal use, set these environment variables:

$ docker run -d --name cloudserver -p 8000:8000 -e SCALITY_ACCESS_KEY_ID=newAccessKey \
-e SCALITY_SECRET_ACCESS_KEY=newSecretKey zenko/cloudserver

Note

This takes precedence over the contents of the authdata.json file. The authdata.json file is ignored.

Note

The ACCESS_KEY and SECRET_KEY environment variables are deprecated.

LOG_LEVEL¶

This variable changes the log level. There are three levels: info, debug, and trace. The default is info. Debug provides more detailed logs, and trace provides the most detailed logs.

$ docker run -d --name cloudserver -p 8000:8000 -e LOG_LEVEL=trace zenko/cloudserver

SSL¶

Set true, this variable runs CloudServer with SSL.

If SSL is set true:

The ENDPOINT environment variable must also be specified.
On Unix-like systems (Linux, OS X, etc.), 127.0.0.1 must be associated with <YOUR_ENDPOINT> in /etc/hosts.

Warning

Self-signed certs with a CA generated within the container are suitable for testing purposes only. Clients cannot trust them, and they may disappear altogether on a container upgrade. The best security practice for production environments is to use an extra container, such as haproxy/nginx/stunnel, for SSL/TLS termination and to pull certificates from a mounted volume, limiting what an exploit on either component can expose.

   $ docker run -d --name cloudserver -p 8000:8000 -e SSL=TRUE -e ENDPOINT=<YOUR_ENDPOINT> \
   zenko/cloudserver

For more information about using ClousdServer with SSL, see `Using SSL <GETTING_STARTED.html#Using SSL>`__

LISTEN_ADDR¶

This variable causes CloudServer and its data and metadata components to listen on the specified address. This allows starting the data or metadata servers as standalone services, for example.

docker run -d --name s3server-data -p 9991:9991 -e LISTEN_ADDR=0.0.0.0
scality/s3server yarn run start_dataserver

DATA_HOST and METADATA_HOST¶

These variables configure the data and metadata servers to use, usually when they are running on another host and only starting the stateless Zenko CloudServer.

$ docker run -d --name cloudserver -e DATA_HOST=cloudserver-data \
-e METADATA_HOST=cloudserver-metadata zenko/cloudserver yarn run start_s3server

REDIS_HOST¶

Use this variable to connect to the redis cache server on another host than localhost.

$ docker run -d --name cloudserver -p 8000:8000 \
-e REDIS_HOST=my-redis-server.example.com zenko/cloudserver

REDIS_PORT¶

Use this variable to connect to the Redis cache server on a port other than the default 6379.

$ docker run -d --name cloudserver -p 8000:8000 \
-e REDIS_PORT=6379 zenko/cloudserver

Tunables and Setup Tips¶

Using Docker Volumes¶

CloudServer runs with a file backend by default, meaning that data is stored inside the CloudServer’s Docker container.

For data and metadata to persist, data and metadata must be hosted in Docker volumes outside the CloudServer’s Docker container. Otherwise, the data and metadata are destroyed when the container is erased.

$ docker run -­v $(pwd)/data:/usr/src/app/localData -­v $(pwd)/metadata:/usr/src/app/localMetadata \
-p 8000:8000 ­-d zenko/cloudserver

This command mounts the ./data host directory to the container at /usr/src/app/localData and the ./metadata host directory to the container at /usr/src/app/localMetaData.

Tip

These host directories can be mounted to any accessible mount point, such as /mnt/data and /mnt/metadata, for example.

Adding, Modifying, or Deleting Accounts or Credentials¶

Create a customized authdata.json file locally based on /conf/authdata.json.
Use Docker volumes to override the default authdata.json through a Docker file mapping.

For example:

$ docker run -v $(pwd)/authdata.json:/usr/src/app/conf/authdata.json -p 8000:8000 -d \
zenko/cloudserver

Specifying a Host Name¶

To specify a host name (for example, s3.domain.name), provide your own config.json file using Docker volumes.

First, add a new key-value pair to the restEndpoints section of your config.json. Make the key the host name you want, and the value the default location_constraint for this endpoint.

For example, s3.example.com is mapped to us-east-1 which is one of the location_constraints listed in your locationConfig.json file here.

For more information about location configuration, see: GETTING STARTED

"restEndpoints": {
    "localhost": "file",
    "127.0.0.1": "file",
    ...
    "cloudserver.example.com": "us-east-1"
},

Next, run CloudServer using a Docker volume:

$ docker run -v $(pwd)/config.json:/usr/src/app/config.json -p 8000:8000 -d zenko/cloudserver

The local config.json file overrides the default one through a Docker file mapping.

Running as an Unprivileged User¶

CloudServer runs as root by default.

To change this, modify the dockerfile and specify a user before the entry point.

The user must exist within the container, and must own the /usr/src/app directory for CloudServer to run.

For example, the following dockerfile lines can be modified:

...
&& groupadd -r -g 1001 scality \
&& useradd -u 1001 -g 1001 -d /usr/src/app -r scality \
&& chown -R scality:scality /usr/src/app

...

USER scality
ENTRYPOINT ["/usr/src/app/docker-entrypoint.sh"]

Continuous Integration with a Docker-Hosted CloudServer¶

When you start the Docker CloudServer image, you can adjust the configuration of the CloudServer instance by passing one or more environment variables on the docker run command line.

To run CloudServer for CI with custom locations (one in-memory, one hosted on AWS), and custom credentials mounted:

$ docker run --name CloudServer -p 8000:8000 \
-v $(pwd)/locationConfig.json:/usr/src/app/locationConfig.json \
-v $(pwd)/authdata.json:/usr/src/app/conf/authdata.json \
-v ~/.aws/credentials:/root/.aws/credentials \
-e S3DATA=multiple -e S3BACKEND=mem zenko/cloudserver

To run CloudServer for CI with custom locations, (one in-memory, one hosted on AWS, and one file), and custom credentials set as environment variables):

$ docker run --name CloudServer -p 8000:8000 \
-v $(pwd)/locationConfig.json:/usr/src/app/locationConfig.json \
-v ~/.aws/credentials:/root/.aws/credentials \
-v $(pwd)/data:/usr/src/app/localData -v $(pwd)/metadata:/usr/src/app/localMetadata \
-e SCALITY_ACCESS_KEY_ID=accessKey1 \
-e SCALITY_SECRET_ACCESS_KEY=verySecretKey1 \
-e S3DATA=multiple -e S3BACKEND=mem zenko/cloudserver

In Production with a Docker-Hosted CloudServer¶

Because data must persist in production settings, CloudServer offers multiple-backend capabilities. This requires a custom endpoint and custom credentials for local storage.

Customize these with:

$ docker run -d --name CloudServer \
-v $(pwd)/data:/usr/src/app/localData -v $(pwd)/metadata:/usr/src/app/localMetadata \
-v $(pwd)/locationConfig.json:/usr/src/app/locationConfig.json \
-v $(pwd)/authdata.json:/usr/src/app/conf/authdata.json \
-v ~/.aws/credentials:/root/.aws/credentials -e S3DATA=multiple \
-e ENDPOINT=custom.endpoint.com \
-p 8000:8000 ­-d zenko/cloudserver \