Polyaxon allows users to manage all logs generated by jobs, builds, and experiments containers in Minio.

Deploy Minio

Before using Minio, you need to deployed it and create a bucket to host your logs.

For that you can use the Helm stable chart: helm install stable/minio --name=plx-minio --namespace=polyaxon

Create a bucket on Minio

You should create a bucket (e.g. plx-logs) where you will host your logs.

In order to use the buckets with Polyaxon, you should create a file containing your access information json object, e.g. minio-s3-key.json. This file should include at least the following information:

{
  "AWS_ENDPOINT_URL": "", 
  "AWS_ACCESS_KEY_ID" : "",
  "AWS_SECRET_ACCESS_KEY": ""
}

When Minio is installed via Helm with default values, it uses the following hard-wired default credentials, which you will use to login to the UI and setup access to Polyaxon:

AccessKey: AKIAIOSFODNN7EXAMPLE
SecretKey: wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY

Your secret content will be:

{
  "AWS_ENDPOINT_URL": "http://plx-minio:9000",
  "AWS_ACCESS_KEY_ID": "AKIAIOSFODNN7EXAMPLE",
  "AWS_SECRET_ACCESS_KEY": "wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY"
}

Create a secret for accessing Minio

You should then create a secret with this access keys information on Kubernetes on the same namespace as Polyaxon deployment:

kubectl create secret generic s3-secret --from-file=s3-secret.json=path/to/s3-key.json -n polyaxon

Use the secret name and secret key in your logs persistence definition

persistence:
  logs:
    store: s3
    bucket: s3://[BUCKET-NAME]
    secret: [SECRET-NAME]
    secretKey: [SECRET-KEY]

e.g.

persistence:
  logs:
    store: s3
    bucket: s3://logs-bucket
    secret: s3-secret
    secretKey: s3-key

Use the secret name and secret key in your logs persistence definition

persistence:
  logs:
    store: s3
    bucket: s3://[BUCKET-NAME]
    secret: [SECRET-NAME]
    secretKey: [SECRET-KEY]

e.g.

persistence:
  logs:
    store: s3
    bucket: s3://logs-bucket
    secret: s3-secret
    secretKey: s3-key

Update/Install Polyaxon deployment

You can now install/upgrade Polyaxon with access the outputs on S3.