Documentation

File Storage Management

There are two steps required in order to manage the files used for the indexing pipeline, etc.

• Getting the temporary S3 credentials.
• Managing the files with S3.

Temporary S3 Credentials for File Management

Use the following endpoint to generate the temporary credentials required for managing your files in the indexing pipeline.

Request

Please refer to KandaSearch REST API Authentication to learn how to obtain the Authentication Token.

You can obtain your projectId by accessing kandasearch.com, going to your project, and clicking on the copy button next to your project ID.

You must have registered your Billing Information and Payment Method Information in KS in order to use this endpoint.

The four actions that can be performed are: LIST, DOWNLOAD, UPLOAD, and DELETE. You will be able to perform the actions you request. The LIST action is provided by default. The UPLOAD action is only available to ADMIN users.

The only directoryType available is AI and it's mandatory.

curl --location 'https://api.kandasearch.com/external/v1/projects/{project-id}/files/storage/credentials' \
--header 'Authorization: Bearer {Auth-Token}' \
--header 'Content-Type: application/json' \
--data '{
  "actions": [
    "DOWNLOAD",
    "UPLOAD",
    "DELETE"
  ],
  "directoryTypes": [
    "AI"
  ]
}'

Response

The HTTP response status will always be 200. However, if the Auth-Token is not provided, the status will be 401. The status key will determine if everything went correctly ("OK") or if there was an error ("ERROR"). In the case of an error, the "message" key will explain the cause of the error.

OK response

{
    "status": "OK",
    "result": {
        "projectId": "ef173abe-457b-4cbc-b96a-94ed9f618324",
        "accessKeyId": "ASIARYRHK...",
        "secretAccessKey": "Tp28sslq6...",
        "sessionToken": "IQoJb3JpZ2luX2VjEL3...",
        "expiration": "2024-04-29T13:34:53Z",
        "actions": [
            "DELETE",
            "DOWNLOAD",
            "UPLOAD"
        ],
        "directoryTypes": [
            "AI"
        ]
    },
    "error": null
}

The expiration time is by default 60 minutes after your request. The date is in UTC.

Error Example

{
    "status": "ERROR",
    "result": null,
    "error": {
        "message": "You are not a member of this project.",
        "origin": "UIB"
    }
}

Managing the S3 files

Setup

Requirements:

• AWS Command Line Interface

To use the credentials from your terminal you will need to make 3 exports using the values from the response. Be careful with the expiration time.

export AWS_ACCESS_KEY_ID="ASIARYRHK..."
export AWS_SECRET_ACCESS_KEY="IQoJb3JpZ2luX2VjEL3..."
export AWS_SESSION_TOKEN="IQoJb3JpZ2luX2VjEL3..."

Actions

All actions require the following path:

s3://kandasearch-prod-backups/{project-id}/user/ai/

List objects in a directory

aws s3 ls s3://kandasearch-prod-backups/{project-id}/user/ai/

Upload an object

aws s3 cp foo.txt s3://kandasearch-prod-backups/{project-id}/user/ai/foo.txt

Download an object

aws s3 cp s3://kandasearch-prod-backups/{project-id}/user/ai/foo.json ./foo.json

Move an object to another directory

aws s3 mv s3://kandasearch-prod-backups/{project-id}/user/ai/foo.txt s3://kandasearch-prod-backups/{project-id}/user/ai/new-folder/foo.txt

Rename an object

aws s3 mv s3://kandasearch-prod-backups/{project-id}/user/ai/foo.txt s3://kandasearch-prod-backups/{project-id}/user/ai/foo-new-name.txt

Delete an object or a directory

Object

aws s3 rm s3://kandasearch-prod-backups/{project-id}/user/ai/foo.txt

Directory

aws s3 rm s3://kandasearch-prod-backups/{project-id}/user/ai/ --recursive