GoRapid Blog

Stories from people who build @GoRapid

PortainerCE API

01 July: Rohit Bansal

Create configs, networks, stacks and registries using portainer api

Portainer exposes an HTTP API that you can use to automate everything you do via the Portainer UI. You may also use Portainer as a gateway (HTTP queries against the Portainer API) to the underlying Docker/Kubernetes API.
I'll be using python functions to call poratiner apis.

PortainerCE API [ Base URL: /api ]

Portainer API is an HTTP API served by Portainer. It is used by the Portainer UI and everything you can do with the UI can be done using the HTTP API.
Examples are available at https://documentation.portainer.io/api/api-examples/
You can find out more about Portainer at http://portainer.io and get some support on Slack.

I'm defining some of the variables which will be used in apis.

portainer_admin_api_url = ''https://test-api.com"

portainer_username = ''test"

portainer_password = "test1234"

Authentication

Most of the API endpoints require to be authenticated as well as some level of authorization to be used.
Portainer API uses JSON Web Token to manage authentication and thus requires you to provide a token in the Authorization header of each request
with the Bearer authentication mechanism.

Below is the python function for the authentication:

import json
import requests

def authenticate():
    portainer_admin_api_url = portainer_admin_api_url
    portainer_username = portainer_username
    portainer_password = portainer_password
    response = requests.post(
        (f"{portainer_admin_api_url}" f"/api/auth"),
        json={
            'username': portainer_username,
            'password': portainer_password
        },
    )
    json_response = response.json()
    token = json_response.get('jwt')
    return token 

So by calling the above function, we'll get jwt_token which will be needed to call other apis. It'll be looking like this:

Bearer XXXXXX 

You can make command line http post request as

http -j POST portainer_admin_api_url/api/auth username=portainer_username password=portainer_password 

Response will be looking like

{
    "jwt": "XXXXXXXXX"
} 
Config

We can call the below python function to create configs. For this, we need jwt_token which will be fetched by calling authentication (above written) function.

def create_config(token, config_file):
    portainer_admin_api_url = portainer_admin_api_url
     with open(config_file, "rb") as f:
         data = f.read()
         data = base64.b64encode(data).decode("utf-8")

     url = (f"{portainer_admin_api_url}"
               f"/api/endpoints/1/docker/configs/create")
     body = {
          'Name': config_name,
          'Data': data
     }
     headers = {'Authorization': f"Bearer {token}"}
     response = requests.post(url, json=body, headers=headers)

     if (response.status_code == 200):
         print('config' + name + 'created successfully')
      else:
             print('Error while creating config ' + name) 

By calling the above function, configs will be created on portainer.

Network

We can call the below python function to create networks which will be attachable. For this, we need jwt_token which will be fetched by calling authentication function.

def create_network(token):
    portainer_admin_api_url = portainer_admin_api_url
    network = network_name
    driver = swarm_driver

    url = (f"{portainer_admin_api_url}"
           f"/api/endpoints/1/docker/networks/create")
    body = {
        'Name': network,
        'Driver': driver,
        'Attachable': True
    }
    headers = {'Authorization': f"Bearer {token}"}
    response = requests.post(url, json=body, headers=headers)
    print(response.status_code)

    if (response.status_code == 200):
        print(f"Network" f" {network} " f"created successfully")
    else:
        print('Error while creating network ' + network) 

By calling the above function, networks will be created on portainer.

Registry

Registry is required to pull the image from cloud like gcp, aws, azure etc

We can call the below python function to create registries.
For this, we need jwt_token which will be fetched by calling authentication (above written) function.
Also we need a service account key which is required to make authentication with cloud like gcp.
This service account key will be generated from IAM's service account of gcp/aws.

Here I''m using gcp. So service account key can be generated by IAM-> service accounts
Also I'm using artifact registry. So url for the registry will be as region-docker.pkg.dev/project_id

  • region is the gcp's project region like asia-southeast-2
  • project_id is the id of your gcp's project

import base64

def create_registry(token):
    portainer_admin_api_url = portainer_admin_api_url
    service_account_key = service_account_key

    with open(service_account_key, "rb") as f:
        data = f.read()
        password = base64.b64encode(data).decode("utf-8")

    url = (f"{portainer_admin_api_url}" f"/api/registries")
    registries = ["registry_name"]


    for registry in registries:
        data = {
            'Name': registry,
            'URL': (f"region-docker.pkg.dev/project_id"
                    f"{registry}"),
            'Type': 3,
            "Authentication": True,
            'Username': '_json_key_base64',
            'Password': password
        }
        headers = {'Authorization': f"Bearer {token}"}
        response = requests.post(url, json=data, headers=headers)

        if (response.status_code == 200):
            print('Registry ' + registry + ' created successfully')
        else:
            print('Error while creating registry ' + registry) 


By calling the above function, registries will be created on portainer.

Below is the http command line to create the registry.

http -j POST portainer_admin_api_url/api/registries "Authorization: jwt_token"  Name=registry_name URL=region-docker.pkg.dev/project_id/registry_name Username=_json_key_base64 Password=encrypted_pwd Type:=3 Authentication:='rue 
Stack

Stacks are collections of services running in a Docker Swarm environment.

Deploying a new stack:

You can deploy a new stack from Portainer using the following options:

  • Web editor: Using our Web Editor, you will capable to define the services for this stack using a docker-compose format.
  • Upload: If you're already have a stack.yml file, you can upload from your computer to deploy that stack.
  • Git Repository: You can use a docker-compose format file hosted in Github.
  • Custom Template: If you already created a template of stacks, you can deploy from this option.

Here I'll be using web editor to create stack using portainer api. To create stack on portainer, we need id of docker-swarm.
We can call the below python function to get swarm id.

def _get_swarm_id(token, portainer_admin_api_url):
    url = (f"{portainer_admin_api_url}"
           f"/api/endpoints/1/docker/swarm")
    headers = {'Authorization': f"Bearer {token}"}
    swarm = requests.get(url, headers=headers)
    swarm_response = swarm.json()
    swarm_id = swarm_response.get('ID')
    return swarm_id 

If you are using environment variable to create stack then call the below function. I've written all the environment variable in a .env file as key=value

from typing import Dict, List
from pathlib import Path
from dotenv import dotenv_values

def _read_env_file_for_stack(stack_name: str) -> List[Dict]:
    dotenv_path = Path(f"path of the .env file")
    env_values = dotenv_values(dotenv_path)

    env_vars = []
    for key, value in env_values.items():
        env_vars.append({
            "name": key,
            "value": value
        })

    return env_vars 

Now we can call below function to create stack which will call the get_swarm_id() method to get swarm_id.

Parameters:

type(integer): Stack deployment type. Possible values: 1 (Swarm stack) or 2 (Compose stack).

Available values : 1, 2

method(string): Stack deployment method. Possible values: file, string or repository.

Available values : string, file, repository

endpointId(integer): Identifier of the endpoint that will be used to deploy the stack

import yaml
import json

def create_stack(token, stack_file):
    portainer_admin_api_url = portainer_admin_api_url
    swarm_id = _get_swarm_id(token, portainer_admin_api_url)
    
    fdata = stack_file.read()
    fdata = yaml.load(fdata, Loader=yaml.FullLoader)
    filedata = json.dumps(fdata)

    url = (f"{portainer_admin_api_url}"
              f"/api/stacks?method=string&type=1&endpointId=1")
    headers = {'Authorization': f"Bearer {token}"}

    data = {
            'Name': stack_name,
            'SwarmID': swarm_id,
            'stackFileContent': filedata
    }

    # if using any environment variables then call _read_env_file_for_stack() method
    env_vars = _read_env_file_for_stack(stack_name)
    if env_vars:
        data['env'] = env_vars

    response = requests.post(url, json=data, headers=headers)

    if (response.status_code == 200):
        print('Stack ' + stack_name + ' created successfully')
    else:
        print('Error while creating stack ' + stack_name) 

By calling the above function, stacks will be created on portainer.

Users list

Below is the http command line to get users list created on portainer

http GET portainer_admin_api_url/api/users "Authorization: jwt_token" 

We can also call the python function get the users list.

def _get_users_list(token):
    token = authenticate()
    url = (f"{PORTAINER_ADMIN_API_URL}"
              f"/api/users")
    headers = {'Authorization': f"Bearer {token}"}
    response = requests.get(url, headers=headers)
    print(response.text) 

If you want to inspect any user then run the below http command line request where user_id will be integer field.

http GET https://staging-portainer.siminvest.co.id/api/users/user_id "Authorization: jwt_token" 
Conclusion:

Today we’ve described python scripts and http command lines to create usefule resources on portainer.

It'll save the time of developer by writing the script instead of creating all resources every time.

Thank You

PortainerCE API

01 July: Rohit Bansal

Create configs, networks, stacks and registries using portainer api

Portainer exposes an HTTP API that you can use to automate everything you do via the Portainer UI. You may also use Portainer as a gateway (HTTP queries against the Portainer API) to the underlying Docker/Kubernetes API.
I'll be using python functions to call poratiner apis.

PortainerCE API [ Base URL: /api ]

Portainer API is an HTTP API served by Portainer. It is used by the Portainer UI and everything you can do with the UI can be done using the HTTP API.
Examples are available at https://documentation.portainer.io/api/api-examples/
You can find out more about Portainer at http://portainer.io and get some support on Slack.

I'm defining some of the variables which will be used in apis.

portainer_admin_api_url = ''https://test-api.com"

portainer_username = ''test"

portainer_password = "test1234"

Authentication

Most of the API endpoints require to be authenticated as well as some level of authorization to be used.
Portainer API uses JSON Web Token to manage authentication and thus requires you to provide a token in the Authorization header of each request
with the Bearer authentication mechanism.

Below is the python function for the authentication:

import json
import requests

def authenticate():
    portainer_admin_api_url = portainer_admin_api_url
    portainer_username = portainer_username
    portainer_password = portainer_password
    response = requests.post(
        (f"{portainer_admin_api_url}" f"/api/auth"),
        json={
            'username': portainer_username,
            'password': portainer_password
        },
    )
    json_response = response.json()
    token = json_response.get('jwt')
    return token 

So by calling the above function, we'll get jwt_token which will be needed to call other apis. It'll be looking like this:

Bearer XXXXXX 

You can make command line http post request as

http -j POST portainer_admin_api_url/api/auth username=portainer_username password=portainer_password 

Response will be looking like

{
    "jwt": "XXXXXXXXX"
} 
Config

We can call the below python function to create configs. For this, we need jwt_token which will be fetched by calling authentication (above written) function.

def create_config(token, config_file):
    portainer_admin_api_url = portainer_admin_api_url
     with open(config_file, "rb") as f:
         data = f.read()
         data = base64.b64encode(data).decode("utf-8")

     url = (f"{portainer_admin_api_url}"
               f"/api/endpoints/1/docker/configs/create")
     body = {
          'Name': config_name,
          'Data': data
     }
     headers = {'Authorization': f"Bearer {token}"}
     response = requests.post(url, json=body, headers=headers)

     if (response.status_code == 200):
         print('config' + name + 'created successfully')
      else:
             print('Error while creating config ' + name) 

By calling the above function, configs will be created on portainer.

Network

We can call the below python function to create networks which will be attachable. For this, we need jwt_token which will be fetched by calling authentication function.

def create_network(token):
    portainer_admin_api_url = portainer_admin_api_url
    network = network_name
    driver = swarm_driver

    url = (f"{portainer_admin_api_url}"
           f"/api/endpoints/1/docker/networks/create")
    body = {
        'Name': network,
        'Driver': driver,
        'Attachable': True
    }
    headers = {'Authorization': f"Bearer {token}"}
    response = requests.post(url, json=body, headers=headers)
    print(response.status_code)

    if (response.status_code == 200):
        print(f"Network" f" {network} " f"created successfully")
    else:
        print('Error while creating network ' + network) 

By calling the above function, networks will be created on portainer.

Registry

Registry is required to pull the image from cloud like gcp, aws, azure etc

We can call the below python function to create registries.
For this, we need jwt_token which will be fetched by calling authentication (above written) function.
Also we need a service account key which is required to make authentication with cloud like gcp.
This service account key will be generated from IAM's service account of gcp/aws.

Here I''m using gcp. So service account key can be generated by IAM-> service accounts
Also I'm using artifact registry. So url for the registry will be as region-docker.pkg.dev/project_id

  • region is the gcp's project region like asia-southeast-2
  • project_id is the id of your gcp's project

import base64

def create_registry(token):
    portainer_admin_api_url = portainer_admin_api_url
    service_account_key = service_account_key

    with open(service_account_key, "rb") as f:
        data = f.read()
        password = base64.b64encode(data).decode("utf-8")

    url = (f"{portainer_admin_api_url}" f"/api/registries")
    registries = ["registry_name"]


    for registry in registries:
        data = {
            'Name': registry,
            'URL': (f"region-docker.pkg.dev/project_id"
                    f"{registry}"),
            'Type': 3,
            "Authentication": True,
            'Username': '_json_key_base64',
            'Password': password
        }
        headers = {'Authorization': f"Bearer {token}"}
        response = requests.post(url, json=data, headers=headers)

        if (response.status_code == 200):
            print('Registry ' + registry + ' created successfully')
        else:
            print('Error while creating registry ' + registry) 


By calling the above function, registries will be created on portainer.

Below is the http command line to create the registry.

http -j POST portainer_admin_api_url/api/registries "Authorization: jwt_token"  Name=registry_name URL=region-docker.pkg.dev/project_id/registry_name Username=_json_key_base64 Password=encrypted_pwd Type:=3 Authentication:='rue 
Stack

Stacks are collections of services running in a Docker Swarm environment.

Deploying a new stack:

You can deploy a new stack from Portainer using the following options:

  • Web editor: Using our Web Editor, you will capable to define the services for this stack using a docker-compose format.
  • Upload: If you're already have a stack.yml file, you can upload from your computer to deploy that stack.
  • Git Repository: You can use a docker-compose format file hosted in Github.
  • Custom Template: If you already created a template of stacks, you can deploy from this option.

Here I'll be using web editor to create stack using portainer api. To create stack on portainer, we need id of docker-swarm.
We can call the below python function to get swarm id.

def _get_swarm_id(token, portainer_admin_api_url):
    url = (f"{portainer_admin_api_url}"
           f"/api/endpoints/1/docker/swarm")
    headers = {'Authorization': f"Bearer {token}"}
    swarm = requests.get(url, headers=headers)
    swarm_response = swarm.json()
    swarm_id = swarm_response.get('ID')
    return swarm_id 

If you are using environment variable to create stack then call the below function. I've written all the environment variable in a .env file as key=value

from typing import Dict, List
from pathlib import Path
from dotenv import dotenv_values

def _read_env_file_for_stack(stack_name: str) -> List[Dict]:
    dotenv_path = Path(f"path of the .env file")
    env_values = dotenv_values(dotenv_path)

    env_vars = []
    for key, value in env_values.items():
        env_vars.append({
            "name": key,
            "value": value
        })

    return env_vars 

Now we can call below function to create stack which will call the get_swarm_id() method to get swarm_id.

Parameters:

type(integer): Stack deployment type. Possible values: 1 (Swarm stack) or 2 (Compose stack).

Available values : 1, 2

method(string): Stack deployment method. Possible values: file, string or repository.

Available values : string, file, repository

endpointId(integer): Identifier of the endpoint that will be used to deploy the stack

import yaml
import json

def create_stack(token, stack_file):
    portainer_admin_api_url = portainer_admin_api_url
    swarm_id = _get_swarm_id(token, portainer_admin_api_url)
    
    fdata = stack_file.read()
    fdata = yaml.load(fdata, Loader=yaml.FullLoader)
    filedata = json.dumps(fdata)

    url = (f"{portainer_admin_api_url}"
              f"/api/stacks?method=string&type=1&endpointId=1")
    headers = {'Authorization': f"Bearer {token}"}

    data = {
            'Name': stack_name,
            'SwarmID': swarm_id,
            'stackFileContent': filedata
    }

    # if using any environment variables then call _read_env_file_for_stack() method
    env_vars = _read_env_file_for_stack(stack_name)
    if env_vars:
        data['env'] = env_vars

    response = requests.post(url, json=data, headers=headers)

    if (response.status_code == 200):
        print('Stack ' + stack_name + ' created successfully')
    else:
        print('Error while creating stack ' + stack_name) 

By calling the above function, stacks will be created on portainer.

Users list

Below is the http command line to get users list created on portainer

http GET portainer_admin_api_url/api/users "Authorization: jwt_token" 

We can also call the python function get the users list.

def _get_users_list(token):
    token = authenticate()
    url = (f"{PORTAINER_ADMIN_API_URL}"
              f"/api/users")
    headers = {'Authorization': f"Bearer {token}"}
    response = requests.get(url, headers=headers)
    print(response.text) 

If you want to inspect any user then run the below http command line request where user_id will be integer field.

http GET https://staging-portainer.siminvest.co.id/api/users/user_id "Authorization: jwt_token" 
Conclusion:

Today we’ve described python scripts and http command lines to create usefule resources on portainer.

It'll save the time of developer by writing the script instead of creating all resources every time.

Thank You

Comments 0