Introduce pre-commit hooks and code cleaning
Some checks failed
ci/woodpecker/manual/woodpecker Pipeline failed
Some checks failed
ci/woodpecker/manual/woodpecker Pipeline failed
This commit is contained in:
parent
305af47086
commit
3c3f37bdc4
4 changed files with 211 additions and 171 deletions
67
.pre-commit-config.yaml
Normal file
67
.pre-commit-config.yaml
Normal file
|
@ -0,0 +1,67 @@
|
||||||
|
# Pre-commit hooks to run tests and ensure code is cleaned.
|
||||||
|
# See https://pre-commit.com for more information
|
||||||
|
---
|
||||||
|
repos:
|
||||||
|
- repo: https://github.com/astral-sh/ruff-pre-commit
|
||||||
|
rev: v0.1.6
|
||||||
|
hooks:
|
||||||
|
- id: ruff
|
||||||
|
args:
|
||||||
|
- --fix
|
||||||
|
- repo: https://github.com/asottile/pyupgrade
|
||||||
|
rev: v3.3.1
|
||||||
|
hooks:
|
||||||
|
- id: pyupgrade
|
||||||
|
args: ["--keep-percent-format", "--py37-plus"]
|
||||||
|
- repo: https://github.com/psf/black
|
||||||
|
rev: 22.12.0
|
||||||
|
hooks:
|
||||||
|
- id: black
|
||||||
|
args: ["--target-version", "py37", "--line-length", "100"]
|
||||||
|
- repo: https://github.com/PyCQA/isort
|
||||||
|
rev: 5.11.5
|
||||||
|
hooks:
|
||||||
|
- id: isort
|
||||||
|
args: ["--profile", "black", "--line-length", "100"]
|
||||||
|
- repo: https://github.com/PyCQA/flake8
|
||||||
|
rev: 6.0.0
|
||||||
|
hooks:
|
||||||
|
- id: flake8
|
||||||
|
args: ["--max-line-length=100"]
|
||||||
|
- repo: https://github.com/codespell-project/codespell
|
||||||
|
rev: v2.2.2
|
||||||
|
hooks:
|
||||||
|
- id: codespell
|
||||||
|
args:
|
||||||
|
- --ignore-words-list=fro,hass
|
||||||
|
- --skip="./.*,*.csv,*.json,*.ambr"
|
||||||
|
- --quiet-level=2
|
||||||
|
exclude_types: [csv, json]
|
||||||
|
- repo: https://github.com/adrienverge/yamllint
|
||||||
|
rev: v1.32.0
|
||||||
|
hooks:
|
||||||
|
- id: yamllint
|
||||||
|
ignore: .github/
|
||||||
|
- repo: https://github.com/pre-commit/mirrors-prettier
|
||||||
|
rev: v2.7.1
|
||||||
|
hooks:
|
||||||
|
- id: prettier
|
||||||
|
- repo: local
|
||||||
|
hooks:
|
||||||
|
- id: pylint
|
||||||
|
name: pylint
|
||||||
|
entry: pylint
|
||||||
|
language: system
|
||||||
|
types: [python]
|
||||||
|
require_serial: true
|
||||||
|
- repo: https://github.com/Lucas-C/pre-commit-hooks-bandit
|
||||||
|
rev: v1.0.5
|
||||||
|
hooks:
|
||||||
|
- id: python-bandit-vulnerability-check
|
||||||
|
name: bandit
|
||||||
|
args: [--skip, "B101", --recursive, mylib]
|
||||||
|
- repo: https://github.com/pre-commit/pre-commit-hooks
|
||||||
|
rev: v4.4.0
|
||||||
|
hooks:
|
||||||
|
- id: check-executables-have-shebangs
|
||||||
|
stages: [manual]
|
46
README.md
46
README.md
|
@ -3,19 +3,20 @@
|
||||||
This docker image could be used as an Woodpecker CI plugin to publish one (or more) Debian package on a Aptly repository using its API. It also could be used with Gitlab CI to define a publishing job.
|
This docker image could be used as an Woodpecker CI plugin to publish one (or more) Debian package on a Aptly repository using its API. It also could be used with Gitlab CI to define a publishing job.
|
||||||
|
|
||||||
This plugin will try to :
|
This plugin will try to :
|
||||||
|
|
||||||
- List all changes files in the specified directory and filter on the specified source package name (if specified)
|
- List all changes files in the specified directory and filter on the specified source package name (if specified)
|
||||||
- Iter on detected changes files and foreach of then:
|
- Iter on detected changes files and foreach of then:
|
||||||
- the changes file is parsed to detect the source package name, the distribution and included files
|
- the changes file is parsed to detect the source package name, the distribution and included files
|
||||||
- the repository name is computed (if not specified). __Format:__ `{prefix}_{distribution}_{component}`. __Note:__ if the default prefix is specified (`.`), it will not be used to compute the repository name.
|
- the repository name is computed (if not specified). **Format:** `{prefix}_{distribution}_{component}`. **Note:** if the default prefix is specified (`.`), it will not be used to compute the repository name.
|
||||||
- the current published distribution is retreived using APTLY Publish API to:
|
- the current published distribution is retrieved using APTLY Publish API to:
|
||||||
- check it was already manally published a first time
|
- check it was already manally published a first time
|
||||||
- check it used a snapshot kind of sources
|
- check it used a snapshot kind of sources
|
||||||
- retreive other components source snapshot
|
- retrieve other components source snapshot
|
||||||
- Upload the changes file and all its included files using APTLY File Upload API in a directory named as the source package
|
- Upload the changes file and all its included files using APTLY File Upload API in a directory named as the source package
|
||||||
- Include the changes file using APTLY Local Repos API
|
- Include the changes file using APTLY Local Repos API
|
||||||
- Compute a snapshot name for the repository based on the current date and the repository name. __Format:__ `YYYYMMDD-HHMMSS_{repository name}`
|
- Compute a snapshot name for the repository based on the current date and the repository name. **Format:** `YYYYMMDD-HHMMSS_{repository name}`
|
||||||
- Create a snapshot of the repository using APTLY Local Repos API
|
- Create a snapshot of the repository using APTLY Local Repos API
|
||||||
- Update the published distribution with this new snapshot as source of the specified component and keeping other components source snapshot.
|
- Update the published distribution with this new snapshot as source of the specified component and keeping other components source snapshot.
|
||||||
|
|
||||||
In case of error, it will exit with a detailed error message (within the limits of what is provided by the Aptly API).
|
In case of error, it will exit with a detailed error message (within the limits of what is provided by the Aptly API).
|
||||||
|
|
||||||
|
@ -45,17 +46,18 @@ pipeline:
|
||||||
force_overwrite: true
|
force_overwrite: true
|
||||||
```
|
```
|
||||||
|
|
||||||
__Parameters:__
|
**Parameters:**
|
||||||
- __api_url:__ Your Aptly API URL (required)
|
|
||||||
- __api_username:__ Username to authenticate on your Aptly API (required)
|
- **api_url:** Your Aptly API URL (required)
|
||||||
- __api_password:__ Password to authenticate on your Aptly API (required)
|
- **api_username:** Username to authenticate on your Aptly API (required)
|
||||||
- __prefix:__ The publishing prefix (optional, default: `.`)
|
- **api_password:** Password to authenticate on your Aptly API (required)
|
||||||
- __repo_component:__ The component name to publish on (optional, default: `main`)
|
- **prefix:** The publishing prefix (optional, default: `.`)
|
||||||
- __repo_name:__ The repository name to publish on. If not specified, it will be computed using the specified prefix and component and the detected package distribution. See above for details.
|
- **repo_component:** The component name to publish on (optional, default: `main`)
|
||||||
- __path:__ Path to the directory where files to publish are stored (optional, default: `dist`)
|
- **repo_name:** The repository name to publish on. If not specified, it will be computed using the specified prefix and component and the detected package distribution. See above for details.
|
||||||
- __source_name:__ Name of the source package to publish (optional, default: all `changes` files are will be publish)
|
- **path:** Path to the directory where files to publish are stored (optional, default: `dist`)
|
||||||
- __max_retries:__ The number of retry in case of error calling the Aptly API (optional, default: no retry)
|
- **source_name:** Name of the source package to publish (optional, default: all `changes` files are will be publish)
|
||||||
- __force_overwrite:__ When publishing, overwrite files in `pool/` directory without notice (optional, default: false)
|
- **max_retries:** The number of retry in case of error calling the Aptly API (optional, default: no retry)
|
||||||
|
- **force_overwrite:** When publishing, overwrite files in `pool/` directory without notice (optional, default: false)
|
||||||
|
|
||||||
## With Gitlab CI
|
## With Gitlab CI
|
||||||
|
|
||||||
|
|
223
aptly-publish
223
aptly-publish
|
@ -9,55 +9,51 @@ import os
|
||||||
import re
|
import re
|
||||||
import sys
|
import sys
|
||||||
|
|
||||||
|
from debian_parser import PackagesParser
|
||||||
from requests import Session
|
from requests import Session
|
||||||
from requests.adapters import HTTPAdapter
|
from requests.adapters import HTTPAdapter
|
||||||
from urllib3.util import Retry
|
from urllib3.util import Retry
|
||||||
|
|
||||||
from debian_parser import PackagesParser
|
|
||||||
|
|
||||||
|
|
||||||
def from_env(name, default=None):
|
def from_env(name, default=None):
|
||||||
""" Retrieve a parameter from environment """
|
"""Retrieve a parameter from environment"""
|
||||||
for var in (f'PLUGIN_{name}', f'APTLY_{name}'):
|
for var in (f"PLUGIN_{name}", f"APTLY_{name}"):
|
||||||
if var in os.environ:
|
if var in os.environ:
|
||||||
return os.environ[var]
|
return os.environ[var]
|
||||||
return default
|
return default
|
||||||
|
|
||||||
|
|
||||||
# Handle parameters from environment
|
# Handle parameters from environment
|
||||||
API_URL = from_env('API_URL', None)
|
API_URL = from_env("API_URL", None)
|
||||||
if not API_URL:
|
if not API_URL:
|
||||||
print('API URL not provided')
|
print("API URL not provided")
|
||||||
sys.exit(1)
|
sys.exit(1)
|
||||||
|
|
||||||
API_USERNAME = from_env('API_USERNAME', None)
|
API_USERNAME = from_env("API_USERNAME", None)
|
||||||
if not API_USERNAME:
|
if not API_USERNAME:
|
||||||
print('API username not provided')
|
print("API username not provided")
|
||||||
sys.exit(1)
|
sys.exit(1)
|
||||||
|
|
||||||
API_PASSWORD = from_env('API_PASSWORD', None)
|
API_PASSWORD = from_env("API_PASSWORD", None)
|
||||||
if not API_PASSWORD:
|
if not API_PASSWORD:
|
||||||
print('API password not provided')
|
print("API password not provided")
|
||||||
sys.exit(1)
|
sys.exit(1)
|
||||||
|
|
||||||
MAX_RETRY = from_env('MAX_RETRIES', None)
|
MAX_RETRY = from_env("MAX_RETRIES", None)
|
||||||
|
|
||||||
REPO_NAME = from_env('REPO_NAME', None)
|
REPO_NAME = from_env("REPO_NAME", None)
|
||||||
PREFIX = from_env('PREFIX', '.')
|
PREFIX = from_env("PREFIX", ".")
|
||||||
REPO_COMPONENT = from_env('REPO_COMPONENT', 'main')
|
REPO_COMPONENT = from_env("REPO_COMPONENT", "main")
|
||||||
INPUT_PATH = from_env('PATH', 'dist')
|
INPUT_PATH = from_env("PATH", "dist")
|
||||||
SOURCE_NAME = from_env('SOURCE_PACKAGE_NAME', None)
|
SOURCE_NAME = from_env("SOURCE_PACKAGE_NAME", None)
|
||||||
FORCE_OVERWRITE = (
|
FORCE_OVERWRITE = from_env("FORCE_OVERWRITE", "false").lower() in ["1", "true", "yes"]
|
||||||
from_env('FORCE_OVERWRITE', "false").lower()
|
|
||||||
in ["1", "true", "yes"]
|
|
||||||
)
|
|
||||||
|
|
||||||
# List changes files
|
# List changes files
|
||||||
changes_files_regex = (
|
changes_files_regex = (
|
||||||
# pylint: disable=consider-using-f-string
|
# pylint: disable=consider-using-f-string
|
||||||
re.compile(r'^%s_.*\.changes$' % SOURCE_NAME)
|
re.compile(r"^%s_.*\.changes$" % SOURCE_NAME)
|
||||||
if SOURCE_NAME else
|
if SOURCE_NAME
|
||||||
re.compile(r'^.*\.changes$')
|
else re.compile(r"^.*\.changes$")
|
||||||
)
|
)
|
||||||
changes_files = []
|
changes_files = []
|
||||||
try:
|
try:
|
||||||
|
@ -75,7 +71,7 @@ except NotADirectoryError:
|
||||||
sys.exit(1)
|
sys.exit(1)
|
||||||
|
|
||||||
if not changes_files:
|
if not changes_files:
|
||||||
print(f'No changes file found in {INPUT_PATH}')
|
print(f"No changes file found in {INPUT_PATH}")
|
||||||
sys.exit(1)
|
sys.exit(1)
|
||||||
|
|
||||||
|
|
||||||
|
@ -83,27 +79,24 @@ if not changes_files:
|
||||||
session = Session()
|
session = Session()
|
||||||
session.auth = (API_USERNAME, API_PASSWORD)
|
session.auth = (API_USERNAME, API_PASSWORD)
|
||||||
if MAX_RETRY:
|
if MAX_RETRY:
|
||||||
retries = Retry(
|
retries = Retry(total=int(MAX_RETRY), status_forcelist=list(range(500, 600)))
|
||||||
total=int(MAX_RETRY),
|
|
||||||
status_forcelist=list(range(500, 600))
|
|
||||||
)
|
|
||||||
session.mount(API_URL, HTTPAdapter(max_retries=retries))
|
session.mount(API_URL, HTTPAdapter(max_retries=retries))
|
||||||
|
|
||||||
|
|
||||||
def get_repo_name(dist):
|
def get_repo_name(dist):
|
||||||
""" Compute and retreive repository name """
|
"""Compute and retrieve repository name"""
|
||||||
if REPO_NAME:
|
if REPO_NAME:
|
||||||
return REPO_NAME
|
return REPO_NAME
|
||||||
value = f'{dist}_{REPO_COMPONENT}'
|
value = f"{dist}_{REPO_COMPONENT}"
|
||||||
if PREFIX != ".":
|
if PREFIX != ".":
|
||||||
value = f'{PREFIX}_{value}'
|
value = f"{PREFIX}_{value}"
|
||||||
return value
|
return value
|
||||||
|
|
||||||
|
|
||||||
def parse_changes_file(filepath):
|
def parse_changes_file(filepath):
|
||||||
""" Parse changes file to detect distribution and included files """
|
"""Parse changes file to detect distribution and included files"""
|
||||||
dirpath = os.path.dirname(filepath)
|
dirpath = os.path.dirname(filepath)
|
||||||
with open(filepath, "r", encoding="utf-8") as file_desc:
|
with open(filepath, encoding="utf-8") as file_desc:
|
||||||
changes_file = file_desc.read()
|
changes_file = file_desc.read()
|
||||||
|
|
||||||
parser = PackagesParser(changes_file)
|
parser = PackagesParser(changes_file)
|
||||||
|
@ -112,198 +105,176 @@ def parse_changes_file(filepath):
|
||||||
files = []
|
files = []
|
||||||
for infos in parser.parse():
|
for infos in parser.parse():
|
||||||
for info in infos:
|
for info in infos:
|
||||||
if info['tag'].lower() == 'files':
|
if info["tag"].lower() == "files":
|
||||||
for line in info['value'].split(' '):
|
for line in info["value"].split(" "):
|
||||||
if not line:
|
if not line:
|
||||||
continue
|
continue
|
||||||
files.append(os.path.join(dirpath, line.split()[-1]))
|
files.append(os.path.join(dirpath, line.split()[-1]))
|
||||||
if info['tag'].lower() == 'distribution':
|
if info["tag"].lower() == "distribution":
|
||||||
if distribution:
|
if distribution:
|
||||||
print(
|
print(
|
||||||
'More than one distribution found in changes file'
|
"More than one distribution found in changes file"
|
||||||
f'{os.path.basename(filepath)}.')
|
f"{os.path.basename(filepath)}."
|
||||||
|
)
|
||||||
sys.exit(1)
|
sys.exit(1)
|
||||||
distribution = info['value']
|
distribution = info["value"]
|
||||||
if info['tag'].lower() == 'source':
|
if info["tag"].lower() == "source":
|
||||||
if package_name:
|
if package_name:
|
||||||
print(
|
print(
|
||||||
'More than one source package name found in changes '
|
"More than one source package name found in changes "
|
||||||
f'file {os.path.basename(filepath)}.')
|
f"file {os.path.basename(filepath)}."
|
||||||
|
)
|
||||||
sys.exit(1)
|
sys.exit(1)
|
||||||
package_name = info['value']
|
package_name = info["value"]
|
||||||
|
|
||||||
if not package_name:
|
if not package_name:
|
||||||
print(
|
print(
|
||||||
'Fail to detect source package name from changes file '
|
"Fail to detect source package name from changes file " f"{os.path.basename(filepath)}."
|
||||||
f'{os.path.basename(filepath)}.')
|
)
|
||||||
sys.exit(1)
|
sys.exit(1)
|
||||||
|
|
||||||
if not distribution:
|
if not distribution:
|
||||||
print(
|
print("Fail to detect distribution from changes file " f"{os.path.basename(filepath)}.")
|
||||||
'Fail to detect distribution from changes file '
|
|
||||||
f'{os.path.basename(filepath)}.')
|
|
||||||
sys.exit(1)
|
sys.exit(1)
|
||||||
|
|
||||||
if not files:
|
if not files:
|
||||||
print(
|
print("No included file found in changes file" f"{os.path.basename(filepath)}.")
|
||||||
'No included file found in changes file'
|
|
||||||
f'{os.path.basename(filepath)}.')
|
|
||||||
sys.exit(1)
|
sys.exit(1)
|
||||||
|
|
||||||
return (package_name, distribution, files)
|
return (package_name, distribution, files)
|
||||||
|
|
||||||
|
|
||||||
def get_published_distribution_other_components_sources(distribution):
|
def get_published_distribution_other_components_sources(distribution):
|
||||||
""" Retreive current published distribution using Aptly API """
|
"""Retrieve current published distribution using Aptly API"""
|
||||||
url = f'{API_URL}/publish'
|
url = f"{API_URL}/publish"
|
||||||
result = session.get(url)
|
result = session.get(url)
|
||||||
if result.status_code != 200:
|
if result.status_code != 200:
|
||||||
print(
|
print(
|
||||||
'Fail to retreive current published distribution '
|
"Fail to retrieve current published distribution "
|
||||||
f'{distribution} using Aptly API (HTTP code: {result.status_code})'
|
f"{distribution} using Aptly API (HTTP code: {result.status_code})"
|
||||||
)
|
)
|
||||||
sys.exit(1)
|
sys.exit(1)
|
||||||
for data in result.json():
|
for data in result.json():
|
||||||
if data['Prefix'] != PREFIX:
|
if data["Prefix"] != PREFIX:
|
||||||
continue
|
continue
|
||||||
if data['Distribution'] != distribution:
|
if data["Distribution"] != distribution:
|
||||||
continue
|
continue
|
||||||
if data['SourceKind'] != 'snapshot':
|
if data["SourceKind"] != "snapshot":
|
||||||
print(
|
print(
|
||||||
f'The distribution {distribution} currently published on '
|
f"The distribution {distribution} currently published on "
|
||||||
f'prefix "{PREFIX}" do not sourcing packages from snapshot(s) '
|
f'prefix "{PREFIX}" do not sourcing packages from snapshot(s) '
|
||||||
f'but from {data["SourceKind"]}.'
|
f'but from {data["SourceKind"]}.'
|
||||||
)
|
)
|
||||||
sys.exit(1)
|
sys.exit(1)
|
||||||
return [
|
return [source for source in data["Sources"] if source["Component"] != REPO_COMPONENT]
|
||||||
source for source in data['Sources']
|
|
||||||
if source['Component'] != REPO_COMPONENT
|
|
||||||
]
|
|
||||||
print(
|
print(
|
||||||
f'Distribution {distribution} seem not currently published on prefix '
|
f"Distribution {distribution} seem not currently published on prefix "
|
||||||
f'"{PREFIX}". Please manually publish it a first time before using '
|
f'"{PREFIX}". Please manually publish it a first time before using '
|
||||||
f'{os.path.basename(sys.argv[0])}.'
|
f"{os.path.basename(sys.argv[0])}."
|
||||||
)
|
)
|
||||||
sys.exit(1)
|
sys.exit(1)
|
||||||
return False
|
|
||||||
|
|
||||||
|
|
||||||
def upload_file(package_name, filepath):
|
def upload_file(package_name, filepath):
|
||||||
""" Upload a file using Aptly API """
|
"""Upload a file using Aptly API"""
|
||||||
url = f'{API_URL}/files/{package_name}'
|
url = f"{API_URL}/files/{package_name}"
|
||||||
with open(filepath, 'rb') as file_desc:
|
with open(filepath, "rb") as file_desc:
|
||||||
result = session.post(url, files={'file': file_desc})
|
result = session.post(url, files={"file": file_desc})
|
||||||
return (
|
return (
|
||||||
result.status_code == 200 and
|
result.status_code == 200
|
||||||
f'{package_name}/{os.path.basename(filepath)}' in result.json()
|
and f"{package_name}/{os.path.basename(filepath)}" in result.json()
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
def include_file(repo_name, package_name, changes_file):
|
def include_file(repo_name, package_name, changes_file):
|
||||||
""" Include a changes file using Aptly API """
|
"""Include a changes file using Aptly API"""
|
||||||
url = (
|
url = f"{API_URL}/repos/{repo_name}/include/{package_name}/" f"{os.path.basename(changes_file)}"
|
||||||
f'{API_URL}/repos/{repo_name}/include/{package_name}/'
|
|
||||||
f'{os.path.basename(changes_file)}'
|
|
||||||
)
|
|
||||||
result = session.post(url)
|
result = session.post(url)
|
||||||
data = result.json()
|
data = result.json()
|
||||||
if data.get('FailedFiles'):
|
if data.get("FailedFiles"):
|
||||||
print()
|
print()
|
||||||
print(f'Some error occurred including {changes_file}:')
|
print(f"Some error occurred including {changes_file}:")
|
||||||
print('Failed files:')
|
print("Failed files:")
|
||||||
for failed_file in data['FailedFiles']:
|
for failed_file in data["FailedFiles"]:
|
||||||
print(f' - {os.path.basename(failed_file)}')
|
print(f" - {os.path.basename(failed_file)}")
|
||||||
if data.get('Report', {}).get('Warnings'):
|
if data.get("Report", {}).get("Warnings"):
|
||||||
print('Warnings:')
|
print("Warnings:")
|
||||||
print(' - %s' % '\n - '.join(data['Report']['Warnings']))
|
print(" - %s" % "\n - ".join(data["Report"]["Warnings"]))
|
||||||
print()
|
print()
|
||||||
return False
|
return False
|
||||||
if not (
|
if not (result.status_code == 200 and data.get("Report", {}).get("Added")):
|
||||||
result.status_code == 200 and
|
print(f"Unknown error occurred including {changes_file}" "See APTLY API logs for details.")
|
||||||
data.get('Report', {}).get('Added')
|
|
||||||
):
|
|
||||||
print(
|
|
||||||
f'Unknown error occurred including {changes_file}'
|
|
||||||
'See APTLY API logs for details.')
|
|
||||||
return False
|
return False
|
||||||
return True
|
return True
|
||||||
|
|
||||||
|
|
||||||
for changes_file in changes_files:
|
for changes_file in changes_files:
|
||||||
print(f'Handle changes file {changes_file}:')
|
print(f"Handle changes file {changes_file}:")
|
||||||
package_name, distribution, filepaths = parse_changes_file(changes_file)
|
package_name, distribution, filepaths = parse_changes_file(changes_file)
|
||||||
filepaths += [changes_file]
|
filepaths += [changes_file]
|
||||||
repo_name = get_repo_name(distribution)
|
repo_name = get_repo_name(distribution)
|
||||||
|
|
||||||
other_components_sources = \
|
other_components_sources = get_published_distribution_other_components_sources(distribution)
|
||||||
get_published_distribution_other_components_sources(distribution)
|
|
||||||
|
|
||||||
print(' - Upload files:')
|
print(" - Upload files:")
|
||||||
for filepath in filepaths:
|
for filepath in filepaths:
|
||||||
if not upload_file(package_name, filepath):
|
if not upload_file(package_name, filepath):
|
||||||
print(
|
print(f" - {filepath}: fail to upload file. See APTLY API logs " "for details.")
|
||||||
f' - {filepath}: fail to upload file. See APTLY API logs '
|
|
||||||
'for details.')
|
|
||||||
sys.exit(1)
|
sys.exit(1)
|
||||||
else:
|
else:
|
||||||
print(f' - {filepath}')
|
print(f" - {filepath}")
|
||||||
|
|
||||||
print(f' - Include changes file {changes_file}:')
|
print(f" - Include changes file {changes_file}:")
|
||||||
if include_file(repo_name, package_name, changes_file):
|
if include_file(repo_name, package_name, changes_file):
|
||||||
print(' - Changes file included')
|
print(" - Changes file included")
|
||||||
else:
|
else:
|
||||||
sys.exit(1)
|
sys.exit(1)
|
||||||
|
|
||||||
# Create a snapshot of the repository
|
# Create a snapshot of the repository
|
||||||
snap_name = datetime.datetime.now().strftime(f'%Y%m%d-%H%M%S_{repo_name}')
|
snap_name = datetime.datetime.now().strftime(f"%Y%m%d-%H%M%S_{repo_name}")
|
||||||
print(f'Create new snapshot "{snap_name}" of repository "{repo_name}"')
|
print(f'Create new snapshot "{snap_name}" of repository "{repo_name}"')
|
||||||
|
|
||||||
url = f'{API_URL}/repos/{repo_name}/snapshots'
|
url = f"{API_URL}/repos/{repo_name}/snapshots"
|
||||||
payload = {'Name': snap_name}
|
payload = {"Name": snap_name}
|
||||||
result = session.post(url, json=payload)
|
result = session.post(url, json=payload)
|
||||||
try:
|
try:
|
||||||
data = result.json()
|
data = result.json()
|
||||||
except Exception: # pylint: disable=broad-except
|
except Exception: # pylint: disable=broad-except
|
||||||
data = {}
|
data = {}
|
||||||
error = (
|
error = (
|
||||||
result.status_code < 200 or
|
result.status_code < 200
|
||||||
result.status_code > 299 or
|
or result.status_code > 299
|
||||||
data.get('Name') != snap_name or
|
or data.get("Name") != snap_name
|
||||||
not data.get('CreatedAt')
|
or not data.get("CreatedAt")
|
||||||
)
|
)
|
||||||
if error:
|
if error:
|
||||||
print(
|
print(
|
||||||
f'Fail to create snapshot "{snap_name}" of repository '
|
f'Fail to create snapshot "{snap_name}" of repository '
|
||||||
f'"{repo_name}". See APTLY API logs for details.')
|
f'"{repo_name}". See APTLY API logs for details.'
|
||||||
|
)
|
||||||
sys.exit(1)
|
sys.exit(1)
|
||||||
|
|
||||||
# Update published snapshot of the distribution
|
# Update published snapshot of the distribution
|
||||||
print(
|
print(
|
||||||
f'Update published snapshot of distribution "{distribution}" to '
|
f'Update published snapshot of distribution "{distribution}" to '
|
||||||
f'"{snap_name}" (prefix: {PREFIX}, component: {REPO_COMPONENT})')
|
f'"{snap_name}" (prefix: {PREFIX}, component: {REPO_COMPONENT})'
|
||||||
|
)
|
||||||
if other_components_sources:
|
if other_components_sources:
|
||||||
print('Note: keep other currently published components:')
|
print("Note: keep other currently published components:")
|
||||||
for source in other_components_sources:
|
for source in other_components_sources:
|
||||||
print(f'- {source["Component"]}: {source["Name"]}')
|
print(f'- {source["Component"]}: {source["Name"]}')
|
||||||
url = f'{API_URL}/publish/:{PREFIX}/{distribution}'
|
url = f"{API_URL}/publish/:{PREFIX}/{distribution}"
|
||||||
payload = {
|
payload = {
|
||||||
'Snapshots': other_components_sources + [
|
"Snapshots": other_components_sources + [{"Component": REPO_COMPONENT, "Name": snap_name}],
|
||||||
{
|
"ForceOverwrite": FORCE_OVERWRITE,
|
||||||
'Component': REPO_COMPONENT,
|
|
||||||
'Name': snap_name
|
|
||||||
}
|
|
||||||
],
|
|
||||||
'ForceOverwrite': FORCE_OVERWRITE,
|
|
||||||
}
|
}
|
||||||
result = session.put(url, json=payload)
|
result = session.put(url, json=payload)
|
||||||
if (
|
if result.status_code < 200 or result.status_code > 299:
|
||||||
result.status_code < 200 or
|
|
||||||
result.status_code > 299
|
|
||||||
):
|
|
||||||
print(
|
print(
|
||||||
'Fail to update published snapshot of distribution '
|
"Fail to update published snapshot of distribution "
|
||||||
f'"{distribution}" to "{snap_name}" (prefix: {PREFIX}, '
|
f'"{distribution}" to "{snap_name}" (prefix: {PREFIX}, '
|
||||||
f'component: {REPO_COMPONENT}). See APTLY API logs for details.')
|
f"component: {REPO_COMPONENT}). See APTLY API logs for details."
|
||||||
|
)
|
||||||
sys.exit(1)
|
sys.exit(1)
|
||||||
|
|
||||||
print("Done.")
|
print("Done.")
|
||||||
|
|
46
docs.md
46
docs.md
|
@ -14,36 +14,36 @@ Woodpecker CI plugin to publish one (or more) Debian package on a Aptly reposito
|
||||||
## Features
|
## Features
|
||||||
|
|
||||||
This plugin will try to :
|
This plugin will try to :
|
||||||
|
|
||||||
- List all changes files in the specified directory and filter on the specified source package name (if specified)
|
- List all changes files in the specified directory and filter on the specified source package name (if specified)
|
||||||
- Iter on detected changes files and foreach of then:
|
- Iter on detected changes files and foreach of then:
|
||||||
- the changes file is parsed to detect the source package name, the distribution and included files
|
- the changes file is parsed to detect the source package name, the distribution and included files
|
||||||
- the repository name is computed (if not specified). __Format:__ `{prefix}_{distribution}_{component}`. __Note:__ if the default prefix is specified (`.`), it will not be used to compute the repository name.
|
- the repository name is computed (if not specified). **Format:** `{prefix}_{distribution}_{component}`. **Note:** if the default prefix is specified (`.`), it will not be used to compute the repository name.
|
||||||
- the current published distribution is retreived using APTLY Publish API to:
|
- the current published distribution is retrieved using APTLY Publish API to:
|
||||||
- check it was already manally published a first time
|
- check it was already manally published a first time
|
||||||
- check it used a snapshot kind of sources
|
- check it used a snapshot kind of sources
|
||||||
- retreive other components source snapshot
|
- retrieve other components source snapshot
|
||||||
- Upload the changes file and all its included files using APTLY File Upload API in a directory named as the source package
|
- Upload the changes file and all its included files using APTLY File Upload API in a directory named as the source package
|
||||||
- Include the changes file using APTLY Local Repos API
|
- Include the changes file using APTLY Local Repos API
|
||||||
- Compute a snapshot name for the repository based on the current date and the repository name. __Format:__ `YYYYMMDD-HHMMSS_{repository name}`
|
- Compute a snapshot name for the repository based on the current date and the repository name. **Format:** `YYYYMMDD-HHMMSS_{repository name}`
|
||||||
- Create a snapshot of the repository using APTLY Local Repos API
|
- Create a snapshot of the repository using APTLY Local Repos API
|
||||||
- Update the published distribution with this new snapshot as source of the specified component and keeping other components source snapshot.
|
- Update the published distribution with this new snapshot as source of the specified component and keeping other components source snapshot.
|
||||||
|
|
||||||
In case of error, it will exit with a detailed error message (within the limits of what is provided by the Aptly API).
|
In case of error, it will exit with a detailed error message (within the limits of what is provided by the Aptly API).
|
||||||
|
|
||||||
## Settings
|
## Settings
|
||||||
|
|
||||||
| Settings Name | Default | Description
|
| Settings Name | Default | Description |
|
||||||
| --------------------------| ----------------- | --------------------------------------------
|
| ---------------- | ------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||||
| `api_url` | *none* | Your Aptly API URL (required)
|
| `api_url` | _none_ | Your Aptly API URL (required) |
|
||||||
| `api_username` | *none* | Username to authenticate on your Aptly API (required)
|
| `api_username` | _none_ | Username to authenticate on your Aptly API (required) |
|
||||||
| `api_password` | *none* | Password to authenticate on your Aptly API (required)
|
| `api_password` | _none_ | Password to authenticate on your Aptly API (required) |
|
||||||
| `prefix` | `.` | The publishing prefix
|
| `prefix` | `.` | The publishing prefix |
|
||||||
| `repo_component` | `main` | The component name to publish on
|
| `repo_component` | `main` | The component name to publish on |
|
||||||
| `repo_name` | `{prefix}_{distribution}_{component}` | The repository name to publish on. If not specified, it will be computed using the specified prefix and component and the detected package distribution. See above for details.
|
| `repo_name` | `{prefix}_{distribution}_{component}` | The repository name to publish on. If not specified, it will be computed using the specified prefix and component and the detected package distribution. See above for details. |
|
||||||
| `path` | `dist` | Path to the directory where files to publish are stored
|
| `path` | `dist` | Path to the directory where files to publish are stored |
|
||||||
| `source_name` | *none* | Name of the source package to publish (optional, default: all `changes` files are will be publish)
|
| `source_name` | _none_ | Name of the source package to publish (optional, default: all `changes` files are will be publish) |
|
||||||
| `max_retries` | *none* | The number of retry in case of error calling the Aptly API (optional, default: no retry)
|
| `max_retries` | _none_ | The number of retry in case of error calling the Aptly API (optional, default: no retry) |
|
||||||
|
|
||||||
|
|
||||||
## Example
|
## Example
|
||||||
|
|
||||||
|
|
Loading…
Reference in a new issue