Following the previous article, I will continue to details the Gitlab CI/CD I’m currently using.
When developing an application, for me you always need to define a standard.
The coding must conform to general rules.
Formatting, syntax checking and best practices are always welcomed.
Even if it means more effort from the dev, you will gain later in code visibility.
So… how to do it ? Let’s talk a bit about Python.
I mainly use black & flake8.
For the tests I’m currently trying to use pytest.
I will guide you to configure them and define automated use with pre-commit.
Please note that the usage of a virtualenv is, as always, recommended:
virtualenv -p `which python3` venv source venv/bin/activate
Here are all the files that I add in my project directory.
It’s how I call the requirements used for dev.
But you can do what you want.
pre-commit>=1.20.0 flake8>=3.7.9 black>=19.10b0
You can then intall the dependencies inside the virtualenv by
pip install -r requirements-dev.txt
This file is used by flake8.
[flake8] exclude = .svn,CVS,.bzr,.hg,.git,__pycache__,.tox,venv,.eggs,build max-line-length = 88 select = C,E,F,W,B,B950 ignore = E203, E501, W503
It allows you to ignore basic issue and use a correct line length for visibility with flake
This file is used by black, repeating similar settings than previously.
[tool.black] line-length = 88 target-version = ["py36", "py37", "py38"] include = '\.pyi?$' exclude = ''' /( \.git | \.eggs | \.idea | \__pycache__ | venv | front | _build | build | dist )/ '''
Those two file will help you to define what are the standard rule you will use while coding.
Now you should be able to use:
black . # Format your file flake8 # Check your syntax
Ok, it’s a good start… but why not automatize that at the commit time ?
Here’s how to do it with pre-commit.
repos: - repo: https://github.com/ambv/black rev: stable hooks: - id: black language_version: python3.6 stages: [commit] - repo: https://github.com/pre-commit/pre-commit-hooks rev: v1.2.3 hooks: - id: flake8 stages: [commit]
Now you can use
pre-commit install to install your commit hook.
And with a simple git commit you now get (at least):
black................................................(no files to check)Skipped Flake8...............................................(no files to check)Skipped
OK…. but you will follow it correctly, but is it the case for everybody ?
Not necessarily, so now we will use Gitlab CI/CD and pipeline to check the code quality.
I prefer separate my CI/CD in function by files, so this step is usually the linting phase for me.
It translate like that.
I mainly use alpine, for it’s portability and small size.
checker: stage: checker image: alpine except: refs: - master - dev - /v*/ before_script: - apk --no-cache add py3-pip python3-dev gcc linux-headers musl-dev libffi-dev openssl-dev git - pip3 install flake8 mccabe pycodestyle pyflakes script: - python3 -m flake8 --statistics --count . allow_failure: true formater: stage: formater image: alpine except: refs: - master - dev - /v*/ before_script: - apk --no-cache add py3-pip python3-dev gcc linux-headers musl-dev libffi-dev openssl-dev git - pip3 install black script: - black --check . allow_failure: false
This allow you to start the job.
include: - local: '.lint-ci.yml' stages: - formater - checker
And « voilà ».
The pipeline will trigger on new commit except on main branches and version tags.
I’m still working on the correct « only » & « except » filter so you’ll have to set your own
preference, but the basics are here, good dev ! 🙂