<\/p>\n
Following the previous article, I will continue to details the Gitlab CI\/CD I’m currently using.<\/p>\n
When developing an application, for me you always need to define a standard.
\nThe coding must conform to general rules.<\/p>\n
Formatting, syntax checking and best practices are always welcomed.
\nEven if it means more effort from the dev, you will gain later in code visibility.<\/p>\n
So… how to do it ? Let’s talk a bit about Python.<\/p>\n
I mainly use black & flake8.
\nFor the tests I’m currently trying to use pytest.
\nI will guide you to configure them and define automated use with pre-commit.<\/p>\n
Please note that the usage of a virtualenv is, as always, recommended:<\/p>\n
virtualenv -p `which python3` venv\nsource venv\/bin\/activate\n<\/code><\/pre>\nFiles Configuration<\/h1>\n
Here are all the files that I add in my project directory.<\/p>\n
requirements-dev.txt<\/h2>\n
It’s how I call the requirements used for dev.
\nBut you can do what you want.<\/p>\n
pre-commit>=1.20.0\nflake8>=3.7.9\nblack>=19.10b0\n<\/code><\/pre>\nYou can then intall the dependencies inside the virtualenv by
\npip install -r requirements-dev.txt<\/code><\/p>\nsetup.cfg<\/h2>\n
This file is used by flake8.<\/p>\n
[flake8]\nexclude = .svn,CVS,.bzr,.hg,.git,__pycache__,.tox,venv,.eggs,build\nmax-line-length = 88\nselect = C,E,F,W,B,B950\nignore = E203, E501, W503\n<\/code><\/pre>\nIt allows you to ignore basic issue and use a correct line length for visibility with flake<\/p>\n
pyproject.toml<\/h2>\n
This file is used by black, repeating similar settings than previously.<\/p>\n
[tool.black]\nline-length = 88\ntarget-version = [\"py36\", \"py37\", \"py38\"]\ninclude = '\\.pyi?$'\nexclude = '''\n\/(\n\\.git\n| \\.eggs\n| \\.idea\n| \\__pycache__\n| venv\n| front\n| _build\n| build\n| dist\n)\/\n'''\n<\/code><\/pre>\nThose two file will help you to define what are the standard rule you will use while coding.
\nNow you should be able to use:<\/p>\n
black . # Format your file\nflake8 # Check your syntax\n<\/code><\/pre>\nOk, it’s a good start… but why not automatize that at the commit time ?
\nHere’s how to do it with pre-commit.<\/p>\n
.pre-commit-config.yaml<\/h2>\nrepos:\n- repo: https:\/\/github.com\/ambv\/black\nrev: stable\nhooks:\n- id: black\nlanguage_version: python3.6\nstages: [commit]\n- repo: https:\/\/github.com\/pre-commit\/pre-commit-hooks\nrev: v1.2.3\nhooks:\n- id: flake8\nstages: [commit]\n<\/code><\/pre>\nNow you can use pre-commit install<\/code> to install your commit hook.
\nAnd with a simple git commit you now get (at least):<\/p>\nblack................................................(no files to check)Skipped\nFlake8...............................................(no files to check)Skipped\n<\/code><\/pre>\nOK…. but you will follow it correctly, but is it the case for everybody ?
\nNot necessarily, so now we will use Gitlab CI\/CD and pipeline to check the code quality.<\/p>\n
CI\/CD<\/h1>\n
I prefer separate my CI\/CD in function by files, so this step is usually the linting phase for me.
\nIt translate like that.<\/p>\n
I mainly use alpine, for it’s portability and small size.<\/p>\n
.lint-ci.yml<\/h2>\nchecker:\nstage: checker\nimage: alpine\nexcept:\nrefs:\n- master\n- dev\n- \/v*\/\nbefore_script:\n- apk --no-cache add py3-pip python3-dev gcc linux-headers musl-dev libffi-dev openssl-dev git\n- pip3 install flake8 mccabe pycodestyle pyflakes\nscript:\n- python3 -m flake8 --statistics --count .\nallow_failure: true\n\nformater:\nstage: formater\nimage: alpine\nexcept:\nrefs:\n- master\n- dev\n- \/v*\/\nbefore_script:\n- apk --no-cache add py3-pip python3-dev gcc linux-headers musl-dev libffi-dev openssl-dev git\n- pip3 install black\nscript:\n- black --check .\nallow_failure: false\n<\/code><\/pre>\n.gitlab-ci.yml<\/h2>\n
This allow you to start the job.<\/p>\n
include:\n- local: '.lint-ci.yml'\n\nstages:\n- formater\n- checker\n<\/code><\/pre>\nAnd “voil\u00e0”.
\nThe pipeline will trigger on new commit except on main branches and version tags.
\nI’m still working on the correct “only” & “except” filter so you’ll have to set your own
\npreference, but the basics are here, good dev ! :)<\/p>","protected":false},"excerpt":{"rendered":"
Introduction Following the previous article, I will continue to details the Gitlab CI\/CD I’m currently using. When developing an application, for me […]<\/p>\n","protected":false},"author":1,"featured_media":2778,"comment_status":"closed","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"ngg_post_thumbnail":0,"footnotes":""},"categories":[245],"tags":[247,241,240,248,242,249,246],"class_list":["post-2774","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-programming","tag-black","tag-cd","tag-ci","tag-flake8","tag-gitlab","tag-lint","tag-python"],"yoast_head":"\n
Gitlab CI & Python Linting - Aerian.fr<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.aerian.fr\/en\/english-gitlab-ci-python-linting\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Gitlab CI & Python Linting - Aerian.fr\" \/>\n<meta property=\"og:description\" content=\"Introduction Following the previous article, I will continue to details the Gitlab CI\/CD I’m currently using. When developing an application, for me […]\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.aerian.fr\/en\/english-gitlab-ci-python-linting\/\" \/>\n<meta property=\"og:site_name\" content=\"Aerian.fr\" \/>\n<meta property=\"article:published_time\" content=\"2020-03-14T11:22:15+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2020-03-19T19:21:17+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/www.aerian.fr\/wp-content\/uploads\/2020\/03\/Ie4QeQRcPvhuNuDVQsroRwbUFINXyE2ywvMwMfBb.png\" \/>\n\t<meta property=\"og:image:width\" content=\"480\" \/>\n\t<meta property=\"og:image:height\" content=\"240\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/png\" \/>\n<meta name=\"author\" content=\"42\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:creator\" content=\"@LinceAerian\" \/>\n<meta name=\"twitter:site\" content=\"@LinceAerian\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"42\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"3 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\/\/www.aerian.fr\/english-gitlab-ci-python-linting\/#article\",\"isPartOf\":{\"@id\":\"https:\/\/www.aerian.fr\/english-gitlab-ci-python-linting\/\"},\"author\":{\"name\":\"42\",\"@id\":\"https:\/\/www.aerian.fr\/#\/schema\/person\/622c3cefbea11a0be741137608b4bf8b\"},\"headline\":\"Gitlab CI & Python Linting\",\"datePublished\":\"2020-03-14T11:22:15+00:00\",\"dateModified\":\"2020-03-19T19:21:17+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\/\/www.aerian.fr\/english-gitlab-ci-python-linting\/\"},\"wordCount\":395,\"publisher\":{\"@id\":\"https:\/\/www.aerian.fr\/#organization\"},\"image\":{\"@id\":\"https:\/\/www.aerian.fr\/english-gitlab-ci-python-linting\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/www.aerian.fr\/wp-content\/uploads\/2020\/03\/Ie4QeQRcPvhuNuDVQsroRwbUFINXyE2ywvMwMfBb.png\",\"keywords\":[\"black\",\"cd\",\"ci\",\"flake8\",\"gitlab\",\"lint\",\"python\"],\"articleSection\":[\"Programming\"],\"inLanguage\":\"en-US\"},{\"@type\":\"WebPage\",\"@id\":\"https:\/\/www.aerian.fr\/en\/english-gitlab-ci-python-linting\/\",\"url\":\"https:\/\/www.aerian.fr\/en\/english-gitlab-ci-python-linting\/\",\"name\":\"Gitlab CI & Python Linting - Aerian.fr\",\"isPartOf\":{\"@id\":\"https:\/\/www.aerian.fr\/en\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\/\/www.aerian.fr\/en\/english-gitlab-ci-python-linting\/#primaryimage\"},\"image\":{\"@id\":\"https:\/\/www.aerian.fr\/english-gitlab-ci-python-linting\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/www.aerian.fr\/wp-content\/uploads\/2020\/03\/Ie4QeQRcPvhuNuDVQsroRwbUFINXyE2ywvMwMfBb.png\",\"datePublished\":\"2020-03-14T11:22:15+00:00\",\"dateModified\":\"2020-03-19T19:21:17+00:00\",\"breadcrumb\":{\"@id\":\"https:\/\/www.aerian.fr\/en\/english-gitlab-ci-python-linting\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[[\"https:\/\/www.aerian.fr\/en\/english-gitlab-ci-python-linting\/\"]]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/www.aerian.fr\/en\/english-gitlab-ci-python-linting\/#primaryimage\",\"url\":\"https:\/\/www.aerian.fr\/wp-content\/uploads\/2020\/03\/Ie4QeQRcPvhuNuDVQsroRwbUFINXyE2ywvMwMfBb.png\",\"contentUrl\":\"https:\/\/www.aerian.fr\/wp-content\/uploads\/2020\/03\/Ie4QeQRcPvhuNuDVQsroRwbUFINXyE2ywvMwMfBb.png\",\"width\":480,\"height\":240,\"caption\":\"Gitlab & Python logos\"},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/www.aerian.fr\/en\/english-gitlab-ci-python-linting\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/www.aerian.fr\/en\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"(English) Gitlab CI & Python Linting\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/www.aerian.fr\/en\/#website\",\"url\":\"https:\/\/www.aerian.fr\/en\/\",\"name\":\"Aerian.fr\",\"description\":\"Welcome to Aerian.fr\",\"publisher\":{\"@id\":\"https:\/\/www.aerian.fr\/en\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/www.aerian.fr\/en\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Organization\",\"@id\":\"https:\/\/www.aerian.fr\/en\/#organization\",\"name\":\"Aerian.fr\",\"url\":\"https:\/\/www.aerian.fr\/en\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/www.aerian.fr\/en\/#\/schema\/logo\/image\/\",\"url\":\"https:\/\/www.aerian.fr\/wp-content\/uploads\/2012\/01\/logo.png\",\"contentUrl\":\"https:\/\/www.aerian.fr\/wp-content\/uploads\/2012\/01\/logo.png\",\"width\":667,\"height\":522,\"caption\":\"Aerian.fr\"},\"image\":{\"@id\":\"https:\/\/www.aerian.fr\/en\/#\/schema\/logo\/image\/\"},\"sameAs\":[\"https:\/\/x.com\/LinceAerian\",\"https:\/\/www.linkedin.com\/in\/marleixmathieu\"]},{\"@type\":\"Person\",\"@id\":\"https:\/\/www.aerian.fr\/#\/schema\/person\/622c3cefbea11a0be741137608b4bf8b\",\"name\":\"42\",\"sameAs\":[\"http:\/\/www.aerian.fr\"],\"url\":\"https:\/\/www.aerian.fr\/en\/author\/lince\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Gitlab CI & Python Linting - Aerian.fr","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.aerian.fr\/en\/english-gitlab-ci-python-linting\/","og_locale":"en_US","og_type":"article","og_title":"Gitlab CI & Python Linting - Aerian.fr","og_description":"Introduction Following the previous article, I will continue to details the Gitlab CI\/CD I’m currently using. When developing an application, for me […]","og_url":"https:\/\/www.aerian.fr\/en\/english-gitlab-ci-python-linting\/","og_site_name":"Aerian.fr","article_published_time":"2020-03-14T11:22:15+00:00","article_modified_time":"2020-03-19T19:21:17+00:00","og_image":[{"width":480,"height":240,"url":"https:\/\/www.aerian.fr\/wp-content\/uploads\/2020\/03\/Ie4QeQRcPvhuNuDVQsroRwbUFINXyE2ywvMwMfBb.png","type":"image\/png"}],"author":"42","twitter_card":"summary_large_image","twitter_creator":"@LinceAerian","twitter_site":"@LinceAerian","twitter_misc":{"Written by":"42","Est. reading time":"3 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/www.aerian.fr\/english-gitlab-ci-python-linting\/#article","isPartOf":{"@id":"https:\/\/www.aerian.fr\/english-gitlab-ci-python-linting\/"},"author":{"name":"42","@id":"https:\/\/www.aerian.fr\/#\/schema\/person\/622c3cefbea11a0be741137608b4bf8b"},"headline":"Gitlab CI & Python Linting","datePublished":"2020-03-14T11:22:15+00:00","dateModified":"2020-03-19T19:21:17+00:00","mainEntityOfPage":{"@id":"https:\/\/www.aerian.fr\/english-gitlab-ci-python-linting\/"},"wordCount":395,"publisher":{"@id":"https:\/\/www.aerian.fr\/#organization"},"image":{"@id":"https:\/\/www.aerian.fr\/english-gitlab-ci-python-linting\/#primaryimage"},"thumbnailUrl":"https:\/\/www.aerian.fr\/wp-content\/uploads\/2020\/03\/Ie4QeQRcPvhuNuDVQsroRwbUFINXyE2ywvMwMfBb.png","keywords":["black","cd","ci","flake8","gitlab","lint","python"],"articleSection":["Programming"],"inLanguage":"en-US"},{"@type":"WebPage","@id":"https:\/\/www.aerian.fr\/en\/english-gitlab-ci-python-linting\/","url":"https:\/\/www.aerian.fr\/en\/english-gitlab-ci-python-linting\/","name":"Gitlab CI & Python Linting - Aerian.fr","isPartOf":{"@id":"https:\/\/www.aerian.fr\/en\/#website"},"primaryImageOfPage":{"@id":"https:\/\/www.aerian.fr\/en\/english-gitlab-ci-python-linting\/#primaryimage"},"image":{"@id":"https:\/\/www.aerian.fr\/english-gitlab-ci-python-linting\/#primaryimage"},"thumbnailUrl":"https:\/\/www.aerian.fr\/wp-content\/uploads\/2020\/03\/Ie4QeQRcPvhuNuDVQsroRwbUFINXyE2ywvMwMfBb.png","datePublished":"2020-03-14T11:22:15+00:00","dateModified":"2020-03-19T19:21:17+00:00","breadcrumb":{"@id":"https:\/\/www.aerian.fr\/en\/english-gitlab-ci-python-linting\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":[["https:\/\/www.aerian.fr\/en\/english-gitlab-ci-python-linting\/"]]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.aerian.fr\/en\/english-gitlab-ci-python-linting\/#primaryimage","url":"https:\/\/www.aerian.fr\/wp-content\/uploads\/2020\/03\/Ie4QeQRcPvhuNuDVQsroRwbUFINXyE2ywvMwMfBb.png","contentUrl":"https:\/\/www.aerian.fr\/wp-content\/uploads\/2020\/03\/Ie4QeQRcPvhuNuDVQsroRwbUFINXyE2ywvMwMfBb.png","width":480,"height":240,"caption":"Gitlab & Python logos"},{"@type":"BreadcrumbList","@id":"https:\/\/www.aerian.fr\/en\/english-gitlab-ci-python-linting\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/www.aerian.fr\/en\/"},{"@type":"ListItem","position":2,"name":"(English) Gitlab CI & Python Linting"}]},{"@type":"WebSite","@id":"https:\/\/www.aerian.fr\/en\/#website","url":"https:\/\/www.aerian.fr\/en\/","name":"Aerian.fr","description":"Welcome to Aerian.fr","publisher":{"@id":"https:\/\/www.aerian.fr\/en\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/www.aerian.fr\/en\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/www.aerian.fr\/en\/#organization","name":"Aerian.fr","url":"https:\/\/www.aerian.fr\/en\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.aerian.fr\/en\/#\/schema\/logo\/image\/","url":"https:\/\/www.aerian.fr\/wp-content\/uploads\/2012\/01\/logo.png","contentUrl":"https:\/\/www.aerian.fr\/wp-content\/uploads\/2012\/01\/logo.png","width":667,"height":522,"caption":"Aerian.fr"},"image":{"@id":"https:\/\/www.aerian.fr\/en\/#\/schema\/logo\/image\/"},"sameAs":["https:\/\/x.com\/LinceAerian","https:\/\/www.linkedin.com\/in\/marleixmathieu"]},{"@type":"Person","@id":"https:\/\/www.aerian.fr\/#\/schema\/person\/622c3cefbea11a0be741137608b4bf8b","name":"42","sameAs":["http:\/\/www.aerian.fr"],"url":"https:\/\/www.aerian.fr\/en\/author\/lince\/"}]}},"_links":{"self":[{"href":"https:\/\/www.aerian.fr\/en\/wp-json\/wp\/v2\/posts\/2774","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.aerian.fr\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.aerian.fr\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.aerian.fr\/en\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.aerian.fr\/en\/wp-json\/wp\/v2\/comments?post=2774"}],"version-history":[{"count":11,"href":"https:\/\/www.aerian.fr\/en\/wp-json\/wp\/v2\/posts\/2774\/revisions"}],"predecessor-version":[{"id":2839,"href":"https:\/\/www.aerian.fr\/en\/wp-json\/wp\/v2\/posts\/2774\/revisions\/2839"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.aerian.fr\/en\/wp-json\/wp\/v2\/media\/2778"}],"wp:attachment":[{"href":"https:\/\/www.aerian.fr\/en\/wp-json\/wp\/v2\/media?parent=2774"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.aerian.fr\/en\/wp-json\/wp\/v2\/categories?post=2774"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.aerian.fr\/en\/wp-json\/wp\/v2\/tags?post=2774"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}