Loading operational +109 −1 Original line number Diff line number Diff line * COMMENT -*- mode: org -*- #+Date: 2014-05-09 Time-stamp: <2018-05-02> Time-stamp: <2019-02-27> #+STARTUP: content * notes :entry: ** 2018-05-02 support test, build and deploy in gocd Loading Loading @@ -48,6 +48,8 @@ Time-stamp: <2018-05-02> * current :entry: ** ** 2019-02-24 fix :url in project.clj create a blog post about this template. ** 2016-09-08 support updating site file, deb scripts etc after project is generated. - the domain in site file could change. - the pkg name could change. Loading Loading @@ -80,6 +82,112 @@ see example ~/.pylintrc ** 2014-05-09 add sphinx support * later :entry: * done :entry: ** 2019-02-24 drop :repositories from project.clj ** 2019-02-27 another problem on new created project. make test fail on new project. futures requires Python '>=2.6, <3' but the running Python is 3.5.3 one pkg in requirements-dev.txt has bad dependencies. pylint has problem. #+BEGIN_SRC sh Collecting futures (from isort>=4.2.5->pylint) Using cached https://pypi.tuna.tsinghua.edu.cn/packages/1f/9e/7b2ff7e965fc654592269f2906ade1c7d705f1bf25b7d469fa153f7d19eb/futures-3.2.0.tar.gz futures requires Python '>=2.6, <3' but the running Python is 3.5.3 #+END_SRC - search: pylint futures requires Python '>=2.6, <3' but the running Python is 3.5.3 search: isort futures broken - try fix isort to older version. isort is used to sort import statements. it has no dependencies. what does that collect line mean? // isort does have dependencies. I fixed isort to 4.2.5 and it works. https://github.com/timothycrosley/isort/blob/4.3.9/setup.py install_requires=['futures; python_version < "3.2"'], it has required futures pkg. the setup.py syntax changed in v4.3.5 v4.3.4 should work fine. old syntax: install_requires = [] if sys.version_info.major == 2: install_requires = ['futures'] ... install_requires=install_requires, new syntax: install_requires=['futures; python_version < "3.2"'], the new syntax doc: Building and Distributing Packages with Setuptools — setuptools 40.8.0 documentation https://setuptools.readthedocs.io/en/latest/setuptools.html#declaring-platform-specific-dependencies Declaring platform specific dependencies PEP 508 -- Dependency specification for Python Software Packages | Python.org https://www.python.org/dev/peps/pep-0508/ Created: 11-Nov-2015 which setuptools version start to support this feature? Why I hit this problem today? This is old stuff. isort 4.3.5 released on Feb 25, 2019. more people will be hit by this soon. does newer pip have the problem? yes. using setuptools 28.8.0 Nov 2016 after upgrading setuptools and pip, still have problem installing latest isort pkg. - reproduce mkdir ~/d/t3 cd ~/d/t3 virtualenv .venv .venv/bin/pip install isort .venv/bin/pip install --no-cache-dir isort .venv/bin/pip uninstall isort this fails on python 3.5.3 on debian stretch. I think the virtualenv is too old. python3 -m venv .venv works fine on stretch live distro. but python3 -m venv also fails on ryzen5. why? It may be my local environment's problem. God know which component's problem. try use no cache. no cache worked! - how to removed the cached version of isort? sylecn@ryzen5:~/d/t3$ du -sh ~/.cache/pip 239M /home/sylecn/.cache/pip sylecn@ryzen5:~/d/t3$ - search: python pip cache and platform specific dependencies - Because it's local cache problem. I will just clear local pip cache. and also upgrade local pip. ** 2019-02-27 after I upgraded lein, new lein install new version has an error. Template resource 'leiningen/new/python/.gitignore' not found. .gitignore file does not get included into a template · Issue #2522 · technomancy/leiningen · GitHub https://github.com/technomancy/leiningen/issues/2522 This is a known problem. ** 2019-02-27 add document in project README on how to do CD. added README-dev.rst ** 2016-07-27 package name should not have -, auto convert it to _. - package name is part of module name, - is not valid there. Loading project.clj +2 −8 Original line number Diff line number Diff line (defproject python/lein-template "1.4.1" (defproject python/lein-template "1.4.2" :description "lein template for a python project" :repositories [["snapshots" {:url "http://devserv.game.yy.com/nexus/content/repositories/snapshots" :creds :gpg}] ["releases" {:url "http://devserv.game.yy.com/nexus/content/repositories/releases" :creds :gpg}]] :url "http://emacsos.com/lein/python" :url "https://blog.emacsos.com/python-project-template" :license {:name "Eclipse Public License" :url "http://www.eclipse.org/legal/epl-v10.html"} :eval-in-leiningen true) src/leiningen/new/python.clj +5 −1 Original line number Diff line number Diff line Loading @@ -39,6 +39,8 @@ :date (let [cal (Calendar/getInstance)] (.format (new SimpleDateFormat "yyyy-MM-dd") (.getTime cal)))} dot-template-files ["gitignore" ] template-files ["Makefile" "fabfile.py" "tox.ini" Loading @@ -48,6 +50,7 @@ "Dockerfile.template" "app.yaml" "README.rst" "README-dev.rst" "utils/versionutils.py" "utils/build-deb" "utils/install-git-hooks" Loading @@ -65,7 +68,7 @@ "git-hooks/post-commit" "git-hooks/post-merge" "git-hooks/pre-commit" ".gitignore"] ] plain-files ["utils/__init__.py" "utils/choose_default_python.sh" "utils/virtualenv-15.1.0/virtualenv.py" Loading @@ -88,6 +91,7 @@ (apply ->files data (concat (map (fn [f] [f ""]) empty-files) (map (fn [f] [(str "." f) (render f data)]) dot-template-files) (map (fn [f] [f (render f data)]) template-files) (map (fn [f] [(str "{{python-pkg-name}}/" f) (render f data)]) package-files) (map (fn [f] [f (io/input-stream (get-template-file f))]) Loading src/leiningen/new/python/README-dev.rst 0 → 100644 +86 −0 Original line number Diff line number Diff line README for developers ========================== How to add configurable variables ---------------------------------------- User configurations can be defined in {{python-pkg-name}}/config.py Default config keys and values are defined in config.py. Runtime configs can be read from config file or environment variable. Config file path is specified in config.py. Config file format is: :: # empty line and comment line are ignored key1=value1 key2=value2 Every config key can be passed in as environment variable as well. The environment variable name is by replace . (dot) and - (hyphen-minus) by _ (underscore), then all upper case. For example, db.host env variable should be DB_HOST. If the same key is defined in more than one place, env var > file config > default config. To use config variable in code, .. code-block:: python from {{python-pkg-name}}.config import CONF CONF.getstr('some.key') CONF.getint('some.key') CONF.getbool('some.key') CONF.getfloat('some.key') How to run tests and deploy projects ----------------------------------------- - In dev environment, .. code-block:: bash make test make run - In production, deploy via deb file, update scripts in deb-scripts/ install fpm the package builder. see https://github.com/jordansissel/fpm install fabric tool. see http://www.fabfile.org/ .. code-block:: bash make test make deb fab deploy - In production, deploy via kubernetes and docker image. in Makefile, update DOCKER_IMAGE_PREFIX and BUILD_DOCKER_IMAGE if necessary, default template use my private docker reg. Run test in docker container and build docker image: define DOCKER_USER and DOCKER_PASSWORD env variable. .. code-block:: bash make ci-build Deploy in kubernetes cluster: define KUBECTL_API and KUBECTL_TOKEN env variable. .. code-block:: bash make deploy default template use ingress to provide https access to the service. In other environment, the app.yaml template should be updated to fit in the environment. src/leiningen/new/python/.gitignore→src/leiningen/new/python/gitignore +0 −0 File moved. View file Loading
operational +109 −1 Original line number Diff line number Diff line * COMMENT -*- mode: org -*- #+Date: 2014-05-09 Time-stamp: <2018-05-02> Time-stamp: <2019-02-27> #+STARTUP: content * notes :entry: ** 2018-05-02 support test, build and deploy in gocd Loading Loading @@ -48,6 +48,8 @@ Time-stamp: <2018-05-02> * current :entry: ** ** 2019-02-24 fix :url in project.clj create a blog post about this template. ** 2016-09-08 support updating site file, deb scripts etc after project is generated. - the domain in site file could change. - the pkg name could change. Loading Loading @@ -80,6 +82,112 @@ see example ~/.pylintrc ** 2014-05-09 add sphinx support * later :entry: * done :entry: ** 2019-02-24 drop :repositories from project.clj ** 2019-02-27 another problem on new created project. make test fail on new project. futures requires Python '>=2.6, <3' but the running Python is 3.5.3 one pkg in requirements-dev.txt has bad dependencies. pylint has problem. #+BEGIN_SRC sh Collecting futures (from isort>=4.2.5->pylint) Using cached https://pypi.tuna.tsinghua.edu.cn/packages/1f/9e/7b2ff7e965fc654592269f2906ade1c7d705f1bf25b7d469fa153f7d19eb/futures-3.2.0.tar.gz futures requires Python '>=2.6, <3' but the running Python is 3.5.3 #+END_SRC - search: pylint futures requires Python '>=2.6, <3' but the running Python is 3.5.3 search: isort futures broken - try fix isort to older version. isort is used to sort import statements. it has no dependencies. what does that collect line mean? // isort does have dependencies. I fixed isort to 4.2.5 and it works. https://github.com/timothycrosley/isort/blob/4.3.9/setup.py install_requires=['futures; python_version < "3.2"'], it has required futures pkg. the setup.py syntax changed in v4.3.5 v4.3.4 should work fine. old syntax: install_requires = [] if sys.version_info.major == 2: install_requires = ['futures'] ... install_requires=install_requires, new syntax: install_requires=['futures; python_version < "3.2"'], the new syntax doc: Building and Distributing Packages with Setuptools — setuptools 40.8.0 documentation https://setuptools.readthedocs.io/en/latest/setuptools.html#declaring-platform-specific-dependencies Declaring platform specific dependencies PEP 508 -- Dependency specification for Python Software Packages | Python.org https://www.python.org/dev/peps/pep-0508/ Created: 11-Nov-2015 which setuptools version start to support this feature? Why I hit this problem today? This is old stuff. isort 4.3.5 released on Feb 25, 2019. more people will be hit by this soon. does newer pip have the problem? yes. using setuptools 28.8.0 Nov 2016 after upgrading setuptools and pip, still have problem installing latest isort pkg. - reproduce mkdir ~/d/t3 cd ~/d/t3 virtualenv .venv .venv/bin/pip install isort .venv/bin/pip install --no-cache-dir isort .venv/bin/pip uninstall isort this fails on python 3.5.3 on debian stretch. I think the virtualenv is too old. python3 -m venv .venv works fine on stretch live distro. but python3 -m venv also fails on ryzen5. why? It may be my local environment's problem. God know which component's problem. try use no cache. no cache worked! - how to removed the cached version of isort? sylecn@ryzen5:~/d/t3$ du -sh ~/.cache/pip 239M /home/sylecn/.cache/pip sylecn@ryzen5:~/d/t3$ - search: python pip cache and platform specific dependencies - Because it's local cache problem. I will just clear local pip cache. and also upgrade local pip. ** 2019-02-27 after I upgraded lein, new lein install new version has an error. Template resource 'leiningen/new/python/.gitignore' not found. .gitignore file does not get included into a template · Issue #2522 · technomancy/leiningen · GitHub https://github.com/technomancy/leiningen/issues/2522 This is a known problem. ** 2019-02-27 add document in project README on how to do CD. added README-dev.rst ** 2016-07-27 package name should not have -, auto convert it to _. - package name is part of module name, - is not valid there. Loading
project.clj +2 −8 Original line number Diff line number Diff line (defproject python/lein-template "1.4.1" (defproject python/lein-template "1.4.2" :description "lein template for a python project" :repositories [["snapshots" {:url "http://devserv.game.yy.com/nexus/content/repositories/snapshots" :creds :gpg}] ["releases" {:url "http://devserv.game.yy.com/nexus/content/repositories/releases" :creds :gpg}]] :url "http://emacsos.com/lein/python" :url "https://blog.emacsos.com/python-project-template" :license {:name "Eclipse Public License" :url "http://www.eclipse.org/legal/epl-v10.html"} :eval-in-leiningen true)
src/leiningen/new/python.clj +5 −1 Original line number Diff line number Diff line Loading @@ -39,6 +39,8 @@ :date (let [cal (Calendar/getInstance)] (.format (new SimpleDateFormat "yyyy-MM-dd") (.getTime cal)))} dot-template-files ["gitignore" ] template-files ["Makefile" "fabfile.py" "tox.ini" Loading @@ -48,6 +50,7 @@ "Dockerfile.template" "app.yaml" "README.rst" "README-dev.rst" "utils/versionutils.py" "utils/build-deb" "utils/install-git-hooks" Loading @@ -65,7 +68,7 @@ "git-hooks/post-commit" "git-hooks/post-merge" "git-hooks/pre-commit" ".gitignore"] ] plain-files ["utils/__init__.py" "utils/choose_default_python.sh" "utils/virtualenv-15.1.0/virtualenv.py" Loading @@ -88,6 +91,7 @@ (apply ->files data (concat (map (fn [f] [f ""]) empty-files) (map (fn [f] [(str "." f) (render f data)]) dot-template-files) (map (fn [f] [f (render f data)]) template-files) (map (fn [f] [(str "{{python-pkg-name}}/" f) (render f data)]) package-files) (map (fn [f] [f (io/input-stream (get-template-file f))]) Loading
src/leiningen/new/python/README-dev.rst 0 → 100644 +86 −0 Original line number Diff line number Diff line README for developers ========================== How to add configurable variables ---------------------------------------- User configurations can be defined in {{python-pkg-name}}/config.py Default config keys and values are defined in config.py. Runtime configs can be read from config file or environment variable. Config file path is specified in config.py. Config file format is: :: # empty line and comment line are ignored key1=value1 key2=value2 Every config key can be passed in as environment variable as well. The environment variable name is by replace . (dot) and - (hyphen-minus) by _ (underscore), then all upper case. For example, db.host env variable should be DB_HOST. If the same key is defined in more than one place, env var > file config > default config. To use config variable in code, .. code-block:: python from {{python-pkg-name}}.config import CONF CONF.getstr('some.key') CONF.getint('some.key') CONF.getbool('some.key') CONF.getfloat('some.key') How to run tests and deploy projects ----------------------------------------- - In dev environment, .. code-block:: bash make test make run - In production, deploy via deb file, update scripts in deb-scripts/ install fpm the package builder. see https://github.com/jordansissel/fpm install fabric tool. see http://www.fabfile.org/ .. code-block:: bash make test make deb fab deploy - In production, deploy via kubernetes and docker image. in Makefile, update DOCKER_IMAGE_PREFIX and BUILD_DOCKER_IMAGE if necessary, default template use my private docker reg. Run test in docker container and build docker image: define DOCKER_USER and DOCKER_PASSWORD env variable. .. code-block:: bash make ci-build Deploy in kubernetes cluster: define KUBECTL_API and KUBECTL_TOKEN env variable. .. code-block:: bash make deploy default template use ingress to provide https access to the service. In other environment, the app.yaml template should be updated to fit in the environment.