The Wayback Machine - http://web.archive.org/web/20221221090136/https://github.com/cookiecutter/cookiecutter-django/issues/3661
Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

'apt-get install git-all' not making it through to 'pip install' of requirements #3661

Open

dantagg opened this issue Mar 31, 2022 · 2 comments
Labels

Comments

@dantagg
Copy link

dantagg commented Mar 31, 2022

What happened?

I updated local django Dockerfile python-build-stage to include

RUN apt-get update && apt-get install --no-install-recommends -y \
# dependencies for building Python packages
build-essential \
# including git \
git-all \
# psycopg2 dependencies
libpq-dev

and updated python-run-stage to include

RUN apt-get update && apt-get install --no-install-recommends -y \
# psycopg2 dependencies
libpq-dev \
# Translations dependencies
gettext \
# including git \
git-all \
# cleaning up unused files
&& apt-get purge -y --auto-remove -o APT::AutoRemove::RecommendsImportant=false \
&& rm -rf /var/lib/apt/lists/*

I did this because one of my pip requirements needs git to install, for example

git+https://github.com/mozilla/django-csp.git

It isn't this, but the one I'm installing is in a private repository.

When I run

docker compose -f local.yml build

it fails with the following error
ERROR: Cannot find command 'git' - do you have 'git' installed and in your PATH?

Looking at the build log, there is an error

 => CACHED [django_fabman_local_docs python-build-stage 1/3] RUN apt-get update && apt-get install --no-install-recommends -y   build-essential   libpq-dev   && apt-  0.0s
 => CACHED [django_fabman_local_docs python-build-stage 2/3] COPY ./requirements /requirements                                                                         0.0s
 => ERROR [django_fabman_local_docs python-build-stage 3/3] RUN pip wheel --no-cache-dir --no-deps --wheel-dir /usr/src/app/wheels    -r /requirements/local.txt -r   16.1s

and then further down there is

=> CANCELED [django_fabman_local_django python-build-stage 1/3] RUN apt-get update && apt-get install --no-install-recommends -y   build-essential   git-all   libp  14.3s

This suggests the pip install is running ahead and being attempted before the apt-get installs have finished and the when pip fails, the APT-GET is cancelled because the whole build has borked.

What should've happened instead?

The build should have completed successfully

Additional details

  • Host system configuration:

    • Version of cookiecutter CLI

    Cookiecutter 1.7.3

    • OS name and version:

    ProductName: macOS
    ProductVersion: 11.6.4
    BuildVersion: 20G417
    Docker Compose version v2.3.3
    Docker version 20.10.13, build a224086
    Python on host: Python 3.10.0 (v3.10.0:b494f5935c
    Python on Docker: Python 3.9.12 (main, Mar 29 2022, 14:20:48) [GCC 10.2.1 20210110] on linux

    • Python version, run python3 -V:
    • Docker version (if using Docker), run docker --version:
    • docker-compose version (if using Docker), run docker-compose --version:
    • ...
  • Options selected and/or replay file:
    On Linux and MacOS: cat ${HOME}/.cookiecutter_replay/cookiecutter-django.json
    (Please, take care to remove sensitive information)

    {
    "cookiecutter": {
    "project_name": "Django Fabman",
    "project_slug": "django_fabman",
    "description": "Service to support the various extensions of Fabman for KWMC Factory",
    "author_name": "Daniel Tagg",
    "domain_name": "kwmc.factory",
    "email": "***@***.com",
    "version": "0.1.0",
    "open_source_license": "Not open source",
    "timezone": "Europe/London",
    "windows": "n",
    "use_pycharm": "y",
    "use_docker": "y",
    "postgresql_version": "14.1",
    "cloud_provider": "None",
    "mail_service": "Other SMTP",
    "use_async": "n",
    "use_drf": "y",
    "frontend_pipeline": "None",
    "use_celery": "n",
    "use_mailhog": "n",
    "use_sentry": "n",
    "use_whitenoise": "y",
    "use_heroku": "n",
    "ci_tool": "Gitlab",
    "keep_local_envs_in_vcs": "n",
    "debug": "n",
    "_template": "gh:cookiecutter/cookiecutter-django"
    }
    
@dantagg dantagg added the bug label Mar 31, 2022
@xjlin0
Copy link

xjlin0 commented May 7, 2022

instead of git-all, how about just git? I also install git in my Dockerfile and it works

ARG PYTHON_VERSION=3.9-slim-bullseye

# define an alias for the specfic python version used in this file.
FROM python:${PYTHON_VERSION} as python

# Python build stage
FROM python as python-build-stage
RUN echo "deb http://deb.debian.org/debian bullseye main" > /etc/apt/sources.list
ARG BUILD_ENVIRONMENT=local

# Install apt packages
RUN apt-get update && apt-get install --no-install-recommends -y \
  # dependencies for building Python packages
  build-essential curl vim wget git    ####### HERE #######

and in requirements/base.py

...
celery==5.2.6  # pyup: < 6.0  # https://github.com/celery/celery
-e git+https://github.com/llazzaro/django-scheduler.git@develop#egg=django-scheduler  ### THIS ###
django-celery-beat==2.2.1  # https://github.com/celery/django-celery-beat
...

@luzfcb
Copy link
Collaborator

luzfcb commented May 7, 2022

@xjlin0

Another option is not to use git (and ssh git clone) at all:

The latest version of pip supports this syntax:

<NAME-OF-THE-PACKAGE>@<FULL-URL-TO-THE-COMPRESSED-PYTHON-PACKAGE>.<tar.gz-or-zip>

For GitHub, it can be translated to:

<NAME-OF-THE-PACKAGE>@https://github.com/<USER>/<REPOSITORY-NAME>/archive/<BRANCH_NAME_OR_COMMIT_HASH>.<tar.gz-or-zip>

Sample:

celery==5.2.6  # pyup: < 6.0  # https://github.com/celery/celery
django-schedule@https://github.com/llazzaro/django-scheduler/archive/develop.tar.gz  ### THIS ###
django-celery-beat==2.2.1  # https://github.com/celery/django-celery-beat

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

3 participants