![]() #0 6.491 raise CalledProcessError(retcode, process.args, #0 6.491 return run(*popenargs, stdout=PIPE, timeout=timeout, check=True, #0 6.491 File "/usr/local/lib/python3.10/subprocess.py", line 421, in check_output #0 6.491 File "/tmp/pip-build-env-zzg3_cq/overlay/lib/python3.10/site-packages/setuptools/build_meta.py", line 335, in run_setup #0 6.491 File "/tmp/pip-build-env-zzg3_cq/overlay/lib/python3.10/site-packages/setuptools/build_meta.py", line 320, in _get_build_requires #0 6.491 return self._get_build_requires(config_settings, requirements=) #0 6.491 File "/tmp/pip-build-env-zzg3_cq/overlay/lib/python3.10/site-packages/setuptools/build_meta.py", line 338, in get_requires_for_build_wheel #0 6.491 File "/home/airflow/.local/lib/python3.10/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 118, in get_requires_for_build_wheel #0 6.491 File "/home/airflow/.local/lib/python3.10/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 335, in main #0 6.491 File "/home/airflow/.local/lib/python3.10/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in #0 6.491 Traceback (most recent call last): #0 6.491 /bin/sh: 1: krb5-config: Permission denied #0 6.491 × Getting requirements to build wheel did not run successfully. #0 6.491 error: subprocess-exited-with-error #0 6.487 Getting requirements to build wheel: finished with status 'error' #0 6.212 Getting requirements to build wheel: started #0 6.210 Installing build dependencies: finished with status 'done' #0 1.892 Installing build dependencies: started This is the error message: > RUN pip install -no-cache-dir -user -r /requirements.txt : I’m not sure what to do to fix this error. However, it seems the HDFS library requires the Kerbedos library, and it causes this error. I tried to include the HDFS library in the requirements.txt, and it seems to install properly. My docker version: Docker version 20.10.17, build 100c701 ![]() script.py will be created in the same directory as the Dockerfile.I’ve struggled to install the apache-airflow-providers-apache-hdfs package into Airflow - Docker 2.5.3 container on Mac. Here’s some sample code we will call script.py from Spacy that iterates over every token in the string Hello World!. SpaCy is a free, open-source library for advanced Natural Language Processing (NLP) in Python. It’s disadvantages are the slower build times and significantly larger final containers. Conda has the advantage of including non-python dependencies. However, because pip can only install Python packages, you may find yourself also having to use your package manager (i.e., apt-get install -y dependency) to install non Python dependencies. In situations where you can find both, using pip and and a your Linux package manager (e.g., apt-get in Debian and Ubuntu) can yield quicker build times and smaller final container sizes. In that case, the decision has been made for you. There may be instances where you can only find directions on how to install an application with one tool or the either. Pip and conda are the two most popular ways to install python packages. Group to use: compute-workshop (if part of multiple groups) Workshop Video ¶ pip (and a Linux package manger) vs anaconda ¶ Connecting to get command line access: ssh to use: workshop, workshop-interactive
0 Comments
Leave a Reply. |