An Old Tree Blooms Anew - Cloud Native Buildpacks

1. Buildpacks, an Old Tree Blooms Anew

The Buildpacks project was first started by Heroku in 2011 and was widely adopted by PaaS platforms such as Cloud Foundry. In an earlier document, Buildpack for PaaS Deployment, I demonstrated how to deploy a Django application to Heroku.

The shortcoming of Buildpacks was that the artifact it produced was in Droplet format, which could not be adapted directly to container platforms.

In January 2018, Pivotal and Heroku started a project called Cloud Native Buildpacks (CNB for short), and it joined the CNCF that October. The goal of the project is to realize a unified application packaging ecosystem.

Put simply, the artifact used to be a Droplet, and now it is an OCI container image that can be deployed on any container platform compatible with OCI images. The working principle is as follows:

Detect. Based on the source code content, automatically detect and match buildpacks.
Analyze. Look for caches from previous builds.
Build. Turn the application’s source code into a runnable package.
Export an OCI-format container image.

2. Who Cloud Native Buildpacks Is For

Cloud Native Buildpacks is a great fit for scenarios that require frequent application packaging, and where the variety of applications is large.

aPaaS platforms

aPaaS is a platform that provides a runtime and often needs to deploy all kinds of applications. Using Cloud Native Buildpacks standardizes the packaging process and quickly turns source code into images.

FaaS platforms

A FaaS platform provides a runtime environment for functions. On container-centric infrastructure, FaaS packages user code into images and then creates containers to run them. Cloud Native Buildpacks can both detect the programming language and framework of user code and quickly compile it into an OCI image, which is exactly what FaaS needs. The FaaS of Google Cloud Platform uses Buildpacks.

Developers

Not every developer is suited to using Cloud Native Buildpacks; learning and maintaining it also has a cost. If you have many applications to maintain and release, then Cloud Native Buildpacks will save you — at the very least you can write far fewer Dockerfiles.

3. Comparison with Similar Application Packaging Tools

Regarding S2I, I wrote a document earlier, Using S2I to Build Cloud Native Applications. In a product I was responsible for developing, S2I was used to package source code into images.

Buildpacks, on the other hand, is a technology I came across in my previous job. At the time I was developing SaaS and needed Buildpacks to package applications.

This is the comparison of similar tools from the Buildpacks community:

Logo
Product Name	Cloud Native Buildpacks	source-to-image (s2i)	Jib	ko
Advanced Caching	Yes	Yes	No	No
Bill-of-Materials	Yes	No	No	No
Modular / Pluggable	Yes	No	N/A †	N/A †
Multi-language	Yes	Yes	No	No
Multi-process	Yes	No	No	No
Minimal app image	Yes	Yes ‡	Yes	Yes
Rebasing	Yes	No	No	No
Reproducibility	Yes	No	Yes	Yes
Reusability	Yes	Yes	N/A †	N/A †
Integrations	_ Azure _ CircleCI _ GitLab _ Google _ Heroku _ Spring Boot _ Tekton _ …	* OpenShift	_ Gradle _ Maven
Governance	CNCF	Red Hat	Google	Google

From the standpoint of ecosystem and features, Buildpacks is better than S2I.

4. Packaging an Application in a Docker Environment

Git must be installed in advance; the following is performed in a Docker environment.

4.1 Installing the Pack Command-Line Tool

Pack is the tool Cloud Native Buildpacks provides for using buildpacks. Here Linux is used as the example for installation:

1
(curl -sSL "https://github.com/buildpacks/pack/releases/download/v0.17.0/pack-v0.17.0-linux.tgz" | sudo tar -C /usr/local/bin/ --no-same-owner -xzv pack)

Check the pack version

1
2
3
pack version

0.17.0+git-d9cb4e7.build-2045

4.2 Packaging an Application

Clone the code

1
git clone https://github.com/shaowenchen/devops-python-sample

Enter the code directory

1
cd devops-python-sample/src

View the code directory

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
tree

.
|-- manage.py
|-- project
|   |-- app1
|   |   |-- admin.py
|   |   |-- apps.py
|   |   |-- __init__.py
|   |   |-- migrations
|   |   |   `-- __init__.py
|   |   |-- models.py
|   |   |-- tests.py
|   |   `-- views.py
|   |-- __init__.py
|   |-- settings.py
|   |-- tests.py
|   |-- urls.py
|   `-- wsgi.py
`-- requirements.txt

3 directories, 14 files

This is a very ordinary Django project structure.

Check the recommended builders

A builder defines the steps for turning source code into an image. If there is a builder you are familiar with, you can use it directly, or you can use the builder recommended by Pack.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
pack builder suggest

Suggested builders:
	Google:                gcr.io/buildpacks/builder:v1      Ubuntu 18 base image with buildpacks for .NET, Go, Java, Node.js, and Python
	Heroku:                heroku/buildpacks:18              heroku-18 base image with buildpacks for Ruby, Java, Node.js, Python, Golang, & PHP
	Heroku:                heroku/buildpacks:20              heroku-20 base image with buildpacks for Ruby, Java, Node.js, Python, Golang, & PHP
	Paketo Buildpacks:     paketobuildpacks/builder:base     Ubuntu bionic base image with buildpacks for Java, .NET Core, NodeJS, Go, Ruby, NGINX and Procfile
	Paketo Buildpacks:     paketobuildpacks/builder:full     Ubuntu bionic base image with buildpacks for Java, .NET Core, NodeJS, Go, PHP, Ruby, Apache HTTPD, NGINX and Procfile
	Paketo Buildpacks:     paketobuildpacks/builder:tiny     Tiny base image (bionic build image, distroless-like run image) with buildpacks for Java Native Image and Go

Tip: Learn more about a specific builder with:
	pack inspect-builder <builder-image>

Run the build; here heroku/buildpacks:20 is used for the build

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
pack build python-sample --builder heroku/buildpacks:20

20: Pulling from heroku/buildpacks
Digest: sha256:128508a60a25ecac8ed16b4507747cceaf627e2617b2e898d98ac303411a010f
Status: Image is up to date for heroku/buildpacks:20
20: Pulling from heroku/pack
Digest: sha256:6f7301c933d3d0d777c3f5b99b5cc9d8e5ee7304e69b5f05989e19e5c4206a95
Status: Image is up to date for heroku/pack:20
===> DETECTING
======== Output: heroku/ruby@0.0.1 ========
no
err:  heroku/ruby@0.0.1 (1)
heroku/python   0.3
heroku/procfile 0.6
===> ANALYZING
Previous image with name "python-sample" not found
===> RESTORING
===> BUILDING
-----> Installing python-3.6.12
-----> Installing pip 20.1.1, setuptools 47.1.1 and wheel 0.34.2
-----> Installing SQLite3
-----> Installing requirements with pip
 !     Your Django version is nearing the end of its community support.
 !     Upgrade to continue to receive security updates and for the best experience with Django.
 !     For more information, check out https://www.djangoproject.com/download/#supported-versions
       Collecting Django==1.11.29
         Downloading Django-1.11.29-py2.py3-none-any.whl (6.9 MB)
       Collecting coverage==4.5.4
         Downloading coverage-4.5.4-cp36-cp36m-manylinux1_x86_64.whl (205 kB)
       Collecting pytz
         Downloading pytz-2021.1-py2.py3-none-any.whl (510 kB)
       Installing collected packages: pytz, Django, coverage
       Successfully installed Django-1.11.29 coverage-4.5.4 pytz-2021.1
-----> $ python manage.py collectstatic --noinput
       61 static files copied to '/workspace/static'.

-----> Discovering process types
       Procfile declares types     -> (none)
===> EXPORTING
Adding layer 'heroku/python:profile'
Adding 1/1 app layer(s)
Adding layer 'launcher'
Adding layer 'config'
Adding label 'io.buildpacks.lifecycle.metadata'
Adding label 'io.buildpacks.build.metadata'
Adding label 'io.buildpacks.project.metadata'
Warning: default process type 'web' not present in list []
*** Images (1de89df98f60):
      python-sample
Adding cache layer 'heroku/python:shim'
Successfully built image python-sample

View the image

1
2
3
docker images |grep python-sample

python-sample                     latest              39a646c853c2        41 years ago        756MB

41 years ago is probably a bug, and the image is a bit large. On the second build I did not see any obvious speedup, so it seems a custom builder is needed to show the advantages in the table above.

Create a container to try it

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
docker run --security-opt apparmor=unconfined --security-opt seccomp=unconfined --rm -p 8000:8000 python-sample  python manage.py runserver 0.0.0.0:8000

Performing system checks...

System check identified no issues (0 silenced).

You have 13 unapplied migration(s). Your project may not work properly until you apply the migrations for app(s): admin, auth, contenttypes, sessions.
Run 'python manage.py migrate' to apply them.
February 04, 2021 - 08:33:02
Django version 1.11.29, using settings 'project.settings'
Starting development server at http://0.0.0.0:8000/
Quit the server with CONTROL-C.

Access the page on port 8000.

5. Summary

This article was mainly a hands-on look at Cloud Native Buildpacks, the evolved version of Buildpacks in the cloud native era.

Like S2I, Buildpacks lets developers define their own builders, and also lets users define some configuration in the source repository, such as specifying the source directory or the interpreter version. Platforms in the PaaS and FaaS category are better off adopting and contributing to these open-source projects than maintaining a similar toolset of their own.