3 min read, 600 words

Creating Debian packages for reproducible Node installations

Update I recently had to repackage puppeteer to update versions and discovered that FPM now correctly drops everything into /usr/lib/. This means the content below is no longer required, but I will leave it up for reference.

While working with my front-end Team at Truveris to refactor our PDF rendering pipeline, we came across and opted to move forward with Puppeteer. This would give us a flexible, fairly complete headless-chromium based tool-chain, with minimal customization required. After a quick PoC lead by the front-end Team, it came time to figure out the best way to provide puppeteer to the required infrastructure, in a operationally friendly manner. This meant packaging puppeteer and its dependencies in a reproducible way, which for us is typically a stable artifact in the form of a standard debian package.

Now, the complexities of operationalizing node and friends, in a way that does not conflict with either creating reproducible builds or immutable(-ish) infrastructure. This meant the typical npm install (or what ever the current flavor of the week is) was not an option for us, as even with npm shrinkwrap it is possible for dependencies to shift over time as they are fetched. Furthermore, the typical node workflow expects every package to have its own self contained dependencies and to not really share anything at all with other components of the local system. This mean either packaging every upstream service of ours that utilizes puppeteer with a self-contained complete copy of the resultant node_modules tree, or finding a more sustainable way to segment off puppeteer into a more deployment friendly dependency.

Fortunately FPM is a thing, which cuts out 90% of the work required to get to our desired state of a globally accessible dependency, shared by many services. There is light at the end of the tunnel, everything should be solved with a single fpm command…

Unfortunately, FPM approaches Node/NPM packaging from the perspective of installing tools or binaries to be run, rather than installing libraries to be included downstream. As such, some post processing of the resultant fpm package is required. Fortunately node didn’t entirely lock us out of such a globally accessible library, and has a couple of options in the default $PATH, so we should be able to relocate the required files to the least offensive default path element, /usr/lib/node.

So, to begin, we are using FPM to fetch from npm, pull in dependencies and build a base package:

$ fpm -s npm -t deb puppeteer

As mentioned before fpm, by default, will place the entire node_modules tree into /usr/share/puppeteer/app/node_modules/, which is unhelpful for us, so some post processing is required.

Extract the new Debian package to a temporary destination:

$ dpkg-deb -R node-puppeteer_1.3.0-dan2_amd64.deb puppeteer-raw
$ cd puppeteer-raw
$  tree -L 2
|   |-- control
|   `-- md5sums
`-- usr
    |-- local
    `-- share

Move the nested node_modules and clean up the now empty directories:

$ mkdir lib
$ mv usr/local/lib/node_modules/ usr/lib/node
$ rm -r usr/local

Fix the permissions for the local install of chromium:

$ find . -perm 700 -exec chmod go+x {} \;

Regenerate DEBIAN/md5sums for pedantic reasons

md5sum $(find . -type f | grep -v '^[.]/DEBIAN/') >DEBIAN/md5sums

Update the build number to the version in DEBIAN/control and Rebuild the package:

$ fakeroot dpkg-deb -b raw/ node-puppeteer_1.5.0-dan1_amd64.deb

Note: standard fakeroot reasoning applies, without fakeroot dpkg-deb will use the current permissions and ownership, which will probably end up not working at all once the package is installed.

Logging and capturing messages from libraries and nested modules Cookiecutter-flask and GitLab-CI

comments powered by Disqus