digitalpulp / ballast
A local development toolset for automating Docker setup and Drupal development.
Installs: 228
Dependents: 0
Suggesters: 0
Security: 0
Stars: 18
Watchers: 4
Forks: 3
Open Issues: 0
Type:project
Requires
- composer/installers: ^1.2
- consolidation/robo: ^1.0
- cweagans/composer-patches: ^1.6.5
- drupal-composer/drupal-scaffold: ^2.5
- drupal/console: ^1.9.2
- drupal/core-recommended: ^8.8.1
- drush/drush: ^9.7
- oomphinc/composer-installers-extender: ^1.1
- webflo/drupal-finder: ^1.0.0
- webmozart/path-util: ^2.3
- zaporylie/composer-drupal-optimizations: ^1.1
Requires (Dev)
- drupal/core-dev: ^8.8.1
Conflicts
README
A local development toolset developed with the support of Digital Pulp.
Key contributors:
A Composer template for Drupal projects with Docker
This project template automates Docker based local development with Drupal Composer workflows. The local development automation is currently only optimized for macOS but Linux and possibly Windows may follow.
- Site dependencies are managed with Composer.
- Setup and management of Docker is automated.
Getting Started
-
First you need to install composer. Note: The instructions below refer to the global composer installation. You might need to replace
composer
withphp composer.phar
(or similar) for your setup. -
Ballast will check your system for needed software. If anything is missing, a list of missing packages will be provided.
-
OS Specific Notes:
-
macOS: Your Docker Sites need a home.
- Choose or create a file folder to hold all the site folders for projects managed with this approach.
- If you have any existing files exported via NFS they must not be in the chosen folder.
- You must also not choose a folder that is a child of an existing NFS export.
- The easiest way forward is
to create a new folder such as
~/DockerSites
.
-
Linux: File permissions are simplest if your user belongs to the same group as the webserver. Nginx runs as group id 82. If this group id does not exist, you should create it and add your user to it. Setting any files that need to be writeable can then be set to 775 (group writeable) so they are writeable by Drupal. You will need to configure your system to forward all requests for
*.test
to the loop back address (localhost). We recommend using dnsmasq, which is well known and should be available via your package manager. The key task is to setaddress=/test/127.0.0.1
in the dnsmasq configuration. Here are some links to helpful blog posts for some common flavors of Linux: -
Windows Linux Subsystem: Build and manage your Ballast sites within Linux. A Windows equivalent to dnsmasq appears to be Acrylic DNS Proxy as described in Setting Up A Windows 10 Development Environment with WSL, PHP & Laravel or if the number of sites are limited, the local FQDN,
our-site.test
, could be pointed to the loopback address in your hosts file:
127.0.0.1 our-site.test
Managing Theme Tasks
There are ahoy commands for running theme tasks in the front-end container, so you can choose to not install node on
your host. When you first setup a site with an established theme, you will probably need to run
ahoy npm install --no-save
to install the node based theme tools. Additional ahoy commands for front-end task can be
found via ahoy --help
.
Initial Setup
After the initial setup, you should delete the Initial Setup section of this README.
In the folder chosen or created under Getting Started, composer create-project digitalpulp/ballast your_project
.
Edit setup/config.yml
There are some project specific values that should be set in this file. The
default location for custom themes is docroot/themes/custom
. This can be
altered by adding a site_theme_path
key to this file and setting it to the
relative path from the project root to parent directory for custom themes.
Edit composer.json
There is also a project specific value to set here, which is the path to
bower components in the custom theme file. Edit the extra section to
add a path to the theme bower_components
directory. An example entry
is shown as the last line in the snippet below:
{ "extra": { "installer-types": ["bower-asset"], "installer-paths": { "docroot/core": ["type:drupal-core"], "docroot/libraries/{$name}": ["type:drupal-library"], "docroot/modules/contrib/{$name}": ["type:drupal-module"], "docroot/profiles/contrib/{$name}": ["type:drupal-profile"], "docroot/themes/contrib/{$name}": ["type:drupal-theme"], "drush/contrib/{$name}": ["type:drupal-drush"], "docroot/themes/custom/theme_name/bower_components/{$name}": ["type:bower-asset"] } } }
In addition, an additional property is available for the drupal-scaffold
key in the extra
property:
{ "file-mapping": { "[web-root]/.htaccess": false, "[web-root]/.eslintrc.json": false, "[web-root]/.ht.router.php": false, "[web-root]/INSTALL.txt": false, "[web-root]/README.txt": false, "[web-root]/autoload.php": false, "[web-root]/example.gitignore": false, "[web-root]/index.php": false, "[web-root]/robots.txt": false, "[web-root]/update.php": false, "[web-root]/web.config": false, "[web-root]/sites/default/settings.php": false } }
These will block that file from being changed when core is updated or added to the project.
Initial Composer Install
You may wish to require an initial line up of contributed modules. (See
Updates and Maintenance below). If you are not adding modules at first,
you may run composer update nothing
to generate an initial
composer.lock
file. Either way, committing the result will speed
setup for other members of your team.
Set a node version in your custom theme.
The front-end container expects a .node-version
file in the theme directory. See the nodenv documentation.
Prepare for Codeship Pro
We use Codeship Pro to deploy
projects built on this template. A free tier is available. All commits
on the develop
branch will be built and deployed.
Docker Environment Variables
There are two environment variables and one path which need to be set in
codeship-services.yml
found in the root of the project repo:
- front-end : environment : THEME_NAME - The folder/machine name of your custom
theme. This folder should be present in
docroot/themes/custom
. If you setsite_theme_path
insetup/config.yml
then also set THEME_PATH here to the same value. - front-end : working_dir - set this value to the full path in the container to
the theme folder. Usually
/var/www/docroot/themes/custom/yourtheme
- deploy : environment : DEPLOY_TARGET - The url of the git remote to which Codeship will push the build artifact.
Encrypted Environment Variables
Deployment credentials should not be stored in the repo in the clear, but Codeship will decrypt these environment variables on each build. You will need to install the Codeship CLI tool Jet to accomplish these steps.
- Create a new Codeship Pro project.
- Open Project Settings and browse to the General tab.
- Scroll down and download the AES Key.
- Move this key into the project root and rename it
codeship.aes
- Create a new file named
env
(Both this file andcodeship.aes
are set to be ignored by git). - Copy or create a private key that matches the public key installed in
your target git remote to the project root or to a
/keys
directory in the project. - Use the ahoy commands to bring up the local project.
- Use advanced command
ahoy key-prep path/to/private_key
to get your private key in a one-line format and appended to theenv
file. - Execute
ssh user@git_remote_url
if you have not accessed this git remote before to add the remote to your~/.ssh/known_hosts
. - Define the environment variables in
env
copying the appropriate line from~/.ssh/known_hosts
, and the name and email to use when commiting the build like this:SSH_PRIVATE_KEY=one-line-key copied from the terminal GIT_KNOWN_HOST=entire line matching the git remote copied from `~/.ssh/known_hosts` GIT_NAME=Codeship-Deploy GIT_EMAIL=name@example.com
- Encrypt the file using Jet:
jet encrypt env env.encrypted
- Remove or move the key created in step 6 - do not commit the private key!
- Commit
env.encrypted
to the repo.
Advanced ahoy
commands for Tech Leads
There are some additional commands in the ahoy.yml
file marked "Advanced" which do
not appear in response to ahoy --help
These are intended for tech leads that may need
to shell in the docker container for some purpose.
Using a Passphrase protected key in the CI Service
Some projects, such as ecommerce projects, should have all ssh keys protected with a passphrase. This is not particularly an issue for keys used by humans, but the key used by a continuous integration service requires automation. The approach used here assumes that the CI environment is using our containers and is therefore created for each build. This example expects the CI service to be Codeship, which you can adapt if you use a different service.
Additional Environment Variables
- Add the passphrase into the
env
file described above asSSHPASS=key
and re-encrypt the file. - Add an environment variable to
environment
section of thedeploy
service in thecodeship-services.yml
file:
GIT_SSH_COMMAND: SSH_AUTH_SOCK=/root/.ssh/ssh_auth_sock ssh
Manually Connect Once
If the key in question has never been used to authenticate to the remote
git service, ssh-agent will still prompt for the passphrase as discussed
in this StackExchange comment.
Use ssh -i
to connect once to the git service from the command line on
your local using the CI key. Supply the passphrase at the prompt and
the git service will be primed for use by your CI user.
Install the Project
composer install
All docker dependencies and Drupal core dependencies along with Drupal core will be installed.
You should commit all files not excluded by the .gitignore file.
What does the template do?
When installing the given composer.json
some tasks are taken care of:
- Drupal will be installed in the
docroot
-directory. - Autoloader is implemented to use the generated composer autoloader in
vendor/autoload.php
, instead of the one provided by Drupal (docroot/vendor/autoload.php
). - Modules (packages of type
drupal-module
) will be placed indocroot/modules/contrib/
- Theme (packages of type
drupal-theme
) will be placed indocroot/themes/contrib/
- Profiles (packages of type
drupal-profile
) will be placed indocroot/profiles/contrib/
- Creates default writable versions of
settings.php
andservices.yml
. - Creates
docroot/sites/default/files
-directory. - Latest version of drush is installed locally for use
at
vendor/bin/drush
. - Latest version of DrupalConsole is installed locally for use
at
vendor/bin/drupal
. - The local machine is checked for dependencies to run the docker
development setup. Any missing dependencies are installed
via homebrew. The following are required for Mac:
- Ahoy
- VirtualBox
- Docker
- Docker Compose
- pre-commit by Yelp
- Docker Machine NFS
- A docker based http-proxy & DNS service is created such that any
docker container with host name ending in
.test
has traffic routed from the host to the proxy. No editing of /etc/hosts required for new projects.
Updates and Maintenance
Updating Drupal Core
This project will attempt to keep all of your Drupal Core files
up-to-date; the project drupal/core-composer-scaffold is used
to ensure that your scaffold files are updated every time drupal/core
is updated. If you customize any of the "scaffolding" files (commonly
.htaccess
), you may need to merge conflicts if any of your modified
files are updated in a new release of Drupal core. See the drupal-scaffold
section above
to block updates to any of these files.
Follow the steps below to update your core files.
- Run
composer update drupal/core-recommended drupal/core-composer-scaffold drupal/core-dev --with-all-dependencies
to update Drupal Core and its dependencies. - Run
git diff
to determine if any of the scaffolding files have changed. Review the files for any changes and restore any customizations to.htaccess
orrobots.txt
. - Commit everything all together in a single commit, so
docroot
will remain in sync with thecore
when checking out branches or runninggit bisect
. - In the event that there are non-trivial conflicts in step 2, you may
wish to perform these steps on a branch, and use
git merge
to combine the updated core files with your customized files. This facilitates the use of a three-way merge tool such as kdiff3. This setup is not necessary if your changes are simple; keeping all of your modifications at the beginning or end of the file is a good strategy to keep merges easy.
Updating and maintaining composer.json
At the Managing Drupal Projects with Composer BOF at DrupalCon
Baltimore, one of the common pain points was merge conflicts in
composer.json
and composer.lock
. It was the strong consensus of
those gathered that for development teams, there should be one
designated maintainer on the team for these files. New modules, updates
and so forth should be requested from the maintainer, who is generally
the project lead.
With composer require ...
you can download new dependencies, including
Drupal contributed modules to your installation. To install the latest
versions of multiple modules:
composer require drupal/block_visibility_groups drupal/config_split drupal/easy_breadcrumb drupal/focal_point drupal/media_entity_image drupal/media_entity_browser drupal/field_formatter drupal/paragraphs drupal/inline_entity_form drupal/pathauto drupal/page_manager drupal/viewsreference
You also can require bower components:
composer require bower-asset/formstone
The stability minimum is set to stable. You will need to flag specific packages if you need a lower stability.
Local Developement Commands
The docker best practice is to work in the host and send commands to a container when needed. This project uses Ahoy as an abstraction tool to further simplify this flow for developers. Ahoy commands work anywhere at or below the root directory of the project.
ahoy harbor
- Build the harbor for your docks. Run this command once after the first time youcomposer install
a dp-docker project.ahoy cast-off
- Launch the global tools needed for local development. Run this command once after you boot your computer.ahoy launch
- Launch this project site.ahoy dock
- Stops this project site and 'returns to port.'ahoy drush command
- Executes command via drush in the site.ahoy drupal command
- Executes command via drupal console in the site.ahoy gulp command
- Executes command via gulp in the site theme folder.ahoy npm command
- Executes command via npm in the site theme folder.ahoy npm-update
- Runs 'npm install' and 'npm-shrinkwrap' in the site theme folder.ahoy compile
- Compile the site theme assets.ahoy rebuild env
- Sync with a server database and compile front end. Pass an environment argument to use with the drush alias (@shortname.env)
Generate composer.json
from existing project
With using the "Composer Generate" drush extension
you can now generate a basic composer.json
file from an existing
project. Note that the generated composer.json
will differ from this
project's file. We recommend comparing the resulting output with this
project's and editing the composer.json to merge them by hand.
FAQ
Should I commit the contrib modules I download?
Composer recommends no. They provide argumentation against but also workrounds if a project decides to do it anyway.
Should I commit the scaffolding files?
The drupal-scaffold plugin can download the scaffold files (like
index.php, update.php, …) to the docroot/ directory of your project. We
generally commit these. If you have not customized those files you could
choose to not check them into your version control system (e.g. git). If
that is the case for your project it might be convenient to
automatically run the drupal-scaffold plugin after every install or
update of your project. You can achieve that by registering
@drupal-scaffold
as post-install and post-update command in your
composer.json:
{ "scripts": { "drupal-scaffold": "DrupalComposer\\DrupalScaffold\\Plugin::scaffold", "post-install-cmd": [ "@drupal-scaffold", "..." ], "post-update-cmd": [ "@drupal-scaffold", "..." ] } }
How can I apply patches to downloaded modules?
If you need to apply patches (depending on the project being modified, a pull request is often a better solution), you can do so with the composer-patches plugin.
To add a patch to drupal module foobar insert the patches section in the extra section of composer.json:
{ "extra": { "patches": { "drupal/foobar": { "Patch description": "URL to patch" } } } }
Appreciation
We are grateful for the following open source projects that made this project possible!