Cloud 5.01

Deployment Process

Understanding build, deploy, and post-deploy phases, the five deployment stages, and how to optimize for minimal downtime.

Exam Critical: Understanding the deployment process (build, deploy, post-deploy phases) is ESSENTIAL for the AD0-E717 exam! Know what happens in each phase, when services are available, and how to optimize deployment.

Deployment Process

mindmap root((Deployment)) Build Phase composer install No database access ece-tools build generate ece-tools build transfer SCD optional here Deploy Phase 60 sec freeze Mount filesystem Services available Config updates SCD default here Post Deploy Phase Site is UP Enable cron Cache warm up Backup configs Five Phases Validation Build Prepare slug Deploy cluster Deployment hooks

Content Organization

Due to the extensive nature of deployment processes, this topic is split into three parts:

  • Part A: Build Phase (ece-tools build:generate and build:transfer)
  • Part B: Deploy Phase (PreDeploy, InstallUpdate, ConfigUpdate, DeployStaticContent)
  • Part C: Post-Deploy Phase, Downtime, Script Extension, and Logs

Deployment Process Overview

Three Main Phases

Phase Key Activities Database Available? Site Status
Build composer install, compile code, optional SCD ✗ NO Site still UP
Deploy Mount FS, configure services, run migrations, SCD (default) ✓ YES Site DOWN (maintenance)
Post-Deploy Enable cron, warm cache, backup configs ✓ YES Site UP

Critical: 60-Second Freeze

All incoming traffic is frozen for 60 seconds at the start of deploy so Cloud can reconfigure routing for the new environment.

Part A: Build Phase

Build Process Overview

The build process is configured in .magento.app.yaml with additional configurations in .magento.env.yaml.

Critical Limitation

Database and services (Redis, Elasticsearch, etc.) are NOT available during the build phase!

Build Hook Configuration

Primary build command in .magento.app.yaml:

hooks:
    build: |
        php ./vendor/bin/ece-tools build

Important: composer install Runs First

Before the build hook executes, Magento's build system automatically runs composer install.

Build Execution Overview

  1. composer install (automatic, before hooks/build)
  2. Run hooks/build commands from .magento.app.yaml
  3. php ./vendor/bin/ece-tools build must be run
  4. Run custom commands (optional)

ece-tools build Command

The ece-tools build command is an alias that runs two commands:

Two Sub-Commands

  • ece-tools build:generate - Generate code and configuration
  • ece-tools build:transfer - Prepare slug for deployment

Practical Tip

You can split the build command into two separate commands if you need to inject code between their execution.

ece-tools build:generate

Build:Generate Sub-Commands

This command executes a series of sub-commands. Understanding the order is important, not memorizing method names.

1. PreBuild

  • Outputs "Verbosity level is..."
  • Cleans generated/code directory
  • Cleans generated/metadata directory

2. SetProductionMode

  • Uses DeployConfigWriter to write to env.php
  • Sets application to production mode

3. ValidateConfiguration

Executes multiple validators:

Validator Purpose Failure Impact
ComposerFile Ensures Zend\Mvc\Controller\ in autoload/psr-4 Critical failure
StageConfig Ensures stage section in .magento.env.yaml is accurate Critical failure
BuildOptionsIni Validates deprecated build_options.ini Critical failure
ConfigFileExists Checks if app/etc/config.php exists Warning (proceeds)
DeprecatedBuildOptionsIni Warns about old build_options.ini usage Warning
StageConfigDeprecatedVariables Warns about deprecated variables (SCD_EXCLUDE_THEMES) Warning
ModulesExist Ensures 'modules' key in app/etc/config.php Critical failure
AppropriateVersion Version check for SCD_STRATEGY, SCD_MAX_EXECUTION_TIME Warning
ScdOptionsIgnorance Warns when SCD_ON_DEMAND/SKIP_SCD ignore other SCD variables Warning
IdealState Checks for optimal deployment configuration Info

Ideal State Conditions

The ideal state is achieved when:

  1. Both SCD_ON_DEMAND and SKIP_SCD are NOT enabled
  2. ece-tools post-deploy hook is present in .magento.app.yaml
  3. SKIP_HTML_MINIFICATION is NOT enabled

Result: Static content deploys during build phase, significantly reducing production downtime!

4. ModuleRefresh

  • Enables all modules

5. ApplyPatches

Applies patches and configures filesystem:

  • copyStaticFile: Copies pub/static.php to pub/front-static.php
  • applyComposerPatches: Applies composer module patches from patches.json
  • applyHotFixes: Applies all .patch files from m2-hotfixes directory

Practical Experience

Patches from the m2-hotfixes directory are applied during the ece-tools build command.

6. MarshallFiles

  • Deletes var/cache directory
  • Copies di.xml files (if Magento < 2.2)

7. CopySampleData

  • If vendor/magento/sample-data-media exists, copies media to pub/media

8. CompileDi

  • Runs bin/magento setup:di:compile
  • Generates dependency injection configuration

9. ComposerDumpAutoload

  • Regenerates Composer's autoload_classmap.php
  • Necessary because additional files may have been added since composer install

10. DeployStaticContent

Optionally deploys static content during build phase (IDEAL for production!):

Conditions for Build-Phase SCD

Static content will deploy during build if ALL conditions are met:

  • SCD_ON_DEMAND must NOT be enabled
  • SKIP_SCD must NOT be enabled
  • Websites and stores must be defined in app/etc/config.php

Important: Use Correct Dump Command

Use php vendor/bin/m2-ece-scd-dump to populate scopes in app/etc/config.php.

Don't use: bin/magento app:config:dump as this dumps ALL configuration!

When conditions are met, runs: bin/magento setup:static-content:deploy

ece-tools build:transfer

Prepare Slug for Deployment

After generation completes, the next step prepares the "slug" (artifact/archive) for deployment.

1. CompressStaticContent

Uses these variables:

  • SCD_COMPRESSION_LEVEL
  • SCD_COMPRESSION_TIMEOUT
  • VERBOSE_COMMANDS

Sample compression command:

timeout -k 30 30 bash -c \
find pub/static -type d -name DELETING_* -prune -o -type f -size +300c \
'(' -name '*.js' -or -name '*.css' -or -name '*.svg' \
    -or -name '*.html' -or -name '*.htm' ')' -print0 \
| xargs -0 -n100 -P16 gzip -q --keep -4

2. ClearInitDirectory

  • Clears contents of init/ directory
  • Removes app/etc/env.php

3. BackupData

Copies static content and writable directories to init directory:

StaticContent

If static content deployed during build, pub/static is moved to init/pub/static

WritableDirectories

These directories are backed up (NOT the mounts from app.yaml):

  • app/etc
  • pub/media
  • var/log
  • var/view_preprocessed

Important Note

During deploy, static content directories are symlinked into the init directory IF SCD_ON_DEMAND is NOT set.

Build Phase Summary

Step What Happens
1. composer install Automatic, before hooks/build
2. build:generate Clean dirs, validate config, compile DI, optional SCD
3. build:transfer Compress static, backup to init/ directory

Part B: Deploy Phase

Deploy Process Overview

The deploy process has two parts:

  1. Mount the filesystem, configure connections and services
  2. Execute deploy commands in .magento.app.yaml

Critical: Traffic Frozen

All incoming traffic is frozen for 60 seconds so Cloud can properly reconfigure routing changes introduced by the new environment.

Deploy Hook Configuration

hooks:
    deploy: |
        php ./vendor/bin/ece-tools deploy

Deploy Phase: PreDeploy

PreDeploy Sub-Commands

1. ConfigUpdate\Cache

Updates cache configuration in app/etc/env.php:

  • If CACHE_CONFIGURATION is valid and matches app/etc/env.php, no changes
  • If using Redis slave (REDIS_USE_SLAVE_CONNECTION), those details won't update in env.php
  • If no Redis relationship exists, nothing happens
  • Otherwise, app/etc/env.php is updated with cache (default) and page_cache info

2. CleanStaticContent

  • If static content was built OR CLEAN_STATIC_FILES variable is missing, nothing happens
  • Otherwise, cleans content from pub/static directory
CLEAN_STATIC_FILES

Helpful during development but can cause problems in production. Use carefully!

3. CleanViewPreprocessed

  • If SKIP_HTML_MINIFICATION is absent (HTML is minified), does nothing
  • Otherwise, var/view_preprocessed directory is cleaned

4. CleanRedisCache

  • Clears all Redis caches (default and page_cache)

5. CleanFileCache

  • Cleans var/cache directory if it exists

6. RestoreWritableDirectories

Copies or symlinks files from init/ directory:

  • init/app/etc/* → copied to root-level locations
  • init/pub/media → copied to root-level

If static content was built in build phase:

  • If HTML is minified, copy init/var/view_preprocessed
  • Copy all files in init/pub/static
Copy Strategy
  • CLEAN_STATIC_FILES set (default): Entire directory copied
  • CLEAN_STATIC_FILES not set: Only subfolders copied (retains existing files)

7. SetProductionMode

  • Enables production mode in app/etc/env.php

8. DisableCron

  • Turns off cron by updating app/etc/env.php

9. ValidateConfiguration

Runs multiple configuration validators:

Validator Purpose Failure Result
DatabaseConfiguration Validates DATABASE_CONFIGURATION with _merge flag Failed deploy
SearchConfiguration Merges/configures SEARCH_CONFIGURATION Failed deploy
ResourceConfiguration Maps RESOURCE_CONFIGURATION connections Failed deploy
SessionConfiguration Configures Redis for SESSION_CONFIGURATION Failed deploy
ElasticSuiteIntegrity If ElasticSearch enabled, ensure proper install Failed deploy
AdminData Validates admin environment variables Warning
PhpVersion Ensures PHP version compatibility Failed deploy
SolrIntegrity Ensures Solr is NOT configured Log message only
ElasticSearchUsage Warns if ES installed but not used Warning
ElasticSearchVersion Checks correct ES version installed Warning
AppropriateVersion Warns about version-specific variables Warning
ScdOptionsIgnorance Checks if SCD should build on deploy Warning
DeprecatedVariables Warns about deprecated vars (VERBOSE_COMMANDS, SCD_EXCLUDE_THEMES) Warning
RawEnvVariable Ensures SCD_THREADS is numeric Warning
MagentoCloudVariables Validates env variables are correct type Warning
JsonFormatVariable Ensures non-array vars can decode as JSON Warning

Deploy Phase: Installation or Update

Two Paths: Install vs Update

If Magento is NOT Installed (Install Path)

1. UnlockCronJobs
  • If Magento ≥ 2.2.2, automatically sets running cron jobs to error status
  • Allows setup upgrade to proceed without cron locks
2. SetCryptKey
  • If no encryption key in app/etc/env.php AND CRYPT_KEY env variable is set, saves to env.php
  • Should only happen on initial install
Warning: Encryption Key

Changing encryption key after installation renders encrypted data useless (API keys, UPS rates, etc.)!

3. InstallUpdate → Setup

Runs setup:install with these notes:

  • Must specify ADMIN_EMAIL environment variable for automated install
  • Sessions saved to database by default
  • Connection info loaded from configuration merged with env variables
4. ConfigUpdate

After install, runs configuration updates (see Update path below)

5. ConfigImport
  • Runs bin/magento app:config:import
  • Imports config from app/etc/env.php and app/etc/config.php into database
6. ResetPassword
  • Notifies admin of admin URL (doesn't actually reset password!)
  • Only runs at install time

If Magento IS Installed (Update Path)

1. ConfigUpdate

Runs multiple configuration updaters:

PrepareConfig

Updates app/etc/env.php with:

  • SCD_ON_DEMAND → static_content_on_demand_in_production
  • SKIP_HTML_MINIFICATION → force_html_minification
  • X_FRAME_CONFIGURATION → x-frame-options
CronConsumersRunner

Updates cron_consumers_runner configuration in app/etc/env.php

DbConnection

If database configured, slaves utilized, and _merge flag set, updates app/etc/env.php with slave database connection info

Amqp

Merges in RabbitMQ details

Session

Configures session storage if Redis enabled (SESSION_CONFIGURATION variable)

SearchEngine

Updates system/default/catalog/search/engine in database (SEARCH_CONFIGURATION). Once set in env.php, can't change in admin.

Urls

If NOT master environment and UPDATE_URLS enabled (default):

  • Updates URL values in core_config_data
  • Updates URLs in app/etc/env.php
  • Uses URLs from ENV_ROUTES environment variable
DocumentRoot

Sets directories/document_root_is_pub to true in app/etc/env.php

Lock

Sets lock provider mount point from MAGENTO_CLOUD_LOCKS_DIR (uses database if not specified)

2. SetAdminUrl
  • If admin URL defined as environment variable, configures backend/frontName
3. Setup
  • Runs bin/magento setup:upgrade --keep-generated

Deploy Phase: DeployStaticContent

Static Content Deployment (Default Location)

Building static content in deploy phase is the DEFAULT (but not ideal for production).

When SCD Runs on Deploy

Condition Result
SCD_ON_DEMAND enabled Static content cleaned, skipped on deploy
SKIP_SCD enabled SCD skipped
SCD in build phase SCD skipped on deploy
CLEAN_STATIC_FILES set Static content cleared
None of above SCD runs on deploy

Generate Static Content

  • Checks if pub/static directory is writable
  • Generates bin/magento setup:static-content:deploy commands
SCD Matrix Strategy

First command deploys all themes EXCLUDING those in SCD_MATRIX:

  • First deploy: Everything EXCEPT matrix themes
  • Subsequent commands: Each theme in matrix for each specified language
  • Purpose: Speed up build by parallelizing specific themes

Variables Used

  • SCD_THREADS: Number of threads
  • SCD_STRATEGY: Deployment strategy
  • SCD_MAX_EXEC_TIME: Applied to --max-execution-time

CompressStaticContent

  • If SCD_ON_DEMAND, SKIP_SCD, or SCD in build phase: no compression
  • Otherwise, compresses pub/static directory

DisableGoogleAnalytics

  • If staging/integration branches and ENABLE_GOOGLE_ANALYTICS disabled (default), GA is disabled

Deploy Phase: DeployCompletion

Final Deploy Steps

Only runs if there is NO post_deploy hook in .magento.app.yaml:

EnableCron

  • Removes cron/enabled flag from app/etc/env.php

Backup

  • Backs up app/etc/env.php and app/etc/config.php
  • Appends .bak extension

CleanCache

  • Runs bin/magento cache:flush

Part C: Post-Deploy Phase

Post-Deploy Hook Configuration

# .magento.app.yaml
hooks:
    # ...
    post_deploy: |
        php ./vendor/bin/ece-tools post-deploy

Site Status

The site is UP when post-deploy hooks run! However, pages may load slowly if SCD_ON_DEMAND is enabled.

Post-Deploy Sub-Commands

1. DebugLogging Validation

  • If this is master branch and debug logging enabled, logs message encouraging you to disable it

2. EnableCron

  • Enables cron in app/etc/env.php

3. Backup

  • Copies app/etc/env.php and app/etc/config.php
  • Appends .bak extension to backups

4. CleanCache

  • Runs bin/magento cache:flush

5. WarmUp

Asynchronously crawls pages on the website:

  • URLs come from stage/post-deploy/WARM_UP_PAGES in .magento.env.yaml
  • Multiple URL pattern types can be used
  • Pre-loads cache for better user experience
TTFB_TESTED_PAGES Variable

Not in ece-tools, but Magento tests specific pages and logs "time to first byte" during post-deploy.

Understanding Downtime

Why Downtime Occurs

Downtime is a result of the deploy process taking longer than it should.

Two Sources of Downtime

  1. 60-second freeze: All incoming traffic is ALWAYS frozen for 60 seconds at start of deploy
  2. Maintenance mode: Site is in maintenance during deploy phase execution

How to Reduce Downtime

Best Practices

  • Move SCD to build phase: Significantly reduces deploy time
  • Use SCD_MATRIX: Parallelize static content generation
  • Optimize database migrations: Keep them fast and efficient
  • Use SCD_ON_DEMAND for development: Skip SCD entirely (not for production!)
  • Enable post-deploy hook: Moves non-critical tasks after site is up

Five Phases of Deployment

Complete Deployment Process

Phase Description Site Status
1. Validation Magento git server checks for valid configuration (not immediate if using GitLab/GitHub) Site UP
2. Build composer install, compile code, optional SCD. No database/services available. Ideally SCD happens here. Site UP
3. Prepare Slug Filesystem prepared (read-only except mounts). If SCD in build, present in init/pub/static. STATIC_CONTENT_SYMLINK creates symlinks. Site UP
4. Deploy Slugs & Cluster Network "down", environment configured with new code. 60-second traffic freeze. Site DOWN
5. Deployment Hooks Part 1: deploy hooks (site DOWN). Part 2: post_deploy hooks (site UP but slow if SCD_ON_DEMAND) Deploy: DOWN, Post: UP

Role of Each Process/Phase

What Each Phase Does

Validation Phase

  • Validates .magento.app.yaml, .magento/services.yaml, .magento/routes.yaml
  • Ensures configuration files are syntactically correct

Build Phase

  • Prepares code for deployment
  • Compiles dependency injection
  • Optionally generates static content (IDEAL!)
  • No downtime impact

Prepare Slug Phase

  • Creates deployment artifact
  • Prepares read-only filesystem
  • Backs up writable directories to init/

Deploy Slugs & Cluster Phase

  • Reconfigures network routing
  • 60-second traffic freeze (unavoidable)
  • Prepares environment for new code

Deployment Hooks Phase

  • deploy: Run migrations, update config, optionally SCD (CAUSES DOWNTIME!)
  • post_deploy: Enable cron, warm cache, backup configs (NO DOWNTIME)

How to Extend Deployment Scripts

Three Methods to Extend ece-tools

Method 1: composer.json psr-4

Use psr-4 section in composer.json to override single classes:

{
    "autoload": {
        "psr-4": {
            "Magento\\MagentoCloud\\": "cloud-patches/src/"
        }
    }
}

Best for: Modifying specific ece-tools classes

Method 2: Git Patches in m2-hotfixes

Create git patches and place in m2-hotfixes/ directory:

  • Patches automatically applied during build phase
  • Works for any composer package

Best for: Patching ece-tools or other vendor packages

Method 3: Custom Commands in Hooks

Add custom commands to hooks in .magento.app.yaml:

hooks:
    build: |
        php ./vendor/bin/ece-tools build:generate
        # Your custom build commands here
        php ./custom/build-script.php
        php ./vendor/bin/ece-tools build:transfer
    deploy: |
        php ./vendor/bin/ece-tools deploy
        # Your custom deploy commands here
    post_deploy: |
        php ./vendor/bin/ece-tools post-deploy
        # Your custom post-deploy commands here
Important Limitations
  • Build phase: Database NOT available
  • Before hooks/build: composer install already ran
  • Deploy phase: Site is DOWN during execution

Best for: Adding functionality without modifying ece-tools

How to Retrieve Logs

Log Locations

Log Type Location Access Method
Deploy Log /var/log/deploy.log SSH, magento-cloud CLI
Build Log Environment message list Cloud Admin UI, magento-cloud CLI
Post-Deploy Log Environment message list Cloud Admin UI, magento-cloud CLI
All Phases Log /app/var/log/cloud.log SSH, magento-cloud CLI

Accessing Logs via CLI

# View deploy log
magento-cloud log deploy

# View all logs
magento-cloud log

# SSH and view logs directly
magento-cloud ssh
cat /var/log/deploy.log
cat /app/var/log/cloud.log

Accessing Logs via UI

  • Navigate to your environment in Cloud Admin
  • Click on deployment activity
  • View build, deploy, and post-deploy logs in separate tabs

Best Practices

Do's
  • Move SCD to build phase for production
  • Use SCD_MATRIX for theme parallelization
  • Use post-deploy hook for non-critical tasks
  • Monitor /var/log/deploy.log for issues
  • Use SCD_ON_DEMAND for development only
  • Optimize database migrations
  • Use m2-hotfixes for patches
  • Test custom hooks in Integration first
Don'ts
  • Don't use SCD_ON_DEMAND in production
  • Don't run SCD in deploy phase for production
  • Don't access database in build phase
  • Don't use CLEAN_STATIC_FILES carelessly
  • Don't forget 60-second freeze always occurs
  • Don't skip post-deploy hook
  • Don't use deprecated variables
  • Don't put long-running tasks in deploy hook

Deployment Optimization Checklist

Ideal Production Configuration

  1. ✓ SCD_ON_DEMAND = NOT enabled
  2. ✓ SKIP_SCD = NOT enabled
  3. ✓ Websites/stores defined in app/etc/config.php
  4. ✓ SCD runs during build phase
  5. ✓ post_deploy hook configured
  6. ✓ SKIP_HTML_MINIFICATION = NOT enabled
  7. ✓ SCD_MATRIX configured for parallel theme generation
  8. ✓ Database migrations optimized

Result

Minimum downtime: Only 60-second freeze + time for database migrations and configuration updates!

Exam Tips

Key Points to Remember

  • Three phases: Build, Deploy, Post-Deploy
  • Build phase: No database/services, composer install first, ece-tools build (generate + transfer)
  • Deploy phase: 60-second freeze, site in maintenance, database available
  • Post-deploy phase: Site is UP, enable cron, cache warm-up
  • composer install: Runs BEFORE hooks/build
  • SCD ideal location: Build phase (reduces downtime)
  • SCD default location: Deploy phase
  • SCD_ON_DEMAND: Skip SCD, generate on first request
  • Patches applied: During build phase (m2-hotfixes directory)
  • Five total phases: Validation, Build, Prepare slug, Deploy cluster, Deployment hooks
  • Deploy logs: /var/log/deploy.log
  • All phases log: /app/var/log/cloud.log
  • Downtime cause: Deploy phase taking too long
  • Reduce downtime: Move SCD to build phase, optimize processes
  • Extend scripts: composer.json psr-4, git patches in m2-hotfixes, custom commands in hooks