Deployment Process
Understanding build, deploy, and post-deploy phases, the five deployment stages, and how to optimize for minimal downtime.
Deployment Process
Content Organization
Due to the extensive nature of deployment processes, this topic is split into three parts:
- Part A: Build Phase (ece-tools build:generate and build:transfer)
- Part B: Deploy Phase (PreDeploy, InstallUpdate, ConfigUpdate, DeployStaticContent)
- Part C: Post-Deploy Phase, Downtime, Script Extension, and Logs
Deployment Process Overview
Three Main Phases
| Phase | Key Activities | Database Available? | Site Status |
|---|---|---|---|
| Build | composer install, compile code, optional SCD | ✗ NO | Site still UP |
| Deploy | Mount FS, configure services, run migrations, SCD (default) | ✓ YES | Site DOWN (maintenance) |
| Post-Deploy | Enable cron, warm cache, backup configs | ✓ YES | Site UP |
Critical: 60-Second Freeze
All incoming traffic is frozen for 60 seconds at the start of deploy so Cloud can reconfigure routing for the new environment.
Part A: Build Phase
Build Process Overview
The build process is configured in .magento.app.yaml with additional configurations in .magento.env.yaml.
Critical Limitation
Database and services (Redis, Elasticsearch, etc.) are NOT available during the build phase!
Build Hook Configuration
Primary build command in .magento.app.yaml:
hooks:
build: |
php ./vendor/bin/ece-tools build
Important: composer install Runs First
Before the build hook executes, Magento's build system automatically runs composer install.
Build Execution Overview
- composer install (automatic, before hooks/build)
- Run hooks/build commands from .magento.app.yaml
- php ./vendor/bin/ece-tools build must be run
- Run custom commands (optional)
ece-tools build Command
The ece-tools build command is an alias that runs two commands:
Two Sub-Commands
ece-tools build:generate- Generate code and configurationece-tools build:transfer- Prepare slug for deployment
Practical Tip
You can split the build command into two separate commands if you need to inject code between their execution.
ece-tools build:generate
Build:Generate Sub-Commands
This command executes a series of sub-commands. Understanding the order is important, not memorizing method names.
1. PreBuild
- Outputs "Verbosity level is..."
- Cleans
generated/codedirectory - Cleans
generated/metadatadirectory
2. SetProductionMode
- Uses DeployConfigWriter to write to
env.php - Sets application to production mode
3. ValidateConfiguration
Executes multiple validators:
| Validator | Purpose | Failure Impact |
|---|---|---|
| ComposerFile | Ensures Zend\Mvc\Controller\ in autoload/psr-4 | Critical failure |
| StageConfig | Ensures stage section in .magento.env.yaml is accurate | Critical failure |
| BuildOptionsIni | Validates deprecated build_options.ini | Critical failure |
| ConfigFileExists | Checks if app/etc/config.php exists | Warning (proceeds) |
| DeprecatedBuildOptionsIni | Warns about old build_options.ini usage | Warning |
| StageConfigDeprecatedVariables | Warns about deprecated variables (SCD_EXCLUDE_THEMES) | Warning |
| ModulesExist | Ensures 'modules' key in app/etc/config.php | Critical failure |
| AppropriateVersion | Version check for SCD_STRATEGY, SCD_MAX_EXECUTION_TIME | Warning |
| ScdOptionsIgnorance | Warns when SCD_ON_DEMAND/SKIP_SCD ignore other SCD variables | Warning |
| IdealState | Checks for optimal deployment configuration | Info |
Ideal State Conditions
The ideal state is achieved when:
- Both SCD_ON_DEMAND and SKIP_SCD are NOT enabled
- ece-tools post-deploy hook is present in .magento.app.yaml
- SKIP_HTML_MINIFICATION is NOT enabled
Result: Static content deploys during build phase, significantly reducing production downtime!
4. ModuleRefresh
- Enables all modules
5. ApplyPatches
Applies patches and configures filesystem:
- copyStaticFile: Copies
pub/static.phptopub/front-static.php - applyComposerPatches: Applies composer module patches from
patches.json - applyHotFixes: Applies all
.patchfiles fromm2-hotfixesdirectory
Practical Experience
Patches from the m2-hotfixes directory are applied during the ece-tools build command.
6. MarshallFiles
- Deletes
var/cachedirectory - Copies di.xml files (if Magento < 2.2)
7. CopySampleData
- If
vendor/magento/sample-data-mediaexists, copies media topub/media
8. CompileDi
- Runs
bin/magento setup:di:compile - Generates dependency injection configuration
9. ComposerDumpAutoload
- Regenerates Composer's
autoload_classmap.php - Necessary because additional files may have been added since composer install
10. DeployStaticContent
Optionally deploys static content during build phase (IDEAL for production!):
Conditions for Build-Phase SCD
Static content will deploy during build if ALL conditions are met:
- SCD_ON_DEMAND must NOT be enabled
- SKIP_SCD must NOT be enabled
- Websites and stores must be defined in app/etc/config.php
Important: Use Correct Dump Command
Use php vendor/bin/m2-ece-scd-dump to populate scopes in app/etc/config.php.
Don't use: bin/magento app:config:dump as this dumps ALL configuration!
When conditions are met, runs: bin/magento setup:static-content:deploy
ece-tools build:transfer
Prepare Slug for Deployment
After generation completes, the next step prepares the "slug" (artifact/archive) for deployment.
1. CompressStaticContent
Uses these variables:
- SCD_COMPRESSION_LEVEL
- SCD_COMPRESSION_TIMEOUT
- VERBOSE_COMMANDS
Sample compression command:
timeout -k 30 30 bash -c \
find pub/static -type d -name DELETING_* -prune -o -type f -size +300c \
'(' -name '*.js' -or -name '*.css' -or -name '*.svg' \
-or -name '*.html' -or -name '*.htm' ')' -print0 \
| xargs -0 -n100 -P16 gzip -q --keep -4
2. ClearInitDirectory
- Clears contents of
init/directory - Removes
app/etc/env.php
3. BackupData
Copies static content and writable directories to init directory:
StaticContent
If static content deployed during build, pub/static is moved to init/pub/static
WritableDirectories
These directories are backed up (NOT the mounts from app.yaml):
app/etcpub/mediavar/logvar/view_preprocessed
Important Note
During deploy, static content directories are symlinked into the init directory IF SCD_ON_DEMAND is NOT set.
Build Phase Summary
| Step | What Happens |
|---|---|
| 1. composer install | Automatic, before hooks/build |
| 2. build:generate | Clean dirs, validate config, compile DI, optional SCD |
| 3. build:transfer | Compress static, backup to init/ directory |
Part B: Deploy Phase
Deploy Process Overview
The deploy process has two parts:
- Mount the filesystem, configure connections and services
- Execute deploy commands in .magento.app.yaml
Critical: Traffic Frozen
All incoming traffic is frozen for 60 seconds so Cloud can properly reconfigure routing changes introduced by the new environment.
Deploy Hook Configuration
hooks:
deploy: |
php ./vendor/bin/ece-tools deploy
Deploy Phase: PreDeploy
PreDeploy Sub-Commands
1. ConfigUpdate\Cache
Updates cache configuration in app/etc/env.php:
- If CACHE_CONFIGURATION is valid and matches app/etc/env.php, no changes
- If using Redis slave (REDIS_USE_SLAVE_CONNECTION), those details won't update in env.php
- If no Redis relationship exists, nothing happens
- Otherwise, app/etc/env.php is updated with cache (default) and page_cache info
2. CleanStaticContent
- If static content was built OR CLEAN_STATIC_FILES variable is missing, nothing happens
- Otherwise, cleans content from
pub/staticdirectory
CLEAN_STATIC_FILES
Helpful during development but can cause problems in production. Use carefully!
3. CleanViewPreprocessed
- If SKIP_HTML_MINIFICATION is absent (HTML is minified), does nothing
- Otherwise,
var/view_preprocesseddirectory is cleaned
4. CleanRedisCache
- Clears all Redis caches (default and page_cache)
5. CleanFileCache
- Cleans
var/cachedirectory if it exists
6. RestoreWritableDirectories
Copies or symlinks files from init/ directory:
- init/app/etc/* → copied to root-level locations
- init/pub/media → copied to root-level
If static content was built in build phase:
- If HTML is minified, copy
init/var/view_preprocessed - Copy all files in
init/pub/static
Copy Strategy
- CLEAN_STATIC_FILES set (default): Entire directory copied
- CLEAN_STATIC_FILES not set: Only subfolders copied (retains existing files)
7. SetProductionMode
- Enables production mode in
app/etc/env.php
8. DisableCron
- Turns off cron by updating
app/etc/env.php
9. ValidateConfiguration
Runs multiple configuration validators:
| Validator | Purpose | Failure Result |
|---|---|---|
| DatabaseConfiguration | Validates DATABASE_CONFIGURATION with _merge flag | Failed deploy |
| SearchConfiguration | Merges/configures SEARCH_CONFIGURATION | Failed deploy |
| ResourceConfiguration | Maps RESOURCE_CONFIGURATION connections | Failed deploy |
| SessionConfiguration | Configures Redis for SESSION_CONFIGURATION | Failed deploy |
| ElasticSuiteIntegrity | If ElasticSearch enabled, ensure proper install | Failed deploy |
| AdminData | Validates admin environment variables | Warning |
| PhpVersion | Ensures PHP version compatibility | Failed deploy |
| SolrIntegrity | Ensures Solr is NOT configured | Log message only |
| ElasticSearchUsage | Warns if ES installed but not used | Warning |
| ElasticSearchVersion | Checks correct ES version installed | Warning |
| AppropriateVersion | Warns about version-specific variables | Warning |
| ScdOptionsIgnorance | Checks if SCD should build on deploy | Warning |
| DeprecatedVariables | Warns about deprecated vars (VERBOSE_COMMANDS, SCD_EXCLUDE_THEMES) | Warning |
| RawEnvVariable | Ensures SCD_THREADS is numeric | Warning |
| MagentoCloudVariables | Validates env variables are correct type | Warning |
| JsonFormatVariable | Ensures non-array vars can decode as JSON | Warning |
Deploy Phase: Installation or Update
Two Paths: Install vs Update
If Magento is NOT Installed (Install Path)
1. UnlockCronJobs
- If Magento ≥ 2.2.2, automatically sets running cron jobs to error status
- Allows setup upgrade to proceed without cron locks
2. SetCryptKey
- If no encryption key in app/etc/env.php AND CRYPT_KEY env variable is set, saves to env.php
- Should only happen on initial install
Warning: Encryption Key
Changing encryption key after installation renders encrypted data useless (API keys, UPS rates, etc.)!
3. InstallUpdate → Setup
Runs setup:install with these notes:
- Must specify ADMIN_EMAIL environment variable for automated install
- Sessions saved to database by default
- Connection info loaded from configuration merged with env variables
4. ConfigUpdate
After install, runs configuration updates (see Update path below)
5. ConfigImport
- Runs
bin/magento app:config:import - Imports config from app/etc/env.php and app/etc/config.php into database
6. ResetPassword
- Notifies admin of admin URL (doesn't actually reset password!)
- Only runs at install time
If Magento IS Installed (Update Path)
1. ConfigUpdate
Runs multiple configuration updaters:
PrepareConfig
Updates app/etc/env.php with:
- SCD_ON_DEMAND → static_content_on_demand_in_production
- SKIP_HTML_MINIFICATION → force_html_minification
- X_FRAME_CONFIGURATION → x-frame-options
CronConsumersRunner
Updates cron_consumers_runner configuration in app/etc/env.php
DbConnection
If database configured, slaves utilized, and _merge flag set, updates app/etc/env.php with slave database connection info
Amqp
Merges in RabbitMQ details
Session
Configures session storage if Redis enabled (SESSION_CONFIGURATION variable)
SearchEngine
Updates system/default/catalog/search/engine in database (SEARCH_CONFIGURATION). Once set in env.php, can't change in admin.
Urls
If NOT master environment and UPDATE_URLS enabled (default):
- Updates URL values in core_config_data
- Updates URLs in app/etc/env.php
- Uses URLs from ENV_ROUTES environment variable
DocumentRoot
Sets directories/document_root_is_pub to true in app/etc/env.php
Lock
Sets lock provider mount point from MAGENTO_CLOUD_LOCKS_DIR (uses database if not specified)
2. SetAdminUrl
- If admin URL defined as environment variable, configures backend/frontName
3. Setup
- Runs
bin/magento setup:upgrade --keep-generated
Deploy Phase: DeployStaticContent
Static Content Deployment (Default Location)
Building static content in deploy phase is the DEFAULT (but not ideal for production).
When SCD Runs on Deploy
| Condition | Result |
|---|---|
| SCD_ON_DEMAND enabled | Static content cleaned, skipped on deploy |
| SKIP_SCD enabled | SCD skipped |
| SCD in build phase | SCD skipped on deploy |
| CLEAN_STATIC_FILES set | Static content cleared |
| None of above | SCD runs on deploy |
Generate Static Content
- Checks if
pub/staticdirectory is writable - Generates
bin/magento setup:static-content:deploycommands
SCD Matrix Strategy
First command deploys all themes EXCLUDING those in SCD_MATRIX:
- First deploy: Everything EXCEPT matrix themes
- Subsequent commands: Each theme in matrix for each specified language
- Purpose: Speed up build by parallelizing specific themes
Variables Used
- SCD_THREADS: Number of threads
- SCD_STRATEGY: Deployment strategy
- SCD_MAX_EXEC_TIME: Applied to --max-execution-time
CompressStaticContent
- If SCD_ON_DEMAND, SKIP_SCD, or SCD in build phase: no compression
- Otherwise, compresses
pub/staticdirectory
DisableGoogleAnalytics
- If staging/integration branches and ENABLE_GOOGLE_ANALYTICS disabled (default), GA is disabled
Deploy Phase: DeployCompletion
Final Deploy Steps
Only runs if there is NO post_deploy hook in .magento.app.yaml:
EnableCron
- Removes cron/enabled flag from app/etc/env.php
Backup
- Backs up app/etc/env.php and app/etc/config.php
- Appends .bak extension
CleanCache
- Runs
bin/magento cache:flush
Part C: Post-Deploy Phase
Post-Deploy Hook Configuration
# .magento.app.yaml
hooks:
# ...
post_deploy: |
php ./vendor/bin/ece-tools post-deploy
Site Status
The site is UP when post-deploy hooks run! However, pages may load slowly if SCD_ON_DEMAND is enabled.
Post-Deploy Sub-Commands
1. DebugLogging Validation
- If this is master branch and debug logging enabled, logs message encouraging you to disable it
2. EnableCron
- Enables cron in app/etc/env.php
3. Backup
- Copies app/etc/env.php and app/etc/config.php
- Appends .bak extension to backups
4. CleanCache
- Runs
bin/magento cache:flush
5. WarmUp
Asynchronously crawls pages on the website:
- URLs come from
stage/post-deploy/WARM_UP_PAGESin .magento.env.yaml - Multiple URL pattern types can be used
- Pre-loads cache for better user experience
TTFB_TESTED_PAGES Variable
Not in ece-tools, but Magento tests specific pages and logs "time to first byte" during post-deploy.
Understanding Downtime
Why Downtime Occurs
Downtime is a result of the deploy process taking longer than it should.
Two Sources of Downtime
- 60-second freeze: All incoming traffic is ALWAYS frozen for 60 seconds at start of deploy
- Maintenance mode: Site is in maintenance during deploy phase execution
How to Reduce Downtime
Best Practices
- Move SCD to build phase: Significantly reduces deploy time
- Use SCD_MATRIX: Parallelize static content generation
- Optimize database migrations: Keep them fast and efficient
- Use SCD_ON_DEMAND for development: Skip SCD entirely (not for production!)
- Enable post-deploy hook: Moves non-critical tasks after site is up
Five Phases of Deployment
Complete Deployment Process
| Phase | Description | Site Status |
|---|---|---|
| 1. Validation | Magento git server checks for valid configuration (not immediate if using GitLab/GitHub) | Site UP |
| 2. Build | composer install, compile code, optional SCD. No database/services available. Ideally SCD happens here. | Site UP |
| 3. Prepare Slug | Filesystem prepared (read-only except mounts). If SCD in build, present in init/pub/static. STATIC_CONTENT_SYMLINK creates symlinks. | Site UP |
| 4. Deploy Slugs & Cluster | Network "down", environment configured with new code. 60-second traffic freeze. | Site DOWN |
| 5. Deployment Hooks | Part 1: deploy hooks (site DOWN). Part 2: post_deploy hooks (site UP but slow if SCD_ON_DEMAND) | Deploy: DOWN, Post: UP |
Role of Each Process/Phase
What Each Phase Does
Validation Phase
- Validates .magento.app.yaml, .magento/services.yaml, .magento/routes.yaml
- Ensures configuration files are syntactically correct
Build Phase
- Prepares code for deployment
- Compiles dependency injection
- Optionally generates static content (IDEAL!)
- No downtime impact
Prepare Slug Phase
- Creates deployment artifact
- Prepares read-only filesystem
- Backs up writable directories to init/
Deploy Slugs & Cluster Phase
- Reconfigures network routing
- 60-second traffic freeze (unavoidable)
- Prepares environment for new code
Deployment Hooks Phase
- deploy: Run migrations, update config, optionally SCD (CAUSES DOWNTIME!)
- post_deploy: Enable cron, warm cache, backup configs (NO DOWNTIME)
How to Extend Deployment Scripts
Three Methods to Extend ece-tools
Method 1: composer.json psr-4
Use psr-4 section in composer.json to override single classes:
{
"autoload": {
"psr-4": {
"Magento\\MagentoCloud\\": "cloud-patches/src/"
}
}
}
Best for: Modifying specific ece-tools classes
Method 2: Git Patches in m2-hotfixes
Create git patches and place in m2-hotfixes/ directory:
- Patches automatically applied during build phase
- Works for any composer package
Best for: Patching ece-tools or other vendor packages
Method 3: Custom Commands in Hooks
Add custom commands to hooks in .magento.app.yaml:
hooks:
build: |
php ./vendor/bin/ece-tools build:generate
# Your custom build commands here
php ./custom/build-script.php
php ./vendor/bin/ece-tools build:transfer
deploy: |
php ./vendor/bin/ece-tools deploy
# Your custom deploy commands here
post_deploy: |
php ./vendor/bin/ece-tools post-deploy
# Your custom post-deploy commands here
Important Limitations
- Build phase: Database NOT available
- Before hooks/build: composer install already ran
- Deploy phase: Site is DOWN during execution
Best for: Adding functionality without modifying ece-tools
How to Retrieve Logs
Log Locations
| Log Type | Location | Access Method |
|---|---|---|
| Deploy Log | /var/log/deploy.log |
SSH, magento-cloud CLI |
| Build Log | Environment message list | Cloud Admin UI, magento-cloud CLI |
| Post-Deploy Log | Environment message list | Cloud Admin UI, magento-cloud CLI |
| All Phases Log | /app/var/log/cloud.log |
SSH, magento-cloud CLI |
Accessing Logs via CLI
# View deploy log
magento-cloud log deploy
# View all logs
magento-cloud log
# SSH and view logs directly
magento-cloud ssh
cat /var/log/deploy.log
cat /app/var/log/cloud.log
Accessing Logs via UI
- Navigate to your environment in Cloud Admin
- Click on deployment activity
- View build, deploy, and post-deploy logs in separate tabs
Best Practices
Do's
- Move SCD to build phase for production
- Use SCD_MATRIX for theme parallelization
- Use post-deploy hook for non-critical tasks
- Monitor /var/log/deploy.log for issues
- Use SCD_ON_DEMAND for development only
- Optimize database migrations
- Use m2-hotfixes for patches
- Test custom hooks in Integration first
Don'ts
- Don't use SCD_ON_DEMAND in production
- Don't run SCD in deploy phase for production
- Don't access database in build phase
- Don't use CLEAN_STATIC_FILES carelessly
- Don't forget 60-second freeze always occurs
- Don't skip post-deploy hook
- Don't use deprecated variables
- Don't put long-running tasks in deploy hook
Deployment Optimization Checklist
Ideal Production Configuration
- ✓ SCD_ON_DEMAND = NOT enabled
- ✓ SKIP_SCD = NOT enabled
- ✓ Websites/stores defined in app/etc/config.php
- ✓ SCD runs during build phase
- ✓ post_deploy hook configured
- ✓ SKIP_HTML_MINIFICATION = NOT enabled
- ✓ SCD_MATRIX configured for parallel theme generation
- ✓ Database migrations optimized
Result
Minimum downtime: Only 60-second freeze + time for database migrations and configuration updates!
Exam Tips
Key Points to Remember
- Three phases: Build, Deploy, Post-Deploy
- Build phase: No database/services, composer install first, ece-tools build (generate + transfer)
- Deploy phase: 60-second freeze, site in maintenance, database available
- Post-deploy phase: Site is UP, enable cron, cache warm-up
- composer install: Runs BEFORE hooks/build
- SCD ideal location: Build phase (reduces downtime)
- SCD default location: Deploy phase
- SCD_ON_DEMAND: Skip SCD, generate on first request
- Patches applied: During build phase (m2-hotfixes directory)
- Five total phases: Validation, Build, Prepare slug, Deploy cluster, Deployment hooks
- Deploy logs: /var/log/deploy.log
- All phases log: /app/var/log/cloud.log
- Downtime cause: Deploy phase taking too long
- Reduce downtime: Move SCD to build phase, optimize processes
- Extend scripts: composer.json psr-4, git patches in m2-hotfixes, custom commands in hooks