AWS Elastic Beanstalk manages your complete environment, creates new nodes with load increase and does the heavy lifting of environment management Job to itself. In turn DEVOPS engineer can not execute any custom activity in Beanstalk managed EC2 instance, like editing cronjobs or deploying library etc. All AWS Elastic Beanstalk managed EC2 instance gets created by Beanstalk and software deployment and other changes left to eb Deploy method.
Though it is not a best practice to use single config file for different environment, however complex enterprise scenario throws situations where one need to use source controlled config files.
In the current context, Crontab need to be controlled for different AWS Elastic Beanstack environment of the same application. In AWS elastic beanstalk cron, need to have different cron to be fired in staging environment, dev environment in comparison to Production environment. E.g. You need to load Production Data every night to testing environment and cleanup all actual customer email and phone number. It is a challenge to have environment specific configurations.
($ /var/log/eb-activity.log )
Here is the step by step, easy to follow guide, to deploy conditional crontab to different elastic beanstalk environment using AWS Elastic Beanstack CLI (Command Line Interface).
- Login to AWS console → Services → Compute→ Elastic Beanstack. If you are comfortable with basics Please jump to step-10.
- You will able to see, all your applications and all environments as well.
- Choose the application and go to one of your environment. Click on the environment.
- On the left navigation bar click on “Configuration”
- Go to software section and click on modify as shown below.
- Look for environment specific PARAM values, which specifies whether it is dev, qa, stage, demo or prod etc. The example below uses PARAM1 for the explanation. You can create your variable as you need and name it.
- Go to your source IDE and in your source root you will have folder “.ebextensions” .
- The “.ebextensions” folder should contain all .config file to deployed to your Elastic Beanstalk. Please add crontab.config file to this folder. The .config files are YAML or JSON file to processed by your eb CLI.
- Point to Watch : Many IDE reformat on commit checkbox is checked by default. If you are checking in .config file, Ensure your Reformat option is un-checked in commit dialogue. Or you have setup done json or YAML in your IDE.
Below given the conditional crontab.config
files: "/tmp/cron_jobs_user": mode: "000744" owner: root group: root content: | # Test cron container_commands: 01_update_dev_cron_job: command: 'if [ "$PARAM1" == "dev" ]; then cat .ebextensions/cron_jobs_user_dev.txt > /tmp/cron_jobs_user; else echo "It is Prod";fi' 02_update_prod_cron_job: command: 'if [ "$PARAM1" == "prod" ]; then cat .ebextensions/cron_jobs_user_prod.txt > /tmp/cron_jobs_user; else echo "It is dev";fi' 03_remove_old_cron_jobs_user: command: "crontab -r -u ec2-user || exit 0" 04_cronjobs_user: command: "crontab /tmp/cron_jobs_user -u ec2-user" leader_only: true
Details of Crontab.Config
Save And Deploy
- Save your crontab.config and commit.
- Now fire your > eb deploy . Watch out for any error or warning.
- Go to your linux or Unix EC2 instance and folder /var/log
- Check eb-activity log. $ tail -f eb-activity.log.
- Your echo and other information will be available in eb-activity log.
- Check your crontab is correct or not $ crontab -l .
Effort is done to make this guide to be as clear as possible. Still you face any issue/ any modification need to be done you can drop a note.
ERROR: The configuration file .ebextensions/crontab.config in application version app-xxx346 contains invalid YAML or JSON. YAML exception: Invalid Yaml: mapping values are not allowed here
in “<reader>”, line 2, column 28: Then your crontab.config has JSON/YAML issue due to indentation/ space not maintained properly. You can check online at http://www.yamllint.com/ before hand.
This is definitely not the best way, however is a working solution. We will explore further to have more sophisticated solution. All suggestions are most welcome.
References & Further Reading: