How to Automate Docker Image Creation for Postgres Using Shell Commands

<p>If you have certain lines of code that repeat over and over again in a project, there's a pretty standard solution for it: using a function.</p> <p>But what about when you have the same project file setup that gets repeated over and over? How do you solve that?</p> <p>Meet bash scripting. Bash scripts are written to execute certain sequences of commands on your machine automatically without you having to do anything. This is known as workflow automation, and that's what I'll be teaching you how to do in this article.</p> <h2 id="heading-table-of-contents">Table Of Contents</h2> <ul> <li><a href="#heading-prerequisites">Prerequisites</a></li> <li><a href="#heading-what-motivated-this-project">What Motivated This Project?</a></li> <li><a href="#heading-introduction">Introduction</a></li> <li><a href="#heading-conditions-in-bash">Conditions in bash</a></li> <li><a href="#heading-interactive-shell">Interactive Shell</a></li> <li><a href="#heading-how-to-create-and-write-to-a-file">How to Create and Write to a File</a></li> <li><a href="#heading-bash-arguments-and-flags">Bash Arguments and Flags</a></li> <li><a href="#heading-how-to-make-your-scripts-executable-and-available-on-your-pcs-path">How to Make Your Scripts Executable and Available on Your PC's Path</a></li> <li><a href="#heading-conclusion">Conclusion</a></li> </ul> <h2 id="heading-prerequisites">Prerequisites</h2> <p>To follow along with this article, you should have:</p> <ul> <li>Git-bash installed on your machine</li> <li>Some experience using a terminal or command prompt to run and execute commands (helpful but not required)</li> <li>Understanding of basic programming concepts (like conditions and loops)</li> <li>Docker installed on your machine</li> <li>Basic understanding of Docker concepts (like images and containers)</li> <li>Experience using Postgres (optional but recommended)</li> </ul> <h2 id="heading-what-motivated-this-project">What Motivated This Project?</h2> <p>I was recently setting up a Docker image for Postgres on a few projects I was working on, and I noticed that I was following the same pattern in creating the images. The only difference was that they had different database credentials and database names.</p> <p>I also realized that a large portion of today's rapidly developed apps tend to use Supabase as their backend, which can come with resource limits such as database storage, bandwidth, concurrent connections, and API request quotas (depending on the service tier and size of the project). And these issues can be solved by setting up your database.</p> <p>So I thought, why not make a CLI tool that could automate creating Docker images for Postgres? That's when I learned about writing executable shell commands using bash to create workflow automations.</p> <p>You can find the full working example used in this guide in this <a href="https://github.com/jabo-arnold-landry/Postgres-docker-image-generator">git-repo</a>.</p> <h2 id="heading-introduction">Introduction</h2> <p>Bash is a shell and scripting language used to write and execute commands or scripts that perform operations on a computer, such as creating directories and files, manipulating files, automating tasks, and managing system processes.</p> <p>In this guide, you'll learn to automate the process of creating a Docker image for a Postgres database so that you don't have to create the image from scratch for each new project.</p> <h3 id="heading-bash-scripting-file-setup">Bash Scripting File Setup</h3> <p>To get started, open you preferred text-editor and create a directory or folder there. Then within that directory, create a file with the extension <code>.sh</code>.</p> <p>At the beginning of a file, type:</p> <pre><code class="language-shell">#!/bin/bash </code></pre> <p><code>#!/bin/bash</code> (known as a "shebang") tells the operating system to run the script using the Bash shell, so the commands inside the file are executed as Bash/terminal commands.</p> <h2 id="heading-conditions-in-bash">Conditions in bash</h2> <p>After you've finished creating a script file and adding shebang, let's start by exploring how if statements are used in bash.</p> <p>The syntax of <code>if</code> looks like this:</p> <pre><code class="language-shell">if [[ condition ]]; then command to execute fi </code></pre> <p>The <code>if</code> in bash is no different from what you're used to in other programming languages:</p> <ul> <li>You use double square braces ([]) to add a condition you're checking against and a semicolon (;).</li> <li>You use <code>then</code> after the <code>if</code> declaration to specify what should be executed if the condition is true.</li> <li>You use the <code>fi</code> to close the condition statement.</li> </ul> <p>When you're writing the condition inside the square braces, make sure you leave a space before the condition and another one after it (as you can see in the code above). If you omit these spaces, you'll get errors.</p> <h3 id="heading-working-example">Working Example:</h3> <p>Since we are automating Postgres database Docker image creation, first check if the file exists and has content. If it does have content, end the script. If not, we'll proceed with the next script.</p> <pre><code class="language-shell">#!/bin/bash if [[ -s docker-compose.yml ]]; then echo "The file already exists" exit 0 fi echo "Next command" </code></pre> <p>In the if condition, we use the flag <code>-s</code> to check if the file exists and has contents. File flags check file availability and permissions, and include but aren't limited to:</p> <ul> <li>-f: used to check if the file exists</li> <li>-s: used to check if the file exists and has contents</li> <li>-d: used to check if a directory exists</li> <li>-r: used to check if a file is readable</li> <li>-w: used to check if a file is writable</li> <li>-x: used to check if a file is executable</li> </ul> <p>In the script, we're using <code>-s</code> to check if a file named <code>docker-compose.yml</code> exists in the directory where we're executing the command. If it's false, we use the <code>echo</code> keyword to print the text "Next command". If it's true, we print the text "The file already exists".</p> <p><code>exit</code> acts as a return keyword for bash scripting, 0 is used for success, and 1 is used for failure.</p> <p>Let's test all this by creating a Docker Compose file and adding some content in the same directory where we're writing the bash script. In your terminal type <code>bash name-of-file.sh</code>, and you should see "The file already exists":</p> <img src="https://cdn.hashnode.com/uploads/covers/69c7bcff7cf27065100ae8be/14d2b684-5e37-4e70-b6be-05386777fd68.png" alt="output after creating the already existing file using bash" style="display: block;" width="600" height="400" loading="lazy"> <p>But if we delete the content in the file and run the command again, we should see the output saying "Next command":</p> <img src="https://cdn.hashnode.com/uploads/covers/69c7bcff7cf27065100ae8be/130698ee-7309-4f3e-9eb0-37632b954ded.png" alt="output of a successful file creation in bash" style="display: block;" width="600" height="400" loading="lazy"> <h2 id="heading-interactive-shell">Interactive Shell</h2> <p>To make an automation dynamic and flexible you will need to receive inputs, take an example of a database name you won't always have the same database name, the name will change based on the project,</p> <p>you do need to setup away to use a given name of the database when creating a new docker image for that you read inputs using the <code>read</code> keyword.</p> <p>There are different types of inputs based on the operation. And there are flags that are used to specify which type of input you're expecting, such as:</p> <ul> <li>-p: for prompting the user to write a text</li> <li>-s: used on passwords when you want the user input to stay hidden when writing it in the terminal</li> <li>-a: used when you're expecting the user to pass in multiple inputs or if you're expecting an array from user.</li> </ul> <p>Here's the syntax for reading user input:</p> <pre><code class="language-shell">read -p 'input_label: ' variable_name </code></pre> <h3 id="heading-working-example">Working Example:</h3> <p>Now let's create scripts to receive user input about the Postgres database image you're going to create.</p> <pre><code class="language-shell">#!/bin/bash # database credentials read -p 'DataBase-name(*): ' db read -p 'DataBase-user(*): ' user read -sp 'DataBase-password(*): ' pwd echo echo "your database is -&gt; $db with a username of $user and a password of $pwd" </code></pre> <p>The script uses <code>-p</code> on the <code>read</code> keyword that prompts a string of text and <code>-sp</code> that prompts the user to enter a string of text that's hidden (because the <code>-s</code> hides what's being typed).</p> <p>If you don't use the <code>-p</code> option together with <code>-s</code>, you won't be able to display a prompt asking the user to enter a password. With <code>-s</code> alone, the input is hidden, but no prompt is shown unless you print one using <code>echo</code> or <code>printf</code>.</p> <p>To print or use the input the user has added, use the dollar sign ($) followed with a variable name for the input you're expecting from the user. In the example above, we named the variable for getting the database name <code>db</code>, so we'll use <code>$db</code> when we want to access the name of the database the user has input.</p> <p>Now if you run the script in the terminal, you should see a sequence of terminal prompts waiting for inputs. Once they're filled, you'll see all the input's values in the terminal:</p> <img src="https://cdn.hashnode.com/uploads/covers/69c7bcff7cf27065100ae8be/9871cea4-07f0-4626-ba2a-f4ca738e447b.png" alt="output on how to get and display user inputs " style="display: block;" width="600" height="400" loading="lazy"> <h3 id="heading-how-to-validate-and-sanitize-input">How to Validate and Sanitize Input</h3> <p>You'll also need to validate input before executing anything. This is important because sometimes crucial field might be missing or mistyped. For example, you might forget to input the database name for the image you're trying to create.</p> <p>When validating input, you'll use one of these flags (among others):</p> <ul> <li>-z: checks if a variable value is empty</li> <li>-n: checks if a variable value is not empty</li> <li>-lt: checks if a number is less than a given number</li> <li>-gt: checks if a number is greater than a given number</li> <li>-eq: checks for equality of two values</li> </ul> <p>You can use the <code>-z</code> flag on every input that you need to make sure isn't empty before moving on to the next step:</p> <pre><code class="language-shell">#!/bin/bash # database credentials read -p 'DataBase-name(*): ' db if [[ -z $db ]] then echo "The database field cannot be empty!" exit 1 fi read -p 'DataBase-user(*): ' user if [[ -z $user ]] then echo "The user field cannot be empty!" exit 1 fi read -sp 'DataBase-password(*): ' pwd if [[ -z $pwd ]] then echo "The password field cannot be empty!" exit 1 fi </code></pre> <p>This script helps you make sure that all critical information about the database is present before proceeding to another input.</p> <h3 id="heading-default-values">Default Values</h3> <p>You may want some fields to use default value (for example, the container name of the image, port, or volume of the container). For such scenarios you can use <code>:=</code> to assign a variable default value:</p> <pre><code class="language-shell">#!/bin/bash # container information read -p 'Container-name: ' container_name read -p 'Container-port(if omitted, the container will use default postgresql default port of 5432): ' port read -p 'Volume-name: ' vol_name echo "container-name-&gt; ${container_name:=cont_one}, port-&gt;${port:=5432}, volume_name-&gt;${vol_name:=backend_data}" </code></pre> <p>When you run the script and skip all three of those inputs, bash will use the default value(s) you've used as fallbacks.</p> <img src="https://cdn.hashnode.com/uploads/covers/69c7bcff7cf27065100ae8be/63c5a5df-10f6-4b9e-b3f1-28460b0db3e4.png" alt="Output for setting up a default value on variables" style="display: block;" width="600" height="400" loading="lazy"> <p>But if you pass value for those inputs, bash uses the provided values. For example, supply all three inputs and see the output you get:</p> <img src="https://cdn.hashnode.com/uploads/covers/69c7bcff7cf27065100ae8be/d62949f4-b87e-448e-b702-393a04ed19ab.png" alt="Receiving user inputs and displaying the recevied input" style="display: block;" width="600" height="400" loading="lazy"> <h2 id="heading-how-to-create-and-write-to-a-file">How to Create and Write to a File</h2> <p>Now that you know how to create a Docker image for a Postgres database, you need to create a <code>docker-compose.yml</code> file that will be executed when starting and stopping your containers.</p> <p>In Bash, there are multiple ways of creating a file:</p> <ul> <li>First, you can use <code>&gt; filename</code>. This creates a file if it doesn't exist. If it exists, its contents are overwritten (replaced).</li> <li>There's also <code>&gt;&gt; filename</code>: This appends contents to a file or creates a file if one doesn't exist.</li> <li>Finally, there's <code>touch filename</code>: This creates a file if it doesn't exist. If the file already exists, its contents are preserved and only the file's timestamps are updated.</li> </ul> <h3 id="heading-how-to-use-gt-to-create-and-write-to-a-file">How to Use <code>&gt;</code> to Create and Write to a File</h3> <p>When you want to create a file and write content to it at the same time, you'll want to use echo in front of <code>&gt;</code> and then add the filename, like this:</p> <pre><code class="language-shell">#!/bin/bash echo Hello&gt; test.txt </code></pre> <p>This will create a <code>test.txt</code> file with "hello" as content inside.</p> <h3 id="heading-how-to-use-gtgt-to-append-contents-to-an-existing-file">How to Use <code>&gt;&gt;</code> to Append Contents to an Existing File</h3> <p>To append contents to an existing file, add the contents before the <code>&gt;&gt;</code> and then follow with the filename you want to append content onto. Let's add Africa after Hello in the <code>test.txt</code> file we created in the previous script.</p> <pre><code class="language-shell">#!/bin/bash echo Africa!&gt;&gt; test.txt </code></pre> <p>Now test.txt will have Hello and Africa! on the new line.</p> <p>Using those symbols, let's create a <code>docker-compose.yml</code> file that will hold contents about our Postgres database setup.</p> <pre><code class="language-shell">#!/bin/bash &gt; docker-compose.yml </code></pre> <p>And just like that you have your docker-compose file created in your directory.</p> <h3 id="heading-how-to-write-multiple-different-contents-to-a-file">How to Write Multiple Different Contents to a File</h3> <p>You can also use the <code>cat</code> keyword to write to a file. Typically, <code>cat</code> is used for reading the contents of a file. But when you combine it with <code>&lt;&lt;EOF</code> and the sign for appending contents to a file <code>&gt;&gt;</code> (that we just discussed), you can add multiple contents to a file in the format you want. The syntax looks like this:</p> <pre><code class="language-shell"> # writing docker compose contents cat &lt;&lt;EOF&gt;&gt; docker-compose.yml # file contents EOF </code></pre> <p>The <code>&lt;&lt;EOF</code> tells bash to write contents to a file until it finds the EOF keyword (end of file). Then you use <code>&gt;&gt; filename</code> to append the contents to the file. With a Docker compose setup it would look like this:</p> <pre><code class="language-shell">#!/bin/bash #file creation and writing &gt; docker-compose.yml cat&lt;&lt;EOF&gt;&gt;docker-compose.yml services: postgres: image: postgres:18-alpine container_name: ${container_name:= $db} environment: POSTGRES_USER: $user POSTGRES_PASSWORD: $pwd POSTGRES_database: $db ports: - ${port:=5432}:5432 volumes: - ${vol_name:=database-data}:/var/postgresql volumes: ${vol_name:=database-data}: EOF </code></pre> <h2 id="heading-bash-arguments-and-flags">Bash Arguments and Flags</h2> <p>With Bash arguments, you can pass values in the same line while executing a script. To use arguments, add the values while executing the script on the same line, like this:</p> <pre><code class="language-shell">bash script val1 val2 </code></pre> <p>To read arguments value(s), use the dollar sign ($) followed by the position in which you inputted your values (positions start at 1, not 0).</p> <p>In the example, if you want to echo the content of the second argument value, you can use <code>echo $2</code>.</p> <p>To understand this better, let's enhance data input experience by passing the database credentials on the fly using arguments:</p> <pre><code class="language-shell">#!/bin/bash # database credentials if [[ -z $1 ]]; then read -p 'DataBase-name(*): ' db else db="$1" fi # database checking and validating if [[ -z $2 ]]; then read -p 'DataBase-user(*): ' user else user="$2" fi # user checking and validating if [[ -z $1 ]]; then read -sp 'DataBase-password(*): ' pwd else pwd=$3 fi # password vaidation and checking </code></pre> <p>The snippet first checks if there's an argument. If there aren't any, you use the prompt to input the value. If there are arguments, you assign the argument value to a variable that represents the given database credentials (database name, password, and so on...). Now you can run the script with arguments:</p> <pre><code class="language-shell">bash image.sh customers ABC-Company secret123! </code></pre> <p>The image that will be created from the above script will have a database of <code>customers</code>, user <code>ABC-Company</code>, and a password of <code>secret123!</code>.</p> <h3 id="heading-bash-flagsoptions">Bash Flags/Options</h3> <p>The script has a section that asks for information about the image that you're creating, but the info can be optional. With bash option/flags, you can skip the step that prompts for container information input using the <code>-D</code> option. This lets you explicitly keep the default container info before the script runs.</p> <p>Here's an example:</p> <pre><code class="language-bash"> #!/bin/bash while getopts "ny" options; do case "$options" in n) echo "You choose no";; y) echo "You choose yes";; *) echo "No such option/flag"; exit 1;; esac done shift $((OPTIND - 1)) </code></pre> <p>In the above script:</p> <ol> <li>The while loop keeps running as long as <code>getopts</code> isn't empty</li> <li><code>getopts</code> gets the passed option in the script</li> <li><code>"ny"</code> are the options we're expecting from user when running the script</li> <li><code>options</code> is a variable that will keep track of the input option between "n" and "y"</li> <li><code>do</code> starts the loop</li> <li><code>case</code> checks if the selected option is in the defined option using the <code>in</code> keyword</li> <li>We then define what happens when we call the script with the option <code>-n</code> or <code>-y</code> and use a double semi-colon (;;) to close the option definition</li> <li>The <code>*</code> is used in case the input options aren't defined.</li> <li><code>esac</code> then closes the case</li> <li><code>done</code> ends the loop so the loop doesn't continue running forever</li> <li><code>shift $((OPTIND - 1))</code> helps avoid a conflict that may occur when the script also expects arguments by shifting the remaining arguments left <em>over</em> the flags, effectively discarding the flags from the parameter list.</li> </ol> <p>You can then apply the syntax in the script to skip the step that asks for container info when <code>-D</code> is passed as an option during the script execution.</p> <pre><code class="language-shell"> #!/bin/bash ask_container_info=true while getopts "D" options; do case "$options" in D) ask_container_info=false;; *) echo "No such option/flag"; exit 1;; esac done shift $((OPTIND - 1)) if [[ $ask_container_info == true ]]; then #Prompt the user to input custom container information read -p 'Container-name: ' container_name read -p 'Container-port(if omitted, the container will use default postgresql default port of 5432): ' port read -p 'Volume-name: ' vol_name fi </code></pre> <p>There is one option here, which is why the option section only has <code>D</code>. <code>ask_container_info</code> starts as true, meaning unless the option is passed, the script will prompt the user to enter the container information. We then change the value of <code>ask_container_info</code> to false when the <code>-D</code> is passed with the script's execution.</p> <h2 id="heading-how-to-make-your-scripts-executable-and-available-on-your-pcs-path">How to Make Your Scripts Executable and Available on Your PC's Path</h2> <p>The next step is to make the script available and executable in every terminal you use on your machine. At the moment, the script just works in the current directory you're using to create and write your scripts.</p> <p>We'll start by making the script file executable and remove the file extension from the file, so we don't need to always call our script with the <code>.sh</code> extension.</p> <h3 id="heading-how-to-make-a-file-executable">How to Make a File Executable</h3> <p>To make a file executable, use the Linux <code>chmod</code> command. It stands for change mode. Mode includes file permissions like write, read, and execute.</p> <pre><code class="language-shell">chmod +x image </code></pre> <p>Here, <code>x</code> makes the file executable, and then you've listed the file name you want to make executable (<code>image</code>).</p> <h3 id="heading-how-to-add-your-directory-to-the-systemmachine-path">How to Add Your Directory to the <code>system/machine</code> Path</h3> <p>To make this available in every terminal you open, regardless of the directory, you'll need to register your scripts in the system paths where all global scripts and commands are stored.</p> <p>To do that, copy the full path of your directory and then in your terminal type the following:</p> <pre><code class="language-shell">export PATH="$PATH:&lt;YOUR_DIRECTORY_LOCATION&gt;" </code></pre> <p>Replace <code>&lt;YOUR_DIRECTORY_LOCATION&gt;</code> with your directory. This command will add the script directory into the current working terminal. But if you try to switch to another terminal and run your script, it'll fail.</p> <p>Now let's configure the directory globally so every terminal will be able to execute the script:</p> <pre><code class="language-shell">nano ~/.bash_profile </code></pre> <p>After this you should see something that looks like this:</p> <img src="https://cdn.hashnode.com/uploads/covers/69c7bcff7cf27065100ae8be/eb9d4588-4d21-43d5-a666-d406cdd9946d.png" alt="Nano editor for bash profiles and its contents" style="display: block;" width="600" height="400" loading="lazy"> <p>You can see in the image above that I already have my path declared in the file (see the selected text).</p> <p>If it's not in you're file, you can copy-paste <code>export PATH="$PATH:&lt;YOUR_DIRECTORY_LOCATION&gt;"</code> inside the file. When you're done, press <code>Ctrl+O</code> and then <code>enter</code> to save. Then press <code>Ctrl+x</code> to exit the nano editor.</p> <p>Then re-load the machine path again to include your scripts there:</p> <pre><code class="language-shell">source ~/.bash_profile </code></pre> <p>After this, you'll be able to run your script in any terminal on your machine using the script file name only (I named my file <code>image</code>):</p> <img src="https://cdn.hashnode.com/uploads/covers/69c7bcff7cf27065100ae8be/09dcf19f-0712-48f7-9ac6-f00723aa267c.png" alt="Output of a successfully script execution for the docker image creation script" style="display: block;" width="600" height="400" loading="lazy"> <h2 id="heading-conclusion">Conclusion</h2> <p>In this guide, you learned how to create your own scripts to automate your workflows. I hope it helps you reduce the amount of energy and time you waste on repetitive tasks that can be automated using script commands.</p> <p>You can find the full working CLI tool that we used as an example in this guide on this <a href="https://github.com/jabo-arnold-landry/Postgres-docker-image-generator">git-repo</a>. There you'll find the installation guide and usage tips. If you find it helpful, please consider starring the repo and sharing it with your collogues who might benefit from it.</p> <p>Thanks!</p>